Add py-render custom code block formater

docs/contributing.md

@@ -315,10 +315,19 @@ words.
 
 #### Code Blocks
 
-All code blocks should use the fenced code block style. If a code block is
-demonstrating Markdown syntax, if can be assigned the `md-render` attribute,
-and both the Markdown source and HTML output will be rendered in a nested set
-of code blocks.
+All code blocks should use the fenced code block style and indicate the
+language of the code contained in the block to ensure proper syntax
+highlighting. 
+
+There are two special types of code blocks which will render output based on
+the content of the code block. See [Rendered Markdown](#rendered-markdown)
+and [Rendered Python](#rendered-python) below.
+
+##### Rendered Markdown
+
+If a code block is demonstrating Markdown syntax, it can be assigned the
+`md-render` attribute in place of the language, and both the Markdown source
+and HTML output will be rendered in a nested set of code blocks.
 
 ```` markdown
 ``` md-render
@@ -388,6 +397,87 @@ The above code block would render as follows:
 Some *Markdown* text.
 ```
 
+##### Rendered Python
+
+If a code block is demonstrating Python code, it can be assigned the
+`py-render` attribute in place of the language, and both the Python code and
+output will be rendered in a nested set of code blocks. The language of the
+output should be specified using the `output-lang` attribute.
+
+```` markdown
+``` py-render { output-lang='html' }
+import markdown
+
+src = 'Some **Markdown** text.'
+fragment = markdown.markdown(src)
+```
+````
+
+The above code block will be rendered as follows:
+
+``` py-render { output-lang='html' }
+import markdown
+
+src = 'Some **Markdown** text.'
+fragment = markdown.markdown(src)
+```
+
+Note that the Python code is executed in an isolated environment. Therefore,
+any imports need to be made to avoid errors. However, as all `py-render`
+blocks on the same page are run within the same environment, an import only
+needs to be made once (before the first use) for all code blocks on the same
+page. Variables assigned in one block will be available in later blocks on the
+same page.
+
+```` markdown
+``` py-render { output-lang='html' }
+from justhtml import JustHTML
+
+html = JustHTML(fragment).to_html()
+```
+````
+
+Notice that in the block above, the variable `fragment` from the previous code
+block is available within this code block. However, `JustHTML` needs to be
+imported as it had not been previously.
+
+``` py-render { output-lang='html' }
+from justhtml import JustHTML
+
+html = JustHTML(fragment).to_html()
+```
+
+Each page contains it's own isolated environment. Objects defined on one page
+will not be available on another page and would need to be redefined.
+
+Output will be generated if one of three conditions are met. Whichever
+condition is encountered first (in decreasing order) is the controlling
+condition.
+
+1. If an error is raised, a traceback will be rendered in the output and
+   highlighted using Pygment's `PythonTracebackLexer` (`py3tb`). 
+2. If the code writes to STDOUT (for example, it passes text to `print()`),
+   then the text sent to STDOUT will be rendered in the output and
+   highlighted using the language assigned to `output-lang`.
+3. If the last line of the code assigns a value to a variable, the value of
+   that variable will be rendered in the output and highlighted using the
+   language assigned to `output-lang`.
+
+If none of the above conditions are met, then the code block will render as
+normal without rendered output. However, the code block will have updated the
+isolated Python environment and any objects created can be referenced in
+later code blocks on the same page.
+
+If a Python code block should not have it's code executed and rendered, then
+simply assign it the `python` attribute. It will then be rendered as a normal
+Python code block. Any objects defined in standard Python code blocks
+will **not** be available to `py-render` style blocks
+
+Code blocks which contain Python Console sessions are not supported by
+`py-render`. They should be assigned `pycon` and contain the code and output
+as copied out of a Python Console session. Any objects defined in Python
+Session code blocks will **not** be available to `py-render` style blocks.
+
 #### Changelog
 
 Any commit/pull request which changes the behavior of the Markdown library in

docs/extensions/code_hilite.md

@@ -262,7 +262,8 @@ markdown.markdown(some_text, extensions=['codehilite'])
 To keep the code block's language in the Pygments generated HTML output, one can provide a custom Pygments formatter
 that takes the `lang_str` option. For example,
 
-```python
+```py-render { output-lang='html' }
+import markdown
 from pygments.formatters import HtmlFormatter
 from markdown.extensions.codehilite import CodeHiliteExtension
 
@@ -285,24 +286,12 @@ some_text = '''\
     print('hellow world')
 '''
 
-markdown.markdown(
+output = markdown.markdown(
     some_text,
     extensions=[CodeHiliteExtension(pygments_formatter=CustomHtmlFormatter)],
 )
 ```
 
-The formatter above will output the following HTML structure for a code block:
-
-```html
-<div class="codehilite">
-    <pre>
-        <code class="language-python">
-        ...
-        </code>
-    </pre>
-</div>
-```
-
 [html formatter]: https://pygments.org/docs/formatters/#HtmlFormatter
 [lexer]: https://pygments.org/docs/lexers/
 [spec]: https://www.w3.org/TR/html5/text-level-semantics.html#the-code-element

docs/library.md

@@ -10,49 +10,26 @@ used by various projects to convert Markdown syntax into HTML.
 ## The Basics
 
 To use markdown as a module, pass a string to the [`markdown.markdown`]
-[markdown.markdown] function.
+[markdown.markdown] function. The string must be a *Unicode* string
+(the default string type in Python).
 
-```python
+``` py-render { output-lang='html' }
 import markdown
-html = markdown.markdown(your_text_string)
-```
-
-The string must be a *Unicode* string (the default string type in Python).
 
-``` python
 src = 'Some **Markdown** text.'
-html = markdown.markdown(src)
+fragment = markdown.markdown(src)
 ```
 
-Python-Markdown only ever outputs an HTML fragment. Therefore, the value of
-`html` above would be:
-
-``` html
-<p>Some <strong>Markdown</strong> text.</p>
-```
+Python-Markdown only ever outputs an HTML fragment. If you need a complete
+HTML document, including `<html>`, `<head>` and `<body>` tags, then you will
+need to pass the output of Python-Markdown into some other tool. For a
+minimal complete document, [JustHTML](https://emilstenstrom.github.io/justhtml/)
+can do that with a single line of code:
 
-If you need a complete HTML document, including `<html>`, `<head>` and
-`<body>` tags, then you will need to pass the output of Python-Markdown into
-some other tool. For a minimal complete document,
-[JustHTML](https://emilstenstrom.github.io/justhtml/) can do that with a
-single line of code:
-
-``` python
+``` py-render { output-lang='html' }
 from justhtml import JustHTML
 
-doc = JustHTML(html)
-```
-
-Assuming the value of `html` from above, the value returned by 
-`doc.to_html()` would be the following string:
-
-``` html
-<html>
-  <head></head>
-  <body>
-    <p>Some <strong>Markdown</strong> text.</p>
-  </body>
-</html>
+html = JustHTML(fragment).to_html()
 ```
 
 For more sophisticated output, you may need to explore the use of a templating

mkdocs.yml

@@ -157,13 +157,12 @@ markdown_extensions:
       pygments_lang_class: true
   - pymdownx.superfences:
       custom_fences:
-        - name: mermaid
-          class: mermaid
-          format: !!python/name:pymdownx.superfences.fence_code_format
         - name: md-render
           class: md-render
           format: !!python/name:tools.superfences_formaters.md_render
-
+        - name: py-render
+          class: py-render
+          format: !!python/name:tools.superfences_formaters.py_render
 
 plugins:
 - search

pyproject.toml

@@ -56,6 +56,7 @@ docs = [
     'mkdocstrings-python==1.16.8',
     'pygments==2.21.0',
     'pymdown-extensions==11.0.2',
+    'justhtml==3.11.2'
 ]
 
 [project.urls]

tools/superfences_formaters.py

@@ -7,6 +7,9 @@
 import markdown
 import yaml
 import re
+import ast
+import sys
+from io import StringIO
 from collections import OrderedDict
 
 
@@ -83,3 +86,92 @@ def md_render(src="", language="", class_name=None, options=None, md="", **kwarg
     source = md.preprocessors['fenced_code_block'].highlight(text, 'markdown', options, md, **kwargs)
     output = md.preprocessors['fenced_code_block'].highlight(html, 'html', result_options, md, **kwargs)
     return f'{source}\n<div class="result">{output}</div>'
+
+
+class PyExecNamespace():
+    def __init__(self, globals=None, locals=None):
+        self.globals = globals or {}
+        self.locals = locals or {}
+
+    def exec(self, source):
+        """ 
+        Execute code in namespace.
+
+        If code outputs to stdout, that output is captured and returned.
+        If nothing it output to stdout, then the last line of code is checked
+        for a variable assignment. If one exists, then the value of that variable
+        is returned. If that fails, then `None` is returned.
+        """
+
+        # Temporarily redirect stdout
+        save_stdout = sys.stdout
+        sys.stdout = StringIO()
+
+        # Run code
+        try:
+            exec(source, self.globals, self.locals)
+        except KeyboardInterrupt:
+            sys.stdout.close()
+            sys.stdout = save_stdout
+            raise
+        except BaseException as exc:
+            sys.stdout.close()
+            sys.stdout = save_stdout
+            import traceback
+            tb = traceback.format_exception(exc, exc, exc.__traceback__.tb_next)
+            return 'traceback', '\n'.join(tb)
+
+        # Retreive anything sent to stdout and restore system default
+        out = sys.stdout.getvalue()
+        sys.stdout.close()
+        sys.stdout = save_stdout
+
+        if out:
+            # Return text sent to stdout
+            return 'stdout', out
+        else:
+            # Nothing sent to stdout. Try to get value of last variable assignment
+            target = None
+            a = ast.parse(source)
+            if a.body:
+                if isinstance(a_last := a.body[-1], ast.Assign):
+                    target = ast.unparse(a_last.targets[0])
+                elif isinstance(a_last, (ast.AnnAssign, ast.AugAssign)):
+                    target = ast.unparse(a_last.target)
+            if target and target in self.locals:
+                return target, self.locals[target]
+        return None, None
+
+
+def py_render(src="", language="", class_name=None, options=None, md="", **kwargs):
+    """ Render Python in a code block and output of the code in a result code block. """
+
+    if not hasattr(md, 'py_namespace'):
+        # This is the first instance of a Python render block on the page. 
+        # Create namespace for this and all future blocks to run in.
+        md.py_namespace = PyExecNamespace()
+    target, result = md.py_namespace.exec(src)
+
+    # Retreive and remove output language from attrs
+    output_lang = kwargs['attrs'].pop('output-lang', '')
+
+    options = options or {}
+    if 'title' not in options:
+        options['title'] = 'Python'
+
+    source = md.preprocessors['fenced_code_block'].highlight(src, 'python', options, md, **kwargs)
+
+    if result is not None:
+        result_options = options.copy()
+        if target == 'traceback':
+            result_options['title'] = 'Error Raised'
+            output_lang = 'py3tb'  # PythonTracebackLexer
+        elif target == 'stdout':
+            result_options['title'] = 'Text Written to STDOUT'
+        elif target is not None:
+            result_options['title'] = f'Value of `{target}`'
+
+        output = md.preprocessors['fenced_code_block'].highlight(result, output_lang, result_options, md, **kwargs)
+        return f'{source}\n<div class="result">{output}</div>'
+    # No result so only render source
+    return source