Preview button on the plugin settings page (M5 part 3b-1)

CLAUDE.md

@@ -101,6 +101,7 @@ Every PR costs txoof review time. Too many small PRs and too few huge ones both
 - **Aim for 300–600 changed lines** (code, tests and docs together; images don't count). Above about 800 lines, split the work: 800 is a guideline and an upper bound, so a PR may go a little over it, but never much. Below that, keep a task in one PR (see "One PR per task" above).
 - **Each PR does one thing that works on its own**, with its tests and docs. `main` is never left half-built.
 - **No PR for one small change** (a typo, a one-line rule). Put it in the next related PR.
+- **Every PR is based on `main`.** No stacked PRs (a PR based on another PR's branch): they cost extra merges and conflict fixes. Split a task into parts that each go straight to `main`, one after the other.
 - **Plan the split before coding.** Show the planned PRs to txoof together with the design questions. One issue may need several PRs; list them in the issue. For example, the scheduler (+1925 lines in #206) could have been four PRs: the new plugins and settings; the scheduler core with its tests (which use a pretend clock); failures, the fallback plugins and config reload; the `paperpi run` command and README.
 - **Review fixes:** small fixes go into the same PR. Fixes that add a new feature go into a follow-up PR, so the first one doesn't keep growing.
 - See the size with `git diff --shortstat origin/main...HEAD`.

README.md

@@ -99,7 +99,7 @@ uv run paperpi run --config paperpi.toml --out screen/ --state-dir state/ --heal
 
 ### The web interface
 
-`paperpi run` also starts the web interface, on port 8080. Open it by the Pi's IP address on a phone or computer on the same home network, for example `http://192.168.1.20:8080` (your router's list of devices shows the Pi's address; on the Pi, `hostname -I` prints it). Names such as `paperpi.local` are refused for now, for safety (see [docs/decisions/web-interface.md](docs/decisions/web-interface.md)). It has three pages for plugins: **Active Plugins** (switch plugins on or off, move them up or down, remove one, open its settings), **Plugin Library** (every plugin that comes with PaperPi; add one at the end of the list) and each plugin's **Settings** (every setting with its help text and default; errors are shown next to the field). A change there is saved in the config file and applies at once. Comments in the file stay, except the comments of a plugin you remove and a comment written after a setting, on the same line, when the web interface changes that setting. If the config file changed since the page was opened, is written in a way the web interface can't change safely, or has a problem that stops PaperPi from using it, nothing is saved and the page says so. A plugin that needs settings (such as `met_no`, which needs a place and your email address) is added switched off, and its Settings page opens to fill them in.
+`paperpi run` also starts the web interface, on port 8080. Open it by the Pi's IP address on a phone or computer on the same home network, for example `http://192.168.1.20:8080` (your router's list of devices shows the Pi's address; on the Pi, `hostname -I` prints it). Names such as `paperpi.local` are refused for now, for safety (see [docs/decisions/web-interface.md](docs/decisions/web-interface.md)). It has three pages for plugins: **Active Plugins** (switch plugins on or off, move them up or down, remove one, open its settings), **Plugin Library** (every plugin that comes with PaperPi; add one at the end of the list) and each plugin's **Settings** (every setting with its help text and default; errors are shown next to the field; **Preview** draws the plugin with the settings in the form before you save them, at your screen's size, without touching the screen; if the real data can't be drawn, it shows sample data and says why). A change there is saved in the config file and applies at once. Comments in the file stay, except the comments of a plugin you remove and a comment written after a setting, on the same line, when the web interface changes that setting. If the config file changed since the page was opened, is written in a way the web interface can't change safely, or has a problem that stops PaperPi from using it, nothing is saved and the page says so. A plugin that needs settings (such as `met_no`, which needs a place and your email address) is added switched off, and its Settings page opens to fill them in.
 
 - **First visit:** the first person to open it sets the web password (at least 8 characters). The browser then stays logged in for a year, until **Log out**. Log out only logs out that browser; a new password logs out every browser.
 - **Forgotten password:** you need access to the Pi itself (a keyboard and screen, or SSH).

docs/decisions/web-interface.md

@@ -45,7 +45,7 @@ In v2 the web interface is the main way to set up and change PaperPi: plugins, s
 - No limit on wrong password attempts.
 
 *Update (M5, issue #238, agreed with txoof on 2026-10-07):*
-- The web server runs **inside `paperpi run`**, in its own thread, not as a separate program. It can tell the scheduler directly to apply a change, it needs one Python process fewer (about 40 MB), and there is one program to install. Previews will draw in a separate process with a time limit, like normal updates, so a slow or broken plugin can't stop the web pages or the screen. If the web interface can't start (port taken), the log says why and the screen keeps running.
+- The web server runs **inside `paperpi run`**, in its own thread, not as a separate program. It can tell the scheduler directly to apply a change, it needs one Python process fewer (about 40 MB), and there is one program to install. Previews draw in a separate process with a time limit, like normal updates, so a slow or broken plugin can't stop the web pages or the screen. If the web interface can't start (port taken), the log says why and the screen keeps running.
 - The password is stored as a **scrypt** hash, which is built into Python (about 0.1 s and 16 MB of memory per check on a Pi; checks run one at a time; no limit on wrong tries, agreed again on 2026-10-07). The log-in cookie is signed with that hash, so a new password logs out every browser.
 - Packages: FastAPI, uvicorn (the web server that runs FastAPI), jinja2 (page templates) and python-multipart (reads forms).
 - **Log-in can be switched off:** `login = false` in `[web]`. Then anyone on the home network can change the settings; the config check gives a hint and the home page says so.
@@ -60,7 +60,14 @@ In v2 the web interface is the main way to set up and change PaperPi: plugins, s
 ### Preview images
 
 - **Home page:** a copy of what is on the e-paper screen now, with the plugin's name and when it was drawn. It updates by itself when the screen changes.
-- **Plugin settings page:** a **Preview** button draws the plugin with the settings in the form, before saving, using the PNG driver (see `display-driver-interface.md`). The real screen is not touched. If the plugin's data source can't be reached, the preview uses the plugin's sample data and says so.
+- **Plugin settings page:** a **Preview** button draws the plugin with the settings in the form, before saving, in a separate process, as normal updates are drawn. The real screen is not touched. If the plugin's data source can't be reached, the preview uses the plugin's sample data and says so.
+- *Update (M5 part 3b-1, agreed with txoof on 2026-10-09):*
+  - The preview draws the block as Save would write it (the form's values applied to a copy of the file text), at the screen's size and for the screen's type from `[display]`. It runs in its own process with a time limit, as normal updates do, in a new, empty storage folder that is deleted afterwards, so it can't change the plugin's saved files.
+  - Real data may take the plugin's own time limit, but at most 45 seconds. When it fails, takes longer, has nothing to show right now, or a required setting is still empty, the preview draws the plugin's sample data (at most 10 seconds) and says why.
+  - One preview is drawn at a time; a second one meanwhile is asked to wait.
+  - The picture is sent inside the page (a `data:` address), so it is never saved as a file. The pages allow such pictures (`img-src 'self' data:`).
+  - For later plugins: an empty secret field (such as an API key) keeps the saved secret, also in a preview. A plugin that has both a secret and a web address the user can change must not send the saved secret to a changed address: then the secret must be typed again (in a preview and when saving), or the plugin allows only its own servers. Network errors in the preview should then say only "could not reach the server", so the preview can't be used to look around the home network. No plugin has such an address yet (checked in the security review of part 3b-1); this matters for e.g. the Dutch public transport plugin.
+  - There is no "send to display" button on the settings page. A **Show on screen now** button, which starts a saved plugin's turn at once, comes with the home page in part 4.
 
 ### Other pages
 

docs/writing-plugins.md

@@ -59,7 +59,7 @@ PLUGIN = Plugin(
 3. `draw(data, context)` returns a dictionary with a value for each block of the layout, for example `{"time": "10:42"}` (or a `Drawn`, see "Moving blocks and colours"). PaperPi draws the layout with epdlib.
 4. The image goes back to PaperPi and the process exits.
 
-For tests and sample images, step 2 is skipped and `sample` is drawn instead. So every plugin can draw an image without network access, and `draw` must work with the sample data.
+For tests and sample images, step 2 is skipped and `sample` is drawn instead. So every plugin can draw an image without network access, and `draw` must work with the sample data. The web interface's Preview also draws the sample data when the real data fails, takes too long, has nothing to show or still needs settings, so make the sample look like real data.
 
 `context` holds:
 

src/paperpi/cli.py

@@ -343,6 +343,8 @@ def load() -> config.Config:
             )
             scheduler.run()
             reports.stopping()
+            if web is not None:
+                web.stop_previews()  # before the clear below, which can take 30 s
             # Only after a stop that was asked for (Ctrl+C, systemctl stop, shutdown or
             # reboot): after an error, the restart draws the picture again anyway. A second
             # stop signal ends the clear at once.

src/paperpi/limits.py

@@ -11,6 +11,13 @@
 #: Highest time limit a user may set for one plugin update.
 PLUGIN_UPDATE_MAX = 600.0
 
+#: Longest a web interface preview may wait for a plugin's real data (the plugin's own time
+#: limit, if that is shorter). Then the preview draws the sample data instead.
+PREVIEW_LIVE = 45.0
+
+#: Longest drawing a plugin's sample data may take for a preview.
+PREVIEW_SAMPLE = 10.0
+
 #: How long a plugin process may take to exit after it has handed over its image.
 PLUGIN_EXIT = 5.0
 

src/paperpi/web/app.py

@@ -10,7 +10,8 @@
 ``/plugins``       Active Plugins: the plugins in the config file; switch on or off, move
                    up or down, remove, settings (``/plugins/<n>/...?id=<id>``, n = place
                    in the file, counted from 0, and the plugin's id to check it is
-                   still there)
+                   still there); ``/plugins/<n>/preview`` draws the settings form's
+                   values without saving them (part of the settings page)
 ``/library``       Plugin Library: every plugin type; ``/library/<type>`` adds one and
                    opens its settings
 ``/static/...``    the style sheet and htmx (a small JavaScript file that updates
@@ -27,6 +28,7 @@
 from __future__ import annotations
 
 import ipaddress
+import logging
 from collections.abc import Awaitable, Callable
 from pathlib import Path
 from typing import Annotated
@@ -46,8 +48,10 @@
 from .helpers import helper_html
 from .password import PasswordError, password_problem
 from .plugins import FormErrors, PluginEditor, library, library_item
+from .preview import Busy, Previewer, PreviewFailed
 
 COOKIE = "paperpi_login"
+log = logging.getLogger(__name__)
 _HERE = Path(__file__).parent
 #: Pages that work without a log-in (and everything in ``/static/``).
 _OPEN = ("/setup", "/login")
@@ -56,10 +60,13 @@
 _QueryId = Annotated[str, Query(alias="id")]
 
 
-def create_app(auth: Auth, editor: PluginEditor | None = None) -> FastAPI:
-    """The web interface, using ``auth`` for the password and log-in and ``editor`` to
-    change the plugins in the config file."""
+def create_app(
+    auth: Auth, editor: PluginEditor | None = None, previewer: Previewer | None = None
+) -> FastAPI:
+    """The web interface, using ``auth`` for the password and log-in, ``editor`` to
+    change the plugins in the config file and ``previewer`` to draw previews."""
     editor = editor or PluginEditor(auth.config_file)
+    previewer = previewer or Previewer()
     app = FastAPI(title="PaperPi", docs_url=None, redoc_url=None, openapi_url=None)
     app.mount("/static", StaticFiles(directory=_HERE / "static"), name="static")
     templates = Jinja2Templates(directory=_HERE / "templates")
@@ -91,18 +98,20 @@ async def refused_or_page(
         path = request.url.path
         if auth.login and path not in _OPEN and not path.startswith("/static/"):
             if auth.password_hash is None:
-                return RedirectResponse("/setup", status_code=303)
+                return _go_to(request, "/setup")
             if not auth.cookie_ok(request.cookies.get(COOKIE)):
-                return RedirectResponse("/login", status_code=303)
+                return _go_to(request, "/login")
         return await call_next(request)
 
     @app.middleware("http")
     async def check(request: Request, call_next: Callable[[Request], Awaitable[Response]]):
         response = await refused_or_page(request, call_next)
         path = request.url.path
         response.headers["X-Frame-Options"] = "DENY"
+        # Previews are pictures sent inside the page (data: addresses), never saved.
         response.headers["Content-Security-Policy"] = (
-            "default-src 'self'; frame-ancestors 'none'; form-action 'self'; base-uri 'none'"
+            "default-src 'self'; img-src 'self' data:; frame-ancestors 'none'; "
+            "form-action 'self'; base-uri 'none'"
         )
         response.headers["Referrer-Policy"] = "same-origin"
         if not path.startswith("/static/"):
@@ -262,9 +271,7 @@ def settings_form(request: Request, index: int, plugin_id: _QueryId = ""):
 
     @app.post("/plugins/{index}/settings", response_class=HTMLResponse)
     async def save_settings(request: Request, index: int):
-        form = await request.form()
-        plugin_id = str(form.get("id", ""))
-        sent = {k: [str(v) for v in form.getlist(k)] for k in form if k != "id"}
+        plugin_id, sent = _settings_form(await request.form())
         try:
             hand_edits = await run_in_threadpool(editor.save_settings, index, plugin_id, sent)
         except FormErrors as error:
@@ -275,6 +282,24 @@ async def save_settings(request: Request, index: int):
             return await run_in_threadpool(plugin_list, request, 409, str(error))
         return settings_saved(index, plugin_id, "saved", hand_edits)
 
+    @app.post("/plugins/{index}/preview", response_class=HTMLResponse)
+    async def preview(request: Request, index: int):
+        # Only a part of the settings page (htmx), so every answer is a 200: htmx doesn't
+        # show the others.
+        plugin_id, sent = _settings_form(await request.form())
+        try:
+            job = await run_in_threadpool(editor.preview_job, index, plugin_id, sent)
+            drawn = await run_in_threadpool(previewer.draw, job)
+        except FormErrors as error:
+            return page(request, "preview.html", errors=error.found.errors)
+        except (EditError, Busy, PreviewFailed) as error:
+            return page(request, "preview.html", problem=str(error))
+        except Exception:  # noqa: BLE001 - e.g. no plugin process could be started
+            log.exception("preview of %s", plugin_id)
+            problem = "The preview could not be drawn. PaperPi's log has the details."
+            return page(request, "preview.html", problem=problem)
+        return page(request, "preview.html", drawn=drawn, job=job)
+
     def settings_saved(index: int, plugin_id: str, done: str, hand_edits: bool):
         values = {"id": plugin_id, "done": done} | ({"hand": "1"} if hand_edits else {})
         return RedirectResponse(f"/plugins/{index}/settings?{urlencode(values)}", 303)
@@ -322,6 +347,20 @@ def logout():
     return app
 
 
+def _go_to(request: Request, path: str) -> Response:
+    """Send the browser to ``path``. For a part of a page that htmx asked for, htmx opens
+    the whole page (else it would show the page inside the part)."""
+    if request.headers.get("hx-request") == "true":
+        return Response(status_code=204, headers={"HX-Redirect": path})
+    return RedirectResponse(path, status_code=303)
+
+
+def _settings_form(form) -> tuple[str, dict[str, list[str]]]:
+    """The plugin ID and the settings of a sent settings form."""
+    plugin_id = str(form.get("id", ""))
+    return plugin_id, {k: [str(v) for v in form.getlist(k)] for k in form if k != "id"}
+
+
 def _number(value: float) -> str:
     """A number as the pages show it: ``604800``, not ``604800.0``."""
     return str(int(value)) if float(value).is_integer() else str(value)