From 2f353125b34d94c098dbc17ab6284f803692956f Mon Sep 17 00:00:00 2001 From: Jack Spiering <46534141+jackspiering@users.noreply.github.com> Date: Sat, 10 Oct 2026 02:33:57 +0200 Subject: [PATCH 1/3] Docs: update AGENTS, CONTRIBUTING, and the templates --- .github/PULL_REQUEST_TEMPLATE.md | 6 +- AGENTS.md | 56 +++++++--- CONTRIBUTING.md | 134 +++++++++++++++++------- templates/service-template/.env | 12 +-- templates/service-template/compose.yaml | 10 +- 5 files changed, 154 insertions(+), 64 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 91a9787a..5996a3a5 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,5 +1,5 @@ # Servicename: context - + ## Description @@ -17,7 +17,9 @@ - [ ] I have performed a self-review of my code and followed the [templates](https://github.com/tailscale-dev/ScaleTail/tree/main/templates/service-template) structure. - [ ] I have added verification that the stack works as expected. - [ ] I have updated necessary documentation (e.g. frontpage [README.md](https://github.com/tailscale-dev/ScaleTail/blob/main/README.md) ). -- [ ] I have selected the correct label(s) for this PR. +- [ ] I ran Compose validation with dummy values for required variables. +- [ ] I ran rumdl on the whole repository. +- [ ] I committed no working credential. ## Additional Context diff --git a/AGENTS.md b/AGENTS.md index 97bf57b4..feb2b1e9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,26 +1,58 @@ # Agent guidance -Follow [CONTRIBUTING.md](CONTRIBUTING.md) and the current service template. +ScaleTail is a collection of Docker Compose stacks. Each directory in `services/` is one stack, which usually runs a self-hosted application behind a Tailscale sidecar container. + +## Where the rules live + +- [CONTRIBUTING.md](CONTRIBUTING.md): how to add, update, verify, and submit a service. Follow it. +- [templates/service-template/](templates/service-template/): the starting point for every service. Its comments explain each setting. +- [documentation/standard-setup.md](documentation/standard-setup.md): the shared setup. A service README lists only what differs, under "Deviations from the standard setup". +- [.github/workflows/](.github/workflows/): the Compose validation and Markdown lint that CI runs. ## Before you edit -- Inspect the target service and its README. -- Confirm ports, commands, volumes, and variables in the image's upstream docs. -- Preserve service-specific exceptions that still apply. +- Read the service's `compose.yaml`, `.env`, and README, including "Deviations from the standard setup". +- Confirm ports, commands, volumes, variables, and health checks in the image's upstream docs. +- Preserve documented exceptions. Some stacks use other Compose service keys or container names, and other containers or `.env` values refer to those names. - Keep changes focused on the requested service or documentation. -- Never commit working auth keys, passwords, tokens, or other credentials. + +## Files that change together + +- Add, rename, or remove a service: the `services//` directory and its row in the root `README.md`. Insert a new row in alphabetical order by the bold name, and leave the other rows alone. +- Make existing users act, such as with a new required secret, a moved volume, or a changed `SERVICE`: add an "Upgrading" section to the service README. `SERVICE` also names the Tailnet device, the containers, and most data folders. +- Change the template: also update `documentation/standard-setup.md` and `CONTRIBUTING.md` where they describe the same setting. + +## Secrets and local data + +- Every `.env` file is tracked and published. Never commit an auth key, password, token, or other credential. +- Leave each secret empty in `.env`. When the stack cannot start without it, read it in `compose.yaml` as `${VAR:?Set VAR in .env}`, and add a `# Required:` comment above it in `.env` that says what to set, such as how to generate it. Keep `TS_AUTHKEY` optional, because only the first start needs it. +- Start test stacks from a copy outside the repository, such as `cp -R services/ /tmp/`, and put real values only in the copy. A running stack writes `./config`, `./ts/state` (with the device key), and its data folders next to `compose.yaml`. +- Before you commit, run `git status` and stage only the files you meant to change. ## Verification -Run `docker compose config --quiet` from each changed service directory. When -possible, start the stack and test its main function through the Tailnet. +From each changed service directory, validate Compose. The command gives each required variable a dummy value, as CI does, so you do not edit `.env`: + +```sh +env $(grep -ohE '\$\{[A-Za-z0-9_]+:\?' compose.yaml .env | sed -E 's/^\$\{//; s/:\?$/=dummy/' | sort -u) docker compose config --quiet +``` -From the repository root, lint the changed Markdown files. Replace the example paths below as needed: +It must exit with status 0. CI fails on "variable is not set" warnings. + +From the repository root, lint all Markdown with the rumdl version that `.github/workflows/linting.yml` pins, currently 0.2.78, and check your changes for whitespace errors: ```sh -rumdl check --config .markdownlint.yml AGENTS.md CONTRIBUTING.md -git diff --check +uvx rumdl@0.2.78 check --config .markdownlint.yml . +git diff --check origin/main ``` -Report the checks you ran. State any checks that credentials, hardware, -permissions, or unavailable tools prevented. +Lint the whole tree, because renaming a service breaks a link in the root `README.md`. `git diff --check origin/main` also covers staged and committed changes, but not untracked files, so stage new files first. + +When you have an auth key, start the copied stack and test its main function through the Tailnet. Run each health check command in its container with `docker exec`. + +Report the checks you ran. State any checks that credentials, hardware, permissions, or unavailable tools prevented. + +## Pull requests + +- Title the pull request `Service: what changes`, such as `Umami: new service`. For several services, list them or use `All services:`. For other areas, use `CI:`, `Docs:`, or `Template:`. +- Fill in the [pull request template](.github/PULL_REQUEST_TEMPLATE.md). Maintainers apply the labels. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 889d1afd..85e50de7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -12,33 +12,63 @@ Thanks for helping improve these Tailscale sidecar examples. This command includes the hidden `.env` file. Use a lowercase directory name. -2. Update `.env` with safe example values. +2. Update `.env`. - Set `SERVICE`, `IMAGE_URL`, `SERVICEPORT`, and the application variables. - Never commit a working auth key, password, token, or other credential. + Set `SERVICE`, usually to the directory name, and set `IMAGE_URL`, + `SERVICEPORT`, and the application variables. Define every variable that + `compose.yaml` uses, unless it has a default such as `${VAR:-value}`. Put + comments on their own line: `VAR= # note` sets `VAR` to `# note`. + + `.env` is published with the repository. Never commit a working auth key, + password, token, or other credential. Leave each secret empty. When the stack + cannot start without it, require it in `compose.yaml`: + + ```yaml + - DB_PASSWORD=${DB_PASSWORD:?Set DB_PASSWORD in .env} + ``` + + Compose then stops with an error until the user sets a value. Above the + empty variable in `.env`, add a comment that starts with `# Required:` and + says what to set, such as + `# Required: signs the sessions. Generate it with: openssl rand -hex 32`. + Keep `TS_AUTHKEY` and optional secrets without `:?`. 3. Adapt `compose.yaml`. - Keep the Compose service keys `tailscale` and `application`. - - Name the containers `tailscale-${SERVICE}` and `app-${SERVICE}`. + - Name the containers `tailscale-${SERVICE}` and `app-${SERVICE}`, unless the + stack documents a deviation under "Deviations from the standard setup". - Keep `network_mode: service:tailscale` on the application. - Keep the application's health-based dependency on `tailscale`. - Add all required persistent volumes. - Add required devices and capabilities explicitly. + - When the stack needs another Compose service key or container name, more + than one application container, or none, list the difference and the + reason under "Deviations from the standard setup". Prefer + `app-${SERVICE}-` for extra containers, such as `app-${SERVICE}-db`. + - Pass `PUID` and `PGID` only when the image reads them, such as LinuxServer.io + images. Keep `TZ`, as the template does. + - Write the image tag explicitly and use the tag upstream recommends. Pin + databases, caches, and search engines to a major or major.minor tag. When + you pin an exact version, say why in a comment. + - Keep application data under `./${SERVICE}-data/`, so that `.gitignore` + covers it. 4. Set the Serve proxy to the application's internal port. - The Serve JSON does not read `SERVICEPORT` from `.env`. Keep runtime variables - escaped, such as `$${TS_CERT_DOMAIN}`. + Hard-code the port in the Serve JSON. Do not use `${SERVICEPORT}`: it is + meant for the optional LAN `ports` block and can differ from the port that + Serve must proxy to. Keep runtime variables escaped, such as + `$${TS_CERT_DOMAIN}`. Keep the `ports` block commented for Tailnet-only access. Document any LAN - port you expose. Remove the Serve configuration when the service does not use - Tailscale Serve. + port you expose. When nothing in the stack uses Tailscale Serve, remove the + Serve configuration: the `TS_SERVE_CONFIG` line, the `configs` entry of the + `tailscale` service, and the top-level `configs` block. 5. Configure the application's health check. - Use the first option that the image supports. The template lists the same - options in the same order. + Use the first option that the image supports. 1. When the image defines its own `HEALTHCHECK`, omit the block and add a `# Healthcheck: defined by the image (...)` comment that names the @@ -54,42 +84,37 @@ Thanks for helping improve these Tailscale sidecar examples. is possible. Omit the block and add a `# Healthcheck: none possible ...` comment that names the reason. - Run the command inside the running container before you commit it. Avoid - `pgrep -f ${SERVICE}`, which breaks when `SERVICE` is renamed. - - A container that runs once and exits needs no check. Omit the block and - add a `# Healthcheck: none needed ...` comment that names the reason. - - For a database, connect over TCP, for example with - `pg_isready -h 127.0.0.1` or `mariadb-admin ping -h 127.0.0.1`. During the - first start, the image runs a temporary server that accepts only socket - connections, so a socket check reports ready too early. When another - service depends on the database, wait with `condition: service_healthy`. - A database image's own `HEALTHCHECK` still comes first. When that check - connects over the socket, say so in the comment. - - Keep the Tailscale health check. + Use the comment text from the template's `compose.yaml`, which covers the + same options, the database case, and the no-check cases. Run the command + inside the running container before you commit it. Keep the Tailscale health + check. 6. Complete the service README. Every service README uses the headings of the template, in the same order. - Do not rename them and do not add others, so that every service reads the - same way. Replace the placeholders in capitals. + Do not rename them, and add no headings other than the optional sections + below, so that every service reads the same way. Replace the placeholders in + capitals. - - **Introduction.** Say what the service does in one or two sentences and - link to the upstream project. Leave feature lists to upstream. + - **Introduction.** The paragraph under the title. Say what the service does + in one or two sentences and link to the upstream project. Leave feature + lists to upstream. - **At a glance.** Give the Tailnet address, the port that Tailscale Serve forwards to, the image, and the data paths on the host. Add a row for - each further port that users connect to, such as DNS or SMTP. + each further port that users connect to, such as DNS or SMTP. Write `None` + for the web interface when there is none, and add an `API` row when Serve + publishes an API. - **Before you start.** List only what the Quick Start does not cover: values that must change in `.env`, secrets to generate, folders to - create, and required host groups or devices. + create, and required host groups or devices. List every required secret + and say that Compose stops with an error until it is set. - **Deviations from the standard setup.** List every difference from [the standard setup](documentation/standard-setup.md) and give the reason. Examples are extra containers, published host ports, a changed or removed Serve configuration, DNS settings, and added capabilities. - **First run.** Describe what the user does after the first start, such - as creating the first account or finding a generated password. + as creating the first account or finding a generated password. When there + is nothing to do, write `Nothing to set up. Open the web interface.` - **Links.** Link to the upstream documentation and source code. When a section has nothing to report, keep the heading and the sentence @@ -101,7 +126,9 @@ Thanks for helping improve these Tailscale sidecar examples. - **Configuration.** Optional settings that users commonly change. - **Troubleshooting.** Known errors and their solutions. - - **Upgrading.** Steps for users of an older version of the stack. + - **Upgrading.** Required when a change makes existing users act, such as a + new required secret, a moved data folder, or a changed `SERVICE`. Say what + to do, as in [FossFLOW](services/fossflow/README.md#upgrading). State only what you confirmed in the Compose file, the upstream documentation, or a running stack. Put guidance that applies to more than @@ -109,7 +136,7 @@ Thanks for helping improve these Tailscale sidecar examples. 7. Add the service to the correct category in the root `README.md`. - Keep the entries in that category alphabetized. + Insert the row in alphabetical order of the bold name, ignoring case. ## Update a service @@ -119,20 +146,44 @@ Thanks for helping improve these Tailscale sidecar examples. - Use `${VARIABLE}` for Compose interpolation, not `$(VARIABLE)`. - Update the service README when ports, paths, setup, or behavior change. - Update the root service list when you add, remove, or rename a service. +- Changing `SERVICE` renames the Tailnet device and its address, the + containers, and the `./${SERVICE}-data` folder. Treat it like a volume change + and add an "Upgrading" section. +- Do not rename existing containers or service keys. Other containers and + `.env` values may refer to them. Preserve valid service-specific exceptions. ## Verify your change -Run Compose validation from each changed service directory: +Run Compose validation from each changed service directory. Stacks that +require your own secrets stop with "required variable ... is missing a value". +Give those variables a dummy value on the command line, as CI does, instead of +editing `.env`: ```sh -docker compose config --quiet +env $(grep -ohE '\$\{[A-Za-z0-9_]+:\?' compose.yaml .env | sed -E 's/^\$\{//; s/:\?$/=dummy/' | sort -u) docker compose config --quiet ``` -This command does not prove that the application works. +The command must exit with status 0 and add no "variable is not set" +warnings. It does not prove that the application works. + +From the repository root, lint all Markdown with the rumdl version that +`.github/workflows/linting.yml` pins, currently 0.2.78: + +```sh +uvx rumdl@0.2.78 check --config .markdownlint.yml . +``` + +After you stage new files, check whitespace with `git diff --check origin/main`. + +When possible, start the stack from a copy outside the repository, such as +`cp -R services/my-service /tmp/my-service`, and put real values only in the +copy. A running stack writes `./config`, `./ts/state` (with the Tailscale +device key), and its data folders next to `compose.yaml`. Before you commit, +run `git status` and stage only the files you changed on purpose. -When possible, start the stack and confirm: +Then confirm: - Tailscale becomes healthy and joins the Tailnet. - The application starts and is reachable through the Tailnet. @@ -141,5 +192,8 @@ When possible, start the stack and confirm: ## Submit a pull request -Follow the pull request template. Report the checks you ran and any checks you -could not run. +Title the pull request `Service: what changes`, such as `Umami: new service`. +For several services, list them or use `All services:`. For other areas, use +`CI:`, `Docs:`, or `Template:`. Fill in the pull request template, and report +the checks you ran and any checks you could not run. Maintainers apply the +labels. diff --git a/templates/service-template/.env b/templates/service-template/.env index 2b2ad66b..9bdc286f 100644 --- a/templates/service-template/.env +++ b/templates/service-template/.env @@ -1,6 +1,5 @@ -#version=1.1 #URL=https://github.com/tailscale-dev/ScaleTail -#COMPOSE_PROJECT_NAME= # Optional: only use when running multiple deployments on the same infrastructure. +# To run a second copy, copy the folder under another name and change SERVICE. # Service Configuration # Service name (e.g., adguard). Used as hostname in Tailscale and for container naming (app-${SERVICE}). @@ -11,17 +10,18 @@ IMAGE_URL= # Network Configuration # Port to expose to local network. Uncomment the "ports:" section in compose.yaml to enable. SERVICEPORT= -DNS_SERVER=9.9.9.9 # Preferred DNS server for Tailscale. Uncomment the "dns:" section in compose.yaml to enable. +# Preferred DNS server for Tailscale. Uncomment the "dns:" section in compose.yaml to enable. +DNS_SERVER=9.9.9.9 # Tailscale Configuration -# Auth key from https://tailscale.com/admin/authkeys. See: https://tailscale.com/kb/1085/auth-keys#generate-an-auth-key for instructions. +# Auth key from https://console.tailscale.com/admin/settings/keys. See: https://tailscale.com/kb/1085/auth-keys#generate-an-auth-key for instructions. TS_AUTHKEY= # Optional Service variables # PUID=1000 -# Time Zone setting for containers -TZ=Europe/Amsterdam # See: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones +# Time Zone setting for containers. See: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones +TZ=Europe/Amsterdam # Any Container environment variables are declared below. See https://docs.docker.com/compose/how-tos/environment-variables/ diff --git a/templates/service-template/compose.yaml b/templates/service-template/compose.yaml index 33734d16..6d489adf 100644 --- a/templates/service-template/compose.yaml +++ b/templates/service-template/compose.yaml @@ -9,7 +9,7 @@ configs: services: # Make sure you have updated/checked the .env file with the correct variables. -# Every variable used in this file must be defined there. +# Every variable used in this file must be defined there, unless it has a default such as ${VAR:-value}. # Tailscale Sidecar Configuration tailscale: image: tailscale/tailscale:latest # Image to be used @@ -53,8 +53,9 @@ services: network_mode: service:tailscale # Sidecar configuration to route the service through Tailscale container_name: app-${SERVICE} # Name for local container management environment: # Variables are declared in .env file. - - PUID=1000 - - PGID=1000 + # Set PUID and PGID only for images that read them, such as LinuxServer.io images. Many images ignore them. + #- PUID=1000 + #- PGID=1000 - TZ=${TZ} #- EXAMPLE_VAR=${EXAMPLE_VAR} volumes: @@ -76,7 +77,8 @@ services: # "# Healthcheck: none possible. The image contains only , with no shell, HTTP client, or health command." # A container that runs once and exits needs no check. Remove this block and replace it with # "# Healthcheck: none needed. ." - # Avoid `pgrep -f ${SERVICE}`: it breaks when SERVICE is renamed, and many images do not include pgrep. + # Database: check over TCP (for example pg_isready -h 127.0.0.1), because the first-start server accepts only socket connections. + # Avoid `pgrep -f` with the service name: it breaks when SERVICE is renamed, and many images do not include pgrep. healthcheck: test: ["CMD", "wget", "--spider", "-q", "http://127.0.0.1:80/"] # Check if the service is responding interval: 1m # How often to perform the check From d20fef02e4855065c8e445bb66147f0bd8e30655 Mon Sep 17 00:00:00 2001 From: Jack Spiering <46534141+jackspiering@users.noreply.github.com> Date: Sat, 10 Oct 2026 13:48:38 +0200 Subject: [PATCH 2/3] Template: keep two database health check rules CONTRIBUTING.md now points to the template comments for the database case. Two rules from the removed paragraph were missing there: a database image's own HEALTHCHECK comes first, and a service that depends on the database waits with condition: service_healthy. --- templates/service-template/compose.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/templates/service-template/compose.yaml b/templates/service-template/compose.yaml index 6d489adf..dff85a0a 100644 --- a/templates/service-template/compose.yaml +++ b/templates/service-template/compose.yaml @@ -78,6 +78,8 @@ services: # A container that runs once and exits needs no check. Remove this block and replace it with # "# Healthcheck: none needed. ." # Database: check over TCP (for example pg_isready -h 127.0.0.1), because the first-start server accepts only socket connections. + # A database image's own HEALTHCHECK still comes first - when it connects over the socket, say so in the comment. + # When another service depends on the database, let it wait with `condition: service_healthy`. # Avoid `pgrep -f` with the service name: it breaks when SERVICE is renamed, and many images do not include pgrep. healthcheck: test: ["CMD", "wget", "--spider", "-q", "http://127.0.0.1:80/"] # Check if the service is responding From cc8b78934b9a1d8d6266c09f968a4aad9903d2de Mon Sep 17 00:00:00 2001 From: Jack Spiering <46534141+jackspiering@users.noreply.github.com> Date: Sat, 10 Oct 2026 14:34:45 +0200 Subject: [PATCH 3/3] Docs: say how to validate the template The documented validation command fails on the template, because SERVICE and IMAGE_URL are empty there. AGENTS.md and CONTRIBUTING.md now give the command with dummy values, as CI uses. --- AGENTS.md | 2 +- CONTRIBUTING.md | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index feb2b1e9..0433614c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,7 +20,7 @@ ScaleTail is a collection of Docker Compose stacks. Each directory in `services/ - Add, rename, or remove a service: the `services//` directory and its row in the root `README.md`. Insert a new row in alphabetical order by the bold name, and leave the other rows alone. - Make existing users act, such as with a new required secret, a moved volume, or a changed `SERVICE`: add an "Upgrading" section to the service README. `SERVICE` also names the Tailnet device, the containers, and most data folders. -- Change the template: also update `documentation/standard-setup.md` and `CONTRIBUTING.md` where they describe the same setting. +- Change the template: also update `documentation/standard-setup.md` and `CONTRIBUTING.md` where they describe the same setting. Validate it from `templates/service-template/` with `SERVICE=dummy IMAGE_URL=dummy docker compose config --quiet`, because the template leaves both values empty. ## Secrets and local data diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 85e50de7..5899e107 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -168,6 +168,10 @@ env $(grep -ohE '\$\{[A-Za-z0-9_]+:\?' compose.yaml .env | sed -E 's/^\$\{//; s/ The command must exit with status 0 and add no "variable is not set" warnings. It does not prove that the application works. +The template leaves `SERVICE` and `IMAGE_URL` empty. To validate it, run +`SERVICE=dummy IMAGE_URL=dummy docker compose config --quiet` from +`templates/service-template/`. + From the repository root, lint all Markdown with the rumdl version that `.github/workflows/linting.yml` pins, currently 0.2.78: