v7.0

docs/configuration/components/automation-worker.md

@@ -0,0 +1,37 @@
+---
+description: Complete reference for automation-worker configuration including workers and PDF generation settings.
+---
+
+# Configuration of automation-worker
+
+This is a cheat sheet for the possible configuration options of the [automation-worker](../../introduction/architecture.md#container-automation-worker).
+It contains all possible settings that can be configured as well as their default values.
+
+The automation-worker is a dedicated component that executes automation rules.
+It reads pending automation tasks from Redis, runs the configured actions (e.g. sending emails, running Python scripts, generating PDFs or triggering AI-powered automations), and publishes the results back to Redis.
+
+The default values provided here are best-effort (not built automatically). They will be used if no value is defined at all.
+
+??? tip "Configuration changes require a restart"
+
+    New configuration options will only apply after a restart of the automation-worker.
+
+## Environment Variables
+
+<!-- md:version 7.0 -->
+
+This section lists the environment variables read by the automation-worker.
+Please read our guide that explains how you can [customize the configuration](../customizations.md) of your SeaTable instance before you proceed.
+
+### Workers
+
+| Environment Variable | Description                                                            | Default |
+| -------------------- | ---------------------------------------------------------------------- | ------- |
+| `AUTOMATION_WORKERS` | Number of worker threads that process automation rule tasks from Redis | 5       |
+
+### PDF Generation
+
+| Environment Variable               | Description                                              | Default |
+| ---------------------------------- | -------------------------------------------------------- | ------- |
+| `CONVERT_PDF_BROWSERS`             | Number of browser processes started to generate PDF files | 2      |
+| `CONVERT_PDF_SESSIONS_PER_BROWSER` | Number of sessions per browser instance                   | 3       |

docs/configuration/components/dtable-server.md

@@ -4,7 +4,7 @@ description: Complete reference for dtable-server configuration including row li
 
 # Configuration of dtable-server
 
-This is a cheat sheet for the possible configuration options of [dtable-server](../../introduction/architecture.md#dtable-server).
+This is a cheat sheet for the possible configuration options of [dtable-server](../../introduction/architecture.md#container-dtable-server).
 It contains all possible settings that can be configured as well as their default values.
 
 The default values provided here are best-effort (not built automatically). They will be used if no value is defined at all.
@@ -17,7 +17,7 @@ The default values provided here are best-effort (not built automatically). They
 
 <!-- md:version 6.2 -->
 
-This section lists the environment variables read by [dtable-server](../../introduction/architecture.md#dtable-server).
+This section lists the environment variables read by [dtable-server](../../introduction/architecture.md#container-dtable-server).
 Please read our guide that explains how you can [customize the configuration](../customizations.md) of your SeaTable instance before you proceed.
 
 ### Automations
@@ -26,6 +26,17 @@ Please read our guide that explains how you can [customize the configuration](..
 | --------------------------------------- | -------------------------------------------------------------------------- | ------- |
 | `AUTOMATION_RATE_LIMIT_PER_BASE_MINUTE` | Limits the number of automations that can be triggered per base per minute | 1000    |
 
+### Caching
+
+<!-- md:version 7.0 -->
+
+The Golang implementation of `dtable-server` supports configuration of the base cache size.
+Eviction happens based on an LRU (least recently used) policy.
+
+| Environment Variable             | Description                   | Default |
+| -------------------------------- | ----------------------------- | ------- |
+| `DTABLE_SERVER_TOTAL_CACHE_SIZE` | Size of the base cache in MB. | 2000    |
+
 ### Persistence
 
 | Environment Variable | Description                                                             | Default |

docs/configuration/customizations.md

@@ -48,14 +48,14 @@ Add `custom-seatable-server.yml` to the `COMPOSE_FILE` variable inside your `.en
 A minimal SeaTable installation contains the following line inside the `.env` file (located at `/opt/seatable-compose/.env`):
 
 ```ini
-COMPOSE_FILE='caddy.yml,seatable-server.yml'
+COMPOSE_FILE='caddy.yml,seatable-server.yml,dtable-server.yml,automation-worker.yml'
 ```
 
 Now simpli add `custom-seatable-server.yml` to the list of filenames that should be merged by Docker Compose.
 Additional filenames should be separated by a comma.
 
 ```ini
-COMPOSE_FILE='caddy.yml,seatable-server.yml,custom-seatable-server.yml'
+COMPOSE_FILE='caddy.yml,seatable-server.yml,dtable-server.yml,automation-worker.yml,custom-seatable-server.yml'
 ```
 
 ### Step 3

docs/configuration/overview.md

@@ -22,7 +22,7 @@ it is sufficient to set/modify these variables directly inside your `.env` file.
 
 ```ini
 # components to be used
-COMPOSE_FILE='caddy.yml,seatable-server.yml' # (1)!
+COMPOSE_FILE='caddy.yml,seatable-server.yml,dtable-server.yml,automation-worker.yml' # (1)!
 COMPOSE_PATH_SEPARATOR=','
 
 # system settings

docs/installation/advanced/s3.md

@@ -103,6 +103,10 @@ Using environment variables has numerous advantages:
 To enable S3 storage for base snapshots, files, pictures and avatars, add `seatable-s3.yml` to the `COMPOSE_FILE` variable inside your `.env` file.
 This instructs Docker-Compose to extend the definition of the `seatable-server` service defined inside `seatable-server.yml`.
 
+!!! info "S3 configuration for `dtable-server` and `automation-worker` (v7.0+)"
+
+    Starting from version 7.0, `seatable-s3.yml` also passes the required S3 environment variables to the `dtable-server` and `automation-worker` containers, not only to `seatable-server`.
+    No additional configuration is required.
 
 Afterwards, you must configure a few environment variables inside your `.env` file. The following table contains examples for the most common object storage providers:
 

docs/installation/basic-setup.md

@@ -126,7 +126,7 @@ Continue setting up your SeaTable server by adjusting only three more variables.
 
     ``` python
     # components to be used
-    COMPOSE_FILE='caddy.yml,seatable-server.yml' # (1)!
+    COMPOSE_FILE='caddy.yml,seatable-server.yml,dtable-server.yml,automation-worker.yml' # (1)!
     COMPOSE_PATH_SEPARATOR=','
 
     # system settings
@@ -152,7 +152,7 @@ Continue setting up your SeaTable server by adjusting only three more variables.
     SECRET_KEY='anothertopsecret'
     ```
 
-    1.  COMPOSE_FILE is a comma-separated list **without spaces**. This list defines which components the server runs. Leave `caddy.yml` and `seatable-server.yml` at the beginning. You will add more components at a later time.
+    1.  COMPOSE_FILE is a comma-separated list **without spaces**. This list defines which components the server runs. Leave `caddy.yml`, `seatable-server.yml`, `dtable-server.yml` and `automation-worker.yml` at the beginning. You will add more components at a later time.
     2.  A [list of timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) is available on Wikipedia.
     3.  SEATABLE_SERVER_HOSTNAME is the (sub)domain (without http:// or https://) under which the server is accessible on (at least) port 80 and 443.
         <br>If you want to use a public IP address (e.g. 5.35.28.112), you can use the free service [nip.io](https://nip.io/). In this case, enter your your-ip.nip.io address (e.g. 5.35.28.112.nip.io).

docs/installation/components/python-pipeline.md

@@ -52,7 +52,7 @@ sed -i "s/COMPOSE_FILE='\(.*\)'/COMPOSE_FILE='\1,python-pipeline.yml'/" /opt/sea
     When manually adding `python-pipeline.yml` to the `COMPOSE_FILE` variable using your preferred text editor, make sure that you do not enter a space (:material-keyboard-space:). After the modification, your `COMPOSE_FILE` variable should look like this:
 
     ```bash
-    COMPOSE_FILE='caddy.yml,seatable-server.yml,python-pipeline.yml'
+    COMPOSE_FILE='caddy.yml,seatable-server.yml,dtable-server.yml,automation-worker.yml,python-pipeline.yml'
     ```
 
 #### Generate a shared secret for secure communication

docs/installation/components/uptime-kuma.md

@@ -19,7 +19,7 @@ nano /opt/seatable-compose/.env
 Your COMPOSE_FILE variable should look something like this:
 
 ```bash
-COMPOSE_FILE='seatable-docker-proxy.yml,seatable-server.yml,uptime-kuma.yml'
+COMPOSE_FILE='caddy.yml,seatable-server.yml,dtable-server.yml,automation-worker.yml,uptime-kuma.yml'
 ```
 
 ### Update the compose project

docs/installation/deployment-approach.md

@@ -34,7 +34,7 @@ You can configure components in the `.env` file, determining which ones to insta
 Example in the `.env` file:
 
 ```bash
-COMPOSE_FILE='caddy.yml,seatable-server.yml'
+COMPOSE_FILE='caddy.yml,seatable-server.yml,dtable-server.yml,automation-worker.yml'
 ```
 
 By adding or removing yml files from this list, you control the composition during runtime, eliminating the need for a single, extensive `docker-compose.yml` file.

docs/installation/faq.md

@@ -60,11 +60,19 @@ description: Troubleshooting tips for common SeaTable Server issues including st
 
 ??? question "Check dtable-server"
 
-    You can check the status of `dtable-server` by executing the following curl request:
+    You can check the status of `dtable-server` by executing the following command:
 
-    ```bash
-    curl https://$SEATABLE_SERVER_HOSTNAME/dtable-server/ping/
-    ```
+    === "Version 6.2 and earlier"
+
+        ```bash
+        curl https://$SEATABLE_SERVER_HOSTNAME/dtable-server/ping/
+        ```
+
+    === "Version 7.0 and later"
+
+        ```bash
+        docker exec -it dtable-server curl http://127.0.0.1:5000/ping/
+        ```
 
     If `dtable-server` does not reply, it might be stuck in a reboot loop.
     This can happen if too many large bases are loaded at once or too many pending operations need to be replayed.
@@ -73,7 +81,7 @@ description: Troubleshooting tips for common SeaTable Server issues including st
     As a workaround, you can increase the timeout for the healthcheck request to give `dtable-server` more time to load bases and apply pending operations.
     This can be achieved by increasing the value of the `DTABLE_SERVER_PING_TIMEOUT` environment variable (the default is `20`).
 
-    **Note:** This variable is not part of the default `seatable-server.yml` file, so you need to include an additional `.yml` file in order to configure this variable.
+    **Note:** This variable is not part of the default `seatable-server.yml` (v6.2 and earlier)/`dtable-server.yml` (v7.0 and later) file, so you need to include an additional `.yml` file in order to configure this variable.
 
 ??? question "Check nginx"
 

docs/introduction/architecture.md

@@ -20,17 +20,26 @@ flowchart TB
             B[seatable-server<br/>80]
             C[mariadb<br/>3306]
             D[redis<br/>6379]
+            E[automation-worker]
+            G[dtable-server<br/>5000]
             A<-->B
             B<-->C
             B<-->D
+            B<-->E
+            B<-->G
+            E<-->C
+            E<-->D
+            E<-->G
+            G<-->C
+            G<-->D
         end
         F@{ shape: bow-rect, label: "Storage"}
     end
 ```
 
 The numbers designate the ports used by the containers. Port 443 in the container `caddy` must be exposed. Port 80 must also be exposed when a Let's Encrypt SSL certificate is to be used.  All other ports are internal ports that are only available within the Docker network.
 
-All Docker containers read from and write to local disk. The containers `caddy`, `seatable-server`, and `mariadb` employ Docker volumes.
+All Docker containers read from and write to local disk. The containers `caddy`, `seatable-server`, `dtable-server`, `automation-worker` and `mariadb` employ Docker volumes.
 
 In an extended setup, additional, optional Docker container can be deployed to add functionality to SeaTable Server. The diagram below describes all Docker containers and their interactions required for a SeaTable Server instance integrated with office editor, Python pipeline, virus scan, and whiteboard.
 
@@ -52,6 +61,8 @@ flowchart TB
             PSc[python-scheduler]
             PSt[python-starter]
             PR[python-runner]
+            AW[automation-worker]
+            DS[dtable-server<br/>5000]
             C<-->SS
             C<-->Tld
             C<-->OO
@@ -67,6 +78,13 @@ flowchart TB
             SS<-->Tld
             SS<-->CAV
             SS<-->PSc
+            SS<-->AW
+            SS<-->DS
+            AW<-->MDB
+            AW<-->R
+            AW<-->DS
+            DS<-->MDB
+            DS<-->R
         end
         F@{ shape: bow-rect, label: "Storage"}
     end
@@ -95,7 +113,6 @@ flowchart LR
     subgraph s[SeaTable Server Container]
         A[nginx<br/>80]
         B[dtable-web<br/>8000]
-        C[dtable-server<br/>5000]
         D[dtable-db<br/>7777]
         E[api-gateway<br/>7780]
         F[dtable-storage-server<br/>6666]
@@ -104,13 +121,9 @@ flowchart LR
         A<-- / -->B
         A<-- /api-gateway -->E
         A<-- /seafhttp -->H
-        B<-->C
         B<-->D
         B<-->F
-        E<-->C
         E<-->D
-        C<-->F
-        C<-->G
         D<-->F
         D<-->G
     end
@@ -126,10 +139,6 @@ All services in the container `seatable-server` connect to the containers `maria
 
 The task of the service dtable-web is to deliver all pages except for the bases themselves. This includes essential features such as the login page, home page, system administration area, team administration, personal settings, and API endpoints. All these functionalities are provided by dtable-web, which is built on the Django framework.
 
-### dtable-server
-
-When accessing a base, you'll be directed to the base editor, which is provided by the dtable-server service. This editor loads the base's content from a JSON file, presenting it in a familiar spreadsheet interface and enabling real-time collaborative work on all data within the base. Each modification is promptly saved to the operation log (stored in MariaDB), and within minutes, these changes are persisted as a JSON file and transmitted to dtable-storage-server for storage in the attached storage system.
-
 ### dtable-db
 
 dtable-db extends the functionality of dtable-server, offering an SQL-like query language to interact with base data. Additionally, it serves as the interface for accessing the Big Data Backend.
@@ -150,6 +159,22 @@ When actions are not executed immediately but with a time delay, SeaTable employ
 
 The api-gateway is a proxy for dtable-server and dtable-db. All API calls for [base operations](https://api.seatable.com/reference/getbaseinfo) are routed through this component. It is also essential for the effective enforcement of API rate and request limits.
 
+## Container dtable-server
+
+<!-- md:version 7.0 -->
+
+When accessing a base, you'll be directed to the base editor, which is provided by the dtable-server service. This editor loads the base's content from a JSON file, presenting it in a familiar spreadsheet interface and enabling real-time collaborative work on all data within the base. Each modification is promptly saved to the operation log (stored in MariaDB), and within minutes, these changes are persisted as a JSON file and transmitted to dtable-storage-server for storage in the attached storage system.
+
+**Note:** Previously, `dtable-server` ran inside the `seatable-server` container. With version 7.0, it has been extracted to a dedicated container.
+
+## Container automation-worker
+
+<!-- md:version 7.0 -->
+
+The `automation-worker` container is a dedicated component that executes automation rules. It reads pending automation tasks from Redis, runs the configured actions (e.g. sending emails, running Python scripts, generating PDFs or triggering AI-powered automations), and publishes the results back to Redis.
+
+The automation-worker connects to the containers `mariadb` and `redis` to read (and write). In addition, it accesses the container `dtable-server` and the inner services of the `seatable-server` container (such as dtable-db and dtable-web) when an automation action interacts with a base.
+
 ## Container mariadb
 
 SeaTable uses MariaDB to store user, group and team information as well as metadata for bases. Additionally, MariaDB stores the operation log. The operation log (saved in the database table `dtable_db.operation_log`) is the base journal. It records all modifications made within all bases of the SeaTable Server instance. (While SeaTable stores all base modifications in MariaDB, it doesn't store the actual base content. Instead, bases are managed within dtable-server and regularly persisted to dtable-storage-server for long-term storage.)

docs/upgrade/extra-upgrade-notice.md

@@ -4,6 +4,44 @@ description: Version-specific upgrade notices and required configuration changes
 
 # Extra upgrade notice
 
+## 7.0
+
+Version v7.0 splits two components out of the `seatable-server` container into **dedicated containers**.
+Both of them are required, so you have to add `dtable-server.yml` and `automation-worker.yml` to the `COMPOSE_FILE` variable inside your `/opt/seatable-compose/.env` file before starting the upgraded instance:
+
+```bash
+COMPOSE_FILE='caddy.yml,seatable-server.yml,dtable-server.yml,automation-worker.yml'
+```
+
+??? warning "dtable-server now runs in its own container"
+
+    Up to v6.2, `dtable-server` was one of the services inside the `seatable-server` container.
+    Starting with v7.0, it runs in a dedicated container and must be activated by adding `dtable-server.yml` to the `COMPOSE_FILE` variable.
+
+    Please also note that customizations of `dtable-server` (e.g. environment variables such as `BASE_MAX_ROWS_LIMIT`) now belong to the `dtable-server` service instead of the `seatable-server` service.
+    If you use a `custom-seatable-server.yml` file, move these settings to a `custom-dtable-server.yml` file and add it to the `COMPOSE_FILE` variable.
+
+    Please refer to the [configuration of dtable-server](../configuration/components/dtable-server.md) for more information.
+
+??? warning "Automation rules require the new automation-worker container"
+
+    Automation rules are no longer executed inside the `seatable-server` container.
+    The new `automation-worker` container is a dedicated component that reads pending automation tasks from Redis, runs the configured actions and publishes the results back to Redis.
+
+    Add `automation-worker.yml` to the `COMPOSE_FILE` variable to activate it.
+
+    Please refer to [this page](../introduction/architecture.md#container-automation-worker) for more information.
+
+??? warning "Ping endpoints of `dtable-db` and `dtable-server` are no longer exposed"
+
+    The `/dtable-server/ping/` and `/dtable-db/ping/` endpoints are not proxied by NGINX anymore and now return a 404 error.
+    If you monitor these URLs from outside, remove them from your monitoring configuration.
+
+??? info "MariaDB starts with an increased `max-allowed-packet`"
+
+    MariaDB is now started with `--max-allowed-packet=32M`, which mitigates issues with very large operations (e.g. importing large tables from XLSX files).
+    This is part of `seatable-server.yml` and applied automatically. No action is required on your side.
+
 ## 6.2
 
 Version v6.2 requires various configuration updates.

mkdocs.yml

@@ -264,6 +264,7 @@ nav:
           - Overview: configuration/overview.md
           - Customizations: configuration/customizations.md
           - Components:
+              - automation-worker: configuration/components/automation-worker.md
               - dtable-api-gateway: configuration/components/dtable-api-gateway.md
               - dtable-db: configuration/components/dtable-db.md
               - dtable-events: configuration/components/dtable-events.md