feat: implement automated database replication and fix RLS subquery & test issues

[mono-xyxy]

Walkthrough - Automated Database Replication Plugin

We have successfully implemented and verified the Automated Database Replication Plugin for StarbaseDB. We also identified and fixed pre-existing state pollution and subquery parsing bugs in the Row Level Security (RLS) test suite and code.

🚀 Changes Made

1. Database Replication Plugin

• New Plugin: Created index.ts implementing the ReplicationPlugin.
• Incremental Polling: Added support for polling both via integer IDs and via updated_at timestamps. If updated_at is available, we use a robust composite tie-breaker sorting query (WHERE updated_at > ? OR (updated_at = ? AND id > ?) ORDER BY updated_at ASC, id ASC) to prevent paging skips on duplicate timestamps.
• Watermark Storage: Persisted sync watermark state in tmp_replication_state with last_synced_id stored as TEXT to natively support UUID/string primary keys alongside auto-incrementing integers.
• Schema Auto-Migration: Implemented dynamic SQLite table creation and column migrations. The plugin checks the source table schema and compares it to the local SQLite table. Any new columns added to the external source are dynamically added to the local table via ALTER TABLE ADD COLUMN.
• Deletion Syncing: Implemented dynamic hard deletion synchronization (syncDeletions) that compares source primary keys to local SQLite primary keys and deletes any rows that were hard deleted in the external source.
• Concurrency & Alarms: Integrated with Durable Object Alarms to yield CPU resources and reschedule tasks in 1-second increments when pagination limits are reached. Controlled concurrency with an in-memory lock.
• Core Integration: Registered ReplicationPlugin in index.ts and configured the environment variables in wrangler.toml and README.md.

2. RLS Bug Fixes

• Test State Pollution: Modified index.test.ts to reset mockConfig.role to 'client' in a beforeEach block, preventing test pollution and ensuring that other test suites do not bypass RLS.
• Subquery Traversals: Modified index.ts to check expr.ast when recursively traversing subqueries in the AST. Previously, subqueries inside FROM and JOIN clauses were skipped.
• Wildcard Authorization: Fixed applyRLSToAst to correctly handle * wildcard policy permissions during INSERT checks.

---

🧪 Validation Results

Automated Unit Tests

We wrote a comprehensive unit test suite in index.test.ts testing:

• Plugin initialization and registration.
• PostgreSQL schema mapping and automatic SQLite table creation.
• Incremental watermark-based tie-breaker polling.
• Type mappings (Date serialization, Boolean-to-Integer coercion, and JSON object serialization).
• Deletion synchronization (handling hard deletes from the external source).

All 161 tests in the repository pass green:

npx vitest run

 Test Files  20 passed (20)
      Tests  161 passed (161)
   Start at  10:30:18
   Duration  5.22s

[mono-xyxy]

ezgif-252daf1324b4047c.mp4