Fix performance issues caused by inefficient DB usage

[varmar05]

Several endpoints were hammering the database with query counts that scaled with the number of files or projects in a request (N+1 problem).

GET /v1/resource/history reduced from 6 + 4N queries to a fixed 7.

• FileHistory.changes() now finds the effective since boundary with a single SQL query (instead of fetching all history and slicing in Python), and eagerly loads version and version.project columns via contains_eager + load_only so no lazy loads occur per row
• FileHistory.diff converted from property (DB query on every access) to cached_property (computed once, stored in dict)
• 'expiration' field excluded from the response schema (not relevant anyway), eliminating the per-row version.project lookup it required

GET /v1/project/<name>?since= reduced from 3 + M + N×M queries to a fixed 5 (where M = versioned files, N = versions in range):

• All FileHistory rows across all versioned files are fetched in a single query with between on version name, then partitioned by file in Python with stop-at-CREATE logic applied per file
• FileDiff records batched in a single query across all files in one pass

POST /v1/project/by_names reduced from up to 100 + 2N queries to 3.

• Workspace names are extracted from the request as a unique set and resolved in a single get_by_names() call (one WHERE name IN (...) query) instead of one get_by_name() call per project entry
• Projects are fetched with a single WHERE (workspace_id, name) IN (...) query using tuple_().in_()
• Project member usernames are fetched in a single query joined from ProjectUser
• get_by_names() added to the Workspace interface

GET /v1/project/paginated reduced from 3 + 3N to 5 (N is paging param)

• selectinload(Project.project_users); _precompute_has_conflict(), plus fixed a pre-existing bug where ws_ids was collected from the unpaginated query object instead of result

GET /v1/project/versions/paginated/<namespace>/<project_name> reduced from 3 + 2N to 4 (N is paging param)

• joinedload(ProjectVersion.project).load_only(...) added to query; workspace names batch

Change strategy for serialization schemas:

• ProjectListSchema.access; insert_usernames uses the pre-fetched users_map instead of issuing a User query per project
• ProjectListSchema.get_has_conflict(); "_has_conflict pre-populated, skipping the expensive Project.files raw SQL query
• ProjectVersionListSchema.namespace; that reads from context["workspaces_map"]
• ProjectVersionListSchema._changes(); "_changes_count" pre-populated, skipping the per-version changes_count() SQL query

[coveralls]

Coverage Report for CI Build 25362674405

Coverage decreased (-0.04%) to 93.106%

Details

• Coverage decreased (-0.04%) from the base build.
• Patch coverage: 8 uncovered changes across 3 files (111 of 119 lines covered, 93.28%).
• 3 coverage regressions across 1 file.

Uncovered Changes

File	Changed	Covered	%
server/mergin/sync/public_api_controller.py	81	76	93.83%
server/mergin/sync/models.py	16	14	87.5%
server/mergin/sync/interfaces.py	2	1	50.0%

Coverage Regressions

3 previously-covered lines in 1 file lost coverage.

File	Lines Losing Coverage	Coverage
server/mergin/sync/models.py	3	92.31%

---

Coverage Stats

Relevant Lines:	9892
Covered Lines:	9210
Line Coverage:	93.11%
Coverage Strength:	0.93 hits per line

---

💛 - Coveralls

[Copilot]

Pull request overview

This PR aims to reduce N+1 database access in the v1 sync API by batching history, project, workspace, and diff lookups instead of resolving them row-by-row.

Changes:

• Reworks file-history loading for /v1/resource/history and /v1/project?...&since= to fetch histories and diffs in bulk.
• Refactors /v1/project/by_names to batch-resolve workspaces, projects, and project members.
• Updates the public v1 schema/tests to stop exposing the expiration field in history responses.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
server/mergin/tests/test_project_controller.py	Updates history endpoint expectations to remove expiration.
server/mergin/sync/workspace.py	Adds get_by_names() to the global workspace handler.
server/mergin/sync/public_api.yaml	Removes expiration from documented v1 history payloads.
server/mergin/sync/public_api_controller.py	Batches history/diff/project/workspace loading in several public API endpoints.
server/mergin/sync/models.py	Changes FileHistory.diff to cached_property and adjusts FileHistory.changes() query loading.
server/mergin/sync/interfaces.py	Extends the workspace handler interface with get_by_names().

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.