Skip to content

Revert "Merge pull request #1649 from CREDO23/kb_git_mvp" - #1650

Merged
CREDO23 merged 1 commit into
MODSetter:mainfrom
CREDO23:revert-kb-git-mvp
Jul 30, 2026
Merged

Revert "Merge pull request #1649 from CREDO23/kb_git_mvp"#1650
CREDO23 merged 1 commit into
MODSetter:mainfrom
CREDO23:revert-kb-git-mvp

Conversation

@CREDO23

@CREDO23 CREDO23 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Reverts #1649. The git-native knowledge base goes back to the feature branch until the follow-up items are fixed; the work is preserved on CREDO23/kb_git_mvp.

Why this is safe

Nothing that merge shipped was reachable in production. Git-native behaviour requires two independent switches, and both default to off:

  • KNOWLEDGE_STORE_ENABLED, an env var that defaults to FALSE
  • workspaces.knowledge_store_enabled, a per-workspace column that defaults to false

Every code path added by #1649 is behind that pair, so with either one off the old write path runs unchanged.

Scope

119 files, all backend, plans, and docs. The frontend is untouched — surfsense_web has zero changes in either the merge or this revert, so a Vercel rebuild produces an identical app.

The tree after this commit is byte-identical to 366f9c18a^1.

Migrations

175 and 176 stay applied on the production database. Both only add columns (workspaces.knowledge_store_enabled, workspaces.last_indexed_revision, chunks.start_line, chunks.end_line), all nullable or defaulted, and no code in this tree reads them.

alembic_version has been moved back to 174 by hand so that this tree's alembic upgrade head is a no-op instead of failing to resolve a revision it no longer contains. Both revisions use ADD COLUMN IF NOT EXISTS, so re-landing this work re-applies them harmlessly over the existing columns.

Test plan

  • git diff 366f9c18a^1 <this commit> is empty
  • Reverted tree's alembic head resolves to 174
  • Deploy: migrate role runs alembic upgrade head as a no-op and exits 0
  • Smoke: workspace list, document read, and an agent turn behave as before the merge

High-level PR Summary

This PR reverts PR #1649, which introduced a git-native knowledge base system. The revert removes a feature branch that replaced the custom virtual-filesystem-over-Postgres approach with Git as the single source of truth for workspace knowledge. The feature was unreachable in production (gated behind two independent switches, both defaulting to off: KNOWLEDGE_STORE_ENABLED environment variable and workspaces.knowledge_store_enabled database column). This revert removes 119 backend files including the complete app/knowledge_store/ module, all associated plans and ADRs, migrations 175 and 176 (which only added nullable columns and will remain applied in production), the dulwich dependency, and extensive test coverage. The work is preserved on the CREDO23/kb_git_mvp feature branch for future re-landing after follow-up items are addressed. After this revert, the tree is byte-identical to the commit before the original merge.

⏱️ Estimated Review Time: 1-3 hours

💡 Review Order Suggestion
Order File Path
1 docs/adr/0001-git-native-knowledge-base.md
2 docs/adr/0002-knowledge-core-ports-and-adapters.md
3 plans/git-native-kb/00-umbrella-plan.md
4 plans/git-native-kb/00b-diagrams.md
5 plans/git-native-kb/00c-shared-contract.md
6 plans/git-native-kb/01-git-storage-core.md
7 plans/git-native-kb/02-git-working-tree-backend.md
8 plans/git-native-kb/03-commit-write-path.md
9 plans/git-native-kb/04-derived-index.md
10 plans/git-native-kb/05-migration.md
11 plans/git-native-kb/05a-seed-runbook.md
12 plans/git-native-kb/06-zero-projection.md
13 surfsense_backend/alembic/versions/175_add_workspace_knowledge_store_flag.py
14 surfsense_backend/alembic/versions/176_add_derived_index_columns.py
15 surfsense_backend/app/knowledge_store/__init__.py
16 surfsense_backend/app/knowledge_store/engines/base.py
17 surfsense_backend/app/knowledge_store/engines/git.py
18 surfsense_backend/app/knowledge_store/store.py
19 surfsense_backend/app/knowledge_store/transaction.py
20 surfsense_backend/app/knowledge_store/write_lock.py
21 surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/backends/git_tree.py
22 surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/middleware/knowledge_store_persistence/commit_turn.py
23 surfsense_backend/app/knowledge_store/index/converge.py
24 surfsense_backend/app/knowledge_store/migrate.py
25 surfsense_backend/app/services/document_revision_recorder.py
26 surfsense_backend/app/config/__init__.py
27 surfsense_backend/app/db.py
28 surfsense_backend/.env.example
29 surfsense_backend/pyproject.toml
30 surfsense_backend/app/celery_app.py
31 surfsense_backend/app/routes/documents_routes.py
32 surfsense_backend/app/routes/editor_routes.py
33 surfsense_backend/app/indexing_pipeline/indexing_pipeline_service.py
34 surfsense_backend/app/indexing_pipeline/chunk_reconciler.py
35 surfsense_backend/app/indexing_pipeline/document_chunker.py
36 surfsense_backend/app/indexing_pipeline/cache/cached_indexing.py
37 surfsense_backend/app/observability/metrics.py
38 surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/middleware/kb_persistence/middleware.py
39 surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/middleware/stack.py
40 surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/backends/resolver.py
41 surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/backends/local_folder.py
42 surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/backends/multi_root_local_folder.py
43 surfsense_backend/app/agents/chat/runtime/path_resolver.py
44 surfsense_backend/app/tasks/celery_tasks/document_reindex_tasks.py
45 surfsense_backend/app/tasks/chat/streaming/agent/event_loop.py

Need help? Join our Discord

Summary by CodeRabbit

  • Refactor

    • Simplified cloud file operations and routing by removing the alternate Git-backed workflow.
    • Standardized document path resolution, file descriptions, and write, move, and delete behavior.
    • Removed line-range metadata from indexing and chunk reconciliation.
  • Documentation

    • Removed architecture decision records, implementation plans, migration guides, and operational runbooks for the retired workflow.
  • Chores

    • Removed related configuration, scheduled maintenance and indexing tasks, database fields, migration tooling, and unused dependency support.
    • Cleaned up associated tests and observability metrics.

This reverts commit 366f9c1, keeping the git-native knowledge base in
dev until the issues queued for the next release are fixed.

Nothing shipped by that merge was reachable in production: git-native
behaviour needs both the KNOWLEDGE_STORE_ENABLED env (default FALSE) and
a per-workspace column (default false), so reverting changes no runtime
behaviour.

Migrations 175 and 176 stay applied in production; alembic_version is
moved back to 174 by hand so this tree's upgrade is a no-op. Both
revisions only add columns, and they use ADD COLUMN IF NOT EXISTS, so
re-landing this work re-applies them harmlessly.
@vercel

vercel Bot commented Jul 30, 2026

Copy link
Copy Markdown

@CREDO23 is attempting to deploy a commit to the Rohan Verma's projects Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 8f98e602-4736-4373-8a95-de88e4345f6e

📥 Commits

Reviewing files that changed from the base of the PR and between 366f9c1 and a8292f5.

⛔ Files ignored due to path filters (1)
  • surfsense_backend/uv.lock is excluded by !**/*.lock
📒 Files selected for processing (118)
  • docs/adr/0001-git-native-knowledge-base.md
  • docs/adr/0002-knowledge-core-ports-and-adapters.md
  • plans/git-native-kb/00-umbrella-plan.md
  • plans/git-native-kb/00b-diagrams.md
  • plans/git-native-kb/00c-shared-contract.md
  • plans/git-native-kb/01-git-storage-core.md
  • plans/git-native-kb/02-git-working-tree-backend.md
  • plans/git-native-kb/03-commit-write-path.md
  • plans/git-native-kb/04-derived-index.md
  • plans/git-native-kb/05-migration.md
  • plans/git-native-kb/05a-seed-runbook.md
  • plans/git-native-kb/06-zero-projection.md
  • surfsense_backend/.env.example
  • surfsense_backend/.gitignore
  • surfsense_backend/alembic/versions/175_add_workspace_knowledge_store_flag.py
  • surfsense_backend/alembic/versions/176_add_derived_index_columns.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/graph/compile_graph_sync.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/middleware/kb_persistence/middleware.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/middleware/knowledge_store_persistence/__init__.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/middleware/knowledge_store_persistence/builder.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/middleware/knowledge_store_persistence/commit_message.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/middleware/knowledge_store_persistence/commit_turn.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/middleware/knowledge_store_persistence/middleware.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/middleware/knowledge_tree/middleware.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/middleware/stack.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/runtime/agent_cache.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/main_agent/runtime/factory.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/backends/git_tree.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/backends/local_folder.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/backends/multi_root_local_folder.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/backends/resolver.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/index.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/middleware/middleware.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/tools/edit_file/index.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/tools/mkdir/index.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/tools/move_file/index.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/tools/read_file/description.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/tools/read_file/index.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/tools/rm/index.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/tools/rmdir/index.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/shared/middleware/filesystem/tools/write_file/index.py
  • surfsense_backend/app/agents/chat/multi_agent_chat/subagents/builtins/knowledge_base/middleware_stack.py
  • surfsense_backend/app/agents/chat/runtime/mention_resolver.py
  • surfsense_backend/app/agents/chat/runtime/path_resolver.py
  • surfsense_backend/app/agents/chat/runtime/references/documents/resolver.py
  • surfsense_backend/app/celery_app.py
  • surfsense_backend/app/config/__init__.py
  • surfsense_backend/app/db.py
  • surfsense_backend/app/indexing_pipeline/cache/cached_indexing.py
  • surfsense_backend/app/indexing_pipeline/chunk_reconciler.py
  • surfsense_backend/app/indexing_pipeline/document_chunker.py
  • surfsense_backend/app/indexing_pipeline/indexing_pipeline_service.py
  • surfsense_backend/app/knowledge_store/__init__.py
  • surfsense_backend/app/knowledge_store/engines/__init__.py
  • surfsense_backend/app/knowledge_store/engines/base.py
  • surfsense_backend/app/knowledge_store/engines/git.py
  • surfsense_backend/app/knowledge_store/identities.py
  • surfsense_backend/app/knowledge_store/index/__init__.py
  • surfsense_backend/app/knowledge_store/index/converge.py
  • surfsense_backend/app/knowledge_store/index/queue.py
  • surfsense_backend/app/knowledge_store/janitor.py
  • surfsense_backend/app/knowledge_store/migrate.py
  • surfsense_backend/app/knowledge_store/settings.py
  • surfsense_backend/app/knowledge_store/store.py
  • surfsense_backend/app/knowledge_store/store_path.py
  • surfsense_backend/app/knowledge_store/transaction.py
  • surfsense_backend/app/knowledge_store/write_lock.py
  • surfsense_backend/app/observability/metrics.py
  • surfsense_backend/app/routes/documents_routes.py
  • surfsense_backend/app/routes/editor_routes.py
  • surfsense_backend/app/services/document_revision_recorder.py
  • surfsense_backend/app/tasks/celery_tasks/document_reindex_tasks.py
  • surfsense_backend/app/tasks/celery_tasks/knowledge_store/__init__.py
  • surfsense_backend/app/tasks/celery_tasks/knowledge_store/drift_monitor_task.py
  • surfsense_backend/app/tasks/celery_tasks/knowledge_store/index_tasks.py
  • surfsense_backend/app/tasks/celery_tasks/knowledge_store/janitor_task.py
  • surfsense_backend/app/tasks/chat/streaming/agent/event_loop.py
  • surfsense_backend/pyproject.toml
  • surfsense_backend/scripts/migrate_knowledge_store.py
  • surfsense_backend/tests/integration/conftest.py
  • surfsense_backend/tests/integration/knowledge_store/__init__.py
  • surfsense_backend/tests/integration/knowledge_store/conftest.py
  • surfsense_backend/tests/integration/knowledge_store/index/__init__.py
  • surfsense_backend/tests/integration/knowledge_store/index/test_converge.py
  • surfsense_backend/tests/integration/knowledge_store/index/test_drift_monitor.py
  • surfsense_backend/tests/integration/knowledge_store/index/test_sweep.py
  • surfsense_backend/tests/integration/knowledge_store/test_commit_turn.py
  • surfsense_backend/tests/integration/knowledge_store/test_document_recorder.py
  • surfsense_backend/tests/integration/knowledge_store/test_fleet_runner.py
  • surfsense_backend/tests/integration/knowledge_store/test_migrate.py
  • surfsense_backend/tests/integration/knowledge_store/test_migrate_placement.py
  • surfsense_backend/tests/integration/knowledge_store/test_paused_turn_keeps_its_copy.py
  • surfsense_backend/tests/integration/knowledge_store/test_store.py
  • surfsense_backend/tests/integration/knowledge_store/test_write_lock.py
  • surfsense_backend/tests/integration/knowledge_store/test_writer_guards.py
  • surfsense_backend/tests/integration/test_document_versioning.py
  • surfsense_backend/tests/unit/agents/new_chat/test_mention_resolver.py
  • surfsense_backend/tests/unit/agents/new_chat/test_path_resolver.py
  • surfsense_backend/tests/unit/indexing_pipeline/test_chunk_reconciler.py
  • surfsense_backend/tests/unit/indexing_pipeline/test_chunk_spans.py
  • surfsense_backend/tests/unit/indexing_pipeline/test_document_hashing.py
  • surfsense_backend/tests/unit/knowledge_store/__init__.py
  • surfsense_backend/tests/unit/knowledge_store/conftest.py
  • surfsense_backend/tests/unit/knowledge_store/engines/__init__.py
  • surfsense_backend/tests/unit/knowledge_store/engines/test_git.py
  • surfsense_backend/tests/unit/knowledge_store/index/__init__.py
  • surfsense_backend/tests/unit/knowledge_store/index/test_index_tasks.py
  • surfsense_backend/tests/unit/knowledge_store/index/test_queue.py
  • surfsense_backend/tests/unit/knowledge_store/test_janitor.py
  • surfsense_backend/tests/unit/knowledge_store/test_settings.py
  • surfsense_backend/tests/unit/knowledge_store/test_transaction.py
  • surfsense_backend/tests/unit/middleware/test_commit_message.py
  • surfsense_backend/tests/unit/middleware/test_filesystem_backends.py
  • surfsense_backend/tests/unit/middleware/test_git_tree_backend.py
  • surfsense_backend/tests/unit/middleware/test_git_tree_tool_staging.py
  • surfsense_backend/tests/unit/middleware/test_knowledge_store_persistence_builder.py
  • surfsense_backend/tests/unit/middleware/test_knowledge_tree.py
  • surfsense_backend/tests/unit/middleware/test_read_file_description.py

📝 Walkthrough

Walkthrough

The PR removes the Git-native knowledge-store architecture, implementation, configuration, migrations, scheduled jobs, agent middleware, line-span indexing support, documentation, and related tests. Cloud filesystem routing now uses the existing Postgres backend path.

Changes

Knowledge-store removal

Layer / File(s) Summary
Architecture, storage, and operational cleanup
docs/adr/*, plans/git-native-kb/*, surfsense_backend/app/knowledge_store/*, surfsense_backend/alembic/versions/*, surfsense_backend/app/celery_app.py
Git-native ADRs, plans, storage engines, transactions, locks, migration tooling, index tasks, janitors, drift monitoring, and migrations are deleted.
Agent filesystem integration
surfsense_backend/app/agents/chat/multi_agent_chat/..., surfsense_backend/app/tasks/chat/streaming/agent/event_loop.py
Knowledge-store flags, persistence middleware, GitTreeBackend routing, and working-copy safety-net commits are removed.
Document paths and versioning
surfsense_backend/app/agents/chat/runtime/..., surfsense_backend/app/routes/documents_routes.py, surfsense_backend/app/tasks/celery_tasks/document_reindex_tasks.py
Document paths use doc_to_virtual_path; metadata-based path conversion and Git-backed version restrictions are removed.
Chunk indexing simplification
surfsense_backend/app/db.py, surfsense_backend/app/indexing_pipeline/*
Chunk line spans and workspace index revisions are removed; embedding and reconciliation operate on text and position tuples.
Tests and dependencies
surfsense_backend/tests/*, surfsense_backend/pyproject.toml, surfsense_backend/.env.example
Knowledge-store tests and fixtures are deleted or simplified, and the dulwich dependency and environment settings are removed.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

Suggested reviewers: modsetter, anishsarkar22

✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Fix failing CI checks
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@CREDO23
CREDO23 merged commit 4e9225b into MODSetter:main Jul 30, 2026
4 of 10 checks passed
CREDO23 added a commit that referenced this pull request Jul 31, 2026
Restores the work from #1649, which was reverted on main in #1650 to keep
it out of the last release. Content is identical to that merge.

This is a revert of the revert (a8292f5) rather than a merge of
kb_git_mvp, deliberately. Merging the branch would make its commits an
ancestor of both dev and main; main's side deleted those files, so the
next merge between the two branches would silently delete them again. A
revert carries the content without the history, so dev and main share no
ancestor that knows about these files, and the eventual dev -> main merge
sees them as added on one side only and keeps them.

Git becomes the source of truth for knowledge base content; Postgres and
pgvector become a derived, rebuildable index. Both switches guarding the
new path default to off: the KNOWLEDGE_STORE_ENABLED env var and the
per-workspace knowledge_store_enabled column, so merging this changes no
runtime behaviour.

Migrations 175 and 176 only add columns and use ADD COLUMN IF NOT EXISTS.
They are already applied on production, where alembic_version was moved
back to 174 during the revert, so they will re-run harmlessly.

Verified on this branch: 0 conflicts against dev, every app.* import in
the restored files resolves, 2251 unit tests pass. The one collection
error (platforms/google_maps) is missing a fixture that is untracked on
both dev and main, and predates this change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant