feat(medcat-trainer): Solr backed full text search of clinical notes

[tomolopolis]

Summary

The Clinical Notes sidebar search on the annotate entities page could only prefix-match note names, and only among docs already loaded in the browser. This adds a Text search mode backed by a Solr full text index of each dataset's documents, and makes the Clinical Notes panel resizable.

Indexing (api/document_search.py)

• One shared SolrCloud collection (DOCUMENT_SEARCH_COLLECTION, default medcattrainer_documents) with fields dataset_id, doc_id, name, and text (text_general). The collection and fields are created on first use.
• The Dataset post_save signal (which covers the admin upload, the create-dataset API, and project import) schedules the index_dataset_task background task once the upload commits. The task delete-by-queries the dataset's docs and re-adds them in batches of 500, so re-uploads replace rather than duplicate.
• Deleting a dataset schedules remove_dataset_task.
• New Datasets admin action, "Index selected datasets for text search", for datasets uploaded before this change.
• DOCUMENT_SEARCH_INDEX_ENABLED=0 turns indexing off.
• collections_available excludes the document collection, so it never shows up as a concept index.

Search (GET /api/search-documents/?dataset=&text=&page=)

• Runs a phrase query on text (case-insensitive via the analyzer), filtered to the dataset, 50 results per page. Results use the same count/next/results shape as DRF and include full document objects.
• Each result carries highlight segments from Solr's unified highlighter. They are split on control-character markers server-side, so the frontend never renders HTML.
• Falls back to a DB icontains match (search_backend: "db") if Solr is unreachable or the dataset isn't fully indexed yet, i.e. the Solr count differs from the DB count.
• Requires authentication.

Frontend

• DocumentSummary.vue: Name / Text toggle under the search box. Name mode is unchanged. Text mode calls /api/search-documents/ (300ms debounce, stale responses ignored), shows a match count, pages with "More Results", and renders the Solr highlight snippet. Without a Solr highlight (DB fallback), it builds a snippet client-side.
• The Clinical Notes panel now sits in a resizable left <pane> of the same splitpanes as the clinical text and the ConceptSummary sidebar (15% default, 10% minimum). The fixed 175px width is replaced with a flex column, so the note cards fill the pane and the list scrolls under the title and search box.
• Replaced the Vue 2 beforeDestroy hook (never fires in Vue 3) with beforeUnmount.

The earlier text__icontains / name__icontains filters on /api/documents/ are kept.

resizable-clinical-notes-pane.mp4

Testing

• Django: full suite passes (293 tests). New test_document_search.py covers indexing requests and batching, search ordering, highlights, pagination, both fallbacks, validation, auth, and the upload/delete signals scheduling tasks on commit.
• Live Solr: LiveSolrDocumentSearchTests (opt-in with MCT_SOLR_LIVE_TESTS=1) passes against Solr 8.11 in cloud mode (-c, as in docker-compose).
• Ran a real manage.py process_tasks worker; it indexed an uploaded dataset into medcattrainer_documents.
• Vitest: full suite passes (167 tests), including a layout test that DocumentSummary, ClinicalText, and the concept sidebar are the three panes.
• Manual browser check with a seeded 30-note project: dragging the new left splitter widened the Clinical Notes pane from about 175px to about 410px with the cards filling it; the right concept splitter still resizes; a Text search for "chest pain" returned 18 highlighted Solr matches and clicking one loaded it.

_{To show artifacts inline, enable in settings.}