Skip to content

Add image annotation support with region notes and viewer tools - #156

Open
devnull03 wants to merge 25 commits into
mainfrom
feat/image-annotations
Open

devnull03 wants to merge 25 commits into
mainfrom
feat/image-annotations

Conversation

@devnull03

@devnull03 devnull03 commented Oct 3, 2026 •

Copy link
Copy Markdown
Owner

Summary

This PR implements image annotation capabilities for the Qrate app, allowing users to mark regions on file images, write notes about them, and manage those annotations through the viewer and Details panel. The feature includes a new annotate mode with drawing and selection tools, region persistence, and integration with the existing diagnostics and history systems.

Key Changes

Viewer & Annotation System

  • Added crates/workspace/src/viewer/annotate.rs — core annotation state management (Annotating, Tool, Mark, Picked, Lit, Reveal) and region drawing/selection logic
  • Added crates/workspace/src/viewer/regions.rs — geometric calculations for region positioning, rotation, and interaction (drag, resize, pin placement) on the displayed page
  • Extended crates/workspace/src/viewer/mod.rs to accept row: Option<RowId> when opening viewers, enabling region notes to be drawn and displayed per row
  • [ / ] step and zoom between a page's regions; the page strip counts each page's regions; a chip warns when the file's size no longer matches its regions; a missing file with regions opens on a dashed stand-in page with Locate file…

Diagnostics & Notes

  • Extended crates/diagnostics/src/lib.rs with NoteMeta struct carrying region, kind, and edit history for authored notes; computed findings remain stateless
  • Added crates/diagnostics/src/note_card.rs — reusable note hover card component showing author, timestamp, and edit status
  • Updated crates/settings/src/project.rs schema: __notes table now includes note_id, region, and kind columns; migration preserves existing notes
  • Updated crates/settings/src/history.rs with Origin::Drawn variant to track region annotation changes
  • Diagnostics::restore_note puts an annotation back from a history entry, including a deleted one

Details Panel & UI

  • Extended crates/workspace/src/panels/details.rs with notes filtering (NotesFilter), per-author filtering, note editing in-place, and region crop display
  • Added subscriptions to follow viewer picks and diagnostics changes, with scroll-to-reveal for selected regions
  • Integrated note card rendering with region thumbnails via new crates/preview/src/crop.rs module

Responsive Titlebars & Panel Headers

  • Added a shared responsive header wrapper for the main window titlebar and dock panel headers; controls collapse into standard dropdown menus when space is limited
  • Kept native window controls visible, truncated the center title, and sized and positioned the main hamburger to match Zed's compact titlebar
  • Preserved panel header alignment and fixed hamburger hit testing so the native window drag region does not intercept clicks
  • Dropdowns use the same actions, checked states, and disabled states as their inline controls, including app menus and panel controls
  • Added panel-owned collapse stages through HeaderStage, PanelHeaders, and PanelHeaderRegistry; the wrapper measures each stage and selects the first that fits
  • Problems collapses progressively: secondary filters move into the menu first, then severity tab names disappear while icons/counts remain, then all controls move into the menu
  • Added an eight-pixel expansion buffer to prevent breakpoint flicker, label tooltips for compact tabs, and focus restoration when resizing dismisses a dropdown
  • Documented the header model in docs/dev/responsive-headers.md

Supporting Changes

  • Added crates/preview/src/crop.rs — extracts and caches region crops from page images for note card display
  • Updated crates/table/src/note.rs to work with the new NoteMeta structure
  • Updated crates/workspace/src/pop_out.rs to pass row context when opening viewers
  • Updated crates/workspace/src/panels/history.rs with annotation() helper, crops on annotation entries, and Restore This Annotation
  • Gallery tiles count their annotations in the bottom-left corner
  • crates/ai/src/traits.rs: allow clippy 1.99's double_must_use, which fires on async_trait's expansion and failed CI on main as well
  • Updated documentation (docs/files-and-photos.md, docs/grid.md, docs/history.md, docs/shortcuts.md) with annotation workflows and keyboard shortcuts
  • Added new SVG icons for annotation tools: map-pin.svg, pencil.svg, rectangle.svg, mouse-pointer-2.svg, square-pen.svg, message-square.svg

Design Mockups

  • Added the design handoff under tmp/design/ (the .dc.html mockups, support.js, and the design chat history)

Implementation Details

  • Region Coordinates: Regions use Region::SCALE (10000) for sub-pixel precision; all viewer calculations work in fractions of the displayed page to handle zoom and rotation
  • Tool State: Annotation mode is app-global (Annotating global) so switching rows preserves the active tool and visibility state
  • Note Identity: Every authored note receives a stable NoteId on first save; computed findings have no identity and are recomputed on project open
  • Crop Caching: Region crops are decoded once and held in memory; the cache key includes file path and region bounds
  • History Integration: Region draws, moves, and resizes are logged as Origin::Drawn changes and can be restored from History

Verification

scripts/ci.sh (fmt, clippy -D warnings, cargo test --workspace) passes locally on Linux, and clippy and fmt pass on stable 1.99. The earlier annotation implementation passed CI on Windows, macOS and Linux (run 225). The responsive header changes passed cargo check -p app --target-dir target/rust-analyzer, Rust formatting checks, and git diff --check; visual resize behavior still needs review in the running app.

Notes were addressed only by their location, so a second note at one
place could not be told apart: set_note removed every note there, and a
logged change could not say which one it touched. Region notes share
their row's location, which would have made that live.

- __notes gains note_id and region (JSON) columns, added in place like
  the provenance columns; a file from before ids numbers its notes on read
- Region: a rectangle or pin in 1/10000ths of the upright page
- Diagnostic.filed becomes note: Option<NoteMeta { id, filed, region }>
- every note write goes through one id-keyed path; file_note, edit_note,
  move_note and apply_note (restore) join set_note, which now corrects
  only the region-less note at its location
- Change::Note records the id and region before and after; entries
  logged before ids still read
- Origin::Drawn for region moves, and History names region changes
The spreadsheet-note import numbers its notes as project::NoteId, which
the wizard's project module did not re-export, so the crate did not
build.
The annotation composer offers Note, Transcription and Question. A note
now keeps that choice, or none when its author picked nothing.

- __notes gains a nullable kind column, added in place like region
- NoteKind on StoredNote and NoteMeta; an unknown stored kind is logged
  and read as none
- Change::Note records kind_before and kind_after, so a restore puts the
  kind back with the text
- file_note and edit_note take the kind; set_note keeps the one it finds
- the store's writes carry text, region and kind as one Body
- a spreadsheet export names a transcription or question in the note's
  scope line
The note hover card and the Notes panel say "edited" and who edited a
note last. Nothing stored that, but every rewording is already in the
log, so it is read from there.

- history::note_edits: each note's newest change of text or kind; a
  move is not a rewording, and filing is not editing
- NoteMeta.edited, read once when the project opens and kept current by
  every write after that
- a note is stamped to the minute from the local clock, the same one
  History reads, rather than with SQLite's date alone
- project::today goes, its one caller gone
Hovering a cell, row number or header with a note now opens one note
card instead of the plain tooltip: the whole note, then its author (or
Unsigned), when it was filed, and "edited" once it has been reworded.
The viewer will use the same card on a region.

- diagnostics::note_card, the one builder both crates mount
- what the old tooltip said, the column's description and the findings
  here, moves under a divider below the note
- a cell's own text, when it was cut short, joins the description
- a row whose only notes are on its file shows the first of those
- places without a note keep the text tooltip
The viewer now knows which row it shows, and draws that row's region
notes over the page wherever the file is open: the overlay, the centre
panel and the pop-out.

- viewer::regions: region to screen and back through zoom, pan, the
  letterbox and rotation, with hit testing, badge placement and handle
  dragging, all gpui-free and tested
- each box gets a dark outer and light inner stroke and a numbered
  badge in its top-left corner; a pin is a numbered dot
- regions number in filing order; larger ones paint first, and hover
  picks a pin, then the smallest box
- a badge that would cover another slides right, then joins its pill
- hovering lightens the fill and opens the note card, below the region
  or above it near the bottom; others drop to 40% when they overlap
- N, or the eye in the toolbar, hides and shows them for every viewer
A turns annotate mode on, from the toolbar's new pen or the key, in
every viewer at once. A tool pill appears on the stage's left edge.

- Rectangle drags out a box with a dashed outline and the rest of the
  page dimmed; a click drops a pin instead. Pin only drops pins.
- on release the composer opens beside the region, right if it fits:
  the row and page, the note, optional Note / Transcription / Question
  chips, Ctrl+Enter or Save, Esc or Cancel to throw the region away
- the first save with no name set asks once, inline, "Sign your notes
  as…"; a name signs this note and every later one, Skip files it
  unsigned, and either way the composer never asks again
- the foot of the stage says how to start until the page has a region
- annotation glyphs from Lucide join the app's own icons
With the Select tool a click picks a region: its badge turns the theme
primary and it grows eight handles. Dragging inside moves it, a handle
resizes it, and both stop at the page's edge. Delete asks first, naming
the region's author, and History keeps a copy to restore.

- Esc steps back one layer at a time: composer, confirmation, drag,
  then the selection
- the selection is published app-wide for the Notes panel to follow
Note cards show the part of the page a region covers. The crop is cut
from the capped copy the cards already decode and held in memory only,
never written to the project; a file that can't be read fails the
source, so a card can draw its own fallback.
The design prototypes and their transcripts, kept beside the code while
the feature is built. Not part of the app.
[ and ] zoom to the previous or next region on the page, with a hint
saying which one is showing. A Reveal global lets the Notes panel turn a
viewer to a region's page and zoom there once the page is laid out.

The page strip counts each page's regions, and a chip says when the
file's size no longer matches the one its regions were drawn on.

A row whose file is gone but has regions still opens: shown_file names
where the file should be, and the viewer draws the regions dashed on a
page of their stored size, with Locate file… and annotating turned off.
Region notes in Details show a crop of their region, or a mini-map when
the file is missing, with #n, page, kind, date, author and "edited".
Hovering lights the region in the viewer and offers edit, history and
delete; clicking reveals it. All / On image / On fields and Mine filter
the list, and a region picked in the viewer is scrolled into view.

History names region changes as annotation added, moved, resized,
edited or deleted, with the crop, and before and after for a move.
The #[must_use] comes from async_trait's expansion of DataReviewer, so
CI's stable clippy failed every platform before reaching anything else.
Diagnostics::restore_note puts a note back the way a change found it:
its region, words and kind, or the whole note under its own id when the
change deleted it. The History panel offers Restore This Annotation (or
Note) on a single note change, and the Details history's restore button
now covers annotation entries as well as cell values.
Copilot AI balanced review requested due to automatic review settings October 3, 2026 07:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants