Auto-populate a new box or line with a segmentation polygon, head/tail points or a tighter box

[mattdawkins]

Two new switches under New Annotation Settings (desktop only — they use the VIAME interactive segmentation service):

• Auto-populate mask — after a brand-new box or head/tail line is drawn, the loaded interactive segmentation model is prompted (a box: its centre; a line: foreground points at 10/30/50/70/90% along it) and the polygon is stored on the detection.
• Auto-populate points — for a box, head/tail are derived from that polygon by VIAME's polygon_keypoints (the same add_keypoints_from_mask hull-extremes code as the SAM2 head/tail utility pipeline). For a line, the box is tightened to the segmentation's bounds.

Either switch triggers the segmentation; only what is still missing on the detection is filled in (an existing polygon or line is left alone), so a user who keeps annotating is not overwritten. Hooked through a new onNewAnnotationGeometry callback in useModeManager that fires only for the first shape of a track being created.

• Stereo — with Auto compute location in other camera on, once the new box or line maps to the other camera successfully the mapped copy gets the same mask/points pass there (segmented on that camera's image). A derived head/tail is ordered to follow the source camera's line, and the stereo length is refreshed when length updates are on.

Progress is shown while annotations are being populated. Empty-mask responses, service errors, and results skipped because the annotation changed are reported in the viewer. A successful mask is saved before head/tail extraction, so a point-extraction failure does not discard it. Media paths and video timestamps are resolved for the annotation’s camera independently of the currently selected camera. Regression tests cover successive annotations, empty responses, partial success, and edits during pending requests.

Needs VIAME/VIAME#299 for the polygon_keypoints command. Helper functions (prompt construction, ring closing, bounds) are unit tested; the end-to-end flow was verified against the running service but not yet clicked through in the app.

🤖 Generated with Claude Code

Multiple polygons per fish

Preserve every returned mask component and hole under one annotation, and derive head/tail from the complete mask. Line-derived bounds cover all components. Simple single-polygon responses remain supported.

Full-mask head/tail extraction requires the companion service update in VIAME/VIAME#300. Older services report extraction failure for multi-component/hole requests while retaining the saved mask, rather than silently extracting from only one component.

Validation: 18 focused DIVE tests and nine companion Python tests pass; DIVE lint and type checking pass. Model inference was not run.

[mattdawkins]

• Stereo: a box mapped to the other camera whose segmented mask overlaps it with IoU below 0.5 is refit to the mask's bounds (fitBoxToMask, mapped copies only; a user-drawn box is never resized)
• Specs for the overlap helper and the refit/keep/user-drawn cases

[mattdawkins]

• Point-click segmentation never reached the keypoint step: the auto-populate trigger only fired for a drawn box or line. A confirmed mask on a brand-new detection (right-click/Enter, or each click in continuous mode) now fires it with a mask source, which skips prediction and runs polygon_keypoints on the stored mask
• Refining an existing detection's mask does not trigger it; an existing head/tail is left alone as before
• Stereo: on finalize the other camera's segmented copy gets the same pass, oriented to the source camera's line, and the length refresh runs when enabled
• Specs for the mode manager trigger and the mask-source populate path

[mattdawkins]

• Stereo box mapping with auto-segmentation on no longer warps the box corners (they sit on the background, so their disparities are unreliable). It waits for the source camera's segmentation, samples up to 5 points inside that mask spread across it (interiorPromptPoints), warps those, segments the other camera from them as foreground prompts, and takes that mask's bounds as the box
• The source mask is used even when only Auto-populate points is on and the mask is not stored
• Falls back to corner warping when the source segmentation, the point warp, or the other-camera segmentation fails; bulk "Warp to All" is unchanged
• Specs for the sampler (spread, holes, thin slivers) and the new prompt source

[mattdawkins]

• Point-segmentation head/tail now updates after every click that reshapes the mask (not only on finalize), on the source camera and, in stereo, on the other camera after each click's stereo segmentation
• A pass only replaces a head/tail it derived itself; once the user moves either point it stops updating
• The "annotation changed while the request was running" notice is no longer shown for a mask reshaped by the next click
• Multi-click mask guarantees (every positive click covered, no negative click covered) are in the service: VIAME PR to follow