stats.json omits all image/video keys when ExternalVideos/H264Remux is enabled #4

Summary

When the writer runs with ExternalVideos / H264Remux, meta/stats.json silently loses every image and video key, while meta/info.json still declares those features. The two files become internally inconsistent, and the dataset is unloadable by LeRobot.

Impact

Downstream lerobot-train fails at make_dataset:

KeyError: observation.images.camera_high

Nothing fails at write time: Merge succeeds, ValidateOutputIntegrity succeeds, and the Inspector reports OK. The defect only surfaces hours later inside a GPU training container.

Reproduction

A v3.0 dataset written through the staging writer with:

lerobot.NewStagingWriter(ctx, lerobot.StagingConfig{
    Version:        ver,
    Features:       feats,   // includes 4 observation.images.* video features
    UseVideos:      true,
    Stats:          lerobot.StatsSampled,
    ExternalVideos: true,
})

Observed on a 10-episode, 4-camera rosbag2 MCAP source (meta/info.json declares observation.images.camera_high, camera_left_wrist, camera_left_wrist_02, camera_right_wrist):

• meta/info.json — 4 video features, dtype=video, shape=[480,640,3], names=["height","width","channels"]
• meta/stats.json — only action, observation.state, observation.gripper, timestamp, and index columns. No image keys at all.

The same source through the decode path produces the image keys correctly.

Suspected mechanism

internal/stats/images.go computes image statistics by decoding sampled frames. On the external-video path the MP4 is produced by remux, so frames are never decoded into images, the statistics accumulator never sees them, and the keys are dropped without a warning.

This looks like an unintended consequence of the remux fast path rather than a deliberate contract: info.json and stats.json are written from the same feature set, so they should not be able to disagree.

Expected

With ExternalVideos: true, meta/stats.json covers every image and video feature declared in meta/info.json, with mean / std / min / max shaped (C, 1, 1), consistent with the decode path within the existing ModeSampled tolerance.

If computing statistics on this path necessarily requires decoding the remuxed video, that is a real trade-off worth discussing — but the current behavior (silently emitting an inconsistent file pair) is worse than either choice.

Notes

• Reproduced against v1.2.1; the relevant code appears unchanged on main.
• A regression test asserting key coverage with ExternalVideos: true would catch this class permanently.
• Happy to contribute the fix and the test.