Skip to content

Read content.otio from otioz bundles in chunks - #2087

Open
jhodges10 wants to merge 1 commit into
AcademySoftwareFoundation:mainfrom
jhodges10:codex/limit-otioz-extraction
Open

jhodges10 wants to merge 1 commit into
AcademySoftwareFoundation:mainfrom
jhodges10:codex/limit-otioz-extraction

Conversation

@jhodges10

@jhodges10 jhodges10 commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Reading the timeline (content.otio) from an otioz bundle allocated the uncompressed size recorded in the zip file before reading any data. That size can be forged, so a 192 byte file allocated 2 GB of memory before failing.

The timeline is now read in chunks, so memory use follows the actual data. This restores the behavior of the Python adapter before #2021, which read the timeline with zipfile. The up-front allocation came in with the C++ port and was not discussed in #1901, #2021, #2060, or #2083.

Two related changes in the same function:

  • The ~2 GB limit on the timeline size is removed. It came from the 32-bit length passed to mz_zip_reader_entry_read(), not from a design decision, and the Python adapter had no such limit.
  • The timeline's CRC is now checked. It was previously ignored, so a corrupt content.otio could be read without an error.

There are no API changes and no new options. Extraction is unchanged and still has no size limits, so large bundles of uncompressed media are unaffected.

An earlier version of this PR added ReadOptions limits for the timeline size, total extracted size, and number of entries. Those were removed: fixed limits would reject legitimate large bundles (for example image sequences with one file per frame, or bundles over 16 GB), and metadata-heavy timelines commonly compress around 100x, so a compression ratio check cannot reliably tell them apart from a zip bomb.

Tests

  • New test_otioz_forged_timeline_size: a bundle whose recorded timeline size is forged to almost 2 GB reads correctly (it fails on main), and a forged CRC is an error (it is accepted on main).
  • C++ tests: all pass, including test_bundle with the 8 GB ZIP64 case.
  • Python tests: all pass except CoreVersionMapGenerationTester, which also fails on main.
  • Manual check: the forged 192 byte file used 2.1 GB of memory on main and 2.5 MB with this change.

Assisted-by: Claude Code / Claude Opus 5.5

🤖 Generated with Claude Code

@jhodges10 jhodges10 added the security Security-related improvements label Oct 30, 2026
@jhodges10

This comment has been minimized.

@jhodges10
jhodges10 force-pushed the codex/limit-otioz-extraction branch 2 times, most recently from 5be297c to 590793b Compare October 30, 2026 19:50
@codecov-commenter

codecov-commenter commented Oct 30, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.72727% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.59%. Comparing base (80c5ff3) to head (83140e1).

Files with missing lines Patch % Lines
src/opentimelineio/bundle.cpp 72.72% 3 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #2087   +/-   ##
=======================================
  Coverage   83.59%   83.59%           
=======================================
  Files         184      184           
  Lines       13632    13628    -4     
  Branches     1271     1272    +1     
=======================================
- Hits        11395    11393    -2     
+ Misses       2064     2062    -2     
  Partials      173      173           
Flag Coverage Δ
py-unittests 83.59% <72.72%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/opentimelineio/bundle.cpp 49.20% <72.72%> (-0.01%) ⬇️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80c5ff3...83140e1. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Reading the timeline from an otioz bundle allocated the uncompressed
size recorded in the zip file before reading any data. That size can be
forged, so a 192 byte file could allocate 2GB of memory. The timeline
is now read in chunks, so memory use follows the actual data, as it did
in the Python adapter before AcademySoftwareFoundation#2021.

This also removes the 2GB limit on the timeline size, which came from
the 32-bit length passed to mz_zip_reader_entry_read() rather than a
design decision, and checks the CRC of the timeline, which was ignored.

Assisted-by: Claude Code / Claude Opus 5.5
Signed-off-by: Jeff Hodges <jeff@jeffhq.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jhodges10 jhodges10 changed the title Limit OTIOZ timeline and extraction expansion Read content.otio from otioz bundles in chunks Oct 31, 2026
@jhodges10
jhodges10 force-pushed the codex/limit-otioz-extraction branch from 590793b to 83140e1 Compare October 31, 2026 01:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

security Security-related improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants