[ArnavAgrawal03]

Core skipped several common attachment types as unsupported: videos (.mp4, .webm), Outlook and MIME emails (.msg, .eml), SQL and other scripts, and .xlsb workbooks. This makes Core read all of them.

Changes

• Code and config files as text. .sql .py .ps1 .sh .bat .cmd .js .ts .css .ini .cfg .conf .toml .properties skip Docling, like .txt. New core/parser/text_files.py honors UTF-8 and UTF-16 byte order marks (SSMS saves .sql as UTF-16 LE), then tries UTF-8, then cp1252.
• .ts is not a video. mimetypes maps .ts to video/mp2t, so _is_video_file no longer guesses from the filename for text extensions, only from content.
• Email. New core/parser/email_parser.py. .eml uses stdlib email; Outlook .msg reads its OLE property streams with olefile (BSD, new dependency). I avoided extract-msg because it's GPL-3.0. Both render From/To/Cc/Date/Subject, the body (HTML converted to text when there's no plain body), and the attachment names. Attachment contents are not parsed. A .msg that isn't an OLE file fails with the reason.
• .xlsb goes through the LibreOffice legacy-Office conversion to .xlsx, like .xls.
• Video transcription with OpenAI. New [parser] transcription_model names a registered model, e.g. openai_gpt_transcribe = { model_name = "openai/gpt-transcribe" }. Startup validates it, as for image_ocr_model.
  • ffprobe checks for an audio track; silent screen recordings skip transcription.
  • ffmpeg writes mono 16 kHz 32 kbps MP3 in 120 s pieces (about 0.5 MB each, under OpenAI's 25 MB limit). Each piece is timed by its real start from ffmpeg's segment list.
  • The call goes through the OpenAI SDK, not LiteLLM. LiteLLM 1.74.7 sends verbose_json to OpenAI models without "gpt-4o" in the name, and gpt-transcribe rejects that with HTTP 400 (checked live, see Testing). It sends json for gpt-* models and verbose_json with segment timestamps for whisper*.
  • Rate limits, timeouts and 5xx raise ConnectionError so the worker retries. A broken audio track logs a warning and the frames still get indexed.
  • transcription_model takes precedence over ASSEMBLYAI_API_KEY.
• Fewer frame captions. New [parser.vision] frame_change_threshold (default 0.025; 0 turns it off). A sampled frame is skipped when no tile of an 8x8 grid over a 128x72 grayscale copy changed by more than the threshold since the last described frame. Measured on synthetic frames: a moved cursor scores 0.015, a new line of text 0.054, compression noise 0.004.
• Timestamps in the text. Frame and transcript lines start with [mm:ss] (h:mm:ss past an hour). video_metadata gains frame_change_threshold and frames_unchanged.
• Docs: DOCKER.md sections 6 (code, config, email) and 7 (video), and commented examples in both toml files.

Testing

• test_text_and_email_parsing 18, test_video_transcription 13, test_video_parser 5, test_legacy_office_parsing 8 passed and 1 skipped (needs LibreOffice), test_image_ocr_model 15, test_docling_picture_text 3, test_week1_ingest_fixes 20, test_logs_no_egress 1: all passed.
• test_document_download.py and test_reranker.py hang locally, the same way on unmodified main, so I didn't get a full-suite run.
• The ffmpeg tests run real ffmpeg/ffprobe on generated videos: an mp4 with audio and a silent VP9 webm (also checks that cv2 reads webm).
• Live OpenAI calls through VideoParser.get_model_transcript on generated speech:
  • 8 s mp4 with gpt-transcribe and json: exact transcript.
  • 4 min 24 s mp4 with speech at 0 s, 125 s and 250 s: three lines timed 0.0, 120.02 and 240.01.
  • gpt-transcribe with verbose_json: 400 response_format 'verbose_json' is not compatible with model 'gpt-transcribe-api-ev3'. Use 'json' or 'text' instead.
  • whisper-1 (verbose_json): segment timestamps, but it made up repeated lines through long silences. gpt-transcribe didn't, so the docs recommend it. whisper-1 shuts down Feb 26, 2027.
• .msg parsing was smoke-tested on six real Outlook messages; every field came out. No content from them is in the repo.
• ruff passes.

Not in this change

• Attachments inside emails are listed by name, not parsed.
• A silent video with frame captioning off yields no text and fails as empty.
• Audio-only files (.mp3, .wav) don't take the video path.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}