Skip to content

feat(transcoder): rewrite on libav (node-av) bindings - #69

Draft
devchaudhary24k wants to merge 4 commits into
devfrom
feat/transcoder-libav
Draft

devchaudhary24k wants to merge 4 commits into
devfrom
feat/transcoder-libav

Conversation

@devchaudhary24k

Copy link
Copy Markdown
Member

Summary

  • Drop subprocess spawn("ffmpeg") in favor of in-process libav via node-av (MIT, prebuilt binaries — no system ffmpeg install required).
  • Decode-once / fan-out ABR pipeline: single demuxer + decoder feeds an N-way split/asplit filter graph; one encoder + HLS muxer per rung. Kills 2-3× duplicate decode work and the h264 threaded-decoder races that flooded stderr.
  • Graceful shutdown via new shared @vidcastx/queue/shutdown helper — SIGINT / SIGTERM / SIGHUP → worker.close(true) so tsx watch restarts release Redis locks cleanly.
  • BullMQ lock tuning: lockDuration: 10m, lockRenewTime: 2m, stalledInterval: 60s — default 30s was flagging long transcodes as stalled.
  • libav log filter (libav-log.ts) suppresses four known-harmless h264 thread-race patterns; everything else flows through with [libav] prefix.
  • Poster fix: yuv420p,setparams=range=pc + color_range: "pc" to silence deprecated pixel-format warning.
  • Hover preview fix: output-packet-count stop condition instead of trim filter (which EOF'd mid-pipeline and silently broke thumbnails).
  • AAC tail drain: encoder.encodeAll(null) so the final 1024-sample partial frame ships instead of being dropped.
  • Extended workers/transcoder/ROADMAP.md with HW-decoder design under §4 and new §8 for structured job logging; TODO summary at bottom.
  • Unrelated chores bundled: db:studio --host=0.0.0.0, routeTree regen, IDEA.md rewrite as product-vision doc.

Test plan

  • pnpm check-types — clean across all 11 workspaces
  • pnpm --filter transcoder lint — clean
  • Smoke: 10s testsrc clip → HLS + poster + preview produced, no reference picture missing spam in stderr
  • Real BullMQ job end-to-end from apps/app upload → transcoder → ready state
  • Observe progress ticks in BullMQ board (should be smooth per-frame, not 2-3 big jumps)
  • Confirm MinIO layout: processed/<org>/<vid>/master.m3u8 + per-rung playlists/segments + thumbnails/ + previews/
  • Video plays end-to-end in studio player
  • Kill worker mid-job with Ctrl+C → confirm clean shutdown log + job returns to queue (no stalled lock)

Notes

  • Target branch: main per request. Do not merge — leaving open for review.
  • Roadmap items §1–§8 in workers/transcoder/ROADMAP.md track the remaining work beyond this PR.

Drop subprocess `spawn("ffmpeg")` in favor of in-process libav via
`node-av` (MIT, prebuilt binaries — no system ffmpeg needed).

- Decode-once / fan-out ABR pipeline: single demuxer + decoder feeds an
  N-way split/asplit filter graph, one encoder+muxer per HLS rung.
  Kills 2-3x duplicate decode work and the h264 threaded-decoder races
  that flooded stderr with "reference picture missing" warnings.
- Poster: `yuv420p,setparams=range=pc` + `color_range: "pc"` to silence
  deprecated pixel-format warning.
- Hover preview: output-packet-count stop condition instead of `trim`
  filter to avoid EOF mid-pipeline.
- AAC tail drain via `encoder.encodeAll(null)` so the final 1024-sample
  partial frame ships instead of being logged as "N frames left".
- Filtered libav log callback (`libav-log.ts`) suppresses four known
  harmless h264 thread-race patterns; everything else flows through
  with a `[libav]` prefix.
- BullMQ worker: `lockDuration` 10m / `lockRenewTime` 2m /
  `stalledInterval` 60s — default 30s lock was flagging long transcodes
  as stalled.
- Shared graceful-shutdown helper `@vidcastx/queue/shutdown` — SIGINT /
  SIGTERM / SIGHUP → `worker.close(true)` so `tsx watch` restarts
  release Redis locks cleanly instead of leaving zombies.
- Extended `workers/transcoder/ROADMAP.md` with HW-decoder design under
  §4 and a new §8 for structured job logging; TODO summary at bottom.
Replaces the hardcoded (1080p/5000k, 720p/2800k, 480p/1400k) ladder
with one scaled from a content-measured probe bitrate. Screencasts
and talking heads stop paying for 5000k they don't need; high-motion
content stops being starved.

- `probeBitrate(inputPath, duration, fps)` runs libx264 `ultrafast`
  CRF=23 across 4 weighted sample windows (head 30s + 3×15s at
  25%/55%/85% of source) and accumulates packet.size. Short-video
  fallback: <60s probes fully, <300s probes twice, >=300s uses the
  4-sample scheme. Total probed footage is bounded (~75s) regardless
  of source length.
- Output is discarded via the existing generator-chain pattern
  (encoder.packets → sum packet.size, no muxer). Same pattern as
  preview.ts.
- `buildStreamVariants(height, fps, probeKbps)` scales per rung:
  1080p = probe × 1.10, 720p × 0.55, 480p × 0.25, then fps multiplier.
  Clamped to [300 kbps, per-rung ceiling]. Rung-drop rule: remove
  any rung within 70% of the rung above. Result: screencasts can
  ship a single rung, vlogs 2-3, action content the full 3.
- `runFFmpegTranscode` logs probe kbps + chosen ladder for
  observability; ROADMAP §1 checkboxes ticked.
@devchaudhary24k
devchaudhary24k changed the base branch from main to dev April 20, 2026 18:52
@devchaudhary24k
devchaudhary24k marked this pull request as draft September 22, 2026 22:23
@devchaudhary24k

Copy link
Copy Markdown
Member Author

Review: parking this as a draft for now

I ran the libav pipeline against the FFmpeg CLI worker on dev, using test clips made with the FFmpeg binary that node-av ships. It type-checks and lints cleanly, but it isn't ready to replace the CLI worker yet.

Blockers

  • Frame loss on any clip with audio. In a 10s clip, 190 of 300 frames came out (video 6.3s long, audio 8.5s). In a 90s clip, only 1536 of 2700 distinct frames. The cause: transcode.ts runs separate video and audio read loops on one demuxer, and node-av 5.2.3 silently drops a stream's packets once its queue reaches 100. libav-log.ts then hides the resulting h264 errors, even though they are real.
  • Seeking uses a stream index instead of -1 (poster.ts, preview.ts, probe.ts). The offset is read as stream time units, so the poster and preview are taken near the end of the clip, and the per-title probe samples the wrong parts.
  • No audio format conversion or downmix before AAC. PCM audio (common in .mov files) fails the whole job.
  • Rotation metadata is ignored. Portrait phone video comes out sideways.
  • Licensing. The prebuilt node-av.node contains a GPLv3 FFmpeg build with libx264 and runs inside the worker process. That needs a decision now that the repo is MIT.

Also

  • The master playlist is written by hand, without CODECS, and BANDWIDTH leaves out audio.
  • close(true) doesn't release the job lock. Combined with the 10-minute lock, a killed job stays stuck for about 10 minutes.
  • probe.ts throws when the duration is unknown, which is common for WebM files recorded in a browser.
  • There's an unvalidated as FFEncoderCodec cast, and errors are logged with only err.message.
  • pnpm 11 allowBuilds covers only linux-x64, so the binding won't load on macOS or arm64.

Worth keeping: forcing 8-bit yuv420p output. The CLI worker on dev currently produces High 10 H.264 from 10-bit sources, which most browsers can't play. Also worth keeping: the shutdown helper idea and per-title encoding once the probe is correct. These are now tasks for the CLI worker in the V1 plan.

Fixing this branch is roughly 2–3 days of work. The branch stays as it is, so it can be picked up again as its own experiment after V1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant