[jantonguirao]

Category:

Other (e.g. Documentation, Tests, Configuration): performance improvement, no functional change.

Description:

FramesDecoderGpu queries the driver with cuvidGetDecoderCaps every time it opens a file, and again whenever it needs a new NVDEC decoder:

• in SelectVideoStream (run when the decoder is constructed, i.e. for every file), to check that NVDEC supports the codec;
• in NVDECCache::GetDecoder, when no cached decoder can be reused and a new one has to be created, to check the codec configuration and its size limits.

The result depends only on the GPU and on the codec, chroma format and bit depth, so it never changes during the process. The call is serialized in the driver, though. With one decoder that is a small, fixed cost per file. When many decoders open files at the same time, the calls queue up in the driver and each one becomes slow, which stalls decoding on every file switch. This affects every user of FramesDecoderGpu (fn.experimental.readers.video, fn.experimental.decoders.video and fn.experimental.inputs.video on the GPU). It matters most for workloads that switch files often (short clips, small sequence_length) with several decoders opening files at once, and it gets worse once a reader runs several decoders concurrently, as planned with the decode lanes (#6563, #6564).

This PR adds frame_dec_gpu_impl::GetDecoderCaps, which caches the CUVIDDECODECAPS per process, keyed by the current device, codec, chroma format and bit depth. The lock is held during the driver query as well, so decoders that start at the same time query each configuration only once instead of all missing the cache together. Both call sites above use it. The static helper FramesDecoderGpu::SupportsHevc, which is not called per file (only the tests use it), still calls the driver directly.

Split out of #6545.

It is based on main and independent of the other PRs. It touches NVDECCache::GetDecoder next to the lines that #6540 changes, so whichever of the two merges second may need a trivial rebase.

Additional information:

Affected modules and functionalities:

• dali/operators/video/frames_decoder_gpu.cc:
  • new frame_dec_gpu_impl::GetDecoderCaps (static mutex and std::map keyed by device, codec, chroma format and bit depth);
  • NVDECCache::GetDecoder and FramesDecoderGpu::SelectVideoStream use it instead of calling cuvidGetDecoderCaps directly.

Key points relevant for the review:

• The key includes the device (cudaGetDevice), so multi-GPU processes and GPUs of different generations get their own entries.
• The single process-wide lock is held during the driver query. This also serializes queries of different configurations, but the driver serializes them anyway, and each configuration is queried only once per process.
• A failed query (CUDA_CALL throws) is not cached; the next call queries again. An unsupported configuration (bIsSupported == 0) is cached like any other result, as it is a property of the hardware.
• The returned capabilities are the same as before, so the support checks and error messages do not change.

Tests:

• Existing tests apply
  • frames_decoder_test.cc: FramesDecoderGpuTest.* (e.g. ConstantFrameRate, VariableFrameRateHevc, InMemoryVfrHevcVideo, RawH264, RawH265, DecoderCacheReusedForCroppedVideo), which open many files with H.264, HEVC and MPEG-4 and go through both cached call sites.
  • decoder/test_video.py: GPU tests of fn.experimental.readers.video and fn.experimental.decoders.video.
• New tests added
  • Python tests
  • GTests
  • Benchmark
  • Other
• N/A

Checklist

Documentation

• Existing documentation applies
• Documentation updated
  • Docstring
  • Doxygen
  • RST
  • Jupyter
  • Other
• N/A

DALI team only

Requirements

• Implements new requirements
• Affects existing requirements
• N/A

REQ IDs: N/A

JIRA TASK: DALI-4935

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[Copilot]

🟢 Approval recommended

The focused change preserves capability checks, correctly synchronizes cache access, and leaves failed queries uncached.

No open issues 🎉

What changed in this PR

This PR reduces repeated NVDEC capability queries when GPU video decoders open files, helping avoid driver contention during concurrent decoding.

Changes:

• Adds a process-wide capability cache keyed by device, codec, chroma format, and bit depth.
• Synchronizes cache access and queries so concurrent requests populate each configuration only once.
• Uses cached capabilities in stream selection and decoder creation while preserving validation and query-failure handling.

File	Description
dali/​operators/​video/​frames_decoder_gpu.cc	Adds the synchronized capability cache and updates both per-file and decoder-creation query sites.

_{Review effort: Balanced}

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; the remaining test-coverage suggestion is non-blocking.

Summary

The PR caches NVDEC capabilities by GPU, codec, chroma format, and bit depth. The latest change lets cache hits share a read lock.

• GPU video decoders reuse capability results for each device and video format.

_{Reviews (2) · Last reviewed commit: "Read the decoder caps cache under a shar..." · Reviewed by Greptile}

[jantonguirao]

!build

[dali-automaton]

CI MESSAGE: [77048240]: BUILD STARTED

[jantonguirao]

!build

[dali-automaton]

CI MESSAGE: [77062151]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [77062151]: BUILD PASSED