fix(whisper): match reference mel preprocessing

[WenJing95]

Background

Nonzero audio edges produced different Whisper mel features because STFT used zero padding. The discarded final frame could also change the log floor.

Exit Criteria

Match reference features at chunk boundaries.

Implementation

Reflect both ends of the waveform and remove the last frame before normalization.

Change categories

• Model or runtime behavior

Validation

Commands and Results

Native reference goldens passed at 80/128 mels; source-quality: 164 passed.

g++ -std=c++17 -O2 -Wall -Wextra -Wpedantic -I. families/whisper/tests/cpp/test_whisper_mel_spectrogram.cpp families/whisper/runtime/whisper_mel_spectrogram.cpp -o /tmp/trtmc-four-whisper-test
/tmp/trtmc-four-whisper-test
python -m tools.community_ci source-quality --base 714f1fc0d567213a7b79a488e9dcfdf405279c6a

Hardware, Environment, and Revisions

8a95b0e: Ubuntu CPU, GCC 13.3; Transformers 5.2.0 reference.

Not Run / Remaining Gaps

TensorRT inference and full CMake suite: TensorRT environment unavailable.

Contributor Self-Review

• I have completed a self-review of this change.

Notes For Future Readers

Reflect after padding/truncating to 30 seconds so short clips keep their zero-padded tail.

Risk level

• Low

Changes are confined to Whisper's center padding and log normalization.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a48f4cba-5f82-4862-b2ed-28b19b407d69

📥 Commits

Reviewing files that changed from the base of the PR and between 714f1fc and 8a95b0e.

📒 Files selected for processing (3)

• families/whisper/runtime/CMakeLists.txt
• families/whisper/runtime/whisper_mel_spectrogram.cpp
• families/whisper/tests/cpp/test_whisper_mel_spectrogram.cpp

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

📝 Summary

Summary

Whisper mel preprocessing now matches reference boundary behavior.

• Reflects both waveform ends without repeating edge samples.
• Removes the final STFT frame before log normalization.
• Adds CTest coverage for 80- and 128-mel outputs.
• Covers boundary padding, trailing-frame removal, empty input, and single-sample input.

Architecture impact

• Family-owned files: Whisper runtime preprocessing, CMake integration, and Whisper tests.
• Shared surfaces: None identified.
• Dependency directions: No new dependencies. The test target uses the Whisper implementation directly.
• Affected consumers: Whisper runtime consumers of extract_mel_spectrogram.
• Unresolved blast-radius questions: TensorRT inference and the full CMake suite were not run because TensorRT was unavailable.

Review outcome

HUMAN REVIEW REQUIRED

The supplied evidence reports passing native reference tests and source-quality checks. Review severity counts are unavailable. TensorRT and full-suite compatibility remain unverified.

Walkthrough

The Whisper mel-spectrogram code now reflects audio edges during center padding and removes the final frame before normalization. A standalone C++ test validates output shapes, values, and edge cases for 80- and 128-mel configurations.

Changes

Whisper mel-spectrogram

Layer / File(s)	Summary
Mel-spectrogram processing updates families/whisper/runtime/whisper_mel_spectrogram.cpp	Center padding now reflects interior waveform samples. The final frame is removed before logarithmic normalization.
Executable test integration families/whisper/tests/cpp/test_whisper_mel_spectrogram.cpp, families/whisper/runtime/CMakeLists.txt	The new test checks shapes, golden values, padding, trailing-frame removal, silence floors, and single-sample input for 80- and 128-mel configurations. CMake builds and registers the test with CTest.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~15 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 8a95b

The mel preprocessing change includes coverage for the changed padding, frame-trimming, and normalization behavior. No actionable merge risk remains.

🚥 Pre-merge checks | ✅ 8 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 30.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 2 files. (1 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (8 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: updating Whisper mel preprocessing to match the reference implementation.
Description check	✅ Passed	The description covers the required background, exit criteria, implementation, behavior category, validation commands and results, environment, remaining gaps, self-review, future notes, and risk rati…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Family Ownership Boundary	✅ Passed	No cross-family dependency is introduced. The changed files are all under families/whisper. The new CTest target in families/whisper/runtime/CMakeLists.txt:44-50 uses only the Whisper test source …
Shared Semantic Neutrality	✅ Passed	PASS: The pull request changes only families/whisper/runtime/CMakeLists.txt, families/whisper/runtime/whisper_mel_spectrogram.cpp, and families/whisper/tests/cpp/test_whisper_mel_spectrogram.cpp…
Benchmark Validation Integrity	✅ Passed	PASS. The PR changes Whisper frontend validation and adds a registered CTest, but it does not introduce an invalid benchmark comparison. The new test compares the mel output at the same mel-bin/frame …
Shared Change Blast Radius	✅ Passed	PASS. The pull request changes only families/whisper/*. The mel implementation is namespaced for Whisper, and its public header and pipeline call contract are unchanged. The new CMake target and CTe…

Full details: Docstring Coverage

Explanation

Docstring coverage is 30.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 2 files. (1 skipped: 1 unsupported.)

---

_{Comment @coderabbitai help to get the list of available commands.}

[yifeif-nv]

Hi @WenJing95 thanks for the contribution. The general direction looks good on this PR. Let me know once you think it is ready to be reviewed and run the internal CI on it. Then we can bring it out of draft mode

[WenJing95]

Ready for review and internal CI. Thanks!