Skip to content

Fix PCM16 and multichannel array input with NumPy CI - #3899

Merged
LauraGPT merged 3 commits into
mainfrom
codex/audio-array-input-ci-20261114
Nov 14, 2026
Merged

LauraGPT merged 3 commits into
mainfrom
codex/audio-array-input-ci-20261114

Conversation

@LauraGPT

@LauraGPT LauraGPT commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Integrate #3890 and #3891, preserving both original contributor commits as ancestors and their implementation/tests unchanged. NumPy PCM16 sound inputs scale to float32; floating 2D sound arrays/tensors reduce to mono before resampling. Non-sound features, integer tensors and reduce_channels=False keep their existing behavior. The documented shorter-axis rule treats square inputs as channels-first.

Both contributor test files now trigger and execute in the existing NumPy 1.26.4 / 2.4.0 CI matrix, for pull requests and main pushes. No independent ONNX/libtorch package change.

Verification

  • Current-main complete workflow selection: 461 private cases passed, zero failures/errors/skips, including 27 CUDA-parametrized cases. Correction: the initial description incorrectly called all 461 cases CPU tests; the private environment exposed CUDA. Eight warnings were retained. This is the workflow scope, not the complete root suite.
  • Original retained controls: baseline 9 failures/4 passes, individual author tests 5/8 passes, combined 13 passes. Previous related 147 CPU passes retained; initial missing private pytorch-wpe failures remain disclosed.
  • Reused real Shanghai L20Z FP32 loader/WavFrontend evidence after exact source-blob and website-only intervening-tree verification: 10 synthetic scenarios, 20 batch-1/2 feature comparisons, CUDA traces and raw waveforms retained. Unchanged GPU cases were not rerun. No pretrained weights or recognition-quality claim; no new BF16 or CUDA-resampling claim.
  • Parsed workflow coverage was missing six selection/event-filter checks before the patch and passes after it. New exact-head and merged-main CI must be separately accepted from actual artifact bytes.
  • Signed+DCO integration, inline code review, no self-approval, protection change or admin bypass. Nano remains paused.

Exact-head artifact acceptance

  • Both NumPy jobs collected 461 cases: 434 passed and 27 skipped per job, with zero failures/errors. Every skip is an explicit CUDA is unavailable case from test_utterance_mvn.py on the CPU runner. All 13 new input regression cases actually ran in both jobs.
  • The 27 exact skipped test identities all passed in the private 461-case run on the same tested source tree. The test code performs real CUDA tensor operations, gradients, and tiny random-model inference; this is not pretrained recognition quality or a full root suite. The separate retained loader/frontend GPU evidence includes profiler traces; the 461-case run did not record a fresh GPU-idle preflight or a new profiler trace.
  • The first private artifact checker incorrectly required zero skips on the CPU runner and rejected the artifact. The revised checker joins exact test identities with private hardware results and rejects wrong skip reasons, skipped new regression cases, missing identities and failures. All 12 checker checks passed. No source change, terminal CI rerun, or reclassification of skips as passes was used.

sohailshk and others added 3 commits November 13, 2026 10:04
load_audio_text_image_video averages the channels of audio files, but
in-memory numpy arrays and tensors passed to generate() were handed on
unchanged. A (1, N) array from torchaudio.load or a stereo array made the
FSMN-VAD frontend assert on the window size and, with (N, 1), raise a
ZeroDivisionError. Reduce 2-D floating-point waveforms to mono (the shorter
axis is the channel axis) before resampling, honoring reduce_channels like
the file path. Empty 2-D input becomes empty 1-D; integer input is left as
before.
…_image_video

A np.int16 array (for example from PyAudio or sounddevice) was converted to
an int16 tensor without scaling. The frontend multiplies float waveforms by
32768, so the signal was overdriven and VAD segments changed, and with a
sample rate other than 16 kHz the resampler raised a TypeError. Scale int16
arrays by 1/32768 like load_bytes does for raw PCM.
Integrate the exact original #3890 and #3891 changes, preserving both author commits as parents. Carry current main forward and exercise both regressions in the existing NumPy matrix.

Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
@LauraGPT
LauraGPT merged commit 173cbee into main Nov 14, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants