Skip to content

fix(onnx): reset the frontend after a short final chunk in streaming Paraformer - #3927

Merged
LauraGPT merged 1 commit into
modelscope:mainfrom
tanishqbaweja:fix/paraformer-online-short-final-reset
Nov 20, 2026
Merged

LauraGPT merged 1 commit into
modelscope:mainfrom
tanishqbaweja:fix/paraformer-online-short-final-reset

Conversation

@tanishqbaweja

Copy link
Copy Markdown
Contributor

Summary

Fixes #3926

funasr_onnx's streaming Paraformer.__call__ (runtime/python/onnxruntime/funasr_onnx/paraformer_online_bin.py:142-147) handles a final chunk with fewer than 960 samples, once the stream has produced features, by decoding the cached overlap frames and returning. It skips extract_feat, and with it the frontend reset that every other final call gets: WavFrontendOnline.extract_fbank calls cache_reset() when is_final (utils/frontend.py:377-378). So input_cache, lfr_splice_cache and reserve_waveforms survived the end of the stream.

The frontend belongs to the model, not to the caller's cache, so the next stream on the same model was framed as a continuation of the previous one: the 240-399 samples left in input_cache went in front of its audio, and its first LFR frame was built from the previous stream's last fbank frame(s). Its first 60 ms feature frame came from the end of the previous stream, and when the left-over sample count was not a multiple of 160, every later 25 ms window moved too. This happens after any stream that has produced features and ends with an empty final call, and with the 600 ms chunking of demo_paraformer_online.py whenever the audio ends with a last chunk of 2 to 959 samples.

This PR resets the frontend in that branch, as the final extract_feat call does in every other case:

if waveforms.shape[1] < 16 * 60 and is_final and len(cache) > 0:
    # extract_feat is skipped here, so reset the frontend as its final call would.
    self.frontend.cache_reset()
    cache["last_chunk"] = True

The C++ runtime already does this in the same branch: ParaformerOnline::Forward calls ResetCache(), which clears reserve_waveforms_, input_cache_ and lfr_splice_cache_, then Reset() (runtime/onnxruntime/src/paraformer-online.cpp:532-539, :391-395). The PyTorch model keeps its frontend state in cache["frontend"] and re-initializes the cache on every final call (funasr/models/paraformer_streaming/model.py:753-754, :547).

The short final call itself decodes the same cached frames as before, and infer does not read the frontend.

The first encoder input of a 0.6 s stream decoded after another stream on the same model (the script in the issue):

final chunk of the previous stream main this PR
0 samples differs from a fresh model (max abs diff 351.5) same as a fresh model
500 samples differs (351.5) same
959 samples differs (351.5) same
960 samples same same
4000 samples same same

Type of change

  • Bug fix
  • Documentation
  • Example or demo
  • Runtime or deployment
  • Benchmark or evaluation
  • Model/training change

Validation

The new test_short_final_after_features_does_not_contaminate_next_stream goes in tests/test_funasr_onnx_paraformer_online_empty_final.py, next to test_short_final_does_not_contaminate_next_stream from #3827. That test covers a short first-and-final call, whose cache is still empty, so it goes through extract_feat and was already reset. The new test uses the class's model: real __call__, WavFrontendOnline, positions and overlap cache, with infer mocked. It decodes 9600 samples of a tone and then a final chunk of 0, 1, 500, 959 or 960 samples. It then decodes a 0.6 s stream of another tone on the same model and checks that the encoder input equals that of the same stream on a fresh model. 960 samples is the control that already goes through extract_feat.

Windows 11, CPU, Python 3.12.15, numpy 1.26.4, onnxruntime 1.32.0, kaldi-native-fbank 1.23.3, librosa 0.11.0. I built wheels from runtime/python/onnxruntime at main (aa3d789) and at this branch, installed each into a fresh environment, and ran the test scripts as test-funasr-onnx-package.yml does:

python tests/test_funasr_onnx_paraformer_online_empty_final.py
  • main package: Ran 13 tests ... FAILED (failures=4). The subtests for 0, 1, 500 and 959 samples fail with Arrays are not equal, Mismatched elements: 12 / 560, Max absolute difference: 14.8243885. The 960-sample subtest passes.
  • This branch: Ran 13 tests ... OK.
  • The other scripts the workflow runs pass with both packages: test_funasr_onnx_release.py (9 tests), test_funasr_onnx_installed.py (13), test_funasr_onnx_paraformer_online_overlap.py (3), test_funasr_onnx_vad_online_short_chunks.py (6), test_funasr_onnx_vad_multiple_inputs.py (6) and test_funasr_onnx_punctuation_empty_input.py (5).

With pytest over those seven files plus test_rich_transcription_postprocess.py, test_rich_transcription_event_segments.py, test_runtime_cmvn.py and test_sentence_postprocess_timestamps.py (PYTHONUTF8=1):

  • main package: 4 failed, 144 passed, 416 subtests passed. The 4 failures are the four subtests above.
  • This branch: 144 passed, 420 subtests passed. The test ids and outcomes are otherwise the same.

Outside the test, with the 80-mel frontend of the issue's script and only infer replaced, I compared every encoder input between main and this branch. The run had 60 models with 4 streams each, 0.6 to 2.5 s of a random tone plus noise per stream. The streams were cut three ways: 9600-sample chunks with the remainder as the final chunk, random chunks with an empty final call, or random chunks with a final chunk of 0-959 samples.

  • On a fresh model, all 240 streams give identical encoder inputs and results on main and this branch.

  • On a reused model, the 130 streams that came after a short final chunk differ from the same stream on a fresh model on main (130/130), and match it on this branch (130/130). The 50 streams that came after a longer final chunk match on both.

  • python -m compileall funasr examples tests

  • Docs or links checked

  • Runtime/deployment command tested

black --line-length=100 (24.4.0, the pre-commit pin) leaves the changed lines alone. It reports the same existing formatting in both files on main and on this branch. git diff --check is clean. No real model was run.

User impact

Users of the funasr_onnx streaming Paraformer who decode several streams with one model, one after another, now get the features of each stream as on a fresh model. That covers a server that serves sessions in turn, or the demo looped over files. Before, after a stream that ended with an empty final call or a final chunk under 960 samples, the next stream started with the end of the previous one. Single streams, and streams that follow a final chunk of 960 samples or more, give the same encoder inputs as before.

Notes for reviewers

  • The short final call still decodes only the cached overlap frames, and the samples of that last chunk and those left in the frontend are not decoded, as before and as in the C++ and PyTorch models. This PR only resets the state.
  • The reset runs before infer, as it does on the other final path, where extract_feat resets the frontend before inference.
  • One model's frontend is still shared by all its streams, so streams on one model still have to run one after another. This PR does not change that.
  • No workflow change: the test is in the file that test-funasr-onnx-package.yml already runs ("Verify empty split-final Paraformer result"). The package version stays at 0.4.11, which the release test checks.
  • No open PR touches paraformer_online_bin.py or utils/frontend.py.

…Paraformer

A final call to the streaming Paraformer with fewer than 960 samples
decodes the overlap frames cached by the earlier calls and returns
without calling extract_feat. extract_feat is what resets
WavFrontendOnline at the end of a stream, so the frontend kept its
cached samples, LFR splice frames and reserved waveform. The frontend
belongs to the model, not to the caller's cache, so the next stream on
the same model was framed as a continuation of the previous one: its
first LFR frame was built from the end of the previous stream, and the
240-399 samples left in the frontend went in front of its audio, which
also moved every later 25 ms window unless their number was a multiple
of 160.

This happens after any stream that has produced features and ends with
an empty final call, and in the demo's 600 ms chunking whenever the
audio ends with a chunk of 2 to 959 samples.

Reset the frontend in that branch, as a final extract_feat call does.
The C++ ParaformerOnline::Forward resets its frontend caches in the
same branch, and the PyTorch model clears its frontend cache on
is_final. The short final call decodes the same frames as before.
@LauraGPT
LauraGPT merged commit dce4ec4 into modelscope:main Nov 20, 2026
4 checks passed
@LauraGPT LauraGPT mentioned this pull request Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

funasr_onnx streaming Paraformer: a final chunk under 960 samples leaves the frontend caches, so the next stream starts with the end of the previous one

2 participants