Skip to content

fix(postprocess): keep a timestamp for symbol tokens in sentence_postprocess - #3929

Merged
LauraGPT merged 1 commit into
modelscope:mainfrom
sohailshk:fix/postprocess-symbol-timestamps
Nov 20, 2026
Merged

LauraGPT merged 1 commit into
modelscope:mainfrom
sohailshk:fix/postprocess-symbol-timestamps

Conversation

@sohailshk

Copy link
Copy Markdown
Contributor

Summary

In sentence_postprocess, a token that is not Chinese, a digit, a BPE piece or letters (the Paraformer-zh vocabulary has . and &) falls into the last branch of the mixed-text path. That branch adds the word but no timestamp in funasr/utils/postprocess_utils.py, and raises ValueError: invalid character in the funasr_onnx and funasr_torch copies. With timestamps requested, every word from that token on is paired with the next token's span and the last word gets none, and a spelled-out acronym after it can raise IndexError:

>>> sentence_postprocess(["价", "格", "是", "3", ".", "5", "元", "好"], [[i*100, i*100+100] for i in range(8)])
# 8 words, 7 timestamps; ".", "5", "元" get the spans of "5", "元", "好"; "好" gets none
>>> sentence_postprocess([".", "x", "y"], [[0, 100], [100, 200], [200, 300]])
('. XY', [[100, 200]], ['.', 'XY'])   # XY should be [100, 300]
>>> sentence_postprocess(["3", ".", "5", "&", "x", "y"], [[i*100, i*100+100] for i in range(6)])
IndexError: list index out of range

Same error as #1283 in funasr_onnx (ValueError: invalid character: .). It reaches users through AutoModel("paraformer-zh"): SeacoParaformer.inference (funasr/models/seaco_paraformer/model.py) calls sentence_postprocess(token, timestamp) and returns the result as result["timestamp"].

The token now gets its own [start, end] in all three copies, so words and timestamps stay one to one.

Type of change

  • Bug fix

Validation

  • python -m compileall funasr examples tests
  • python -m pytest tests/test_sentence_postprocess_timestamps.py -q: 14 failed subtests (3 new tests over the three copies), 13 passed on current main; 13 passed (57 subtests) with the fix.
  • CI text-postprocessing set (test_postprocess_hotwords.py, test_rich_transcription_postprocess.py, test_sentence_postprocess_timestamps.py): 44 passed, 6 skipped (fuzzy hotword tests, optional deps not installed; the same 6 skip on main).

User impact

Word-level timestamp output for Paraformer-style models stays aligned with the words when the text contains a decimal point, & or similar tokens; funasr_onnx and funasr_torch no longer raise on them.

Notes for reviewers

  • One branch changed per copy; token lists without such tokens take the same paths as before.
  • The symbol branch handles the space after a Latin word like the Chinese branch does (it removes it), so ["ok", ".", "好"] gives ok.好 without timestamps and keeps 好 at its own span with them.
  • Merges cleanly with fix(postprocess): keep a trailing @@ piece instead of dropping it #3836 (it edits the end of the same function).
  • The implementation and tests were written with AI assistance.

…process

Tokens such as "." and "&" fall into the last branch of the mixed-text path, which added the word but no timestamp (funasr) or raised ValueError (funasr_onnx, funasr_torch). Every word from that token on was paired with the next token's span and the last word got none, and acronyms after the symbol could raise IndexError. Give the token its own timestamp in all three copies, and handle the blank after a Latin word like the Chinese branch does.
@LauraGPT
LauraGPT merged commit 8d3db4c into modelscope:main Nov 20, 2026
8 checks passed
LauraGPT added a commit that referenced this pull request Nov 21, 2026
Preserve per-utterance Whisper frontend lengths and core symbol-token timestamps (#3918, #3929). Standalone ONNX/Torch packages are not released by this root-package change.

Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
@LauraGPT LauraGPT mentioned this pull request Nov 21, 2026
LauraGPT added a commit that referenced this pull request Nov 21, 2026
Preserve per-utterance Whisper frontend lengths and core symbol-token timestamps (#3918, #3929). Standalone ONNX/Torch packages are not released by this root-package change.

Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants