Microsoft.ML.Tokenizers.Tests.EnglishRobertaTests.TestTokenizerEncoding(text: \"the brown fox jumped over the lazy dog!\", expectedTokens: [\"the\", \"Ġbrown\", \"Ġfox\", \"Ġjumped\", \"Ġover\", ···], expectedOffsets: [Tuple (0, 3), Tuple (3, 6), Tuple (9, 4), Tuple (13, 7), Tuple (20, 5), ···], expectedIds: [1169, 7586, 21831, 11687, 625, ···])
Exception Message
System.ArgumentOutOfRangeException : value must be non-negative
Parameter name: value
Stack Trace
at System.Index.ThrowValueArgumentOutOfRange_NeedNonNegNumException()
at Microsoft.ML.Tokenizers.EnglishRobertaTokenizer.ModifyTokenListOffsets(IReadOnlyList`1 tokens, Span`1 indexMapping) in D:\a\_work\1\s\src\Microsoft.ML.Tokenizers\Model\EnglishRobertaTokenizer.cs:line 913
at Microsoft.ML.Tokenizers.EnglishRobertaTokenizer.EncodeInternal(ReadOnlySpan`1 text) in D:\a\_work\1\s\src\Microsoft.ML.Tokenizers\Model\EnglishRobertaTokenizer.cs:line 397
at Microsoft.ML.Tokenizers.EnglishRobertaTokenizer.EncodeToTokens(String text, ReadOnlySpan`1 textSpan, EncodeSettings settings) in D:\a\_work\1\s\src\Microsoft.ML.Tokenizers\Model\EnglishRobertaTokenizer.cs:line 337
at Microsoft.ML.Tokenizers.Tokenizer.EncodeToTokens(ReadOnlySpan`1 text, String& normalizedText, Boolean considerPreTokenization, Boolean considerNormalization) in D:\a\_work\1\s\src\Microsoft.ML.Tokenizers\Tokenizer.cs:line 166
at Microsoft.ML.Tokenizers.Tests.EnglishRobertaTests.TestTokenizerEncoding(String text, String[] expectedTokens, ValueTuple`2[] expectedOffsets, Int32[] expectedIds) in D:\a\_work\1\s\test\Microsoft.ML.Tokenizers.Tests\EnglishRobertaTests.cs:line 179
Build Information
Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1599215
Build error leg or test failing: Microsoft.ML.Tokenizers.Tests.EnglishRobertaTests.TestTokenizerEncoding
Pull request: #7704
Error Message
DO NOT USE JSON BELOW IF THIS IS A BUILD BREAK otherwise build analysis will allow pull requests to merge that break the build worse. For a build break, do not use this issue form. Make a regular new issue.
Fill the error message using step by step known issues guidance.
{ "ErrorMessage": ["EnglishRobertaTests.TestTokenizerEncoding", "ArgumentOutOfRangeException : value must be non-negative"], "BuildRetry": false, "ExcludeConsoleLog": false }Known issue validation
Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1599215
Error message validated:
[EnglishRobertaTests.TestTokenizerEncoding ArgumentOutOfRangeException : value must be non-negative]Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 9/16/2026 2:17:25 PM UTC
Report
Summary