fix: convert a character_not_in_repertoire error to an invalid-class error - #866
Merged
Merged
Conversation
…error PostgreSQL rejects text that is not valid UTF-8 or contains a NUL byte with `22021 character_not_in_repertoire`. Every `Postgrex.Error` clause in `handle_raised_error/4` ends in a constraint lookup, which finds nothing for that code, so a create, update or filter with such a value returned `Ash.Error.Unknown` and the API extensions answered 500. Only the data layer knows this storage has that limit; ETS and Mnesia store the value. The error names no column, so for an update the attribute is found by checking the changeset's changes for a value that is not valid UTF-8 or contains a NUL byte and each becomes `InvalidAttribute`. A create's rescue only has the placeholder changeset `bulk_create/3` builds for it, so it becomes `InvalidChanges`. A query becomes `InvalidFilterValue`. Closes ash-project#854
7 tasks
Contributor
|
🚀 Thank you for your contribution! 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #854.
@g3nke11 claimed this on 2026-09-17. I linked this branch on the issue on 2026-09-22 and offered to open it if they had moved on; there has been no reply since, so I am opening it now. Happy to close it if they would rather finish their own.
PostgreSQL rejects text that is not valid UTF-8 or contains a NUL byte with
22021 character_not_in_repertoire. EveryPostgrex.Errorclause inhandle_raised_error/4ends inEcto.Adapters.Postgres.Connection.to_constraints/2, which finds nothing for that code, so a create, update or filter with such a value returnsAsh.Error.Unknown, and the API extensions answer 500. Only the data layer knows this storage has that limit; ETS and Mnesia store the value.This adds a clause matching
%Postgrex.Error{postgres: %{code: :character_not_in_repertoire}}. It matches on the error code, not on message text. It sits before the existingPostgrex.Errorclauses for the{:bulk_create, _}and{:ecto_changeset, _, _}contexts, which would otherwise capture it first.Ash.Error.Query.InvalidFilterValue.Ash.Error.Changes.InvalidAttributefor each change whose value failsString.valid?/1or contains<<0>>. The error names no column, so the attribute comes from the changes.Ash.Error.Changes.InvalidChanges. The rescue there only has the placeholder changeset thatbulk_create/3builds for it, with no changes, so the attribute cannot be named.The message is Postgres's own (
invalid byte sequence for encoding "UTF8": 0x00). One thing I would change once theno_value?option you suggested on #863 exists: the query case has no value to put onInvalidFilterValue, so it currently renders asInvalid filter value `nil`: ..., the same as #863. Happy to adjust it here along with that.Tests:
test/repertoire_error_test.exscovers a filter, an update with a NUL byte and with invalid UTF-8 (asserting the attribute), a create, and an ordinary-text control. Onmainthe four error cases returnAsh.Error.Unknown; with the change they pass. Full suite 1011 passed onmain, where the same threeJoinSubquerySortTest/UniqAggregateSortTestcases from #858 fail with and without this change, and 983 passed with the patch on v2.13.1.mix format --check-formatted,mix credo --strict,mix sobelowandmix dialyzerclean.The branch is based on current
main, and it also merges cleanly with #863 in either order; I checkedmainplus both together, and the full suite passes on that with only the three #858 failures.Found by an AI agent working with a human fuzz-testing their own application; the reproduction outside this repo is https://github.com/grempe/ash-fuzz-repros/blob/main/test/ash_postgres/nul_byte_in_text_test.exs, where all eight cases pass against this branch, across Ash, AshJsonApi, AshGraphql and ash_lua.
Contributor checklist
Leave anything that you believe does not apply unchecked.