Windows agent sessions can silently replace Unicode punctuation with ? before signing messages

[santimon]

Windows agent sessions can silently replace Unicode punctuation with ? before signing Buzz messages

Summary

On Windows, an agent session that sends generated message content from Windows PowerShell 5.1 through buzz messages send --content - without explicitly setting UTF-8 pipe encoding can silently replace non-ASCII characters with ASCII question marks before the Nostr event is built and signed.

This is easy to miss because the command succeeds, the relay accepts the event, and Buzz Desktop faithfully renders the already-corrupted event. The corruption is permanent in message history and can vary between per-channel sessions of the same agent.

Observed behavior

Intended text:

The standalone laptop is a solid call — it solves the availability concern.
No desktop environment needed — less to patch, less to break.

Stored/rendered text:

The standalone laptop is a solid call ? it solves the availability concern.
No desktop environment needed ? less to patch, less to break.

The substitution repeats at every intended em dash. This is not mojibake such as —, and it is not a Desktop rendering error: the outbound PowerShell/native-process boundary has already converted U+2014 to U+003F before submission.

Environment

• Windows 11
• Windows PowerShell 5.1.26100.9168
• Buzz managed ACP agent sessions
• Buzz CLI invoked by the agent harness
• One independent agent session per channel

Reproduction

From Windows PowerShell 5.1, pipe Unicode prose to the CLI without first setting $OutputEncoding to UTF-8:

$message = 'alpha — beta “quoted” café'
$message | buzz messages send --channel <channel-uuid> --content -

Fetch the resulting signed event and inspect its content code points. Depending on the active legacy encoding, unsupported characters can be stored as ? (U+003F) or otherwise corrupted.

The immediate workaround is:

$OutputEncoding = [Console]::OutputEncoding = [Text.UTF8Encoding]::new()
$message | buzz messages send --channel <channel-uuid> --content -

However, a prompt/workspace instruction is insufficient because each channel has an independent, potentially long-running agent session. One session can comply while another session of the same agent continues using the unsafe encoding.

Expected behavior

Every Windows agent session should preserve generated Unicode text byte-for-byte through CLI submission without relying on prompt compliance or per-command shell setup.

Proposed fix

Initialize every Windows PowerShell process spawned by buzz-acp with explicit BOM-less UTF-8 input/output encoding before agent commands execute:

$OutputEncoding = [Console]::OutputEncoding = [Text.UTF8Encoding]::new()

Also consider removing the shell encoding boundary from human-facing message submission:

• provide a buzz messages send --content-file <path> option that reads UTF-8 explicitly; or
• have the harness invoke the CLI directly and write UTF-8 bytes to stdin rather than constructing PowerShell command arguments.

Regression test

Add a Windows end-to-end test that sends the following through the same harness path used by managed agents:

em dash — curly quotes “test” accented café CJK 測試 emoji 🐝

Fetch the accepted event and assert exact Unicode scalar values in content, not only successful command exit or visual rendering. Run the test for two independent channel sessions to cover per-session initialization.

Impact

• Signed message history can be permanently altered before relay ingestion.
• The failure is silent: the CLI reports success.
• Multiple sessions of one agent can behave differently.
• Human readers may incorrectly attribute the problem to Desktop rendering or relay storage.