Skip to content

[core] fix(Chatwoot): sanitize attachment base64 content in consumer … - #2202

Open
danielkv wants to merge 1 commit into
devlikeapro:corefrom
Booksuite:dguolo/redis-memory-bloat-fix
Open

[core] fix(Chatwoot): sanitize attachment base64 content in consumer …#2202
danielkv wants to merge 1 commit into
devlikeapro:corefrom
Booksuite:dguolo/redis-memory-bloat-fix

Conversation

@danielkv

Copy link
Copy Markdown

Summary

Fixes high Redis memory usage caused by storing full base64-encoded media attachments in BullMQ job returnvalue fields within the Chatwoot integration consumer.


Problem

When a WhatsApp media message (e.g., a 150MB PDF or video) is processed by WAHAMessageAnyConsumer:

  1. MessageBaseHandler.handle() calls buildChatWootMessage(), which reads the media file into a base64 string under attachments[].content.
  2. After sending the payload to Chatwoot via conversation.send(message), handle() returns message directly.
  3. BullMQ serializes the return value of processJob() into Redis under bull:chatwoot.waha:<jobId> (returnvalue).
  4. As a result, each processed media job retains 200MB+ of base64 data in Redis until cleaned up by retention policies (REMOVE_ON_COMPLETE_AGE/COUNT).

Solution

Sanitize attachments[].content to an empty string ('') in the object returned by handle().

  • Chatwoot API call (conversation.send): Receives the full message object with valid base64 media content intact.
  • BullMQ return value stored in Redis: Contains lightweight attachment metadata (filename, encoding) with empty content, reducing job footprint in Redis from hundreds of MBs to ~500 bytes.

Changes Included

  • src/apps/chatwoot/consumers/waha/base.ts: Updated MessageBaseHandler.handle() return value to sanitize attachments[].content.
  • src/apps/chatwoot/consumers/waha/base.test.ts: Added unit test verifying that conversation.send() receives the original base64 payload while the returned job result strips it.

Tested

  • corepack yarn lint
  • corepack yarn build

Verification

Ran unit tests locally:

npx jest --selectProjects unit src/apps/chatwoot/consumers/waha/base.test.ts

Output:

PASS unit src/apps/chatwoot/consumers/waha/base.test.ts
  MessageBaseHandler
    ✓ should sanitize attachment content to empty string on handle return

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant