Skip to content

feat(vordr): auto-flag low-effort top-level comments#3903

Merged
idoshamun merged 3 commits into
mainfrom
eng-1571-auto-vordr-low-effort-comments-on-creation
May 24, 2026
Merged

feat(vordr): auto-flag low-effort top-level comments#3903
idoshamun merged 3 commits into
mainfrom
eng-1571-auto-vordr-low-effort-comments-on-creation

Conversation

@idoshamun
Copy link
Copy Markdown
Member

Summary

Adds a synchronous isLowEffortComment(content) check to top-level comment creation. When it returns true, the comment is auto-flagged through the existing Vordr flags.vordr path (no new column) and a telemetry counter + log line is emitted so we can monitor the rate post-launch.

Key decisions

  • Pure module at src/common/lowEffortComment.ts with no DB or network dependencies — vocab + normalization rules live in one place for easy tuning.
  • Top-level only: wired into commentOnPost; commentOnComment (replies) is intentionally untouched per the spec.
  • Templated squad welcomes excluded via a leading regex check so auto-generated welcomes don't get flagged.
  • OR'd into existing vordr path: vordr: lowEffort || (await checkWithVordr(...)) — same column, same downstream behavior; no schema change.
  • Telemetry: reuses the existing counters.api.vordr counter with reason: 'low_effort' (consistent with vordr_ip / vordr_word) plus a comment_low_effort_autoflagged log line carrying userId, commentId, postId.
  • Baseline from prod data suggests ~22% of top-level comments will match.

Test plan

  • Unit tests at __tests__/common/lowEffortComment.test.ts — 26 cases covering positive, negative, and edge scenarios from the spec
  • __tests__/comments.ts mutation commentOnPost + vordr integration suites pass with no snapshot drift
  • pnpm run lint clean
  • pnpm run build clean
  • Watch comment_low_effort_autoflagged log volume + vordr counter (reason=low_effort) post-deploy to tune vocab

Closes ENG-1571


Created by Huginn 🐦‍⬛

idoshamun and others added 2 commits May 24, 2026 13:00
Adds a pure `isLowEffortComment` filter that runs synchronously on every
new top-level comment. When it matches, the comment's vordr flag is set
inline through the existing flags mechanism and a counter/log line is
emitted so we can monitor the auto-flag rate. Replies and templated
squad welcomes are exempt.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drop dead empty-string entry and duplicate vocab tokens (`deserved`,
  `damnn`, `goo`) — Set already dedups but the source list shouldn't.
- Inline normalize() as a fluent chain; collapse the two length checks
  into one (0 < 4) and drop the unused `norm`/`words` dual return.
- Tighten the resolver wire-in by reusing `commentId` / `content`
  instead of re-reading them from `createdComment`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@pulumi
Copy link
Copy Markdown

pulumi Bot commented May 24, 2026

🍹 The Update (preview) for dailydotdev/api/prod (at 295b2d2) was successful.

✨ Neo Explanation

Routine deployment of a new image adding low-effort comment auto-flagging, with migration jobs cycling to the new commit. ✅ Low Risk

This is a standard application deployment rolling out commit 295b2d2 (previously 798dc603). The new image contains the low-effort comment detection feature — a vocabulary-based classifier that short-circuits the Vordr moderation check for comments like "nice" or "👍", logging and incrementing a telemetry counter instead of calling the external filter.

The migration Jobs (both DB and Clickhouse) are being cycled to the new commit hash, which is how this pipeline runs schema migrations on each deploy. No schema changes are visible in the diff, but the migration jobs will run against the new image regardless.

🔵 Info — The lowEffort flag now short-circuits checkWithVordr, meaning low-effort comments are flagged purely by the local classifier without calling the Vordr service. This is an observable behavioral change: Vordr will see fewer comment checks, and the vordr counter on affected comments will be set to true without Vordr's input. Verify that downstream systems consuming the vordr flag on comments handle this correctly and that the telemetry reason: 'low_effort' counter is wired up as expected.

Resource Changes

    Name                                                       Type                           Operation
~   vpc-native-worker-job-deployment                           kubernetes:apps/v1:Deployment  update
+   vpc-native-api-clickhouse-migration-7384d8c9               kubernetes:batch/v1:Job        create
~   vpc-native-update-current-streak-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-private-deployment                              kubernetes:apps/v1:Deployment  update
~   vpc-native-daily-digest-cron                               kubernetes:batch/v1:CronJob    update
-   vpc-native-api-db-migration-798dc603                       kubernetes:batch/v1:Job        delete
~   vpc-native-clean-zombie-users-cron                         kubernetes:batch/v1:CronJob    update
~   vpc-native-hourly-notification-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-deployment                                      kubernetes:apps/v1:Deployment  update
~   vpc-native-check-analytics-report-cron                     kubernetes:batch/v1:CronJob    update
~   vpc-native-update-source-public-threshold-cron             kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-analytics-clickhouse-cron          kubernetes:batch/v1:CronJob    update
~   vpc-native-update-trending-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-post-analytics-clickhouse-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-channel-digests-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-validate-active-users-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-expire-super-agent-trial-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-updated-sync-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-generate-search-invites-cron                    kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-user-posts-analytics-refresh-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-update-highlighted-views-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-temporal-deployment                             kubernetes:apps/v1:Deployment  update
~   vpc-native-calculate-top-readers-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-bg-deployment                                   kubernetes:apps/v1:Deployment  update
~   vpc-native-user-profile-analytics-history-clickhouse-cron  kubernetes:batch/v1:CronJob    update
~   vpc-native-update-achievement-rarity-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-images-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-user-companies-cron                kubernetes:batch/v1:CronJob    update
-   vpc-native-api-clickhouse-migration-798dc603               kubernetes:batch/v1:Job        delete
+   vpc-native-api-db-migration-7384d8c9                       kubernetes:batch/v1:Job        create
~   vpc-native-clean-expired-better-auth-sessions-cron         kubernetes:batch/v1:CronJob    update
~   vpc-native-materialize-monthly-best-post-archives-cron     kubernetes:batch/v1:CronJob    update
~   vpc-native-materialize-yearly-best-post-archives-cron      kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-old-notifications-cron                    kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-opportunities-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-rotate-daily-quests-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-stale-user-transactions-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-deployment                  kubernetes:apps/v1:Deployment  update
~   vpc-native-rotate-weekly-quests-cron                       kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-channel-highlights-cron                   kubernetes:batch/v1:CronJob    update
... and 12 other changes

Two FPs surfaced during data validation:
- '@user welcome as well!!!' (and 'welcome aboard', 'welcome back', …) —
  the templated-welcome filter was anchored to 'welcome to <X>'. Broaden
  to 'welcome\\b' so any greeting continuation is exempt.
- Comments containing a markdown image/GIF embed (e.g. tenor/klipy GIFs,
  with or without surrounding text) were being flagged because after URL
  strip only 'GIF' or trailing junk remained. Add an IMAGE_EMBED guard
  that exempts any comment containing '![alt](url)' (also tolerates the
  angle-bracketed URL form '![alt](<url>)').

Adds Jest cases for both FP classes plus a few collapse-variant positives
(Damnnnnnn, Greatt!!!).
@idoshamun
Copy link
Copy Markdown
Member Author

Pushed 295b2d2 to apply two FP fixes that came up after the original ticket was filed:

  • Templated welcome regex broadened from welcome to <X> to just welcome\b so variants like welcome aboard, welcome back, welcome as well are also exempt.
  • Image/GIF embed carve-out — any comment containing ![alt](url) markdown (with or without surrounding text, including the angle-bracketed URL form ![alt](<url>)) is now exempt. Was triggering FPs on tenor/klipy GIF reactions.

Test suite extended with explicit cases for both classes (@user welcome aboard/back/as well, GIF with trailing ??, GIF with leading text, angle-bracketed URL form) plus a couple of letter-collapse positives (Damnnnnnn, Greatt!!!).

@idoshamun idoshamun merged commit f19358a into main May 24, 2026
9 checks passed
@idoshamun idoshamun deleted the eng-1571-auto-vordr-low-effort-comments-on-creation branch May 24, 2026 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant