Skip to content

fix: use canonical highlight history for bragi evaluation#3901

Merged
idoshamun merged 1 commit into
mainfrom
codex/canonical-highlight-history
May 24, 2026
Merged

fix: use canonical highlight history for bragi evaluation#3901
idoshamun merged 1 commit into
mainfrom
codex/canonical-highlight-history

Conversation

@idoshamun
Copy link
Copy Markdown
Member

@idoshamun idoshamun commented May 24, 2026

Summary

  • fetch Bragi evaluation history from global highlights_canonical instead of legacy post_highlight rows
  • generate/evaluate canonical highlights once, then collect one canonical projection tagged with the channels selected for publish
  • fan out saved canonical rows into legacy post_highlight rows per channel using channel settings and snapshots
  • keep canonical updatedAt behavior covered and update channel-highlight tests for global canonical history

Testing

  • NODE_ENV=test npx jest tests/cron/channelHighlights.ts --testEnvironment=node --runInBand
  • npx eslint src/common/channelHighlight/generate.ts src/common/channelHighlight/publish.ts src/common/channelHighlight/queries.ts tests/cron/channelHighlights.ts --max-warnings 0
  • npx tsc --noEmit

Note: pnpm currently fails before running scripts in this checkout with packages field missing or empty, so verification used npx directly.

@pulumi
Copy link
Copy Markdown

pulumi Bot commented May 24, 2026

🍹 The Update (preview) for dailydotdev/api/prod (at f567ec9) was successful.

✨ Neo Explanation

Routine image rollout deploying a significant internal refactor of the channel highlights pipeline (per-channel → single global canonical evaluation), plus a new Pub/Sub subscription. No stateful resource replacements or deletions; the main thing to verify is that the new `major-headline-added-notification` subscription has a ready consumer. ✅ Low Risk

This PR refactors the channel highlights generation pipeline from a per-channel model to a single global canonical run. The generateChannelHighlights function (which ran one evaluation per configured channel) is replaced by generateHighlights, which runs a single unified evaluation against HighlightsCanonical as the source of truth, then fans out to legacy per-channel PostHighlight records. The evaluateChannelHighlights function is renamed to evaluateHighlights and now always uses fixed global channel/targetAudience constants rather than per-channel config. Run tracking shifts from one ChannelHighlightRun per channel to a single global run per execution.

The deployment rolls out a new container image (de9a17a2) across all workloads, runs fresh DB and Clickhouse migration jobs, and adds a new Pub/Sub subscription (api-sub-api.major-headline-added-notification).

🔵 Info — The new api.major-headline-added-notification Pub/Sub subscription will begin receiving messages immediately after creation. Confirm a corresponding consumer/push endpoint is ready to handle messages before or at deploy time, or messages will accumulate unprocessed.

🔵 Info — The first cron execution after deploy will have no prior global channel run record, so lastFetchedAt will be null and fetchStart will fall back to the full horizon window. This is a one-time wider fetch, not a bug, but worth monitoring for elevated DB query load on the first run.

Resource Changes

    Name                                                    Type                           Operation
~   vpc-native-clean-zombie-opportunities-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-update-views-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-old-notifications-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-update-tag-materialized-views-cron           kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-images-cron                     kubernetes:batch/v1:CronJob    update
~   vpc-native-update-source-public-threshold-cron          kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-cron                     kubernetes:batch/v1:CronJob    update
~   vpc-native-update-achievement-rarity-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-temporal-deployment                          kubernetes:apps/v1:Deployment  update
~   vpc-native-user-profile-updated-sync-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-post-analytics-history-day-clickhouse-cron   kubernetes:batch/v1:CronJob    update
~   vpc-native-rotate-weekly-quests-cron                    kubernetes:batch/v1:CronJob    update
~   vpc-native-sync-subscription-with-cio-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-check-analytics-report-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-ws-deployment                                kubernetes:apps/v1:Deployment  update
~   vpc-native-clean-channel-highlights-cron                kubernetes:batch/v1:CronJob    update
~   vpc-native-update-trending-cron                         kubernetes:batch/v1:CronJob    update
~   vpc-native-update-highlighted-views-cron                kubernetes:batch/v1:CronJob    update
~   vpc-native-update-current-streak-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-user-posts-analytics-refresh-cron            kubernetes:batch/v1:CronJob    update
~   vpc-native-worker-job-deployment                        kubernetes:apps/v1:Deployment  update
~   vpc-native-user-profile-analytics-clickhouse-cron       kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-user-companies-cron             kubernetes:batch/v1:CronJob    update
~   vpc-native-channel-highlights-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-deployment               kubernetes:apps/v1:Deployment  update
-   vpc-native-api-db-migration-798dc603                    kubernetes:batch/v1:Job        delete
~   vpc-native-post-analytics-clickhouse-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-expire-super-agent-trial-cron                kubernetes:batch/v1:CronJob    update
~   vpc-native-materialize-monthly-best-post-archives-cron  kubernetes:batch/v1:CronJob    update
-   vpc-native-api-clickhouse-migration-798dc603            kubernetes:batch/v1:Job        delete
~   vpc-native-bg-deployment                                kubernetes:apps/v1:Deployment  update
~   vpc-native-squad-posts-analytics-refresh-cron           kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-gifted-plus-cron                       kubernetes:batch/v1:CronJob    update
+   vpc-native-api-db-migration-de9a17a2                    kubernetes:batch/v1:Job        create
~   vpc-native-private-deployment                           kubernetes:apps/v1:Deployment  update
~   vpc-native-clean-zombie-users-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-hourly-notification-cron                     kubernetes:batch/v1:CronJob    update
+   vpc-native-api-clickhouse-migration-de9a17a2            kubernetes:batch/v1:Job        create
~   vpc-native-update-tags-str-cron                         kubernetes:batch/v1:CronJob    update
~   vpc-native-deployment                                   kubernetes:apps/v1:Deployment  update
~   vpc-native-clean-expired-better-auth-sessions-cron      kubernetes:batch/v1:CronJob    update
... and 13 other changes

@idoshamun idoshamun force-pushed the codex/canonical-highlight-history branch 8 times, most recently from cb7a0c6 to 5d29acf Compare May 24, 2026 12:55
@idoshamun idoshamun force-pushed the codex/canonical-highlight-history branch from 5d29acf to f567ec9 Compare May 24, 2026 13:03
@idoshamun idoshamun merged commit 3571d00 into main May 24, 2026
9 checks passed
@idoshamun idoshamun deleted the codex/canonical-highlight-history branch May 24, 2026 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant