Stop guessing TikTok video ids from captions

[paulocastellano]

Problem

Nightwatch #70: LogicException: The remote id is already attached to another TryPost publication. in CollectPublicationMetrics.

When TikTok had not reported a post's public video id yet, TikTokPublisher fell back to video/list and took the newest video whose caption started the same way, created up to a day earlier. On channels whose captions all open alike (bcwc.cloud #BCWC…, #TeacherLife…) the new post's video was not listed yet, so each post took the previous post's video. When TikTok later reported the real id for that previous post, the video was already held by the next one and reconciliation threw.

On the prod clone: 17 posts on 4 channels carry another post's video (wrong metrics and link), and some of their real videos were imported as duplicate posts. Publishing itself was never affected: each video went out once, correctly.

Fix

TikTok sends publicaly_available_post_id only after moderation, which usually takes under a minute but can take hours (Get Post Status). On the clone it arrived weeks later. So TryPost now waits for that id instead of guessing:

• No more caption matching in TikTokPublisher. Until TikTok reports the id, a post keeps its publish_id and the profile URL, and stays published.
• ResolveTikTokVideoId, dispatched by PublishToSocialPlatform one minute after a public TikTok post is published without its video id, asks TikTokPublisher::publicVideoId(). Moderation usually ends within a minute, so this resolves most posts. social:resolve-tiktok-video-ids (every 15 minutes, a database query that calls TikTok only for posts whose recheck is due) catches slower reviews: on every run through a post's first hour, hourly through its first day, then daily for 30 days, following the Google Business reconcile pattern (last_reconciled_at). Only posts public to everyone are asked (TikTok never reports an id for followers, friends or private posts), and posts on channels that are not connected are skipped. The check is recorded before the id is assigned, so a failure follows the same cadence instead of repeating every run; a network timeout counts as not reported yet. Only a numeric id counts as a video id, the same rule that decides a post still awaits one.
• AssignTikTokVideoId writes the id for the resolver and the metrics job, under a lock on the post (a post deleted meanwhile is left alone): it sets the video URL on the post and on its analytics publication (keeping the current one when the channel has no username) and moves the analytics publication through reconcileRemoteId, which merges snapshots and drops an imported duplicate. A post published before it had an analytics publication (30 of the 71 waiting posts on the clone) gets one first through SyncTryPostPublication, so it gets the same merge and the same conflict guard. The only other writer is ImportExternalPosts::claimedBySentPost (exact text, single candidate, deferred when ambiguous); for TikTok it now only claims for posts still on a publish_id, so it can no longer move a resolved post to a native video with the same caption (pre-existing gap).
• The LogicException stays as an invariant guard: a video another TryPost post holds is never assigned. The resolver logs it once a day instead of failing the job; the repair command fixes those posts. A video discovered at the same moment (unique-index race) leaves the post for the next check.
• AssignTikTokVideoId retries its transaction on a deadlock (it locks the post before the publication; the importer locks them the other way).
• TikTokAnalytics is deleted. It had no caller since the analytics pipeline (feat: independent social posts, workspace analytics and media editing #374) replaced it; its per-post metrics read was dead code. Its one live test (reading saved TikTok metrics through ReadPublicationAnalytics) moved to PersistedPostMetricsReadTest.
• Post::scopePublishedToTikTok() for the query the sweep and the repair share.
• Rule recorded in .ai/rules/tiktok.md.

Repairing existing data

Run right after the deploy: php artisan tiktok:repair-video-ids --dry-run, then without --dry-run. Both phases read the video's creation time from analytics_publications.provider_published_at, which discovery rewrites with TikTok's own value; on the prod clone the dry run found the 17 expected posts, so check the production dry run reports a similar number. The dry run executes everything in a transaction and rolls it back, so its numbers are exact; media file deletions are only queued after a commit and are dropped with it. This is a one-off; remove it in a follow-up once it has run.

Phase 1 — a post holding the previous post's video. Only posts that still have a channel are considered. A post is repaired only when both hold:

1. The video it holds was created right before another post of the channel was published.
2. Exactly one video was created right before the post itself.

The second check matters: posts whose video was created long before published_at can be correct slow publishes (TikTok's status took 11–48 minutes) and are left alone. Repairs are filtered until they settle: a repair is kept only when its own video is claimed by no other repair and is free (unlinked, imported, or held by a post that is itself repaired). An imported copy of the returned video is deleted.

Phase 2 — a video several posts of a channel hold. Only published TryPost posts and imported posts take part. The TryPost post published right after the video was created keeps it, but only when that video is the only one created in the two minutes before it (the same evidence phase 1 requires, through one shared lookup); otherwise the video is left alone, so a native video is never deleted on weak evidence. Each other TryPost post gets its own video back when exactly one free video was created right before it (deleting its imported copy), and only otherwise loses the id and points at the profile again (the importer can still link its real video by exact text later). Imported copies of the shared video are deleted. Each video is settled with the posts holding it at that moment, since settling an earlier video can hand one of them its own video back. These came from the old post page caption match, which scanned newest first and could hand a post a later post's video.

Then the command dispatches the resolver for every post still on a publish_id on a connected channel, including those older than 30 days.

Run on the prod clone (queue sync, no worker, so nothing else in the queue ran; storage on the local bucket):

• Phase 1: 17 posts got their own video back; 8 imported copies deleted.
• Phase 2: 6 posts released, 1 imported copy deleted.
• Resolver: 68 posts asked TikTok, 7 got their real id (including July posts: old publish_ids still answer). Most of the rest have no public id yet or had an expired token locally (the local TikTok app cannot refresh production tokens; production refreshes them every 15 minutes).
• After: 0 video ids shared by two posts of a channel, 0 publications pointing at a different video than their post, 0 failed jobs; a second dry run finds nothing to repair.

Tests

• New: ResolveTikTokVideoIdTest (resolves and moves the publication, waits while there is no id, skips private/resolved/imported posts, deleted post, sweep cadence and 30-day cap), AssignTikTokVideoIdTest (no publication, no username, conflict writes nothing) and RepairTikTokVideoIdsCommandTest (imported duplicate, a run of consecutive wrong posts, cascade and double-claim guards, slow publish untouched, shared videos with and without a single owner, an orphaned copy settled in the same run, dry run rolled back with media deletions deferred to a commit). The guard tests fail against the single-pass filter.
• The metrics job test now asserts the video URL.
• TikTokPublisherTest: publicVideoId (string/integer id, still in moderation, not a numeric id, rejected or unavailable status fetch, network timeout, rejected token refresh); the publish test asserts video/list is never called. Caption-match tests removed with the code.
• The sweep skips imported, failed and retrying posts; a video held by another post records the check and is not re-dispatched before its cadence; a second repair run finds nothing.
• TikTok fixtures that used video_123 as the public video id now use numeric ids, as TikTok's int64 field does.
• Full suite in parallel (PostgreSQL): 9606 passed. Affected tests on MySQL: passed.