Skip to content

fix(media-use): serialize parallel prefs record calls - #5460

Closed
JoAdo07 wants to merge 1 commit into
heygen-com:mainfrom
JoAdo07:fix/prefs-parallel-record
Closed

JoAdo07 wants to merge 1 commit into
heygen-com:mainfrom
JoAdo07:fix/prefs-parallel-record

Conversation

@JoAdo07

@JoAdo07 JoAdo07 commented Oct 11, 2026

Copy link
Copy Markdown

Summary

Parallel prefs.mjs record calls shared one fixed tmp name, so the loser's rename died with ENOENT, and read-modify-write races dropped keys (#5333). Tmp names now carry the pid (as telemetry.mjs does), and both read-modify-write paths (recordProjectTier, recordUserSighting) run under the existing withFileLock with a re-read inside — including creating the parent dir first, which the lock itself needs.

Test plan

  • New skills/media-use/scripts/prefs-concurrency.test.mjs: the issue's exact 4-parallel repro (all exit 0, all 4 keys kept) + sequential accumulation. 2/2 pass; the parallel case fails on unpatched code (verified via stash).
  • node --check clean. Repo checks (check-comment-citations, ratchet, package suites) could not run here — no bun on this machine and npm install isn't supported for workspace ops — CI will run them.

Closes #5333

Unique pid tmp names plus the existing withFileLock around both read-modify-write paths. Closes heygen-com#5333.

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking up #5333. The fix works: in a probe of 25 trials × 8 parallel node prefs.mjs record processes, this head kept every key and every user sighting, while main lost keys in about half the trials.

#5366 fixes the same issue with the same withFileLock approach, and we're going with that one, so I'd suggest closing this one as a duplicate. Here's why #5366 is ahead as submitted:

  • It patches both copies of prefs-store.mjs. This PR only changes skills/media-use/scripts/lib/prefs-store.mjs, so scripts/check-media-use-copy-parity.test.mjs fails ("standalone and CLI copies differ").
  • It regenerates skills-manifest.json. Here the media-use hash is stale, so the "Skills: manifest in sync" check would fail.
  • Its tests race separate processes behind a start barrier and fail on main every time. They also cover the user sighting ledger, re-reading after a held lock, and abandoned locks. With the fix reverted, prefs-concurrency.test.mjs failed on main in only 3 of 5 runs.

Smaller things that would also trip CI here: oxfmt wants prefs-store.mjs:144 wrapped and the two JSON.parse( blocks in the test collapsed. The comment at prefs-store.mjs:75 ("used to share…") is history narration, which the Comments check rejects.

Thanks again for the clear write-up and the repro.

— Rames

@JoAdo07

JoAdo07 commented Oct 22, 2026

Copy link
Copy Markdown
Author

Closing this in favor of #5366, which covers the same race with the copy-parity and manifest updates called out in review. Thanks for the detailed feedback and for validating the fix.

@JoAdo07 JoAdo07 closed this Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

media-use prefs.mjs: parallel record calls fail with ENOENT on a shared preferences.json.tmp and lose keys

2 participants