[workstonedai-collab]

The change

Optimize sqlite_utils.utils.dedupe_keys by caching the next suffix to try for each repeated original key. This avoids repeatedly scanning already-reserved suffixes when many duplicate column names occur.

The implementation preserves the existing behavior for duplicate aliases and adds coverage for generator inputs, repeated suffix bases, and a scaling regression.

Closes #912: #912

Before

The previous implementation restarted suffix search at _2 for every duplicate key. For a large number of duplicate names, this repeatedly formatted and checked suffixes that had already been rejected.

After

The implementation remembers the next suffix for each repeated base key and continues from there. Candidate names are still checked against both existing names and names reserved by the original input, so the result remains unique.

Testing

The following results were reported for this change:

• 1501 passed, 16 skipped
• changed-module suite: 74 passed
• mypy over 10 source files: passed
• Black and flake8: passed
• Complexity regression: the old implementation performed 1,999,000 format calls, exceeding the budget of 2,000; the new implementation passes.

The synthetic benchmark reported these medians over 5 runs on Python 3.12, macOS arm64:

duplicates-500: baseline=0.006974s current=0.000058s
duplicates-2000: baseline=0.119233s current=0.000218s
duplicates-5000: baseline=0.780338s current=0.000614s
reserved-suffix-500: baseline=0.024951s current=0.000124s
reserved-suffix-2000: baseline=0.420098s current=0.000463s

These are helper microbenchmarks for dedupe_keys, not end-to-end database performance measurements.

Independent equivalence check: all 19,531 sequences of length 0–6 over id, id_2, id_3, id_2_2, and x matched the frozen baseline exactly.

AI assistance: MiniMax-M3.1-Flash implemented the optimization, tests and benchmark, including repairs after test feedback. Codex reviewed the diff and ran the listed validation.

---

📚 Documentation preview 📚: https://sqlite-utils--913.org.readthedocs.build/en/913/