Fix performance regression in v2.55 related to having many packfiles - #971
Merged
Conversation
dscho
force-pushed
the
fix-perf-regression-in-v2.55
branch
from
August 4, 2026 23:12
0562289 to
70c8828
Compare
dscho
marked this pull request as ready for review
August 5, 2026 07:37
There was a problem hiding this comment.
Pull request overview
This PR addresses a performance regression when repositories contain very large numbers of packfiles by removing an O(n) “dedupe/removal” step from the hot path used while initially loading packfiles into a packfile_store. It also extends the perf suite to cover the rev-parse --short scenario that motivated the report.
Changes:
- Add a fast-path helper to append a new pack entry to a
packfile_listwithout scanning/removing. - Use that helper when adding packs to a
packfile_store, avoiding quadratic behavior during bulk pack discovery/loading. - Add a perf test that measures
git rev-parse --short HEADwith 10,000 packs.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
t/perf/p5303-many-packs.sh |
Adds a perf benchmark for rev-parse --short under extreme packfile counts. |
packfile.c |
Introduces packfile_list_append_new() and uses it in packfile_store_add_pack() to avoid O(n²) pack insertion behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
mjcheetham
previously approved these changes
Aug 5, 2026
Since 589127c (packfile: move list of packs into the packfile store, 2025-10-30), there is a performance regression when many packfiles need to be loaded: `packfile_store_add_pack()` now calls `packfile_list_remove_internal()` to detect whether the packfile was _already_ in the list, if if so, move it to the end of the list. This function linearly scans the existing list before every insertion. Newly loading N packs therefore has complexity O(N²). In one reported use case (#970), N equals 37,815 and caused a slow-down of a simple `git rev-parse --short HEAD` (which is regularly executed as part of `GIT_PS1`) from 0.4s to 4.5s. In another, heavily exercised CI scenario, clone times increased from under 2 minutes to over half an hour. Let's fix this by establishing a fast path for known-new packfiles. The keen reader will note that there is currently only a single, "known-new" caller of the `packfile_list_append()` function, and wonder why not simply remove this check whether the packfile already exists in the list? Originally, when above-mentioned commit introduced that logic, there was a second caller in `prepare_midx()`, which would have required that check, but that caller was removed in 6aff1f2 (packfile: always add packfiles to MRU when adding a pack, 2025-10-30). Still, the function is declared in a header file, and to avoid any problems with in-flight or downstream callers, it is safer to extend the signature to be explicit whether or not to skip that check. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho
force-pushed
the
fix-perf-regression-in-v2.55
branch
from
August 5, 2026 09:35
70c8828 to
11f7294
Compare
dscho
enabled auto-merge
August 5, 2026 09:37
mjcheetham
approved these changes
Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When there are many packfiles in the repository (or in an alternate of it), a regression introduced into Git v2.53 introduced complexity quadratic in the number of the packfiles. In one instance, causing a slow-down of a simple
git rev-parse --short HEAD(as used inGIT_PS1) from 0.4s to 4.5s. Let's fix that by avoiding the quadratic behavior in the most common case: loading the packfiles afresh.In another (internally-reported) instance, clone times that increased to over 30 minutes were reported to reduce to under 2 minutes with this patch.
This closes #970.