Skip to content

repository: single config/config text object (repo version 5) - #10377

Draft
ThomasWaldmann wants to merge 1 commit into
borgbackup:masterfrom
ThomasWaldmann:repo-config-file
Draft

ThomasWaldmann wants to merge 1 commit into
borgbackup:masterfrom
ThomasWaldmann:repo-config-file

Conversation

@ThomasWaldmann

@ThomasWaldmann ThomasWaldmann commented Sep 16, 2026

Copy link
Copy Markdown
Member

Follow-up to #10371 .. #10374, which emptied the manifest. Opening a repository loaded four store objects: config/readme, config/version, config/id and config/manifest (only read for its key type byte). Now it loads one:

# This is a Borg Backup repository.
# See https://borgbackup.readthedocs.io/

[repository]
version = 5
id = <64 hex digits>
encryption = aes256-ocb
id_hash = sha256

The config object

  • config/config is a plain text INI object (configparser), the readme text lives on as comment lines. encryption / id_hash record the crypto suite by the --encryption / --id-hash names; key_factory() selects the key class from them without reading any repository object. Both names are recorded or none: a config with one but not the other is invalid, an unsupported suite raises InvalidRepositoryConfig. Key storage (keyfile vs repokey) stays a property of each key.
  • Repository version 5, only 5 accepted. No code for the old layout, no migration (betas are for new repos only): an older beta repository fails to open with "repository version 4 is not supported".

No manifest object for borg 2 repositories. Manifest is only the in-memory container for key, repo_objs, repository and archives. Removed: write() and its callers, get_manifest() / put_manifest(), the manifest checks and rebuild in borg check, borg debug dump-manifest, key_from_repository() and the ROBJ_MANIFEST type (no stored object ever had it, borg 1.x objects carry no type at all). borg 1.x repositories are still read via their manifest (legacy_key_factory), since it holds their archives list. NoManifestError stays defined (never raised) so rc 26 remains reserved.

repo-create leaves nothing behind when it fails. Repository.create() writes the config (version, id, no key info) by default, so a repository created via the Python API is an openable key/value store. borg repo-create alone uses create_config=False and writes the config once, last, after the key exists, so nothing that looks like a repository exists until it succeeds: any failure or interruption before that destroys the store and, in keyfile mode, the keyfile just written; a failure inside create() itself destroys the store as well.

A store without config (only possible after a hard kill, or a repository that lost its config):

  • on open it is reported as not a valid repository (InvalidRepository); DoesNotExist is only for a missing store backend, the rest:// case from repo-info on a non-existent remote repo raises BackendDoesNotExist #10365;
  • repo-create on it says what borg knows, with the new IncompleteRepository error (rc 11): no repository config, so not a borg 2 repository or the leftover of an interrupted repo-create (the store backend refuses any non-empty directory, so this is all borg can tell);
  • borg repo-delete --force destroys it, but only if it looks like the leftover of an interrupted repo-create (a chunk index, but no packs and no archives), so that ssh/rest/s3 users can remove a leftover without other access to the storage, and a repository that merely lost its config is never destroyed;
  • key_factory() on a config without key info raises RepositoryKeyInfoMissing (rc 54).

Cache config: the [cache]/[integrity] cross-check on the manifest id could never detect anything (every borg that can open the repository knows the integrity section) and is dropped, together with its warning and test.

Docs: a new "Repository config" section replaces the manifest section, including how to recreate a lost config by hand (check --repair can not restore it); the layout list, packs (no migration from 4 to 5 either) and security internals (where the protection against a swapped crypto suite lives: the security directory and the unknown-unencrypted prompt for swaps to a non-encrypting suite, the security directory for swaps between encrypting suites), the repo-info example, the error list (rc 11, 54) and the remaining manifest mentions are updated. Sphinx builds without warnings.

Tests: manifest-specific tests removed; new tests for the config round trip, a store without config (open: invalid, create: incomplete, repo-delete --force: destroyed only if a leftover), a repository that lost its config (invalid, not destroyed), bad configs (wrong version, no section, missing / invalid / half-present entries, not text), key_class_for over all creatable suites, repo-create failing while writing the config or inside create() leaving neither store nor keyfile behind, and a plain data directory refused by repo-create and repo-delete --force.

Full suite passes locally (FUSE mount tests deselected, macFUSE is unavailable on this machine).

🤖 Generated with Claude Code

@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.03483% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.12%. Comparing base (3b905e0) to head (c7a58a1).
⚠️ Report is 35 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/borg/repository.py 93.68% 5 Missing and 1 partial ⚠️
src/borg/archiver/repo_delete_cmd.py 83.33% 2 Missing and 1 partial ⚠️
src/borg/archiver/_common.py 75.00% 1 Missing and 1 partial ⚠️
src/borg/cache.py 50.00% 1 Missing and 1 partial ⚠️
src/borg/archiver/debug_cmd.py 85.71% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10377      +/-   ##
==========================================
- Coverage   88.12%   88.12%   -0.01%     
==========================================
  Files         103      103              
  Lines       18928    18842      -86     
  Branches     2923     2921       -2     
==========================================
- Hits        16681    16604      -77     
+ Misses       1556     1552       -4     
+ Partials      691      686       -5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Comment thread docs/internals/packs.rst Outdated
Comment thread src/borg/crypto/key.py Outdated
Comment thread src/borg/legacy/archives.py
Comment thread src/borg/testsuite/archiver/compact_cmd_test.py Outdated
Comment thread src/borg/repository.py
@ThomasWaldmann
ThomasWaldmann marked this pull request as draft September 16, 2026 04:31
@ThomasWaldmann ThomasWaldmann changed the title repository: single config/config text object replaces readme, version, id and manifest (repo version 5) repository: single config/config text object (repo version 5) Sep 16, 2026
…, id and manifest (repo version 5)

Opening a repository loaded four store objects: config/readme (sanity check),
config/version, config/id and config/manifest (only read for its key type
byte since the manifest lost all other content). Now there is one plain text
INI object, config/config, e.g.:

    # This is a Borg Backup repository.
    # See https://borgbackup.readthedocs.io/

    [repository]
    version = 5
    id = <64 hex digits>
    encryption = aes256-ocb
    id_hash = sha256

The crypto suite of the key is recorded by the --encryption / --id-hash
names, so key_factory() selects the key class from the config without
reading any repository object. Both names are recorded or none; a config
with one but not the other is invalid, an unsupported suite raises
InvalidRepositoryConfig. The repository version is 5; only version 5 is
accepted, there is no code to read the old layout and no migration (betas
are for new repositories only).

The manifest object is gone for borg 2 repositories: Manifest is now only
the in-memory container for key, repo_objs, repository and archives; its
write() and all callers, get_manifest()/put_manifest(), the manifest checks
and rebuild in "borg check", "borg debug dump-manifest", the key type
detection from stored objects (key_from_repository) and the ROBJ_MANIFEST
type are removed. borg 1.x repositories are still read via their manifest
(legacy_key_factory), as it holds their archives list. NoManifestError is
kept (never raised) so that rc 26 stays reserved.

Repository.create() writes the config (version, id, no key info) right
away, so a repository created via the Python API is an openable key/value
store. "borg repo-create" alone defers it (create_config=False) and writes
the config once, last, after the key was created, so nothing that looks
like a repository exists until repo-create succeeds: any failure or
interruption before that destroys the store (and the keyfile, in keyfile
mode), and a failure inside create() destroys the store as well.

A store without repository config (only possible after a hard kill) is
reported as not a valid repository (InvalidRepository; DoesNotExist is only
for a missing store backend, the rest:// case from borgbackup#10365), repo-create on
it says what borg knows (IncompleteRepository, rc 11: no repository config,
not a borg 2 repository or the leftover of an interrupted repo-create), and
"borg repo-delete --force" destroys it if it looks like such a leftover (a
chunk index, but no packs and no archives), so that ssh/rest/s3 users can
remove it without other access to the storage. key_factory() on a config
without key info raises RepositoryKeyInfoMissing (rc 54).

The cache config's [cache]/[integrity] cross-check (manifest id) could
never detect anything and is dropped.

Docs: new "Repository config" section replaces the manifest section
(including how to recreate a lost config by hand, as check --repair can
not), the layout list, packs and security internals (where the protection
against a swapped crypto suite lives), repo-info example, error list and the
remaining manifest mentions are updated; no migration from version 4 to 5.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant