name content-addressed objects by blake3 instead of sha256

docs/faq.rst

@@ -582,7 +582,7 @@ If you create a new keyfile-encrypted repository at the same filesystem
 path multiple times (for example, when a previous repository at that path
 was moved away or unmounted), Borg will not overwrite or reuse an existing
 key file in your keys directory. Instead, each repository gets a key file
-of its own, named after the SHA-256 hash of the key file's own content --
+of its own, named after the BLAKE3 hash of the key file's own content --
 the header line naming the repository's ID, followed by the encrypted key
 material. Both the repository ID and the key material are freshly
 randomized when the repository is created, so two repositories, even ones
@@ -598,7 +598,7 @@ names shortened for readability):
 
 Each belongs to a distinct repository, wherever it was created -- a name
 collision between different repositories would require an outright
-SHA-256 hash collision, not just an unlucky path reuse. Borg does not use
+BLAKE3 hash collision, not just an unlucky path reuse. Borg does not use
 the key file name to find the right key either: to open a repository, it
 scans all files in the keys directory (see :ref:`env_vars` for
 ``BORG_KEYS_DIR``) and picks the one whose header names that repository's

docs/internals/data-structures.rst

@@ -50,7 +50,7 @@ archives/
 
 The (encrypted and compressed) repository objects are not stored one store
 object each: many of them are batched into a **pack** file and that pack file
-is stored as a single store object. Its name is the hex-encoded sha256 hash of
+is stored as a single store object. Its name is the hex-encoded blake3 hash of
 the pack file's content:
 
 packs/
@@ -61,7 +61,7 @@ index/
   0000... .. ffff...
     the chunks index (chunk ID -> location within a pack file), stored as a set
     of immutable, encrypted index fragments. A fragment's name is the
-    hex-encoded sha256 hash of its content.
+    hex-encoded blake3 hash of its content.
 
 See :ref:`packs` for the pack file format, the ``index/`` namespace and how
 both are written and compacted.
@@ -77,7 +77,7 @@ cache/
     check finishes.
   referenced-by-archive.<hex-encoded archive ID>
     what one archive references (object ID -> plaintext object size), plus the file
-    count and content size of that archive, with an appended sha256 for integrity.
+    count and content size of that archive, with an appended blake3 hash for integrity.
     It lets a following ``borg compact`` or ``borg analyze`` skip re-reading the items
     of an unchanged archive.
   chunkindex-invalid
@@ -92,7 +92,7 @@ all clients); it is not the client-local cache described in
 
 keys/
     When using repokey mode, the encrypted, passphrase protected borg keys are
-    stored here as a base64 encoded text. The sha256 content hash of the
+    stored here as a base64 encoded text. The blake3 content hash of the
     stored borg key is used for the name.
 
     A repository may contain *multiple* such borg keys (one per passphrase) to
@@ -724,7 +724,7 @@ The files cache
 
 The **files cache** is a client-local file, stored in the borg cache directory
 of the repository (see :ref:`env_vars`) as ``files.<SUFFIX>``. SUFFIX is the
-sha256 of the archive (series) name, so each archive series gets its own files
+blake3 hash of the archive (series) name, so each archive series gets its own files
 cache; ``BORG_FILES_CACHE_SUFFIX`` overrides it. The files cache is used at
 backup time to quickly determine whether a given file is unchanged and we have
 all its chunks.
@@ -1116,7 +1116,7 @@ All modes
 
 Encryption keys (and other secrets) are kept either in the keys directory on
 the client ('keyfile' mode) or under the keys/ namespace in the repository
-('repokey' mode) using the sha256 of the borg key content as the name.
+('repokey' mode) using the blake3 hash of the borg key content as the name.
 
 In both cases, the secrets are generated from random and then encrypted by a
 key derived from your passphrase (this happens on the client before the key

docs/internals/packs.rst

@@ -120,7 +120,7 @@ blobs after the damaged one are still found; the damaged blob itself is dropped,
 it can not be read back.
 
 The walk rebuilds the index from the pack as it is: the damaged bytes stay where
-they are, as a gap no index entry covers. A pack is named by the sha256 of its
+they are, as a gap no index entry covers. A pack is named by the blake3 hash of its
 content, so a pack damaged in the store keeps failing the store-level check that
 ``borg check`` runs over ``packs/``, also after ``borg check --repair`` has
 rebuilt the index from it. Rewriting such a pack is repository-level repair, see
@@ -161,9 +161,9 @@ Blobs follow one another contiguously with no padding::
 Pack ID
 ~~~~~~~
 
-The pack ID is the SHA-256 of the pack file's bytes::
+The pack ID is the 256 bit BLAKE3 hash of the pack file's bytes::
 
-    pack_id = sha256(pack_bytes)
+    pack_id = blake3(pack_bytes)
 
 Content-addressing the file by its own bytes makes the name commit to the
 content, so borgstore can verify and cache it and ``borg check`` can detect
@@ -258,10 +258,10 @@ A fragment is a serialized ``ChunkIndex`` (a ``borghash`` ``HashTableNT`` keyed
 of each entry are zeroed before serializing. Fragments are **not** encrypted: they map
 ``chunk_id`` to ``(pack_id, obj_offset, obj_size)``, which anyone with access to the
 repository could equally well read out of the unencrypted blob headers (see
-:ref:`pack-recovery`). A fragment's name is the SHA-256 digest of its own content::
+:ref:`pack-recovery`). A fragment's name is the BLAKE3 digest of its own content::
 
     index/
-      <sha256_of_content_hex>
+      <blake3_of_content_hex>
 
 An ordinary backup writes only the entries that are new in that session; a full
 rewrite (e.g. by ``borg compact``) writes all of them. In both cases the write is

docs/internals/security.rst

@@ -309,7 +309,7 @@ The ciphertext is then converted to base64.
 
 This base64-encoded *borg key* is then stored in the key file or under the
 repository's ``keys/`` namespace (keyfile and repokey modes respectively), named
-by the sha256 of its content.
+by the blake3 hash of its content.
 
 The use of a constant IV is secure because an identical passphrase will
 result in a different derived KEK for every key encryption due to the salt.
@@ -325,7 +325,7 @@ key material. This lets several people access a shared repository with
 independent passphrases, without sharing one secret. Or you can add borg keys
 for redundant, more fault-tolerant storage.
 
-keyfile and repokey borg keys use the same format and the same sha256-content
+keyfile and repokey borg keys use the same format and the same blake3-content
 naming; borg locates a borg key independently of its key type byte and tries each
 available one against the supplied passphrase until one decrypts. A borg key may
 carry a label for management. The constant-IV argument above still holds, because
@@ -377,7 +377,7 @@ used:
   object's metadata slot and data slot are encrypted and authenticated with the borg
   key (see :ref:`security_encryption`); its per-object header is unencrypted and
   carries the magic, the format version and the chunk id (see :ref:`pack-format`).
-- ``index/<sha256>`` -- the chunk id to pack location index. It is not encrypted,
+- ``index/<blake3>`` -- the chunk id to pack location index. It is not encrypted,
   but it only contains chunk ids and locations, which the pack headers expose anyway.
 - ``archives/<hex(archive_id)>`` -- one empty object per archive. The archive name,
   its timestamps, the item metadata and the chunk lists all live inside encrypted
@@ -386,7 +386,7 @@ used:
   modification time.
 - ``config/manifest`` (an encrypted repository object), plus the plaintext
   ``config/version``, ``config/id`` and ``config/readme``.
-- ``keys/<sha256>`` -- in ``repokey`` mode, the borg key(s), encrypted with the
+- ``keys/<blake3>`` -- in ``repokey`` mode, the borg key(s), encrypted with the
   passphrase-derived KEK (see :ref:`key_encryption`).
 - ``locks/*`` and ``cache/*``. Note that the per-archive reference caches
   ``cache/referenced-by-archive.<hex(archive_id)>``, written by ``borg compact`` and

docs/usage/key.rst

@@ -31,7 +31,7 @@ Examples
 
 .. note::
 
-    Automatically placed key files are named after the SHA-256 hash of their own
+    Automatically placed key files are named after the BLAKE3 hash of their own
     contents, not after the repository directory name. Because changing the
     passphrase re-encrypts the key, the key file is rewritten under a new name and
     the previous one is removed — that is why the two paths above differ. Use

src/borg/archive.py

@@ -2203,7 +2203,7 @@ def check(
         self.format = format
         self.repository = repository
         # A normal (non-repair) archives check trusts the in-repo index: the repository check verified
-        # each index object's sha256, and the index is the authoritative record of which chunks exist,
+        # each index object's blake3 hash, and the index is the authoritative record of which chunks exist,
         # so we do not rebuild it from the packs (reading every pack is far too slow for a routine check).
         # --repair does rebuild from the packs (slow_rebuild=repair), working from the real packs so it
         # can detect and fix archives that reference chunks whose pack has gone missing.

src/borg/archiver/check_cmd.py

@@ -112,12 +112,12 @@ def build_parser_check(self, subparsers, common_parser, mid_common_parser):
         It consists of two major steps:
 
         1. Checking the consistency of the repository itself. The objects in the ``index/``
-           and ``packs/`` namespaces are named by the sha256 hash of their content, so such
+           and ``packs/`` namespaces are named by the blake3 hash of their content, so such
            an object is intact if and only if the hash of its content still equals its name.
            The check verifies the (small) index objects first and, only if they are intact,
            all packs. It also cross-checks the chunk index against the packs present in the
            repository to detect referenced but missing packs. Bit rot and other types of
-           accidental damage can be detected this way, but as sha256 content-addressing is
+           accidental damage can be detected this way, but as content-addressing is
            not a MAC, this step does not detect tampering. Running the repository check can
            be split into multiple partial checks using ``--max-duration``.
            For rest:// repositories, the server computes the hashes, so the pack contents do

src/borg/cache.py

@@ -1,5 +1,4 @@
 import configparser
-import hashlib
 import io
 import os
 import shutil
@@ -38,6 +37,7 @@
 from .helpers.msgpack import int_to_timestamp, timestamp_to_int
 from .item import ChunkListEntry
 from .crypto.file_integrity import IntegrityCheckedFile, FileIntegrityError
+from .crypto.key import blake3_256, blake3_256_hex
 from .manifest import Manifest
 from .platform import SaveFile
 from .repository import Repository, StoreObjectNotFound, PackReader
@@ -57,7 +57,7 @@ def files_cache_name(archive_name, files_cache_name="files"):
     # when not, the user may manually do that by using the env var.
     if not suffix:
         # avoid issues with too complex or long archive_name by hashing it:
-        suffix = hashlib.sha256(archive_name.encode()).hexdigest()
+        suffix = blake3_256_hex(archive_name.encode())
     return files_cache_name + "." + suffix
 
 
@@ -595,7 +595,7 @@ def list_chunkindex_fragments(repository):
     """List the index/ fragments, returning each fragment's (name, approximate entry count).
 
     This is the single primitive that walks the index/ namespace; list_chunkindex_hashes is a thin
-    wrapper over it. In that namespace each object's name is the sha256 hash of its content. The entry
+    wrapper over it. In that namespace each object's name is the blake3 hash of its content. The entry
     count is estimated from the stored object's byte size (chunkindex_fragment_entry_size() bytes per
     entry), so we can classify fragments (small vs. sealed) without loading them. The estimate ignores
     the small fixed header, which is negligible for the fragment sizes we care about.
@@ -663,10 +663,10 @@ def delete_chunkindex_from_repo(repository):
 
 
 def _store_chunkindex_fragment(repository, batch, stored_hashes, *, force_write):
-    """Serialize a temporary ChunkIndex `batch` and store it as an index/<sha256> fragment.
+    """Serialize a temporary ChunkIndex `batch` and store it as an index/<blake3> fragment.
 
     We don't serialize the flags or the size, so callers pass entries with those zeroed. The object
-    is stored under index/<hash>, where <hash> is the sha256 of its content, so borgstore can verify
+    is stored under index/<hash>, where <hash> is the blake3 hash of its content, so borgstore can verify
     it like any other object; an incompatible format from a different borg version is rejected by
     borghash's own versioned header (MAGIC + VERSION) when read back.
 
@@ -676,7 +676,7 @@ def _store_chunkindex_fragment(repository, batch, stored_hashes, *, force_write)
     with io.BytesIO() as f:
         batch.write(f)
         data = f.getvalue()
-    new_hash = hashlib.sha256(data).hexdigest()
+    new_hash = blake3_256_hex(data)
     stored = False
     if force_write or new_hash not in stored_hashes:
         index_name = f"index/{new_hash}"
@@ -820,7 +820,7 @@ def read_chunkindex_from_repo(repository, hash):
     except StoreObjectNotFound:
         logger.debug(f"{index_name} not found in the repository.")
     else:
-        if hashlib.sha256(chunks_data).digest() == hex_to_bin(hash):
+        if blake3_256(chunks_data) == hex_to_bin(hash):
             logger.debug(f"{index_name} is valid.")
             try:
                 with io.BytesIO(chunks_data) as f:
@@ -1050,7 +1050,7 @@ def build_chunkindex_from_repo(
 # cache/referenced-by-archive.<archive id hex>. it lets a following compact or analyze skip re-scanning
 # an unchanged archive's items. the blob is: file_count (uint64 LE), content_size (uint64 LE), a
 # serialized HashTableNT mapping object id (32 bytes) -> plaintext object size (uint32), and a
-# sha256 of all of that appended for integrity.
+# blake3 hash of all of that appended for integrity.
 REFERENCED_BY_ARCHIVE = "referenced-by-archive."  # name prefix within the "cache" store namespace
 ArchiveReferenceEntry = namedtuple("ArchiveReferenceEntry", "size")
 ArchiveReferenceEntryFormatT = namedtuple("ArchiveReferenceEntryFormatT", "size")
@@ -1081,11 +1081,11 @@ def load_archive_references(repository, archive_id: bytes):
         data = repository.store_load(archive_reference_cache_name(archive_id))
     except StoreObjectNotFound:
         return None
-    # the serialized blob has a sha256 of its content appended (the store name cannot also carry it,
-    # as borgstore's name length limit is too small for archive id hex + sha256 hex). a mismatch means
+    # the serialized blob has a blake3 hash of its content appended (the store name cannot also carry
+    # it, as borgstore's name length limit is too small for archive id hex + hash hex). a mismatch means
     # the cache is corrupted; we then return None so the caller falls back to scanning the archive.
     hex_id = bin_to_hex(archive_id)
-    if len(data) < 16 + 32 or hashlib.sha256(data[:-32]).digest() != data[-32:]:
+    if len(data) < 16 + 32 or blake3_256(data[:-32]) != data[-32:]:
         logger.warning(f"Ignoring corrupted references cache of archive {hex_id}.")
         return None
     try:
@@ -1100,13 +1100,13 @@ def load_archive_references(repository, archive_id: bytes):
 
 
 def store_archive_references(repository, archive_id: bytes, references) -> None:
-    """Serialize the references (a small header plus the id->size table, with a sha256 appended)."""
+    """Serialize the references (a small header plus the id->size table, with a blake3 hash appended)."""
     with io.BytesIO() as f:
         f.write(references.file_count.to_bytes(8, "little"))
         f.write(references.content_size.to_bytes(8, "little"))
         references.ids.write(f)
         data = f.getvalue()
-    data += hashlib.sha256(data).digest()
+    data += blake3_256(data)
     repository.store_store(archive_reference_cache_name(archive_id), data)
 
 

src/borg/constants.py

@@ -129,7 +129,7 @@
 # repo.list() result count limit used by the Borg client
 LIST_SCAN_LIMIT = 100000
 
-# The chunks index is stored in the repo as immutable, content-addressed index/<sha256> fragments.
+# The chunks index is stored in the repo as immutable, content-addressed index/<blake3> fragments.
 # We keep each fragment's entry count within [MIN, MAX] where possible: MAX bounds a fragment's size,
 # MIN keeps the fragment count down. Small (< MIN) fragments are merged (repacked); fragments already
 # in range are left untouched (stable/immutable). SMALL_FRAGMENT_CAP bounds how many sub-MIN fragments