Skip to content

Reuse cached OCI layers and configs - #393

Open
Suzu1Dev wants to merge 3 commits into
sysprog21:mainfrom
Suzu1Dev:fix/oci-reuse-cached-layers
Open

Suzu1Dev wants to merge 3 commits into
sysprog21:mainfrom
Suzu1Dev:fix/oci-reuse-cached-layers

Conversation

@Suzu1Dev

@Suzu1Dev Suzu1Dev commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Repeated pulls open layer sources before checking the local blob cache and fetch the config during platform selection even when it is cached. Reuse verified layers and config blobs before opening their sources, including the config read used for platform checks.

Cached blobs still undergo digest and size verification. Platform mismatches and corrupt cached configs remain errors; missing blobs still require their sources. Pulls still fetch manifests from the registry.

Local tests cover cached layers with unavailable sources and full pulls of single images and indexes with and without declared platforms. A cold pull makes one config GET; a repeated pull makes none even when that endpoint is configured to return 404. Platform mismatch tests require the expected error for each selection path.

Validation on macOS arm64 with Go 1.25.3:

  • make oci-lint (formatting and Darwin/Linux vet)
  • GOFLAGS=-count=1 make oci-test (race tests with local fixtures)
  • make elfuse-oci and ./build/elfuse-oci --help
  • git diff --check origin/main

Supersedes #394.

Opening a layer's compressed reader starts a registry request even when
its digest is already stored locally. Delay that request until the store
has checked its existing blob so repeated pulls can reuse a verified
layer without reopening its source.
@jserv
jserv requested a review from henrybear327 September 23, 2026 02:03
cubic-dev-ai[bot]

This comment was marked as resolved.

Comment thread cmd/oci/store.go Outdated
return v1.Descriptor{}, err
}
if err := s.writeBlob(ctx, v1.Descriptor{MediaType: types.OCIConfigJSON, Digest: configHash, Size: int64(len(config))}, io.NopCloser(bytes.NewReader(config))); err != nil {
if err := s.writeBlob(ctx, v1.Descriptor{MediaType: types.OCIConfigJSON, Digest: configHash, Size: int64(len(config))}, func() (io.ReadCloser, error) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

img.RawConfigFile() still runs before this cache check, and on a remote.Image that is a registry blob GET: remoteImage.RawConfigFile falls through to fetchBlob unless the manifest carries Config.Data. A repeated pull of an already-cached image therefore still costs one request for the config, so the layers are lazy but the config is not. Taking Digest and Size from img.Manifest().Config and moving RawConfigFile() into the closure applies the same deferral here, reasonable as a follow-up if you want to keep this change to layers.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The config cache follow-up is in #394 . It reuses the cached config for platform checks and defers source reads until a cache miss.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: The follow-up is merged right here now.

@Suzu1Dev
Suzu1Dev marked this pull request as draft September 23, 2026 02:24
Platform selection reads the config before publication, so deferring
only the config write still issues a blob GET on every pull. Use
verified cached config bytes for platform checks and defer source reads
during publication until the manifest descriptor misses the cache.

Exercise repeated pulls against a local registry with config downloads
disabled. Keep platform mismatches and corrupt cached blobs rejected.
@Suzu1Dev
Suzu1Dev marked this pull request as ready for review September 23, 2026 03:34
cubic-dev-ai[bot]

This comment was marked as resolved.

A declared index with a matching platform must reject a mismatched
config. Accepting the platformless index's no-match error can hide a
fallthrough that skips that rejection. Require each mode's expected
error for both cached and uncached configs.
@Suzu1Dev Suzu1Dev changed the title Reuse cached OCI layers before opening sources Reuse cached OCI layers and configs Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants