Skip to content

Move config structs into a leaf storage/config package - #33

Merged
u9g merged 1 commit into
mainfrom
jason/split-config-package
Sep 16, 2026
Merged

u9g merged 1 commit into
mainfrom
jason/split-config-package

Conversation

@u9g

@u9g u9g commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Anyone who embeds storage.S3Config (or the other provider configs) in their own config struct, without ever creating a Storage, currently pulls in the GCS, S3, Azure and Aliyun SDKs. Measured from livekit/cloud-config, which only decodes YAML into cloud/pkg/config.CloudConfig for validation, the one *storage.S3Config field is responsible for ~390 compiled packages and ~526 MB of module cache that nothing else needs (44% of its module cache).

This moves the config structs into github.com/livekit/storage/config, whose only external import is azcore (for AzureConfig.TokenCredential). The root package keeps the exact same API:

  • storage.S3Config etc. are type aliases for the new package's types, so existing code compiles unchanged and the types are identical.
  • storage.New(conf Config) dispatches with a type switch instead of an unexported interface method (methods can't be attached to aliases). Config is still an interface with an unexported marker method, so the argument stays type-checked at compile time.

Pure refactor, no behavior change. go build, go vet, go test ./... pass.

Follow-ups once this merges: cloud-protocol/replay/config and then cloud switch their import to storage/config; with both, cloud-config drops to 0 GCS/S3 packages and its module cache shrinks from ~1.2 GB to ~0.7 GB raw.

Consumers that only embed S3Config & co. in their own config (e.g.
livekit/cloud's pkg/config, and transitively cloud-config's validator)
currently pull in every provider SDK: ~390 packages and ~500 MB of
module cache for one struct field. The root package keeps the same API
via type aliases; New dispatches with a type switch instead of an
unexported interface method.

@matkam matkam left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. Clean refactor, and the aliases keep the API identical.

Verified locally at 7b86f754

  • The struct definitions in config/config.go are byte-identical to the ones removed from config.go. No YAML tag drift, no field reorder.
  • go build ./..., go vet ./... and go test ./... pass.
  • go list -deps ./config reaches zero GCS, S3 and Aliyun packages. The only third-party dependency left is azcore, which pulls 24 Azure packages and 9 golang.org/x packages. That is 33, not the 50 in the description. My method excludes the standard library and vendor/ paths. Which method did you use? It is worth making the two numbers agree.

1. The type switch drops the compiler's exhaustiveness check (non-blocking)

The old Config interface forced every provider config to carry a newStorage method. A new config type could not compile without one. The type switch drops that guarantee. A new struct in config/ with a storageConfig() method compiles, and then fails at run time in the default arm.

Add a sentinel error and a table test:

var ErrUnsupportedConfig = errors.New("unsupported storage config type")
// default: return nil, fmt.Errorf("%w %T", ErrUnsupportedConfig, conf)

The test must call New for each config type and assert that the error is not ErrUnsupportedConfig.

2. Embedded configs no longer dispatch (non-blocking today)

Method promotion makes the type switch narrower than the old dispatch:

type Outer struct{ *storage.S3Config }

Outer satisfies Config in both versions. Before, New(&Outer{...}) reached NewS3 through the promoted method. Now it falls to default and fails at run time. This is the pattern your own description names, so it is worth guarding.

No caller regresses today. I checked every importer of github.com/livekit/storage in the org: egress, cloud-egress, cloud-sip, cloud, cloud-protocol, private-cloud and livekit-omni. All of them use named fields. The only storage.New call outside this repo, egress/test/download.go, passes c.S3, c.GCP, c.Azure or c.AliOSS.

Include an embedded config in the test from finding 1. That is the exact case the switch lost.

Nit

config/config.go says "Copyright 2024". The repo uses the year the file was created. See local.go (2025) and error.go (2026).

Mirrors

private-cloud carries source copies of cloud-protocol and cloud-egress under replace directives. It does not pick this up from a version bump. Whoever syncs that repo must carry the change across. livekit-omni is a read-only sync for tooling, so it needs nothing.

Not worth chasing

The 33 packages that remain come from azcore, which AzureConfig.TokenCredential requires. Leave them.

@u9g
u9g merged commit 705386e into main Sep 16, 2026
9 checks passed
@u9g
u9g deleted the jason/split-config-package branch September 16, 2026 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants