Move config structs into a leaf storage/config package - #33
Conversation
Consumers that only embed S3Config & co. in their own config (e.g. livekit/cloud's pkg/config, and transitively cloud-config's validator) currently pull in every provider SDK: ~390 packages and ~500 MB of module cache for one struct field. The root package keeps the same API via type aliases; New dispatches with a type switch instead of an unexported interface method.
matkam
left a comment
There was a problem hiding this comment.
Approve. Clean refactor, and the aliases keep the API identical.
Verified locally at 7b86f754
- The struct definitions in
config/config.goare byte-identical to the ones removed fromconfig.go. No YAML tag drift, no field reorder. go build ./...,go vet ./...andgo test ./...pass.go list -deps ./configreaches zero GCS, S3 and Aliyun packages. The only third-party dependency left isazcore, which pulls 24 Azure packages and 9golang.org/xpackages. That is 33, not the 50 in the description. My method excludes the standard library andvendor/paths. Which method did you use? It is worth making the two numbers agree.
1. The type switch drops the compiler's exhaustiveness check (non-blocking)
The old Config interface forced every provider config to carry a newStorage method. A new config type could not compile without one. The type switch drops that guarantee. A new struct in config/ with a storageConfig() method compiles, and then fails at run time in the default arm.
Add a sentinel error and a table test:
var ErrUnsupportedConfig = errors.New("unsupported storage config type")
// default: return nil, fmt.Errorf("%w %T", ErrUnsupportedConfig, conf)The test must call New for each config type and assert that the error is not ErrUnsupportedConfig.
2. Embedded configs no longer dispatch (non-blocking today)
Method promotion makes the type switch narrower than the old dispatch:
type Outer struct{ *storage.S3Config }Outer satisfies Config in both versions. Before, New(&Outer{...}) reached NewS3 through the promoted method. Now it falls to default and fails at run time. This is the pattern your own description names, so it is worth guarding.
No caller regresses today. I checked every importer of github.com/livekit/storage in the org: egress, cloud-egress, cloud-sip, cloud, cloud-protocol, private-cloud and livekit-omni. All of them use named fields. The only storage.New call outside this repo, egress/test/download.go, passes c.S3, c.GCP, c.Azure or c.AliOSS.
Include an embedded config in the test from finding 1. That is the exact case the switch lost.
Nit
config/config.go says "Copyright 2024". The repo uses the year the file was created. See local.go (2025) and error.go (2026).
Mirrors
private-cloud carries source copies of cloud-protocol and cloud-egress under replace directives. It does not pick this up from a version bump. Whoever syncs that repo must carry the change across. livekit-omni is a read-only sync for tooling, so it needs nothing.
Not worth chasing
The 33 packages that remain come from azcore, which AzureConfig.TokenCredential requires. Leave them.
Anyone who embeds
storage.S3Config(or the other provider configs) in their own config struct, without ever creating aStorage, currently pulls in the GCS, S3, Azure and Aliyun SDKs. Measured fromlivekit/cloud-config, which only decodes YAML intocloud/pkg/config.CloudConfigfor validation, the one*storage.S3Configfield is responsible for ~390 compiled packages and ~526 MB of module cache that nothing else needs (44% of its module cache).This moves the config structs into
github.com/livekit/storage/config, whose only external import isazcore(forAzureConfig.TokenCredential). The root package keeps the exact same API:storage.S3Configetc. are type aliases for the new package's types, so existing code compiles unchanged and the types are identical.storage.New(conf Config)dispatches with a type switch instead of an unexported interface method (methods can't be attached to aliases).Configis still an interface with an unexported marker method, so the argument stays type-checked at compile time.Pure refactor, no behavior change.
go build,go vet,go test ./...pass.Follow-ups once this merges:
cloud-protocol/replay/configand thencloudswitch their import tostorage/config; with both,cloud-configdrops to 0 GCS/S3 packages and its module cache shrinks from ~1.2 GB to ~0.7 GB raw.