Skip to content

Refactor AvroSchemaResolver: WriterSchemas enum + WriterSchemaKey#37051

Merged
martykulma merged 1 commit into
MaterializeInc:mainfrom
martykulma:maz-aws-glue-avroschemaresolver
Jun 16, 2026
Merged

Refactor AvroSchemaResolver: WriterSchemas enum + WriterSchemaKey#37051
martykulma merged 1 commit into
MaterializeInc:mainfrom
martykulma:maz-aws-glue-avroschemaresolver

Conversation

@martykulma

@martykulma martykulma commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Generalize the handling of writer schemas and their keys.

When MZ reads a message from Kafka, it will parse the wire format header to determine what schema the message was written with. It may have been the reader schema, but it may be a different schema (the writer schema) that MZ will need to dynamically look up.

@martykulma martykulma force-pushed the maz-aws-glue-avroschemaresolver branch 4 times, most recently from 3731e93 to c00677a Compare June 15, 2026 21:17
@martykulma martykulma marked this pull request as ready for review June 16, 2026 13:05
@martykulma martykulma requested a review from a team as a code owner June 16, 2026 13:05
@martykulma martykulma requested a review from a team June 16, 2026 13:05

@patrickwwbutler patrickwwbutler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with an opinion comment on the class name of WriterSchemas!

Comment thread src/interchange/src/avro/schema.rs Outdated
/// without a registry to fetch from). Each variant owns its cache type by
/// construction, so the resolver cannot mis-route a key to the wrong
/// cache.
pub enum WriterSchemas {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just at first glance the name WriterSchemas feels a bit ambiguous to me, because it kind of implies that the schemas themselves literally exist in this object (which I guess they kind of do when there's a cache?), when it seems like it's more generally a way to get the schemas?

@martykulma martykulma force-pushed the maz-aws-glue-avroschemaresolver branch 2 times, most recently from 7e761ea to 002e27d Compare June 16, 2026 19:53
@martykulma martykulma force-pushed the maz-aws-glue-avroschemaresolver branch from 002e27d to 16c7ea6 Compare June 16, 2026 19:55
@martykulma martykulma merged commit 6e5d1dd into MaterializeInc:main Jun 16, 2026
159 checks passed
@martykulma martykulma deleted the maz-aws-glue-avroschemaresolver branch June 16, 2026 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants