DataFusion's Substrait consumer doesn't handle ReadRel.ExtensionTable today. That means custom table reads need to go through ExtensionLeaf, even when the Substrait plan is modeling a read.
I think ExtensionTable should have a resolver hook that returns a DataFusion LogicalPlan, with the normal ReadRel filter/projection handling applied around the resolved plan. Substrait evaluates ReadRel.filter before ReadRel.projection, so that ordering matters here: https://substrait.io/relations/logical_relations/#read-filtering
There may be room to reuse more of the named-table path, but I think this should preserve the existing named-table lowering for compatibility and only generalize the parts needed for non-named reads.
DataFusion's Substrait consumer doesn't handle
ReadRel.ExtensionTabletoday. That means custom table reads need to go throughExtensionLeaf, even when the Substrait plan is modeling a read.I think
ExtensionTableshould have a resolver hook that returns a DataFusionLogicalPlan, with the normalReadRelfilter/projection handling applied around the resolved plan. Substrait evaluatesReadRel.filterbeforeReadRel.projection, so that ordering matters here: https://substrait.io/relations/logical_relations/#read-filteringThere may be room to reuse more of the named-table path, but I think this should preserve the existing named-table lowering for compatibility and only generalize the parts needed for non-named reads.