docs: rewrite 'Encode metadata, not a handle to a live object' section

Closes #1726

Author: Awesome06

docs/source/extension-guide/checklist.md

@@ -91,8 +91,9 @@ publish. Each links to the page that explains it.
       derived from it is in use. This is the rule most likely to arrive as a
       bug report against your library. → {ref}`extension_sessions`
 - [ ] **Your production codec serializes durable metadata**, not a
-      process-local token. The examples in this repository use tokens to make
-      ownership observable; that is a demonstration, not a pattern.
+      process-local token. While one arm of one codec in this repository uses a
+      token to make ownership observable, that is a marked workaround for an
+      upstream defect, not a pattern.
       → {ref}`extension_codec_durable_metadata`
 - [ ] **You have integration tests across a real FFI boundary.** The two
       example crates in this repository are the pattern: build the cdylib,

docs/source/extension-guide/codecs.md

@@ -58,15 +58,24 @@ Your payload has to be enough to rebuild the object somewhere your process is
 not. Write the metadata a fresh instance can be constructed from — a path, a
 connection string, a schema, the options the object was created with.
 
-The example codecs in this repository do not do this, and it is worth knowing
-before copying them. They keep a process-local `HashMap` of live providers and
-encode an integer token into it: encoding inserts, decoding removes. That makes
-Rust type identity observable across three separately loaded libraries in one
-test, which is what the examples exist to show. It also means a decode consumes
-its token, so the same bytes cannot be decoded twice, one encoded plan cannot
-fan out to several readers, and a plan that never reaches a decoder keeps its
-provider alive for the life of the process. A real codec has none of those
-properties because it does not park the object anywhere.
+For working examples, see the logical codec in [`datafusion-ffi-example`] and
+the codecs in `examples/distributed/storage-library`.
+
+You may see a codec that uses a process-local `HashMap` of live objects and
+encodes an integer token into it: encoding inserts, decoding removes. Do not copy
+this pattern. It means a decode consumes its token, so the same bytes cannot be
+decoded twice, one encoded plan cannot fan out to several readers, and a plan
+that never reaches a decoder keeps its object alive for the life of the process.
+A real codec has none of those properties because it does not park the object
+anywhere.
+
+A token registry is a consequence, not a choice. A codec that downcasts to its
+own concrete types is never handed something it cannot describe. The only place
+this repository uses one is the `ForeignExecutionPlan` arm of the physical codec
+in [`datafusion-ffi-example`]. It is a marked workaround for an upstream defect
+([apache/datafusion#25152](https://github.com/apache/datafusion/issues/25152))
+and will be deleted once that is fixed. For why a codec would ever claim a foreign
+node it does not own in the first place, see {ref}`extension_codec_order`.
 
 (extension_codec_ids)=
 
@@ -151,7 +160,9 @@ after it. The query still succeeds. What changes is which library wrote the
 bytes — so a plan that has to decode in another process now needs whichever
 library happened to win, not the one whose node it is.
 `MyPhysicalExtensionCodec` in [`datafusion-ffi-example`] claims this way, and
-the query-planner example's test suite pins the consequence.
+the query-planner example's test suite pins the consequence. See
+{ref}`extension_codec_durable_metadata` for why that arm exists and when it will
+be removed.
 
 Two rules of thumb: