docs: point the non-Sphinx consumers at the new pages

Four links in the two example READMEs pointed into `docs/source/.../ffi.md`
with heading anchors. Sphinx does not link-check those, so deleting the page
would have left them silently 404ing. They now use published URLs, which also
fixes a second problem: a relative path into `docs/source` only renders on
github.com and is broken for anyone reading the README from crates.io, an
sdist, or a vendored copy. `grep -rn "docs/source" examples/` is now empty.

`.ai/skills/ffi-capsule-protocol/SKILL.md` named the deleted page as "where the
truth is", and AGENTS.md sends agents to that skill before they touch a capsule
getter. It now points at the specific extension-guide pages and separately at
`ffi-internals.md`.

`llms.txt` filed the whole subject under "Optional" as "extending the Python
bindings" — the wrong shelf for the headline feature of a major release, since
that section means "skippable on a tight context budget". It gains an
Extensions and distribution section, `datafusion.extensions` and
`datafusion.ipc` in the API list, both FFI example crates, and the corrected
`distributing-work` URL.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: timsaucer

.ai/skills/ffi-capsule-protocol/SKILL.md

@@ -223,7 +223,12 @@ pins that; changing it should be deliberate.
 
 ## Where the truth is
 
-- `docs/source/contributor-guide/ffi.md` — the protocol, the fork caveat.
+- `docs/source/extension-guide/` — the protocol, for the library author.
+  `capsule-protocol.md` has the hook convention and what the getter argument
+  actually is; `codecs.md`, `bundles.md`, and `query-planners.md` have the
+  per-component rules; `index.md` lists all 18 hooks.
+- `docs/source/contributor-guide/ffi-internals.md` — why the framing is shaped
+  this way, including the weak-`Arc` scheme and the one-level rebind.
 - `docs/source/user-guide/upgrade-guides.md` — every past migration.
 - `crates/core/src/codec.rs` — the codec chain: the envelope, identity dispatch,
   and the two unframed cases from Rule 8.

docs/source/llms.txt

@@ -16,21 +16,33 @@
 - [Common operations](https://datafusion.apache.org/python/user-guide/common-operations/index.html): select, filter, join, aggregate, window, expressions, and functions.
 - [SQL](https://datafusion.apache.org/python/user-guide/sql.html): running SQL against registered tables.
 - [Configuration](https://datafusion.apache.org/python/user-guide/configuration.html): session and runtime options.
+- [Using extension libraries](https://datafusion.apache.org/python/user-guide/extensions.html): installing a compiled extension and registering its tables, functions, or execution backend with `SessionContext.with_extensions`.
+
+## Extensions and distribution
+
+- [Using extension libraries](https://datafusion.apache.org/python/user-guide/extensions.html): which extensions register directly and which need `with_extensions`, plus the two failure modes (a collected context, a version mismatch).
+- [Distributing work](https://datafusion.apache.org/python/user-guide/distributing-work/index.html): choosing between shipping expressions to a worker pool and installing a distributed query engine.
+- [Shipping expressions to workers](https://datafusion.apache.org/python/user-guide/distributing-work/expressions.html): pickling `Expr` to `multiprocessing` or Ray, what travels inline vs by name, `datafusion.ipc` worker/sender contexts, and the security model.
+- [Extension Guide](https://datafusion.apache.org/python/extension-guide/index.html): the PyCapsule protocol for shipping table providers, catalogs, functions, extension codecs, and query planners from a Rust cdylib. Includes the full `__datafusion_*__` hook reference (18 hooks) and an author checklist.
 
 ## DataFrame API reference
 
 - [`datafusion.dataframe.DataFrame`](https://datafusion.apache.org/python/autoapi/datafusion/dataframe/index.html): the lazy DataFrame builder (`select`, `filter`, `aggregate`, `join`, `sort`, `limit`, set operations).
 - [`datafusion.expr`](https://datafusion.apache.org/python/autoapi/datafusion/expr/index.html): expression tree nodes (`Expr`, `Window`, `WindowFrame`, `GroupingSet`).
 - [`datafusion.functions`](https://datafusion.apache.org/python/autoapi/datafusion/functions/index.html): 290+ scalar, aggregate, and window functions.
 - [`datafusion.context.SessionContext`](https://datafusion.apache.org/python/autoapi/datafusion/context/index.html): session entry point, data loading, SQL execution.
+- [`datafusion.extensions`](https://datafusion.apache.org/python/autoapi/datafusion/extensions/index.html): `SessionExtensionComponents`, `SessionExtensionExportable`, `SessionPlannerExportable`, `QueryPlannerExportable` — the extension-bundle protocol.
+- [`datafusion.ipc`](https://datafusion.apache.org/python/autoapi/datafusion/ipc/index.html): worker and sender context slots for shipping expressions between processes.
 
 ## Examples
 
 - [TPC-H queries (GitHub)](https://github.com/apache/datafusion-python/tree/main/examples/tpch): canonical translations of TPC-H Q01–Q22 to idiomatic DataFrame code, each with reference SQL embedded in the module docstring.
 - [Other examples (GitHub)](https://github.com/apache/datafusion-python/tree/main/examples): UDF/UDAF/UDWF, Substrait, Pandas/Polars interop, S3 reads.
+- [FFI extension example (GitHub)](https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example): a Rust cdylib exporting table providers, catalogs, functions, and extension codecs. Has build commands in its README.
+- [FFI query planner example (GitHub)](https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-query-planner-example): a Rust cdylib exporting a query planner and a complete extension bundle.
 
 ## Optional
 
-- [Contributor guide](https://datafusion.apache.org/python/contributor-guide/introduction.html): building from source, extending the Python bindings.
+- [Contributor guide](https://datafusion.apache.org/python/contributor-guide/index.html): building from source and changing datafusion-python itself.
 - [Upgrade guides](https://datafusion.apache.org/python/user-guide/upgrade-guides.html): migration notes between releases.
 - [Upstream Rust `DataFusion`](https://datafusion.apache.org/): the underlying query engine.

examples/datafusion-ffi-example/README.md

@@ -49,6 +49,6 @@ ctx = ctx.with_physical_extension_codec(provider_physical_codec)
 ctx.set_query_planner(planner)
 ```
 
-Installing a codec after the planner rebuilds the planner against it, so this order is a recommendation rather than a requirement. Planner-last states the ownership flow more clearly. The exception is a planner that wraps a fallback: the rebuild reaches the installed planner only, not the fallback inside it, so codecs-first is a requirement there. See [Rebinding a planner's codecs is one level deep](../../docs/source/contributor-guide/ffi.md#rebinding-a-planners-codecs-is-one-level-deep), which also covers why re-installing a planner rebinds the session to the codecs of whichever handle it was installed on.
+Installing a codec after the planner rebuilds the planner against it, so this order is a recommendation rather than a requirement. Planner-last states the ownership flow more clearly. The exception is a planner that wraps a fallback: the rebuild reaches the installed planner only, not the fallback inside it, so codecs-first is a requirement there. See [Rebinding a planner's codecs is one level deep](https://datafusion.apache.org/python/extension-guide/query-planners.html#install-codecs-before-a-layered-planner), which also covers why re-installing a planner rebinds the session to the codecs of whichever handle it was installed on.
 
-For the limits behind that choice — how the codec chain dispatches, which node kinds survive the boundary, and what a derived context shares with the context it came from — see [Query Planners Across Multiple Libraries](../../docs/source/contributor-guide/ffi.md#query-planners-across-multiple-libraries) in the contributor guide.
+For the limits behind that choice — how the codec chain dispatches, which node kinds survive the boundary, and what a derived context shares with the context it came from — see the [Extension Guide](https://datafusion.apache.org/python/extension-guide/index.html).

examples/datafusion-ffi-query-planner-example/README.md

@@ -78,6 +78,6 @@ ctx.set_query_planner(MyQueryPlanner())
 
 `MyPlannerConfig` is transferred through the foreign session. `MyQueryPlanner` reads `ffi_query_planner.max_rows`, creates the plan with `DefaultPhysicalPlanner`, and adds a built-in `GlobalLimitExec`. The test changes the setting with `SET` and verifies the new row limit.
 
-The provider's codec chain is attached to the planner when it is installed and is also used to decode the returned physical plan in `datafusion-python`. Extension codecs compose: each `with_logical_extension_codec` / `with_physical_extension_codec` call appends to the session's codec chain, and each payload records which codec wrote it, so several libraries can install codecs on the same session and the order between them does not affect decoding. This planner owns no serializable types of its own and deliberately uses only built-in physical nodes. Install the codecs before the planner where possible; installing a codec afterwards rebuilds the planner against the new chain, but planner-last order is easier to audit. That rebuild is one level deep — a planner constructed with `fallback=` keeps the codecs its fallback was imported with — so codecs-first is a requirement rather than a preference once planners are layered. See [Rebinding a planner's codecs is one level deep](../../docs/source/contributor-guide/ffi.md#rebinding-a-planners-codecs-is-one-level-deep).
+The provider's codec chain is attached to the planner when it is installed and is also used to decode the returned physical plan in `datafusion-python`. Extension codecs compose: each `with_logical_extension_codec` / `with_physical_extension_codec` call appends to the session's codec chain, and each payload records which codec wrote it, so several libraries can install codecs on the same session and the order between them does not affect decoding. This planner owns no serializable types of its own and deliberately uses only built-in physical nodes. Install the codecs before the planner where possible; installing a codec afterwards rebuilds the planner against the new chain, but planner-last order is easier to audit. That rebuild is one level deep — a planner constructed with `fallback=` keeps the codecs its fallback was imported with — so codecs-first is a requirement rather than a preference once planners are layered. See [Rebinding a planner's codecs is one level deep](https://datafusion.apache.org/python/extension-guide/query-planners.html#install-codecs-before-a-layered-planner).
 
-For the limits behind that choice — how the codec chain dispatches, which node kinds survive the boundary, and what a derived context shares with the context it came from — see [Query Planners Across Multiple Libraries](../../docs/source/contributor-guide/ffi.md#query-planners-across-multiple-libraries) in the contributor guide.
+For the limits behind that choice — how the codec chain dispatches, which node kinds survive the boundary, and what a derived context shares with the context it came from — see the [Extension Guide](https://datafusion.apache.org/python/extension-guide/index.html).