docs(tutorials): index the 4.x demo notebooks

docs/source/content/tutorials.md

@@ -14,26 +14,72 @@
 
 ## Demos
 
+### Core Demos
+
+The two core notebooks, which pair with the links in [Where To Start](#where-to-start) above.
+
+- [**Main Demo**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Main_Demo.ipynb) - The main demo. This is where to start if you're new to TransformerLens. Shows a lot of great features for getting started, including available models, how to access model activations, and generally useful features you should know about.
+
+- [**Exploratory Analysis**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Exploratory_Analysis_Demo.ipynb) - Probably the best place to start, after the Main Demo. Demonstrates how to use TransformerLens to perform exploratory analysis - focuses less on rigor and more on getting a grasp of what's going on quickly. Uses a lot of useful interpretability techniques like logit attribution and activation patching. Steal liberally from this!
+
+### Interpretability techniques
+
 - [**Activation Patching in TransformerLens**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Activation_Patching_in_TL_Demo.ipynb) - Accompanies the [Exploratory Analysis Demo](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Exploratory_Analysis_Demo.ipynb). This demo explains how to use [Activation Patching](https://dynalist.io/d/n2ZWtnoYHrU1s4vnFSAQ519J#z=qeWBvs-R-taFfcCq-S_hgMqx) in TransformerLens, a mechanistic interpretability technique that uses causal intervention to identify which activations in a model matter for producing an output.
 
 - [**Attribution Patching**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Attribution_Patching_Demo.ipynb) - [Attribution Patching](https://www.neelnanda.io/mechanistic-interpretability/attribution-patching) is an incomplete project that uses gradients to take a linear approximation to activation patching. It's a good approximation when patching in small activations like the outputs of individual attention heads, and bad when patching in large activations like a residual stream.
 
-- [**Exploratory Analysis**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Exploratory_Analysis_Demo.ipynb) - Probably the best place to start, after the Main Demo. Demonstrates how to use TransformerLens to perform exploratory analysis - focuses less on rigor and more on getting a grasp of what's going on quickly. Uses a lot of useful interpretability techniques like logit attribution and activation patching. Steal liberally from this!
+- [**Direct Logit Attribution**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Direct_Logit_Attribution_Demo.ipynb) - Direct Logit Attribution (DLA) measures how much each attention head, MLP neuron and embedding contributes to a target logit. This demo shows the `project_output_onto` argument of `ActivationCache.get_full_resid_decomposition`, which folds the projection into the decomposition so the per-neuron intermediate tensor, which can run to many GB, is never materialized.
 
-- [**Grokking**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Grokking_Demo.ipynb) - "Grokking" is a phenomenon where a model can learn to memorise the training data (minimising training loss) but then, if trained for a lot longer, can learn to generalise, leading to a sharp decrease in test loss as well. This demo shows training a model on the task of modular addition, verifying that it groks, and doing analysis. The demo is light on explanation, so you'll probably want to pair it with [Neel's video series](https://www.youtube.com/watch?v=ob4vuiqG2Go) on the paper it's based on.
+- [**Direct Path Patching**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/direct_path_patching_ioi.ipynb) - Isolates a single head-to-head edge: patches only a source head's contribution into the query, key or value input of a destination head, leaving every other component's view of the source unchanged. Validated on the Indirect Object Identification (IOI) task with `get_act_patch_direct_path`.
 
 - [**Head Detector**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Head_Detector_Demo.ipynb) - Shows how to use TransformerLens to automatically detect several common types of attention head, as well as create your own custom detection algorithms to find your own!
 
 - [**Interactive Neuroscope**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Interactive_Neuroscope.ipynb) - Very hacky demo, but this is a feature, not a bug. Shows how to quickly create useful web-based visualisations of data, even if you're not a professional front-end developer. This demo creates an interactive Neuroscope - a visualization of a neuron's activations on text that will dynamically update as you edit the text.
 
-- [**LLaMA**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/LLaMA.ipynb) - Converts Meta's LLaMA model (7B parameter version for now until multi-GPU support is added) to TransformerLens.
+- [**Patchscopes & Generation with Patching**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Patchscopes_Generation_Demo.ipynb) - Implements [Patchscopes](https://arxiv.org/abs/2401.06102) (read a hidden representation from a source prompt and patch it into a target prompt), extends it to generating multiple tokens with patching, and applies it to logit lens, entity description and zero-shot feature extraction on GPT-2 small.
 
-- [**Main Demo**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Main_Demo.ipynb) - The main demo. This is where to start if you're new to TransformerLens. Shows a lot of great features for getting started, including available models, how to access model activations, and generally useful features you should know about.
+- [**SVD Interpreter Demo**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/SVD_Interpreter_Demo.ipynb) - Based on the [Conjecture post](https://www.lesswrong.com/posts/mkbGjzxD8d8XqKHzA/the-singular-value-decompositions-of-transformer-weight#Directly_editing_SVD_representations) about how the singular value decompositions of transformer matrices are surprisingly interpretable, this demo shows how to use TransformerLens to reproduce this and investigate further.
 
-- [**No Position Experiment**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/No_Position_Experiment.ipynb) - The accompanying notebook to Neel's [real-time research video](https://www.youtube.com/watch?v=yo4QvDn-vsU). Trains a model with no positional embeddings to predict the previous token, and makes a start at analysing what's going on there!
+The Backward Lens, Jacobian Lens and SVD Circuits notebooks are linked from [Choosing an analysis tool](analysis_tools.md) and [Fitting a Jacobian lens](jacobian_lens_fitting.md), and the SwiGLU Gate-Sign notebook from [The Hook System](hook_system.md).
+
+### Models and architectures
+
+- [**BERT**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/BERT.ipynb) - Uses BERT through `TransformerBridge` for masked language modelling and next sentence prediction, then inspects its activation cache and ablates an attention head with a hook.
+
+- [**Gemma 3 Multimodal**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Gemma3_Multimodal.ipynb) - Loads the vision-language Gemma 3 (`google/gemma-3-4b-it`: a SigLIP vision encoder plus the Gemma 3 language model, the architecture MedGemma uses) through `TransformerBridge`, generates text from an image and a prompt, and captures vision-language activations with `run_with_cache()`. The checkpoint is gated on Hugging Face.
+
+- [**GPT-OSS-20B**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/GPT_OSS_Demo.ipynb) - Loads OpenAI's [GPT-OSS-20B](https://huggingface.co/openai/gpt-oss-20b) through `TransformerBridge` and walks through caching, expert-routing inspection, the logit lens, attention patterns and activation patching. Peak RAM is about 56 GB.
+
+- [**LLaMA**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/LLaMA.ipynb) - Loads Meta's LLaMA models through `TransformerBridge`: either original LLaMA weights you have converted to the HuggingFace format, or the gated `meta-llama/Llama-3.2-1B` from the Hub. Compares logits with the HuggingFace model, then reads from and writes to hooks.
+
+- [**LLaMA (Quantized)**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/LLaMA_GPU_Quantized.ipynb) - The LLaMA demo with `meta-llama/Llama-3.2-1B` loaded in 4-bit to reduce memory use (`bitsandbytes` on CUDA, `optimum-quanto` on MPS/CPU), plus per-head Q/K/V hooks via `use_split_qkv_input`.
+
+- [**LLaVA**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/LLaVA.ipynb) - Loads LLaVA (`llava-hf/llava-1.5-7b-hf`: a CLIP vision encoder plus a LLaMA language model) through `TransformerBridge`, generates image descriptions, and extracts and visualizes attention patterns.
 
 - [**Othello-GPT**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Othello_GPT.ipynb) - This is a demo notebook porting the weights of the Othello-GPT Model from the excellent [Emergent World Representations](https://arxiv.org/pdf/2210.13382.pdf) paper to TransformerLens. Neel's [sequence on investigating this](https://www.lesswrong.com/s/nhGNHyJHbrofpPbRG) is also well worth reading if you're interested in this topic!
 
-- [**SVD Interpreter Demo**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/SVD_Interpreter_Demo.ipynb) - Based on the [Conjecture post](https://www.lesswrong.com/posts/mkbGjzxD8d8XqKHzA/the-singular-value-decompositions-of-transformer-weight#Directly_editing_SVD_representations) about how the singular value decompositions of transformer matrices are surprisingly interpretable, this demo shows how to use TransformerLens to reproduce this and investigate further.
+- [**Qwen**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Qwen.ipynb) - Loads `Qwen/Qwen2-0.5B` through `TransformerBridge` and checks its output against the HuggingFace model. The first-generation Qwen section is commented out because that model's remote code needs `transformers_stream_generator`, which is incompatible with `transformers>=5.0.0`.
+
+- [**StableLM**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/stable_lm.ipynb) - Compares Stability AI's base and instruction-tuned StableLM models (`stablelm-base-alpha-3b`, `stablelm-tuned-alpha-3b`) to show the effect of instruction fine-tuning, then explores a "Capture The Flag" prompt-injection attack on the tuned model.
+
+- [**T5**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/T5.ipynb) - Loads `google-t5/t5-small` through `TransformerBridge`, runs it on token, string and list-of-string inputs, generates text, and visualizes encoder and decoder attention patterns and top-k tokens with CircuitsVis.
 
 - [**Tracr to TransformerLens**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Tracr_to_Transformer_Lens_Demo.ipynb) - [Tracr](https://github.com/deepmind/tracr) is a cool new DeepMind tool that compiles a written program in [RASP](https://arxiv.org/abs/2106.06981) to transformer weights.This is a (hacky!) script to convert Tracr weights from the JAX form to a TransformerLens `TransformerBridge` in PyTorch.
+
+### Backends and integrations
+
+- [**HuggingFace-style Evals on TransformerBridge**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Bridge_Evals_Demo.ipynb) - Two ways to run benchmark evaluations on a `TransformerBridge` model: pass `bridge.original_model` to an existing framework such as `lm-eval-harness`, or write a short eval loop with hooks active to measure how an intervention (ablation, patching, steering) changes accuracy.
+
+- [**Inspect Driver**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Inspect_Bridge_Demo.ipynb) - Uses `boot_inspect` to turn a model served through [`inspect_ai`](https://inspect.aisi.org.uk) into a bridge, so `run_with_cache`, named hook points and interventions work over the Inspect boundary. Runs on CPU.
+
+- [**Inspect Driver (vLLM Provider)**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Inspect_vLLM_Provider_Demo.ipynb) - The Inspect driver demo served by vLLM (`provider="tl_bridge_vllm"`), so parallel `inspect_eval` samples stream through one resident model for dataset-scale evals. It serves fewer hook points than the HF provider and needs an NVIDIA GPU.
+
+- [**LIT Integration**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/LIT_Integration_Demo.ipynb) - Uses Google's [Learning Interpretability Tool (LIT)](https://pair-code.github.io/lit/) with TransformerLens for interactive analysis: attention patterns, token predictions, the embedding projector and token salience.
+
+### Training and research
+
+- [**Grokking**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Grokking_Demo.ipynb) - "Grokking" is a phenomenon where a model can learn to memorise the training data (minimising training loss) but then, if trained for a lot longer, can learn to generalise, leading to a sharp decrease in test loss as well. This demo shows training a model on the task of modular addition, verifying that it groks, and doing analysis. The demo is light on explanation, so you'll probably want to pair it with [Neel's video series](https://www.youtube.com/watch?v=ob4vuiqG2Go) on the paper it's based on.
+
+- [**No Position Experiment**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/No_Position_Experiment.ipynb) - The accompanying notebook to Neel's [real-time research video](https://www.youtube.com/watch?v=yo4QvDn-vsU). Trains a model with no positional embeddings to predict the previous token, and makes a start at analysing what's going on there!
+
+- [**Real-time Training Telemetry**](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Realtime_Training_Telemetry_Demo.ipynb) - Extracts and plots mechanistic metrics from the `ActivationCache` during a training loop to find the window where phase transitions, such as induction-head formation, begin. Trains a 2-layer toy model for 500 steps on CPU.