Trajectory Geometry of Transformer Representations Across Layers

Mechanistic interpretability of multilingual reasoning in transformers. 170+ causal intervention experiments across 4 model families.

Prosthetic cognition architecture for AI agents. Deterministic scaffolding over probabilistic reasoning.

Visualizing Modern LLM Mechanics, Loss Landscapes & HPC Topologies

A 2-D map of GPT-2's token embeddings you can poke at. Pick two axes, project the vocabulary, and see where analogies, cyclic features, and hubness show up (and where the 2-D view is lying to you). Companion to a writeup on which "cyclic" concepts actually form circles.

Code for 'Exploring the Impact of a Transformer's Latent Space Geometry on Downstream Task Performance' (arXiv:2406.12159)

Personal learning notebook on latent space engineering, representation geometry, and exploratory research notes.

Reproducibility package for "Context Is King: How In-Context Specification Shapes the Geometry of Concepts" — code, cached data, and interactive 3D explorers.

An empirical and geometric analysis of Neural Collapse under different optimizers on CIFAR-10

Mechanistic interpretability of transformer hallucinations via attention flow, residual stream geometry, and head-level attribution analysis.

Com la tokenització fractura la morfologia catalana i si una segmentació conscient dels morfemes recupera la geometria. Provat en 3 llengües indoeuropees (català, castellà, anglès): el català es fragmenta ~1,7× més que l'anglès; forçar el tall morfèmic recupera la composicionalitat (robust a portadora i replicat en castellà).

Code for 'Reliable Measures of Spread in High Dimensional Latent Spaces' (ICML 2023)

Effective rank, RankMe, E1, CKA and anisotropy on transformer hidden states are determined by one direction. The exact identity, and the attention sink behind it.