Measuring Geometry in High Dimensions

Working with high-dimensional distributions requires reliable measurement tools, and our intuitions about shape and spread break down above a handful of dimensions. Much of my early work focused on this foundational challenge: how do we quantify properties like spread and clustering in spaces where we can't visualize the data?

I found that widely-used measures of distributional spread in NLP (Average Cosine Similarity and I(V)) are unreliable as relative measures, failing to produce expected rankings on synthetic distributions with known properties. I introduced two alternatives, Eigenvalue Early Enrichment (EEE) and Vasicek Ratio MSE (VRM), drawing on methods from astronomy, ecology, and spatial statistics. This work underscored a broader challenge: the tools we use to describe high-dimensional geometry shape the conclusions we draw from it, and many commonly used tools are not up to the task.

An open question that continues to motivate my work is whether there exists a measure (or family of measures) that can capture the kind of complex, recursive structure described by my nooks-and-crannies hypothesis. Existing measures of spread and clustering capture aspects of this structure, but none fully characterize it. I believe that recent work on feature neighborhoods and geometric structure in the mechanistic interpretability community may offer promising directions.

Latent Space Geometry and Model Performance

If the geometric organization of a model's latent space matters for downstream function, we should be able to find measurable properties that predict performance. To test this, I applied a range of geometric measures, including the spread measures above and several quantization-based clustering measures, to the latent spaces of BERT-family models with varying degrees of weight perturbation.

I found that quantized cell density (Point Patchiness), a measure of variability in cluster distributions borrowed from ecology, predicts GLUE benchmark performance with r = 0.9 across perturbed BERT-small models. This suggests that the fine-grained patchwork structure of contextual representations is meaningfully related to a model's ability to learn downstream tasks, supporting the nooks-and-crannies intuition that organizational structure, independent of specific content, benefits transfer learning.

This work also revealed that adding noise to model weights does not smoothly degrade latent space geometry. Many geometric measures showed non-monotonic, "zig-zag" patterns as weight noise increased. This unexpected finding points to a complex, nonlinear relationship between weight space and representation space that I believe warrants further investigation, potentially through geometric analysis of the weight distributions themselves.

Weight Distributions and Overparameterization

The nonlinear relationship between weights and latent geometry led me to examine the weight side of the equation directly from three angles:

Pre-training Dynamics. I explored how pre-training data scale, training task, and hyperparameter configuration shape the distribution of weights across transformer layers. The central finding is that training scale and hyperparameter choices affect weight distributions far more than training task.

Overparameterization. I also observed that pre-trained weights remain surprisingly close to their initialized values even after extensive training, connecting to the lottery ticket hypothesis and raising questions about which weights are actually doing meaningful work. I explored methods for masking based on weight movement as initial work on understanding overparameterization through weight movement patterns.

Initialization. Building on work showing that non-standard pre-training processes can still produce surprisingly strong benchmarking performance, I began exploring how different model initializations and pre-training configurations affect benchmark performance throughout training as initial work toward understanding the distributional characteristics of weights that are beneficial for model learning.

This work points toward several open questions, including whether the geometry of weight space itself predicts downstream performance, and whether targeted weight initialization and/or pruning could produce favorable latent geometry before linguistic training begins.

Refusal Steering in Mixture-of-Experts Models

In recent work, I shift from encoder-only models to decoder-only generative models, investigating how safety alignment, specifically refusal behavior, is implemented in Mixture-of-Experts (MoE) architectures.

I extended ActAdd refusal steering to three open-source MoE models and introduced expert-aware steering methods that isolate the contributions of individual experts. While expert-aware steering was less effective than aggregate steering, this result was analytically productive: it allowed me to probe where refusal behavior resides within the architecture and how different components contribute.

Several findings emerged from this analysis. Refusal-specific expert routing patterns do not predict steering effectiveness, suggesting that the model's detection of harmful content and its behavioral response operate through different mechanisms. Refusal behavior appears to be distributed across both the attention and feed-forward sublayers, with evidence of two distinct pathways: an internal pathway (FFN-mediated, learned during post-training) and a contextual pathway (attention-mediated, driven by system prompts). And the nonmonotonic relationship between steering coefficient and attack success rate suggests that refusal behavior may occupy a nonlinear subspace, with evidence of entanglement with other post-training behaviors like chain-of-thought production.

These findings connect to broader questions about superposition and feature geometry in post-trained models, and have practical implications for alignment robustness: if refusal is distributed and entangled rather than localized and linear, approaches to safety alignment may need to account for this complexity. From the nooks-and-crannies perspective, these observations suggest that post-training behavioral features may be threaded through the existing organizational structure of the latent space rather than occupying distinct, separable regions.

Cyclical Manifolds in Interpretability Features

A newer thread of work extends my interest in latent space geometry into circuit-tracing and feature-extraction methods specifically. In a preliminary exploration on Gemma-2-2b, I found that Anthropic's pre-trained cross-layer transcoder (CLT) features preserve and sharpen a known cyclical manifold structure (the geometric encoding of days-of-the-week) that has been observed in raw model activations and output behavior. The CLT features produced a cleaner, more distinctly separated cyclical structure than either the raw activations or output probabilities, suggesting the feature extraction process is doing exactly what it was designed to do: surfacing cleaner, more interpretable semantic structure from the residual stream. Future work in this vein will apply the manifold steering methods of Wurgaft et al. (2026) in CLT space.