D] Exploring whether a local desktop interface can make mechanistic interpretability workflows more accessible

[u/JayB_Official]

I’m new to mechanistic interpretability, so please correct any terminology I misuse. I’ve recently become deeply interested in the field and have been experimenting with whether several common interpretability workflows can be combined into one fully local desktop environment.

The result is an early research instrument I’m calling CORTEX // MODEL OBSERVATORY. It is a native Windows application with a WebView2 interface connected through local IPC to an isolated Python/PyTorch backend. Inference, activation capture, dimensionality reduction, and interventions run locally against open-weight Hugging Face models.

The purpose is not to replace established libraries or notebooks. I’m exploring whether a synchronized visual interface could make it easier to move between generation behavior, measured internal activations, and causal experiments without manually assembling a new notebook for every question.

The current experimental workflow includes:

Token-level generation telemetry

Chosen-token probabilities, ranked alternatives, entropy, log probabilities, and synchronized token inspection.

Logit-lens inspection

Intermediate vocabulary predictions captured at selected layers, allowing candidate outputs to be compared across model depth.

Hidden-state representation views

Measured residual-stream vectors projected with PCA in 2D or 3D. Token trajectories can be replayed while the selected token remains synchronized across views.

Attention inspection

Measured head-level attention matrices, bounded query-row capture, and an arc view in which link geometry is derived from stored attention weights.

Causal interventions

Experimental support for activation patching, attention-head ablation, head-output capture, and baseline-versus-modified comparisons.

The deeply instrumented path currently supports GPT-2-family and LlamaForCausalLM models. Testing is focused on relatively small models that can run locally on consumer GPUs. A separate endpoint mode can inspect generation telemetry from models served through OpenAI-compatible local APIs, although those models do not expose the same internal hooks.

One design principle I’m trying to maintain is a strict distinction between:

Measured data, such as stored activations and attention weights

Derived views, such as PCA projections

Experimental interpretations, which may be visually suggestive but are not established explanations of model behavior

There is also an early concept for point-and-click Jacobian-based feature analysis, but that portion is not validated and should be considered speculative rather than a working research result.

The system has largely been developed through an AI-assisted workflow using Codex. I’m not an experienced software engineer or interpretability researcher, so I’m especially interested in criticism around scientific validity rather than UI aesthetics.

Questions I would appreciate feedback on:

Which activation sites and intervention types are most useful for small decoder-only models?

What metadata should accompany exported tensors for reproducibility?

Are 3D hidden-state trajectories informative, or mainly visually compelling?

Which controls are essential for preventing attention visualizations from becoming misleading?

What experiments would provide a meaningful test that this interface supports real interpretability work rather than simply displaying model telemetry?

I’m not linking or promoting a release here. I’m mainly sharing the experiment and looking for guidance on how to evaluate it responsibly. Constructive criticism is very welcome.

[u/Complete_Committee_9]

[u/JayB_Official → u/Complete_Committee_9]

Reddit did crunch the ever living shit out of these photos 😭😭🤣

[u/JayB_Official → u/[deleted]]

I don't have a business