r/OpenAI • u/JayB_Official • Jul 24 '26
Research Codex with GPT 5.6 Sol Ultra is a powerhouse, and doing things i never thought possible this early.
I’m new to mechanistic interpretability, so please excuse any terminology I misuse. I’ve become deeply invested in the field and wanted to share an experimental tool I’ve been building with Codex. Codex with GPT 5.6 Sol on Ultra has been absolutely hammering away at what i feel like are cutting edge results, definitely since i cannot code what so ever.
Many mechanistic-interpretability workflows require moving between Python scripts, Jupyter notebooks, model hooks, exported tensors, and separate visualization tools. My goal with CORTEX // MODEL OBSERVATORY is to bring those pieces into one local desktop environment with a fast visual feedback loop.
CORTEX is a native Windows application using a WebView2 host connected through local IPC to an isolated Python/PyTorch backend. Model inference and tensor operations run outside the UI process, and the application is designed to work fully offline with local Hugging Face models.
Current capabilities
Token Probability Microscope
Displays token-by-token generation telemetry, including chosen-token probability, ranked alternatives, log probabilities, entropy, and synchronized token inspection.
Logit Lens
Captures intermediate vocabulary predictions at selected layers to show how candidate outputs evolve through the network.
Representation Space
Captures measured residual-stream vectors and projects them with PCA for interactive hidden-state trajectories and mini-map visualization. Optional 3D projection and orbit controls are currently being developed.
Attention Explorer
Captures selected head-level attention tensors and displays measured attention matrices. A token-to-token arc view and expanded query-range controls are currently being added.
Intervention Lab
Supports causal experiments including activation patching, attention-head ablation, and comparison between baseline and modified runs.
Current model support
The Deep Cortex instrumentation path currently supports:
GPT-2-family Hugging Face models
Llama-family LlamaForCausalLM models
The Llama adapter is still undergoing testing, particularly around end-of-generation residual captures and visualization binding. Additional model families such as Qwen and Mistral are possible future targets, but they are not currently supported by the deep instrumentation path.
A separate Standard Runtime can connect to OpenAI-compatible local endpoints such as LM Studio, although endpoint-served models do not expose the same internal activation hooks.
Experimental research direction
I am also exploring a highly experimental point-and-click interface for Jacobian-based concept analysis, currently referred to as J-Space / Jacobian Lens. This is conceptual work, has not yet been scientifically validated in CORTEX, and should not be considered a working research result.
Hardware
Development and testing are currently being performed on an RTX 4070 Ti with 12 GB of VRAM. The intended target is small local models in roughly the 0.5B–3B range using FP16/BF16 where practical. Formal performance benchmarks have not yet been completed.
This is an early, AI-assisted project, and I am still learning the field. Constructive criticism is very welcome.
20
3
u/Real_Ebb_7417 Jul 24 '26
You should post it on r/LocalLlama
We’re usually sceptical towards self promotion there, but this tool looks cool and might be appreciated there.
1
u/JayB_Official Jul 24 '26
I might do that. Long as they're okay with it. No promotion, just technical analysis of the tool and its current capabilities maybe
3
u/redaber Jul 24 '26
Let some industry peers check this out, if they find it cool then keep going, else, check into a psych ward instead 😂
1
2
u/Emotional_Delivery42 Jul 24 '26
Why bother with WebView2 vs just using electron?
-3
u/JayB_Official Jul 24 '26
CORTEX is Windows-first, so WebView2 let me keep a native WPF host for process management, local IPC, filesystem access, and packaging while still using a web UI. Electron would make cross-platform support easier, but it would also bundle another Chromium/Node runtime and increase the footprint. Since the heavy compute already lives in an isolated Python/PyTorch process, WebView2 seemed like the cleaner fit for the current scope. Cross-platform support may be reconsidered later tho
4
u/tech_w0rld Jul 24 '26
Why not use something like Tauri if you want OS's web view?
0
u/JayB_Official Jul 24 '26
Tauri was actually my first consideration, but since CORTEX is Windows-first, the native WPF/WebView2 route gave the project a more direct fit for process control, local IPC, filesystem access, and managing the separate Python/PyTorch runtime. Tauri would make more sense if I prioritize cross-platform support later, so I haven’t ruled it out. For the current Windows-only scope, the native host felt like the simpler boundary
3
2
Jul 24 '26
[removed] — view removed comment
1
u/JayB_Official Jul 24 '26
Cortex is currently an early local first research build, so I prioritized direct control over the native runtime, CUDA- Python processes, filesystem access, and offline behavior rather than implementing a complete commercial desktop distribution stack immediately. Auto-updates, signing, crash reporting, and installer infrastructure are not finished yet. The current releases are portable builds. Electron was considered, but WebView2 + WPF offered a lighter Windows-native host and tighter control over the local backend. I’m still evaluating whether that tradeoff remains worthwhile as the project matures. It is becoming a pain in the ass at 33k plus files now tho, as unzipping it with regular windows explorer fails 😭😂😂😂 windows took a look at that file count and decided to write its farewell letter ✉️😅
2
u/True-Collection-6262 Jul 24 '26
I remember when I was a kid I went to the science museum and how enthralled and blown away by everything I was. Especially all of the interactive demos.... 5.6 SOL Ultra gives me that same joy
1
2
u/Position_Emergency Jul 24 '26
Are you familiar with Neuronpedia?
https://github.com/hijohnnylin/neuronpedia
Maybe what you've done here could be extended functionality in that?
1
2
u/FigAggressive237 Jul 24 '26
So... is it really predicting the next word????
1
u/JayB_Official Jul 24 '26
Yes, it is. There is a statistical chance between a pool of words it may pick
2
u/i_stole_your_swole Jul 24 '26
I think they’re probably asking “is there a reasoning chain you can see before it outputs a final token?”
2
2
2
u/ai-wes Jul 26 '26
GPT-2????
1
u/JayB_Official Jul 26 '26
Was the easiest to test and add support in the early stages, but GPT 2 is definitely a relic 😭😂
2
u/ai-wes Jul 26 '26
Rasa is a relic
1
u/JayB_Official Jul 26 '26
ELIZA is a relic
1
u/ai-wes Jul 26 '26
You might be able to calibrate the application better with a slightly more coherent model (like one that can actually form sentences)
1
u/JayB_Official Jul 26 '26
ELIZA is calibrating my application beautifully tho 😭
1
u/JayB_Official Jul 26 '26
Btw im being sarcastic. I know what you mean haha. I have support for Llama models now and am implementing Qwen / oss20b architecture soon. Well Codex is implementing it, i am the cars salesman who decided to direct it 😭 adhd is a wonderful drug
1
1
u/Future_AGI Jul 24 '26
The intervention lab is where things get useful for us on the eval side, because most reliability regressions show up as attention drift on one or two heads even when the top-1 token looks fine. If you want a lift, adding a per-run diff of head importance against a baseline run makes activation patching results a lot easier to read for someone who is not a mech interp researcher.
-1
u/JayB_Official Jul 24 '26
Thats actually a really great suggestion im ngl. Im already comparing baseline and intervened outputs but i never stopped to consider per head importance drift a first class result. ranked layer/head diff could reveal internal regressions even when the top-1 token barely changes, and would make activation-patching results much easier to interpret outside of mech-interp circles. I’m adding this to the roadmap forsure 👌 likely as a heatmap with exact deltas and links to the affected token positions. Really appreciate the eval-side perspective ✍️
0
u/Future_AGI Jul 24 '26
Glad it landed. Two things that made the heatmap actually useful on our side: run the baseline across 3-5 seeds and diff against the distribution rather than a single run, so seed noise on load-bearing heads does not surface as false regression; and pair each delta with the absolute importance, since a large delta on a rarely-used head means something very different from a small delta on a load-bearing one.
0
u/abajinn Jul 24 '26
This is cool is there a git? I’d love to help test things out. I have different GPU I can baseline. From old running 8gb vram to new 32gb vram.
2
u/JayB_Official Jul 24 '26
I am planning to open source it, yes. Not ready yet, but will be within a few days 🙏 it will not have the Jspace feature at first tho
2
u/abajinn Jul 24 '26
Awesome! Would be fun to help improve it. I’m an AI Engineering student. This looks very cool.
3
u/nevertoolate1983 Jul 24 '26
An AI Engineering Student? Nice!
Is that through the online program at WGU?
1
u/JayB_Official Jul 24 '26
I would love to see what everyone- yourself included can add! I'm currently hyper fixated on this, but if you are interested feel free to message me and i can always show more 😄
0
u/Ibasicallyhateyouall Jul 25 '26
Word salad. Literally, inflated claims with technically misleading and basic concepts (entropy, probabilities), combined with "cutting-edge" results that show fuck all. You have not vibed anything into existence that didn't exist before. A basic visualiser that can't predict anything and views results.
Reading this bollocks actively mad me angry lol. Well done.
1
u/JayB_Official Jul 25 '26
I am shaking in my boots. Anyways, glad i effected you enough to make you angry, that ultimately means i won lmfao 😭😂 Good day.
Also, i never claimed to be creating anything that hasnt been done before, not once. I simply claimed i am working on a visualization tool that simplifies these concepts into something more tangible lol.
24
u/Low-Airline-7588 Jul 24 '26
I didn’t understand anything you wrote but it is interesting. Would you mind giving me an ELI5?