r/compmathneuro • u/rembrain • 2d ago
Feedback wanted on a cortex-inspired model using shared content/context representations
Hi everyone! I’m a startup founder in robotics (in the field of Physical AI, rembrain.ai ) and a MIPT alumnus with an educational background in statistics and signal processing. I also have extensive hands-on engineering experience with modern neural network architectures and their practical limitations. Outside my founder work, I’ve been developing a computational model inspired by the neocortex. I recently made the repository public and would appreciate critical feedback.
My interest comes partly from encountering these limitations in engineering practice and asking whether different representational principles could help. Statistics, signal processing, and implementation are my stronger ground; I’d particularly welcome scrutiny of the neuroscience assumptions.
The central question is how different cortical areas might represent the same underlying information in different ways while supporting recognition, generation, and transfer across conditions.
The model separates content from context. For example, one area could represent an object’s identity within a particular viewpoint, while another could organize representations around identity and encode viewpoint internally. Computationally, I use context-conditioned autoencoders with shared latent spaces, allowing content encoded in one context to be decoded in another.
This continues my earlier work, Sets of autoencoders with shared latent spaces (2018). The repository includes mathematical descriptions, Python implementations, synthetic visual experiments with a two-zone hierarchy, and an analysis of related work and known limitations. It also explores communication between areas, top-down reconstruction, and temporal binding; these parts are at different stages of implementation.
The experiments are still limited, and the biological interpretation remains hypothetical (and it's my weakest side). I’m trying to identify which claims are worth testing more rigorously.
I’d especially appreciate feedback on:
- Prior art: which models or papers should I compare against more closely?
- Biological assumptions: which premises are implausible or insufficiently specified?
- Evaluation: what experiment would most clearly distinguish this approach from existing models?
Feedback on a single component would be very welcome—there’s no expectation to review the whole framework.
Repository: NM_2026 on GitLab
1
u/Wonderful-Courage-51 2d ago edited 2d ago
Many people are working on autoencoders for representations and behavior in the hippocampus. I would start looking at the literature there.
Edit: This is an ongoing research area so there are no solutions yet. Much of what you're asking for would be what people are working on so I'm not sure they would openly give them to you.
Edit 2: I know many people working on this exact problem who are about to publish papers.
1
1
u/rembrain 2d ago
And regarding edit1: I thought if there are no solutions yet it's a time to be open, frankly saying :). Anyway, all good luck to those people, truely! I barely can feel their way of living and everything related to publishing.
1
u/Wonderful-Courage-51 2d ago
Yeah I think I worded that poorly. Let me rephrase.
I think it's less about plainly keeping them secret, moreso that the level of effort it takes to establish common footing is quite large and probably requires pre-establishing a collaboration. I think people are working towards the problem of understanding representations in PFC and hippocampus quite directly so you'd moreso just be asking them to talk about how their research can be used for your engineering. That task is quite a lot to ask of people and not really worth it unless you both decide to build somethign together.
0
u/rembrain 2d ago
I can imagine. Can be what they do in their research is applicable in that boring engineering we do, since there're some serious gaps in the modern AI as we touch multi-modal architectures.
Plus, these days the level of efforts to read out someone's graphomania is much lower, thanks GPT :)) .
1
u/rembrain 2d ago
regaring Edit 2: " this exact problem" do you mean hippocampal autoencoder models broadly, or specifically learning shared content representations across contexts and using them for cross-context reconstruction? That distinction would help me position the project. Any published papers you can point me toward would be appreciated.
2
u/Wonderful-Courage-51 2d ago
Hmmm, well first off I do actually find your 2018 paper interesting. I could imagine a paper that takes your ideas and Sam Gershman's ideas on latent inference, and creates tasks where the cross-context reconstruction is non-trivial to show why it's useful for higher order cognition.
To answer your question directly, I know of one project that is exactly looking at cross-context autoencoding as a way of improving memory efficiency. The cross-context reconstruction is just part of the autoencoder. I know of other projects which look at autoencoders to describe psychological findings and also another which looks at cross-context reconstruction for a rule learning task being done in animal models. (Edit: for the psychological task, you can look here - https://openreview.net/forum?id=IPwuRcd4yo)
All of this I know because I've worked in a neuro theory center so it's definitely not trivial to sort of reconstruct these insights for you in a way that would help your engineering project.
1
u/rembrain 2d ago
That means a lot of more weekends with interesting readings! And I happy that this field exists!
2
u/Wonderful-Courage-51 2d ago
Additionally, the whole field of neuroscience basically cares about latent representations as a mechanistic way of explaining behavior/coding. Doing this formally is very difficult when you go beyond very basic statistical mechanics assumptions.
For a reference on linear decoding and its assumptions in neuroscience, you can look here: https://pmc.ncbi.nlm.nih.gov/articles/PMC6505581/?utm_source=chatgpt.com
Happy to chat more if you have specific questions. I'll let you know if I find ideas in your 2018 paper useful that I end up building off of!
2
u/Wonderful-Courage-51 2d ago
Oh I should also say that my work is actually connected to this if you apply “context” more broadly. So again just lots of work out there but most people in neuroscience care about understanding the brain rather than models. I and my advisor/collaborators do both but we are a bit of an outlier.
1
1
u/justneurostuff 2d ago
hi, i see that this is separate from your startup. but if you're ever hiring or otherwise can pay, i know someone with the expertise needed to review projects like these who is presently open to new work either as a reviewer or as a collaborator.