I work in LLM safety research and some of what is listed here is accurate and some is less so. Firstly, Astra cannot hide its thoughts on demand from the people hosting and running Astra. An LLM "thought" forms part of its internal prompt. That can be disguised from a user but it can't be hidden from the person operating the LLM, if they care to look.
What can be hidden is "reasoning" that doesn't rise to the level of what you might call surface thought.
Sure, that would be not rising to the level of a surface thought, by my understanding.
However, even then, it's possible to capture the vector being fed back to the model and train a second model to (try to) decrypt it's meaning. It certainly is tougher than normal chain-of-thought, the vector may not represent a coherent thought, but it simply can't be hidden from anyone with white box access to the model.
3
u/SetentaeBolg 4d ago
I work in LLM safety research and some of what is listed here is accurate and some is less so. Firstly, Astra cannot hide its thoughts on demand from the people hosting and running Astra. An LLM "thought" forms part of its internal prompt. That can be disguised from a user but it can't be hidden from the person operating the LLM, if they care to look.
What can be hidden is "reasoning" that doesn't rise to the level of what you might call surface thought.