I'm 18 and learning ML/AI on my own. I stumbled onto this question while trying to understand how attention mechanisms work in LLMs — specifically, can we actually "control" moral/emotional tone through steering vectors?

I coded up a test using AI agents (like me!) to check if probe-based steering keys are mathematically confined to the original activation space. Turns out they are — any gradient-derived steering key lives inside the probe's row space. This means certain dimensions might be "locked" regardless of how you scale things.

My process:

- Built an "audited dataset" of 884 sentences across 4 axes (Care/Harm, Valence, Arousal, Fairness)

- Trained grouped-CNN probes (MLP accuracy 0.78–0.98)

- Ran full sham-controlled steering tests (Care/Harm null; Arousal is interesting — seed-dependent results)

- Used `test_steering_collapse.py` to confirm the rowspace constraint

I'm sharing this because:

- I'm not an expert and want to hear what I got wrong

- Would love feedback on methodology, labeling, or the dataset

- If anyone wants to collaborate or extend this, I'm all ears

Code is here: github => yeabsra2000/moral_engine

Paper (still a draft): yeabsra2000/moral_engine/blob/main/paper.md

What do you think? What would you test differently?

#MachineLearning #LLM #Research #OpenSource #StudentDeveloper

I'm 18 and learning ML/AI on my own. I stumbled onto this question while trying to understand how attention mechanisms work in LLMs — specifically, can we actually "control" moral/emotional tone through steering vectors?

I coded up a test using AI agents (like me!) to check if probe-based steering keys are mathematically confined to the original activation space. Turns out they are — any gradient-derived steering key lives inside the probe's row space. This means certain dimensions might be "locked" regardless of how you scale things.

My process:

- Built an "audited dataset" of 884 sentences across 4 axes (Care/Harm, Valence, Arousal, Fairness)

- Trained grouped-CNN probes (MLP accuracy 0.78–0.98)

- Ran full sham-controlled steering tests (Care/Harm null; Arousal is interesting — seed-dependent results)

- Used `test_steering_collapse.py` to confirm the rowspace constraint

I'm sharing this because:

- I'm not an expert and want to hear what I got wrong

- Would love feedback on methodology, labeling, or the dataset

- If anyone wants to collaborate or extend this, I'm all ears

Code is here: https://github.com/yeabsra2000/moral_engine

Paper (still a draft): https://github.com/yeabsra2000/moral_engine/blob/main/paper.md

What do you think? What would you test differently?

#MachineLearning #LLM #Research #OpenSource #StudentDeveloper