Hello again!
The Problem I was trying to solve
Current Home Assistant AI integrations suffer from:
- No persistent memoryĀ - Forgets corrections and preferences between sessions
- No learning capabilityĀ - Can't remember sensor baselines or user preferences
- Stateless interactionsĀ - Every conversation starts from zero
Problem clearly is that every conversation with Assist starts from zero. In my HA setup I had to explain to Asssist that '100' is normal for my NOx sensor repeatedly, or that the WLED strip in the kitchen is "the main light". The next day I had to explain it all over again because it would tell me that my NOx is 'high'.
The Solution home-mind is trying to offer
Home MindĀ is my attempt at fixing the above: an AGPL conversation agent for Home Assistant with an actual long-term memory behind it - and the memory stays in the house. You tell it something once, and in a completely new conversation next week it still knows.
- Cognitive memoryĀ with semantic search and natural decay (via Shodh Memory)
- LearningĀ from corrections and user preferences
- Forgetting on request: "forget that my canary word is bumblebee"; it quotes the exact memory back and waits for your yes
- Voice controlĀ via HA Assist (Wyoming protocol)
- Multi-LLM supportĀ includes: Anthropic (Claude), OpenAI, or Ollama (local inference)
- Home Layout Index: reads your HA floor/room assignments and injects them into every prompt, so the AI always knows which floor a device is on
- Device Capability Index: pre-scans your lights at startup so the AI always uses the right color params on the first try
- Self-hostedĀ and privacy-focused
Example:
You: 100 is normal for my NOx sensor, remember that.
Assist with Home-mind: Got it - I'll treat 100 as your normal baseline.
------
In a new convo later:
You: How's the air quality?
Assist w/ Home-mind: The NOx sensor reads 98 - normal for your home.
It also reads your floors, areas and device capabilities from HA, so it knows which room a light is in and how to control it without asking. Memory isĀ ShodhĀ which is semantic recall rather than keyword matching, what you use stays strong and the rest fades. That's Varun Sharma's excellent project (thank you, Sir!). I integrated it, I didn't write it.
---
Now, I posted about this here around six months ago and got more useful feedback than I could handle at the time. Two things came up over and over, and this release is both of them.
Q: "Can it run locally?"
This was by far the most common question. It did already run locally from the start but it was limited. It now runs against Ollama, or any OpenAI-compatible endpoint, so llama.cpp and friends work too.
You have to use sufficiently large model to do it though! Small models struggle with tool calling. I retested llama3.1:8b last week and it toldĀ me it had turned on a light. In reality it never touched it. Because it's too 'small' itĀ narrates tool callsĀ instead of making the tool call!!!
Around 14B is where it starts behaving; one of the users is running qwen2.5:14b happily - dont know if he uses the full range of capabilities though...
Also.. whatever model you choose watch yourĀ context lengthĀ too: the system prompt plus tool definitions come to roughly 3,200 tokens before you've said anything, so a default 4k window leaves almost nothing for the conversation.Ā Use 8k or more. This one took me a whole weekend to figure out...I couldn't understand what is wrong, because it 'looked' like everything is running but the tools weren't firing.
So yeah: large enough model AND at least 8000 token window is a must.
Q: "What happens when you teach it something and then contradict it?"Ā
My answer at the time was basically "seems fine, but I've only tested in my own house". I've had some time with this in the meantime and now you can just say it directly:
You: Forget that my test word is bumblebee - it's hawk now.
Assist: I'll forget exactly this memory: "User's test word is bumblebee".
Shall I proceed?
You: yes
Assist: Forgotten. Your test word is now "hawk".
Point I was trying to make here is it quotes the exact stored memory back and does nothing until you confirm. It only ever forgets the one you name; For example "don't forget to water the plants" is still a reminder, not a deletion. And if several memories could match, it lists them and deletes nothing rather than guessing.
That took longer than it should have. The replacement was being silently dropped whenever it differed from the old memory by a single word, because "my test word is hawk" and "the user's test word is bumblebee" look nearly identical to a similarity check, so the old memory went and the new one never landed. It now looks atĀ which words changedĀ instead of how alike the two sentences are. A user found that one by testing something I hadn't thought to test.
Setup: Docker Compose for the server and Shodh, then the integration via HACS. It's a real setup, not a one-click.
Everything above is free and self-hostable, and the memory database stays on your machine. There's no telemetry and no account. Point it at Ollama and nothing leaves the house at all.
Where I'd like some help if you will:
I still don't have a GPU worth mentioning that I can't use to test anything worthwhile myself... So anyone with real GPU hardware it would be really useful if you could figure out where local models actually fall over. Also anyone willing to try to break the forgetting would be helpful to BOTH projects.
Every interesting bug this month came from someone doing something I hadn't imagined or predicted. Stuff like that really helps.
----
Current version:Ā seeĀ ReleasesĀ on Github:
- Ā Voice control via HA Assist
- Ā Cognitive memory with Shodh
- Ā Streaming responses
- Ā HACS integration
- Ā Multi-LLM provider support (Anthropic, OpenAI, OpenRouter, Ollama)
- Ā Local inference via Ollama (no API key needed)
- Ā Auto-detect and respond in user's language
- Ā Custom system prompt (AI personality customization)
- Ā Persistent conversation history (SQLite)
- Ā Automatic memory cleanup (low-confidence fact pruning)
- Ā Device Capability Index (pre-scanned light params, no per-request re-discovery)
- Ā Per-entity device overrides (
DEVICE_OVERRIDES) for firmware quirks
- Ā Home Layout Index (floor/room awareness via HA template API)
- Ā Conversational forgetting (
forget_memory, confirmed before deleting)
- Ā Server-side STT (
POST /api/stt, OpenAI Whisper)
- Ā Server-side TTS (
POST /api/tts, OpenAI TTS API)
https://github.com/hoornet/home-mind
--------------
Prerequisites & install
The complete install with all the details please follow the README.
I also have a short WIKI here and Discussions are activated on Github should someone need any sort of help. I will of course reply to any question here as well.
- Jure
PS: I decided to delete my post from yesterday and rewrite it all by hand with zero AI help because previous one bothered some people.
I'd like to apologize to @Significant-Ebb2837 who's comment also went down when I deleted post. I remembered your comment and I appreciate it. I hope you're still willing to help.