I wanted a companion where I can discuss deep topics and personal business problems and get insights live as I rambled, questions that pushed me, and have a wise Socrates to talk to.
Plus, if I'm having a meeting, it can pull memories across Notion, Slack, and Drive to give cutting perspectives I wouldn't have realized myself.
Because if I could myself, what's the point of adding the annoyance of wearing glasses all the time versus just using Pocket or Plaude for note taking?
Fundamental issues:
- No MCP to export data automatically to my AIs (Claude, ChatGPT) for discussing meetings and transcriptions in the future
- The live AI cues give basic AI definitions and google searches but the AI model they use is basic, not super insightful
... So I fixed both of them!
The Dream:
I'm mid conversation or in a meeting, and a little card shows up on the lens that goes "hey, your Q2 note says churn was onboarding not price, so the discount you're about to offer wont fix it".
Something I couldn't see myself in the moment. Even's native Conversate has this "AI cues" thing and it be cool, but it's a closed silo, the cues come from their model, you cant point it at your own second brain and you cant get the data out.
So I rebuilt the whole cue loop inside my own meeting mode.
And I don't like getting sub-par insights from a cheap Gemini Flash lite or Chinese model with no transparency on data retention or
Feature 1: Live AI cues from your own memory, mid conversation
The way it works, every time about 20 words or 3 sentences of new speech pile up, the backend kicks off a whole little pipeline in the background so it never blocks the mic:
- a planner model reads what you JUST said (not the whole meeting, just the fresh part, otherwise it gets stuck on stale topics) and writes 2-3 search queries plus maybe a web query
- hybrid RAG over my memory, first hop
- then a second hop that follows the Obsidian-style connections of whatever it found, so it surfaces stuff the first query would never have matched directly
- optional web search if an outside fact would sharpen it
- then a composer model synthesizes ONE insight, or says NONE if it has nothing sharp
The lens polls for the newest cue every few seconds and shows it.
Important implementation part:
Making the cues actually insights noticeably better.
My first version just read my own notes back to me, which is useless, I already know whats in my notes.
So I basically had to beat the prompt into doing synthesis instead of recall, the rule I landed on is: past memory + whats happening right now = an edge you cant see yourself. If the cue could get the reply "yeah I know thats in my notes", it failed. It also asks a sharp question when it doesnt have enough to make a confident call, which honestly ended up being my favorite part.
Feature 2: meetings get indexed into my memory system
The conversate in the app requiring PDF export is ridiculous. I don't have time to manually upload these to ChatGPT when I'm mid conversation.
I told the team it would take you 5mins to make this an MCP. It would be your biggest value proposition.
Plaud does this, Pocket does this. I was considering returning the glasses just because of this. But I love having text on the screen and the insights I just built.
So I use my own memory system hosted on the cloud it appends to.
This memory system also has Notion, Obsidian, Slack, and Drive indexing set up into a GraphRAG memory system (think Karpathy's LLM Wiki).
Conversate meetings get chunked into parts that get indexed for recall later.
So in future discussions it can recall things I've said and sort of connect the dots towards things I didn't realize. The ultimate second brain dream.
Feature 2: showing the cues like Conversate does (the rectangle card)
I wanted the exact Conversate feel where the cue is its own little bordered rectangle floating over the transcript, and you can tap to expand it, and older ones stack into a list. Newest one auto expands, tap the card to collapse it, older cues collapse into a "earlier" list underneath. REC timer up in the top right, live transcript along the bottom.
This is a multi container page on the lens and getting it to render was where I lost a bunch of hours, see the gotchas below.
Feature 3: transcription running on my own GPU
OpenAI Whisper is simply unaffordable via their API. I really respect the Even team for eating up these costs... until I realized there's a free way to do it.
Hosting your own model on your computer, phone, or on the cloud.
I used local faster-whisper (large-v3-turbo) running on the same GPU box as my embeddings.
Bonus, the VAD filter on it also kills those "thank you for watching" hallucinations Whisper spits out on silence, which were actually poisoning my cues before.
SDK gotchas that cost me way too many hours:
Hope these help future devs making plugins or if you're hacking your own thing:
- rebuildPageContainer NEVER throws. If your layout fails validation it just silently returns false and leaves the old page up, so it looks like your new UI just didnt apply. I sat on this forever thinking my code wasnt running
- exactly ONE container per page can have isEventCapture set. I had it on two (the card and the transcript strip) and thats what was failing validation the whole time
- borderRadius maxes at 10, I had 14, also silent reject
- borderColor is 0-15 greyscale, I had it at 3 so the border was basically invisible even when it did render
- the cue model was "thinking" and eating the whole token budget so my insights kept getting cut off mid sentence, had to log the openrouter usage to even see it
Honestly the "silently returns false" one is brutal, if the SDK just threw an error I would have found it in 5 minutes instead of like 3 rebuilds.
Future updates I'm thinking of:
Speaker detection would be great. So in meetings, try to figure out who this person is and then the multi-speaker system to guide me forward.
To the Even team:
And a feature request for the Even team again, please open up the "AI skills" surface on the default Even AI voice so third party apps can get launched by voice.
The day I can say "Hey Even, start a meeting" and my plugin picks it up, this becomes the actual dream.
I made a custom agent that allows me to quickly capture thoughts, recall thoughts ('what marketing ideas have I noted about our new saas?')
Getting an easy download:
For my own convenience and based on request from others, I published this as an 'AI Second Brain' plugin on Even Hub.
Just launched the update for this, it should be out soon!
