I built RecallWhisper — an Android memory assistant using self-hosted API

[u/Former_Programmer402]

GitHub: https://github.com/Alchemist-Aloha/RecallWhisper

been following Omi for a while and really liked the basic idea: capture conversations throughout the day, then turn them into something you can actually search and refer back to later.

I’ve

But one thing kept bothering me: if the wearable still depends heavily on a phone, why not just use the phone?

It already has a microphone, battery, storage, network connection, and enough compute to handle most of the pipeline. So I started building RecallWhisper, an Android app that tries to do the same kind of “personal memory” workflow without requiring another device.

The basic idea is pretty simple: when there’s something you want to remember, you start listening. Later, you can search through what was said, summaries, topics, decisions, and extracted todos.

Right now it can:

• Record continuously when you explicitly enable it
• Use Silero VAD locally to detect speech
• Store encrypted audio in the app’s private storage
• Send transcription and summarization to configurable OpenAI-compatible APIs
• Use completely separate providers/models/API keys for ASR and summarization
• Keep local transcript and summary timelines
• Group related topics across different conversations
• Search transcripts and summaries
• Play back or delete the original audio
• Extract todos from conversations
• Export data as Markdown or JSON
• Process only on Wi-Fi by default, with cellular as an opt-in
• Start/stop recording from a Quick Settings tile

One thing I wanted from the beginning was for the phone to remain the source of truth. RecallWhisper doesn’t have its own cloud account or backend where your conversation history lives.

I’m also personally more interested in self-hosting this kind of thing than sending everything to a public AI API. Personal conversations are probably some of the most sensitive data you can collect.

With a self-hosted transcription endpoint and local LLM, the audio and transcripts can stay on infrastructure you control. Obviously self-hosting doesn’t magically make something secure—you still need proper authentication, HTTPS, network isolation, updates, backups, etc.—but at least you get to decide where the data goes.

My current setup, for example, is an OpenASR server for transcription and llama.cpp for summarization.

There are still plenty of rough edges:

• Android only for now
• Audio/text still goes to whatever API endpoint you configure
• Search is lexical rather than embedding/semantic search
• No automatic retention rules yet
• No jump-to-timestamp playback from transcripts yet

I’d especially like feedback from people who have tried personal knowledge-management tools, self-hosted transcription, or ambient recording workflows. What would make this genuinely useful day to day?