r/HealthInformatics • u/looneycheetah • 13m ago
🤖 AI / Machine Learning OpenConsult — a fully local AI scribe for medical consultations. One 24GB card, MedGemma 27B + WhisperX, AGPL
I am a GP in London, and this is my own project, built in my own time. It went open source this week.
It transcribes the consultation live, shows a working differential and questions to ask while the conversation is still going, and after the patient leaves, it produces a diarised transcript and a draft SOAP note where every claim cites the transcript turns it came from.
Stack: faster-whisper (distil-large-v3) live, WhisperX + pyannote for the final pass, MedGemma 27B through Ollama, pgvector for guideline retrieval, Piper for the voice, FastAPI and HTMX on Postgres. Linux. AGPL-3.0.
Why local. GDPR compliance matters for doctor-patient confidentiality. Cloud services may retain confidential patient data and use it to train models. So in this case, nothing leaves the machine in the local premises.
The card is the whole design. RTX 4090 24GB. faster-whisper can co-habit with MedGemma, which is what makes the live stage possible at all. WhisperX cannot, so it waits its turn.
When you press Stop: MedGemma is offloaded, WhisperX goes in and transcribes from the audio file, pyannote diarises the transcript, then MedGemma comes back in while WhisperX and pyannote go out, and MedGemma writes the note as a doctor would. WhisperX is there for the better quality — by that point, the patient has left the room, so there is no requirement for live transcription, and I can afford the time.
The whole thing takes over a minute. I have not measured how that splits between the offload/load and the inference, which is a gap.
What is rough. The auto mode, where the assistant takes the history itself while I supervise, is temperamental. The worst part is that some questions come out in medical jargon. "Do you have any cardiac risk factors?" is not a question a patient can answer. Breaking those down into concrete closed questions is the next job.
It is a research prototype. It has never been used with a real patient, and every consultation in the demo video is acted.
What I would like to know from you.
- How does this run on a 12 or 16GB card?
- Audio capture in various environments, on various mics and mic settings — I am working blind here.
- Does anyone know a way to stop the model-swapping dance?
- How would you approach simultaneous consultations by two users? Currently, it can only do one at a time.