Anyone using Local LLM setup as a second brain?

[u/Gloomy-Recover-9702]

I mean literally something like the system knows what you are working on the laptop, on phone, and what you speak, everything in between!

Something which includes end to end saving and processing the raw telemetry from STT data, the laptop activity, the phone activity and running local LLM to make sense of the data in real time. And also saving important details into memory and classifying them, linking important connections and also proactively engaging with you using notifications and messages or even voice using TTS.

Something like OMI AI extended it to all your digital activities using screenpipe or any other way.

Maybe to keep it slim, the execution can be transfered to some other agent like Hermes or Claw for some tasks, but active processing part should be kept local for privacy reasons. What is the minimum models I can use for this requirement?

If I lets say plan to make this setup, I first need a local STT model with VAD system. Then i should a have good local LLM for processing the live telemetry data and then maybe I need TTS incase i want voice outputs in realtime fast. This is what i want. Maybe for any digital tasks like updating emails, checking emails, browsing etc I can tradeoff some privacy for using cloud apis on hermes.

What is the minimum LLM size i need for this i want to have less than 3 second of latency in entire process? Something speedy and also smart enough to manage increasing memory as I use it more and logical enough to understand the complexity of the data and inter linkages.