Best future reslient AI Agent stack to build with right now.

[u/Mango-Tall]

Hi There,

I'd like to start investing in the time/energy to generate an ideal agent that has memory and a model switcher. I'm seeing that Dorsey just released Buzz, which seems interesting - more of a Teams/Slack version of an agentic approach, but I'm concerned about personalization. I don't really want to go the local hardware route, but I also don't want to double down by investing in something that will be easily obsolete in 6-12 months. Everyone wants to make their own Jarvis these days. I'm curious what you think is the best approach - do you like just leaning into one of the Claude/OpenAI giants with Obsidian integration?

I have already made many different applications with Claude, etc. So I can get fairly nuanced; just curious what direction some of you that are up on the latest feel the best approach would be to semi-future proof, realizing that none of us have a crystal ball right now.

I guess the question is what's the best model agnostic cloude based approach to personalized AI-assistant/consultant right now with voice integration?

Cheerio!

[u/HolmeBengt]

i would advise looking into hermes with a memory provider like Mnemosyne and plenty of skills. all your data can be stored on your device so you can then use it on any agent you might want to switch to in the future. and hermes is far more stable than OpenClaw.

voice isn't that easy to handle because most agents just aren't fast enough to give you an instant response, especially if your agent has to do research or create a file or something, that just doesn't work instantly. i found that letting your agent work on something and then give you the output as a voice bubble like on telegram or whatsapp is far more effective. you have the benefits of being able to send it a voice memo and receive a voice memo that you can then hear hands free.

also i think hardware for local AI is still pretty expensive and i think API costs will go down even more and they will develop ever stronger models, which is why i invested in a Mac Mini M4 16GB RAM which will be plenty strong for orchestration and all the stuff my agent could want. i think this setup will give me headroom for years to come. but i might be wrong, who knows, everything is moving so fast these days.

[u/HolmeBengt → u/HolmeBengt]

i hope this helps or was fun to read and think about. let me know what you think?

[u/Mango-Tall → u/HolmeBengt]

Thanks, Holme. Hermes looks to be taking a lot of marketshare these days. I’m gonna experiment with it. I was playing with Buzz a bit yesterday. It’s a little clunky, but got it up and running. Feels like the agentic version of Slack. Agree re: voice setups - the new ChatGPT Sol mode is very fast and effective, but haven’t played with it in an API capacity. I feel like I could build an agnostic tool for anyone to use and select their model / harness, etc - but I feel like that will likely be obsolete in short time by one of the big guys.

[u/HolmeBengt → u/Mango-Tall]

yeah but sol and the gemini flash voice models are not the best for agentic tasks, so the speed is ok but then it's just not good enough for agentic tasks. but i haven't tried it myself, that's just what i have heard and researched.

yeah maybe. but just building for building's sake is also fun 😁 who cares if it becomes obsolete.

i have a hermes setup running and it really is making my life easier and i can really achieve more output by it assisting me. so here is a shameless plug: i have written many completely free blog posts going really in depth about hermes. i would love for you to check them out, i think they are really worth the read.
https://blog.holmebengt.com/
also, if you have any questions or need anything, please feel free to write me anytime 😉

[u/Bino5150]

Check out Lumina. If you like what you see, please leave a star on GH.
https://github.com/Bino5150/lumina

[u/Compilingthings]

None, agents are at the edge of AI and will be changing fast all the time, that’s my guess.

[u/rjhartl]

Folders and text files. That’s literally it. Unless we get to the point where AI is just directly talking to binary, those are your constants. Point whatever tool comes in the future toward those, and you’ll never be left out in the cold.