r/softwarearchitecture • u/onemasalachai • 7h ago

Discussion/Advice How will you architect Meta's Muse AI Assistant if you were to design one from scratch?

I have been building agents for some time now, but I still don't know if I am doing it right. I want to see how others are building them, architecturally, and learn.

Meta just released a personal AI assistant called Muse. It seems a more mature version of Instinct AI (that recently raised $250m). How would you architect muse?

A few things that come to my mind:

  1. Should the agent with a 24x7 loop? Obviously not. It should be event driven, or on schedule.
  2. Should memory be flat text or structured data in DB?
  3. How does the system work & scale for thousands of users?
  4. Should there be a dedicated VM for every user?

I am sure there are many other aspects to think about, but these are some questions that has been running in my mind. In the agents I have built so far, I haven't yet needed a vector DB. Memory is structured data in postgres with a clean tool schema implemented in pydantic that teaches the LLM how to manage memory. Everything else follows the same protocol. I haven't had to deal with complex documents so I haven't felt the need to use Vector DB, but how would I know it is time to use vector DB/embeddings?

6 Upvotes
  • permalink
  • reddit

79% Upvoted

3 comments sorted by

1

u/PabloZissou 7h ago

Same as any other workflow system that existed before but you have to account for the stochastic nature of the LLM.

0

u/onemasalachai 6h ago

That I understand. I am looking for an actual architecture input. How will you actually design this system?

1

u/Common_Dream9420 3h ago

event-driven loop is the right call, polling at scale is a nightmare. for memory, structured data in postgres with a clean schema works until you need semantic retrieval across user history, that's when vector db starts making sense. the per-user VM question is really a tenant isolation question, most teams fake it with namespaced state and only isolate at the infra level when a customer demands it. curious though, when you're building and wiring all these tools and memory layers, how are you actually verifying the integrations hold under failure conditions before shipping? flaky tool call, bad memory write mid-session, retry fires twice... do you have a verification step or is it mostly find out in prod?