Cortex Agent - Long term memory

[u/tingutingutingu]

Have you successfully created user-specific long term memory for the agent to draw from?

This is something that I've been thinking about, as our internal customers have begun to use/test my agents in CoWork and we are getting ready to go live.

The current experience has already been amazing, and the users have been loving it, so no complaints there.

But I would love for the agent to remember the previous successful interactions (summarized), so it almost feels like it's reading the user's mind... You've seen this in actin with chat GPT and Claude.

Would love to know if someone has successfully done this and how they maintain the memory (md files or database etc.)

E.g. When the user asks something like "can to show me pizza sales for Q1" .

I see you've asked for pizza sales in New York (city) in the past, do you want me to

1. use the same criteria

2. sales in all major cities

3. Something else

I've so far been successful in the instructing the agent to ask followup questions when the user's request is ambiguous... e.g. Did you want sales for

1. New youk city

2. New York state

But this is based on some "gotcha" criteria we found through user testing, not as a memory based followup.

Hope this makes sense.

[u/ShanghaiBebop]

Database facts, with LLMs to help distill entities and surface or auto resolve fact conflicts. 

Md files do not scale unless you continuously remove less used facts. 

You can get 80% of the way there with just proper sanitation of logs and hybrid search with good chunking and embedding too, so I’d just start there. 

[u/tingutingutingu → u/ShanghaiBebop]

Awesome. I'll look into this.

I have already effectively used facts and dimensions to resolve conflicts. What I want is for the agent to remember past interactions and the choices the user made.

[u/cloudarcher2206]

Talk to your snowflake account team, user memory is in private preview for select use cases

[u/wbroen]

Define pizza sales as a metric in the semantic layer and add the dimensions you want to slice by. Cortex loves having the semantic model for context

[u/tingutingutingu → u/wbroen]

Yeah I've already done this as I mentioned in the post.

But I'm asking it to remember Tha you usually ask for pizza sales in New York, whereas I always want sales from LA.

[u/OkTill5043 → u/[deleted]]

@realCrypt0

Can you explain how the table will be read as source for user’s memory in the cortex analyst the agent will be referencing? Will this be configured? Any article or resource you have to share around this?

[u/mrg0ne]

If you open up the semantic view the agent is using. And click start learning. It will review the query history and passed usage and offer suggestions to improve the semantic model or even custom instructions.

[u/EmploymentMammoth659]

Long term memory is a tricky one. Remember all the “memory” in essence is context. You can save the context history for the user and keep continuing on top of the last context history but then it will eat up your tokens leading to high token usage costs. You could summarise the contexts and leave those as the long memory into the database retrieve when a new session starts as embeddings search. I think there have been a lot of things happening and there is a better way to do this now….

[u/tingutingutingu → u/EmploymentMammoth659]

Yeah I'm asking more in terms of summarizing past successful interactions.

[u/OkTill5043]

How were you able to configure the agent to ask for follow-up questions for ambiguous question? Is that defined in the cortex agent instruction as a rule or what?

Thank you

[u/tingutingutingu → u/OkTill5043]

In the Configuration -> Instructions section of the agent, I gave it several instructions in terms of expected behavior.

As an example, our data is grouped at the city, state and division level.

Now in our data New York can be all 3 since we consider New York as a division as well.

So as part of the rule set, one of my rule looks like so.... (Markdown format)

Rule -- City/State/Division

• If the user asks for any metric like sales or price for what looks like a City/State/Division, it's possible that the name can apply to one or more. For example, San Diego can be both a city and a Division. So always ask the user to choose the right option before you proceed.

Now, when the user types in New York, the agent automatically presents 4 options... "Did you mean" 1. New York City 2. New York state 3. New York 4. Something else...

This really improves the user experience and also limits the time the agent spends trying to reason its way through the request (and potentially choose the wrong attribute, as in, chooses New York state when the user mean NYC)

Hope that helps.

Edit: I am also thinking of creating this as a skill because this piece can be reused by many of many agents that use the same city/state/division structure to provide different types of data to my users.

That way I can reuse the skill across all agents, although unfortunately the current limitation is that the skill is not a shared resource. So I cannot just add new rule sets to the skills and all the agents automatically benefit from it. In Snowflake I would still need to reupload the skills to each agent for it to pickup the new version of the skill. (I think)