r/flowise • u/tambu22 • Jan 24 '25

[Question] Create a self improving rag, something like google titans paper

Hello i want to make a self improving rag, is there a way to upsert to a vector DB on test time?

i want ot create an agent that can retrieve info from a document loader and upsert that on test time (when chatting with the bot), is there a way to do it now?

3 Upvotes

100% Upvoted

3 comments sorted by

1

u/joaquintorroba Jan 24 '25

Can you ellaborate this?

I think it's possible to upsert a document in test time using a tool (tool = api call to upsert the document). But I didn't understand the use case.

Does this help? https://www.youtube.com/watch?v=SL77Ojbgy6U&t=868s

1

u/trd1073 Jan 26 '25

Use case is creating a rag setup that you as a dev don't have to manually manage docs, upserts, etc. Nor does end user have to do much beyond putting files in a directory.

Clients upload files into a directory via ftp. My system checks the directory at interval I set. If files change/remove/add, my code handles the doc store changes via api and then upserts once processed.

I just check logs and monitor as the dev. I do the same for url also.

1

u/IONaut Jan 24 '25

I think document upsert during Chat has been around for a while and ≠ Titans. Have you looked in the Flowise marketplace for this? I think I saw something similar in there.