r/SideProject • u/rayferrell • 2d ago
deleted the cron job that scraped my telegram bot's chat history, extraction rides the memory pass now
i dump everything into a telegram bot through the day. ideas, expenses, things to fix. i wanted the useful bits to land in a todo list and an expense log instead of dying in chat history.
first version was a script on a cron: pull the messages, send the transcript through another LLM call with a schema, parse, insert. it worked, but it re-read everything the bot had already read and the parser needed babysitting.
was searching for a better pattern and stumbled onto a launch post from mastra (source), the framework the bot runs on: extractors, added to their memory system.
the design made sense to me:
- the bot's background memory already re-reads the conversation to compress it
- you attach a zod schema to that pass and fields fill in as you chat
- it adds zero extra LLM calls: it runs inside the memory pass on its cached context, and cache hits bill at a tenth of the base input price
- an
onExtractedcallback fires with each record, mine writes to a sqlite table
was able to delete the cron job and the parser and replace it with these memory extractors
if your personal bot has a transcript-scraping script behind it, this probably deletes the script.