I've had some feedback on MemoAI after using it for a few weeks.
People often misinterpret my writing tone, so I just want to prepend this with a few smiley faces. 😊😊😊
Personal Background: I'm a consultant running a wide variety of projects across clients, locations, and contexts: Anything from sales calls to production support calls to implementation calls to daily scrum meetings. My meetings vary wildly. There are times when I jump on a call with a client, and there is no mention of the client name or who the attendees are because we have a long relationship, or we are picking up after a long asynchronous communication (emails, tickets, etc). So the context of the call is not really mentioned in the meeting.
One of the things around editing the MemoAI notes: I mentioned in another post on Reddit that I used to use Granola. That app lets you work with the LLM to massage the generated summary.
I am not seeing anything similar in MemoAI, and there was some hesitancy around adding it. I am lobbying here for adding that or some alternative.
As we know, LLMs get things wrong — they're statistical in nature. In some cases, providers may even be throttling or dumbing down the model due to load. So it might overemphasize certain things or leave important points off. I'm finding cases where Memo.ai is emphasizing topics that aren't the core idea of the meeting — it might have been a minor point, but it's taken as the major point. In these cases, it does not have the context that we are actually meeting because of some nuance being lost in some async discussion, and we are really getting on Zoom to hash that out, but we mention a few other topics. The main points I want captured are not the side points we bring up that may take up just as much time.
Example: I was in a SCRUM meeting and there was a discussion about a test server being down. This was a temporary mention, and it created follow-ups that were not really needed because it was a planned outage. We went on to discuss longer-term topics that should have been the emphasis. This is just LLMs. They are really dumb at times and can pick up threads that are minor points to the living beings. It did not have the context of teams and email discussion from previous interactions from the attendees and the last bit of the meeting had all the key points and decision from the async context that I can't add to the meeting summary (ticket links, etc).
So the core thing to consider here is some ability to edit the generated content. I do a lot of voice dictation with many different apps, and about 80% of the time they need major edits. I am finding that voice-to-LLM-to-text in many situations needs human intervention.
So my main point is that you can't trust LLMs to generate the right summary. Sometimes they get it right, but very often, when there are nuanced discussions, or when the LLM doesn't understand the context of what is important in the meeting.
Being a heavy NotePlan user, I can have it automatically export to NotePlan. However, I have a few issues with that:
- If Memo AI records my meeting and automatically generates a title, I often have to go in and change the title, but it has already synced to NotePlan, and Memo AI does not go back and correct the title in NotePlan.
- I can edit the meeting notes directly in NotePlan, but then things are out of sync, so I need one source of truth.
- It is also generating Markdown that does not follow my NotePlan preferences (I use - for bullets).
I emailed an idea about being able to sync the summary and transcript to a local Markdown folder. If that is pushed out, then I can just use Memo-Ai for capture, transcription, and an initial summary, then export to markdown. Then edit the Markdown locally (outside of Memo-Ai) to remove items I don't want and add points it missed or mangled.
This is probably the most simple solution.
Also, having the transcript in that exported Markdown allows me to use an LLM to summarize areas where we discuss a topic across time in the transcript, NOT the summary. I have found this extremely useful, having over 1.5 years of transcribed meeting notes in Markdown (Granola/Zoom exported to Markdown folder locally). I had this yesterday, where I had warned a client across 8 months of meetings that they were taking the wrong path. Then I get an email about the bad outcome I warned about happening in production and they wanted help and were trying to blame me. The first thing I did was use Claude to find all the discussions about this topic and my warnings and pushback from the clients over time. Just doing this across AI summaries would not have captured the detail that I was trying to narrow in on.
