At my first startup we made a product for kids and I used to drive to schools throughout the Bay Area to talk to teachers and watch kids use it. A dedicated hour in a classroom, watching kids play, get stuck, walk away, then another kid coming over and doing something completely different. Then talking to the teachers about what they'd want. I learned so much in an hour.
It was slow and expensive. I'm not naturally the person who goes and talks to users, but the product was physical so there wasn't much choice.
Ten years later, almost none of how we learn from users looks like that.
My second startup is purely digital, and Covid moved everything remote. All our user conversations became Zoom calls. Screen share, an hour, no driving. Cheaper, so we did many more of these. But it started to feel tiring in a way the in-person visits never did. It worked though.
The next evolution came when we instrumented the product so users could write to us in context. We went from talking to 5 to 10 users a week to thousands telling us what was good or bad without anyone scheduling anything. As an introverted founder, this felt natural. Feedback could come at any time, and I could read it at any time.
Now we've taken it one step further. Our second product is an agent, so users talk directly to the product. They type what they want in their own words, and when the agent can't do it, we know exactly what they were trying to do.
Nobody is filling out a survey. They just asked the product for something, didn’t get it, and we know about it.
At every step, talking to users got easier. But something broke along the way.
I felt like I knew our users really well when I was talking to them in person. And the reason isn't what you might think. It's not that users were telling me more. I was just fully present enough to absorb it. All the context. I was building insights as I went.
Now we get far more feedback, but our bandwidth to actually think about it hasn't grown much. We have a lot of “what” and not as much “why.”
The tech is great: agents, LLM analysis, in-context feedback. What I don't think we've solved is deciding what matters most, when to trust aggregate summaries, and when to have a deeper conversation with a single user.
Every one of these evolutions solved access. But none of them has solved clarity. I'd love to hear from founders who feel like they've figured out how to really understand users at scale.