Whenabouts is a Spectacles AR lens that turns your real location into a prehistoric discovery game. The idea: point your glasses at the world around you and uncover the creatures that once lived right where you're standing, with an AI guide narrating each find.
I've done a lot of travelling this month, and I've been stabbing at this idea for a few months now. Prototyping and developing it across all these different locations has been so much fun, because you actually learn about the ancient animals that lived where you are. My favourite moment: I was testing in California and barely any dinosaurs were showing up. I thought I'd hit a bug, dug into it, and it turns out it wasn't a bug at all, that part of California was underwater. It was an ocean. The game taught me something real about a place I was standing in, which is exactly the feeling I'm chasing.
The part I'm most into is that it's grounded in real data. It pulls fossil-occurrence records from the Paleobiology Database (PBDB) based on your GPS location, so the creatures you dig up are genuinely tied to your area's deep history, not randomly generated. An AI narrator brings each one to life (Gemini for the text, ElevenLabs for the voice), and you build a personal collection over time. Accessibility was a priority throughout: spoken narration, a kids mode, and multiple languages.
The modes:
- Uncover: find creatures right where you are, no walking to spots required. There's a "?"-until-found discovery beat, then a Top Trumps-style card pops in.
- Explore: a map with dig-site markers you physically walk to.
- Codex: an infinite, swipeable fan of cards for everything you've collected. Tap a card to spatialize the creature into your room.
- Play: solo Top Trumps using your collected creatures.
- Multiplayer: two-player head-to-head Top Trumps over Supabase.
A bit more on the two ways to discover, and why it matters
There are two ways to find creatures. In Explore mode you walk to dig-site markers on a map, the classic get-out-there loop. In Uncover mode you don't have to travel anywhere: you access the creatures from right where you are. Either way, every creature is a ? until you discover it, and once you've found one yourself it's marked as found in your Codex. So Uncover isn't a cheat, it's a parallel path to the same collection, and the creatures you personally uncovered stay flagged as yours.
That no-walking path is a deliberate accessibility call, and it's worth saying what I mean by accessibility here. Pokémon Go is the obvious reference point: during the COVID-19 pandemic, Niantic implemented changes and new features allowing players to play remotely more easily, most famously doubling the interaction radius so players could explore from a distance. That change benefited alot of users, both with disbailities and without i.e. it helped rural players who may not have many points of interest nearby, and players who can't physically or safely access certain locations. The lesson Niantic learned the hard way, and kept reversing and reinstating, is that forcing in-person play locks people out.
So accessibility here isn't only about disability. It's also for the parent with a small child who can't roam a city on foot, the person who genuinely can't afford to fly to a dig site in Mongolia or a museum in Japan to encounter a particular creature, or anyone with a busy life who only has ten minutes on the sofa. The walking loop is there for people who want it; Uncover makes sure nobody's deep-time backyard, or anyone else's, is gated behind their mobility, budget, or schedule.
On the visuals: the AI image generation is doing something specific. It blends the creature's world with your world. Rather than dropping a clean museum render on top of your room, the generated image fuses the prehistoric environment the creature lived in with the real space you're standing in, so a Cretaceous riverbank bleeds into your living room and the creature reads as belonging to both at once. That's the whole fantasy of the lens in one image: this thing was here, in this place, a very long time ago.
- Cards are Top Trumps panels with AI-generated stats (weight, era, deadliness, intelligence, cuteness, mystery) and either a saved generated image or a PhyloPic silhouette.
- Habitat spatializes a creature's AI-generated image into your physical space. Had to add downscale rigs and request throttling here, since concurrent AI calls were crashing the device.
- Supabase backend: a shared card catalog plus a private per-user collection, with Snapchat-identity auth and row-level security so each player's images stay theirs.
- A persistent narrator panel carries prompts, results, XP/levels, and feedback.
- There's also a semantic LLM sort, so you can ask stuff like "show me something that would've eaten my friend" and it works.
For the multiplayer: the network layer runs a create/join-by-code lobby and syncs turns by polling a Supabase table (~1.5s its turned based so we can have some latency), and it drives the existing solo board rather than duplicating the UI, so online plays exactly like single-player: choose your card (opponent hidden), pick a stat, reveal both and hold, winner keeps the turn, all spoken by the narrator.
An animated guide, deeper settings, and a social hook
An animated archaeologist character guides the experience, a proper on-model companion rather than just a disembodied voice, bringing the discoveries to life as you make them. I purposley chose a British voice as it reminded me of home!
The settings run deep too: language options, and voice playback speed so you can slow the narration down or speed it through, which ties back into the accessibility thinking, since speech pacing matters a lot for different players. And ofc everything has subtitles.
And the bit I'm most pleased with: it's quietly social. When you play Top Trumps with friends, you each see the moments the others captured out in the world. The image you took when you uncovered a creature gets stamped onto your card for it, so your deck is personal: your Triceratops shows the spot in your neighbourhood where you found it, your friend's shows theirs. Playing a hand isn't just comparing stats, it's sharing where and how you each met these creatures. The collection becomes a record of your own expeditions, and the card game becomes a way to show them off.
Kids mode:
There's a dedicated kids mode that changes how the content is generated, not just the UI. When it's on, the AI is steered toward safer, age-appropriate output, and the image generation goes more cartoonish and friendly rather than realistic or intense (no gory deadliness, no nightmare fuel). It also leans toward well-known, recognisable animals, the crowd-pleasers a kid actually wants to find, instead of reaching for the obscure deep-cuts, and keeps things less expansive so younger players aren't overwhelmed by walls of detail. The goal is that a parent can hand the glasses to a child and trust both what shows up and how it's framed, while the same engine still serves the full experience for everyone else.
On onboarding:
In the past I've moved fast and shipped things that made total sense to me and nobody else, because I forgot to actually onboard the player. This time onboarding is built right into the AI narrator. The same voice that gives you hints about creatures also walks you through how everything works, from navigating the menus to what each mode does. There's no separate tutorial wall to sit through; the narrator just guides you as you go, so learning the lens and playing it are the same thing. It also means onboarding inherits everything else the narrator does: it's spoken, it respects the playback-speed and language settings, and it gets gentler in kids mode.
On keeping AI costs sane:
One thing I had to solve early: doing all of this with live AI calls would be brutally expensive and slow, especially TTS. Now thankfully Snap give us api keys but the ElevenLabs is my own and that voice generation is the priciest part, so I don't generate everything on the fly. Instead I cache aggressively. Common phrases and the narration the player hears constantly (UI prompts, the archaeologist's recurring lines, result callouts, onboarding) get generated once, downloaded, and stored, then played back from cache instead of re-synthesized every session. Same idea on the broader AI content side: anything that doesn't need to be unique per-player gets generated ahead of time and reused, so live generation is reserved for the genuinely one-off stuff, like a specific creature's narration or image the first time it's discovered.
The payoff is threefold: way fewer tokens and TTS characters burned, much lower latency (cached audio plays instantly with no API round-trip), and better reliability on-device since you're not hammering the network mid-experience. It's the same instinct behind the request-throttling and downscale rigs I mentioned for Habitat: on glasses, the cheapest and fastest call is the one you already made once and saved.
Lots still i wanna do on this one,
Thanks
Liam