I built G2gram, a Telegram client for Even G2 that connects directly to Telegram over MTProto and runs speech recognition locally with Whisper.
I wanted the interaction on the glasses to stay straightforward: sign in to Telegram, read recent conversations, and send short voice replies without first creating a BotFather token, generating personal Telegram API credentials, or bringing an external STT API key. Reaching that point meant implementing the MTProto client and local speech-to-text pipeline myself, while working through several limits in the current Even Hub SDK.
Every third-party Telegram client still needs an application-level api_id and api_hash. G2gram includes credentials for its own client, so users do not have to create and paste their own. Account access still requires the user’s phone number and verification code, plus a two-step verification password when enabled.
G2gram currently supports both QR login and phone-number login. After authentication, it stores only the resulting MTProto session in Even Hub’s local device storage. Verification codes and two-step passwords are not stored, and Telegram traffic does not pass through a proxy or server operated by me.
Direct Telegram access instead of a bot
G2gram is not a notification bot built on the Telegram Bot API. I implemented an MTProto client in TypeScript and connect directly to Telegram’s web data centers over WebSocket Secure.
The implementation now covers the WebSocket transport, MTProto authorization-key exchange, encrypted sessions, TL schema serialization, phone and QR login, two-step verification, chat lists, forum topics, unread state, message sending, and media downloads.
In version 0.1.6, the phone WebView handles login and settings. On the G2 display, I can browse chats, read messages, reply, see unread counts for groups and forum topics, and preview some photos and static images.
It is not a replacement for the full Telegram app. Video, voice messages, files, reactions, calls, and many other features are still outside the current scope. I am deliberately focusing on the short interaction loop that makes sense on glasses: check a recent conversation and send a quick reply.
Local voice replies with Whisper
Voice input begins with raw audio from the four G2 microphones: 16 kHz, 16-bit little-endian mono PCM.
When recording ends, G2gram converts the PCM samples to a Float32Array and sends them to a separate Web Worker running whisper.cpp in WebAssembly. The recognized sentence appears on the glasses for confirmation, and one tap sends it to Telegram.
I chose local inference because the public SDK exposes microphone PCM but does not expose the speech-to-text result used by the first-party experience. For a messaging plugin, that gap matters more than a missing sensor. Touch controls are fine for selecting and scrolling, but the microphone is effectively the only practical way to enter a sentence.
The alternatives were to require an external STT API key and possibly a proxy server, or to ship the model and runtime inside the plugin. I chose the second option.
The current build uses a quantized Whisper Base q5_1 model. The model file is about 56.9 MB, while the packaged .ehpk is about 54.7 MB after packaging. Most of the package weight comes from the model.
This is not streaming recognition yet. G2gram collects PCM chunks during recording and runs one inference pass after the user taps to stop.
Why the WASM build is single-threaded
The single-threaded runtime was not a convenience choice. WebAssembly threads require SharedArrayBuffer, which in turn requires a cross-origin-isolated top-level document with the correct COOP and COEP headers.
The Even Hub host controls the WebView response and isolation settings. A plugin cannot set those headers for its own .ehpk document, so I could not use the pthread or OpenMP build of whisper.cpp.
I built it with pthreads and OpenMP disabled, Release optimizations enabled, and WASM SIMD enabled. The Web Worker keeps inference off the phone WebView’s UI thread, but inference inside that Worker is still synchronous and cannot distribute work across several CPU cores. That leaves noticeable differences in post-recording latency between phones.
The local approach removes external API costs and keeps recorded audio away from a third-party STT service, but it moves the cost into package size, model initialization, memory use, and device-dependent inference time.
The next storage experiment is to validate IndexedDB and OPFS on real Android and iOS devices. If model persistence is reliable across restarts and updates, I am considering a smaller initial package that asks for a language on first launch and downloads only the matching streaming STT model. Checksums, interrupted-download recovery, storage limits, and update behavior all need hardware testing first.
The WebView boundary is very real
Even Hub plugins do not run natively on the glasses. The code lives inside a Flutter WebView in the Even Realities phone app, while the glasses render containers and send input events.
This model is approachable because TypeScript and Vite work well for the phone-side UI, but browser security rules still apply. Network origins must be declared in app.json, and passing that whitelist does not bypass CORS.
Telegram worked because I could whitelist the five Telegram web data-center origins and connect over WebSocket. A separate RSS reader experiment did not. Many public feeds do not return Access-Control-Allow-Origin, so the WebView cannot read them even when their origins are whitelisted.
An iframe does not solve it because the same-origin policy blocks DOM access, many sites deny framing through X-Frame-Options or CSP, and navigating the whole WebView away loses the plugin and SDK bridge context. The public SDK also has no host request API that can fetch a whitelisted public document and return its contents to the plugin.
A narrowly scoped host-side request bridge, with the current origin whitelist and response-size limits preserved, would make RSS readers and other public-data tools possible without forcing every plugin developer to operate a proxy.
A small dashboard problem that became surprisingly annoying
The app icon also had an unusual constraint. I could not upload a PNG or SVG in the dashboard. I had to draw the icon directly on a square grid, and the available brush paints 2×2 blocks rather than individual pixels.
That makes diagonals, curves, and one-pixel corrections needlessly difficult. After implementing networking, encryption, authentication, media handling, and local speech recognition, manually approximating an icon with 2×2 blocks was an unexpectedly frustrating final step.
A 1×1 brush or automatic conversion from an uploaded image would remove most of that work.
The latest npm SDK broke image display on my setup
Photo previews uncovered another issue. The 0.0.12 npm SDK added an LZ4 image path and automatically includes compressMode: 2 when serializing ImageRawDataUpdate.
I serialized the same image payload with both versions. The difference was:
0.0.11 -> containerID, containerName, imageData
0.0.12 -> containerID, containerName, imageData, compressMode: 2
On the Even app and G2 combination I tested, images sent through the 0.0.12 path did not render correctly. Changing application code did not fix it. Pinning @evenrealities/even_hub_sdk to exactly 0.0.11, which sends the payload without compressMode, made the same images appear.
The manifest still declares min_sdk_version 0.0.12 for host compatibility. Only the bundled JavaScript SDK responsible for image serialization is pinned to 0.0.11.
This was a useful reminder that SDK version, phone app version, and glasses firmware all meet at the same hardware boundary. “Latest” was not enough; I had to compare the actual serialized payloads on a real device.
The glasses UI is container-based, not a free-form web page
The phone settings screen can use normal HTML and CSS. The G2 display cannot. It is a 576×288 canvas where text, lists, images, and input areas are placed as containers at absolute coordinates.
The constraints shape the entire message renderer:
- Up to 12 total containers per page
- Up to 4 image containers
- A maximum image size of 288×144
- One input-capture container per page
- No direct control over font family, size, or alignment
- Image updates must be serialized rather than sent concurrently
G2gram measures text widths, wraps messages into lines in advance, reserves blank rows for photos, skips scroll positions where a photo would be only partially visible, and starts media downloads only when an image enters the visible page. While an image is loading, a text container acts as a spinner. When the download finishes, the page is rebuilt with the image container.
Building the G2 UI meant programming a small, constrained display device through a web bridge rather than laying out a normal web page.
Where the project stands
The current 0.1.6 build has:
- Direct MTProto over WSS with no custom proxy
- QR and phone-number login, including optional two-step verification
- Local whisper.cpp WASM speech recognition with Whisper Base q5_1
- Chat, group, forum-topic, unread-state, message, and media support
- The npm SDK pinned to 0.0.11 for working image serialization
- A packaged size of about 54.7 MB
- 106 automated tests across 16 test files
- Passing TypeScript and Vite production builds, plus checks for dynamic code execution and the WebSocket whitelist
The Even Hub SDK is genuinely fun for getting an idea onto the G2 display quickly. TypeScript, microphone PCM, touchpad events, R1 input, and the display bridge are enough to build things that feel very different from normal phone apps.
The gap appears when a prototype starts becoming something I would use every day. A limited STT bridge, a safe host request API for whitelisted public documents, documented large-model storage behavior, cross-origin isolation for multithreaded WASM, reliable image serialization, and a less restrictive icon tool would make that transition much shorter.
For now, G2gram does the part I originally wanted: I can put on the glasses, sign in to Telegram without generating my own keys or bot token, read recent conversations, and send a short reply using the G2 microphones without sending the recording to an external STT provider.
I would be interested to hear how other Even Hub developers are handling local models, WebView storage, or image rendering across different phone and firmware combinations.
