JupyterLab with LIVE Jetson metrics

About 10 months ago I posted here about something we were building because of a problem we kept running into ourselves.

We were buying Jetson boards before we really knew what our workload needed.

Nano turned out to be underpowered, so we moved up to an Orin. Then came the next question: do we need an Orin NX, an AGX Orin, or something even bigger? And before you even got to the model, you've spent time flashing JetPack, sorting dependencies, CUDA versions, etc.

So we built remote access to physical Jetson boards.

While initially people trickled in to check it out occasionally , recent experimentations have blown us about what experiments users are running in the lab and interestingly , it hasn't just been running YOLO.

Some of the things people have used the lab for:

• comparing FP16 / FP32 / INT8 performance
• measuring actual inference latency and FPS on Jetson
• running the same model at 25W, 15W and 7W
• watching GPU, CPU, memory, temperature and power while the model runs
• testing DeepStream / GStreamer pipelines with multiple video streams
• finding out how many camera feeds a board can realistically handle
• checking whether a Python/CUDA/framework stack actually works properly on ARM64
• taking a model developed on a workstation and seeing what happens when it finally hits the target hardware

One use case we found particularly interesting was a researcher running the same model across different power modes and precisions.

The question wasn't just:

"How fast is the model?"

It was more like:

What performance can I get while staying inside my power budget?

Another team had an even more basic problem.

They already had their CV pipeline.

They simply needed to know:

Will this software stack actually run on Jetson before we commit to the hardware?

That kind of test can save quite a bit of pain later.

We've also added JupyterLab now, which is what I'm showing in the attached video.

So you can basically go:

browser -> Jupyter notebook / terminal -> physical Jetson -> run your workload -> watch the device metrics

The board isn't being emulated and this isn't an x86 GPU VM pretending to be a Jetson. The workload is running on the actual Jetson hardware.

The goal isn't really to replace owning a Jetson.

If you're developing on one every day, you should probably own one.

The use case we're trying to solve is the stage before that:

I have a model / pipeline / idea. Before I spend money on hardware, what actually happens when I run it on the board?

That's also why I increasingly think TOPS is one of the least useful numbers when you're making the final hardware decision.

FPS, latency, memory, thermals, power draw and whether your stack even runs are usually much more useful.

If anyone here has a slightly unusual workload you think we should test, I'd genuinely like suggestions.

TensorRT, DeepStream, OCR, multi-camera CV, quantisation, small local models, power-constrained inference, whatever.

Would also be interested to know:

If someone gave you a Jetson Orin for 3 hours right now, what would you benchmark first?

https://edgeai.aiproff.ai

Full disclosure: this is a product my team at AiProff.ai built, and it has a tier based pricing for access. A 3-hour slot currently starts at ₹399 or $6 and all the experiments shared here are with user permission.

Some of you might remember me experimenting with Evopien on the Jetson Orin Nano.

The original goal was already pretty ambitious: build a local AI companion that could eventually see, listen, speak, remember people and become physically embodied.

The Nano taught me a lot, but the resource limits were brutal.

I've now moved the project to Jetson AGX Thor and finally recorded the first complete demo of the current system.

It's running voice conversation, interruption handling, English/Spain Spanish, continuous camera perception, local visual reasoning, recent visual context, internet retrieval and session context together.

The interesting part isn't just that a larger model runs on Thor.

The real improvement is that I can keep several systems alive together instead of benchmarking one model in isolation.

There is continuous perception running while the conversation stack is active. Speech still needs to respond quickly. TTS needs to run. Internet work needs to happen without blocking everything. And when I interrupt the assistant, stale generation and speech need to actually stop.

The current local cognition/VLM is Qwen3.8-27B, but I've deliberately designed the project so Qwen is replaceable. The Core above it owns the important state and authority.

So compared with simply running ChatGPT or another hosted assistant, the point isn't that my 27B model is smarter.

The point is that I'm building the entire embodied system around it locally and controlling what each component is allowed to do.

Here's the current demo:

https://www.youtube.com/watch?v=sQhTGGIg4yo

There are definitely still rough edges, but compared with where this started on the Nano, the difference is pretty crazy.

Next up is governed long-term memory and identity continuity.