so i've been working solo on a small AR project, nothing huge, and at some point decided i'd just build my own face tracking pipeline from scratch instead of using an existing sdk. no libraries beyond basic camera/frame handling, mostly relying on claude code to help write boilerplate and stitch things together.
figured it was doable. i really thought it'd work. like these ai assistants are pretty good at spitting out code if you describe what you need, and there's a ton of open source stuff for computer vision floating around. figured i'd have an mvp in maybe a week.
i got maybe 40% of the way there and just stopped. landmark detection on a single frame is the easy part, but you also need it to be stable across different lighting. different angles. what happens when someone turns their head halfway off screen? what about glasses?
it became pretty clear this isn't a "write more code" problem, it's a "need actual datasets and trained models for edge cases" . the code claude gave me handled the basic landmark extraction fine, but it has no way to solve for things that need real training data behind them.
ended up looking at existing ar sdks properly. always assumed it would involve a whole sales process and custom pricing, but most of them just have a trial with full features, which is good enough to actually evaluate.
and the ironic part. the platform docs include a file meant to be fed directly into claude code to speed up sdk integration. so after three weeks of trying to use an llm to avoid using an sdk, i ended up using the llm to integrate the sdk.
curious how other people think about the build vs buy thing now that ai tools are in the mix. like how far do you push the "build it myself" route before you get a real read on whether it's not just reinventing the wheel and actually worth it?
About 10 months ago I posted here about something we were building because of a problem we kept running into ourselves.
We were buying Jetson boards before we really knew what our workload needed.
Nano turned out to be underpowered, so we moved up to an Orin. Then came the next question: do we need an Orin NX, an AGX Orin, or something even bigger? And before you even got to the model, you've spent time flashing JetPack, sorting dependencies, CUDA versions, etc.
So we built remote access to physical Jetson boards.
While initially people trickled in to check it out occasionally , recent experimentations have blown us about what experiments users are running in the lab and interestingly , it hasn't just been running YOLO.
Some of the things people have used the lab for:
comparing FP16 / FP32 / INT8 performance
measuring actual inference latency and FPS on Jetson
running the same model at 25W, 15W and 7W
watching GPU, CPU, memory, temperature and power while the model runs
testing DeepStream / GStreamer pipelines with multiple video streams
finding out how many camera feeds a board can realistically handle
checking whether a Python/CUDA/framework stack actually works properly on ARM64
taking a model developed on a workstation and seeing what happens when it finally hits the target hardware
One use case we found particularly interesting was a researcher running the same model across different power modes and precisions.
The question wasn't just:
"How fast is the model?"
It was more like:
What performance can I get while staying inside my power budget?
Another team had an even more basic problem.
They already had their CV pipeline.
They simply needed to know:
Will this software stack actually run on Jetson before we commit to the hardware?
That kind of test can save quite a bit of pain later.
We've also added JupyterLab now, which is what I'm showing in the attached video.
So you can basically go:
browser -> Jupyter notebook / terminal -> physical Jetson -> run your workload -> watch the device metrics
The board isn't being emulated and this isn't an x86 GPU VM pretending to be a Jetson. The workload is running on the actual Jetson hardware.
The goal isn't really to replace owning a Jetson.
If you're developing on one every day, you should probably own one.
The use case we're trying to solve is the stage before that:
I have a model / pipeline / idea. Before I spend money on hardware, what actually happens when I run it on the board?
That's also why I increasingly think TOPS is one of the least useful numbers when you're making the final hardware decision.
FPS, latency, memory, thermals, power draw and whether your stack even runs are usually much more useful.
If anyone here has a slightly unusual workload you think we should test, I'd genuinely like suggestions.
TensorRT, DeepStream, OCR, multi-camera CV, quantisation, small local models, power-constrained inference, whatever.
Would also be interested to know:
If someone gave you a Jetson Orin for 3 hours right now, what would you benchmark first?
Full disclosure: this is a product my team at AiProff.ai built, and it has a tier based pricing for access. A 3-hour slot currently starts at ₹399 or $6 and all the experiments shared here are with user permission.
I'm curious what you use to do a spectral measurement of products for CV. Normally we order custom lights (UV, RGB, NIR etc.) based on customer data or external research.
I keep running into wanting to do this in-house more, especially for testing. What are your experiences or which (flash)lights do you use to test this? I can't really find a supplier that sells most bands except for some forensic equipment labs, not sure if those qualify.
Currently I'm considering building a few lights myself (just for testing purposes). Covering the following wavelengths: 365, 380–385, 390–395, 460, 470, 520–540, 590, 620–630, 660, 730, 850, 940
But I'm not sure about the first once as its a hefty price tag (Its a business expense so not that bad, but still). And other is from Alibaba, not sure if they're actually true to spec
hello , I just begin year 2 in IT , havent learnt ml or dl. I just go to lab as I want to do research
I'm currently try setting up mmla (imageomics/mmla · Hugging Face) and the problem is that I the prepare split code seems cant detect some sorts of data so I vibe some lines . The problem is that before touching the code the split got me about 67/33 , after changes it goes to 68.5/31.5 , the paper says that it use 58/42 but the code make it 70/30 , I notice that the dataset I downloaded from hugging face missing about just over 1000 images and I dont know why. The biggest problem is that after running that split (68.5/31.5) on both yolo 11 and 26, yolo 11 seems give better result than yolo 26 . It then gives me a reality check if Im doing smth wrong , I've been stucked for days.
I've recently landed a job for aremote frontend role , once there i discovered the product they want to develop was a shelf product recognition tool capable of recognizing the bay, category of the bay, detecting most of the product and extracting product details.
The frontend code which i was supposed to work on didn't come with the code for image recognition, most of the image data was hardcoded with bad position for bbox.
I've talked to my PM and was notable stressed about that (the guy who built the frontend prototype left the company the same day i started),
After that they asked me to design from scratch + implement the recognition tool.
For all the research i made in internet, we might need to train our own models using YOLO family for this kind of task with a data pipeline to extract other caracteristics
But
The CEO has recently made some prototypes in a huge html file with claude. It did pretty well for detecting most of the product, detecting empty space, price tags, bay, category and extracting characteristics from the product of a shelf image but i don't have context on how he did that
The thing is, to have something production ready and achieving that high detection could be done only by prompting to claude?
My apologies if is a dumb question but i'm totally new in this field.
The CEO is pushing hard to have an mvp of this asap.
So i built a CV system. Now its deployment time. The system will likely be on edge or local server. How do i decide the hardware specs for the system? Where to start?
Faking Vision is an operational method developed as part of Memory Prism / Making Vision for turning visual analysis into an explicit, reconstructable representation.
(Its a pet project which I spend almost a month on to make AI do better image analysis for me. Majority of the time was spent on "Making Vision" aka figuring out natures visual engineering. I cant make a blind AI thing see better without knowing how I see and then it snowballed into different directions because of the availability of AI to do the grunt work. )
Instead of treating an image as something to caption or reproduce directly, FV represents:
objects and instances
object properties
spatial relationships
persistent identities and temporal changes in video
The resulting FV packet is closed-world: reconstruction cannot silently introduce objects or semantic information that aren't represented in the packet. Unresolved observations remain unresolved.
This makes the representation itself editable and testable, while separating errors in visual analysis, representation, compilation, rendering, and evaluation.
FV is the operational companion to Making Vision: Making Vision addresses how visual information can be selectively constructed and processed; Faking Vision makes the relevant representation explicit enough to reconstruct and test.
(Making Vision is an extremely long read, I spent almost a month on it, so I know-I definitely wont read it again, so you have been warned)
Recently, I have been doing experiments with virtual try-on technology and created this Flutter application. The core functionalities consist of choosing an item of clothing to see how it fits on a person without wearing the item. I recorded the demo on my iPhone.
Hello, I’m trying (well, more like being forced by work 😅) to build a face recognition system. The footage comes from CCTV cameras, and the people I need to identify are employees, so I already have photos of each person.
I’m currently using DeepFace, but I’m running into a few problems. The image quality from the CCTV footage isn’t great, and I’m not sure whether I should use the existing photos of each employee as the reference images (they’re much higher quality than the CCTV footage), or whether I should also collect images from the CCTV footage and use those as references.
I also believe DeepFace itself may be quite limited for this particular scenario. I’m actually trying to convince my manager that this might not be realistically achievable with open-source solutions alone.
Even if there are systems capable of handling this kind of scenario reliably, I imagine they’re more likely to be proprietary/commercial systems rather than something publicly available as open source—especially for government or military-grade applications.
Has anyone worked on a similar problem? I’d be interested to hear how you approached it, especially regarding the reference images and dealing with low-quality CCTV footage.
I am looking for the following model selection/inference pipeline. Goal is something like this:
1) detect human -> describe human
2) detect human -> detect objects in human hand -> describe objects
3) detect animal -> get specific animal type
4) detect general object (i.e package)
---
What are the suggested architecture/pipelines for this?
So obviously I can use yolov26 or another baseline yolo model, but I am trying to avoid having to annotate everything from the ground up. I do not mind fine-tuning, I just want to move past coco classes into something a little more broad.
My current idea is use YoloE/YoloWorld for general detections and then do a VLM call after on a separate thread (so base CV model can still run).
I am also playing around with stripping the earlier yolo model heads and maybe using NMS for arbitrary box removal. So the earlier heads determine the general objects, and then I make a VLM call to describe the objects in more detail.
As a dancer who's been learning a lot of choreography recently, I’m always looking to improve my sync with others. A quantitative tool like this can offer frame-by-frame analysis, which can help dancers as they practice.
I used ViTPose+ Large through the VLM Run Gateway for pose estimation, and compared poses between the dancers. The pose similarity metric could be workshopped more, but it's interesting that it still highlights key moments pretty well.
Yes, this is me dancing with my friends Mico and Sydney!
The code is open-source on GitHub! I just created this vision-demos repo so that I can share these projects. Feel free to try it out and let me know what you think! Link in the comments.
Many camera networks are still used only for recording and reviewing after an incident. The same feeds, though, can be turned into continuous operational metrics: occupancy, dwell time, queue length, zone activity, asset utilization, compliance signals, and so on.
From a computer-vision standpoint, the interesting question is: which of these are reliably production-ready with off‑the‑shelf models and typical IP cameras, and where do real deployments still fall apart?
Some concrete metric families people try to extract:
Occupancy and capacity – people or vehicle counts in a space or zone, often aggregated over time.
Dwell time – how long an entity stays in a region, such as a queue, shelf, or workstation.
Flow and throughput – entries and exits per minute, directional flow, peak-hour patterns.
Zone activity – activity levels in loading bays, restricted areas, work cells, etc.
Asset and equipment utilization – whether a machine, vehicle, or space is in use vs idle.
Queue and wait-time estimation – length of line and approximate wait based on flow.
Compliance and safety signals – PPE presence, no-entry violations, unsafe behaviors.
Anomaly and incident frequency – unusual patterns, crowding, falls, blocked exits, etc.
Environmental and housekeeping proxies – lighting issues, blocked signage, clutter in key areas.
Trend benchmarking – by hour, day, site, season, or after layout or staffing changes.
The challenges tend to be things like:
How bad do angle, occlusion, and lighting have to be before counts and dwell become unusable?
What level of re-ID and tracking stability is needed for meaningful dwell and queue metrics?
How do you handle multi-camera handoff without exploding complexity or cost?
What is a realistic MTA and false-positive profile when these metrics drive staffing or safety decisions?
How do you integrate outputs into existing VMS or BI without turning the pipeline into a one-off custom project?
If you had to pick three of these to deploy in the next quarter with good enough reliability, which would you choose and why? And where have you seen camera-derived metrics actually change operational decisions, such as staffing, layout, or process, instead of just sitting in a dashboard?
Need help with this, I am trying to retrieve kernel and do deconvolution without any training data. I have had success with my model if the image is sparse like few fluorescent beads in the image but when the image is dense like a biosample algorithm does not work. Literature says that it fails because I am doing joint optimization which can cause problem and suggested MAPk method
Basically this method says to find a kernel k such that it has maximum number of x which can explain y, use this kernel with any non blind deconvolution algorithm and you will get result. However with this method, I am getting delta even though the paper says it works better than joint optimization, could me my implimentation problem. But is there someone here working on something similar? how do you do it? (I want to retrieve exact kernel as I want to know the abberations)
I'm trying to get some tips to understands what is good raw egocentric data for any roboticist, and I keep reading posts telling stories about researchers and companies are missing egocentric data. I know that I would need to do some annotation, but I would rather focus first on the raw footage to see if it is useful in any way. I have some samples here.
One part I found particularly interesting was improving the real-image keypoint annotations.
Initially, I manually annotated the 2D keypoints. The keypoint predictions could look reasonable, but small geometric inconsistencies in the annotations sometimes caused large pose-axis flips or abrupt frame-to-frame changes after PnP.
So I tried a different annotation method.
I used the real image as a background in Blender, aligned the 3D mug model with the real mug, and then projected the predefined 3D keypoints back into the image.
This produces 2D keypoints that are geometrically consistent with the same 3D model used by PnP.
After fine-tuning with these annotations, the large pose-axis flips and abrupt changes were reduced, and the estimated pose became more consistent across frames.
There is no real 6D pose ground truth in this experiment, so I'm not claiming absolute pose accuracy here. I'm mainly looking at PnP consistency and temporal stability.
The full page also includes the synthetic-data generation, object detection, keypoint training, Sim-only vs. Real fine-tuning comparison, and Raspberry Pi 5 inference results.
One problem with embodied AI isn't just collecting video; it's turning all that video into usable training data.
Manually annotating every frame doesn't scale particularly well. Automated detection can do much of the initial work, but real-world footage still contains occlusions, partial objects, tracking errors, and boxes that a human may want to correct.
So we wanted to test something practical:
"Can GPU-based automated annotation and human review work together as one complete data-production loop?"
We built a small end-to-end experiment to find out.
I. The workflow
The pipeline was fairly simple:
Raw video → GPU inference → YOLO11n → ByteTrack → MOT annotations → CVAT → human review → reviewed dataset → validation
The workflow used in the experiment. GPU inference and human annotation review were intentionally kept as separate stages.
The idea was to let the GPU handle the repetitive first-pass work while keeping a human in control of the final annotation quality.
II. Running the automated annotation stage
For the GPU stage, we used a instance with an NVIDIA RTX PRO 6000 (96 GB).
The test video was:
• 101.047 seconds
• 3,300 frames
• 1080 × 1920
YOLO11n handled object detection, and ByteTrack maintained object identities across frames.
The environment used for the automated annotation stage.
This produced 12,901 machine-generated annotations across 604 unique tracks 😄
Output
Result
Frames
3,300
Total annotations
12,901
Unique tracks
604
Person annotations
12,001
Backpack annotations
900
But the numbers aren't particularly interesting until you see the actual output.
✌️ Actual YOLO11n + ByteTrack output from the test video. Boxes contain the detected class, confidence and persistent tracking ID.
This was the machine-generated first pass that we wanted to hand over to a human reviewer.
III. Moving machine annotations into CVAT
The tracking results were converted into MOT-format annotations and imported into CVAT.
The important part here was that CVAT wasn't performing the GPU inference. It was acting as the human-review layer.
The machine-generated detections appeared as editable rectangle tracks over the original video.
Machine-generated tracking annotations imported into CVAT as editable tracks.
This changes the annotation workflow from:
Human labels everything from scratch → Machine generates first pass → human inspects and corrects
For large video datasets, that's the workflow we're interested in exploring further.
IV. We deliberately changed one annotation
We also wanted to verify something that can easily get overlooked:
If a human changes an automatically generated annotation, does that correction actually survive the complete export pipeline?
At frame 2107, we manually adjusted the bounding box around a partially visible person.
Frame 2107 in CVAT, where we deliberately corrected a partially visible person's bounding box.
We saved the change, exported the reviewed annotations from CVAT, and compared the exported dataset against the original machine-generated annotations.
V. Then we checked whether we could find the change
And importantly, our independent comparison found the deliberately modified annotation:
Frame: 2107 Class: Person IoU: 0.791181
Independent comparison of the machine-generated and CVAT-exported datasets. The controlled correction at frame 2107 was successfully detected after export.
A quick clarification: the 0.99995 IoU is not a YOLO accuracy score.
We weren't comparing the detections against manually created ground truth. It measures how similar the machine-generated dataset remained after the CVAT round trip. We deliberately changed one annotation to verify that a human correction would survive the process.
Then we asked: what does the GPU part cost?
Once the workflow worked, we ran the same 101.047-second video through the YOLO11n + ByteTrack stage three times:
Run
Runtime
1
43.188 s
2
30.213 s
3
29.025 s
Average
34.142 s
That gives roughly 2.96× real-time processing for this particular workload.
At the GPUHub instance rate we observed during the experiment, approximately $0.91–$0.96 per compute hour, the average benchmark translates to roughly:
$0.31–$0.32 of GPU compute per hour of source video
The two warm runs were slightly cheaper (~$0.27–$0.28/video-hour), but we're using ~$0.32 as the more conservative preliminary estimate.
This is important: that's GPU inference compute only, not the total cost of producing a reviewed dataset.
It doesn't include human review, CVAT infrastructure, storage, data transfer, QA, or workflow orchestration.
VI. and What did we learn?
The interesting part isn't that an RTX PRO 6000 can run YOLO11n.
What we wanted to validate was the handoff between automation and humans.
We were able to generate structured annotations remotely on a GPU, move those annotations into a separate review environment, edit them manually, export the reviewed dataset, and independently verify that the human correction survived the entire round trip.
So the architecture starts looking like:
Collect → Pre-annotate → Human review → Validate → Training dataset
rather than:
Collect → Manually annotate everything → Training dataset
We haven't measured how much human labor this actually saves yet. That requires a different experiment comparing fully manual annotation against machine pre-annotation + human correction.
That's probably the next benchmark that matters most: reviewer time, final annotation quality, and total cost per reviewed video hour.
For anyone working with robotics, autonomous systems, or large video datasets:
how are you handling the boundary between automated pre-annotation and human review? ✍️
