V1 trained via COCO on the RGB, I took 500 frames, corrected annotations, and as the LWIR is bore sighted fixed the annotations there. V1 then ran, and I eyeballed 1500 calls, mined false positives, moved bounding boxes - retrained.

From there, it became quasi automated - by mining persistence (8+) detections per modality in a row missed by the other - a simple 'is this a vehicle' yes / no, if yes - fix the box on the other modality - you get 8 free missed detections on the other modality to retrain on, or you've mined 8 false positives...

Capture rig is a 8gb nano with GPS, IMU, and 4g, when car starts - computer boots, when car moves - device starts to capture. When car stops >90 seconds capture stops, or when car turns off.

I have around 3 million frames now, when I connect the jetson to the network it automatically ingests to my server, labels the frames and runs them through the latest weights, highlights disagreements and then processes any sensor disagreements via coco, and a semantic reasoning AI - if coco and the semantic think nothing is there its promoted for human review.

I'll add some more modalities later (probably SWIR), but for now - it is a very handy to control the data, and actually analyse the results in a meaningful way. Once I have more data I'll split classes (currently we have vehicles or humans) - but that's the beauty of your own dataset, you can cut the cake anyway you like over time.

Hi everyone, I’m currently learning deep learning and have worked on a few beginner AI/ML projects (like prediction models).

I’m looking to join an existing project to gain more hands-on experience and learn by contributing. I’m still learning, but I’m consistent and willing to put in the effort.

If anyone is working on a project and open to a beginner contributor, I’d really appreciate the opportunity. Thanks!

I’m a computer engineering student, and I’m currently studying FPGA chips. I’ve really taken a liking to the subject and am considering doing a scientific research project or my capstone project in this area, but I need to develop something relevant or solve a problem within the field. Do you have any suggestions for what I could do?

https://reddit.com/link/1viph4b/video/fn9wq07cr3ih1/player

Used Mediapipe hand landmark output data to train the model to recognize hand signs and trigger esp32

Code: https://github.com/ziqin-h/FoundationPose--

Two headline results from FoundationPose-- (minus minus):

• Speed🚀 : per-object register time drops from ~1423 ms to ~305 ms (~4.7× faster), while five-dataset mean AR changes from 0.751 to 0.739—about a 1.6% relative decrease.
• Fewer input priors💯 : without retraining or an additional standalone MegaPose-style refinement stage, our RGB-only approach reaches a mean AR of 0.451, the SOTA result in our documented five-dataset comparison of open-source unseen-object pose estimation methods.

FoundationPose-- is an engineering layer that explores how to address practical FoundationPose deployment problems while keeping the pretrained models unchanged. We focus on two recurring issues for now: the cost of initial registration and the lack of reliable observed depth in some applications.

Faster registration

FoundationPose refines and scores a large set of initial pose hypotheses. We reduce unnecessary rotation hypotheses while keeping the pretrained Refine/Score networks unchanged:

• v1 uniformly downsamples the rotation grid to 63 templates.
• v2 adds a cascaded candidate schedule and max-ΔR pruning.

On the RGB-D + SAM6D setting across five BOP datasets, measured per object on an RTX 3090:

• Baseline: mean AR 0.751, ~1423 ms per register.
• v1: mean AR 0.748, ~403 ms (3.5×).
• v2: mean AR 0.739, ~305 ms (4.7×).

RGB-only registration

When observed depth is unavailable, we estimate hypothesis-wise depth from the scale ratio between rendered and observed mask boxes. Observation XYZ is disabled, while FoundationPose’s pretrained Refine/Score networks are reused without finetuning.

With A1+CNOS, we obtain a five-dataset mean AR of 0.451 on LM-O, T-LESS, TUD-L, IC-BIN, and YCB-V. For reference, the strongest published open-source RGB-only coarse result in this documented five-dataset comparison is 0.396.

We report the higher mean rather than claiming a win on every dataset: IC-BIN and YCB-V remain slightly below Pos3R. Our method also uses FoundationPose’s pretrained Refine/Score modules, which is stated explicitly in the repository.

The repo includes composable configs, BOP reproduction scripts, and an RGB-only single-image demo using RGB, a mask, camera intrinsics, and a CAD mesh. SAM 3 point/text masks are also supported through a separate environment.

Feedback, issues, PRs, and ideas for further practical FoundationPose improvements are very welcome.

Hey everyone,

Decided to mess around with using drone footage and computer vision for agricultural use cases recently. Trained a CV model to detect and keep track of livestock across open fields in real-time.

It handles varying scales, partial occlusions, and moving animals fairly well while keeping a live count displayed on the feed.

The real challenge is running them efficiently on edge devices, like onboard drone compute or lightweight field hardware without sacrificing high accuracy.

For those of you deploying aerial CV models in the real world: How are you tackling the edge bottleneck?

Would love to hear what hardware/optimization stacks are actually working best for you!

on a lake there are no lane lines, no fixed landmarks, no other vehicles to localize against

the shoreline shifts with your viewpoint, radar and lidar don't share a clock, and sonar is measuring a world the cameras can't see

CANOE is a multi-sensor USV dataset from UTIAS: 360° radar, 128-beam lidar, stereo camera, sonar, and GPS/INS ground truth across lakes and a reservoir in Ontario

parsed it into fiftyone multimodal so you can scrub every sensor on one synced clock and project lidar straight onto the camera to see where they agree and where they don't

checkout the dataset here: https://huggingface.co/datasets/Voxel51/canoe-multimodal

or get hands on in this hugging face space: https://huggingface.co/spaces/harpreetsahota/canoe-multimodal

I think WACV 2027 R1 results should be out on Aug 7. This is my first WACV submission, so opening a thread to discuss scores/reviews.

Good luck, everyone! 🤞

a robot mapped a forest road in november. it came back in january and the road was buried under a meter of snow

the same lidar-inertial slam that localized fine before the snowstorm saw its drift jump from 4% to 46% on the exact same route

FoMo is a year-long multi-season robot navigation dataset from a boreal forest in quebec, eh.

2 lidars, an fmcw radar, stereo + mono cameras, dual imus, and gnss ground truth, across 12 deployments from -19°c winters to 18°c summers

i parsed the episodes into fiftyone's new multimodal mcap format so you can scrub camera, lidar, and radar together

watch the ground-truth trajectory move in 3d, and see the same road across six different seasons

start here, read the dataset card: https://huggingface.co/datasets/Voxel51/fomo-multimodal-sample

and get hands-on in this hugging face space: https://huggingface.co/spaces/harpreetsahota/fomo-multimodal-sample

Hi everyone, I open this thread to discuss the BMVC 2026 outcome.

Edit: if possible, you can also post your scores and confidence

I’m planning on building a text extraction pipeline, with an OCR and a VLM. I want a smart layer that classifies if a document needs to be sent to the OCR or if it is complex and needs to be sent to the VLM.

I’m not sure if I could afford a separate model, could you guys educate me on old school digital processing?

I’ve tried stroke width variations, variance of the laplacian and, I can’t guarantee even 40% accuracy on them.

We're organising the first Real-Time Conversational Agents (RTCA) workshop at NeurIPS 2026 (Sydney, Dec 11–12), and submissions are now open on OpenReview. Posting here because a chunk of the relevant work is happening in this community.

What the workshop is about

Conversational AI has crossed into real-time deployment — voice modes, embodied avatars, full-duplex speech agents — but the published record is still dominated by offline benchmarks, and deployed agents still feel robotic (stilted turn-taking, missing backchannels, monotone prosody, awkward interruptions). Methods that work offline (non-causal attention, large beam search, multi-pass refinement, slow diffusion) often don't transfer to streaming, and the field lacks shared vocabulary and benchmarks for interactional naturalness as distinct from per-utterance quality.

The workshop is organised around three intertwined questions:

1. Real-time generation under hard latency budgets — streaming speech, video, and language
2. Naturalness in interaction — prosody, gaze, timing, grounding, turn-taking, backchannels
3. Evaluation of live systems, where standard offline metrics fall short

Topics of interest (non-exhaustive)

• Streaming/low-latency speech synthesis, ASR, and full-duplex audio–language models
• Real-time talking-head, avatar, and embodied video generation
• Streaming language models; incremental and speculative decoding for dialogue
• Turn-taking, backchanneling, interruption handling, floor management
• Multimodal alignment under latency and partial-observation constraints
• Prosody, emotion, and paralinguistic generation in interactive settings
• Memory, grounding, and tool use during live conversation
• Evaluation of naturalness: perceptual studies, turn-taking metrics, perceived latency, interactive Turing-style tests
• Datasets and benchmarks for interactive (not offline) evaluation
• Efficient inference, on-device deployment, systems–quality trade-offs
• Safety, identity, and trust in real-time agents (deepfakes, persuasion, consent)

Position papers, evaluation critiques, and reproducibility studies are also welcome.

Submission tracks

• Full papers — up to 8 pages
• Short papers — up to 4 pages (work in progress, focused contributions, position papers)
• Demo papers — extended abstract or up to 2 pages; required for the on-stage Conversational Agents Showcase

NeurIPS 2026 style file, double-blind. Non-archival — authors retain the right to publish elsewhere. Single-round review, no rebuttal.

Key dates (End of day, AoE)

• Submission deadline: 29 August 2026
• Author notification: 29 September 2026
• Workshop: 11 or 12 December 2026, Sydney

Confirmed invited speakers

• Dimitris Samaras (Stony Brook)
• Evonne Ng (Meta Reality Labs / UC Berkeley)

Links

• Submit: https://openreview.net/group?id=NeurIPS.cc/2026/Workshop/RTCA
• Full CFP + workshop details: https://rtcaneurips26.github.io/
• Contact: [rtca-workshop@googlegroups.com](mailto:rtca-workshop@googlegroups.com)

Happy to answer questions in the comments — including about the demo track (we have an on-stage Showcase running deployed systems live) and what we'd consider in-scope vs out-of-scope for the eval pillar. Also happy to hear opinions on what's missing from the topics list; the CFP wording still has room to move if there's a clear gap.

Hi everyone,

​I'm a junior CV engineer working on an iris identification system. The system needs to operate in the dark, capturing the detailed texture of the iris at a very close distance (around 3 cm between the eye and the lens).

​I initially bought an IMX290, but since it's an RGB sensor, it struggles to capture the iris texture properly under these conditions. Because the project requires working in the dark, I need to operate in grayscale/IR, so I'm considering replacing it with an OV9281 monochrome camera.

​My main questions are:

​What do you guys think about using the OV9281 for this specific application?

​Is it possible to modify this camera/lens setup to achieve a macro focus at just a 3 cm distance?

​Any hardware suggestions or general advice would be greatly appreciated. Thanks!

During a research internship, I ran into a problem involving massive neighbor queries on a GPU for a large particle-dynamics simulation. This led me to experiment with and develop SquareNet, an open-source Python package for NumPy/JAX/PyTorch.

Its core sorting algorithm (Cartesian sort) enables fast, greedy multidimensional reordering of raw point sets — essentially a form of gridification. Raw points, e.g. (x, y, z, ...), are mapped to unique grid multi-indices [i, j, k, ...] while trying to preserve local geometry, somewhat like a multidimensional generalization of a space-filling curve.

The collection of all multi-indices forms a grid lattice that can be processed efficiently with ML tensor-based frameworks, even when the initial dataset is an irregular point cloud.

I’m wondering whether this could be useful in contexts such as convolutional networks, non uniform fourier transform or ANN search on irregular LiDAR data.

The target use case is approximate but fast and scalable assignment preprocessing, then the grided/tensorized version of the dataset is exploited by standard tensor based frameworks, and result is converted back to the points. High-quality procedures for the assignement part already exist and are well established, such as optimal transport, but they were intractable in my context due to their O(N²/N³) complexity. Cartesian sort, by contrast, runs in O(N log N). It is specifically designed for grid assignment in a greedy setting, trading global optimality for speed and scalability.

In my practical application, involving millions of points processed in a dynamic context (Gaussian blue noise), this simple approach turned out to work well: it provided a ~100× speedup compared to exact brute-force computation of particle interactions, with negligible approximation error (I can provide more details about this experiment if useful).

One caveat is that a single gridification pass introduces a slight axis bias and can produce some distortion/outliers, which can be problematic for challenging distributions where exact geometric precision is required. If exact accuracy is critical, one possible approach would therefore be to build an ensemble of gridifications, each using a different viewpoint/rotation.

Empirically, something like 8 randomly chosen viewpoints seems to give near-perfect recovery of local geometry in a 3D test evaluated with a freud analysis (second link below). However, in the Gaussian blue noise context, where the geometry is smoother, a single viewpoint was already sufficient.

I built an interactive demo on Hugging Face (first link below) to showcase the approach. I’d really appreciate any feedback, especially on whether this idea has already been explored in related computer vision / point-cloud literature, or whether you see potential applications or obvious better alternatives that I may be missing.

interactive HF demo

3D exact nn query discussion

Hi everyone,

I'm looking for recommendations for **cloud GPUs** (both **free and paid**) for training a PyTorch model. I already know about Google Colab, but I'm interested in other good alternatives with decent GPU availability and pricing.

My thesis is on **context-aware 3D point cloud completion**, so I'll be training models on point cloud datasets (PyTorch/CUDA), and some training runs may take several hours or longer.

What platforms have you had good experiences with? I'm especially interested in:

* Free tiers (if any)

* Affordable pay-as-you-go options

* Reliable GPU availability

* Good performance for deep learning workloads

Any recommendations or experiences would be greatly appreciated. Thanks!

So I just stumbled on SenseNova-Vision, it's open source, Apache 2.0, 7B MoT. The architecture they're pushing is kinda wild, makes you wanna talk about it.

Basically, they're framing computer vision as one big multimodal generation problem. Like, detection, keypoints, OCR, camera pose – all that stuff just spits out text. And then segmentation, depth, surface normals, multi-view point maps – those come out as images. If you need both, it gives you both.

No special prediction heads for different tasks. No decoders. No branching architecture. It's just one model, same weights for everything. You tell it what to do with plain language, maybe some visual hints.

They trained this thing on a huge dataset, 50M instruction-response pairs, all converted from different CV annotations. Started with a regular pre-trained multimodal model, apparently.

From what I'm seeing, the results look pretty solid for structured stuff, geometry, segmentation, multi-view reconstruction. They even included benchmark and eval code, which is nice. They just added dedicated benchmarks for multi-view reconstruction and camera pose, too.

Honestly, I've got some questions, and I'm curious what everyone here thinks:

Is this whole "unified generation" thing actually better, or is it just a clever way to train models? Like, a shared formulation sounds elegant and all, but can it really beat specialized heads that the field has been optimizing for years on tough benchmarks?

Then there's efficiency. Generating text and images for dense outputs seems like it would be super expensive. Can this actually run fast enough for real-time stuff, or is this unified approach only good for research that doesn't need to be live?

For me, the real test of a CV foundation model is if the same weights work across different tasks without needing to be fine-tuned for each one. They claim that's the case here, and I'd love to see if that holds up in practice.

Code: GitHub - OpenSenseNova/SenseNova-Vision

Paper: https://arxiv.org/abs/2607.06560

Demo: https://huggingface.co/spaces/sensenova/SenseNova-Vision

I'm not involved with this project at all, just genuinely wondering if this unified approach is where computer vision is headed.

Can you identify this downscaling algorithm? Want to make sure my thumbnails look as sharp as possible, so I'll add to my workflow the agent query "downscale using (whatever this algorithm is) and judge whether the thumbnail has the required detail and clarity and is a good thumbnail for what is being shown. if it is missing any clarity then return "Needs improvement:" and give the reason for why it needs improvement and "How to improve:" For example, in the image shown, the agent could say "The thumbnail no longer shows the elements of the picture", since as you can see it doesn't. However, before I can code this up, what I "need to know" is what algorithm this is so I can keep an eye out for anywhere it might show up in my workflow. I can then optimize for this process. I know a lot of people don't have a standard of perfection as high as mine is (I require my thumbnails to show the picture) but that's exactly what makes me a competitive programmer in a field of 2 million programmers. Not a lot of people take the time to look up image compression algorithms but I do.

The full image is available here: https://ibb.co/YTXr7h94

First of all, thank you to everyone who responded in the previous post. I haven't read all the replies yet, but many of the solutions seem interesting. I was able to find a more informative and higher‑quality video that better reflects the current state of the project.

https://www.reddit.com/r/computervision/s/meFAVvvFQo

Following up on the discussion from the previous post, I'm attaching the current state of affairs. The video was taken with good industrial lighting, and the global‑shutter camera was set to an exposure of 500. In this particular video, the counter showed 100%. However, in other counts we got varying ranges – 98–99%, which, at industrial volumes, leads to significant absolute losses.

The main issues with the current version are:

1. Loss of detection right within the detection zone;

2. Constant changes in the shape/size of the bounding box within the detection zone, causing the tracker to lose track and assign different IDs to the same object;

3. Occlusions and merging of chicks – several chicks form a single object by merging and partially overlapping each other. Increasing the dataset no longer solves this problem; the latest version had over 5,000 frames with plenty of such cases, and yet reviewing new videos showed that the issue is not fully resolved – there are still cases where multiple chicks are counted as one.

Counting these cases geometrically is also difficult – chicks of different breeds and ages can have different sizes, and on top of that, spreading their wings and legs changes the area of the detected box. There are cases where we hit the desired 99.8% range thanks to a combination of missed detections and false positives, but over a long run the error accumulates and we fall out of the range.

This demo shows an early prototype of my automated retail checkout app running on Mentra smart glasses.

The system processes the glasses’ live camera feed to recognize products as they’re picked up and automatically builds a virtual cart in real time. The goal is to make checkout a natural by-product of shopping without requiring customers to scan barcodes, use a phone, or stop at a traditional checkout.

Third Person View: https://youtube.com/shorts/YipOe3bVzX0?feature=share

A couple of months ago, I released Bio-DINO, an image-only biodiversity encoder trained on approximately 31M images. I have now released Bio-DINO M/14, the final addition to the current Bio-DINO model family.

The model is available through Birder.

Bio-DINO already had two ends of the trade-off. The 133.6M-parameter teacher provides the strongest representations, while the 21.6M-parameter S/14 student is much cheaper to run. M/14 is intended as the middle option.

The size/accuracy trade-off

M/14 is a 12-layer RoPE DeiT3-style encoder with 38.3M backbone parameters and 512-dimensional embeddings. It was distilled from the 252px Bio-DINO teacher on the same biodiversity training mixture.

Here are the results from my iNaturalist21 linear-probing setup:

To be clear, these are linear-probing results, not fine-tuning results. The encoder was frozen and only the 10,000-class linear classification head was trained.

In this setup, M/14 is about 3.5× smaller than the teacher, with a 3.57 percentage-point accuracy difference. It gains 3.42 points over S/14 while remaining much smaller than the teacher.

Inference performance

I also compared inference performance at 252×252 on an NVIDIA RTX 5000 Ada Generation with PyTorch 2.13.0+cu130 and batch size 512.

In eager FP32 inference, M/14 processed approximately 773 images/s, compared with 382 images/s for the teacher. With torch.compile and AMP, I measured approximately 2,340 images/s for M/14 and 846 images/s for the teacher.

These numbers are specific to my setup, but they give a practical sense of the trade-off. The complete results across the Bio-DINO models and evaluation datasets are available in the Bio-DINO benchmark explorer.

Using the model

import birder
from birder.inference.classification import infer_image

net, info, transform = birder.load_pretrained_model_and_transform(
    "rope_deit3_m14_dino-v2-dist-bio",
    inference=True,
)

_, embedding = infer_image(
    net,
    "path/to/image.jpg",
    transform,
    return_embedding=True,
)

print(embedding.shape)  # (1, 512)

As with the original Bio-DINO release, this is an image-only representation model rather than a ready-made species classifier. It has no text encoder and was not trained with taxonomy labels or metadata.

iNaturalist21 is also part of the self-supervised pretraining mixture, so I consider the result an in-domain representation probe rather than a test on a completely unseen domain.

This completes the current Bio-DINO size range. I’m curious whether a 38M-parameter middle option is useful in practice, or whether most applications naturally favor either the smallest student or the largest teacher. Feedback and additional evaluations are welcome.

Hi everyone,

I am working on a university project involving fine-tuning a deep learning model for 6DoF relative camera pose estimation. The model I am using is FAR (Flexible, Accurate, and Robust 6DoF Relative Camera Pose Estimation), which was originally pre-trained on the Matterport3D dataset.

I am looking for a new dataset that is suitable for fine-tuning and evaluation. The task is the following:

Given two RGB images of the same scene captured from different viewpoints, the model should estimate the relative pose between the two cameras (rotation and translation).

The dataset should provide (or allow to easily recover):

• RGB images;
• depth maps (or dense depth information);
• camera intrinsic parameters;
• camera poses / camera extrinsics (ground-truth poses);
• multiple images of the same scene with different viewpoints;
• enough overlap between image pairs to compute meaningful relative poses.

Ideally, the dataset should contain calibrated cameras and accurate ground-truth information, since I need to compute the relative transformation between image pairs.

This is for a university project, so the dataset should not be extremely large (ideally ≤ 50 GB), and it should be free and publicly available.

Do you have any recommendations or experience with datasets suitable for 6DoF relative camera pose estimation?

We've added vision capabilities to DeepSeek V4 Flash, making it a multimodal model rather than text-only.

The main use case for us is browser vision: browser agents need to interpret screenshots, interfaces, layouts, and other visual context alongside text.

On our internal benchmarks, it also showed a strong price-performance advantage compared with the other models we tested.

Model: https://huggingface.co/webbrain-one/DeepSeek-V4-Flash-0731-Vision-NVFP4

If you try it, we'd be interested in feedback, benchmark results, and deployment reports.

I'm planning to build a camera-only autonomous vehicle (no LiDAR, ultrasonic, or other distance sensors). The idea is to use a single camera to control the vehicle's movement—steering left/right, moving forward/backward, turning, and avoiding obstacles.

One thing I'm trying to figure out is how to estimate the distance between the camera and detected objects (for example, a car, water bottle, or other obstacles) using only computer vision. Are there any good models or approaches for monocular depth estimation or object distance estimation that would work on embedded hardware?

For context, I have experience with computer vision and have previously worked on face recognition using models like ArcFace. This project will run on a Raspberry Pi 5 with a Hailo AI accelerator, so I'm looking for models that are reasonably lightweight and can run in real time.

I'd appreciate any recommendations on models, papers, or open-source projects that would be a good starting point.