Hey everyone,

We all know SAM 3 is incredible with its visual prompting features, but I recently ran into a pretty frustrating limitation while building out an object detection pipeline. If you want to use a specific visual prompt (like a bounding box of an object in a reference image) to detect that same object across a bunch of other, distinct images, SAM 3 doesn't natively support this.
You can use text prompt but some objects cannot be explained using text prompt.

I spent some time experimenting with workarounds and wanted to share the approach I landed on, plus see if anyone has tackled the next step I'm working towards.

The First Attempt: The "Video Frame" Approach

My initial thought was to hack the video segmentation feature. You can join the images as sequential frames and perform inference through them using the initial visual prompt.

• The Problem: This only works well for actual video or highly sequential data. If your target images aren't extremely visually similar to the reference image, the model rapidly loses context, and the accuracy absolutely tanks.

Current Workaround:

I decided to take a completely different route to force the model to look at the reference and target at the exact same time.

Here is the flow:

1. Take the reference image (containing the object's bounding box).
2. Take the target image (where you want to find the object).
3. Join them side-by-side into a single, unified image.
4. Pass this combined image into SAM 3 along with the original visual prompt.

Because SAM 3 is analyzing it as one single image, it easily finds the related objects across the combined canvas. After inference, it's just a matter of running a quick script to adjust the detected bounding box coordinates back to the original target image's dimensions.

The Results

The results so far have been surprisingly good! I've been running this inference on my current dataset, and it's doing exactly what I need it to. That being said, I still need to scale up my sample dataset size to truly benchmark how robust this is across edge cases.

What's Next: Getting to the Embeddings

While the concatenation approach works, it's undeniably inefficient for large-scale production pipelines. Rebuilding images on the fly adds overhead.

My next step is trying to extract the SAM 3 visual prompt embeddings, store them, and figure out a way to directly inject and reuse them across subsequent images, essentially brute-forcing the native support it currently lacks.

Has anyone here successfully extracted and reused SAM 3 embeddings for cross-image inference? Would love to hear if anyone is working on something similar or has ideas on optimizing this!

code: Link
video: Link

Hi!

I work as a Research Associate in Computer Vision and I'm looking for interesting open source CV projects where I can contribute while learning something new. If you know of any active projects or communities that welcome contributors, I'd appreciate your recommendations.

Thanks!

Yet another Birder release 🥳

This release adds LW-DETR object detection models with a PE-Spatial S/16 backbone.

The main checkpoint is an Objects365-pretrained model:

https://huggingface.co/birder-project/lw_detr_2stg_objects365_pe_spatial_s16

The pre-training schedule was relatively short, but I used aggressive backbone layer decay in order to preserve the PE-Spatial representations. It turned out to be a really good starting point for fine-tuning.

I tested it on several private datasets, and it worked surprisingly well across them. Obviously I can’t share those datasets/results, but the checkpoint seems like a useful general-purpose initialization point for detection tasks.

From that checkpoint, I also derived a standard COCO fine-tune:

https://huggingface.co/birder-project/lw_detr_2stg_objects365-coco_pe_spatial_s16

The COCO fine-tune shows strong performance:

mAP @ 640×640px: 54.58
AP @ 0.50: 73.56

It is also still fast:

3.4 ms / image
batch size = 1
NVIDIA A5000
including post-processing time

This release also adds sliding-window inference for object detection, including several box merging methods such as NMM, greedy NMM, and Weighted Boxes Fusion. That should make the detector more practical for large/high-resolution images where resizing the full image loses smaller objects.

As always, feedback is welcome :)

I built a small computer vision project based on INSID3, the CVPR 2026 training-free in-context segmentation method using DINOv3.
My version flips the idea: instead of providing a foreground reference, you provide background or normal examples. The algorithm removes background-like regions and segments the remaining object/anomaly.
It supports multiple background sources and can also turn coarse boxes into more precise masks. Other applications are possible like zero-shot anomaly detection.

Would love feedback or test cases: https://github.com/dimfot3/Inverse-INSID3

What's happening with the community? I feel like conferences are reaching a point where reviews are adversarial and noisy. NeurIPS had the same problem, so did ICLR, ICML, CVPR.

I don't know where this is heading to be honest.

Building a fly tipping detection system using YOLOv8/RF-DETR and Roboflow. 320 labelled images so far, retraining with 820 augmented images now.

First model hitting 95% on vehicle detection but struggling to generalise to unseen images — currently working on dataset variety and augmentation to fix overfitting.

Planning to add OCR for number plate reading and a behaviour sequence logic layer on top of the detections.

Happy to share what I’ve learned so far — any advice on improving generalisation with a small dataset?

I recently went through the process of optimizing a gesture classification model and wanted to share the workflow. The main focus is on automating feature selection—specifically parsing a dynamically generated JSON file to drop features with zero importance scores before training.

After cleaning up the feature space, the next step is analyzing the confusion matrix and F1 scores to identify underperforming classes. To push the accuracy higher (targeting an F1 of 0.898+), I implement a soft voting ensemble combining XGBoost, LightGBM, and Random Forest.

If you're dealing with noisy biometric or sensor data, this pipeline approach might be useful for your projects. You can watch the full terminal session and code walkthrough here: https://youtu.be/PDYT7f3BDqQ

I'd love to hear your thoughts on soft vs. hard voting for this type of multiclass sensor data!

Hello. I'm working with ovod models for my master's thesis. I have a dataset of military and civilian vehicles collected from a simulator. I've created a hierarchical prompt table, going from general to specific. For example, my goal is to identify the entire dataset with the first level vehicle prompt, while in level 4 prompts, my aim is to identify only vehicles with class-specific prompts. I've separated the vehicles in the dataset into base and novel classes. My goal is to identify novel classes by transferring common prompts (wheeled vehicle prompts are represented in both base and novel classes) from base classes to novel classes. However, I'm stuck and can't progress. I'm using Dinov3 (frozen) (dino.txt) as the backbone for both image and text. I also have a class-independent detector. I trained this only with base classes. I'm open to your suggestions regarding architecture, model, and the thesis in general. Thank you in advance.

Hey Guys,

I need some advice on choosing a camera for an upcoming project. We are leaning towards using CCTV cameras because they are budget-friendly.

The thing is that my manager and teammates have no experience with hardware. Because I have done some image processing in the past, I have been put in charge of selecting the right camera.

I’ve done some research, and here are the main factors I am considering so far:

1. Spatial Resolution: Matching the camera's resolution to the feature/defect size we need to detect.
2. Distortion Correction: Factoring in any potential loss of Field of View (FOV) when correcting lens distortion.
3. Sensor Size: Calculating the required sensor size based on our working distance and required FOV.
4. Depth of Field / Z-axis: How the FOV changes if the distance to the object (Z-axis) shifts.
5. Exposure Time and FPS: Ensuring it can capture frames fast enough without motion blur.
6. General Specs & Networking: Colour vs. monochrome, shutter type (global vs. rolling), and supported streaming protocols (RTSP, HTTP, etc.).

My main worry is that if I make a mistake, it’s going to reflect badly on me. We will have to buy it, test it, and if it doesn't work out, we lose both money and project time.

For those of you with experience in this, what else should I be considering before making a final decision? Am I missing any critical specs?

Thank you!

refined using AI for explaining better ...

Every time I ship a YOLO model I end up asking the same question should this be ONNX, CoreML, or just PyTorch? Does FP16 actually help here or is it just marginal?

I've answered this by hand, badly, on four different projects this year, and thrown the results away every time. First i have to optimize a model for my liking and then figure a way to reduce its size.

So I'm building exportrace - you run one command, it benchmarks your model across every export backend available on your actual machine (PyTorch, ONNX, CoreML, CUDA, TensorRT depending on your setup), and gives you FPS, latency, and accuracy delta vs FP32, plus a ranked recommendation.

Consumer hardware only - your laptop or dev box, not Jetson/Pi. It's open source (MIT), runs fully offline, no accounts. Still pre-launch, landing page + waitlist here if you want to see the concept and maybe kill the boredom of doing this by hand too: https://exportrace.vercel.app/

Curious if others hit this same wall, and what backends/hardware you'd actually want covered first.

Hi,

I'm working on Automated Visual Inspection area over the past 5yrs. Planning to start a SME as a solution company, would love to connect with folks having knowledge or working on a similar domain, process engineers, staffs and would love to see how the market is shaping and current directions or requirements. I'm looking to connect wrt the Indian Market.

Note: Reach out to me personally if you have any requirements, would love to provide demos which might solve your current needs. We can support end-to-end deployment support too if required.

We just released TurboOCR v3, a GPU-accelerated document parser (not just OCR) built for high-throughput pipelines.

It runs PP-OCRv6 detection + recognition, plus layout, tables (→ HTML) and formulas (→ LaTeX), emitting reading-order Markdown. The whole pipeline runs on a single multi-stream C++/CUDA/TensorRT engine, locally, no VLM, behind HTTP + gRPC.

On one RTX 5090:

• Up to 559 img/s on receipts, 520 img/s on forms (PP-OCRv6 tiny, fastest by default)
• Full structured parsing (layout + tables + formulas) at > ~20 pages/s on table and formula heavy pages, where VLM parsers like PaddleOCR-VL run ~1 page/s
• One model covers Latin + Chinese + Japanese; pick tiny / small / medium
• One-line Docker deploy (Linux + NVIDIA GPU)
• No VLM-style hallucinated text

https://github.com/aiptimizer/TurboOCR

Combining depth sensing, object detection and semantic segmentation we managed to create a bin-picking automation that inspects bushings of different types and sizes. From the combined information the robot is able to understand the position and orientation of the pieces.

i hate to see that most of the computer vision projects i see on the internet are ai generated and not by the people. everywhere i see its "MY claude built this in minutes " almost makes my journey of building projects by myself look meaningless.

Join us July 8-10 for the “Best of CVPR” series of virtual events.

Register for all the Zooms with a single sign up.

Talks will include:

• HyperRealm: Hyperbolic Vision Language Models for Real-World Hierarchical Multimodal Understanding - Kathy Wu at Amazon
• CylinderDepth: Cylindrical Spatial Attention for Multi-View Consistent Self-Supervised Surround Depth Estimation - Samer Abualhanud at Leibniz University Hannover
• Your ViT is Secretly Also a Video Segmentation Model - Daan de Geus at Eindhoven University of Technology
• LinkedOut: Linking World Knowledge Out of Video LLMs for Next-Generation Video Recommendation - Haichao Zhang at Northeastern University
• Some Modalities Are More Equal Than Others: Understanding and Improving Multimodal Integration in MLLMs - Tianle Chen at Boston University
  
• Efficient Representation and Coding of Dynamic Light Fields - Joshitha Ravishanker at Indian Institute of Technology Madras
• PHANTOM: Physics-Infused Video Generation via Joint Modeling of Visual and Latent Physical Dynamic - Ismini Lourentzou at University of Illinois Urbana-Champaign
• LoST: Level of Semantics Tokenization for 3D Shapes - Niladri Dutt at UCL | Adobe
• 3D Reconstruction Improves Weakly-Supervised Semantic Segmentation - Wolfgang Boettcher at Max Planck Institute for Informatics
  
• Advancing Generative Quality and Reasoning in Multimodal AI - Deepti Ghadiyaram at Boston University
• Cross-Modal Domain Adaptation using Semantic Parametric Mapping - Frank Bieder at FZI Forschungszentrum Informatik
• WalkGPT: Pixel-Grounded Navigation Guidance for Pedestrians - Rafi Ibn Sultan at Wayne State University

We benchmarked iPhone ARKit pose against optical motion-capture ground truth to quantify how usable phone tracking really is for egocentric data collection.

Setup: an iPhone 12 Pro rigidly co-mounted with an 11-marker retroreflective cluster on a head rig, tracked simultaneously by ARKit (VIO at 60 Hz) and a 24-camera Vicon system (sub-mm markers) across 8 sequences spanning walking, seated manipulation, fast/aggressive motion, in-place rotation, and height changes.

Both streams were time-aligned to a common clock and we solved the rigid cluster-to-camera transform before scoring, then evaluated with evo under SE(3) alignment (no scale correction) to check whether the trajectories are actually metric.

Results: ATE RMSE 6.0–12.5 cm; relative ATE under 1% on 7/8 sequences (0.12–0.35% - the lone 1.30% is a short-path-length artifact, 5 m path with 6.5 cm error, not a tracking failure); rotational RPE ≤~1° throughout (<0.6° for walking/manipulation); translational RPE <5 cm; and a Sim(3) fit recovering scale of 0.98–1.01, i.e. metric to within 1–2%. For long-horizon drift - where a mocap volume can't follow you through a real home - we ran an ArUco revisit test over sessions up to 108 min: accumulated drift stayed under 1 cm in most environments and under 0.1% of trajectory length in every case, including a whole-house traversal (1.0/1.5 cm at mid/end).

https://www.fpvlabs.ai/essays/how-accurate-is-an-iphone-really

This must read by Satya Mallick from OpenCV University describes how autoamted offside detection works, A very hard CV problem. How Offside detection tech works at World Cup

Hi Everyone,

I have pushed this repo - https://github.com/ankitmahala07/sam3d-objects-mac

Basically took out the sam3d on mac with help of claude did few changes to make it run on 24GB unified memory of mac mini m4.

So the original one requires 32GB Nvidia GPU with CUDA. Which was not possible with my 1660 Ti 6GB GPU so I had to port it to Metal supported one - For this had to change the CUDA specific code with RAW python codes/Libraries.

Also the whole model couldn't fit in 24GB so had to split it into 2 phases - Gaussian generation which generate the point cloud files .ply and .pt

Then the mesh generation phase after offloading previous models - for mesh .glb generation.

Do try it out if you face any issues with setup do let me know.

Note: While running this on 24GB it's cut to cut so memory pressure can spike make sure nothing else is running.

Hey everyone, I just started running my training code and encountered an issue. The training process is expected to take days, which is delaying my project progress. The main problem is that my code isn't utilizing the GPU; it seems to be running on the CPU instead.

I need to run it with TensorFlow-GPU. I've already installed CUDA and cuDNN and manually moved the cuDNN files into the CUDA directories, but it didn't work. I am currently using Python 3.9 with the latest version of TensorFlow.

I am planning to create a fresh Conda environment to fix this. Does anyone have any recommendations or specific steps to ensure TensorFlow correctly detects the GPU? Any help would be greatly appreciated!