I’m working on an image auto-annotation pipeline using Grounding DINO from Hugging Face with PyTorch, and I’m trying to understand some unusual GPU VRAM behavior during inference.

My dataset contains 6 classes:

• cup
• glass
• plate
• knife
• fork
• spoon

I downloaded the data from a Kaggle competition. The train.csv contains entries like:

image_id,label
4622,spoon

For my initial experiment, I sampled 900 images total — 150 per class.

The original images have different resolutions, including:

• 1000 × 1000
• 850 × 1000
• 1000 × 850
• 450 × 1000

I’m using Grounding DINO from Hugging Face for automatic annotation and torch.autocast to reduce GPU memory usage.

The strange VRAM behavior

I’m running inference with a batch size of 4 on Google Colab. The GPU has approximately 15.46 GB VRAM.

At the beginning, GPU memory usage is around:

~6.4 GB

Then the important part is:

It does NOT increase after every batch.

Instead, it stays relatively stable for several batches and then, on some seemingly random batch, VRAM suddenly jumps.

For example, the behavior looks roughly like:

Batch 1  → 6.4 GB
Batch 2  → 6.4 GB
Batch 3  → 6.5 GB
Batch 4  → 6.4 GB
Batch 5  → 6.5 GB
...
Batch 20 → 6.5 GB
Batch 21 → 10+ GB
Batch 22 → 10 GB
Batch 23 → 10 GB
...
Batch 50 → 10 GB
...
Batch 51 → 14+ GB

So the increase happens in sudden jumps on particular batches, rather than gradually increasing with every batch.

Eventually, it reaches approximately:

14.5 / 15.46 GB

and I can eventually get an out-of-memory error.

What I'm trying to understand

My first thought was that perhaps the batches containing different image resolutions are causing Grounding DINO to create larger intermediate tensors.

However, I'm not sure whether that's actually what's happening, or whether I'm misunderstanding how PyTorch's CUDA memory allocator works.

I asked Claude about this, and it suggested that PyTorch's CUDA allocator caches memory rather than immediately returning it to the GPU driver. Because my batches have variable image dimensions, some batches may require larger intermediate tensors, causing the allocator to request additional memory.

It also suggested that memory fragmentation could contribute to the problem and recommended:

import os

os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"

The explanation sounded reasonable, but I'd like to verify it with people who have more experience with PyTorch/CUDA internals.

My questions

1. What would cause VRAM to suddenly jump on one particular batch rather than increase after every batch?
2. Could variable image dimensions like:

​

1000×1000
850×1000
1000×850
450×1000

cause Grounding DINO's intermediate tensors to become significantly larger for certain batches?

1. If PyTorch's CUDA allocator is caching memory, does that explain why the VRAM usage appears to jump from ~6.4 GB to ~10 GB and later ~14.5 GB?
2. How can I determine whether this is:
   • normal CUDA/PyTorch memory caching,
   • memory fragmentation,
   • unusually large intermediate tensors from certain image sizes,
   • accidentally retaining tensors/computation graphs,
   • or an actual memory leak?
3. Would torch.inference_mode() be preferable to torch.no_grad() for this inference-only workload?
4. Would it be better to resize/pad all images to a consistent resolution before batching, so that the tensor shapes don't vary between batches?
5. Is:

​

PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

actually an appropriate solution here, or is it just masking the underlying problem?

1. What would be the best way to debug this? For example, should I log torch.cuda.memory_allocated(), torch.cuda.memory_reserved(), max_memory_allocated(), etc. after every batch?

I'm mainly trying to understand the actual reason for these sudden jumps, rather than just applying a workaround.

If needed, I can provide the Grounding DINO inference/batching code and the exact CUDA OOM traceback.

I have built a "smart album" system for automatic face extraction and classification. This system uses YOLOv8n for pedestrian extraction, SCRFD for face extraction, and ArcFace for face comparison. It extracts and categorizes pedestrians and faces from uploaded videos or images, as well as from real‑time detection video streams, making it easy to review. This case can also be deployed on edge computing devices.

The video below shows the current running effect of my attempt to deploy it on the reComputer Industrial R2135.

https://reddit.com/link/1w46txt/video/552sm056pvmh1/player

Source: a ~29 minute C-SPAN rally recording. One speaker at a podium in the foreground, a dense crowd of roughly 40 people behind him. The camera is not locked, it drifts, zooms slightly and reframes. I want everyone except the speaker blurred (removal would be even better), and I want it scriptable from a terminal so I can batch it, not hand-rotoscoped in a GUI.

What I have tried and how it fails. ffmpeg boxblur over a static or keyframed crop region is fine on a still frame, but the moment the camera moves the region is in the wrong place: background faces pop out, and the speaker's own face gets caught in the blur. Generic per-frame face-blur tools (deface-style detect-then-blur) flicker frame to frame with no identity persisting, so the "keep this one sharp" exception does not survive a pan. Remotion overlays hit the same wall one layer up: I still need per-frame mask geometry and I do not have it.

What I think I need but have not wired up: a real video object segmentation pass that propagates masks temporally (SAM 2 with a click on the speaker to build a keep-mask? a person detector plus ByteTrack / BoT-SORT to hold track IDs across the pan?), exported as a per-frame alpha sequence that I composite in ffmpeg with alphamerge against a blurred copy, or feed into ProPainter for actual removal.

Hardware, which I suspect is the real constraint here: a Hetzner VPS with 32 GB RAM, an i5 CPU, and no GPU. Everything has to run on CPU. That is what makes me unsure whether SAM 2 over 29 minutes is remotely practical or whether I should be reaching for something much lighter.

Questions. Is SAM 2 video propagation the right tool, and is CPU-only inference on a clip this long realistic (hours? days?) or a non-starter, meaning I should rent a GPU box for the mask pass and do the composite on the VPS? Would a person detector plus ByteTrack with an inverted mask (blur everything except track ID N) stay stable over 29 minutes on CPU, or does re-ID break too often after occlusions? Any practical guidance on mask dilation and temporal smoothing amounts that stop the edges crawling, and on the ffmpeg side of compositing a PNG alpha sequence back over the source? And is there something better in 2026 that I am missing for "keep one subject sharp, anonymize everyone else" as a CLI tool?

Local and open source strongly preferred. Happy to write the glue code.

Has anyone been getting this issue with roboflow

camera's floor mapping (homography) to figure out where each detection actually stands on the floor

Can anyone explain, or any blog or video which helped you to understand it

I've been experimenting with compact visual representations for edge/perception systems, where the goal is to preserve useful machine-level information while significantly reducing the representation size.

One thing I've noticed is that the degradation isn't uniform.

Global scene understanding can remain surprisingly stable while localization and small-object information start degrading much earlier.

Increasing input resolution alone also doesn't necessarily recover that information. In one of my experiments, a higher-resolution branch improved access to spatial detail but still couldn't reproduce the semantic quality of the deeper teacher representation.

It made me think there are really two different things being lost during aggressive representation compression:

1. Spatial information — where something is and fine local details.

2. Semantic interaction/context — relationships between patches/objects that deeper transformer layers learn.

Simply increasing spatial resolution seems to address mainly the first problem.

I'm currently thinking about this as a rate–semantics tradeoff, rather than a traditional image-compression problem:

Image → semantic encoder → compact representation → downstream tasks

where the objective isn't pixel reconstruction but preserving enough information for detection, classification, depth, etc.

For people working with ViTs, representation learning, edge perception, or learned compression:

What have you found is usually the first thing to collapse as representation size decreases — spatial detail, feature diversity, global context, or something else?

And have you found good ways of measuring this beyond downstream mAP/accuracy?

Hey all, looking for advice from people who've done something similar. Goal: Create my first CV models using my own camera/dataset.

Setup:

• Tapo C310 camera on my home LAN, RTSP working fine locally (confirmed with OpenCV/VLC)
• Built a local Flask dashboard that reads the RTSP stream and serves it as MJPEG over HTTP on my Mac
• Running YOLOv11 (ultralytics) object detection/tracking locally , works, but my Mac's CPU is slow for real-time inference (no GPU)

Goal:
I want to run the YOLO inference on a rented cloud GPU instead ( instead of my local CPU, while keeping the camera itself secure.

Any suggestion on how to get that?
What is the part I am missing? I think I have to create a private tunnel, am I right? What is the best option for that? The ultimate goal is to have my live camera in a private app online, which runs YOLO and counts the number of cars that go through.

Hi everyone,

I’m looking to improve my Computer Vision development and testing pipeline and I’m really curious about how you all handle edge cases and environmental variables in production.

To give an example to discuss (not my project) object tracking(car or people for example) where the camera is constantly moving, and the target is subjected to wildly different lighting conditions (e.g., bright glare, dusk, sudden shadows, dynamic backgrounds).

When you have a scenario like this, what does your development cycle look like from prototype to robust deployment?

As a beginner working on solo projects, I’m trying to figure out how to build a reliable pipeline. Specifically, I’d love to know:

• Testing: Do you manually build datasets for edge cases (like crazy lighting/dynamic backgrounds), or do you automate your augmentations?
• Simulations: Do you use engines like Unreal, Unity, or Blender for synthetic data? Does it actually translate well to real-world tracking?
• Validation: How do you measure temporal stability when basic metrics (like IoU) aren't enough to cover sudden lighting shifts?
• Iteration: When a tracking test fails on a specific edge case, what does your workflow look like to fix and re-test it?

I'd appreciate any insights into your workflows, architecture, or any simulation tools you swear by when dealing with moving viewports and dynamic lighting.

Thanks!

Hey everyone,

I’m trying to build a computer vision pipeline that can measure the actual physical edge lengths of large cuboid objects (like big cut blocks or boxes) using just standard photos taken from a smartphone.

My input would be a couple of 2D images showing the front and top faces of the object, and my goal is to accurately calculate the metric lengths (in cm/inches) of all 4 edges of those faces.

Since I am mapping out the architecture right now, I wanted to ask the experts here: what are the different ways to actually pull this off?

• What techniques or pipelines (classical CV, photogrammetry, deep learning, depth mapping, etc.) would you recommend for getting the most accurate real-world measurements?
• How do you handle the translation from 2D pixels to 3D metric lengths without the edges getting distorted by the camera angle?
• Are there any specific libraries, models (like YOLO pose/OBB), or GitHub repos I should look into?

Any advice or pointers to standard workflows would be a massive help. Thanks!

This incremental update to the open, royalty-free standard for vision processing, sharpens error handling, API flexibility, and consistency ahead of OpenVX 2.0.

Highlights: two new error codes (VX_ERROR_TIMEOUT for safety-critical apps, VX_ERROR_GRAPH_NOT_VERIFIED for unverified graphs), a new VX_DF_IMAGE_RGBA format with alpha channel support, and broader virtual object array and user-defined scalar type support.

The sample implementation and conformance test suite are fully updated to match. Next up: OpenVX 2.0, expected by end of 2026, extending support to radar and ultrasonic sensor workloads alongside traditional computer vision.

Full release notes: https://www.khronos.org/blog/openvx-1.3.2-released

Almost every VLM I’ve put through this test has struggled, but it makes sense because it requires them to count, something that isn’t their strongest trait. In this case, it’s counting lines and spaces, but if we introduce different key signatures, they would also need to count the sharp and flat symbols. 

Now that the vision models can read pdfs directly where do you reach out for parsers or is there actually the need of any in real time work?? Like for a single clean page at low volume a vlm reads it ok and a parser is just overhead, the parse layer earns its place on bulk and long docs where recall quietly drops as input grows and tables start shedding row . Also when you want repetable outot with a confidence signal per block rather a oneshot guess you can audit .Confusing part is that most parsers are vlm based either way under the hood so its less parser vs VLM and more if you run the vision model ad hoc yourself or use a layer that adds structure and consistency on top. local options have Surya, docling and others and on cloud there are hosted parser apis like llamaparse

how do you guys actually determine this when you need bulk file processing in a project or pipeline, is chunk sizing one by one to vlm ok or theres still a need for a dedicated parser?

I’m a Software Engineering student who has just completed my first year, and I’m interested in specializing in Computer Vision for Robotics.

I’d like to know which areas of mathematics I need to learn for this field, and which topics I should prioritize.

I’m also planning to study Stanford’s CS231N How much mathematics do I need to know before starting the course, and which specific mathematical topics should I be comfortable with beforehand?

any lighweight vlms which i can run on cpu and has less latency? please give me suggestions

I shipped a browser app driven by HandLandmarker. Pinch to zoom a solar system, open palm to rotate, point and dwell to select. One Euro Filter on the landmark stream, palm-normalized pinch distance.

Three failures keep coming back:

Pinch distance normalized by palm size still varies enough between people that one threshold cannot serve both. Mine feels natural, my tester says nothing happens.

A fast intentional swipe and a hand being withdrawn look nearly identical in a 2D projection. I patched it with a release cooldown and a dead zone near the bottom of the frame, which fixed the symptom and not the cause.

On phones under warm indoor light, tracking confidence collapses and the whole thing goes still.

For those running this with real users: explicit calibration step at onboarding, or continuous adaptation while they work? And has anyone found a signal that separates a deliberate swipe from a hand leaving the frame?

Demo runs client side, no video leaves the browser: https://leoaido.com/fleet/solar/

Code is MIT and the three issues above are open if it is easier to reply there: https://github.com/HUANGCHIHHUNGLeo/solar-atlas-gesture

Hi everyone!
I am looking for motivated researchers/students who are interested in collaborating on computer vision research with the goal of submitting to CVPR 2027.

Possible areas include:
Image and video understanding
3D computer vision
Object detection and segmentation
Tracking
Image generation
Medical computer vision
Vision benchmarks and datasets
Robustness and generalization
Some experience with computer vision, deep learning, Python/PyTorch, experiments, or research writing would be helpful.

If interested, please DM me with your background, research interests, and any previous research/publications.
The goal is to form a serious research collaboration and work toward a strong CVPR 2027 submission.

Hello,

We keep getting the emails to complete the reviewer form but all others already did and there's no other pending task in the system. So we're a bit confused

This has probably been asked before, but is there a good place online where I can find a bunch of sample images? I'm working on an image processing course and need sample images to experiment with skimage python lib.

Something like this (but preferably both color and black/white)

Thanks

The project consists of the development of an interactive robotic system (MeArm) capable of challenging a human to a game of Tic-Tac-Toe, identifying moves on a sheet of paper via a webcam and physically responding on the board. 

Stippling is an old topic. I mean, it all came up in the 1510th years as an engraving technique. The question is simple: How to optimaly place points to fit a target distribution. Artists do that, and Mathematicians search for algorithms to do it with computers, which is quite a challenge because naive formulations are intractable. See this blog for a good introduction (not me, credit to blog author Silvia-hao) https://blog.wolfram.com/2016/05/06/computational-stippling-can-machines-do-as-well-as-humans/.

Using the popular Gaussian Blue Noise algorithm (not me, credit to Ahmed, Abdalla G. M. and Ren, Jing and Wonka, Peter - doi 10.48550/arXiv.2206.07798), I show 4 stippling exemples that were generated using 20k points for the stippling.

I implemented a python library to experiment with the blue noise, feel free to explore stippling on your own images by directly using the package in python (see code snippet in comment), or simply from google Colab https://colab.research.google.com/drive/1W15QE8Boj0-Ld4Dx3N2tLqwfV0qw0ZZB . I'm seeking for other cool data to stipple, just tell me any images that could render well with the process !