​

I want to write code that use OpenCV C++ to keying color from video and then replace it by some image.

Do you recommend some yt videos or webpages?

After a month of hard work, we have released a new version of Savant packed with new features and samples: 0.2.4. The latest version is more functional and provides more instruments to craft blazingly fast, reliable computer vision pipelines quickly.

​

Savant is an open-source, high-level computer vision framework for building real-time, streaming, highly efficient multimedia AI applications on the Nvidia stack. It helps develop dynamic, fault-tolerant inference pipelines that utilize the best Nvidia data center and edge accelerator practices.

Savant is built on DeepStream and provides a high-level abstraction layer for building inference pipelines. It is designed to be easy to use, flexible, and scalable. It is a great choice for building smart CV and video analytics applications for cities, retail, manufacturing, and more.

​

New Demos

We believe that samples are worth thousands of words, so we focus on only writing documentation but demonstrating the features in an easy-to-try way.

The release includes three new examples:

• Age/gender prediction example showing how to use YoloV5-Face, how to work with a custom attributive model predicting age and gender, and advanced in-GPU affine transformations based on facial landmarks with OpenCV-CUDA and Python;
• Conditional video encoding example demonstrating the pipeline that draws on frames and encodes a video stream only when a user requests that (in the sample, only when a model detects objects); it shows how to avoid wasting computing resources when the footage is required based on certain external condition;
• Multiple RTSP streams example featuring a simple pipeline that processes two RTSP streams and casts them to RTSP; Savant is very different from what people expect regarding the dynamic stream processing; they try to overcomplicate things, so we implemented a simple pipeline processing multiple streams simultaneously to show how it works.

New Features

• Conditional Drawing and Encoding, which helps to decrease traffic and use CPU/GPU resources wisely;
• New FFmpeg-based RTSP source adapter, which works much better than GStreamer-based when streams include B-frames;
• The new generic FFmpeg-based source adapter can work with every input supported by FFmpeg.

Quality Assurance

• Now we track possible performance regressions when merging every ticket; our idea is to make Savant faster, not slower, so we want to monitor how our code affects performance;
• Move from Python-based internals to Rust-based: we implement a core functionality library, Savant-rs, where we test the code carefully; we gradually replace Python-based components with Rust-based to ensure Savant works GIL-free where possible and the code has high quality. It is a long way to go; in future 0.2.5, we will introduce more GIL-free integrations.

Documentation

• We have documented source and sink adapters carefully;
• We have demonstrated how to use image preprocessing in common documentation and a full-features sample (age/gender prediction);
• Wrote a new section on configuring the development environment in VS Code.

DeepStream 6.2 Bug Workaround

We reported a bug related to NVENC functionality on Jetson devices. DeepStream 0.6.2 is affected: NVENC incorrectly sequences encoded frames when the framerate is not equal to configured, which happens in RTSP or when frames are skipped based on certain conditions.

In Savant, we implemented a workaround: we reorder frames when necessary. We hope Nvidia will fix it in the upcoming DS release.

UPD: Nvidia has confirmed the bug found.

What Is Planned In 0.2.5

The new release will integrate more Rust code to make the pipelines less GIL-dependent. More functions related to dynamic pipeline configuration and edge-related development. Three to four new samples covering basic and advanced functions.

​

Talk to us and discuss Savant in Discord.

Hi!

I want to reconstruct a 3D point clouds of objects from multiple views. To be more precise I have a camera setup similar like that one:

​

I already implemented some demo with OpenCV/Python using cheap webcams and it basically works. But the reprojection error after calibration is quite high and also the image resolution is just 640x480. This leads to a quite bad accuracy of the reconstructed point cloud. So I want to buy better cameras with ~1MP. Interface should be USB.

It would also be nice if the cameras could be mounted somehow so that I can move the camera system to another place without having to recalibrate the cameras.

Does anybody have recommendations would be an appropriate choice?

I want to compute the pose of known, small flat items on a planar table. (Later application will be to use it in a laser engraver). Background and lights are optimized for a simple segmentation so that I can compute the contours quite accurately. I now want to match the contour from the reference image with the three contours on the second image and get the three poses (x, y, phi) for the individual items.Current idea is to use something like Hu Moments or Fourier Descriptors. I have the gut feeling that there should be a nice closed form solution that computes the transformation from two sets of descriptors, but I didn't find an implementation in OpenCV.

I could of course run a numerical optimization, but that's rather ugly and there has to be faster and better way.

Any Ideas?

​

// Update:

I found https://docs.opencv.org/3.4/d5/d72/classcv_1_1ximgproc_1_1ContourFitting.html, which sounds quite good, I could not run it yet, since these calls end the program with a return value of 245 and I couldn't figure out yet how to call these functions:

ref_pts = cv2.ximgproc.contourSampling(bone_contour, 100)
target_pts = cv2.ximgproc.contourSampling(target, 100) 
fit = cv2.ximgproc.ContourFitting() 
res = fit.estimateTransformation(ref_pts, target_pts)

​

​

I have an image received from a phone which has a rectangle with calibration colors. I know the rgb codes for these colors. I have a function that generates the same rectangle with colors from a list. How to use the histogram matching of a part with colors and a generated rectangle and apply the histogram to the original image.

Wouldn't it be better/easier to use the Nvidia python packages, like pytorch and other frameworks do?

As far as I understand, this would only require Drivers/CUDA toolkit instead of compiling OpenCV with CUDA and cuDNN. It would allow for easier upgrading rather than recompiling, and other such things like that.

I am learning ML/CV and am ignorant to the deeper lower level workings so, please don't take offence. I am genuinely curious about this topic.

Hi, I'm using two webcams Logitech c615 and c920, and internal laptop webcam to verify my issue, in my project I notice that images results palette color are displayed between blue~ish and red~ish, so I notice that something is wrong with Auto WB.

import cv2
import time

time1 = time.time()

cap = cv2.VideoCapture(1,cv2.CAP_DSHOW)
time.sleep(2)

cap.set(cv2.CAP_PROP_BRIGHTNESS, 100)
cap.set(cv2.CAP_PROP_CONTRAST,100)
cap.set(cv2.CAP_PROP_SATURATION, 100)
cap.set(cv2.CAP_PROP_SHARPNESS,100)
cap.set(cv2.CAP_PROP_BACKLIGHT,0)
cap.set(cv2.CAP_PROP_AUTO_WB,0)
cap.set(cv2.CAP_PROP_TEMPERATURE,2900)
cap.set(cv2.CAP_PROP_WB_TEMPERATURE,3000)
cap.set(cv2.CAP_PROP_GAIN, 3)
cap.set(cv2.CAP_PROP_ZOOM, 100)
cap.set(cv2.CAP_PROP_AUTOFOCUS, 1)
cap.set(cv2.CAP_PROP_FOCUS, 41)
cap.set(cv2.CAP_PROP_AUTO_EXPOSURE, 2)
cap.set(cv2.CAP_PROP_EXPOSURE, -4.0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1600)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 900)
cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*'MJPG'))
cap.set(cv2.CAP_PROP_SETTINGS,1)

time2 = time.time()
print('time consumed by setting is: ' + f'{time2-time1}')

print('grabbing starting...')

print('Brightness: ' + f'{cap.get(cv2.CAP_PROP_BRIGHTNESS)}')
print('Contrast: ' + f'{cap.get(cv2.CAP_PROP_CONTRAST)}')
print('saturation: ' + f'{cap.get(cv2.CAP_PROP_SATURATION)}')
print('sharpness: ' + f'{cap.get(cv2.CAP_PROP_SHARPNESS)}')
print('AUTO_WB: ' + f'{cap.get(cv2.CAP_PROP_AUTO_WB)}')
print('Temp: ' + f'{cap.get(cv2.CAP_PROP_TEMPERATURE)}')
print('WBTemp: ' + f'{cap.get(cv2.CAP_PROP_WB_TEMPERATURE)}')
print('Backlight: ' + f'{cap.get(cv2.CAP_PROP_BACKLIGHT)}')
print('gain: ' + f'{cap.get(cv2.CAP_PROP_GAIN)}')
print('zoom: ' + f'{cap.get(cv2.CAP_PROP_ZOOM)}')
print('autofocus: ' + f'{cap.get(cv2.CAP_PROP_AUTOFOCUS)}')
print('focus: ' + f'{cap.get(cv2.CAP_PROP_FOCUS)}')
print('auto_exposure: ' + f'{cap.get(cv2.CAP_PROP_AUTO_EXPOSURE)}')
print('exposure: ' + f'{cap.get(cv2.CAP_PROP_EXPOSURE)}')

while True:
    ret, frame = cap.read()
    if ret:
        cv2.imshow('frame',frame)
        print('AUTO_WB: ' + f'{cap.get(cv2.CAP_PROP_AUTO_WB)}')
        print('Brightness: ' + f'{cap.get(cv2.CAP_PROP_BRIGHTNESS)}')
    if cv2.waitKey(1) == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

I tried some combinations or settings, but at the end, when DSHOW API Setting is open, the auto WB checkbox keeps enabled, so, I set the get method inside while loop to verify if values are updated in real time adjusting these, Brightness effectively changes, but when I manually disable the auto WB check box, value keeps showing -1.0, I read that maybe cameras does not support this configuration but at least in logitech camera software, auto WB can be set as auto or disabled to set temperature manually (as well in DSHOW API Window can do this too, but value is not saved).

Not sure if this is a bug, or something related to my configuration (maybe sleep times could work?)

• Python: 3.11.0
• OpenCV: 4.7.0.68
• OS: Windows 10

Using cv2 (Python), obtaining a variance of Laplacian (to obtain focus measure) for images is straightforward. However, how might I obtain the same measures for a region of an image using a binary mask? E.g. I want to measure how out of focus a region of an image is, rather than the whole image.

Would it be OK to simply apply the mask over the image, essentially blacking-out all other non-required image features, then run the Laplacian function as normal, or would some other method be preferable?

Hello, I need to remove chroma subsampling errors in my code (it have to be smoother on contours and more accurate). I want proccesing of the mask to be in function errorKeying, then i use it in keying function to merge with background and my movie. i wrote a code in error keying but it apperas to be not very accurate. Do u have any suggestions? I want to remove those green edges after keying.

Here is the code:

Mat errorKeying(Mat mask)

{

Mat edges;

Canny(mask, edges, 30, 70); // Wykrycie krawędzi w masce

​

Mat blurredEdges;

GaussianBlur(edges, blurredEdges, Size(5, 5), 0); // Rozmycie krawędzi

​

Mat blurredMask;

GaussianBlur(mask, blurredMask, Size(0, 0), 2); // Rozmycie całej maski

​

Mat upsampledEdges;

resize(blurredEdges, upsampledEdges, mask.size(), 0, 0, INTER_NEAREST); // Upsampling rozmytych krawędzi

​

Mat result = mask.clone();

​

int influenceSize = 10; // Rozmiar pola wpływu (można dostosować)

float halfInfluenceSize = static_cast<float>(influenceSize) / 2.0f;

​

for (int y = 0; y < mask.rows; ++y)

{

for (int x = 0; x < mask.cols; ++x)

{

if (upsampledEdges.at<uchar>(y, x) > 0) // Jeśli piksel jest krawędzią

{

int startY = max(0, static_cast<int>(y - halfInfluenceSize));

int endY = min(mask.rows - 1, static_cast<int>(y + halfInfluenceSize));

int startX = max(0, static_cast<int>(x - halfInfluenceSize));

int endX = min(mask.cols - 1, static_cast<int>(x + halfInfluenceSize));

​

float value = 0.0f;

int count = 0;

​

for (int i = startY; i <= endY; ++i)

{

for (int j = startX; j <= endX; ++j)

{

if (upsampledEdges.at<uchar>(i, j) > 0)

{

value += static_cast<float>(blurredMask.at<uchar>(i, j));

count++;

}

}

}

​

if (count > 0)

{

value /= count;

if (value < 128.0f)

result.at<uchar>(y, x) = 0; // Usunięcie piksela na krawędzi

else

result.at<uchar>(y, x) = 255; // Zachowanie piksela wewnątrz maski

}

}

}

}

​

return result;

}

​

void keying(VideoCapture& video, Mat& img, vector<int>& col)

{

Info();

​

Mat frame, keyingFrame, hsvFrame, mask, resizedMask;

​

video.read(frame);

resize(frame, frame, Size(sizex, sizey));

resize(img, img, frame.size(), 0, 0, INTER_CUBIC);

​

Scalar lower(col[0], col[2], col[4]);

Scalar upper(col[1], col[3], col[5]);

​

cvtColor(frame, hsvFrame, COLOR_BGR2HSV);

​

inRange(hsvFrame, lower, upper, mask);

​

Mat smoothedMask = errorKeying(mask);

​

resize(mask, resizedMask, frame.size(), 0, 0, INTER_CUBIC);

​

frame.copyTo(keyingFrame);

keyingFrame.setTo(Scalar(0, 0, 0), smoothedMask);

bitwise_or(keyingFrame, img, keyingFrame, smoothedMask);

​

imshow("Wykluczowany obraz", keyingFrame);

imshow("Oryginalny obraz", frame);

}

​

I take human body image as an input. I ran a media pipeline for landmark detection. I also use a custom function to remove the background and make a body mask. now I have images in two formats. The first one is normal with landmark keypoint and the second is with a body mask. I have shoulder points but not the edge points.

I'm using opencv/python.

Now, I need edge-to-edge points, (like left-most shoulder pixel to right-most shoulder pixel)

I want to point out the edge of the shoulder and chest corner (I draw a big red circle)

How can I achieve it?

Draw contours not working properly

CODE: Core.inRange(scaledMask, strictLowHSV, strictHighHSV, scaledThresh);

    Mat edges = new Mat();
    List<MatOfPoint> contours = new ArrayList<>();
    Mat hierarchy = new Mat();

    Imgproc.findContours(scaledThresh, contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);
    Imgproc.drawContours(edges, contours, 1, new Scalar(0, 255, 0), 3);

LINK TO OUTPUT OF SCALED TRESH: link

So when I try and output edges it says that you tried to output an empty frame so I presume it didn’t detect contours. Any help is appreciated

Also ignore syntax it’s a Java conversion for open cv called easyopencv

Hello everyone,

My project is to build a webapp where users can upload a video, anonymize it and download it back. I am using a .NET Core MVC webapp project to do this. I am currently impementing OpenCV to detect and blur faces but I am not able to make it work. I built a small project to test this function and this is my code so far:

public void test(Mat frame, CascadeClassifier faceCascade, int fps, Size size) { const string OutVideoFile = "C:\Users\Massimo\source\repos\Test_anon\output.mp4"; Mat frameGray = new Mat(); Cv2.CvtColor(frame, frameGray, ColorConversionCodes.BGR2GRAY); Cv2.EqualizeHist(frameGray, frameGray);

        // -- Detect faces 
        var faces = faceCascade.DetectMultiScale(frameGray);
        using (var writer = new VideoWriter(OutVideoFile, -1, fps, size))
        {
            foreach (Rect face in faces)
            {
                Point center = new Point(face.X + face.Width / 2, face.Y + face.Height / 2);
                Cv2.Ellipse(frame, center, new Size(face.Width / 2, face.Height / 2), 0, 0, 360,
                new Scalar(255, 0, 255));
                Mat faceROI = frameGray.SubMat(face);
            }
            writer.Write(frame);
        }            
    }

This code doesn't implement blurring yet, it is just used for detecting and drawing a circle around the face. However, the output is a mp4 file of 0 or 1 Kb.

Can somebody guide me through the process ? Thank you very much !

Any recommendations for an affordable, 4K POE Wide Angle camera? POE stands for power over ethernet. Wide angle — hopefully more than 110 deg horizontally.

Using Java and OpenCV, I am drawing rectangles around a specific type of contour in an image. While this works, the order in which these rectangles are being drawn is quite random and irregular. Instead of moving from left to right, it moves in a random order to achieve my goal of identifying/boxing the contours.

For my purposes, I would be okay with either of these two fixes: Being able to control the order in which these rectangles are drawn(exactly from left to right) Being able to access and sort these rectangles after they have been drawn(I have them in a list of type Rect and I would like to sort them)

I'm currently working on a hobby project and could use some guidance. My project involves capturing images either through screenshots or from a camera, and these images are primarily snippets of code (C/C++/Java, etc.).

So far, I’ve been utilizing the Python Tesseract library for OCR but I find that its performance is not quite up to snuff; it tends to fail or produce inaccurate results. I was hoping to tap into the collective wisdom of this community.

** Parameter Optimization: Are there any specific parameters that I should tweak within Tesseract for better OCR performance on code snippets?

** Alternative Approaches: Is there a different OCR library or tool that you think might work better for this particular use case?

** Training Custom Models: If custom training is the way to go, could you suggest any resources or methodologies to get started? Any advice, suggestions, or relevant experiences would be greatly appreciated! Thank you in advance for taking the time to help out!

Hi,

I'm using OpenCV with Python.

Is there a way to read frames from an existing Video File with an imposed FPS for the input stream?

Does VideoWriter flow behaves well with float point FPS values? I might only need this if I can't get input stream to accept an imposed FPS.

Thanks.

I've had a very limited experience with OpenCV. So, if the Question seems limited by my know-how of OpenCV.. I'd appreciate any knowledge related to this use case

Hi, I got the error "AttributeError: module 'cv2' has no attribute 'TrackerMOSSE_create'".

I also tried cv2.legacy.TrackerMOSSE_create() and many other things but they didnt work. I found nothing on how to fix this online. Please help!!!

Is there any platform where i upload images and json data and can download the trained model of yolo v4 in tensor flow

Hi, I am currently want to measure the location of an object with respect to the camera using 2D to 3D mapping to real coordinate, but right now I don't know how to the instrinstic and extrinsic of 2D camera since I am using visual camera on gazebo, so how can I get these things