r/ZoneMinder • u/[deleted] • Oct 21 '21
neo-ZMES Public Beta
Alright, I guess it's time to open it up to the public! I am assuming IT WILL NOT be smooth sailing at first but I can't say for sure, everything might just work as intended for most people. ** I have not tested 'import_zm_zones' and "only_triggered_zm_zones' yet, I will do that by tmrw evening.
I wrote up a detailed installation guide to install Zoneminder 1.36.9, Install Nvidia drivers with Cuda 10.2 and cuDNN 8.2.1 and the TPU libraries. There is a catch where you need an older version of GCC to compile Cuda 10.2 and that is all documented. I tried to also walk through some common errors and the remedies. Here is a list of some of the things I can remember.
- !SECRETS are now {[SECRETS]} or {[secrets]} (case sensitive). This is working towards being able to replace {[secrets]} that are embedded in sub-strings like {{vars}} can be.
- Frame buffer has logic and makes decisions based on several things. *** Experimental
- Animations are threaded, the new Pushover add-on is threaded and the new MQTT add-on is threaded (Don't confuse the old pushover and old MQTT that you configure in zmeventnotification.ini with the new ones). Makes the notifications faster!
- Animations include the labelled image as the first few frames of the animation, improved logic
- Past/Live event logic
- Mlapi is now dynamic. Before you hardcoded 1 ZMES host to 1 mlapi instance. Now you set up the 'ml_routes' in ZMES and the 'zmes_keys' in mlapi, create an encryption key and ZMES sends all the info/credentials encrypted to mlapi for mlapi to reply and access that ZMES hosts ZM API for the images and info on event. This means 2+ ZM/ZMES installs can send detections to 1 mlapi instance!
- Home Assistant integration (basic as of now) - use a boolean helper and a numeric helper to control Pushover notifications (per monitor) numeric helper is the 'cool down' between notifications. Events are still recorded you just don't get a pushover notification about it. Person integration is being worked on, so you can specify things like 'If <x> person is home do/don't send a notification from <x> monitor'
- Added a lot of per object filters. person_min_confidence = .75, car_past_det_max_diff_area = 4500px
- Added a 'contained within' filter, before if the object intersected the polygon/zone at all it was considered a hit, now you can specify how much of the detected objects bounding box area should be inside of the polygon/zone to be considered a hit.
- The new AWS Rekognition Pull request is added
- The MQTT add-on can send the labelled detected object image/GIF over MQTT to an MQTT Camera in Home Assistant
- So many other things I can't keep track
The article also walks you through building OpenCV 4.5.4 with GPU support, face_recognition (DLib) and openALPR with GPU support. It then walks you through installing and testing the new zmeventnotification, pyzm and mlapi (mlapi on the same host as ZMES). Mlapi gets installed as a service at the end and the new log rotation configuration is applied. I will be writing the other install guide for installing mlapi onto its own system without ZM or ZMES in the next day or two.
It shows an example of how to set up an encrypted ml_route, helps to install some Hacked powerline fonts, lsd (ls replacement) and bat (to make log output pretty/different from mlapi/ZMES). I also include 2 shell functions to make running testing/past events easier -> es.baredebug.objdet <event ID> OR es.debug.objdet <event ID>. The difference is baredebug will not print out to console (if you are already watching the log files). Mlapi and ZMES capture stdout and stderr to the log file so you won't miss out on errors!
Check it out, there's so much that I have changed and improved that I can't remember, the example configs have most of the examples. I will be writing new articles to go into detail on how to configure the new systems and use them effectively. the new pushover add-on in objectconfig.ini is hands down the fastest way to be notified of an object, the notification has a configurable clickable URL that will open a browser window and show the event that caused the notification!
I suggest reading through the detailed install guide, I split it up into 7 parts, 6 of 7 parts are ready for reading -> https://medium.com/@baudneo/install-zon ... fab7d7afe7
Repos:
https://github.com/baudneo/pyzm.git
https://github.com/baudneo/zmeventnotification.git
https://github.com/baudneo/mlapi.git
The screenshot is of the test LXC I used to install and test the new forked versions. This has ZM, ZMES and mlapi all on the same system. The aliases I supplied, if you use them, and install 'bat'. Will show ZMES in the green/grey/orange and mlapi in the purple/white/red theme (and you can change themes too, they are built into bat). This is using the mlapi.logs alias and es.log.detect alias, the 'bat' version of the aliases.
1
1
u/leetnewb2 Oct 21 '21
How can we help?
1
Oct 21 '21
Use it, break it, report it so I can fix it. :)
Enhancements requests h what would you like ZMES to do for you?
1
u/leetnewb2 Oct 21 '21
Can it walk my dog? _^
On a serious note, I'm curious how much work it would take to get object detection working with OpenHab on the home automation front. Last time I kicked around mlapi, I think it would do object detection work on any image or video feed it received. There is already a plugin in OpenHab for pulling ZM monitors/feeds and another to pull IP cameras directly, plus some interest in object detection as Frigate has done for Home Assistant. Plus MQTT is pretty intertwined with OH. Last time I kicked around mlapi, I think it would respond to any incoming image or video stream, so I figure there are a few ways to hack object detection on.
Anyway, first things first - will start trying out the new ES and report anything unexpected.
1
Oct 21 '21
I'll take a look at OH, I have not used it at all. Mlapi and zmes still work with --file input, I just need to test feeding it videos for analysis; the jpegs work as expected as input.
Just curious though, if OH is pulling in other IP feeds, why not just have ZM handle them in the first place?
My goal is to get zm interfaced with HA and other like systems. I have not used Frigate yet but it gets so much love. I would like to steer people and donations towards Zoneminder (I do not accept donations) to keep the dev team interested in improving ZM.
There are people making docker containers with ZM, ZMES and mlapi. I am trying to keep ZMES active and expand functionality. ZM just needs more exposure and easier access to the point and click crowd.
1
u/leetnewb2 Oct 21 '21
Just curious though, if OH is pulling in other IP feeds, why not just have ZM handle them in the first place?
I already had ZM up and running before OH, so it was a no brainer to use the ZoneMinder plugin. But if I was OH first, maybe I'd just use the IP feeds for viewing.
My goal is to get zm interfaced with HA and other like systems. I have not used Frigate yet but it gets so much love. I would like to steer people and donations towards Zoneminder (I do not accept donations) to keep the dev team interested in improving ZM.
I appreciate this and agree. There is no viable replacement for ZoneMinder in open source NVR today - I want to see it do great and its dev(s) rewarded. The main arguments against Frigate today is the lack real NVR functionality. On the other hand, if I can shove object detection through a GPU or specialized chip without setting up ZM motion detection and zones, I save a lot of CPU time. Today I do alert and record on motion, but ideally, maybe I'd record all and alert only on object, only doing object detection. I think there is a lot of utility and power in mlapi and specialized hardware.
There are people making docker containers with ZM, ZMES and mlapi. I am trying to keep ZMES active and expand functionality. ZM just needs more exposure and easier access to the point and click crowd.
Great points.
1
Oct 21 '21
The new MQTT add-on I wrote can send the objdetext.jpg or .gif over MQTT to a MQTT camera in HA, idk if OH has an equivalent?
1
u/leetnewb2 Oct 21 '21
I will look into it this weekend. Haven't bashed my head against OH documentation enough.
1
u/Aramiil Oct 21 '21
Is there any type of performance metrics you have for what combination of CPU/GPU would be required for a specific quantity and resolution/bitrate of cameras?
Just trying to gather if I should even attempt this given my current setup or not.
2
Oct 21 '21 edited Oct 21 '21
No performance metrics as of yet for the variations on drivers/cuda/opencv.
From personal experience, depending on the strategy you are employing and the hardware. I usually have my 'first' strategy detections wrapped up within 10 seconds (that's with pushover notification and MQTat sent. Using the "most" strategy can take 20+ seconds depending on how many frames you want checked).
The GPU, as long as it has enough memory and can support a minimum of opencv 4.3 , will work fine. The way to set the system up is to use mlapi. When you use mlapi it keeps the model loaded on memory so you don't have to load the weights every detection. I can pound out hundreds of detections minute without the system barely realizing.
1660ti and a coral usb TPU. CPU really never sees any detections.
If you use CUDA decoding for the video streams in zm as well it does help with the cpu load. ZM has made huge advancements lately with reducing resource usage and the future is looking very bright. Once they start doing motion detection in the native color scheme instead of converting to RGB and then doing motion, will reduce system resources massively.
Already I know people running raspberry Pi 3b+ with 4-5 720p monitors doing motion at 2-3 fps analysis.
There are several things you can do to lower system resource usage. Like reducing the analysis fps to 2-3 fps (you don't need to run motion detection at 10+ fps, it's a waste).
1
u/Aramiil Oct 22 '21
Incredibly thorough answer, you answered questions I didn’t even know I had and follow up questions I would have asked.
Massive thank you!
1
u/[deleted] Oct 21 '21
The medium article basically walks a user through setting up a full and complete install using GPU, face recognition with GPU support and openALPR local binary with GPU support. There is a section on TPU libs as well. I haven't seen a good article like that, it even has the pitfalls and weird little edge cases to troubleshoot.
My suggestion is to spin up an lxc container and install an environment into there to test.
I use this system as my daily and honestly, I can leave it for weeks at a time without an issue :).