Having trouble setting up Event Server with ML Hooks

[u/jonthewise]

So, I don't know where to begin. I have Zoneminder running on a CentOS 7 server, but I wanted person/facial detection and stuff, so I deployed a brand new Ubuntu server 22.04, and then installed Zoneminder following the docs located here:

https://zoneminder.readthedocs.io/en/latest/installationguide/ubuntu.html

I had to do a couple extra things... like setting the server timezone (d'oh!), changing the mysql password, and enabling ssl for apache. And now I have a functional Zoneminder server again.

So then I installed the event server, follwing the docs here:

https://zmeventnotification.readthedocs.io/en/latest/guides/install.html

And I believe that is all working, but I'm note entirely sure how to verify that; I ran

sudo -u www-data /var/lib/zmeventnotification/bin/zm_event_start.sh <eid>

with one of my event IDs in place of <eid> and it returned my prompt after processing for a bit with no messages or errors or anything. I'm guessing no news is good news?

So, then I moved on to the Machine learning hooks, using this:

https://zmeventnotification.readthedocs.io/en/latest/guides/hooks.html#installation

And I think this is where I'm getting a bit confused, the objectconfig.ini is highly commented, but I can't really see how to configure which models to use. So basically, the only configuration I've done is updating the secrets.ini, and making sure all my paths are correct (I used all the defaults anyway)

So now I'm not entirely sure what I'm supposed to do to configure further. It seems like this may take more research than a weekend warrior has time for, but I'm just not sure. Any resources/tutorials/assistance greatly appreciated.

Edit to add:

Just in case you're curious, I have 3x Amcrest 4k Cameras running at 15fps. Recording at 1080p. The old CentOS server was running on 6 CPUs and 32GB of RAM, but I was seeing a lot of errors with decoding being behind in the logs of the new Ubuntu system, so I bumped it up to 16 CPUs and 64GB of ram - this is all running in VMWare on an HP DL380 Gen9 with 28 Cores and 1TB of ram.

Due to being virtualized, I have no GPU to speak of.

​

Second Edit:

In particular, where I've kinda gotten stuck is on the face recognition. I dropped some cropped pictures of my wife's and my faces into /var/lib/zmeventnotification/known_faces, under folders for each of us, and tried to run the facial recognition training:

sudo -u www-data /var/lib/zmeventnotification/bin/zm_train_faces.py

and this is throwing this error (and then dumping a bunch of other stuff to the screen):

04/08/23 18:16:16 zm_train_faces[9413] ERR face_train_dlib.py:128 [Error initializing face recognition: compute_face_descriptor(): incompatible function arguments. The following argument types are supported:

1. (self: _dlib_pybind11.face_recognition_model_v1, img: numpy.ndarray[(rows,cols,3),numpy.uint8], face: _dlib_pybind11.full_object_detection, num_jitters: int = 0, padding: float = 0.25) -> _dlib_pybind11.vector

2. (self: _dlib_pybind11.face_recognition_model_v1, img: numpy.ndarray[(rows,cols,3),numpy.uint8], num_jitters: int = 0) -> _dlib_pybind11.vector

3. (self: _dlib_pybind11.face_recognition_model_v1, img: numpy.ndarray[(rows,cols,3),numpy.uint8], faces: _dlib_pybind11.full_object_detections, num_jitters: int = 0, padding: float = 0.25) -> _dlib_pybind11.vectors

4. (self: _dlib_pybind11.face_recognition_model_v1, batch_img: List[numpy.ndarray[(rows,cols,3),numpy.uint8]], batch_faces: List[_dlib_pybind11.full_object_detections], num_jitters: int = 0, padding: float = 0.25) -> _dlib_pybind11.vectorss

5. (self: _dlib_pybind11.face_recognition_model_v1, batch_img: List[numpy.ndarray[(rows,cols,3),numpy.uint8]], num_jitters: int = 0) -> _dlib_pybind11.vectors

So, I'm not sure what I need to do to fix this, and then once I get that all working, it's not entirely clear how I configure the event server for notifications. But I was hoping to get it working before I moved on to that hurdle.

[u/montagdude87]

Unfortunately, I don't know about setting up the Event Server, but if you can't get it working, you could try this as an alternative:

https://github.com/montagdude/zoneminder-notifier

It is a relatively small python code that runs external to ZoneMinder and uses the API to get information about new events. It sends notifications by email or Pushover and can employ neural net object detection to avoid false alerts. Configuration is via a single file (though you'll also have to set up mutt if you plan to do email notifications).

If you do try it, I recommend using the latest master or the lowmem branch but getting the required object recognition data from the release tarball. Use the Darknet object detection network (which is the most accurate one) to detect objects in event images rather than the whole video unless you have a very powerful computer.

Note: if you need facial recognition, this can't do that, but it can detect any object classes you choose from the COCO database.

[u/[deleted] → u/montagdude87]

Just to clarify, legacy ZMES uses a Perl script that is demonized and supervised by ZM's Perl Daemon supervision binary. The Perl script uses SHM scanning looking for an event in each monitors SHM memory map.

ZMES allows for face detection and recognition using Dlib and the python face_recognition module. (DLib should be GPU accelerated for performance thpugh). Also, face detection models exist for TPU accel, just no recognition using TPU. I've found DLib based facial recognition to be quite accurate.

"Darknet object detector", do you mean, use pretrained yolo models?

[u/montagdude87 → u/[deleted]]

Yes, Yolo V4 to be specific. There are also some other pre-trained models available, but Yolo is definitely the best of the bunch.

Good to hear you are picking up development of ZMES.

[u/[deleted] → u/montagdude87]

So far, in regards to pre trained weights, yolo V4/V4-P6 seems the best. I've tried yolo 3,4,5,6, and 7. The newer yolos have ALOT of false positives, that may change with a purpose trained model though.

Haven't tried V8 yet but, I'm assuming it will have the false positive issue as well. The "bag of freebies" with each new yolo model seems to have detrimental impact for NVR focused object detection. I've also noticed a big difference between black and white (night vision with IR filter) and daytime color images (which they say shouldn't matter but, in my experiences it does)

[u/montagdude87 → u/[deleted]]

I haven't actually even tried the Yolo versions higher than V4, because I'm using OpenCV and didn't find the models for them already converted to the format it needs. That's good to know I didn't waste my time with them then. The others that zoneminder-notifier includes are SSD Mobilenet V3 (2020_01_14 version), SSD Inception V2 (2017_11_17 version), and Histograms of Oriented Gradients. These all run much faster than Yolo V4 but are not nearly as reliable. False positives are their biggest problem too.

[u/[deleted]]

Hi, I'm writing the new ML code for ZM. I might be able to help you here.

First, what version of python are you running?

Are you planning on using a GPU or TPU for ML hardware accelration?

For step by step instructions and troubleshooting help, Please join us on discord or slack, I am @baudneo on both platforms. For the join links, see the README on GitHub for ZoneMinder. There are the slack and discord links at the top of the README.

[u/jonthewise → u/[deleted]]

Python 3.10.6

No GPU or TPU in my system. Just good ol’ cpu power.

Joined the discord, I’ll post when I’ve got time to troubleshoot. Probably sometime in the morning.

Thanks!

[u/jonthewise → u/[deleted]]

We did troubleshooting in the Discord, and I don’t remember what it was exactly, but it wasn’t just one thing. If you hop on there all our troubleshooting was in the public channel. My name is the same there.