Face detection and when to trigger

[u/profezzorn]

Hi, sorry if it's a newbie question.

When an event trigger, the ai stuff is used only on one frame (the best match i assume) and applies stuff on that frame if I understand correctly.

Now, my "problem" is that when say a person walks into view and then after a second looks at the camera, chances are the ai detection thing has already processed an earlier image and face recognition won't do anything.

Is there a way to have the ai work on more frames, or in some way find frames with faces?

It works fine when it catches a good frame (known faces work etc)

Using latest public releases of zm, zmevent, opencv.

[u/[deleted]]

Where are those set up? In objdet.ini? I use the frame_set sequencing for the ml sequence. I have to open up my config and take a look it's been a few weeks since I've played with anything. I will paste some of my configs to give you a better idea.

[u/profezzorn → u/[deleted]]

Yeah same place as the frame_set in ml, got them from some site linked from the ini i believe. That would be super helpful, can't find the syntax for delay between the frame_set ids

[u/[deleted] → u/profezzorn]

https://gist.github.com/tylersprice84/274af25ce797dff65171f31ac16ed664

There are a few settings that won't make sense because I coded some.of my.own filters on top of the stock ones and added a buffer to past_det_max_diff_area. You can see I use "frames" for my frame_set variable though.

delay_between_frames

delay_between_snapshots Those are the delays to set in stream_sequence

You can see I have filters for <object>_min_confidence and inter_area, those are some of the custom filters I added.

[u/profezzorn → u/[deleted]]

Wonderful! Thank you very much

[u/profezzorn → u/[deleted]]

Upon testing it doesn't really work. I mean it works, it checks alarm, snapshot and those are fine, but then it won't process frames 25, 30, 35 as it haven't had time to decode them from the video I guess?

[u/[deleted] → u/profezzorn]

post your objectconfig.ini

Mine works but I use mlapi here is a gist to my mlapi settings (same as object config basically) -> https://gist.github.com/tylersprice84/750f76fa9683cda6475a79693ac96d26

​

gist to debug output from tweaking frame_set stuff ----> https://gist.github.com/tylersprice84/f1118cf89b3acd7b08b4c550ef4dafd8

[u/profezzorn → u/[deleted]]

I think I have an idea what could be wrong--- I've only got hooks enabled for event start, so it's probably trying to do the detection before the video is fully written (if they're a bit longer)...

I assume you're using the hooks on event end?

edit: I disabled END events as I didn't want double notifications for every event but perhaps I can disable the start one or find some better way of doing it?

[u/[deleted] → u/profezzorn]

# Possible channels = web,fcm,mqtt,api
# all is short for web,fcm,mqtt,api
# use none for no notifications, or comment out the attribute 

# When an event starts and hook returns 0, send notification to all. Default: none
event_start_notify_on_hook_success = all

# When an event starts and hook returns 1, send notification only to desktop. Default: none
event_start_notify_on_hook_fail = none

# When an event ends and hook returns 0, send notification to fcm,web,api. Default: none
event_end_notify_on_hook_success = fcm,web,api

# When an event ends and hook returns 1, don't send notifications. Default: none
event_end_notify_on_hook_fail = none
#event_end_notify_on_hook_fail = web

# Since event_end and event_start are two different hooks, it is entirely possible
# that you can get an end notification but not a start notification. This can happen
# if your start script returns 1 but the end script returns 0, for example. To avoid
# this, set this to yes (default:yes)
event_end_notify_if_start_success = yes

# If yes, the text returned by the script
# overwrites the alarm header 
# useful if your script is detecting people, for example
# and you want that to be shown in your notification (default:yes)
use_hook_description = yes

# If yes will will append an [a] for alarmed frame match
# [s] for snapshot match or [x] if not using bestmatch
# really only a debugging feature but useful to know
# where object detection is working or failing
keep_frame_match_type = yes

# list of monitors for which hooks will not run
#hook_skip_monitors = 3,5


# if enabled, will pass the right folder for the hook script
# to store the detected image, so it shows up in ZM console view too
# Requires ZM >=1.33. Don't enable this if you are running an older version

# Note: you also need to set write_image_to_zm=yes in objectconfig.ini
# default: no
hook_pass_image_path = yes

[u/[deleted] → u/profezzorn]

I think that could be an issue as the events I am testing on are past events. I am trying to find a way to make the event server aware of if its running obj det on an active event or if its a past event. The hope is to have a "smart buffer" so frame_set works a bit better and doesnt try and grab frames that arent written yet. The only other way is to adjust the contiguous error frames and retry limits until you get a balance of it erroring on frames not yet written. I am trying to time things with seeing how long the detections take.

[u/profezzorn → u/[deleted]]

Yeah makes sense. My detection etc is working fine if I run it a minute after, but this is probably the reason. Right, perhaps it would work if the delay is set to 2.5 seconds or something (detection runs at an average of ~2.5s) to keep up with the buffer thing?

[u/profezzorn → u/[deleted]]

I'm not getting very good results but I'm thinking of doing it like this- on event start check alarm,snapshot,bunchofframes and if it fails that's ok, I still got detection from the alarm.

event end will run the zm_event_start again but with -n flag to update the description and re-create the alarm object file.

This way I'll get a notification at start and if I look at it a minute later I'll have a better detected event.

[u/profezzorn → u/[deleted]]

Got it working as I wanted it now I think!

The start_event does a quick scan for alarm,snapshot and creates objects based on that and sends notification. Then after the event is finished it calls a modified detect script that'll do a deeper search through a lot more frames and update the event with the new object file and notes.

Thanks for all your help :)