Zoneminder Record High-res H264 streams 24/7 with low CPU Load

README.txt

>>>>>>>>>>>>>>>>>>>>> BIG UPDATE <<<<<<<<<<<<<<<<<<<<<<<<<<<<
On zoneminder version 1.35.16 and later there is a DecodingEnabled setting for each monitor - this script is therefore of little use anymore



>>>>>>>>>>>>>>>>>>>>> ORIGINAL README <<<<<<<<<<<<<<<<<<<<<<<<<<<<
The original Zoneminder Forum thread about all this is here:

https://forums.zoneminder.com/viewtopic.php?f=9&t=27537&p=107235#p107235

Many thanks to Matt Watts for his improvements.



Notes.

You will need to install mediainfo (apt-get install mediainfo).

By default, zmrecord is set to save the continuous recordings in 600
second (10 minute) chunks.  Change the value of RECORD_TIME at the top
of zmrecord.sh to alter this.

Test

sudo zmrecord.sh <Monitor_Id>

and you should see recording files appearing in <Monitor Storage Path>/<Monitor-ID>

eg if Monitor 7's Storage path is /mnt/bigraid/zoneminder/recordings the
mp4 files should appear in:

/mnt/bigraid/zoneminder/recordings/events/7/<event-id>/...


Installation 

sudo cp zmrecord.sh /opt/zmrecord.sh
sudo chmod 0554 /opt/zmrecord.sh
sudo cp [email protected] /lib/systemd/system

Then for the monitors you wish to record do:

systemctl enable zmrecord@<MonitorID..>.service
systemctl start zmrecord@<MonitorID..>.service

eg for Monitors 2,3 and 6

systemctl enable [email protected] [email protected] [email protected]
systemctl start [email protected] [email protected] [email protected]

zmrecord.sh

#!/bin/bash

# Original script posted here https://forums.zoneminder.com/viewtopic.php?f=9&t=27537  by russell_i_brown 

# Install to /opt/zmrecord.sh

#       Grab a video stream from an IP camera into a temp file.
#       Inject an 'Events' record into the ZM Database.
#       Create the directory in the zm storage area
#       Move the mp4 to the right place
#
#       Call with the Camera Id number as arg1:
#
#       zmrecord.sh 12
#
#
#       You might have to install mediainfo which I use to grab
#       the frame count.
#
#       Version 0.1     -  russell_i_brown	18/ 9/18        First hack
#       Version 0.2     -  russell_i_brown	19/ 9/18        Read monitor guff from DB
#       Version 0.3     -  russell_i_brown	19/ 9/18        Set StorageId in DB
#       Version 0.4     -  matt_watts		10/12/19        Use FFMPEG segment output to ensure frames are not missed between segments
#       Version 0.5	    -  matt_watts		12/12/19	    Move video segments immediatley into database / storage tree while live and keep db stats updated
#	    Version 0.6     -  matt_watts		16/12/19	    
#                                - Replace FFMPEG list output with simple folder polling allowing some crash/script kill recovery
#								- Use FFMPEG empty_moov flag to allow playing in-recording segment in the web ui and preventing
#								  corrupt video on crash/script kill
#								- Continuous event database updates as segment is being wrote
#								- Log some errors and events direct to Zoneminders DB Log
#       Version 0.7     -  russell_i_brown	31/01/20        Use Event Prefix as requested by tuxmos
#								Handle Monitor with a StorageId of 0 by defaulting to 1 if blank  (ZM 1.34 issue?)
#								Calculate the Bandwidth so the ZM console looks sane.
#       Version 0.8     -  ZM Update has changes Events.StartTime to Events.StartDateTime and Events.EndTime to Events.EndDate time
#                          otherwise same as 0.7

#########################################
# 	Recording chunk time in seconds
#	This script will record for this number of seconds at a time
#	600 (10 minutes) is what I use.
RECORD_TIME=600
#
#########################################

trap "kill 0" SIGINT

# ZM Log
function zm_log() {
	CODE="$1"
	CODEVAL="$2"
	MESSAGE="`echo $3|tr -d '"'`"
	TIMESTAMP=`date +%s`
	SQL="INSERT INTO Logs ( Component,File,ServerId,Pid,Level,Code,Line,TimeKey, Message ) 
	     VALUES           ( 'zmrecord.sh', 'zmrecord.sh', NULL, $$, $CODEVAL, \"$CODE\", 0, $TIMESTAMP, \"$MESSAGE\" );"
	echo $SQL | mysql -NBqr zm
}


function log_inf() {
	zm_log "INF" "2" "$@"
}

function log_err() {
	zm_log "ERR" "-2" "$@"	
}

# Run the ffmpeg process and keep stats file up to date while it runs - pass ffmpeg errors to zm log
function run_ffmpeg() {
	MONITOR_ID="$1"
	CAM_URL="$2"
	OUTPUT_STATS_FN="$3"
	RECTIME="$4"
	OUTPUT_FN="$5"
	
	rm "$OUTPUT_STATS_FN".errorpipe 2>/dev/null
	mkfifo "$OUTPUT_STATS_FN".errorpipe
	rm "$OUTPUT_STATS_FN".statpipe 2>/dev/null
	mkfifo "$OUTPUT_STATS_FN".statpipe

	cat "$OUTPUT_STATS_FN".errorpipe | \
	while read ERR; do
		log_err "Monitor: $MONITOR_ID FFMPEG: $ERR"
	done &
	PID_ERR_READ=$!

	cat "$OUTPUT_STATS_FN".statpipe | \
	while read LN; do
		PARM="`echo $LN | cut -d '=' -f1`"
                VAL="`echo $LN | cut -d '=' -f2`"
                if [ "$PARM" = "fps" ]; then FFMPEG_FPS=$VAL; fi
                if [ "$PARM" = "bitrate" ]; then FFMPEG_BITRATE=$VAL; fi
                if [ "$PARM" = "frame" ]; then FFMPEG_FRAME=$VAL; fi
                echo "`date +%s` $FFMPEG_FPS $FFMPEG_BITRATE $FFMPEG_FRAME" > "$OUTPUT_STATS_FN"
	done &
	PID_STAT_READ=$!
	
	ffmpeg -y -loglevel error -i "$CAM_URL" \
		-c:v copy -f segment \
		-c:a aac \
               -segment_time $RECTIME -segment_atclocktime 1 \
	       -segment_format_options movflags=empty_moov \
	       -strftime 1 "$OUTPUT_FN" \
	       -progress - -nostats 2>"$OUTPUT_STATS_FN".errorpipe 1>"$OUTPUT_STATS_FN".statpipe &
	PID_FFMPEG=$!

	# Wait for exit signal or child pid death
	RUNNING=1
	trap 'RUNNING=0' EXIT
	trap 'RUNNING=0' INT
	trap 'RUNNING=0' TERM
	while [ "$RUNNING" = "1" ]; do
		if ! kill -0 "$PID_FFMPEG" 2>/dev/null; then RUNNING=0; fi
		if ! kill -0 "$PID_ERR_READ" 2>/dev/null; then RUNNING=0; fi
		if ! kill -0 "$PID_STAT_READ" 2>/dev/null; then RUNNING=0; fi
		sleep 1
	done
	# Shutdown the subprocess - we want to make really really sure that FFMPEG has indeed died before leaving!
	# its not 100% perfect and the output from netcams can really freeze it occassionaly to the point SIGINT
	# does not work and the subprocess exiting will leave a zombie ffmpeg behind
	# if it does not shutdown in 4 seconds force kill it
	kill $PID_FFMPEG $PID_ERR_READ $PID_STAT_READ 2>/dev/null
	KILL_TIMEOUT=0
	RUNNING=1
	trap 'RUNNING=1' EXIT
	trap 'RUNNING=1' INT
	trap 'RUNNING=1' TERM
	while [ "$RUNNING" = "1" ]; do
		KILL_TIMEOUT=$((KILL_TIMEOUT+1))
		if ! kill -0 "$PID_FFMPEG" 2>/dev/null; then RUNNING=0; fi
		if [ "$KILL_TIMEOUT" -gt 4 ]; then
			kill -9 $PID_FFMPEG 2>/dev/null
		fi
		sleep 1
	done
	rm "$OUTPUT_STATS_FN".errorpipe 2>/dev/null
	rm "$OUTPUT_STATS_FN".statpipe 2>/dev/null
	
	log_err Monitor: $MONITOR_ID FFMPEG Has Terminated
}

# Scan the tmp folder for <timestamp>.mp4 files
# Move them into the correct tree and create a DB entry ensuring that on premature script termination what data was
# captured so far is in place
# leave an eventid and symbolic link in the tmp folder 
# the eventid keeps the info for this script to poll the video in its new location
# the symbolic link allows ffmpeg to re-open the file. FFMPEG does this when quit earlier than expected to add the moov atom (faststart)
function addIncompleteVideosToDB() {
	TMPFOLDER="$1"
        MONITOR_ID="$2"
        STORAGEID="$3"
        RECORD_TIME="$4"
        WIDTH="$5"
        HEIGHT="$6"
	STORE_PATH="$7"
	EVENT_PREFIX="$8"
	# iterate all new mp4 files found
	for FILEPATH in "$TMPFOLDER"/*.mp4; do
		[ -f "$FILEPATH" ] || continue
		# get beginning timestamp from ffmpeg created filename
		FILENAME=${FILEPATH##*/}
		TIMESTAMP=${FILENAME%%.mp4}
		# create new db entry for video segment
		SQL="	INSERT INTO Events 
			(	MonitorId,StorageId,Name,Cause,StartDateTime,EndDateTime,Width,Height,Length,Frames,
				Videoed,DiskSpace,Scheme,StateId,AlarmFrames,SaveJPEGs,Notes)
			VALUES
			(	$MONITOR_ID,\"$STORAGEID\",\"New Event\",\"Continuous\",FROM_UNIXTIME($TIMESTAMP),
				NULL,$WIDTH,$HEIGHT,1,1,1,1,\"Medium\",1,0,0,\"Injected from ffmpeg capture\");
                     	SET @last_id = LAST_INSERT_ID();
                     	UPDATE Events SET Name=CONCAT(\"$EVENT_PREFIX\",@last_id),DefaultVideo=CONCAT(@last_id,\"-video.mp4\") where Id=@last_id;
                     	SELECT @last_id;"
		THIS_ID=`echo $SQL | mysql -NBqr zm`
		# if mysql returned a new eventid then move the video into the correct place
		# ffmpeg will still retain its handle to the file and continue writing
		# leave an eventid reference to the video in the tmp folder - this will allow 
		# polling of the file until it needs completing 
		if ! [ "$THIS_ID" -le 0 ]; then
			DATE_PATH=$STORE_PATH/`date -d @$TIMESTAMP --rfc-3339=date`
			THIS_PATH=$DATE_PATH/$THIS_ID
			NEW_VIDEO_PATH="$THIS_PATH"/$THIS_ID-video.mp4
			DESCFILE="$TMPFOLDER"/"$TIMESTAMP".eventid
	                mkdir -p $THIS_PATH
        	        mv -f "$FILEPATH" "$NEW_VIDEO_PATH"
			chown www-data:www-data "$NEW_VIDEO_PATH" "$THIS_PATH" "$DATE_PATH"
			echo "$THIS_PATH/$THIS_ID-video.mp4" > "$DESCFILE"
			log_inf "Monitor: $MONITOR_ID - New segment : $THIS_PATH/$THIS_ID-video.mp4 Event $THIS_ID"
		fi
	done
}

# Scan the tmp folder for <timestamp>.eventid files
# By checking which ones point to an mp4 that is not open ( by ffmpeg ) we can determine
# the segment is complete and close it. This can also pick up stray recordings after say a PC crash
function addCompleteVideosToDB() {
	TMPFOLDER="$1"
        MONITOR_ID="$2"
        STORAGEID="$3"
        RECORD_TIME="$4"
        WIDTH="$5"
        HEIGHT="$6"
	STORE_PATH="$7"
	# iterate all eventid files in the tmp folder
	for EVENTID_FILEPATH in "$TMPFOLDER"/*.eventid; do
		[ -f "$EVENTID_FILEPATH" ] || continue		
		EVENTID_FILENAME=${EVENTID_FILEPATH##*/}
		TIMESTAMP=${EVENTID_FILENAME%%.eventid}
		[[ $TIMESTAMP =~ ^[0-9]+$ ]] || continue
		VIDEO_PATH="`cat "$EVENTID_FILEPATH"`"
		VIDEO_FILENAME=${VIDEO_PATH##*/}
		EVENTID=${VIDEO_FILENAME%%-video.mp4}
		# if the video is not longer open for writing remove the eventid file, this will be the last update
		LASTUPDATE=0
		if [ "`fuser "$VIDEO_PATH" 2>/dev/null`" = "" ]; then
			log_inf "Monitor: $MONITOR_ID Event : $EVENTID - Segment complete : $VIDEO_PATH"
			rm "$EVENTID_FILEPATH"
			LASTUPDATE=1
		fi
		# get video stats and validate them
		# only spit out actual errors if this is the final update (video finished), an in-writing video
		# is likley to not be stat ready early in its life
		FRAMES=`mediainfo --Output="Video;%FrameCount%" "$VIDEO_PATH"`
		if ! [[ $FRAMES =~ ^[0-9]+$ ]]; then
			if [ $LASTUPDATE -eq 1 ]; then
				log_err "Monitor: $MONITOR_ID Event : $EVENTID ERROR: mediainfo --Output=Video;%FrameCount% $VIDEO_PATH returned '$FRAMES', probable corrupt video"
			fi
			FRAMES=1
		fi
		FILESIZE=`stat -c%s "$VIDEO_PATH"`
		if ! [[ $FILESIZE =~ ^[0-9]+$ ]]; then
			if [ $LASTUPDATE -eq 1 ]; then
				log_err "Monitor: $MONITOR_ID Event : $EVENTID ERROR: stat -c%s $VIDEO_PATH returned '$FILESIZE', integer expecected"
			fi
			FILESIZE=1
		fi
		END_TIMESTAMP=`stat -c %Y "$VIDEO_PATH"`
		if ! [[ $END_TIMESTAMP =~ ^[0-9]+$ ]]; then
			if [ $LASTUPDATE -eq 1 ]; then
				log_err "Monitor: $MONITOR_ID Event : $EVENTID ERROR: stat -c %Y $VIDEO_PATH returned '$END_TIMESTAMP', integer expecected"
			fi
			LENGTH=RECORD_TIME
			END_TIMESTAMP=$((TIMESTAMP+1))
		fi
		# update the db with stats
		SQL="	UPDATE Events SET 
				Frames=$FRAMES,
				EndDateTime=FROM_UNIXTIME($END_TIMESTAMP),
				DiskSpace=$FILESIZE
				WHERE Id=$EVENTID;
			UPDATE Events SET Length=UNIX_TIMESTAMP(EndDateTime)-UNIX_TIMESTAMP(StartDateTime) 
			WHERE StartDateTime IS NOT NULL AND EndDateTime IS NOT NULL AND Id=$EVENTID;"
		echo $SQL | mysql -NBqr zm
	done
}

# For Bandwidth Calc
LAST_FPS_TIME=0
LAST_FILESIZE=0
LAST_BW=0

# Keep the monitor status up to date so fps etc can be viewed in the zm console
function updateMonitorStatus() {
	MONITOR_ID="$1"
	STATS_FILE="$2"
	SHUTTINGDOWN="$3"
	STATUS="NotRunning"
	FPS=0
	BW=0
	if ! [ "$SHUTTINGDOWN" = "1" ]; then
		if [ -f "$STATS_FILE" ]; then
			IFS=' ' read -r TS FPS BW FRAME <<< `cat "$STATS_FILE"`
			if  [ -z "${TS//[0-9]}" ] && [ -n "$TS" ]; then
				AGE=$(($(date +%s)-TS))
			else
				AGE=1000
			fi
			if [ "$AGE" -lt 4 ]; then
				STATUS="Connected"
			fi
		fi
	fi
	# Calc CaptureBandwith in the same way as ZM in zm_monitor.cpp:
	# 	unsigned int new_capture_bandwidth = (new_camera_bytes - last_camera_bytes)/(now-last_fps_time);
	# Unfortunately, BASH can't trap a divide by zero so we have to check
	# Get Filesize for Bandwidth calc.  Ffmpeg buffers quite a lot so see if the file has changed
	# size before calculating the Bandwidth.
	FILESIZE=`stat -c%s "$NEW_VIDEO_PATH" 2>/dev/null`
	if [ ${FILESIZE:-0} -gt ${LAST_FILESIZE:-0} ]
	then
		ELAPSED_SECONDS=$(($TS - $LAST_FPS_TIME))
		if [ $ELAPSED_SECONDS > 0 ]
		then
			FILESIZE_INCREASE=$(( ${FILESIZE:-0} - ${LAST_FILESIZE:-0} ))
			LAST_BW=$(( $FILESIZE_INCREASE / $ELAPSED_SECONDS ))
			LAST_FPS_TIME=$TS
			LAST_FILESIZE=$FILESIZE
		fi
	fi
	if [ "$FPS" == "" ]; then
		FPS=0
	fi
	SQL="	REPLACE INTO Monitor_Status (MonitorId, Status, CaptureFPS, CaptureBandwidth) 
		VALUES 
		('$MONITOR_ID','$STATUS', '$FPS', ${LAST_BW:-0})"
	echo $SQL | mysql -NBqr zm
}
# Main function - 
function zmRecord() {
	MONITOR_ID=$1
	# Read Monitor Info From DB
	IFS=$'\t'
	read -r WIDTH HEIGHT READPATH STORAGEID EVENT_PREFIX <<< `mysql -NBqr zm -e "select Width,Height,Path,StorageId,EventPrefix from Monitors where Id=$MONITOR_ID"`
	read -r STORAGE <<< `mysql -NBqr zm -e "select Path from Storage where Id=$STORAGEID"`
	# Hmmmm.... on a fresh install of ZM 1.34, the Monitor has a StorageId of 0 but the Storage table
	# starts at Id 1.  Handle a blank Storage Path.
	if [ -z "$STORAGE" -a $STORAGEID -eq 0 ]
	then
		read -r STORAGE <<< `mysql -NBqr zm -e "select Path from Storage where Id=1"`
	fi
	STORE_PATH=$STORAGE/$MONITOR_ID
	if [ -z "$READPATH" -o -z "$STORE_PATH" -o -z "$STORAGE" ]; then
		log_err "$0 Monitor Data Error. Got: Width $WIDTH Height $HEIGHT ReadPath $READPATH StorePath $STORE_PATH"
		exit 1
	fi
	# create the working folders & files
	# stats txt is constantly updated and placed in ram ( /dev/shm )
	TMPFOLDER="$STORE_PATH/../$MONITOR_ID-zmrecordtmp"
        FFMPEG_STAT_OUTPUT_FILE=/dev/shm/zmrecord-m$MONITOR_ID-stats.txt
	FFMPEG_OUTPUT_FILENAME="$TMPFOLDER/%s.mp4"
	mkdir -p "$TMPFOLDER"
	rm "$FFMPEG_STAT_OUTPUT_FILE" 2>/dev/null
	touch "$FFMPEG_STAT_OUTPUT_FILE"
	chmod a+r "$FFMPEG_STAT_OUTPUT_FILE"
	# Main loop exit trap triggered by EXIT / INT / TERM signals
	RUNNING=1
	trap 'RUNNING=0' EXIT
	trap 'RUNNING=0' INT
	trap 'RUNNING=0' TERM
	# Main loop - start the ffmpeg processes and monitor
	# regularly run updates to check ffmpegs video and stat output and transfer
	# to zm database / file tree
	log_inf "Monitor: $MONITOR_ID Starting zmrecord.sh"
	FFMPEG_PID=-100
	while [ "$RUNNING" = "1" ]; do
		# testing this script one time my tmp folder vanished ( zm cleanup? ) 
		# and this script was unable to recover so I added this
		mkdir -p "$TMPFOLDER"
		# start subprocesses / restart them if they stopped
		if ! kill -0 "$FFMPEG_PID" 2>/dev/null; then
			log_inf "Monitor: $MONITOR_ID Starting FFMPEG process"
			run_ffmpeg "$MONITOR_ID" "$READPATH" "$FFMPEG_STAT_OUTPUT_FILE" "$RECORD_TIME" "$FFMPEG_OUTPUT_FILENAME" &
			FFMPEG_PID=$!
		fi
		# check ffmpeg is actually working by monitoring the frame count
		FFMPEG_STATS_FRAMES=`cat "$FFMPEG_STAT_OUTPUT_FILE" | cut -d " " -f4`
		if [ "$PREV_FFMPEG_STATS_FRAMES" = "$FFMPEG_STATS_FRAMES" ]; then
			if [ "$PREV_FFMPEG_STATS_FRAMES_SAMECOUNT" = "" ]; then
				PREV_FFMPEG_STATS_FRAMES_SAMECOUNT=0
			fi
			PREV_FFMPEG_STATS_FRAMES_SAMECOUNT=$((PREV_FFMPEG_STATS_FRAMES_SAMECOUNT+1))
		else
			PREV_FFMPEG_STATS_FRAMES_SAMECOUNT=0
		fi
		PREV_FFMPEG_STATS_FRAMES="$FFMPEG_STATS_FRAMES"
		if [ "$PREV_FFMPEG_STATS_FRAMES_SAMECOUNT" -gt 5 ]; then
			if kill -0 "$FFMPEG_PID" 2>/dev/null; then
				log_err "Monitor: $MONITOR_ID FFMPEG has stalled - killing"
				kill $FFMPEG_PID
				PREV_FFMPEG_STATS_FRAMES_SAMECOUNT=0
			fi
		fi
		# run the video / db update polling functions
		addCompleteVideosToDB "$TMPFOLDER" "$MONITOR_ID" "$STORAGEID" "$RECORD_TIME" "$WIDTH" "$HEIGHT" "$STORE_PATH" "$EVENT_PREFIX"
		sleep 4
		updateMonitorStatus "$MONITOR_ID" "$FFMPEG_STAT_OUTPUT_FILE" 0
		addIncompleteVideosToDB "$TMPFOLDER" "$MONITOR_ID" "$STORAGEID" "$RECORD_TIME" "$WIDTH" "$HEIGHT" "$STORE_PATH"	"$EVENT_PREFIX"
	done
	# Finish up
	if ! kill -0 "$FFMPEG_PID" 2>/dev/null; then
		kill "$FFMPEG_PID" 2>/dev/null
	fi
	addCompleteVideosToDB "$TMPFOLDER" "$MONITOR_ID" "$STORAGEID" "$RECORD_TIME" "$WIDTH" "$HEIGHT" "$STORE_PATH" "$EVENT_PREFIX"
	updateMonitorStatus "$MONITOR_ID" "$FFMPEG_STAT_OUTPUT_FILE" 1
	log_inf "Monitor: $MONITOR_ID Ended zmrecord.sh"
}
if [ $# -ne 1 ]; then
        echo "Syntax: $0 <Monitor Id>"
        exit 1
fi
MONITOR_ID=$1
zmRecord $MONITOR_ID

[email protected]

#
#  Fire up zmrecord for multiple cameras.
#  
#  Enable with:
#
#       systemctl enable zmrecord@<MonitorId>.service
#
#  the recording should start
#
#  Start each recording process automatically at boot with:
#  
#       systemctl start zmrecord@<MonitorId>.service
#
#

[Unit]
Description=Record Stream from Camera %I
After=zoneminder.service

[Service]
Type=simple
ExecStart=/opt/zmrecord.sh %I
Restart=on-failure

[Install]
WantedBy=multi-user.target

Hi Matt,

Thanks for making those improvements - very nice.

I've just upgrade my ZM to 1.34 and revisited this, noticed your improvements and a request to honour EventPrefix on the forum so I forked your version, made the changes for EventPrefix, added some Bandwidth calcs just to keep the ZM console looking sane and handled a curious 1.34 issue on Storage Areas Ids.

Anyway, my version (0.7) is on the forked copy: https://gist.github.com/ruffle-b/5ab39b9b6f456d694f7665678698bf5f

I don't think Gist supports formal pulls but feel free to merge my changes and call your copy the master... or I can leave my fork live; not bothered which.

Thank you - I have merged the changes into this one. I dont think it will hurt to keep your fork live as I do not go on github ( or the PC ) much when all the stuff is working so well 💯
Again thanks for putting this all forward - the Zoneminder server is purring along at about 40% CPU with 3 HD streams managed by this script and their respective sub-streams being motion analysed traditionally with zoneminder. It was completley saturated before! Looking foward to even more CPU savings to come by using the cameras own motion detection to trigger Zoneminder.
I am now in a position to replace this old high wattage CPU ( Q6600 ) with a nice quiet low wattage APU which is kind of where we all want to be with a 24/7 running PC :-)