A camera that detects motion should be able to sound a speaker, and the
speaker is rarely the camera: it is a separate device with its own address,
credentials, stream and Controls entry. Model it as a monitor and let a
monitor's alarm drive actions on other monitors.
Add Monitors.DeviceClass enum('Camera','Speaker'). This is what the device
is, as distinct from Type, which selects the capture backend - an IP speaker
still captures over Ffmpeg like any other RTSP device, so Type could not
carry the distinction.
Add the MonitorActions table: MonitorId is the monitor that triggers,
TargetMonitorId the device acted on, and the two are frequently different.
TriggerOn covers EventStart, EventEnd, Alarm and Manual.
Which actions a device is offered is decided by its Controls row - CanLight,
CanIndicatorLight, CanAudioPlay - so a device can only be asked to do what it
has been measured to do. The editor filters on this and the save path
re-checks it, because the request is not to be trusted.
Execution goes straight to the target's zmcontrol socket rather than forking
zmcontrol.pl per action: the daemon already accepts a line of JSON there, and
it is the same path the control panel uses. ActionCommandName maps the DB
enum onto a method name as a whitelist, so nothing out of the database
reaches the control daemon uninspected. Actions are fire-and-forget - a
speaker that is offline is logged and skipped, never allowed to hold up event
handling.
Alarm actions fire only on the genuine entry into alarm, not on the
ALERT->ALARM re-entry, which would re-sound a speaker within one incident.
EventEnd runs on the calling thread before the event is handed to the closing
thread, which does not capture `this`.
Manual actions appear as buttons on the watch page, and are the reason the
control panel is now shown for a monitor that has actions but no control of
its own. Firing one sends only the action id; the command and target are
rebuilt server side, and Control rights are required on the target device and
not merely on the monitor the action hangs off.
Also add --file to zmcontrol.pl, without which audioPlay could not be driven
from the command line. The web path was unaffected as it bypasses GetOptions.
Tests: tests/zm_monitor_action.cpp covers the command whitelist, the message
format (including that file id 0 is a real id and that a stale AudioFile is
never passed to a command that takes none), trigger names, and that every
value of the ActionType enum maps to a command.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
(cherry picked from commit b561341e988af69c3a46db854da410309f7cfa82)
An IP speaker is controllable but has none of the capabilities the Controls
table models: nothing moves, focuses or lights up. It plays a sound file it
already holds, selected by a numeric id, and carries an output volume.
Add CanAudioPlay, MinAudioFile, MaxAudioFile and CanAudioVolume to Controls,
with migration zm_update-1.39.21.sql. CanAudioPlay renders one button per
sound id plus a stop button; MinAudioFile/MaxAudioFile bound that range;
CanAudioVolume renders the volume pair. The sound id travels in the command
name (audioPlay12), the way presets already do, because a control button has
no way to attach a separate parameter.
Add ZoneMinder::Control::IPSpeaker driving the device's own HTTP interface,
and a Controls entry for it. Playback goes over the vendor interface rather
than ONVIF because the ONVIF audio output service on this firmware only
describes the output and offers no way to start a stored file.
Measured against a device reporting ONVIF Manufacturer "IPSpeaker" and
firmware CS20-V3.3.45N:
- File ids fall in two windows, 10-14 built-in and 20-30 operator uploads.
Anything else is refused with result -2, and an id in a window with no file
uploaded with result -3, so valid_fileid refuses only the former: an empty
slot is a legitimate id the operator may fill later.
- config=audio.set replaces the whole audio section, so a partial write
reverts the microphone, codec list and echo-cancellation settings. Every
volume change reads the section and echoes it back with the new level.
audio.get also returns outmute, which audio.set rejects, so it is excluded
from the field list.
- The volume reads back and round-trips, but the device's RTSP audio is a
pre-volume tap of the playback signal: it is unchanged with outvolume at 0,
so it cannot confirm the loudspeaker's acoustic output. CanAudioVolume is
set on the strength of the setting persisting, and both the POD and the
seed row say the acoustic effect is unconfirmed.
The seed row covers the built-in window only, because a fresh device has
nothing uploaded and the firmware refuses an empty slot.
Adding columns to Controls extends the 43 positional INSERTs in
db/controls.sql, which carry no column list and so must match the column
count exactly.
Tests: scripts/ZoneMinder/t/ip_speaker.t, 32 assertions covering file id
windows, volume clamping and stepping, request building and the whole-section
write. All pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
(cherry picked from commit d07e552a3c58373f7f0ccebf9aa28adb4d51baa3)
The original fix replaced the hardcoded 000 token in getCamParams and
_setImaging, but get_config/set_config landed upstream afterwards and
carry three more imaging requests that still address VideoSource 000:
the ImagingSettings and ImagingOptions entries in %config_types, and the
SetImagingSettings body in set_config.
Cameras that number their sources differently reject all three. On an
AMLINK AL5M-T5171EW the video source is 00000 and a request for 000 comes
back as "The requested VideoSource does not exist.", so the imaging half
of the config API is unusable on those cameras even with the earlier fix
applied.
The query bodies now carry a __VIDEO_SOURCE_TOKEN__ placeholder, matching
how __PROFILE_TOKEN__ already works, and get_config substitutes it only
when the body contains it so unrelated categories do not pay for a
GetVideoSources round trip. set_config calls _video_source_token()
directly, which is cached after the first lookup.
The added tests assert on the module source because %config_types is a
file-scoped lexical. That is deliberate: the failure this guards against
is a new imaging call being written with a literal token again, which is
exactly how get_config/set_config reintroduced the bug.
Verified against both live AMLINK units: GetVideoSources returns 00000 on
each, and GetImagingSettings answers for 00000 while faulting for 000.
Perl suite 13 files / 191 assertions pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
getCamParams() and _setImaging() addressed the imaging service with a
literal VideoSourceToken of 000. That token is a different namespace from
the media ProfileToken carried in ControlDevice, so it could not be
configured around: on an AMLINK AL5M-T5171EW the media profiles are
MediaProfile00000 while the video source is 00000, and the camera answers
a request for 000 with "The requested VideoSource does not exist."
Brightness (Iris) and Contrast (White) control were unusable on any camera
that does not happen to number its first video source 000.
Read the token from GetVideoSources on first use and cache it. The
fallback to 000 is applied per call rather than cached, so a camera that
is unreachable when the control daemon starts gets another chance instead
of being pinned to the wrong token for the life of the daemon.
Parsing is split out as video_source_token_from_xml() so it can be tested
without a camera, matching the pattern used by the HikVision light
helpers.
Verified against a live AMLINK AL5M-T5171EW: token discovered as 00000,
irisAbsOpen(step 10) moved Brightness 50 -> 60, restored to 50.
(cherry picked from commit 7f1935d69d58b3e2bbf71a54fe11b1a6b4bc30af)
Object::save() returns the error string on failure and '' on success, so
`if ($manufacturer->save())` took the success branch only when the save had
failed. ManufacturerId and ModelId were therefore assigned from an unsaved
object and left unset whenever the save actually worked.
Invert the test, capture the error string, and log it. Model.pm needs
ZoneMinder::Logger imported for Error().
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both loops called $dbh->ping() and discarded the result. ping() only tests
the connection; DBD::mysql does not reconnect on its own since
mysql_auto_reconnect is off. zmupdate's update-check loop sleeps 3600s
between iterations, so on a server with wait_timeout below an hour the
handle is already closed on wake-up and the following zmDbDo fails with
"The client was disconnected by the server because of inactivity".
zmDbConnect() already pings and reconnects when needed, and assigns the
package global $dbh that zmDbDo uses, so call that instead. zmupdate keeps
passing mysql_multi_statements so a reconnect matches the original handle.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Upstream's schema-drift fix landed db/zm_update-1.39.25.sql and bumped
version.txt to 1.39.25 while the trigger consolidation was using the same
number for db/zm_update-1.39.25.sql.in.
Git does not see this as a conflict -- the two files have different names in
the source tree -- but configure_file generates the .sql.in into
zm_update-1.39.25.sql, which is the same install target as the tracked .sql.
Confirmed by staging an install off the merge: the installed
zm_update-1.39.25.sql was upstream's column reconciliation and the trigger
consolidation was simply absent, with no error anywhere. An upgrade would
have applied one of the two and silently skipped the other.
Renumbered to 1.39.26, version.txt to match, and the two zmstats.pl.in
comments that name the migration updated. Upstream's 1.39.25 is untouched.
Verified by staging an install again: both files are now present and hold
what they should. The migration was re-run end to end at the new number
against a scratch database -- drops the eight cascade triggers, leaves four,
and the 29 trigger assertions pass afterwards.
ai_server carries the old 1.39.25 numbering and will need the same treatment
when master is next merged into it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
The guard skipped the delay when a filter was named. zmpkg starts this as
"zmdc.pl start zmfilter.pl --filter_id=N --daemon", so it always names one --
the boot path never waited, and the only invocation that did was a hand-run
scan of every filter. Exactly backwards from what the delay is for.
--daemon is what marks the unattended case, so gate on that instead, still
skipping when there is a terminal on stdin.
Measured with Filter::Execute stubbed to return no events, so nothing was
deleted or emailed, timing the first call to it:
invocation before after
--filter_id=1 --daemon (zmpkg) 0.1s 5.1s
--filter_id=1 (hand-run) 0.1s 0.1s
no args, scan all filters 5.1s 0.1s
So filters now start 5 seconds later at boot, which is the point of the
constant, and a hand-run returns immediately.
Also sets zmstats' START_DELAY to 5. The previous commit's message said it
did this and it did not: the edit was written against "=> 5" when the file
said "=> 30", so it silently matched nothing while the interactive skip
beside it landed. Verified this time by running the generated script both
ways -- non-interactive logs "starting in 5 seconds", interactive logs
nothing and proceeds.
That run also closes the gap the previous commit noted: startedInteractively
resolves at runtime in the generated zmstats.pl. It had only been checked
with perl -Tc, which does not catch an unimported sub called with parens --
an earlier run against a stale module path failed with exactly that error and
sent me looking.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
zmwatch slept 30 seconds before its first pass. The reason is real: when it
starts, zmc has not necessarily created its shared memory or written a
heartbeat, and neither state is distinguishable from a camera that has died,
so the first pass would restart every monitor.
A fixed sleep is the wrong shape for that. It is not derived from anything --
ZM_WATCH_CHECK_INTERVAL is 10 and ZM_WATCH_MAX_DELAY is 45, so the delay was
three check intervals and shorter than the staleness threshold it was
standing in for. On a busy host with many cameras, or one camera slow to
answer, 30 seconds is not enough and the fleet gets restarted anyway. It also
blocks every other check, and makes running zmwatch by hand a 30 second wait.
Each monitor now gets until ZM_WATCH_MAX_DELAY after zmwatch started -- the
same threshold used to judge a heartbeat stale -- to appear, and only if we
have never yet seen it healthy. Once seen healthy it is judged immediately, so
a camera that dies later is caught exactly as fast as before, and every check
other than the restart runs from the first pass.
Verified against the two monitors on a machine with no daemons running, which
is the boot state, with Monitor::control stubbed so nothing was actually
started. Before: both monitors restarted on the first pass after the sleep,
and again every 10 seconds. After: no restart for 45 seconds, then both
restarted on the first pass past the grace, at 50 seconds. So a monitor that
never comes up is still caught, ~20 seconds later than before at boot and at
the same speed as before thereafter.
Also adds ZoneMinder::General::startedInteractively and uses it to skip the
start delay in zmstats.pl and zmfilter.pl when a person ran them. zmdc.pl
reopens STDIN on /dev/null for everything it starts, so a terminal on STDIN
reliably means a hand-run; zmtelemetry.pl already relies on this. A run from
cron or a unit file has no terminal either and still waits, which is the right
way round. zmstats keeps a delay because it is polite while zmc and zma are
competing for the machine, but nothing in it races with startup -- it connects
to the database first and has its own reconnect loop -- so the value is now 5
rather than 30.
Note zmfilter's existing guard is inverted with respect to that intent:
zmpkg starts it as "--filter_id=N --daemon", and the guard skips the delay
when a filter is named, so the daemon path never waited and only a manual
scan-all did. Left as it is here beyond adding the interactive case; worth a
separate look.
The zmstats and zmfilter changes are not runtime tested. Running either
against a live install prunes and rewrites rows, and there was no throwaway
install to point them at; startedInteractively is verified directly in both
directions, and all three generated scripts pass perl -Tc.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
setContrast takes 'constrast'. Spelling the parameter 'contrast', as
the field is named everywhere else including in getImageSetting's own
answer, is accepted with result=0, leaves the real parameter unset and
applies 0 for it. Contrast 0 is a black picture, so set_config would
have blacked out any camera whose contrast a template touched, while
reporting success.
Found by doing it to a live FI9853EP: the camera kept serving video
with a correct timestamp overlay and no error anywhere, and the only
sign was getImageSetting answering contrast 0 where an untouched
camera of the same model answered 50.
field_set entries are now [command, parameter] pairs rather than a bare
command with the parameter assumed from the field name, since the
firmware gives no indication when it is handed a name it does not read.
The same check against an untouched camera clears brightness, hue,
saturation and sharpness: those were written with their documented
names during the same session and kept their values.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
set_time wrote unconditionally. Because the offset it corrects only
moves at a daylight saving change, running it from cron - which is the
point of it being idempotent - meant thousands of writes a year to the
camera's flash for the two that matter.
Read the section first and return success without writing when every
field we would set already holds the wanted value. Measured against an
FI9853EP: a repeat run now issues one getSystemTime and no write, while
a moved offset still reads, merges and writes once.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
FI9821W_Y2k, FI9831W and FOSCAMR2C all talk to the same
/cgi-bin/CGIProxy.fcgi endpoint that FoscamHD now implements settings
for, so reparent them onto it instead of ZoneMinder::Control.
Method resolution keeps each module's own new, open, sendCmd and PTZ
commands and picks up get_config, set_config, set_time, cgi and
rtsp_url from FoscamHD, so how these cameras move is unchanged.
FoscamHD resolves the address and credentials on demand rather than in
open() precisely so this works: these modules have their own open()
which builds a UserAgent without working out either.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
Everything from the FI98xx generation onward answers a settings API on
/cgi-bin/CGIProxy.fcgi, but nothing in the tree reads or writes it - the
Foscam modules we ship only move the camera. That left no way to see or
change a camera's settings except its web ui, which on these needs an
IE6 plugin.
Adds get_config/set_config over 14 sections, plus set_time to point a
camera at an NTP server.
Two behaviours were measured against an FI9853EP on firmware 2.22.2.15
and drive the implementation:
Whole-section writers do not merge. A setSystemTime that leaves out
timeFormat and timeZone sets both to 0 rather than leaving them alone;
a partial write was observed to reset timeFormat 1 -> 0 and timeZone
14400 -> 0. set_config therefore reads the section back and writes the
merged result, never the diff on its own.
setSystemTime refuses isDst=1 outright, answering -1, so a daylight
saving offset has to be folded into timeZone. timeZone is seconds west
of UTC - EDT, UTC-4, is 14400 - which is the POSIX sign and the
opposite of a tm_gmtoff. Because DST lives in timeZone, set_time has
to be re-run when the offset changes; it is idempotent so cron is fine.
VideoStreamParam is read-only because getVideoStreamParam answers
resolution0..3 while setVideoStreamParam takes a single streamType plus
unsuffixed fields, so the read shape cannot be merged back into a
write. IPInfo is read-only because a wrong value takes the camera off
the network, where this module can no longer reach it. ImageSetting
has one command per field; setDenoiseLevel answers -3 on this firmware
so denoiseLevel is readable but not writable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
Events_Hour, Events_Day, Events_Week and Events_Month each carried their
own update and delete trigger, and every one issued a separate UPDATE
against the same Event_Summaries row. A single Event delete therefore
locked that row five times: once from event_delete_trigger and once from
each bucket's cascade. That is the deadlock zmstats.pl hits when it bulk
deletes aged rows out of Events_Hour while zmc and zma are writing.
event_update_trigger and event_delete_trigger now modify the bucket tables
themselves and apply one consolidated UPDATE, using ROW_COUNT() after each
bucket statement to tell whether the event was still in that bucket so
aged-out events do not over-adjust the counters. Measured on a scratch
database, an Event delete goes from 5 Event_Summaries row updates to 1.
This also fixes a drift bug. The old event_update_trigger kept
ArchivedEventDiskSpace correct only in a branch that cannot be reached: it
sits under IF (NEW.Archived != OLD.Archived) and requires both to be
false. The branch that does run when an already-archived event grows
updated Events_Archived but not Event_Summaries, so the archived total
drifted for the life of the install. Reproduced on a scratch database:
after archiving a 250-byte event and growing it to 999,
ArchivedEventDiskSpace still read 100.
BEHAVIOUR CHANGE. A direct DELETE against a bucket table no longer adjusts
Event_Summaries at all, and that is how zmstats.pl prunes. zmstats.pl
already resyncs HourEvents/DayEvents/WeekEvents/MonthEvents and their disk
space columns from COUNT(*)/SUM(DiskSpace) on any pass where it pruned, so
those four pairs become eventually consistent within one
ZM_STATS_UPDATE_INTERVAL instead of exact at every instant. The Total and
Archived columns stay exact, because only the Events triggers touch them.
The zmstats.pl comments are updated to describe the new arrangement; its
code is unchanged.
Migration is db/zm_update-1.39.25.sql.in: drop the eight cascade triggers,
resync Event_Summaries from the events themselves so the new triggers start
from ground truth and the archived drift above is repaired, then source
db/triggers.sql. version.txt goes to 1.39.25. No schema change, so
zm_create.sql.in needs no edit -- it already sources triggers.sql.
Ported from the ai_server branch, where this migration sits at 1.39.7, a
number master passed long ago and an existing install would never run.
Repackaged above master's tip. The ai_server version also unwinds a
views-and-SWR experiment that only ever existed on that branch, which is
dropped here as a no-op on master, and carries an unrelated START_DELAY
change, which is not taken.
Tests: tests/perl/test_event_summaries_triggers.pl, 29 assertions against a
real server, skipped when no scratch database is configured. Verified they
fail on the current triggers, on exactly the two claims above: the archived
drift, and 5 row updates per delete. Also verified the migration repairs a
drifted install, is idempotent across a second run, and leaves a migrated
install with byte-identical triggers to a fresh one. Generated zmstats.pl
passes perl -Tc.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
The C++ loader reads back only the rows where Value differs from DefaultValue
and takes compiled-in defaults for the rest, so the two columns have to be
written in the same form. Both writers passed Value through a boolean
conversion and DefaultValue through none: a boolean left alone was stored as
Value '1' against DefaultValue 'yes', never compared equal, and was read back
on every start. The filter did nothing for the 77 boolean rows.
Both columns now go through ConfigData::dbValue. It lives there because the
two writers - saveConfigToDB for an existing install and zmconfgen for the
zm_create.sql of a fresh one - have to agree, and drifting apart is what
caused this. An option with no default is stored as its type's empty value,
matching the empty string initialiseConfig already gives it.
The generated zm_create.sql now has Value equal to DefaultValue for all 259
rows, so a fresh install reads no Config rows at all. The generated
zm_config_defines.h is byte identical, confirming the compiled-in defaults
are unaffected by the representation change.
tests/perl/test_config_default_value.pl checks the invariant over every
option in ConfigData; it needs no database, since ConfigData does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
ConfigData wrote boolean defaults and requires clauses as 'yes'/'no' while
the Config table has always held '1'/'0', so every writer and reader had to
convert between the two. yes/no is not a boolean value and the web UI never
showed it - booleans render as a checkbox, which reads Value directly and
ignores the Hint - so the string form bought nothing and only existed to be
translated away again.
Converts the 75 boolean defaults, the 59 requires clauses that test them, and
the boolean entry in %types. OPT_FFMPEG is substituted into a boolean default
by cmake, so it changes with them.
Requires clauses reach the Config table byte for byte as before: they were
already converted to 1/0 on the way in, and the web UI string-compares them
against the stored Value.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
SIGHUP means reload. zmc responds by closing its events, disconnecting the
camera and reconnecting; the perl daemons respond by exiting so zmdc restarts
them. zmdc.pl logrot hupped every managed process, so the nightly logrotate
run cost about 8 seconds of capture on a default install.
Rotating a log file only needs the daemon to drop its file handle, so use a
separate signal for it. SIGWINCH is otherwise unused, is ignored by default and
exists on every supported platform.
- Logger (C++) installs a SIGWINCH handler beside its USR1/USR2 handler. The
handler only sets a flag; the next logPrint closes the file and the write
reopens it at the original path. This covers every C++ binary without
touching any daemon's main loop.
- Logger.pm registers WINCH alongside HUP in logSetSignal, which logInit
already calls, so the scripts that install their own HUP handler still
rotate.
- zmdc.pl logrot sends WINCH. The logrotate config is unchanged - it still
calls zmpkg.pl logrot.
Filter.pm and FilterTerm.php justified MAX_EVENT_DAYS by events not outliving
the nightly HUP, which is no longer what bounds them; cite SectionLength.
Tests: tests/zm_logger_rotate.cpp and tests/perl/test_log_rotate_signal.pl both
log, rename the file out from under the process, confirm writes still land in
the renamed file, signal WINCH and confirm the original path is written again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NDhTBPj9xEaT52pRmAufaP
DBD::mysql 5 refuses to connect to a MariaDB server and upstream does not
consider that a bug, so a MariaDB system needs DBD::MariaDB. Prefer it when
present; it drives both servers. ZM_DB_TYPE cannot select it, which is why
setting it to MariaDB does not work: the same value builds the PHP PDO dsn,
where the only valid driver is mysql whichever server is in use.
Swapping the dsn scheme is not enough on its own. DBD::MariaDB spells
DBD::mysql's mysql_* parameters mariadb_* and ignores parameters it does not
recognise, so a dsn built with the wrong prefix does not fail: it drops the
socket path and the TLS settings and connects over plain TCP. Build the dsn in
one place from the chosen driver, renaming caller supplied options too, since
zmupdate.pl asks for mysql_multi_statements.
Read the id of an inserted row through DBI's last_insert_id rather than the
mysql_insertid handle attribute, which DBD::MariaDB does not have and would
answer undef for, leaving events and objects with no Id.
DBD::MariaDB is always utf8mb4, so it takes no mysql_enable_utf8mb4 attribute.
Let cmake accept either driver instead of requiring DBD::mysql, checking in the
same order, and warn when neither is installed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019URmtYqza6Rzi6F7cmabSm
A DateTime lower bound was written as
COALESCE(E.EndDateTime, '9999-12-31 23:59:59') >= T1
reading "an event that has not ended yet never ends". EndDateTime is NULL for
an event that is still recording, but also for one zmc was killed part way
through, and that stays NULL forever - so every abandoned event matched every
window from then on. Wrapping the column in a function also meant no index
could range over it: with the only MonitorId key being MonitorId alone, every
montage review request read all of that monitor's events and filtered them in
memory, however narrow the window.
Length tells the two cases apart. It is flushed during recording, so a live
event's effective end keeps advancing while an abandoned one's is frozen at
whatever was recorded. Use the same CASE the SELECT list in ajax/events.php
already uses, and bound it below by StartDateTime.
The lower bound now emits three conjuncts:
E.StartDateTime >= DATE_SUB(T1, INTERVAL 1 DAY)
AND (E.EndDateTime IS NULL OR E.EndDateTime >= T1)
AND <effective end> >= T1
The floor bounds the scan for a window in the past and drops events abandoned
more than a day earlier - events do not outlive the nightly logrotate SIGHUP,
which stops and restarts them, so a day is comfortably beyond any real event.
The middle conjunct is implied by the third and exists only to give the
optimiser a second indexable handle: it is narrow exactly when the floor is
wide, so between them a window at either end of the retention period has
something cheap to range over. Verified the optimiser picks correctly for both,
unprompted. The third is the residual that gets the semantics right.
Add the two composite keys those conjuncts need. Two range columns cannot both
narrow one B-tree, and these are mirror images - EndDateTime >= T1 is
open-ended upwards, StartDateTime <= T2 downwards:
Events_MonitorId_StartDateTime_idx (MonitorId, StartDateTime)
Events_EndDateTime_MonitorId_idx (EndDateTime, MonitorId)
MonitorId leads the first because it is the equality; past a range column later
columns can no longer narrow the scan, and (StartDateTime, MonitorId) measured
3.5x slower for the same rows. EndDateTime leads the second so it also covers
zmaudit's hunt for events that were never closed, which has no monitor to scope
it - 9 rows with a key against a 22,948 row table scan without.
Both replacements are added before the keys they supersede are dropped:
Events_MonitorId_idx (MonitorId) is now a leftmost prefix of the new key.
Events_EndDateTime_DiskSpace (EndDateTime, DiskSpace) existed for the scan
that hunted events with no DiskSpace set; DiskSpace is set when the event is
finalised in C++, so nothing scans for it any more.
Net index count on Events is unchanged.
Measured on a 7,167 event monitor. Default one hour window: 7,055 rows
examined / 25.4ms -> 173 rows / 4ms. Window scrubbed a week back, which
persists in the zmFilter_StartDateTime cookie: 6,995 rows / 28ms -> 19 rows /
0.15ms.
Adds migration db/zm_update-1.39.22.sql. Extends t/filter_sql.t; the PHP and Perl were
checked to emit identical SQL, and the semantics checked against a table of
seven event shapes - abandoned long ago, abandoned recently, overlapping,
inside, before, still recording, and still recording for 17 hours.
A filter with LockRows set selected its entire result set FOR UPDATE inside one
transaction, so every lock the per-event work went on to take was held until
the run committed:
Events[Id] -> Events_Hour/Day/Week/Month[EventId] -> Event_Summaries[MonitorId] -> Storage[Id]
Because the locks accumulated across events, two filters deadlocked: one held
Event_Summaries for a monitor while waiting on a Storage row, the other held
that Storage row while waiting on Event_Summaries for its next event. No
per-event lock ordering can fix that while both rows stay locked for the length
of the batch.
Holding Event_Summaries for the whole run also blocked zmc from opening a new
event on any monitor the filter had touched, since creating an event updates
that row. The transaction spanned ffmpeg encodes, uploads and executed commands
as well, so it could be held open for minutes.
zmfilter now claims one event at a time in Events_Lock and releases it when it
is done with that event, so no InnoDB lock is held across the work. The
per-event body moves into checkFilterEvent.
skip_locked now adds NOT EXISTS over Events_Lock to the filter query rather
than SKIP LOCKED. The exclusion has to happen in the query: a filter whose
whole result set was held elsewhere would otherwise fill its LIMIT with events
it could only skip, and make no progress. It no longer depends on MariaDB 10.6
/ MySQL 8.0.1, so the UI no longer disables the option on older servers.
Also drops the two dbh->commit() calls in the AutoCopy branch. With no
transaction open they would warn, and before this they were silently ending the
batch transaction mid-loop, so AutoCopy filters never had the guarantee
LockRows was supposed to give them.
filterdebug.php was appending a bare ' SKIP LOCKED' after the LIMIT, which is
not valid SQL; it now renders the real clause in the right position.
Adds t/event_lock.t and t/filter_sql.t.
Events_Lock holds advisory locks over events, so that two filters do not work
on the same event at the same time. Claiming an event is a single autocommitted
INSERT of one row, which means no InnoDB lock is held while the filter actually
works on the event.
No foreign key to Events on purpose: it would make every event delete take a
lock in this table, which is the coupling this exists to avoid. Rows left
behind for deleted events are harmless and expire.
ZM_FILTER_LOCK_TIMEOUT (default 3600) is how long a claim lasts. A filter that
is killed part way through an event cannot release anything, so claims have to
expire on their own. It needs to be longer than the slowest thing a filter does
to a single event, which is why it is configurable rather than fixed.
Adds migration db/zm_update-1.39.21.sql.
lock_and_load + save is a read-modify-write, and outside a transaction the
SELECT ... FOR UPDATE autocommits and drops its lock immediately. The absolute
value written afterwards then clobbers any adjustment zmc, zmaudit or another
filter made in between.
Replace both call sites with ZoneMinder::Storage::adjust_diskspace, a single
relative UPDATE that is atomic on its own and holds the row lock only for the
length of that statement. It reads the new total back into the object rather
than adding the delta locally, because ZoneMinder::Object caches objects for
the life of the process and its accessors never re-load, so the cached value
can be arbitrarily stale and adjusting it locally would preserve the error.
Keeping the adjustment to one statement also keeps Storage out of the lock
chain that deleting an event walks:
Events[Id] -> Events_Hour/Day/Week/Month[EventId] -> Event_Summaries[MonitorId]
Adds t/storage_diskspace.t.
The two subs decide whether zmpkg.pl hands start/stop/restart to systemd or
runs the daemons itself, which is what keeps them out of the web server's
cgroup and mount namespace. Living in a script they could not be reached by
require_ok, so the cgroup matching had no test.
Split the matching into cgroup_in_service, which takes the text of a cgroup
file rather than reading one, and cover it: cgroup v2's single line and v1's
several, a sub-cgroup of a delegated service, a unit whose name merely starts
with ours, and empty or undef input.
The test found that the delimiter in m|/\Q$unit\E\.service(?:/|$)| ended the
match at the alternation, so the module did not compile.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019URmtYqza6Rzi6F7cmabSm
Verified every added endpoint against an Amcrest IP5M-1190EW running
2.810.00AC004.0.R. A full get_config() returns 52 sections in ~24s and
the set_config() key form round-trips, but several guesses were wrong:
networkInterface.cgi does not exist, netApp.cgi?action=getInterfaces is
the real endpoint.
userManager.cgi reports a real per-user Id starting at 1, so get_users
was handing callers the array position instead of the number the camera
actually recognises.
modifyUser rejects a partial record with 400. update_user now reads the
user back and sends the whole record with the changes merged over it.
getSoftwareVersion and getHardwareVersion both answer with a bare
'version=', so device_info's flat merge lost one of them. The software
one is a single line with the build appended - '2.810.00AC004.0.R,
build:2023-09-04' - so getVersion splits it and returns the build
separately in list context.
The configManager list is now the set that model actually implements.
UPnP and Login are gone, they answer 400; Lighting_V2, WLan, NAS, Record,
Alarm and the VideoIn* sections are added.
That model has no getCurrentProtocolCaps and no coaxialControlIO, and it
answers OK to Lighting/Lighting_V2 writes and then ignores them, so its
WhiteLight is not drivable over http. Documented rather than papered
over. update_firmware stays the one unverified path, deliberately.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LzHrvNtF13vyYBAJjktam6
set_config prepended the section name to keys that already carried it,
so NTP.Address went out as NTP.NTP.Address. get_config only strips the
'table.' prefix, which leaves keys in exactly the form setConfig wants,
so drop the extra prefix and let reads and writes round-trip.
uri_encode was called but URI::Encode was never imported, and the plain
call leaves reserved characters alone, which corrupts passwords in a
query string. Import it and always escape reserved characters.
get_config now reads configManager.cgi sections as well as the handful
of dedicated cgis, retries once on a 401 so the fresh digest nonce gets
used, strips the 'table.' prefix, and tries any unrecognised name as a
configManager section so a template can pull in sections the module
doesn't list.
set_config fell off the end of its loop returning undef on success.
reboot POSTed system_reset=1 to setparam.cgi, a Vivotek endpoint, on a
relative url through the raw user agent. ZoneMinder::General was used
but never required.
Adds device info, time, users, snapshot/rtsp/profiles/probe, white light
and siren, and the remaining PTZ stops and auto pan, matching the method
names cameratool.pl probes for with ->can().
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LzHrvNtF13vyYBAJjktam6
The openbsd branch and master both changed the top fallback in CpuUsage
and diverged. Combine them:
- Keep master's /proc/stat diagnostics: report why /proc/stat could not
be read (ProcSubset=pid mount namespace vs open failure), default the
fields split out of top output so an empty result does not warn under
-w, and only complain once per process.
- Keep the openbsd branch's parse_bsd_top_cpu(): OpenBSD top prints CPU
states in a format neither the FreeBSD nor the Linux grep matches, so
when the greps produce no numbers, parse raw top output. It handles
both the aggregate "CPU states:" line and per-core "CPU0 states:"
lines, averaging over the lines seen, and does not depend on padding.
- Carry over scripts/ZoneMinder/t/server_cpu.t covering it.
Tests: prove -Iscripts/ZoneMinder/lib scripts/ZoneMinder/t/server_cpu.t
-> 16/16 pass. perl -c on Server.pm is clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kfiom9RefM75GTZFkqpJo2
zmpkg.pl hands start/stop/restart to systemd via zmsystemctl.pl, so that pid 1
forks the daemons and they get zoneminder.service's cgroup and mount namespace
rather than the web server's. The detection defeated itself: systemdRunning()
read pid 1's name out of /proc, which a web server running with
ProtectProc=invisible hides from the web user. Starting from the web ui
therefore concluded systemd was absent, skipped the delegation and ran zmdc
inside the web server's namespace, where ProcSubset=pid hides /proc/stat and
every figure zmstats records is wrong.
Use -d /run/systemd/system, which is what sd_booted(3) does and stays readable
however /proc is mounted.
calledBysystem() had to change with it or the two would combine into a start
loop, systemd running zmpkg which asks systemd to run zmpkg. It read the
parent's name from /proc, equally hidden, and the parent is pid 1 only until
it re-parents us. Ask our own cgroup whether we are already the service
zmsystemctl.pl would start. Not INVOCATION_ID: systemd sets it for every
descendant of a unit, so anything forked from the web ui inherits the web
server's copy and would wrongly look systemd-started.
Also check the exit status of the zmsystemctl.pl call. It was discarded, so a
refused pkexec left the command cleared and nothing started, with nothing
logged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019URmtYqza6Rzi6F7cmabSm
/proc/stat is invisible to any process in a mount namespace set up with
ProcSubset=pid, which is systemd's default hardening for apache. Daemons
started from the web ui are forked off mod_php and inherit it, so zmstats
fell through to the top fallback, which reads /proc/stat as well and so
produced no output either.
Separate the missing-file case from the failed-open case so the message
names the likely cause instead of printing a $! that a failed -e never set.
Emit the warning once per process rather than once per
ZM_STATS_UPDATE_INTERVAL. The condition does not clear itself, so it was
filling zmstats.log with the same three lines a minute.
split() on an empty $top_output returns an empty list, leaving all four
values undef, so each fallback cycle also emitted four "Use of
uninitialized value" warnings under -w. Default them before use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019URmtYqza6Rzi6F7cmabSm
zmwatch fetches its monitor list once at the top of a pass and then walks it.
Deleting a monitor mid-pass stops its zmc from the web ui, so by the time
zmwatch reaches that monitor its shared memory is gone, zmMemVerify fails and
zmwatch calls control('restart') from the now-stale list. zmc comes back for a
monitor that is marked Deleted, so no later pass ever looks at it again and
nothing stops it until zmpkg.pl restart. That is the orphaned zmc left behind
after deleting a monitor from the console.
Re-read Deleted from the database in ZoneMinder::Monitor::control() before
running a start or a restart, and skip the command if the monitor has been
deleted or the row is gone. control() is the single path every restart goes
through, so this covers any caller working from a list it fetched earlier.
Stopping is deliberately still allowed, since that is how an orphan gets
cleaned up.
Tests: tests/perl/test_monitor_control_deleted.pl stubs zmDbFetchOne and
runCommand and checks that start/restart still run for a live monitor, are
skipped for a deleted one and for one removed from the database, and that stop
is unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NDhTBPj9xEaT52pRmAufaP
Adds a web config option whose contents are rendered on the login view,
between the title and the username and password fields. Useful for a site
notice, an acceptable use or legal warning, or a note identifying which
installation this is when running more than one.
Defaults to empty, and nothing is emitted when it is empty or whitespace, so
existing installs look exactly as they do now.
The text is escaped rather than interpreted. The login page is served before
anyone has authenticated, so it is not somewhere to emit admin-supplied markup,
and no other config value in the skin is output unescaped either. Escaping runs
before nl2br so the only tags reaching the browser are the line breaks we add
ourselves; reversing that order would turn the setting into stored XSS.
Uses the text type, so the Options UI renders a textarea and the value can span
lines. The Config.Value column is already text and options.php normalises CRLF
to LF on save, so no schema change is needed and nl2br sees consistent
newlines.
Guarded with defined() to match the surrounding code, so the view still renders
where the database predates the option.
Styled in base, classic and dark. Text contrast is 7.0:1 light and 7.5:1 dark,
both above WCAG AA, and long unbroken tokens wrap rather than widening the
fixed-width form.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The capture daemons and this watchdog share the database. If the DB (or
the host/network) drops out, every zmc stops updating its shared-memory
heartbeat at the same instant. When the DB connection recovers, the next
check pass saw the whole fleet as simultaneously stale and restarted --
and, for cameras with control capability, rebooted -- every monitor, even
though none of them individually failed.
Track wall-clock time between check passes. If we just came out of a DB
reconnect, or far more time elapsed than a normal check interval (host
stall/clock step), skip restart+reboot actions for one pass so capture
daemons can refresh their heartbeats. A genuinely dead camera is still
stale next pass and gets handled normally -- reboot behaviour is
preserved, just no longer fired on a false fleet-wide positive.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HoNiNxwgyaHV29CbiueCUf
The mp4 is written as the event records, so its mtime is when recording
finished, not when it started. Set EndDateTime to the mtime and
StartDateTime to duration seconds before it, so Length agrees with
EndDateTime - StartDateTime.
The directory mtime fallback also ran unconditionally after both
branches, overwriting the times just computed and the Deep-scheme
path-derived start. Only use it when there are neither capture jpgs nor
an mp4.
Adds t/event_recover_timestamps.t, which builds a 3 second mp4 with
ffmpeg and checks the recovered times against its mtime.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BV1gDc9T6D6H8gtTcxwLvk
The DateTime filter attribute was taught the "event overlaps this
instant/window" idiom in web/includes/FilterTerm.php: a lower bound (>=/>)
compares COALESCE(EndDateTime, '9999-12-31 23:59:59') so still-running events
are included, while an upper bound keeps comparing StartDateTime. The Perl
Filter.pm still emitted E.StartDateTime for that attribute, so a saved filter
using DateTime with >=/> selected different events in the web UI than in
zmfilter.pl, which drives automated delete/email/execute actions.
Apply the same column selection in ZoneMinder::Filter::Sql so the daemon and
the web agree. Verified by generating SQL from both sides for =, <, <=, >, >=:
the column expressions are now byte-identical.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KQipwf632JGgNH4W7p8cqs
Replace the single alarm_image slot with an analysis_image_buffer ring of
image_buffer_count Images living in the already-reserved alarm_images SHM
region. Successive WriteAlarmImage calls rotate through the ring and
publish last_analysis_index last (after the bytes and per-slot format),
so a reader sampling last_analysis_index always sees a fully written
slot. GetAlarmImage returns that slot, syncing its AVPixelFormat from the
per-slot analysis_image_pixelformats array.
SharedData gains last_analysis_index and analysis_image_count (plus 8
bytes of padding to keep the 16-byte-multiple layout), making it 888
bytes. The Perl (Memory.pm) and PHP (Monitor.php) SHM readers are updated
in lockstep, and a static_assert(sizeof(SharedData)==888) in zm_monitor.h
guards the layout against silent drift.
This lets multiple in-flight analysis/annotated frames be buffered and
streamed in sync rather than always overwriting one slot, and gives the
AI object-detection work a place to publish annotated frames.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The C++ and PHP parsers for zm.conf and /etc/zm/conf.d/*.conf used fgets
with a 512-byte buffer, silently truncating any line longer than that.
There was also no way to split a value across multiple lines, so editors
hit the cap with no workaround.
Accept a trailing backslash (with optional whitespace before the newline)
as a line-continuation marker. Leading whitespace on continuation lines
is stripped so users can indent for readability without it leaking into
the value. Three parsers all read the same files and must agree:
- src/zm_config.cpp: switch to std::ifstream + std::getline so a single
physical line is no longer capped at 512 bytes, then join continuation
lines before running the existing pointer-based parser
- scripts/ZoneMinder/lib/ZoneMinder/Config.pm.in: accumulate a logical
line across trailing-backslash physical lines
- web/includes/config.php.in: drop the 512-byte fgets cap and accumulate
in the same way
tests/zm_config.cpp covers three cases against the C++ parser: a
three-segment continuation joins to "firstsecondthird" with leading
whitespace stripped, a bare backslash inside a value is preserved
(C:\Users\zm), and a 1500-byte single line survives intact.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
#4914 unconditionally drops Javascript errors and CSP violations sourced from
browser extensions. Some operators want to know when a plugin is touching the
ZoneMinder tab, so gate the suppression behind a config entry.
Add ZM_LOG_BROWSER_EXTENSIONS (boolean, default no) in the logging category.
It is exposed to JS by the existing non-private-config loop in skin.js.php as
the string '0'/'1' (the same convention as ZM_LOG_INJECT), so logger.js only
filters extension sources when it is '0'. Default behaviour is unchanged:
extension noise stays out of the log.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
zmwatch only gated the camera reboot attempt on CanReboot(), so it called
$control->open() even when the camera was unreachable - the common reason a
monitor has no image since startup - and blocked until the connection timed
out, logging an error each pass.
Add a ping check before open(). Move the host resolution into Control so callers
don't have to dig the ip out of the Path: add Control::host(), which returns the
cached host or derives it from the monitor's ControlAddress/Path via the shared
guess_credentials() (parsing only, no network i/o), and have ping() fall back to
it. ping() still accepts an explicit ip.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The hand-rolled regex in open() mis-parsed a stream Path that carried no
credentials, e.g. rtsp://10.0.0.4:554/cam: the greedy [^:@]+ consumed the host
as the username, 554 as the password, and ADDRESS backtracked to the single
digit 4. The control daemon then dialled http://...@4, producing
"Can't connect to 4:80 (Connection timed out)".
Replace the regex with the shared URI-based Control.pm guess_credentials(),
which handles ControlAddress and Path, converts rtsp to http, and falls back to
the Monitor User/Pass. Create the LWP::UserAgent before parsing since
guess_credentials() sets credentials on it. The Grandstream login wire protocol
(challenge/authcode and old-style fallback) is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On systems with many cameras the Grandstream control log entries gave no
indication of which monitor they referred to. Prefix every Debug/Warning/Error
message with the monitor Id and Name. Also fix two message typos
(challengstring, UNable).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Isaac Connor