Upstream's schema-drift fix landed db/zm_update-1.39.25.sql and bumped
version.txt to 1.39.25 while the trigger consolidation was using the same
number for db/zm_update-1.39.25.sql.in.
Git does not see this as a conflict -- the two files have different names in
the source tree -- but configure_file generates the .sql.in into
zm_update-1.39.25.sql, which is the same install target as the tracked .sql.
Confirmed by staging an install off the merge: the installed
zm_update-1.39.25.sql was upstream's column reconciliation and the trigger
consolidation was simply absent, with no error anywhere. An upgrade would
have applied one of the two and silently skipped the other.
Renumbered to 1.39.26, version.txt to match, and the two zmstats.pl.in
comments that name the migration updated. Upstream's 1.39.25 is untouched.
Verified by staging an install again: both files are now present and hold
what they should. The migration was re-run end to end at the new number
against a scratch database -- drops the eight cascade triggers, leaves four,
and the 29 trigger assertions pass afterwards.
ai_server carries the old 1.39.25 numbering and will need the same treatment
when master is next merged into it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
The guard skipped the delay when a filter was named. zmpkg starts this as
"zmdc.pl start zmfilter.pl --filter_id=N --daemon", so it always names one --
the boot path never waited, and the only invocation that did was a hand-run
scan of every filter. Exactly backwards from what the delay is for.
--daemon is what marks the unattended case, so gate on that instead, still
skipping when there is a terminal on stdin.
Measured with Filter::Execute stubbed to return no events, so nothing was
deleted or emailed, timing the first call to it:
invocation before after
--filter_id=1 --daemon (zmpkg) 0.1s 5.1s
--filter_id=1 (hand-run) 0.1s 0.1s
no args, scan all filters 5.1s 0.1s
So filters now start 5 seconds later at boot, which is the point of the
constant, and a hand-run returns immediately.
Also sets zmstats' START_DELAY to 5. The previous commit's message said it
did this and it did not: the edit was written against "=> 5" when the file
said "=> 30", so it silently matched nothing while the interactive skip
beside it landed. Verified this time by running the generated script both
ways -- non-interactive logs "starting in 5 seconds", interactive logs
nothing and proceeds.
That run also closes the gap the previous commit noted: startedInteractively
resolves at runtime in the generated zmstats.pl. It had only been checked
with perl -Tc, which does not catch an unimported sub called with parens --
an earlier run against a stale module path failed with exactly that error and
sent me looking.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
zmwatch slept 30 seconds before its first pass. The reason is real: when it
starts, zmc has not necessarily created its shared memory or written a
heartbeat, and neither state is distinguishable from a camera that has died,
so the first pass would restart every monitor.
A fixed sleep is the wrong shape for that. It is not derived from anything --
ZM_WATCH_CHECK_INTERVAL is 10 and ZM_WATCH_MAX_DELAY is 45, so the delay was
three check intervals and shorter than the staleness threshold it was
standing in for. On a busy host with many cameras, or one camera slow to
answer, 30 seconds is not enough and the fleet gets restarted anyway. It also
blocks every other check, and makes running zmwatch by hand a 30 second wait.
Each monitor now gets until ZM_WATCH_MAX_DELAY after zmwatch started -- the
same threshold used to judge a heartbeat stale -- to appear, and only if we
have never yet seen it healthy. Once seen healthy it is judged immediately, so
a camera that dies later is caught exactly as fast as before, and every check
other than the restart runs from the first pass.
Verified against the two monitors on a machine with no daemons running, which
is the boot state, with Monitor::control stubbed so nothing was actually
started. Before: both monitors restarted on the first pass after the sleep,
and again every 10 seconds. After: no restart for 45 seconds, then both
restarted on the first pass past the grace, at 50 seconds. So a monitor that
never comes up is still caught, ~20 seconds later than before at boot and at
the same speed as before thereafter.
Also adds ZoneMinder::General::startedInteractively and uses it to skip the
start delay in zmstats.pl and zmfilter.pl when a person ran them. zmdc.pl
reopens STDIN on /dev/null for everything it starts, so a terminal on STDIN
reliably means a hand-run; zmtelemetry.pl already relies on this. A run from
cron or a unit file has no terminal either and still waits, which is the right
way round. zmstats keeps a delay because it is polite while zmc and zma are
competing for the machine, but nothing in it races with startup -- it connects
to the database first and has its own reconnect loop -- so the value is now 5
rather than 30.
Note zmfilter's existing guard is inverted with respect to that intent:
zmpkg starts it as "--filter_id=N --daemon", and the guard skips the delay
when a filter is named, so the daemon path never waited and only a manual
scan-all did. Left as it is here beyond adding the interactive case; worth a
separate look.
The zmstats and zmfilter changes are not runtime tested. Running either
against a live install prunes and rewrites rows, and there was no throwaway
install to point them at; startedInteractively is verified directly in both
directions, and all three generated scripts pass perl -Tc.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
setContrast takes 'constrast'. Spelling the parameter 'contrast', as
the field is named everywhere else including in getImageSetting's own
answer, is accepted with result=0, leaves the real parameter unset and
applies 0 for it. Contrast 0 is a black picture, so set_config would
have blacked out any camera whose contrast a template touched, while
reporting success.
Found by doing it to a live FI9853EP: the camera kept serving video
with a correct timestamp overlay and no error anywhere, and the only
sign was getImageSetting answering contrast 0 where an untouched
camera of the same model answered 50.
field_set entries are now [command, parameter] pairs rather than a bare
command with the parameter assumed from the field name, since the
firmware gives no indication when it is handed a name it does not read.
The same check against an untouched camera clears brightness, hue,
saturation and sharpness: those were written with their documented
names during the same session and kept their values.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
set_time wrote unconditionally. Because the offset it corrects only
moves at a daylight saving change, running it from cron - which is the
point of it being idempotent - meant thousands of writes a year to the
camera's flash for the two that matter.
Read the section first and return success without writing when every
field we would set already holds the wanted value. Measured against an
FI9853EP: a repeat run now issues one getSystemTime and no write, while
a moved offset still reads, merges and writes once.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
FI9821W_Y2k, FI9831W and FOSCAMR2C all talk to the same
/cgi-bin/CGIProxy.fcgi endpoint that FoscamHD now implements settings
for, so reparent them onto it instead of ZoneMinder::Control.
Method resolution keeps each module's own new, open, sendCmd and PTZ
commands and picks up get_config, set_config, set_time, cgi and
rtsp_url from FoscamHD, so how these cameras move is unchanged.
FoscamHD resolves the address and credentials on demand rather than in
open() precisely so this works: these modules have their own open()
which builds a UserAgent without working out either.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
Everything from the FI98xx generation onward answers a settings API on
/cgi-bin/CGIProxy.fcgi, but nothing in the tree reads or writes it - the
Foscam modules we ship only move the camera. That left no way to see or
change a camera's settings except its web ui, which on these needs an
IE6 plugin.
Adds get_config/set_config over 14 sections, plus set_time to point a
camera at an NTP server.
Two behaviours were measured against an FI9853EP on firmware 2.22.2.15
and drive the implementation:
Whole-section writers do not merge. A setSystemTime that leaves out
timeFormat and timeZone sets both to 0 rather than leaving them alone;
a partial write was observed to reset timeFormat 1 -> 0 and timeZone
14400 -> 0. set_config therefore reads the section back and writes the
merged result, never the diff on its own.
setSystemTime refuses isDst=1 outright, answering -1, so a daylight
saving offset has to be folded into timeZone. timeZone is seconds west
of UTC - EDT, UTC-4, is 14400 - which is the POSIX sign and the
opposite of a tm_gmtoff. Because DST lives in timeZone, set_time has
to be re-run when the offset changes; it is idempotent so cron is fine.
VideoStreamParam is read-only because getVideoStreamParam answers
resolution0..3 while setVideoStreamParam takes a single streamType plus
unsuffixed fields, so the read shape cannot be merged back into a
write. IPInfo is read-only because a wrong value takes the camera off
the network, where this module can no longer reach it. ImageSetting
has one command per field; setDenoiseLevel answers -3 on this firmware
so denoiseLevel is readable but not writable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
Events_Hour, Events_Day, Events_Week and Events_Month each carried their
own update and delete trigger, and every one issued a separate UPDATE
against the same Event_Summaries row. A single Event delete therefore
locked that row five times: once from event_delete_trigger and once from
each bucket's cascade. That is the deadlock zmstats.pl hits when it bulk
deletes aged rows out of Events_Hour while zmc and zma are writing.
event_update_trigger and event_delete_trigger now modify the bucket tables
themselves and apply one consolidated UPDATE, using ROW_COUNT() after each
bucket statement to tell whether the event was still in that bucket so
aged-out events do not over-adjust the counters. Measured on a scratch
database, an Event delete goes from 5 Event_Summaries row updates to 1.
This also fixes a drift bug. The old event_update_trigger kept
ArchivedEventDiskSpace correct only in a branch that cannot be reached: it
sits under IF (NEW.Archived != OLD.Archived) and requires both to be
false. The branch that does run when an already-archived event grows
updated Events_Archived but not Event_Summaries, so the archived total
drifted for the life of the install. Reproduced on a scratch database:
after archiving a 250-byte event and growing it to 999,
ArchivedEventDiskSpace still read 100.
BEHAVIOUR CHANGE. A direct DELETE against a bucket table no longer adjusts
Event_Summaries at all, and that is how zmstats.pl prunes. zmstats.pl
already resyncs HourEvents/DayEvents/WeekEvents/MonthEvents and their disk
space columns from COUNT(*)/SUM(DiskSpace) on any pass where it pruned, so
those four pairs become eventually consistent within one
ZM_STATS_UPDATE_INTERVAL instead of exact at every instant. The Total and
Archived columns stay exact, because only the Events triggers touch them.
The zmstats.pl comments are updated to describe the new arrangement; its
code is unchanged.
Migration is db/zm_update-1.39.25.sql.in: drop the eight cascade triggers,
resync Event_Summaries from the events themselves so the new triggers start
from ground truth and the archived drift above is repaired, then source
db/triggers.sql. version.txt goes to 1.39.25. No schema change, so
zm_create.sql.in needs no edit -- it already sources triggers.sql.
Ported from the ai_server branch, where this migration sits at 1.39.7, a
number master passed long ago and an existing install would never run.
Repackaged above master's tip. The ai_server version also unwinds a
views-and-SWR experiment that only ever existed on that branch, which is
dropped here as a no-op on master, and carries an unrelated START_DELAY
change, which is not taken.
Tests: tests/perl/test_event_summaries_triggers.pl, 29 assertions against a
real server, skipped when no scratch database is configured. Verified they
fail on the current triggers, on exactly the two claims above: the archived
drift, and 5 row updates per delete. Also verified the migration repairs a
drifted install, is idempotent across a second run, and leaves a migrated
install with byte-identical triggers to a fresh one. Generated zmstats.pl
passes perl -Tc.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
The C++ loader reads back only the rows where Value differs from DefaultValue
and takes compiled-in defaults for the rest, so the two columns have to be
written in the same form. Both writers passed Value through a boolean
conversion and DefaultValue through none: a boolean left alone was stored as
Value '1' against DefaultValue 'yes', never compared equal, and was read back
on every start. The filter did nothing for the 77 boolean rows.
Both columns now go through ConfigData::dbValue. It lives there because the
two writers - saveConfigToDB for an existing install and zmconfgen for the
zm_create.sql of a fresh one - have to agree, and drifting apart is what
caused this. An option with no default is stored as its type's empty value,
matching the empty string initialiseConfig already gives it.
The generated zm_create.sql now has Value equal to DefaultValue for all 259
rows, so a fresh install reads no Config rows at all. The generated
zm_config_defines.h is byte identical, confirming the compiled-in defaults
are unaffected by the representation change.
tests/perl/test_config_default_value.pl checks the invariant over every
option in ConfigData; it needs no database, since ConfigData does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
ConfigData wrote boolean defaults and requires clauses as 'yes'/'no' while
the Config table has always held '1'/'0', so every writer and reader had to
convert between the two. yes/no is not a boolean value and the web UI never
showed it - booleans render as a checkbox, which reads Value directly and
ignores the Hint - so the string form bought nothing and only existed to be
translated away again.
Converts the 75 boolean defaults, the 59 requires clauses that test them, and
the boolean entry in %types. OPT_FFMPEG is substituted into a boolean default
by cmake, so it changes with them.
Requires clauses reach the Config table byte for byte as before: they were
already converted to 1/0 on the way in, and the web UI string-compares them
against the stored Value.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
SIGHUP means reload. zmc responds by closing its events, disconnecting the
camera and reconnecting; the perl daemons respond by exiting so zmdc restarts
them. zmdc.pl logrot hupped every managed process, so the nightly logrotate
run cost about 8 seconds of capture on a default install.
Rotating a log file only needs the daemon to drop its file handle, so use a
separate signal for it. SIGWINCH is otherwise unused, is ignored by default and
exists on every supported platform.
- Logger (C++) installs a SIGWINCH handler beside its USR1/USR2 handler. The
handler only sets a flag; the next logPrint closes the file and the write
reopens it at the original path. This covers every C++ binary without
touching any daemon's main loop.
- Logger.pm registers WINCH alongside HUP in logSetSignal, which logInit
already calls, so the scripts that install their own HUP handler still
rotate.
- zmdc.pl logrot sends WINCH. The logrotate config is unchanged - it still
calls zmpkg.pl logrot.
Filter.pm and FilterTerm.php justified MAX_EVENT_DAYS by events not outliving
the nightly HUP, which is no longer what bounds them; cite SectionLength.
Tests: tests/zm_logger_rotate.cpp and tests/perl/test_log_rotate_signal.pl both
log, rename the file out from under the process, confirm writes still land in
the renamed file, signal WINCH and confirm the original path is written again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NDhTBPj9xEaT52pRmAufaP
DBD::mysql 5 refuses to connect to a MariaDB server and upstream does not
consider that a bug, so a MariaDB system needs DBD::MariaDB. Prefer it when
present; it drives both servers. ZM_DB_TYPE cannot select it, which is why
setting it to MariaDB does not work: the same value builds the PHP PDO dsn,
where the only valid driver is mysql whichever server is in use.
Swapping the dsn scheme is not enough on its own. DBD::MariaDB spells
DBD::mysql's mysql_* parameters mariadb_* and ignores parameters it does not
recognise, so a dsn built with the wrong prefix does not fail: it drops the
socket path and the TLS settings and connects over plain TCP. Build the dsn in
one place from the chosen driver, renaming caller supplied options too, since
zmupdate.pl asks for mysql_multi_statements.
Read the id of an inserted row through DBI's last_insert_id rather than the
mysql_insertid handle attribute, which DBD::MariaDB does not have and would
answer undef for, leaving events and objects with no Id.
DBD::MariaDB is always utf8mb4, so it takes no mysql_enable_utf8mb4 attribute.
Let cmake accept either driver instead of requiring DBD::mysql, checking in the
same order, and warn when neither is installed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019URmtYqza6Rzi6F7cmabSm
A DateTime lower bound was written as
COALESCE(E.EndDateTime, '9999-12-31 23:59:59') >= T1
reading "an event that has not ended yet never ends". EndDateTime is NULL for
an event that is still recording, but also for one zmc was killed part way
through, and that stays NULL forever - so every abandoned event matched every
window from then on. Wrapping the column in a function also meant no index
could range over it: with the only MonitorId key being MonitorId alone, every
montage review request read all of that monitor's events and filtered them in
memory, however narrow the window.
Length tells the two cases apart. It is flushed during recording, so a live
event's effective end keeps advancing while an abandoned one's is frozen at
whatever was recorded. Use the same CASE the SELECT list in ajax/events.php
already uses, and bound it below by StartDateTime.
The lower bound now emits three conjuncts:
E.StartDateTime >= DATE_SUB(T1, INTERVAL 1 DAY)
AND (E.EndDateTime IS NULL OR E.EndDateTime >= T1)
AND <effective end> >= T1
The floor bounds the scan for a window in the past and drops events abandoned
more than a day earlier - events do not outlive the nightly logrotate SIGHUP,
which stops and restarts them, so a day is comfortably beyond any real event.
The middle conjunct is implied by the third and exists only to give the
optimiser a second indexable handle: it is narrow exactly when the floor is
wide, so between them a window at either end of the retention period has
something cheap to range over. Verified the optimiser picks correctly for both,
unprompted. The third is the residual that gets the semantics right.
Add the two composite keys those conjuncts need. Two range columns cannot both
narrow one B-tree, and these are mirror images - EndDateTime >= T1 is
open-ended upwards, StartDateTime <= T2 downwards:
Events_MonitorId_StartDateTime_idx (MonitorId, StartDateTime)
Events_EndDateTime_MonitorId_idx (EndDateTime, MonitorId)
MonitorId leads the first because it is the equality; past a range column later
columns can no longer narrow the scan, and (StartDateTime, MonitorId) measured
3.5x slower for the same rows. EndDateTime leads the second so it also covers
zmaudit's hunt for events that were never closed, which has no monitor to scope
it - 9 rows with a key against a 22,948 row table scan without.
Both replacements are added before the keys they supersede are dropped:
Events_MonitorId_idx (MonitorId) is now a leftmost prefix of the new key.
Events_EndDateTime_DiskSpace (EndDateTime, DiskSpace) existed for the scan
that hunted events with no DiskSpace set; DiskSpace is set when the event is
finalised in C++, so nothing scans for it any more.
Net index count on Events is unchanged.
Measured on a 7,167 event monitor. Default one hour window: 7,055 rows
examined / 25.4ms -> 173 rows / 4ms. Window scrubbed a week back, which
persists in the zmFilter_StartDateTime cookie: 6,995 rows / 28ms -> 19 rows /
0.15ms.
Adds migration db/zm_update-1.39.22.sql. Extends t/filter_sql.t; the PHP and Perl were
checked to emit identical SQL, and the semantics checked against a table of
seven event shapes - abandoned long ago, abandoned recently, overlapping,
inside, before, still recording, and still recording for 17 hours.
A filter with LockRows set selected its entire result set FOR UPDATE inside one
transaction, so every lock the per-event work went on to take was held until
the run committed:
Events[Id] -> Events_Hour/Day/Week/Month[EventId] -> Event_Summaries[MonitorId] -> Storage[Id]
Because the locks accumulated across events, two filters deadlocked: one held
Event_Summaries for a monitor while waiting on a Storage row, the other held
that Storage row while waiting on Event_Summaries for its next event. No
per-event lock ordering can fix that while both rows stay locked for the length
of the batch.
Holding Event_Summaries for the whole run also blocked zmc from opening a new
event on any monitor the filter had touched, since creating an event updates
that row. The transaction spanned ffmpeg encodes, uploads and executed commands
as well, so it could be held open for minutes.
zmfilter now claims one event at a time in Events_Lock and releases it when it
is done with that event, so no InnoDB lock is held across the work. The
per-event body moves into checkFilterEvent.
skip_locked now adds NOT EXISTS over Events_Lock to the filter query rather
than SKIP LOCKED. The exclusion has to happen in the query: a filter whose
whole result set was held elsewhere would otherwise fill its LIMIT with events
it could only skip, and make no progress. It no longer depends on MariaDB 10.6
/ MySQL 8.0.1, so the UI no longer disables the option on older servers.
Also drops the two dbh->commit() calls in the AutoCopy branch. With no
transaction open they would warn, and before this they were silently ending the
batch transaction mid-loop, so AutoCopy filters never had the guarantee
LockRows was supposed to give them.
filterdebug.php was appending a bare ' SKIP LOCKED' after the LIMIT, which is
not valid SQL; it now renders the real clause in the right position.
Adds t/event_lock.t and t/filter_sql.t.
Events_Lock holds advisory locks over events, so that two filters do not work
on the same event at the same time. Claiming an event is a single autocommitted
INSERT of one row, which means no InnoDB lock is held while the filter actually
works on the event.
No foreign key to Events on purpose: it would make every event delete take a
lock in this table, which is the coupling this exists to avoid. Rows left
behind for deleted events are harmless and expire.
ZM_FILTER_LOCK_TIMEOUT (default 3600) is how long a claim lasts. A filter that
is killed part way through an event cannot release anything, so claims have to
expire on their own. It needs to be longer than the slowest thing a filter does
to a single event, which is why it is configurable rather than fixed.
Adds migration db/zm_update-1.39.21.sql.
lock_and_load + save is a read-modify-write, and outside a transaction the
SELECT ... FOR UPDATE autocommits and drops its lock immediately. The absolute
value written afterwards then clobbers any adjustment zmc, zmaudit or another
filter made in between.
Replace both call sites with ZoneMinder::Storage::adjust_diskspace, a single
relative UPDATE that is atomic on its own and holds the row lock only for the
length of that statement. It reads the new total back into the object rather
than adding the delta locally, because ZoneMinder::Object caches objects for
the life of the process and its accessors never re-load, so the cached value
can be arbitrarily stale and adjusting it locally would preserve the error.
Keeping the adjustment to one statement also keeps Storage out of the lock
chain that deleting an event walks:
Events[Id] -> Events_Hour/Day/Week/Month[EventId] -> Event_Summaries[MonitorId]
Adds t/storage_diskspace.t.
The two subs decide whether zmpkg.pl hands start/stop/restart to systemd or
runs the daemons itself, which is what keeps them out of the web server's
cgroup and mount namespace. Living in a script they could not be reached by
require_ok, so the cgroup matching had no test.
Split the matching into cgroup_in_service, which takes the text of a cgroup
file rather than reading one, and cover it: cgroup v2's single line and v1's
several, a sub-cgroup of a delegated service, a unit whose name merely starts
with ours, and empty or undef input.
The test found that the delimiter in m|/\Q$unit\E\.service(?:/|$)| ended the
match at the alternation, so the module did not compile.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019URmtYqza6Rzi6F7cmabSm
Verified every added endpoint against an Amcrest IP5M-1190EW running
2.810.00AC004.0.R. A full get_config() returns 52 sections in ~24s and
the set_config() key form round-trips, but several guesses were wrong:
networkInterface.cgi does not exist, netApp.cgi?action=getInterfaces is
the real endpoint.
userManager.cgi reports a real per-user Id starting at 1, so get_users
was handing callers the array position instead of the number the camera
actually recognises.
modifyUser rejects a partial record with 400. update_user now reads the
user back and sends the whole record with the changes merged over it.
getSoftwareVersion and getHardwareVersion both answer with a bare
'version=', so device_info's flat merge lost one of them. The software
one is a single line with the build appended - '2.810.00AC004.0.R,
build:2023-09-04' - so getVersion splits it and returns the build
separately in list context.
The configManager list is now the set that model actually implements.
UPnP and Login are gone, they answer 400; Lighting_V2, WLan, NAS, Record,
Alarm and the VideoIn* sections are added.
That model has no getCurrentProtocolCaps and no coaxialControlIO, and it
answers OK to Lighting/Lighting_V2 writes and then ignores them, so its
WhiteLight is not drivable over http. Documented rather than papered
over. update_firmware stays the one unverified path, deliberately.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LzHrvNtF13vyYBAJjktam6
set_config prepended the section name to keys that already carried it,
so NTP.Address went out as NTP.NTP.Address. get_config only strips the
'table.' prefix, which leaves keys in exactly the form setConfig wants,
so drop the extra prefix and let reads and writes round-trip.
uri_encode was called but URI::Encode was never imported, and the plain
call leaves reserved characters alone, which corrupts passwords in a
query string. Import it and always escape reserved characters.
get_config now reads configManager.cgi sections as well as the handful
of dedicated cgis, retries once on a 401 so the fresh digest nonce gets
used, strips the 'table.' prefix, and tries any unrecognised name as a
configManager section so a template can pull in sections the module
doesn't list.
set_config fell off the end of its loop returning undef on success.
reboot POSTed system_reset=1 to setparam.cgi, a Vivotek endpoint, on a
relative url through the raw user agent. ZoneMinder::General was used
but never required.
Adds device info, time, users, snapshot/rtsp/profiles/probe, white light
and siren, and the remaining PTZ stops and auto pan, matching the method
names cameratool.pl probes for with ->can().
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LzHrvNtF13vyYBAJjktam6
The openbsd branch and master both changed the top fallback in CpuUsage
and diverged. Combine them:
- Keep master's /proc/stat diagnostics: report why /proc/stat could not
be read (ProcSubset=pid mount namespace vs open failure), default the
fields split out of top output so an empty result does not warn under
-w, and only complain once per process.
- Keep the openbsd branch's parse_bsd_top_cpu(): OpenBSD top prints CPU
states in a format neither the FreeBSD nor the Linux grep matches, so
when the greps produce no numbers, parse raw top output. It handles
both the aggregate "CPU states:" line and per-core "CPU0 states:"
lines, averaging over the lines seen, and does not depend on padding.
- Carry over scripts/ZoneMinder/t/server_cpu.t covering it.
Tests: prove -Iscripts/ZoneMinder/lib scripts/ZoneMinder/t/server_cpu.t
-> 16/16 pass. perl -c on Server.pm is clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kfiom9RefM75GTZFkqpJo2
zmpkg.pl hands start/stop/restart to systemd via zmsystemctl.pl, so that pid 1
forks the daemons and they get zoneminder.service's cgroup and mount namespace
rather than the web server's. The detection defeated itself: systemdRunning()
read pid 1's name out of /proc, which a web server running with
ProtectProc=invisible hides from the web user. Starting from the web ui
therefore concluded systemd was absent, skipped the delegation and ran zmdc
inside the web server's namespace, where ProcSubset=pid hides /proc/stat and
every figure zmstats records is wrong.
Use -d /run/systemd/system, which is what sd_booted(3) does and stays readable
however /proc is mounted.
calledBysystem() had to change with it or the two would combine into a start
loop, systemd running zmpkg which asks systemd to run zmpkg. It read the
parent's name from /proc, equally hidden, and the parent is pid 1 only until
it re-parents us. Ask our own cgroup whether we are already the service
zmsystemctl.pl would start. Not INVOCATION_ID: systemd sets it for every
descendant of a unit, so anything forked from the web ui inherits the web
server's copy and would wrongly look systemd-started.
Also check the exit status of the zmsystemctl.pl call. It was discarded, so a
refused pkexec left the command cleared and nothing started, with nothing
logged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019URmtYqza6Rzi6F7cmabSm
/proc/stat is invisible to any process in a mount namespace set up with
ProcSubset=pid, which is systemd's default hardening for apache. Daemons
started from the web ui are forked off mod_php and inherit it, so zmstats
fell through to the top fallback, which reads /proc/stat as well and so
produced no output either.
Separate the missing-file case from the failed-open case so the message
names the likely cause instead of printing a $! that a failed -e never set.
Emit the warning once per process rather than once per
ZM_STATS_UPDATE_INTERVAL. The condition does not clear itself, so it was
filling zmstats.log with the same three lines a minute.
split() on an empty $top_output returns an empty list, leaving all four
values undef, so each fallback cycle also emitted four "Use of
uninitialized value" warnings under -w. Default them before use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019URmtYqza6Rzi6F7cmabSm
zmwatch fetches its monitor list once at the top of a pass and then walks it.
Deleting a monitor mid-pass stops its zmc from the web ui, so by the time
zmwatch reaches that monitor its shared memory is gone, zmMemVerify fails and
zmwatch calls control('restart') from the now-stale list. zmc comes back for a
monitor that is marked Deleted, so no later pass ever looks at it again and
nothing stops it until zmpkg.pl restart. That is the orphaned zmc left behind
after deleting a monitor from the console.
Re-read Deleted from the database in ZoneMinder::Monitor::control() before
running a start or a restart, and skip the command if the monitor has been
deleted or the row is gone. control() is the single path every restart goes
through, so this covers any caller working from a list it fetched earlier.
Stopping is deliberately still allowed, since that is how an orphan gets
cleaned up.
Tests: tests/perl/test_monitor_control_deleted.pl stubs zmDbFetchOne and
runCommand and checks that start/restart still run for a live monitor, are
skipped for a deleted one and for one removed from the database, and that stop
is unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NDhTBPj9xEaT52pRmAufaP
Adds a web config option whose contents are rendered on the login view,
between the title and the username and password fields. Useful for a site
notice, an acceptable use or legal warning, or a note identifying which
installation this is when running more than one.
Defaults to empty, and nothing is emitted when it is empty or whitespace, so
existing installs look exactly as they do now.
The text is escaped rather than interpreted. The login page is served before
anyone has authenticated, so it is not somewhere to emit admin-supplied markup,
and no other config value in the skin is output unescaped either. Escaping runs
before nl2br so the only tags reaching the browser are the line breaks we add
ourselves; reversing that order would turn the setting into stored XSS.
Uses the text type, so the Options UI renders a textarea and the value can span
lines. The Config.Value column is already text and options.php normalises CRLF
to LF on save, so no schema change is needed and nl2br sees consistent
newlines.
Guarded with defined() to match the surrounding code, so the view still renders
where the database predates the option.
Styled in base, classic and dark. Text contrast is 7.0:1 light and 7.5:1 dark,
both above WCAG AA, and long unbroken tokens wrap rather than widening the
fixed-width form.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The capture daemons and this watchdog share the database. If the DB (or
the host/network) drops out, every zmc stops updating its shared-memory
heartbeat at the same instant. When the DB connection recovers, the next
check pass saw the whole fleet as simultaneously stale and restarted --
and, for cameras with control capability, rebooted -- every monitor, even
though none of them individually failed.
Track wall-clock time between check passes. If we just came out of a DB
reconnect, or far more time elapsed than a normal check interval (host
stall/clock step), skip restart+reboot actions for one pass so capture
daemons can refresh their heartbeats. A genuinely dead camera is still
stale next pass and gets handled normally -- reboot behaviour is
preserved, just no longer fired on a false fleet-wide positive.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HoNiNxwgyaHV29CbiueCUf
The mp4 is written as the event records, so its mtime is when recording
finished, not when it started. Set EndDateTime to the mtime and
StartDateTime to duration seconds before it, so Length agrees with
EndDateTime - StartDateTime.
The directory mtime fallback also ran unconditionally after both
branches, overwriting the times just computed and the Deep-scheme
path-derived start. Only use it when there are neither capture jpgs nor
an mp4.
Adds t/event_recover_timestamps.t, which builds a 3 second mp4 with
ffmpeg and checks the recovered times against its mtime.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BV1gDc9T6D6H8gtTcxwLvk
The DateTime filter attribute was taught the "event overlaps this
instant/window" idiom in web/includes/FilterTerm.php: a lower bound (>=/>)
compares COALESCE(EndDateTime, '9999-12-31 23:59:59') so still-running events
are included, while an upper bound keeps comparing StartDateTime. The Perl
Filter.pm still emitted E.StartDateTime for that attribute, so a saved filter
using DateTime with >=/> selected different events in the web UI than in
zmfilter.pl, which drives automated delete/email/execute actions.
Apply the same column selection in ZoneMinder::Filter::Sql so the daemon and
the web agree. Verified by generating SQL from both sides for =, <, <=, >, >=:
the column expressions are now byte-identical.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KQipwf632JGgNH4W7p8cqs
Replace the single alarm_image slot with an analysis_image_buffer ring of
image_buffer_count Images living in the already-reserved alarm_images SHM
region. Successive WriteAlarmImage calls rotate through the ring and
publish last_analysis_index last (after the bytes and per-slot format),
so a reader sampling last_analysis_index always sees a fully written
slot. GetAlarmImage returns that slot, syncing its AVPixelFormat from the
per-slot analysis_image_pixelformats array.
SharedData gains last_analysis_index and analysis_image_count (plus 8
bytes of padding to keep the 16-byte-multiple layout), making it 888
bytes. The Perl (Memory.pm) and PHP (Monitor.php) SHM readers are updated
in lockstep, and a static_assert(sizeof(SharedData)==888) in zm_monitor.h
guards the layout against silent drift.
This lets multiple in-flight analysis/annotated frames be buffered and
streamed in sync rather than always overwriting one slot, and gives the
AI object-detection work a place to publish annotated frames.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The C++ and PHP parsers for zm.conf and /etc/zm/conf.d/*.conf used fgets
with a 512-byte buffer, silently truncating any line longer than that.
There was also no way to split a value across multiple lines, so editors
hit the cap with no workaround.
Accept a trailing backslash (with optional whitespace before the newline)
as a line-continuation marker. Leading whitespace on continuation lines
is stripped so users can indent for readability without it leaking into
the value. Three parsers all read the same files and must agree:
- src/zm_config.cpp: switch to std::ifstream + std::getline so a single
physical line is no longer capped at 512 bytes, then join continuation
lines before running the existing pointer-based parser
- scripts/ZoneMinder/lib/ZoneMinder/Config.pm.in: accumulate a logical
line across trailing-backslash physical lines
- web/includes/config.php.in: drop the 512-byte fgets cap and accumulate
in the same way
tests/zm_config.cpp covers three cases against the C++ parser: a
three-segment continuation joins to "firstsecondthird" with leading
whitespace stripped, a bare backslash inside a value is preserved
(C:\Users\zm), and a 1500-byte single line survives intact.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
#4914 unconditionally drops Javascript errors and CSP violations sourced from
browser extensions. Some operators want to know when a plugin is touching the
ZoneMinder tab, so gate the suppression behind a config entry.
Add ZM_LOG_BROWSER_EXTENSIONS (boolean, default no) in the logging category.
It is exposed to JS by the existing non-private-config loop in skin.js.php as
the string '0'/'1' (the same convention as ZM_LOG_INJECT), so logger.js only
filters extension sources when it is '0'. Default behaviour is unchanged:
extension noise stays out of the log.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
zmwatch only gated the camera reboot attempt on CanReboot(), so it called
$control->open() even when the camera was unreachable - the common reason a
monitor has no image since startup - and blocked until the connection timed
out, logging an error each pass.
Add a ping check before open(). Move the host resolution into Control so callers
don't have to dig the ip out of the Path: add Control::host(), which returns the
cached host or derives it from the monitor's ControlAddress/Path via the shared
guess_credentials() (parsing only, no network i/o), and have ping() fall back to
it. ping() still accepts an explicit ip.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The hand-rolled regex in open() mis-parsed a stream Path that carried no
credentials, e.g. rtsp://10.0.0.4:554/cam: the greedy [^:@]+ consumed the host
as the username, 554 as the password, and ADDRESS backtracked to the single
digit 4. The control daemon then dialled http://...@4, producing
"Can't connect to 4:80 (Connection timed out)".
Replace the regex with the shared URI-based Control.pm guess_credentials(),
which handles ControlAddress and Path, converts rtsp to http, and falls back to
the Monitor User/Pass. Create the LWP::UserAgent before parsing since
guess_credentials() sets credentials on it. The Grandstream login wire protocol
(challenge/authcode and old-style fallback) is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On systems with many cameras the Grandstream control log entries gave no
indication of which monitor they referred to. Prefix every Debug/Warning/Error
message with the monitor Id and Name. Also fix two message typos
(challengstring, UNable).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a ZM_DB_SSL_VERIFY_SERVER_CERT setting so a database connection that uses
ZM_DB_SSL_CA_CERT can talk to a server with a self-signed or otherwise
non-matching certificate. When enabled, verification is by identity (the cert
must chain to the CA and its CN/SAN must match ZM_DB_HOST), consistent across
the C++ daemons, the PHP web interface, the CakePHP API and the Perl scripts.
This re-does the reverted #3817. That PR broke the build because it called
mysql_options(MYSQL_OPT_SSL_VERIFY_SERVER_CERT, ...), and that enum was removed
from the MySQL 8.0 C client in favour of MYSQL_OPT_SSL_MODE; it also passed a
c_str() where a my_bool* was expected, and referenced the PHP constant
unconditionally (fatal on PHP 8 for an upgraded install whose zm.conf predates
the option).
The option that controls server-cert verification differs by client library and
the symbols are enum values, not macros, so CMake feature-detects them by
compiling:
- HAVE_MYSQL_OPT_SSL_MODE (MySQL 5.7.11+/8.0, MariaDB Connector/C 3.1+)
- HAVE_MYSQL_OPT_SSL_VERIFY_SERVER_CERT (older MariaDB/MySQL)
zm_db.cpp uses SSL_MODE_VERIFY_IDENTITY / SSL_MODE_REQUIRED when the former is
available, else falls back to the latter with a proper my_bool.
Value handling is three-way in every layer: a truthy value verifies, a false-y
value (0/false/no/off) skips verification, and an empty/unset value leaves the
client default in place so existing installs are unchanged on upgrade. PHP, the
API datasource (via PDO flags) and the Perl DSN are all guarded with defined()
checks. Fresh installs default to 1.
Documents the full ZM_DB_* connection and SSL settings, including the hostname
verification gotcha when connecting by IP, in docs/userguide/configfiles.rst.
refs #3816
open() is contracted to return true or false so callers (zmcontrol.pl,
zmwatch.pl) can tell whether the camera is reachable, but it always ended
with an assignment that evaluated truthy and reported success regardless.
When the initial login probe failed it also rebuilt BASE_URL in the old
basic-auth style and returned without ever testing that connection.
Return 0 on a failed probe and 1 only after a successful exchange. Check
is_success() on the authcode response in the modern path, and actually issue
and check a request on the old-style URL in the fallback path so success
means we can talk to the camera. Log the previously unused ResCode.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two spots wrote temp files outside ZoneMinder's controlled temp tree:
- zmvideo.pl concat mode built its ffmpeg concat list at a predictable
path, /tmp/<concat_name>.concat.lst, in world-writable /tmp. A
predictable name there is open to a symlink/race and leaks monitor and
event names. Create it instead with File::Temp (randomized name, atomic
O_EXCL) inside ZM_TMPDIR. The list entries are absolute paths, so its
location does not affect ffmpeg's resolution.
- web/ajax/training.php created its detection scratch image with
tempnam(sys_get_temp_dir(), ...), escaping ZM's temp tree and its
cleanup. Use tempnam(ZM_DIR_TEMP, ...) so it stays under the configured
temp dir.
Both now resolve to the per-distro temp dir (e.g. /var/lib/zoneminder/temp
on RedHat, /var/tmp/zm on Debian), keeping scratch files inside the tree
that packaging and systemd hardening already cover.
refs #2915
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ZoneMinder Perl modules carried a GPL-2-or-later notice in their file
header but ended with an h2xs-scaffolding POD footer stating the code could
be used "under the same terms as Perl itself" (Artistic-1.0-or-GPL-1+). The
two statements contradict each other.
Every affected file has the same copyright holder in both the header and the
footer, and was contributed to ZoneMinder under the GPL-2+ header; the
Perl-terms line is inherited boilerplate that was never the intended license.
The project as a whole is GPL-2+ (see COPYING).
Replace the contradictory POD paragraph with a one-line pointer to the GPL so
each file states a single, consistent license without duplicating the full
notice already present in the header. Copyright and author attribution lines
are left unchanged.
Affects 60 .pm and .pm.in files under scripts/ZoneMinder/lib/ZoneMinder.
fixes#817
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
