ZM_AUTH_HASH_SECRET signs the JWT access and refresh tokens, and its
default is a fixed string in the public source. Nothing generated a
per-install value and tokens signed with the default verified normally,
so on an install with auth on and the secret untouched anyone could sign
an admin token and be accepted by the web UI, the API and zms.
- ZoneMinder::Config::saveConfigToDB() now replaces an empty or default
secret with 32 random bytes from /dev/urandom, hex encoded. Package
installs and upgrades run zmupdate.pl -f, which saves the config, so
existing installs get a secret on upgrade. A secret the admin set is
left alone.
- validateToken() in PHP and zmLoadTokenUser() in C++ refuse to verify
tokens while the secret is empty or the default, and the API refuses to
issue them, as it already did for an empty secret.
- zmLoadTokenUser() no longer writes the signing key to the debug log.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GenerateVideo used DefaultVideo as the ffmpeg input relative to the event
directory and built the output file from Name with only whitespace
replaced, so an Events=Edit user could make zmvideo read or write any
path the account can reach. zmfilter's generateImage passed
event_path/DefaultVideo to ffmpeg -i, and the %EV% email tag attached
event_path/DefaultVideo, which could mail out any readable file.
Strip everything up to the last path separator from DefaultVideo in all
three places, and reduce Name to [-A-Za-z0-9_.] with no leading dot
before using it as the video filename.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 5904023f13448e557dc205edb9440972e4a6482d)
executeCommand substituted %TAG% placeholders in AutoExecuteCmd with raw
event and monitor fields and ran the result with qx(), so the shell parsed
those values. Several are writable by non-admin users: event Name, Cause
and Notes (Events=Edit via ajax/event.php and the API), event tags
(Events=Edit), monitor Name (Monitors=Edit), and Cause/Notes are also set
by zmtrigger clients on TCP 6802. An event named $(cmd) therefore ran cmd
as the web/zm user whenever an admin's AutoExecute command used %EN%,
%EC%, %ED%, %ETAGS%, %MN% and similar tags.
Each tag is now substituted on its own. Values made only of word
characters, spaces and /.,:+=@%- are inserted as before, so commands using
%EID%, %MID%, %EPATH% and similar keep working unchanged. Any other value
is exported as an environment variable ZM_TAG_n and referenced in a form
the shell expands without re-parsing, chosen from the quoting context the
admin placed the tag in: "${ZM_TAG_n}" when bare, ${ZM_TAG_n} inside
double quotes and '"${ZM_TAG_n}"' inside single quotes. Such a value
now always reaches the command literally; a bare one is no longer split
into several words.
substituteTags itself is unchanged, so email and message bodies behave
as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit b9891a4a443f1f5e0d8faa7026518a7463ec9440)
With ZM_AUTH_HASH_IPS on, the client address bound into the auth hash was
taken from the left-most X-Forwarded-For value whenever the header was
present, both when PHP generated the hash (getRemoteAddr()) and when PHP or
zms validated it (getAuthUser(), zmLoadAuthUser()). The header is client
controlled, so anyone holding a leaked hash could replay it from anywhere by
sending the address it was bound to.
Add ZM_AUTH_TRUSTED_PROXIES, a list of exact reverse proxy addresses.
X-Forwarded-For is now used only when REMOTE_ADDR is one of them, and is read
from the right, skipping hops that are themselves listed proxies, so values a
client prepends are never chosen. With the option empty, the default, the
header is ignored and REMOTE_ADDR is used.
PHP (web/includes/Network.php getRemoteAddr()) and C++ (ClientAddress() in
zm_utils, used by zmLoadAuthUser()) implement the same rule so generation and
validation continue to agree. Every PHP caller already routes through
getRemoteAddr(), so session.php and auth.php need no change.
Reverse proxy users who enable ZM_AUTH_HASH_IPS must list their proxy in the
new option; until they do, hashes bind to the proxy address, which still
validates but no longer distinguishes clients. This is the behaviour change
that the #4921 work avoided by trusting the header.
The option is added through ConfigData only, like other recent options;
zmupdate.pl --freshen inserts it, zms falls back to the compiled-in default and
PHP treats an undefined constant as empty, so no schema migration is needed.
Tests: ClientAddress Catch2 case; tests/php/test_remote_addr.php updated for
the trusted-proxy rule.
refs GHSA-72rf-54rm-798c
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit acea5889dec0826f595cb736147a5fdc9c94a2a9)
GHSA-pxq8-5c8j-xf3r reports the event Name reaching a shell through
Event::GenerateVideo. That sink is already closed by "fix: run the event
video encoder without a shell" (GHSA-pfph-4j9j-7cv7), which runs ffmpeg as a
list. Reviewing the rest of the Perl video path turned up one more place an
event field reaches a shell: zmfilter's generateImage, used to attach frames
to filter emails and messages when no capture jpeg exists, built
ffmpeg -nostdin -ss <Delta> -i '<event path>/<DefaultVideo>' ...
and ran it with qx(). DefaultVideo is settable by an operator with
Events=Edit, so a single quote in it ends the quoted argument. The -r check
before it means the named file has to exist, but GenerateVideo itself will
create one whose name is the event Name, so the two fields together are
enough. Driving generateImage directly with DefaultVideo set to
"a';touch${IFS}<marker>;'.mp4" (and that file present) created the marker
before this change.
ffmpeg is now run through a list-form piped open, so there is no shell and
the path is one argument whatever it holds. stdout is still collected for the
debug log as qx() did; if ffmpeg cannot be started, status is reported as
127, as the shell did for a missing command. With the change the same input
passes the literal path to ffmpeg and creates no marker.
See GHSA-pxq8-5c8j-xf3r.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 0f700d30e8b5e67705243029b57005236ccbc3ff)
zmfilter.pl rebuilds SQL from the JSON the web tier stores verbatim in
Filters.Query_json, so a saved filter's attribute, operator and value all
arrive at the statement exactly as they were written. None of the three were
checked here, while web/includes/FilterTerm.php checks all three: values were
wrapped in quotes by hand, which a value containing one closes; the final
attribute branch concatenated 'E.'.$attr; and the generic operator branch
emitted the operator as written.
Any account that can save a filter could therefore have the daemon run SQL of
their choosing with its own credentials, against a table set that includes
Users. The daemon also deletes and moves events, so this is not only a read.
Confirmed against the module before the change, building SQL from stored terms
and checking whether the payload landed outside a string literal:
val = "x' OR 1=1 -- " -> outside, injected
attr = "Id=1 OR 1=1 -- " -> outside, injected
op = "= 1 OR 1=1 -- " -> outside, injected
Attributes now have to match what the PHP accepts, operators have to be one of
the sixteen the UI can produce, and a filter carrying anything else is refused
rather than run. Values go through DBI's quote().
Numbers are left unquoted so the statements the existing filters produce are
unchanged: a bare number cannot carry SQL, and quoting it would have altered
every numeric comparison in the file for no gain. Verified by generating the
SQL for seven representative filters before and after -- purge-when-full,
delete-after-a-day, LIKE on a name, IN on monitors, a monitor name, a score
range and an IS NULL -- and diffing: identical apart from the timestamp a
relative date resolves to.
After the change the three payloads above are contained in a literal, or the
filter is refused.
See GHSA-p8h3-4x5c-cv7p.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBHBB95RBX7D9p8ge2WDZb
(cherry picked from commit bc11642fea844bc86bac0080da42e4514d5a4d76)
Event::GenerateVideo built an ffmpeg command line as a string and ran it
through qx(), and two of the values in it are event fields an operator with
Events=Edit can set through the API. DefaultVideo was interpolated with no
quoting at all. The name behind the output filename only has its whitespace
replaced, so a single quote in it closed the quoting that was there. Either one
gave arbitrary command execution as the account the daemons run as.
Both confirmed against the module before the change, driving GenerateVideo
directly:
DefaultVideo = "x.mp4; touch <marker>; echo" -> marker created
Name = "evt';touch${IFS}<marker>;echo'" -> marker created
The second needs ${IFS} rather than spaces because the name has whitespace
substituted before it is used, which is the whole of the sanitising that was
being relied on.
The command is now a list passed to exec, so there is no shell to escape from
whatever those fields hold; ffmpeg's own output still lands in the event's
ffmpeg.log. The two configured option strings are admin-set and hold several
options each, so they are split on whitespace to become separate arguments;
shell quoting inside them is no longer honoured.
After the change both payloads run ffmpeg with the injection as a literal
argument and create no marker.
See GHSA-pfph-4j9j-7cv7.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBHBB95RBX7D9p8ge2WDZb
(cherry picked from commit 5fc491728c679bc7a8ff4b51183983e7fe184200)
The long options --shell/--command are util-linux only. OpenBSD su takes
-s for the shell and passes arguments after the login name to it, and
util-linux su accepts the same form.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reverts the previous commit's always-on measurement. Decoding audio for every
monitor that has it spends CPU on a number almost nothing reads, which is the
wrong trade even though it did solve the chicken and egg of picking a
threshold without ever seeing a level.
Measure when something is actually going to use the reading instead:
- AudioDetection is on, as before, so nothing changes for a monitor that
scores on audio; or
- somebody asked. SharedData gains audio_level_until, a wall clock second
the capture thread keeps measuring up to. The monitor editor's new level
meter pushes it forward while it is on screen and the measurement lapses a
few seconds after the page is left, so nothing has to send a stop and a
crashed browser cannot leave a monitor decoding forever.
When the reading stops being wanted the decoder is released and the published
level and peak are cleared, so a stale number is not left looking current and
an old peak does not land on the next frame row written.
audio_level_until is carved out of analysis_pad rather than appended, so
SharedData stays 888 bytes and no existing offset moves; the static_asserts,
Memory.pm and Monitor.php are updated together and all three now agree the
field is at +880.
The meter itself is on the audio settings, shown whether or not
AudioDetection is checked, because the level is what you need in order to
choose a threshold. It draws the threshold currently in the input as a mark on
the bar so a reading can be judged against it before saving, and a monitor
whose zmc is not running reads "no reading" rather than a confident 0, which
would be indistinguishable from silence.
Frames.AudioLevel is therefore 0 again on monitors that do not score on audio.
That is what the graph already treats as "no audio data", so it draws no line
rather than a flat one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JpiSWBmtQkR5bcgpHWY4ME
A speaker that has dropped off the network fails on every command until
somebody fixes it, so that is the one error in IPSpeaker that repeats forever
rather than once. On a monitor deliberately marked unimportant it is noise,
and it buries the failures worth acting on.
Nothing about that is specific to speakers, so the policy goes in Logger as
importanceLevel, with ErrorImportance and WarningImportance alongside the
plain Error and Warning for callers to use. It takes the importance value
itself rather than a monitor, so Logger needs to know nothing about monitors
and callers that have some other notion of how much something matters can
still use it.
zmwatch.pl has been open coding the same idea as
WARNING+$monitor->ImportanceNumber() in three places; it now calls
WarningImportance instead, which is the same arithmetic and so leaves its
levels exactly as they were, including the Not important case that lands in
DEBUG1. That case is pinned by a test rather than quietly corrected: it is
long standing behaviour and not this change's business. zmwatch no longer
needs the logger object it was keeping for logPrint, so logInit() is called
bare there as it is in every other script.
Anything that is not a number counts as Normal, so a caller with no monitor
to ask still reports in full: not knowing how much a monitor matters is no
reason to hide its faults.
IPSpeaker is then a one line change at the call site. Only the failure to
reach the device is weighed; a refusal or unparseable content means the device
answered and something is really wrong, which is worth an error however
unimportant the monitor is, and does not repeat the same way.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JpiSWBmtQkR5bcgpHWY4ME
LWP::UserAgent defaults to 180 seconds. Actions for a monitor are serialised
through one control daemon, so a speaker that has dropped off the network holds
that daemon for three minutes per attempt and every command queued behind it
waits. AMLink already sets a timeout; this did not.
Ten seconds, matching AMLink. The speaker answers in milliseconds when it is
reachable at all, so this only ever bounds the case where it is not - which is
the case right now.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
The camera does not report an expired session as a JSON error. It answers with
a bare printable string, "Invalid session in request", where a masked base64
payload belongs. Unmasking that and base64 decoding it produces noise, so a
routine timeout was logged as "malformed JSON string ... at character offset
0" - which named neither the session nor the camera's own explanation.
Detect it before unmasking, log what the camera actually said, and treat it as
a lost session so the existing recovery re-establishes it. session_error is
pure so the wording seen on the wire is pinned by a test rather than by a
camera happening to be in the right state.
refs #4423
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
zmcontrol sent keepAlive only from the branch that runs when select() times
out. A monitor taking a steady stream of commands never reaches that branch,
so it never pinged at all, and cameras that expire a session on a timer
refresh it on the keepAlive rather than on ordinary requests. The monitors
with the most traffic were therefore the ones that lost their session.
Measured on monitor 1: it pinged twice while idle, then took a command every
three seconds for three minutes and the camera rejected the next one 181
seconds after the last ping. Its light commands were being dropped for as long
as the motion kept coming, which is exactly when they matter.
Move the decision into Control::keepAliveDue, timed from the last ping, and
check it at the top of the loop so the 'next' paths in the command handling
cannot skip it. A clock step backwards resets the baseline rather than holding
the ping off until real time catches up.
refs #4423
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
An undecodable reply means the camera and this module no longer agree about
the session, but nothing put that right. Dahua_RPC re-logs-in only when it gets
a parseable error back, so rpc_call returning undef left the session poisoned:
in the logs a CoaxialControlIO.control failure is followed by every later
keepAlive and logout on that session failing the same way, until something else
forces a login. A light switched on by an alarm stays on for that whole period.
Split the single attempt out as rpc_once, recording why it failed, and have
rpc_call re-login and retry once when the failure was a decode failure on a
masked Request.
The guards matter more than the retry:
- bootstrap channels never recover, since login() issues them
- login() calls logout(), a Request on the dead session, so a re-entrancy guard
stops its own decode failure starting another recovery
- one retry only, and a 5 second backoff, so a camera that always answers
unreadably is not met with a login per command
refs #4423
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
The decode failure diagnostic appended its fields after $@, which ends in a
newline, so the payload length, alignment, unmasked-decode result and leading
bytes were pushed onto a second line where nothing looked for them.
$@ was also read after the unmasked-decode eval had already reset it, so the
error reported was that eval's outcome rather than the failure being described.
Add log_safe to flatten a message before logging, use it for the three places
that interpolate $@, capture the real error before running another eval, and
report the decoded byte count and the decoded leading bytes - the base64 alone
does not say whether what failed to parse was truncated or simply not ours.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
The audio_fifo_path field was retired alongside video_fifo_path when the
media FIFOs were replaced by the stream socket. video_fifo_path was
repurposed as stream_socket_path, but a single socket carries both
streams so there is no separate audio path to publish; reserved_path2
stays reserved.
Spell out the convention where the field is defined, so a future reader
does not remove or reorder it (which would shift every later field for
out-of-tree shm readers) and knows the slot is free to reuse at the same
64-byte width. Align the PHP Monitor object and ZoneMinder::Memory notes.
refs #5143
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4UcdJLt1bxwdpcigGxZRD
Address monitor-side stream socket review findings:
- zmc starts the stream socket before the first connect attempt, and
SendStreamHealthEvent records the health state even when the socket is
not up yet, so connection/prime faults during startup are observable to
a consumer instead of being lost until the first successful prime.
- PrimeCapture announces audio whenever the camera has a decodable audio
stream (Capture forwards audio packets unconditionally), and clears a
previously announced stream when a re-prime no longer sees it.
- Publish the media stream socket path in the monitor shared-memory
block (reusing the retired video_fifo_path field as stream_socket_path,
same offset and size) so consumers discover it without hard-coding the
convention or reading the producer's zm.conf; the PHP Monitor object
and the ZoneMinder::Memory Perl module expose the renamed field.
- Move the wall-clock microseconds helper out of zm_monitor.cpp into
zm_time.h as SystemClockMicros(), where time helpers belong.
refs #5143
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4UcdJLt1bxwdpcigGxZRD
Rename video_fifo_path/audio_fifo_path to reserved_path1/2, keeping the
struct layout and 864-byte size unchanged so existing shared memory
mappers (and out-of-tree shm readers) keep working through the
transition. The fields are zeroed at startup as before, scrubbing any
stale path strings left by a pre-upgrade zmc.
The stream socket deliberately publishes nothing via shared memory: its
path is the documented convention PATH_SOCKS/stream_{monitor_id}.sock,
derivable from static config, and codec parameters travel in the HELLO
handshake instead of shm fields.
Perl (Memory.pm) and PHP shared memory maps updated to match by name;
offsets are unchanged.
Tests: full suite passes via ctest; php -l clean on Monitor.php.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The scripts run under -T, so they cannot trust the caller's PATH and set
their own. Sixteen of them hardcoded /bin:/usr/bin:/usr/local/bin, which
assumes everything they shell out to lives under /usr or /usr/local.
ZoneMinder::General::findDbCommand looks for a database client on that
PATH, and zmupdate.pl and zmcamtool.pl run what it finds. So an install
whose client sits anywhere else cannot apply schema changes:
sh: mysql: command not found
Command 'mysql -u'zmuser' ... ' exited with status: 127
even with the client on the caller's PATH. Homebrew on Apple Silicon is
the case that surfaced it - the client is in /opt/homebrew/bin - but a
--prefix=/opt install on Linux has the same shape, as does anything that
keeps its database client outside the FHS locations.
Replaced the literal with @ZM_SCRIPT_PATH@, defaulting to the same three
directories plus wherever cmake actually found a client, and overridable
for packagers who want to pin it. Warn at configure time when no client
is found at all, since that failure otherwise appears much later and
says something unrelated.
Nothing changes for an install whose client is already under /usr/bin:
the directory is only appended when it is not in the list, so the
default stays exactly as it was.
Memory.pm is deliberately left alone. It has a narrower PATH of
/bin:/usr/bin, and the only command it runs is uname through an absolute
path from ZM_PATH_UNAME, so it does not need widening.
Verified on macOS across the three cases: with the client in
/opt/homebrew/bin the directory is appended; -DZM_SCRIPT_PATH= is
respected verbatim; and pointing detection at /usr/bin/mariadb leaves
the default untouched. Build clean, suite 146 cases.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5KL9Xbi7K5aGsauLtd8tG
scripts/ZoneMinder/lib/ZoneMinder/Control/ held both ONVIF.pm and onvif.pm, the
only pair of paths in the tree differing solely in case. On a case insensitive
filesystem git can materialise just one of them, so a macOS clone reports the
loser as modified forever and git add -A commits one module's contents over the
other.
The runtime consequence is worse than the checkout noise. Every seeded Controls
row uses Protocol='ONVIF', and zmcontrol.pl builds the module name from that
column. Where onvif.pm is the file that survived, require ZoneMinder::Control::ONVIF
still succeeds because the filename matches, but it defines the lowercase package,
so the bless lands in an empty ::ONVIF and the first method call dies.
ONVIF.pm has replaced onvif.pm since the unified module landed. Of the subs only
onvif.pm defines, all but horizontalPatrol and horizontalPatrolStop exist there
under underscore-prefixed names, and every ONVIF-protocol Controls row has
CanAutoScan=0, so nothing can reach those two.
Nothing ever moved existing installs onto the new protocol name, so add
zm_update-1.39.29.sql to do it. zm_update-1.35.23.sql sent Protocol='onvif' to
FoscamCGI, which was right in 2021 when onvif.pm held the Foscam CGI protocol and
was copied to FoscamCGI.pm, but onvif.pm was afterwards replaced with a real ONVIF
implementation and the seed row restored, so rows written since belong on ONVIF.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The session leak is fixed - no "too many connections" since the restart - but
decode failures continue, so there is a second cause. The old message named
only the symptom, which is why the first diagnosis chased the wrong thing.
The failure now reports the payload length, whether that length is a multiple
of four as base64 requires, whether the reply decodes without unmasking, and
the leading bytes. Those separate the plausible causes: a camera answering
unmasked, a truncated reply, and a reply masked with a key we no longer share.
If the reply does decode unmasked it is returned rather than discarded, since
an unmasked answer is still an answer.
What is already ruled out, so the next person does not repeat it: a stale
session still answers masked and decodes cleanly, just without a result field;
LWP's decoded_content is byte-identical to content here; and the masked payload
did not collide with the </cmd> terminator in 48 replies across six sessions.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
The control daemon filled its log with
failed to decode the reply to CoaxialControlIO.control: malformed JSON
on every lightOn, while the light itself still worked. Three faults in this
module compounding, all mine.
Nothing ever logged out. close() was inherited from Dahua_RPC, which only sets
a flag, so every login abandoned a session. The camera counts connections and
reclaims them slowly, and zmcontrol keeps one object for the life of the daemon
while Dahua_RPC re-logins whenever the camera times the session out - the log
shows that happening every half hour. Each one leaked a slot until the camera
answered global.login with "too many connections!", which it is still doing
here hours later.
login() then made a transient failure permanent. It cleared session and
mask_key up front, so a login that failed left the object with no key. Several
Dahua_RPC callers re-login on error and carry on without checking the result,
so every later command was sent unmasked, came back masked, and failed to
base64-decode. The logged error therefore described the symptom two steps
downstream of the cause and never mentioned the session at all.
So:
- logout() gives the session back, and close() calls it
- login() releases the previous session before taking another
- rpc_call refuses to send on the masked channel with no key, and says the
session is gone rather than emitting an unreadable decode error
- a login refused for "too many connections" is reported as that
global.logout, with no parameters on the ordinary Request channel, confirmed
against the camera's own web bundle rather than guessed.
Tested: 5 new assertions that a Request with no key returns undef and puts
nothing on the wire, while Login, OutsideCmd and the key exchange - which
legitimately have no key yet - are still sent. Perl suite 15 files / 266
assertions.
Not yet verified against the camera: it is still refusing logins with "too many
connections" from the sessions already leaked, so the logout path could not be
exercised end to end. It needs a retry once the camera has reclaimed them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
Points the camera at an NTP server and sets how often it syncs. The cameras
were on time.windows.com once a day, or in one case at an address on the LAN
that does not answer NTP at all, so it was not syncing.
UpdatePeriod is in minutes and 1 is accepted - established by writing it and
reading it back, since the firmware has no getConfigCaps to ask. That is as
often as the camera will go, so it can be held within a minute of the server.
Only Address, Enable and UpdatePeriod are written. TimeZone and TimeZoneDesc
are deliberately left alone: the camera's clock already reads correctly and
the index is not a standard one, 26 for "Middletime" here against a factory
default of 25 "Easterntime". Changing a timezone we do not understand to fix a
sync interval would be a poor trade.
The write is a whole-section merge that echoes back every field the camera
returned, so a partial write cannot silently drop one, and set_time skips the
write entirely when the settings already match. The camera holds this in
flash and the method is meant to be safe to re-run - from cron, for instance -
so an unconditional write would spend erase cycles for nothing. Same reasoning
as FoscamHD::set_time.
Tested: 12 new assertions over the merge and the change detection, including
that untouched fields survive, that the caller's hash is not mutated, and that
1440 against "1440" does not read as a change - which would otherwise rewrite
flash on every run. Verified against both cameras: one syncing from
time.windows.com every 1440 minutes and one pointed at a LAN address that does
not answer NTP at all every 60, both now on the local server every 1, and
re-running was a no-op. Perl suite 15 files / 261 assertions.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
These cameras have a white light that ONVIF does not expose at all: imaging
offers only Brightness/ColorSaturation/Contrast/Sharpness, GetRelayOutputs is
empty with RelayOutputs="0", and there is no PTZ service and so no auxiliary
commands. The light lives behind a vendor JSON-RPC tunnel.
The vocabulary turns out to be Dahua's -- CoaxialControlIO for the light,
magicBox for reboot -- so this subclasses Dahua_RPC and replaces only open,
rpc_call and login. lightOn, lightOff, lightStatus and reboot are inherited
unchanged, and Dahua_RPC's lightStatus already parses the
params.status.WhiteLight these cameras return.
The transport is the whole of the work, and none of it is discoverable by
probing:
* The endpoint is /Onvif/device_service with a capital O. The lowercase
/onvif/device_service is the real ONVIF SOAP service; the capitalised
spelling is a separate vendor tunnel sharing the path. /RPC2, /RPC3,
/OutsideCmd and every /cgi-bin/* return 404 on this model, which is why
the API looked absent.
* Requests are <body><cmdType>T</cmdType><cmd>P</cmd></body>. Without
cmdType the camera drops the connection rather than returning an error.
* P is base64 of the JSON-RPC object, XOR-masked with a session key once one
exists. Masking is mandatory: an unmasked Request is refused even on a
freshly authenticated session, so the key exchange is not optional.
* That key needs a three step bootstrap -- log in, fetch the device RSA
public key over the unauthenticated OutsideCmd channel, then send an AES
key encrypted under it and decrypt the mask key from the reply. The salt
doubles as the AES key and must be 32 decimal digits; 16 is accepted by
the AES step and then rejected by getGeneralKey.
IO polarity matches Dahua_RPC: IO 1 is on, IO 2 is off. Worth stating because
the camera's own web UI is a toggle that does not reveal which is which, and
because a floodlight cannot be verified optically in daylight -- an RTSP
brightness comparison appeared to say the opposite and was exposure drift.
CoaxialControlIO.getStatus is the reliable witness.
Tested: 26 assertions over the pure transport helpers -- mask symmetry and
cycling, NUL bytes surviving the mask, the envelope, stripping the CRLF the
device pads its replies with, and the channel routing, which fails silently on
the wire if wrong. Verified live against both units: open negotiates a 32 byte
mask key and lightStatus reports Off -> On -> Off around lightOn/lightOff. Perl
suite 15 files / 249 assertions pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
ZoneMinder has carried audio for years without ever listening to it: the
packets go into the event and nothing reads them. AudioDetector decodes
them in the capture thread and reports a 0-100 level, which Analyse()
turns into score alongside motion, ONVIF and Amcrest.
The level is dBFS-derived, not a raw amplitude ratio. Linear RMS is
unusable as a setting: ordinary speech sits at 1-3% of full scale, so
every sound worth catching would be crammed into the bottom two points of
the range and no operator could tune it. Mapping -60..0 dBFS onto 0..100
puts speech around 43 instead.
Three columns on Monitors: AudioDetection to enable it, AudioThreshold
for the level to alarm at, and AudioAlarmScore for what it contributes.
AudioAlarmScore defaults to 9, matching what an ONVIF or Amcrest alarm
already adds. A threshold of 0 means off, so enabling detection without
choosing a threshold cannot alarm on silence -- a plain level >= threshold
test would alarm on every packet in that state.
The level and the alarm flag go in SharedData's two spare bytes, renamed
from reserved1/reserved2. Offsets and sizes are unchanged, so the
888-byte cross-process layout and the Memory.pm and Monitor.php offset
tables all still agree; the readers are renamed in the same commit so the
level is available to the web UI.
The decoder is opened lazily on the first audio packet rather than at
camera setup, so a monitor with detection off never carries one and a
stream that gains audio on reconnect still gets picked up. Scoring reads
the capture thread's most recent level rather than scoring per audio
packet, because the score belongs to a video frame and audio packets do
not arrive in step with them.
Tested: 242 assertions over the pure helpers - the RMS of both sample
formats, the dB scale's monotonicity and endpoints, full-scale clamping
of decoder overshoot, and the threshold-0 case. The repo's Catch2 harness
needs v3 and only v2 is installed here, so tests/zm_audio_detector.cpp
was compiled against a v2 shim to check it, and an identical set of
assertions was run standalone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
A camera that detects motion should be able to sound a speaker, and the
speaker is rarely the camera: it is a separate device with its own address,
credentials, stream and Controls entry. Model it as a monitor and let a
monitor's alarm drive actions on other monitors.
Add Monitors.DeviceClass enum('Camera','Speaker'). This is what the device
is, as distinct from Type, which selects the capture backend - an IP speaker
still captures over Ffmpeg like any other RTSP device, so Type could not
carry the distinction.
Add the MonitorActions table: MonitorId is the monitor that triggers,
TargetMonitorId the device acted on, and the two are frequently different.
TriggerOn covers EventStart, EventEnd, Alarm and Manual.
Which actions a device is offered is decided by its Controls row - CanLight,
CanIndicatorLight, CanAudioPlay - so a device can only be asked to do what it
has been measured to do. The editor filters on this and the save path
re-checks it, because the request is not to be trusted.
Execution goes straight to the target's zmcontrol socket rather than forking
zmcontrol.pl per action: the daemon already accepts a line of JSON there, and
it is the same path the control panel uses. ActionCommandName maps the DB
enum onto a method name as a whitelist, so nothing out of the database
reaches the control daemon uninspected. Actions are fire-and-forget - a
speaker that is offline is logged and skipped, never allowed to hold up event
handling.
Alarm actions fire only on the genuine entry into alarm, not on the
ALERT->ALARM re-entry, which would re-sound a speaker within one incident.
EventEnd runs on the calling thread before the event is handed to the closing
thread, which does not capture `this`.
Manual actions appear as buttons on the watch page, and are the reason the
control panel is now shown for a monitor that has actions but no control of
its own. Firing one sends only the action id; the command and target are
rebuilt server side, and Control rights are required on the target device and
not merely on the monitor the action hangs off.
Also add --file to zmcontrol.pl, without which audioPlay could not be driven
from the command line. The web path was unaffected as it bypasses GetOptions.
Tests: tests/zm_monitor_action.cpp covers the command whitelist, the message
format (including that file id 0 is a real id and that a stale AudioFile is
never passed to a command that takes none), trigger names, and that every
value of the ActionType enum maps to a command.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
(cherry picked from commit b561341e988af69c3a46db854da410309f7cfa82)
An IP speaker is controllable but has none of the capabilities the Controls
table models: nothing moves, focuses or lights up. It plays a sound file it
already holds, selected by a numeric id, and carries an output volume.
Add CanAudioPlay, MinAudioFile, MaxAudioFile and CanAudioVolume to Controls,
with migration zm_update-1.39.21.sql. CanAudioPlay renders one button per
sound id plus a stop button; MinAudioFile/MaxAudioFile bound that range;
CanAudioVolume renders the volume pair. The sound id travels in the command
name (audioPlay12), the way presets already do, because a control button has
no way to attach a separate parameter.
Add ZoneMinder::Control::IPSpeaker driving the device's own HTTP interface,
and a Controls entry for it. Playback goes over the vendor interface rather
than ONVIF because the ONVIF audio output service on this firmware only
describes the output and offers no way to start a stored file.
Measured against a device reporting ONVIF Manufacturer "IPSpeaker" and
firmware CS20-V3.3.45N:
- File ids fall in two windows, 10-14 built-in and 20-30 operator uploads.
Anything else is refused with result -2, and an id in a window with no file
uploaded with result -3, so valid_fileid refuses only the former: an empty
slot is a legitimate id the operator may fill later.
- config=audio.set replaces the whole audio section, so a partial write
reverts the microphone, codec list and echo-cancellation settings. Every
volume change reads the section and echoes it back with the new level.
audio.get also returns outmute, which audio.set rejects, so it is excluded
from the field list.
- The volume reads back and round-trips, but the device's RTSP audio is a
pre-volume tap of the playback signal: it is unchanged with outvolume at 0,
so it cannot confirm the loudspeaker's acoustic output. CanAudioVolume is
set on the strength of the setting persisting, and both the POD and the
seed row say the acoustic effect is unconfirmed.
The seed row covers the built-in window only, because a fresh device has
nothing uploaded and the firmware refuses an empty slot.
Adding columns to Controls extends the 43 positional INSERTs in
db/controls.sql, which carry no column list and so must match the column
count exactly.
Tests: scripts/ZoneMinder/t/ip_speaker.t, 32 assertions covering file id
windows, volume clamping and stepping, request building and the whole-section
write. All pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
(cherry picked from commit d07e552a3c58373f7f0ccebf9aa28adb4d51baa3)
The original fix replaced the hardcoded 000 token in getCamParams and
_setImaging, but get_config/set_config landed upstream afterwards and
carry three more imaging requests that still address VideoSource 000:
the ImagingSettings and ImagingOptions entries in %config_types, and the
SetImagingSettings body in set_config.
Cameras that number their sources differently reject all three. On an
AMLINK AL5M-T5171EW the video source is 00000 and a request for 000 comes
back as "The requested VideoSource does not exist.", so the imaging half
of the config API is unusable on those cameras even with the earlier fix
applied.
The query bodies now carry a __VIDEO_SOURCE_TOKEN__ placeholder, matching
how __PROFILE_TOKEN__ already works, and get_config substitutes it only
when the body contains it so unrelated categories do not pay for a
GetVideoSources round trip. set_config calls _video_source_token()
directly, which is cached after the first lookup.
The added tests assert on the module source because %config_types is a
file-scoped lexical. That is deliberate: the failure this guards against
is a new imaging call being written with a literal token again, which is
exactly how get_config/set_config reintroduced the bug.
Verified against both live AMLINK units: GetVideoSources returns 00000 on
each, and GetImagingSettings answers for 00000 while faulting for 000.
Perl suite 13 files / 191 assertions pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
getCamParams() and _setImaging() addressed the imaging service with a
literal VideoSourceToken of 000. That token is a different namespace from
the media ProfileToken carried in ControlDevice, so it could not be
configured around: on an AMLINK AL5M-T5171EW the media profiles are
MediaProfile00000 while the video source is 00000, and the camera answers
a request for 000 with "The requested VideoSource does not exist."
Brightness (Iris) and Contrast (White) control were unusable on any camera
that does not happen to number its first video source 000.
Read the token from GetVideoSources on first use and cache it. The
fallback to 000 is applied per call rather than cached, so a camera that
is unreachable when the control daemon starts gets another chance instead
of being pinned to the wrong token for the life of the daemon.
Parsing is split out as video_source_token_from_xml() so it can be tested
without a camera, matching the pattern used by the HikVision light
helpers.
Verified against a live AMLINK AL5M-T5171EW: token discovered as 00000,
irisAbsOpen(step 10) moved Brightness 50 -> 60, restored to 50.
(cherry picked from commit 7f1935d69d58b3e2bbf71a54fe11b1a6b4bc30af)
Object::save() returns the error string on failure and '' on success, so
`if ($manufacturer->save())` took the success branch only when the save had
failed. ManufacturerId and ModelId were therefore assigned from an unsaved
object and left unset whenever the save actually worked.
Invert the test, capture the error string, and log it. Model.pm needs
ZoneMinder::Logger imported for Error().
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both loops called $dbh->ping() and discarded the result. ping() only tests
the connection; DBD::mysql does not reconnect on its own since
mysql_auto_reconnect is off. zmupdate's update-check loop sleeps 3600s
between iterations, so on a server with wait_timeout below an hour the
handle is already closed on wake-up and the following zmDbDo fails with
"The client was disconnected by the server because of inactivity".
zmDbConnect() already pings and reconnects when needed, and assigns the
package global $dbh that zmDbDo uses, so call that instead. zmupdate keeps
passing mysql_multi_statements so a reconnect matches the original handle.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Upstream's schema-drift fix landed db/zm_update-1.39.25.sql and bumped
version.txt to 1.39.25 while the trigger consolidation was using the same
number for db/zm_update-1.39.25.sql.in.
Git does not see this as a conflict -- the two files have different names in
the source tree -- but configure_file generates the .sql.in into
zm_update-1.39.25.sql, which is the same install target as the tracked .sql.
Confirmed by staging an install off the merge: the installed
zm_update-1.39.25.sql was upstream's column reconciliation and the trigger
consolidation was simply absent, with no error anywhere. An upgrade would
have applied one of the two and silently skipped the other.
Renumbered to 1.39.26, version.txt to match, and the two zmstats.pl.in
comments that name the migration updated. Upstream's 1.39.25 is untouched.
Verified by staging an install again: both files are now present and hold
what they should. The migration was re-run end to end at the new number
against a scratch database -- drops the eight cascade triggers, leaves four,
and the 29 trigger assertions pass afterwards.
ai_server carries the old 1.39.25 numbering and will need the same treatment
when master is next merged into it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
The guard skipped the delay when a filter was named. zmpkg starts this as
"zmdc.pl start zmfilter.pl --filter_id=N --daemon", so it always names one --
the boot path never waited, and the only invocation that did was a hand-run
scan of every filter. Exactly backwards from what the delay is for.
--daemon is what marks the unattended case, so gate on that instead, still
skipping when there is a terminal on stdin.
Measured with Filter::Execute stubbed to return no events, so nothing was
deleted or emailed, timing the first call to it:
invocation before after
--filter_id=1 --daemon (zmpkg) 0.1s 5.1s
--filter_id=1 (hand-run) 0.1s 0.1s
no args, scan all filters 5.1s 0.1s
So filters now start 5 seconds later at boot, which is the point of the
constant, and a hand-run returns immediately.
Also sets zmstats' START_DELAY to 5. The previous commit's message said it
did this and it did not: the edit was written against "=> 5" when the file
said "=> 30", so it silently matched nothing while the interactive skip
beside it landed. Verified this time by running the generated script both
ways -- non-interactive logs "starting in 5 seconds", interactive logs
nothing and proceeds.
That run also closes the gap the previous commit noted: startedInteractively
resolves at runtime in the generated zmstats.pl. It had only been checked
with perl -Tc, which does not catch an unimported sub called with parens --
an earlier run against a stale module path failed with exactly that error and
sent me looking.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
zmwatch slept 30 seconds before its first pass. The reason is real: when it
starts, zmc has not necessarily created its shared memory or written a
heartbeat, and neither state is distinguishable from a camera that has died,
so the first pass would restart every monitor.
A fixed sleep is the wrong shape for that. It is not derived from anything --
ZM_WATCH_CHECK_INTERVAL is 10 and ZM_WATCH_MAX_DELAY is 45, so the delay was
three check intervals and shorter than the staleness threshold it was
standing in for. On a busy host with many cameras, or one camera slow to
answer, 30 seconds is not enough and the fleet gets restarted anyway. It also
blocks every other check, and makes running zmwatch by hand a 30 second wait.
Each monitor now gets until ZM_WATCH_MAX_DELAY after zmwatch started -- the
same threshold used to judge a heartbeat stale -- to appear, and only if we
have never yet seen it healthy. Once seen healthy it is judged immediately, so
a camera that dies later is caught exactly as fast as before, and every check
other than the restart runs from the first pass.
Verified against the two monitors on a machine with no daemons running, which
is the boot state, with Monitor::control stubbed so nothing was actually
started. Before: both monitors restarted on the first pass after the sleep,
and again every 10 seconds. After: no restart for 45 seconds, then both
restarted on the first pass past the grace, at 50 seconds. So a monitor that
never comes up is still caught, ~20 seconds later than before at boot and at
the same speed as before thereafter.
Also adds ZoneMinder::General::startedInteractively and uses it to skip the
start delay in zmstats.pl and zmfilter.pl when a person ran them. zmdc.pl
reopens STDIN on /dev/null for everything it starts, so a terminal on STDIN
reliably means a hand-run; zmtelemetry.pl already relies on this. A run from
cron or a unit file has no terminal either and still waits, which is the right
way round. zmstats keeps a delay because it is polite while zmc and zma are
competing for the machine, but nothing in it races with startup -- it connects
to the database first and has its own reconnect loop -- so the value is now 5
rather than 30.
Note zmfilter's existing guard is inverted with respect to that intent:
zmpkg starts it as "--filter_id=N --daemon", and the guard skips the delay
when a filter is named, so the daemon path never waited and only a manual
scan-all did. Left as it is here beyond adding the interactive case; worth a
separate look.
The zmstats and zmfilter changes are not runtime tested. Running either
against a live install prunes and rewrites rows, and there was no throwaway
install to point them at; startedInteractively is verified directly in both
directions, and all three generated scripts pass perl -Tc.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
setContrast takes 'constrast'. Spelling the parameter 'contrast', as
the field is named everywhere else including in getImageSetting's own
answer, is accepted with result=0, leaves the real parameter unset and
applies 0 for it. Contrast 0 is a black picture, so set_config would
have blacked out any camera whose contrast a template touched, while
reporting success.
Found by doing it to a live FI9853EP: the camera kept serving video
with a correct timestamp overlay and no error anywhere, and the only
sign was getImageSetting answering contrast 0 where an untouched
camera of the same model answered 50.
field_set entries are now [command, parameter] pairs rather than a bare
command with the parameter assumed from the field name, since the
firmware gives no indication when it is handed a name it does not read.
The same check against an untouched camera clears brightness, hue,
saturation and sharpness: those were written with their documented
names during the same session and kept their values.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
set_time wrote unconditionally. Because the offset it corrects only
moves at a daylight saving change, running it from cron - which is the
point of it being idempotent - meant thousands of writes a year to the
camera's flash for the two that matter.
Read the section first and return success without writing when every
field we would set already holds the wanted value. Measured against an
FI9853EP: a repeat run now issues one getSystemTime and no write, while
a moved offset still reads, merges and writes once.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
FI9821W_Y2k, FI9831W and FOSCAMR2C all talk to the same
/cgi-bin/CGIProxy.fcgi endpoint that FoscamHD now implements settings
for, so reparent them onto it instead of ZoneMinder::Control.
Method resolution keeps each module's own new, open, sendCmd and PTZ
commands and picks up get_config, set_config, set_time, cgi and
rtsp_url from FoscamHD, so how these cameras move is unchanged.
FoscamHD resolves the address and credentials on demand rather than in
open() precisely so this works: these modules have their own open()
which builds a UserAgent without working out either.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
Everything from the FI98xx generation onward answers a settings API on
/cgi-bin/CGIProxy.fcgi, but nothing in the tree reads or writes it - the
Foscam modules we ship only move the camera. That left no way to see or
change a camera's settings except its web ui, which on these needs an
IE6 plugin.
Adds get_config/set_config over 14 sections, plus set_time to point a
camera at an NTP server.
Two behaviours were measured against an FI9853EP on firmware 2.22.2.15
and drive the implementation:
Whole-section writers do not merge. A setSystemTime that leaves out
timeFormat and timeZone sets both to 0 rather than leaving them alone;
a partial write was observed to reset timeFormat 1 -> 0 and timeZone
14400 -> 0. set_config therefore reads the section back and writes the
merged result, never the diff on its own.
setSystemTime refuses isDst=1 outright, answering -1, so a daylight
saving offset has to be folded into timeZone. timeZone is seconds west
of UTC - EDT, UTC-4, is 14400 - which is the POSIX sign and the
opposite of a tm_gmtoff. Because DST lives in timeZone, set_time has
to be re-run when the offset changes; it is idempotent so cron is fine.
VideoStreamParam is read-only because getVideoStreamParam answers
resolution0..3 while setVideoStreamParam takes a single streamType plus
unsuffixed fields, so the read shape cannot be merged back into a
write. IPInfo is read-only because a wrong value takes the camera off
the network, where this module can no longer reach it. ImageSetting
has one command per field; setDenoiseLevel answers -3 on this firmware
so denoiseLevel is readable but not writable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
Events_Hour, Events_Day, Events_Week and Events_Month each carried their
own update and delete trigger, and every one issued a separate UPDATE
against the same Event_Summaries row. A single Event delete therefore
locked that row five times: once from event_delete_trigger and once from
each bucket's cascade. That is the deadlock zmstats.pl hits when it bulk
deletes aged rows out of Events_Hour while zmc and zma are writing.
event_update_trigger and event_delete_trigger now modify the bucket tables
themselves and apply one consolidated UPDATE, using ROW_COUNT() after each
bucket statement to tell whether the event was still in that bucket so
aged-out events do not over-adjust the counters. Measured on a scratch
database, an Event delete goes from 5 Event_Summaries row updates to 1.
This also fixes a drift bug. The old event_update_trigger kept
ArchivedEventDiskSpace correct only in a branch that cannot be reached: it
sits under IF (NEW.Archived != OLD.Archived) and requires both to be
false. The branch that does run when an already-archived event grows
updated Events_Archived but not Event_Summaries, so the archived total
drifted for the life of the install. Reproduced on a scratch database:
after archiving a 250-byte event and growing it to 999,
ArchivedEventDiskSpace still read 100.
BEHAVIOUR CHANGE. A direct DELETE against a bucket table no longer adjusts
Event_Summaries at all, and that is how zmstats.pl prunes. zmstats.pl
already resyncs HourEvents/DayEvents/WeekEvents/MonthEvents and their disk
space columns from COUNT(*)/SUM(DiskSpace) on any pass where it pruned, so
those four pairs become eventually consistent within one
ZM_STATS_UPDATE_INTERVAL instead of exact at every instant. The Total and
Archived columns stay exact, because only the Events triggers touch them.
The zmstats.pl comments are updated to describe the new arrangement; its
code is unchanged.
Migration is db/zm_update-1.39.25.sql.in: drop the eight cascade triggers,
resync Event_Summaries from the events themselves so the new triggers start
from ground truth and the archived drift above is repaired, then source
db/triggers.sql. version.txt goes to 1.39.25. No schema change, so
zm_create.sql.in needs no edit -- it already sources triggers.sql.
Ported from the ai_server branch, where this migration sits at 1.39.7, a
number master passed long ago and an existing install would never run.
Repackaged above master's tip. The ai_server version also unwinds a
views-and-SWR experiment that only ever existed on that branch, which is
dropped here as a no-op on master, and carries an unrelated START_DELAY
change, which is not taken.
Tests: tests/perl/test_event_summaries_triggers.pl, 29 assertions against a
real server, skipped when no scratch database is configured. Verified they
fail on the current triggers, on exactly the two claims above: the archived
drift, and 5 row updates per delete. Also verified the migration repairs a
drifted install, is idempotent across a second run, and leaves a migrated
install with byte-identical triggers to a fresh one. Generated zmstats.pl
passes perl -Tc.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
The C++ loader reads back only the rows where Value differs from DefaultValue
and takes compiled-in defaults for the rest, so the two columns have to be
written in the same form. Both writers passed Value through a boolean
conversion and DefaultValue through none: a boolean left alone was stored as
Value '1' against DefaultValue 'yes', never compared equal, and was read back
on every start. The filter did nothing for the 77 boolean rows.
Both columns now go through ConfigData::dbValue. It lives there because the
two writers - saveConfigToDB for an existing install and zmconfgen for the
zm_create.sql of a fresh one - have to agree, and drifting apart is what
caused this. An option with no default is stored as its type's empty value,
matching the empty string initialiseConfig already gives it.
The generated zm_create.sql now has Value equal to DefaultValue for all 259
rows, so a fresh install reads no Config rows at all. The generated
zm_config_defines.h is byte identical, confirming the compiled-in defaults
are unaffected by the representation change.
tests/perl/test_config_default_value.pl checks the invariant over every
option in ConfigData; it needs no database, since ConfigData does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
ConfigData wrote boolean defaults and requires clauses as 'yes'/'no' while
the Config table has always held '1'/'0', so every writer and reader had to
convert between the two. yes/no is not a boolean value and the web UI never
showed it - booleans render as a checkbox, which reads Value directly and
ignores the Hint - so the string form bought nothing and only existed to be
translated away again.
Converts the 75 boolean defaults, the 59 requires clauses that test them, and
the boolean entry in %types. OPT_FFMPEG is substituted into a boolean default
by cmake, so it changes with them.
Requires clauses reach the Config table byte for byte as before: they were
already converted to 1/0 on the way in, and the web UI string-compares them
against the stored Value.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
