scripts/ZoneMinder/lib/ZoneMinder/Control/ held both ONVIF.pm and onvif.pm, the
only pair of paths in the tree differing solely in case. On a case insensitive
filesystem git can materialise just one of them, so a macOS clone reports the
loser as modified forever and git add -A commits one module's contents over the
other.
The runtime consequence is worse than the checkout noise. Every seeded Controls
row uses Protocol='ONVIF', and zmcontrol.pl builds the module name from that
column. Where onvif.pm is the file that survived, require ZoneMinder::Control::ONVIF
still succeeds because the filename matches, but it defines the lowercase package,
so the bless lands in an empty ::ONVIF and the first method call dies.
ONVIF.pm has replaced onvif.pm since the unified module landed. Of the subs only
onvif.pm defines, all but horizontalPatrol and horizontalPatrolStop exist there
under underscore-prefixed names, and every ONVIF-protocol Controls row has
CanAutoScan=0, so nothing can reach those two.
Nothing ever moved existing installs onto the new protocol name, so add
zm_update-1.39.29.sql to do it. zm_update-1.35.23.sql sent Protocol='onvif' to
FoscamCGI, which was right in 2021 when onvif.pm held the Foscam CGI protocol and
was copied to FoscamCGI.pm, but onvif.pm was afterwards replaced with a real ONVIF
implementation and the seed row restored, so rows written since belong on ONVIF.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The session leak is fixed - no "too many connections" since the restart - but
decode failures continue, so there is a second cause. The old message named
only the symptom, which is why the first diagnosis chased the wrong thing.
The failure now reports the payload length, whether that length is a multiple
of four as base64 requires, whether the reply decodes without unmasking, and
the leading bytes. Those separate the plausible causes: a camera answering
unmasked, a truncated reply, and a reply masked with a key we no longer share.
If the reply does decode unmasked it is returned rather than discarded, since
an unmasked answer is still an answer.
What is already ruled out, so the next person does not repeat it: a stale
session still answers masked and decodes cleanly, just without a result field;
LWP's decoded_content is byte-identical to content here; and the masked payload
did not collide with the </cmd> terminator in 48 replies across six sessions.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
The control daemon filled its log with
failed to decode the reply to CoaxialControlIO.control: malformed JSON
on every lightOn, while the light itself still worked. Three faults in this
module compounding, all mine.
Nothing ever logged out. close() was inherited from Dahua_RPC, which only sets
a flag, so every login abandoned a session. The camera counts connections and
reclaims them slowly, and zmcontrol keeps one object for the life of the daemon
while Dahua_RPC re-logins whenever the camera times the session out - the log
shows that happening every half hour. Each one leaked a slot until the camera
answered global.login with "too many connections!", which it is still doing
here hours later.
login() then made a transient failure permanent. It cleared session and
mask_key up front, so a login that failed left the object with no key. Several
Dahua_RPC callers re-login on error and carry on without checking the result,
so every later command was sent unmasked, came back masked, and failed to
base64-decode. The logged error therefore described the symptom two steps
downstream of the cause and never mentioned the session at all.
So:
- logout() gives the session back, and close() calls it
- login() releases the previous session before taking another
- rpc_call refuses to send on the masked channel with no key, and says the
session is gone rather than emitting an unreadable decode error
- a login refused for "too many connections" is reported as that
global.logout, with no parameters on the ordinary Request channel, confirmed
against the camera's own web bundle rather than guessed.
Tested: 5 new assertions that a Request with no key returns undef and puts
nothing on the wire, while Login, OutsideCmd and the key exchange - which
legitimately have no key yet - are still sent. Perl suite 15 files / 266
assertions.
Not yet verified against the camera: it is still refusing logins with "too many
connections" from the sessions already leaked, so the logout path could not be
exercised end to end. It needs a retry once the camera has reclaimed them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
Points the camera at an NTP server and sets how often it syncs. The cameras
were on time.windows.com once a day, or in one case at an address on the LAN
that does not answer NTP at all, so it was not syncing.
UpdatePeriod is in minutes and 1 is accepted - established by writing it and
reading it back, since the firmware has no getConfigCaps to ask. That is as
often as the camera will go, so it can be held within a minute of the server.
Only Address, Enable and UpdatePeriod are written. TimeZone and TimeZoneDesc
are deliberately left alone: the camera's clock already reads correctly and
the index is not a standard one, 26 for "Middletime" here against a factory
default of 25 "Easterntime". Changing a timezone we do not understand to fix a
sync interval would be a poor trade.
The write is a whole-section merge that echoes back every field the camera
returned, so a partial write cannot silently drop one, and set_time skips the
write entirely when the settings already match. The camera holds this in
flash and the method is meant to be safe to re-run - from cron, for instance -
so an unconditional write would spend erase cycles for nothing. Same reasoning
as FoscamHD::set_time.
Tested: 12 new assertions over the merge and the change detection, including
that untouched fields survive, that the caller's hash is not mutated, and that
1440 against "1440" does not read as a change - which would otherwise rewrite
flash on every run. Verified against both cameras: one syncing from
time.windows.com every 1440 minutes and one pointed at a LAN address that does
not answer NTP at all every 60, both now on the local server every 1, and
re-running was a no-op. Perl suite 15 files / 261 assertions.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
These cameras have a white light that ONVIF does not expose at all: imaging
offers only Brightness/ColorSaturation/Contrast/Sharpness, GetRelayOutputs is
empty with RelayOutputs="0", and there is no PTZ service and so no auxiliary
commands. The light lives behind a vendor JSON-RPC tunnel.
The vocabulary turns out to be Dahua's -- CoaxialControlIO for the light,
magicBox for reboot -- so this subclasses Dahua_RPC and replaces only open,
rpc_call and login. lightOn, lightOff, lightStatus and reboot are inherited
unchanged, and Dahua_RPC's lightStatus already parses the
params.status.WhiteLight these cameras return.
The transport is the whole of the work, and none of it is discoverable by
probing:
* The endpoint is /Onvif/device_service with a capital O. The lowercase
/onvif/device_service is the real ONVIF SOAP service; the capitalised
spelling is a separate vendor tunnel sharing the path. /RPC2, /RPC3,
/OutsideCmd and every /cgi-bin/* return 404 on this model, which is why
the API looked absent.
* Requests are <body><cmdType>T</cmdType><cmd>P</cmd></body>. Without
cmdType the camera drops the connection rather than returning an error.
* P is base64 of the JSON-RPC object, XOR-masked with a session key once one
exists. Masking is mandatory: an unmasked Request is refused even on a
freshly authenticated session, so the key exchange is not optional.
* That key needs a three step bootstrap -- log in, fetch the device RSA
public key over the unauthenticated OutsideCmd channel, then send an AES
key encrypted under it and decrypt the mask key from the reply. The salt
doubles as the AES key and must be 32 decimal digits; 16 is accepted by
the AES step and then rejected by getGeneralKey.
IO polarity matches Dahua_RPC: IO 1 is on, IO 2 is off. Worth stating because
the camera's own web UI is a toggle that does not reveal which is which, and
because a floodlight cannot be verified optically in daylight -- an RTSP
brightness comparison appeared to say the opposite and was exposure drift.
CoaxialControlIO.getStatus is the reliable witness.
Tested: 26 assertions over the pure transport helpers -- mask symmetry and
cycling, NUL bytes surviving the mask, the envelope, stripping the CRLF the
device pads its replies with, and the channel routing, which fails silently on
the wire if wrong. Verified live against both units: open negotiates a 32 byte
mask key and lightStatus reports Off -> On -> Off around lightOn/lightOff. Perl
suite 15 files / 249 assertions pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
ZoneMinder has carried audio for years without ever listening to it: the
packets go into the event and nothing reads them. AudioDetector decodes
them in the capture thread and reports a 0-100 level, which Analyse()
turns into score alongside motion, ONVIF and Amcrest.
The level is dBFS-derived, not a raw amplitude ratio. Linear RMS is
unusable as a setting: ordinary speech sits at 1-3% of full scale, so
every sound worth catching would be crammed into the bottom two points of
the range and no operator could tune it. Mapping -60..0 dBFS onto 0..100
puts speech around 43 instead.
Three columns on Monitors: AudioDetection to enable it, AudioThreshold
for the level to alarm at, and AudioAlarmScore for what it contributes.
AudioAlarmScore defaults to 9, matching what an ONVIF or Amcrest alarm
already adds. A threshold of 0 means off, so enabling detection without
choosing a threshold cannot alarm on silence -- a plain level >= threshold
test would alarm on every packet in that state.
The level and the alarm flag go in SharedData's two spare bytes, renamed
from reserved1/reserved2. Offsets and sizes are unchanged, so the
888-byte cross-process layout and the Memory.pm and Monitor.php offset
tables all still agree; the readers are renamed in the same commit so the
level is available to the web UI.
The decoder is opened lazily on the first audio packet rather than at
camera setup, so a monitor with detection off never carries one and a
stream that gains audio on reconnect still gets picked up. Scoring reads
the capture thread's most recent level rather than scoring per audio
packet, because the score belongs to a video frame and audio packets do
not arrive in step with them.
Tested: 242 assertions over the pure helpers - the RMS of both sample
formats, the dB scale's monotonicity and endpoints, full-scale clamping
of decoder overshoot, and the threshold-0 case. The repo's Catch2 harness
needs v3 and only v2 is installed here, so tests/zm_audio_detector.cpp
was compiled against a v2 shim to check it, and an identical set of
assertions was run standalone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
An IP speaker is controllable but has none of the capabilities the Controls
table models: nothing moves, focuses or lights up. It plays a sound file it
already holds, selected by a numeric id, and carries an output volume.
Add CanAudioPlay, MinAudioFile, MaxAudioFile and CanAudioVolume to Controls,
with migration zm_update-1.39.21.sql. CanAudioPlay renders one button per
sound id plus a stop button; MinAudioFile/MaxAudioFile bound that range;
CanAudioVolume renders the volume pair. The sound id travels in the command
name (audioPlay12), the way presets already do, because a control button has
no way to attach a separate parameter.
Add ZoneMinder::Control::IPSpeaker driving the device's own HTTP interface,
and a Controls entry for it. Playback goes over the vendor interface rather
than ONVIF because the ONVIF audio output service on this firmware only
describes the output and offers no way to start a stored file.
Measured against a device reporting ONVIF Manufacturer "IPSpeaker" and
firmware CS20-V3.3.45N:
- File ids fall in two windows, 10-14 built-in and 20-30 operator uploads.
Anything else is refused with result -2, and an id in a window with no file
uploaded with result -3, so valid_fileid refuses only the former: an empty
slot is a legitimate id the operator may fill later.
- config=audio.set replaces the whole audio section, so a partial write
reverts the microphone, codec list and echo-cancellation settings. Every
volume change reads the section and echoes it back with the new level.
audio.get also returns outmute, which audio.set rejects, so it is excluded
from the field list.
- The volume reads back and round-trips, but the device's RTSP audio is a
pre-volume tap of the playback signal: it is unchanged with outvolume at 0,
so it cannot confirm the loudspeaker's acoustic output. CanAudioVolume is
set on the strength of the setting persisting, and both the POD and the
seed row say the acoustic effect is unconfirmed.
The seed row covers the built-in window only, because a fresh device has
nothing uploaded and the firmware refuses an empty slot.
Adding columns to Controls extends the 43 positional INSERTs in
db/controls.sql, which carry no column list and so must match the column
count exactly.
Tests: scripts/ZoneMinder/t/ip_speaker.t, 32 assertions covering file id
windows, volume clamping and stepping, request building and the whole-section
write. All pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
(cherry picked from commit d07e552a3c58373f7f0ccebf9aa28adb4d51baa3)
The original fix replaced the hardcoded 000 token in getCamParams and
_setImaging, but get_config/set_config landed upstream afterwards and
carry three more imaging requests that still address VideoSource 000:
the ImagingSettings and ImagingOptions entries in %config_types, and the
SetImagingSettings body in set_config.
Cameras that number their sources differently reject all three. On an
AMLINK AL5M-T5171EW the video source is 00000 and a request for 000 comes
back as "The requested VideoSource does not exist.", so the imaging half
of the config API is unusable on those cameras even with the earlier fix
applied.
The query bodies now carry a __VIDEO_SOURCE_TOKEN__ placeholder, matching
how __PROFILE_TOKEN__ already works, and get_config substitutes it only
when the body contains it so unrelated categories do not pay for a
GetVideoSources round trip. set_config calls _video_source_token()
directly, which is cached after the first lookup.
The added tests assert on the module source because %config_types is a
file-scoped lexical. That is deliberate: the failure this guards against
is a new imaging call being written with a literal token again, which is
exactly how get_config/set_config reintroduced the bug.
Verified against both live AMLINK units: GetVideoSources returns 00000 on
each, and GetImagingSettings answers for 00000 while faulting for 000.
Perl suite 13 files / 191 assertions pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTCzCbvGt8xKQRNCSA7o8
getCamParams() and _setImaging() addressed the imaging service with a
literal VideoSourceToken of 000. That token is a different namespace from
the media ProfileToken carried in ControlDevice, so it could not be
configured around: on an AMLINK AL5M-T5171EW the media profiles are
MediaProfile00000 while the video source is 00000, and the camera answers
a request for 000 with "The requested VideoSource does not exist."
Brightness (Iris) and Contrast (White) control were unusable on any camera
that does not happen to number its first video source 000.
Read the token from GetVideoSources on first use and cache it. The
fallback to 000 is applied per call rather than cached, so a camera that
is unreachable when the control daemon starts gets another chance instead
of being pinned to the wrong token for the life of the daemon.
Parsing is split out as video_source_token_from_xml() so it can be tested
without a camera, matching the pattern used by the HikVision light
helpers.
Verified against a live AMLINK AL5M-T5171EW: token discovered as 00000,
irisAbsOpen(step 10) moved Brightness 50 -> 60, restored to 50.
(cherry picked from commit 7f1935d69d58b3e2bbf71a54fe11b1a6b4bc30af)
Object::save() returns the error string on failure and '' on success, so
`if ($manufacturer->save())` took the success branch only when the save had
failed. ManufacturerId and ModelId were therefore assigned from an unsaved
object and left unset whenever the save actually worked.
Invert the test, capture the error string, and log it. Model.pm needs
ZoneMinder::Logger imported for Error().
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
zmwatch slept 30 seconds before its first pass. The reason is real: when it
starts, zmc has not necessarily created its shared memory or written a
heartbeat, and neither state is distinguishable from a camera that has died,
so the first pass would restart every monitor.
A fixed sleep is the wrong shape for that. It is not derived from anything --
ZM_WATCH_CHECK_INTERVAL is 10 and ZM_WATCH_MAX_DELAY is 45, so the delay was
three check intervals and shorter than the staleness threshold it was
standing in for. On a busy host with many cameras, or one camera slow to
answer, 30 seconds is not enough and the fleet gets restarted anyway. It also
blocks every other check, and makes running zmwatch by hand a 30 second wait.
Each monitor now gets until ZM_WATCH_MAX_DELAY after zmwatch started -- the
same threshold used to judge a heartbeat stale -- to appear, and only if we
have never yet seen it healthy. Once seen healthy it is judged immediately, so
a camera that dies later is caught exactly as fast as before, and every check
other than the restart runs from the first pass.
Verified against the two monitors on a machine with no daemons running, which
is the boot state, with Monitor::control stubbed so nothing was actually
started. Before: both monitors restarted on the first pass after the sleep,
and again every 10 seconds. After: no restart for 45 seconds, then both
restarted on the first pass past the grace, at 50 seconds. So a monitor that
never comes up is still caught, ~20 seconds later than before at boot and at
the same speed as before thereafter.
Also adds ZoneMinder::General::startedInteractively and uses it to skip the
start delay in zmstats.pl and zmfilter.pl when a person ran them. zmdc.pl
reopens STDIN on /dev/null for everything it starts, so a terminal on STDIN
reliably means a hand-run; zmtelemetry.pl already relies on this. A run from
cron or a unit file has no terminal either and still waits, which is the right
way round. zmstats keeps a delay because it is polite while zmc and zma are
competing for the machine, but nothing in it races with startup -- it connects
to the database first and has its own reconnect loop -- so the value is now 5
rather than 30.
Note zmfilter's existing guard is inverted with respect to that intent:
zmpkg starts it as "--filter_id=N --daemon", and the guard skips the delay
when a filter is named, so the daemon path never waited and only a manual
scan-all did. Left as it is here beyond adding the interactive case; worth a
separate look.
The zmstats and zmfilter changes are not runtime tested. Running either
against a live install prunes and rewrites rows, and there was no throwaway
install to point them at; startedInteractively is verified directly in both
directions, and all three generated scripts pass perl -Tc.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
setContrast takes 'constrast'. Spelling the parameter 'contrast', as
the field is named everywhere else including in getImageSetting's own
answer, is accepted with result=0, leaves the real parameter unset and
applies 0 for it. Contrast 0 is a black picture, so set_config would
have blacked out any camera whose contrast a template touched, while
reporting success.
Found by doing it to a live FI9853EP: the camera kept serving video
with a correct timestamp overlay and no error anywhere, and the only
sign was getImageSetting answering contrast 0 where an untouched
camera of the same model answered 50.
field_set entries are now [command, parameter] pairs rather than a bare
command with the parameter assumed from the field name, since the
firmware gives no indication when it is handed a name it does not read.
The same check against an untouched camera clears brightness, hue,
saturation and sharpness: those were written with their documented
names during the same session and kept their values.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
set_time wrote unconditionally. Because the offset it corrects only
moves at a daylight saving change, running it from cron - which is the
point of it being idempotent - meant thousands of writes a year to the
camera's flash for the two that matter.
Read the section first and return success without writing when every
field we would set already holds the wanted value. Measured against an
FI9853EP: a repeat run now issues one getSystemTime and no write, while
a moved offset still reads, merges and writes once.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
FI9821W_Y2k, FI9831W and FOSCAMR2C all talk to the same
/cgi-bin/CGIProxy.fcgi endpoint that FoscamHD now implements settings
for, so reparent them onto it instead of ZoneMinder::Control.
Method resolution keeps each module's own new, open, sendCmd and PTZ
commands and picks up get_config, set_config, set_time, cgi and
rtsp_url from FoscamHD, so how these cameras move is unchanged.
FoscamHD resolves the address and credentials on demand rather than in
open() precisely so this works: these modules have their own open()
which builds a UserAgent without working out either.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
Everything from the FI98xx generation onward answers a settings API on
/cgi-bin/CGIProxy.fcgi, but nothing in the tree reads or writes it - the
Foscam modules we ship only move the camera. That left no way to see or
change a camera's settings except its web ui, which on these needs an
IE6 plugin.
Adds get_config/set_config over 14 sections, plus set_time to point a
camera at an NTP server.
Two behaviours were measured against an FI9853EP on firmware 2.22.2.15
and drive the implementation:
Whole-section writers do not merge. A setSystemTime that leaves out
timeFormat and timeZone sets both to 0 rather than leaving them alone;
a partial write was observed to reset timeFormat 1 -> 0 and timeZone
14400 -> 0. set_config therefore reads the section back and writes the
merged result, never the diff on its own.
setSystemTime refuses isDst=1 outright, answering -1, so a daylight
saving offset has to be folded into timeZone. timeZone is seconds west
of UTC - EDT, UTC-4, is 14400 - which is the POSIX sign and the
opposite of a tm_gmtoff. Because DST lives in timeZone, set_time has
to be re-run when the offset changes; it is idempotent so cron is fine.
VideoStreamParam is read-only because getVideoStreamParam answers
resolution0..3 while setVideoStreamParam takes a single streamType plus
unsuffixed fields, so the read shape cannot be merged back into a
write. IPInfo is read-only because a wrong value takes the camera off
the network, where this module can no longer reach it. ImageSetting
has one command per field; setDenoiseLevel answers -3 on this firmware
so denoiseLevel is readable but not writable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BJjpSYbZRM8ucGW9HbgtR
The C++ loader reads back only the rows where Value differs from DefaultValue
and takes compiled-in defaults for the rest, so the two columns have to be
written in the same form. Both writers passed Value through a boolean
conversion and DefaultValue through none: a boolean left alone was stored as
Value '1' against DefaultValue 'yes', never compared equal, and was read back
on every start. The filter did nothing for the 77 boolean rows.
Both columns now go through ConfigData::dbValue. It lives there because the
two writers - saveConfigToDB for an existing install and zmconfgen for the
zm_create.sql of a fresh one - have to agree, and drifting apart is what
caused this. An option with no default is stored as its type's empty value,
matching the empty string initialiseConfig already gives it.
The generated zm_create.sql now has Value equal to DefaultValue for all 259
rows, so a fresh install reads no Config rows at all. The generated
zm_config_defines.h is byte identical, confirming the compiled-in defaults
are unaffected by the representation change.
tests/perl/test_config_default_value.pl checks the invariant over every
option in ConfigData; it needs no database, since ConfigData does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
ConfigData wrote boolean defaults and requires clauses as 'yes'/'no' while
the Config table has always held '1'/'0', so every writer and reader had to
convert between the two. yes/no is not a boolean value and the web UI never
showed it - booleans render as a checkbox, which reads Value directly and
ignores the Hint - so the string form bought nothing and only existed to be
translated away again.
Converts the 75 boolean defaults, the 59 requires clauses that test them, and
the boolean entry in %types. OPT_FFMPEG is substituted into a boolean default
by cmake, so it changes with them.
Requires clauses reach the Config table byte for byte as before: they were
already converted to 1/0 on the way in, and the web UI string-compares them
against the stored Value.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
SIGHUP means reload. zmc responds by closing its events, disconnecting the
camera and reconnecting; the perl daemons respond by exiting so zmdc restarts
them. zmdc.pl logrot hupped every managed process, so the nightly logrotate
run cost about 8 seconds of capture on a default install.
Rotating a log file only needs the daemon to drop its file handle, so use a
separate signal for it. SIGWINCH is otherwise unused, is ignored by default and
exists on every supported platform.
- Logger (C++) installs a SIGWINCH handler beside its USR1/USR2 handler. The
handler only sets a flag; the next logPrint closes the file and the write
reopens it at the original path. This covers every C++ binary without
touching any daemon's main loop.
- Logger.pm registers WINCH alongside HUP in logSetSignal, which logInit
already calls, so the scripts that install their own HUP handler still
rotate.
- zmdc.pl logrot sends WINCH. The logrotate config is unchanged - it still
calls zmpkg.pl logrot.
Filter.pm and FilterTerm.php justified MAX_EVENT_DAYS by events not outliving
the nightly HUP, which is no longer what bounds them; cite SectionLength.
Tests: tests/zm_logger_rotate.cpp and tests/perl/test_log_rotate_signal.pl both
log, rename the file out from under the process, confirm writes still land in
the renamed file, signal WINCH and confirm the original path is written again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NDhTBPj9xEaT52pRmAufaP
DBD::mysql 5 refuses to connect to a MariaDB server and upstream does not
consider that a bug, so a MariaDB system needs DBD::MariaDB. Prefer it when
present; it drives both servers. ZM_DB_TYPE cannot select it, which is why
setting it to MariaDB does not work: the same value builds the PHP PDO dsn,
where the only valid driver is mysql whichever server is in use.
Swapping the dsn scheme is not enough on its own. DBD::MariaDB spells
DBD::mysql's mysql_* parameters mariadb_* and ignores parameters it does not
recognise, so a dsn built with the wrong prefix does not fail: it drops the
socket path and the TLS settings and connects over plain TCP. Build the dsn in
one place from the chosen driver, renaming caller supplied options too, since
zmupdate.pl asks for mysql_multi_statements.
Read the id of an inserted row through DBI's last_insert_id rather than the
mysql_insertid handle attribute, which DBD::MariaDB does not have and would
answer undef for, leaving events and objects with no Id.
DBD::MariaDB is always utf8mb4, so it takes no mysql_enable_utf8mb4 attribute.
Let cmake accept either driver instead of requiring DBD::mysql, checking in the
same order, and warn when neither is installed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019URmtYqza6Rzi6F7cmabSm
A DateTime lower bound was written as
COALESCE(E.EndDateTime, '9999-12-31 23:59:59') >= T1
reading "an event that has not ended yet never ends". EndDateTime is NULL for
an event that is still recording, but also for one zmc was killed part way
through, and that stays NULL forever - so every abandoned event matched every
window from then on. Wrapping the column in a function also meant no index
could range over it: with the only MonitorId key being MonitorId alone, every
montage review request read all of that monitor's events and filtered them in
memory, however narrow the window.
Length tells the two cases apart. It is flushed during recording, so a live
event's effective end keeps advancing while an abandoned one's is frozen at
whatever was recorded. Use the same CASE the SELECT list in ajax/events.php
already uses, and bound it below by StartDateTime.
The lower bound now emits three conjuncts:
E.StartDateTime >= DATE_SUB(T1, INTERVAL 1 DAY)
AND (E.EndDateTime IS NULL OR E.EndDateTime >= T1)
AND <effective end> >= T1
The floor bounds the scan for a window in the past and drops events abandoned
more than a day earlier - events do not outlive the nightly logrotate SIGHUP,
which stops and restarts them, so a day is comfortably beyond any real event.
The middle conjunct is implied by the third and exists only to give the
optimiser a second indexable handle: it is narrow exactly when the floor is
wide, so between them a window at either end of the retention period has
something cheap to range over. Verified the optimiser picks correctly for both,
unprompted. The third is the residual that gets the semantics right.
Add the two composite keys those conjuncts need. Two range columns cannot both
narrow one B-tree, and these are mirror images - EndDateTime >= T1 is
open-ended upwards, StartDateTime <= T2 downwards:
Events_MonitorId_StartDateTime_idx (MonitorId, StartDateTime)
Events_EndDateTime_MonitorId_idx (EndDateTime, MonitorId)
MonitorId leads the first because it is the equality; past a range column later
columns can no longer narrow the scan, and (StartDateTime, MonitorId) measured
3.5x slower for the same rows. EndDateTime leads the second so it also covers
zmaudit's hunt for events that were never closed, which has no monitor to scope
it - 9 rows with a key against a 22,948 row table scan without.
Both replacements are added before the keys they supersede are dropped:
Events_MonitorId_idx (MonitorId) is now a leftmost prefix of the new key.
Events_EndDateTime_DiskSpace (EndDateTime, DiskSpace) existed for the scan
that hunted events with no DiskSpace set; DiskSpace is set when the event is
finalised in C++, so nothing scans for it any more.
Net index count on Events is unchanged.
Measured on a 7,167 event monitor. Default one hour window: 7,055 rows
examined / 25.4ms -> 173 rows / 4ms. Window scrubbed a week back, which
persists in the zmFilter_StartDateTime cookie: 6,995 rows / 28ms -> 19 rows /
0.15ms.
Adds migration db/zm_update-1.39.22.sql. Extends t/filter_sql.t; the PHP and Perl were
checked to emit identical SQL, and the semantics checked against a table of
seven event shapes - abandoned long ago, abandoned recently, overlapping,
inside, before, still recording, and still recording for 17 hours.
A filter with LockRows set selected its entire result set FOR UPDATE inside one
transaction, so every lock the per-event work went on to take was held until
the run committed:
Events[Id] -> Events_Hour/Day/Week/Month[EventId] -> Event_Summaries[MonitorId] -> Storage[Id]
Because the locks accumulated across events, two filters deadlocked: one held
Event_Summaries for a monitor while waiting on a Storage row, the other held
that Storage row while waiting on Event_Summaries for its next event. No
per-event lock ordering can fix that while both rows stay locked for the length
of the batch.
Holding Event_Summaries for the whole run also blocked zmc from opening a new
event on any monitor the filter had touched, since creating an event updates
that row. The transaction spanned ffmpeg encodes, uploads and executed commands
as well, so it could be held open for minutes.
zmfilter now claims one event at a time in Events_Lock and releases it when it
is done with that event, so no InnoDB lock is held across the work. The
per-event body moves into checkFilterEvent.
skip_locked now adds NOT EXISTS over Events_Lock to the filter query rather
than SKIP LOCKED. The exclusion has to happen in the query: a filter whose
whole result set was held elsewhere would otherwise fill its LIMIT with events
it could only skip, and make no progress. It no longer depends on MariaDB 10.6
/ MySQL 8.0.1, so the UI no longer disables the option on older servers.
Also drops the two dbh->commit() calls in the AutoCopy branch. With no
transaction open they would warn, and before this they were silently ending the
batch transaction mid-loop, so AutoCopy filters never had the guarantee
LockRows was supposed to give them.
filterdebug.php was appending a bare ' SKIP LOCKED' after the LIMIT, which is
not valid SQL; it now renders the real clause in the right position.
Adds t/event_lock.t and t/filter_sql.t.
Events_Lock holds advisory locks over events, so that two filters do not work
on the same event at the same time. Claiming an event is a single autocommitted
INSERT of one row, which means no InnoDB lock is held while the filter actually
works on the event.
No foreign key to Events on purpose: it would make every event delete take a
lock in this table, which is the coupling this exists to avoid. Rows left
behind for deleted events are harmless and expire.
ZM_FILTER_LOCK_TIMEOUT (default 3600) is how long a claim lasts. A filter that
is killed part way through an event cannot release anything, so claims have to
expire on their own. It needs to be longer than the slowest thing a filter does
to a single event, which is why it is configurable rather than fixed.
Adds migration db/zm_update-1.39.21.sql.
lock_and_load + save is a read-modify-write, and outside a transaction the
SELECT ... FOR UPDATE autocommits and drops its lock immediately. The absolute
value written afterwards then clobbers any adjustment zmc, zmaudit or another
filter made in between.
Replace both call sites with ZoneMinder::Storage::adjust_diskspace, a single
relative UPDATE that is atomic on its own and holds the row lock only for the
length of that statement. It reads the new total back into the object rather
than adding the delta locally, because ZoneMinder::Object caches objects for
the life of the process and its accessors never re-load, so the cached value
can be arbitrarily stale and adjusting it locally would preserve the error.
Keeping the adjustment to one statement also keeps Storage out of the lock
chain that deleting an event walks:
Events[Id] -> Events_Hour/Day/Week/Month[EventId] -> Event_Summaries[MonitorId]
Adds t/storage_diskspace.t.
The two subs decide whether zmpkg.pl hands start/stop/restart to systemd or
runs the daemons itself, which is what keeps them out of the web server's
cgroup and mount namespace. Living in a script they could not be reached by
require_ok, so the cgroup matching had no test.
Split the matching into cgroup_in_service, which takes the text of a cgroup
file rather than reading one, and cover it: cgroup v2's single line and v1's
several, a sub-cgroup of a delegated service, a unit whose name merely starts
with ours, and empty or undef input.
The test found that the delimiter in m|/\Q$unit\E\.service(?:/|$)| ended the
match at the alternation, so the module did not compile.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019URmtYqza6Rzi6F7cmabSm
Verified every added endpoint against an Amcrest IP5M-1190EW running
2.810.00AC004.0.R. A full get_config() returns 52 sections in ~24s and
the set_config() key form round-trips, but several guesses were wrong:
networkInterface.cgi does not exist, netApp.cgi?action=getInterfaces is
the real endpoint.
userManager.cgi reports a real per-user Id starting at 1, so get_users
was handing callers the array position instead of the number the camera
actually recognises.
modifyUser rejects a partial record with 400. update_user now reads the
user back and sends the whole record with the changes merged over it.
getSoftwareVersion and getHardwareVersion both answer with a bare
'version=', so device_info's flat merge lost one of them. The software
one is a single line with the build appended - '2.810.00AC004.0.R,
build:2023-09-04' - so getVersion splits it and returns the build
separately in list context.
The configManager list is now the set that model actually implements.
UPnP and Login are gone, they answer 400; Lighting_V2, WLan, NAS, Record,
Alarm and the VideoIn* sections are added.
That model has no getCurrentProtocolCaps and no coaxialControlIO, and it
answers OK to Lighting/Lighting_V2 writes and then ignores them, so its
WhiteLight is not drivable over http. Documented rather than papered
over. update_firmware stays the one unverified path, deliberately.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LzHrvNtF13vyYBAJjktam6
set_config prepended the section name to keys that already carried it,
so NTP.Address went out as NTP.NTP.Address. get_config only strips the
'table.' prefix, which leaves keys in exactly the form setConfig wants,
so drop the extra prefix and let reads and writes round-trip.
uri_encode was called but URI::Encode was never imported, and the plain
call leaves reserved characters alone, which corrupts passwords in a
query string. Import it and always escape reserved characters.
get_config now reads configManager.cgi sections as well as the handful
of dedicated cgis, retries once on a 401 so the fresh digest nonce gets
used, strips the 'table.' prefix, and tries any unrecognised name as a
configManager section so a template can pull in sections the module
doesn't list.
set_config fell off the end of its loop returning undef on success.
reboot POSTed system_reset=1 to setparam.cgi, a Vivotek endpoint, on a
relative url through the raw user agent. ZoneMinder::General was used
but never required.
Adds device info, time, users, snapshot/rtsp/profiles/probe, white light
and siren, and the remaining PTZ stops and auto pan, matching the method
names cameratool.pl probes for with ->can().
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LzHrvNtF13vyYBAJjktam6
The openbsd branch and master both changed the top fallback in CpuUsage
and diverged. Combine them:
- Keep master's /proc/stat diagnostics: report why /proc/stat could not
be read (ProcSubset=pid mount namespace vs open failure), default the
fields split out of top output so an empty result does not warn under
-w, and only complain once per process.
- Keep the openbsd branch's parse_bsd_top_cpu(): OpenBSD top prints CPU
states in a format neither the FreeBSD nor the Linux grep matches, so
when the greps produce no numbers, parse raw top output. It handles
both the aggregate "CPU states:" line and per-core "CPU0 states:"
lines, averaging over the lines seen, and does not depend on padding.
- Carry over scripts/ZoneMinder/t/server_cpu.t covering it.
Tests: prove -Iscripts/ZoneMinder/lib scripts/ZoneMinder/t/server_cpu.t
-> 16/16 pass. perl -c on Server.pm is clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kfiom9RefM75GTZFkqpJo2
/proc/stat is invisible to any process in a mount namespace set up with
ProcSubset=pid, which is systemd's default hardening for apache. Daemons
started from the web ui are forked off mod_php and inherit it, so zmstats
fell through to the top fallback, which reads /proc/stat as well and so
produced no output either.
Separate the missing-file case from the failed-open case so the message
names the likely cause instead of printing a $! that a failed -e never set.
Emit the warning once per process rather than once per
ZM_STATS_UPDATE_INTERVAL. The condition does not clear itself, so it was
filling zmstats.log with the same three lines a minute.
split() on an empty $top_output returns an empty list, leaving all four
values undef, so each fallback cycle also emitted four "Use of
uninitialized value" warnings under -w. Default them before use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019URmtYqza6Rzi6F7cmabSm
zmwatch fetches its monitor list once at the top of a pass and then walks it.
Deleting a monitor mid-pass stops its zmc from the web ui, so by the time
zmwatch reaches that monitor its shared memory is gone, zmMemVerify fails and
zmwatch calls control('restart') from the now-stale list. zmc comes back for a
monitor that is marked Deleted, so no later pass ever looks at it again and
nothing stops it until zmpkg.pl restart. That is the orphaned zmc left behind
after deleting a monitor from the console.
Re-read Deleted from the database in ZoneMinder::Monitor::control() before
running a start or a restart, and skip the command if the monitor has been
deleted or the row is gone. control() is the single path every restart goes
through, so this covers any caller working from a list it fetched earlier.
Stopping is deliberately still allowed, since that is how an orphan gets
cleaned up.
Tests: tests/perl/test_monitor_control_deleted.pl stubs zmDbFetchOne and
runCommand and checks that start/restart still run for a live monitor, are
skipped for a deleted one and for one removed from the database, and that stop
is unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NDhTBPj9xEaT52pRmAufaP
Adds a web config option whose contents are rendered on the login view,
between the title and the username and password fields. Useful for a site
notice, an acceptable use or legal warning, or a note identifying which
installation this is when running more than one.
Defaults to empty, and nothing is emitted when it is empty or whitespace, so
existing installs look exactly as they do now.
The text is escaped rather than interpreted. The login page is served before
anyone has authenticated, so it is not somewhere to emit admin-supplied markup,
and no other config value in the skin is output unescaped either. Escaping runs
before nl2br so the only tags reaching the browser are the line breaks we add
ourselves; reversing that order would turn the setting into stored XSS.
Uses the text type, so the Options UI renders a textarea and the value can span
lines. The Config.Value column is already text and options.php normalises CRLF
to LF on save, so no schema change is needed and nl2br sees consistent
newlines.
Guarded with defined() to match the surrounding code, so the view still renders
where the database predates the option.
Styled in base, classic and dark. Text contrast is 7.0:1 light and 7.5:1 dark,
both above WCAG AA, and long unbroken tokens wrap rather than widening the
fixed-width form.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The mp4 is written as the event records, so its mtime is when recording
finished, not when it started. Set EndDateTime to the mtime and
StartDateTime to duration seconds before it, so Length agrees with
EndDateTime - StartDateTime.
The directory mtime fallback also ran unconditionally after both
branches, overwriting the times just computed and the Deep-scheme
path-derived start. Only use it when there are neither capture jpgs nor
an mp4.
Adds t/event_recover_timestamps.t, which builds a 3 second mp4 with
ffmpeg and checks the recovered times against its mtime.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BV1gDc9T6D6H8gtTcxwLvk
The DateTime filter attribute was taught the "event overlaps this
instant/window" idiom in web/includes/FilterTerm.php: a lower bound (>=/>)
compares COALESCE(EndDateTime, '9999-12-31 23:59:59') so still-running events
are included, while an upper bound keeps comparing StartDateTime. The Perl
Filter.pm still emitted E.StartDateTime for that attribute, so a saved filter
using DateTime with >=/> selected different events in the web UI than in
zmfilter.pl, which drives automated delete/email/execute actions.
Apply the same column selection in ZoneMinder::Filter::Sql so the daemon and
the web agree. Verified by generating SQL from both sides for =, <, <=, >, >=:
the column expressions are now byte-identical.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KQipwf632JGgNH4W7p8cqs
Replace the single alarm_image slot with an analysis_image_buffer ring of
image_buffer_count Images living in the already-reserved alarm_images SHM
region. Successive WriteAlarmImage calls rotate through the ring and
publish last_analysis_index last (after the bytes and per-slot format),
so a reader sampling last_analysis_index always sees a fully written
slot. GetAlarmImage returns that slot, syncing its AVPixelFormat from the
per-slot analysis_image_pixelformats array.
SharedData gains last_analysis_index and analysis_image_count (plus 8
bytes of padding to keep the 16-byte-multiple layout), making it 888
bytes. The Perl (Memory.pm) and PHP (Monitor.php) SHM readers are updated
in lockstep, and a static_assert(sizeof(SharedData)==888) in zm_monitor.h
guards the layout against silent drift.
This lets multiple in-flight analysis/annotated frames be buffered and
streamed in sync rather than always overwriting one slot, and gives the
AI object-detection work a place to publish annotated frames.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The C++ and PHP parsers for zm.conf and /etc/zm/conf.d/*.conf used fgets
with a 512-byte buffer, silently truncating any line longer than that.
There was also no way to split a value across multiple lines, so editors
hit the cap with no workaround.
Accept a trailing backslash (with optional whitespace before the newline)
as a line-continuation marker. Leading whitespace on continuation lines
is stripped so users can indent for readability without it leaking into
the value. Three parsers all read the same files and must agree:
- src/zm_config.cpp: switch to std::ifstream + std::getline so a single
physical line is no longer capped at 512 bytes, then join continuation
lines before running the existing pointer-based parser
- scripts/ZoneMinder/lib/ZoneMinder/Config.pm.in: accumulate a logical
line across trailing-backslash physical lines
- web/includes/config.php.in: drop the 512-byte fgets cap and accumulate
in the same way
tests/zm_config.cpp covers three cases against the C++ parser: a
three-segment continuation joins to "firstsecondthird" with leading
whitespace stripped, a bare backslash inside a value is preserved
(C:\Users\zm), and a 1500-byte single line survives intact.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
#4914 unconditionally drops Javascript errors and CSP violations sourced from
browser extensions. Some operators want to know when a plugin is touching the
ZoneMinder tab, so gate the suppression behind a config entry.
Add ZM_LOG_BROWSER_EXTENSIONS (boolean, default no) in the logging category.
It is exposed to JS by the existing non-private-config loop in skin.js.php as
the string '0'/'1' (the same convention as ZM_LOG_INJECT), so logger.js only
filters extension sources when it is '0'. Default behaviour is unchanged:
extension noise stays out of the log.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
zmwatch only gated the camera reboot attempt on CanReboot(), so it called
$control->open() even when the camera was unreachable - the common reason a
monitor has no image since startup - and blocked until the connection timed
out, logging an error each pass.
Add a ping check before open(). Move the host resolution into Control so callers
don't have to dig the ip out of the Path: add Control::host(), which returns the
cached host or derives it from the monitor's ControlAddress/Path via the shared
guess_credentials() (parsing only, no network i/o), and have ping() fall back to
it. ping() still accepts an explicit ip.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The hand-rolled regex in open() mis-parsed a stream Path that carried no
credentials, e.g. rtsp://10.0.0.4:554/cam: the greedy [^:@]+ consumed the host
as the username, 554 as the password, and ADDRESS backtracked to the single
digit 4. The control daemon then dialled http://...@4, producing
"Can't connect to 4:80 (Connection timed out)".
Replace the regex with the shared URI-based Control.pm guess_credentials(),
which handles ControlAddress and Path, converts rtsp to http, and falls back to
the Monitor User/Pass. Create the LWP::UserAgent before parsing since
guess_credentials() sets credentials on it. The Grandstream login wire protocol
(challenge/authcode and old-style fallback) is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On systems with many cameras the Grandstream control log entries gave no
indication of which monitor they referred to. Prefix every Debug/Warning/Error
message with the monitor Id and Name. Also fix two message typos
(challengstring, UNable).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a ZM_DB_SSL_VERIFY_SERVER_CERT setting so a database connection that uses
ZM_DB_SSL_CA_CERT can talk to a server with a self-signed or otherwise
non-matching certificate. When enabled, verification is by identity (the cert
must chain to the CA and its CN/SAN must match ZM_DB_HOST), consistent across
the C++ daemons, the PHP web interface, the CakePHP API and the Perl scripts.
This re-does the reverted #3817. That PR broke the build because it called
mysql_options(MYSQL_OPT_SSL_VERIFY_SERVER_CERT, ...), and that enum was removed
from the MySQL 8.0 C client in favour of MYSQL_OPT_SSL_MODE; it also passed a
c_str() where a my_bool* was expected, and referenced the PHP constant
unconditionally (fatal on PHP 8 for an upgraded install whose zm.conf predates
the option).
The option that controls server-cert verification differs by client library and
the symbols are enum values, not macros, so CMake feature-detects them by
compiling:
- HAVE_MYSQL_OPT_SSL_MODE (MySQL 5.7.11+/8.0, MariaDB Connector/C 3.1+)
- HAVE_MYSQL_OPT_SSL_VERIFY_SERVER_CERT (older MariaDB/MySQL)
zm_db.cpp uses SSL_MODE_VERIFY_IDENTITY / SSL_MODE_REQUIRED when the former is
available, else falls back to the latter with a proper my_bool.
Value handling is three-way in every layer: a truthy value verifies, a false-y
value (0/false/no/off) skips verification, and an empty/unset value leaves the
client default in place so existing installs are unchanged on upgrade. PHP, the
API datasource (via PDO flags) and the Perl DSN are all guarded with defined()
checks. Fresh installs default to 1.
Documents the full ZM_DB_* connection and SSL settings, including the hostname
verification gotcha when connecting by IP, in docs/userguide/configfiles.rst.
refs #3816
open() is contracted to return true or false so callers (zmcontrol.pl,
zmwatch.pl) can tell whether the camera is reachable, but it always ended
with an assignment that evaluated truthy and reported success regardless.
When the initial login probe failed it also rebuilt BASE_URL in the old
basic-auth style and returned without ever testing that connection.
Return 0 on a failed probe and 1 only after a successful exchange. Check
is_success() on the authcode response in the modern path, and actually issue
and check a request on the old-style URL in the fallback path so success
means we can talk to the camera. Log the previously unused ResCode.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
