Commit Graph
19 Commits
Author SHA1 Message Date
Isaac ConnorandClaude Opus 5 1e38c674fe fix: collapse the five Event_Summaries writes per event into one
Events_Hour, Events_Day, Events_Week and Events_Month each carried their
own update and delete trigger, and every one issued a separate UPDATE
against the same Event_Summaries row. A single Event delete therefore
locked that row five times: once from event_delete_trigger and once from
each bucket's cascade. That is the deadlock zmstats.pl hits when it bulk
deletes aged rows out of Events_Hour while zmc and zma are writing.

event_update_trigger and event_delete_trigger now modify the bucket tables
themselves and apply one consolidated UPDATE, using ROW_COUNT() after each
bucket statement to tell whether the event was still in that bucket so
aged-out events do not over-adjust the counters. Measured on a scratch
database, an Event delete goes from 5 Event_Summaries row updates to 1.

This also fixes a drift bug. The old event_update_trigger kept
ArchivedEventDiskSpace correct only in a branch that cannot be reached: it
sits under IF (NEW.Archived != OLD.Archived) and requires both to be
false. The branch that does run when an already-archived event grows
updated Events_Archived but not Event_Summaries, so the archived total
drifted for the life of the install. Reproduced on a scratch database:
after archiving a 250-byte event and growing it to 999,
ArchivedEventDiskSpace still read 100.

BEHAVIOUR CHANGE. A direct DELETE against a bucket table no longer adjusts
Event_Summaries at all, and that is how zmstats.pl prunes. zmstats.pl
already resyncs HourEvents/DayEvents/WeekEvents/MonthEvents and their disk
space columns from COUNT(*)/SUM(DiskSpace) on any pass where it pruned, so
those four pairs become eventually consistent within one
ZM_STATS_UPDATE_INTERVAL instead of exact at every instant. The Total and
Archived columns stay exact, because only the Events triggers touch them.
The zmstats.pl comments are updated to describe the new arrangement; its
code is unchanged.

Migration is db/zm_update-1.39.25.sql.in: drop the eight cascade triggers,
resync Event_Summaries from the events themselves so the new triggers start
from ground truth and the archived drift above is repaired, then source
db/triggers.sql. version.txt goes to 1.39.25. No schema change, so
zm_create.sql.in needs no edit -- it already sources triggers.sql.

Ported from the ai_server branch, where this migration sits at 1.39.7, a
number master passed long ago and an existing install would never run.
Repackaged above master's tip. The ai_server version also unwinds a
views-and-SWR experiment that only ever existed on that branch, which is
dropped here as a no-op on master, and carries an unrelated START_DELAY
change, which is not taken.

Tests: tests/perl/test_event_summaries_triggers.pl, 29 assertions against a
real server, skipped when no scratch database is configured. Verified they
fail on the current triggers, on exactly the two claims above: the archived
drift, and 5 row updates per delete. Also verified the migration repairs a
drifted install, is idempotent across a second run, and leaves a migrated
install with byte-identical triggers to a fresh one. Generated zmstats.pl
passes perl -Tc.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Y6FieTwEXuLhhR4e2yiax
2026-09-05 12:23:39 -04:00
Isaac Connor 1323e9f023 docs: reference the actual Event::Event constructor in lock-order block
Previous wording named a fictional Event::createNotification function;
the bucket+ES insert sequence in src/zm_event.cpp lives in the Event
constructor (line 46). Point future readers at the right symbol so they
can grep and verify the path.
2026-05-16 23:14:37 -04:00
Isaac Connor e254ed41d2 docs: expand triggers.sql lock-order block with full writer list
Adds zm_event.cpp's create path (Events INSERT -> bucket INSERTs ->
Event_Summaries UPDATE; event_insert_trigger itself is commented out so
zmc does this directly), and lists zmaudit.pl alongside zmstats.pl, so
the comment enumerates every writer obligated to follow the canonical
order rather than just the trigger paths.
2026-05-16 22:44:14 -04:00
Isaac ConnorandClaude Opus 4.7 fb230ebbba docs: document canonical Events lock acquisition order in triggers.sql
Records the InnoDB X-lock order that all writers (zma trigger path, zmstats
prune+resync, zmfilter/Event::delete) must follow to avoid the deadlock cycles
fixed in the surrounding commits. Comment only; no behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 16:48:18 -04:00
Isaac Connor d64c5fa2c8 Remove events_insert_trigger 2026-01-24 11:16:07 -05:00
Isaac Connor 7c42215f2c Alter event_update_trigger to only update summary tables if there is a diskspace diff 2025-12-02 15:08:06 -05:00
Isaac Connor ed57267258 Remove updating the Storage table in triggers. Do those updates manually because there is frequently too much contention around the Storage table and filters get locked on it. 2025-08-07 14:08:45 -04:00
Isaac Connor cb35a6dd90 Handle case when OLD.DiskSpace is NULL 2024-07-09 08:12:52 -04:00
Isaac Connor 35b3d236e0 Fix typo in triggers 2021-04-09 19:47:08 -04:00
Isaac Connor 4e5ef01156 Fixup event summary column population, update triggers 2021-04-09 16:46:36 -04:00
Isaac Connor c5ca123042 Move event summaries to their own table to reduce contention on Monitor_Status 2021-04-09 10:16:16 -04:00
Isaac Connor 63c78b43f8 Change event stats to Monitor_Status 2020-11-06 13:27:00 -05:00
Isaac Connor adbd3486bf Rename StartTime, EndTime in Events to StartDateTime and EndDateTime 2020-11-04 13:49:39 -05:00
Isaac Connor 75b4f4f2b3 Add a few missing GREATESTs in the triggers. 2019-04-30 17:29:18 -04:00
Isaac Connor 3c56b32b08 Use GREATEST function prevent negative values in event counts 2019-04-24 10:38:08 -04:00
Isaac Connor 914c608dbc get rid of stored procedure. Do everything in the Trigger 2018-04-20 08:54:25 -04:00
Isaac Connor 0ad01f09bd refix triggers updating EventCounts. remove unsigned from eventcounts and diskspace totals. 2018-02-20 22:48:23 -05:00
Isaac Connor e4ec1712f4 remove the table creation/dropping and a if/elseif fix 2018-02-18 11:48:00 -05:00
Isaac Connor 94212eda2f fix triggers 2018-02-18 11:01:33 -05:00