Commit Graph
270 Commits
Author SHA1 Message Date
Isaac Connor eee9c0f4ac fix(packetqueue): destroy packets after releasing mutex; drop dead code
clear() was destroying ZMPackets while holding the queue mutex, unlike
queuePacket() and clearPackets() which defer destruction to a stack
vector that goes out of scope after the lock is released. ZMPacket
teardown frees Image, AVFrame, and AVPacket data and is expensive,
especially on PASSTHROUGH queues that may hold hundreds of MB. Doing it
under the mutex risked stalling capture and analysis threads during
shutdown or monitor reconfigure. Adopt the same deferred-destroy
pattern.

Also remove dead code identified in the audit:
- PacketQueue::unlock(ZMPacketLock*): defined but no callers; would
  UAF if anyone passed a non-heap pointer.
- monitor_ field and setMonitor(): set never read, no callers.
- analysis_it field: declared, never used (Monitor::analysis_it is
  the live one).
- get_stream_it(int) declaration: never defined.
- max_video_packet_count comment: clarify 0 means unlimited (the
  setter clamps negatives to 0 and the limit check is > 0).
2026-06-01 11:54:32 -04:00
Isaac ConnorandClaude Opus 4.7 b2b50aba5d perf: skip clearPackets early-returns and allow drop-to-iterator-keyframe
Two related changes to PacketQueue::clearPackets, called by the analysis
thread on every video packet:

1. Lock-free call-site gate (should_try_clear) on the analysis path.
   In keep_keyframes mode the existing early-return at the top of
   clearPackets discards most non-keyframe video packets after acquiring
   the queue mutex. Add an inline lock-free check at the call site so
   non-keyframe packets skip the mutex acquire entirely. clear_packets_pending_
   is now std::atomic<bool> so it can be read without the lock; a stale
   read is harmless (at worst we make one extra cheap early-returning call).
   The !keep_keyframes path always returns true from the gate because that
   mode pops one packet at a time on every video packet.

2. Iterator boundary in the scan loop changed from >= to >. Setting
   next_front to a packet that an iterator points at is safe because
   clearPackets deletes strictly before next_front, so the iterator's
   own packet stays in the queue. Previously, an event-start (or other)
   iterator landing exactly on a keyframe blocked the leading GOP from
   being dropped until the iterator advanced; now we can include that
   keyframe as next_front while the iterator continues to point at it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 19:39:04 -04:00
Isaac ConnorandClaude Opus 4.6 b196585b0d perf: reduce lock contention in PacketQueue
Remove per-packet ZMPacketLock trylock() from clearPackets() scan loops
and queuePacket() GOP deletion — the iterator check is sufficient since
threads only access packets through their own shared_ptr.

Add a monotonic uint64_t queue_index to ZMPacket, assigned on enqueue.
clearPackets() now finds the earliest iterator position with a single
min() over the 2-3 iterators, then uses one integer comparison per
scanned packet instead of walking all iterators per packet.

Defer packet destruction outside the mutex in both clearPackets() and
queuePacket() by collecting removed shared_ptrs into a local vector
and releasing the lock before they are destroyed.

Raise per-packet deletion Debug(1) to Debug(4) in both paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 15:38:07 -05:00
Isaac ConnorandClaude Opus 4.6 0882a3ad1e fix: resolve Event::Run thread hang preventing zmc clean shutdown
Event::Run could block indefinitely in PacketQueue methods during normal
event closing (closeEvent from analysis thread), because their wait
predicates only check deleting/zm_terminate, not Event's terminate_ flag.

Three changes fix this:
- get_packet_no_wait: return immediately when iterator at end instead of
  blocking on condition variable (makes it truly non-blocking)
- Event::Run: use increment_it(wait=false) since deletePacket can advance
  the iterator to end() during AddPacket_ without the queue lock
- Event::Stop: call packetqueue->notify_all() to wake timed waits so
  Run() checks terminate_ promptly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 23:00:59 -05:00
Isaac Connor 56588939e7 Make clearPackets return false if it can't remove any packets. So that we try again even if we aren't looking at a keyframe. 2026-02-11 22:35:50 -05:00
Isaac ConnorandClaude Opus 4.6 21d0e68673 perf: add monotonic queue_index to ZMPacket for O(1) iterator lookup
Add a uint64_t queue_index field to ZMPacket, assigned by PacketQueue
on enqueue via a monotonic counter. This lets clearPackets() find the
earliest iterator-pointed packet with a single min() over the 2-3
iterators, then check each scanned packet with one integer comparison
instead of searching a vector of shared_ptrs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 23:41:48 -05:00
Isaac ConnorandClaude Opus 4.6 7d91347b76 perf: reduce lock contention in PacketQueue::clearPackets()
Remove per-packet ZMPacketLock trylock() from clearPackets() scan loops
and queuePacket() GOP deletion — the iterator check is sufficient since
threads only access packets through their own shared_ptr after obtaining
it from the queue.

Pre-compute the set of iterator-pointed packets once before scanning
instead of calling is_there_an_iterator_pointing_to_packet() per packet,
reducing O(packets * iterators) to O(iterators) for the lookup setup.

Batch packet destruction outside the mutex by collecting removed
shared_ptrs into a local vector and releasing the lock before they are
destroyed, so expensive ZMPacket destructors don't block queuePacket()
and get_packet_and_increment_it().

Raise per-packet deletion Debug(1) to Debug(4) to cut string formatting
on the hot path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 23:35:58 -05:00
Isaac ConnorandClaude Opus 4.6 3039fde4b3 fix: count begin packet in get_event_start_packet_it pre_event walk
The backward walk loop exits when the iterator reaches pktQueue.begin()
without counting that packet as a video frame. This off-by-one causes
the "Hit end of packetqueue before satisfying pre_event_count" warning
even when the queue has enough packets. Check the begin packet after
the loop exits.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 22:32:56 -05:00
Isaac Connor ae42c3c94d Wait on the packetqueue condition instead of sleeping. Should help analysis keep up with decoding better and offer faster shutdown. 2026-02-04 20:44:17 -05:00
Isaac ConnorandClaude Opus 4.5 0f917ae1ed refactor: restructure Decode() and fix PacketQueue locking issues
Monitor::Decode():
- Reorganize into 5 clear phases with descriptive comments
- Phase 1: Receive decoded frame from decoder
- Phase 2: Get and send new packet to decoder
- Phase 3: Convert decoded frame to Image
- Phase 4: Prepare Y-channel for analysis
- Phase 5: Process RGB image (deinterlace, rotate, privacy, timestamp)
- Extract applyOrientation() and applyDeinterlacing() helper functions
- Keep slow send_packet detection timing for diagnostics

PacketQueue locking fixes:
- Move lock acquisition before accessing shared state in queuePacket()
- Keep lock held while iterating in stop()
- Add lock to addStream()
- Remove duplicate packet_counts allocation in clear()

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 11:18:50 -05:00
Isaac Connor 5c402ccab4 Add wait parameter to increment_it. Sometimes we should wait and sometimes not. 2026-01-27 09:30:28 -05:00
Isaac Connor ae97d4bd4b Make incremement at end a debug. 2026-01-27 08:05:04 -05:00
Isaac Connor d4d01cb0d9 Use RAII for packet_counts 2026-01-16 12:28:57 -05:00
Isaac Connor 7a94717cf4 Add locking to size() and packet_count() and has_out_of_order_packets 2026-01-16 12:22:33 -05:00
Isaac Connor 5561041c70 Add a lambda to prevent spurious wakeups 2026-01-16 12:11:42 -05:00
Isaac Connor 1a3e36424e Set frames_since_last_keyrrame = 0 instead of 1 2026-01-16 12:06:22 -05:00
Isaac Connor 9f0b5c8cd6 Handle impossible case where snapshot-it points to end. Can't -- on end. WOuld need to be a reverse iterator. 2026-01-16 12:02:42 -05:00
Isaac Connor 5204a24345 Fix doulbe free of it. 2026-01-16 11:55:35 -05:00
Isaac Connor 2e2690c7c6 Move lock guards to protect deleting basically to satisfy AI. It is technically correct. 2026-01-16 10:49:31 -05:00
Isaac Connor 14bb739c00 DOn't need to lock the new keyframe, just frames before it. Fix not break if it is locked 2026-01-15 16:32:32 -05:00
Isaac Connor 9615969314 We may need () 2026-01-15 16:32:32 -05:00
Isaac Connor 0de8c7b4ad Only warn about being at end of queue in incremement_it if we are not deleting to prevent red herring logging 2026-01-15 16:32:32 -05:00
Isaac Connor 448056a809 Use lock_guard instead of unique_lock where appropriate 2026-01-15 16:32:32 -05:00
Isaac Connor 37d5430adc Turn off a debug 2026-01-15 16:32:32 -05:00
nvme 792fb4de75 Reduce logging 2026-01-15 16:32:32 -05:00
Isaac Connor 7c5b24cbab Must start at second packet. Cleanup 2026-01-15 16:32:32 -05:00
nvme 6b424088e3 When every frame is a keyframe, need to set max_keyframe_interval=1 2026-01-15 16:32:32 -05:00
Isaac Connor d82bfbfb5e Remove dead code, fix not deleting packets when full 2026-01-15 16:32:32 -05:00
Isaac Connor ed2f809df0 Improve debug 2026-01-15 16:32:32 -05:00
Isaac Connor 0932ab71dd Maybe fix not freeing audio packets 2026-01-15 16:32:32 -05:00
Isaac Connor f173deae29 Notify while holding the lock 2026-01-15 16:32:32 -05:00
Isaac Connor c05a9462da Get lock earlier 2026-01-15 16:32:32 -05:00
Isaac Connor 56628240b8 Optimise std_shared_ptr accesses 2026-01-15 16:32:32 -05:00
Isaac Connor 6e6b3d097b Fix lock around deleting causinganalysis to hang 2026-01-15 16:32:32 -05:00
Isaac Connor cb23ce65e8 We don't need to lock the packets to delete them. 2026-01-15 16:32:32 -05:00
Isaac Connor a10f65d574 Always delete frames when we don't care about keyframes 2026-01-15 16:32:32 -05:00
Isaac Connor 1d8bbcb916 Improve debug 2026-01-15 16:32:32 -05:00
Isaac Connor eb2c13e151 Log when failed to lock packet. 2026-01-15 16:32:32 -05:00
Isaac Connor 3734e2c213 Log the # of packets removed 2026-01-15 16:32:32 -05:00
Isaac Connor e38e88f2d9 iterator at the beginning is ok, make it a debug 2026-01-15 16:32:32 -05:00
Isaac Connor 376052684d Maybe fix memleak on iterators 2026-01-15 16:32:32 -05:00
Isaac Connor 131b5caee0 Update to RAII style ZMPacketLock. Rename snap to packet in zm_monitor. Keep hw_frame in packet." 2026-01-15 12:46:23 -05:00
Isaac Connor f45c085bf6 Fix clearing packets in queuePacket 2026-01-15 12:46:23 -05:00
Isaac Connor ef35a1f007 If every frame is a keyframe, we didn't set keyframe interval, so handle it as a special case 2025-09-24 12:26:06 -04:00
Isaac Connor bb356a8b96 Add debug of new keyframe interval 2025-09-06 10:28:32 -04:00
Isaac Connor 09c767174f Try out a better method of counting keyframes 2025-09-06 10:27:34 -04:00
Isaac Connor 3c4dca78f0 Log when failed to lock packet. 2025-09-06 10:26:32 -04:00
Isaac Connor eb09e0ad33 Log the # of packets removed 2025-09-06 10:17:41 -04:00
Isaac Connor 3be4e1440e Reduce debug logging 2025-09-06 10:17:19 -04:00
Isaac Connor 727e1fe5bb Add debug to not keeping keyframes case 2025-09-06 10:14:29 -04:00