mirror of
https://github.com/Screenly/Anthias.git
synced 2026-07-30 09:15:51 -04:00
531bf3e4c8
* feat(viewer): pi3-64 hardware video playback (30fps, zero drops) + audio Fixes #3084 (pi3-64 video renders black). On the Raspberry Pi 3 arm64 board (VideoCore IV, eglfs) QtMultimedia's VideoOutput cannot present a video node on the vc4 GL driver — the frame is black — and compositing a full-screen video widget through eglfs GL caps at ~9 fps regardless. This adds a pi3-64-only path that keeps decode + scanout on the hardware and composites the video on a vc4 DRM overlay plane, reaching a stable 30 fps with zero ongoing dropped frames, with audio. Video (VideoView::gstPlayOverlay): - filesrc ! qtdemux ! h264parse ! v4l2h264dec (bcm2835 HW decoder) ! queue ! kmssink on a vc4 overlay plane, using eglfs's own DRM master fd + connector via QPlatformNativeInterface; the plane is HW-composited with the eglfs UI plane. - The decoder is driven capture-io-mode=mmap so kmssink COPIES each frame into its own scanout buffer instead of pinning the decoder's DPB pool — that pin deadlocked every plain decoder-direct attempt. The decoder emits I420 at full rate; the alternative ISP convert to NV12 is a second M2M pass that caps at ~18 fps, so it is only a fallback (ANTHIAS_GST_ISP_CONVERT=1). - Falls back to a CPU-raster appsink path, then to QMediaPlayer, if the DRM resources or an overlay plane can't be resolved. Audio (VideoView::gstStartAudio) runs in a SEPARATE pipeline: - filesrc ! qtdemux audio ! decodebin ! audioconvert ! audioresample ! alsasink on the HDMI card. Independent clock/preroll/bus so no audio condition (missing track, slow device, decode gap) can stall the video — a shared pipeline froze the video at the first frame. - Adds gstreamer1.0-libav for the audio decoders (avdec_ac3/aac/mp3/…); decodebin reported a missing plug-in on AC3 with only faad (AAC). Also: only pi3-64 links GStreamer (AnthiasViewer.pro CONFIG, Dockerfile build + runtime deps); _build_webview_env enables the path via ANTHIAS_VIDEO_RASTER/OVERLAY; start_viewer chowns the GStreamer registry cache to avoid a per-launch rescan; main.cpp installs a fatal-signal backtrace handler. Validated on a real Pi 3 (1080p30 H.264/AC3, HDMI): 30 fps, dropped flat at the startup ramp only, clean across the loop boundary, HDMI PCM RUNNING, respawns=1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build(viewer): pin pi3-64 gstreamer plugins-base/bad/alsa to RPi +rpt3 The RPi archive ships RPi-tuned +rpt3 builds of gstreamer plugins-base, plugins-bad and alsa (the archive + keyring are already configured by _rpt1-ffmpeg-pin.j2 — pi3-64 is in its board list). plugins-bad carries the vc4 kmssink the overlay-plane path scans out through, so the RPi build is the right one for production parity with Pi OS. plugins-good (the V4L2 HW decode/convert elements) and libav (audio decoders) aren't in the RPi archive, so they stay on Debian. The +rpt3 plugins share upstream 1.26.2 with Debian's libgstreamer1.0-0 core (not in the RPi archive), so the ABI matches. Validated on a real Pi 3 (clean reboot): base/bad/alsa install as 1.26.2*+rpt3*, good/libav as Debian; 30fps zero ongoing drops, audio playing, respawns=1 — unchanged from the Debian-plugins build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(viewer): address Copilot review on the pi3-64 video path - main.cpp: scope the fatal-signal backtrace handler to the pi3-64 build (#ifdef ANTHIAS_GSTREAMER) instead of installing it fleet-wide. It was added to debug the pi3-64 overlay crashes; other boards keep the default crash behaviour. (It already re-raised via SIG_DFL, so core dumps were never suppressed — but this keeps the PR to its board scope.) - gstPlay: normalise a file:// URI to a filesystem path before handing it to filesrc (Anthias passes bare local paths today; this is defensive). - Start audio once in gstPlay so BOTH the overlay AND the appsink-raster fallback get sound — previously the raster fallback played silent (audio was only started in gstPlayOverlay). gstStartAudio now escapes its own path; gstPlayOverlay drops the unused options param. - play(): log when gstPlay exhausts every GStreamer path (there is no QMediaPlayer fallback on this board) instead of returning silently. Validated on a real Pi 3: video 30fps zero ongoing drops + audio (3.5mm) still play; pi3-64 (with GStreamer) and pi4-64 (without) both compile. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(viewer): address Copilot review round 2 on the pi3-64 video path - videoview.cpp: clamp ANTHIAS_GST_QUEUE — a 0/negative/non-numeric value made queueBuffers 0, i.e. an unbounded queue (max-size-buffers/time/bytes all 0) → OOM on the 1 GB board; fall back to the default 4. - gstStartAudio: check gst_element_set_state()'s result; on GST_STATE_CHANGE_FAILURE log + tear the audio pipeline down instead of leaving a dead pipeline reporting "started". - containerFps: was written from the kmssink pad-probe (streaming thread) and read by sampleStats (GUI thread) as a plain double — a data race. Make the handoff atomic (QAtomicInteger fps×1000 via containerFps() / setContainerFps()). - main.cpp: the fatal-signal handler now uses only async-signal-safe calls (write() + backtrace_symbols_fd) instead of fprintf/fflush. - test_viewer.py: also assert ANTHIAS_VIDEO_OVERLAY is set on pi3-64 and cleared on other boards. Validated on a real Pi 3: 30fps zero ongoing drops, container-fps=30.00, audio (3.5mm) playing; pi3-64 (with GStreamer) and pi4-64 (without) both compile. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(viewer): address Copilot review round 3 on the pi3-64 video path - sampleStats: on the gstMode appsink-raster fallback, query the GStreamer pipeline position instead of QMediaPlayer::position() (the player isn't the one playing there, so position-ms was misleading). - gstPlayOverlay: guard a null QGuiApplication::primaryScreen() before using it for the DRM native-resource lookups; document why driFd<=0 is the correct "not available" test here (eglfs returns the fd through a void* API — a real DRM master fd is always >2, "no fd" comes back as 0). - main.cpp: soften the crash-handler comment — backtrace()/ backtrace_symbols_fd() are glibc extensions, not formally guaranteed async-signal-safe (best-effort last-gasp diagnostic). Compiles on pi3-64 (with GStreamer) and pi4-64 (without); Pi 3 still plays 30fps with audio. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(viewer): address Copilot review round 4 on the pi3-64 video path - gstPlay: treat a missing appsink (gst_bin_get_by_name null) as a hard failure — otherwise the pipeline goes PLAYING to a silent black screen. - play(): only start the stats sampler when gstPlay() succeeds, so a failed start doesn't emit SAMPLE lines with position=-1/elapsed=-1. - gstStartAudio: escape the ALSA device string before interpolating it into the gst_parse_launch description (same as the filesrc path). - gstStop: drop the stashed frame (under the frame mutex) and reset the drain flag, so a late-queued onGstFrame() can't repaint a stale frame after teardown. pi3-64 compiles; changes are pi3-64-scoped (#ifdef ANTHIAS_GSTREAMER) so other boards are unaffected. Pi 3 still plays 30fps with audio. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(viewer): log real position in the gst STOP line (Copilot round 5) The gstMode STOP stats line hard-coded position-ms=0, which read like playback never advanced. Query the pipeline position (it's still up at that point; gstStop() runs after) and log it, or -1 if the query fails. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(viewer): recover from pipeline errors + watch the appsink bus (round 6) - anthias_gst_bus_loop: on GST_MESSAGE_ERROR, tear the pipeline down (queued VideoView::stop()) and return FALSE to drop the watch, instead of logging and leaving a dead pipeline on a stale frame — the next asset re-shows and rebuilds. - gstPlay (appsink-raster path): install the same bus watch so pipeline errors are surfaced/recovered and EOS looping goes through one path (gstRestartLoop). Drops the now-redundant appsink EOS callback (anthias_gst_on_eos); appsink posts EOS to the bus like any GstBaseSink. Validated on a real Pi 3: overlay path still plays 30fps, loops cleanly across the 60s boundary (no respawn), audio playing. pi3-64-scoped (#ifdef ANTHIAS_GSTREAMER); other boards unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(viewer): fix stale pi3-64 video comments (Copilot round 7) Comment-only. The code evolved past several comments: - appsink note said sync=true; the pipeline uses sync=false (GUI-side coalescing paces the paint) — corrected. - gstStop said the appsink path has no bus watch; it now installs one (round 6) — corrected. - two comments claimed decoder-direct I420 "renders blue" on the vc4 plane, but decoder-direct I420 is the working default; reworded to describe the formats/mmap-copy accurately without the (thrashed-board artifact) "blue" assertion. No code change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(viewer): telemetry accuracy + drop dead playbin code (Copilot round 8) - STOP: on the overlay path report gstRawSamples as frames-rendered (the Qt frame counters stay 0 there, kmssink's tally is the real count). - appsink-raster path: latch the source framerate from the negotiated caps (latchSourceFps) so its SAMPLE computes expected/dropped instead of reporting -1. - Remove dead gstUri / gstRequeueUri (leftover playbin about-to-finish requeue path; the current pipelines don't use playbin). - Fix the gstRestartLoop header comment (loops via the pipeline bus watch, not the removed appsink EOS callback). Compiles on pi3-64 + pi4-64; overlay hot path unchanged from the already-validated build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(viewer): reset gstRawSamples per playback for both paths (round 9) gstRawSamples was reset only in gstPlayOverlay, so the appsink-raster path carried the previous asset's cumulative count into its stats. Reset it once in gstPlay() right after teardown (covers overlay + appsink); drop the now-redundant reset in gstPlayOverlay. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(viewer): tighten raster drain coalescing + clear stale surface (round 10) - onGstFrame(): release gstDrainPending at the END of the drain, not the start. pushGstFrame() uses testAndSetOrdered(0,1) so exactly one drain is ever in flight; clearing at the end keeps it that way (frames arriving mid-drain just update gstLatestFrame) — bounded GUI event queue, no unbounded QImage backlog. - gstPlay(): clear the raster surface right after teardown so the previous asset's last frame can't linger (stale-frame flash) while the new pipeline prerolls or if it fails to start. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(viewer): sigaction crash handler + ordered drain release (round 11) - main.cpp: install the fatal-signal handler via sigaction() with SA_RESETHAND instead of signal(). The handler no longer calls signal() (not async-signal-safe) — SA_RESETHAND restores the default disposition so the re-raise takes the default path (core dump / exit). - onGstFrame: release gstDrainPending with storeRelease (was storeRelaxed) so it pairs with pushGstFrame's testAndSetOrdered acquire — the drain's writes are visible before the next drain can be queued. (Copilot also flagged toImage() in onRasterPainted; that is the dormant QMediaPlayer-raster path — pi3-64 uses gstMode, which feeds frames via onGstFrame/setImage and never runs that toImage-in-paint path.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(viewer): clarify pi3-64 presenter comments (Copilot round 12) Comment-only. The two "CPU-raster presenter" comments predate the overlay default; reworded to: pi3-64 uses the non-VideoOutput presenter — the GStreamer vc4 overlay-plane path (preferred, ANTHIAS_VIDEO_OVERLAY) with the CPU-raster blit as fallback. (Copilot also suggested a QtTest that constructs VideoView with ANTHIAS_VIDEO_RASTER=1; that headless C++ harness is a reasonable follow-up — the env wiring is already covered by tests/test_viewer.py.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>