chore: drop pyzmq + libzmq, finalize ZMQ→Redis migration

With both legs of the viewer signalling path on Redis (PR1: command bus, PR2: reply bus), the pyzmq runtime dependency and the libzmq* build deps are no longer used. - pyproject.toml: remove pyzmq==23.2.1 from server, viewer, wifi-connect, and mypy dep groups (4 places). - uv.lock: regenerated; pyzmq + transitive py drop out. - tools/image_builder/{__main__,utils}.py: remove libzmq3-dev / libzmq5-dev / libzmq5 from the base apt list and from the viewer context's apt list. docker/uv-builder.j2 likewise drops libzmq3-dev from both the prebuilt-uv branch and the pip-fallback branch (32-bit ARM). The rendered docker/Dockerfile.* artifacts are gitignored, so no committed Dockerfile churn here — they regenerate cleanly via `python -m tools.image_builder --dockerfiles-only`. - send_zmq_message.py → send_viewer_message.py. The script already publishes via Redis (fixed in the PR1 follow-up); rename + update callers (bin/start_wifi_connect.sh, docker/Dockerfile.wifi-connect.j2) now that the ZMQ name is misleading. - bin/start_server.sh: drop the stale "single-worker because ZmqPublisher binds 10001" comment. The publisher is now a Redis client — no port bind, multi-worker is safe whenever the operator wants to opt in (not changed in this PR). - CLAUDE.md: update the architecture description (ZMQ ports 10001 / 5558 are gone, Redis carries the viewer signalling traffic now). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-10 09:08:09 -04:00 · 2026-04-27 17:06:04 +00:00
parent eceba40467
commit a9be1d3779
10 changed files with 8 additions and 65 deletions
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -12,11 +12,11 @@ Anthias runs as a set of Docker containers:

 - **anthias-server** (port 80 in prod, 8000 in dev) — uvicorn (ASGI) serving the Django web app, REST API, the React frontend's static assets (via WhiteNoise), uploaded media at `/anthias_assets/`, and the WebSocket endpoint at `/ws` (Django Channels with a Redis-backed channel layer). Always plain HTTP — TLS is opt-in and handled by the **anthias-caddy** sidecar that `bin/enable_ssl.sh` installs as a compose override (Caddy local CA by default, or auto Let's Encrypt with `--domain`, or BYO cert with `--cert`/`--key`).
 - **anthias-celery** — Async task queue (asset downloads, cleanup). Publishes asset-update events back to the WebSocket consumers via the Channels Redis layer.
- **anthias-viewer** — Drives the display, receives instructions via ZMQ, talks to anthias-server over HTTP.
- **redis** (port 6379) — Celery broker + result backend, and Channels channel layer.
+- **anthias-viewer** — Drives the display, receives instructions over the Redis pub/sub `anthias.viewer` channel, talks to anthias-server over HTTP.
+- **redis** (port 6379) — Celery broker + result backend, Channels channel layer, and the viewer signalling bus (pub/sub channel + per-correlation-ID reply lists).
 - **webview** — Qt-based browser for rendering content on the display; fetches `/anthias_assets/` from anthias-server.

-Inter-service communication uses ZMQ (port 10001 publisher, 5558 collector) for the viewer signalling path; WebSocket fan-out from Celery to browsers goes via Channels/Redis. The primary database is SQLite stored at `~/.anthias/anthias.db`, with configuration in `~/.anthias/anthias.conf`. (Pre-rebrand installations have these at `~/.screenly/screenly.db` and `~/.screenly/screenly.conf`; `bin/migrate_legacy_paths.sh` migrates them on upgrade and leaves back-compat symlinks.)
+Inter-service messaging is all Redis: WebSocket fan-out from Celery to browsers goes via Channels/Redis, and server↔viewer commands/replies use Redis pub/sub on `anthias.viewer` with BLPOP on `anthias.reply.<correlation-id>` for the few request-reply paths. The primary database is SQLite stored at `~/.anthias/anthias.db`, with configuration in `~/.anthias/anthias.conf`. (Pre-rebrand installations have these at `~/.screenly/screenly.db` and `~/.screenly/screenly.conf`; `bin/migrate_legacy_paths.sh` migrates them on upgrade and leaves back-compat symlinks.)

 ### Key Directories

@@ -24,9 +24,9 @@ Inter-service communication uses ZMQ (port 10001 publisher, 5558 collector) for
 - `anthias_django/` — Django project settings, URLs, ASGI/WSGI
 - `api/` — REST API (views, serializers, URLs for v1, v1.1, v1.2, v2)
 - `static/src/` — TypeScript/React frontend (components, Redux store, hooks, tests)
- `viewer/` — Viewer service (scheduling, media player, ZMQ communication)
+- `viewer/` — Viewer service (scheduling, media player, Redis pub/sub messaging)
 - `webview/` — C++ Qt-based WebView (Qt5 for Pi 1-4, Qt6 for Pi 5/x86)
- `lib/` — Shared Python utilities (auth, device helpers, diagnostics, ZMQ)
+- `lib/` — Shared Python utilities (auth, device helpers, diagnostics)
 - `docker/` — Dockerfile Jinja2 templates for each service
 - `tests/` — Python unit/integration tests
 - `bin/` — Shell scripts for install, dev setup, testing, upgrades
--- a/bin/start_server.sh
+++ b/bin/start_server.sh
@@ -30,10 +30,6 @@ else
        ./manage.py dbbackup --noinput --clean
 fi

-# Single-worker on purpose: ZmqPublisher.get_instance() in api/views/*
-# binds tcp://0.0.0.0:10001, which would EADDRINUSE across multiple
-# workers. Hoist the publisher into a sidecar (or move it to the
-# Channels layer) before adding `--workers N` here.
 UVICORN_BIND_HOST="${LISTEN:-0.0.0.0}"
 UVICORN_BIND_PORT="${PORT:-8080}"

--- a/bin/start_wifi_connect.sh
+++ b/bin/start_wifi_connect.sh
@@ -31,7 +31,7 @@ while [[ true ]]; do
        echo "Skipping setting up Wifi-Connect Access Point."

        if [[ "$IS_CONNECTED" = 'false' ]]; then
-            python send_zmq_message.py --action='show_splash'
+            python send_viewer_message.py --action='show_splash'
        fi

        exit 0
@@ -41,7 +41,7 @@ while [[ true ]]; do
        echo "Connect to the Access Point and configure the SSID and Passphrase for the network to connect to."

        if [[ "$IS_CONNECTED" = '' ]]; then
-            python send_zmq_message.py --action='setup_wifi'
+            python send_viewer_message.py --action='setup_wifi'
        fi

        IS_CONNECTED='false'
--- a/docker/Dockerfile.wifi-connect.j2
+++ b/docker/Dockerfile.wifi-connect.j2
@@ -30,7 +30,7 @@ COPY --from=uv-builder /venv /venv
 ENV PATH="/venv/bin:$PATH"
 ENV VIRTUAL_ENV="/venv"

-COPY send_zmq_message.py ./
+COPY send_viewer_message.py ./

 RUN touch /var/lib/misc/dnsmasq.leases

--- a/docker/uv-builder.j2
+++ b/docker/uv-builder.j2
@@ -21,7 +21,6 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
        build-essential \
        libffi-dev \
        libssl-dev \
-        libzmq3-dev \
        python3 \
        python3-dev \
        python3-pip \
@@ -42,7 +41,6 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
        build-essential \
        libffi-dev \
        libssl-dev \
-        libzmq3-dev \
        python3 \
        python3-dev \
 {% for dep in builder_extra_apt | default([]) %}
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -59,7 +59,6 @@ server = [
    "python-dateutil==2.9.0.post0",
    "pytz==2025.2",
    "PyYAML==6.0.2",
-    "pyzmq==23.2.1",
    "redis==7.4.0",
    "requests[security]==2.33.1",
    "setuptools==79.0.1",
@@ -90,7 +89,6 @@ viewer = [
    "python-dateutil==2.9.0.post0",
    "python-vlc==3.0.21203",
    "pytz==2025.2",
-    "pyzmq==23.2.1",
    "redis==7.4.0",
    "requests[security]==2.33.1",
    "tenacity==9.1.2",
@@ -102,7 +100,6 @@ wifi-connect = [
    "Cython==3.2.4",
    "future==1.0.0",
    "netifaces==0.11.0",
-    "pyzmq==23.2.1",
    "redis==7.4.0",
 ]
 dev = [
@@ -148,7 +145,6 @@ mypy = [
    "djangorestframework==3.16.1",
    "drf-spectacular==0.29.0",
    "pytz==2025.2",
-    "pyzmq==23.2.1",
    "whitenoise==6.8.2",
 ]

--- a/send_viewer_message.py
+++ b/send_viewer_message.py
--- a/tools/image_builder/main.py
+++ b/tools/image_builder/main.py
@@ -65,9 +65,6 @@ def build_image(
        'libcec-dev ',
        'libffi-dev',
        'libssl-dev',
-        'libzmq3-dev',
-        'libzmq5-dev',
-        'libzmq5',
        'lsb-release',
        'mplayer',
        'net-tools',
--- a/tools/image_builder/utils.py
+++ b/tools/image_builder/utils.py
@@ -195,7 +195,6 @@ def get_viewer_context(board: str) -> dict[str, Any]:
        'libsqlite3-dev',
        'libsrtp2-dev',
        'libssl-dev',
-        'libzmq3-dev',
        'libswscale-dev',
        'libsystemd-dev',
        'libts-dev',
@@ -232,8 +231,6 @@ def get_viewer_context(board: str) -> dict[str, Any]:
        'libxslt1-dev',
        'libxss-dev',
        'libxtst-dev',
-        'libzmq5-dev',
-        'libzmq5',
        'net-tools',
        'procps',
        'psmisc',
--- a/uv.lock
+++ b/uv.lock
@@ -157,7 +157,6 @@ mypy = [
    { name = "pygit2" },
    { name = "python-on-whales" },
    { name = "pytz" },
-    { name = "pyzmq" },
    { name = "requests" },
    { name = "ruff" },
    { name = "tenacity" },
@@ -199,7 +198,6 @@ server = [
    { name = "python-dateutil" },
    { name = "pytz" },
    { name = "pyyaml" },
-    { name = "pyzmq" },
    { name = "redis" },
    { name = "requests" },
    { name = "setuptools" },
@@ -245,7 +243,6 @@ test = [
    { name = "python-vlc" },
    { name = "pytz" },
    { name = "pyyaml" },
-    { name = "pyzmq" },
    { name = "redis" },
    { name = "requests" },
    { name = "selenium" },
@@ -281,7 +278,6 @@ viewer = [
    { name = "python-dateutil" },
    { name = "python-vlc" },
    { name = "pytz" },
-    { name = "pyzmq" },
    { name = "redis" },
    { name = "requests" },
    { name = "sh" },
@@ -293,7 +289,6 @@ wifi-connect = [
    { name = "cython" },
    { name = "future" },
    { name = "netifaces" },
-    { name = "pyzmq" },
    { name = "redis" },
 ]

@@ -363,7 +358,6 @@ mypy = [
    { name = "pygit2", specifier = "==1.19.1" },
    { name = "python-on-whales", specifier = "==0.79.0" },
    { name = "pytz", specifier = "==2025.2" },
-    { name = "pyzmq", specifier = "==23.2.1" },
    { name = "requests", specifier = "==2.33.1" },
    { name = "ruff", specifier = "==0.14.10" },
    { name = "tenacity", specifier = "==9.1.2" },
@@ -405,7 +399,6 @@ server = [
    { name = "python-dateutil", specifier = "==2.9.0.post0" },
    { name = "pytz", specifier = "==2025.2" },
    { name = "pyyaml", specifier = "==6.0.2" },
-    { name = "pyzmq", specifier = "==23.2.1" },
    { name = "redis", specifier = "==7.4.0" },
    { name = "requests", extras = ["security"], specifier = "==2.33.1" },
    { name = "setuptools", specifier = "==79.0.1" },
@@ -451,7 +444,6 @@ test = [
    { name = "python-vlc", specifier = "==3.0.21203" },
    { name = "pytz", specifier = "==2025.2" },
    { name = "pyyaml", specifier = "==6.0.2" },
-    { name = "pyzmq", specifier = "==23.2.1" },
    { name = "redis", specifier = "==7.4.0" },
    { name = "requests", extras = ["security"], specifier = "==2.33.1" },
    { name = "selenium", specifier = "==4.36.0" },
@@ -487,7 +479,6 @@ viewer = [
    { name = "python-dateutil", specifier = "==2.9.0.post0" },
    { name = "python-vlc", specifier = "==3.0.21203" },
    { name = "pytz", specifier = "==2025.2" },
-    { name = "pyzmq", specifier = "==23.2.1" },
    { name = "redis", specifier = "==7.4.0" },
    { name = "requests", extras = ["security"], specifier = "==2.33.1" },
    { name = "sh", specifier = "==2.2.2" },
@@ -499,7 +490,6 @@ wifi-connect = [
    { name = "cython", specifier = "==3.2.4" },
    { name = "future", specifier = "==1.0.0" },
    { name = "netifaces", specifier = "==0.11.0" },
-    { name = "pyzmq", specifier = "==23.2.1" },
    { name = "redis", specifier = "==7.4.0" },
 ]

@@ -1546,15 +1536,6 @@ wheels = [
    { url = "https://files.pythonhosted.org/packages/3e/73/2ce007f4198c80fcf2cb24c169884f833fe93fbc03d55d302627b094ee91/psutil-7.2.1-cp37-abi3-win_arm64.whl", hash = "sha256:0d67c1822c355aa6f7314d92018fb4268a76668a536f133599b91edd48759442", size = 133836, upload-time = "2025-12-29T08:26:43.086Z" },
 ]

-[[package]]
-name = "py"
-version = "1.11.0"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/98/ff/fec109ceb715d2a6b4c4a85a61af3b40c723a961e8828319fbcb15b868dc/py-1.11.0.tar.gz", hash = "sha256:51c75c4126074b472f746a24399ad32f6053d1b34b68d2fa41e558e6f4a98719", size = 207796, upload-time = "2021-11-04T17:17:01.377Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/f6/f0/10642828a8dfb741e5f3fbaac830550a518a775c7fff6f04a007259b0548/py-1.11.0-py2.py3-none-any.whl", hash = "sha256:607c53218732647dff4acdfcd50cb62615cedf612e72d1724fb1a0cc6405b378", size = 98708, upload-time = "2021-11-04T17:17:00.152Z" },
-]
-
 [[package]]
 name = "pyasn1"
 version = "0.6.3"
@@ -1903,28 +1884,6 @@ wheels = [
    { url = "https://files.pythonhosted.org/packages/fa/de/02b54f42487e3d3c6efb3f89428677074ca7bf43aae402517bc7cca949f3/PyYAML-6.0.2-cp313-cp313-win_amd64.whl", hash = "sha256:8388ee1976c416731879ac16da0aff3f63b286ffdd57cdeb95f3f2e085687563", size = 156446, upload-time = "2024-08-06T20:33:04.33Z" },
 ]

-[[package]]
-name = "pyzmq"
-version = "23.2.1"
-source = { registry = "https://pypi.org/simple" }
-dependencies = [
-    { name = "cffi", marker = "implementation_name == 'pypy'" },
-    { name = "py", marker = "implementation_name == 'pypy'" },
-]
-sdist = { url = "https://files.pythonhosted.org/packages/72/37/d5603f352522e249e44ee767a8a59b3fe7cf7f708a94fd40a637c6890add/pyzmq-23.2.1.tar.gz", hash = "sha256:2b381aa867ece7d0a82f30a0c7f3d4387b7cf2e0697e33efaa5bed6c5784abcd", size = 1218264, upload-time = "2022-08-12T09:01:12.157Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/b4/e3/081b1a02af85b60e19191ea5d41b422ea7a0126043b7c54b5f3c2ce8e065/pyzmq-23.2.1-cp311-cp311-macosx_10_15_universal2.whl", hash = "sha256:e753eee6d3b93c5354e8ba0a1d62956ee49355f0a36e00570823ef64e66183f5", size = 1829830, upload-time = "2022-08-12T09:17:58.12Z" },
-    { url = "https://files.pythonhosted.org/packages/38/e7/3ff3068074316da9990e889382b7a5f08728ebca893a2b8e018c1531633f/pyzmq-23.2.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:f07016e3cf088dbfc6e7c5a7b3f540db5c23b0190d539e4fd3e2b5e6beffa4b5", size = 1244536, upload-time = "2022-08-12T09:18:12.296Z" },
-    { url = "https://files.pythonhosted.org/packages/dc/24/3ac8d73b09a6db9d26d4c6fb454101dbc9e1ea1e6df50cd50041decbd895/pyzmq-23.2.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4805af9614b0b41b7e57d17673459facf85604dac502a5a9244f6e8c9a4de658", size = 880172, upload-time = "2022-08-12T09:12:29.324Z" },
-    { url = "https://files.pythonhosted.org/packages/a7/bb/54986ab979a4f964c6b76817b6fdd7403a9cf76f368a0c67ec6bb65a783d/pyzmq-23.2.1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:39dd252b683816935702825e5bf775df16090619ced9bb4ba68c2d0b6f0c9b18", size = 1130741, upload-time = "2022-08-12T09:11:08.068Z" },
-    { url = "https://files.pythonhosted.org/packages/4d/b4/804256cc2a3668b152c5ddf65bf319ae8d922ef6bfadee24e0ef23f096d6/pyzmq-23.2.1-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:84678153432241bcdca2210cf4ff83560b200556867aea913ffbb960f5d5f340", size = 1075742, upload-time = "2022-08-12T09:18:41.418Z" },
-    { url = "https://files.pythonhosted.org/packages/6b/36/0546d92a57ca60a456f801f4e1a55932c8c6e324e1b77f6faca660728b4e/pyzmq-23.2.1-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:90d88f9d9a2ae6cfb1dc4ea2d1710cdf6456bc1b9a06dd1bb485c5d298f2517e", size = 1407475, upload-time = "2022-08-12T09:12:32.562Z" },
-    { url = "https://files.pythonhosted.org/packages/a7/2c/9ff05957a40dc87889a584457a6c700e77ad3f623e4f5e33a29dfcc2ea59/pyzmq-23.2.1-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:794871988c34727c7f79bdfe2546e6854ae1fa2e1feb382784f23a9c6c63ecb3", size = 1737821, upload-time = "2022-08-12T09:13:42.82Z" },
-    { url = "https://files.pythonhosted.org/packages/b6/a0/79ae7abdc28eecc261806a9ebbfb03f05709a501350009c5925e0ed73a29/pyzmq-23.2.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:c56b1a62a1fb87565343c57b6743fd5da6e138b8c6562361d7d9b5ce4acf399a", size = 1626767, upload-time = "2022-08-12T09:13:44.173Z" },
-    { url = "https://files.pythonhosted.org/packages/cd/88/dcbe0bfc6b744badda43322e2538ab269451e0f298d3e4616da92a25dfa3/pyzmq-23.2.1-cp311-cp311-win32.whl", hash = "sha256:c3ebf1668664d20c8f7d468955f18379b7d1f7bc8946b13243d050fa3888c7ff", size = 887707, upload-time = "2022-08-12T09:09:56.207Z" },
-    { url = "https://files.pythonhosted.org/packages/b7/90/da3761e269f7d5a7916e98364f55745ca1a979a6316a265fc4b4ac26e144/pyzmq-23.2.1-cp311-cp311-win_amd64.whl", hash = "sha256:ec9803aca9491fd6f0d853d2a6147f19f8deaaa23b1b713d05c5d09e56ea7142", size = 1139705, upload-time = "2022-08-12T09:08:42.647Z" },
-]
-
 [[package]]
 name = "redis"
 version = "7.4.0"