From 8041fc30e451b4514eef5830dd345281fc980d40 Mon Sep 17 00:00:00 2001 From: Viktor Petersson Date: Mon, 27 Apr 2026 19:50:33 +0100 Subject: [PATCH 1/2] ci: switch primary registry to ghcr, drop legacy srly-ose namespace (#2761) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Make `ghcr.io/screenly/anthias-*` the canonical source for Anthias container images and demote Docker Hub's `screenly/anthias-*` to a parallel mirror during the migration window. The legacy `screenly/srly-ose-*` namespace is dropped entirely (matrix push + latest-* mirror). The compose templates are flipped to ghcr in the same change so `bin/upgrade_containers.sh` regenerates with ghcr on the next run. Why --- Two motivations stack: 1. Docker Hub's anonymous-pull rate limit (100 pulls / 6h per IP) bites end-users when a fleet of devices behind one NAT all run `bin/upgrade_containers.sh` at once, not just CI. GHCR has no such limit for public packages, and storage is free unlimited. Authed pushes from CI also get a much higher quota under the GitHub Actions token than under our shared Docker Hub bot. 2. d568602's publish-latest hit Docker Hub's 429 rate limit on retag #52 (`srly-ose-redis:latest-pi3`) — the legacy namespace doubled the manifest GETs in the loop and bought no real back-compat in exchange. `docker-compose.yml.tmpl` has pointed installs at `screenly/anthias-*` since 2023-02 (b9998438), and `bin/upgrade_containers.sh` regenerates compose from the template on every upgrade, so any device that has run an upgrade in the past three years is on `screenly/anthias-*` already. What ships ---------- * `tools/image_builder/__main__.py` — `namespaces` becomes `['ghcr.io/screenly/anthias', 'screenly/anthias']`. GHCR is listed first so it's the primary push target; Docker Hub is the parallel mirror. The buildx matrix now pushes both `-` tags to both registries on every build. * `.github/workflows/docker-build.yaml` — adds job-scoped `permissions: { contents: read, packages: write }` on `buildx` and `publish-latest` (not at workflow level, so `run-tests` doesn't inherit), plus a `Login to GitHub Container Registry` step using `${{ github.actor }}` + `${{ secrets.GITHUB_TOKEN }}` in both jobs. The publish-latest mirror loop iterates over both namespaces (GHCR first) inside the same retry-wrapped retag block, so `latest-` advances atomically across both registries or not at all. * `docker/labels.j2` — new shared partial that emits the OCI image labels (`source`, `url`, `licenses`, `title`, `description`). `image.source` is the load-bearing one for GHCR: it links the package to its source repo, which makes the package inherit the repo's visibility and grants repo collaborators push/delete access. * `docker/Dockerfile.{base,redis,viewer}.j2` — include the new partial. `Dockerfile.base.j2` covers server / celery / wifi-connect / test (which all `{% include 'Dockerfile.base.j2' %}`); `redis.j2` and `viewer.j2` have their own production-stage `FROM` so include `labels.j2` directly. * `docker-compose.yml.tmpl`, `docker-compose.balena.yml.tmpl`, `docker-compose.balena.dev.yml.tmpl` — flip 15 `image:` lines from `screenly/anthias-*` to `ghcr.io/screenly/anthias-*`. Devices pick this up on next `bin/upgrade_containers.sh` (the script regenerates `docker-compose.yml` from the template). Retry-with-backoff seatbelt around `imagetools` calls (originally added in 8099a14a) is preserved. Deployment notes ---------------- After this lands, the docker-build workflow will run on master and publish to GHCR for the first time. Before merging, set `Screenly`'s default-new-package visibility to "Public" at https://github.com/organizations/Screenly/settings/packages so the five new packages don't land private. (`org.opencontainers.image.source` auto-links each package to this repo but does not set visibility.) Migration-window risk: between merge and `publish-latest` completion (~80 min), `ghcr.io/screenly/anthias-*:latest-` tags don't exist yet. Devices that run `bin/upgrade_containers.sh` in that window will fail to pull and stay on their existing containers (no auto-fallback to Docker Hub). They'll pull successfully on the next upgrade attempt. To minimise impact, merge during a low-fleet-upgrade window. Phase 3 (months later, separate PR): stop publishing `latest-*` to Docker Hub once enough fleet has rotated through an upgrade. `-` tags on Docker Hub stay around indefinitely for explicit pins. Co-authored-by: Claude Opus 4.7 (1M context) --- .github/workflows/docker-build.yaml | 78 ++++++++++++++++++++++++++--- docker-compose.balena.dev.yml.tmpl | 10 ++-- docker-compose.balena.yml.tmpl | 10 ++-- docker-compose.yml.tmpl | 10 ++-- docker/Dockerfile.base.j2 | 1 + docker/Dockerfile.redis.j2 | 2 + docker/Dockerfile.viewer.j2 | 2 + docker/labels.j2 | 19 +++++++ tools/image_builder/__main__.py | 19 ++++++- 9 files changed, 128 insertions(+), 23 deletions(-) create mode 100644 docker/labels.j2 diff --git a/.github/workflows/docker-build.yaml b/.github/workflows/docker-build.yaml index 6b903367..bdc381d6 100644 --- a/.github/workflows/docker-build.yaml +++ b/.github/workflows/docker-build.yaml @@ -37,6 +37,15 @@ jobs: buildx: needs: run-tests + # Scoped per-job (not at workflow level) so `run-tests` and any + # future read-only job don't inherit `packages: write`. `buildx` + # needs it so `docker login ghcr.io` with GITHUB_TOKEN can push + # ghcr.io/screenly/anthias-*. `contents: read` is the implicit + # default but pinned explicitly so a future workflow edit can't + # silently lose checkout access. + permissions: + contents: read + packages: write strategy: # Don't cancel sibling jobs on the first failure: any platform that # has already finished building its image will have pushed the @@ -106,6 +115,14 @@ jobs: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} + - name: Login to GitHub Container Registry + if: success() && github.event_name == 'push' + uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + - name: Build Container env: DOCKER_BUILDKIT: 1 @@ -132,13 +149,31 @@ jobs: # checks every - tag is resolvable before any # retag fires, so a missing source tag fails the job before it # mutates the registry. `imagetools create` re-points the registry - # tag to the existing manifest without re-uploading layers, so the - # full ~98 retags (7 boards × 7 services × 2 namespaces) finish in - # under two minutes. + # tag to the existing manifest without re-uploading layers, and each + # call is wrapped in 5-attempt exponential backoff so a transient + # 429 / 5xx doesn't strand latest-* half-mirrored. + # + # Two registries: `ghcr.io/screenly/anthias-*` is the new primary + # (rate-limit-friendly, free unlimited storage for public packages, + # auth via the workflow-issued GITHUB_TOKEN with `packages: write`). + # `screenly/anthias-*` on Docker Hub stays in the loop as a parallel + # mirror during the migration window so devices that haven't yet + # picked up the compose-template flip keep getting `latest-*` + # advanced. GHCR is mirrored first so the canonical source is up to + # date even if Docker Hub flakes on a later retag. The legacy + # `screenly/srly-ose-*` namespace was dropped: every device that + # has run `upgrade_containers.sh` since 2023-02 (b9998438) is on + # `screenly/anthias-*` already, so the parallel `srly-ose-*` push + # was buying no real back-compat in exchange for half the manifest + # GETs (one of two reasons d568602's publish hit Docker Hub's 429). publish-latest: needs: buildx if: github.event_name == 'push' runs-on: ubuntu-24.04 + # See `buildx.permissions` above for rationale; same scoping reason. + permissions: + contents: read + packages: write steps: - name: Checkout uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 @@ -152,13 +187,44 @@ jobs: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} + - name: Login to GitHub Container Registry + uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + - name: Mirror short-hash tags to latest- run: | set -euo pipefail GIT_SHORT_HASH=$(git rev-parse --short=7 HEAD) BOARDS=(pi1 pi2 pi3 pi4 pi4-64 pi5 x86) SERVICES=(server celery redis viewer wifi-connect) - NAMESPACES=(screenly/anthias screenly/srly-ose) + # GHCR first so the canonical primary is current even if the + # Docker Hub mirror later in the loop flakes. + NAMESPACES=(ghcr.io/screenly/anthias screenly/anthias) + + # Wrap a registry op in 5 attempts of exponential backoff + # (2s, 4s, 8s, 16s) so a transient 429 / 5xx doesn't strand + # `latest-*` half-mirrored. Both `imagetools inspect` and + # `imagetools create` are idempotent: inspect is read-only, + # and create overwrites the tag with the same manifest + # digest on retry. + retry() { + local attempt + for attempt in 1 2 3 4 5; do + if "$@"; then + return 0 + fi + if [ "${attempt}" -lt 5 ]; then + local delay=$((2 ** attempt)) + echo "Attempt ${attempt} failed; retrying in ${delay}s..." >&2 + sleep "${delay}" + fi + done + echo "Giving up after 5 attempts: $*" >&2 + return 1 + } echo "::group::Preflight: verify every - source tag exists" for namespace in "${NAMESPACES[@]}"; do @@ -166,7 +232,7 @@ jobs: for board in "${BOARDS[@]}"; do src="${namespace}-${service}:${GIT_SHORT_HASH}-${board}" echo "Verifying ${src}" - docker buildx imagetools inspect --raw "${src}" >/dev/null + retry docker buildx imagetools inspect --raw "${src}" >/dev/null done done done @@ -179,7 +245,7 @@ jobs: src="${namespace}-${service}:${GIT_SHORT_HASH}-${board}" dst="${namespace}-${service}:latest-${board}" echo "Mirroring ${src} -> ${dst}" - docker buildx imagetools create -t "${dst}" "${src}" + retry docker buildx imagetools create -t "${dst}" "${src}" done done done diff --git a/docker-compose.balena.dev.yml.tmpl b/docker-compose.balena.dev.yml.tmpl index 92763ea7..c1fea4dd 100644 --- a/docker-compose.balena.dev.yml.tmpl +++ b/docker-compose.balena.dev.yml.tmpl @@ -3,7 +3,7 @@ version: "2" services: anthias-wifi-connect: - image: screenly/anthias-wifi-connect:${GIT_SHORT_HASH}-${BOARD} + image: ghcr.io/screenly/anthias-wifi-connect:${GIT_SHORT_HASH}-${BOARD} build: context: . dockerfile: ./docker/Dockerfile.wifi-connect @@ -21,7 +21,7 @@ services: io.balena.features.firmware: "1" anthias-server: - image: screenly/anthias-server:${GIT_SHORT_HASH}-${BOARD} + image: ghcr.io/screenly/anthias-server:${GIT_SHORT_HASH}-${BOARD} build: context: . dockerfile: ./docker/Dockerfile.server @@ -43,7 +43,7 @@ services: io.balena.features.supervisor-api: '1' anthias-viewer: - image: screenly/anthias-viewer:${GIT_SHORT_HASH}-${BOARD} + image: ghcr.io/screenly/anthias-viewer:${GIT_SHORT_HASH}-${BOARD} build: context: . dockerfile: ./docker/Dockerfile.viewer @@ -65,7 +65,7 @@ services: io.balena.features.supervisor-api: '1' anthias-celery: - image: screenly/anthias-celery:${GIT_SHORT_HASH}-${BOARD} + image: ghcr.io/screenly/anthias-celery:${GIT_SHORT_HASH}-${BOARD} build: context: . dockerfile: ./docker/Dockerfile.celery @@ -85,7 +85,7 @@ services: io.balena.features.supervisor-api: '1' redis: - image: screenly/anthias-redis:${GIT_SHORT_HASH}-${BOARD} + image: ghcr.io/screenly/anthias-redis:${GIT_SHORT_HASH}-${BOARD} build: context: . dockerfile: ./docker/Dockerfile.redis diff --git a/docker-compose.balena.yml.tmpl b/docker-compose.balena.yml.tmpl index 3cd9ed8a..a580e1ed 100644 --- a/docker-compose.balena.yml.tmpl +++ b/docker-compose.balena.yml.tmpl @@ -3,7 +3,7 @@ version: "2" services: anthias-wifi-connect: - image: screenly/anthias-wifi-connect:${GIT_SHORT_HASH}-${BOARD} + image: ghcr.io/screenly/anthias-wifi-connect:${GIT_SHORT_HASH}-${BOARD} depends_on: - anthias-viewer environment: @@ -18,7 +18,7 @@ services: io.balena.features.firmware: "1" anthias-server: - image: screenly/anthias-server:${GIT_SHORT_HASH}-${BOARD} + image: ghcr.io/screenly/anthias-server:${GIT_SHORT_HASH}-${BOARD} ports: - 80:8080 environment: @@ -37,7 +37,7 @@ services: io.balena.features.supervisor-api: '1' anthias-viewer: - image: screenly/anthias-viewer:${GIT_SHORT_HASH}-${BOARD} + image: ghcr.io/screenly/anthias-viewer:${GIT_SHORT_HASH}-${BOARD} depends_on: - anthias-server environment: @@ -56,7 +56,7 @@ services: io.balena.features.supervisor-api: '1' anthias-celery: - image: screenly/anthias-celery:${GIT_SHORT_HASH}-${BOARD} + image: ghcr.io/screenly/anthias-celery:${GIT_SHORT_HASH}-${BOARD} depends_on: - anthias-server - redis @@ -73,7 +73,7 @@ services: io.balena.features.supervisor-api: '1' redis: - image: screenly/anthias-redis:${GIT_SHORT_HASH}-${BOARD} + image: ghcr.io/screenly/anthias-redis:${GIT_SHORT_HASH}-${BOARD} ports: - 127.0.0.1:6379:6379 restart: always diff --git a/docker-compose.yml.tmpl b/docker-compose.yml.tmpl index c868c2d8..efd99927 100644 --- a/docker-compose.yml.tmpl +++ b/docker-compose.yml.tmpl @@ -2,7 +2,7 @@ services: anthias-wifi-connect: - image: screenly/anthias-wifi-connect:${DOCKER_TAG}-${DEVICE_TYPE} + image: ghcr.io/screenly/anthias-wifi-connect:${DOCKER_TAG}-${DEVICE_TYPE} build: context: . dockerfile: docker/Dockerfile.wifi-connect @@ -21,7 +21,7 @@ services: target: /run/dbus/system_bus_socket anthias-server: - image: screenly/anthias-server:${DOCKER_TAG}-${DEVICE_TYPE} + image: ghcr.io/screenly/anthias-server:${DOCKER_TAG}-${DEVICE_TYPE} build: context: . dockerfile: docker/Dockerfile.server @@ -51,7 +51,7 @@ services: io.balena.features.supervisor-api: '1' anthias-viewer: - image: screenly/anthias-viewer:${DOCKER_TAG}-${DEVICE_TYPE} + image: ghcr.io/screenly/anthias-viewer:${DOCKER_TAG}-${DEVICE_TYPE} build: context: . dockerfile: docker/Dockerfile.viewer @@ -79,7 +79,7 @@ services: io.balena.features.supervisor-api: '1' anthias-celery: - image: screenly/anthias-celery:${DOCKER_TAG}-${DEVICE_TYPE} + image: ghcr.io/screenly/anthias-celery:${DOCKER_TAG}-${DEVICE_TYPE} build: context: . dockerfile: docker/Dockerfile.celery @@ -103,7 +103,7 @@ services: io.balena.features.supervisor-api: '1' redis: - image: screenly/anthias-redis:${DOCKER_TAG}-${DEVICE_TYPE} + image: ghcr.io/screenly/anthias-redis:${DOCKER_TAG}-${DEVICE_TYPE} build: context: . dockerfile: docker/Dockerfile.redis diff --git a/docker/Dockerfile.base.j2 b/docker/Dockerfile.base.j2 index ddc7fecf..e5dd0c2c 100644 --- a/docker/Dockerfile.base.j2 +++ b/docker/Dockerfile.base.j2 @@ -19,3 +19,4 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ # https://github.com/balena-io-library/base-images/issues/562 RUN c_rehash +{% include 'labels.j2' %} diff --git a/docker/Dockerfile.redis.j2 b/docker/Dockerfile.redis.j2 index f817be77..25d03a8a 100644 --- a/docker/Dockerfile.redis.j2 +++ b/docker/Dockerfile.redis.j2 @@ -19,5 +19,7 @@ ENV GIT_BRANCH={{ git_branch }} RUN sed -i 's/^bind.*/bind 0.0.0.0/g' /etc/redis/redis.conf RUN sed -i 's/^protected-mode.*/protected-mode no/g' /etc/redis/redis.conf +{% include 'labels.j2' %} + CMD ["redis-server", "--protected-mode", "no"] diff --git a/docker/Dockerfile.viewer.j2 b/docker/Dockerfile.viewer.j2 index fe44d43f..18998ec0 100644 --- a/docker/Dockerfile.viewer.j2 +++ b/docker/Dockerfile.viewer.j2 @@ -70,4 +70,6 @@ WORKDIR /usr/src/app RUN mkdir -p /usr/src/app COPY . /usr/src/app/ +{% include 'labels.j2' %} + CMD ["bash", "./bin/start_viewer.sh"] diff --git a/docker/labels.j2 b/docker/labels.j2 new file mode 100644 index 00000000..df4ef6e1 --- /dev/null +++ b/docker/labels.j2 @@ -0,0 +1,19 @@ +{# OCI image labels (https://specs.opencontainers.org/image-spec/annotations/). + `image.source` is the load-bearing one for GitHub Container Registry: + GHCR uses it to link the package to its source repo, which makes the + package inherit the repo's visibility and grants repo collaborators + push/delete access. Without it, packages stay private even when the + repo is public, and the package page on GitHub has no link back. #} +{% set service_descriptions = { + 'server': 'Anthias web server (uvicorn + Django + Channels)', + 'celery': 'Anthias background task worker (asset downloads, cleanup, display power)', + 'redis': 'Redis broker for Anthias Celery and Channels', + 'viewer': 'Anthias display/viewer service', + 'wifi-connect': 'Anthias WiFi captive-portal helper', + 'test': 'Anthias test runner', +} %} +LABEL org.opencontainers.image.source="https://github.com/Screenly/Anthias" +LABEL org.opencontainers.image.url="https://anthias.screenly.io" +LABEL org.opencontainers.image.licenses="GPL-2.0" +LABEL org.opencontainers.image.title="anthias-{{ service }}" +LABEL org.opencontainers.image.description="{{ service_descriptions.get(service, 'Anthias ' + service) }}" diff --git a/tools/image_builder/__main__.py b/tools/image_builder/__main__.py index 488b034f..606eee2e 100644 --- a/tools/image_builder/__main__.py +++ b/tools/image_builder/__main__.py @@ -109,6 +109,7 @@ def build_image( 'git_branch': git_branch, 'git_hash': git_hash, 'git_short_hash': git_short_hash, + 'service': service, 'target_platform': target_platform, **context, }, @@ -239,8 +240,22 @@ def main( # Build Docker images for service_name in services_to_build: - # Define tag components - namespaces = ['screenly/anthias', 'screenly/srly-ose'] + # Define tag components. + # + # GHCR is listed first because it is the primary, canonical source + # for Anthias images going forward — `bin/upgrade_containers.sh` + # regenerates compose from `docker-compose.yml.tmpl`, so flipping + # the template (separate change) flips every device on next + # upgrade. Docker Hub stays in the list as a parallel push during + # the migration window so devices that haven't yet picked up the + # template flip keep getting `latest-*` advanced. + # + # The legacy `screenly/srly-ose-*` namespace was dropped: every + # device that has run `upgrade_containers.sh` since 2023-02 + # (b9998438) is on `screenly/anthias-*`, and stale `srly-ose-*` + # `latest-*` mirroring (one of two reasons d568602 hit Docker + # Hub's 429) gives no real back-compat in exchange. + namespaces = ['ghcr.io/screenly/anthias', 'screenly/anthias'] version_suffix = ( f'{board}-64' if board == 'pi4' and platform == 'linux/arm64/v8' From bf6e9a17414176b8f6f7dab22e43833bf0c9b01f Mon Sep 17 00:00:00 2001 From: Viktor Petersson Date: Tue, 28 Apr 2026 08:08:19 +0100 Subject: [PATCH 2/2] fix(viewer): unbreak django.setup() in viewer container (#2762) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(viewer): unbreak django.setup() in viewer container The mypy commit (93e55018) added `import django_stubs_ext` and `django_stubs_ext.monkeypatch()` to anthias_django/settings.py, but `django-stubs-ext` is only in the `server`/`test` dependency groups, not `viewer`. The viewer also tries to load every entry in `INSTALLED_APPS` at django.setup() time, which pulls in `channels`, `rest_framework`, `drf_spectacular`, `dbbackup` — none of which the viewer ships or uses (it never serves HTTP). Both failure modes were hidden by a bare `try: django.setup() ... except Exception: pass` in viewer/__init__.py, leaving `connect_to_redis` undefined for the next module-level statement. End result on real hardware (Pi and x86): File "/usr/src/app/viewer/__init__.py", line 63, in r = connect_to_redis() NameError: name 'connect_to_redis' is not defined — a misleading symptom three layers downstream of the actual ModuleNotFoundError. Changes: * `anthias_django/settings.py`: - Make `import django_stubs_ext` + `monkeypatch()` optional. The codebase has zero runtime usages of `QuerySet[Asset]`-style subscriptable Django generics (and no `from __future__ import annotations`), so the patch is currently a no-op anyway. mypy + django-stubs still pick it up at type-check time because the dev group ships it. - Gate `INSTALLED_APPS` on `ANTHIAS_SERVICE=viewer`. The viewer only needs `anthias_app` + `contenttypes` + `auth` for ORM access to the Asset model. Server/celery/test don't set the env var and keep the full 12-app list. * `docker/Dockerfile.viewer.j2`: set `ENV ANTHIAS_SERVICE="viewer"`. * `viewer/__init__.py`: drop the bare `try: ... except Exception: pass`. Any future import or django.setup() failure now surfaces as a real traceback instead of a confusing NameError downstream. * `celery_tasks.py`: same defensive cleanup. Celery uses the server dep group so it doesn't fail today, but the antipattern would mask the same class of regression — fix it before it bites. Verified inside docker on x86: rebuilt all three images (server/celery/viewer); each module imports cleanly. Server still loads the full INSTALLED_APPS (12 apps incl. channels, DRF, dbbackup) and django_stubs_ext.monkeypatch() still runs. Viewer reaches the loop entry point (Qt browser launch then fails on the headless build host, expected and unrelated). Co-Authored-By: Claude Opus 4.7 (1M context) * refactor(settings): split INSTALLED_APPS into base + http-only Reverses the if/else from the previous commit so the structure matches the intent: every Django consumer (server, celery, viewer, test) gets the same minimal base — ORM, contenttypes, auth — and HTTP-serving services additively opt into the web stack on top of that. Why this is better than the if/else: * Single source of truth for "what does any Django consumer need" — no risk of the two branches drifting. * Adding a future lightweight service (e.g. a one-shot migration runner) is a no-op: it gets the right base by default. * The web-only apps are listed exactly once and clearly tagged as HTTP-only, instead of being interleaved with base apps in the full-mode branch. Verified inside docker (viewer image, ANTHIAS_SERVICE=viewer): 3-app list as before. With ANTHIAS_SERVICE unset (server-equivalent path): 12-app list, identical contents to pre-refactor master, just with base apps now leading. `manage.py check` reports no issues — the ordering change (channels was first; now anthias_app/contenttypes/ auth lead) is benign because Anthias drives ASGI via uvicorn, not the runserver shadow that channels' first-app position used to matter for. Co-Authored-By: Claude Opus 4.7 (1M context) * review: address Copilot feedback on PR #2762 Three findings, all valid: * Comment claimed `django_stubs_ext.monkeypatch()` was a no-op because no runtime code subscripts Django generics. That's wrong: `anthias_app/admin.py` defines `class AssetAdmin(admin.ModelAdmin [Asset])` at module level, which raises TypeError on the server without the patch. Rewrite the comment to be honest about the runtime dependency so a future contributor doesn't delete the patch thinking it's dead. * `except ImportError: pass` was too broad — it would also swallow a partially-installed django_stubs_ext (e.g. a missing internal submodule). Narrow to `ModuleNotFoundError` and only swallow when `exc.name == 'django_stubs_ext'`; re-raise otherwise so unrelated import failures surface. * The same comment claimed the viewer image doesn't ship drf-spectacular or django-dbbackup, but the viewer dep group still listed both. The gated INSTALLED_APPS no longer references their apps and viewer code never imports them, so drop them from the viewer group instead of fixing the comment to admit they were there. Re-locked uv.lock. Verified inside docker after rebuilding the viewer image: viewer loads cleanly, INSTALLED_APPS = 3, and `importlib.util.find_spec` confirms drf_spectacular / dbbackup / django_stubs_ext are all absent from the viewer venv. Co-Authored-By: Claude Opus 4.7 (1M context) --------- Co-authored-by: Claude Opus 4.7 (1M context) --- anthias_django/settings.py | 49 ++++++++++++++++++++++++++++--------- celery_tasks.py | 21 +++++++--------- docker/Dockerfile.viewer.j2 | 1 + pyproject.toml | 2 -- uv.lock | 4 --- viewer/__init__.py | 27 +++++++++----------- 6 files changed, 59 insertions(+), 45 deletions(-) diff --git a/anthias_django/settings.py b/anthias_django/settings.py index 7d0ec502..743c6842 100644 --- a/anthias_django/settings.py +++ b/anthias_django/settings.py @@ -12,12 +12,24 @@ import secrets from os import getenv from pathlib import Path -import django_stubs_ext import pytz from settings import settings as device_settings -django_stubs_ext.monkeypatch() +# django_stubs_ext.monkeypatch() makes Django generic classes +# subscriptable at runtime, and the server side of this repo relies on +# that — anthias_app/admin.py defines `class AssetAdmin(admin.ModelAdmin +# [Asset])` at import time, which raises TypeError without the patch. +# Keep the import optional so the viewer image (and any future service +# that doesn't ship django-stubs-ext) can still load this settings +# module; do not remove the patch as a no-op. +try: + import django_stubs_ext + + django_stubs_ext.monkeypatch() +except ModuleNotFoundError as exc: + if exc.name != 'django_stubs_ext': + raise # Build paths inside the project like this: BASE_DIR / 'subdir'. BASE_DIR = Path(__file__).resolve().parent.parent @@ -66,21 +78,34 @@ ALLOWED_HOSTS = [ # Application definition +# Apps every Django consumer needs: ORM access to the Asset model, +# plus the contenttypes + auth tables those models implicitly depend +# on. Loaded by every service that calls django.setup() — server, +# celery, viewer, test. INSTALLED_APPS = [ - 'channels', 'anthias_app.apps.AnthiasAppConfig', - 'drf_spectacular', - 'rest_framework', - 'api.apps.ApiConfig', - 'django.contrib.admin', - 'django.contrib.auth', 'django.contrib.contenttypes', - 'django.contrib.sessions', - 'django.contrib.messages', - 'django.contrib.staticfiles', - 'dbbackup', + 'django.contrib.auth', ] +# Apps only the HTTP-serving services need (REST API, OpenAPI schema, +# Channels for WebSockets, the admin UI, sessions/messages, static +# files, DB backups). The viewer never serves HTTP, so it skips these +# at django.setup() time and the viewer image doesn't have to ship +# the packages they live in. Server/celery/test images are unaffected. +if getenv('ANTHIAS_SERVICE') != 'viewer': + INSTALLED_APPS += [ + 'channels', + 'drf_spectacular', + 'rest_framework', + 'api.apps.ApiConfig', + 'django.contrib.admin', + 'django.contrib.sessions', + 'django.contrib.messages', + 'django.contrib.staticfiles', + 'dbbackup', + ] + MIDDLEWARE = [ 'django.middleware.security.SecurityMiddleware', 'whitenoise.middleware.WhiteNoiseMiddleware', diff --git a/celery_tasks.py b/celery_tasks.py index 99574194..2d3f8eaa 100755 --- a/celery_tasks.py +++ b/celery_tasks.py @@ -8,20 +8,17 @@ import sh from celery import Celery from tenacity import Retrying, stop_after_attempt, wait_fixed -try: - django.setup() +django.setup() - # Place imports that uses Django in this block. +# Place imports that uses Django in this block. - from lib import diagnostics - from lib.utils import ( - connect_to_redis, - is_balena_app, - reboot_via_balena_supervisor, - shutdown_via_balena_supervisor, - ) -except Exception: - pass +from lib import diagnostics # noqa: E402 +from lib.utils import ( # noqa: E402 + connect_to_redis, + is_balena_app, + reboot_via_balena_supervisor, + shutdown_via_balena_supervisor, +) __author__ = 'Screenly, Inc' diff --git a/docker/Dockerfile.viewer.j2 b/docker/Dockerfile.viewer.j2 index 18998ec0..a7e26174 100644 --- a/docker/Dockerfile.viewer.j2 +++ b/docker/Dockerfile.viewer.j2 @@ -61,6 +61,7 @@ ENV GIT_SHORT_HASH={{ git_short_hash }} ENV GIT_BRANCH={{ git_branch }} ENV DEVICE_TYPE={{ board }} ENV DJANGO_SETTINGS_MODULE="anthias_django.settings" +ENV ANTHIAS_SERVICE="viewer" RUN useradd -g video viewer diff --git a/pyproject.toml b/pyproject.toml index 7587c4ab..fe597b33 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -79,8 +79,6 @@ viewer = [ "cryptography==46.0.7", "Cython==3.2.4", "Django==4.2.30", - "django-dbbackup==4.2.1", - "drf-spectacular==0.29.0", "future==1.0.0", "idna==3.11", "Jinja2==3.1.6", diff --git a/uv.lock b/uv.lock index 7436c2a1..f663af91 100644 --- a/uv.lock +++ b/uv.lock @@ -270,8 +270,6 @@ viewer = [ { name = "cryptography" }, { name = "cython" }, { name = "django" }, - { name = "django-dbbackup" }, - { name = "drf-spectacular" }, { name = "future" }, { name = "idna" }, { name = "jinja2" }, @@ -476,8 +474,6 @@ viewer = [ { name = "cryptography", specifier = "==46.0.7" }, { name = "cython", specifier = "==3.2.4" }, { name = "django", specifier = "==4.2.30" }, - { name = "django-dbbackup", specifier = "==4.2.1" }, - { name = "drf-spectacular", specifier = "==0.29.0" }, { name = "future", specifier = "==1.0.0" }, { name = "idna", specifier = "==3.11" }, { name = "jinja2", specifier = "==3.1.6" }, diff --git a/viewer/__init__.py b/viewer/__init__.py index dda5152d..c9c211cc 100644 --- a/viewer/__init__.py +++ b/viewer/__init__.py @@ -32,23 +32,20 @@ from viewer.utils import ( watchdog, ) -try: - django.setup() +django.setup() - # Place imports that uses Django in this block. +# Place imports that uses Django in this block. - from lib.utils import ( - connect_to_redis, - get_balena_device_info, - get_node_ip, - is_balena_app, - string_to_bool, - url_fails, - ) - from viewer.scheduling import Scheduler - from viewer.zmq import ZMQ_HOST_PUB_URL, ZmqSubscriber -except Exception: - pass +from lib.utils import ( # noqa: E402 + connect_to_redis, + get_balena_device_info, + get_node_ip, + is_balena_app, + string_to_bool, + url_fails, +) +from viewer.scheduling import Scheduler # noqa: E402 +from viewer.zmq import ZMQ_HOST_PUB_URL, ZmqSubscriber # noqa: E402 __author__ = 'Screenly, Inc'