Files
Anthias/docs/developer-documentation.md
Viktor Petersson 5e00c8ba25 refactor(docker): drop celery image, restore base apt layer dedup (#2776)
* refactor(docker): drop celery image, restore base apt layer dedup

- Delete Dockerfile.celery.j2; compose now runs celery on the
  anthias-server image with a `command:` override.
- Make viewer extend Dockerfile.base.j2 (mirroring test); drop 17
  packages duplicated between viewer and base_apt_dependencies, plus
  4 within-list duplicates.
- Move `# syntax=docker/dockerfile:1.4` to line 1 of every rendered
  Dockerfile. It previously lived in uv-builder.j2 line 1 and got
  bumped mid-file for server by the bun-builder prelude, silently
  disabling the 1.4 frontend and breaking cache-key parity with
  viewer — the actual blocker for layer dedup.
- Collapse CI matrix from (board × service) to (board) so all
  services for a board build on the same runner with the same
  buildkit cache, producing byte-identical apt layer digests at the
  registry.
- Add ENV DJANGO_SETTINGS_MODULE to the server image so the merged
  image runs both server and celery CMDs.
- Update all five compose templates (prod, balena prod, balena dev,
  dev, test) to redirect anthias-celery at the server image with a
  command: override. dev compose pins an explicit `image:` tag so
  both services share the locally-built SHA.
- Remove old anthias-celery / srly-ose-celery containers in
  upgrade_containers.sh so the recreated container can take the name.

Verified end-to-end on x86: server and viewer apt layers share a
single digest; SHARED SIZE jumps from 132 MB to 1.216 GB; merged
image runs both workloads in compose (celery task round-trips
through Redis to SUCCESS).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf(docker): cache buildkit layers in GHCR registry across CI runs

Add a --cache-backend / $BUILDX_CACHE_BACKEND option to
tools.image_builder with two modes:

- `local` (default): writes to /tmp/.buildx-cache/<board>/.
  Unchanged from before; right for local dev.
- `registry`: pushes BuildKit cache to
  ghcr.io/screenly/anthias-<service>:buildcache-<board>. Reuses the
  GHCR login already done by docker-build.yaml, no extra tokens or
  third-party actions needed.

Wire CI to use registry mode on push events (master) so subsequent
runs of the same board pull cached layers — the ~825 MB extracted
apt install per service goes from ~3 min cold to a few seconds
warm. workflow_dispatch on a non-master branch falls back to local
mode (effectively no-cache) so manual runs can't pollute the master
cache.

Drop the old actions/cache@v5 step that mirrored
/tmp/.buildx-cache/<board> through actions/cache — registry cache
is per-step rather than one big tarball, so it survives the GitHub
Actions cache 10 GB-per-repo eviction better.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(image-builder): move local cache out of /tmp to user XDG cache dir

SonarCloud python:S5443 flagged the previous /tmp/.buildx-cache/
default as a security hotspot — `/tmp` is world-writable, so on a
multi-user host another account could in principle tamper with the
buildkit cache. Switch to $XDG_CACHE_HOME/anthias-buildx/<board>/
(default ~/.cache/anthias-buildx/), which is per-user by default
and follows XDG Base Directory convention.

CI is unaffected: docker-build.yaml uses --cache-backend=registry
on push events, which pushes cache to GHCR and never touches the
local path. Local dev users with stale state in
/tmp/.buildx-cache/<board>/ can rm it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(docker): correct cache-backend comments to match real behavior

Two doc fixes per Copilot review on #2776:

- tools/image_builder/__main__.py: the cache-backend rationale
  block still referenced /tmp/.buildx-cache/<board>; update to
  $XDG_CACHE_HOME/anthias-buildx/<board> so it matches the
  implementation moved in 529a50e0.
- .github/workflows/docker-build.yaml: the env comment claimed
  pull-request builds read from the registry cache, but this
  workflow has no pull_request trigger — non-push runs are
  workflow_dispatch, which both falls through to local cache and
  skips `docker login ghcr.io`, so it has no GHCR auth at all.
  Rewrite the comment around the push / workflow_dispatch split
  the code actually implements.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(docker): address Copilot review on registry cache + test compose

- tools/image_builder/__main__.py: comment in the registry-cache
  branch said the cache namespace was "picked from the build's tag
  list", but the implementation hardcodes
  ghcr.io/screenly/anthias-{service}. Rewrite the comment to
  describe what the code actually does and call out the hardcode
  so a future namespaces refactor doesn't silently break cache.
- docker-compose.test.yml: anthias-celery had its own `build:`
  block pointing at Dockerfile.test, claiming "reuses the test
  image" — but compose builds two separate images per service
  even with identical context, defeating the dedup intent. Mirror
  the docker-compose.dev.yml pattern: pin anthias-test to an
  explicit `image: anthias-test:dev` tag and have anthias-celery
  reference the same tag with no `build:`. Also bind-mount the
  source into celery so it picks up code changes (matches
  anthias-test's existing volume).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(image-builder): read-only registry cache without --push

Per Copilot review: --cache-backend=registry previously tried to
push cache to ghcr.io/... regardless of --push, so a local invocation
without GHCR auth would fail mid-build with a confusing registry
error. Split the behavior:

- Reads (cache_from) are always set when registry mode is active —
  the anthias-* GHCR packages are public, so warm-starting off CI's
  cache without auth works and helps local dev.
- Writes (cache_to) only happen when --push is also set, since
  that's when the workflow has authenticated to GHCR. Without
  --push, log a yellow warning and skip cache_to.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(docker): set DJANGO_SETTINGS_MODULE in test image for celery worker

Per Copilot review on #2776 (suppressed-due-to-low-confidence note,
but the bug is real): docker-compose.test.yml runs the celery
worker from anthias-test:dev. celery_tasks.py calls django.setup()
at module import time, which needs DJANGO_SETTINGS_MODULE in the
environment. The pre-refactor Dockerfile.celery.j2 set it
explicitly; this PR moved that ENV to Dockerfile.server.j2 only,
so the production celery (running on the server image) is fine but
the test celery would have crashed with ImproperlyConfigured.

Set the same ENV in Dockerfile.test.j2. Server and test images
both ship a usable Django environment for any process that imports
anthias_django.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 15:21:43 +01:00

9.8 KiB
Raw Blame History

Developer documentation

Understanding the components that make up Anthias

Here is a high-level overview of the different components that make Anthias:

Anthias Diagram Overview

These components and their dependencies are mostly installed and handled with Ansible and Docker.

  • The web app component (anthias-server) is the single HTTP entrypoint, served by uvicorn (ASGI). It runs the Django front-end + REST API, serves static assets via WhiteNoise, streams uploaded media at /anthias_assets/, and exposes the WebSocket endpoint at /ws via Django Channels. Always plain HTTP — TLS is opt-in via the anthias-caddy sidecar that bin/enable_ssl.sh installs (Caddy local CA by default, or Let's Encrypt with --domain).
  • The viewer (anthias-viewer) is what drives the screen (e.g., shows web page, image or video). It fetches media from anthias-server over HTTP.
  • The Celery (anthias-celery) component is for asynchronously queueing and executing tasks outside the HTTP request-response cycle (e.g., yt-dlp downloads, asset cleanup). It pushes asset-update events to connected WebSocket clients via the Redis-backed Channels layer.
  • Redis (redis) is used as the Celery broker/result backend and as the Channels channel layer.
  • The database component uses SQLite for storing the assets information.

Dockerized development environment

To simplify development of the server module of Anthias, we've created a Docker container. This is intended to run on your local machine with the Anthias repository mounted as a volume.

Important

  • Make sure that you have installed Docker on your machine before proceeding.
  • Anthias is using Docker's buildx for the image builds. This is used both for cross compilation as well as for local caching. You might need to run docker buildx create --use first.

Assuming you're in the source code repository, simply run:

$ ./bin/start_development_server.sh

# The console output was truncated for brevity.
# ...

[+] Running 4/4
 ✔ Network anthias_default                Created                            0.1s
 ✔ Container anthias-redis-1              Started                            0.2s
 ✔ Container anthias-anthias-server-1     Started                            0.3s
 ✔ Container anthias-anthias-celery-1     Started                            0.4s

Note

Running the script will install Python 3.11, pyenv, and uv inside a Docker container on your machine. This is to ensure that the development environment is consistent across different machines.

The script currently supports Debian-based systems and macOS.

unning the command above will start the development server and you should be able to access the web interface at http://localhost:8000.

To stop the development server, run the following:

docker compose -f docker-compose.dev.yml down

Building containers on the Raspberry Pi

Note

Make sure that you have Docker installed on the device before proceeding.

$ ENVIRONMENT=production \
    ./bin/generate_dev_mode_dockerfiles.sh
$ MODE=build \
    ./bin/upgrade_containers.sh

Django admin site

Create a superuser account:

$ export COMPOSE_FILE=docker-compose.dev.yml
$ docker compose exec anthias-server \
    python manage.py createsuperuser
# You will be prompted to enter a username, an email address, and a password.

Once you have created a superuser account, you can open the Django admin site at http://localhost:8000/admin/ (with a trailing slash) and login with the credentials you just created.

Testing

Running the unit tests

Build and start the containers.

$ uv run python -m tools.image_builder \
  --dockerfiles-only \
  --disable-cache-mounts \
  --service redis \
  --service test
$ docker compose \
    -f docker-compose.test.yml up -d --build

Run the unit tests.

$ docker compose \
    -f docker-compose.test.yml \
    exec anthias-test bash ./bin/prepare_test_environment.sh -s

# Integration and non-integration tests should be run separately as the
# former doesn't run as expected when run together with the latter.

$ docker compose \
    -f docker-compose.test.yml \
    exec anthias-test ./manage.py test --exclude-tag=integration

$ docker compose \
    -f docker-compose.test.yml \
    exec anthias-test ./manage.py test --tag=integration

The QA checklist

We've also provided a checklist that can serve as a guide for testing Anthias manually.

Generating CSS and JS files

To get started, you need to start the development server first. See this section for details.

Starting the bundler in development mode

To start Bun in development (watch) mode, run the following command:

$ docker compose -f docker-compose.dev.yml exec anthias-server \
    bun run dev

This runs bun build --watch for JS/TS and sass --watch for SCSS in parallel. Making changes to the TypeScript, TSX, or SCSS files will automatically trigger a recompilation, generating the corresponding bundle and CSS files.

Formatting and linting TypeScript code

To run the linting and formatting checks on the TypeScript code, run the following command:

$ docker compose -f docker-compose.dev.yml exec anthias-server \
    bun run lint:check
$ docker compose -f docker-compose.dev.yml exec anthias-server \
    bun run format:check

If you want to fix the linting errors and formatting issues, run the following command:

$ docker compose -f docker-compose.dev.yml exec anthias-server \
    bun run lint:fix
$ docker compose -f docker-compose.dev.yml exec anthias-server \
    bun run format:fix

Closing the transpiler

Just press Ctrl-C to close the bundler watch.

Linting Python code locally

The project uses ruff for linting the Python codebase. While the linter is being run on the CI/CD pipeline, you can also run it locally. There are several ways to do this.

Run the linter using act

act lets you run GitHub Actions locally. This is useful for testing the CI/CD pipeline locally. Installation instructions can be found here.

After installing and setting up act, run the following command:

$ act -W .github/workflows/python-lint.yaml

The command above will run the linter on the all the Python files in the repository. If you want to run the linter on a specific file, you can try the commands in the next section.

Running the linter using uv

You have to install uv first. You can find the installation instructions here.

After installing uv, run the following commands:

# Install the dependencies
$ uv venv
$ uv pip install --group dev-host
$ uv run ruff check .

To run the linter on a specific file, run the following command:

$ uv run ruff check /path/to/file.py

Managing releases

Creating a new release

Check what the latest release is:

$ git pull
$ git tag

# Running the `git tag` command should output something like this:
# 0.16
# ...
# v0.18.6

Create a new release:

$ git tag -a v0.18.7 -m "Test new automated disk images"

Push release:

$ git push origin v0.18.7

Delete a broken release

$ git tag -d v0.18.5                         [±master ✓]
Deleted tag 'v0.18.5' (was 9b86c39)

$ git push --delete origin v0.18.5           [±master ✓]

Directories and files explained

In this section, we'll explain the different directories and files that are present in a Raspberry Pi with Anthias installed.

/home/${USER}/anthias/

  • All of the files and folders from the Github repo should be cloned into this directory.
  • On installations created before the rename, this directory is /home/${USER}/screenly/ — the installer migrates it to anthias/ on upgrade and leaves a back-compat symlink at the old path for one release.

/home/${USER}/.anthias/

  • default_assets.yml — configuration file which contains the default assets that get added to the assets list if enabled
  • anthias.conf — configuration file for web interface settings
  • anthias.db database file containing current assets information.
  • On pre-rename installations this directory is ~/.screenly/ containing screenly.conf / screenly.db; the installer migrates them.

/etc/systemd/system/

  • anthias-host-agent.service — starts the Python script host_agent.py, which subscribes from the Redis component and performs a system call to shutdown or reboot the device when the message is received.

/etc/sudoers.d/anthias_overrides

  • sudoers configuration file that allows pi user to execute certain sudo commands without being a superuser (i.e., root)

/usr/share/plymouth/themes/anthias

  • anthias.plymouth — Plymouth config file (sets module name, ImageDir and ScriptFile dir)
  • anthias.script plymouth script file that loads and scales the splash screen image during the boot process
  • splashscreen.png — the spash screen image that is displayed during the boot process

Debugging the Anthias WebView

export QT_LOGGING_DEBUG=1
export QT_LOGGING_RULES="*.debug=true"
export QT_QPA_EGLFS_DEBUG=1

The Anthias WebView is a custom-built web browser based on the Qt toolkit framework. The browser is assembled with a Dockerfile and built by a webview/build_qt#.sh script.

For further info on these files and more, visit the following link: https://github.com/Screenly/Anthias/tree/master/webview