Files
Anthias/docs/developer-documentation.md
Viktor Petersson c7ec6ea771 chore(build): replace webpack, npm, and jest with bun (#2746)
* chore(deps): manage Python deps via uv dependency-groups

Replaces the six service-scoped requirements*.txt files with
PEP 735 dependency-groups in pyproject.toml and rebuilds every
Docker image as a two-stage build: a uv-builder stage (using the
official ghcr.io/astral-sh/uv image, with a pip fallback for
armv6) produces /venv via `uv sync --group <svc>`, which the
runtime stage copies in. uv.lock becomes authoritative for all
services. requirements/requirements.host.txt is kept as a
committed, auto-generated artifact (`uv export --group host`) so
bin/install.sh and the Ansible role keep working; a python-lint
CI step enforces it stays in sync.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(deps): bump Django, cryptography, pyOpenSSL, and 5 others

- Django 4.2.29 → 4.2.30 (latest 4.2 LTS)
- cryptography 3.3.2 → 46.0.7 (capped by pyOpenSSL 26's `cryptography<47`;
  cryptography 47 is incompatible with the latest pyOpenSSL)
- pyOpenSSL 19.1.0 → 26.0.0 (required by newer cryptography ABI —
  pyOpenSSL 19 crashed at import against cryptography ≥ ~3.4)
- requests 2.32.5 → 2.33.1 (aligned across every group, including
  docker-image-builder and local)
- pyasn1 0.6.2 → 0.6.3
- redis 7.1.0 → 7.4.0
- Cython 3.2.3 → 3.2.4
- sh 1.8 → 2.2.2 (major bump; usages in celery_tasks.py, bin/wait.py,
  lib/utils.py stick to the stable `sh.<cmd>` + `sh.ErrorReturnCode_N`
  API — verified still works)
- python-vlc 3.0.20123 → 3.0.21203

`mako` and `flatted` were requested but skipped: `mako` was already
removed from the project (9535745e), and `flatted` is an npm dep in
`package-lock.json`, not a Python dep.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(deps): bump wheel from 0.38.1 to 0.46.2

Closes Dependabot PR #2651.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(deps): manage Python deps via uv dependency-groups

Replaces the six service-scoped requirements*.txt files with
PEP 735 dependency-groups in pyproject.toml and rebuilds every
Docker image as a two-stage build: a uv-builder stage (using the
official ghcr.io/astral-sh/uv image, with a pip fallback for
armv6) produces /venv via `uv sync --group <svc>`, which the
runtime stage copies in. uv.lock becomes authoritative for all
services. requirements/requirements.host.txt is kept as a
committed, auto-generated artifact (`uv export --group host`) so
bin/install.sh and the Ansible role keep working; a python-lint
CI step enforces it stays in sync.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(deps): bump Django, cryptography, pyOpenSSL, and 5 others

- Django 4.2.29 → 4.2.30 (latest 4.2 LTS)
- cryptography 3.3.2 → 46.0.7 (capped by pyOpenSSL 26's `cryptography<47`;
  cryptography 47 is incompatible with the latest pyOpenSSL)
- pyOpenSSL 19.1.0 → 26.0.0 (required by newer cryptography ABI —
  pyOpenSSL 19 crashed at import against cryptography ≥ ~3.4)
- requests 2.32.5 → 2.33.1 (aligned across every group, including
  docker-image-builder and local)
- pyasn1 0.6.2 → 0.6.3
- redis 7.1.0 → 7.4.0
- Cython 3.2.3 → 3.2.4
- sh 1.8 → 2.2.2 (major bump; usages in celery_tasks.py, bin/wait.py,
  lib/utils.py stick to the stable `sh.<cmd>` + `sh.ErrorReturnCode_N`
  API — verified still works)
- python-vlc 3.0.20123 → 3.0.21203

`mako` and `flatted` were requested but skipped: `mako` was already
removed from the project (9535745e), and `flatted` is an npm dep in
`package-lock.json`, not a Python dep.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(deps): bump wheel from 0.38.1 to 0.46.2

Closes Dependabot PR #2651.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: adapt sh 2.x API changes in wait.py and viewer

Two real breakages uncovered by auditing every `sh.*` call site
against the sh 1.x → 2.x API:

- bin/wait.py: `sh.grep(sh.route(), 'default')` no longer pipes
  in sh 2.x — the inner command stringifies to its stdout and
  becomes a literal argument to grep, producing
  `grep '<route_output>' default` and an ErrorReturnCode_2. Use
  the idiomatic `sh.grep('default', _in=sh.route())` instead.

- viewer/__init__.py: `browser.process.alive` is gone in sh 2.x
  (`OProc` no longer exposes it). Use `browser.process.is_alive()[0]`,
  which returns the `(alive_bool, exit_code)` tuple.

Plus two review nits:
- Add trailing newline to docs/migrating-assets-to-screenly.md
- Use `diff -u` in the requirements.host.txt CI drift check so
  failures print a readable unified diff.

Verified against sh==2.2.2 inside the rebuilt server image:
- `sh.grep('default', _in=sh.echo('…'))` pipes correctly
- `cmd.process.is_alive()` → `(True, None)` while running,
  `(False, 0)` after wait()
- `cmd.process.stdout.decode('utf-8')` still works on `_bg=True`
  processes

83/83 unit tests + 12/12 integration tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(docker): serialize apt cache access with sharing=locked

The multi-stage uv-builder + runtime layout means two RUN steps can
race on BuildKit's shared `/var/cache/apt` cache mount. apt requires
an exclusive lock on /var/cache/apt/archives, so a concurrent
apt-get in the sibling stage causes the build to fail with
`E: Could not get lock /var/cache/apt/archives/lock`.

BuildKit's default cache mount sharing mode is `shared` (unrestricted
concurrent access). Switching to `sharing=locked` makes BuildKit
serialize access across stages, matching apt's locking model.

Discovered while cross-compiling `pi4-64` under QEMU, where the
slower emulated apt-get in stage 1 overlapped with the host-speed
apt-get in stage 2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: fix ansible-lint and sbom workflows

**ansible-lint** (broken since 2026-04-08, #2732):
- `ansible-community/ansible-lint-action@main` repo is gone (404),
  so every run failed with "Unable to resolve action".
- Rewrite the workflow to use setup-uv + `uv run ansible-lint` from
  a new `ansible-lint==26.4.0` entry in the `dev-host` dependency
  group — matches the uv-based pattern already used by
  `python-lint.yaml`.
- Add `.ansible-lint` config with a skip list covering 19
  pre-existing violations in `ansible/` roles
  (`var-naming[no-role-prefix]`, `risky-shell-pipe`, `no-free-form`)
  so the workflow can go green today; follow-up PRs should drive
  the skip list down.
- Extend the path triggers to fire on config, workflow, and lock
  changes — not just `ansible/**`.

**sbom** (broken since 2026-04-02):
- The `sbomify/github-action` renamed `SBOM_FILE` to `LOCK_FILE` for
  lockfile inputs. Every run has been failing with "`uv.lock` is a
  lock file, not an SBOM. Please use LOCK_FILE instead of SBOM_FILE."
- Rename both `SBOM_FILE` envs (`package-lock.json` and `uv.lock`)
  to `LOCK_FILE`.

Verified locally: `uv run ansible-lint ansible/` passes (0
failures, 0 warnings).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(build): replace webpack, npm, and jest with bun

Collapses the JS toolchain to a single tool. Bun handles installs
(replacing npm), bundling via `bun build` + `sass` CLI (replacing
webpack + ts-loader + babel + mini-css-extract-plugin), and testing
via `bun test` (replacing jest + ts-jest + jest-fixed-jsdom). Dev/test
Dockerfiles pull the bun binary from the official `oven/bun` image via
`COPY --from=`; production uses `oven/bun` as a builder stage.

Removes 18 devDependencies and 5 config files; adds only `bunfig.toml`
and `@happy-dom/global-registrator`.

Drive-by fix: `FormData` was imported as a value from `@/types` in
two files but is a type-only interface shadowing the browser global.
Webpack+ts-loader silently erased it; Bun's bundler surfaced the bug.
Converted to `import type`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(docker): symlink bunx to bun in dev and test images

`bunx` is a symlink to `bun` in the official `oven/bun` image, so the
single-file `COPY --from=oven/bun:...-slim /usr/local/bin/bun` missed it.
Result: `bun run dev:css` / `bun run build:css` failed with
`bunx: command not found` inside dev and test containers.

Recreate the symlink after the copy. Production is unaffected because
its builder stage uses `FROM oven/bun` (bunx already present).

Caught by full end-to-end build verification.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: SHA-pin all external GitHub Actions

Addresses SonarCloud rule githubactions:S7637 ("Use full commit SHA
hash for this dependency") and brings the repo in line with the
hardened CI guidance from OpenSSF, CISA, and GitHub itself: tag refs
like @v7 or @master are mutable and can be retargeted by the action
owner or via compromise. Pinning to a full commit SHA removes that
supply-chain risk.

Every `uses:` reference to an external action across all 13 workflow
files is now pinned by SHA, with the original tag preserved as an
inline comment so the intent remains readable:

    uses: actions/checkout@de0fac2e45 # v6

Dependabot's github-actions ecosystem (already configured in
.github/dependabot.yml) recognises this `<SHA> # <tag>` format and
will update both the SHA and the comment together on future version
bumps, so we don't lose automated update coverage.

Scope: 21 distinct external actions × 73 total use sites across
ansible-lint, build-balena-disk-image, build-webview, codeql-analysis,
deploy-website, docker-build, generate-openapi-schema, javascript-lint,
lint-workflows, python-lint, sbom, and test-runner. Local workflow
references (./.github/workflows/...) left untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs,chore: address review feedback on bun migration

- Update CLAUDE.md and docs/developer-documentation.md to replace
  npm/webpack/jest references with bun equivalents. The old webpack
  ProvidePlugin bullet was superseded by tsconfig's react-jsx runtime;
  restate that.
- Add comments in setupTests.ts explaining (1) why Bun's native fetch
  is stashed and restored around happy-dom's GlobalRegistrator (so MSW
  can intercept) and (2) why testing-library is imported dynamically
  after registration (so `screen` binds to a live document.body).
- Narrow the production builder SCSS COPY back to `*.scss` and drop
  the unused `bunfig.toml` copy (it's only consumed by `bun test`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(dev): fail-fast when a watcher crashes in `bun run dev`

`wait` without arguments returns the last-exiting job's status, so a
crashing JS or CSS watcher could leave the script reporting success.
Track each watcher's PID, use `wait -n` to exit on the first failure,
and kill the survivor via a trap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 06:53:56 +01:00

9.7 KiB
Raw Blame History

Developer documentation

Understanding the components that make up Anthias

Here is a high-level overview of the different components that make Anthias:

Anthias Diagram Overview

These components and their dependencies are mostly installed and handled with Ansible and Docker.

  • The NGINX component (anthias-nginx) forwards requests to the backend and serves static files. It also acts as a reverse proxy.
  • The viewer (anthias-viewer) is what drives the screen (e.g., shows web page, image or video).
  • The web app component (anthias-server) — which consists of the front-end and back-end code is what the user interacts with via browser.
  • The Celery (anthias-celery) component is for aynschronouslt queueing and executing tasks outside the HTTP request-response cycle (e.g., doing assets cleanup).
  • The WebSocket (anthias-websocket) component is used for forwarding requests from NGINX to the backend.
  • Redis (redis) is used as a database, cache and message broker.
  • The database component uses SQLite for storing the assets information.

Dockerized development environment

To simplify development of the server module of Anthias, we've created a Docker container. This is intended to run on your local machine with the Anthias repository mounted as a volume.

Important

  • Make sure that you have installed Docker on your machine before proceeding.
  • Anthias is using Docker's buildx for the image builds. This is used both for cross compilation as well as for local caching. You might need to run docker buildx create --use first.

Assuming you're in the source code repository, simply run:

$ ./bin/start_development_server.sh

# The console output was truncated for brevity.
# ...

[+] Running 6/6
 ✔ Network anthias_default                Created                            0.1s
 ✔ Container anthias-redis-1              Started                            0.2s
 ✔ Container anthias-anthias-server-1     Started                            0.2s
 ✔ Container anthias-anthias-celery-1     Started                            0.3s
 ✔ Container anthias-anthias-websocket-1  Started                            0.4s
 ✔ Container anthias-anthias-nginx-1      Started                            0.5s

Note

Running the script will install Python 3.11, pyenv, and uv inside a Docker container on your machine. This is to ensure that the development environment is consistent across different machines.

The script currently supports Debian-based systems and macOS.

unning the command above will start the development server and you should be able to access the web interface at http://localhost:8000.

To stop the development server, run the following:

docker compose -f docker-compose.dev.yml down

Building containers on the Raspberry Pi

Note

Make sure that you have Docker installed on the device before proceeding.

$ ENVIRONMENT=production \
    ./bin/generate_dev_mode_dockerfiles.sh
$ MODE=build \
    ./bin/upgrade_containers.sh

Django admin site

Create a superuser account:

$ export COMPOSE_FILE=docker-compose.dev.yml
$ docker compose exec anthias-server \
    python manage.py createsuperuser
# You will be prompted to enter a username, an email address, and a password.

Once you have created a superuser account, you can open the Django admin site at http://localhost:8000/admin/ (with a trailing slash) and login with the credentials you just created.

Testing

Running the unit tests

Build and start the containers.

$ uv run python -m tools.image_builder \
  --dockerfiles-only \
  --disable-cache-mounts \
  --service celery \
  --service redis \
  --service test
$ docker compose \
    -f docker-compose.test.yml up -d --build

Run the unit tests.

$ docker compose \
    -f docker-compose.test.yml \
    exec anthias-test bash ./bin/prepare_test_environment.sh -s

# Integration and non-integration tests should be run separately as the
# former doesn't run as expected when run together with the latter.

$ docker compose \
    -f docker-compose.test.yml \
    exec anthias-test ./manage.py test --exclude-tag=integration

$ docker compose \
    -f docker-compose.test.yml \
    exec anthias-test ./manage.py test --tag=integration

The QA checklist

We've also provided a checklist that can serve as a guide for testing Anthias manually.

Generating CSS and JS files

To get started, you need to start the development server first. See this section for details.

Starting the bundler in development mode

To start Bun in development (watch) mode, run the following command:

$ docker compose -f docker-compose.dev.yml exec anthias-server \
    bun run dev

This runs bun build --watch for JS/TS and sass --watch for SCSS in parallel. Making changes to the TypeScript, TSX, or SCSS files will automatically trigger a recompilation, generating the corresponding bundle and CSS files.

Formatting and linting TypeScript code

To run the linting and formatting checks on the TypeScript code, run the following command:

$ docker compose -f docker-compose.dev.yml exec anthias-server \
    bun run lint:check
$ docker compose -f docker-compose.dev.yml exec anthias-server \
    bun run format:check

If you want to fix the linting errors and formatting issues, run the following command:

$ docker compose -f docker-compose.dev.yml exec anthias-server \
    bun run lint:fix
$ docker compose -f docker-compose.dev.yml exec anthias-server \
    bun run format:fix

Closing the transpiler

Just press Ctrl-C to close the bundler watch.

Linting Python code locally

The project uses ruff for linting the Python codebase. While the linter is being run on the CI/CD pipeline, you can also run it locally. There are several ways to do this.

Run the linter using act

act lets you run GitHub Actions locally. This is useful for testing the CI/CD pipeline locally. Installation instructions can be found here.

After installing and setting up act, run the following command:

$ act -W .github/workflows/python-lint.yaml

The command above will run the linter on the all the Python files in the repository. If you want to run the linter on a specific file, you can try the commands in the next section.

Running the linter using uv

You have to install uv first. You can find the installation instructions here.

After installing uv, run the following commands:

# Install the dependencies
$ uv venv
$ uv pip install --group dev-host
$ uv run ruff check .

To run the linter on a specific file, run the following command:

$ uv run ruff check /path/to/file.py

Managing releases

Creating a new release

Check what the latest release is:

$ git pull
$ git tag

# Running the `git tag` command should output something like this:
# 0.16
# ...
# v0.18.6

Create a new release:

$ git tag -a v0.18.7 -m "Test new automated disk images"

Push release:

$ git push origin v0.18.7

Delete a broken release

$ git tag -d v0.18.5                         [±master ✓]
Deleted tag 'v0.18.5' (was 9b86c39)

$ git push --delete origin v0.18.5           [±master ✓]

Directories and files explained

In this section, we'll explain the different directories and files that are present in a Raspberry Pi with Anthias installed.

home/${USER}/screenly/

  • All of the files and folders from the Github repo should be cloned into this directory.

/home/${USER}/.screenly/

  • default_assets.yml — configuration file which contains the default assets that get added to the assets list if enabled
  • initialized — tells whether access point service (for Wi-Fi connectivity) runs or not
  • screenly.conf — configuration file for web interface settings
  • screenly.db database file containing current assets information.

/etc/systemd/system/

  • wifi-connect.service — starts the Balena wifi-connect program to dynamically set the Wi-Fi config on the device via the captive portal
  • anthias-host-agent.service — starts the Python script host_agent.py, which subscribes from the Redis component and performs a system call to shutdown or reboot the device when the message is received.

/etc/sudoers.d/screenly_overrides

  • sudoers configuration file that allows pi user to execute certain sudo commands without being a superuser (i.e., root)

/usr/share/plymouth/themes/anthias

  • anthias.plymouth — Plymouth config file (sets module name, ImageDir and ScriptFile dir)
  • anthias.script plymouth script file that loads and scales the splash screen image during the boot process
  • splashscreen.png — the spash screen image that is displayed during the boot process

Debugging the Anthias WebView

export QT_LOGGING_DEBUG=1
export QT_LOGGING_RULES="*.debug=true"
export QT_QPA_EGLFS_DEBUG=1

The Anthias WebView is a custom-built web browser based on the Qt toolkit framework. The browser is assembled with a Dockerfile and built by a webview/build_qt#.sh script.

For further info on these files and more, visit the following link: https://github.com/Screenly/Anthias/tree/master/webview