chore(deps): manage Python deps via uv dependency-groups (#2744)

* chore(deps): manage Python deps via uv dependency-groups

Replaces the six service-scoped requirements*.txt files with
PEP 735 dependency-groups in pyproject.toml and rebuilds every
Docker image as a two-stage build: a uv-builder stage (using the
official ghcr.io/astral-sh/uv image, with a pip fallback for
armv6) produces /venv via `uv sync --group <svc>`, which the
runtime stage copies in. uv.lock becomes authoritative for all
services. requirements/requirements.host.txt is kept as a
committed, auto-generated artifact (`uv export --group host`) so
bin/install.sh and the Ansible role keep working; a python-lint
CI step enforces it stays in sync.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(deps): bump Django, cryptography, pyOpenSSL, and 5 others

- Django 4.2.29 → 4.2.30 (latest 4.2 LTS)
- cryptography 3.3.2 → 46.0.7 (capped by pyOpenSSL 26's `cryptography<47`;
  cryptography 47 is incompatible with the latest pyOpenSSL)
- pyOpenSSL 19.1.0 → 26.0.0 (required by newer cryptography ABI —
  pyOpenSSL 19 crashed at import against cryptography ≥ ~3.4)
- requests 2.32.5 → 2.33.1 (aligned across every group, including
  docker-image-builder and local)
- pyasn1 0.6.2 → 0.6.3
- redis 7.1.0 → 7.4.0
- Cython 3.2.3 → 3.2.4
- sh 1.8 → 2.2.2 (major bump; usages in celery_tasks.py, bin/wait.py,
  lib/utils.py stick to the stable `sh.<cmd>` + `sh.ErrorReturnCode_N`
  API — verified still works)
- python-vlc 3.0.20123 → 3.0.21203

`mako` and `flatted` were requested but skipped: `mako` was already
removed from the project (9535745e), and `flatted` is an npm dep in
`package-lock.json`, not a Python dep.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(deps): bump wheel from 0.38.1 to 0.46.2

Closes Dependabot PR #2651.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: adapt sh 2.x API changes in wait.py and viewer

Two real breakages uncovered by auditing every `sh.*` call site
against the sh 1.x → 2.x API:

- bin/wait.py: `sh.grep(sh.route(), 'default')` no longer pipes
  in sh 2.x — the inner command stringifies to its stdout and
  becomes a literal argument to grep, producing
  `grep '<route_output>' default` and an ErrorReturnCode_2. Use
  the idiomatic `sh.grep('default', _in=sh.route())` instead.

- viewer/__init__.py: `browser.process.alive` is gone in sh 2.x
  (`OProc` no longer exposes it). Use `browser.process.is_alive()[0]`,
  which returns the `(alive_bool, exit_code)` tuple.

Plus two review nits:
- Add trailing newline to docs/migrating-assets-to-screenly.md
- Use `diff -u` in the requirements.host.txt CI drift check so
  failures print a readable unified diff.

Verified against sh==2.2.2 inside the rebuilt server image:
- `sh.grep('default', _in=sh.echo('…'))` pipes correctly
- `cmd.process.is_alive()` → `(True, None)` while running,
  `(False, 0)` after wait()
- `cmd.process.stdout.decode('utf-8')` still works on `_bg=True`
  processes

83/83 unit tests + 12/12 integration tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(docker): serialize apt cache access with sharing=locked

The multi-stage uv-builder + runtime layout means two RUN steps can
race on BuildKit's shared `/var/cache/apt` cache mount. apt requires
an exclusive lock on /var/cache/apt/archives, so a concurrent
apt-get in the sibling stage causes the build to fail with
`E: Could not get lock /var/cache/apt/archives/lock`.

BuildKit's default cache mount sharing mode is `shared` (unrestricted
concurrent access). Switching to `sharing=locked` makes BuildKit
serialize access across stages, matching apt's locking model.

Discovered while cross-compiling `pi4-64` under QEMU, where the
slower emulated apt-get in stage 1 overlapped with the host-speed
apt-get in stage 2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: fix ansible-lint and sbom workflows

**ansible-lint** (broken since 2026-04-08, #2732):
- `ansible-community/ansible-lint-action@main` repo is gone (404),
  so every run failed with "Unable to resolve action".
- Rewrite the workflow to use setup-uv + `uv run ansible-lint` from
  a new `ansible-lint==26.4.0` entry in the `dev-host` dependency
  group — matches the uv-based pattern already used by
  `python-lint.yaml`.
- Add `.ansible-lint` config with a skip list covering 19
  pre-existing violations in `ansible/` roles
  (`var-naming[no-role-prefix]`, `risky-shell-pipe`, `no-free-form`)
  so the workflow can go green today; follow-up PRs should drive
  the skip list down.
- Extend the path triggers to fire on config, workflow, and lock
  changes — not just `ansible/**`.

**sbom** (broken since 2026-04-02):
- The `sbomify/github-action` renamed `SBOM_FILE` to `LOCK_FILE` for
  lockfile inputs. Every run has been failing with "`uv.lock` is a
  lock file, not an SBOM. Please use LOCK_FILE instead of SBOM_FILE."
- Rename both `SBOM_FILE` envs (`package-lock.json` and `uv.lock`)
  to `LOCK_FILE`.

Verified locally: `uv run ansible-lint ansible/` passes (0
failures, 0 warnings).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: SHA-pin all external GitHub Actions

Addresses SonarCloud rule githubactions:S7637 ("Use full commit SHA
hash for this dependency") and brings the repo in line with the
hardened CI guidance from OpenSSF, CISA, and GitHub itself: tag refs
like @v7 or @master are mutable and can be retargeted by the action
owner or via compromise. Pinning to a full commit SHA removes that
supply-chain risk.

Every `uses:` reference to an external action across all 13 workflow
files is now pinned by SHA, with the original tag preserved as an
inline comment so the intent remains readable:

    uses: actions/checkout@de0fac2e45 # v6

Dependabot's github-actions ecosystem (already configured in
.github/dependabot.yml) recognises this `<SHA> # <tag>` format and
will update both the SHA and the comment together on future version
bumps, so we don't lose automated update coverage.

Scope: 21 distinct external actions × 73 total use sites across
ansible-lint, build-balena-disk-image, build-webview, codeql-analysis,
deploy-website, docker-build, generate-openapi-schema, javascript-lint,
lint-workflows, python-lint, sbom, and test-runner. Local workflow
references (./.github/workflows/...) left untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(viewer): use RunningCommand.is_alive() instead of OProc tuple

OProc.is_alive() returns (bool, exit_code); RunningCommand.is_alive()
wraps that and returns just the bool. The wrapper is clearer than
indexing into the tuple.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Viktor Petersson
2026-04-25 06:48:36 +01:00
committed by GitHub
parent 277b58d969
commit ee12387b06
38 changed files with 2182 additions and 277 deletions

12
.ansible-lint Normal file
View File

@@ -0,0 +1,12 @@
---
# ansible-lint configuration
# https://ansible.readthedocs.io/projects/lint/configuring/
# Rules listed below are pre-existing violations in the ansible/ roles
# that we've chosen to defer rather than fix as part of the uv
# dependency-group migration. Follow-up PRs should drive these down
# and remove entries from the skip list.
skip_list:
- var-naming[no-role-prefix]
- risky-shell-pipe
- no-free-form

View File

@@ -37,5 +37,6 @@ pip-log.txt
.sass-cache/*
node_modules
venv
.venv
website/
docker-compose.*

View File

@@ -8,21 +8,42 @@ on:
- production
paths:
- 'ansible/**'
- '.ansible-lint'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/ansible-lint.yaml'
pull_request:
branches:
- master
- production
paths:
- 'ansible/**'
- '.ansible-lint'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/ansible-lint.yaml'
jobs:
build:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Set up Python 3.11
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: '3.11'
- name: Install uv
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
with:
version: '0.9.17'
- name: Install dependencies
run: |
uv venv
uv pip install --group dev-host
- name: Run ansible-lint
uses: ansible-community/ansible-lint-action@main
with:
path: "ansible/"
run: |
uv run ansible-lint ansible/

View File

@@ -24,7 +24,7 @@ jobs:
run: |
echo ${{ inputs.docker-tag }} > docker-tag
- name: Upload docker tag file
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: docker-tag
path: docker-tag
@@ -45,7 +45,7 @@ jobs:
attestations: write
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Install zstd
run: |
@@ -53,7 +53,7 @@ jobs:
sudo apt-get install -y zstd
- name: Setup Node.js
uses: actions/setup-node@v6
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: '22.21.0'
@@ -136,7 +136,7 @@ jobs:
ls -la ./*raspberry*
- name: Upload artifacts
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: balena-images-${{ matrix.board }}
path: |
@@ -145,7 +145,7 @@ jobs:
./*raspberry*.json
- name: Attest
uses: actions/attest-build-provenance@v4
uses: actions/attest-build-provenance@a2bbfa25375fe432b6a289bc6b6cd05ecd0c4c32 # v4
with:
subject-path: '${{ github.workspace }}/*raspberry*.img.zst'
@@ -156,26 +156,26 @@ jobs:
contents: write
steps:
- name: Download docker tag
uses: actions/download-artifact@v8
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: docker-tag
path: .
- name: Download OpenAPI schema
uses: actions/download-artifact@v8
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: anthias-api-schema
path: .
- name: Download balena images
uses: actions/download-artifact@v8
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
pattern: balena-images-*
path: .
merge-multiple: true
- name: Create release
uses: ncipollo/release-action@v1.21.0
uses: ncipollo/release-action@339a81892b84b4eeb0f6e744e4574d79d0d9b8dd # v1.21.0
with:
allowUpdates: true
generateReleaseNotes: true

View File

@@ -24,18 +24,18 @@ jobs:
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v4
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4
- name: Login to Docker Hub
if: success() && github.event_name != 'pull_request'
uses: docker/login-action@v4
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
@@ -72,17 +72,17 @@ jobs:
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v4
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4
- name: Cache build layers
uses: actions/cache@v5
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
id: cache
with:
path: /tmp/.cache
@@ -123,7 +123,7 @@ jobs:
~/tmp/qt-build/qt5-*.tar.gz.sha256 \
./build
- uses: actions/upload-artifact@v7
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: webview-${{ matrix.board }}
path: ~/tmp/qt-build/
@@ -136,13 +136,13 @@ jobs:
board: ['pi5', 'x86']
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v4
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4
- name: Set environment variables
run: |
@@ -175,7 +175,7 @@ jobs:
~/tmp-${{ matrix.board }}/build/release/webview-*.tar.gz.sha256 \
./build
- uses: actions/upload-artifact@v7
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: webview-${{ matrix.board }}
path: ./build
@@ -195,12 +195,12 @@ jobs:
id-token: write
attestations: write
steps:
- uses: actions/download-artifact@v8
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: webview-${{ matrix.board }}
path: ./build
- name: Create a release
uses: softprops/action-gh-release@v3
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3
with:
prerelease: true
files: |

View File

@@ -40,11 +40,11 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
uses: github/codeql-action/init@95e58e9a2cdfd71adc6e0353d5c52f41a045d225 # v4
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
@@ -55,7 +55,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v4
uses: github/codeql-action/autobuild@95e58e9a2cdfd71adc6e0353d5c52f41a045d225 # v4
# Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
@@ -69,4 +69,4 @@ jobs:
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4
uses: github/codeql-action/analyze@95e58e9a2cdfd71adc6e0353d5c52f41a045d225 # v4

View File

@@ -28,12 +28,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Setup Pages
uses: actions/configure-pages@v6
uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6
- uses: actions/setup-python@v6
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: '3.9'
cache: 'pip'
@@ -59,7 +59,7 @@ jobs:
find _site
- name: Upload artifact
uses: actions/upload-pages-artifact@v5
uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5
with:
path: website/_site
@@ -73,4 +73,4 @@ jobs:
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v5
uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5

View File

@@ -46,15 +46,15 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: ${{ matrix.python-version }}
- name: Install uv
uses: astral-sh/setup-uv@v7
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
with:
version: '0.9.17'
@@ -64,11 +64,11 @@ jobs:
uv pip install --group docker-image-builder
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v4
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4
with:
driver-opts: |
image=moby/buildkit:latest
@@ -79,7 +79,7 @@ jobs:
docker buildx inspect --bootstrap
- name: Cache Docker layers
uses: actions/cache@v5
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
id: cache
with:
path: /tmp/.buildx-cache/${{ matrix.board }}-${{ matrix.service }}
@@ -93,7 +93,7 @@ jobs:
- name: Login to Docker Hub
if: success() && github.event_name == 'push'
uses: docker/login-action@v4
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
@@ -121,7 +121,7 @@ jobs:
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Set Docker tag
run: |
@@ -152,7 +152,7 @@ jobs:
balena-deploy/docker-compose.yml
fi
- uses: balena-io/deploy-to-balena-action@master
- uses: balena-io/deploy-to-balena-action@638e3085dfe40b8c3cef2a34fe9d0874e572de4e # master
id: build
continue-on-error: true
with:
@@ -167,7 +167,7 @@ jobs:
# Balena deploy often fails with 'ESOCKETTIMEDOUT'.
# This adds some retry logic.
- uses: balena-io/deploy-to-balena-action@master
- uses: balena-io/deploy-to-balena-action@638e3085dfe40b8c3cef2a34fe9d0874e572de4e # master
id: build-retry
if: ${{ failure() && steps.build.conclusion == 'failure' }}
with:

View File

@@ -56,15 +56,15 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Set up Python 3.11
uses: actions/setup-python@v6
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: "3.11"
- name: Install uv
uses: astral-sh/setup-uv@v7
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
with:
version: '0.9.17'
@@ -74,7 +74,7 @@ jobs:
uv pip install --group docker-image-builder
- name: Cache Docker layers
uses: actions/cache@v5
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
id: cache
with:
path: /tmp/.buildx-cache
@@ -102,7 +102,7 @@ jobs:
--file anthias-api-schema.json
- name: Upload the OpenAPI Schema
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: anthias-api-schema
path: anthias-api-schema.json

View File

@@ -24,10 +24,10 @@ jobs:
lint:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Setup Node.js
uses: actions/setup-node@v6
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: '20'
cache: 'npm'

View File

@@ -25,7 +25,7 @@ jobs:
actionlint-version: ['1.7.7']
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Download actionlint
run: |

View File

@@ -7,6 +7,7 @@ on:
paths:
- pyproject.toml
- uv.lock
- 'requirements/requirements.host.txt'
- '**/*.py'
- '.github/workflows/python-lint.yaml'
pull_request:
@@ -15,6 +16,7 @@ on:
paths:
- pyproject.toml
- uv.lock
- 'requirements/requirements.host.txt'
- '**/*.py'
- '.github/workflows/python-lint.yaml'
@@ -25,15 +27,15 @@ jobs:
matrix:
python-version: ["3.11"]
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: ${{ matrix.python-version }}
- name: Install uv
uses: astral-sh/setup-uv@v7
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
with:
version: '0.9.17'
@@ -49,3 +51,11 @@ jobs:
- name: Run Ruff formatting checks
run: |
uv run ruff format --check .
- name: Verify requirements/requirements.host.txt is in sync
run: |
uv export --group host --no-hashes --no-annotate \
--no-emit-project --no-default-groups \
-o /tmp/requirements.host.txt
diff -u <(grep -v '^#' requirements/requirements.host.txt) \
<(grep -v '^#' /tmp/requirements.host.txt)

View File

@@ -22,20 +22,20 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Build and upload SBOM
uses: sbomify/github-action@master
uses: sbomify/github-action@ac8b0d423c90447f3011ef8ab5f9ff2e854c6931 # master
env:
TOKEN: ${{ secrets.SBOMIFY_TOKEN }}
COMPONENT_ID: 'LCkvzm8qaL'
SBOM_FILE: 'package-lock.json'
LOCK_FILE: 'package-lock.json'
AUGMENT: true
UPLOAD: true
OUTPUT_FILE: js.cdx.json
- name: Attest
uses: actions/attest-build-provenance@v4
uses: actions/attest-build-provenance@a2bbfa25375fe432b6a289bc6b6cd05ecd0c4c32 # v4
with:
subject-path: '${{ github.workspace }}/*.cdx.json'
@@ -49,19 +49,19 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Build and upload SBOM
uses: sbomify/github-action@master
uses: sbomify/github-action@ac8b0d423c90447f3011ef8ab5f9ff2e854c6931 # master
env:
TOKEN: ${{ secrets.SBOMIFY_TOKEN }}
COMPONENT_ID: 'Vhc4zm8pdV'
SBOM_FILE: 'uv.lock'
LOCK_FILE: 'uv.lock'
AUGMENT: true
UPLOAD: true
OUTPUT_FILE: python.cdx.json
- name: Attest
uses: actions/attest-build-provenance@v4
uses: actions/attest-build-provenance@a2bbfa25375fe432b6a289bc6b6cd05ecd0c4c32 # v4
with:
subject-path: '${{ github.workspace }}/*.cdx.json'

View File

@@ -21,15 +21,15 @@ jobs:
COMPOSE_FILE: docker-compose.test.yml
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Set up Python ${{ inputs.python-version }}
uses: actions/setup-python@v6
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: ${{ inputs.python-version }}
- name: Install uv
uses: astral-sh/setup-uv@v7
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
with:
version: '0.9.17'
@@ -73,7 +73,7 @@ jobs:
- name: Run Python integration tests
if: inputs.test-type == 'python'
uses: nick-fields/retry@v4
uses: nick-fields/retry@ad984534de44a9489a53aefd81eb77f87c70dc60 # v4
with:
timeout_minutes: 5
max_attempts: 5
@@ -83,7 +83,7 @@ jobs:
- name: Upload coverage reports to Codecov
if: inputs.test-type == 'python'
uses: codecov/codecov-action@v6
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

View File

@@ -9,7 +9,7 @@ import sh
# wait for default route
def is_routing_up():
try:
sh.grep(sh.route(), 'default')
sh.grep('default', _in=sh.route())
return True
except sh.ErrorReturnCode_1:
return False

View File

@@ -1,12 +1,9 @@
# syntax=docker/dockerfile:1.4
# vim: ft=dockerfile
FROM {{ base_image }}:{{ base_image_tag }}
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/var/cache/apt \
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
{% endif %}
apt-get update && \
apt-get -y install --no-install-recommends \
@@ -22,13 +19,3 @@ RUN --mount=type=cache,target=/var/cache/apt \
# https://github.com/balena-io-library/base-images/issues/562
RUN c_rehash
# We need this to ensure that the wheels can be built.
# Otherwise we get "invalid command 'bdist_wheel'"
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/root/.cache/pip \
{% endif %}
pip3 install --upgrade pip --break-system-packages && \
pip3 install wheel --break-system-packages

View File

@@ -1,12 +1,10 @@
{% include 'uv-builder.j2' %}
{% include 'Dockerfile.base.j2' %}
COPY requirements/requirements.txt /tmp/requirements.txt
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/root/.cache/pip \
{% endif %}
pip3 install -r /tmp/requirements.txt --break-system-packages
COPY --from=uv-builder /venv /venv
ENV PATH="/venv/bin:$PATH"
ENV VIRTUAL_ENV="/venv"
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

View File

@@ -6,7 +6,7 @@ FROM {{ base_image }}:{{ base_image_tag }}
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/var/cache/apt \
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
{% endif %}
apt-get update && \
apt-get -y install --no-install-recommends \

View File

@@ -6,7 +6,7 @@ FROM {{ base_image }}:{{ base_image_tag }}
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/var/cache/apt \
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
{% endif %}
apt-get update && \
apt-get -y install --no-install-recommends \

View File

@@ -19,15 +19,13 @@ COPY ./static/src/ /app/static/src/
RUN npm run build
{% endif %}
{% include 'uv-builder.j2' %}
{% include 'Dockerfile.base.j2' %}
COPY requirements/requirements.txt /tmp/requirements.txt
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/root/.cache/pip \
{% endif %}
pip3 install -r /tmp/requirements.txt --break-system-packages
COPY --from=uv-builder /venv /venv
ENV PATH="/venv/bin:$PATH"
ENV VIRTUAL_ENV="/venv"
RUN mkdir -p /usr/src/app
COPY . /usr/src/app/

View File

@@ -1,3 +1,5 @@
{% include 'uv-builder.j2' %}
{% include 'Dockerfile.base.j2' %}
# vim: ft=dockerfile
@@ -23,7 +25,7 @@ ENV DEBIAN_FRONTEND noninteractive
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/var/cache/apt \
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
{% endif %}
apt-get -y install \
{% for dependency in apt_dependencies %}
@@ -40,25 +42,14 @@ RUN --mount=type=cache,target=/var/cache/apt \
# WORKDIR /opt/ffmpeg
# COPY --from=builder /opt/ffmpeg/ffserver ./
ADD requirements/requirements.txt /tmp/requirements.txt
ADD requirements/requirements.dev.txt /tmp/requirements.dev.txt
ADD requirements/requirements.viewer.txt /tmp/requirements.viewer.txt
RUN wget {{ chromedriver_dl_url }} -O /tmp/chromedriver.zip && \
unzip -o /tmp/chromedriver.zip -d /opt
RUN wget {{ chrome_dl_url }} -O /tmp/chrome.zip && \
unzip -o /tmp/chrome.zip -d /opt
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/root/.cache/pip \
{% endif %}
pip3 install \
-r /tmp/requirements.txt \
-r /tmp/requirements.dev.txt \
-r /tmp/requirements.viewer.txt \
--break-system-packages
COPY --from=uv-builder /venv /venv
ENV PATH="/venv/bin:$PATH"
ENV VIRTUAL_ENV="/venv"
RUN mkdir -p /usr/src/app
COPY . /usr/src/app

View File

@@ -1,5 +1,4 @@
# syntax=docker/dockerfile:1.4
# vim: ft=dockerfile
{% include 'uv-builder.j2' %}
FROM {{ base_image }}:{{ base_image_tag }}
@@ -7,7 +6,7 @@ FROM {{ base_image }}:{{ base_image_tag }}
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/var/cache/apt \
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
{% endif %}
apt-get update && \
apt-get -y install --no-install-recommends \
@@ -19,25 +18,9 @@ RUN --mount=type=cache,target=/var/cache/apt \
{% endif %}
{% endfor %}
# We need this to ensure that the wheels can be built.
# Otherwise we get "invalid command 'bdist_wheel'"
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/root/.cache/pip \
{% endif %}
pip3 install --upgrade pip --break-system-packages && \
pip3 install wheel --break-system-packages
# Install Python requirements
COPY requirements/requirements.viewer.txt /tmp/requirements.txt
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/root/.cache/pip \
{% endif %}
pip3 install -r /tmp/requirements.txt --break-system-packages
COPY --from=uv-builder /venv /venv
ENV PATH="/venv/bin:$PATH"
ENV VIRTUAL_ENV="/venv"
# Works around issue with `curl`
# https://github.com/balena-io-library/base-images/issues/562

View File

@@ -1,12 +1,10 @@
{% include 'uv-builder.j2' %}
{% include 'Dockerfile.base.j2' %}
COPY requirements/requirements-websocket.txt /tmp/requirements.txt
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/root/.cache/pip \
{% endif %}
pip3 install -r /tmp/requirements.txt --break-system-packages
COPY --from=uv-builder /venv /venv
ENV PATH="/venv/bin:$PATH"
ENV VIRTUAL_ENV="/venv"
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

View File

@@ -1,9 +1,11 @@
{% include 'uv-builder.j2' %}
{% include 'Dockerfile.base.j2' %}
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/var/cache/apt \
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
{% endif %}
apt-get -y install --no-install-recommends \
{% for dependency in apt_dependencies %}
@@ -24,13 +26,9 @@ RUN if [[ ! -z "{{ archive_url }}" ]]; then \
unzip -o /tmp/wifi_connect.zip -d /usr/src/app && \
rm /tmp/wifi_connect.zip; fi
COPY requirements/requirements.wifi-connect.txt /tmp/requirements.txt
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/root/.cache/pip \
{% endif %}
pip3 install -r /tmp/requirements.txt --break-system-packages
COPY --from=uv-builder /venv /venv
ENV PATH="/venv/bin:$PATH"
ENV VIRTUAL_ENV="/venv"
COPY send_zmq_message.py ./

64
docker/uv-builder.j2 Normal file
View File

@@ -0,0 +1,64 @@
# syntax=docker/dockerfile:1.4
# vim: ft=dockerfile
FROM {{ base_image }}:{{ base_image_tag }} AS uv-builder
{% if board in ['pi1', 'pi2'] %}
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
{% endif %}
apt-get update && \
apt-get install -y --no-install-recommends \
build-essential \
libffi-dev \
libssl-dev \
libzmq3-dev \
python3 \
python3-dev \
python3-pip \
{% for dep in builder_extra_apt | default([]) %}
{{ dep }} \
{% endfor %}
ca-certificates && \
pip3 install uv --break-system-packages
{% else %}
COPY --from=ghcr.io/astral-sh/uv:0.9.17 /uv /uvx /usr/local/bin/
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
{% endif %}
apt-get update && \
apt-get install -y --no-install-recommends \
build-essential \
libffi-dev \
libssl-dev \
libzmq3-dev \
python3 \
python3-dev \
{% for dep in builder_extra_apt | default([]) %}
{{ dep }} \
{% endfor %}
ca-certificates
{% endif %}
ENV UV_LINK_MODE=copy \
UV_COMPILE_BYTECODE=1 \
UV_PYTHON=/usr/bin/python3.11 \
UV_PROJECT_ENVIRONMENT=/venv
WORKDIR /app
COPY pyproject.toml uv.lock /app/
{% if uv_system_site_packages %}
RUN uv venv --system-site-packages --python /usr/bin/python3.11 /venv
{% endif %}
{% if disable_cache_mounts %}
RUN \
{% else %}
RUN --mount=type=cache,target=/root/.cache/uv \
{% endif %}
uv sync --frozen --no-install-project --no-default-groups \
--group {{ uv_group }}

View File

@@ -9,23 +9,12 @@ To get started, SSH to your Raspberry Pi running Anthias. For instance:
$ ssh pi@raspberrypi
```
Go to the project root directory and create a Python virtual environment, if you haven't created one.
Go to the project root directory and install the dependencies required by
the assets migration script using [uv](https://docs.astral.sh/uv/):
```bash
$ cd ~/screenly
$ python -m venv venv/
```
Activate the virtual environment. You need to do this everytime right before you run the script.
```bash
$ source ./venv/bin/activate
```
Install the dependencies required by the assets migration script.
```bash
$ pip install -r requirements/requirements.local.txt
$ uv sync --group local
```
Before running the script, you should prepare the following:
@@ -35,5 +24,5 @@ Before running the script, you should prepare the following:
Run the assets migration script. Follow through the instructions & prompts carefully.
```bash
$ python tools/migrate_assets_to_screenly.py
```
$ uv run python tools/migrate_assets_to_screenly.py
```

View File

@@ -8,12 +8,122 @@ dependencies = []
[dependency-groups]
dev-host = [
"ansible-lint==26.4.0",
"ruff==0.14.10",
]
docker-image-builder = [
"click==8.1.7",
"jinja2==3.1.6",
"pygit2==1.19.1",
"requests==2.32.5",
"requests==2.33.1",
"python-on-whales==0.79.0",
]
server = [
"cec==0.2.8",
"celery==5.2.2",
"certifi==2025.10.5",
"cffi==2.0.0",
"configparser==7.2.0",
"cryptography==46.0.7",
"Cython==3.2.4",
"Django==4.2.30",
"djangorestframework==3.16.1",
"django-dbbackup==4.2.1",
"drf-spectacular==0.29.0",
"future==1.0.0",
"gevent-websocket==0.10.1",
"gevent==25.9.1",
"gunicorn==23.0.0",
"hurry.filesize==0.9",
"importlib-metadata==4.13.0",
"Jinja2==3.1.6",
"jsonschema==4.25.1",
"kombu==5.5.4",
"netifaces==0.11.0",
"psutil==7.2.1",
"pyasn1==0.6.3",
"pydbus==0.6.0",
"pyOpenSSL==26.0.0",
"python-dateutil==2.9.0.post0",
"pytz==2025.2",
"PyYAML==6.0.2",
"pyzmq==23.2.1",
"redis==7.4.0",
"requests[security]==2.33.1",
"setuptools==79.0.1",
"tenacity==9.1.2",
"sh==2.2.2",
"six==1.17.0",
"urllib3==2.6.3",
"wheel==0.46.2",
"yt-dlp==2026.2.21",
]
viewer = [
"cec==0.2.8",
"certifi==2025.10.5",
"configparser==7.2.0",
"cryptography==46.0.7",
"Cython==3.2.4",
"Django==4.2.30",
"django-dbbackup==4.2.1",
"drf-spectacular==0.29.0",
"future==1.0.0",
"idna==3.11",
"Jinja2==3.1.6",
"jsonschema==4.25.1",
"netifaces==0.11.0",
"pydbus==0.6.0",
"python-dateutil==2.9.0.post0",
"python-vlc==3.0.21203",
"pytz==2025.2",
"pyzmq==23.2.1",
"redis==7.4.0",
"requests[security]==2.33.1",
"tenacity==9.1.2",
"sh==2.2.2",
"uptime==3.0.1",
"urllib3==2.6.3",
]
websocket = [
"Cython==3.2.4",
"future==1.0.0",
"gevent-websocket==0.10.1",
"gevent==25.9.1",
"Jinja2==3.1.6",
"pytz==2025.2",
"pyzmq==23.2.1",
]
wifi-connect = [
"Cython==3.2.4",
"future==1.0.0",
"netifaces==0.11.0",
"pyzmq==23.2.1",
"redis==7.4.0",
]
dev = [
"future==1.0.0",
"mock==5.2.0",
"pep8==1.7.1",
"selenium==4.36.0",
"splinter==0.21.0",
"time-machine==2.15.0",
"unittest-parametrize==1.8.0",
]
host = [
"ansible-core==2.18.3",
"getmac==0.9.5",
"netifaces2==0.0.22",
"redis==7.4.0",
"requests[security]==2.33.1",
"tenacity==9.1.2",
]
local = [
"click==8.1.7",
"requests==2.33.1",
"tenacity==9.1.2",
]
test = [
{ include-group = "server" },
{ include-group = "viewer" },
{ include-group = "dev" },
]

View File

@@ -1,7 +0,0 @@
Cython==3.2.3
future==1.0.0
gevent-websocket==0.10.1
gevent==25.9.1
Jinja2==3.1.6
pytz==2025.2
pyzmq==23.2.1

View File

@@ -1,7 +0,0 @@
future==1.0.0
mock==5.2.0
pep8==1.7.1
selenium==4.36.0
splinter==0.21.0
time-machine==2.15.0
unittest-parametrize==1.8.0

View File

@@ -1,8 +1,21 @@
# vim: ft=requirements
# This file was autogenerated by uv via the following command:
# uv export --group host --no-hashes --no-annotate --no-emit-project --no-default-groups -o requirements/requirements.host.txt
ansible-core==2.18.3
async-timeout==5.0.1 ; python_full_version < '3.11.3'
certifi==2025.10.5
cffi==2.0.0 ; platform_python_implementation != 'PyPy'
charset-normalizer==3.4.4
cryptography==46.0.7
getmac==0.9.5
idna==3.11
jinja2==3.1.6
markupsafe==3.0.3
netifaces2==0.0.22
redis==7.1.0
requests[security]==2.32.3
packaging==26.1
pycparser==2.23 ; implementation_name != 'PyPy' and platform_python_implementation != 'PyPy'
pyyaml==6.0.2
redis==7.4.0
requests==2.33.1
resolvelib==1.0.1
tenacity==9.1.2
urllib3==2.6.3

View File

@@ -1,5 +0,0 @@
# vim: ft=requirements
click==8.1.7
requests==2.32.5
tenacity==9.1.2

View File

@@ -1,37 +0,0 @@
cec==0.2.8
celery==5.2.2
certifi==2025.10.5
cffi==2.0.0
configparser==7.2.0
cryptography==3.3.2
Cython==3.2.3
Django==4.2.29
djangorestframework==3.16.1
django-dbbackup==4.2.1
drf-spectacular==0.29.0
future==1.0.0
gevent-websocket==0.10.1
gevent==25.9.1
gunicorn==23.0.0
hurry.filesize==0.9
importlib-metadata==4.13.0
Jinja2==3.1.6
jsonschema==4.25.1
kombu==5.5.4
netifaces==0.11.0
psutil==7.2.1
pyasn1==0.6.2
pydbus==0.6.0
pyOpenSSL==19.1.0
python-dateutil==2.9.0.post0
pytz==2025.2
PyYAML==6.0.2
pyzmq==23.2.1
redis==7.1.0
requests[security]==2.32.3
tenacity==9.1.2
sh==1.8
six==1.17.0
urllib3==2.6.3
wheel==0.38.1
yt-dlp==2026.2.21

View File

@@ -1,26 +0,0 @@
# vim: ft=requirements
cec==0.2.8
certifi==2025.10.5
configparser==7.2.0
cryptography==3.3.2
Cython==3.2.3
Django==4.2.29
django-dbbackup==4.2.1
drf-spectacular==0.29.0
future==1.0.0
idna==3.11
Jinja2==3.1.6
jsonschema==4.25.1
netifaces==0.11.0
pydbus==0.6.0
python-dateutil==2.9.0.post0
python-vlc==3.0.20123
pytz==2025.2
pyzmq==23.2.1
redis==7.1.0
requests[security]==2.32.3
tenacity==9.1.2
sh==1.8
uptime==3.0.1
urllib3==2.6.3

View File

@@ -1,5 +0,0 @@
Cython==3.2.3
future==1.0.0
netifaces==0.11.0
pyzmq==23.2.1
redis==7.1.0

View File

@@ -15,6 +15,7 @@ from tools.image_builder.utils import (
get_build_parameters,
get_docker_tag,
get_test_context,
get_uv_builder_context,
get_viewer_context,
get_wifi_connect_context,
)
@@ -93,6 +94,8 @@ def build_image(
elif service == 'wifi-connect':
context.update(get_wifi_connect_context(target_platform))
context.update(get_uv_builder_context(service))
generate_dockerfile(
service,
{

View File

@@ -76,6 +76,36 @@ def generate_dockerfile(service: str, context: dict) -> None:
f.write(dockerfile)
def get_uv_builder_context(service: str) -> dict:
service_to_group = {
'server': 'server',
'celery': 'server',
'websocket': 'websocket',
'wifi-connect': 'wifi-connect',
'viewer': 'viewer',
'test': 'test',
}
uv_group = service_to_group.get(service)
if uv_group is None:
return {}
groups_needing_native_build_libs = {'server', 'viewer', 'test'}
builder_extra_apt = []
if uv_group in groups_needing_native_build_libs:
builder_extra_apt = [
'libcec-dev',
'libdbus-1-dev',
'libdbus-glib-1-dev',
]
return {
'uv_group': uv_group,
'builder_extra_apt': builder_extra_apt,
'uv_system_site_packages': service in {'viewer', 'test'},
}
def get_test_context() -> dict:
chrome_dl_url = (
'https://storage.googleapis.com/chrome-for-testing-public/'

1810
uv.lock generated
View File

File diff suppressed because it is too large Load Diff

View File

@@ -164,7 +164,7 @@ def load_browser():
def view_webpage(uri):
global current_browser_url
if browser is None or not browser.process.alive:
if browser is None or not browser.is_alive():
load_browser()
if current_browser_url is not uri:
browser_bus.loadPage(uri)
@@ -175,7 +175,7 @@ def view_webpage(uri):
def view_image(uri):
global current_browser_url
if browser is None or not browser.process.alive:
if browser is None or not browser.is_alive():
load_browser()
if current_browser_url is not uri:
browser_bus.loadImage(uri)