mirror of
https://github.com/Screenly/Anthias.git
synced 2026-06-10 17:18:43 -04:00
da53e045a6801b5056fb448e68e55da7bf106913
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Viktor Petersson
|
c28da3b294 |
fix(server): compare against latest release tag, not branch HEAD (#2831)
* fix(server): compare against latest release tag, not branch HEAD `is_up_to_date()` previously compared the device's baked-in `GIT_HASH` to the master branch's HEAD SHA, with a GHCR-image-digest fallback. That answered "is your commit at master tip?" instead of "is there a newer release?", so devices on the last release artefact flipped to "out of date" the moment master moved one commit forward, and any GitHub API hiccup silently flipped the indicator back to "up to date". Rewrite around `/repos/Screenly/Anthias/releases/latest`: - Cache the `tag_name` for 24h; honour the existing 5-minute error backoff key. - Compare with `packaging.version.Version` so CalVer ordering is numeric (`2026.10.0 > 2026.5.0`). - On API failure or malformed remote tag, return the last successfully-computed verdict (persisted in Redis on every fresh check); fall back to `False` if there's no cached verdict — never optimistic. - Suppress the indicator on dev/branch builds whose local CalVer doesn't parse. - Drop `fetch_remote_hash`, `remote_branch_available`, `is_running_latest_published_image` and the GHCR helpers. Closes #2819 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style: ruff-format new github module + tests Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server): scope verdict cache by local release, back off on malformed 200s Two follow-ups to the release-tag rewrite: - Scope the last-verdict cache key by the installed CalVer (`is-up-to-date:last-verdict:<release>`). After an upgrade during a GitHub outage, the previous version's verdict no longer applies; an unscoped cache could keep the indicator wrong until GitHub came back. Falling back to the no-cache path (False) for an upgraded release is the safer choice the issue's "first-run pessimistic" rule already prescribed. - Trip the existing 5-minute backoff on malformed 200 responses (bad JSON, missing/non-string `tag_name`). Without this, an upstream payload regression would re-fire the GitHub call on every page render until the body was fixed. Also pin `packaging==26.1` directly in the server group instead of relying on its transitive arrival via celery → kombu. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server): don't cache an unparseable release tag Per Copilot review: `_fetch_latest_release_tag()` previously cached any non-empty `tag_name` for 24h before validating parseability. A hand-edited release returning `nightly` (or similar) would pin `is_up_to_date()` to the fallback verdict for the full TTL even after upstream corrected the tag. Validate `_parse_version(tag) is not None` before writing the cache; treat an unparseable tag as a malformed-body error and trip the existing 5-minute backoff. Once the backoff clears, the next request re-fetches and picks up the corrected tag. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Viktor Petersson
|
133ec78ff0 |
refactor(packaging): adopt src/ layout with split server/viewer packages (#2817)
* refactor(packaging): adopt src/ layout with split server/viewer packages
Move all Python source under src/ following modern packaging conventions.
Server, viewer, host-agent, and shared common code now live as four
top-level packages with clear excision boundaries — anthias_viewer can
be removed wholesale when the rewrite-out-of-Python lands without
touching the server.
src/anthias_common/ shared: errors, utils, internal_auth, device_helper
src/anthias_server/ Django app, REST API, Celery tasks, manage.py
lib/ server-only: auth, backup_helper, diagnostics, github, telemetry
src/anthias_viewer/ player runtime (was viewer/)
src/anthias_host_agent/ systemd-driven host shim (was host_agent.py)
tools/raspberry_pi_imager/ moved from repo root
tests/conftest.py moved from repo root
pyproject.toml gets [build-system], setuptools src/ discovery, and an
anthias-manage console script. Django AppConfigs keep label='anthias_app'
and label='api' so existing migration dependency tuples don't move.
BASE_DIR computed from parents[3] to keep templates/static at repo root.
mypy_path set to ["src", "stubs"] with explicit_package_bases.
Dockerfile templates set PYTHONPATH=/usr/src/app/src; bin/start_*.sh
and CI workflows use python -m anthias_server.manage / python -m
anthias_viewer instead of bare ./manage.py and python -m viewer.
Ansible host-agent unit invokes python -m anthias_host_agent.
Verified end-to-end in the docker test container:
- 430 unit tests pass (matches baseline)
- 7 integration tests pass, 5 skipped (matches baseline)
- ruff, mypy clean
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style: ruff format the new src/ tree
The longer post-rename module paths (anthias_common.internal_auth vs
lib.internal_auth, etc.) pushed several import lines past 79 chars, so
ruff format had to wrap them. Apply that formatting and split the one
multi-import in anthias_viewer/__init__.py into per-symbol lines so the
existing # noqa: E402 sits on the `from` line where ruff expects it,
without needing a re-anchor when format wraps the parens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: realign sonar + gitignore comment to src/ layout
sonar-project.properties still pointed at the pre-refactor top-level
packages (anthias_app, anthias_django, api, lib, viewer, ...) and
their old per-file coverage.exclusions paths, which would have
produced empty Sonar runs and stale exclusions. Collapse sources to
`src` and rewrite the exclusions to the new src/anthias_*/ paths.
Also fix the stale path reference in .gitignore's comment for the
test DB (now src/anthias_server/django_project/settings.py).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: gitignore .claude/ and untrack the lock file I just leaked
Previous commit accidentally pulled in .claude/scheduled_tasks.lock
because .claude was in .dockerignore but not .gitignore. Add the
pattern to .gitignore and drop the file from the index.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(dockerignore): exclude pytest cache, __pycache__ dirs, and the local test DB
Three entries that were missing relative to the new src/ layout:
- .anthias-test.db (and -journal/-wal/-shm siblings) — created at the
repo root by src/anthias_server/django_project/settings.py when a
developer runs the host pytest suite. Without this exclude, the
next docker build COPY . bakes the file into /usr/src/app/.
- **/__pycache__ — *.py[co] only matched the .pyc/.pyo files, leaving
the empty cache directories to ship.
- .pytest_cache — host-side, regenerable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(urls): preserve 'anthias_app' URL namespace, not just the app label
Copilot caught that the import-rewrite swept up the URL namespace too:
app_name in src/anthias_server/app/urls.py changed from 'anthias_app'
to 'anthias_server.app', which leaves templates/login.html's
{% url 'anthias_app:login' %} pointing at a namespace that no longer
exists — NoReverseMatch at render time when an unauthenticated request
hits the login page.
The namespace is the same kind of stable user-facing identifier as the
AppConfig label (which we already kept as 'anthias_app'). Restore it,
and revert the two reverse() callers in lib/auth.py and app/views.py
that the rewrite changed in lockstep.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ci): update --confcutdir to the new tools/raspberry_pi_imager path
Copilot caught that the earlier sweep missed --confcutdir=raspberry_pi_imager
(no trailing slash) — replace_all of "raspberry_pi_imager/" only matched
path-with-slash forms. Without confcutdir, pytest walks back up looking
for conftests and discovers the repo-root tests/conftest.py, which
applies the Anthias-specific Django/Redis stubs to the rpi-imager test
run on the website-deploy workflow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Viktor Petersson
|
f96016ca95 |
refactor(tests): migrate Python test suite to pytest (#2801)
* refactor(tests): migrate Python test suite from Django runner to pytest
Replaces ./manage.py test with pytest as the single Python test runner
for speed (xdist parallelization), readability (function-style + bare
assert), and to drop the mock and unittest_parametrize extra deps in
favor of stdlib unittest.mock + pytest.mark.parametrize.
- pyproject.toml: drop mock and unittest-parametrize from dev (and
types-mock from dev-host); add pytest, pytest-django, pytest-mock,
pytest-xdist; add [tool.pytest.ini_options] with
DJANGO_SETTINGS_MODULE, testpaths covering tests/, api/tests/ and
anthias_app/, and the integration marker.
- All 13 Python test modules under tests/ and api/tests/ converted
to function-based pytest with @pytest.mark.django_db where Django
ORM is touched, @pytest.mark.parametrize replacing
unittest_parametrize, and @pytest.mark.integration replacing
Django @tag('integration'). Mocks now import unittest.mock.
- tests/test_app.py: integration tests use
@pytest.mark.django_db(transaction=True) since the original
unittest.TestCase did not wrap in a Django transaction (the live
server uses the same SQLite DB).
- .github/workflows/test-runner.yml + CLAUDE.md: invocations
switched to `pytest -n auto -m "not integration"` and
`pytest -m integration`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(coverage): add tests for lib/auth.py
Cover NoAuth, BasicAuth, the abstract Auth base, password hashing
round-trip, the legacy SHA256 detector, and the @authorized decorator's
three branches (no auth, auth-required redirect, view passthrough).
The Authorization header path exercises malformed base64, missing
colon, unsupported scheme, and the RFC 7617 colon-in-password case.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(coverage): cover GitHub + GHCR helpers in lib/github.py
remote_branch_available: happy path (200 → True), 404 → False, network
exception triggers backoff, 5xx triggers backoff, cache hits short-
circuit, and active backoff suppresses requests. Locks in #2797's
direct-branch endpoint behavior via a URL substring assertion.
fetch_remote_hash: missing GIT_BRANCH, cache hit, branch-unavailable
short-circuit, happy path, and request-exception handling.
_get_ghcr_anonymous_token: happy path, RequestException, ValueError on
JSON decode, missing token field, non-string token field.
_get_ghcr_manifest_digest: happy path, 404 (no backoff), 5xx + 429
(with backoff), RequestException, missing/empty Docker-Content-Digest
header (with backoff).
is_running_latest_published_image: missing inputs, all three cache
verdicts ('1'/'0'/'?'), backoff active, no token, no latest digest,
current digest 404 caches '?', match → True, mismatch → False, and
cache key scoped per (device_type, short_hash).
handle_github_error: with and without exc.response.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(coverage): add tests for lib/diagnostics.py
Cover the env-var helpers (get_git_branch / get_git_short_hash /
get_git_hash), the file-backed helpers (get_uptime, get_debian_version,
get_load_avg, get_utc_isodate), the device-helper-backed accessors
(get_raspberry_code, get_raspberry_model), the CEC-via-subprocess
get_display_power on True/False/CEC error/Unknown/empty stdout/timeout,
and try_connectivity in all-OK / all-Error / mixed configurations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(coverage): add tests for viewer ↔ server Redis messaging
ViewerPublisher.send_to_viewer publishes the namespaced 'viewer …'
payload on VIEWER_CHANNEL; ReplySender pushes JSON onto the per-
correlation-ID list and sets the 30s TTL; ReplyCollector.recv_json
covers the BLPOP path with second-rounding, the LPOP non-blocking
path (timeout_ms <= 0), and ReplyTimeoutError on no reply. Singleton
get_instance() caches the first instance and rejects double-init.
ViewerSubscriber._consume dispatches commands with/without parameters,
skips wrong-topic and non-string payloads, falls back to the 'unknown'
handler when registered, and is a no-op when no handler matches.
run() signals viewer-subscriber-ready, falls into _consume, and on
ConnectionError marks unready before sleeping.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(coverage): add pytest tests for anthias_app/views_files.py
Cover _client_ip directly (REMOTE_ADDR happy path, IPv6, malformed,
missing), the require_client_in decorator (allow/reject/malformed/
multi-CIDR), and both views (anthias_assets, static_with_mime) for
the in-CIDR happy path, out-of-CIDR rejection, traversal blocking,
missing files, directory requests, symlink escape, and the mime
override allowlist.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(coverage): cover reboot/shutdown/get_display_power Celery tasks
reboot_anthias and shutdown_anthias now have both code paths exercised:
on Balena, the supervisor helper is invoked (via tenacity Retrying);
off Balena, an 'r.publish('hostcmd', …)' is issued. get_display_power
delegates to lib.diagnostics and persists the verdict + 1h TTL on
Redis. send_telemetry_task is a thin wrapper. cleanup also gains a
test for the early-return path when settings['assetdir'] is missing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(coverage): expand lib/utils.py coverage and add device_helper tests
lib/utils: cover string_to_bool truthy/falsy/invalid, validate_url
across http/https/rtsp/rtmp + invalid forms, the env-var booleans
(is_ci / is_balena_app / is_demo_node / is_docker), the Balena
supervisor helpers (get_balena_device_info / reboot / shutdown /
version OK + error), JSON datetime serialisation, the perfect-paper-
password generator (length and no-symbols variant), and the
template-handle-unicode non-string fallback.
lib/device_helper: parse_cpu_info on a representative Pi 4 cpuinfo
fixture and on a minimal stub, plus get_device_type's full mapping
(pi5/Compute Module 5, pi4/Compute Module 4, pi3/Compute Module 3,
pi2, pi1) with x86 fallback when /proc/device-tree/model is missing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci(coverage): wire up pytest-cov and enforce 80% line+branch coverage
Add pytest-cov==6.0.0 to the dev group, configure [tool.coverage.run]
with branch coverage and the focused source list (lib, viewer, api,
anthias_app, celery_tasks, settings) plus omits for boilerplate
(migrations, asgi/wsgi/urls/routing, image_builder, host_agent,
viewer/__main__, etc.), and set fail_under = 80 in [tool.coverage.report]
so CI fails when coverage regresses.
The test-runner workflow now passes --cov flags to both the unit and
integration jobs (--cov-append on the second so the totals merge), and
points Codecov at the resulting coverage.xml that lands on the runner
workspace via the /usr/src/app bind-mount. CLAUDE.md gets a coverage
example next to the existing pytest invocation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tests): satisfy mypy on coverage tests
Mypy under strict settings flagged 47 errors in the recently-added
coverage tests. They fall into a few mechanical buckets, all in test
files only — no source changes:
- Replace `patch.object(<module>, '<attr>', ...)` with the string
form `patch('<dotted.path>', ...)` for `utils.os`, `utils.requests`,
`diagnostics.device_helper`, `diagnostics.utils`, and
`celery_tasks_module.diagnostics`. mypy refuses to introspect the
re-exported attribute even though it exists at runtime.
- Switch the `request.session = {}` ignore code from `[attr-defined]`
(which mypy reports as unused) to `[assignment]` — the actual error
is "dict cannot be assigned to SessionBase".
- Drop unused ignore comments that mypy flagged
(`tests/test_messaging.py` `_redis = fake_redis`,
`tests/test_github.py` `exc.response = None`,
`tests/test_auth.py` `view()` / `view() -> str`).
- Stop assigning the result of `NoAuth.authenticate()` (declared
`-> None`); call it as a statement instead.
Verified locally with `uv run mypy .` (Success: no issues found in 105
source files), `uv run ruff check .`, and `uv run ruff format --check .`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci(sonar): add sonar-project.properties to silence test-file false positives
Sonar's default analysis treated everything under the repo as
sonar.sources, including the new coverage tests. That tripped a
batch of S5332 / S2068 / S1244 / S5443 false positives on fixtures
that intentionally exercise insecure protocols, hard-coded test
credentials, exact float comparisons, and tempfile.TemporaryDirectory
patterns.
Pin sonar.sources to the actual source dirs and mark tests/ +
api/tests/ as sonar.tests so the narrower rule set applies. Wire up
sonar.python.coverage.reportPaths so the coverage.xml emitted by
pytest-cov is consumed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(tests): make unit suite locally runnable + force-mock externals
The unit suite previously assumed Docker: settings.py hard-coded the
SQLite test DB to /data/.anthias/test.db, viewer modules required
PyGObject from python3-gi, and connect_to_redis() pointed at the
hostname `redis`. Developers had no way to run pytest from the host.
Changes
- anthias_django/settings.py: when ENVIRONMENT=test, default the
SQLite path to BASE_DIR/.anthias-test.db. CI containers preserve the
/data location via the new ANTHIAS_TEST_DB_PATH env var.
- docker-compose.test.yml: set ANTHIAS_TEST_DB_PATH=/data/.anthias/test.db
so CI keeps its historic DB layout without churn.
- .gitignore: cover .anthias-test.db and its WAL siblings.
- conftest.py (new, repo root): force ENVIRONMENT=test, stub gi /
gi.repository / pydbus in sys.modules so viewer/__init__.py imports
without the distro PyGObject stack, replace lib.utils.connect_to_redis
with a dict-backed MagicMock factory at conftest load time AND via an
autouse fixture (so module-level `r = connect_to_redis()` bindings in
celery_tasks / lib.github / lib.telemetry / api.views.* never hit a
real socket), and ensure settings['assetdir'] exists once per session
so legacy cleanup fixtures don't FileNotFoundError out.
- CLAUDE.md: document the local pytest recipe alongside the Docker one.
External-call audit
- requests.get/head/post/put/delete: all unit-test references inside
patch() / patch.object() — clean.
- connect_to_redis() / module-level `r`: every reference is either
patched per-test or mocked by the new conftest fixture.
- subprocess: only tests/test_migrate_legacy_paths.py runs a real
subprocess, which is the migration shell script under test (local,
no network) — left as-is per the task spec.
- DBus / gi: stubbed at conftest level; no test exercises a live bus.
Local run: `uv sync --group test && uv run pytest -m "not integration"`
yields 359 passed, 12 deselected (libcec-dev required on host for the
cec wheel build).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style(tests): apply ruff format to conftest.py
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tests): address Copilot review + drop unused type:ignore in conftest
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(tests): drop type:ignore / noqa suppressions in tests + conftest
Replace bare-dict assignments to ``request.session`` in test_auth.py with
``SessionStore()`` from the signed_cookies backend (typed correctly,
no DB write). Use ``cast()`` for the singleton-sentinel assignments in
test_messaging.py instead of ``# type: ignore[assignment]``.
In conftest.py, swap the ``try: import gi`` probe for
``importlib.util.find_spec``, and use ``setattr()`` for the dynamic
attribute writes onto stubbed modules — both stop tripping mypy
without suppression. Always stub ``pydbus`` when ``gi`` is missing
(real pydbus does ``from gi.repository import GLib, GObject`` and our
minimal stub doesn't fake ``GObject``).
Drop the dead ``# noqa: E501`` markers from three test signatures (E501
isn't in the active ruff rule selection, so they were suppressing
nothing) and rename the one signature that genuinely was over 79 chars
(``test__if_git_branch_env_does_not_exist__is_up_to_date_should_return_true``
→ ``test_returns_true_when_git_branch_env_missing``).
Verified: ``ruff check``, ``ruff format --check``, ``mypy``, and the
non-integration ``pytest`` run (359 passed) are all clean with zero
suppressions remaining in tests/ + conftest.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci(sonar): silence test-only rule false positives
S2068 (hard-coded credential), S1244 (float equality), S5864
(identity check), and S7492 (comprehension unpack) flag patterns
that are intentional in test fixtures. Suppress them under
tests/** and api/tests/** via sonar.issue.ignore.multicriteria
rather than scattering NOSONAR comments.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci(sonar): fix issue-ignore globs to actually match top-level test files
`tests/**/*.py` is Ant-style "tests/<dir>/.../<file>.py" — it
requires at least one intermediate directory and so does not
match `tests/test_auth.py`. Switch all four exclusions to
`**/test_*.py`, which matches every test file under the project
regardless of nesting and is the only pattern Sonar's
PathMatcher reliably honors for top-level test directories.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tests): unblock integration DB sharing, settings xdist race, sonar
Three independent issues surfaced when CI ran
|