Files
sabnzbd/tests/data/tavern/api_queue_empty.yaml
mnightingale 35d5355f49 Replace CherryPy with Uvicorn and Starlette (#3550)
* Migrate web interface from CherryPy to Uvicorn/Starlette

Squashed rebase of feature/uvicorn (34 commits) onto develop, reconciled
with ~3 months of intervening develop changes.

Replaces the CherryPy webserver and request handling with Uvicorn/Starlette
across the API, web interface, RSS, config pages and related modules.

Reconciliation with develop during the rebase:
- api.py: kept develop's security/behaviour fixes (orphan path-traversal
  guard, expanded log redaction incl. host_whitelist and
  remote_label_replacement, get_dconfig single-return, get_retryable_jobs,
  connections default, translated NNTP test errors) on top of the Starlette
  request/response rewrite.
- interface.py: ported the RSS route handlers to develop's DB-backed
  RSSRepository API (process_feed, rss_repository / find_job_by_url /
  clear_feed / clear_downloaded / flag_downloaded).
- misc.py: kept develop's hachoir-based get_media_duration.
- requirements.txt: dropped the CherryPy stack, adopted develop's newer pins.

Also applied ruff --fix (PEP 604 unions, builtin generics) to align with
develop's lint config.

Verified: ruff check, black --check, and the affected test suites
(9413 passed, 1 skipped) all pass.

* Update starlette/uvicorn versions

* Fix race issues in global rss state

* Fix test race in server shutdown

The uvicorn migration turned /shutdown (and the shutdown API) into fire-and-forget: it spawned shutdown_program() in a background thread and replied immediately, whereas develop ran it synchronously and only replied once halt() had persisted all state. Because the module-scoped test teardown doesn't wait for the process to exit, the next module's clean_cache_dir wiped the shared cache dir (and reused the fixed port) while the previous instance was still saving state and holding the port — producing the three intermittent failures (deleted sabnzbd.log → "File log disabled or not found"; un-persisted [sorters] → KeyError; stale instance → missing wizard .quoteBlock).

* Fix robots and description, add favicon

* Remove remains of http basic auth

* Setup Starlette once configuration is available, fix static file relative cwd and url_base config

* abort_and_show_error when webserver fails to start

* Guard stopping webserver that never started

* Delegate XFF handling to ProxyHeadersMiddleware

* Merged params at request.state.params instead of modifying private apis

* Both shutdown routes share implementation and do not block event loop

* Run sync handlers via run_in_threadpool and facilitate eventual migration to async

* Pool database connections

* Online backup of database due to WAL changes

* Fix exception on None request.client (test clients or unix sockets)

* Fix flakey tests due to process not fully shutting down

* Restore X-Frame-Options behaviour via middleware

* Fix set_config_default with multiple keywords

* Remove broken logging call

* Restore api logging functionality

* Cache-Control: no-store

* Login only via POST

* Remove 401 (basic-auth) and add 404 handling via redirect

* Fix crash when shutdown not an int

* Use BaseRedirectResponse helper

* Remove trailing slashes from wizard routes

* URL helper, absolute URLs everywhere, fixes issues with nested navigation

* Fix scheduler adding multiple daysofweek

* Restore CherryPy api behaviour merging body with query params (body wins)

* Clearer documentation of get_request_params and request_params

* First stage supporting gradual api async

* Fix rss ajax consuming flash

* Restore access log functionality

* Hostname check in middleware

* Request logging in middleware

* Param parsing in middleware

* Security checks in middleware

* secured_expose is now purely route registration

* Lookup api handler once per request

* Fix flakey alert dialogs

* Trigger restart via BackgroundTask

* Restore CherryPy first param wins and get/post consistency

* Remove dead code

* Secure cookies based on protocol the client used

* Fix various issues with port_is_free

1. port_is_free answered the wrong question. It connect-probed ("is something answering?") rather than bind-probed ("can I bind?"). A port could report free and then kill startup at uvicorn's bind().
2. The bind-all remap crossed address families. :: was mapped to 127.0.0.1, probing IPv4 for an IPv6 bind — a regression against portend, which maps :: → ::1.
3. The call sites passed the wrong host. browserhost is a client-reachable address; the thing that has to be bindable is web_host.
4. Errors were swallowed. A bare except OSError hid gaierror, so an unresolvable host reported "free".
5. find_free_port had a port-0 trap. Under a bind-probe, currentport=0 always succeeds and returned 0 — the old failure sentinel. Now guarded, and None instead of 0.
6. Ports 80/443 were misdiagnosed. EACCES was folded into "occupied", producing ten futile probes and a panic claiming another program held the port. PermissionError now propagates to a dedicated panic explaining the actual remedies.
7. The tests were largely tautological. Three tests covering one branch, an IPv6 test with no IPv6 in it, a timeout test that never engaged the timeout, TOCTOU-prone fixed-range probes, no SO_REUSEADDR on the helper listener, and nothing asserting the property that matters — that "free" implies bindable.
8. A portability bug I introduced, then fixed. I'd baked Linux SO_REUSEADDR overlap semantics into four assertions; macOS differs. Now platform-aware, with the IPv6 regression re-covered by checking the socket family directly.

* Claim the bind address for uvicorn on startup, resolves "49" in err handling from cherrypy

* Rename function BaseRedirectResponse to base_redirect_response

* Restore error response on change web directory

* Add missing typings

* Fix return type of retry job for future types

* A better fix for xdist compatibility - test overwrote db_path

* Secure session cookies (rss flash)

* Inline or remove some functions

* Retry job futuretype behaviour

* Sneak a worksteal fix in

* Test and fix retry_job futuretype behaviour
2026-08-11 13:27:51 +01:00

120 lines
3.9 KiB
YAML
Executable File

---
test_name: Check empty queue format (json output)
strict:
- json:on
stages:
- name: queue format empty
request:
url: "http://{SAB_HOST}:{SAB_PORT}/api"
method: GET
params:
mode: queue
apikey: "{SAB_APIKEY}"
response:
status_code: 200
headers:
content-type: !re_match "application/json"
content-type: !re_search "charset=(UTF|utf)-8"
cache-control: "no-store"
pragma: "no-cache"
access-control-allow-origin: "*"
json:
queue:
version: "{SAB_VERSION}"
paused: !anybool
pause_int: !re_match "[0-9]*"
paused_all: !anybool
diskspace1: !re_match "[0-9.]*"
diskspace2: !re_match "[0-9.]*"
diskspace1_norm: !re_search "[0-9][0-9.]*.?(\ [A-Z])?"
diskspace2_norm: !re_search "[0-9][0-9.]*.?(\ [A-Z])?"
diskspacetotal1: !re_match "[0-9.]*"
diskspacetotal2: !re_match "[0-9.]*"
speedlimit: !re_match "[0-9]*"
speedlimit_abs: !re_match "[0-9.]*" # Value may be empty if unset so !anyint won't work
have_warnings: !re_match "[0-9]*"
finishaction: null
quota: !re_search "[0-9][0-9.]*.?(\ [A-Z])?"
have_quota: !anybool
left_quota: !re_search "[0-9][0-9.]*.?(\ [A-Z])?"
cache_art: !re_search "[0-9]*"
cache_size: !re_search "[0-9][0-9.]*.?(\ [A-Z])?"
kbpersec: !re_match "[0-9.]*"
speed: !re_search "[0-9][0-9.]*.?(\ [A-Z])?"
mbleft: !re_match "[0-9.]*"
mb: !re_match "[0-9.]*"
sizeleft: "0 B"
size: "0 B"
noofslots_total: 0
status: "Idle"
timeleft: "0:00:00"
noofslots: 0
start: 0
limit: 0
finish: 0
slots: []
---
test_name: Check empty queue format (xml output)
stages:
- name: queue format empty
request:
url: "http://{SAB_HOST}:{SAB_PORT}/api"
method: GET
params:
mode: queue
apikey: "{SAB_APIKEY}"
output: xml
response:
status_code: 200
headers:
content-type: !re_match "text/xml"
content-type: !re_search "charset=(UTF|utf)-8"
cache-control: "no-store"
pragma: "no-cache"
access-control-allow-origin: "*"
content-length: !re_match "[0-9]+"
verify_response_with:
function: tavalidate:assert_xml
extra_kwargs:
strict: True # don't accept extra tags in the response
expected: |
<?xml version="1.0" encoding="UTF-8" ?>
<queue><version>!anystr</version>
<paused>!anybool</paused>
<pause_int>!anyint</pause_int>
<paused_all>!anybool</paused_all>
<diskspace1>!anyfloat</diskspace1>
<diskspace2>!anyfloat</diskspace2>
<diskspace1_norm>!anystr</diskspace1_norm>
<diskspace2_norm>!anystr</diskspace2_norm>
<diskspacetotal1>!anyfloat</diskspacetotal1>
<diskspacetotal2>!anyfloat</diskspacetotal2>
<speedlimit>!anyint</speedlimit>
<speedlimit_abs>!anystr</speedlimit_abs>
<have_warnings>!anyint</have_warnings>
<finishaction>!anything</finishaction>
<quota>!anystr</quota>
<have_quota>!anybool</have_quota>
<left_quota>!anystr</left_quota>
<cache_art>0</cache_art>
<cache_size>0 B</cache_size>
<kbpersec>!anyfloat</kbpersec>
<speed>0 </speed>
<mbleft>0.00</mbleft>
<mb>0.00</mb>
<sizeleft>0 B</sizeleft>
<size>0 B</size>
<noofslots_total>0</noofslots_total>
<status>Idle</status>
<timeleft>0:00:00</timeleft>
<noofslots>0</noofslots>
<start>0</start>
<limit>0</limit>
<finish>0</finish>
<slots></slots>
</queue>