mirror of
https://github.com/sabnzbd/sabnzbd.git
synced 2026-09-23 03:15:22 -04:00
* Migrate web interface from CherryPy to Uvicorn/Starlette
Squashed rebase of feature/uvicorn (34 commits) onto develop, reconciled
with ~3 months of intervening develop changes.
Replaces the CherryPy webserver and request handling with Uvicorn/Starlette
across the API, web interface, RSS, config pages and related modules.
Reconciliation with develop during the rebase:
- api.py: kept develop's security/behaviour fixes (orphan path-traversal
guard, expanded log redaction incl. host_whitelist and
remote_label_replacement, get_dconfig single-return, get_retryable_jobs,
connections default, translated NNTP test errors) on top of the Starlette
request/response rewrite.
- interface.py: ported the RSS route handlers to develop's DB-backed
RSSRepository API (process_feed, rss_repository / find_job_by_url /
clear_feed / clear_downloaded / flag_downloaded).
- misc.py: kept develop's hachoir-based get_media_duration.
- requirements.txt: dropped the CherryPy stack, adopted develop's newer pins.
Also applied ruff --fix (PEP 604 unions, builtin generics) to align with
develop's lint config.
Verified: ruff check, black --check, and the affected test suites
(9413 passed, 1 skipped) all pass.
* Update starlette/uvicorn versions
* Fix race issues in global rss state
* Fix test race in server shutdown
The uvicorn migration turned /shutdown (and the shutdown API) into fire-and-forget: it spawned shutdown_program() in a background thread and replied immediately, whereas develop ran it synchronously and only replied once halt() had persisted all state. Because the module-scoped test teardown doesn't wait for the process to exit, the next module's clean_cache_dir wiped the shared cache dir (and reused the fixed port) while the previous instance was still saving state and holding the port — producing the three intermittent failures (deleted sabnzbd.log → "File log disabled or not found"; un-persisted [sorters] → KeyError; stale instance → missing wizard .quoteBlock).
* Fix robots and description, add favicon
* Remove remains of http basic auth
* Setup Starlette once configuration is available, fix static file relative cwd and url_base config
* abort_and_show_error when webserver fails to start
* Guard stopping webserver that never started
* Delegate XFF handling to ProxyHeadersMiddleware
* Merged params at request.state.params instead of modifying private apis
* Both shutdown routes share implementation and do not block event loop
* Run sync handlers via run_in_threadpool and facilitate eventual migration to async
* Pool database connections
* Online backup of database due to WAL changes
* Fix exception on None request.client (test clients or unix sockets)
* Fix flakey tests due to process not fully shutting down
* Restore X-Frame-Options behaviour via middleware
* Fix set_config_default with multiple keywords
* Remove broken logging call
* Restore api logging functionality
* Cache-Control: no-store
* Login only via POST
* Remove 401 (basic-auth) and add 404 handling via redirect
* Fix crash when shutdown not an int
* Use BaseRedirectResponse helper
* Remove trailing slashes from wizard routes
* URL helper, absolute URLs everywhere, fixes issues with nested navigation
* Fix scheduler adding multiple daysofweek
* Restore CherryPy api behaviour merging body with query params (body wins)
* Clearer documentation of get_request_params and request_params
* First stage supporting gradual api async
* Fix rss ajax consuming flash
* Restore access log functionality
* Hostname check in middleware
* Request logging in middleware
* Param parsing in middleware
* Security checks in middleware
* secured_expose is now purely route registration
* Lookup api handler once per request
* Fix flakey alert dialogs
* Trigger restart via BackgroundTask
* Restore CherryPy first param wins and get/post consistency
* Remove dead code
* Secure cookies based on protocol the client used
* Fix various issues with port_is_free
1. port_is_free answered the wrong question. It connect-probed ("is something answering?") rather than bind-probed ("can I bind?"). A port could report free and then kill startup at uvicorn's bind().
2. The bind-all remap crossed address families. :: was mapped to 127.0.0.1, probing IPv4 for an IPv6 bind — a regression against portend, which maps :: → ::1.
3. The call sites passed the wrong host. browserhost is a client-reachable address; the thing that has to be bindable is web_host.
4. Errors were swallowed. A bare except OSError hid gaierror, so an unresolvable host reported "free".
5. find_free_port had a port-0 trap. Under a bind-probe, currentport=0 always succeeds and returned 0 — the old failure sentinel. Now guarded, and None instead of 0.
6. Ports 80/443 were misdiagnosed. EACCES was folded into "occupied", producing ten futile probes and a panic claiming another program held the port. PermissionError now propagates to a dedicated panic explaining the actual remedies.
7. The tests were largely tautological. Three tests covering one branch, an IPv6 test with no IPv6 in it, a timeout test that never engaged the timeout, TOCTOU-prone fixed-range probes, no SO_REUSEADDR on the helper listener, and nothing asserting the property that matters — that "free" implies bindable.
8. A portability bug I introduced, then fixed. I'd baked Linux SO_REUSEADDR overlap semantics into four assertions; macOS differs. Now platform-aware, with the IPv6 regression re-covered by checking the socket family directly.
* Claim the bind address for uvicorn on startup, resolves "49" in err handling from cherrypy
* Rename function BaseRedirectResponse to base_redirect_response
* Restore error response on change web directory
* Add missing typings
* Fix return type of retry job for future types
* A better fix for xdist compatibility - test overwrote db_path
* Secure session cookies (rss flash)
* Inline or remove some functions
* Retry job futuretype behaviour
* Sneak a worksteal fix in
* Test and fix retry_job futuretype behaviour
166 lines
5.7 KiB
YAML
Executable File
166 lines
5.7 KiB
YAML
Executable File
---
|
|
test_name: Check general queue format (json output)
|
|
|
|
strict:
|
|
- json:on
|
|
|
|
stages:
|
|
- name: queue format single entry
|
|
request:
|
|
url: "http://{SAB_HOST}:{SAB_PORT}/api"
|
|
method: GET
|
|
params:
|
|
mode: queue
|
|
apikey: "{SAB_APIKEY}"
|
|
limit: 1
|
|
response:
|
|
status_code: 200
|
|
headers:
|
|
content-type: !re_match "application/json"
|
|
content-type: !re_search "charset=(UTF|utf)-8"
|
|
cache-control: "no-store"
|
|
pragma: "no-cache"
|
|
access-control-allow-origin: "*"
|
|
json:
|
|
queue:
|
|
version: "{SAB_VERSION}"
|
|
paused: !anybool
|
|
pause_int: !re_match "[0-9]*"
|
|
paused_all: !anybool
|
|
diskspace1: !re_match "[0-9.]*"
|
|
diskspace2: !re_match "[0-9.]*"
|
|
diskspace1_norm: !re_search "[0-9][0-9.]*.?(\ [A-Z])?"
|
|
diskspace2_norm: !re_search "[0-9][0-9.]*.?(\ [A-Z])?"
|
|
diskspacetotal1: !re_match "[0-9.]*"
|
|
diskspacetotal2: !re_match "[0-9.]*"
|
|
speedlimit: !re_match "[0-9]*"
|
|
speedlimit_abs: !re_match "[0-9.]*" # Value may be empty if unset so !anyint won't work
|
|
have_warnings: !re_match "[0-9]*"
|
|
finishaction: !anything
|
|
quota: !re_search "[0-9][0-9.]*.?(\ [A-Z])?"
|
|
have_quota: !anybool
|
|
left_quota: !re_search "[0-9][0-9.]*.?(\ [A-Z])?"
|
|
cache_art: !re_search "[0-9]*"
|
|
cache_size: !re_search "[0-9][0-9.]*.?(\ [A-Z])?"
|
|
kbpersec: !re_match "[0-9.]*"
|
|
speed: !re_search "[0-9][0-9.]*.?(\ [A-Z])?"
|
|
mbleft: !re_match "[0-9.]*"
|
|
mb: !re_match "[0-9.]*"
|
|
sizeleft: !re_search "[0-9][0-9.]*.?(\ [A-Z]+)?"
|
|
size: !re_search "[0-9][0-9.]*.?(\ [A-Z]+)?"
|
|
noofslots_total: !anyint
|
|
status: "Paused"
|
|
timeleft: "0:00:00"
|
|
noofslots: !anyint
|
|
start: !anyint
|
|
limit: !anyint
|
|
finish: !anyint
|
|
slots: !anylist
|
|
index: !re_match "[0-9]+"
|
|
nzo_id: "!anystr"
|
|
unpackopts: !re_match "-?[0-9]+"
|
|
priority: "!anystr"
|
|
script: "!anystr"
|
|
filename: "!anystr"
|
|
labels: !anylist
|
|
password: "!anystr"
|
|
cat: "!anystr"
|
|
mbleft: !re_match "[0-9.]*"
|
|
mb: !re_match "[0-9.]*"
|
|
size: !re_search "[0-9][0-9.]*.?(\ [A-Z]+)?"
|
|
sizeleft: !re_search "[0-9][0-9.]*.?(\ [A-Z]+)?"
|
|
percentage: "[0-9.]+"
|
|
mbmissing: !re_search "[0-9][0-9.]*.?(\ [A-Z]+)?"
|
|
direct_unpack: !re_match "[0-9]+"
|
|
status: "Paused"
|
|
timeleft: "0:00:00"
|
|
avg_age: "!anystr"
|
|
time_added: !anyint
|
|
|
|
---
|
|
test_name: Check general queue format (xml output)
|
|
|
|
stages:
|
|
- name: queue format single entry
|
|
request:
|
|
url: "http://{SAB_HOST}:{SAB_PORT}/api"
|
|
method: GET
|
|
params:
|
|
mode: queue
|
|
apikey: "{SAB_APIKEY}"
|
|
output: xml
|
|
limit: 1
|
|
response:
|
|
status_code: 200
|
|
headers:
|
|
content-type: !re_match "text/xml"
|
|
content-type: !re_search "charset=(UTF|utf)-8"
|
|
cache-control: "no-store"
|
|
pragma: "no-cache"
|
|
access-control-allow-origin: "*"
|
|
content-length: !re_match "[0-9]+"
|
|
verify_response_with:
|
|
function: tavalidate:assert_xml
|
|
extra_kwargs:
|
|
strict: True
|
|
expected: |
|
|
<?xml version="1.0" encoding="UTF-8" ?>
|
|
<queue>
|
|
<version>!anystr</version>
|
|
<paused>!anybool</paused>
|
|
<pause_int>!anyint</pause_int>
|
|
<paused_all>!anybool</paused_all>
|
|
<diskspace1>!anyfloat</diskspace1>
|
|
<diskspace2>!anyfloat</diskspace2>
|
|
<diskspace1_norm>!anystr</diskspace1_norm>
|
|
<diskspace2_norm>!anystr</diskspace2_norm>
|
|
<diskspacetotal1>!anyfloat</diskspacetotal1>
|
|
<diskspacetotal2>!anyfloat</diskspacetotal2>
|
|
<speedlimit>!anyint</speedlimit>
|
|
<speedlimit_abs>!anystr</speedlimit_abs>
|
|
<have_warnings>!anyint</have_warnings>
|
|
<finishaction>!anything</finishaction>
|
|
<quota>!anystr</quota>
|
|
<have_quota>!anybool</have_quota>
|
|
<left_quota>!anystr</left_quota>
|
|
<cache_art>!anyint</cache_art>
|
|
<cache_size>!anystr</cache_size>
|
|
<kbpersec>!anyfloat</kbpersec>
|
|
<speed>!anystr</speed>
|
|
<mbleft>!anyfloat</mbleft>
|
|
<mb>!anyfloat</mb>
|
|
<sizeleft>!anystr</sizeleft>
|
|
<size>!anystr</size>
|
|
<noofslots_total>!anyint</noofslots_total>
|
|
<status>!anystr</status>
|
|
<timeleft>0:00:00</timeleft>
|
|
<noofslots>!anyint</noofslots>
|
|
<start>!anyint</start>
|
|
<limit>!anyint</limit>
|
|
<finish>!anyint</finish>
|
|
<slots>
|
|
<slot>
|
|
<index>!anyint</index>
|
|
<nzo_id>!anystr</nzo_id>
|
|
<unpackopts>!anyint</unpackopts>
|
|
<priority>!anystr</priority>
|
|
<script>!anystr</script>
|
|
<filename>!anystr</filename>
|
|
<labels>!anything</labels>
|
|
<password>!anystr</password>
|
|
<cat>!anystr</cat>
|
|
<mbleft>!anyfloat</mbleft>
|
|
<mb>!anyfloat</mb>
|
|
<size>!anystr</size>
|
|
<sizeleft>!anystr</sizeleft>
|
|
<percentage>!anyint</percentage>
|
|
<mbmissing>!anyfloat</mbmissing>
|
|
<direct_unpack>!anything</direct_unpack>
|
|
<status>!anystr</status>
|
|
<timeleft>0:00:00</timeleft>
|
|
<avg_age>!anystr</avg_age>
|
|
<time_added>!anyint</time_added>
|
|
</slot>
|
|
</slots>
|
|
</queue>
|