* Migrate web interface from CherryPy to Uvicorn/Starlette
Squashed rebase of feature/uvicorn (34 commits) onto develop, reconciled
with ~3 months of intervening develop changes.
Replaces the CherryPy webserver and request handling with Uvicorn/Starlette
across the API, web interface, RSS, config pages and related modules.
Reconciliation with develop during the rebase:
- api.py: kept develop's security/behaviour fixes (orphan path-traversal
guard, expanded log redaction incl. host_whitelist and
remote_label_replacement, get_dconfig single-return, get_retryable_jobs,
connections default, translated NNTP test errors) on top of the Starlette
request/response rewrite.
- interface.py: ported the RSS route handlers to develop's DB-backed
RSSRepository API (process_feed, rss_repository / find_job_by_url /
clear_feed / clear_downloaded / flag_downloaded).
- misc.py: kept develop's hachoir-based get_media_duration.
- requirements.txt: dropped the CherryPy stack, adopted develop's newer pins.
Also applied ruff --fix (PEP 604 unions, builtin generics) to align with
develop's lint config.
Verified: ruff check, black --check, and the affected test suites
(9413 passed, 1 skipped) all pass.
* Update starlette/uvicorn versions
* Fix race issues in global rss state
* Fix test race in server shutdown
The uvicorn migration turned /shutdown (and the shutdown API) into fire-and-forget: it spawned shutdown_program() in a background thread and replied immediately, whereas develop ran it synchronously and only replied once halt() had persisted all state. Because the module-scoped test teardown doesn't wait for the process to exit, the next module's clean_cache_dir wiped the shared cache dir (and reused the fixed port) while the previous instance was still saving state and holding the port — producing the three intermittent failures (deleted sabnzbd.log → "File log disabled or not found"; un-persisted [sorters] → KeyError; stale instance → missing wizard .quoteBlock).
* Fix robots and description, add favicon
* Remove remains of http basic auth
* Setup Starlette once configuration is available, fix static file relative cwd and url_base config
* abort_and_show_error when webserver fails to start
* Guard stopping webserver that never started
* Delegate XFF handling to ProxyHeadersMiddleware
* Merged params at request.state.params instead of modifying private apis
* Both shutdown routes share implementation and do not block event loop
* Run sync handlers via run_in_threadpool and facilitate eventual migration to async
* Pool database connections
* Online backup of database due to WAL changes
* Fix exception on None request.client (test clients or unix sockets)
* Fix flakey tests due to process not fully shutting down
* Restore X-Frame-Options behaviour via middleware
* Fix set_config_default with multiple keywords
* Remove broken logging call
* Restore api logging functionality
* Cache-Control: no-store
* Login only via POST
* Remove 401 (basic-auth) and add 404 handling via redirect
* Fix crash when shutdown not an int
* Use BaseRedirectResponse helper
* Remove trailing slashes from wizard routes
* URL helper, absolute URLs everywhere, fixes issues with nested navigation
* Fix scheduler adding multiple daysofweek
* Restore CherryPy api behaviour merging body with query params (body wins)
* Clearer documentation of get_request_params and request_params
* First stage supporting gradual api async
* Fix rss ajax consuming flash
* Restore access log functionality
* Hostname check in middleware
* Request logging in middleware
* Param parsing in middleware
* Security checks in middleware
* secured_expose is now purely route registration
* Lookup api handler once per request
* Fix flakey alert dialogs
* Trigger restart via BackgroundTask
* Restore CherryPy first param wins and get/post consistency
* Remove dead code
* Secure cookies based on protocol the client used
* Fix various issues with port_is_free
1. port_is_free answered the wrong question. It connect-probed ("is something answering?") rather than bind-probed ("can I bind?"). A port could report free and then kill startup at uvicorn's bind().
2. The bind-all remap crossed address families. :: was mapped to 127.0.0.1, probing IPv4 for an IPv6 bind — a regression against portend, which maps :: → ::1.
3. The call sites passed the wrong host. browserhost is a client-reachable address; the thing that has to be bindable is web_host.
4. Errors were swallowed. A bare except OSError hid gaierror, so an unresolvable host reported "free".
5. find_free_port had a port-0 trap. Under a bind-probe, currentport=0 always succeeds and returned 0 — the old failure sentinel. Now guarded, and None instead of 0.
6. Ports 80/443 were misdiagnosed. EACCES was folded into "occupied", producing ten futile probes and a panic claiming another program held the port. PermissionError now propagates to a dedicated panic explaining the actual remedies.
7. The tests were largely tautological. Three tests covering one branch, an IPv6 test with no IPv6 in it, a timeout test that never engaged the timeout, TOCTOU-prone fixed-range probes, no SO_REUSEADDR on the helper listener, and nothing asserting the property that matters — that "free" implies bindable.
8. A portability bug I introduced, then fixed. I'd baked Linux SO_REUSEADDR overlap semantics into four assertions; macOS differs. Now platform-aware, with the IPv6 regression re-covered by checking the socket family directly.
* Claim the bind address for uvicorn on startup, resolves "49" in err handling from cherrypy
* Rename function BaseRedirectResponse to base_redirect_response
* Restore error response on change web directory
* Add missing typings
* Fix return type of retry job for future types
* A better fix for xdist compatibility - test overwrote db_path
* Secure session cookies (rss flash)
* Inline or remove some functions
* Retry job futuretype behaviour
* Sneak a worksteal fix in
* Test and fix retry_job futuretype behaviour
* Speed up tests with pytest-xdist
* Ignore request failures when the server disappeared
* Fix tests which modify module globals
* Fix lang change leaking
* Fix transaction compilation order breaking tests
* Always load a clean config and fix related tests
* No module globals
* Fix other test classes assigning module level by making calls
* Replace from sabnzbd.cfg import so monkeypatch works
* More global poisoning
* Fix dependant tests which worksteal breaks
* More global vars
* Drop loadscope until more failures are resolved
* Due to how pytest-xdist is implemented, the -s/--capture=no option does not work
* monkeypatch db_path
* Also patch startup_done
* loadscope
* Apply memory limits from cgroups
* Apply cgroup limits to article cache
* Fix this bug
* _physical_memory return None when undetermined
* Mark platform on test group
* Remove MemoryInfo dataclass
* Cap memory with reservation
* Keep the slightly broken 32 vs 64 bit code
* 512 *MEBI because the numbers look nicer
* Fix flaky sort test by comparing at the same resolution
* Fix consistency of values printed at reduced precision
* Replace the float-string roundtrip with rounding and adjust to next unit if required
* Comment styling
* Verify RAR5 passwords when header encryption is not used
* Make flow a bit closer to rarfile
* Spelling
* Add note to trusting header verification
* Spelling
* Delegate to rarfile 4.3 where possible and cache rar*_s2k
* Explicit types instead of TypeVar
* Remove duplicate test and add a header encryption with correct password
* Verify after parse
* Rename method and explicit cache size
* Remove dead code
* Clearer verify after parse
* Performance early exit
* Clarify early exit
* Update comment, usually same password and salt
* Copy parse doc
* Use media duration for accurate sample detection
Enhances the `is_sample()` logic to reduce false positives for files
whose names contain "sample" or "proof" but are actually full-length
content.
Integrates the `hachoir` library to parse media files and extract their
duration. Files are only considered samples if their duration is less
than or equal to `SAMPLE_MAX_DURATION` (default 2 minutes).
If media duration cannot be determined (e.g., not a media file or
parsing fails), the detection falls back to the name-based approach.
* Refactor `is_sample` function to accept single path argument
* Extend Cleanup List with filename and path pattern matching
Previously limited to cleaning up files based only on their extensions, the 'Cleanup List' now supports more flexible matching. Users can specify:
- Exact filenames (e.g., `Thumbs.db`)
- Wildcard filename patterns (e.g., `*.tmp`, `cleanup.*`)
- Relative path patterns (e.g., `images/*`, `*/test.jpg`)
This enhancement provides greater control and flexibility for automated post-processing cleanup.
* Remove UI delays
* Everything using set_config, set_platform, or pyfakefs
* tmp_path
* Scroll to top seems unnecessary but make it wait until it's done
* Remove sleep from test_rss_basic_flow
* Remove sleeps from test_daemonizing
* Define pytest markers
* Reduce sleep in test_queue_repair
* Remove sleeps from clean_cache_dir
* Suppress failures to connect during startup
* Reduce cache sleeps
* Reduce sleeps checking app started
* Reduce sleep removing cache dir
* replace series/date/movie sorters with a generic sorter
* fix test_eval_sort on windoze
* unbreak and de-uglify the fix
* add special setting for season pack sorting
* remove unused import
* replace series/date/movie sorters with a generic sorter
* fix test_eval_sort on windoze
* unbreak and de-uglify the fix
* add special setting for season pack sorting
* remove unused import
* correct type for sort_type entries
* standardize ui
* add visual hints for drag-n-drop
* move presets directly below sort string field
* replace hex with ascii letters to avoid random occurences of (cd|e)[0-9]+
* Some styling things
---------
Co-authored-by: Safihre <safihre@sabnzbd.org>
Michael Nightingale