Move the rendering of the script log "More" link from the backend's `stage_log` content to the frontend. The backend now provides a `has_script_log` flag, allowing the frontend to dynamically display the link and fetch the script log on demand.
Includes a database migration to clean up old embedded links in existing `stage_log` entries. This improves separation of concerns, simplifies `stage_log` data, and allows for more flexible frontend rendering.
Previous HTML stripping could allow malformed or unclosed tags to pass through, potentially leading to Cross-Site Scripting (XSS) vulnerabilities if completed by the browser.
This change updates the regex to aggressively remove all HTML tags, even incomplete ones. The `knockout-extensions.js` code also now universally escapes HTML, removing a potential bypass. New tests confirm this behavior.
On Windows, creating symbolic links requires either an elevated shell or Developer Mode. This introduces a runtime check to verify if symlinks can be created. Tests that rely on symlink creation are now skipped if this check fails, preventing unnecessary test failures in environments without the required permissions.
These tests rely on a specific behavior of '..' in paths involving symbolic links, which differs on Windows where '..' is collapsed before the filesystem resolves links.
Previously, UI access for login-bypassed clients relied on issuing a stateless "anonymous session" cookie to enable CSRF protection. This commit removes that cookie mechanism.
Instead, when the login is bypassed (e.g., for local clients without credentials), the CSRF token is now bound to a stable, internal "anonymous" identity. This maintains robust CSRF protection for browser interactions without requiring an explicit session cookie, simplifying overall session management.
* Add a session store that keeps web-UI logins in the admin folder
* Authenticate the web interface with session cookies and CSRF tokens
* Stop embedding the apikey in the web interface
* Cover the session, CSRF and apikey changes
The `mode` parameter in API calls enables specific behaviors and key bypasses (e.g., 'version', 'auth', NZB-key). Previously, this logic was applied universally by `check_apikey`. This change ensures such `mode`-based logic is only honored when accessing the dedicated `/api` endpoint, preventing potential API key bypasses on other web UI routes.
Prevent directory creation in `renamer` to overwrite `__ADMIN__`
Check the whole new path for `__ADMIN__` in `renamer`
Drop `__ADMIN__` from names that are allowed to have sub-directories
Resolve the path before checking it against `__ADMIN__`
* Add SafeUnpickler to guard against pickle-attacks
* Harden pickle unpickler with explicit allowlist
The previous `SafeUnpickler` allowed any class from `sabnzbd.*` by wildcard, which could still enable deserialization attacks if a "gadget class" (e.g., with a malicious `__del__` method) was present within our own package.
This commit renames the class to `RestrictedUnpickler` and changes its logic to only allow classes explicitly defined in `_SAFE_GLOBALS`. This significantly enhances security by preventing the unpickling of any unlisted classes, including those from within `sabnzbd`. Adds `os.stat_result` and `sabnzbd.nzb.*` classes to the allowlist for compatibility.
* Use sabctools.FileWriter for article writes
* Keep the FileWriter open on the NzbFile, bounded by an LRU
* Stream decoded articles straight to their file
* Fail the article, not the connection, when a streaming write fails
* Pause on a disk error from a streamed write
A full disk cannot be fixed by fetching the article again, so mirror the assembler
and pause instead of spending its retries and failing the job as incomplete.
* Treat an exhausted quota as a full disk
* Use sabctools 9.7.0
* Refuse unbracketed IPv6 addresses in the Host header
* Bracket the IPv6 client address logged with warnings
* Decide the Secure cookie attribute from the request scope
* Reproduce bug
* Let sanitize_filename keep par2 sub-directories
Par2 sets can store their files inside a folder, always using a forward slash as
separator no matter which platform created the set. We replaced that slash with an
underscore, so the name could never match what par2 called the file.
With allow_subdirs the separator is kept and every part is sanitized on its own.
The result stays local to the folder it is used in: empty parts, "." and ".." are
dropped, so neither a leading slash nor any amount of traversal can point outside.
* Write files into the sub-directory par2 names them in
The name from the par2 pack is what we already rename files to, but the separator
was stripped so the file landed flat with an underscore. Post-processing then had
to move it into place afterwards, which is how the volumes ended up somewhere the
unpacker did not look.
Assemble straight into the sub-directory instead. get_unique_filepath() creates the
folder and returns the name relative to the download folder, so nzf.filename now
holds that relative name. analyze_rar_filename() takes the basename, so a setname
still matches what rar_unpack() derives from a path with setname_from_path().
* Match existing files in sub-directories on retry
check_existing_files() listed the job folder without descending into it, so on a
retry the volumes par2 put in a folder of their own were never matched and got
downloaded again.
List the sub-directories too, skipping the admin folder, and match on the name
relative to the job folder. Rename bookkeeping is normalized to the platform
separator, par2 always reports a forward slash.
* Find rar sets in the job's sub-directories
* Wait for the direct unpacker to finish
* Update nzf.setname and nzf.vol after deobfuscating names
* Fix double extraction
* Ruff
* Only purge unconfigured rss feeds at startup
* Only perform remove obsolete actions if the feed was read
* Update seen_at so downloaded items are not removed while still in the feed
* Increase hardcoded retention to 7 days
Jobs are listed in the history output as soon as they enter the
post-processing queue, and a stopped job has its status set to Failed at
the very start of process_job. The duplicate alternative is only released
near the end of it, so waiting for the job to show up in the history as
failed does not guarantee the alternative was already released. The gap
is tiny when running serially, but wide enough to fail the test under the
load of pytest-xdist. Wait for the label to be dropped instead.
Also move the helpers shared by both adding-NZB test classes into a base
class in testhelper, so the clean tests no longer have to import the
other test module to copy its methods.
* Migrate web interface from CherryPy to Uvicorn/Starlette
Squashed rebase of feature/uvicorn (34 commits) onto develop, reconciled
with ~3 months of intervening develop changes.
Replaces the CherryPy webserver and request handling with Uvicorn/Starlette
across the API, web interface, RSS, config pages and related modules.
Reconciliation with develop during the rebase:
- api.py: kept develop's security/behaviour fixes (orphan path-traversal
guard, expanded log redaction incl. host_whitelist and
remote_label_replacement, get_dconfig single-return, get_retryable_jobs,
connections default, translated NNTP test errors) on top of the Starlette
request/response rewrite.
- interface.py: ported the RSS route handlers to develop's DB-backed
RSSRepository API (process_feed, rss_repository / find_job_by_url /
clear_feed / clear_downloaded / flag_downloaded).
- misc.py: kept develop's hachoir-based get_media_duration.
- requirements.txt: dropped the CherryPy stack, adopted develop's newer pins.
Also applied ruff --fix (PEP 604 unions, builtin generics) to align with
develop's lint config.
Verified: ruff check, black --check, and the affected test suites
(9413 passed, 1 skipped) all pass.
* Update starlette/uvicorn versions
* Fix race issues in global rss state
* Fix test race in server shutdown
The uvicorn migration turned /shutdown (and the shutdown API) into fire-and-forget: it spawned shutdown_program() in a background thread and replied immediately, whereas develop ran it synchronously and only replied once halt() had persisted all state. Because the module-scoped test teardown doesn't wait for the process to exit, the next module's clean_cache_dir wiped the shared cache dir (and reused the fixed port) while the previous instance was still saving state and holding the port — producing the three intermittent failures (deleted sabnzbd.log → "File log disabled or not found"; un-persisted [sorters] → KeyError; stale instance → missing wizard .quoteBlock).
* Fix robots and description, add favicon
* Remove remains of http basic auth
* Setup Starlette once configuration is available, fix static file relative cwd and url_base config
* abort_and_show_error when webserver fails to start
* Guard stopping webserver that never started
* Delegate XFF handling to ProxyHeadersMiddleware
* Merged params at request.state.params instead of modifying private apis
* Both shutdown routes share implementation and do not block event loop
* Run sync handlers via run_in_threadpool and facilitate eventual migration to async
* Pool database connections
* Online backup of database due to WAL changes
* Fix exception on None request.client (test clients or unix sockets)
* Fix flakey tests due to process not fully shutting down
* Restore X-Frame-Options behaviour via middleware
* Fix set_config_default with multiple keywords
* Remove broken logging call
* Restore api logging functionality
* Cache-Control: no-store
* Login only via POST
* Remove 401 (basic-auth) and add 404 handling via redirect
* Fix crash when shutdown not an int
* Use BaseRedirectResponse helper
* Remove trailing slashes from wizard routes
* URL helper, absolute URLs everywhere, fixes issues with nested navigation
* Fix scheduler adding multiple daysofweek
* Restore CherryPy api behaviour merging body with query params (body wins)
* Clearer documentation of get_request_params and request_params
* First stage supporting gradual api async
* Fix rss ajax consuming flash
* Restore access log functionality
* Hostname check in middleware
* Request logging in middleware
* Param parsing in middleware
* Security checks in middleware
* secured_expose is now purely route registration
* Lookup api handler once per request
* Fix flakey alert dialogs
* Trigger restart via BackgroundTask
* Restore CherryPy first param wins and get/post consistency
* Remove dead code
* Secure cookies based on protocol the client used
* Fix various issues with port_is_free
1. port_is_free answered the wrong question. It connect-probed ("is something answering?") rather than bind-probed ("can I bind?"). A port could report free and then kill startup at uvicorn's bind().
2. The bind-all remap crossed address families. :: was mapped to 127.0.0.1, probing IPv4 for an IPv6 bind — a regression against portend, which maps :: → ::1.
3. The call sites passed the wrong host. browserhost is a client-reachable address; the thing that has to be bindable is web_host.
4. Errors were swallowed. A bare except OSError hid gaierror, so an unresolvable host reported "free".
5. find_free_port had a port-0 trap. Under a bind-probe, currentport=0 always succeeds and returned 0 — the old failure sentinel. Now guarded, and None instead of 0.
6. Ports 80/443 were misdiagnosed. EACCES was folded into "occupied", producing ten futile probes and a panic claiming another program held the port. PermissionError now propagates to a dedicated panic explaining the actual remedies.
7. The tests were largely tautological. Three tests covering one branch, an IPv6 test with no IPv6 in it, a timeout test that never engaged the timeout, TOCTOU-prone fixed-range probes, no SO_REUSEADDR on the helper listener, and nothing asserting the property that matters — that "free" implies bindable.
8. A portability bug I introduced, then fixed. I'd baked Linux SO_REUSEADDR overlap semantics into four assertions; macOS differs. Now platform-aware, with the IPv6 regression re-covered by checking the socket family directly.
* Claim the bind address for uvicorn on startup, resolves "49" in err handling from cherrypy
* Rename function BaseRedirectResponse to base_redirect_response
* Restore error response on change web directory
* Add missing typings
* Fix return type of retry job for future types
* A better fix for xdist compatibility - test overwrote db_path
* Secure session cookies (rss flash)
* Inline or remove some functions
* Retry job futuretype behaviour
* Sneak a worksteal fix in
* Test and fix retry_job futuretype behaviour
* Speed up tests with pytest-xdist
* Ignore request failures when the server disappeared
* Fix tests which modify module globals
* Fix lang change leaking
* Fix transaction compilation order breaking tests
* Always load a clean config and fix related tests
* No module globals
* Fix other test classes assigning module level by making calls
* Replace from sabnzbd.cfg import so monkeypatch works
* More global poisoning
* Fix dependant tests which worksteal breaks
* More global vars
* Drop loadscope until more failures are resolved
* Due to how pytest-xdist is implemented, the -s/--capture=no option does not work
* monkeypatch db_path
* Also patch startup_done
* loadscope
* Apply memory limits from cgroups
* Apply cgroup limits to article cache
* Fix this bug
* _physical_memory return None when undetermined
* Mark platform on test group
* Remove MemoryInfo dataclass
* Cap memory with reservation
* Keep the slightly broken 32 vs 64 bit code
* 512 *MEBI because the numbers look nicer
* Fix flaky sort test by comparing at the same resolution
* Fix consistency of values printed at reduced precision
* Replace the float-string roundtrip with rounding and adjust to next unit if required
* Comment styling
* feat: add 'Sort by Remaining Size' queue sort option
Add a new 'remaining_bytes' sort field that sorts the queue by absolute
remaining bytes (bytes - bytes_tried), complementing the existing
'remaining' sort which uses percentage downloaded and the 'size' sort
which uses total bytes.
Changes:
- nzbqueue.py: add remaining_bytes sort field, fix update_sort_order()
to parse field+direction from config instead of hardcoding 'remaining'
- skintext.py: add Glitter-sortRemainingBytesAsc/Desc translation strings,
update auto_sort explanation text
- Glitter UI: add dropdown entries and JS handlers for both directions
- Config: add auto_sort options for remaining_bytes asc/desc
- Tests: parametrize remaining_bytes asc/desc against sizeleft slot field
* style: wrap long string for Black formatting
* fix: limit remaining-size sorting to ascending
---------
Co-authored-by: QuixThe2nd <QuixThe2nd@users.noreply.github.com>
* RSS age rule
* Support years (y) and months (mo) and do not approximate durations in seconds
* Better handling of entries with no age
* Formatting
* Allow >=, =>, <=, and =< aliases
* Implement feedback
* Fix existing issue with From SxxEyy in default row
* Test unitless ages
Safihre