mirror of
https://github.com/sabnzbd/sabnzbd.git
synced 2026-09-08 11:44:34 -04:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e39d1e323a | ||
|
|
091b9cfafd | ||
|
|
08aaca09df | ||
|
|
f99fb0202f | ||
|
|
5636283f31 | ||
|
|
0ccb23b9d3 | ||
|
|
ee9a6c0b6c | ||
|
|
e3308e61b2 | ||
|
|
53e6ae479d | ||
|
|
a2b3ed3de1 | ||
|
|
fb8b181205 | ||
|
|
45873d4cbb | ||
|
|
f32847229a | ||
|
|
72630c30ae | ||
|
|
990dce2232 | ||
|
|
99447d7a62 | ||
|
|
6455a44ed4 | ||
|
|
cb05869e0c | ||
|
|
936b2db7df |
No files matched your search
@@ -104,7 +104,8 @@ jobs:
|
||||
# We need the official Python, because the GA ones only support newer macOS versions
|
||||
# The deployment target is picked up by the Python build tools automatically
|
||||
# If updated, make sure to also set LSMinimumSystemVersion in SABnzbd.spec
|
||||
PYTHON_VERSION: "3.14.4"
|
||||
# renovate: datasource=python-version depName=python
|
||||
PYTHON_VERSION: "3.14.5"
|
||||
MACOSX_DEPLOYMENT_TARGET: "10.15"
|
||||
# We need to force compile for universal2 support
|
||||
CFLAGS: -arch x86_64 -arch arm64
|
||||
|
||||
+40
-11
@@ -1,8 +1,37 @@
|
||||
Release Notes - SABnzbd 5.0.0 Release Candidate 3
|
||||
Release Notes - SABnzbd 5.0.4 Release Candidate 1
|
||||
=========================================================
|
||||
|
||||
This is the third (and hopefully final) release candidate of version 5.0.
|
||||
Please note the breaking changes described below!
|
||||
In this fifth major release of SABnzbd, we included:
|
||||
|
||||
* Smarter (and potentially faster) downloads with NNTP Pipelining and Direct Write.
|
||||
* Reliability gains from a reworked cache and safer processing.
|
||||
* A significant number of bug fixes and platform updates.
|
||||
|
||||
## Features and bug fixes in 5.0.4
|
||||
|
||||
* RSS feed modifications no longe require clicking `Apply Filters`.
|
||||
* Fixed RSS priority handling.
|
||||
* Server Test could block the queue.
|
||||
* Use reduced progress bar width in Compact layout.
|
||||
* Replaced unreliable directory access check.
|
||||
* Prevent crash in idle-jobs check.
|
||||
* Jobs with invalid date would show as `Fetching`.
|
||||
* Windows and macOS: Updated Python to 3.14.5 and Unrar to 7.22.
|
||||
|
||||
## Features and bug fixes in 5.0.1, 5.0.2 and 5.0.3
|
||||
|
||||
* Fetching NZB's failed on new installations.
|
||||
* Queue entries from older versions now load correctly after upgrading.
|
||||
* `nzo_id` now uses GUIDs to prevent duplicates.
|
||||
* Updating to 5.0 now succeeds even when duplicate `nzo_id` values exist.
|
||||
* URL fetching no longer crashes when duplicate jobs are present.
|
||||
* Default RSS priority is now applied correctly.
|
||||
* Duplicate filenames no longer cause queue stalls.
|
||||
* IPv6 binding now works correctly for the Web interface.
|
||||
* The `On queue finish` action now executes reliably.
|
||||
* Saving the post-processing queue no longer throws errors.
|
||||
* Directory creation now handles existing directories gracefully.
|
||||
* Windows and macOS: Updated Unrar to 7.21 and 7zip to 26.01.
|
||||
|
||||
## New features in 5.0.0
|
||||
|
||||
@@ -10,6 +39,7 @@ Please note the breaking changes described below!
|
||||
requests, significantly improving speeds on high-latency connections.
|
||||
Read more here: https://sabnzbd.org/wiki/advanced/nntp-pipelining
|
||||
New servers will have `Articles per request` set to `2` by default.
|
||||
Existing servers need to be manually changed to use NNTP Pipelining.
|
||||
* Implemented Direct Write to optimize assembly of downloaded files.
|
||||
Read more here: https://sabnzbd.org/wiki/advanced/direct-write
|
||||
* Post-processing scripts will always be executed, even for failed jobs.
|
||||
@@ -17,16 +47,16 @@ Please note the breaking changes described below!
|
||||
* Removed Special setting `empty_postproc` as it is no longer needed.
|
||||
* Complete redesign of article cache.
|
||||
* Improved disk speed measurement in Status window.
|
||||
* Reduce delays between jobs during post-processing.
|
||||
* Improve handling of disks getting full.
|
||||
* Diskspace check will now included Category-specific folders.
|
||||
* If a download only has `.nzb` files inside, the new downloads
|
||||
* Reduced delays between jobs during post-processing.
|
||||
* Improved handling of disks getting full.
|
||||
* Diskspace check will now include Category-specific folders.
|
||||
* If a download only contains `.nzb` files, the new downloads
|
||||
will include the name of the original download.
|
||||
* No longer show tracebacks in the browser, only in the logs.
|
||||
* Dropped support for Python 3.8.
|
||||
* Windows: Added Windows ARM (portable) release.
|
||||
* Windows and macOS: Added HTML version of the Release Notes.
|
||||
* Windows and macOS: Update Python to 3.14.4, Unrar to 7.20,
|
||||
* Windows and macOS: Updated Python to 3.14.4, Unrar to 7.20,
|
||||
par2cmdline-turbo to 1.4 and 7zip to 26.00.
|
||||
|
||||
## Bug fixes in 5.0.0
|
||||
@@ -39,11 +69,11 @@ Please note the breaking changes described below!
|
||||
* Handle passwords that exceed Unrar's password limit.
|
||||
* Aborted Direct Unpack could result in no files being unpacked.
|
||||
* Encrypted RAR detection could sometimes fail.
|
||||
* Unwanted Extension check was unreliable on passworded downloads.
|
||||
* Unwanted extension check was unreliable on passworded downloads.
|
||||
* No longer exclusively lock all directory operations.
|
||||
* Improved handling of non-NFC unicode filenames.
|
||||
* Sorting of files inside jobs was inconsistent.
|
||||
* Linux: No longer break test-data in release package.
|
||||
* Linux: No longer break test-data in the release package.
|
||||
* Windows: Tray icon disappears after Explorer restart.
|
||||
* macOS: Slow to start on some network setups.
|
||||
|
||||
@@ -52,7 +82,6 @@ Please note the breaking changes described below!
|
||||
* Default `Minimum Free Space for Temporary Download Folder` set to `500M`.
|
||||
* Enabled `verify_xff_header` setting by default.
|
||||
|
||||
|
||||
## Upgrade notices
|
||||
|
||||
* You can directly upgrade from version 3.0.0 and newer.
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
<tr>
|
||||
<th style="width: 25px;"></th>
|
||||
<th></th>
|
||||
<th class="table-status-header" data-bind="css: { 'table-header-status-smaller' : extraHistoryColumns().length }"></th>
|
||||
<th class="table-status-header" data-bind="css: { 'table-header-status-smaller' : extraHistoryColumns().length || displayCompact() }"></th>
|
||||
<!-- ko foreach: extraHistoryColumns -->
|
||||
<th class="table-header-extra"></th>
|
||||
<!-- /ko -->
|
||||
|
||||
@@ -65,7 +65,7 @@
|
||||
<!-- ko foreach: extraQueueColumns -->
|
||||
<th class="table-header-extra"></th>
|
||||
<!-- /ko -->
|
||||
<th class="table-header-progress" data-bind="css: { 'table-header-progress-smaller' : extraQueueColumns().length }"></th>
|
||||
<th class="table-header-progress" data-bind="css: { 'table-header-progress-smaller' : extraQueueColumns().length || displayCompact() }"></th>
|
||||
<th style="width: 85px;"></th>
|
||||
<th style="width: 60px;"></th>
|
||||
</tr>
|
||||
|
||||
@@ -549,7 +549,7 @@ function QueueModel(parent, data) {
|
||||
self.index = ko.observable(data.index);
|
||||
self.status = ko.observable(data.status);
|
||||
self.labels = ko.observableArray(data.labels);
|
||||
self.isGrabbing = ko.observable(data.status === 'Grabbing' || data.avg_age === '-')
|
||||
self.isGrabbing = ko.observable(data.status === 'Grabbing' || !parseFloat(data.mb)) // Paused URL-grab jobs report status 'Paused', not 'Grabbing'
|
||||
self.isFetchingBlocks = data.status === 'Fetching' || data.priority === 'Repair' // No need to update
|
||||
self.totalMB = ko.observable(parseFloat(data.mb));
|
||||
self.remainingMB = ko.observable(parseFloat(data.mbleft))
|
||||
@@ -669,7 +669,7 @@ function QueueModel(parent, data) {
|
||||
self.password(data.password);
|
||||
self.index(data.index);
|
||||
self.status(data.status)
|
||||
self.isGrabbing(data.status === 'Grabbing' || data.avg_age === '-')
|
||||
self.isGrabbing(data.status === 'Grabbing' || !parseFloat(data.mb))
|
||||
self.totalMB(parseFloat(data.mb));
|
||||
self.remainingMB(parseFloat(data.mbleft));
|
||||
self.missingMB(parseFloat(data.mbmissing))
|
||||
|
||||
@@ -42,6 +42,18 @@
|
||||
<url type="faq">https://sabnzbd.org/wiki/faq</url>
|
||||
<url type="contact">https://sabnzbd.org/live-chat.html</url>
|
||||
<releases>
|
||||
<release version="5.0.4" date="2026-06-11" type="stable">
|
||||
<url type="details">https://github.com/sabnzbd/sabnzbd/releases/tag/5.0.4</url>
|
||||
</release>
|
||||
<release version="5.0.3" date="2026-05-15" type="stable">
|
||||
<url type="details">https://github.com/sabnzbd/sabnzbd/releases/tag/5.0.3</url>
|
||||
</release>
|
||||
<release version="5.0.2" date="2026-05-14" type="stable">
|
||||
<url type="details">https://github.com/sabnzbd/sabnzbd/releases/tag/5.0.2</url>
|
||||
</release>
|
||||
<release version="5.0.1" date="2026-05-01" type="stable">
|
||||
<url type="details">https://github.com/sabnzbd/sabnzbd/releases/tag/5.0.1</url>
|
||||
</release>
|
||||
<release version="5.0.0" date="2026-05-01" type="stable">
|
||||
<url type="details">https://github.com/sabnzbd/sabnzbd/releases/tag/5.0.0</url>
|
||||
</release>
|
||||
|
||||
+1
-1
@@ -521,7 +521,7 @@ def delayed_startup_actions():
|
||||
|
||||
# Do an extra purge of the history on startup to ensure timely removal on systems that
|
||||
# aren't on 24/7 and typically don't benefit from the daily scheduled call at midnight
|
||||
database.scheduled_history_purge()
|
||||
sabnzbd.database.scheduled_history_purge()
|
||||
|
||||
# Start SSDP and Bonjour if SABnzbd isn't listening on localhost only
|
||||
if sabnzbd.cfg.enable_broadcast() and not misc.is_localhost(cfg.web_host()):
|
||||
|
||||
@@ -380,7 +380,7 @@ class Downloader(Thread):
|
||||
@synchronized(DOWNLOADER_LOCK)
|
||||
def add_socket(self, nw: NewsWrapper):
|
||||
"""Add a socket to be watched for read or write availability"""
|
||||
if nw.nntp:
|
||||
if nw.nntp and not nw.blocking:
|
||||
nw.server.idle_threads.discard(nw)
|
||||
nw.server.busy_threads.add(nw)
|
||||
try:
|
||||
@@ -402,7 +402,7 @@ class Downloader(Thread):
|
||||
@synchronized(DOWNLOADER_LOCK)
|
||||
def remove_socket(self, nw: NewsWrapper):
|
||||
"""Remove a socket to be watched"""
|
||||
if nw.nntp:
|
||||
if nw.nntp and not nw.blocking:
|
||||
nw.server.busy_threads.discard(nw)
|
||||
nw.server.idle_threads.add(nw)
|
||||
nw.timeout = None
|
||||
|
||||
+20
-14
@@ -384,30 +384,36 @@ def real_path(loc: str, path: str) -> str:
|
||||
|
||||
|
||||
def create_real_path(
|
||||
name: str, loc: str, path: str, apply_permissions: bool = False, writable: bool = True
|
||||
name: str,
|
||||
loc: str,
|
||||
path: str,
|
||||
apply_permissions: bool = False,
|
||||
writable: bool = True,
|
||||
) -> tuple[bool, str, Optional[str]]:
|
||||
"""When 'path' is relative, create join of 'loc' and 'path'
|
||||
When 'path' is absolute, create normalized path
|
||||
'name' is used for logging.
|
||||
Optional 'umask' will be applied.
|
||||
Optional permissions will be applied.
|
||||
'writable' means that an existing folder should be writable
|
||||
Returns ('success', 'full path', 'error_msg')
|
||||
"""
|
||||
if path:
|
||||
my_dir = real_path(loc, path)
|
||||
if not os.path.exists(my_dir):
|
||||
if not create_all_dirs(my_dir, apply_permissions):
|
||||
msg = T("Cannot create directory %s") % clip_path(my_dir)
|
||||
real_dir = real_path(loc, path)
|
||||
if not os.path.exists(real_dir):
|
||||
if not create_all_dirs(real_dir, apply_permissions):
|
||||
msg = T("Cannot create directory %s") % clip_path(real_dir)
|
||||
logging.error(msg)
|
||||
return False, my_dir, msg
|
||||
return False, real_dir, msg
|
||||
|
||||
checks = (os.W_OK + os.R_OK) if writable else os.R_OK
|
||||
if os.access(my_dir, checks):
|
||||
return True, my_dir, None
|
||||
else:
|
||||
msg = T("%s directory: %s error accessing") % (name, clip_path(my_dir))
|
||||
# os.access() uses the real UID/GID via the access(2) syscall, which can
|
||||
# return false negatives, for example on NFS mounts with UID mapping or root-squashing.
|
||||
# Verify access by performing actual I/O operations (read/write) instead.
|
||||
if not os.path.isdir(real_dir) or (writable and not directory_is_writable_with_file(real_dir, "sab_test.txt")):
|
||||
msg = T("%s directory: %s error accessing") % (name, clip_path(real_dir))
|
||||
logging.error(msg)
|
||||
return False, my_dir, msg
|
||||
return False, real_dir, msg
|
||||
else:
|
||||
return True, real_dir, None
|
||||
else:
|
||||
return False, path, None
|
||||
|
||||
@@ -1229,7 +1235,7 @@ def directory_is_writable_with_file(mydir: str, myfilename: str) -> bool:
|
||||
return False
|
||||
try:
|
||||
with open(filename, "w") as f:
|
||||
f.write("Some random content")
|
||||
f.write("Some random content to test directory and file permissions")
|
||||
os.remove(filename)
|
||||
return True
|
||||
except Exception:
|
||||
|
||||
@@ -1268,7 +1268,7 @@ class ConfigRss:
|
||||
self.__refresh_force = False
|
||||
self.__refresh_ignore = False
|
||||
if self.__evaluate:
|
||||
msg = sabnzbd.RSSReader.run_feed(
|
||||
msg = sabnzbd.RSSReader.process_feed(
|
||||
active_feed,
|
||||
download=self.__refresh_download,
|
||||
force=self.__refresh_force,
|
||||
@@ -1514,7 +1514,7 @@ class ConfigRss:
|
||||
"""Download NZB from provider (Download button)"""
|
||||
feed = kwargs.get("feed")
|
||||
url = kwargs.get("url")
|
||||
if att := sabnzbd.RSSReader.lookup_url(feed, url):
|
||||
if att := sabnzbd.RSSReader.find_job_by_url(feed, url):
|
||||
nzbname = kwargs.get("nzbname")
|
||||
pp = att.get("pp")
|
||||
cat = att.get("cat")
|
||||
@@ -1950,7 +1950,7 @@ def GetRssLog(feed):
|
||||
job["title"] = job["title"]
|
||||
job["skip"] = "*" * int(job.get("status", "").endswith("*"))
|
||||
# These fields could be empty
|
||||
job["cat"] = job.get("cat", "")
|
||||
job["cat"] = job.get("cat", "") or T("Default")
|
||||
job["size"] = job.get("size", "")
|
||||
job["infourl"] = job.get("infourl", "")
|
||||
|
||||
@@ -1987,7 +1987,7 @@ def GetRssLog(feed):
|
||||
|
||||
return job
|
||||
|
||||
jobs = sabnzbd.RSSReader.show_result(feed).values()
|
||||
jobs = sabnzbd.RSSReader.get_feed_jobs(feed).values()
|
||||
good, bad, done = ([], [], [])
|
||||
for job in jobs:
|
||||
if job["status"][0] == "G":
|
||||
|
||||
+1
-1
@@ -751,7 +751,7 @@ def get_memory() -> int:
|
||||
return mem_info["TotalPhys"]
|
||||
elif sabnzbd.MACOS:
|
||||
# Use system-call to extract total memory on macOS
|
||||
system_output = run_command(["sysctl", "-n", "hw.memsize"]).strip()
|
||||
return int(run_command(["sysctl", "-n", "hw.memsize"]).strip())
|
||||
else:
|
||||
try:
|
||||
with open("/proc/meminfo") as f:
|
||||
|
||||
@@ -371,7 +371,7 @@ class NewsWrapper:
|
||||
server = self.server
|
||||
|
||||
# Do not pipeline requests until authentication is completed (connected)
|
||||
if self.ready or not self._response_queue:
|
||||
if not self.blocking and (self.ready or not self._response_queue):
|
||||
server_ready = (
|
||||
server.active
|
||||
and not server.restart
|
||||
|
||||
+16
-9
@@ -29,7 +29,7 @@ from typing import Union, Optional
|
||||
import sabnzbd
|
||||
from sabnzbd.nzb import Article, NzbObject
|
||||
from sabnzbd.misc import exit_sab, cat_to_opts, int_conv, caller_name, safe_lower, duplicate_warning
|
||||
from sabnzbd.filesystem import get_admin_path, remove_all, globber_full, remove_file, is_valid_script
|
||||
from sabnzbd.filesystem import get_admin_path, remove_all, globber_full, remove_file, is_valid_script, create_all_dirs
|
||||
from sabnzbd.nzbparser import process_single_nzb
|
||||
from sabnzbd.panic import panic_queue
|
||||
from sabnzbd.decorators import NzbQueueLocker
|
||||
@@ -62,6 +62,11 @@ class NzbQueue:
|
||||
self.__nzo_list: list[NzbObject] = []
|
||||
self.__nzo_table: dict[str, NzbObject] = {}
|
||||
|
||||
# Make sure the future-dir exists so the URLGrabber can download files
|
||||
# This will also create admin_dir if it doesn't exist already by calling get_path on it
|
||||
if not os.path.exists(future_dir := os.path.join(cfg.admin_dir.get_path(), FUTURE_Q_FOLDER)):
|
||||
create_all_dirs(future_dir)
|
||||
|
||||
def read_queue(self, repair: int):
|
||||
"""Read queue from disk, supporting repair modes
|
||||
0 = no repairs
|
||||
@@ -901,15 +906,17 @@ class NzbQueue:
|
||||
break
|
||||
|
||||
if nzf.all_servers_in_try_list(active_servers):
|
||||
# Check for articles where all active servers have already been tried
|
||||
# Collect articles where all active servers have already been tried.
|
||||
# Snapshot under lock to avoid RuntimeError from dict mutation when
|
||||
# register_article later calls nzf.remove_article (see #3431).
|
||||
with nzf.lock:
|
||||
for article in nzf.articles:
|
||||
if article.all_servers_in_try_list(active_servers):
|
||||
logging.debug(
|
||||
"Removing article %s with bad trylist in file %s", article, nzf.filename
|
||||
)
|
||||
nzo.increase_bad_articles_counter("missing_articles")
|
||||
sabnzbd.NzbQueue.register_article(article, success=False)
|
||||
articles_to_remove = [
|
||||
article for article in nzf.articles if article.all_servers_in_try_list(active_servers)
|
||||
]
|
||||
for article in articles_to_remove:
|
||||
logging.debug("Removing article %s with bad trylist in file %s", article, nzf.filename)
|
||||
nzo.increase_bad_articles_counter("missing_articles")
|
||||
sabnzbd.NzbQueue.register_article(article, success=False)
|
||||
|
||||
if not nzf.assembled and not nzf.articles:
|
||||
logging.debug("Not assembled but no remaining articles for file %s", nzf.filename)
|
||||
|
||||
+186
-242
@@ -26,7 +26,7 @@ import datetime
|
||||
import threading
|
||||
import urllib.parse
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Union, Optional
|
||||
from typing import Union, Optional, Any
|
||||
|
||||
import sabnzbd
|
||||
from sabnzbd.constants import RSS_FILE_NAME, DEFAULT_PRIORITY
|
||||
@@ -120,8 +120,8 @@ class NormalisedEntry:
|
||||
|
||||
# Maybe the newznab also provided SxxExx info
|
||||
try:
|
||||
season = re.findall(r"\d+", entry["newznab"]["season"])[0]
|
||||
episode = re.findall(r"\d+", entry["newznab"]["episode"])[0]
|
||||
season = int_conv(re.findall(r"\d+", entry["newznab"]["season"])[0])
|
||||
episode = int_conv(re.findall(r"\d+", entry["newznab"]["episode"])[0])
|
||||
except (KeyError, IndexError):
|
||||
season = episode = 0
|
||||
|
||||
@@ -177,7 +177,7 @@ class NormalisedEntry:
|
||||
episode=job.get("episode", 0),
|
||||
)
|
||||
|
||||
def is_duplicate(self, jobs: dict[str, dict]) -> bool:
|
||||
def is_duplicate(self, jobs: dict[str, dict[str, Any]]) -> bool:
|
||||
"""Check if a job with the same title and size already exists in another feed"""
|
||||
for job_link, job in jobs.items():
|
||||
# Allow 5% size deviation because indexers might have small differences for same release
|
||||
@@ -269,10 +269,10 @@ class FeedRule:
|
||||
return False
|
||||
|
||||
# Episode / season rules
|
||||
elif self.type == "F" and not self.ep_match(season, episode, self.regex):
|
||||
elif self.type == "F" and not self.episode_matches(season, episode, self.regex):
|
||||
logging.debug("Filter rejected on rule %d (episode too early)", rule_index)
|
||||
return False
|
||||
elif self.type == "S" and self.ep_match(season, episode, self.regex, title):
|
||||
elif self.type == "S" and self.episode_matches(season, episode, self.regex, title):
|
||||
logging.debug("Filter matched on rule %d (show SxxEyy match)", rule_index)
|
||||
return True
|
||||
|
||||
@@ -296,7 +296,7 @@ class FeedRule:
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def ep_match(season: int, episode: int, expr: str, title: Optional[str] = None):
|
||||
def episode_matches(season: int, episode: int, expr: str, title: Optional[str] = None):
|
||||
"""Return True if season, episode is at or above expected
|
||||
Optionally `title` can be matched
|
||||
"""
|
||||
@@ -374,36 +374,24 @@ class FeedConfig:
|
||||
episode: int,
|
||||
) -> FeedEvaluation:
|
||||
"""Evaluate rules for a single RSS entry."""
|
||||
is_match: bool = False
|
||||
matched_rule: Optional[FeedRule] = None
|
||||
matched_index: int = 0
|
||||
cur_season: int = season
|
||||
cur_episode: int = episode
|
||||
entry_cat = category
|
||||
rule_matched: bool = False
|
||||
last_rule: Optional[FeedRule] = None
|
||||
last_rule_index: int = 0
|
||||
feed_season: int = season
|
||||
feed_episode: int = episode
|
||||
|
||||
# Start from feed defaults for options.
|
||||
my_category: Optional[str] = self.default_category
|
||||
my_pp: Optional[str] = self.default_pp
|
||||
my_script: Optional[str] = self.default_script
|
||||
my_priority: Optional[int] = self.default_priority
|
||||
|
||||
# If there are no rules; return early
|
||||
if not self.rules:
|
||||
return FeedEvaluation(
|
||||
matched=is_match,
|
||||
rule_index=matched_index,
|
||||
season=int_conv(cur_season),
|
||||
episode=int_conv(cur_episode),
|
||||
category=my_category,
|
||||
pp=my_pp,
|
||||
script=my_script,
|
||||
priority=my_priority,
|
||||
)
|
||||
resolved_cat: Optional[str] = self.default_category
|
||||
resolved_pp: Optional[int] = self.default_pp
|
||||
resolved_script: Optional[str] = self.default_script
|
||||
resolved_priority: Optional[int] = self.default_priority
|
||||
|
||||
# Fill in missing season / episode information when F/S rules exist
|
||||
if self.has_type("F", "S") and (not cur_season or not cur_episode):
|
||||
if self.has_type("F", "S") and (not feed_season or not feed_episode):
|
||||
show_analysis = sabnzbd.sorting.BasicAnalyzer(title)
|
||||
cur_season = show_analysis.info.get("season_num")
|
||||
cur_episode = show_analysis.info.get("episode_num")
|
||||
feed_season = int_conv(show_analysis.info.get("season_num"))
|
||||
feed_episode = int_conv(show_analysis.info.get("episode_num"))
|
||||
|
||||
# Match against all filters until a positive or negative match
|
||||
for idx, rule in enumerate(self.rules):
|
||||
@@ -412,76 +400,70 @@ class FeedConfig:
|
||||
|
||||
outcome = rule.matches(
|
||||
title=title,
|
||||
category=category,
|
||||
category=entry_cat,
|
||||
size=size,
|
||||
season=cur_season,
|
||||
episode=cur_episode,
|
||||
season=feed_season,
|
||||
episode=feed_episode,
|
||||
rule_index=idx,
|
||||
)
|
||||
|
||||
if outcome is None:
|
||||
continue
|
||||
|
||||
matched_index = idx
|
||||
is_match = outcome
|
||||
matched_rule = rule if outcome else None
|
||||
last_rule = rule
|
||||
last_rule_index = idx
|
||||
rule_matched = outcome
|
||||
break
|
||||
|
||||
if matched_rule is None:
|
||||
base_category = (
|
||||
cat_convert(category) if category and self.default_category is None else self.default_category
|
||||
)
|
||||
rule_has_category = bool(last_rule and last_rule.category)
|
||||
|
||||
# Category resolution
|
||||
if not rule_matched and self.default_category:
|
||||
effective_category = self.default_category
|
||||
elif rule_matched and rule_has_category:
|
||||
effective_category = last_rule.category
|
||||
elif entry_cat and not self.default_category:
|
||||
effective_category = cat_convert(entry_cat)
|
||||
else:
|
||||
base_category = matched_rule.category or cat_convert(category) or self.default_category
|
||||
effective_category = resolved_cat
|
||||
|
||||
my_category, my_pp, my_script, my_priority = self._resolve_options(
|
||||
base_category=base_category,
|
||||
rule=matched_rule,
|
||||
)
|
||||
|
||||
return FeedEvaluation(
|
||||
matched=is_match,
|
||||
rule_index=matched_index,
|
||||
season=int_conv(cur_season),
|
||||
episode=int_conv(cur_episode),
|
||||
category=my_category,
|
||||
pp=my_pp,
|
||||
script=my_script,
|
||||
priority=my_priority,
|
||||
)
|
||||
|
||||
def _resolve_options(
|
||||
self,
|
||||
*,
|
||||
base_category: Optional[str],
|
||||
rule: Optional[FeedRule],
|
||||
) -> tuple[Optional[str], Optional[int], Optional[str], Optional[int]]:
|
||||
"""Resolve options for a feed rule."""
|
||||
if base_category:
|
||||
cat, cat_pp, cat_script, cat_prio = cat_to_opts(base_category)
|
||||
# Category-derived defaults
|
||||
if effective_category:
|
||||
resolved_cat, cat_pp, cat_script, cat_prio = cat_to_opts(effective_category)
|
||||
cat_pp = _normalise_pp(cat_pp)
|
||||
cat_script = _normalise_str_or_none(cat_script)
|
||||
cat_prio = _normalise_priority(cat_prio)
|
||||
else:
|
||||
cat = cat_pp = cat_script = cat_prio = None
|
||||
resolved_cat = cat_pp = cat_script = cat_prio = None
|
||||
|
||||
pp = first_not_none(
|
||||
rule.pp if rule else None,
|
||||
cat_pp,
|
||||
self.default_pp,
|
||||
)
|
||||
script = first_not_none(
|
||||
rule.script if rule else None,
|
||||
cat_script,
|
||||
self.default_script,
|
||||
)
|
||||
priority = first_not_none(
|
||||
rule.priority if rule else None,
|
||||
cat_prio,
|
||||
self.default_priority,
|
||||
)
|
||||
# PP resolution
|
||||
if last_rule and last_rule.pp is not None:
|
||||
resolved_pp = last_rule.pp
|
||||
elif not (rule_has_category or entry_cat):
|
||||
resolved_pp = cat_pp
|
||||
|
||||
return cat, pp, script, priority
|
||||
# Script resolution
|
||||
if last_rule and last_rule.script is not None:
|
||||
resolved_script = last_rule.script
|
||||
elif not (rule_has_category or entry_cat):
|
||||
resolved_script = cat_script
|
||||
|
||||
# Priority resolution
|
||||
if last_rule and last_rule.priority not in (DEFAULT_PRIORITY, None):
|
||||
resolved_priority = last_rule.priority
|
||||
elif not ((last_rule and last_rule.priority != DEFAULT_PRIORITY) or entry_cat):
|
||||
resolved_priority = cat_prio
|
||||
|
||||
return FeedEvaluation(
|
||||
matched=rule_matched,
|
||||
rule_index=last_rule_index,
|
||||
season=feed_season,
|
||||
episode=feed_episode,
|
||||
category=resolved_cat,
|
||||
pp=resolved_pp,
|
||||
script=resolved_script,
|
||||
priority=resolved_priority,
|
||||
)
|
||||
|
||||
|
||||
class RSSReader:
|
||||
@@ -529,7 +511,7 @@ class RSSReader:
|
||||
self.shutdown = True
|
||||
|
||||
@synchronized(RSS_LOCK)
|
||||
def run_feed(
|
||||
def process_feed(
|
||||
self,
|
||||
feed: str,
|
||||
download: bool = False,
|
||||
@@ -547,93 +529,12 @@ class RSSReader:
|
||||
new_downloads: list[str] = []
|
||||
|
||||
# Configuration
|
||||
uris, filters, first, jobs, config_error = self.configure_rss(feed, ignore_first)
|
||||
if config_error:
|
||||
return config_error
|
||||
|
||||
# Fetch & parse RSS
|
||||
if readout:
|
||||
entries, msg = self.fetch_rss(feed, uris)
|
||||
else:
|
||||
entries, msg = (jobs, "")
|
||||
|
||||
# Error in readout or no new readout
|
||||
if readout and not entries:
|
||||
return msg
|
||||
|
||||
# Normalise entries, evaluate rules and apply side effects
|
||||
for entry in entries:
|
||||
if self.shutdown:
|
||||
return ""
|
||||
|
||||
try:
|
||||
if readout:
|
||||
normalised = NormalisedEntry.from_feed_entry(entry)
|
||||
if not normalised:
|
||||
continue
|
||||
# Skip duplicates across multiple feeds
|
||||
if len(uris) > 1 and self.is_duplicate(normalised, jobs):
|
||||
continue
|
||||
else:
|
||||
normalised = NormalisedEntry.from_job_entry(entry, jobs)
|
||||
except (AttributeError, IndexError):
|
||||
last_uri = uris[-1] if uris else ""
|
||||
logging.info(T("Incompatible feed") + " " + last_uri)
|
||||
logging.info("Traceback: ", exc_info=True)
|
||||
return T("Incompatible feed")
|
||||
|
||||
if not normalised.link:
|
||||
continue
|
||||
|
||||
# Track all valid links so obsolete ones can be cleaned up later
|
||||
new_links.append(normalised.link)
|
||||
|
||||
evaluation, should_download, is_starred = self._evaluate_entry(
|
||||
entry=normalised,
|
||||
jobs=jobs,
|
||||
filters=filters,
|
||||
first=first,
|
||||
download=download,
|
||||
force=force,
|
||||
readout=readout,
|
||||
)
|
||||
if evaluation is None:
|
||||
continue
|
||||
|
||||
downloaded = self._process_entry(
|
||||
feed=feed,
|
||||
entry=normalised,
|
||||
jobs=jobs,
|
||||
evaluation=evaluation,
|
||||
should_download=should_download,
|
||||
is_starred=is_starred,
|
||||
)
|
||||
if downloaded:
|
||||
new_downloads.append(normalised.title)
|
||||
|
||||
# Send email if wanted and not "forced"
|
||||
if new_downloads and cfg.email_rss() and not force:
|
||||
emailer.rss_mail(feed, new_downloads)
|
||||
|
||||
self.remove_obsolete(jobs, new_links)
|
||||
|
||||
return msg
|
||||
|
||||
def configure_rss(
|
||||
self, feed: str, ignore_first: bool
|
||||
) -> tuple[list[str], Optional[FeedConfig], bool, dict, Optional[str]]:
|
||||
"""Prepare configuration and state for a feed run.
|
||||
|
||||
Returns (uris, filters, first, jobs, error_message).
|
||||
If `error_message` is not empty, the caller should abort and return it.
|
||||
"""
|
||||
# Preparations, get options
|
||||
try:
|
||||
feeds = config.get_rss()[feed]
|
||||
except KeyError:
|
||||
logging.error(T('Incorrect RSS feed description "%s"'), feed)
|
||||
logging.info("Traceback: ", exc_info=True)
|
||||
return [], None, False, {}, T('Incorrect RSS feed description "%s"') % feed
|
||||
return T('Incorrect RSS feed description "%s"') % feed
|
||||
|
||||
uris = feeds.uri()
|
||||
filters = FeedConfig.from_config(feeds)
|
||||
@@ -646,7 +547,72 @@ class RSSReader:
|
||||
self.jobs[feed] = {}
|
||||
jobs = self.jobs[feed]
|
||||
|
||||
return uris, filters, first, jobs, ""
|
||||
# Fetch & parse RSS
|
||||
if readout:
|
||||
entries, msg = self.fetch_rss(feed, uris)
|
||||
else:
|
||||
entries, msg = (jobs, "")
|
||||
|
||||
# Error in readout or no new readout
|
||||
if readout and not entries:
|
||||
return msg
|
||||
|
||||
# Normalise entries, evaluate rules and apply side effects
|
||||
for raw_entry in entries:
|
||||
if self.shutdown:
|
||||
return ""
|
||||
|
||||
try:
|
||||
if readout:
|
||||
feed_entry = NormalisedEntry.from_feed_entry(raw_entry)
|
||||
if not feed_entry:
|
||||
continue
|
||||
# Skip duplicates across multiple feeds
|
||||
if len(uris) > 1 and feed_entry.is_duplicate(jobs):
|
||||
continue
|
||||
else:
|
||||
feed_entry = NormalisedEntry.from_job_entry(raw_entry, jobs)
|
||||
except (AttributeError, IndexError):
|
||||
last_uri = uris[-1] if uris else ""
|
||||
logging.info(T("Incompatible feed") + " " + last_uri)
|
||||
logging.info("Traceback: ", exc_info=True)
|
||||
return T("Incompatible feed")
|
||||
if not feed_entry.link:
|
||||
continue
|
||||
|
||||
# Track all valid links so obsolete ones can be cleaned up later
|
||||
new_links.append(feed_entry.link)
|
||||
|
||||
evaluation, should_download, is_starred = self._evaluate_entry(
|
||||
feed_entry=feed_entry,
|
||||
jobs=jobs,
|
||||
filters=filters,
|
||||
first=first,
|
||||
download=download,
|
||||
force=force,
|
||||
readout=readout,
|
||||
)
|
||||
if evaluation is None:
|
||||
continue
|
||||
|
||||
downloaded = self._process_entry(
|
||||
feed=feed,
|
||||
feed_entry=feed_entry,
|
||||
jobs=jobs,
|
||||
evaluation=evaluation,
|
||||
should_download=should_download,
|
||||
is_starred=is_starred,
|
||||
)
|
||||
if downloaded:
|
||||
new_downloads.append(feed_entry.title)
|
||||
|
||||
# Send email if wanted and not "forced"
|
||||
if new_downloads and cfg.email_rss() and not force:
|
||||
emailer.rss_mail(feed, new_downloads)
|
||||
|
||||
self.remove_obsolete(jobs, new_links)
|
||||
|
||||
return msg
|
||||
|
||||
@staticmethod
|
||||
def patch_feedparser():
|
||||
@@ -769,103 +735,89 @@ class RSSReader:
|
||||
@staticmethod
|
||||
def _evaluate_entry(
|
||||
*,
|
||||
entry: NormalisedEntry,
|
||||
feed_entry: NormalisedEntry,
|
||||
jobs: dict,
|
||||
filters: FeedConfig,
|
||||
first: bool,
|
||||
download: bool,
|
||||
force: bool,
|
||||
readout: bool,
|
||||
) -> tuple[Optional[FeedEvaluation], Optional[bool], Optional[bool]]:
|
||||
) -> tuple[Optional[FeedEvaluation], bool, bool]:
|
||||
"""Evaluate a normalised entry against filters
|
||||
|
||||
Returns a tuple (evaluation, should_download, star) or None if the entry should be skipped.
|
||||
"""
|
||||
link = entry.link
|
||||
link = feed_entry.link
|
||||
job = jobs.get(link)
|
||||
job_status = job.get("status", " ")[0] if job else "N"
|
||||
|
||||
if job_status not in "NGB" and not (job_status == "X" and readout):
|
||||
return None, None, None
|
||||
return None, False, False
|
||||
|
||||
# Match this title against all filters
|
||||
logging.debug("Trying title=%r, size=%d", entry.title, entry.size)
|
||||
logging.debug("Trying title=%r, size=%d", feed_entry.title, feed_entry.size)
|
||||
evaluation = filters.evaluate(
|
||||
title=entry.title,
|
||||
category=entry.category,
|
||||
size=entry.size,
|
||||
season=entry.season,
|
||||
episode=entry.episode,
|
||||
title=feed_entry.title,
|
||||
category=feed_entry.category,
|
||||
size=feed_entry.size,
|
||||
season=feed_entry.season,
|
||||
episode=feed_entry.episode,
|
||||
)
|
||||
|
||||
is_starred = job and job.get("status", "").endswith("*")
|
||||
is_starred = bool(job and job.get("status", "").endswith("*"))
|
||||
star = first or is_starred
|
||||
should_download = (download and not first and not is_starred) or force
|
||||
|
||||
return evaluation, should_download, star
|
||||
|
||||
@staticmethod
|
||||
def update_job_entry(jobs: dict, update: ResolvedEntry) -> None:
|
||||
def update_job_entry(jobs: dict, resolved_entry: ResolvedEntry) -> None:
|
||||
"""Update the stored job entry"""
|
||||
jobs[update.link] = {
|
||||
"title": update.title,
|
||||
"url": update.link,
|
||||
"infourl": update.infourl,
|
||||
"cat": update.cat,
|
||||
"pp": update.pp,
|
||||
"script": update.script,
|
||||
"prio": str(update.priority) if update.priority is not None else str(DEFAULT_PRIORITY),
|
||||
"orgcat": update.orgcat,
|
||||
"size": update.size,
|
||||
"age": update.age,
|
||||
jobs[resolved_entry.link] = {
|
||||
"title": resolved_entry.title,
|
||||
"url": resolved_entry.link,
|
||||
"infourl": resolved_entry.infourl,
|
||||
"cat": resolved_entry.cat,
|
||||
"pp": resolved_entry.pp,
|
||||
"script": resolved_entry.script,
|
||||
"prio": resolved_entry.priority if resolved_entry.priority is not None else DEFAULT_PRIORITY,
|
||||
"orgcat": resolved_entry.orgcat,
|
||||
"size": resolved_entry.size,
|
||||
"age": resolved_entry.age,
|
||||
"time": time.time(),
|
||||
"rule": str(update.rule),
|
||||
"season": str(update.season),
|
||||
"episode": str(update.episode),
|
||||
"status": update.status,
|
||||
"rule": resolved_entry.rule,
|
||||
"season": resolved_entry.season,
|
||||
"episode": resolved_entry.episode,
|
||||
"status": resolved_entry.status,
|
||||
}
|
||||
|
||||
if update.status == "D":
|
||||
jobs[update.link]["time_downloaded"] = time.localtime()
|
||||
if resolved_entry.status == "D":
|
||||
jobs[resolved_entry.link]["time_downloaded"] = time.localtime()
|
||||
|
||||
@staticmethod
|
||||
def enqueue_download(feed: str, update: ResolvedEntry) -> None:
|
||||
if not update.download:
|
||||
def enqueue_download(feed: str, resolved_entry: ResolvedEntry) -> None:
|
||||
if not resolved_entry.download:
|
||||
return
|
||||
|
||||
nzbname = None if special_rss_site(update.link) else update.title
|
||||
nzbname = None if special_rss_site(resolved_entry.link) else resolved_entry.title
|
||||
|
||||
logging.info("Adding %s (%s) to queue", update.link, update.title)
|
||||
logging.info("Adding %s (%s) to queue", resolved_entry.link, resolved_entry.title)
|
||||
sabnzbd.urlgrabber.add_url(
|
||||
update.link,
|
||||
pp=update.pp,
|
||||
script=update.script,
|
||||
cat=update.cat,
|
||||
priority=update.priority,
|
||||
resolved_entry.link,
|
||||
pp=resolved_entry.pp,
|
||||
script=resolved_entry.script,
|
||||
cat=resolved_entry.cat,
|
||||
priority=resolved_entry.priority,
|
||||
nzbname=nzbname,
|
||||
nzo_info={"RSS": feed},
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def is_duplicate(entry: NormalisedEntry, jobs: dict[str, dict]) -> bool:
|
||||
"""Check if a job with the same title and size already exists in another feed"""
|
||||
for job_link, job in jobs.items():
|
||||
# Allow 5% size deviation because indexers might have small differences for same release
|
||||
if (
|
||||
job.get("title") == entry.title
|
||||
and entry.link != job_link
|
||||
and (job.get("size") * 0.95) < entry.size < (job.get("size") * 1.05)
|
||||
):
|
||||
logging.info("Ignoring job %s from other feed", entry.title)
|
||||
return True
|
||||
return False
|
||||
|
||||
def _process_entry(
|
||||
self,
|
||||
*,
|
||||
feed: str,
|
||||
jobs: dict[str, dict],
|
||||
entry: NormalisedEntry,
|
||||
feed_entry: NormalisedEntry,
|
||||
evaluation: FeedEvaluation,
|
||||
should_download: bool,
|
||||
is_starred: bool,
|
||||
@@ -883,15 +835,15 @@ class RSSReader:
|
||||
else:
|
||||
status = "B"
|
||||
|
||||
update = ResolvedEntry(
|
||||
link=entry.link,
|
||||
title=entry.title,
|
||||
infourl=entry.infourl,
|
||||
size=entry.size,
|
||||
age=entry.age,
|
||||
resolved_entry = ResolvedEntry(
|
||||
link=feed_entry.link,
|
||||
title=feed_entry.title,
|
||||
infourl=feed_entry.infourl,
|
||||
size=feed_entry.size,
|
||||
age=feed_entry.age,
|
||||
season=evaluation.season,
|
||||
episode=evaluation.episode,
|
||||
orgcat=entry.category,
|
||||
orgcat=feed_entry.category,
|
||||
cat=evaluation.category,
|
||||
pp=evaluation.pp,
|
||||
script=evaluation.script,
|
||||
@@ -901,8 +853,8 @@ class RSSReader:
|
||||
download=(status == "D"),
|
||||
)
|
||||
|
||||
self.update_job_entry(jobs, update)
|
||||
self.enqueue_download(feed, update)
|
||||
self.update_job_entry(jobs, resolved_entry)
|
||||
self.enqueue_download(feed, resolved_entry)
|
||||
|
||||
return bool(evaluation.matched and should_download)
|
||||
|
||||
@@ -918,7 +870,7 @@ class RSSReader:
|
||||
if feeds[feed].enable():
|
||||
logging.info('Starting scheduled RSS read-out for "%s"', feed)
|
||||
active = True
|
||||
self.run_feed(feed, download=True, ignore_first=True)
|
||||
self.process_feed(feed, download=True, ignore_first=True)
|
||||
# Wait 15 seconds, else sites may get irritated
|
||||
for _ in range(15):
|
||||
if self.shutdown:
|
||||
@@ -935,7 +887,7 @@ class RSSReader:
|
||||
logging.info("Finished scheduled RSS read-outs")
|
||||
|
||||
@synchronized(RSS_LOCK)
|
||||
def show_result(self, feed):
|
||||
def get_feed_jobs(self, feed):
|
||||
if feed in self.jobs:
|
||||
try:
|
||||
return self.jobs[feed]
|
||||
@@ -969,7 +921,7 @@ class RSSReader:
|
||||
lst[link]["time_downloaded"] = time.localtime()
|
||||
|
||||
@synchronized(RSS_LOCK)
|
||||
def lookup_url(self, feed, url):
|
||||
def find_job_by_url(self, feed, url):
|
||||
if url and feed in self.jobs:
|
||||
lst = self.jobs[feed]
|
||||
for link in lst:
|
||||
@@ -1025,14 +977,6 @@ def _normalise_pp(value) -> Optional[int]:
|
||||
return None
|
||||
|
||||
|
||||
def first_not_none(*args):
|
||||
"""Return first value which is not None"""
|
||||
for a in args:
|
||||
if a is not None:
|
||||
return a
|
||||
return None
|
||||
|
||||
|
||||
def special_rss_site(url: str) -> bool:
|
||||
"""Return True if url describes an RSS site with odd titles"""
|
||||
return cfg.rss_filenames() or match_str(url, cfg.rss_odd_titles())
|
||||
+1
-1
@@ -6,5 +6,5 @@
|
||||
# You MUST use double quotes (so " and not ')
|
||||
# Do not forget to update the appdata file for every major release!
|
||||
|
||||
__version__ = "5.1.0-dev"
|
||||
__version__ = "5.0.4RC1"
|
||||
__baseline__ = "unknown"
|
||||
@@ -0,0 +1,16 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<rss version="2.0">
|
||||
<channel>
|
||||
<title>RSS feed</title>
|
||||
<description>RSS feed</description>
|
||||
<link>https://sabnzbd.org/</link>
|
||||
<item>
|
||||
<title>TITLE</title>
|
||||
<link>http://LINK</link>
|
||||
<comments>COMMENTS</comments>
|
||||
<pubDate>Tue, 20 May 2025 18:21:01 +0000</pubDate>
|
||||
<guid isPermaLink="true">https://sabnzbd.org/rss_feed_category</guid>
|
||||
<category>TV > HD</category>
|
||||
</item>
|
||||
</channel>
|
||||
</rss>
|
||||
@@ -443,6 +443,34 @@ class TestAddingNZBs:
|
||||
):
|
||||
self._priority_tester(prio_def_cat, prio_add, prio_add_cat, prio_preq, prio_preq_cat, prio_meta_cat)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"prio_def_cat, prio_add, prio_add_cat, prio_preq, prio_preq_cat, prio_meta_cat, expected_priority, expected_status",
|
||||
[
|
||||
# Added category priority should apply when no explicit priority is set.
|
||||
(LOW_PRIORITY, None, HIGH_PRIORITY, None, None, None, HIGH_PRIORITY, "Queued"),
|
||||
# Pre-queue category should override category supplied during add.
|
||||
(LOW_PRIORITY, None, LOW_PRIORITY, None, HIGH_PRIORITY, None, HIGH_PRIORITY, "Queued"),
|
||||
# Explicit priority on add should beat implicit category from pre-queue.
|
||||
(LOW_PRIORITY, NORMAL_PRIORITY, None, None, HIGH_PRIORITY, None, NORMAL_PRIORITY, "Queued"),
|
||||
# NZB metadata category should be used if no category is provided later.
|
||||
(LOW_PRIORITY, None, None, None, None, HIGH_PRIORITY, HIGH_PRIORITY, "Queued"),
|
||||
],
|
||||
)
|
||||
def test_adding_nzbs_category_priority_precedence(
|
||||
self,
|
||||
prio_def_cat,
|
||||
prio_add,
|
||||
prio_add_cat,
|
||||
prio_preq,
|
||||
prio_preq_cat,
|
||||
prio_meta_cat,
|
||||
expected_priority,
|
||||
expected_status,
|
||||
):
|
||||
job = self._prep_priority_tester(prio_def_cat, prio_add, prio_add_cat, prio_preq, prio_preq_cat, prio_meta_cat)
|
||||
assert job["priority"] == ALL_PRIOS.get(expected_priority)
|
||||
assert job["status"] == expected_status
|
||||
|
||||
def test_adding_nzbs_partial(self):
|
||||
"""Test adding parts of an NZB file, cut off somewhere in the middle to simulate
|
||||
the effects of an interrupted download or bad hardware. Should fail, of course."""
|
||||
|
||||
@@ -112,6 +112,34 @@ class TestNzbQueue:
|
||||
# Try list restored
|
||||
assert sabnzbd.Downloader.servers[0] in list(joba.files[0].articles)[0].try_list
|
||||
|
||||
def test_stop_idle_jobs_no_crash_on_exhausted_articles(self):
|
||||
"""Regression test: stop_idle_jobs must not raise RuntimeError when
|
||||
register_article removes an article from nzf.articles (a dict) while
|
||||
the same dict is being iterated. Introduced by commit 44d94226e when
|
||||
nzf.articles was changed from list to dict but the protective [:] copy
|
||||
was dropped from the iteration in stop_idle_jobs."""
|
||||
server = sabnzbd.Downloader.servers[0]
|
||||
|
||||
nzo = make_dummy_nzo("stall-test", files=1, articles=3)
|
||||
nzf = nzo.files[0]
|
||||
|
||||
# Load all articles into memory (only first is loaded at NzbFile init)
|
||||
nzf.finish_import()
|
||||
|
||||
q = NzbQueue()
|
||||
# add() resets try lists, so saturate them after adding
|
||||
q.add(nzo)
|
||||
sabnzbd.NzbQueue = q
|
||||
|
||||
# Saturate all try-lists so stop_idle_jobs enters the article-removal branch
|
||||
nzo.add_to_try_list(server)
|
||||
nzf.add_to_try_list(server)
|
||||
for article in list(nzf.articles):
|
||||
article.add_to_try_list(server)
|
||||
|
||||
# Must not raise RuntimeError: dictionary changed size during iteration
|
||||
q.stop_idle_jobs()
|
||||
|
||||
@pytest.mark.skipif(not sabnzbd.WINDOWS, reason="Legacy 3.0.0 queue fixture contains Windows-specific paths")
|
||||
def test_restore_legacy_queue_format_3_0_0(self, tmp_path, monkeypatch):
|
||||
fixture_path = Path(SAB_DATA_DIR) / "test_3_0_0_queue_format"
|
||||
|
||||
+139
-39
@@ -21,16 +21,18 @@ tests.test_misc - Testing functions in misc.py
|
||||
|
||||
import datetime
|
||||
import time
|
||||
import uuid
|
||||
from typing import Optional
|
||||
|
||||
import configobj
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
from werkzeug import Response
|
||||
from xml.etree.ElementTree import Element, SubElement, tostring
|
||||
|
||||
import sabnzbd.rss as rss
|
||||
import sabnzbd.config
|
||||
from sabnzbd.constants import DEFAULT_PRIORITY, LOW_PRIORITY, HIGH_PRIORITY, FORCE_PRIORITY
|
||||
from sabnzbd.rss import FeedEvaluation, FeedConfig
|
||||
from sabnzbd.constants import DEFAULT_PRIORITY, LOW_PRIORITY, HIGH_PRIORITY, FORCE_PRIORITY, PAUSED_PRIORITY
|
||||
from tests.testhelper import httpserver_handler_data_dir
|
||||
|
||||
|
||||
@@ -44,7 +46,7 @@ class TestRSS:
|
||||
pp: Optional[str] = None,
|
||||
script: Optional[str] = None,
|
||||
priority: Optional[int] = None,
|
||||
filters: list[tuple[str, str, str, str, str, int, str]] = None,
|
||||
filters: Optional[list[tuple[str, str, str, str, str, int, str]]] = None,
|
||||
):
|
||||
"""Setup the basic settings to get things going"""
|
||||
values: dict = {"uri": feed_url}
|
||||
@@ -79,7 +81,7 @@ class TestRSS:
|
||||
|
||||
# Start the RSS reader
|
||||
rss_obj = rss.RSSReader()
|
||||
rss_obj.run_feed(feed_name)
|
||||
rss_obj.process_feed(feed_name)
|
||||
|
||||
# Is the feed processed?
|
||||
assert feed_name in rss_obj.jobs
|
||||
@@ -91,8 +93,8 @@ class TestRSS:
|
||||
assert job_data["infourl"] == "https://nzbgeek.info/geekseek.php?guid=FakeKey"
|
||||
assert job_data["orgcat"] == "TV > HD"
|
||||
assert job_data["cat"] == "tv"
|
||||
assert job_data["episode"] == "3"
|
||||
assert job_data["season"] == "4"
|
||||
assert job_data["episode"] == 3
|
||||
assert job_data["season"] == 4
|
||||
assert job_data["size"] == 1209464000
|
||||
|
||||
# feedparser returns UTC so SABnzbd converts to locale
|
||||
@@ -106,7 +108,7 @@ class TestRSS:
|
||||
|
||||
# Start the RSS reader
|
||||
rss_obj = rss.RSSReader()
|
||||
rss_obj.run_feed(feed_name)
|
||||
rss_obj.process_feed(feed_name)
|
||||
|
||||
# Is the feed processed?
|
||||
assert feed_name in rss_obj.jobs
|
||||
@@ -119,8 +121,8 @@ class TestRSS:
|
||||
assert job_data["infourl"] == "https://nzbfinder.ws/details/FakeKey"
|
||||
assert job_data["orgcat"] == "Movies > HD"
|
||||
assert job_data["cat"] == "movies"
|
||||
assert job_data["episode"] == "720"
|
||||
assert job_data["season"] == "2018"
|
||||
assert job_data["episode"] == 720
|
||||
assert job_data["season"] == 2018
|
||||
assert job_data["size"] == 5164539914
|
||||
|
||||
# feedparser returns UTC so SABnzbd converts to locale
|
||||
@@ -136,7 +138,7 @@ class TestRSS:
|
||||
|
||||
# Start the RSS reader
|
||||
rss_obj = rss.RSSReader()
|
||||
rss_obj.run_feed(feed_name)
|
||||
rss_obj.process_feed(feed_name)
|
||||
|
||||
# Is the feed processed?
|
||||
assert feed_name in rss_obj.jobs
|
||||
@@ -161,7 +163,7 @@ class TestRSS:
|
||||
|
||||
# Start the RSS reader
|
||||
rss_obj = rss.RSSReader()
|
||||
rss_obj.run_feed(feed_name)
|
||||
rss_obj.process_feed(feed_name)
|
||||
|
||||
# Is the feed processed?
|
||||
assert feed_name in rss_obj.jobs
|
||||
@@ -175,7 +177,7 @@ class TestRSS:
|
||||
|
||||
# Start the RSS reader
|
||||
rss_obj = rss.RSSReader()
|
||||
rss_obj.run_feed(feed_name)
|
||||
rss_obj.process_feed(feed_name)
|
||||
|
||||
# Is the feed processed?
|
||||
assert feed_name in rss_obj.jobs
|
||||
@@ -204,7 +206,7 @@ class TestRSS:
|
||||
1000,
|
||||
0,
|
||||
0,
|
||||
FeedEvaluation(matched=True, rule_index=0, season=0, episode=0),
|
||||
dict(rule=0, season=0, episode=0),
|
||||
),
|
||||
(
|
||||
(None, None, None, None),
|
||||
@@ -214,7 +216,7 @@ class TestRSS:
|
||||
1000,
|
||||
0,
|
||||
0,
|
||||
FeedEvaluation(matched=True, rule_index=1, season=0, episode=0),
|
||||
dict(rule=1, season=0, episode=0),
|
||||
),
|
||||
(
|
||||
(None, None, None, None),
|
||||
@@ -224,7 +226,7 @@ class TestRSS:
|
||||
1000,
|
||||
0,
|
||||
0,
|
||||
FeedEvaluation(matched=True, rule_index=1, season=5, episode=2),
|
||||
dict(rule=1, season=5, episode=2),
|
||||
),
|
||||
(
|
||||
(None, None, None, None),
|
||||
@@ -234,17 +236,17 @@ class TestRSS:
|
||||
1000,
|
||||
0,
|
||||
0,
|
||||
FeedEvaluation(matched=False, rule_index=0, season=1, episode=2),
|
||||
dict(rule=0, season=1, episode=2),
|
||||
),
|
||||
(
|
||||
(None, None, None, LOW_PRIORITY),
|
||||
[],
|
||||
[("", "", "", "A", "*", "", "")],
|
||||
"Title",
|
||||
None,
|
||||
1000,
|
||||
0,
|
||||
0,
|
||||
FeedEvaluation(matched=True, rule_index=0, season=0, episode=0, priority=LOW_PRIORITY),
|
||||
dict(rule=0, season=0, episode=0, prio=LOW_PRIORITY),
|
||||
),
|
||||
(
|
||||
(None, None, None, LOW_PRIORITY),
|
||||
@@ -254,7 +256,7 @@ class TestRSS:
|
||||
1000,
|
||||
0,
|
||||
0,
|
||||
FeedEvaluation(matched=True, rule_index=0, season=0, episode=0, priority=HIGH_PRIORITY),
|
||||
dict(rule=0, season=0, episode=0, prio=HIGH_PRIORITY),
|
||||
),
|
||||
(
|
||||
(None, 1, None, None),
|
||||
@@ -264,7 +266,7 @@ class TestRSS:
|
||||
1000,
|
||||
0,
|
||||
0,
|
||||
FeedEvaluation(matched=True, rule_index=0, season=0, episode=0, pp=1),
|
||||
dict(rule=0, season=0, episode=0, pp=None),
|
||||
),
|
||||
(
|
||||
(None, 1, None, None),
|
||||
@@ -274,44 +276,74 @@ class TestRSS:
|
||||
1000,
|
||||
0,
|
||||
0,
|
||||
FeedEvaluation(matched=True, rule_index=0, season=0, episode=0, pp=3),
|
||||
dict(rule=0, season=0, episode=0, pp=3),
|
||||
),
|
||||
( # category overrides
|
||||
("tv", 1, DEFAULT_PRIORITY, ""),
|
||||
("tv", 1, "", DEFAULT_PRIORITY),
|
||||
[("evaluator", "", "", "A", "*", "", "1")],
|
||||
"Title",
|
||||
None,
|
||||
1000,
|
||||
0,
|
||||
0,
|
||||
FeedEvaluation(
|
||||
matched=True,
|
||||
rule_index=0,
|
||||
dict(
|
||||
rule=0,
|
||||
season=0,
|
||||
episode=0,
|
||||
category="evaluator",
|
||||
pp=3,
|
||||
script="evaluator.py",
|
||||
priority=FORCE_PRIORITY,
|
||||
cat="evaluator",
|
||||
pp=1,
|
||||
script=None,
|
||||
prio=DEFAULT_PRIORITY,
|
||||
),
|
||||
),
|
||||
( # category with rule overrides
|
||||
("tv", 1, DEFAULT_PRIORITY, ""),
|
||||
("tv", 1, "", DEFAULT_PRIORITY),
|
||||
[("evaluator", "2", "override.py", "A", "*", "", "1")],
|
||||
"Title",
|
||||
None,
|
||||
1000,
|
||||
0,
|
||||
0,
|
||||
FeedEvaluation(
|
||||
matched=True,
|
||||
rule_index=0,
|
||||
dict(
|
||||
rule=0,
|
||||
season=0,
|
||||
episode=0,
|
||||
category="evaluator",
|
||||
cat="evaluator",
|
||||
pp=2,
|
||||
script="override.py",
|
||||
priority=FORCE_PRIORITY,
|
||||
prio=DEFAULT_PRIORITY,
|
||||
),
|
||||
),
|
||||
(
|
||||
("", "", "", PAUSED_PRIORITY),
|
||||
[("", "", "", "A", "*", "", "")],
|
||||
"Title",
|
||||
"TV > HD",
|
||||
1000,
|
||||
0,
|
||||
0,
|
||||
dict(
|
||||
rule=0,
|
||||
season=0,
|
||||
episode=0,
|
||||
cat="tv",
|
||||
prio=PAUSED_PRIORITY,
|
||||
),
|
||||
),
|
||||
(
|
||||
("", "", "", PAUSED_PRIORITY),
|
||||
[("", "", "", "F", "", "", "")],
|
||||
"Title",
|
||||
"TV > HD",
|
||||
1000,
|
||||
3,
|
||||
5,
|
||||
dict(
|
||||
rule=0,
|
||||
season=3,
|
||||
episode=5,
|
||||
cat="tv",
|
||||
prio=PAUSED_PRIORITY,
|
||||
),
|
||||
),
|
||||
],
|
||||
@@ -326,8 +358,67 @@ class TestRSS:
|
||||
size: int,
|
||||
season: int,
|
||||
episode: int,
|
||||
expected_match: FeedEvaluation,
|
||||
expected_match: dict,
|
||||
):
|
||||
def build_xml_response(
|
||||
title: str, category: Optional[str], size: Optional[int], season: Optional[int], episode: Optional[int]
|
||||
):
|
||||
root = Element("rss", version="2.0")
|
||||
|
||||
channel = SubElement(root, "channel")
|
||||
SubElement(channel, "title").text = "RSS feed"
|
||||
SubElement(channel, "description").text = "RSS feed"
|
||||
SubElement(channel, "link").text = "https://sabnzbd.org/"
|
||||
|
||||
item = SubElement(channel, "item")
|
||||
|
||||
SubElement(item, "title").text = title
|
||||
SubElement(item, "link").text = "http://LINK"
|
||||
SubElement(item, "comments").text = "COMMENTS"
|
||||
SubElement(item, "pubDate").text = "Tue, 20 May 2025 18:21:01 +0000"
|
||||
|
||||
guid = SubElement(item, "guid")
|
||||
guid.set("isPermaLink", "true")
|
||||
guid.text = uuid.uuid4().hex
|
||||
|
||||
# optional fields
|
||||
if category is not None:
|
||||
SubElement(item, "category").text = category
|
||||
|
||||
if size is not None:
|
||||
SubElement(item, "size").text = str(size)
|
||||
|
||||
if season is not None:
|
||||
SubElement(
|
||||
item,
|
||||
"newznab:attr",
|
||||
{
|
||||
"name": "season",
|
||||
"value": str(season),
|
||||
},
|
||||
)
|
||||
|
||||
if episode is not None:
|
||||
SubElement(
|
||||
item,
|
||||
"newznab:attr",
|
||||
{
|
||||
"name": "episode",
|
||||
"value": str(episode),
|
||||
},
|
||||
)
|
||||
|
||||
xml_bytes = tostring(root, encoding="utf-8")
|
||||
|
||||
return xml_bytes
|
||||
|
||||
httpserver.expect_request("/evaluator.xml").respond_with_handler(
|
||||
lambda request: Response(
|
||||
build_xml_response(title=title, category=category, size=size, season=season, episode=episode),
|
||||
status=200,
|
||||
content_type="application/rss+xml",
|
||||
)
|
||||
)
|
||||
default_category, default_pp, default_script, default_priority = defaults
|
||||
feed_name = "Evaluator"
|
||||
self.setup_rss(
|
||||
@@ -348,7 +439,16 @@ class TestRSS:
|
||||
},
|
||||
)
|
||||
|
||||
feed_cfg = FeedConfig.from_config(sabnzbd.config.get_rss()[feed_name])
|
||||
result_match = feed_cfg.evaluate(title=title, category=category, size=size, season=season, episode=episode)
|
||||
# Start the RSS reader
|
||||
rss_obj = rss.RSSReader()
|
||||
rss_obj.process_feed(feed_name)
|
||||
|
||||
assert result_match == expected_match
|
||||
# Is the feed processed?
|
||||
assert feed_name in rss_obj.jobs
|
||||
assert "http://LINK" in rss_obj.jobs[feed_name]
|
||||
|
||||
# Check some job-data
|
||||
job_data = rss_obj.jobs[feed_name]["http://LINK"]
|
||||
for k, v in expected_match.items():
|
||||
assert k in job_data
|
||||
assert job_data[k] == v, f"Expected {k!r}: {job_data[k]!r} == {v!r}"
|
||||
Reference in new issue
Block a user