mirror of
https://github.com/Cleanuparr/Cleanuparr.git
synced 2026-01-01 02:18:36 -05:00
Compare commits
12 Commits
fix_qbit_p
...
v1.5.18
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
bd28c7ab05 | ||
|
|
720279df65 | ||
|
|
2d4ec648b8 | ||
|
|
704fdaca4a | ||
|
|
b134136e51 | ||
|
|
5ca717d7e0 | ||
|
|
7068ee5e5a | ||
|
|
9f770473e5 | ||
|
|
5fe0f5750a | ||
|
|
b8ce225ccc | ||
|
|
f21f7388b7 | ||
|
|
a1354f231a |
2
.github/ISSUE_TEMPLATE/1-bug.yml
vendored
2
.github/ISSUE_TEMPLATE/1-bug.yml
vendored
@@ -18,7 +18,7 @@ body:
|
||||
required: true
|
||||
- label: Ensured I am using the latest version.
|
||||
required: true
|
||||
- label: Enabled debug logging.
|
||||
- label: Enabled verbose logging.
|
||||
required: true
|
||||
- type: textarea
|
||||
id: what-happened
|
||||
|
||||
2
.github/ISSUE_TEMPLATE/3-help.yml
vendored
2
.github/ISSUE_TEMPLATE/3-help.yml
vendored
@@ -18,7 +18,7 @@ body:
|
||||
required: true
|
||||
- label: Ensured I am using the latest version.
|
||||
required: true
|
||||
- label: Enabled debug logging.
|
||||
- label: Enabled verbose logging.
|
||||
required: true
|
||||
- type: textarea
|
||||
id: description
|
||||
|
||||
56
README.md
56
README.md
@@ -12,6 +12,7 @@ cleanuperr was created primarily to address malicious files, such as `*.lnk` or
|
||||
> **Features:**
|
||||
> - Strike system to mark stalled or downloads stuck in metadata downloading.
|
||||
> - Remove and block downloads that reached a maximum number of strikes.
|
||||
> - Remove and block downloads that have a low download speed or high estimated completion time.
|
||||
> - Remove downloads blocked by qBittorrent or by cleanuperr's **content blocker**.
|
||||
> - Trigger a search for downloads removed from the *arrs.
|
||||
> - Clean up downloads that have been seeding for a certain amount of time.
|
||||
@@ -91,11 +92,11 @@ I've seen a few discussions on this type of naming and I've decided that I didn'
|
||||
#### 2. **Queue cleaner** will:
|
||||
- Run every 5 minutes (or configured cron, or right after `content blocker`).
|
||||
- Process all items in the *arr queue.
|
||||
- Check each queue item if it is **stalled (download speed is 0)**, **stuck in metadata downloading** or **failed to be imported**.
|
||||
- Check each queue item if it is **stalled (download speed is 0)**, **stuck in metadata downloading**, **failed to be imported** or **slow**.
|
||||
- If it is, the item receives a **strike** and will continue to accumulate strikes every time it meets any of these conditions.
|
||||
- Check each queue item if it meets one of the following condition in the download client:
|
||||
- **Marked as completed, but 0 bytes have been downloaded** (due to files being blocked by qBittorrent or the **content blocker**).
|
||||
- All associated files of are marked as **unwanted/skipped**.
|
||||
- All associated files are marked as **unwanted/skipped/do not download**.
|
||||
- If the item **DOES NOT** match the above criteria, it will be skipped.
|
||||
- If the item **DOES** match the criteria or has received the **maximum number of strikes**:
|
||||
- It will be removed from the *arr's queue and blocked.
|
||||
@@ -168,39 +169,62 @@ services:
|
||||
- ./cleanuperr/logs:/var/logs
|
||||
- ./cleanuperr/ignored.txt:/ignored.txt
|
||||
environment:
|
||||
# general settings
|
||||
- TZ=America/New_York
|
||||
- DRY_RUN=false
|
||||
- HTTP_MAX_RETRIES=0
|
||||
- HTTP_TIMEOUT=100
|
||||
|
||||
# logging
|
||||
- LOGGING__LOGLEVEL=Information
|
||||
- LOGGING__FILE__ENABLED=false
|
||||
- LOGGING__FILE__PATH=/var/logs/
|
||||
- LOGGING__ENHANCED=true
|
||||
|
||||
# job triggers
|
||||
- TRIGGERS__QUEUECLEANER=0 0/5 * * * ?
|
||||
- TRIGGERS__CONTENTBLOCKER=0 0/5 * * * ?
|
||||
- TRIGGERS__DOWNLOADCLEANER=0 0 * * * ?
|
||||
|
||||
# queue cleaner
|
||||
- QUEUECLEANER__ENABLED=true
|
||||
- QUEUECLEANER__IGNORED_DOWNLOADS_PATH=/ignored.txt
|
||||
- QUEUECLEANER__RUNSEQUENTIALLY=true
|
||||
|
||||
# failed imports
|
||||
- QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES=5
|
||||
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE=false
|
||||
- QUEUECLEANER__IMPORT_FAILED_DELETE_PRIVATE=false
|
||||
# - QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__0=title mismatch
|
||||
# - QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__1=manual import required
|
||||
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__0=title mismatch
|
||||
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__1=manual import required
|
||||
|
||||
# stalled downloads
|
||||
- QUEUECLEANER__STALLED_MAX_STRIKES=5
|
||||
- QUEUECLEANER__STALLED_RESET_STRIKES_ON_PROGRESS=false
|
||||
- QUEUECLEANER__STALLED_IGNORE_PRIVATE=false
|
||||
- QUEUECLEANER__STALLED_DELETE_PRIVATE=false
|
||||
|
||||
# slow downloads
|
||||
- QUEUECLEANER__SLOW_MAX_STRIKES=5
|
||||
- QUEUECLEANER__SLOW_RESET_STRIKES_ON_PROGRESS=true
|
||||
- QUEUECLEANER__SLOW_IGNORE_PRIVATE=false
|
||||
- QUEUECLEANER__SLOW_DELETE_PRIVATE=false
|
||||
- QUEUECLEANER__SLOW_MIN_SPEED=1MB
|
||||
- QUEUECLEANER__SLOW_MAX_TIME=20
|
||||
- QUEUECLEANER__SLOW_IGNORE_ABOVE_SIZE=60GB
|
||||
|
||||
# content blocker
|
||||
- CONTENTBLOCKER__ENABLED=true
|
||||
- CONTENTBLOCKER__IGNORED_DOWNLOADS_PATH=/ignored.txt
|
||||
- CONTENTBLOCKER__IGNORE_PRIVATE=false
|
||||
- CONTENTBLOCKER__DELETE_PRIVATE=false
|
||||
|
||||
# download cleaner
|
||||
- DOWNLOADCLEANER__ENABLED=true
|
||||
- DOWNLOADCLEANER__IGNORED_DOWNLOADS_PATH=/ignored.txt
|
||||
- DOWNLOADCLEANER__DELETE_PRIVATE=false
|
||||
|
||||
# categories to seed until max ratio or min seed time has been reached
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__NAME=tv-sonarr
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MAX_RATIO=-1
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MIN_SEED_TIME=0
|
||||
@@ -212,17 +236,22 @@ services:
|
||||
|
||||
- DOWNLOAD_CLIENT=none
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=disabled
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=qBittorrent
|
||||
# - QBITTORRENT__URL=http://localhost:8080
|
||||
# - QBITTORRENT__URL_BASE=myCustomPath
|
||||
# - QBITTORRENT__USERNAME=user
|
||||
# - QBITTORRENT__PASSWORD=pass
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=deluge
|
||||
# - DELUGE__URL_BASE=myCustomPath
|
||||
# - DELUGE__URL=http://localhost:8112
|
||||
# - DELUGE__PASSWORD=testing
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=transmission
|
||||
# - TRANSMISSION__URL=http://localhost:9091
|
||||
# - TRANSMISSION__URL_BASE=myCustomPath
|
||||
# - TRANSMISSION__USERNAME=test
|
||||
# - TRANSMISSION__PASSWORD=testing
|
||||
|
||||
@@ -253,6 +282,7 @@ services:
|
||||
|
||||
- NOTIFIARR__ON_IMPORT_FAILED_STRIKE=true
|
||||
- NOTIFIARR__ON_STALLED_STRIKE=true
|
||||
- NOTIFIARR__ON_SLOW_STRIKE=true
|
||||
- NOTIFIARR__ON_QUEUE_ITEM_DELETED=true
|
||||
- NOTIFIARR__ON_DOWNLOAD_CLEANED=true
|
||||
- NOTIFIARR__API_KEY=notifiarr_secret
|
||||
@@ -268,7 +298,21 @@ services:
|
||||
|
||||
> [!TIP]
|
||||
> ### Run as a Windows Service
|
||||
> Check out this stackoverflow answer on how to do it: https://stackoverflow.com/a/15719678
|
||||
> 1. Download latest nssm build from `https://nssm.cc/builds`.
|
||||
> 2. Unzip `nssm.exe` in `C:\example\directory`.
|
||||
> 3. Open a terminal with Administrator rights and execute these commands:
|
||||
> ```
|
||||
> nssm.exe install Cleanuperr "C:\example\directory\cleanuperr.exe"
|
||||
> nssm.exe set Cleanuperr AppDirectory "C:\example\directory\"
|
||||
> nssm.exe set Cleanuperr AppStdout "C:\example\directory\cleanuperr.log"
|
||||
> nssm.exe set Cleanuperr AppStderr "C:\example\directory\cleanuperr.crash.log"
|
||||
> nssm.exe set Cleanuperr AppRotateFiles 1
|
||||
> nssm.exe set Cleanuperr AppRotateOnline 1
|
||||
> nssm.exe set Cleanuperr AppRotateBytes 10485760
|
||||
> nssm.exe set Cleanuperr AppRotateFiles 10
|
||||
> nssm.exe set Cleanuperr Start SERVICE_AUTO_START
|
||||
> nssm.exe start Cleanuperr
|
||||
> ```
|
||||
|
||||
### <img src="https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/brands/linux.svg" height="20" style="vertical-align: middle;"> <span style="vertical-align: middle;">Linux</span>
|
||||
|
||||
@@ -296,7 +340,7 @@ services:
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Some people have experienced problems when trying to execute cleanuperr on MacOS because the system actively blocked the file for not being signed.
|
||||
> As per [this](), you may need to also execute this command:
|
||||
> As per [this comment](https://stackoverflow.com/a/77907937), you may need to also execute this command:
|
||||
> ```
|
||||
> codesign --sign - --force --preserve-metadata=entitlements,requirements,flags,runtime /example/directory/cleanuperr
|
||||
> ```
|
||||
|
||||
@@ -14,7 +14,7 @@ deployment:
|
||||
value: "false"
|
||||
|
||||
- name: LOGGING__LOGLEVEL
|
||||
value: Debug
|
||||
value: Verbose
|
||||
- name: LOGGING__FILE__ENABLED
|
||||
value: "true"
|
||||
- name: LOGGING__FILE__PATH
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
using Common.Exceptions;
|
||||
using Common.Exceptions;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.DownloadClient;
|
||||
|
||||
@@ -8,6 +9,9 @@ public sealed record DelugeConfig : IConfig
|
||||
|
||||
public Uri? Url { get; init; }
|
||||
|
||||
[ConfigurationKeyName("URL_BASE")]
|
||||
public string UrlBase { get; init; } = string.Empty;
|
||||
|
||||
public string? Password { get; init; }
|
||||
|
||||
public void Validate()
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
using Common.Exceptions;
|
||||
using Common.Exceptions;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.DownloadClient;
|
||||
|
||||
@@ -8,6 +9,9 @@ public sealed class QBitConfig : IConfig
|
||||
|
||||
public Uri? Url { get; init; }
|
||||
|
||||
[ConfigurationKeyName("URL_BASE")]
|
||||
public string UrlBase { get; init; } = string.Empty;
|
||||
|
||||
public string? Username { get; init; }
|
||||
|
||||
public string? Password { get; init; }
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
using Common.Exceptions;
|
||||
using Common.Exceptions;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.DownloadClient;
|
||||
|
||||
@@ -8,6 +9,9 @@ public record TransmissionConfig : IConfig
|
||||
|
||||
public Uri? Url { get; init; }
|
||||
|
||||
[ConfigurationKeyName("URL_BASE")]
|
||||
public string UrlBase { get; init; } = "transmission";
|
||||
|
||||
public string? Username { get; init; }
|
||||
|
||||
public string? Password { get; init; }
|
||||
|
||||
@@ -10,13 +10,16 @@ public abstract record NotificationConfig
|
||||
[ConfigurationKeyName("ON_STALLED_STRIKE")]
|
||||
public bool OnStalledStrike { get; init; }
|
||||
|
||||
[ConfigurationKeyName("ON_SLOW_STRIKE")]
|
||||
public bool OnSlowStrike { get; init; }
|
||||
|
||||
[ConfigurationKeyName("ON_QUEUE_ITEM_DELETED")]
|
||||
public bool OnQueueItemDeleted { get; init; }
|
||||
|
||||
[ConfigurationKeyName("ON_DOWNLOAD_CLEANED")]
|
||||
public bool OnDownloadCleaned { get; init; }
|
||||
|
||||
public bool IsEnabled => OnImportFailedStrike || OnStalledStrike || OnQueueItemDeleted || OnDownloadCleaned;
|
||||
public bool IsEnabled => OnImportFailedStrike || OnStalledStrike || OnSlowStrike || OnQueueItemDeleted || OnDownloadCleaned;
|
||||
|
||||
public abstract bool IsValid();
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
using Common.Exceptions;
|
||||
using Common.CustomDataTypes;
|
||||
using Common.Exceptions;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.QueueCleaner;
|
||||
@@ -37,17 +38,74 @@ public sealed record QueueCleanerConfig : IJobConfig, IIgnoredDownloadsConfig
|
||||
|
||||
[ConfigurationKeyName("STALLED_DELETE_PRIVATE")]
|
||||
public bool StalledDeletePrivate { get; init; }
|
||||
|
||||
[ConfigurationKeyName("SLOW_MAX_STRIKES")]
|
||||
public ushort SlowMaxStrikes { get; init; }
|
||||
|
||||
[ConfigurationKeyName("SLOW_RESET_STRIKES_ON_PROGRESS")]
|
||||
public bool SlowResetStrikesOnProgress { get; init; }
|
||||
|
||||
[ConfigurationKeyName("SLOW_IGNORE_PRIVATE")]
|
||||
public bool SlowIgnorePrivate { get; init; }
|
||||
|
||||
[ConfigurationKeyName("SLOW_DELETE_PRIVATE")]
|
||||
public bool SlowDeletePrivate { get; init; }
|
||||
|
||||
[ConfigurationKeyName("SLOW_MIN_SPEED")]
|
||||
public string SlowMinSpeed { get; init; } = string.Empty;
|
||||
|
||||
public ByteSize SlowMinSpeedByteSize => string.IsNullOrEmpty(SlowMinSpeed) ? new ByteSize(0) : ByteSize.Parse(SlowMinSpeed);
|
||||
|
||||
[ConfigurationKeyName("SLOW_MAX_TIME")]
|
||||
public double SlowMaxTime { get; init; }
|
||||
|
||||
[ConfigurationKeyName("SLOW_IGNORE_ABOVE_SIZE")]
|
||||
public string SlowIgnoreAboveSize { get; init; } = string.Empty;
|
||||
|
||||
public ByteSize? SlowIgnoreAboveSizeByteSize => string.IsNullOrEmpty(SlowIgnoreAboveSize) ? null : ByteSize.Parse(SlowIgnoreAboveSize);
|
||||
|
||||
public void Validate()
|
||||
{
|
||||
if (ImportFailedMaxStrikes is > 0 and < 3)
|
||||
{
|
||||
throw new ValidationException("the minimum value for IMPORT_FAILED_MAX_STRIKES must be 3");
|
||||
throw new ValidationException($"the minimum value for {SectionName}__IMPORT_FAILED_MAX_STRIKES must be 3");
|
||||
}
|
||||
|
||||
if (StalledMaxStrikes is > 0 and < 3)
|
||||
{
|
||||
throw new ValidationException("the minimum value for STALLED_MAX_STRIKES must be 3");
|
||||
throw new ValidationException($"the minimum value for {SectionName}__STALLED_MAX_STRIKES must be 3");
|
||||
}
|
||||
|
||||
if (SlowMaxStrikes is > 0 and < 3)
|
||||
{
|
||||
throw new ValidationException($"the minimum value for {SectionName}__SLOW_MAX_STRIKES must be 3");
|
||||
}
|
||||
|
||||
if (SlowMaxStrikes > 0)
|
||||
{
|
||||
bool isSlowSpeedSet = !string.IsNullOrEmpty(SlowMinSpeed);
|
||||
|
||||
if (isSlowSpeedSet && ByteSize.TryParse(SlowMinSpeed, out _) is false)
|
||||
{
|
||||
throw new ValidationException($"invalid value for {SectionName}__SLOW_MIN_SPEED");
|
||||
}
|
||||
|
||||
if (SlowMaxTime < 0)
|
||||
{
|
||||
throw new ValidationException($"invalid value for {SectionName}__SLOW_MAX_TIME");
|
||||
}
|
||||
|
||||
if (!isSlowSpeedSet && SlowMaxTime is 0)
|
||||
{
|
||||
throw new ValidationException($"either {SectionName}__SLOW_MIN_SPEED or {SectionName}__SLOW_MAX_STRIKES must be set");
|
||||
}
|
||||
|
||||
bool isSlowIgnoreAboveSizeSet = !string.IsNullOrEmpty(SlowIgnoreAboveSize);
|
||||
|
||||
if (isSlowIgnoreAboveSizeSet && ByteSize.TryParse(SlowIgnoreAboveSize, out _) is false)
|
||||
{
|
||||
throw new ValidationException($"invalid value for {SectionName}__SLOW_IGNORE_ABOVE_SIZE");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
115
code/Common/CustomDataTypes/ByteSize.cs
Normal file
115
code/Common/CustomDataTypes/ByteSize.cs
Normal file
@@ -0,0 +1,115 @@
|
||||
using System.Globalization;
|
||||
|
||||
namespace Common.CustomDataTypes;
|
||||
|
||||
public readonly struct ByteSize : IComparable<ByteSize>, IEquatable<ByteSize>
|
||||
{
|
||||
public long Bytes { get; }
|
||||
|
||||
private const long BytesPerKB = 1024;
|
||||
private const long BytesPerMB = 1024 * 1024;
|
||||
private const long BytesPerGB = 1024 * 1024 * 1024;
|
||||
|
||||
public ByteSize(long bytes)
|
||||
{
|
||||
if (bytes < 0)
|
||||
{
|
||||
throw new ArgumentOutOfRangeException(nameof(bytes), "bytes can not be negative");
|
||||
}
|
||||
|
||||
Bytes = bytes;
|
||||
}
|
||||
|
||||
public static ByteSize FromKilobytes(double kilobytes) => new((long)(kilobytes * BytesPerKB));
|
||||
public static ByteSize FromMegabytes(double megabytes) => new((long)(megabytes * BytesPerMB));
|
||||
public static ByteSize FromGigabytes(double gigabytes) => new((long)(gigabytes * BytesPerGB));
|
||||
|
||||
public static ByteSize Parse(string input)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(input))
|
||||
{
|
||||
throw new ArgumentNullException(nameof(input));
|
||||
}
|
||||
|
||||
input = input.Trim().ToUpperInvariant();
|
||||
double value;
|
||||
if (input.EndsWith("KB", StringComparison.InvariantCultureIgnoreCase))
|
||||
{
|
||||
value = double.Parse(input[..^2], CultureInfo.InvariantCulture);
|
||||
return FromKilobytes(value);
|
||||
}
|
||||
|
||||
if (input.EndsWith("MB", StringComparison.InvariantCultureIgnoreCase))
|
||||
{
|
||||
value = double.Parse(input[..^2], CultureInfo.InvariantCulture);
|
||||
return FromMegabytes(value);
|
||||
}
|
||||
|
||||
if (input.EndsWith("GB", StringComparison.InvariantCultureIgnoreCase))
|
||||
{
|
||||
value = double.Parse(input[..^2], CultureInfo.InvariantCulture);
|
||||
return FromGigabytes(value);
|
||||
}
|
||||
|
||||
throw new FormatException("invalid size format | only KB, MB and GB are supported");
|
||||
}
|
||||
|
||||
public static bool TryParse(string? input, out ByteSize? result)
|
||||
{
|
||||
result = default;
|
||||
|
||||
if (string.IsNullOrWhiteSpace(input))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
input = input.Trim().ToUpperInvariant();
|
||||
|
||||
if (input.EndsWith("KB", StringComparison.InvariantCultureIgnoreCase) &&
|
||||
double.TryParse(input[..^2], NumberStyles.Float, CultureInfo.InvariantCulture, out double kb))
|
||||
{
|
||||
result = FromKilobytes(kb);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (input.EndsWith("MB", StringComparison.InvariantCultureIgnoreCase) &&
|
||||
double.TryParse(input[..^2], NumberStyles.Float, CultureInfo.InvariantCulture, out double mb))
|
||||
{
|
||||
result = FromMegabytes(mb);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (input.EndsWith("GB", StringComparison.InvariantCultureIgnoreCase) &&
|
||||
double.TryParse(input[..^2], NumberStyles.Float, CultureInfo.InvariantCulture, out double gb))
|
||||
{
|
||||
result = FromGigabytes(gb);
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
public override string ToString() =>
|
||||
Bytes switch
|
||||
{
|
||||
>= BytesPerGB => $"{Bytes / (double)BytesPerGB:0.##} GB",
|
||||
>= BytesPerMB => $"{Bytes / (double)BytesPerMB:0.##} MB",
|
||||
_ => $"{Bytes / (double)BytesPerKB:0.##} KB"
|
||||
};
|
||||
|
||||
public int CompareTo(ByteSize other) => Bytes.CompareTo(other.Bytes);
|
||||
public bool Equals(ByteSize other) => Bytes == other.Bytes;
|
||||
|
||||
public override bool Equals(object? obj) => obj is ByteSize other && Equals(other);
|
||||
public override int GetHashCode() => Bytes.GetHashCode();
|
||||
|
||||
public static bool operator ==(ByteSize left, ByteSize right) => left.Equals(right);
|
||||
public static bool operator !=(ByteSize left, ByteSize right) => !(left == right);
|
||||
public static bool operator <(ByteSize left, ByteSize right) => left.Bytes < right.Bytes;
|
||||
public static bool operator >(ByteSize left, ByteSize right) => left.Bytes > right.Bytes;
|
||||
public static bool operator <=(ByteSize left, ByteSize right) => left.Bytes <= right.Bytes;
|
||||
public static bool operator >=(ByteSize left, ByteSize right) => left.Bytes >= right.Bytes;
|
||||
|
||||
public static ByteSize operator +(ByteSize left, ByteSize right) => new(left.Bytes + right.Bytes);
|
||||
public static ByteSize operator -(ByteSize left, ByteSize right) => new(Math.Max(left.Bytes - right.Bytes, 0));
|
||||
}
|
||||
66
code/Common/CustomDataTypes/SmartTimeSpan.cs
Normal file
66
code/Common/CustomDataTypes/SmartTimeSpan.cs
Normal file
@@ -0,0 +1,66 @@
|
||||
using System.Text;
|
||||
|
||||
namespace Common.CustomDataTypes;
|
||||
|
||||
public readonly struct SmartTimeSpan : IComparable<SmartTimeSpan>, IEquatable<SmartTimeSpan>
|
||||
{
|
||||
public TimeSpan Time { get; }
|
||||
|
||||
public SmartTimeSpan(TimeSpan time)
|
||||
{
|
||||
Time = time;
|
||||
}
|
||||
|
||||
public override string ToString()
|
||||
{
|
||||
if (Time == TimeSpan.Zero)
|
||||
{
|
||||
return "0 seconds";
|
||||
}
|
||||
|
||||
StringBuilder sb = new();
|
||||
|
||||
if (Time.Days > 0)
|
||||
{
|
||||
sb.Append($"{Time.Days} day{(Time.Days > 1 ? "s" : "")} ");
|
||||
}
|
||||
|
||||
if (Time.Hours > 0)
|
||||
{
|
||||
sb.Append($"{Time.Hours} hour{(Time.Hours > 1 ? "s" : "")} ");
|
||||
}
|
||||
|
||||
if (Time.Minutes > 0)
|
||||
{
|
||||
sb.Append($"{Time.Minutes} minute{(Time.Minutes > 1 ? "s" : "")} ");
|
||||
}
|
||||
|
||||
if (Time.Seconds > 0)
|
||||
{
|
||||
sb.Append($"{Time.Seconds} second{(Time.Seconds > 1 ? "s" : "")}");
|
||||
}
|
||||
|
||||
return sb.ToString().TrimEnd();
|
||||
}
|
||||
|
||||
public static SmartTimeSpan FromMinutes(double minutes) => new(TimeSpan.FromMinutes(minutes));
|
||||
public static SmartTimeSpan FromSeconds(double seconds) => new(TimeSpan.FromSeconds(seconds));
|
||||
public static SmartTimeSpan FromHours(double hours) => new(TimeSpan.FromHours(hours));
|
||||
public static SmartTimeSpan FromDays(double days) => new(TimeSpan.FromDays(days));
|
||||
|
||||
public int CompareTo(SmartTimeSpan other) => Time.CompareTo(other.Time);
|
||||
public bool Equals(SmartTimeSpan other) => Time.Equals(other.Time);
|
||||
|
||||
public override bool Equals(object? obj) => obj is SmartTimeSpan other && Equals(other);
|
||||
public override int GetHashCode() => Time.GetHashCode();
|
||||
|
||||
public static bool operator ==(SmartTimeSpan left, SmartTimeSpan right) => left.Equals(right);
|
||||
public static bool operator !=(SmartTimeSpan left, SmartTimeSpan right) => !left.Equals(right);
|
||||
public static bool operator <(SmartTimeSpan left, SmartTimeSpan right) => left.Time < right.Time;
|
||||
public static bool operator >(SmartTimeSpan left, SmartTimeSpan right) => left.Time > right.Time;
|
||||
public static bool operator <=(SmartTimeSpan left, SmartTimeSpan right) => left.Time <= right.Time;
|
||||
public static bool operator >=(SmartTimeSpan left, SmartTimeSpan right) => left.Time >= right.Time;
|
||||
|
||||
public static SmartTimeSpan operator +(SmartTimeSpan left, SmartTimeSpan right) => new(left.Time + right.Time);
|
||||
public static SmartTimeSpan operator -(SmartTimeSpan left, SmartTimeSpan right) => new(left.Time - right.Time);
|
||||
}
|
||||
@@ -2,7 +2,13 @@
|
||||
|
||||
public enum DeleteReason
|
||||
{
|
||||
None,
|
||||
Stalled,
|
||||
ImportFailed,
|
||||
AllFilesBlocked
|
||||
DownloadingMetadata,
|
||||
SlowSpeed,
|
||||
SlowTime,
|
||||
AllFilesSkipped,
|
||||
AllFilesSkippedByQBit,
|
||||
AllFilesBlocked,
|
||||
}
|
||||
@@ -3,5 +3,8 @@
|
||||
public enum StrikeType
|
||||
{
|
||||
Stalled,
|
||||
ImportFailed
|
||||
DownloadingMetadata,
|
||||
ImportFailed,
|
||||
SlowSpeed,
|
||||
SlowTime,
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
namespace Domain.Models.Cache;
|
||||
|
||||
public sealed record CacheItem
|
||||
public sealed record StalledCacheItem
|
||||
{
|
||||
/// <summary>
|
||||
/// The amount of bytes that have been downloaded.
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
namespace Domain.Models.Deluge.Response;
|
||||
|
||||
public sealed record TorrentStatus
|
||||
public sealed record DownloadStatus
|
||||
{
|
||||
public string? Hash { get; init; }
|
||||
|
||||
@@ -12,8 +12,14 @@ public sealed record TorrentStatus
|
||||
|
||||
public ulong Eta { get; init; }
|
||||
|
||||
[JsonProperty("download_payload_rate")]
|
||||
public long DownloadSpeed { get; init; }
|
||||
|
||||
public bool Private { get; init; }
|
||||
|
||||
[JsonProperty("total_size")]
|
||||
public long Size { get; init; }
|
||||
|
||||
[JsonProperty("total_done")]
|
||||
public long TotalDone { get; init; }
|
||||
|
||||
@@ -25,6 +25,7 @@ public static class MainDI
|
||||
{
|
||||
config.AddConsumer<NotificationConsumer<FailedImportStrikeNotification>>();
|
||||
config.AddConsumer<NotificationConsumer<StalledStrikeNotification>>();
|
||||
config.AddConsumer<NotificationConsumer<SlowStrikeNotification>>();
|
||||
config.AddConsumer<NotificationConsumer<QueueItemDeletedNotification>>();
|
||||
config.AddConsumer<NotificationConsumer<DownloadCleanedNotification>>();
|
||||
|
||||
@@ -34,6 +35,7 @@ public static class MainDI
|
||||
{
|
||||
e.ConfigureConsumer<NotificationConsumer<FailedImportStrikeNotification>>(context);
|
||||
e.ConfigureConsumer<NotificationConsumer<StalledStrikeNotification>>(context);
|
||||
e.ConfigureConsumer<NotificationConsumer<SlowStrikeNotification>>(context);
|
||||
e.ConfigureConsumer<NotificationConsumer<QueueItemDeletedNotification>>(context);
|
||||
e.ConfigureConsumer<NotificationConsumer<DownloadCleanedNotification>>(context);
|
||||
e.ConcurrentMessageLimit = 1;
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
"HTTP_MAX_RETRIES": 0,
|
||||
"HTTP_TIMEOUT": 10,
|
||||
"Logging": {
|
||||
"LogLevel": "Debug",
|
||||
"LogLevel": "Verbose",
|
||||
"Enhanced": true,
|
||||
"File": {
|
||||
"Enabled": false,
|
||||
@@ -34,7 +34,14 @@
|
||||
"STALLED_MAX_STRIKES": 5,
|
||||
"STALLED_RESET_STRIKES_ON_PROGRESS": true,
|
||||
"STALLED_IGNORE_PRIVATE": true,
|
||||
"STALLED_DELETE_PRIVATE": false
|
||||
"STALLED_DELETE_PRIVATE": false,
|
||||
"SLOW_MAX_STRIKES": 5,
|
||||
"SLOW_RESET_STRIKES_ON_PROGRESS": true,
|
||||
"SLOW_IGNORE_PRIVATE": false,
|
||||
"SLOW_DELETE_PRIVATE": false,
|
||||
"SLOW_MIN_SPEED": "1MB",
|
||||
"SLOW_MAX_TIME": 20,
|
||||
"SLOW_IGNORE_ABOVE_SIZE": "4GB"
|
||||
},
|
||||
"DownloadCleaner": {
|
||||
"Enabled": false,
|
||||
@@ -52,15 +59,18 @@
|
||||
"DOWNLOAD_CLIENT": "qbittorrent",
|
||||
"qBittorrent": {
|
||||
"Url": "http://localhost:8080",
|
||||
"URL_BASE": "",
|
||||
"Username": "test",
|
||||
"Password": "testing"
|
||||
},
|
||||
"Deluge": {
|
||||
"Url": "http://localhost:8112",
|
||||
"URL_BASE": "",
|
||||
"Password": "testing"
|
||||
},
|
||||
"Transmission": {
|
||||
"Url": "http://localhost:9091",
|
||||
"URL_BASE": "transmission",
|
||||
"Username": "test",
|
||||
"Password": "testing"
|
||||
},
|
||||
@@ -107,6 +117,7 @@
|
||||
"Notifiarr": {
|
||||
"ON_IMPORT_FAILED_STRIKE": true,
|
||||
"ON_STALLED_STRIKE": true,
|
||||
"ON_SLOW_STRIKE": true,
|
||||
"ON_QUEUE_ITEM_DELETED": true,
|
||||
"ON_DOWNLOAD_CLEANED": true,
|
||||
"API_KEY": "",
|
||||
|
||||
@@ -42,15 +42,18 @@
|
||||
"DOWNLOAD_CLIENT": "none",
|
||||
"qBittorrent": {
|
||||
"Url": "http://localhost:8080",
|
||||
"URL_BASE": "",
|
||||
"Username": "",
|
||||
"Password": ""
|
||||
},
|
||||
"Deluge": {
|
||||
"Url": "http://localhost:8112",
|
||||
"URL_BASE": "",
|
||||
"Password": "testing"
|
||||
},
|
||||
"Transmission": {
|
||||
"Url": "http://localhost:9091",
|
||||
"URL_BASE": "transmission",
|
||||
"Username": "test",
|
||||
"Password": "testing"
|
||||
},
|
||||
@@ -97,6 +100,7 @@
|
||||
"Notifiarr": {
|
||||
"ON_IMPORT_FAILED_STRIKE": false,
|
||||
"ON_STALLED_STRIKE": false,
|
||||
"ON_SLOW_STRIKE": false,
|
||||
"ON_QUEUE_ITEM_DELETED": false,
|
||||
"ON_DOWNLOAD_CLEANED": false,
|
||||
"API_KEY": "",
|
||||
|
||||
@@ -39,7 +39,7 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||
});
|
||||
|
||||
// Act
|
||||
sut.ResetStrikesOnProgress("test-hash", 100);
|
||||
sut.ResetStalledStrikesOnProgress("test-hash", 100);
|
||||
|
||||
// Assert
|
||||
_fixture.Cache.ReceivedCalls().ShouldBeEmpty();
|
||||
@@ -50,19 +50,19 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||
{
|
||||
// Arrange
|
||||
const string hash = "test-hash";
|
||||
CacheItem cacheItem = new CacheItem { Downloaded = 100 };
|
||||
StalledCacheItem stalledCacheItem = new StalledCacheItem { Downloaded = 100 };
|
||||
|
||||
_fixture.Cache.TryGetValue(Arg.Any<object>(), out Arg.Any<object?>())
|
||||
.Returns(x =>
|
||||
{
|
||||
x[1] = cacheItem;
|
||||
x[1] = stalledCacheItem;
|
||||
return true;
|
||||
});
|
||||
|
||||
TestDownloadService sut = _fixture.CreateSut();
|
||||
|
||||
// Act
|
||||
sut.ResetStrikesOnProgress(hash, 200);
|
||||
sut.ResetStalledStrikesOnProgress(hash, 200);
|
||||
|
||||
// Assert
|
||||
_fixture.Cache.Received(1).Remove(CacheKeys.Strike(StrikeType.Stalled, hash));
|
||||
@@ -73,20 +73,20 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||
{
|
||||
// Arrange
|
||||
const string hash = "test-hash";
|
||||
CacheItem cacheItem = new CacheItem { Downloaded = 200 };
|
||||
StalledCacheItem stalledCacheItem = new StalledCacheItem { Downloaded = 200 };
|
||||
|
||||
_fixture.Cache
|
||||
.TryGetValue(Arg.Any<object>(), out Arg.Any<object?>())
|
||||
.Returns(x =>
|
||||
{
|
||||
x[1] = cacheItem;
|
||||
x[1] = stalledCacheItem;
|
||||
return true;
|
||||
});
|
||||
|
||||
TestDownloadService sut = _fixture.CreateSut();
|
||||
|
||||
// Act
|
||||
sut.ResetStrikesOnProgress(hash, 100);
|
||||
sut.ResetStalledStrikesOnProgress(hash, 100);
|
||||
|
||||
// Assert
|
||||
_fixture.Cache.DidNotReceive().Remove(Arg.Any<object>());
|
||||
@@ -98,27 +98,6 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||
public StrikeAndCheckLimitTests(DownloadServiceFixture fixture) : base(fixture)
|
||||
{
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ShouldDelegateCallToStriker()
|
||||
{
|
||||
// Arrange
|
||||
const string hash = "test-hash";
|
||||
const string itemName = "test-item";
|
||||
_fixture.Striker.StrikeAndCheckLimit(hash, itemName, 3, StrikeType.Stalled)
|
||||
.Returns(true);
|
||||
|
||||
TestDownloadService sut = _fixture.CreateSut();
|
||||
|
||||
// Act
|
||||
bool result = await sut.StrikeAndCheckLimit(hash, itemName);
|
||||
|
||||
// Assert
|
||||
result.ShouldBeTrue();
|
||||
await _fixture.Striker
|
||||
.Received(1)
|
||||
.StrikeAndCheckLimit(hash, itemName, 3, StrikeType.Stalled);
|
||||
}
|
||||
}
|
||||
|
||||
public class ShouldCleanDownloadTests : DownloadServiceTests
|
||||
|
||||
@@ -3,6 +3,7 @@ using System.Text.RegularExpressions;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Domain.Enums;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.DownloadClient;
|
||||
@@ -35,7 +36,7 @@ public class TestDownloadService : DownloadService
|
||||
|
||||
public override void Dispose() { }
|
||||
public override Task LoginAsync() => Task.CompletedTask;
|
||||
public override Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads) => Task.FromResult(new StalledResult());
|
||||
public override Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads) => Task.FromResult(new DownloadCheckResult());
|
||||
public override Task<BlockFilesResult> BlockUnwantedFilesAsync(string hash, BlocklistType blocklistType,
|
||||
ConcurrentBag<string> patterns, ConcurrentBag<Regex> regexes, IReadOnlyList<string> ignoredDownloads) => Task.FromResult(new BlockFilesResult());
|
||||
public override Task DeleteDownload(string hash) => Task.CompletedTask;
|
||||
@@ -44,7 +45,6 @@ public class TestDownloadService : DownloadService
|
||||
IReadOnlyList<string> ignoredDownloads) => Task.CompletedTask;
|
||||
|
||||
// Expose protected methods for testing
|
||||
public new void ResetStrikesOnProgress(string hash, long downloaded) => base.ResetStrikesOnProgress(hash, downloaded);
|
||||
public new Task<bool> StrikeAndCheckLimit(string hash, string itemName) => base.StrikeAndCheckLimit(hash, itemName);
|
||||
public new void ResetStalledStrikesOnProgress(string hash, long downloaded) => base.ResetStalledStrikesOnProgress(hash, downloaded);
|
||||
public new SeedingCheckResult ShouldCleanDownload(double ratio, TimeSpan seedingTime, Category category) => base.ShouldCleanDownload(ratio, seedingTime, category);
|
||||
}
|
||||
@@ -4,7 +4,7 @@ namespace Infrastructure.Extensions;
|
||||
|
||||
public static class DelugeExtensions
|
||||
{
|
||||
public static bool ShouldIgnore(this TorrentStatus download, IReadOnlyList<string> ignoredDownloads)
|
||||
public static bool ShouldIgnore(this DownloadStatus download, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
foreach (string value in ignoredDownloads)
|
||||
{
|
||||
|
||||
@@ -10,7 +10,7 @@ public static class CacheKeys
|
||||
public static string BlocklistPatterns(InstanceType instanceType) => $"{instanceType.ToString()}_patterns";
|
||||
public static string BlocklistRegexes(InstanceType instanceType) => $"{instanceType.ToString()}_regexes";
|
||||
|
||||
public static string Item(string hash) => $"item_{hash}";
|
||||
public static string StrikeItem(string hash, StrikeType strikeType) => $"item_{hash}_{strikeType.ToString()}";
|
||||
|
||||
public static string IgnoredDownloads(string name) => $"{name}_ignored";
|
||||
}
|
||||
@@ -13,7 +13,7 @@
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="FLM.QBittorrent" Version="1.0.0" />
|
||||
<PackageReference Include="FLM.Transmission" Version="1.0.2" />
|
||||
<PackageReference Include="FLM.Transmission" Version="1.0.3" />
|
||||
<PackageReference Include="Mapster" Version="7.4.0" />
|
||||
<PackageReference Include="MassTransit" Version="8.3.6" />
|
||||
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="9.0.2" />
|
||||
|
||||
@@ -43,9 +43,11 @@ public abstract class ArrClient : IArrClient
|
||||
|
||||
public virtual async Task<QueueListResponse> GetQueueItemsAsync(ArrInstance arrInstance, int page)
|
||||
{
|
||||
Uri uri = new(arrInstance.Url, GetQueueUrlPath(page));
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/{GetQueueUrlPath().TrimStart('/')}";
|
||||
uriBuilder.Query = GetQueueUrlQuery(page);
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uri);
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||
@@ -56,7 +58,7 @@ public abstract class ArrClient : IArrClient
|
||||
}
|
||||
catch
|
||||
{
|
||||
_logger.LogError("queue list failed | {uri}", uri);
|
||||
_logger.LogError("queue list failed | {uri}", uriBuilder.Uri);
|
||||
throw;
|
||||
}
|
||||
|
||||
@@ -65,7 +67,7 @@ public abstract class ArrClient : IArrClient
|
||||
|
||||
if (queueResponse is null)
|
||||
{
|
||||
throw new Exception($"unrecognized queue list response | {uri} | {responseBody}");
|
||||
throw new Exception($"unrecognized queue list response | {uriBuilder.Uri} | {responseBody}");
|
||||
}
|
||||
|
||||
return queueResponse;
|
||||
@@ -112,13 +114,20 @@ public abstract class ArrClient : IArrClient
|
||||
return false;
|
||||
}
|
||||
|
||||
public virtual async Task DeleteQueueItemAsync(ArrInstance arrInstance, QueueRecord record, bool removeFromClient)
|
||||
public virtual async Task DeleteQueueItemAsync(
|
||||
ArrInstance arrInstance,
|
||||
QueueRecord record,
|
||||
bool removeFromClient,
|
||||
DeleteReason deleteReason
|
||||
)
|
||||
{
|
||||
Uri uri = new(arrInstance.Url, GetQueueDeleteUrlPath(record.Id, removeFromClient));
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/{GetQueueDeleteUrlPath(record.Id).TrimStart('/')}";
|
||||
uriBuilder.Query = GetQueueDeleteUrlQuery(removeFromClient);
|
||||
|
||||
try
|
||||
{
|
||||
using HttpRequestMessage request = new(HttpMethod.Delete, uri);
|
||||
using HttpRequestMessage request = new(HttpMethod.Delete, uriBuilder.Uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
HttpResponseMessage? response = await _dryRunInterceptor.InterceptAsync<HttpResponseMessage>(SendRequestAsync, request);
|
||||
@@ -126,15 +135,16 @@ public abstract class ArrClient : IArrClient
|
||||
|
||||
_logger.LogInformation(
|
||||
removeFromClient
|
||||
? "queue item deleted | {url} | {title}"
|
||||
: "queue item removed from arr | {url} | {title}",
|
||||
? "queue item deleted with reason {reason} | {url} | {title}"
|
||||
: "queue item removed from arr with reason {reason} | {url} | {title}",
|
||||
deleteReason.ToString(),
|
||||
arrInstance.Url,
|
||||
record.Title
|
||||
);
|
||||
}
|
||||
catch
|
||||
{
|
||||
_logger.LogError("queue delete failed | {uri} | {title}", uri, record.Title);
|
||||
_logger.LogError("queue delete failed | {uri} | {title}", uriBuilder.Uri, record.Title);
|
||||
throw;
|
||||
}
|
||||
}
|
||||
@@ -152,9 +162,13 @@ public abstract class ArrClient : IArrClient
|
||||
return true;
|
||||
}
|
||||
|
||||
protected abstract string GetQueueUrlPath(int page);
|
||||
protected abstract string GetQueueUrlPath();
|
||||
|
||||
protected abstract string GetQueueDeleteUrlPath(long recordId, bool removeFromClient);
|
||||
protected abstract string GetQueueUrlQuery(int page);
|
||||
|
||||
protected abstract string GetQueueDeleteUrlPath(long recordId);
|
||||
|
||||
protected abstract string GetQueueDeleteUrlQuery(bool removeFromClient);
|
||||
|
||||
protected virtual void SetApiKey(HttpRequestMessage request, string apiKey)
|
||||
{
|
||||
|
||||
@@ -11,7 +11,7 @@ public interface IArrClient
|
||||
|
||||
Task<bool> ShouldRemoveFromQueue(InstanceType instanceType, QueueRecord record, bool isPrivateDownload);
|
||||
|
||||
Task DeleteQueueItemAsync(ArrInstance arrInstance, QueueRecord record, bool removeFromClient);
|
||||
Task DeleteQueueItemAsync(ArrInstance arrInstance, QueueRecord record, bool removeFromClient, DeleteReason deleteReason);
|
||||
|
||||
Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items);
|
||||
|
||||
|
||||
@@ -27,29 +27,42 @@ public class LidarrClient : ArrClient, ILidarrClient
|
||||
{
|
||||
}
|
||||
|
||||
protected override string GetQueueUrlPath(int page)
|
||||
protected override string GetQueueUrlPath()
|
||||
{
|
||||
return $"/api/v1/queue?page={page}&pageSize=200&includeUnknownArtistItems=true&includeArtist=true&includeAlbum=true";
|
||||
return "/api/v1/queue";
|
||||
}
|
||||
|
||||
protected override string GetQueueDeleteUrlPath(long recordId, bool removeFromClient)
|
||||
protected override string GetQueueUrlQuery(int page)
|
||||
{
|
||||
string path = $"/api/v1/queue/{recordId}?blocklist=true&skipRedownload=true&changeCategory=false";
|
||||
return $"page={page}&pageSize=200&includeUnknownArtistItems=true&includeArtist=true&includeAlbum=true";
|
||||
}
|
||||
|
||||
path += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
|
||||
protected override string GetQueueDeleteUrlPath(long recordId)
|
||||
{
|
||||
return $"/api/v1/queue/{recordId}";
|
||||
}
|
||||
|
||||
return path;
|
||||
protected override string GetQueueDeleteUrlQuery(bool removeFromClient)
|
||||
{
|
||||
string query = "blocklist=true&skipRedownload=true&changeCategory=false";
|
||||
query += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
|
||||
|
||||
return query;
|
||||
}
|
||||
|
||||
public override async Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items)
|
||||
{
|
||||
if (items?.Count is null or 0) return;
|
||||
if (items?.Count is null or 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
Uri uri = new(arrInstance.Url, "/api/v1/command");
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v1/command";
|
||||
|
||||
foreach (var command in GetSearchCommands(items))
|
||||
{
|
||||
using HttpRequestMessage request = new(HttpMethod.Post, uri);
|
||||
using HttpRequestMessage request = new(HttpMethod.Post, uriBuilder.Uri);
|
||||
request.Content = new StringContent(
|
||||
JsonConvert.SerializeObject(command, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore }),
|
||||
Encoding.UTF8,
|
||||
@@ -132,8 +145,11 @@ public class LidarrClient : ArrClient, ILidarrClient
|
||||
|
||||
private async Task<List<Album>?> GetAlbumsAsync(ArrInstance arrInstance, List<long> albumIds)
|
||||
{
|
||||
Uri uri = new(arrInstance.Url, $"api/v1/album?{string.Join('&', albumIds.Select(x => $"albumIds={x}"))}");
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uri);
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v1/album";
|
||||
uriBuilder.Query = string.Join('&', albumIds.Select(x => $"albumIds={x}"));
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using var response = await _httpClient.SendAsync(request);
|
||||
|
||||
@@ -27,18 +27,27 @@ public class RadarrClient : ArrClient, IRadarrClient
|
||||
{
|
||||
}
|
||||
|
||||
protected override string GetQueueUrlPath(int page)
|
||||
protected override string GetQueueUrlPath()
|
||||
{
|
||||
return $"/api/v3/queue?page={page}&pageSize=200&includeUnknownMovieItems=true&includeMovie=true";
|
||||
return "/api/v3/queue";
|
||||
}
|
||||
|
||||
protected override string GetQueueDeleteUrlPath(long recordId, bool removeFromClient)
|
||||
protected override string GetQueueUrlQuery(int page)
|
||||
{
|
||||
string path = $"/api/v3/queue/{recordId}?blocklist=true&skipRedownload=true&changeCategory=false";
|
||||
|
||||
path += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
|
||||
return $"page={page}&pageSize=200&includeUnknownMovieItems=true&includeMovie=true";
|
||||
}
|
||||
|
||||
return path;
|
||||
protected override string GetQueueDeleteUrlPath(long recordId)
|
||||
{
|
||||
return $"/api/v3/queue/{recordId}";
|
||||
}
|
||||
|
||||
protected override string GetQueueDeleteUrlQuery(bool removeFromClient)
|
||||
{
|
||||
string query = "blocklist=true&skipRedownload=true&changeCategory=false";
|
||||
query += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
|
||||
|
||||
return query;
|
||||
}
|
||||
|
||||
public override async Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items)
|
||||
@@ -50,14 +59,16 @@ public class RadarrClient : ArrClient, IRadarrClient
|
||||
|
||||
List<long> ids = items.Select(item => item.Id).ToList();
|
||||
|
||||
Uri uri = new(arrInstance.Url, "/api/v3/command");
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/command";
|
||||
|
||||
RadarrCommand command = new()
|
||||
{
|
||||
Name = "MoviesSearch",
|
||||
MovieIds = ids,
|
||||
};
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Post, uri);
|
||||
using HttpRequestMessage request = new(HttpMethod.Post, uriBuilder.Uri);
|
||||
request.Content = new StringContent(
|
||||
JsonConvert.SerializeObject(command),
|
||||
Encoding.UTF8,
|
||||
@@ -135,8 +146,10 @@ public class RadarrClient : ArrClient, IRadarrClient
|
||||
|
||||
private async Task<Movie?> GetMovie(ArrInstance arrInstance, long movieId)
|
||||
{
|
||||
Uri uri = new(arrInstance.Url, $"api/v3/movie/{movieId}");
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uri);
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/movie/{movieId}";
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||
|
||||
@@ -28,18 +28,27 @@ public class SonarrClient : ArrClient, ISonarrClient
|
||||
{
|
||||
}
|
||||
|
||||
protected override string GetQueueUrlPath(int page)
|
||||
protected override string GetQueueUrlPath()
|
||||
{
|
||||
return $"/api/v3/queue?page={page}&pageSize=200&includeUnknownSeriesItems=true&includeSeries=true&includeEpisode=true";
|
||||
return "/api/v3/queue";
|
||||
}
|
||||
|
||||
protected override string GetQueueDeleteUrlPath(long recordId, bool removeFromClient)
|
||||
|
||||
protected override string GetQueueUrlQuery(int page)
|
||||
{
|
||||
string path = $"/api/v3/queue/{recordId}?blocklist=true&skipRedownload=true&changeCategory=false";
|
||||
return $"page={page}&pageSize=200&includeUnknownSeriesItems=true&includeSeries=true&includeEpisode=true";
|
||||
}
|
||||
|
||||
path += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
|
||||
protected override string GetQueueDeleteUrlPath(long recordId)
|
||||
{
|
||||
return $"/api/v3/queue/{recordId}";
|
||||
}
|
||||
|
||||
return path;
|
||||
protected override string GetQueueDeleteUrlQuery(bool removeFromClient)
|
||||
{
|
||||
string query = "blocklist=true&skipRedownload=true&changeCategory=false";
|
||||
query += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
|
||||
|
||||
return query;
|
||||
}
|
||||
|
||||
public override async Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items)
|
||||
@@ -49,11 +58,12 @@ public class SonarrClient : ArrClient, ISonarrClient
|
||||
return;
|
||||
}
|
||||
|
||||
Uri uri = new(arrInstance.Url, "/api/v3/command");
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/command";
|
||||
|
||||
foreach (SonarrCommand command in GetSearchCommands(items.Cast<SonarrSearchItem>().ToHashSet()))
|
||||
{
|
||||
using HttpRequestMessage request = new(HttpMethod.Post, uri);
|
||||
using HttpRequestMessage request = new(HttpMethod.Post, uriBuilder.Uri);
|
||||
request.Content = new StringContent(
|
||||
JsonConvert.SerializeObject(command, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore }),
|
||||
Encoding.UTF8,
|
||||
@@ -199,8 +209,11 @@ public class SonarrClient : ArrClient, ISonarrClient
|
||||
|
||||
private async Task<List<Episode>?> GetEpisodesAsync(ArrInstance arrInstance, List<long> episodeIds)
|
||||
{
|
||||
Uri uri = new(arrInstance.Url, $"api/v3/episode?{string.Join('&', episodeIds.Select(x => $"episodeIds={x}"))}");
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uri);
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/episode";
|
||||
uriBuilder.Query = string.Join('&', episodeIds.Select(x => $"episodeIds={x}"));
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||
@@ -212,8 +225,10 @@ public class SonarrClient : ArrClient, ISonarrClient
|
||||
|
||||
private async Task<Series?> GetSeriesAsync(ArrInstance arrInstance, long seriesId)
|
||||
{
|
||||
Uri uri = new(arrInstance.Url, $"api/v3/series/{seriesId}");
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uri);
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/series/{seriesId}";
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||
|
||||
@@ -142,7 +142,7 @@ public sealed class ContentBlocker : GenericHandler
|
||||
removeFromClient = false;
|
||||
}
|
||||
|
||||
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient);
|
||||
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient, DeleteReason.AllFilesBlocked);
|
||||
await _notifier.NotifyQueueItemDeleted(removeFromClient, DeleteReason.AllFilesBlocked);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -27,12 +27,15 @@ public sealed class DelugeClient
|
||||
"label",
|
||||
"seeding_time",
|
||||
"ratio",
|
||||
"trackers"
|
||||
"trackers",
|
||||
"download_payload_rate",
|
||||
"total_size"
|
||||
];
|
||||
|
||||
public DelugeClient(IOptions<DelugeConfig> config, IHttpClientFactory httpClientFactory)
|
||||
{
|
||||
_config = config.Value;
|
||||
_config.Validate();
|
||||
_httpClient = httpClientFactory.CreateClient(nameof(DelugeService));
|
||||
}
|
||||
|
||||
@@ -77,18 +80,31 @@ public sealed class DelugeClient
|
||||
return torrents.FirstOrDefault();
|
||||
}
|
||||
|
||||
public async Task<TorrentStatus?> GetTorrentStatus(string hash)
|
||||
public async Task<DownloadStatus?> GetTorrentStatus(string hash)
|
||||
{
|
||||
return await SendRequest<TorrentStatus?>(
|
||||
"web.get_torrent_status",
|
||||
hash,
|
||||
Fields
|
||||
);
|
||||
try
|
||||
{
|
||||
return await SendRequest<DownloadStatus?>(
|
||||
"web.get_torrent_status",
|
||||
hash,
|
||||
Fields
|
||||
);
|
||||
}
|
||||
catch (DelugeClientException e)
|
||||
{
|
||||
// Deluge returns an error when the torrent is not found
|
||||
if (e.Message == "AttributeError: 'NoneType' object has no attribute 'call'")
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
public async Task<List<TorrentStatus>?> GetStatusForAllTorrents()
|
||||
public async Task<List<DownloadStatus>?> GetStatusForAllTorrents()
|
||||
{
|
||||
Dictionary<string, TorrentStatus>? downloads = await SendRequest<Dictionary<string, TorrentStatus>?>(
|
||||
Dictionary<string, DownloadStatus>? downloads = await SendRequest<Dictionary<string, DownloadStatus>?>(
|
||||
"core.get_torrents_status",
|
||||
"",
|
||||
Fields
|
||||
@@ -121,8 +137,12 @@ public sealed class DelugeClient
|
||||
{
|
||||
StringContent content = new StringContent(json);
|
||||
content.Headers.ContentType = new MediaTypeWithQualityHeaderValue("application/json");
|
||||
|
||||
var responseMessage = await _httpClient.PostAsync(new Uri(_config.Url, "/json"), content);
|
||||
|
||||
UriBuilder uriBuilder = new(_config.Url);
|
||||
uriBuilder.Path = string.IsNullOrEmpty(_config.UrlBase)
|
||||
? $"{uriBuilder.Path.TrimEnd('/')}/json"
|
||||
: $"{uriBuilder.Path.TrimEnd('/')}/{_config.UrlBase.TrimStart('/').TrimEnd('/')}/json";
|
||||
var responseMessage = await _httpClient.PostAsync(uriBuilder.Uri, content);
|
||||
responseMessage.EnsureSuccessStatusCode();
|
||||
|
||||
var responseJson = await responseMessage.Content.ReadAsStringAsync();
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Globalization;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Attributes;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.DownloadClient;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Common.CustomDataTypes;
|
||||
using Domain.Enums;
|
||||
using Domain.Models.Deluge.Response;
|
||||
using Infrastructure.Extensions;
|
||||
@@ -50,14 +52,14 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
public override async Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
hash = hash.ToLowerInvariant();
|
||||
|
||||
DelugeContents? contents = null;
|
||||
StalledResult result = new();
|
||||
DownloadCheckResult result = new();
|
||||
|
||||
TorrentStatus? download = await _client.GetTorrentStatus(hash);
|
||||
DownloadStatus? download = await _client.GetTorrentStatus(hash);
|
||||
|
||||
if (download?.Hash is null)
|
||||
{
|
||||
@@ -65,6 +67,8 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
return result;
|
||||
}
|
||||
|
||||
result.IsPrivate = download.Private;
|
||||
|
||||
if (ignoredDownloads.Count > 0 && download.ShouldIgnore(ignoredDownloads))
|
||||
{
|
||||
_logger.LogInformation("skip | download is ignored | {name}", download.Name);
|
||||
@@ -79,6 +83,7 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
{
|
||||
_logger.LogDebug(exception, "failed to find torrent {hash} in the download client", hash);
|
||||
}
|
||||
|
||||
|
||||
bool shouldRemove = contents?.Contents?.Count > 0;
|
||||
|
||||
@@ -92,17 +97,15 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
|
||||
if (shouldRemove)
|
||||
{
|
||||
result.DeleteReason = DeleteReason.AllFilesBlocked;
|
||||
// remove if all files are unwanted
|
||||
result.ShouldRemove = true;
|
||||
result.DeleteReason = DeleteReason.AllFilesSkipped;
|
||||
return result;
|
||||
}
|
||||
|
||||
result.ShouldRemove = shouldRemove || await IsItemStuckAndShouldRemove(download);
|
||||
result.IsPrivate = download.Private;
|
||||
// remove if download is stuck
|
||||
(result.ShouldRemove, result.DeleteReason) = await EvaluateDownloadRemoval(download);
|
||||
|
||||
if (!shouldRemove && result.ShouldRemove)
|
||||
{
|
||||
result.DeleteReason = DeleteReason.Stalled;
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -114,7 +117,7 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
{
|
||||
hash = hash.ToLowerInvariant();
|
||||
|
||||
TorrentStatus? download = await _client.GetTorrentStatus(hash);
|
||||
DownloadStatus? download = await _client.GetTorrentStatus(hash);
|
||||
BlockFilesResult result = new();
|
||||
|
||||
if (download?.Hash is null)
|
||||
@@ -123,9 +126,6 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
return result;
|
||||
}
|
||||
|
||||
var ceva = await _client.GetTorrentExtended(hash);
|
||||
|
||||
|
||||
if (ignoredDownloads.Count > 0 && download.ShouldIgnore(ignoredDownloads))
|
||||
{
|
||||
_logger.LogInformation("skip | download is ignored | {name}", download.Name);
|
||||
@@ -222,7 +222,7 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
public override async Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes,
|
||||
IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
foreach (TorrentStatus download in downloads)
|
||||
foreach (DownloadStatus download in downloads)
|
||||
{
|
||||
if (string.IsNullOrEmpty(download.Hash))
|
||||
{
|
||||
@@ -295,33 +295,90 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
await _client.ChangeFilesPriority(hash, sortedPriorities);
|
||||
}
|
||||
|
||||
private async Task<bool> IsItemStuckAndShouldRemove(TorrentStatus status)
|
||||
private async Task<(bool, DeleteReason)> EvaluateDownloadRemoval(DownloadStatus status)
|
||||
{
|
||||
(bool ShouldRemove, DeleteReason Reason) result = await CheckIfSlow(status);
|
||||
|
||||
if (result.ShouldRemove)
|
||||
{
|
||||
return result;
|
||||
}
|
||||
|
||||
return await CheckIfStuck(status);
|
||||
}
|
||||
|
||||
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfSlow(DownloadStatus download)
|
||||
{
|
||||
if (_queueCleanerConfig.SlowMaxStrikes is 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.State is null || !download.State.Equals("Downloading", StringComparison.InvariantCultureIgnoreCase))
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.DownloadSpeed <= 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.SlowIgnorePrivate && download.Private)
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip slow check | download is private | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.Size > (_queueCleanerConfig.SlowIgnoreAboveSizeByteSize?.Bytes ?? long.MaxValue))
|
||||
{
|
||||
_logger.LogDebug("skip slow check | download is too large | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
ByteSize minSpeed = _queueCleanerConfig.SlowMinSpeedByteSize;
|
||||
ByteSize currentSpeed = new ByteSize(download.DownloadSpeed);
|
||||
SmartTimeSpan maxTime = SmartTimeSpan.FromHours(_queueCleanerConfig.SlowMaxTime);
|
||||
SmartTimeSpan currentTime = SmartTimeSpan.FromSeconds(download.Eta);
|
||||
|
||||
return await CheckIfSlow(
|
||||
download.Hash!,
|
||||
download.Name!,
|
||||
minSpeed,
|
||||
currentSpeed,
|
||||
maxTime,
|
||||
currentTime
|
||||
);
|
||||
}
|
||||
|
||||
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfStuck(DownloadStatus status)
|
||||
{
|
||||
if (_queueCleanerConfig.StalledMaxStrikes is 0)
|
||||
{
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.StalledIgnorePrivate && status.Private)
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip stalled check | download is private | {name}", status.Name);
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (status.State is null || !status.State.Equals("Downloading", StringComparison.InvariantCultureIgnoreCase))
|
||||
{
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (status.Eta > 0)
|
||||
{
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
ResetStrikesOnProgress(status.Hash!, status.TotalDone);
|
||||
|
||||
return await StrikeAndCheckLimit(status.Hash!, status.Name!);
|
||||
ResetStalledStrikesOnProgress(status.Hash!, status.TotalDone);
|
||||
|
||||
return (await _striker.StrikeAndCheckLimit(status.Hash!, status.Name!, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled), DeleteReason.Stalled);
|
||||
}
|
||||
|
||||
private static void ProcessFiles(Dictionary<string, DelugeFileOrDirectory>? contents, Action<string, DelugeFileOrDirectory> processFile)
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
namespace Infrastructure.Verticals.DownloadClient;
|
||||
|
||||
public sealed record StalledResult
|
||||
public sealed record DownloadCheckResult
|
||||
{
|
||||
/// <summary>
|
||||
/// True if the download should be removed; otherwise false.
|
||||
@@ -3,6 +3,7 @@ using System.Text.RegularExpressions;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Common.CustomDataTypes;
|
||||
using Common.Helpers;
|
||||
using Domain.Enums;
|
||||
using Domain.Models.Cache;
|
||||
@@ -60,7 +61,7 @@ public abstract class DownloadService : IDownloadService
|
||||
|
||||
public abstract Task LoginAsync();
|
||||
|
||||
public abstract Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads);
|
||||
public abstract Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads);
|
||||
|
||||
/// <inheritdoc/>
|
||||
public abstract Task<BlockFilesResult> BlockUnwantedFilesAsync(string hash,
|
||||
@@ -78,32 +79,104 @@ public abstract class DownloadService : IDownloadService
|
||||
public abstract Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes,
|
||||
IReadOnlyList<string> ignoredDownloads);
|
||||
|
||||
protected void ResetStrikesOnProgress(string hash, long downloaded)
|
||||
protected void ResetStalledStrikesOnProgress(string hash, long downloaded)
|
||||
{
|
||||
if (!_queueCleanerConfig.StalledResetStrikesOnProgress)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (_cache.TryGetValue(CacheKeys.Item(hash), out CacheItem? cachedItem) && cachedItem is not null && downloaded > cachedItem.Downloaded)
|
||||
|
||||
if (_cache.TryGetValue(CacheKeys.StrikeItem(hash, StrikeType.Stalled), out StalledCacheItem? cachedItem) &&
|
||||
cachedItem is not null && downloaded > cachedItem.Downloaded)
|
||||
{
|
||||
// cache item found
|
||||
_cache.Remove(CacheKeys.Strike(StrikeType.Stalled, hash));
|
||||
_logger.LogDebug("resetting strikes for {hash} due to progress", hash);
|
||||
_logger.LogDebug("resetting stalled strikes for {hash} due to progress", hash);
|
||||
}
|
||||
|
||||
_cache.Set(CacheKeys.Item(hash), new CacheItem { Downloaded = downloaded }, _cacheOptions);
|
||||
_cache.Set(CacheKeys.StrikeItem(hash, StrikeType.Stalled), new StalledCacheItem { Downloaded = downloaded }, _cacheOptions);
|
||||
}
|
||||
|
||||
protected void ResetSlowSpeedStrikesOnProgress(string downloadName, string hash)
|
||||
{
|
||||
if (!_queueCleanerConfig.SlowResetStrikesOnProgress)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
string key = CacheKeys.Strike(StrikeType.SlowSpeed, hash);
|
||||
|
||||
if (!_cache.TryGetValue(key, out object? value) || value is null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
_cache.Remove(key);
|
||||
_logger.LogDebug("resetting slow speed strikes due to progress | {name}", downloadName);
|
||||
}
|
||||
|
||||
protected void ResetSlowTimeStrikesOnProgress(string downloadName, string hash)
|
||||
{
|
||||
if (!_queueCleanerConfig.SlowResetStrikesOnProgress)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
string key = CacheKeys.Strike(StrikeType.SlowTime, hash);
|
||||
|
||||
if (!_cache.TryGetValue(key, out object? value) || value is null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
_cache.Remove(key);
|
||||
_logger.LogDebug("resetting slow time strikes due to progress | {name}", downloadName);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Strikes an item and checks if the limit has been reached.
|
||||
/// </summary>
|
||||
/// <param name="hash">The torrent hash.</param>
|
||||
/// <param name="itemName">The name or title of the item.</param>
|
||||
/// <returns>True if the limit has been reached; otherwise, false.</returns>
|
||||
protected async Task<bool> StrikeAndCheckLimit(string hash, string itemName)
|
||||
protected async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfSlow(
|
||||
string downloadHash,
|
||||
string downloadName,
|
||||
ByteSize minSpeed,
|
||||
ByteSize currentSpeed,
|
||||
SmartTimeSpan maxTime,
|
||||
SmartTimeSpan currentTime
|
||||
)
|
||||
{
|
||||
return await _striker.StrikeAndCheckLimit(hash, itemName, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled);
|
||||
if (minSpeed.Bytes > 0 && currentSpeed < minSpeed)
|
||||
{
|
||||
_logger.LogTrace("slow speed | {speed}/s | {name}", currentSpeed.ToString(), downloadName);
|
||||
|
||||
bool shouldRemove = await _striker
|
||||
.StrikeAndCheckLimit(downloadHash, downloadName, _queueCleanerConfig.SlowMaxStrikes, StrikeType.SlowSpeed);
|
||||
|
||||
if (shouldRemove)
|
||||
{
|
||||
return (true, DeleteReason.SlowSpeed);
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
ResetSlowSpeedStrikesOnProgress(downloadName, downloadHash);
|
||||
}
|
||||
|
||||
if (maxTime.Time > TimeSpan.Zero && currentTime > maxTime)
|
||||
{
|
||||
_logger.LogTrace("slow estimated time | {time} | {name}", currentTime.ToString(), downloadName);
|
||||
|
||||
bool shouldRemove = await _striker
|
||||
.StrikeAndCheckLimit(downloadHash, downloadName, _queueCleanerConfig.SlowMaxStrikes, StrikeType.SlowTime);
|
||||
|
||||
if (shouldRemove)
|
||||
{
|
||||
return (true, DeleteReason.SlowTime);
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
ResetSlowTimeStrikesOnProgress(downloadName, downloadHash);
|
||||
}
|
||||
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
protected SeedingCheckResult ShouldCleanDownload(double ratio, TimeSpan seedingTime, Category category)
|
||||
|
||||
@@ -28,7 +28,7 @@ public class DummyDownloadService : DownloadService
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
|
||||
public override Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
public override Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
@@ -15,7 +15,7 @@ public interface IDownloadService : IDisposable
|
||||
/// </summary>
|
||||
/// <param name="hash">The download hash.</param>
|
||||
/// <param name="ignoredDownloads">Downloads to ignore from processing.</param>
|
||||
public Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads);
|
||||
public Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads);
|
||||
|
||||
/// <summary>
|
||||
/// Blocks unwanted files from being fully downloaded.
|
||||
|
||||
@@ -5,6 +5,7 @@ using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.DownloadClient;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Common.CustomDataTypes;
|
||||
using Common.Helpers;
|
||||
using Domain.Enums;
|
||||
using Infrastructure.Extensions;
|
||||
@@ -45,7 +46,11 @@ public class QBitService : DownloadService, IQBitService
|
||||
{
|
||||
_config = config.Value;
|
||||
_config.Validate();
|
||||
_client = new(httpClientFactory.CreateClient(Constants.HttpClientWithRetryName), _config.Url);
|
||||
UriBuilder uriBuilder = new(_config.Url);
|
||||
uriBuilder.Path = string.IsNullOrEmpty(_config.UrlBase)
|
||||
? uriBuilder.Path
|
||||
: $"{uriBuilder.Path.TrimEnd('/')}/{_config.UrlBase.TrimStart('/')}";
|
||||
_client = new(httpClientFactory.CreateClient(Constants.HttpClientWithRetryName), uriBuilder.Uri);
|
||||
}
|
||||
|
||||
public override async Task LoginAsync()
|
||||
@@ -59,9 +64,9 @@ public class QBitService : DownloadService, IQBitService
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
public override async Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
StalledResult result = new();
|
||||
DownloadCheckResult result = new();
|
||||
TorrentInfo? download = (await _client.GetTorrentListAsync(new TorrentListQuery { Hashes = [hash] }))
|
||||
.FirstOrDefault();
|
||||
|
||||
@@ -92,30 +97,25 @@ public class QBitService : DownloadService, IQBitService
|
||||
bool.TryParse(dictValue?.ToString(), out bool boolValue)
|
||||
&& boolValue;
|
||||
|
||||
// if all files were blocked by qBittorrent
|
||||
if (download is { CompletionOn: not null, Downloaded: null or 0 })
|
||||
{
|
||||
result.ShouldRemove = true;
|
||||
result.DeleteReason = DeleteReason.AllFilesBlocked;
|
||||
return result;
|
||||
}
|
||||
|
||||
IReadOnlyList<TorrentContent>? files = await _client.GetTorrentContentsAsync(hash);
|
||||
|
||||
// if all files are marked as skip
|
||||
if (files?.Count is > 0 && files.All(x => x.Priority is TorrentContentPriority.Skip))
|
||||
{
|
||||
result.ShouldRemove = true;
|
||||
result.DeleteReason = DeleteReason.AllFilesBlocked;
|
||||
|
||||
// if all files were blocked by qBittorrent
|
||||
if (download is { CompletionOn: not null, Downloaded: null or 0 })
|
||||
{
|
||||
result.DeleteReason = DeleteReason.AllFilesSkippedByQBit;
|
||||
return result;
|
||||
}
|
||||
|
||||
// remove if all files are unwanted
|
||||
result.DeleteReason = DeleteReason.AllFilesSkipped;
|
||||
return result;
|
||||
}
|
||||
|
||||
result.ShouldRemove = await IsItemStuckAndShouldRemove(download, result.IsPrivate);
|
||||
|
||||
if (result.ShouldRemove)
|
||||
{
|
||||
result.DeleteReason = DeleteReason.Stalled;
|
||||
}
|
||||
(result.ShouldRemove, result.DeleteReason) = await EvaluateDownloadRemoval(download, result.IsPrivate);
|
||||
|
||||
return result;
|
||||
}
|
||||
@@ -333,30 +333,96 @@ public class QBitService : DownloadService, IQBitService
|
||||
_client.Dispose();
|
||||
}
|
||||
|
||||
private async Task<bool> IsItemStuckAndShouldRemove(TorrentInfo torrent, bool isPrivate)
|
||||
private async Task<(bool, DeleteReason)> EvaluateDownloadRemoval(TorrentInfo torrent, bool isPrivate)
|
||||
{
|
||||
(bool ShouldRemove, DeleteReason Reason) result = await CheckIfSlow(torrent, isPrivate);
|
||||
|
||||
if (result.ShouldRemove)
|
||||
{
|
||||
return result;
|
||||
}
|
||||
|
||||
return await CheckIfStuck(torrent, isPrivate);
|
||||
}
|
||||
|
||||
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfSlow(TorrentInfo download, bool isPrivate)
|
||||
{
|
||||
if (_queueCleanerConfig.SlowMaxStrikes is 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.State is not (TorrentState.Downloading or TorrentState.ForcedDownload))
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.DownloadSpeed <= 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.SlowIgnorePrivate && isPrivate)
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip slow check | download is private | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.Size > (_queueCleanerConfig.SlowIgnoreAboveSizeByteSize?.Bytes ?? long.MaxValue))
|
||||
{
|
||||
_logger.LogDebug("skip slow check | download is too large | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
ByteSize minSpeed = _queueCleanerConfig.SlowMinSpeedByteSize;
|
||||
ByteSize currentSpeed = new ByteSize(download.DownloadSpeed);
|
||||
SmartTimeSpan maxTime = SmartTimeSpan.FromHours(_queueCleanerConfig.SlowMaxTime);
|
||||
SmartTimeSpan currentTime = new SmartTimeSpan(download.EstimatedTime ?? TimeSpan.Zero);
|
||||
|
||||
return await CheckIfSlow(
|
||||
download.Hash,
|
||||
download.Name,
|
||||
minSpeed,
|
||||
currentSpeed,
|
||||
maxTime,
|
||||
currentTime
|
||||
);
|
||||
}
|
||||
|
||||
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfStuck(TorrentInfo torrent, bool isPrivate)
|
||||
{
|
||||
if (_queueCleanerConfig.StalledMaxStrikes is 0)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.StalledIgnorePrivate && isPrivate)
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip stalled check | download is private | {name}", torrent.Name);
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (torrent.State is not TorrentState.StalledDownload and not TorrentState.FetchingMetadata
|
||||
and not TorrentState.ForcedFetchingMetadata)
|
||||
{
|
||||
// ignore other states
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.StalledIgnorePrivate && isPrivate)
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip stalled check | download is private | {name}", torrent.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (torrent.State is TorrentState.StalledDownload)
|
||||
{
|
||||
_logger.LogTrace("stalled download | {name}", torrent.Name);
|
||||
|
||||
ResetStalledStrikesOnProgress(torrent.Hash, torrent.Downloaded ?? 0);
|
||||
|
||||
return (await _striker.StrikeAndCheckLimit(torrent.Hash, torrent.Name, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled), DeleteReason.Stalled);
|
||||
}
|
||||
|
||||
ResetStrikesOnProgress(torrent.Hash, torrent.Downloaded ?? 0);
|
||||
|
||||
return await StrikeAndCheckLimit(torrent.Hash, torrent.Name);
|
||||
_logger.LogTrace("downloading metadata | {name}", torrent.Name);
|
||||
|
||||
return (await _striker.StrikeAndCheckLimit(torrent.Hash, torrent.Name, _queueCleanerConfig.StalledMaxStrikes, StrikeType.DownloadingMetadata), DeleteReason.DownloadingMetadata);
|
||||
}
|
||||
|
||||
private async Task<IReadOnlyList<TorrentTracker>> GetTrackersAsync(string hash)
|
||||
|
||||
@@ -5,6 +5,7 @@ using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.DownloadClient;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Common.CustomDataTypes;
|
||||
using Common.Helpers;
|
||||
using Domain.Enums;
|
||||
using Infrastructure.Extensions;
|
||||
@@ -26,7 +27,6 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
{
|
||||
private readonly TransmissionConfig _config;
|
||||
private readonly Client _client;
|
||||
private TorrentInfo[]? _torrentsCache;
|
||||
|
||||
private static readonly string[] Fields =
|
||||
[
|
||||
@@ -42,7 +42,9 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
TorrentFields.DOWNLOAD_DIR,
|
||||
TorrentFields.SECONDS_SEEDING,
|
||||
TorrentFields.UPLOAD_RATIO,
|
||||
TorrentFields.TRACKERS
|
||||
TorrentFields.TRACKERS,
|
||||
TorrentFields.RATE_DOWNLOAD,
|
||||
TorrentFields.TOTAL_SIZE
|
||||
];
|
||||
|
||||
public TransmissionService(
|
||||
@@ -64,9 +66,13 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
{
|
||||
_config = config.Value;
|
||||
_config.Validate();
|
||||
UriBuilder uriBuilder = new(_config.Url);
|
||||
uriBuilder.Path = string.IsNullOrEmpty(_config.UrlBase)
|
||||
? $"{uriBuilder.Path.TrimEnd('/')}/rpc"
|
||||
: $"{uriBuilder.Path.TrimEnd('/')}/{_config.UrlBase.TrimStart('/').TrimEnd('/')}/rpc";
|
||||
_client = new(
|
||||
httpClientFactory.CreateClient(Constants.HttpClientWithRetryName),
|
||||
new Uri(_config.Url, "/transmission/rpc").ToString(),
|
||||
uriBuilder.Uri.ToString(),
|
||||
login: _config.Username,
|
||||
password: _config.Password
|
||||
);
|
||||
@@ -78,9 +84,9 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
public override async Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
StalledResult result = new();
|
||||
DownloadCheckResult result = new();
|
||||
TorrentInfo? download = await GetTorrentAsync(hash);
|
||||
|
||||
if (download is null)
|
||||
@@ -115,17 +121,15 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
|
||||
if (shouldRemove)
|
||||
{
|
||||
// remove if all files are unwanted
|
||||
result.ShouldRemove = true;
|
||||
result.DeleteReason = DeleteReason.AllFilesBlocked;
|
||||
return result;
|
||||
}
|
||||
|
||||
// remove if all files are unwanted or download is stuck
|
||||
result.ShouldRemove = shouldRemove || await IsItemStuckAndShouldRemove(download);
|
||||
// remove if download is stuck
|
||||
(result.ShouldRemove, result.DeleteReason) = await EvaluateDownloadRemoval(download);
|
||||
|
||||
if (!shouldRemove && result.ShouldRemove)
|
||||
{
|
||||
result.DeleteReason = DeleteReason.Stalled;
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -334,60 +338,96 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
});
|
||||
}
|
||||
|
||||
private async Task<bool> IsItemStuckAndShouldRemove(TorrentInfo torrent)
|
||||
private async Task<(bool, DeleteReason)> EvaluateDownloadRemoval(TorrentInfo torrent)
|
||||
{
|
||||
(bool ShouldRemove, DeleteReason Reason) result = await CheckIfSlow(torrent);
|
||||
|
||||
if (result.ShouldRemove)
|
||||
{
|
||||
return result;
|
||||
}
|
||||
|
||||
return await CheckIfStuck(torrent);
|
||||
}
|
||||
|
||||
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfSlow(TorrentInfo download)
|
||||
{
|
||||
if (_queueCleanerConfig.SlowMaxStrikes is 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.Status is not 4)
|
||||
{
|
||||
// not in downloading state
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.RateDownload <= 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.SlowIgnorePrivate && download.IsPrivate is true)
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip slow check | download is private | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.TotalSize > (_queueCleanerConfig.SlowIgnoreAboveSizeByteSize?.Bytes ?? long.MaxValue))
|
||||
{
|
||||
_logger.LogDebug("skip slow check | download is too large | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
ByteSize minSpeed = _queueCleanerConfig.SlowMinSpeedByteSize;
|
||||
ByteSize currentSpeed = new ByteSize(download.RateDownload ?? long.MaxValue);
|
||||
SmartTimeSpan maxTime = SmartTimeSpan.FromHours(_queueCleanerConfig.SlowMaxTime);
|
||||
SmartTimeSpan currentTime = SmartTimeSpan.FromSeconds(download.Eta ?? 0);
|
||||
|
||||
return await CheckIfSlow(
|
||||
download.HashString!,
|
||||
download.Name!,
|
||||
minSpeed,
|
||||
currentSpeed,
|
||||
maxTime,
|
||||
currentTime
|
||||
);
|
||||
}
|
||||
|
||||
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfStuck(TorrentInfo download)
|
||||
{
|
||||
if (_queueCleanerConfig.StalledMaxStrikes is 0)
|
||||
{
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.StalledIgnorePrivate && (torrent.IsPrivate ?? false))
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip stalled check | download is private | {name}", torrent.Name);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (torrent.Status is not 4)
|
||||
if (download.Status is not 4)
|
||||
{
|
||||
// not in downloading state
|
||||
return false;
|
||||
}
|
||||
|
||||
if (torrent.Eta > 0)
|
||||
{
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
ResetStrikesOnProgress(torrent.HashString!, torrent.DownloadedEver ?? 0);
|
||||
|
||||
return await StrikeAndCheckLimit(torrent.HashString!, torrent.Name!);
|
||||
if (download.RateDownload > 0 || download.Eta > 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.StalledIgnorePrivate && (download.IsPrivate ?? false))
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip stalled check | download is private | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
ResetStalledStrikesOnProgress(download.HashString!, download.DownloadedEver ?? 0);
|
||||
|
||||
return (await _striker.StrikeAndCheckLimit(download.HashString!, download.Name!, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled), DeleteReason.Stalled);
|
||||
}
|
||||
|
||||
private async Task<TorrentInfo?> GetTorrentAsync(string hash)
|
||||
{
|
||||
TorrentInfo? torrent = _torrentsCache?
|
||||
.FirstOrDefault(x => x.HashString.Equals(hash, StringComparison.InvariantCultureIgnoreCase));
|
||||
|
||||
if (_torrentsCache is null || torrent is null)
|
||||
{
|
||||
// refresh cache
|
||||
_torrentsCache = (await _client.TorrentGetAsync(Fields))
|
||||
?.Torrents;
|
||||
}
|
||||
|
||||
if (_torrentsCache?.Length is null or 0)
|
||||
{
|
||||
_logger.LogDebug("could not list torrents | {url}", _config.Url);
|
||||
}
|
||||
|
||||
torrent = _torrentsCache?.FirstOrDefault(x => x.HashString.Equals(hash, StringComparison.InvariantCultureIgnoreCase));
|
||||
|
||||
if (torrent is null)
|
||||
{
|
||||
_logger.LogDebug("could not find torrent | {hash} | {url}", hash, _config.Url);
|
||||
}
|
||||
|
||||
return torrent;
|
||||
}
|
||||
private async Task<TorrentInfo?> GetTorrentAsync(string hash) =>
|
||||
(await _client.TorrentGetAsync(Fields, hash))
|
||||
?.Torrents
|
||||
?.FirstOrDefault();
|
||||
}
|
||||
@@ -27,6 +27,9 @@ public sealed class NotificationConsumer<T> : IConsumer<T> where T : Notificatio
|
||||
case StalledStrikeNotification stalledMessage:
|
||||
await _notificationService.Notify(stalledMessage);
|
||||
break;
|
||||
case SlowStrikeNotification slowMessage:
|
||||
await _notificationService.Notify(slowMessage);
|
||||
break;
|
||||
case QueueItemDeletedNotification queueItemDeleteMessage:
|
||||
await _notificationService.Notify(queueItemDeleteMessage);
|
||||
break;
|
||||
|
||||
@@ -6,6 +6,8 @@ public interface INotificationFactory
|
||||
|
||||
List<INotificationProvider> OnStalledStrikeEnabled();
|
||||
|
||||
List<INotificationProvider> OnSlowStrikeEnabled();
|
||||
|
||||
List<INotificationProvider> OnQueueItemDeletedEnabled();
|
||||
|
||||
List<INotificationProvider> OnDownloadCleanedEnabled();
|
||||
|
||||
@@ -12,6 +12,8 @@ public interface INotificationProvider
|
||||
Task OnFailedImportStrike(FailedImportStrikeNotification notification);
|
||||
|
||||
Task OnStalledStrike(StalledStrikeNotification notification);
|
||||
|
||||
Task OnSlowStrike(SlowStrikeNotification notification);
|
||||
|
||||
Task OnQueueItemDeleted(QueueItemDeletedNotification notification);
|
||||
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
namespace Infrastructure.Verticals.Notifications.Models;
|
||||
|
||||
public sealed record SlowStrikeNotification : ArrNotification
|
||||
{
|
||||
}
|
||||
@@ -32,6 +32,11 @@ public class NotifiarrProvider : NotificationProvider
|
||||
await _proxy.SendNotification(BuildPayload(notification, WarningColor), _config);
|
||||
}
|
||||
|
||||
public override async Task OnSlowStrike(SlowStrikeNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification, WarningColor), _config);
|
||||
}
|
||||
|
||||
public override async Task OnQueueItemDeleted(QueueItemDeletedNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification, ImportantColor), _config);
|
||||
|
||||
@@ -24,6 +24,11 @@ public class NotificationFactory : INotificationFactory
|
||||
ActiveProviders()
|
||||
.Where(n => n.Config.OnStalledStrike)
|
||||
.ToList();
|
||||
|
||||
public List<INotificationProvider> OnSlowStrikeEnabled() =>
|
||||
ActiveProviders()
|
||||
.Where(n => n.Config.OnSlowStrike)
|
||||
.ToList();
|
||||
|
||||
public List<INotificationProvider> OnQueueItemDeletedEnabled() =>
|
||||
ActiveProviders()
|
||||
|
||||
@@ -18,6 +18,8 @@ public abstract class NotificationProvider : INotificationProvider
|
||||
public abstract Task OnFailedImportStrike(FailedImportStrikeNotification notification);
|
||||
|
||||
public abstract Task OnStalledStrike(StalledStrikeNotification notification);
|
||||
|
||||
public abstract Task OnSlowStrike(SlowStrikeNotification notification);
|
||||
|
||||
public abstract Task OnQueueItemDeleted(QueueItemDeletedNotification notification);
|
||||
|
||||
|
||||
@@ -48,11 +48,16 @@ public class NotificationPublisher : INotificationPublisher
|
||||
switch (strikeType)
|
||||
{
|
||||
case StrikeType.Stalled:
|
||||
case StrikeType.DownloadingMetadata:
|
||||
await _dryRunInterceptor.InterceptAsync(Notify<StalledStrikeNotification>, notification.Adapt<StalledStrikeNotification>());
|
||||
break;
|
||||
case StrikeType.ImportFailed:
|
||||
await _dryRunInterceptor.InterceptAsync(Notify<FailedImportStrikeNotification>, notification.Adapt<FailedImportStrikeNotification>());
|
||||
break;
|
||||
case StrikeType.SlowSpeed:
|
||||
case StrikeType.SlowTime:
|
||||
await _dryRunInterceptor.InterceptAsync(Notify<SlowStrikeNotification>, notification.Adapt<SlowStrikeNotification>());
|
||||
break;
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
|
||||
@@ -44,6 +44,21 @@ public class NotificationService
|
||||
}
|
||||
}
|
||||
|
||||
public async Task Notify(SlowStrikeNotification notification)
|
||||
{
|
||||
foreach (INotificationProvider provider in _notificationFactory.OnSlowStrikeEnabled())
|
||||
{
|
||||
try
|
||||
{
|
||||
await provider.OnSlowStrike(notification);
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
_logger.LogWarning(exception, "failed to send notification | provider {provider}", provider.Name);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public async Task Notify(QueueItemDeletedNotification notification)
|
||||
{
|
||||
foreach (INotificationProvider provider in _notificationFactory.OnQueueItemDeletedEnabled())
|
||||
|
||||
@@ -77,6 +77,8 @@ public sealed class QueueCleaner : GenericHandler
|
||||
|
||||
QueueRecord record = group.First();
|
||||
|
||||
_logger.LogTrace("processing | {title} | {id}", record.Title, record.DownloadId);
|
||||
|
||||
if (!arrClient.IsRecordValid(record))
|
||||
{
|
||||
continue;
|
||||
@@ -91,7 +93,7 @@ public sealed class QueueCleaner : GenericHandler
|
||||
// push record to context
|
||||
ContextProvider.Set(nameof(QueueRecord), record);
|
||||
|
||||
StalledResult stalledCheckResult = new();
|
||||
DownloadCheckResult downloadCheckResult = new();
|
||||
|
||||
if (record.Protocol is "torrent" && _downloadClientConfig.DownloadClient is not Common.Enums.DownloadClient.Disabled)
|
||||
{
|
||||
@@ -102,14 +104,14 @@ public sealed class QueueCleaner : GenericHandler
|
||||
}
|
||||
|
||||
// stalled download check
|
||||
stalledCheckResult = await _downloadService.ShouldRemoveFromArrQueueAsync(record.DownloadId, ignoredDownloads);
|
||||
downloadCheckResult = await _downloadService.ShouldRemoveFromArrQueueAsync(record.DownloadId, ignoredDownloads);
|
||||
}
|
||||
|
||||
// failed import check
|
||||
bool shouldRemoveFromArr = await arrClient.ShouldRemoveFromQueue(instanceType, record, stalledCheckResult.IsPrivate);
|
||||
DeleteReason deleteReason = stalledCheckResult.ShouldRemove ? stalledCheckResult.DeleteReason : DeleteReason.ImportFailed;
|
||||
bool shouldRemoveFromArr = await arrClient.ShouldRemoveFromQueue(instanceType, record, downloadCheckResult.IsPrivate);
|
||||
DeleteReason deleteReason = downloadCheckResult.ShouldRemove ? downloadCheckResult.DeleteReason : DeleteReason.ImportFailed;
|
||||
|
||||
if (!shouldRemoveFromArr && !stalledCheckResult.ShouldRemove)
|
||||
if (!shouldRemoveFromArr && !downloadCheckResult.ShouldRemove)
|
||||
{
|
||||
_logger.LogInformation("skip | {title}", record.Title);
|
||||
continue;
|
||||
@@ -119,20 +121,26 @@ public sealed class QueueCleaner : GenericHandler
|
||||
|
||||
bool removeFromClient = true;
|
||||
|
||||
if (stalledCheckResult.IsPrivate)
|
||||
if (downloadCheckResult.IsPrivate)
|
||||
{
|
||||
if (stalledCheckResult.ShouldRemove && !_config.StalledDeletePrivate)
|
||||
{
|
||||
removeFromClient = false;
|
||||
}
|
||||
bool isStalledWithoutPruneFlag =
|
||||
downloadCheckResult.DeleteReason is DeleteReason.Stalled &&
|
||||
!_config.StalledDeletePrivate;
|
||||
|
||||
bool isSlowWithoutPruneFlag =
|
||||
downloadCheckResult.DeleteReason is DeleteReason.SlowSpeed or DeleteReason.SlowTime &&
|
||||
!_config.SlowDeletePrivate;
|
||||
|
||||
bool shouldKeepDueToDeleteRules = downloadCheckResult.ShouldRemove && (isStalledWithoutPruneFlag || isSlowWithoutPruneFlag);
|
||||
bool shouldKeepDueToImportRules = shouldRemoveFromArr && !_config.ImportFailedDeletePrivate;
|
||||
|
||||
if (shouldRemoveFromArr && !_config.ImportFailedDeletePrivate)
|
||||
if (shouldKeepDueToDeleteRules || shouldKeepDueToImportRules)
|
||||
{
|
||||
removeFromClient = false;
|
||||
}
|
||||
}
|
||||
|
||||
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient);
|
||||
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient, deleteReason);
|
||||
await _notifier.NotifyQueueItemDeleted(removeFromClient, deleteReason);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -178,7 +178,7 @@ services:
|
||||
- TZ=Europe/Bucharest
|
||||
- DRY_RUN=false
|
||||
|
||||
- LOGGING__LOGLEVEL=Debug
|
||||
- LOGGING__LOGLEVEL=Verbose
|
||||
- LOGGING__FILE__ENABLED=true
|
||||
- LOGGING__FILE__PATH=/var/logs
|
||||
- LOGGING__ENHANCED=true
|
||||
@@ -193,14 +193,25 @@ services:
|
||||
- QUEUECLEANER__ENABLED=true
|
||||
- QUEUECLEANER__IGNORED_DOWNLOADS_PATH=/ignored
|
||||
- QUEUECLEANER__RUNSEQUENTIALLY=true
|
||||
|
||||
- QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES=5
|
||||
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE=true
|
||||
- QUEUECLEANER__IMPORT_FAILED_DELETE_PRIVATE=false
|
||||
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__0=file is a sample
|
||||
|
||||
- QUEUECLEANER__STALLED_MAX_STRIKES=5
|
||||
- QUEUECLEANER__STALLED_RESET_STRIKES_ON_PROGRESS=true
|
||||
- QUEUECLEANER__STALLED_IGNORE_PRIVATE=true
|
||||
- QUEUECLEANER__STALLED_DELETE_PRIVATE=false
|
||||
|
||||
- QUEUECLEANER__SLOW_MAX_STRIKES=5
|
||||
- QUEUECLEANER__SLOW_RESET_STRIKES_ON_PROGRESS=true
|
||||
- QUEUECLEANER__SLOW_IGNORE_PRIVATE=false
|
||||
- QUEUECLEANER__SLOW_DELETE_PRIVATE=false
|
||||
- QUEUECLEANER__SLOW_MIN_SPEED=1MB
|
||||
- QUEUECLEANER__SLOW_MAX_TIME=20
|
||||
- QUEUECLEANER__SLOW_IGNORE_ABOVE_SIZE=1KB
|
||||
|
||||
- CONTENTBLOCKER__ENABLED=true
|
||||
- CONTENTBLOCKER__IGNORED_DOWNLOADS_PATH=/ignored
|
||||
- CONTENTBLOCKER__IGNORE_PRIVATE=true
|
||||
@@ -251,12 +262,13 @@ services:
|
||||
- LIDARR__INSTANCES__0__URL=http://lidarr:8686
|
||||
- LIDARR__INSTANCES__0__APIKEY=7f677cfdc074414397af53dd633860c5
|
||||
|
||||
# - NOTIFIARR__ON_IMPORT_FAILED_STRIKE=true
|
||||
# - NOTIFIARR__ON_STALLED_STRIKE=true
|
||||
# - NOTIFIARR__ON_QUEUE_ITEM_DELETED=true
|
||||
# - NOTIFIARR__ON_DOWNLOAD_CLEANED=true
|
||||
# - NOTIFIARR__API_KEY=notifiarr_secret
|
||||
# - NOTIFIARR__CHANNEL_ID=discord_channel_id
|
||||
- NOTIFIARR__ON_IMPORT_FAILED_STRIKE=true
|
||||
- NOTIFIARR__ON_STALLED_STRIKE=true
|
||||
- NOTIFIARR__ON_SLOW_STRIKE=true
|
||||
- NOTIFIARR__ON_QUEUE_ITEM_DELETED=true
|
||||
- NOTIFIARR__ON_DOWNLOAD_CLEANED=true
|
||||
- NOTIFIARR__API_KEY=notifiarr_secret
|
||||
- NOTIFIARR__CHANNEL_ID=discord_channel_id
|
||||
volumes:
|
||||
- ./data/cleanuperr/logs:/var/logs
|
||||
- ./data/cleanuperr/ignored_downloads:/ignored
|
||||
|
||||
90
variables.md
90
variables.md
@@ -160,7 +160,7 @@
|
||||
> If not set to `0`, the minimum value is `3`.
|
||||
|
||||
#### **`QUEUECLEANER__STALLED_RESET_STRIKES_ON_PROGRESS`**
|
||||
- Controls whether to remove strikes if any download progress was made since last checked.
|
||||
- Controls whether to remove the given strikes if any download progress was made since last checked.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
@@ -174,7 +174,7 @@
|
||||
- Required: No.
|
||||
|
||||
#### **`QUEUECLEANER__STALLED_DELETE_PRIVATE`**
|
||||
- Controls whether to delete stalled private downloads from the download client.
|
||||
- Controls whether stalled downloads from private trackers should be removed from the download client.
|
||||
- Has no effect if `QUEUECLEANER__STALLED_IGNORE_PRIVATE` is `true`.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
@@ -184,6 +184,65 @@
|
||||
> [!WARNING]
|
||||
> Setting `QUEUECLEANER__STALLED_DELETE_PRIVATE=true` means you don't care about seeding, ratio, H&R and potentially losing your private tracker account.
|
||||
|
||||
#### **`QUEUECLEANER__SLOW_MAX_STRIKES`**
|
||||
- Number of strikes before removing a slow download.
|
||||
- Set to `0` to never remove slow downloads.
|
||||
- A strike is given when an item is slow.
|
||||
- Type: Integer
|
||||
- Default: `0`
|
||||
- Required: No.
|
||||
> [!NOTE]
|
||||
> If not set to `0`, the minimum value is `3`.
|
||||
|
||||
#### **`QUEUECLEANER__SLOW_RESET_STRIKES_ON_PROGRESS`**
|
||||
- Controls whether to remove the given strikes if the download speed or estimated time are not slow anymore.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
#### **`QUEUECLEANER__SLOW_IGNORE_PRIVATE`**
|
||||
- Controls whether to ignore slow downloads from private trackers.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
#### **`QUEUECLEANER__SLOW_DELETE_PRIVATE`**
|
||||
- Controls whether slow downloads from private trackers should be removed from the download client.
|
||||
- Has no effect if `QUEUECLEANER__SLOW_IGNORE_PRIVATE` is `true`.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
> [!WARNING]
|
||||
> Setting `QUEUECLEANER__SLOW_DELETE_PRIVATE=true` means you don't care about seeding, ratio, H&R and potentially losing your private tracker account.
|
||||
|
||||
#### **`QUEUECLEANER__SLOW_MIN_SPEED`**
|
||||
- The minimum speed a download should have.
|
||||
- Downloads receive strikes if their speed falls bellow this value.
|
||||
- If not specified, downloads will not receive strikes for slow download speed.
|
||||
- Type: String.
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
- Value examples: `1.5KB`, `400KB`, `2MB`
|
||||
|
||||
#### **`QUEUECLEANER__SLOW_MAX_TIME`**
|
||||
- The maximum estimated hours a download should take to finish.
|
||||
- Downloads receive strikes if their estimated finish time is above this value.
|
||||
- If not specified (or `0`), downloads will not receive strikes for slow estimated finish time.
|
||||
- Type: Integer.
|
||||
- Default: `0`.
|
||||
- Required: No.
|
||||
|
||||
#### **`QUEUECLEANER__SLOW_IGNORE_ABOVE_SIZE`**
|
||||
- Downloads above this size will not be removed for being slow.
|
||||
- Type: String.
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
- Value examples: `10KB`, `200MB`, `3GB`.
|
||||
|
||||
#
|
||||
|
||||
### Content Blocker settings
|
||||
@@ -378,6 +437,12 @@
|
||||
- Default: `http://localhost:8080`.
|
||||
- Required: No.
|
||||
|
||||
#### **`QBITTORRENT__URL_BASE`**
|
||||
- Adds a prefix to the qBittorrent url, such as `[QBITTORRENT__URL]/[QBITTORRENT__URL_BASE]/api`.
|
||||
- Type: String.
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
#### **`QBITTORRENT__USERNAME`**
|
||||
- Username for qBittorrent authentication.
|
||||
- Type: String.
|
||||
@@ -396,6 +461,12 @@
|
||||
- Default: `http://localhost:8112`.
|
||||
- Required: No.
|
||||
|
||||
#### **`DELUGE__URL_BASE`**
|
||||
- Adds a prefix to the deluge json url, such as `[DELUGE__URL]/[DELUGE__URL_BASE]/json`.
|
||||
- Type: String.
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
#### **`DELUGE__PASSWORD`**
|
||||
- Password for Deluge authentication.
|
||||
- Type: String.
|
||||
@@ -408,6 +479,12 @@
|
||||
- Default: `http://localhost:9091`.
|
||||
- Required: No.
|
||||
|
||||
#### **`TRANSMISSION__URL_BASE`**
|
||||
- Adds a prefix to the Transmission rpc url, such as `[TRANSMISSION__URL]/[TRANSMISSION__URL_BASE]/rpc`.
|
||||
- Type: String.
|
||||
- Default: `transmission`.
|
||||
- Required: No.
|
||||
|
||||
#### **`TRANSMISSION__USERNAME`**
|
||||
- Username for Transmission authentication.
|
||||
- Type: String.
|
||||
@@ -598,7 +675,14 @@
|
||||
- Required: No.
|
||||
|
||||
#### **`NOTIFIARR__ON_STALLED_STRIKE`**
|
||||
- Controls whether to notify when an item receives a stalled download strike.
|
||||
- Controls whether to notify when an item receives a stalled download strike. This includes strikes for being stuck while downloading metadata.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
#### **`NOTIFIARR__ON_SLOW_STRIKE`**
|
||||
- Controls whether to notify when an item receives a slow download strike. This includes strikes for having a low download speed or slow estimated finish time.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
|
||||
Reference in New Issue
Block a user