Compare commits

...

15 Commits

Author SHA1 Message Date
Flaminel
75b001cf6a Add Apprise support (#124) 2025-05-01 21:00:01 +03:00
Flaminel
479ca7884e Fix crashing when tracker url is malformed (#121) 2025-04-28 16:48:54 +03:00
Flaminel
00d8910118 Update README.md 2025-04-12 23:43:44 +03:00
Flaminel
bd28c7ab05 Fix missing notifications for new strike types (#112) 2025-04-08 22:20:51 +03:00
Flaminel
720279df65 Update README.md 2025-04-08 18:14:01 +03:00
Flaminel
2d4ec648b8 Update README.md 2025-04-06 18:10:46 +03:00
Flaminel
704fdaca4a Add cleanup for slow downloads (#110) 2025-04-06 13:28:05 +03:00
Flaminel
b134136e51 Update README.md 2025-03-29 01:11:41 +02:00
Flaminel
5ca717d7e0 Update README.md 2025-03-27 19:53:57 +02:00
Flaminel
7068ee5e5a Update README.md 2025-03-26 13:30:55 +02:00
Flaminel
9f770473e5 Remove Transmission downloads cache (#105) 2025-03-26 00:26:10 +02:00
Flaminel
5fe0f5750a Fix qBit queued items being processed (#102) 2025-03-21 23:06:31 +02:00
Flaminel
b8ce225ccc Fix Deluge service crashing when download is not found (#97) 2025-03-20 00:09:58 +02:00
Flaminel
f21f7388b7 Add download client customizable url base (#43) 2025-03-20 00:09:24 +02:00
Flaminel
a1354f231a Add base path support for arrs (#96) 2025-03-20 00:08:51 +02:00
59 changed files with 1373 additions and 276 deletions

View File

@@ -18,7 +18,7 @@ body:
required: true
- label: Ensured I am using the latest version.
required: true
- label: Enabled debug logging.
- label: Enabled verbose logging.
required: true
- type: textarea
id: what-happened

View File

@@ -18,7 +18,7 @@ body:
required: true
- label: Ensured I am using the latest version.
required: true
- label: Enabled debug logging.
- label: Enabled verbose logging.
required: true
- type: textarea
id: description

View File

@@ -12,6 +12,7 @@ cleanuperr was created primarily to address malicious files, such as `*.lnk` or
> **Features:**
> - Strike system to mark stalled or downloads stuck in metadata downloading.
> - Remove and block downloads that reached a maximum number of strikes.
> - Remove and block downloads that have a low download speed or high estimated completion time.
> - Remove downloads blocked by qBittorrent or by cleanuperr's **content blocker**.
> - Trigger a search for downloads removed from the *arrs.
> - Clean up downloads that have been seeding for a certain amount of time.
@@ -91,11 +92,11 @@ I've seen a few discussions on this type of naming and I've decided that I didn'
#### 2. **Queue cleaner** will:
- Run every 5 minutes (or configured cron, or right after `content blocker`).
- Process all items in the *arr queue.
- Check each queue item if it is **stalled (download speed is 0)**, **stuck in metadata downloading** or **failed to be imported**.
- Check each queue item if it is **stalled (download speed is 0)**, **stuck in metadata downloading**, **failed to be imported** or **slow**.
- If it is, the item receives a **strike** and will continue to accumulate strikes every time it meets any of these conditions.
- Check each queue item if it meets one of the following condition in the download client:
- **Marked as completed, but 0 bytes have been downloaded** (due to files being blocked by qBittorrent or the **content blocker**).
- All associated files of are marked as **unwanted/skipped**.
- All associated files are marked as **unwanted/skipped/do not download**.
- If the item **DOES NOT** match the above criteria, it will be skipped.
- If the item **DOES** match the criteria or has received the **maximum number of strikes**:
- It will be removed from the *arr's queue and blocked.
@@ -130,10 +131,14 @@ I've seen a few discussions on this type of naming and I've decided that I didn'
1. Set `QUEUECLEANER__ENABLED` to `true`.
2. Set `QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES` to a desired value.
3. Optionally set failed import message patterns to ignore using `QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__<NUMBER>`.
4. Set `DOWNLOAD_CLIENT` to `none`.
4. Set `DOWNLOAD_CLIENT` to `none`(works only for usenet) or `disabled` (works for both usenet and torrent).
> [!WARNING]
> When `DOWNLOAD_CLIENT=none`, no other action involving a download client would work (e.g. content blocking, removing stalled downloads, excluding private trackers).
> [!IMPORTANT]
> When `DOWNLOAD_CLIENT=disabled`, no other action involving a download client would work (e.g. content blocking, removing stalled downloads, excluding private trackers).
>
> When the download client is set to `disabled`, the queue cleaner will be able to remove items that are failed to be imported even if there is no download client configured. This means that all downloads, including private ones, will be completely removed.
>
> Setting `DOWNLOAD_CLIENT=disabled` means you don't care about seeding, ratio, H&R and potentially losing your private tracker account.
## Usage
@@ -168,39 +173,62 @@ services:
- ./cleanuperr/logs:/var/logs
- ./cleanuperr/ignored.txt:/ignored.txt
environment:
# general settings
- TZ=America/New_York
- DRY_RUN=false
- HTTP_MAX_RETRIES=0
- HTTP_TIMEOUT=100
# logging
- LOGGING__LOGLEVEL=Information
- LOGGING__FILE__ENABLED=false
- LOGGING__FILE__PATH=/var/logs/
- LOGGING__ENHANCED=true
# job triggers
- TRIGGERS__QUEUECLEANER=0 0/5 * * * ?
- TRIGGERS__CONTENTBLOCKER=0 0/5 * * * ?
- TRIGGERS__DOWNLOADCLEANER=0 0 * * * ?
# queue cleaner
- QUEUECLEANER__ENABLED=true
- QUEUECLEANER__IGNORED_DOWNLOADS_PATH=/ignored.txt
- QUEUECLEANER__RUNSEQUENTIALLY=true
# failed imports
- QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES=5
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE=false
- QUEUECLEANER__IMPORT_FAILED_DELETE_PRIVATE=false
# - QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__0=title mismatch
# - QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__1=manual import required
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__0=title mismatch
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__1=manual import required
# stalled downloads
- QUEUECLEANER__STALLED_MAX_STRIKES=5
- QUEUECLEANER__STALLED_RESET_STRIKES_ON_PROGRESS=false
- QUEUECLEANER__STALLED_IGNORE_PRIVATE=false
- QUEUECLEANER__STALLED_DELETE_PRIVATE=false
# slow downloads
- QUEUECLEANER__SLOW_MAX_STRIKES=5
- QUEUECLEANER__SLOW_RESET_STRIKES_ON_PROGRESS=true
- QUEUECLEANER__SLOW_IGNORE_PRIVATE=false
- QUEUECLEANER__SLOW_DELETE_PRIVATE=false
- QUEUECLEANER__SLOW_MIN_SPEED=1MB
- QUEUECLEANER__SLOW_MAX_TIME=20
- QUEUECLEANER__SLOW_IGNORE_ABOVE_SIZE=60GB
# content blocker
- CONTENTBLOCKER__ENABLED=true
- CONTENTBLOCKER__IGNORED_DOWNLOADS_PATH=/ignored.txt
- CONTENTBLOCKER__IGNORE_PRIVATE=false
- CONTENTBLOCKER__DELETE_PRIVATE=false
# download cleaner
- DOWNLOADCLEANER__ENABLED=true
- DOWNLOADCLEANER__IGNORED_DOWNLOADS_PATH=/ignored.txt
- DOWNLOADCLEANER__DELETE_PRIVATE=false
# categories to seed until max ratio or min seed time has been reached
- DOWNLOADCLEANER__CATEGORIES__0__NAME=tv-sonarr
- DOWNLOADCLEANER__CATEGORIES__0__MAX_RATIO=-1
- DOWNLOADCLEANER__CATEGORIES__0__MIN_SEED_TIME=0
@@ -212,17 +240,22 @@ services:
- DOWNLOAD_CLIENT=none
# OR
# - DOWNLOAD_CLIENT=disabled
# OR
# - DOWNLOAD_CLIENT=qBittorrent
# - QBITTORRENT__URL=http://localhost:8080
# - QBITTORRENT__URL_BASE=myCustomPath
# - QBITTORRENT__USERNAME=user
# - QBITTORRENT__PASSWORD=pass
# OR
# - DOWNLOAD_CLIENT=deluge
# - DELUGE__URL_BASE=myCustomPath
# - DELUGE__URL=http://localhost:8112
# - DELUGE__PASSWORD=testing
# OR
# - DOWNLOAD_CLIENT=transmission
# - TRANSMISSION__URL=http://localhost:9091
# - TRANSMISSION__URL_BASE=myCustomPath
# - TRANSMISSION__USERNAME=test
# - TRANSMISSION__PASSWORD=testing
@@ -253,10 +286,19 @@ services:
- NOTIFIARR__ON_IMPORT_FAILED_STRIKE=true
- NOTIFIARR__ON_STALLED_STRIKE=true
- NOTIFIARR__ON_SLOW_STRIKE=true
- NOTIFIARR__ON_QUEUE_ITEM_DELETED=true
- NOTIFIARR__ON_DOWNLOAD_CLEANED=true
- NOTIFIARR__API_KEY=notifiarr_secret
- NOTIFIARR__CHANNEL_ID=discord_channel_id
- APPRISE__ON_IMPORT_FAILED_STRIKE=true
- APPRISE__ON_STALLED_STRIKE=true
- APPRISE__ON_SLOW_STRIKE=true
- APPRISE__ON_QUEUE_ITEM_DELETED=true
- APPRISE__ON_DOWNLOAD_CLEANED=true
- APPRISE__URL=http://apprise:8000
- APPRISE__KEY=myConfigKey
```
### <img src="https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/brands/windows.svg" height="20" style="vertical-align: middle;"> <span style="vertical-align: middle;">Windows</span>
@@ -268,7 +310,21 @@ services:
> [!TIP]
> ### Run as a Windows Service
> Check out this stackoverflow answer on how to do it: https://stackoverflow.com/a/15719678
> 1. Download latest nssm build from `https://nssm.cc/builds`.
> 2. Unzip `nssm.exe` in `C:\example\directory`.
> 3. Open a terminal with Administrator rights and execute these commands:
> ```
> nssm.exe install Cleanuperr "C:\example\directory\cleanuperr.exe"
> nssm.exe set Cleanuperr AppDirectory "C:\example\directory\"
> nssm.exe set Cleanuperr AppStdout "C:\example\directory\cleanuperr.log"
> nssm.exe set Cleanuperr AppStderr "C:\example\directory\cleanuperr.crash.log"
> nssm.exe set Cleanuperr AppRotateFiles 1
> nssm.exe set Cleanuperr AppRotateOnline 1
> nssm.exe set Cleanuperr AppRotateBytes 10485760
> nssm.exe set Cleanuperr AppRotateFiles 10
> nssm.exe set Cleanuperr Start SERVICE_AUTO_START
> nssm.exe start Cleanuperr
> ```
### <img src="https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/brands/linux.svg" height="20" style="vertical-align: middle;"> <span style="vertical-align: middle;">Linux</span>
@@ -296,7 +352,7 @@ services:
> [!IMPORTANT]
> Some people have experienced problems when trying to execute cleanuperr on MacOS because the system actively blocked the file for not being signed.
> As per [this](), you may need to also execute this command:
> As per [this comment](https://stackoverflow.com/a/77907937), you may need to also execute this command:
> ```
> codesign --sign - --force --preserve-metadata=entitlements,requirements,flags,runtime /example/directory/cleanuperr
> ```

View File

@@ -14,7 +14,7 @@ deployment:
value: "false"
- name: LOGGING__LOGLEVEL
value: Debug
value: Verbose
- name: LOGGING__FILE__ENABLED
value: "true"
- name: LOGGING__FILE__PATH

View File

@@ -1,4 +1,5 @@
using Common.Exceptions;
using Common.Exceptions;
using Microsoft.Extensions.Configuration;
namespace Common.Configuration.DownloadClient;
@@ -8,6 +9,9 @@ public sealed record DelugeConfig : IConfig
public Uri? Url { get; init; }
[ConfigurationKeyName("URL_BASE")]
public string UrlBase { get; init; } = string.Empty;
public string? Password { get; init; }
public void Validate()

View File

@@ -1,4 +1,5 @@
using Common.Exceptions;
using Common.Exceptions;
using Microsoft.Extensions.Configuration;
namespace Common.Configuration.DownloadClient;
@@ -8,6 +9,9 @@ public sealed class QBitConfig : IConfig
public Uri? Url { get; init; }
[ConfigurationKeyName("URL_BASE")]
public string UrlBase { get; init; } = string.Empty;
public string? Username { get; init; }
public string? Password { get; init; }

View File

@@ -1,4 +1,5 @@
using Common.Exceptions;
using Common.Exceptions;
using Microsoft.Extensions.Configuration;
namespace Common.Configuration.DownloadClient;
@@ -8,6 +9,9 @@ public record TransmissionConfig : IConfig
public Uri? Url { get; init; }
[ConfigurationKeyName("URL_BASE")]
public string UrlBase { get; init; } = "transmission";
public string? Username { get; init; }
public string? Password { get; init; }

View File

@@ -10,13 +10,16 @@ public abstract record NotificationConfig
[ConfigurationKeyName("ON_STALLED_STRIKE")]
public bool OnStalledStrike { get; init; }
[ConfigurationKeyName("ON_SLOW_STRIKE")]
public bool OnSlowStrike { get; init; }
[ConfigurationKeyName("ON_QUEUE_ITEM_DELETED")]
public bool OnQueueItemDeleted { get; init; }
[ConfigurationKeyName("ON_DOWNLOAD_CLEANED")]
public bool OnDownloadCleaned { get; init; }
public bool IsEnabled => OnImportFailedStrike || OnStalledStrike || OnQueueItemDeleted || OnDownloadCleaned;
public bool IsEnabled => OnImportFailedStrike || OnStalledStrike || OnSlowStrike || OnQueueItemDeleted || OnDownloadCleaned;
public abstract bool IsValid();
}

View File

@@ -1,4 +1,5 @@
using Common.Exceptions;
using Common.CustomDataTypes;
using Common.Exceptions;
using Microsoft.Extensions.Configuration;
namespace Common.Configuration.QueueCleaner;
@@ -37,17 +38,74 @@ public sealed record QueueCleanerConfig : IJobConfig, IIgnoredDownloadsConfig
[ConfigurationKeyName("STALLED_DELETE_PRIVATE")]
public bool StalledDeletePrivate { get; init; }
[ConfigurationKeyName("SLOW_MAX_STRIKES")]
public ushort SlowMaxStrikes { get; init; }
[ConfigurationKeyName("SLOW_RESET_STRIKES_ON_PROGRESS")]
public bool SlowResetStrikesOnProgress { get; init; }
[ConfigurationKeyName("SLOW_IGNORE_PRIVATE")]
public bool SlowIgnorePrivate { get; init; }
[ConfigurationKeyName("SLOW_DELETE_PRIVATE")]
public bool SlowDeletePrivate { get; init; }
[ConfigurationKeyName("SLOW_MIN_SPEED")]
public string SlowMinSpeed { get; init; } = string.Empty;
public ByteSize SlowMinSpeedByteSize => string.IsNullOrEmpty(SlowMinSpeed) ? new ByteSize(0) : ByteSize.Parse(SlowMinSpeed);
[ConfigurationKeyName("SLOW_MAX_TIME")]
public double SlowMaxTime { get; init; }
[ConfigurationKeyName("SLOW_IGNORE_ABOVE_SIZE")]
public string SlowIgnoreAboveSize { get; init; } = string.Empty;
public ByteSize? SlowIgnoreAboveSizeByteSize => string.IsNullOrEmpty(SlowIgnoreAboveSize) ? null : ByteSize.Parse(SlowIgnoreAboveSize);
public void Validate()
{
if (ImportFailedMaxStrikes is > 0 and < 3)
{
throw new ValidationException("the minimum value for IMPORT_FAILED_MAX_STRIKES must be 3");
throw new ValidationException($"the minimum value for {SectionName}__IMPORT_FAILED_MAX_STRIKES must be 3");
}
if (StalledMaxStrikes is > 0 and < 3)
{
throw new ValidationException("the minimum value for STALLED_MAX_STRIKES must be 3");
throw new ValidationException($"the minimum value for {SectionName}__STALLED_MAX_STRIKES must be 3");
}
if (SlowMaxStrikes is > 0 and < 3)
{
throw new ValidationException($"the minimum value for {SectionName}__SLOW_MAX_STRIKES must be 3");
}
if (SlowMaxStrikes > 0)
{
bool isSlowSpeedSet = !string.IsNullOrEmpty(SlowMinSpeed);
if (isSlowSpeedSet && ByteSize.TryParse(SlowMinSpeed, out _) is false)
{
throw new ValidationException($"invalid value for {SectionName}__SLOW_MIN_SPEED");
}
if (SlowMaxTime < 0)
{
throw new ValidationException($"invalid value for {SectionName}__SLOW_MAX_TIME");
}
if (!isSlowSpeedSet && SlowMaxTime is 0)
{
throw new ValidationException($"either {SectionName}__SLOW_MIN_SPEED or {SectionName}__SLOW_MAX_STRIKES must be set");
}
bool isSlowIgnoreAboveSizeSet = !string.IsNullOrEmpty(SlowIgnoreAboveSize);
if (isSlowIgnoreAboveSizeSet && ByteSize.TryParse(SlowIgnoreAboveSize, out _) is false)
{
throw new ValidationException($"invalid value for {SectionName}__SLOW_IGNORE_ABOVE_SIZE");
}
}
}
}

View File

@@ -0,0 +1,115 @@
using System.Globalization;
namespace Common.CustomDataTypes;
public readonly struct ByteSize : IComparable<ByteSize>, IEquatable<ByteSize>
{
public long Bytes { get; }
private const long BytesPerKB = 1024;
private const long BytesPerMB = 1024 * 1024;
private const long BytesPerGB = 1024 * 1024 * 1024;
public ByteSize(long bytes)
{
if (bytes < 0)
{
throw new ArgumentOutOfRangeException(nameof(bytes), "bytes can not be negative");
}
Bytes = bytes;
}
public static ByteSize FromKilobytes(double kilobytes) => new((long)(kilobytes * BytesPerKB));
public static ByteSize FromMegabytes(double megabytes) => new((long)(megabytes * BytesPerMB));
public static ByteSize FromGigabytes(double gigabytes) => new((long)(gigabytes * BytesPerGB));
public static ByteSize Parse(string input)
{
if (string.IsNullOrWhiteSpace(input))
{
throw new ArgumentNullException(nameof(input));
}
input = input.Trim().ToUpperInvariant();
double value;
if (input.EndsWith("KB", StringComparison.InvariantCultureIgnoreCase))
{
value = double.Parse(input[..^2], CultureInfo.InvariantCulture);
return FromKilobytes(value);
}
if (input.EndsWith("MB", StringComparison.InvariantCultureIgnoreCase))
{
value = double.Parse(input[..^2], CultureInfo.InvariantCulture);
return FromMegabytes(value);
}
if (input.EndsWith("GB", StringComparison.InvariantCultureIgnoreCase))
{
value = double.Parse(input[..^2], CultureInfo.InvariantCulture);
return FromGigabytes(value);
}
throw new FormatException("invalid size format | only KB, MB and GB are supported");
}
public static bool TryParse(string? input, out ByteSize? result)
{
result = default;
if (string.IsNullOrWhiteSpace(input))
{
return false;
}
input = input.Trim().ToUpperInvariant();
if (input.EndsWith("KB", StringComparison.InvariantCultureIgnoreCase) &&
double.TryParse(input[..^2], NumberStyles.Float, CultureInfo.InvariantCulture, out double kb))
{
result = FromKilobytes(kb);
return true;
}
if (input.EndsWith("MB", StringComparison.InvariantCultureIgnoreCase) &&
double.TryParse(input[..^2], NumberStyles.Float, CultureInfo.InvariantCulture, out double mb))
{
result = FromMegabytes(mb);
return true;
}
if (input.EndsWith("GB", StringComparison.InvariantCultureIgnoreCase) &&
double.TryParse(input[..^2], NumberStyles.Float, CultureInfo.InvariantCulture, out double gb))
{
result = FromGigabytes(gb);
return true;
}
return false;
}
public override string ToString() =>
Bytes switch
{
>= BytesPerGB => $"{Bytes / (double)BytesPerGB:0.##} GB",
>= BytesPerMB => $"{Bytes / (double)BytesPerMB:0.##} MB",
_ => $"{Bytes / (double)BytesPerKB:0.##} KB"
};
public int CompareTo(ByteSize other) => Bytes.CompareTo(other.Bytes);
public bool Equals(ByteSize other) => Bytes == other.Bytes;
public override bool Equals(object? obj) => obj is ByteSize other && Equals(other);
public override int GetHashCode() => Bytes.GetHashCode();
public static bool operator ==(ByteSize left, ByteSize right) => left.Equals(right);
public static bool operator !=(ByteSize left, ByteSize right) => !(left == right);
public static bool operator <(ByteSize left, ByteSize right) => left.Bytes < right.Bytes;
public static bool operator >(ByteSize left, ByteSize right) => left.Bytes > right.Bytes;
public static bool operator <=(ByteSize left, ByteSize right) => left.Bytes <= right.Bytes;
public static bool operator >=(ByteSize left, ByteSize right) => left.Bytes >= right.Bytes;
public static ByteSize operator +(ByteSize left, ByteSize right) => new(left.Bytes + right.Bytes);
public static ByteSize operator -(ByteSize left, ByteSize right) => new(Math.Max(left.Bytes - right.Bytes, 0));
}

View File

@@ -0,0 +1,66 @@
using System.Text;
namespace Common.CustomDataTypes;
public readonly struct SmartTimeSpan : IComparable<SmartTimeSpan>, IEquatable<SmartTimeSpan>
{
public TimeSpan Time { get; }
public SmartTimeSpan(TimeSpan time)
{
Time = time;
}
public override string ToString()
{
if (Time == TimeSpan.Zero)
{
return "0 seconds";
}
StringBuilder sb = new();
if (Time.Days > 0)
{
sb.Append($"{Time.Days} day{(Time.Days > 1 ? "s" : "")} ");
}
if (Time.Hours > 0)
{
sb.Append($"{Time.Hours} hour{(Time.Hours > 1 ? "s" : "")} ");
}
if (Time.Minutes > 0)
{
sb.Append($"{Time.Minutes} minute{(Time.Minutes > 1 ? "s" : "")} ");
}
if (Time.Seconds > 0)
{
sb.Append($"{Time.Seconds} second{(Time.Seconds > 1 ? "s" : "")}");
}
return sb.ToString().TrimEnd();
}
public static SmartTimeSpan FromMinutes(double minutes) => new(TimeSpan.FromMinutes(minutes));
public static SmartTimeSpan FromSeconds(double seconds) => new(TimeSpan.FromSeconds(seconds));
public static SmartTimeSpan FromHours(double hours) => new(TimeSpan.FromHours(hours));
public static SmartTimeSpan FromDays(double days) => new(TimeSpan.FromDays(days));
public int CompareTo(SmartTimeSpan other) => Time.CompareTo(other.Time);
public bool Equals(SmartTimeSpan other) => Time.Equals(other.Time);
public override bool Equals(object? obj) => obj is SmartTimeSpan other && Equals(other);
public override int GetHashCode() => Time.GetHashCode();
public static bool operator ==(SmartTimeSpan left, SmartTimeSpan right) => left.Equals(right);
public static bool operator !=(SmartTimeSpan left, SmartTimeSpan right) => !left.Equals(right);
public static bool operator <(SmartTimeSpan left, SmartTimeSpan right) => left.Time < right.Time;
public static bool operator >(SmartTimeSpan left, SmartTimeSpan right) => left.Time > right.Time;
public static bool operator <=(SmartTimeSpan left, SmartTimeSpan right) => left.Time <= right.Time;
public static bool operator >=(SmartTimeSpan left, SmartTimeSpan right) => left.Time >= right.Time;
public static SmartTimeSpan operator +(SmartTimeSpan left, SmartTimeSpan right) => new(left.Time + right.Time);
public static SmartTimeSpan operator -(SmartTimeSpan left, SmartTimeSpan right) => new(left.Time - right.Time);
}

View File

@@ -2,7 +2,13 @@
public enum DeleteReason
{
None,
Stalled,
ImportFailed,
AllFilesBlocked
DownloadingMetadata,
SlowSpeed,
SlowTime,
AllFilesSkipped,
AllFilesSkippedByQBit,
AllFilesBlocked,
}

View File

@@ -3,5 +3,8 @@
public enum StrikeType
{
Stalled,
ImportFailed
DownloadingMetadata,
ImportFailed,
SlowSpeed,
SlowTime,
}

View File

@@ -1,6 +1,6 @@
namespace Domain.Models.Cache;
public sealed record CacheItem
public sealed record StalledCacheItem
{
/// <summary>
/// The amount of bytes that have been downloaded.

View File

@@ -2,7 +2,7 @@
namespace Domain.Models.Deluge.Response;
public sealed record TorrentStatus
public sealed record DownloadStatus
{
public string? Hash { get; init; }
@@ -12,8 +12,14 @@ public sealed record TorrentStatus
public ulong Eta { get; init; }
[JsonProperty("download_payload_rate")]
public long DownloadSpeed { get; init; }
public bool Private { get; init; }
[JsonProperty("total_size")]
public long Size { get; init; }
[JsonProperty("total_done")]
public long TotalDone { get; init; }
@@ -29,5 +35,5 @@ public sealed record TorrentStatus
public sealed record Tracker
{
public required Uri Url { get; init; }
public required string Url { get; init; }
}

View File

@@ -25,6 +25,7 @@ public static class MainDI
{
config.AddConsumer<NotificationConsumer<FailedImportStrikeNotification>>();
config.AddConsumer<NotificationConsumer<StalledStrikeNotification>>();
config.AddConsumer<NotificationConsumer<SlowStrikeNotification>>();
config.AddConsumer<NotificationConsumer<QueueItemDeletedNotification>>();
config.AddConsumer<NotificationConsumer<DownloadCleanedNotification>>();
@@ -34,6 +35,7 @@ public static class MainDI
{
e.ConfigureConsumer<NotificationConsumer<FailedImportStrikeNotification>>(context);
e.ConfigureConsumer<NotificationConsumer<StalledStrikeNotification>>(context);
e.ConfigureConsumer<NotificationConsumer<SlowStrikeNotification>>(context);
e.ConfigureConsumer<NotificationConsumer<QueueItemDeletedNotification>>(context);
e.ConfigureConsumer<NotificationConsumer<DownloadCleanedNotification>>(context);
e.ConcurrentMessageLimit = 1;

View File

@@ -1,4 +1,5 @@
using Infrastructure.Verticals.Notifications;
using Infrastructure.Verticals.Notifications.Apprise;
using Infrastructure.Verticals.Notifications.Notifiarr;
namespace Executable.DependencyInjection;
@@ -8,8 +9,11 @@ public static class NotificationsDI
public static IServiceCollection AddNotifications(this IServiceCollection services, IConfiguration configuration) =>
services
.Configure<NotifiarrConfig>(configuration.GetSection(NotifiarrConfig.SectionName))
.Configure<AppriseConfig>(configuration.GetSection(AppriseConfig.SectionName))
.AddTransient<INotifiarrProxy, NotifiarrProxy>()
.AddTransient<INotificationProvider, NotifiarrProvider>()
.AddTransient<IAppriseProxy, AppriseProxy>()
.AddTransient<INotificationProvider, AppriseProvider>()
.AddTransient<INotificationPublisher, NotificationPublisher>()
.AddTransient<INotificationFactory, NotificationFactory>()
.AddTransient<NotificationService>();

View File

@@ -3,7 +3,7 @@
"HTTP_MAX_RETRIES": 0,
"HTTP_TIMEOUT": 10,
"Logging": {
"LogLevel": "Debug",
"LogLevel": "Verbose",
"Enhanced": true,
"File": {
"Enabled": false,
@@ -34,7 +34,14 @@
"STALLED_MAX_STRIKES": 5,
"STALLED_RESET_STRIKES_ON_PROGRESS": true,
"STALLED_IGNORE_PRIVATE": true,
"STALLED_DELETE_PRIVATE": false
"STALLED_DELETE_PRIVATE": false,
"SLOW_MAX_STRIKES": 5,
"SLOW_RESET_STRIKES_ON_PROGRESS": true,
"SLOW_IGNORE_PRIVATE": false,
"SLOW_DELETE_PRIVATE": false,
"SLOW_MIN_SPEED": "1MB",
"SLOW_MAX_TIME": 20,
"SLOW_IGNORE_ABOVE_SIZE": "4GB"
},
"DownloadCleaner": {
"Enabled": false,
@@ -52,15 +59,18 @@
"DOWNLOAD_CLIENT": "qbittorrent",
"qBittorrent": {
"Url": "http://localhost:8080",
"URL_BASE": "",
"Username": "test",
"Password": "testing"
},
"Deluge": {
"Url": "http://localhost:8112",
"URL_BASE": "",
"Password": "testing"
},
"Transmission": {
"Url": "http://localhost:9091",
"URL_BASE": "transmission",
"Username": "test",
"Password": "testing"
},
@@ -107,9 +117,19 @@
"Notifiarr": {
"ON_IMPORT_FAILED_STRIKE": true,
"ON_STALLED_STRIKE": true,
"ON_SLOW_STRIKE": true,
"ON_QUEUE_ITEM_DELETED": true,
"ON_DOWNLOAD_CLEANED": true,
"API_KEY": "",
"CHANNEL_ID": ""
},
"Apprise": {
"ON_IMPORT_FAILED_STRIKE": true,
"ON_STALLED_STRIKE": true,
"ON_SLOW_STRIKE": true,
"ON_QUEUE_ITEM_DELETED": true,
"ON_DOWNLOAD_CLEANED": true,
"URL": "http://localhost:8000",
"KEY": ""
}
}

View File

@@ -42,15 +42,18 @@
"DOWNLOAD_CLIENT": "none",
"qBittorrent": {
"Url": "http://localhost:8080",
"URL_BASE": "",
"Username": "",
"Password": ""
},
"Deluge": {
"Url": "http://localhost:8112",
"URL_BASE": "",
"Password": "testing"
},
"Transmission": {
"Url": "http://localhost:9091",
"URL_BASE": "transmission",
"Username": "test",
"Password": "testing"
},
@@ -97,9 +100,19 @@
"Notifiarr": {
"ON_IMPORT_FAILED_STRIKE": false,
"ON_STALLED_STRIKE": false,
"ON_SLOW_STRIKE": false,
"ON_QUEUE_ITEM_DELETED": false,
"ON_DOWNLOAD_CLEANED": false,
"API_KEY": "",
"CHANNEL_ID": ""
},
"Apprise": {
"ON_IMPORT_FAILED_STRIKE": false,
"ON_STALLED_STRIKE": false,
"ON_SLOW_STRIKE": false,
"ON_QUEUE_ITEM_DELETED": false,
"ON_DOWNLOAD_CLEANED": false,
"URL": "",
"KEY": ""
}
}

View File

@@ -39,7 +39,7 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
});
// Act
sut.ResetStrikesOnProgress("test-hash", 100);
sut.ResetStalledStrikesOnProgress("test-hash", 100);
// Assert
_fixture.Cache.ReceivedCalls().ShouldBeEmpty();
@@ -50,19 +50,19 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
{
// Arrange
const string hash = "test-hash";
CacheItem cacheItem = new CacheItem { Downloaded = 100 };
StalledCacheItem stalledCacheItem = new StalledCacheItem { Downloaded = 100 };
_fixture.Cache.TryGetValue(Arg.Any<object>(), out Arg.Any<object?>())
.Returns(x =>
{
x[1] = cacheItem;
x[1] = stalledCacheItem;
return true;
});
TestDownloadService sut = _fixture.CreateSut();
// Act
sut.ResetStrikesOnProgress(hash, 200);
sut.ResetStalledStrikesOnProgress(hash, 200);
// Assert
_fixture.Cache.Received(1).Remove(CacheKeys.Strike(StrikeType.Stalled, hash));
@@ -73,20 +73,20 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
{
// Arrange
const string hash = "test-hash";
CacheItem cacheItem = new CacheItem { Downloaded = 200 };
StalledCacheItem stalledCacheItem = new StalledCacheItem { Downloaded = 200 };
_fixture.Cache
.TryGetValue(Arg.Any<object>(), out Arg.Any<object?>())
.Returns(x =>
{
x[1] = cacheItem;
x[1] = stalledCacheItem;
return true;
});
TestDownloadService sut = _fixture.CreateSut();
// Act
sut.ResetStrikesOnProgress(hash, 100);
sut.ResetStalledStrikesOnProgress(hash, 100);
// Assert
_fixture.Cache.DidNotReceive().Remove(Arg.Any<object>());
@@ -98,27 +98,6 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
public StrikeAndCheckLimitTests(DownloadServiceFixture fixture) : base(fixture)
{
}
[Fact]
public async Task ShouldDelegateCallToStriker()
{
// Arrange
const string hash = "test-hash";
const string itemName = "test-item";
_fixture.Striker.StrikeAndCheckLimit(hash, itemName, 3, StrikeType.Stalled)
.Returns(true);
TestDownloadService sut = _fixture.CreateSut();
// Act
bool result = await sut.StrikeAndCheckLimit(hash, itemName);
// Assert
result.ShouldBeTrue();
await _fixture.Striker
.Received(1)
.StrikeAndCheckLimit(hash, itemName, 3, StrikeType.Stalled);
}
}
public class ShouldCleanDownloadTests : DownloadServiceTests

View File

@@ -3,6 +3,7 @@ using System.Text.RegularExpressions;
using Common.Configuration.ContentBlocker;
using Common.Configuration.DownloadCleaner;
using Common.Configuration.QueueCleaner;
using Domain.Enums;
using Infrastructure.Interceptors;
using Infrastructure.Verticals.ContentBlocker;
using Infrastructure.Verticals.DownloadClient;
@@ -35,7 +36,7 @@ public class TestDownloadService : DownloadService
public override void Dispose() { }
public override Task LoginAsync() => Task.CompletedTask;
public override Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads) => Task.FromResult(new StalledResult());
public override Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads) => Task.FromResult(new DownloadCheckResult());
public override Task<BlockFilesResult> BlockUnwantedFilesAsync(string hash, BlocklistType blocklistType,
ConcurrentBag<string> patterns, ConcurrentBag<Regex> regexes, IReadOnlyList<string> ignoredDownloads) => Task.FromResult(new BlockFilesResult());
public override Task DeleteDownload(string hash) => Task.CompletedTask;
@@ -44,7 +45,6 @@ public class TestDownloadService : DownloadService
IReadOnlyList<string> ignoredDownloads) => Task.CompletedTask;
// Expose protected methods for testing
public new void ResetStrikesOnProgress(string hash, long downloaded) => base.ResetStrikesOnProgress(hash, downloaded);
public new Task<bool> StrikeAndCheckLimit(string hash, string itemName) => base.StrikeAndCheckLimit(hash, itemName);
public new void ResetStalledStrikesOnProgress(string hash, long downloaded) => base.ResetStalledStrikesOnProgress(hash, downloaded);
public new SeedingCheckResult ShouldCleanDownload(double ratio, TimeSpan seedingTime, Category category) => base.ShouldCleanDownload(ratio, seedingTime, category);
}

View File

@@ -1,10 +1,11 @@
using Domain.Models.Deluge.Response;
using Infrastructure.Helpers;
namespace Infrastructure.Extensions;
public static class DelugeExtensions
{
public static bool ShouldIgnore(this TorrentStatus download, IReadOnlyList<string> ignoredDownloads)
public static bool ShouldIgnore(this DownloadStatus download, IReadOnlyList<string> ignoredDownloads)
{
foreach (string value in ignoredDownloads)
{
@@ -18,7 +19,7 @@ public static class DelugeExtensions
return true;
}
if (download.Trackers.Any(x => x.Url.Host.EndsWith(value, StringComparison.InvariantCultureIgnoreCase)))
if (download.Trackers.Any(x => UriService.GetDomain(x.Url)?.EndsWith(value, StringComparison.InvariantCultureIgnoreCase) ?? false))
{
return true;
}

View File

@@ -1,4 +1,5 @@
using QBittorrent.Client;
using Infrastructure.Helpers;
using QBittorrent.Client;
namespace Infrastructure.Extensions;
@@ -29,9 +30,16 @@ public static class QBitExtensions
public static bool ShouldIgnore(this TorrentTracker tracker, IReadOnlyList<string> ignoredDownloads)
{
string? trackerUrl = UriService.GetDomain(tracker.Url);
if (trackerUrl is null)
{
return false;
}
foreach (string value in ignoredDownloads)
{
if (tracker.Url.Host.EndsWith(value, StringComparison.InvariantCultureIgnoreCase))
if (trackerUrl.EndsWith(value, StringComparison.InvariantCultureIgnoreCase))
{
return true;
}

View File

@@ -1,4 +1,5 @@
using Transmission.API.RPC.Entity;
using Infrastructure.Helpers;
using Transmission.API.RPC.Entity;
namespace Infrastructure.Extensions;
@@ -19,7 +20,7 @@ public static class TransmissionExtensions
}
bool? hasIgnoredTracker = download.Trackers?
.Any(x => new Uri(x.Announce!).Host.EndsWith(value, StringComparison.InvariantCultureIgnoreCase));
.Any(x => UriService.GetDomain(x.Announce)?.EndsWith(value, StringComparison.InvariantCultureIgnoreCase) ?? false);
if (hasIgnoredTracker is true)
{

View File

@@ -10,7 +10,7 @@ public static class CacheKeys
public static string BlocklistPatterns(InstanceType instanceType) => $"{instanceType.ToString()}_patterns";
public static string BlocklistRegexes(InstanceType instanceType) => $"{instanceType.ToString()}_regexes";
public static string Item(string hash) => $"item_{hash}";
public static string StrikeItem(string hash, StrikeType strikeType) => $"item_{hash}_{strikeType.ToString()}";
public static string IgnoredDownloads(string name) => $"{name}_ignored";
}

View File

@@ -0,0 +1,37 @@
using System.Text.RegularExpressions;
namespace Infrastructure.Helpers;
public static class UriService
{
public static string? GetDomain(string? input)
{
if (string.IsNullOrWhiteSpace(input))
{
return null;
}
// add "http://" if scheme is missing to help Uri.TryCreate
if (!input.StartsWith("http", StringComparison.OrdinalIgnoreCase))
{
input = "http://" + input;
}
if (Uri.TryCreate(input, UriKind.Absolute, out var uri))
{
return uri.Host;
}
// url might be malformed
var regex = new Regex(@"^(?:https?:\/\/)?([^\/\?:]+)", RegexOptions.IgnoreCase);
var match = regex.Match(input);
if (match.Success)
{
return match.Groups[1].Value;
}
// could not extract
return null;
}
}

View File

@@ -12,8 +12,8 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="FLM.QBittorrent" Version="1.0.0" />
<PackageReference Include="FLM.Transmission" Version="1.0.2" />
<PackageReference Include="FLM.QBittorrent" Version="1.0.1" />
<PackageReference Include="FLM.Transmission" Version="1.0.3" />
<PackageReference Include="Mapster" Version="7.4.0" />
<PackageReference Include="MassTransit" Version="8.3.6" />
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="9.0.2" />

View File

@@ -43,9 +43,11 @@ public abstract class ArrClient : IArrClient
public virtual async Task<QueueListResponse> GetQueueItemsAsync(ArrInstance arrInstance, int page)
{
Uri uri = new(arrInstance.Url, GetQueueUrlPath(page));
UriBuilder uriBuilder = new(arrInstance.Url);
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/{GetQueueUrlPath().TrimStart('/')}";
uriBuilder.Query = GetQueueUrlQuery(page);
using HttpRequestMessage request = new(HttpMethod.Get, uri);
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
SetApiKey(request, arrInstance.ApiKey);
using HttpResponseMessage response = await _httpClient.SendAsync(request);
@@ -56,7 +58,7 @@ public abstract class ArrClient : IArrClient
}
catch
{
_logger.LogError("queue list failed | {uri}", uri);
_logger.LogError("queue list failed | {uri}", uriBuilder.Uri);
throw;
}
@@ -65,7 +67,7 @@ public abstract class ArrClient : IArrClient
if (queueResponse is null)
{
throw new Exception($"unrecognized queue list response | {uri} | {responseBody}");
throw new Exception($"unrecognized queue list response | {uriBuilder.Uri} | {responseBody}");
}
return queueResponse;
@@ -112,13 +114,20 @@ public abstract class ArrClient : IArrClient
return false;
}
public virtual async Task DeleteQueueItemAsync(ArrInstance arrInstance, QueueRecord record, bool removeFromClient)
public virtual async Task DeleteQueueItemAsync(
ArrInstance arrInstance,
QueueRecord record,
bool removeFromClient,
DeleteReason deleteReason
)
{
Uri uri = new(arrInstance.Url, GetQueueDeleteUrlPath(record.Id, removeFromClient));
UriBuilder uriBuilder = new(arrInstance.Url);
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/{GetQueueDeleteUrlPath(record.Id).TrimStart('/')}";
uriBuilder.Query = GetQueueDeleteUrlQuery(removeFromClient);
try
{
using HttpRequestMessage request = new(HttpMethod.Delete, uri);
using HttpRequestMessage request = new(HttpMethod.Delete, uriBuilder.Uri);
SetApiKey(request, arrInstance.ApiKey);
HttpResponseMessage? response = await _dryRunInterceptor.InterceptAsync<HttpResponseMessage>(SendRequestAsync, request);
@@ -126,15 +135,16 @@ public abstract class ArrClient : IArrClient
_logger.LogInformation(
removeFromClient
? "queue item deleted | {url} | {title}"
: "queue item removed from arr | {url} | {title}",
? "queue item deleted with reason {reason} | {url} | {title}"
: "queue item removed from arr with reason {reason} | {url} | {title}",
deleteReason.ToString(),
arrInstance.Url,
record.Title
);
}
catch
{
_logger.LogError("queue delete failed | {uri} | {title}", uri, record.Title);
_logger.LogError("queue delete failed | {uri} | {title}", uriBuilder.Uri, record.Title);
throw;
}
}
@@ -152,9 +162,13 @@ public abstract class ArrClient : IArrClient
return true;
}
protected abstract string GetQueueUrlPath(int page);
protected abstract string GetQueueUrlPath();
protected abstract string GetQueueDeleteUrlPath(long recordId, bool removeFromClient);
protected abstract string GetQueueUrlQuery(int page);
protected abstract string GetQueueDeleteUrlPath(long recordId);
protected abstract string GetQueueDeleteUrlQuery(bool removeFromClient);
protected virtual void SetApiKey(HttpRequestMessage request, string apiKey)
{

View File

@@ -11,7 +11,7 @@ public interface IArrClient
Task<bool> ShouldRemoveFromQueue(InstanceType instanceType, QueueRecord record, bool isPrivateDownload);
Task DeleteQueueItemAsync(ArrInstance arrInstance, QueueRecord record, bool removeFromClient);
Task DeleteQueueItemAsync(ArrInstance arrInstance, QueueRecord record, bool removeFromClient, DeleteReason deleteReason);
Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items);

View File

@@ -27,29 +27,42 @@ public class LidarrClient : ArrClient, ILidarrClient
{
}
protected override string GetQueueUrlPath(int page)
protected override string GetQueueUrlPath()
{
return $"/api/v1/queue?page={page}&pageSize=200&includeUnknownArtistItems=true&includeArtist=true&includeAlbum=true";
return "/api/v1/queue";
}
protected override string GetQueueDeleteUrlPath(long recordId, bool removeFromClient)
protected override string GetQueueUrlQuery(int page)
{
string path = $"/api/v1/queue/{recordId}?blocklist=true&skipRedownload=true&changeCategory=false";
return $"page={page}&pageSize=200&includeUnknownArtistItems=true&includeArtist=true&includeAlbum=true";
}
path += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
protected override string GetQueueDeleteUrlPath(long recordId)
{
return $"/api/v1/queue/{recordId}";
}
return path;
protected override string GetQueueDeleteUrlQuery(bool removeFromClient)
{
string query = "blocklist=true&skipRedownload=true&changeCategory=false";
query += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
return query;
}
public override async Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items)
{
if (items?.Count is null or 0) return;
if (items?.Count is null or 0)
{
return;
}
Uri uri = new(arrInstance.Url, "/api/v1/command");
UriBuilder uriBuilder = new(arrInstance.Url);
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v1/command";
foreach (var command in GetSearchCommands(items))
{
using HttpRequestMessage request = new(HttpMethod.Post, uri);
using HttpRequestMessage request = new(HttpMethod.Post, uriBuilder.Uri);
request.Content = new StringContent(
JsonConvert.SerializeObject(command, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore }),
Encoding.UTF8,
@@ -132,8 +145,11 @@ public class LidarrClient : ArrClient, ILidarrClient
private async Task<List<Album>?> GetAlbumsAsync(ArrInstance arrInstance, List<long> albumIds)
{
Uri uri = new(arrInstance.Url, $"api/v1/album?{string.Join('&', albumIds.Select(x => $"albumIds={x}"))}");
using HttpRequestMessage request = new(HttpMethod.Get, uri);
UriBuilder uriBuilder = new(arrInstance.Url);
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v1/album";
uriBuilder.Query = string.Join('&', albumIds.Select(x => $"albumIds={x}"));
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
SetApiKey(request, arrInstance.ApiKey);
using var response = await _httpClient.SendAsync(request);

View File

@@ -27,18 +27,27 @@ public class RadarrClient : ArrClient, IRadarrClient
{
}
protected override string GetQueueUrlPath(int page)
protected override string GetQueueUrlPath()
{
return $"/api/v3/queue?page={page}&pageSize=200&includeUnknownMovieItems=true&includeMovie=true";
return "/api/v3/queue";
}
protected override string GetQueueDeleteUrlPath(long recordId, bool removeFromClient)
protected override string GetQueueUrlQuery(int page)
{
string path = $"/api/v3/queue/{recordId}?blocklist=true&skipRedownload=true&changeCategory=false";
path += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
return $"page={page}&pageSize=200&includeUnknownMovieItems=true&includeMovie=true";
}
return path;
protected override string GetQueueDeleteUrlPath(long recordId)
{
return $"/api/v3/queue/{recordId}";
}
protected override string GetQueueDeleteUrlQuery(bool removeFromClient)
{
string query = "blocklist=true&skipRedownload=true&changeCategory=false";
query += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
return query;
}
public override async Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items)
@@ -50,14 +59,16 @@ public class RadarrClient : ArrClient, IRadarrClient
List<long> ids = items.Select(item => item.Id).ToList();
Uri uri = new(arrInstance.Url, "/api/v3/command");
UriBuilder uriBuilder = new(arrInstance.Url);
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/command";
RadarrCommand command = new()
{
Name = "MoviesSearch",
MovieIds = ids,
};
using HttpRequestMessage request = new(HttpMethod.Post, uri);
using HttpRequestMessage request = new(HttpMethod.Post, uriBuilder.Uri);
request.Content = new StringContent(
JsonConvert.SerializeObject(command),
Encoding.UTF8,
@@ -135,8 +146,10 @@ public class RadarrClient : ArrClient, IRadarrClient
private async Task<Movie?> GetMovie(ArrInstance arrInstance, long movieId)
{
Uri uri = new(arrInstance.Url, $"api/v3/movie/{movieId}");
using HttpRequestMessage request = new(HttpMethod.Get, uri);
UriBuilder uriBuilder = new(arrInstance.Url);
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/movie/{movieId}";
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
SetApiKey(request, arrInstance.ApiKey);
using HttpResponseMessage response = await _httpClient.SendAsync(request);

View File

@@ -28,18 +28,27 @@ public class SonarrClient : ArrClient, ISonarrClient
{
}
protected override string GetQueueUrlPath(int page)
protected override string GetQueueUrlPath()
{
return $"/api/v3/queue?page={page}&pageSize=200&includeUnknownSeriesItems=true&includeSeries=true&includeEpisode=true";
return "/api/v3/queue";
}
protected override string GetQueueDeleteUrlPath(long recordId, bool removeFromClient)
protected override string GetQueueUrlQuery(int page)
{
string path = $"/api/v3/queue/{recordId}?blocklist=true&skipRedownload=true&changeCategory=false";
return $"page={page}&pageSize=200&includeUnknownSeriesItems=true&includeSeries=true&includeEpisode=true";
}
path += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
protected override string GetQueueDeleteUrlPath(long recordId)
{
return $"/api/v3/queue/{recordId}";
}
return path;
protected override string GetQueueDeleteUrlQuery(bool removeFromClient)
{
string query = "blocklist=true&skipRedownload=true&changeCategory=false";
query += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
return query;
}
public override async Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items)
@@ -49,11 +58,12 @@ public class SonarrClient : ArrClient, ISonarrClient
return;
}
Uri uri = new(arrInstance.Url, "/api/v3/command");
UriBuilder uriBuilder = new(arrInstance.Url);
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/command";
foreach (SonarrCommand command in GetSearchCommands(items.Cast<SonarrSearchItem>().ToHashSet()))
{
using HttpRequestMessage request = new(HttpMethod.Post, uri);
using HttpRequestMessage request = new(HttpMethod.Post, uriBuilder.Uri);
request.Content = new StringContent(
JsonConvert.SerializeObject(command, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore }),
Encoding.UTF8,
@@ -199,8 +209,11 @@ public class SonarrClient : ArrClient, ISonarrClient
private async Task<List<Episode>?> GetEpisodesAsync(ArrInstance arrInstance, List<long> episodeIds)
{
Uri uri = new(arrInstance.Url, $"api/v3/episode?{string.Join('&', episodeIds.Select(x => $"episodeIds={x}"))}");
using HttpRequestMessage request = new(HttpMethod.Get, uri);
UriBuilder uriBuilder = new(arrInstance.Url);
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/episode";
uriBuilder.Query = string.Join('&', episodeIds.Select(x => $"episodeIds={x}"));
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
SetApiKey(request, arrInstance.ApiKey);
using HttpResponseMessage response = await _httpClient.SendAsync(request);
@@ -212,8 +225,10 @@ public class SonarrClient : ArrClient, ISonarrClient
private async Task<Series?> GetSeriesAsync(ArrInstance arrInstance, long seriesId)
{
Uri uri = new(arrInstance.Url, $"api/v3/series/{seriesId}");
using HttpRequestMessage request = new(HttpMethod.Get, uri);
UriBuilder uriBuilder = new(arrInstance.Url);
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/series/{seriesId}";
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
SetApiKey(request, arrInstance.ApiKey);
using HttpResponseMessage response = await _httpClient.SendAsync(request);

View File

@@ -142,7 +142,7 @@ public sealed class ContentBlocker : GenericHandler
removeFromClient = false;
}
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient);
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient, DeleteReason.AllFilesBlocked);
await _notifier.NotifyQueueItemDeleted(removeFromClient, DeleteReason.AllFilesBlocked);
}
});

View File

@@ -27,12 +27,15 @@ public sealed class DelugeClient
"label",
"seeding_time",
"ratio",
"trackers"
"trackers",
"download_payload_rate",
"total_size"
];
public DelugeClient(IOptions<DelugeConfig> config, IHttpClientFactory httpClientFactory)
{
_config = config.Value;
_config.Validate();
_httpClient = httpClientFactory.CreateClient(nameof(DelugeService));
}
@@ -77,18 +80,31 @@ public sealed class DelugeClient
return torrents.FirstOrDefault();
}
public async Task<TorrentStatus?> GetTorrentStatus(string hash)
public async Task<DownloadStatus?> GetTorrentStatus(string hash)
{
return await SendRequest<TorrentStatus?>(
"web.get_torrent_status",
hash,
Fields
);
try
{
return await SendRequest<DownloadStatus?>(
"web.get_torrent_status",
hash,
Fields
);
}
catch (DelugeClientException e)
{
// Deluge returns an error when the torrent is not found
if (e.Message == "AttributeError: 'NoneType' object has no attribute 'call'")
{
return null;
}
throw;
}
}
public async Task<List<TorrentStatus>?> GetStatusForAllTorrents()
public async Task<List<DownloadStatus>?> GetStatusForAllTorrents()
{
Dictionary<string, TorrentStatus>? downloads = await SendRequest<Dictionary<string, TorrentStatus>?>(
Dictionary<string, DownloadStatus>? downloads = await SendRequest<Dictionary<string, DownloadStatus>?>(
"core.get_torrents_status",
"",
Fields
@@ -121,8 +137,12 @@ public sealed class DelugeClient
{
StringContent content = new StringContent(json);
content.Headers.ContentType = new MediaTypeWithQualityHeaderValue("application/json");
var responseMessage = await _httpClient.PostAsync(new Uri(_config.Url, "/json"), content);
UriBuilder uriBuilder = new(_config.Url);
uriBuilder.Path = string.IsNullOrEmpty(_config.UrlBase)
? $"{uriBuilder.Path.TrimEnd('/')}/json"
: $"{uriBuilder.Path.TrimEnd('/')}/{_config.UrlBase.TrimStart('/').TrimEnd('/')}/json";
var responseMessage = await _httpClient.PostAsync(uriBuilder.Uri, content);
responseMessage.EnsureSuccessStatusCode();
var responseJson = await responseMessage.Content.ReadAsStringAsync();

View File

@@ -1,10 +1,12 @@
using System.Collections.Concurrent;
using System.Globalization;
using System.Text.RegularExpressions;
using Common.Attributes;
using Common.Configuration.ContentBlocker;
using Common.Configuration.DownloadCleaner;
using Common.Configuration.DownloadClient;
using Common.Configuration.QueueCleaner;
using Common.CustomDataTypes;
using Domain.Enums;
using Domain.Models.Deluge.Response;
using Infrastructure.Extensions;
@@ -50,14 +52,14 @@ public class DelugeService : DownloadService, IDelugeService
}
/// <inheritdoc/>
public override async Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
public override async Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
{
hash = hash.ToLowerInvariant();
DelugeContents? contents = null;
StalledResult result = new();
DownloadCheckResult result = new();
TorrentStatus? download = await _client.GetTorrentStatus(hash);
DownloadStatus? download = await _client.GetTorrentStatus(hash);
if (download?.Hash is null)
{
@@ -65,6 +67,8 @@ public class DelugeService : DownloadService, IDelugeService
return result;
}
result.IsPrivate = download.Private;
if (ignoredDownloads.Count > 0 && download.ShouldIgnore(ignoredDownloads))
{
_logger.LogInformation("skip | download is ignored | {name}", download.Name);
@@ -79,6 +83,7 @@ public class DelugeService : DownloadService, IDelugeService
{
_logger.LogDebug(exception, "failed to find torrent {hash} in the download client", hash);
}
bool shouldRemove = contents?.Contents?.Count > 0;
@@ -92,17 +97,15 @@ public class DelugeService : DownloadService, IDelugeService
if (shouldRemove)
{
result.DeleteReason = DeleteReason.AllFilesBlocked;
// remove if all files are unwanted
result.ShouldRemove = true;
result.DeleteReason = DeleteReason.AllFilesSkipped;
return result;
}
result.ShouldRemove = shouldRemove || await IsItemStuckAndShouldRemove(download);
result.IsPrivate = download.Private;
// remove if download is stuck
(result.ShouldRemove, result.DeleteReason) = await EvaluateDownloadRemoval(download);
if (!shouldRemove && result.ShouldRemove)
{
result.DeleteReason = DeleteReason.Stalled;
}
return result;
}
@@ -114,7 +117,7 @@ public class DelugeService : DownloadService, IDelugeService
{
hash = hash.ToLowerInvariant();
TorrentStatus? download = await _client.GetTorrentStatus(hash);
DownloadStatus? download = await _client.GetTorrentStatus(hash);
BlockFilesResult result = new();
if (download?.Hash is null)
@@ -123,9 +126,6 @@ public class DelugeService : DownloadService, IDelugeService
return result;
}
var ceva = await _client.GetTorrentExtended(hash);
if (ignoredDownloads.Count > 0 && download.ShouldIgnore(ignoredDownloads))
{
_logger.LogInformation("skip | download is ignored | {name}", download.Name);
@@ -222,7 +222,7 @@ public class DelugeService : DownloadService, IDelugeService
public override async Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes,
IReadOnlyList<string> ignoredDownloads)
{
foreach (TorrentStatus download in downloads)
foreach (DownloadStatus download in downloads)
{
if (string.IsNullOrEmpty(download.Hash))
{
@@ -295,33 +295,90 @@ public class DelugeService : DownloadService, IDelugeService
await _client.ChangeFilesPriority(hash, sortedPriorities);
}
private async Task<bool> IsItemStuckAndShouldRemove(TorrentStatus status)
private async Task<(bool, DeleteReason)> EvaluateDownloadRemoval(DownloadStatus status)
{
(bool ShouldRemove, DeleteReason Reason) result = await CheckIfSlow(status);
if (result.ShouldRemove)
{
return result;
}
return await CheckIfStuck(status);
}
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfSlow(DownloadStatus download)
{
if (_queueCleanerConfig.SlowMaxStrikes is 0)
{
return (false, DeleteReason.None);
}
if (download.State is null || !download.State.Equals("Downloading", StringComparison.InvariantCultureIgnoreCase))
{
return (false, DeleteReason.None);
}
if (download.DownloadSpeed <= 0)
{
return (false, DeleteReason.None);
}
if (_queueCleanerConfig.SlowIgnorePrivate && download.Private)
{
// ignore private trackers
_logger.LogDebug("skip slow check | download is private | {name}", download.Name);
return (false, DeleteReason.None);
}
if (download.Size > (_queueCleanerConfig.SlowIgnoreAboveSizeByteSize?.Bytes ?? long.MaxValue))
{
_logger.LogDebug("skip slow check | download is too large | {name}", download.Name);
return (false, DeleteReason.None);
}
ByteSize minSpeed = _queueCleanerConfig.SlowMinSpeedByteSize;
ByteSize currentSpeed = new ByteSize(download.DownloadSpeed);
SmartTimeSpan maxTime = SmartTimeSpan.FromHours(_queueCleanerConfig.SlowMaxTime);
SmartTimeSpan currentTime = SmartTimeSpan.FromSeconds(download.Eta);
return await CheckIfSlow(
download.Hash!,
download.Name!,
minSpeed,
currentSpeed,
maxTime,
currentTime
);
}
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfStuck(DownloadStatus status)
{
if (_queueCleanerConfig.StalledMaxStrikes is 0)
{
return false;
return (false, DeleteReason.None);
}
if (_queueCleanerConfig.StalledIgnorePrivate && status.Private)
{
// ignore private trackers
_logger.LogDebug("skip stalled check | download is private | {name}", status.Name);
return false;
return (false, DeleteReason.None);
}
if (status.State is null || !status.State.Equals("Downloading", StringComparison.InvariantCultureIgnoreCase))
{
return false;
return (false, DeleteReason.None);
}
if (status.Eta > 0)
{
return false;
return (false, DeleteReason.None);
}
ResetStrikesOnProgress(status.Hash!, status.TotalDone);
return await StrikeAndCheckLimit(status.Hash!, status.Name!);
ResetStalledStrikesOnProgress(status.Hash!, status.TotalDone);
return (await _striker.StrikeAndCheckLimit(status.Hash!, status.Name!, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled), DeleteReason.Stalled);
}
private static void ProcessFiles(Dictionary<string, DelugeFileOrDirectory>? contents, Action<string, DelugeFileOrDirectory> processFile)

View File

@@ -2,7 +2,7 @@
namespace Infrastructure.Verticals.DownloadClient;
public sealed record StalledResult
public sealed record DownloadCheckResult
{
/// <summary>
/// True if the download should be removed; otherwise false.

View File

@@ -3,6 +3,7 @@ using System.Text.RegularExpressions;
using Common.Configuration.ContentBlocker;
using Common.Configuration.DownloadCleaner;
using Common.Configuration.QueueCleaner;
using Common.CustomDataTypes;
using Common.Helpers;
using Domain.Enums;
using Domain.Models.Cache;
@@ -60,7 +61,7 @@ public abstract class DownloadService : IDownloadService
public abstract Task LoginAsync();
public abstract Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads);
public abstract Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads);
/// <inheritdoc/>
public abstract Task<BlockFilesResult> BlockUnwantedFilesAsync(string hash,
@@ -78,32 +79,104 @@ public abstract class DownloadService : IDownloadService
public abstract Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes,
IReadOnlyList<string> ignoredDownloads);
protected void ResetStrikesOnProgress(string hash, long downloaded)
protected void ResetStalledStrikesOnProgress(string hash, long downloaded)
{
if (!_queueCleanerConfig.StalledResetStrikesOnProgress)
{
return;
}
if (_cache.TryGetValue(CacheKeys.Item(hash), out CacheItem? cachedItem) && cachedItem is not null && downloaded > cachedItem.Downloaded)
if (_cache.TryGetValue(CacheKeys.StrikeItem(hash, StrikeType.Stalled), out StalledCacheItem? cachedItem) &&
cachedItem is not null && downloaded > cachedItem.Downloaded)
{
// cache item found
_cache.Remove(CacheKeys.Strike(StrikeType.Stalled, hash));
_logger.LogDebug("resetting strikes for {hash} due to progress", hash);
_logger.LogDebug("resetting stalled strikes for {hash} due to progress", hash);
}
_cache.Set(CacheKeys.Item(hash), new CacheItem { Downloaded = downloaded }, _cacheOptions);
_cache.Set(CacheKeys.StrikeItem(hash, StrikeType.Stalled), new StalledCacheItem { Downloaded = downloaded }, _cacheOptions);
}
protected void ResetSlowSpeedStrikesOnProgress(string downloadName, string hash)
{
if (!_queueCleanerConfig.SlowResetStrikesOnProgress)
{
return;
}
string key = CacheKeys.Strike(StrikeType.SlowSpeed, hash);
if (!_cache.TryGetValue(key, out object? value) || value is null)
{
return;
}
_cache.Remove(key);
_logger.LogDebug("resetting slow speed strikes due to progress | {name}", downloadName);
}
protected void ResetSlowTimeStrikesOnProgress(string downloadName, string hash)
{
if (!_queueCleanerConfig.SlowResetStrikesOnProgress)
{
return;
}
string key = CacheKeys.Strike(StrikeType.SlowTime, hash);
if (!_cache.TryGetValue(key, out object? value) || value is null)
{
return;
}
_cache.Remove(key);
_logger.LogDebug("resetting slow time strikes due to progress | {name}", downloadName);
}
/// <summary>
/// Strikes an item and checks if the limit has been reached.
/// </summary>
/// <param name="hash">The torrent hash.</param>
/// <param name="itemName">The name or title of the item.</param>
/// <returns>True if the limit has been reached; otherwise, false.</returns>
protected async Task<bool> StrikeAndCheckLimit(string hash, string itemName)
protected async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfSlow(
string downloadHash,
string downloadName,
ByteSize minSpeed,
ByteSize currentSpeed,
SmartTimeSpan maxTime,
SmartTimeSpan currentTime
)
{
return await _striker.StrikeAndCheckLimit(hash, itemName, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled);
if (minSpeed.Bytes > 0 && currentSpeed < minSpeed)
{
_logger.LogTrace("slow speed | {speed}/s | {name}", currentSpeed.ToString(), downloadName);
bool shouldRemove = await _striker
.StrikeAndCheckLimit(downloadHash, downloadName, _queueCleanerConfig.SlowMaxStrikes, StrikeType.SlowSpeed);
if (shouldRemove)
{
return (true, DeleteReason.SlowSpeed);
}
}
else
{
ResetSlowSpeedStrikesOnProgress(downloadName, downloadHash);
}
if (maxTime.Time > TimeSpan.Zero && currentTime > maxTime)
{
_logger.LogTrace("slow estimated time | {time} | {name}", currentTime.ToString(), downloadName);
bool shouldRemove = await _striker
.StrikeAndCheckLimit(downloadHash, downloadName, _queueCleanerConfig.SlowMaxStrikes, StrikeType.SlowTime);
if (shouldRemove)
{
return (true, DeleteReason.SlowTime);
}
}
else
{
ResetSlowTimeStrikesOnProgress(downloadName, downloadHash);
}
return (false, DeleteReason.None);
}
protected SeedingCheckResult ShouldCleanDownload(double ratio, TimeSpan seedingTime, Category category)

View File

@@ -28,7 +28,7 @@ public class DummyDownloadService : DownloadService
return Task.CompletedTask;
}
public override Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
public override Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
{
throw new NotImplementedException();
}

View File

@@ -15,7 +15,7 @@ public interface IDownloadService : IDisposable
/// </summary>
/// <param name="hash">The download hash.</param>
/// <param name="ignoredDownloads">Downloads to ignore from processing.</param>
public Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads);
public Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads);
/// <summary>
/// Blocks unwanted files from being fully downloaded.

View File

@@ -5,6 +5,7 @@ using Common.Configuration.ContentBlocker;
using Common.Configuration.DownloadCleaner;
using Common.Configuration.DownloadClient;
using Common.Configuration.QueueCleaner;
using Common.CustomDataTypes;
using Common.Helpers;
using Domain.Enums;
using Infrastructure.Extensions;
@@ -45,7 +46,11 @@ public class QBitService : DownloadService, IQBitService
{
_config = config.Value;
_config.Validate();
_client = new(httpClientFactory.CreateClient(Constants.HttpClientWithRetryName), _config.Url);
UriBuilder uriBuilder = new(_config.Url);
uriBuilder.Path = string.IsNullOrEmpty(_config.UrlBase)
? uriBuilder.Path
: $"{uriBuilder.Path.TrimEnd('/')}/{_config.UrlBase.TrimStart('/')}";
_client = new(httpClientFactory.CreateClient(Constants.HttpClientWithRetryName), uriBuilder.Uri);
}
public override async Task LoginAsync()
@@ -59,9 +64,9 @@ public class QBitService : DownloadService, IQBitService
}
/// <inheritdoc/>
public override async Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
public override async Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
{
StalledResult result = new();
DownloadCheckResult result = new();
TorrentInfo? download = (await _client.GetTorrentListAsync(new TorrentListQuery { Hashes = [hash] }))
.FirstOrDefault();
@@ -92,30 +97,25 @@ public class QBitService : DownloadService, IQBitService
bool.TryParse(dictValue?.ToString(), out bool boolValue)
&& boolValue;
// if all files were blocked by qBittorrent
if (download is { CompletionOn: not null, Downloaded: null or 0 })
{
result.ShouldRemove = true;
result.DeleteReason = DeleteReason.AllFilesBlocked;
return result;
}
IReadOnlyList<TorrentContent>? files = await _client.GetTorrentContentsAsync(hash);
// if all files are marked as skip
if (files?.Count is > 0 && files.All(x => x.Priority is TorrentContentPriority.Skip))
{
result.ShouldRemove = true;
result.DeleteReason = DeleteReason.AllFilesBlocked;
// if all files were blocked by qBittorrent
if (download is { CompletionOn: not null, Downloaded: null or 0 })
{
result.DeleteReason = DeleteReason.AllFilesSkippedByQBit;
return result;
}
// remove if all files are unwanted
result.DeleteReason = DeleteReason.AllFilesSkipped;
return result;
}
result.ShouldRemove = await IsItemStuckAndShouldRemove(download, result.IsPrivate);
if (result.ShouldRemove)
{
result.DeleteReason = DeleteReason.Stalled;
}
(result.ShouldRemove, result.DeleteReason) = await EvaluateDownloadRemoval(download, result.IsPrivate);
return result;
}
@@ -333,36 +333,102 @@ public class QBitService : DownloadService, IQBitService
_client.Dispose();
}
private async Task<bool> IsItemStuckAndShouldRemove(TorrentInfo torrent, bool isPrivate)
private async Task<(bool, DeleteReason)> EvaluateDownloadRemoval(TorrentInfo torrent, bool isPrivate)
{
(bool ShouldRemove, DeleteReason Reason) result = await CheckIfSlow(torrent, isPrivate);
if (result.ShouldRemove)
{
return result;
}
return await CheckIfStuck(torrent, isPrivate);
}
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfSlow(TorrentInfo download, bool isPrivate)
{
if (_queueCleanerConfig.SlowMaxStrikes is 0)
{
return (false, DeleteReason.None);
}
if (download.State is not (TorrentState.Downloading or TorrentState.ForcedDownload))
{
return (false, DeleteReason.None);
}
if (download.DownloadSpeed <= 0)
{
return (false, DeleteReason.None);
}
if (_queueCleanerConfig.SlowIgnorePrivate && isPrivate)
{
// ignore private trackers
_logger.LogDebug("skip slow check | download is private | {name}", download.Name);
return (false, DeleteReason.None);
}
if (download.Size > (_queueCleanerConfig.SlowIgnoreAboveSizeByteSize?.Bytes ?? long.MaxValue))
{
_logger.LogDebug("skip slow check | download is too large | {name}", download.Name);
return (false, DeleteReason.None);
}
ByteSize minSpeed = _queueCleanerConfig.SlowMinSpeedByteSize;
ByteSize currentSpeed = new ByteSize(download.DownloadSpeed);
SmartTimeSpan maxTime = SmartTimeSpan.FromHours(_queueCleanerConfig.SlowMaxTime);
SmartTimeSpan currentTime = new SmartTimeSpan(download.EstimatedTime ?? TimeSpan.Zero);
return await CheckIfSlow(
download.Hash,
download.Name,
minSpeed,
currentSpeed,
maxTime,
currentTime
);
}
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfStuck(TorrentInfo torrent, bool isPrivate)
{
if (_queueCleanerConfig.StalledMaxStrikes is 0)
{
return false;
}
if (_queueCleanerConfig.StalledIgnorePrivate && isPrivate)
{
// ignore private trackers
_logger.LogDebug("skip stalled check | download is private | {name}", torrent.Name);
return false;
return (false, DeleteReason.None);
}
if (torrent.State is not TorrentState.StalledDownload and not TorrentState.FetchingMetadata
and not TorrentState.ForcedFetchingMetadata)
{
// ignore other states
return false;
return (false, DeleteReason.None);
}
if (_queueCleanerConfig.StalledIgnorePrivate && isPrivate)
{
// ignore private trackers
_logger.LogDebug("skip stalled check | download is private | {name}", torrent.Name);
return (false, DeleteReason.None);
}
if (torrent.State is TorrentState.StalledDownload)
{
_logger.LogTrace("stalled download | {name}", torrent.Name);
ResetStalledStrikesOnProgress(torrent.Hash, torrent.Downloaded ?? 0);
return (await _striker.StrikeAndCheckLimit(torrent.Hash, torrent.Name, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled), DeleteReason.Stalled);
}
ResetStrikesOnProgress(torrent.Hash, torrent.Downloaded ?? 0);
return await StrikeAndCheckLimit(torrent.Hash, torrent.Name);
_logger.LogTrace("downloading metadata | {name}", torrent.Name);
return (await _striker.StrikeAndCheckLimit(torrent.Hash, torrent.Name, _queueCleanerConfig.StalledMaxStrikes, StrikeType.DownloadingMetadata), DeleteReason.DownloadingMetadata);
}
private async Task<IReadOnlyList<TorrentTracker>> GetTrackersAsync(string hash)
{
return (await _client.GetTorrentTrackersAsync(hash))
.Where(x => !x.Url.ToString().Contains("**"))
.Where(x => x.Url.Contains("**"))
.ToList();
}
}

View File

@@ -5,6 +5,7 @@ using Common.Configuration.ContentBlocker;
using Common.Configuration.DownloadCleaner;
using Common.Configuration.DownloadClient;
using Common.Configuration.QueueCleaner;
using Common.CustomDataTypes;
using Common.Helpers;
using Domain.Enums;
using Infrastructure.Extensions;
@@ -26,7 +27,6 @@ public class TransmissionService : DownloadService, ITransmissionService
{
private readonly TransmissionConfig _config;
private readonly Client _client;
private TorrentInfo[]? _torrentsCache;
private static readonly string[] Fields =
[
@@ -42,7 +42,9 @@ public class TransmissionService : DownloadService, ITransmissionService
TorrentFields.DOWNLOAD_DIR,
TorrentFields.SECONDS_SEEDING,
TorrentFields.UPLOAD_RATIO,
TorrentFields.TRACKERS
TorrentFields.TRACKERS,
TorrentFields.RATE_DOWNLOAD,
TorrentFields.TOTAL_SIZE
];
public TransmissionService(
@@ -64,9 +66,13 @@ public class TransmissionService : DownloadService, ITransmissionService
{
_config = config.Value;
_config.Validate();
UriBuilder uriBuilder = new(_config.Url);
uriBuilder.Path = string.IsNullOrEmpty(_config.UrlBase)
? $"{uriBuilder.Path.TrimEnd('/')}/rpc"
: $"{uriBuilder.Path.TrimEnd('/')}/{_config.UrlBase.TrimStart('/').TrimEnd('/')}/rpc";
_client = new(
httpClientFactory.CreateClient(Constants.HttpClientWithRetryName),
new Uri(_config.Url, "/transmission/rpc").ToString(),
uriBuilder.Uri.ToString(),
login: _config.Username,
password: _config.Password
);
@@ -78,9 +84,9 @@ public class TransmissionService : DownloadService, ITransmissionService
}
/// <inheritdoc/>
public override async Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
public override async Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
{
StalledResult result = new();
DownloadCheckResult result = new();
TorrentInfo? download = await GetTorrentAsync(hash);
if (download is null)
@@ -115,17 +121,15 @@ public class TransmissionService : DownloadService, ITransmissionService
if (shouldRemove)
{
// remove if all files are unwanted
result.ShouldRemove = true;
result.DeleteReason = DeleteReason.AllFilesBlocked;
return result;
}
// remove if all files are unwanted or download is stuck
result.ShouldRemove = shouldRemove || await IsItemStuckAndShouldRemove(download);
// remove if download is stuck
(result.ShouldRemove, result.DeleteReason) = await EvaluateDownloadRemoval(download);
if (!shouldRemove && result.ShouldRemove)
{
result.DeleteReason = DeleteReason.Stalled;
}
return result;
}
@@ -334,60 +338,96 @@ public class TransmissionService : DownloadService, ITransmissionService
});
}
private async Task<bool> IsItemStuckAndShouldRemove(TorrentInfo torrent)
private async Task<(bool, DeleteReason)> EvaluateDownloadRemoval(TorrentInfo torrent)
{
(bool ShouldRemove, DeleteReason Reason) result = await CheckIfSlow(torrent);
if (result.ShouldRemove)
{
return result;
}
return await CheckIfStuck(torrent);
}
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfSlow(TorrentInfo download)
{
if (_queueCleanerConfig.SlowMaxStrikes is 0)
{
return (false, DeleteReason.None);
}
if (download.Status is not 4)
{
// not in downloading state
return (false, DeleteReason.None);
}
if (download.RateDownload <= 0)
{
return (false, DeleteReason.None);
}
if (_queueCleanerConfig.SlowIgnorePrivate && download.IsPrivate is true)
{
// ignore private trackers
_logger.LogDebug("skip slow check | download is private | {name}", download.Name);
return (false, DeleteReason.None);
}
if (download.TotalSize > (_queueCleanerConfig.SlowIgnoreAboveSizeByteSize?.Bytes ?? long.MaxValue))
{
_logger.LogDebug("skip slow check | download is too large | {name}", download.Name);
return (false, DeleteReason.None);
}
ByteSize minSpeed = _queueCleanerConfig.SlowMinSpeedByteSize;
ByteSize currentSpeed = new ByteSize(download.RateDownload ?? long.MaxValue);
SmartTimeSpan maxTime = SmartTimeSpan.FromHours(_queueCleanerConfig.SlowMaxTime);
SmartTimeSpan currentTime = SmartTimeSpan.FromSeconds(download.Eta ?? 0);
return await CheckIfSlow(
download.HashString!,
download.Name!,
minSpeed,
currentSpeed,
maxTime,
currentTime
);
}
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfStuck(TorrentInfo download)
{
if (_queueCleanerConfig.StalledMaxStrikes is 0)
{
return false;
return (false, DeleteReason.None);
}
if (_queueCleanerConfig.StalledIgnorePrivate && (torrent.IsPrivate ?? false))
{
// ignore private trackers
_logger.LogDebug("skip stalled check | download is private | {name}", torrent.Name);
return false;
}
if (torrent.Status is not 4)
if (download.Status is not 4)
{
// not in downloading state
return false;
}
if (torrent.Eta > 0)
{
return false;
return (false, DeleteReason.None);
}
ResetStrikesOnProgress(torrent.HashString!, torrent.DownloadedEver ?? 0);
return await StrikeAndCheckLimit(torrent.HashString!, torrent.Name!);
if (download.RateDownload > 0 || download.Eta > 0)
{
return (false, DeleteReason.None);
}
if (_queueCleanerConfig.StalledIgnorePrivate && (download.IsPrivate ?? false))
{
// ignore private trackers
_logger.LogDebug("skip stalled check | download is private | {name}", download.Name);
return (false, DeleteReason.None);
}
ResetStalledStrikesOnProgress(download.HashString!, download.DownloadedEver ?? 0);
return (await _striker.StrikeAndCheckLimit(download.HashString!, download.Name!, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled), DeleteReason.Stalled);
}
private async Task<TorrentInfo?> GetTorrentAsync(string hash)
{
TorrentInfo? torrent = _torrentsCache?
.FirstOrDefault(x => x.HashString.Equals(hash, StringComparison.InvariantCultureIgnoreCase));
if (_torrentsCache is null || torrent is null)
{
// refresh cache
_torrentsCache = (await _client.TorrentGetAsync(Fields))
?.Torrents;
}
if (_torrentsCache?.Length is null or 0)
{
_logger.LogDebug("could not list torrents | {url}", _config.Url);
}
torrent = _torrentsCache?.FirstOrDefault(x => x.HashString.Equals(hash, StringComparison.InvariantCultureIgnoreCase));
if (torrent is null)
{
_logger.LogDebug("could not find torrent | {hash} | {url}", hash, _config.Url);
}
return torrent;
}
private async Task<TorrentInfo?> GetTorrentAsync(string hash) =>
(await _client.TorrentGetAsync(Fields, hash))
?.Torrents
?.FirstOrDefault();
}

View File

@@ -0,0 +1,27 @@
using Common.Configuration.Notification;
namespace Infrastructure.Verticals.Notifications.Apprise;
public sealed record AppriseConfig : NotificationConfig
{
public const string SectionName = "Apprise";
public Uri? Url { get; init; }
public string? Key { get; init; }
public override bool IsValid()
{
if (Url is null)
{
return false;
}
if (string.IsNullOrEmpty(Key?.Trim()))
{
return false;
}
return true;
}
}

View File

@@ -0,0 +1,31 @@
using System.ComponentModel.DataAnnotations;
namespace Infrastructure.Verticals.Notifications.Apprise;
public sealed record ApprisePayload
{
[Required]
public string Title { get; init; }
[Required]
public string Body { get; init; }
public string Type { get; init; } = NotificationType.Info.ToString().ToLowerInvariant();
public string Format { get; init; } = FormatType.Text.ToString().ToLowerInvariant();
}
public enum NotificationType
{
Info,
Success,
Warning,
Failure
}
public enum FormatType
{
Text,
Markdown,
Html
}

View File

@@ -0,0 +1,90 @@
using System.Text;
using Infrastructure.Verticals.Notifications.Models;
using Microsoft.Extensions.Options;
namespace Infrastructure.Verticals.Notifications.Apprise;
public sealed class AppriseProvider : NotificationProvider
{
private readonly AppriseConfig _config;
private readonly IAppriseProxy _proxy;
public AppriseProvider(IOptions<AppriseConfig> config, IAppriseProxy proxy)
: base(config)
{
_config = config.Value;
_proxy = proxy;
}
public override string Name => "Apprise";
public override async Task OnFailedImportStrike(FailedImportStrikeNotification notification)
{
await _proxy.SendNotification(BuildPayload(notification, NotificationType.Warning), _config);
}
public override async Task OnStalledStrike(StalledStrikeNotification notification)
{
await _proxy.SendNotification(BuildPayload(notification, NotificationType.Warning), _config);
}
public override async Task OnSlowStrike(SlowStrikeNotification notification)
{
await _proxy.SendNotification(BuildPayload(notification, NotificationType.Warning), _config);
}
public override async Task OnQueueItemDeleted(QueueItemDeletedNotification notification)
{
await _proxy.SendNotification(BuildPayload(notification, NotificationType.Warning), _config);
}
public override async Task OnDownloadCleaned(DownloadCleanedNotification notification)
{
await _proxy.SendNotification(BuildPayload(notification, NotificationType.Warning), _config);
}
private static ApprisePayload BuildPayload(ArrNotification notification, NotificationType notificationType)
{
StringBuilder body = new();
body.AppendLine(notification.Description);
body.AppendLine();
body.AppendLine($"Instance type: {notification.InstanceType.ToString()}");
body.AppendLine($"Url: {notification.InstanceUrl}");
body.AppendLine($"Download hash: {notification.Hash}");
foreach (NotificationField field in notification.Fields ?? [])
{
body.AppendLine($"{field.Title}: {field.Text}");
}
ApprisePayload payload = new()
{
Title = notification.Title,
Body = body.ToString(),
Type = notificationType.ToString().ToLowerInvariant(),
};
return payload;
}
private static ApprisePayload BuildPayload(Notification notification, NotificationType notificationType)
{
StringBuilder body = new();
body.AppendLine(notification.Description);
body.AppendLine();
foreach (NotificationField field in notification.Fields ?? [])
{
body.AppendLine($"{field.Title}: {field.Text}");
}
ApprisePayload payload = new()
{
Title = notification.Title,
Body = body.ToString(),
Type = notificationType.ToString().ToLowerInvariant(),
};
return payload;
}
}

View File

@@ -0,0 +1,34 @@
using System.Text;
using Common.Helpers;
using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
namespace Infrastructure.Verticals.Notifications.Apprise;
public sealed class AppriseProxy : IAppriseProxy
{
private readonly HttpClient _httpClient;
public AppriseProxy(IHttpClientFactory httpClientFactory)
{
_httpClient = httpClientFactory.CreateClient(Constants.HttpClientWithRetryName);
}
public async Task SendNotification(ApprisePayload payload, AppriseConfig config)
{
string content = JsonConvert.SerializeObject(payload, new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
});
UriBuilder uriBuilder = new(config.Url);
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/notify/{config.Key}";
using HttpRequestMessage request = new(HttpMethod.Post, uriBuilder.Uri);
request.Method = HttpMethod.Post;
request.Content = new StringContent(content, Encoding.UTF8, "application/json");
using HttpResponseMessage response = await _httpClient.SendAsync(request);
response.EnsureSuccessStatusCode();
}
}

View File

@@ -0,0 +1,6 @@
namespace Infrastructure.Verticals.Notifications.Apprise;
public interface IAppriseProxy
{
Task SendNotification(ApprisePayload payload, AppriseConfig config);
}

View File

@@ -27,6 +27,9 @@ public sealed class NotificationConsumer<T> : IConsumer<T> where T : Notificatio
case StalledStrikeNotification stalledMessage:
await _notificationService.Notify(stalledMessage);
break;
case SlowStrikeNotification slowMessage:
await _notificationService.Notify(slowMessage);
break;
case QueueItemDeletedNotification queueItemDeleteMessage:
await _notificationService.Notify(queueItemDeleteMessage);
break;

View File

@@ -6,6 +6,8 @@ public interface INotificationFactory
List<INotificationProvider> OnStalledStrikeEnabled();
List<INotificationProvider> OnSlowStrikeEnabled();
List<INotificationProvider> OnQueueItemDeletedEnabled();
List<INotificationProvider> OnDownloadCleanedEnabled();

View File

@@ -12,6 +12,8 @@ public interface INotificationProvider
Task OnFailedImportStrike(FailedImportStrikeNotification notification);
Task OnStalledStrike(StalledStrikeNotification notification);
Task OnSlowStrike(SlowStrikeNotification notification);
Task OnQueueItemDeleted(QueueItemDeletedNotification notification);

View File

@@ -0,0 +1,5 @@
namespace Infrastructure.Verticals.Notifications.Models;
public sealed record SlowStrikeNotification : ArrNotification
{
}

View File

@@ -12,6 +12,8 @@ public class NotifiarrProvider : NotificationProvider
private const string WarningColor = "f0ad4e";
private const string ImportantColor = "bb2124";
private const string Logo = "https://github.com/flmorg/cleanuperr/blob/main/Logo/48.png?raw=true";
public override string Name => "Notifiarr";
public NotifiarrProvider(IOptions<NotifiarrConfig> config, INotifiarrProxy proxy)
: base(config)
@@ -20,8 +22,6 @@ public class NotifiarrProvider : NotificationProvider
_proxy = proxy;
}
public override string Name => "Notifiarr";
public override async Task OnFailedImportStrike(FailedImportStrikeNotification notification)
{
await _proxy.SendNotification(BuildPayload(notification, WarningColor), _config);
@@ -32,6 +32,11 @@ public class NotifiarrProvider : NotificationProvider
await _proxy.SendNotification(BuildPayload(notification, WarningColor), _config);
}
public override async Task OnSlowStrike(SlowStrikeNotification notification)
{
await _proxy.SendNotification(BuildPayload(notification, WarningColor), _config);
}
public override async Task OnQueueItemDeleted(QueueItemDeletedNotification notification)
{
await _proxy.SendNotification(BuildPayload(notification, ImportantColor), _config);

View File

@@ -5,7 +5,7 @@ using Newtonsoft.Json.Serialization;
namespace Infrastructure.Verticals.Notifications.Notifiarr;
public class NotifiarrProxy : INotifiarrProxy
public sealed class NotifiarrProxy : INotifiarrProxy
{
private readonly HttpClient _httpClient;

View File

@@ -24,6 +24,11 @@ public class NotificationFactory : INotificationFactory
ActiveProviders()
.Where(n => n.Config.OnStalledStrike)
.ToList();
public List<INotificationProvider> OnSlowStrikeEnabled() =>
ActiveProviders()
.Where(n => n.Config.OnSlowStrike)
.ToList();
public List<INotificationProvider> OnQueueItemDeletedEnabled() =>
ActiveProviders()

View File

@@ -18,6 +18,8 @@ public abstract class NotificationProvider : INotificationProvider
public abstract Task OnFailedImportStrike(FailedImportStrikeNotification notification);
public abstract Task OnStalledStrike(StalledStrikeNotification notification);
public abstract Task OnSlowStrike(SlowStrikeNotification notification);
public abstract Task OnQueueItemDeleted(QueueItemDeletedNotification notification);

View File

@@ -48,11 +48,16 @@ public class NotificationPublisher : INotificationPublisher
switch (strikeType)
{
case StrikeType.Stalled:
case StrikeType.DownloadingMetadata:
await _dryRunInterceptor.InterceptAsync(Notify<StalledStrikeNotification>, notification.Adapt<StalledStrikeNotification>());
break;
case StrikeType.ImportFailed:
await _dryRunInterceptor.InterceptAsync(Notify<FailedImportStrikeNotification>, notification.Adapt<FailedImportStrikeNotification>());
break;
case StrikeType.SlowSpeed:
case StrikeType.SlowTime:
await _dryRunInterceptor.InterceptAsync(Notify<SlowStrikeNotification>, notification.Adapt<SlowStrikeNotification>());
break;
}
}
catch (Exception ex)

View File

@@ -44,6 +44,21 @@ public class NotificationService
}
}
public async Task Notify(SlowStrikeNotification notification)
{
foreach (INotificationProvider provider in _notificationFactory.OnSlowStrikeEnabled())
{
try
{
await provider.OnSlowStrike(notification);
}
catch (Exception exception)
{
_logger.LogWarning(exception, "failed to send notification | provider {provider}", provider.Name);
}
}
}
public async Task Notify(QueueItemDeletedNotification notification)
{
foreach (INotificationProvider provider in _notificationFactory.OnQueueItemDeletedEnabled())

View File

@@ -77,6 +77,8 @@ public sealed class QueueCleaner : GenericHandler
QueueRecord record = group.First();
_logger.LogTrace("processing | {title} | {id}", record.Title, record.DownloadId);
if (!arrClient.IsRecordValid(record))
{
continue;
@@ -91,7 +93,7 @@ public sealed class QueueCleaner : GenericHandler
// push record to context
ContextProvider.Set(nameof(QueueRecord), record);
StalledResult stalledCheckResult = new();
DownloadCheckResult downloadCheckResult = new();
if (record.Protocol is "torrent" && _downloadClientConfig.DownloadClient is not Common.Enums.DownloadClient.Disabled)
{
@@ -102,14 +104,14 @@ public sealed class QueueCleaner : GenericHandler
}
// stalled download check
stalledCheckResult = await _downloadService.ShouldRemoveFromArrQueueAsync(record.DownloadId, ignoredDownloads);
downloadCheckResult = await _downloadService.ShouldRemoveFromArrQueueAsync(record.DownloadId, ignoredDownloads);
}
// failed import check
bool shouldRemoveFromArr = await arrClient.ShouldRemoveFromQueue(instanceType, record, stalledCheckResult.IsPrivate);
DeleteReason deleteReason = stalledCheckResult.ShouldRemove ? stalledCheckResult.DeleteReason : DeleteReason.ImportFailed;
bool shouldRemoveFromArr = await arrClient.ShouldRemoveFromQueue(instanceType, record, downloadCheckResult.IsPrivate);
DeleteReason deleteReason = downloadCheckResult.ShouldRemove ? downloadCheckResult.DeleteReason : DeleteReason.ImportFailed;
if (!shouldRemoveFromArr && !stalledCheckResult.ShouldRemove)
if (!shouldRemoveFromArr && !downloadCheckResult.ShouldRemove)
{
_logger.LogInformation("skip | {title}", record.Title);
continue;
@@ -119,20 +121,26 @@ public sealed class QueueCleaner : GenericHandler
bool removeFromClient = true;
if (stalledCheckResult.IsPrivate)
if (downloadCheckResult.IsPrivate)
{
if (stalledCheckResult.ShouldRemove && !_config.StalledDeletePrivate)
{
removeFromClient = false;
}
bool isStalledWithoutPruneFlag =
downloadCheckResult.DeleteReason is DeleteReason.Stalled &&
!_config.StalledDeletePrivate;
bool isSlowWithoutPruneFlag =
downloadCheckResult.DeleteReason is DeleteReason.SlowSpeed or DeleteReason.SlowTime &&
!_config.SlowDeletePrivate;
bool shouldKeepDueToDeleteRules = downloadCheckResult.ShouldRemove && (isStalledWithoutPruneFlag || isSlowWithoutPruneFlag);
bool shouldKeepDueToImportRules = shouldRemoveFromArr && !_config.ImportFailedDeletePrivate;
if (shouldRemoveFromArr && !_config.ImportFailedDeletePrivate)
if (shouldKeepDueToDeleteRules || shouldKeepDueToImportRules)
{
removeFromClient = false;
}
}
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient);
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient, deleteReason);
await _notifier.NotifyQueueItemDeleted(removeFromClient, deleteReason);
}
});

View File

@@ -178,7 +178,7 @@ services:
- TZ=Europe/Bucharest
- DRY_RUN=false
- LOGGING__LOGLEVEL=Debug
- LOGGING__LOGLEVEL=Verbose
- LOGGING__FILE__ENABLED=true
- LOGGING__FILE__PATH=/var/logs
- LOGGING__ENHANCED=true
@@ -193,14 +193,25 @@ services:
- QUEUECLEANER__ENABLED=true
- QUEUECLEANER__IGNORED_DOWNLOADS_PATH=/ignored
- QUEUECLEANER__RUNSEQUENTIALLY=true
- QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES=5
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE=true
- QUEUECLEANER__IMPORT_FAILED_DELETE_PRIVATE=false
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__0=file is a sample
- QUEUECLEANER__STALLED_MAX_STRIKES=5
- QUEUECLEANER__STALLED_RESET_STRIKES_ON_PROGRESS=true
- QUEUECLEANER__STALLED_IGNORE_PRIVATE=true
- QUEUECLEANER__STALLED_DELETE_PRIVATE=false
- QUEUECLEANER__SLOW_MAX_STRIKES=5
- QUEUECLEANER__SLOW_RESET_STRIKES_ON_PROGRESS=true
- QUEUECLEANER__SLOW_IGNORE_PRIVATE=false
- QUEUECLEANER__SLOW_DELETE_PRIVATE=false
- QUEUECLEANER__SLOW_MIN_SPEED=1MB
- QUEUECLEANER__SLOW_MAX_TIME=20
- QUEUECLEANER__SLOW_IGNORE_ABOVE_SIZE=1KB
- CONTENTBLOCKER__ENABLED=true
- CONTENTBLOCKER__IGNORED_DOWNLOADS_PATH=/ignored
- CONTENTBLOCKER__IGNORE_PRIVATE=true
@@ -253,10 +264,19 @@ services:
# - NOTIFIARR__ON_IMPORT_FAILED_STRIKE=true
# - NOTIFIARR__ON_STALLED_STRIKE=true
# - NOTIFIARR__ON_SLOW_STRIKE=true
# - NOTIFIARR__ON_QUEUE_ITEM_DELETED=true
# - NOTIFIARR__ON_DOWNLOAD_CLEANED=true
# - NOTIFIARR__API_KEY=notifiarr_secret
# - NOTIFIARR__CHANNEL_ID=discord_channel_id
# - APPRISE__ON_IMPORT_FAILED_STRIKE=true
# - APPRISE__ON_STALLED_STRIKE=true
# - APPRISE__ON_SLOW_STRIKE=true
# - APPRISE__ON_QUEUE_ITEM_DELETED=true
# - APPRISE__ON_DOWNLOAD_CLEANED=true
# - APPRISE__URL=http://localhost:8000
# - APPRISE__KEY=mykey
volumes:
- ./data/cleanuperr/logs:/var/logs
- ./data/cleanuperr/ignored_downloads:/ignored

View File

@@ -6,6 +6,8 @@
- [Download Client settings](#download-client-settings)
- [Arr settings](#arr-settings)
- [Notification settings](#notification-settings)
- [Notifiarr settings](#notifiarr__api_key)
- [Apprise settings](#apprise__url)
- [Advanced settings](#advanced-settings)
#
@@ -160,7 +162,7 @@
> If not set to `0`, the minimum value is `3`.
#### **`QUEUECLEANER__STALLED_RESET_STRIKES_ON_PROGRESS`**
- Controls whether to remove strikes if any download progress was made since last checked.
- Controls whether to remove the given strikes if any download progress was made since last checked.
- Type: Boolean
- Possible values: `true`, `false`
- Default: `false`
@@ -174,7 +176,7 @@
- Required: No.
#### **`QUEUECLEANER__STALLED_DELETE_PRIVATE`**
- Controls whether to delete stalled private downloads from the download client.
- Controls whether stalled downloads from private trackers should be removed from the download client.
- Has no effect if `QUEUECLEANER__STALLED_IGNORE_PRIVATE` is `true`.
- Type: Boolean
- Possible values: `true`, `false`
@@ -184,6 +186,65 @@
> [!WARNING]
> Setting `QUEUECLEANER__STALLED_DELETE_PRIVATE=true` means you don't care about seeding, ratio, H&R and potentially losing your private tracker account.
#### **`QUEUECLEANER__SLOW_MAX_STRIKES`**
- Number of strikes before removing a slow download.
- Set to `0` to never remove slow downloads.
- A strike is given when an item is slow.
- Type: Integer
- Default: `0`
- Required: No.
> [!NOTE]
> If not set to `0`, the minimum value is `3`.
#### **`QUEUECLEANER__SLOW_RESET_STRIKES_ON_PROGRESS`**
- Controls whether to remove the given strikes if the download speed or estimated time are not slow anymore.
- Type: Boolean
- Possible values: `true`, `false`
- Default: `false`
- Required: No.
#### **`QUEUECLEANER__SLOW_IGNORE_PRIVATE`**
- Controls whether to ignore slow downloads from private trackers.
- Type: Boolean
- Possible values: `true`, `false`
- Default: `false`
- Required: No.
#### **`QUEUECLEANER__SLOW_DELETE_PRIVATE`**
- Controls whether slow downloads from private trackers should be removed from the download client.
- Has no effect if `QUEUECLEANER__SLOW_IGNORE_PRIVATE` is `true`.
- Type: Boolean
- Possible values: `true`, `false`
- Default: `false`
- Required: No.
> [!WARNING]
> Setting `QUEUECLEANER__SLOW_DELETE_PRIVATE=true` means you don't care about seeding, ratio, H&R and potentially losing your private tracker account.
#### **`QUEUECLEANER__SLOW_MIN_SPEED`**
- The minimum speed a download should have.
- Downloads receive strikes if their speed falls bellow this value.
- If not specified, downloads will not receive strikes for slow download speed.
- Type: String.
- Default: Empty.
- Required: No.
- Value examples: `1.5KB`, `400KB`, `2MB`
#### **`QUEUECLEANER__SLOW_MAX_TIME`**
- The maximum estimated hours a download should take to finish.
- Downloads receive strikes if their estimated finish time is above this value.
- If not specified (or `0`), downloads will not receive strikes for slow estimated finish time.
- Type: Integer.
- Default: `0`.
- Required: No.
#### **`QUEUECLEANER__SLOW_IGNORE_ABOVE_SIZE`**
- Downloads above this size will not be removed for being slow.
- Type: String.
- Default: Empty.
- Required: No.
- Value examples: `10KB`, `200MB`, `3GB`.
#
### Content Blocker settings
@@ -378,6 +439,12 @@
- Default: `http://localhost:8080`.
- Required: No.
#### **`QBITTORRENT__URL_BASE`**
- Adds a prefix to the qBittorrent url, such as `[QBITTORRENT__URL]/[QBITTORRENT__URL_BASE]/api`.
- Type: String.
- Default: Empty.
- Required: No.
#### **`QBITTORRENT__USERNAME`**
- Username for qBittorrent authentication.
- Type: String.
@@ -396,6 +463,12 @@
- Default: `http://localhost:8112`.
- Required: No.
#### **`DELUGE__URL_BASE`**
- Adds a prefix to the deluge json url, such as `[DELUGE__URL]/[DELUGE__URL_BASE]/json`.
- Type: String.
- Default: Empty.
- Required: No.
#### **`DELUGE__PASSWORD`**
- Password for Deluge authentication.
- Type: String.
@@ -408,6 +481,12 @@
- Default: `http://localhost:9091`.
- Required: No.
#### **`TRANSMISSION__URL_BASE`**
- Adds a prefix to the Transmission rpc url, such as `[TRANSMISSION__URL]/[TRANSMISSION__URL_BASE]/rpc`.
- Type: String.
- Default: `transmission`.
- Required: No.
#### **`TRANSMISSION__USERNAME`**
- Username for Transmission authentication.
- Type: String.
@@ -598,7 +677,14 @@
- Required: No.
#### **`NOTIFIARR__ON_STALLED_STRIKE`**
- Controls whether to notify when an item receives a stalled download strike.
- Controls whether to notify when an item receives a stalled download strike. This includes strikes for being stuck while downloading metadata.
- Type: Boolean
- Possible values: `true`, `false`
- Default: `false`
- Required: No.
#### **`NOTIFIARR__ON_SLOW_STRIKE`**
- Controls whether to notify when an item receives a slow download strike. This includes strikes for having a low download speed or slow estimated finish time.
- Type: Boolean
- Possible values: `true`, `false`
- Default: `false`
@@ -618,6 +704,53 @@
- Default: `false`
- Required: No.
#### **`APPRISE__URL`**
- [Apprise url](https://github.com/caronc/apprise-api) where to send notifications.
- Type: String
- Default: Empty.
- Required: No.
#### **`APPRISE__KEY`**
- [Apprise configuration key](https://github.com/caronc/apprise-api?tab=readme-ov-file#screenshots) containing all 3rd party notification providers which Cleanuperr would notify.
- Type: String
- Default: Empty.
- Required: No.
#### **`APPRISE__ON_IMPORT_FAILED_STRIKE`**
- Controls whether to notify when an item receives a failed import strike.
- Type: Boolean
- Possible values: `true`, `false`
- Default: `false`
- Required: No.
#### **`APPRISE__ON_STALLED_STRIKE`**
- Controls whether to notify when an item receives a stalled download strike. This includes strikes for being stuck while downloading metadata.
- Type: Boolean
- Possible values: `true`, `false`
- Default: `false`
- Required: No.
#### **`APPRISE__ON_SLOW_STRIKE`**
- Controls whether to notify when an item receives a slow download strike. This includes strikes for having a low download speed or slow estimated finish time.
- Type: Boolean
- Possible values: `true`, `false`
- Default: `false`
- Required: No.
#### **`APPRISE__ON_QUEUE_ITEM_DELETED`**
- Controls whether to notify when a queue item is deleted.
- Type: Boolean
- Possible values: `true`, `false`
- Default: `false`
- Required: No.
#### **`APPRISE__ON_DOWNLOAD_CLEANED`**
- Controls whether to notify when a download is cleaned.
- Type: Boolean
- Possible values: `true`, `false`
- Default: `false`
- Required: No.
#
### Advanced settings