mirror of
https://github.com/Cleanuparr/Cleanuparr.git
synced 2025-12-23 22:18:39 -05:00
Add download cleaner and dry run (#58)
This commit is contained in:
Marius Nechifor
168
README.md
168
README.md
@@ -61,7 +61,7 @@ This tool is actively developed and still a work in progress, so using the `late
|
||||
2. **Queue cleaner** will:
|
||||
- Run every 5 minutes (or configured cron, or right after `content blocker`).
|
||||
- Process all items in the *arr queue.
|
||||
- Check each queue item if it is **stalled (download speed is 0)**, **stuck in matadata downloading** or **failed to be imported**.
|
||||
- Check each queue item if it is **stalled (download speed is 0)**, **stuck in metadata downloading** or **failed to be imported**.
|
||||
- If it is, the item receives a **strike** and will continue to accumulate strikes every time it meets any of these conditions.
|
||||
- Check each queue item if it meets one of the following condition in the download client:
|
||||
- **Marked as completed, but 0 bytes have been downloaded** (due to files being blocked by qBittorrent or the **content blocker**).
|
||||
@@ -71,6 +71,9 @@ This tool is actively developed and still a work in progress, so using the `late
|
||||
- It will be removed from the *arr's queue and blocked.
|
||||
- It will be deleted from the download client.
|
||||
- A new search will be triggered for the *arr item.
|
||||
3. **Download cleaner** will:
|
||||
- Run every hour (or configured cron).
|
||||
- Automatically clean up downloads that have been seeding for a certain amount of time.
|
||||
|
||||
# Setup
|
||||
|
||||
@@ -114,6 +117,8 @@ services:
|
||||
volumes:
|
||||
- ./cleanuperr/logs:/var/logs
|
||||
environment:
|
||||
- DRY_RUN=false
|
||||
|
||||
- LOGGING__LOGLEVEL=Information
|
||||
- LOGGING__FILE__ENABLED=false
|
||||
- LOGGING__FILE__PATH=/var/logs/
|
||||
@@ -121,6 +126,7 @@ services:
|
||||
|
||||
- TRIGGERS__QUEUECLEANER=0 0/5 * * * ?
|
||||
- TRIGGERS__CONTENTBLOCKER=0 0/5 * * * ?
|
||||
- TRIGGERS__DOWNLOADCLEANER=0 0 * * * ?
|
||||
|
||||
- QUEUECLEANER__ENABLED=true
|
||||
- QUEUECLEANER__RUNSEQUENTIALLY=true
|
||||
@@ -138,6 +144,17 @@ services:
|
||||
- CONTENTBLOCKER__IGNORE_PRIVATE=false
|
||||
- CONTENTBLOCKER__DELETE_PRIVATE=false
|
||||
|
||||
- DOWNLOADCLEANER__ENABLED=true
|
||||
- DOWNLOADCLEANER__DELETE_PRIVATE=false
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__NAME=tv-sonarr
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MAX_RATIO=-1
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MIN_SEED_TIME=0
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MAX_SEED_TIME=240
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__NAME=radarr
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MAX_RATIO=-1
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MIN_SEED_TIME=0
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MAX_SEED_TIME=240
|
||||
|
||||
- DOWNLOAD_CLIENT=none
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=qBittorrent
|
||||
@@ -179,139 +196,25 @@ services:
|
||||
- LIDARR__INSTANCES__1__URL=http://radarr:8687
|
||||
- LIDARR__INSTANCES__1__APIKEY=secret6
|
||||
|
||||
# - NOTIFIARR__ON_IMPORT_FAILED_STRIKE=false
|
||||
# - NOTIFIARR__ON_STALLED_STRIKE=false
|
||||
# - NOTIFIARR__ON_QUEUE_ITEM_DELETE=false
|
||||
# - NOTIFIARR__API_KEY=notifiarr_secret
|
||||
# - NOTIFIARR__CHANNEL_ID=discord_channel_id
|
||||
- NOTIFIARR__ON_IMPORT_FAILED_STRIKE=true
|
||||
- NOTIFIARR__ON_STALLED_STRIKE=true
|
||||
- NOTIFIARR__ON_QUEUE_ITEM_DELETED=true
|
||||
- NOTIFIARR__ON_DOWNLOAD_CLEANED=true
|
||||
- NOTIFIARR__API_KEY=notifiarr_secret
|
||||
- NOTIFIARR__CHANNEL_ID=discord_channel_id
|
||||
```
|
||||
|
||||
## Environment variables
|
||||
|
||||
### General variables
|
||||
<details>
|
||||
<summary>Click here</summary>
|
||||
|
||||
| Variable | Required | Description | Default value |
|
||||
|---|---|---|---|
|
||||
| LOGGING__LOGLEVEL | No | Can be `Verbose`, `Debug`, `Information`, `Warning`, `Error` or `Fatal`. | `Information` |
|
||||
| LOGGING__FILE__ENABLED | No | Enable or disable logging to file. | false |
|
||||
| LOGGING__FILE__PATH | No | Directory where to save the log files. | empty |
|
||||
| LOGGING__ENHANCED | No | Enhance logs whenever possible.<br>A more detailed description is provided. [here](variables.md#LOGGING__ENHANCED) | true |
|
||||
|||||
|
||||
| TRIGGERS__QUEUECLEANER | Yes if queue cleaner is enabled | - [Quartz cron trigger](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html).<br>- Can be a max of 6h interval.<br>- **Is ignored if `QUEUECLEANER__RUNSEQUENTIALLY=true` and `CONTENTBLOCKER__ENABLED=true`**. | 0 0/5 * * * ? |
|
||||
| TRIGGERS__CONTENTBLOCKER | Yes if content blocker is enabled | - [Quartz cron trigger](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html).<br>- Can be a max of 6h interval. | 0 0/5 * * * ? |
|
||||
|||||
|
||||
| QUEUECLEANER__ENABLED | No | Enable or disable the queue cleaner. | true |
|
||||
| QUEUECLEANER__RUNSEQUENTIALLY | No | If set to true, the queue cleaner will run after the content blocker instead of running in parallel, streamlining the cleaning process. | true |
|
||||
| QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES | No | - After how many strikes should a failed import be removed.<br>- 0 means never. | 0 |
|
||||
| QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE | No | Whether to ignore failed imports from private trackers. | false |
|
||||
| QUEUECLEANER__IMPORT_FAILED_DELETE_PRIVATE | No | - Whether to delete failed imports of private downloads from the download client.<br>- Does not have any effect if `QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE` is `true`.<br>- **Set this to `true` if you don't care about seeding, ratio, H&R and potentially losing your tracker account.** | false |
|
||||
| QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__0 | No | - First pattern to look for when an import is failed.<br>- If the specified message pattern is found, the item is skipped. | empty |
|
||||
| QUEUECLEANER__STALLED_MAX_STRIKES | No | - After how many strikes should a stalled download be removed.<br>- 0 means never. | 0 |
|
||||
| QUEUECLEANER__STALLED_RESET_STRIKES_ON_PROGRESS | No | Whether to remove strikes if any download progress was made since last checked. | false |
|
||||
| QUEUECLEANER__STALLED_IGNORE_PRIVATE | No | Whether to ignore stalled downloads from private trackers. | false |
|
||||
| QUEUECLEANER__STALLED_DELETE_PRIVATE | No | - Whether to delete stalled private downloads from the download client.<br>- Does not have any effect if `QUEUECLEANER__STALLED_IGNORE_PRIVATE` is `true`.<br>- **Set this to `true` if you don't care about seeding, ratio, H&R and potentially losing your tracker account.** | false |
|
||||
|||||
|
||||
| CONTENTBLOCKER__ENABLED | No | Enable or disable the content blocker. | false |
|
||||
| CONTENTBLOCKER__IGNORE_PRIVATE | No | Whether to ignore downloads from private trackers. | false |
|
||||
| CONTENTBLOCKER__DELETE_PRIVATE | No | - Whether to delete private downloads that have all files blocked from the download client.<br>- Does not have any effect if `CONTENTBLOCKER__IGNORE_PRIVATE` is `true`.<br>- **Set this to `true` if you don't care about seeding, ratio, H&R and potentially losing your tracker account.** | false |
|
||||
</details>
|
||||
|
||||
### Download client variables
|
||||
<details>
|
||||
<summary>Click here</summary>
|
||||
|
||||
| Variable | Required | Description | Default value |
|
||||
|---|---|---|---|
|
||||
| DOWNLOAD_CLIENT | No | Download client that is used by *arrs<br>Can be `qbittorrent`, `deluge`, `transmission` or `none` | `none` |
|
||||
| QBITTORRENT__URL | No | qBittorrent instance url | http://localhost:8112 |
|
||||
| QBITTORRENT__USERNAME | No | qBittorrent user | empty |
|
||||
| QBITTORRENT__PASSWORD | No | qBittorrent password | empty |
|
||||
|||||
|
||||
| DELUGE__URL | No | Deluge instance url | http://localhost:8080 |
|
||||
| DELUGE__PASSWORD | No | Deluge password | empty |
|
||||
|||||
|
||||
| TRANSMISSION__URL | No | Transmission instance url | http://localhost:9091 |
|
||||
| TRANSMISSION__USERNAME | No | Transmission user | empty |
|
||||
| TRANSMISSION__PASSWORD | No | Transmission password | empty |
|
||||
</details>
|
||||
|
||||
### Arr variables
|
||||
<details>
|
||||
<summary>Click here</summary>
|
||||
|
||||
| Variable | Required | Description | Default value |
|
||||
|---|---|---|---|
|
||||
| SONARR__ENABLED | No | Enable or disable Sonarr cleanup | false |
|
||||
| SONARR__BLOCK__TYPE | No | Block type<br>Can be `blacklist` or `whitelist` | `blacklist` |
|
||||
| SONARR__BLOCK__PATH | No | Path to the blocklist (local file or url)<br>Needs to be json compatible | empty |
|
||||
| SONARR__SEARCHTYPE | No | What to search for after removing a queue item<br>Can be `Episode`, `Season` or `Series` | `Episode` |
|
||||
| SONARR__INSTANCES__0__URL | No | First Sonarr instance url | http://localhost:8989 |
|
||||
| SONARR__INSTANCES__0__APIKEY | No | First Sonarr instance API key | empty |
|
||||
|||||
|
||||
| RADARR__ENABLED | No | Enable or disable Radarr cleanup | false |
|
||||
| RADARR__BLOCK__TYPE | No | Block type<br>Can be `blacklist` or `whitelist` | `blacklist` |
|
||||
| RADARR__BLOCK__PATH | No | Path to the blocklist (local file or url)<br>Needs to be json compatible | empty |
|
||||
| RADARR__INSTANCES__0__URL | No | First Radarr instance url | http://localhost:7878 |
|
||||
| RADARR__INSTANCES__0__APIKEY | No | First Radarr instance API key | empty |
|
||||
|||||
|
||||
| LIDARR__ENABLED | No | Enable or disable LIDARR cleanup | false |
|
||||
| LIDARR__BLOCK__TYPE | No | Block type<br>Can be `blacklist` or `whitelist` | `blacklist` |
|
||||
| LIDARR__BLOCK__PATH | No | Path to the blocklist (local file or url)<br>Needs to be json compatible | empty |
|
||||
| LIDARR__INSTANCES__0__URL | No | First LIDARR instance url | http://localhost:8686 |
|
||||
| LIDARR__INSTANCES__0__APIKEY | No | First LIDARR instance API key | empty |
|
||||
</details>
|
||||
|
||||
### Notifications variables
|
||||
<details>
|
||||
<summary>Click here</summary>
|
||||
|
||||
| Variable | Required | Description | Default value |
|
||||
|---|---|---|---|
|
||||
| NOTIFIARR__ON_IMPORT_FAILED_STRIKE | No | Notify on failed import strike. | false |
|
||||
| NOTIFIARR__ON_STALLED_STRIKE | No | Notify on stalled download strike. | false |
|
||||
| NOTIFIARR__ON_QUEUE_ITEM_DELETE | No | Notify on deleting a queue item. | false |
|
||||
| NOTIFIARR__API_KEY | No | Notifiarr API key.<br>Requires Notifiarr's `Passthrough` integration to work. | empty |
|
||||
| NOTIFIARR__CHANNEL_ID | No | Discord channel id for notifications. | empty |
|
||||
</details>
|
||||
|
||||
|
||||
### Advanced variables
|
||||
<details>
|
||||
<summary>Click here</summary>
|
||||
|
||||
| Variable | Required | Description | Default value |
|
||||
|---|---|---|---|
|
||||
| HTTP_MAX_RETRIES | No | The number of times to retry a failed HTTP call (to *arrs, download clients etc.) | 0 |
|
||||
| HTTP_TIMEOUT | No | The number of seconds to wait before failing an HTTP call (to *arrs, download clients etc.) | 100 |
|
||||
</details>
|
||||
|
||||
#
|
||||
|
||||
> [!NOTE]
|
||||
> 1. The queue cleaner and content blocker can be enabled or disabled separately, if you want to run only one of them.
|
||||
> 2. Only one download client can be enabled at a time. If you have more than one download client, you should deploy multiple instances of cleanuperr.
|
||||
> 3. The blocklists (blacklist/whitelist) should have a single pattern on each line and supports the following:
|
||||
> ```
|
||||
> *example // file name ends with "example"
|
||||
> example* // file name starts with "example"
|
||||
> *example* // file name has "example" in the name
|
||||
> example // file name is exactly the word "example"
|
||||
> regex:<ANY_REGEX> // regex that needs to be marked at the start of the line with "regex:"
|
||||
> ```
|
||||
> 4. Multiple Sonarr/Radarr/Lidarr instances can be specified using this format, where `<NUMBER>` starts from `0`:
|
||||
> ```
|
||||
> SONARR__INSTANCES__<NUMBER>__URL
|
||||
> SONARR__INSTANCES__<NUMBER>__APIKEY
|
||||
> ```
|
||||
> 5. Multiple failed import patterns can be specified using this format, where `<NUMBER>` starts from 0:
|
||||
> ```
|
||||
> QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__<NUMBER>
|
||||
> ```
|
||||
> 6. [This blacklist](https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/blacklist) and [this whitelist](https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/whitelist) can be used for Sonarr and Radarr, but they are not suitable for other *arrs.
|
||||
|
||||
#
|
||||
Jump to:
|
||||
- [General settings](variables.md#general-settings)
|
||||
- [Queue Cleaner settings](variables.md#queue-cleaner-settings)
|
||||
- [Content Blocker settings](variables.md#content-blocker-settings)
|
||||
- [Download Cleaner settings](variables.md#download-cleaner-settings)
|
||||
- [Download Client settings](variables.md#download-client-settings)
|
||||
- [Arr settings](variables.md#arr-settings)
|
||||
- [Notification settings](variables.md#notification-settings)
|
||||
- [Advanced settings](variables.md#advanced-settings)
|
||||
|
||||
### Binaries (if you're not using Docker)
|
||||
|
||||
@@ -328,9 +231,10 @@ Special thanks for inspiration go to:
|
||||
- [ThijmenGThN/swaparr](https://github.com/ThijmenGThN/swaparr)
|
||||
- [ManiMatter/decluttarr](https://github.com/ManiMatter/decluttarr)
|
||||
- [PaeyMoopy/sonarr-radarr-queue-cleaner](https://github.com/PaeyMoopy/sonarr-radarr-queue-cleaner)
|
||||
- [Sonarr](https://github.com/Sonarr/Sonarr) & [Radarr](https://github.com/Radarr/Radarr) for the logo
|
||||
- [Sonarr](https://github.com/Sonarr/Sonarr) & [Radarr](https://github.com/Radarr/Radarr)
|
||||
|
||||
# Buy me a coffee
|
||||
If I made your life just a tiny bit easier, consider buying me a coffee!
|
||||
|
||||
<a href="https://buymeacoffee.com/flaminel" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Buy Me A Coffee" style="height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a>
|
||||
<a href="https://buymeacoffee.com/flaminel" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Buy Me A Coffee" style="height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a>
|
||||
|
||||
|
||||
6
code/Common/Attributes/DryRunSafeguardAttribute.cs
Normal file
6
code/Common/Attributes/DryRunSafeguardAttribute.cs
Normal file
@@ -0,0 +1,6 @@
|
||||
namespace Common.Attributes;
|
||||
|
||||
[AttributeUsage(AttributeTargets.Method, Inherited = true)]
|
||||
public class DryRunSafeguardAttribute : Attribute
|
||||
{
|
||||
}
|
||||
@@ -7,7 +7,7 @@
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.Extensions.Configuration.Abstractions" Version="9.0.0" />
|
||||
<PackageReference Include="Microsoft.Extensions.Configuration.Abstractions" Version="9.0.2" />
|
||||
<PackageReference Include="Serilog" Version="4.2.0" />
|
||||
</ItemGroup>
|
||||
|
||||
|
||||
@@ -11,9 +11,9 @@ public abstract record ArrConfig
|
||||
public required List<ArrInstance> Instances { get; init; }
|
||||
}
|
||||
|
||||
public record Block
|
||||
public readonly record struct Block
|
||||
{
|
||||
public BlocklistType Type { get; set; }
|
||||
public BlocklistType Type { get; init; }
|
||||
|
||||
public string? Path { get; set; }
|
||||
public string? Path { get; init; }
|
||||
}
|
||||
45
code/Common/Configuration/DownloadCleaner/Category.cs
Normal file
45
code/Common/Configuration/DownloadCleaner/Category.cs
Normal file
@@ -0,0 +1,45 @@
|
||||
using Common.Exceptions;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.DownloadCleaner;
|
||||
|
||||
public sealed record Category : IConfig
|
||||
{
|
||||
public required string Name { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Max ratio before removing a download.
|
||||
/// </summary>
|
||||
[ConfigurationKeyName("MAX_RATIO")]
|
||||
public required double MaxRatio { get; init; } = -1;
|
||||
|
||||
/// <summary>
|
||||
/// Min number of hours to seed before removing a download, if the ratio has been met.
|
||||
/// </summary>
|
||||
[ConfigurationKeyName("MIN_SEED_TIME")]
|
||||
public required double MinSeedTime { get; init; } = 0;
|
||||
|
||||
/// <summary>
|
||||
/// Number of hours to seed before removing a download.
|
||||
/// </summary>
|
||||
[ConfigurationKeyName("MAX_SEED_TIME")]
|
||||
public required double MaxSeedTime { get; init; } = -1;
|
||||
|
||||
public void Validate()
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(Name))
|
||||
{
|
||||
throw new ValidationException($"{nameof(Name)} can not be empty");
|
||||
}
|
||||
|
||||
if (MaxRatio < 0 && MaxSeedTime < 0)
|
||||
{
|
||||
throw new ValidationException($"both {nameof(MaxRatio)} and {nameof(MaxSeedTime)} are disabled");
|
||||
}
|
||||
|
||||
if (MinSeedTime < 0)
|
||||
{
|
||||
throw new ValidationException($"{nameof(MinSeedTime)} can not be negative");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
using Common.Exceptions;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.DownloadCleaner;
|
||||
|
||||
public sealed record DownloadCleanerConfig : IJobConfig
|
||||
{
|
||||
public const string SectionName = "DownloadCleaner";
|
||||
|
||||
public bool Enabled { get; init; }
|
||||
|
||||
public List<Category>? Categories { get; init; }
|
||||
|
||||
[ConfigurationKeyName("DELETE_PRIVATE")]
|
||||
public bool DeletePrivate { get; set; }
|
||||
|
||||
public void Validate()
|
||||
{
|
||||
if (!Enabled)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (Categories?.Count is null or 0)
|
||||
{
|
||||
throw new ValidationException("no categories configured");
|
||||
}
|
||||
|
||||
if (Categories?.GroupBy(x => x.Name).Any(x => x.Count() > 1) is true)
|
||||
{
|
||||
throw new ValidationException("duplicated categories found");
|
||||
}
|
||||
|
||||
Categories?.ForEach(x => x.Validate());
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,6 @@
|
||||
namespace Common.Configuration.DownloadClient;
|
||||
using Common.Exceptions;
|
||||
|
||||
namespace Common.Configuration.DownloadClient;
|
||||
|
||||
public sealed record DelugeConfig : IConfig
|
||||
{
|
||||
@@ -12,7 +14,7 @@ public sealed record DelugeConfig : IConfig
|
||||
{
|
||||
if (Url is null)
|
||||
{
|
||||
throw new ArgumentNullException(nameof(Url));
|
||||
throw new ValidationException($"{nameof(Url)} is empty");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,6 @@
|
||||
namespace Common.Configuration.DownloadClient;
|
||||
using Common.Exceptions;
|
||||
|
||||
namespace Common.Configuration.DownloadClient;
|
||||
|
||||
public sealed class QBitConfig : IConfig
|
||||
{
|
||||
@@ -14,7 +16,7 @@ public sealed class QBitConfig : IConfig
|
||||
{
|
||||
if (Url is null)
|
||||
{
|
||||
throw new ArgumentNullException(nameof(Url));
|
||||
throw new ValidationException($"{nameof(Url)} is empty");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,6 @@
|
||||
namespace Common.Configuration.DownloadClient;
|
||||
using Common.Exceptions;
|
||||
|
||||
namespace Common.Configuration.DownloadClient;
|
||||
|
||||
public record TransmissionConfig : IConfig
|
||||
{
|
||||
@@ -14,7 +16,7 @@ public record TransmissionConfig : IConfig
|
||||
{
|
||||
if (Url is null)
|
||||
{
|
||||
throw new ArgumentNullException(nameof(Url));
|
||||
throw new ValidationException($"{nameof(Url)} is empty");
|
||||
}
|
||||
}
|
||||
}
|
||||
9
code/Common/Configuration/General/DryRunConfig.cs
Normal file
9
code/Common/Configuration/General/DryRunConfig.cs
Normal file
@@ -0,0 +1,9 @@
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.General;
|
||||
|
||||
public sealed record DryRunConfig
|
||||
{
|
||||
[ConfigurationKeyName("DRY_RUN")]
|
||||
public bool IsDryRun { get; init; }
|
||||
}
|
||||
@@ -1,8 +1,9 @@
|
||||
using Microsoft.Extensions.Configuration;
|
||||
using Common.Exceptions;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.General;
|
||||
|
||||
public class HttpConfig : IConfig
|
||||
public sealed record HttpConfig : IConfig
|
||||
{
|
||||
[ConfigurationKeyName("HTTP_MAX_RETRIES")]
|
||||
public ushort MaxRetries { get; init; }
|
||||
@@ -14,7 +15,7 @@ public class HttpConfig : IConfig
|
||||
{
|
||||
if (Timeout is 0)
|
||||
{
|
||||
throw new ArgumentException("HTTP_TIMEOUT must be greater than 0");
|
||||
throw new ValidationException("HTTP_TIMEOUT must be greater than 0");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -7,4 +7,6 @@ public sealed class TriggersConfig
|
||||
public required string QueueCleaner { get; init; }
|
||||
|
||||
public required string ContentBlocker { get; init; }
|
||||
|
||||
public required string DownloadCleaner { get; init; }
|
||||
}
|
||||
@@ -10,10 +10,13 @@ public abstract record NotificationConfig
|
||||
[ConfigurationKeyName("ON_STALLED_STRIKE")]
|
||||
public bool OnStalledStrike { get; init; }
|
||||
|
||||
[ConfigurationKeyName("ON_QUEUE_ITEM_DELETE")]
|
||||
public bool OnQueueItemDelete { get; init; }
|
||||
[ConfigurationKeyName("ON_QUEUE_ITEM_DELETED")]
|
||||
public bool OnQueueItemDeleted { get; init; }
|
||||
|
||||
[ConfigurationKeyName("ON_DOWNLOAD_CLEANED")]
|
||||
public bool OnDownloadCleaned { get; init; }
|
||||
|
||||
public bool IsEnabled => OnImportFailedStrike || OnStalledStrike || OnQueueItemDelete;
|
||||
public bool IsEnabled => OnImportFailedStrike || OnStalledStrike || OnQueueItemDeleted || OnDownloadCleaned;
|
||||
|
||||
public abstract bool IsValid();
|
||||
}
|
||||
12
code/Common/Exceptions/ValidationException.cs
Normal file
12
code/Common/Exceptions/ValidationException.cs
Normal file
@@ -0,0 +1,12 @@
|
||||
namespace Common.Exceptions;
|
||||
|
||||
public sealed class ValidationException : Exception
|
||||
{
|
||||
public ValidationException()
|
||||
{
|
||||
}
|
||||
|
||||
public ValidationException(string message) : base(message)
|
||||
{
|
||||
}
|
||||
}
|
||||
8
code/Domain/Enums/CleanReason.cs
Normal file
8
code/Domain/Enums/CleanReason.cs
Normal file
@@ -0,0 +1,8 @@
|
||||
namespace Domain.Enums;
|
||||
|
||||
public enum CleanReason
|
||||
{
|
||||
None,
|
||||
MaxRatioReached,
|
||||
MaxSeedTimeReached,
|
||||
}
|
||||
@@ -16,4 +16,11 @@ public sealed record TorrentStatus
|
||||
|
||||
[JsonProperty("total_done")]
|
||||
public long TotalDone { get; init; }
|
||||
|
||||
public string? Label { get; init; }
|
||||
|
||||
[JsonProperty("seeding_time")]
|
||||
public long SeedingTime { get; init; }
|
||||
|
||||
public float Ratio { get; init; }
|
||||
}
|
||||
@@ -1,6 +1,8 @@
|
||||
using Common.Configuration.Arr;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.DownloadClient;
|
||||
using Common.Configuration.General;
|
||||
using Common.Configuration.Logging;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
|
||||
@@ -10,8 +12,10 @@ public static class ConfigurationDI
|
||||
{
|
||||
public static IServiceCollection AddConfiguration(this IServiceCollection services, IConfiguration configuration) =>
|
||||
services
|
||||
.Configure<DryRunConfig>(configuration)
|
||||
.Configure<QueueCleanerConfig>(configuration.GetSection(QueueCleanerConfig.SectionName))
|
||||
.Configure<ContentBlockerConfig>(configuration.GetSection(ContentBlockerConfig.SectionName))
|
||||
.Configure<DownloadCleanerConfig>(configuration.GetSection(DownloadCleanerConfig.SectionName))
|
||||
.Configure<DownloadClientConfig>(configuration)
|
||||
.Configure<QBitConfig>(configuration.GetSection(QBitConfig.SectionName))
|
||||
.Configure<DelugeConfig>(configuration.GetSection(DelugeConfig.SectionName))
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
using Common.Configuration.Logging;
|
||||
using Domain.Enums;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.DownloadCleaner;
|
||||
using Infrastructure.Verticals.QueueCleaner;
|
||||
using Serilog;
|
||||
using Serilog.Events;
|
||||
@@ -33,7 +34,7 @@ public static class LoggingDI
|
||||
const string consoleOutputTemplate = $"[{{@t:yyyy-MM-dd HH:mm:ss.fff}} {{@l:u3}}]{jobNameTemplate}{instanceNameTemplate} {{@m}}\n{{@x}}";
|
||||
const string fileOutputTemplate = $"{{@t:yyyy-MM-dd HH:mm:ss.fff zzz}} [{{@l:u3}}]{jobNameTemplate}{instanceNameTemplate} {{@m:lj}}\n{{@x}}";
|
||||
LogEventLevel level = LogEventLevel.Information;
|
||||
List<string> names = [nameof(ContentBlocker), nameof(QueueCleaner)];
|
||||
List<string> names = [nameof(ContentBlocker), nameof(QueueCleaner), nameof(DownloadCleaner)];
|
||||
int jobPadding = names.Max(x => x.Length) + 2;
|
||||
names = [InstanceType.Sonarr.ToString(), InstanceType.Radarr.ToString(), InstanceType.Lidarr.ToString()];
|
||||
int arrPadding = names.Max(x => x.Length) + 2;
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
using System.Net;
|
||||
using Castle.DynamicProxy;
|
||||
using Common.Configuration.General;
|
||||
using Common.Helpers;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Verticals.DownloadClient.Deluge;
|
||||
using Infrastructure.Verticals.Notifications.Consumers;
|
||||
using Infrastructure.Verticals.Notifications.Models;
|
||||
@@ -25,7 +27,8 @@ public static class MainDI
|
||||
{
|
||||
config.AddConsumer<NotificationConsumer<FailedImportStrikeNotification>>();
|
||||
config.AddConsumer<NotificationConsumer<StalledStrikeNotification>>();
|
||||
config.AddConsumer<NotificationConsumer<QueueItemDeleteNotification>>();
|
||||
config.AddConsumer<NotificationConsumer<QueueItemDeletedNotification>>();
|
||||
config.AddConsumer<NotificationConsumer<DownloadCleanedNotification>>();
|
||||
|
||||
config.UsingInMemory((context, cfg) =>
|
||||
{
|
||||
@@ -33,12 +36,14 @@ public static class MainDI
|
||||
{
|
||||
e.ConfigureConsumer<NotificationConsumer<FailedImportStrikeNotification>>(context);
|
||||
e.ConfigureConsumer<NotificationConsumer<StalledStrikeNotification>>(context);
|
||||
e.ConfigureConsumer<NotificationConsumer<QueueItemDeleteNotification>>(context);
|
||||
e.ConfigureConsumer<NotificationConsumer<QueueItemDeletedNotification>>(context);
|
||||
e.ConfigureConsumer<NotificationConsumer<DownloadCleanedNotification>>(context);
|
||||
e.ConcurrentMessageLimit = 1;
|
||||
e.PrefetchCount = 1;
|
||||
});
|
||||
});
|
||||
});
|
||||
})
|
||||
.AddDryRunInterceptor();
|
||||
|
||||
private static IServiceCollection AddHttpClients(this IServiceCollection services, IConfiguration configuration)
|
||||
{
|
||||
@@ -86,4 +91,31 @@ public static class MainDI
|
||||
.OrResult(response => !response.IsSuccessStatusCode && response.StatusCode != HttpStatusCode.Unauthorized)
|
||||
.WaitAndRetryAsync(config.MaxRetries, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)))
|
||||
);
|
||||
|
||||
private static IServiceCollection AddDryRunInterceptor(this IServiceCollection services)
|
||||
{
|
||||
services
|
||||
.Where(s => s.ServiceType != typeof(IDryRunService) && typeof(IDryRunService).IsAssignableFrom(s.ServiceType))
|
||||
.ToList()
|
||||
.ForEach(service =>
|
||||
{
|
||||
services.Decorate(service.ServiceType, (target, svc) =>
|
||||
{
|
||||
ProxyGenerator proxyGenerator = new();
|
||||
DryRunAsyncInterceptor interceptor = svc.GetRequiredService<DryRunAsyncInterceptor>();
|
||||
|
||||
object implementation = proxyGenerator.CreateClassProxyWithTarget(
|
||||
service.ServiceType,
|
||||
target,
|
||||
interceptor
|
||||
);
|
||||
|
||||
((IInterceptedService)target).Proxy = implementation;
|
||||
|
||||
return implementation;
|
||||
});
|
||||
});
|
||||
|
||||
return services;
|
||||
}
|
||||
}
|
||||
@@ -1,10 +1,12 @@
|
||||
using Common.Configuration;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.General;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Common.Helpers;
|
||||
using Executable.Jobs;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.DownloadCleaner;
|
||||
using Infrastructure.Verticals.Jobs;
|
||||
using Infrastructure.Verticals.QueueCleaner;
|
||||
using Quartz;
|
||||
@@ -59,6 +61,12 @@ public static class QuartzDI
|
||||
{
|
||||
q.AddJob<QueueCleaner>(queueCleanerConfig, triggersConfig.QueueCleaner);
|
||||
}
|
||||
|
||||
DownloadCleanerConfig? downloadCleanerConfig = configuration
|
||||
.GetRequiredSection(DownloadCleanerConfig.SectionName)
|
||||
.Get<DownloadCleanerConfig>();
|
||||
|
||||
q.AddJob<DownloadCleaner>(downloadCleanerConfig, triggersConfig.DownloadCleaner);
|
||||
}
|
||||
|
||||
private static void AddJob<T>(
|
||||
@@ -109,7 +117,7 @@ public static class QuartzDI
|
||||
|
||||
if (triggerValue > Constants.TriggerMaxLimit)
|
||||
{
|
||||
throw new Exception($"{trigger} should have a fire time of maximum 1 hour");
|
||||
throw new Exception($"{trigger} should have a fire time of maximum {Constants.TriggerMaxLimit.TotalHours} hours");
|
||||
}
|
||||
|
||||
if (triggerValue > StaticConfiguration.TriggerValue)
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
using Infrastructure.Verticals.Arr;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Verticals.Arr;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.DownloadCleaner;
|
||||
using Infrastructure.Verticals.DownloadClient;
|
||||
using Infrastructure.Verticals.DownloadClient.Deluge;
|
||||
using Infrastructure.Verticals.DownloadClient.QBittorrent;
|
||||
@@ -13,12 +15,14 @@ public static class ServicesDI
|
||||
{
|
||||
public static IServiceCollection AddServices(this IServiceCollection services) =>
|
||||
services
|
||||
.AddTransient<DryRunAsyncInterceptor>()
|
||||
.AddTransient<SonarrClient>()
|
||||
.AddTransient<RadarrClient>()
|
||||
.AddTransient<LidarrClient>()
|
||||
.AddTransient<QueueCleaner>()
|
||||
.AddTransient<ContentBlocker>()
|
||||
.AddTransient<FilenameEvaluator>()
|
||||
.AddTransient<DownloadCleaner>()
|
||||
.AddTransient<IFilenameEvaluator, FilenameEvaluator>()
|
||||
.AddTransient<DummyDownloadService>()
|
||||
.AddTransient<QBitService>()
|
||||
.AddTransient<DelugeService>()
|
||||
@@ -26,5 +30,5 @@ public static class ServicesDI
|
||||
.AddTransient<ArrQueueIterator>()
|
||||
.AddTransient<DownloadServiceFactory>()
|
||||
.AddSingleton<BlocklistProvider>()
|
||||
.AddSingleton<Striker>();
|
||||
.AddSingleton<IStriker, Striker>();
|
||||
}
|
||||
@@ -10,9 +10,9 @@
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.Extensions.Hosting" Version="9.0.0" />
|
||||
<PackageReference Include="Microsoft.Extensions.Http" Version="9.0.0" />
|
||||
<PackageReference Include="Microsoft.Extensions.Http.Polly" Version="9.0.0" />
|
||||
<PackageReference Include="Microsoft.Extensions.Hosting" Version="9.0.2" />
|
||||
<PackageReference Include="Microsoft.Extensions.Http" Version="9.0.2" />
|
||||
<PackageReference Include="Microsoft.Extensions.Http.Polly" Version="9.0.2" />
|
||||
<PackageReference Include="Quartz" Version="3.13.1" />
|
||||
<PackageReference Include="Quartz.Extensions.DependencyInjection" Version="3.13.1" />
|
||||
<PackageReference Include="Quartz.Extensions.Hosting" Version="3.13.1" />
|
||||
|
||||
@@ -6,7 +6,7 @@ namespace Executable.Jobs;
|
||||
|
||||
[DisallowConcurrentExecution]
|
||||
public sealed class GenericJob<T> : IJob
|
||||
where T : GenericHandler
|
||||
where T : IHandler
|
||||
{
|
||||
private readonly ILogger<GenericJob<T>> _logger;
|
||||
private readonly T _handler;
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
{
|
||||
"DRY_RUN": true,
|
||||
"HTTP_MAX_RETRIES": 0,
|
||||
"HTTP_TIMEOUT": 10,
|
||||
"Logging": {
|
||||
@@ -11,7 +12,8 @@
|
||||
},
|
||||
"Triggers": {
|
||||
"QueueCleaner": "0/10 * * * * ?",
|
||||
"ContentBlocker": "0/10 * * * * ?"
|
||||
"ContentBlocker": "0/10 * * * * ?",
|
||||
"DownloadCleaner": "0/10 * * * * ?"
|
||||
},
|
||||
"ContentBlocker": {
|
||||
"Enabled": true,
|
||||
@@ -32,6 +34,18 @@
|
||||
"STALLED_IGNORE_PRIVATE": true,
|
||||
"STALLED_DELETE_PRIVATE": false
|
||||
},
|
||||
"DownloadCleaner": {
|
||||
"Enabled": false,
|
||||
"DELETE_PRIVATE": false,
|
||||
"CATEGORIES": [
|
||||
{
|
||||
"Name": "tv-sonarr",
|
||||
"MAX_RATIO": -1,
|
||||
"MIN_SEED_TIME": 0,
|
||||
"MAX_SEED_TIME": -1
|
||||
}
|
||||
]
|
||||
},
|
||||
"DOWNLOAD_CLIENT": "qbittorrent",
|
||||
"qBittorrent": {
|
||||
"Url": "http://localhost:8080",
|
||||
@@ -90,7 +104,8 @@
|
||||
"Notifiarr": {
|
||||
"ON_IMPORT_FAILED_STRIKE": true,
|
||||
"ON_STALLED_STRIKE": true,
|
||||
"ON_QUEUE_ITEM_DELETE": true,
|
||||
"ON_QUEUE_ITEM_DELETED": true,
|
||||
"ON_DOWNLOAD_CLEANED": true,
|
||||
"API_KEY": "",
|
||||
"CHANNEL_ID": ""
|
||||
}
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
{
|
||||
"DRY_RUN": false,
|
||||
"HTTP_MAX_RETRIES": 0,
|
||||
"HTTP_TIMEOUT": 100,
|
||||
"Logging": {
|
||||
@@ -11,7 +12,8 @@
|
||||
},
|
||||
"Triggers": {
|
||||
"QueueCleaner": "0 0/5 * * * ?",
|
||||
"ContentBlocker": "0 0/5 * * * ?"
|
||||
"ContentBlocker": "0 0/5 * * * ?",
|
||||
"DownloadCleaner": "0 0 * * * ?"
|
||||
},
|
||||
"ContentBlocker": {
|
||||
"Enabled": false,
|
||||
@@ -29,6 +31,11 @@
|
||||
"STALLED_IGNORE_PRIVATE": false,
|
||||
"STALLED_DELETE_PRIVATE": false
|
||||
},
|
||||
"DownloadCleaner": {
|
||||
"Enabled": false,
|
||||
"DELETE_PRIVATE": false,
|
||||
"CATEGORIES": []
|
||||
},
|
||||
"DOWNLOAD_CLIENT": "none",
|
||||
"qBittorrent": {
|
||||
"Url": "http://localhost:8080",
|
||||
@@ -87,7 +94,8 @@
|
||||
"Notifiarr": {
|
||||
"ON_IMPORT_FAILED_STRIKE": false,
|
||||
"ON_STALLED_STRIKE": false,
|
||||
"ON_QUEUE_ITEM_DELETE": false,
|
||||
"ON_QUEUE_ITEM_DELETED": false,
|
||||
"ON_DOWNLOAD_CLEANED": false,
|
||||
"API_KEY": "",
|
||||
"CHANNEL_ID": ""
|
||||
}
|
||||
|
||||
33
code/Infrastructure.Tests/Infrastructure.Tests.csproj
Normal file
33
code/Infrastructure.Tests/Infrastructure.Tests.csproj
Normal file
@@ -0,0 +1,33 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net9.0</TargetFramework>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<IsPackable>false</IsPackable>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="coverlet.collector" Version="6.0.4">
|
||||
<PrivateAssets>all</PrivateAssets>
|
||||
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
|
||||
</PackageReference>
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.13.0" />
|
||||
<PackageReference Include="NSubstitute" Version="5.3.0" />
|
||||
<PackageReference Include="Shouldly" Version="4.3.0" />
|
||||
<PackageReference Include="xunit" Version="2.9.3" />
|
||||
<PackageReference Include="xunit.runner.visualstudio" Version="3.0.2">
|
||||
<PrivateAssets>all</PrivateAssets>
|
||||
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
|
||||
</PackageReference>
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<Using Include="Xunit"/>
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\Infrastructure\Infrastructure.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -0,0 +1,20 @@
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using NSubstitute;
|
||||
|
||||
namespace Infrastructure.Tests.Verticals.ContentBlocker;
|
||||
|
||||
public class FilenameEvaluatorFixture
|
||||
{
|
||||
public ILogger<FilenameEvaluator> Logger { get; }
|
||||
|
||||
public FilenameEvaluatorFixture()
|
||||
{
|
||||
Logger = Substitute.For<ILogger<FilenameEvaluator>>();
|
||||
}
|
||||
|
||||
public FilenameEvaluator CreateSut()
|
||||
{
|
||||
return new FilenameEvaluator(Logger);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,219 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Shouldly;
|
||||
|
||||
namespace Infrastructure.Tests.Verticals.ContentBlocker;
|
||||
|
||||
public class FilenameEvaluatorTests : IClassFixture<FilenameEvaluatorFixture>
|
||||
{
|
||||
private readonly FilenameEvaluatorFixture _fixture;
|
||||
|
||||
public FilenameEvaluatorTests(FilenameEvaluatorFixture fixture)
|
||||
{
|
||||
_fixture = fixture;
|
||||
}
|
||||
|
||||
public class PatternTests : FilenameEvaluatorTests
|
||||
{
|
||||
public PatternTests(FilenameEvaluatorFixture fixture) : base(fixture) { }
|
||||
|
||||
[Fact]
|
||||
public void WhenNoPatterns_ShouldReturnTrue()
|
||||
{
|
||||
// Arrange
|
||||
var sut = _fixture.CreateSut();
|
||||
var patterns = new ConcurrentBag<string>();
|
||||
var regexes = new ConcurrentBag<Regex>();
|
||||
|
||||
// Act
|
||||
var result = sut.IsValid("test.txt", BlocklistType.Blacklist, patterns, regexes);
|
||||
|
||||
// Assert
|
||||
result.ShouldBeTrue();
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("test.txt", "test.txt", true)] // Exact match
|
||||
[InlineData("test.txt", "*.txt", true)] // End wildcard
|
||||
[InlineData("test.txt", "test.*", true)] // Start wildcard
|
||||
[InlineData("test.txt", "*test*", true)] // Both wildcards
|
||||
[InlineData("test.txt", "other.txt", false)] // No match
|
||||
public void Blacklist_ShouldMatchPatterns(string filename, string pattern, bool shouldBeBlocked)
|
||||
{
|
||||
// Arrange
|
||||
var sut = _fixture.CreateSut();
|
||||
var patterns = new ConcurrentBag<string> { pattern };
|
||||
var regexes = new ConcurrentBag<Regex>();
|
||||
|
||||
// Act
|
||||
var result = sut.IsValid(filename, BlocklistType.Blacklist, patterns, regexes);
|
||||
|
||||
// Assert
|
||||
result.ShouldBe(!shouldBeBlocked);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("test.txt", "test.txt", true)] // Exact match
|
||||
[InlineData("test.txt", "*.txt", true)] // End wildcard
|
||||
[InlineData("test.txt", "test.*", true)] // Start wildcard
|
||||
[InlineData("test.txt", "*test*", true)] // Both wildcards
|
||||
[InlineData("test.txt", "other.txt", false)] // No match
|
||||
public void Whitelist_ShouldMatchPatterns(string filename, string pattern, bool shouldBeAllowed)
|
||||
{
|
||||
// Arrange
|
||||
var sut = _fixture.CreateSut();
|
||||
var patterns = new ConcurrentBag<string> { pattern };
|
||||
var regexes = new ConcurrentBag<Regex>();
|
||||
|
||||
// Act
|
||||
var result = sut.IsValid(filename, BlocklistType.Whitelist, patterns, regexes);
|
||||
|
||||
// Assert
|
||||
result.ShouldBe(shouldBeAllowed);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("TEST.TXT", "test.txt")]
|
||||
[InlineData("test.txt", "TEST.TXT")]
|
||||
public void ShouldBeCaseInsensitive(string filename, string pattern)
|
||||
{
|
||||
// Arrange
|
||||
var sut = _fixture.CreateSut();
|
||||
var patterns = new ConcurrentBag<string> { pattern };
|
||||
var regexes = new ConcurrentBag<Regex>();
|
||||
|
||||
// Act
|
||||
var result = sut.IsValid(filename, BlocklistType.Blacklist, patterns, regexes);
|
||||
|
||||
// Assert
|
||||
result.ShouldBeFalse();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void MultiplePatterns_ShouldMatchAny()
|
||||
{
|
||||
// Arrange
|
||||
var sut = _fixture.CreateSut();
|
||||
var patterns = new ConcurrentBag<string>
|
||||
{
|
||||
"other.txt",
|
||||
"*.pdf",
|
||||
"test.*"
|
||||
};
|
||||
var regexes = new ConcurrentBag<Regex>();
|
||||
|
||||
// Act
|
||||
var result = sut.IsValid("test.txt", BlocklistType.Blacklist, patterns, regexes);
|
||||
|
||||
// Assert
|
||||
result.ShouldBeFalse();
|
||||
}
|
||||
}
|
||||
|
||||
public class RegexTests : FilenameEvaluatorTests
|
||||
{
|
||||
public RegexTests(FilenameEvaluatorFixture fixture) : base(fixture) { }
|
||||
|
||||
[Fact]
|
||||
public void WhenNoRegexes_ShouldReturnTrue()
|
||||
{
|
||||
// Arrange
|
||||
var sut = _fixture.CreateSut();
|
||||
var patterns = new ConcurrentBag<string>();
|
||||
var regexes = new ConcurrentBag<Regex>();
|
||||
|
||||
// Act
|
||||
var result = sut.IsValid("test.txt", BlocklistType.Blacklist, patterns, regexes);
|
||||
|
||||
// Assert
|
||||
result.ShouldBeTrue();
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(@"test\d+\.txt", "test123.txt", true)]
|
||||
[InlineData(@"test\d+\.txt", "test.txt", false)]
|
||||
public void Blacklist_ShouldMatchRegexes(string pattern, string filename, bool shouldBeBlocked)
|
||||
{
|
||||
// Arrange
|
||||
var sut = _fixture.CreateSut();
|
||||
var patterns = new ConcurrentBag<string>();
|
||||
var regexes = new ConcurrentBag<Regex> { new Regex(pattern, RegexOptions.IgnoreCase) };
|
||||
|
||||
// Act
|
||||
var result = sut.IsValid(filename, BlocklistType.Blacklist, patterns, regexes);
|
||||
|
||||
// Assert
|
||||
result.ShouldBe(!shouldBeBlocked);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(@"test\d+\.txt", "test123.txt", true)]
|
||||
[InlineData(@"test\d+\.txt", "test.txt", false)]
|
||||
public void Whitelist_ShouldMatchRegexes(string pattern, string filename, bool shouldBeAllowed)
|
||||
{
|
||||
// Arrange
|
||||
var sut = _fixture.CreateSut();
|
||||
var patterns = new ConcurrentBag<string>();
|
||||
var regexes = new ConcurrentBag<Regex> { new Regex(pattern, RegexOptions.IgnoreCase) };
|
||||
|
||||
// Act
|
||||
var result = sut.IsValid(filename, BlocklistType.Whitelist, patterns, regexes);
|
||||
|
||||
// Assert
|
||||
result.ShouldBe(shouldBeAllowed);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(@"TEST\d+\.TXT", "test123.txt")]
|
||||
[InlineData(@"test\d+\.txt", "TEST123.TXT")]
|
||||
public void ShouldBeCaseInsensitive(string pattern, string filename)
|
||||
{
|
||||
// Arrange
|
||||
var sut = _fixture.CreateSut();
|
||||
var patterns = new ConcurrentBag<string>();
|
||||
var regexes = new ConcurrentBag<Regex> { new Regex(pattern, RegexOptions.IgnoreCase) };
|
||||
|
||||
// Act
|
||||
var result = sut.IsValid(filename, BlocklistType.Blacklist, patterns, regexes);
|
||||
|
||||
// Assert
|
||||
result.ShouldBeFalse();
|
||||
}
|
||||
}
|
||||
|
||||
public class CombinedTests : FilenameEvaluatorTests
|
||||
{
|
||||
public CombinedTests(FilenameEvaluatorFixture fixture) : base(fixture) { }
|
||||
|
||||
[Fact]
|
||||
public void WhenBothPatternsAndRegexes_ShouldMatchBoth()
|
||||
{
|
||||
// Arrange
|
||||
var sut = _fixture.CreateSut();
|
||||
var patterns = new ConcurrentBag<string> { "*.txt" };
|
||||
var regexes = new ConcurrentBag<Regex> { new Regex(@"test\d+", RegexOptions.IgnoreCase) };
|
||||
|
||||
// Act
|
||||
var result = sut.IsValid("test123.txt", BlocklistType.Blacklist, patterns, regexes);
|
||||
|
||||
// Assert
|
||||
result.ShouldBeFalse();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WhenPatternMatchesButRegexDoesNot_ShouldReturnFalse()
|
||||
{
|
||||
// Arrange
|
||||
var sut = _fixture.CreateSut();
|
||||
var patterns = new ConcurrentBag<string> { "*.txt" };
|
||||
var regexes = new ConcurrentBag<Regex> { new Regex(@"test\d+", RegexOptions.IgnoreCase) };
|
||||
|
||||
// Act
|
||||
var result = sut.IsValid("other.txt", BlocklistType.Whitelist, patterns, regexes);
|
||||
|
||||
// Assert
|
||||
result.ShouldBeFalse();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.DownloadClient;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using NSubstitute;
|
||||
|
||||
namespace Infrastructure.Tests.Verticals.DownloadClient;
|
||||
|
||||
public class DownloadServiceFixture : IDisposable
|
||||
{
|
||||
public ILogger<DownloadService> Logger { get; set; }
|
||||
public IMemoryCache Cache { get; set; }
|
||||
public IStriker Striker { get; set; }
|
||||
|
||||
public DownloadServiceFixture()
|
||||
{
|
||||
Logger = Substitute.For<ILogger<DownloadService>>();
|
||||
Cache = Substitute.For<IMemoryCache>();
|
||||
Striker = Substitute.For<IStriker>();
|
||||
}
|
||||
|
||||
public TestDownloadService CreateSut(
|
||||
QueueCleanerConfig? queueCleanerConfig = null,
|
||||
ContentBlockerConfig? contentBlockerConfig = null
|
||||
)
|
||||
{
|
||||
queueCleanerConfig ??= new QueueCleanerConfig
|
||||
{
|
||||
Enabled = true,
|
||||
RunSequentially = true,
|
||||
StalledResetStrikesOnProgress = true,
|
||||
StalledMaxStrikes = 3
|
||||
};
|
||||
|
||||
var queueCleanerOptions = Substitute.For<IOptions<QueueCleanerConfig>>();
|
||||
queueCleanerOptions.Value.Returns(queueCleanerConfig);
|
||||
|
||||
contentBlockerConfig ??= new ContentBlockerConfig
|
||||
{
|
||||
Enabled = true
|
||||
};
|
||||
|
||||
var contentBlockerOptions = Substitute.For<IOptions<ContentBlockerConfig>>();
|
||||
contentBlockerOptions.Value.Returns(contentBlockerConfig);
|
||||
|
||||
var downloadCleanerOptions = Substitute.For<IOptions<DownloadCleanerConfig>>();
|
||||
downloadCleanerOptions.Value.Returns(new DownloadCleanerConfig());
|
||||
|
||||
var filenameEvaluator = Substitute.For<IFilenameEvaluator>();
|
||||
var notifier = Substitute.For<NotificationPublisher>();
|
||||
|
||||
return new TestDownloadService(
|
||||
Logger,
|
||||
queueCleanerOptions,
|
||||
contentBlockerOptions,
|
||||
downloadCleanerOptions,
|
||||
Cache,
|
||||
filenameEvaluator,
|
||||
Striker,
|
||||
notifier
|
||||
);
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
// Cleanup if needed
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,235 @@
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Domain.Enums;
|
||||
using Domain.Models.Cache;
|
||||
using Infrastructure.Helpers;
|
||||
using Infrastructure.Verticals.Context;
|
||||
using Infrastructure.Verticals.DownloadClient;
|
||||
using NSubstitute;
|
||||
using NSubstitute.ClearExtensions;
|
||||
using Shouldly;
|
||||
|
||||
namespace Infrastructure.Tests.Verticals.DownloadClient;
|
||||
|
||||
public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||
{
|
||||
private readonly DownloadServiceFixture _fixture;
|
||||
|
||||
public DownloadServiceTests(DownloadServiceFixture fixture)
|
||||
{
|
||||
_fixture = fixture;
|
||||
_fixture.Cache.ClearSubstitute();
|
||||
_fixture.Striker.ClearSubstitute();
|
||||
}
|
||||
|
||||
public class ResetStrikesOnProgressTests : DownloadServiceTests
|
||||
{
|
||||
public ResetStrikesOnProgressTests(DownloadServiceFixture fixture) : base(fixture)
|
||||
{
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WhenStalledStrikeDisabled_ShouldNotResetStrikes()
|
||||
{
|
||||
// Arrange
|
||||
TestDownloadService sut = _fixture.CreateSut(queueCleanerConfig: new()
|
||||
{
|
||||
Enabled = true,
|
||||
RunSequentially = true,
|
||||
StalledResetStrikesOnProgress = false,
|
||||
});
|
||||
|
||||
// Act
|
||||
sut.ResetStrikesOnProgress("test-hash", 100);
|
||||
|
||||
// Assert
|
||||
_fixture.Cache.ReceivedCalls().ShouldBeEmpty();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WhenProgressMade_ShouldResetStrikes()
|
||||
{
|
||||
// Arrange
|
||||
const string hash = "test-hash";
|
||||
CacheItem cacheItem = new CacheItem { Downloaded = 100 };
|
||||
|
||||
_fixture.Cache.TryGetValue(Arg.Any<object>(), out Arg.Any<object?>())
|
||||
.Returns(x =>
|
||||
{
|
||||
x[1] = cacheItem;
|
||||
return true;
|
||||
});
|
||||
|
||||
TestDownloadService sut = _fixture.CreateSut();
|
||||
|
||||
// Act
|
||||
sut.ResetStrikesOnProgress(hash, 200);
|
||||
|
||||
// Assert
|
||||
_fixture.Cache.Received(1).Remove(CacheKeys.Strike(StrikeType.Stalled, hash));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WhenNoProgress_ShouldNotResetStrikes()
|
||||
{
|
||||
// Arrange
|
||||
const string hash = "test-hash";
|
||||
CacheItem cacheItem = new CacheItem { Downloaded = 200 };
|
||||
|
||||
_fixture.Cache
|
||||
.TryGetValue(Arg.Any<object>(), out Arg.Any<object?>())
|
||||
.Returns(x =>
|
||||
{
|
||||
x[1] = cacheItem;
|
||||
return true;
|
||||
});
|
||||
|
||||
TestDownloadService sut = _fixture.CreateSut();
|
||||
|
||||
// Act
|
||||
sut.ResetStrikesOnProgress(hash, 100);
|
||||
|
||||
// Assert
|
||||
_fixture.Cache.DidNotReceive().Remove(Arg.Any<object>());
|
||||
}
|
||||
}
|
||||
|
||||
public class StrikeAndCheckLimitTests : DownloadServiceTests
|
||||
{
|
||||
public StrikeAndCheckLimitTests(DownloadServiceFixture fixture) : base(fixture)
|
||||
{
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ShouldDelegateCallToStriker()
|
||||
{
|
||||
// Arrange
|
||||
const string hash = "test-hash";
|
||||
const string itemName = "test-item";
|
||||
_fixture.Striker.StrikeAndCheckLimit(hash, itemName, 3, StrikeType.Stalled)
|
||||
.Returns(true);
|
||||
|
||||
TestDownloadService sut = _fixture.CreateSut();
|
||||
|
||||
// Act
|
||||
bool result = await sut.StrikeAndCheckLimit(hash, itemName);
|
||||
|
||||
// Assert
|
||||
result.ShouldBeTrue();
|
||||
await _fixture.Striker
|
||||
.Received(1)
|
||||
.StrikeAndCheckLimit(hash, itemName, 3, StrikeType.Stalled);
|
||||
}
|
||||
}
|
||||
|
||||
public class ShouldCleanDownloadTests : DownloadServiceTests
|
||||
{
|
||||
public ShouldCleanDownloadTests(DownloadServiceFixture fixture) : base(fixture)
|
||||
{
|
||||
ContextProvider.Set("downloadName", "test-download");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WhenRatioAndMinSeedTimeReached_ShouldReturnTrue()
|
||||
{
|
||||
// Arrange
|
||||
Category category = new()
|
||||
{
|
||||
Name = "test",
|
||||
MaxRatio = 1.0,
|
||||
MinSeedTime = 1,
|
||||
MaxSeedTime = -1
|
||||
};
|
||||
const double ratio = 1.5;
|
||||
TimeSpan seedingTime = TimeSpan.FromHours(2);
|
||||
|
||||
TestDownloadService sut = _fixture.CreateSut();
|
||||
|
||||
// Act
|
||||
var result = sut.ShouldCleanDownload(ratio, seedingTime, category);
|
||||
|
||||
// Assert
|
||||
result.ShouldSatisfyAllConditions(
|
||||
() => result.ShouldClean.ShouldBeTrue(),
|
||||
() => result.Reason.ShouldBe(CleanReason.MaxRatioReached)
|
||||
);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WhenRatioReachedAndMinSeedTimeNotReached_ShouldReturnFalse()
|
||||
{
|
||||
// Arrange
|
||||
Category category = new()
|
||||
{
|
||||
Name = "test",
|
||||
MaxRatio = 1.0,
|
||||
MinSeedTime = 3,
|
||||
MaxSeedTime = -1
|
||||
};
|
||||
const double ratio = 1.5;
|
||||
TimeSpan seedingTime = TimeSpan.FromHours(2);
|
||||
|
||||
TestDownloadService sut = _fixture.CreateSut();
|
||||
|
||||
// Act
|
||||
var result = sut.ShouldCleanDownload(ratio, seedingTime, category);
|
||||
|
||||
// Assert
|
||||
result.ShouldSatisfyAllConditions(
|
||||
() => result.ShouldClean.ShouldBeFalse(),
|
||||
() => result.Reason.ShouldBe(CleanReason.None)
|
||||
);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WhenMaxSeedTimeReached_ShouldReturnTrue()
|
||||
{
|
||||
// Arrange
|
||||
Category category = new()
|
||||
{
|
||||
Name = "test",
|
||||
MaxRatio = -1,
|
||||
MinSeedTime = 0,
|
||||
MaxSeedTime = 1
|
||||
};
|
||||
const double ratio = 0.5;
|
||||
TimeSpan seedingTime = TimeSpan.FromHours(2);
|
||||
|
||||
TestDownloadService sut = _fixture.CreateSut();
|
||||
|
||||
// Act
|
||||
SeedingCheckResult result = sut.ShouldCleanDownload(ratio, seedingTime, category);
|
||||
|
||||
// Assert
|
||||
result.ShouldSatisfyAllConditions(
|
||||
() => result.ShouldClean.ShouldBeTrue(),
|
||||
() => result.Reason.ShouldBe(CleanReason.MaxSeedTimeReached)
|
||||
);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WhenNeitherConditionMet_ShouldReturnFalse()
|
||||
{
|
||||
// Arrange
|
||||
Category category = new()
|
||||
{
|
||||
Name = "test",
|
||||
MaxRatio = 2.0,
|
||||
MinSeedTime = 0,
|
||||
MaxSeedTime = 3
|
||||
};
|
||||
const double ratio = 1.0;
|
||||
TimeSpan seedingTime = TimeSpan.FromHours(1);
|
||||
|
||||
TestDownloadService sut = _fixture.CreateSut();
|
||||
|
||||
// Act
|
||||
var result = sut.ShouldCleanDownload(ratio, seedingTime, category);
|
||||
|
||||
// Assert
|
||||
result.ShouldSatisfyAllConditions(
|
||||
() => result.ShouldClean.ShouldBeFalse(),
|
||||
() => result.Reason.ShouldBe(CleanReason.None)
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.DownloadClient;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
namespace Infrastructure.Tests.Verticals.DownloadClient;
|
||||
|
||||
public class TestDownloadService : DownloadService
|
||||
{
|
||||
public TestDownloadService(
|
||||
ILogger<DownloadService> logger,
|
||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
||||
IOptions<DownloadCleanerConfig> downloadCleanerConfig,
|
||||
IMemoryCache cache,
|
||||
IFilenameEvaluator filenameEvaluator,
|
||||
IStriker striker,
|
||||
NotificationPublisher notifier)
|
||||
: base(logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig,
|
||||
cache, filenameEvaluator, striker, notifier)
|
||||
{
|
||||
}
|
||||
|
||||
public override void Dispose() { }
|
||||
public override Task LoginAsync() => Task.CompletedTask;
|
||||
public override Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash) => Task.FromResult(new StalledResult());
|
||||
public override Task<BlockFilesResult> BlockUnwantedFilesAsync(string hash, BlocklistType blocklistType,
|
||||
ConcurrentBag<string> patterns, ConcurrentBag<Regex> regexes) => Task.FromResult(new BlockFilesResult());
|
||||
public override Task DeleteDownload(string hash) => Task.CompletedTask;
|
||||
public override Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories) => Task.FromResult<List<object>?>(null);
|
||||
public override Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes) => Task.CompletedTask;
|
||||
|
||||
// Expose protected methods for testing
|
||||
public new void ResetStrikesOnProgress(string hash, long downloaded) => base.ResetStrikesOnProgress(hash, downloaded);
|
||||
public new Task<bool> StrikeAndCheckLimit(string hash, string itemName) => base.StrikeAndCheckLimit(hash, itemName);
|
||||
public new SeedingCheckResult ShouldCleanDownload(double ratio, TimeSpan seedingTime, Category category) => base.ShouldCleanDownload(ratio, seedingTime, category);
|
||||
}
|
||||
@@ -12,13 +12,15 @@
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Castle.Core.AsyncInterceptor" Version="2.1.0" />
|
||||
<PackageReference Include="FLM.QBittorrent" Version="1.0.0" />
|
||||
<PackageReference Include="FLM.Transmission" Version="1.0.2" />
|
||||
<PackageReference Include="Mapster" Version="7.4.0" />
|
||||
<PackageReference Include="MassTransit" Version="8.3.6" />
|
||||
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="9.0.0" />
|
||||
<PackageReference Include="Microsoft.Extensions.Http" Version="9.0.0" />
|
||||
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="9.0.2" />
|
||||
<PackageReference Include="Microsoft.Extensions.Http" Version="9.0.2" />
|
||||
<PackageReference Include="Quartz" Version="3.13.1" />
|
||||
<PackageReference Include="Scrutor" Version="6.0.1" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
|
||||
49
code/Infrastructure/Interceptors/DryRunInterceptor.cs
Normal file
49
code/Infrastructure/Interceptors/DryRunInterceptor.cs
Normal file
@@ -0,0 +1,49 @@
|
||||
using System.Reflection;
|
||||
using Castle.DynamicProxy;
|
||||
using Common.Attributes;
|
||||
using Common.Configuration.General;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
namespace Infrastructure.Interceptors;
|
||||
|
||||
public class DryRunAsyncInterceptor : AsyncInterceptorBase
|
||||
{
|
||||
private readonly ILogger<DryRunAsyncInterceptor> _logger;
|
||||
private readonly DryRunConfig _config;
|
||||
|
||||
public DryRunAsyncInterceptor(ILogger<DryRunAsyncInterceptor> logger, IOptions<DryRunConfig> config)
|
||||
{
|
||||
_logger = logger;
|
||||
_config = config.Value;
|
||||
}
|
||||
|
||||
protected override async Task InterceptAsync(IInvocation invocation, IInvocationProceedInfo proceedInfo, Func<IInvocation, IInvocationProceedInfo, Task> proceed)
|
||||
{
|
||||
MethodInfo? method = invocation.MethodInvocationTarget ?? invocation.Method;
|
||||
if (IsDryRun(method))
|
||||
{
|
||||
_logger.LogInformation("[DRY RUN] skipping method: {name}", method.Name);
|
||||
return;
|
||||
}
|
||||
|
||||
await proceed(invocation, proceedInfo);
|
||||
}
|
||||
|
||||
protected override async Task<TResult> InterceptAsync<TResult>(IInvocation invocation, IInvocationProceedInfo proceedInfo, Func<IInvocation, IInvocationProceedInfo, Task<TResult>> proceed)
|
||||
{
|
||||
MethodInfo? method = invocation.MethodInvocationTarget ?? invocation.Method;
|
||||
if (IsDryRun(method))
|
||||
{
|
||||
_logger.LogInformation("[DRY RUN] skipping method: {name}", method.Name);
|
||||
return default!;
|
||||
}
|
||||
|
||||
return await proceed(invocation, proceedInfo);
|
||||
}
|
||||
|
||||
private bool IsDryRun(MethodInfo method)
|
||||
{
|
||||
return method.GetCustomAttributes(typeof(DryRunSafeguardAttribute), true).Any() && _config.IsDryRun;
|
||||
}
|
||||
}
|
||||
5
code/Infrastructure/Interceptors/IDryRunService.cs
Normal file
5
code/Infrastructure/Interceptors/IDryRunService.cs
Normal file
@@ -0,0 +1,5 @@
|
||||
namespace Infrastructure.Interceptors;
|
||||
|
||||
public interface IDryRunService : IInterceptedService
|
||||
{
|
||||
}
|
||||
6
code/Infrastructure/Interceptors/IInterceptedService.cs
Normal file
6
code/Infrastructure/Interceptors/IInterceptedService.cs
Normal file
@@ -0,0 +1,6 @@
|
||||
namespace Infrastructure.Interceptors;
|
||||
|
||||
public interface IInterceptedService
|
||||
{
|
||||
public object Proxy { get; set; }
|
||||
}
|
||||
21
code/Infrastructure/Interceptors/InterceptedService.cs
Normal file
21
code/Infrastructure/Interceptors/InterceptedService.cs
Normal file
@@ -0,0 +1,21 @@
|
||||
namespace Infrastructure.Interceptors;
|
||||
|
||||
public class InterceptedService : IInterceptedService
|
||||
{
|
||||
private object? _proxy;
|
||||
|
||||
public object Proxy
|
||||
{
|
||||
get
|
||||
{
|
||||
if (_proxy is null)
|
||||
{
|
||||
throw new Exception("Proxy is not set");
|
||||
}
|
||||
|
||||
return _proxy;
|
||||
}
|
||||
|
||||
set => _proxy = value;
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
using Common.Attributes;
|
||||
using Common.Configuration.Arr;
|
||||
using Common.Configuration.Logging;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
@@ -5,6 +6,8 @@ using Common.Helpers;
|
||||
using Domain.Enums;
|
||||
using Domain.Models.Arr;
|
||||
using Domain.Models.Arr.Queue;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Verticals.Arr.Interfaces;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
@@ -12,24 +15,30 @@ using Newtonsoft.Json;
|
||||
|
||||
namespace Infrastructure.Verticals.Arr;
|
||||
|
||||
public abstract class ArrClient
|
||||
public abstract class ArrClient : InterceptedService, IArrClient, IDryRunService
|
||||
{
|
||||
protected readonly ILogger<ArrClient> _logger;
|
||||
protected readonly HttpClient _httpClient;
|
||||
protected readonly LoggingConfig _loggingConfig;
|
||||
protected readonly QueueCleanerConfig _queueCleanerConfig;
|
||||
protected readonly Striker _striker;
|
||||
protected readonly IStriker _striker;
|
||||
|
||||
/// <summary>
|
||||
/// Constructor to be used by interceptors.
|
||||
/// </summary>
|
||||
protected ArrClient()
|
||||
{
|
||||
}
|
||||
|
||||
protected ArrClient(
|
||||
ILogger<ArrClient> logger,
|
||||
IHttpClientFactory httpClientFactory,
|
||||
IOptions<LoggingConfig> loggingConfig,
|
||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||
Striker striker
|
||||
IStriker striker
|
||||
)
|
||||
{
|
||||
_logger = logger;
|
||||
_striker = striker;
|
||||
_httpClient = httpClientFactory.CreateClient(Constants.HttpClientWithRetryName);
|
||||
_loggingConfig = loggingConfig.Value;
|
||||
_queueCleanerConfig = queueCleanerConfig.Value;
|
||||
@@ -110,16 +119,14 @@ public abstract class ArrClient
|
||||
public virtual async Task DeleteQueueItemAsync(ArrInstance arrInstance, QueueRecord record, bool removeFromClient)
|
||||
{
|
||||
Uri uri = new(arrInstance.Url, GetQueueDeleteUrlPath(record.Id, removeFromClient));
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Delete, uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||
|
||||
try
|
||||
{
|
||||
response.EnsureSuccessStatusCode();
|
||||
using HttpRequestMessage request = new(HttpMethod.Delete, uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using var _ = await ((ArrClient)Proxy).SendRequestAsync(request);
|
||||
|
||||
_logger.LogInformation(
|
||||
removeFromClient
|
||||
? "queue item deleted | {url} | {title}"
|
||||
@@ -157,6 +164,16 @@ public abstract class ArrClient
|
||||
request.Headers.Add("x-api-key", apiKey);
|
||||
}
|
||||
|
||||
[DryRunSafeguard]
|
||||
protected virtual async Task<HttpResponseMessage> SendRequestAsync(HttpRequestMessage request)
|
||||
{
|
||||
HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||
|
||||
response.EnsureSuccessStatusCode();
|
||||
|
||||
return response;
|
||||
}
|
||||
|
||||
private bool HasIgnoredPatterns(QueueRecord record)
|
||||
{
|
||||
if (_queueCleanerConfig.ImportFailedIgnorePatterns?.Count is null or 0)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
using Common.Configuration;
|
||||
using Common.Configuration.Arr;
|
||||
using Domain.Models.Arr.Queue;
|
||||
using Infrastructure.Verticals.Arr.Interfaces;
|
||||
using Microsoft.Extensions.Logging;
|
||||
|
||||
namespace Infrastructure.Verticals.Arr;
|
||||
@@ -14,7 +15,7 @@ public sealed class ArrQueueIterator
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
public async Task Iterate(ArrClient arrClient, ArrInstance arrInstance, Func<IReadOnlyList<QueueRecord>, Task> action)
|
||||
public async Task Iterate(IArrClient arrClient, ArrInstance arrInstance, Func<IReadOnlyList<QueueRecord>, Task> action)
|
||||
{
|
||||
const ushort maxPage = 100;
|
||||
ushort page = 1;
|
||||
|
||||
19
code/Infrastructure/Verticals/Arr/Interfaces/IArrClient.cs
Normal file
19
code/Infrastructure/Verticals/Arr/Interfaces/IArrClient.cs
Normal file
@@ -0,0 +1,19 @@
|
||||
using Common.Configuration.Arr;
|
||||
using Domain.Enums;
|
||||
using Domain.Models.Arr;
|
||||
using Domain.Models.Arr.Queue;
|
||||
|
||||
namespace Infrastructure.Verticals.Arr.Interfaces;
|
||||
|
||||
public interface IArrClient
|
||||
{
|
||||
Task<QueueListResponse> GetQueueItemsAsync(ArrInstance arrInstance, int page);
|
||||
|
||||
Task<bool> ShouldRemoveFromQueue(InstanceType instanceType, QueueRecord record, bool isPrivateDownload);
|
||||
|
||||
Task DeleteQueueItemAsync(ArrInstance arrInstance, QueueRecord record, bool removeFromClient);
|
||||
|
||||
Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items);
|
||||
|
||||
bool IsRecordValid(QueueRecord record);
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
namespace Infrastructure.Verticals.Arr.Interfaces;
|
||||
|
||||
public interface ILidarrClient : IArrClient
|
||||
{
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
namespace Infrastructure.Verticals.Arr.Interfaces;
|
||||
|
||||
public interface IRadarrClient : IArrClient
|
||||
{
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
namespace Infrastructure.Verticals.Arr.Interfaces;
|
||||
|
||||
public interface ISonarrClient : IArrClient
|
||||
{
|
||||
}
|
||||
@@ -5,6 +5,7 @@ using Common.Configuration.QueueCleaner;
|
||||
using Domain.Models.Arr;
|
||||
using Domain.Models.Arr.Queue;
|
||||
using Domain.Models.Lidarr;
|
||||
using Infrastructure.Verticals.Arr.Interfaces;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
@@ -12,14 +13,19 @@ using Newtonsoft.Json;
|
||||
|
||||
namespace Infrastructure.Verticals.Arr;
|
||||
|
||||
public sealed class LidarrClient : ArrClient
|
||||
public class LidarrClient : ArrClient, ILidarrClient
|
||||
{
|
||||
/// <inheritdoc/>
|
||||
public LidarrClient()
|
||||
{
|
||||
}
|
||||
|
||||
public LidarrClient(
|
||||
ILogger<LidarrClient> logger,
|
||||
IHttpClientFactory httpClientFactory,
|
||||
IOptions<LoggingConfig> loggingConfig,
|
||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||
Striker striker
|
||||
IStriker striker
|
||||
) : base(logger, httpClientFactory, loggingConfig, queueCleanerConfig, striker)
|
||||
{
|
||||
}
|
||||
@@ -54,13 +60,12 @@ public sealed class LidarrClient : ArrClient
|
||||
);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using var response = await _httpClient.SendAsync(request);
|
||||
string? logContext = await ComputeCommandLogContextAsync(arrInstance, command);
|
||||
|
||||
try
|
||||
{
|
||||
response.EnsureSuccessStatusCode();
|
||||
|
||||
using var _ = await ((LidarrClient)Proxy).SendRequestAsync(request);
|
||||
|
||||
_logger.LogInformation("{log}", GetSearchLog(arrInstance.Url, command, true, logContext));
|
||||
}
|
||||
catch
|
||||
|
||||
@@ -5,6 +5,7 @@ using Common.Configuration.QueueCleaner;
|
||||
using Domain.Models.Arr;
|
||||
using Domain.Models.Arr.Queue;
|
||||
using Domain.Models.Radarr;
|
||||
using Infrastructure.Verticals.Arr.Interfaces;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
@@ -12,14 +13,19 @@ using Newtonsoft.Json;
|
||||
|
||||
namespace Infrastructure.Verticals.Arr;
|
||||
|
||||
public sealed class RadarrClient : ArrClient
|
||||
public class RadarrClient : ArrClient, IRadarrClient
|
||||
{
|
||||
/// <inheritdoc/>
|
||||
public RadarrClient()
|
||||
{
|
||||
}
|
||||
|
||||
public RadarrClient(
|
||||
ILogger<ArrClient> logger,
|
||||
IHttpClientFactory httpClientFactory,
|
||||
IOptions<LoggingConfig> loggingConfig,
|
||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||
Striker striker
|
||||
IStriker striker
|
||||
) : base(logger, httpClientFactory, loggingConfig, queueCleanerConfig, striker)
|
||||
{
|
||||
}
|
||||
@@ -62,12 +68,11 @@ public sealed class RadarrClient : ArrClient
|
||||
);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||
string? logContext = await ComputeCommandLogContextAsync(arrInstance, command);
|
||||
|
||||
try
|
||||
{
|
||||
response.EnsureSuccessStatusCode();
|
||||
using var _ = await ((RadarrClient)Proxy).SendRequestAsync(request);
|
||||
|
||||
_logger.LogInformation("{log}", GetSearchLog(arrInstance.Url, command, true, logContext));
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@ using Common.Configuration.QueueCleaner;
|
||||
using Domain.Models.Arr;
|
||||
using Domain.Models.Arr.Queue;
|
||||
using Domain.Models.Sonarr;
|
||||
using Infrastructure.Verticals.Arr.Interfaces;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
@@ -13,14 +14,19 @@ using Series = Domain.Models.Sonarr.Series;
|
||||
|
||||
namespace Infrastructure.Verticals.Arr;
|
||||
|
||||
public sealed class SonarrClient : ArrClient
|
||||
public class SonarrClient : ArrClient, ISonarrClient
|
||||
{
|
||||
/// <inheritdoc/>
|
||||
public SonarrClient()
|
||||
{
|
||||
}
|
||||
|
||||
public SonarrClient(
|
||||
ILogger<SonarrClient> logger,
|
||||
IHttpClientFactory httpClientFactory,
|
||||
IOptions<LoggingConfig> loggingConfig,
|
||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||
Striker striker
|
||||
IStriker striker
|
||||
) : base(logger, httpClientFactory, loggingConfig, queueCleanerConfig, striker)
|
||||
{
|
||||
}
|
||||
@@ -58,13 +64,12 @@ public sealed class SonarrClient : ArrClient
|
||||
);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||
string? logContext = await ComputeCommandLogContextAsync(arrInstance, command, command.SearchType);
|
||||
|
||||
try
|
||||
{
|
||||
response.EnsureSuccessStatusCode();
|
||||
|
||||
using var _ = await ((SonarrClient)Proxy).SendRequestAsync(request);
|
||||
|
||||
_logger.LogInformation("{log}", GetSearchLog(command.SearchType, arrInstance.Url, command, true, logContext));
|
||||
}
|
||||
catch
|
||||
|
||||
@@ -7,6 +7,7 @@ using Domain.Enums;
|
||||
using Domain.Models.Arr;
|
||||
using Domain.Models.Arr.Queue;
|
||||
using Infrastructure.Verticals.Arr;
|
||||
using Infrastructure.Verticals.Arr.Interfaces;
|
||||
using Infrastructure.Verticals.Context;
|
||||
using Infrastructure.Verticals.DownloadClient;
|
||||
using Infrastructure.Verticals.Jobs;
|
||||
@@ -36,7 +37,6 @@ public sealed class ContentBlocker : GenericHandler
|
||||
BlocklistProvider blocklistProvider,
|
||||
DownloadServiceFactory downloadServiceFactory,
|
||||
NotificationPublisher notifier
|
||||
|
||||
) : base(
|
||||
logger, downloadClientConfig,
|
||||
sonarrConfig, radarrConfig, lidarrConfig,
|
||||
@@ -76,7 +76,7 @@ public sealed class ContentBlocker : GenericHandler
|
||||
using var _ = LogContext.PushProperty("InstanceName", instanceType.ToString());
|
||||
|
||||
HashSet<SearchItem> itemsToBeRefreshed = [];
|
||||
ArrClient arrClient = GetClient(instanceType);
|
||||
IArrClient arrClient = GetClient(instanceType);
|
||||
BlocklistType blocklistType = _blocklistProvider.GetBlocklistType(instanceType);
|
||||
ConcurrentBag<string> patterns = _blocklistProvider.GetPatterns(instanceType);
|
||||
ConcurrentBag<Regex> regexes = _blocklistProvider.GetRegexes(instanceType);
|
||||
@@ -131,7 +131,7 @@ public sealed class ContentBlocker : GenericHandler
|
||||
}
|
||||
|
||||
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient);
|
||||
await _notifier.NotifyQueueItemDelete(removeFromClient, DeleteReason.AllFilesBlocked);
|
||||
await _notifier.NotifyQueueItemDeleted(removeFromClient, DeleteReason.AllFilesBlocked);
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ using Microsoft.Extensions.Logging;
|
||||
|
||||
namespace Infrastructure.Verticals.ContentBlocker;
|
||||
|
||||
public sealed class FilenameEvaluator
|
||||
public class FilenameEvaluator : IFilenameEvaluator
|
||||
{
|
||||
private readonly ILogger<FilenameEvaluator> _logger;
|
||||
|
||||
@@ -31,7 +31,6 @@ public sealed class FilenameEvaluator
|
||||
{
|
||||
BlocklistType.Blacklist => !patterns.Any(pattern => MatchesPattern(filename, pattern)),
|
||||
BlocklistType.Whitelist => patterns.Any(pattern => MatchesPattern(filename, pattern)),
|
||||
_ => true
|
||||
};
|
||||
}
|
||||
|
||||
@@ -46,7 +45,6 @@ public sealed class FilenameEvaluator
|
||||
{
|
||||
BlocklistType.Blacklist => !regexes.Any(regex => regex.IsMatch(filename)),
|
||||
BlocklistType.Whitelist => regexes.Any(regex => regex.IsMatch(filename)),
|
||||
_ => true
|
||||
};
|
||||
}
|
||||
|
||||
@@ -76,6 +74,6 @@ public sealed class FilenameEvaluator
|
||||
);
|
||||
}
|
||||
|
||||
return filename == pattern;
|
||||
return filename.Equals(pattern, StringComparison.InvariantCultureIgnoreCase);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
|
||||
namespace Infrastructure.Verticals.ContentBlocker;
|
||||
|
||||
public interface IFilenameEvaluator
|
||||
{
|
||||
bool IsValid(string filename, BlocklistType type, ConcurrentBag<string> patterns, ConcurrentBag<Regex> regexes);
|
||||
}
|
||||
@@ -17,8 +17,8 @@ public static class ContextProvider
|
||||
return _asyncLocalDict.Value?.TryGetValue(key, out object? value) is true ? value : null;
|
||||
}
|
||||
|
||||
public static T? Get<T>(string key) where T : class
|
||||
public static T Get<T>(string key) where T : class
|
||||
{
|
||||
return Get(key) as T;
|
||||
return Get(key) as T ?? throw new Exception($"failed to get \"{key}\" from context");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
using Common.Configuration.Arr;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.DownloadClient;
|
||||
using Domain.Enums;
|
||||
using Domain.Models.Arr.Queue;
|
||||
using Infrastructure.Verticals.Arr;
|
||||
using Infrastructure.Verticals.Arr.Interfaces;
|
||||
using Infrastructure.Verticals.DownloadClient;
|
||||
using Infrastructure.Verticals.Jobs;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using Serilog.Context;
|
||||
|
||||
namespace Infrastructure.Verticals.DownloadCleaner;
|
||||
|
||||
public sealed class DownloadCleaner : GenericHandler
|
||||
{
|
||||
private readonly DownloadCleanerConfig _config;
|
||||
private readonly HashSet<string> _excludedHashes = [];
|
||||
|
||||
public DownloadCleaner(
|
||||
ILogger<DownloadCleaner> logger,
|
||||
IOptions<DownloadCleanerConfig> config,
|
||||
IOptions<DownloadClientConfig> downloadClientConfig,
|
||||
IOptions<SonarrConfig> sonarrConfig,
|
||||
IOptions<RadarrConfig> radarrConfig,
|
||||
IOptions<LidarrConfig> lidarrConfig,
|
||||
SonarrClient sonarrClient,
|
||||
RadarrClient radarrClient,
|
||||
LidarrClient lidarrClient,
|
||||
ArrQueueIterator arrArrQueueIterator,
|
||||
DownloadServiceFactory downloadServiceFactory,
|
||||
NotificationPublisher notifier
|
||||
) : base(
|
||||
logger, downloadClientConfig,
|
||||
sonarrConfig, radarrConfig, lidarrConfig,
|
||||
sonarrClient, radarrClient, lidarrClient,
|
||||
arrArrQueueIterator, downloadServiceFactory,
|
||||
notifier
|
||||
)
|
||||
{
|
||||
_config = config.Value;
|
||||
_config.Validate();
|
||||
}
|
||||
|
||||
public override async Task ExecuteAsync()
|
||||
{
|
||||
if (_config.Categories?.Count is null or 0)
|
||||
{
|
||||
_logger.LogWarning("no categories configured");
|
||||
return;
|
||||
}
|
||||
|
||||
await _downloadService.LoginAsync();
|
||||
|
||||
List<object>? downloads = await _downloadService.GetAllDownloadsToBeCleaned(_config.Categories);
|
||||
|
||||
if (downloads?.Count is null or 0)
|
||||
{
|
||||
_logger.LogDebug("no downloads found in the download client");
|
||||
return;
|
||||
}
|
||||
|
||||
// wait for the downloads to appear in the arr queue
|
||||
await Task.Delay(10 * 1000);
|
||||
|
||||
await ProcessArrConfigAsync(_sonarrConfig, InstanceType.Sonarr, true);
|
||||
await ProcessArrConfigAsync(_radarrConfig, InstanceType.Radarr, true);
|
||||
await ProcessArrConfigAsync(_lidarrConfig, InstanceType.Lidarr, true);
|
||||
|
||||
await _downloadService.CleanDownloads(downloads, _config.Categories, _excludedHashes);
|
||||
}
|
||||
|
||||
protected override async Task ProcessInstanceAsync(ArrInstance instance, InstanceType instanceType)
|
||||
{
|
||||
using var _ = LogContext.PushProperty("InstanceName", instanceType.ToString());
|
||||
|
||||
IArrClient arrClient = GetClient(instanceType);
|
||||
|
||||
await _arrArrQueueIterator.Iterate(arrClient, instance, async items =>
|
||||
{
|
||||
var groups = items
|
||||
.Where(x => !string.IsNullOrEmpty(x.DownloadId))
|
||||
.GroupBy(x => x.DownloadId)
|
||||
.ToList();
|
||||
|
||||
foreach (QueueRecord record in groups.Select(group => group.First()))
|
||||
{
|
||||
_excludedHashes.Add(record.DownloadId.ToLowerInvariant());
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
public override void Dispose()
|
||||
{
|
||||
_downloadService.Dispose();
|
||||
}
|
||||
}
|
||||
@@ -62,6 +62,26 @@ public sealed class DelugeClient
|
||||
await ListTorrentsExtended(new Dictionary<string, string> { { "hash", hash } });
|
||||
return torrents.FirstOrDefault();
|
||||
}
|
||||
|
||||
public async Task<TorrentStatus?> GetTorrentStatus(string hash)
|
||||
{
|
||||
return await SendRequest<TorrentStatus?>(
|
||||
"web.get_torrent_status",
|
||||
hash,
|
||||
new[] { "hash", "state", "name", "eta", "private", "total_done", "label", "seeding_time", "ratio" }
|
||||
);
|
||||
}
|
||||
|
||||
public async Task<List<TorrentStatus>?> GetStatusForAllTorrents()
|
||||
{
|
||||
Dictionary<string, TorrentStatus>? downloads = await SendRequest<Dictionary<string, TorrentStatus>?>(
|
||||
"core.get_torrents_status",
|
||||
"",
|
||||
new[] { "hash", "state", "name", "eta", "private", "total_done", "label", "seeding_time", "ratio" }
|
||||
);
|
||||
|
||||
return downloads?.Values.ToList();
|
||||
}
|
||||
|
||||
public async Task<DelugeContents?> GetTorrentFiles(string hash)
|
||||
{
|
||||
@@ -78,9 +98,9 @@ public sealed class DelugeClient
|
||||
await SendRequest<DelugeResponse<object>>("core.set_torrent_options", hash, filePriorities);
|
||||
}
|
||||
|
||||
public async Task<DelugeResponse<object>> DeleteTorrent(string hash)
|
||||
public async Task DeleteTorrents(List<string> hashes)
|
||||
{
|
||||
return await SendRequest<DelugeResponse<object>>("core.remove_torrents", new List<string> { hash }, true);
|
||||
await SendRequest<DelugeResponse<object>>("core.remove_torrents", hashes, true);
|
||||
}
|
||||
|
||||
private async Task<String> PostJson(String json)
|
||||
|
||||
@@ -1,21 +1,31 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Attributes;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.DownloadClient;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Domain.Enums;
|
||||
using Domain.Models.Deluge.Response;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.Context;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using MassTransit.Configuration;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
namespace Infrastructure.Verticals.DownloadClient.Deluge;
|
||||
|
||||
public sealed class DelugeService : DownloadServiceBase
|
||||
public class DelugeService : DownloadService, IDelugeService
|
||||
{
|
||||
private readonly DelugeClient _client;
|
||||
|
||||
/// <inheritdoc/>
|
||||
public DelugeService()
|
||||
{
|
||||
}
|
||||
|
||||
public DelugeService(
|
||||
ILogger<DelugeService> logger,
|
||||
@@ -23,10 +33,12 @@ public sealed class DelugeService : DownloadServiceBase
|
||||
IHttpClientFactory httpClientFactory,
|
||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
||||
IOptions<DownloadCleanerConfig> downloadCleanerConfig,
|
||||
IMemoryCache cache,
|
||||
FilenameEvaluator filenameEvaluator,
|
||||
Striker striker
|
||||
) : base(logger, queueCleanerConfig, contentBlockerConfig, cache, filenameEvaluator, striker)
|
||||
IFilenameEvaluator filenameEvaluator,
|
||||
IStriker striker,
|
||||
NotificationPublisher notifier
|
||||
) : base(logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig, cache, filenameEvaluator, striker, notifier)
|
||||
{
|
||||
config.Value.Validate();
|
||||
_client = new (config, httpClientFactory);
|
||||
@@ -45,7 +57,7 @@ public sealed class DelugeService : DownloadServiceBase
|
||||
DelugeContents? contents = null;
|
||||
StalledResult result = new();
|
||||
|
||||
TorrentStatus? status = await GetTorrentStatus(hash);
|
||||
TorrentStatus? status = await _client.GetTorrentStatus(hash);
|
||||
|
||||
if (status?.Hash is null)
|
||||
{
|
||||
@@ -98,7 +110,7 @@ public sealed class DelugeService : DownloadServiceBase
|
||||
{
|
||||
hash = hash.ToLowerInvariant();
|
||||
|
||||
TorrentStatus? status = await GetTorrentStatus(hash);
|
||||
TorrentStatus? status = await _client.GetTorrentStatus(hash);
|
||||
BlockFilesResult result = new();
|
||||
|
||||
if (status?.Hash is null)
|
||||
@@ -178,17 +190,89 @@ public sealed class DelugeService : DownloadServiceBase
|
||||
return result;
|
||||
}
|
||||
|
||||
await _client.ChangeFilesPriority(hash, sortedPriorities);
|
||||
await ((DelugeService)Proxy).ChangeFilesPriority(hash, sortedPriorities);
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
public override async Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories)
|
||||
{
|
||||
return (await _client.GetStatusForAllTorrents())
|
||||
?.Where(x => !string.IsNullOrEmpty(x.Hash))
|
||||
.Where(x => x.State?.Equals("seeding", StringComparison.InvariantCultureIgnoreCase) is true)
|
||||
.Where(x => categories.Any(cat => cat.Name.Equals(x.Label, StringComparison.InvariantCultureIgnoreCase)))
|
||||
.Cast<object>()
|
||||
.ToList();
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes)
|
||||
{
|
||||
foreach (TorrentStatus download in downloads)
|
||||
{
|
||||
if (string.IsNullOrEmpty(download.Hash))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
Category? category = categoriesToClean
|
||||
.FirstOrDefault(x => x.Name.Equals(download.Label, StringComparison.InvariantCultureIgnoreCase));
|
||||
|
||||
if (category is null)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
if (excludedHashes.Any(x => x.Equals(download.Hash, StringComparison.InvariantCultureIgnoreCase)))
|
||||
{
|
||||
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!_downloadCleanerConfig.DeletePrivate && download.Private)
|
||||
{
|
||||
_logger.LogDebug("skip | download is private | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
ContextProvider.Set("downloadName", download.Name);
|
||||
ContextProvider.Set("hash", download.Hash);
|
||||
|
||||
TimeSpan seedingTime = TimeSpan.FromSeconds(download.SeedingTime);
|
||||
SeedingCheckResult result = ShouldCleanDownload(download.Ratio, seedingTime, category);
|
||||
|
||||
if (!result.ShouldClean)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
await ((DelugeService)Proxy).DeleteDownload(download.Hash);
|
||||
|
||||
_logger.LogInformation(
|
||||
"download cleaned | {reason} reached | {name}",
|
||||
result.Reason is CleanReason.MaxRatioReached
|
||||
? "MAX_RATIO & MIN_SEED_TIME"
|
||||
: "MAX_SEED_TIME",
|
||||
download.Name
|
||||
);
|
||||
|
||||
await _notifier.NotifyDownloadCleaned(download.Ratio, seedingTime, category.Name, result.Reason);
|
||||
}
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task Delete(string hash)
|
||||
[DryRunSafeguard]
|
||||
public override async Task DeleteDownload(string hash)
|
||||
{
|
||||
hash = hash.ToLowerInvariant();
|
||||
|
||||
await _client.DeleteTorrent(hash);
|
||||
await _client.DeleteTorrents([hash]);
|
||||
}
|
||||
|
||||
[DryRunSafeguard]
|
||||
protected virtual async Task ChangeFilesPriority(string hash, List<int> sortedPriorities)
|
||||
{
|
||||
await _client.ChangeFilesPriority(hash, sortedPriorities);
|
||||
}
|
||||
|
||||
private async Task<bool> IsItemStuckAndShouldRemove(TorrentStatus status)
|
||||
@@ -219,15 +303,6 @@ public sealed class DelugeService : DownloadServiceBase
|
||||
|
||||
return await StrikeAndCheckLimit(status.Hash!, status.Name!);
|
||||
}
|
||||
|
||||
private async Task<TorrentStatus?> GetTorrentStatus(string hash)
|
||||
{
|
||||
return await _client.SendRequest<TorrentStatus?>(
|
||||
"web.get_torrent_status",
|
||||
hash,
|
||||
new[] { "hash", "state", "name", "eta", "private", "total_done" }
|
||||
);
|
||||
}
|
||||
|
||||
private static void ProcessFiles(Dictionary<string, DelugeFileOrDirectory>? contents, Action<string, DelugeFileOrDirectory> processFile)
|
||||
{
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
namespace Infrastructure.Verticals.DownloadClient.Deluge;
|
||||
|
||||
public interface IDelugeService : IDownloadService
|
||||
{
|
||||
}
|
||||
183
code/Infrastructure/Verticals/DownloadClient/DownloadService.cs
Normal file
183
code/Infrastructure/Verticals/DownloadClient/DownloadService.cs
Normal file
@@ -0,0 +1,183 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Common.Helpers;
|
||||
using Domain.Enums;
|
||||
using Domain.Models.Cache;
|
||||
using Infrastructure.Helpers;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.Context;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
namespace Infrastructure.Verticals.DownloadClient;
|
||||
|
||||
public abstract class DownloadService : InterceptedService, IDownloadService
|
||||
{
|
||||
protected readonly ILogger<DownloadService> _logger;
|
||||
protected readonly QueueCleanerConfig _queueCleanerConfig;
|
||||
protected readonly ContentBlockerConfig _contentBlockerConfig;
|
||||
protected readonly DownloadCleanerConfig _downloadCleanerConfig;
|
||||
protected readonly IMemoryCache _cache;
|
||||
protected readonly IFilenameEvaluator _filenameEvaluator;
|
||||
protected readonly IStriker _striker;
|
||||
protected readonly MemoryCacheEntryOptions _cacheOptions;
|
||||
protected readonly NotificationPublisher _notifier;
|
||||
|
||||
/// <summary>
|
||||
/// Constructor to be used by interceptors.
|
||||
/// </summary>
|
||||
protected DownloadService()
|
||||
{
|
||||
}
|
||||
|
||||
protected DownloadService(
|
||||
ILogger<DownloadService> logger,
|
||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
||||
IOptions<DownloadCleanerConfig> downloadCleanerConfig,
|
||||
IMemoryCache cache,
|
||||
IFilenameEvaluator filenameEvaluator,
|
||||
IStriker striker,
|
||||
NotificationPublisher notifier)
|
||||
{
|
||||
_logger = logger;
|
||||
_queueCleanerConfig = queueCleanerConfig.Value;
|
||||
_contentBlockerConfig = contentBlockerConfig.Value;
|
||||
_downloadCleanerConfig = downloadCleanerConfig.Value;
|
||||
_cache = cache;
|
||||
_filenameEvaluator = filenameEvaluator;
|
||||
_striker = striker;
|
||||
_notifier = notifier;
|
||||
_cacheOptions = new MemoryCacheEntryOptions()
|
||||
.SetSlidingExpiration(StaticConfiguration.TriggerValue + Constants.CacheLimitBuffer);
|
||||
}
|
||||
|
||||
public abstract void Dispose();
|
||||
|
||||
public abstract Task LoginAsync();
|
||||
|
||||
public abstract Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash);
|
||||
|
||||
/// <inheritdoc/>
|
||||
public abstract Task<BlockFilesResult> BlockUnwantedFilesAsync(
|
||||
string hash,
|
||||
BlocklistType blocklistType,
|
||||
ConcurrentBag<string> patterns,
|
||||
ConcurrentBag<Regex> regexes
|
||||
);
|
||||
|
||||
/// <inheritdoc/>
|
||||
public abstract Task DeleteDownload(string hash);
|
||||
|
||||
/// <inheritdoc/>
|
||||
public abstract Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories);
|
||||
|
||||
/// <inheritdoc/>
|
||||
public abstract Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes);
|
||||
|
||||
protected void ResetStrikesOnProgress(string hash, long downloaded)
|
||||
{
|
||||
if (!_queueCleanerConfig.StalledResetStrikesOnProgress)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (_cache.TryGetValue(CacheKeys.Item(hash), out CacheItem? cachedItem) && cachedItem is not null && downloaded > cachedItem.Downloaded)
|
||||
{
|
||||
// cache item found
|
||||
_cache.Remove(CacheKeys.Strike(StrikeType.Stalled, hash));
|
||||
_logger.LogDebug("resetting strikes for {hash} due to progress", hash);
|
||||
}
|
||||
|
||||
_cache.Set(CacheKeys.Item(hash), new CacheItem { Downloaded = downloaded }, _cacheOptions);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Strikes an item and checks if the limit has been reached.
|
||||
/// </summary>
|
||||
/// <param name="hash">The torrent hash.</param>
|
||||
/// <param name="itemName">The name or title of the item.</param>
|
||||
/// <returns>True if the limit has been reached; otherwise, false.</returns>
|
||||
protected async Task<bool> StrikeAndCheckLimit(string hash, string itemName)
|
||||
{
|
||||
return await _striker.StrikeAndCheckLimit(hash, itemName, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled);
|
||||
}
|
||||
|
||||
protected SeedingCheckResult ShouldCleanDownload(double ratio, TimeSpan seedingTime, Category category)
|
||||
{
|
||||
// check ratio
|
||||
if (DownloadReachedRatio(ratio, seedingTime, category))
|
||||
{
|
||||
return new()
|
||||
{
|
||||
ShouldClean = true,
|
||||
Reason = CleanReason.MaxRatioReached
|
||||
};
|
||||
}
|
||||
|
||||
// check max seed time
|
||||
if (DownloadReachedMaxSeedTime(seedingTime, category))
|
||||
{
|
||||
return new()
|
||||
{
|
||||
ShouldClean = true,
|
||||
Reason = CleanReason.MaxSeedTimeReached
|
||||
};
|
||||
}
|
||||
|
||||
return new();
|
||||
}
|
||||
|
||||
private bool DownloadReachedRatio(double ratio, TimeSpan seedingTime, Category category)
|
||||
{
|
||||
if (category.MaxRatio < 0)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
string downloadName = ContextProvider.Get<string>("downloadName");
|
||||
TimeSpan minSeedingTime = TimeSpan.FromHours(category.MinSeedTime);
|
||||
|
||||
if (category.MinSeedTime > 0 && seedingTime < minSeedingTime)
|
||||
{
|
||||
_logger.LogDebug("skip | download has not reached MIN_SEED_TIME | {name}", downloadName);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (ratio < category.MaxRatio)
|
||||
{
|
||||
_logger.LogDebug("skip | download has not reached MAX_RATIO | {name}", downloadName);
|
||||
return false;
|
||||
}
|
||||
|
||||
// max ration is 0 or reached
|
||||
return true;
|
||||
}
|
||||
|
||||
private bool DownloadReachedMaxSeedTime(TimeSpan seedingTime, Category category)
|
||||
{
|
||||
if (category.MaxSeedTime < 0)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
string downloadName = ContextProvider.Get<string>("downloadName");
|
||||
TimeSpan maxSeedingTime = TimeSpan.FromHours(category.MaxSeedTime);
|
||||
|
||||
if (category.MaxSeedTime > 0 && seedingTime < maxSeedingTime)
|
||||
{
|
||||
_logger.LogDebug("skip | download has not reached MAX_SEED_TIME | {name}", downloadName);
|
||||
return false;
|
||||
}
|
||||
|
||||
// max seed time is 0 or reached
|
||||
return true;
|
||||
}
|
||||
}
|
||||
@@ -1,90 +0,0 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Common.Helpers;
|
||||
using Domain.Enums;
|
||||
using Domain.Models.Cache;
|
||||
using Infrastructure.Helpers;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
namespace Infrastructure.Verticals.DownloadClient;
|
||||
|
||||
public abstract class DownloadServiceBase : IDownloadService
|
||||
{
|
||||
protected readonly ILogger<DownloadServiceBase> _logger;
|
||||
protected readonly QueueCleanerConfig _queueCleanerConfig;
|
||||
protected readonly ContentBlockerConfig _contentBlockerConfig;
|
||||
protected readonly IMemoryCache _cache;
|
||||
protected readonly FilenameEvaluator _filenameEvaluator;
|
||||
protected readonly Striker _striker;
|
||||
protected readonly MemoryCacheEntryOptions _cacheOptions;
|
||||
|
||||
protected DownloadServiceBase(
|
||||
ILogger<DownloadServiceBase> logger,
|
||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
||||
IMemoryCache cache,
|
||||
FilenameEvaluator filenameEvaluator,
|
||||
Striker striker
|
||||
)
|
||||
{
|
||||
_logger = logger;
|
||||
_queueCleanerConfig = queueCleanerConfig.Value;
|
||||
_contentBlockerConfig = contentBlockerConfig.Value;
|
||||
_cache = cache;
|
||||
_filenameEvaluator = filenameEvaluator;
|
||||
_striker = striker;
|
||||
_cacheOptions = new MemoryCacheEntryOptions()
|
||||
.SetSlidingExpiration(StaticConfiguration.TriggerValue + Constants.CacheLimitBuffer);
|
||||
}
|
||||
|
||||
public abstract void Dispose();
|
||||
|
||||
public abstract Task LoginAsync();
|
||||
|
||||
public abstract Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash);
|
||||
|
||||
/// <inheritdoc/>
|
||||
public abstract Task<BlockFilesResult> BlockUnwantedFilesAsync(
|
||||
string hash,
|
||||
BlocklistType blocklistType,
|
||||
ConcurrentBag<string> patterns,
|
||||
ConcurrentBag<Regex> regexes
|
||||
);
|
||||
|
||||
/// <inheritdoc/>
|
||||
public abstract Task Delete(string hash);
|
||||
|
||||
protected void ResetStrikesOnProgress(string hash, long downloaded)
|
||||
{
|
||||
if (!_queueCleanerConfig.StalledResetStrikesOnProgress)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (_cache.TryGetValue(CacheKeys.Item(hash), out CacheItem? cachedItem) && cachedItem is not null && downloaded > cachedItem.Downloaded)
|
||||
{
|
||||
// cache item found
|
||||
_cache.Remove(CacheKeys.Strike(StrikeType.Stalled, hash));
|
||||
_logger.LogDebug("resetting strikes for {hash} due to progress", hash);
|
||||
}
|
||||
|
||||
_cache.Set(CacheKeys.Item(hash), new CacheItem { Downloaded = downloaded }, _cacheOptions);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Strikes an item and checks if the limit has been reached.
|
||||
/// </summary>
|
||||
/// <param name="hash">The torrent hash.</param>
|
||||
/// <param name="itemName">The name or title of the item.</param>
|
||||
/// <returns>True if the limit has been reached; otherwise, false.</returns>
|
||||
protected async Task<bool> StrikeAndCheckLimit(string hash, string itemName)
|
||||
{
|
||||
return await _striker.StrikeAndCheckLimit(hash, itemName, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled);
|
||||
}
|
||||
}
|
||||
@@ -1,18 +1,20 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
namespace Infrastructure.Verticals.DownloadClient;
|
||||
|
||||
public sealed class DummyDownloadService : DownloadServiceBase
|
||||
public sealed class DummyDownloadService : DownloadService
|
||||
{
|
||||
public DummyDownloadService(ILogger<DownloadServiceBase> logger, IOptions<QueueCleanerConfig> queueCleanerConfig, IOptions<ContentBlockerConfig> contentBlockerConfig, IMemoryCache cache, FilenameEvaluator filenameEvaluator, Striker striker) : base(logger, queueCleanerConfig, contentBlockerConfig, cache, filenameEvaluator, striker)
|
||||
public DummyDownloadService(ILogger<DownloadService> logger, IOptions<QueueCleanerConfig> queueCleanerConfig, IOptions<ContentBlockerConfig> contentBlockerConfig, IOptions<DownloadCleanerConfig> downloadCleanerConfig, IMemoryCache cache, IFilenameEvaluator filenameEvaluator, IStriker striker, NotificationPublisher notifier) : base(logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig, cache, filenameEvaluator, striker, notifier)
|
||||
{
|
||||
}
|
||||
|
||||
@@ -35,7 +37,17 @@ public sealed class DummyDownloadService : DownloadServiceBase
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
public override Task Delete(string hash)
|
||||
public override Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories)
|
||||
{
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
public override Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes)
|
||||
{
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
public override Task DeleteDownload(string hash)
|
||||
{
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Infrastructure.Interceptors;
|
||||
|
||||
namespace Infrastructure.Verticals.DownloadClient;
|
||||
|
||||
public interface IDownloadService : IDisposable
|
||||
public interface IDownloadService : IDisposable, IDryRunService
|
||||
{
|
||||
public Task LoginAsync();
|
||||
|
||||
@@ -29,8 +31,23 @@ public interface IDownloadService : IDisposable
|
||||
ConcurrentBag<Regex> regexes
|
||||
);
|
||||
|
||||
/// <summary>
|
||||
/// Fetches all downloads.
|
||||
/// </summary>
|
||||
/// <param name="categories">The categories by which to filter the downloads.</param>
|
||||
/// <returns>A list of downloads for the provided categories.</returns>
|
||||
Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories);
|
||||
|
||||
/// <summary>
|
||||
/// Cleans the downloads.
|
||||
/// </summary>
|
||||
/// <param name="downloads"></param>
|
||||
/// <param name="categoriesToClean">The categories that should be cleaned.</param>
|
||||
/// <param name="excludedHashes">The hashes that should not be cleaned.</param>
|
||||
public abstract Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes);
|
||||
|
||||
/// <summary>
|
||||
/// Deletes a download item.
|
||||
/// </summary>
|
||||
public Task Delete(string hash);
|
||||
public Task DeleteDownload(string hash);
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
namespace Infrastructure.Verticals.DownloadClient.QBittorrent;
|
||||
|
||||
public interface IQBitService : IDownloadService
|
||||
{
|
||||
}
|
||||
@@ -1,34 +1,46 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Attributes;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.DownloadClient;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Common.Helpers;
|
||||
using Domain.Enums;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.Context;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using QBittorrent.Client;
|
||||
using Category = Common.Configuration.DownloadCleaner.Category;
|
||||
|
||||
namespace Infrastructure.Verticals.DownloadClient.QBittorrent;
|
||||
|
||||
public sealed class QBitService : DownloadServiceBase
|
||||
public class QBitService : DownloadService, IQBitService
|
||||
{
|
||||
private readonly QBitConfig _config;
|
||||
private readonly QBittorrentClient _client;
|
||||
|
||||
/// <inheritdoc/>
|
||||
public QBitService()
|
||||
{
|
||||
}
|
||||
|
||||
public QBitService(
|
||||
ILogger<QBitService> logger,
|
||||
IHttpClientFactory httpClientFactory,
|
||||
IOptions<QBitConfig> config,
|
||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
||||
IOptions<DownloadCleanerConfig> downloadCleanerConfig,
|
||||
IMemoryCache cache,
|
||||
FilenameEvaluator filenameEvaluator,
|
||||
Striker striker
|
||||
) : base(logger, queueCleanerConfig, contentBlockerConfig, cache, filenameEvaluator, striker)
|
||||
IFilenameEvaluator filenameEvaluator,
|
||||
IStriker striker,
|
||||
NotificationPublisher notifier
|
||||
) : base(logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig, cache, filenameEvaluator, striker, notifier)
|
||||
{
|
||||
_config = config.Value;
|
||||
_config.Validate();
|
||||
@@ -188,17 +200,98 @@ public sealed class QBitService : DownloadServiceBase
|
||||
|
||||
foreach (int fileIndex in unwantedFiles)
|
||||
{
|
||||
await _client.SetFilePriorityAsync(hash, fileIndex, TorrentContentPriority.Skip);
|
||||
await ((QBitService)Proxy).SkipFile(hash, fileIndex);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories) =>
|
||||
(await _client.GetTorrentListAsync(new()
|
||||
{
|
||||
Filter = TorrentListFilter.Seeding
|
||||
}))
|
||||
?.Where(x => !string.IsNullOrEmpty(x.Hash))
|
||||
.Where(x => categories.Any(cat => cat.Name.Equals(x.Category, StringComparison.InvariantCultureIgnoreCase)))
|
||||
.Cast<object>()
|
||||
.ToList();
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task Delete(string hash)
|
||||
public override async Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes)
|
||||
{
|
||||
foreach (TorrentInfo download in downloads)
|
||||
{
|
||||
if (string.IsNullOrEmpty(download.Hash))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
Category? category = categoriesToClean
|
||||
.FirstOrDefault(x => download.Category.Equals(x.Name, StringComparison.InvariantCultureIgnoreCase));
|
||||
|
||||
if (category is null)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
if (excludedHashes.Any(x => x.Equals(download.Hash, StringComparison.InvariantCultureIgnoreCase)))
|
||||
{
|
||||
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!_downloadCleanerConfig.DeletePrivate)
|
||||
{
|
||||
TorrentProperties? torrentProperties = await _client.GetTorrentPropertiesAsync(download.Hash);
|
||||
|
||||
bool isPrivate = torrentProperties.AdditionalData.TryGetValue("is_private", out var dictValue) &&
|
||||
bool.TryParse(dictValue?.ToString(), out bool boolValue)
|
||||
&& boolValue;
|
||||
|
||||
if (isPrivate)
|
||||
{
|
||||
_logger.LogDebug("skip | download is private | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
ContextProvider.Set("downloadName", download.Name);
|
||||
ContextProvider.Set("hash", download.Hash);
|
||||
|
||||
SeedingCheckResult result = ShouldCleanDownload(download.Ratio, download.SeedingTime ?? TimeSpan.Zero, category);
|
||||
|
||||
if (!result.ShouldClean)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
await ((QBitService)Proxy).DeleteDownload(download.Hash);
|
||||
|
||||
_logger.LogInformation(
|
||||
"download cleaned | {reason} reached | {name}",
|
||||
result.Reason is CleanReason.MaxRatioReached
|
||||
? "MAX_RATIO & MIN_SEED_TIME"
|
||||
: "MAX_SEED_TIME",
|
||||
download.Name
|
||||
);
|
||||
|
||||
await _notifier.NotifyDownloadCleaned(download.Ratio, download.SeedingTime ?? TimeSpan.Zero, category.Name, result.Reason);
|
||||
}
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
[DryRunSafeguard]
|
||||
public override async Task DeleteDownload(string hash)
|
||||
{
|
||||
await _client.DeleteAsync(hash, deleteDownloadedData: true);
|
||||
}
|
||||
|
||||
[DryRunSafeguard]
|
||||
protected virtual async Task SkipFile(string hash, int fileIndex)
|
||||
{
|
||||
await _client.SetFilePriorityAsync(hash, fileIndex, TorrentContentPriority.Skip);
|
||||
}
|
||||
|
||||
public override void Dispose()
|
||||
{
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
using Domain.Enums;
|
||||
|
||||
namespace Infrastructure.Verticals.DownloadClient;
|
||||
|
||||
public sealed record SeedingCheckResult
|
||||
{
|
||||
public bool ShouldClean { get; set; }
|
||||
public CleanReason Reason { get; set; }
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
namespace Infrastructure.Verticals.DownloadClient.Transmission;
|
||||
|
||||
public interface ITransmissionService : IDownloadService
|
||||
{
|
||||
}
|
||||
@@ -1,12 +1,16 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Attributes;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.DownloadClient;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Common.Helpers;
|
||||
using Domain.Enums;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.Context;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
@@ -16,22 +20,29 @@ using Transmission.API.RPC.Entity;
|
||||
|
||||
namespace Infrastructure.Verticals.DownloadClient.Transmission;
|
||||
|
||||
public sealed class TransmissionService : DownloadServiceBase
|
||||
public class TransmissionService : DownloadService, ITransmissionService
|
||||
{
|
||||
private readonly TransmissionConfig _config;
|
||||
private readonly Client _client;
|
||||
private TorrentInfo[]? _torrentsCache;
|
||||
|
||||
/// <inheritdoc/>
|
||||
public TransmissionService()
|
||||
{
|
||||
}
|
||||
|
||||
public TransmissionService(
|
||||
IHttpClientFactory httpClientFactory,
|
||||
ILogger<TransmissionService> logger,
|
||||
IOptions<TransmissionConfig> config,
|
||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
||||
IOptions<DownloadCleanerConfig> downloadCleanerConfig,
|
||||
IMemoryCache cache,
|
||||
FilenameEvaluator filenameEvaluator,
|
||||
Striker striker
|
||||
) : base(logger, queueCleanerConfig, contentBlockerConfig, cache, filenameEvaluator, striker)
|
||||
IFilenameEvaluator filenameEvaluator,
|
||||
IStriker striker,
|
||||
NotificationPublisher notifier
|
||||
) : base(logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig, cache, filenameEvaluator, striker, notifier)
|
||||
{
|
||||
_config = config.Value;
|
||||
_config.Validate();
|
||||
@@ -164,16 +175,96 @@ public sealed class TransmissionService : DownloadServiceBase
|
||||
|
||||
_logger.LogDebug("changing priorities | torrent {hash}", hash);
|
||||
|
||||
await _client.TorrentSetAsync(new TorrentSettings
|
||||
{
|
||||
Ids = [ torrent.Id ],
|
||||
FilesUnwanted = unwantedFiles.ToArray(),
|
||||
});
|
||||
await ((TransmissionService)Proxy).SetUnwantedFiles(torrent.Id, unwantedFiles.ToArray());
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
public override async Task Delete(string hash)
|
||||
/// <inheritdoc/>
|
||||
public override async Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories)
|
||||
{
|
||||
string[] fields = [
|
||||
TorrentFields.FILES,
|
||||
TorrentFields.FILE_STATS,
|
||||
TorrentFields.HASH_STRING,
|
||||
TorrentFields.ID,
|
||||
TorrentFields.ETA,
|
||||
TorrentFields.NAME,
|
||||
TorrentFields.STATUS,
|
||||
TorrentFields.IS_PRIVATE,
|
||||
TorrentFields.DOWNLOADED_EVER,
|
||||
TorrentFields.DOWNLOAD_DIR,
|
||||
TorrentFields.SECONDS_SEEDING,
|
||||
TorrentFields.UPLOAD_RATIO
|
||||
];
|
||||
|
||||
return (await _client.TorrentGetAsync(fields))
|
||||
?.Torrents
|
||||
?.Where(x => !string.IsNullOrEmpty(x.HashString))
|
||||
.Where(x => x.Status is 5 or 6)
|
||||
.Where(x => categories
|
||||
.Any(cat => x.DownloadDir?.EndsWith(cat.Name, StringComparison.InvariantCultureIgnoreCase) is true)
|
||||
)
|
||||
.Cast<object>()
|
||||
.ToList();
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes)
|
||||
{
|
||||
foreach (TorrentInfo download in downloads)
|
||||
{
|
||||
if (string.IsNullOrEmpty(download.HashString))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
Category? category = categoriesToClean
|
||||
.FirstOrDefault(x => download.DownloadDir?.EndsWith(x.Name, StringComparison.InvariantCultureIgnoreCase) is true);
|
||||
|
||||
if (category is null)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
if (excludedHashes.Any(x => x.Equals(download.HashString, StringComparison.InvariantCultureIgnoreCase)))
|
||||
{
|
||||
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!_downloadCleanerConfig.DeletePrivate && download.IsPrivate is true)
|
||||
{
|
||||
_logger.LogDebug("skip | download is private | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
ContextProvider.Set("downloadName", download.Name);
|
||||
ContextProvider.Set("hash", download.HashString);
|
||||
|
||||
TimeSpan seedingTime = TimeSpan.FromSeconds(download.SecondsSeeding ?? 0);
|
||||
SeedingCheckResult result = ShouldCleanDownload(download.uploadRatio ?? 0, seedingTime, category);
|
||||
|
||||
if (!result.ShouldClean)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
await ((TransmissionService)Proxy).RemoveDownloadAsync(download.Id);
|
||||
|
||||
_logger.LogInformation(
|
||||
"download cleaned | {reason} reached | {name}",
|
||||
result.Reason is CleanReason.MaxRatioReached
|
||||
? "MAX_RATIO & MIN_SEED_TIME"
|
||||
: "MAX_SEED_TIME",
|
||||
download.Name
|
||||
);
|
||||
|
||||
await _notifier.NotifyDownloadCleaned(download.uploadRatio ?? 0, seedingTime, category.Name, result.Reason);
|
||||
}
|
||||
}
|
||||
|
||||
public override async Task DeleteDownload(string hash)
|
||||
{
|
||||
TorrentInfo? torrent = await GetTorrentAsync(hash);
|
||||
|
||||
@@ -189,6 +280,22 @@ public sealed class TransmissionService : DownloadServiceBase
|
||||
{
|
||||
}
|
||||
|
||||
[DryRunSafeguard]
|
||||
protected virtual async Task RemoveDownloadAsync(long downloadId)
|
||||
{
|
||||
await _client.TorrentRemoveAsync([downloadId], true);
|
||||
}
|
||||
|
||||
[DryRunSafeguard]
|
||||
protected virtual async Task SetUnwantedFiles(long downloadId, long[] unwantedFiles)
|
||||
{
|
||||
await _client.TorrentSetAsync(new TorrentSettings
|
||||
{
|
||||
Ids = [downloadId],
|
||||
FilesUnwanted = unwantedFiles,
|
||||
});
|
||||
}
|
||||
|
||||
private async Task<bool> IsItemStuckAndShouldRemove(TorrentInfo torrent)
|
||||
{
|
||||
if (_queueCleanerConfig.StalledMaxStrikes is 0)
|
||||
|
||||
8
code/Infrastructure/Verticals/ItemStriker/IStriker.cs
Normal file
8
code/Infrastructure/Verticals/ItemStriker/IStriker.cs
Normal file
@@ -0,0 +1,8 @@
|
||||
using Domain.Enums;
|
||||
|
||||
namespace Infrastructure.Verticals.ItemStriker;
|
||||
|
||||
public interface IStriker
|
||||
{
|
||||
Task<bool> StrikeAndCheckLimit(string hash, string itemName, ushort maxStrikes, StrikeType strikeType);
|
||||
}
|
||||
@@ -8,7 +8,7 @@ using Microsoft.Extensions.Logging;
|
||||
|
||||
namespace Infrastructure.Verticals.ItemStriker;
|
||||
|
||||
public class Striker
|
||||
public sealed class Striker : IStriker
|
||||
{
|
||||
private readonly ILogger<Striker> _logger;
|
||||
private readonly IMemoryCache _cache;
|
||||
|
||||
@@ -4,6 +4,7 @@ using Domain.Enums;
|
||||
using Domain.Models.Arr;
|
||||
using Domain.Models.Arr.Queue;
|
||||
using Infrastructure.Verticals.Arr;
|
||||
using Infrastructure.Verticals.Arr.Interfaces;
|
||||
using Infrastructure.Verticals.DownloadClient;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Logging;
|
||||
@@ -11,16 +12,16 @@ using Microsoft.Extensions.Options;
|
||||
|
||||
namespace Infrastructure.Verticals.Jobs;
|
||||
|
||||
public abstract class GenericHandler : IDisposable
|
||||
public abstract class GenericHandler : IHandler, IDisposable
|
||||
{
|
||||
protected readonly ILogger<GenericHandler> _logger;
|
||||
protected readonly DownloadClientConfig _downloadClientConfig;
|
||||
protected readonly SonarrConfig _sonarrConfig;
|
||||
protected readonly RadarrConfig _radarrConfig;
|
||||
protected readonly LidarrConfig _lidarrConfig;
|
||||
protected readonly SonarrClient _sonarrClient;
|
||||
protected readonly RadarrClient _radarrClient;
|
||||
protected readonly LidarrClient _lidarrClient;
|
||||
protected readonly ISonarrClient _sonarrClient;
|
||||
protected readonly IRadarrClient _radarrClient;
|
||||
protected readonly ILidarrClient _lidarrClient;
|
||||
protected readonly ArrQueueIterator _arrArrQueueIterator;
|
||||
protected readonly IDownloadService _downloadService;
|
||||
protected readonly NotificationPublisher _notifier;
|
||||
@@ -31,9 +32,9 @@ public abstract class GenericHandler : IDisposable
|
||||
IOptions<SonarrConfig> sonarrConfig,
|
||||
IOptions<RadarrConfig> radarrConfig,
|
||||
IOptions<LidarrConfig> lidarrConfig,
|
||||
SonarrClient sonarrClient,
|
||||
RadarrClient radarrClient,
|
||||
LidarrClient lidarrClient,
|
||||
ISonarrClient sonarrClient,
|
||||
IRadarrClient radarrClient,
|
||||
ILidarrClient lidarrClient,
|
||||
ArrQueueIterator arrArrQueueIterator,
|
||||
DownloadServiceFactory downloadServiceFactory,
|
||||
NotificationPublisher notifier
|
||||
@@ -68,7 +69,7 @@ public abstract class GenericHandler : IDisposable
|
||||
|
||||
protected abstract Task ProcessInstanceAsync(ArrInstance instance, InstanceType instanceType);
|
||||
|
||||
private async Task ProcessArrConfigAsync(ArrConfig config, InstanceType instanceType)
|
||||
protected async Task ProcessArrConfigAsync(ArrConfig config, InstanceType instanceType, bool throwOnFailure = false)
|
||||
{
|
||||
if (!config.Enabled)
|
||||
{
|
||||
@@ -84,11 +85,16 @@ public abstract class GenericHandler : IDisposable
|
||||
catch (Exception exception)
|
||||
{
|
||||
_logger.LogError(exception, "failed to clean {type} instance | {url}", instanceType, arrInstance.Url);
|
||||
|
||||
if (throwOnFailure)
|
||||
{
|
||||
throw;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
protected ArrClient GetClient(InstanceType type) =>
|
||||
protected IArrClient GetClient(InstanceType type) =>
|
||||
type switch
|
||||
{
|
||||
InstanceType.Sonarr => _sonarrClient,
|
||||
|
||||
6
code/Infrastructure/Verticals/Jobs/IHandler.cs
Normal file
6
code/Infrastructure/Verticals/Jobs/IHandler.cs
Normal file
@@ -0,0 +1,6 @@
|
||||
namespace Infrastructure.Verticals.Jobs;
|
||||
|
||||
public interface IHandler
|
||||
{
|
||||
Task ExecuteAsync();
|
||||
}
|
||||
@@ -27,9 +27,12 @@ public sealed class NotificationConsumer<T> : IConsumer<T> where T : Notificatio
|
||||
case StalledStrikeNotification stalledMessage:
|
||||
await _notificationService.Notify(stalledMessage);
|
||||
break;
|
||||
case QueueItemDeleteNotification queueItemDeleteMessage:
|
||||
case QueueItemDeletedNotification queueItemDeleteMessage:
|
||||
await _notificationService.Notify(queueItemDeleteMessage);
|
||||
break;
|
||||
case DownloadCleanedNotification downloadCleanedNotification:
|
||||
await _notificationService.Notify(downloadCleanedNotification);
|
||||
break;
|
||||
default:
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
@@ -6,5 +6,7 @@ public interface INotificationFactory
|
||||
|
||||
List<INotificationProvider> OnStalledStrikeEnabled();
|
||||
|
||||
List<INotificationProvider> OnQueueItemDeleteEnabled();
|
||||
List<INotificationProvider> OnQueueItemDeletedEnabled();
|
||||
|
||||
List<INotificationProvider> OnDownloadCleanedEnabled();
|
||||
}
|
||||
@@ -13,5 +13,7 @@ public interface INotificationProvider
|
||||
|
||||
Task OnStalledStrike(StalledStrikeNotification notification);
|
||||
|
||||
Task OnQueueItemDelete(QueueItemDeleteNotification notification);
|
||||
Task OnQueueItemDeleted(QueueItemDeletedNotification notification);
|
||||
|
||||
Task OnDownloadCleaned(DownloadCleanedNotification notification);
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
using Domain.Enums;
|
||||
|
||||
namespace Infrastructure.Verticals.Notifications.Models;
|
||||
|
||||
public record ArrNotification : Notification
|
||||
{
|
||||
public required InstanceType InstanceType { get; init; }
|
||||
|
||||
public required Uri InstanceUrl { get; init; }
|
||||
|
||||
public required string Hash { get; init; }
|
||||
|
||||
public Uri? Image { get; init; }
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
namespace Infrastructure.Verticals.Notifications.Models;
|
||||
|
||||
public sealed record DownloadCleanedNotification : Notification
|
||||
{
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
namespace Infrastructure.Verticals.Notifications.Models;
|
||||
|
||||
public sealed record FailedImportStrikeNotification : Notification
|
||||
public sealed record FailedImportStrikeNotification : ArrNotification
|
||||
{
|
||||
}
|
||||
@@ -1,20 +1,12 @@
|
||||
using Domain.Enums;
|
||||
namespace Infrastructure.Verticals.Notifications.Models;
|
||||
|
||||
namespace Infrastructure.Verticals.Notifications.Models;
|
||||
|
||||
public record Notification
|
||||
public abstract record Notification
|
||||
{
|
||||
public required InstanceType InstanceType { get; init; }
|
||||
|
||||
public required Uri InstanceUrl { get; init; }
|
||||
|
||||
public required string Hash { get; init; }
|
||||
|
||||
public required string Title { get; init; }
|
||||
|
||||
public required string Description { get; init; }
|
||||
|
||||
public Uri? Image { get; init; }
|
||||
|
||||
public List<NotificationField>? Fields { get; init; }
|
||||
|
||||
public NotificationLevel Level { get; init; }
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
namespace Infrastructure.Verticals.Notifications.Models;
|
||||
|
||||
public enum NotificationLevel
|
||||
{
|
||||
Test,
|
||||
Information,
|
||||
Warning,
|
||||
Important
|
||||
}
|
||||
@@ -1,5 +0,0 @@
|
||||
namespace Infrastructure.Verticals.Notifications.Models;
|
||||
|
||||
public sealed record QueueItemDeleteNotification : Notification
|
||||
{
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
namespace Infrastructure.Verticals.Notifications.Models;
|
||||
|
||||
public sealed record QueueItemDeletedNotification : ArrNotification
|
||||
{
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
namespace Infrastructure.Verticals.Notifications.Models;
|
||||
|
||||
public sealed record StalledStrikeNotification : Notification
|
||||
public sealed record StalledStrikeNotification : ArrNotification
|
||||
{
|
||||
}
|
||||
@@ -1,4 +1,3 @@
|
||||
using Domain.Enums;
|
||||
using Infrastructure.Verticals.Notifications.Models;
|
||||
using Mapster;
|
||||
using Microsoft.Extensions.Options;
|
||||
@@ -12,6 +11,7 @@ public class NotifiarrProvider : NotificationProvider
|
||||
|
||||
private const string WarningColor = "f0ad4e";
|
||||
private const string ImportantColor = "bb2124";
|
||||
private const string Logo = "https://github.com/flmorg/cleanuperr/blob/main/Logo/48.png?raw=true";
|
||||
|
||||
public NotifiarrProvider(IOptions<NotifiarrConfig> config, INotifiarrProxy proxy)
|
||||
: base(config)
|
||||
@@ -32,12 +32,17 @@ public class NotifiarrProvider : NotificationProvider
|
||||
await _proxy.SendNotification(BuildPayload(notification, WarningColor), _config);
|
||||
}
|
||||
|
||||
public override async Task OnQueueItemDelete(QueueItemDeleteNotification notification)
|
||||
public override async Task OnQueueItemDeleted(QueueItemDeletedNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification, ImportantColor), _config);
|
||||
}
|
||||
|
||||
private NotifiarrPayload BuildPayload(Notification notification, string color)
|
||||
public override async Task OnDownloadCleaned(DownloadCleanedNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification), _config);
|
||||
}
|
||||
|
||||
private NotifiarrPayload BuildPayload(ArrNotification notification, string color)
|
||||
{
|
||||
NotifiarrPayload payload = new()
|
||||
{
|
||||
@@ -47,7 +52,7 @@ public class NotifiarrProvider : NotificationProvider
|
||||
Text = new()
|
||||
{
|
||||
Title = notification.Title,
|
||||
Icon = "https://github.com/flmorg/cleanuperr/blob/main/Logo/48.png?raw=true",
|
||||
Icon = Logo,
|
||||
Description = notification.Description,
|
||||
Fields = new()
|
||||
{
|
||||
@@ -62,7 +67,7 @@ public class NotifiarrProvider : NotificationProvider
|
||||
},
|
||||
Images = new()
|
||||
{
|
||||
Thumbnail = new Uri("https://github.com/flmorg/cleanuperr/blob/main/Logo/48.png?raw=true"),
|
||||
Thumbnail = new Uri(Logo),
|
||||
Image = notification.Image
|
||||
}
|
||||
}
|
||||
@@ -72,4 +77,32 @@ public class NotifiarrProvider : NotificationProvider
|
||||
|
||||
return payload;
|
||||
}
|
||||
|
||||
private NotifiarrPayload BuildPayload(DownloadCleanedNotification notification)
|
||||
{
|
||||
NotifiarrPayload payload = new()
|
||||
{
|
||||
Discord = new()
|
||||
{
|
||||
Color = ImportantColor,
|
||||
Text = new()
|
||||
{
|
||||
Title = notification.Title,
|
||||
Icon = Logo,
|
||||
Description = notification.Description,
|
||||
Fields = notification.Fields?.Adapt<List<Field>>() ?? []
|
||||
},
|
||||
Ids = new Ids
|
||||
{
|
||||
Channel = _config.ChannelId
|
||||
},
|
||||
Images = new()
|
||||
{
|
||||
Thumbnail = new Uri(Logo)
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
return payload;
|
||||
}
|
||||
}
|
||||
@@ -25,8 +25,13 @@ public class NotificationFactory : INotificationFactory
|
||||
.Where(n => n.Config.OnStalledStrike)
|
||||
.ToList();
|
||||
|
||||
public List<INotificationProvider> OnQueueItemDeleteEnabled() =>
|
||||
public List<INotificationProvider> OnQueueItemDeletedEnabled() =>
|
||||
ActiveProviders()
|
||||
.Where(n => n.Config.OnQueueItemDelete)
|
||||
.Where(n => n.Config.OnQueueItemDeleted)
|
||||
.ToList();
|
||||
|
||||
public List<INotificationProvider> OnDownloadCleanedEnabled() =>
|
||||
ActiveProviders()
|
||||
.Where(n => n.Config.OnDownloadCleaned)
|
||||
.ToList();
|
||||
}
|
||||
@@ -19,5 +19,7 @@ public abstract class NotificationProvider : INotificationProvider
|
||||
|
||||
public abstract Task OnStalledStrike(StalledStrikeNotification notification);
|
||||
|
||||
public abstract Task OnQueueItemDelete(QueueItemDeleteNotification notification);
|
||||
public abstract Task OnQueueItemDeleted(QueueItemDeletedNotification notification);
|
||||
|
||||
public abstract Task OnDownloadCleaned(DownloadCleanedNotification notification);
|
||||
}
|
||||
@@ -1,6 +1,9 @@
|
||||
using Common.Configuration.Arr;
|
||||
using System.Globalization;
|
||||
using Common.Attributes;
|
||||
using Common.Configuration.Arr;
|
||||
using Domain.Enums;
|
||||
using Domain.Models.Arr.Queue;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Verticals.Context;
|
||||
using Infrastructure.Verticals.Notifications.Models;
|
||||
using Mapster;
|
||||
@@ -9,27 +12,35 @@ using Microsoft.Extensions.Logging;
|
||||
|
||||
namespace Infrastructure.Verticals.Notifications;
|
||||
|
||||
public sealed class NotificationPublisher
|
||||
public class NotificationPublisher : InterceptedService, IDryRunService
|
||||
{
|
||||
private readonly ILogger<NotificationPublisher> _logger;
|
||||
private readonly IBus _messageBus;
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Constructor to be used by interceptors.
|
||||
/// </summary>
|
||||
public NotificationPublisher()
|
||||
{
|
||||
}
|
||||
|
||||
public NotificationPublisher(ILogger<NotificationPublisher> logger, IBus messageBus)
|
||||
{
|
||||
_logger = logger;
|
||||
_messageBus = messageBus;
|
||||
}
|
||||
|
||||
public async Task NotifyStrike(StrikeType strikeType, int strikeCount)
|
||||
[DryRunSafeguard]
|
||||
public virtual async Task NotifyStrike(StrikeType strikeType, int strikeCount)
|
||||
{
|
||||
try
|
||||
{
|
||||
QueueRecord record = GetRecordFromContext();
|
||||
InstanceType instanceType = GetInstanceTypeFromContext();
|
||||
Uri instanceUrl = GetInstanceUrlFromContext();
|
||||
Uri? imageUrl = GetImageFromContext(record, instanceType);
|
||||
QueueRecord record = ContextProvider.Get<QueueRecord>(nameof(QueueRecord));
|
||||
InstanceType instanceType = (InstanceType)ContextProvider.Get<object>(nameof(InstanceType));
|
||||
Uri instanceUrl = ContextProvider.Get<Uri>(nameof(ArrInstance) + nameof(ArrInstance.Url));
|
||||
Uri imageUrl = GetImageFromContext(record, instanceType);
|
||||
|
||||
Notification notification = new()
|
||||
ArrNotification notification = new()
|
||||
{
|
||||
InstanceType = instanceType,
|
||||
InstanceUrl = instanceUrl,
|
||||
@@ -56,14 +67,15 @@ public sealed class NotificationPublisher
|
||||
}
|
||||
}
|
||||
|
||||
public async Task NotifyQueueItemDelete(bool removeFromClient, DeleteReason reason)
|
||||
[DryRunSafeguard]
|
||||
public virtual async Task NotifyQueueItemDeleted(bool removeFromClient, DeleteReason reason)
|
||||
{
|
||||
QueueRecord record = GetRecordFromContext();
|
||||
InstanceType instanceType = GetInstanceTypeFromContext();
|
||||
Uri instanceUrl = GetInstanceUrlFromContext();
|
||||
Uri? imageUrl = GetImageFromContext(record, instanceType);
|
||||
QueueRecord record = ContextProvider.Get<QueueRecord>(nameof(QueueRecord));
|
||||
InstanceType instanceType = (InstanceType)ContextProvider.Get<object>(nameof(InstanceType));
|
||||
Uri instanceUrl = ContextProvider.Get<Uri>(nameof(ArrInstance) + nameof(ArrInstance.Url));
|
||||
Uri imageUrl = GetImageFromContext(record, instanceType);
|
||||
|
||||
Notification notification = new()
|
||||
QueueItemDeletedNotification notification = new()
|
||||
{
|
||||
InstanceType = instanceType,
|
||||
InstanceUrl = instanceUrl,
|
||||
@@ -74,20 +86,29 @@ public sealed class NotificationPublisher
|
||||
Fields = [new() { Title = "Removed from download client?", Text = removeFromClient ? "Yes" : "No" }]
|
||||
};
|
||||
|
||||
await _messageBus.Publish(notification.Adapt<QueueItemDeleteNotification>());
|
||||
await _messageBus.Publish(notification);
|
||||
}
|
||||
|
||||
private static QueueRecord GetRecordFromContext() =>
|
||||
ContextProvider.Get<QueueRecord>(nameof(QueueRecord)) ?? throw new Exception("failed to get record from context");
|
||||
[DryRunSafeguard]
|
||||
public virtual async Task NotifyDownloadCleaned(double ratio, TimeSpan seedingTime, string categoryName, CleanReason reason)
|
||||
{
|
||||
DownloadCleanedNotification notification = new()
|
||||
{
|
||||
Title = $"Cleaned item from download client with reason: {reason}",
|
||||
Description = ContextProvider.Get<string>("downloadName"),
|
||||
Fields =
|
||||
[
|
||||
new() { Title = "Hash", Text = ContextProvider.Get<string>("hash").ToLowerInvariant() },
|
||||
new() { Title = "Category", Text = categoryName.ToLowerInvariant() },
|
||||
new() { Title = "Ratio", Text = $"{ratio.ToString(CultureInfo.InvariantCulture)}%" },
|
||||
new() { Title = "Seeding hours", Text = $"{Math.Round(seedingTime.TotalHours, 0).ToString(CultureInfo.InvariantCulture)}h" }
|
||||
],
|
||||
Level = NotificationLevel.Important
|
||||
};
|
||||
|
||||
await _messageBus.Publish(notification);
|
||||
}
|
||||
|
||||
private static InstanceType GetInstanceTypeFromContext() =>
|
||||
(InstanceType)(ContextProvider.Get<object>(nameof(InstanceType)) ??
|
||||
throw new Exception("failed to get instance type from context"));
|
||||
|
||||
private static Uri GetInstanceUrlFromContext() =>
|
||||
ContextProvider.Get<Uri>(nameof(ArrInstance) + nameof(ArrInstance.Url)) ??
|
||||
throw new Exception("failed to get instance url from context");
|
||||
|
||||
private static Uri GetImageFromContext(QueueRecord record, InstanceType instanceType) =>
|
||||
instanceType switch
|
||||
{
|
||||
|
||||
@@ -44,13 +44,28 @@ public class NotificationService
|
||||
}
|
||||
}
|
||||
|
||||
public async Task Notify(QueueItemDeleteNotification notification)
|
||||
public async Task Notify(QueueItemDeletedNotification notification)
|
||||
{
|
||||
foreach (INotificationProvider provider in _notificationFactory.OnQueueItemDeleteEnabled())
|
||||
foreach (INotificationProvider provider in _notificationFactory.OnQueueItemDeletedEnabled())
|
||||
{
|
||||
try
|
||||
{
|
||||
await provider.OnQueueItemDelete(notification);
|
||||
await provider.OnQueueItemDeleted(notification);
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
_logger.LogWarning(exception, "failed to send notification | provider {provider}", provider.Name);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public async Task Notify(DownloadCleanedNotification notification)
|
||||
{
|
||||
foreach (INotificationProvider provider in _notificationFactory.OnDownloadCleanedEnabled())
|
||||
{
|
||||
try
|
||||
{
|
||||
await provider.OnDownloadCleaned(notification);
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
|
||||
@@ -5,6 +5,7 @@ using Domain.Enums;
|
||||
using Domain.Models.Arr;
|
||||
using Domain.Models.Arr.Queue;
|
||||
using Infrastructure.Verticals.Arr;
|
||||
using Infrastructure.Verticals.Arr.Interfaces;
|
||||
using Infrastructure.Verticals.Context;
|
||||
using Infrastructure.Verticals.DownloadClient;
|
||||
using Infrastructure.Verticals.Jobs;
|
||||
@@ -48,7 +49,7 @@ public sealed class QueueCleaner : GenericHandler
|
||||
using var _ = LogContext.PushProperty("InstanceName", instanceType.ToString());
|
||||
|
||||
HashSet<SearchItem> itemsToBeRefreshed = [];
|
||||
ArrClient arrClient = GetClient(instanceType);
|
||||
IArrClient arrClient = GetClient(instanceType);
|
||||
|
||||
// push to context
|
||||
ContextProvider.Set(nameof(ArrInstance) + nameof(ArrInstance.Url), instance.Url);
|
||||
@@ -113,7 +114,7 @@ public sealed class QueueCleaner : GenericHandler
|
||||
}
|
||||
|
||||
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient);
|
||||
await _notifier.NotifyQueueItemDelete(removeFromClient, deleteReason);
|
||||
await _notifier.NotifyQueueItemDeleted(removeFromClient, deleteReason);
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
@@ -8,6 +8,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Infrastructure", "Infrastru
|
||||
EndProject
|
||||
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Domain", "Domain\Domain.csproj", "{8871592A-B260-4B15-8EF8-6AB24480DE5D}"
|
||||
EndProject
|
||||
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Infrastructure.Tests", "Infrastructure.Tests\Infrastructure.Tests.csproj", "{F2DBB5FD-8D93-45D0-B211-5E9A8C14B684}"
|
||||
EndProject
|
||||
Global
|
||||
GlobalSection(SolutionConfigurationPlatforms) = preSolution
|
||||
Debug|Any CPU = Debug|Any CPU
|
||||
@@ -30,5 +32,9 @@ Global
|
||||
{8871592A-B260-4B15-8EF8-6AB24480DE5D}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{8871592A-B260-4B15-8EF8-6AB24480DE5D}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
{8871592A-B260-4B15-8EF8-6AB24480DE5D}.Release|Any CPU.Build.0 = Release|Any CPU
|
||||
{F2DBB5FD-8D93-45D0-B211-5E9A8C14B684}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{F2DBB5FD-8D93-45D0-B211-5E9A8C14B684}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{F2DBB5FD-8D93-45D0-B211-5E9A8C14B684}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
{F2DBB5FD-8D93-45D0-B211-5E9A8C14B684}.Release|Any CPU.Build.0 = Release|Any CPU
|
||||
EndGlobalSection
|
||||
EndGlobal
|
||||
|
||||
@@ -1,2 +0,0 @@
|
||||
{
|
||||
}
|
||||
@@ -148,6 +148,8 @@ services:
|
||||
- ./data/lidarr/config:/config
|
||||
- ./data/lidarr/music:/music
|
||||
- ./data/qbittorrent/downloads:/downloads
|
||||
# - ./data/deluge/downloads:/downloads
|
||||
# - ./data/transmission/downloads:/downloads
|
||||
ports:
|
||||
- 8686:8686
|
||||
restart: unless-stopped
|
||||
@@ -163,6 +165,8 @@ services:
|
||||
- ./data/readarr/config:/config
|
||||
- ./data/readarr/books:/books
|
||||
- ./data/qbittorrent/downloads:/downloads
|
||||
# - ./data/deluge/downloads:/downloads
|
||||
# - ./data/transmission/downloads:/downloads
|
||||
ports:
|
||||
- 8787:8787
|
||||
restart: unless-stopped
|
||||
@@ -171,6 +175,8 @@ services:
|
||||
image: ghcr.io/flmorg/cleanuperr:latest
|
||||
container_name: cleanuperr
|
||||
environment:
|
||||
- DRY_RUN=false
|
||||
|
||||
- LOGGING__LOGLEVEL=Debug
|
||||
- LOGGING__FILE__ENABLED=true
|
||||
- LOGGING__FILE__PATH=/var/logs
|
||||
@@ -181,6 +187,7 @@ services:
|
||||
|
||||
- TRIGGERS__QUEUECLEANER=0/30 * * * * ?
|
||||
- TRIGGERS__CONTENTBLOCKER=0/30 * * * * ?
|
||||
- TRIGGERS__DOWNLOADCLEANER=0/30 * * * * ?
|
||||
|
||||
- QUEUECLEANER__ENABLED=true
|
||||
- QUEUECLEANER__RUNSEQUENTIALLY=true
|
||||
@@ -196,17 +203,28 @@ services:
|
||||
- CONTENTBLOCKER__IGNORE_PRIVATE=true
|
||||
- CONTENTBLOCKER__DELETE_PRIVATE=false
|
||||
|
||||
- DOWNLOADCLEANER__ENABLED=true
|
||||
- DOWNLOADCLEANER__DELETE_PRIVATE=false
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__NAME=tv-sonarr
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MAX_RATIO=-1
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MIN_SEED_TIME=0
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MAX_SEED_TIME=0.01
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__NAME=radarr
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MAX_RATIO=-1
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MIN_SEED_TIME=0
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MAX_SEED_TIME=0.01
|
||||
|
||||
- DOWNLOAD_CLIENT=qbittorrent
|
||||
- QBITTORRENT__URL=http://qbittorrent:8080
|
||||
- QBITTORRENT__USERNAME=test
|
||||
- QBITTORRENT__PASSWORD=testing
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=deluge
|
||||
# - DELUGE__URL=http://localhost:8112
|
||||
# - DELUGE__URL=http://deluge:8112
|
||||
# - DELUGE__PASSWORD=testing
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=transmission
|
||||
# - TRANSMISSION__URL=http://localhost:9091
|
||||
# - TRANSMISSION__URL=http://transmission:9091
|
||||
# - TRANSMISSION__USERNAME=test
|
||||
# - TRANSMISSION__PASSWORD=testing
|
||||
|
||||
@@ -231,7 +249,7 @@ services:
|
||||
|
||||
# - NOTIFIARR__ON_IMPORT_FAILED_STRIKE=true
|
||||
# - NOTIFIARR__ON_STALLED_STRIKE=true
|
||||
# - NOTIFIARR__ON_QUEUE_ITEM_DELETE=true
|
||||
# - NOTIFIARR__ON_QUEUE_ITEM_DELETED=true
|
||||
# - NOTIFIARR__API_KEY=notifiarr_secret
|
||||
# - NOTIFIARR__CHANNEL_ID=discord_channel_id
|
||||
volumes:
|
||||
|
||||
525
variables.md
525
variables.md
@@ -1,11 +1,524 @@
|
||||
## LOGGING__ENHANCED
|
||||
## Table of contents
|
||||
- [General settings](variables.md#general-settings)
|
||||
- [Queue Cleaner settings](variables.md#queue-cleaner-settings)
|
||||
- [Content Blocker settings](variables.md#content-blocker-settings)
|
||||
- [Download Cleaner settings](variables.md#download-cleaner-settings)
|
||||
- [Download Client settings](variables.md#download-client-settings)
|
||||
- [Arr settings](variables.md#arr-settings)
|
||||
- [Notification settings](variables.md#notification-settings)
|
||||
- [Advanced settings](variables.md#advanced-settings)
|
||||
|
||||
Some logs may contain information that is hard to read. Enhancing these logs usually comes with the cost of additional calls to the APIs.
|
||||
#
|
||||
|
||||
If enabled, logs like this
|
||||
### General settings
|
||||
|
||||
```movie search triggered | http://localhost:7878/ | movie ids: 1, 2```
|
||||
**`DRY_RUN`**
|
||||
- When enabled, simulates irreversible operations (like deletions and notifications) without making actual changes.
|
||||
- Type: Boolean.
|
||||
- Possible values: `true`, `false`.
|
||||
- Default: `false`.
|
||||
- Required: No.
|
||||
|
||||
will transform into
|
||||
**`LOGGING__LOGLEVEL`**
|
||||
- Controls the detail level of application logs.
|
||||
- Type: String.
|
||||
- Possible values: `Verbose`, `Debug`, `Information`, `Warning`, `Error`, `Fatal`.
|
||||
- Default: `Information`.
|
||||
- Required: No.
|
||||
|
||||
```movie search triggered | http://localhost:7878/ | [Speak No Evil][The Wild Robot]```
|
||||
**`LOGGING__FILE__ENABLED`**
|
||||
- Enables logging to a file.
|
||||
- Type: Boolean.
|
||||
- Possible values: `true`, `false`.
|
||||
- Default: `false`.
|
||||
- Required: No.
|
||||
|
||||
**`LOGGING__FILE__PATH`**
|
||||
- Directory where log files will be saved.
|
||||
- Type: String.
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
**`LOGGING__ENHANCED`**
|
||||
- Provides more detailed descriptions in logs whenever possible.
|
||||
- Type: Boolean.
|
||||
- Possible values: `true`, `false`.
|
||||
- Default: `true`.
|
||||
- Required: No.
|
||||
</details>
|
||||
|
||||
#
|
||||
|
||||
### Queue Cleaner settings
|
||||
|
||||
**`TRIGGERS__QUEUECLEANER`**
|
||||
- Cron schedule for the queue cleaner job.
|
||||
- Type: String - [Quartz cron format](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html).
|
||||
- Default: `0 0/5 * * * ?` (every 5 minutes).
|
||||
- Required: Yes if queue cleaner is enabled.
|
||||
|
||||
> [!NOTE]
|
||||
> - Maximum interval is 6 hours.
|
||||
> - Ignored if `QUEUECLEANER__RUNSEQUENTIALLY=true` and `CONTENTBLOCKER__ENABLED=true`.
|
||||
|
||||
**`QUEUECLEANER__ENABLED`**
|
||||
- Enables or disables the queue cleaning functionality.
|
||||
- When enabled, processes all items in the *arr queue.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `true`
|
||||
- Required: No.
|
||||
|
||||
**`QUEUECLEANER__RUNSEQUENTIALLY`**
|
||||
- Controls whether queue cleaner runs after content blocker instead of in parallel.
|
||||
- When `true`, streamlines the cleaning process by running immediately after content blocker.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `true`
|
||||
- Required: No.
|
||||
|
||||
**`QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES`**
|
||||
- Number of strikes before removing a failed import.
|
||||
- Set to `0` to never remove failed imports.
|
||||
- A strike is given when an item is stalled, stuck in metadata downloading, or failed to be imported.
|
||||
- Type: Integer
|
||||
- Possible values: `0` or greater
|
||||
- Default: `0`
|
||||
- Required: No.
|
||||
|
||||
**`QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE`**
|
||||
- Controls whether to ignore failed imports from private trackers.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
**`QUEUECLEANER__IMPORT_FAILED_DELETE_PRIVATE`**
|
||||
- Controls whether to delete failed imports from private trackers from the download client.
|
||||
- Has no effect if `QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE` is `true`.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
> [!WARNING]
|
||||
> Setting `QUEUECLEANER__IMPORT_FAILED_DELETE_PRIVATE=true` means you don't care about seeding, ratio, H&R and potentially losing your tracker account.
|
||||
|
||||
**`QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS`**
|
||||
- Patterns to look for in failed import messages that should be ignored.
|
||||
- Multiple patterns can be specified using incrementing numbers starting from 0.
|
||||
- Type: String array
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
- Example:
|
||||
```yaml
|
||||
QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__0: "title mismatch"
|
||||
QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__1: "manual import required"
|
||||
```
|
||||
|
||||
**`QUEUECLEANER__STALLED_MAX_STRIKES`**
|
||||
- Number of strikes before removing a stalled download.
|
||||
- Set to `0` to never remove stalled downloads.
|
||||
- A strike is given when download speed is 0.
|
||||
- Type: Integer
|
||||
- Possible values: `0` or greater
|
||||
- Default: `0`
|
||||
- Required: No.
|
||||
|
||||
**`QUEUECLEANER__STALLED_RESET_STRIKES_ON_PROGRESS`**
|
||||
- Controls whether to remove strikes if any download progress was made since last checked.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
**`QUEUECLEANER__STALLED_IGNORE_PRIVATE`**
|
||||
- Controls whether to ignore stalled downloads from private trackers.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
**`QUEUECLEANER__STALLED_DELETE_PRIVATE`**
|
||||
- Controls whether to delete stalled private downloads from the download client.
|
||||
- Has no effect if `QUEUECLEANER__STALLED_IGNORE_PRIVATE` is `true`.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
> [!WARNING]
|
||||
> Setting `QUEUECLEANER__STALLED_DELETE_PRIVATE=true` means you don't care about seeding, ratio, H&R and potentially losing your tracker account.
|
||||
|
||||
#
|
||||
|
||||
### Content Blocker settings
|
||||
|
||||
**`TRIGGERS__CONTENTBLOCKER`**
|
||||
- Cron schedule for the content blocker job.
|
||||
- Type: String - [Quartz cron format](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html).
|
||||
- Default: `0 0/5 * * * ?` (every 5 minutes).
|
||||
- Required: No.
|
||||
|
||||
> [!NOTE]
|
||||
> - Maximum interval is 6 hours.
|
||||
|
||||
**`CONTENTBLOCKER__ENABLED`**
|
||||
- Enables or disables the content blocker functionality.
|
||||
- When enabled, processes all items in the *arr queue and marks unwanted files.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
**`CONTENTBLOCKER__IGNORE_PRIVATE`**
|
||||
- Controls whether to ignore downloads from private trackers.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
**`CONTENTBLOCKER__DELETE_PRIVATE`**
|
||||
- Controls whether to delete private downloads that have all files blocked from the download client.
|
||||
- Has no effect if `CONTENTBLOCKER__IGNORE_PRIVATE` is `true`.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
> [!WARNING]
|
||||
> Setting `CONTENTBLOCKER__DELETE_PRIVATE=true` means you don't care about seeding, ratio, H&R and potentially losing your tracker account.
|
||||
|
||||
#
|
||||
|
||||
### Download Cleaner settings
|
||||
|
||||
**`TRIGGERS__DOWNLOADCLEANER`**
|
||||
- Cron schedule for the download cleaner job.
|
||||
- Type: String - [Quartz cron format](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html).
|
||||
- Default: `0 0 * * * ?` (every hour).
|
||||
- Required: No.
|
||||
|
||||
> [!NOTE]
|
||||
> - Maximum interval is 6 hours.
|
||||
|
||||
**`DOWNLOADCLEANER__ENABLED`**
|
||||
- Enables or disables the download cleaner functionality.
|
||||
- When enabled, automatically cleans up downloads that have been seeding for a certain amount of time.
|
||||
- Type: Boolean.
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
**`DOWNLOADCLEANER__DELETE_PRIVATE`**
|
||||
- Controls whether to delete private downloads.
|
||||
- Type: Boolean.
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
> [!WARNING]
|
||||
> Setting `DOWNLOADCLEANER__DELETE_PRIVATE=true` means you don't care about seeding, ratio, H&R and potentially losing your tracker account.
|
||||
|
||||
**`DOWNLOADCLEANER__CATEGORIES__0__NAME`**
|
||||
- Name of the category to clean.
|
||||
- Type: String.
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
> [!NOTE]
|
||||
> The category name must match the category that was set in the *arr.
|
||||
> For qBittorrent, the category name is the name of the download category.
|
||||
> For Deluge, the category name is the name of the label.
|
||||
> For Transmission, the category name is the name of the download location.
|
||||
|
||||
**`DOWNLOADCLEANER__CATEGORIES__0__MAX_RATIO`**
|
||||
- Maximum ratio to reach before removing a download.
|
||||
- Type: Decimal.
|
||||
- Possible values: `-1` or greater (`-1` means no limit or disabled).
|
||||
- Default: `-1`
|
||||
- Required: No.
|
||||
|
||||
**`DOWNLOADCLEANER__CATEGORIES__0__MIN_SEED_TIME`**
|
||||
- Minimum number of hours to seed before removing a download, if the ratio has been met.
|
||||
- Used with `MAX_RATIO` to ensure a minimum seed time.
|
||||
- Type: Decimal.
|
||||
- Possible values: `0` or greater.
|
||||
- Default: `0`
|
||||
- Required: No.
|
||||
|
||||
**`DOWNLOADCLEANER__CATEGORIES__0__MAX_SEED_TIME`**
|
||||
- Maximum number of hours to seed before removing a download.
|
||||
- Type: Decimal.
|
||||
- Possible values: `-1` or greater (`-1` means no limit or disabled).
|
||||
- Default: `-1`
|
||||
- Required: No.
|
||||
|
||||
> [!NOTE]
|
||||
> A download is cleaned when any of (`MAX_RATIO` & `MIN_SEED_TIME`) or `MAX_SEED_TIME` is reached.
|
||||
|
||||
> [!NOTE]
|
||||
> Multiple categories can be specified using this format, where `<NUMBER>` starts from 0:
|
||||
> ```yaml
|
||||
> DOWNLOADCLEANER__CATEGORIES__<NUMBER>__NAME
|
||||
> DOWNLOADCLEANER__CATEGORIES__<NUMBER>__MAX_RATIO
|
||||
> DOWNLOADCLEANER__CATEGORIES__<NUMBER>__MIN_SEED_TIME
|
||||
> DOWNLOADCLEANER__CATEGORIES__<NUMBER>__MAX_SEED_TIME
|
||||
> ```
|
||||
|
||||
#
|
||||
|
||||
### Download Client settings
|
||||
|
||||
**`DOWNLOAD_CLIENT`**
|
||||
- Specifies which download client is used by *arrs.
|
||||
- Type: String.
|
||||
- Possible values: `none`, `qbittorrent`, `deluge`, `transmission`.
|
||||
- Default: `none`
|
||||
- Required: No.
|
||||
|
||||
> [!NOTE]
|
||||
> Only one download client can be enabled at a time. If you have more than one download client, you should deploy multiple instances of cleanuperr.
|
||||
|
||||
**`QBITTORRENT__URL`**
|
||||
- URL of the qBittorrent instance.
|
||||
- Type: String.
|
||||
- Default: `http://localhost:8080`.
|
||||
- Required: No.
|
||||
|
||||
**`QBITTORRENT__USERNAME`**
|
||||
- Username for qBittorrent authentication.
|
||||
- Type: String.
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
**`QBITTORRENT__PASSWORD`**
|
||||
- Password for qBittorrent authentication.
|
||||
- Type: String.
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
**`DELUGE__URL`**
|
||||
- URL of the Deluge instance.
|
||||
- Type: String.
|
||||
- Default: `http://localhost:8112`.
|
||||
- Required: No.
|
||||
|
||||
**`DELUGE__PASSWORD`**
|
||||
- Password for Deluge authentication.
|
||||
- Type: String.
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
**`TRANSMISSION__URL`**
|
||||
- URL of the Transmission instance.
|
||||
- Type: String.
|
||||
- Default: `http://localhost:9091`.
|
||||
- Required: No.
|
||||
|
||||
**`TRANSMISSION__USERNAME`**
|
||||
- Username for Transmission authentication.
|
||||
- Type: String.
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
**`TRANSMISSION__PASSWORD`**
|
||||
- Password for Transmission authentication.
|
||||
- Type: String.
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
#
|
||||
|
||||
### Arr settings
|
||||
|
||||
**`SONARR__ENABLED`**
|
||||
- Enables or disables Sonarr cleanup.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
**`SONARR__BLOCK__TYPE`**
|
||||
- Determines how file blocking works for Sonarr.
|
||||
- Type: String
|
||||
- Possible values: `blacklist`, `whitelist`
|
||||
- Default: `blacklist`
|
||||
- Required: No.
|
||||
|
||||
**`SONARR__BLOCK__PATH`**
|
||||
- Path to the blocklist file (local file or URL).
|
||||
- Must be JSON compatible.
|
||||
- Type: String
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
**`SONARR__SEARCHTYPE`**
|
||||
- Determines what to search for after removing a queue item.
|
||||
- Type: String
|
||||
- Possible values: `Episode`, `Season`, `Series`
|
||||
- Default: `Episode`
|
||||
- Required: No.
|
||||
|
||||
**`SONARR__INSTANCES__0__URL`**
|
||||
- URL of the Sonarr instance.
|
||||
- Type: String
|
||||
- Default: `http://localhost:8989`
|
||||
- Required: No.
|
||||
|
||||
**`SONARR__INSTANCES__0__APIKEY`**
|
||||
- API key for the Sonarr instance.
|
||||
- Type: String
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
**`RADARR__ENABLED`**
|
||||
- Enables or disables Radarr cleanup.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
**`RADARR__BLOCK__TYPE`**
|
||||
- Determines how file blocking works for Radarr.
|
||||
- Type: String
|
||||
- Possible values: `blacklist`, `whitelist`
|
||||
- Default: `blacklist`
|
||||
- Required: No.
|
||||
|
||||
**`RADARR__BLOCK__PATH`**
|
||||
- Path to the blocklist file (local file or URL).
|
||||
- Must be JSON compatible.
|
||||
- Type: String
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
**`RADARR__INSTANCES__0__URL`**
|
||||
- URL of the Radarr instance.
|
||||
- Type: String
|
||||
- Default: `http://localhost:7878`
|
||||
- Required: No.
|
||||
|
||||
**`RADARR__INSTANCES__0__APIKEY`**
|
||||
- API key for the Radarr instance.
|
||||
- Type: String
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
**`LIDARR__ENABLED`**
|
||||
- Enables or disables Lidarr cleanup.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
**`LIDARR__BLOCK__TYPE`**
|
||||
- Determines how file blocking works for Lidarr.
|
||||
- Type: String
|
||||
- Possible values: `blacklist`, `whitelist`
|
||||
- Default: `blacklist`
|
||||
- Required: No.
|
||||
|
||||
**`LIDARR__BLOCK__PATH`**
|
||||
- Path to the blocklist file (local file or URL).
|
||||
- Must be JSON compatible.
|
||||
- Type: String
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
**`LIDARR__INSTANCES__0__URL`**
|
||||
- URL of the Lidarr instance.
|
||||
- Type: String
|
||||
- Default: `http://localhost:8686`
|
||||
- Required: No.
|
||||
|
||||
**`LIDARR__INSTANCES__0__APIKEY`**
|
||||
- API key for the Lidarr instance.
|
||||
- Type: String
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
> [!NOTE]
|
||||
> Multiple instances can be specified for each *arr using this format, where `<NUMBER>` starts from 0:
|
||||
> ```yaml
|
||||
> <ARR>__INSTANCES__<NUMBER>__URL
|
||||
> <ARR>__INSTANCES__<NUMBER>__APIKEY
|
||||
> ```
|
||||
|
||||
> [!NOTE]
|
||||
> The blocklists (blacklist/whitelist) support the following patterns:
|
||||
> ```
|
||||
> *example // file name ends with "example"
|
||||
> example* // file name starts with "example"
|
||||
> *example* // file name has "example" in the name
|
||||
> example // file name is exactly the word "example"
|
||||
> regex:<ANY_REGEX> // regex that needs to be marked at the start of the line with "regex:"
|
||||
> ```
|
||||
|
||||
> [!NOTE]
|
||||
> [This blacklist](https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/blacklist) and [this whitelist](https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/whitelist) can be used for Sonarr and Radarr, but they are not suitable for other *arrs.
|
||||
|
||||
#
|
||||
|
||||
### Notification settings
|
||||
|
||||
**`NOTIFIARR__API_KEY`**
|
||||
- Notifiarr API key for sending notifications.
|
||||
- Requires Notifiarr's [`Passthrough`](https://notifiarr.wiki/en/Website/Integrations/Passthrough) integration to work.
|
||||
- Type: String
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
**`NOTIFIARR__CHANNEL_ID`**
|
||||
- Discord channel ID where notifications will be sent.
|
||||
- Type: String
|
||||
- Default: Empty.
|
||||
- Required: No.
|
||||
|
||||
**`NOTIFIARR__ON_IMPORT_FAILED_STRIKE`**
|
||||
- Controls whether to notify when an item receives a failed import strike.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
**`NOTIFIARR__ON_STALLED_STRIKE`**
|
||||
- Controls whether to notify when an item receives a stalled download strike.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
**`NOTIFIARR__ON_QUEUE_ITEM_DELETED`**
|
||||
- Controls whether to notify when a queue item is deleted.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
**`NOTIFIARR__ON_DOWNLOAD_CLEANED`**
|
||||
- Controls whether to notify when a download is cleaned.
|
||||
- Type: Boolean
|
||||
- Possible values: `true`, `false`
|
||||
- Default: `false`
|
||||
- Required: No.
|
||||
|
||||
#
|
||||
|
||||
### Advanced settings
|
||||
|
||||
**`HTTP_MAX_RETRIES`**
|
||||
- The number of times to retry a failed HTTP call.
|
||||
- Applies to calls to *arrs, download clients, and other services.
|
||||
- Type: Integer
|
||||
- Possible values: `0` or greater
|
||||
- Default: `0`
|
||||
- Required: No.
|
||||
|
||||
**`HTTP_TIMEOUT`**
|
||||
- The number of seconds to wait before failing an HTTP call.
|
||||
- Applies to calls to *arrs, download clients, and other services.
|
||||
- Type: Integer
|
||||
- Possible values: Greater than `0`
|
||||
- Default: `100`
|
||||
- Required: No.
|
||||
Reference in New Issue
Block a user