mirror of
https://github.com/Cleanuparr/Cleanuparr.git
synced 2025-12-30 09:28:22 -05:00
Compare commits
25 Commits
fix_qbit_p
...
v1.6.2
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9463d7587f | ||
|
|
7d2bf41bec | ||
|
|
93bb8cc18d | ||
|
|
449d9e623f | ||
|
|
3a50d9be3c | ||
|
|
693f80fe6a | ||
|
|
8cfc73213a | ||
|
|
6fbae768a4 | ||
|
|
8e9d0127e0 | ||
|
|
b92d70769a | ||
|
|
75b001cf6a | ||
|
|
479ca7884e | ||
|
|
00d8910118 | ||
|
|
bd28c7ab05 | ||
|
|
720279df65 | ||
|
|
2d4ec648b8 | ||
|
|
704fdaca4a | ||
|
|
b134136e51 | ||
|
|
5ca717d7e0 | ||
|
|
7068ee5e5a | ||
|
|
9f770473e5 | ||
|
|
5fe0f5750a | ||
|
|
b8ce225ccc | ||
|
|
f21f7388b7 | ||
|
|
a1354f231a |
1
.github/FUNDING.yml
vendored
Normal file
1
.github/FUNDING.yml
vendored
Normal file
@@ -0,0 +1 @@
|
||||
github: Flaminel
|
||||
2
.github/ISSUE_TEMPLATE/1-bug.yml
vendored
2
.github/ISSUE_TEMPLATE/1-bug.yml
vendored
@@ -18,7 +18,7 @@ body:
|
||||
required: true
|
||||
- label: Ensured I am using the latest version.
|
||||
required: true
|
||||
- label: Enabled debug logging.
|
||||
- label: Enabled verbose logging.
|
||||
required: true
|
||||
- type: textarea
|
||||
id: what-happened
|
||||
|
||||
2
.github/ISSUE_TEMPLATE/3-help.yml
vendored
2
.github/ISSUE_TEMPLATE/3-help.yml
vendored
@@ -18,7 +18,7 @@ body:
|
||||
required: true
|
||||
- label: Ensured I am using the latest version.
|
||||
required: true
|
||||
- label: Enabled debug logging.
|
||||
- label: Enabled verbose logging.
|
||||
required: true
|
||||
- type: textarea
|
||||
id: description
|
||||
|
||||
49
.github/workflows/docs.yml
vendored
Normal file
49
.github/workflows/docs.yml
vendored
Normal file
@@ -0,0 +1,49 @@
|
||||
name: Deploy Docusaurus to GitHub Pages
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'docs/**'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pages: write
|
||||
id-token: write
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20.x
|
||||
cache: yarn
|
||||
cache-dependency-path: docs/yarn.lock
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: docs
|
||||
run: yarn install --frozen-lockfile
|
||||
|
||||
- name: Build Docusaurus
|
||||
working-directory: docs
|
||||
run: yarn build
|
||||
|
||||
- name: Upload artifact
|
||||
uses: actions/upload-pages-artifact@v3
|
||||
with:
|
||||
path: docs/build
|
||||
retention-days: 1
|
||||
|
||||
- name: Deploy to GitHub Pages
|
||||
id: deployment
|
||||
uses: actions/deploy-pages@v4
|
||||
329
README.md
329
README.md
@@ -1,53 +1,25 @@
|
||||
_Love this project? Give it a ⭐️ and let others know!_
|
||||
|
||||
# <img width="24px" src="./Logo/256.png" alt="cleanuperr"></img> cleanuperr
|
||||
# <img width="24px" src="./Logo/256.png" alt="cleanuperr"></img> Cleanuperr
|
||||
|
||||
[](https://discord.gg/sWggpnmGNY)
|
||||
|
||||
cleanuperr is a tool for automating the cleanup of unwanted or blocked files in Sonarr, Radarr, and supported download clients like qBittorrent. It removes incomplete or blocked downloads, updates queues, and enforces blacklists or whitelists to manage file selection. After removing blocked content, cleanuperr can also trigger a search to replace the deleted shows/movies.
|
||||
Cleanuperr is a tool for automating the cleanup of unwanted or blocked files in Sonarr, Radarr, and supported download clients like qBittorrent. It removes incomplete or blocked downloads, updates queues, and enforces blacklists or whitelists to manage file selection. After removing blocked content, Cleanuperr can also trigger a search to replace the deleted shows/movies.
|
||||
|
||||
cleanuperr was created primarily to address malicious files, such as `*.lnk` or `*.zipx`, that were getting stuck in Sonarr/Radarr and required manual intervention. Some of the reddit posts that made cleanuperr come to life can be found [here](https://www.reddit.com/r/sonarr/comments/1gqnx16/psa_sonarr_downloaded_a_virus/), [here](https://www.reddit.com/r/sonarr/comments/1gqwklr/sonar_downloaded_a_mkv_file_which_looked_like_a/), [here](https://www.reddit.com/r/sonarr/comments/1gpw2wa/downloaded_waiting_to_import/) and [here](https://www.reddit.com/r/sonarr/comments/1gpi344/downloads_not_importing_no_files_found/).
|
||||
Cleanuperr was created primarily to address malicious files, such as `*.lnk` or `*.zipx`, that were getting stuck in Sonarr/Radarr and required manual intervention. Some of the reddit posts that made Cleanuperr come to life can be found [here](https://www.reddit.com/r/sonarr/comments/1gqnx16/psa_sonarr_downloaded_a_virus/), [here](https://www.reddit.com/r/sonarr/comments/1gqwklr/sonar_downloaded_a_mkv_file_which_looked_like_a/), [here](https://www.reddit.com/r/sonarr/comments/1gpw2wa/downloaded_waiting_to_import/) and [here](https://www.reddit.com/r/sonarr/comments/1gpi344/downloads_not_importing_no_files_found/).
|
||||
|
||||
> [!IMPORTANT]
|
||||
> **Features:**
|
||||
> - Strike system to mark stalled or downloads stuck in metadata downloading.
|
||||
> - Remove and block downloads that reached a maximum number of strikes.
|
||||
> - Remove downloads blocked by qBittorrent or by cleanuperr's **content blocker**.
|
||||
> - Remove and block downloads that have a low download speed or high estimated completion time.
|
||||
> - Remove downloads blocked by qBittorrent or by Cleanuperr's **content blocker**.
|
||||
> - Trigger a search for downloads removed from the *arrs.
|
||||
> - Clean up downloads that have been seeding for a certain amount of time.
|
||||
> - Notify on strike or download removal.
|
||||
> - Ignore certain torrent hashes, categories, tags or trackers from processing.
|
||||
> - Ignore certain torrent hashes, categories, tags or trackers from being processed by Cleanuperr.
|
||||
|
||||
cleanuperr supports both qBittorrent's built-in exclusion features and its own blocklist-based system. Binaries for all platforms are provided, along with Docker images for easy deployment.
|
||||
|
||||
> [!WARNING]
|
||||
> This tool is actively developed and still a work in progress, so using the `latest` Docker tag may result in breaking changes. Join the Discord server if you want to reach out to me quickly (or just stay updated on new releases) so we can squash those pesky bugs together:
|
||||
>
|
||||
> https://discord.gg/sWggpnmGNY
|
||||
|
||||
## Table of contents:
|
||||
- [Naming choice](#naming-choice)
|
||||
- [Quick Start](#quick-start)
|
||||
- [How it works](#how-it-works)
|
||||
- [Content blocker](#1-content-blocker-will)
|
||||
- [Queue cleaner](#2-queue-cleaner-will)
|
||||
- [Download cleaner](#3-download-cleaner-will)
|
||||
- [Setup](#setup-examples)
|
||||
- [Usage](#usage)
|
||||
- [Docker](#docker)
|
||||
- [Environment Variables](#environment-variables)
|
||||
- [Docker Compose](#docker-compose-example)
|
||||
- [Windows](#windows)
|
||||
- [Linux](#linux)
|
||||
- [MacOS](#macos)
|
||||
- [FreeBSD](#freebsd)
|
||||
- [Credits](#credits)
|
||||
|
||||
## Naming choice
|
||||
|
||||
I've had people asking why it's `cleanuperr` and not `cleanuparr` and that I should change it. This name was intentional.
|
||||
|
||||
I've seen a few discussions on this type of naming and I've decided that I didn't deserve the `arr` moniker since `cleanuperr` is not a fork of `NZB.Drone` and it does not have any affiliation with the arrs. I still wanted to keep the naming style close enough though, to suggest a correlation between them.
|
||||
Cleanuperr supports both qBittorrent's built-in exclusion features and its own blocklist-based system. Binaries for all platforms are provided, along with Docker images for easy deployment.
|
||||
|
||||
## Quick Start
|
||||
|
||||
@@ -62,292 +34,17 @@ I've seen a few discussions on this type of naming and I've decided that I didn'
|
||||
> 3. **Manual Installation (if you're not using Docker)**
|
||||
> Go to [Windows](#windows), [Linux](#linux) or [MacOS](#macos).
|
||||
|
||||
> [!TIP]
|
||||
> Refer to the [Environment variables](#environment-variables) section for detailed configuration instructions and the [Setup examples](#setup-examples) section for an in-depth explanation of the cleanup process.
|
||||
# Docs
|
||||
|
||||
Docs can be found [here](https://flmorg.github.io/cleanuperr/).
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Only the **latest versions** of the following apps are supported, or earlier versions that have the same API as the latest version:
|
||||
> - qBittorrent
|
||||
> - Deluge
|
||||
> - Transmission
|
||||
> - Sonarr
|
||||
> - Radarr
|
||||
> - Lidarr
|
||||
# <img width="24px" src="./Logo/256.png" alt="Cleanuperr"> Cleanuperr <svg width="14px" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 384 512"><!--!Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free Copyright 2025 Fonticons, Inc.--><path d="M376.6 84.5c11.3-13.6 9.5-33.8-4.1-45.1s-33.8-9.5-45.1 4.1L192 206 56.6 43.5C45.3 29.9 25.1 28.1 11.5 39.4S-3.9 70.9 7.4 84.5L150.3 256 7.4 427.5c-11.3 13.6-9.5 33.8 4.1 45.1s33.8 9.5 45.1-4.1L192 306 327.4 468.5c11.3 13.6 31.5 15.4 45.1 4.1s15.4-31.5 4.1-45.1L233.7 256 376.6 84.5z"/></svg> Huntarr <img width="24px" src="https://github.com/plexguide/Huntarr.io/blob/main/frontend/static/logo/256.png?raw=true" alt Huntarr></img>
|
||||
|
||||
# How it works
|
||||
Think of **Cleanuperr** as the janitor of your server; it keeps your download queue spotless, removes clutter, and blocks malicious files. Now imagine combining that with **Huntarr**, the compulsive librarian who finds missing and upgradable media to complete your collection
|
||||
|
||||
#### 1. **Content blocker** will:
|
||||
- Run every 5 minutes (or configured cron).
|
||||
- Process all items in the *arr queue.
|
||||
- Find the corresponding item from the download client for each queue item.
|
||||
- Mark the files that were found in the queue as **unwanted/skipped** if:
|
||||
- They **are listed in the blacklist**, or
|
||||
- They **are not included in the whitelist**.
|
||||
- If **all files** of a download **are unwanted**:
|
||||
- It will be removed from the *arr's queue and blocked.
|
||||
- It will be deleted from the download client.
|
||||
- A new search will be triggered for the *arr item.
|
||||
#### 2. **Queue cleaner** will:
|
||||
- Run every 5 minutes (or configured cron, or right after `content blocker`).
|
||||
- Process all items in the *arr queue.
|
||||
- Check each queue item if it is **stalled (download speed is 0)**, **stuck in metadata downloading** or **failed to be imported**.
|
||||
- If it is, the item receives a **strike** and will continue to accumulate strikes every time it meets any of these conditions.
|
||||
- Check each queue item if it meets one of the following condition in the download client:
|
||||
- **Marked as completed, but 0 bytes have been downloaded** (due to files being blocked by qBittorrent or the **content blocker**).
|
||||
- All associated files of are marked as **unwanted/skipped**.
|
||||
- If the item **DOES NOT** match the above criteria, it will be skipped.
|
||||
- If the item **DOES** match the criteria or has received the **maximum number of strikes**:
|
||||
- It will be removed from the *arr's queue and blocked.
|
||||
- It will be deleted from the download client.
|
||||
- A new search will be triggered for the *arr item.
|
||||
#### 3. **Download cleaner** will:
|
||||
- Run every hour (or configured cron).
|
||||
- Automatically clean up downloads that have been seeding for a certain amount of time.
|
||||
While **Huntarr** fills in the blanks and improves what you already have, **Cleanuperr** makes sure that only clean downloads get through. If you're aiming for a reliable and self-sufficient setup, **Cleanuperr** and **Huntarr** will take your automated media stack to another level.
|
||||
|
||||
# Setup examples
|
||||
|
||||
## Using qBittorrent's built-in feature (works only with qBittorrent)
|
||||
|
||||
1. Go to qBittorrent -> Options -> Downloads -> make sure `Excluded file names` is checked -> Paste an exclusion list that you have copied.
|
||||
- [blacklist](https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/blacklist), or
|
||||
- [permissive blacklist](https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/blacklist_permissive), or
|
||||
- create your own
|
||||
2. qBittorrent will block files from being downloaded. In the case of malicious content, **nothing is downloaded and the torrent is marked as complete**.
|
||||
3. Start **cleanuperr** with `QUEUECLEANER__ENABLED` set to `true`.
|
||||
4. The **queue cleaner** will perform a cleanup process as described in the [How it works](#how-it-works) section.
|
||||
|
||||
## Using cleanuperr's blocklist (works with all supported download clients)
|
||||
|
||||
1. Set both `QUEUECLEANER__ENABLED` and `CONTENTBLOCKER__ENABLED` to `true` in your environment variables.
|
||||
2. Configure and enable either a **blacklist** or a **whitelist** as described in the [Arr variables](variables.md#Arr-settings) section.
|
||||
3. Once configured, cleanuperr will perform the following tasks:
|
||||
- Execute the **content blocker** job, as explained in the [How it works](#how-it-works) section.
|
||||
- Execute the **queue cleaner** job, as explained in the [How it works](#how-it-works) section.
|
||||
|
||||
## Using cleanuperr just for failed *arr imports (works for Usenet users as well)
|
||||
|
||||
1. Set `QUEUECLEANER__ENABLED` to `true`.
|
||||
2. Set `QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES` to a desired value.
|
||||
3. Optionally set failed import message patterns to ignore using `QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__<NUMBER>`.
|
||||
4. Set `DOWNLOAD_CLIENT` to `none`.
|
||||
|
||||
> [!WARNING]
|
||||
> When `DOWNLOAD_CLIENT=none`, no other action involving a download client would work (e.g. content blocking, removing stalled downloads, excluding private trackers).
|
||||
|
||||
## Usage
|
||||
|
||||
### <img src="https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/brands/docker.svg" height="20" style="vertical-align: middle;"> <span style="vertical-align: middle;">Docker</span>
|
||||
|
||||
|
||||
### **Environment variables**
|
||||
|
||||
**Jump to:**
|
||||
- [General settings](variables.md#general-settings)
|
||||
- [Queue Cleaner settings](variables.md#queue-cleaner-settings)
|
||||
- [Content Blocker settings](variables.md#content-blocker-settings)
|
||||
- [Download Cleaner settings](variables.md#download-cleaner-settings)
|
||||
- [Download Client settings](variables.md#download-client-settings)
|
||||
- [Arr settings](variables.md#arr-settings)
|
||||
- [Notification settings](variables.md#notification-settings)
|
||||
- [Advanced settings](variables.md#advanced-settings)
|
||||
|
||||
### Docker compose example
|
||||
|
||||
> [!NOTE]
|
||||
>
|
||||
> This example contains all settings and should be modified to fit your needs.
|
||||
|
||||
```
|
||||
version: "3.3"
|
||||
services:
|
||||
cleanuperr:
|
||||
image: ghcr.io/flmorg/cleanuperr:latest
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ./cleanuperr/logs:/var/logs
|
||||
- ./cleanuperr/ignored.txt:/ignored.txt
|
||||
environment:
|
||||
- TZ=America/New_York
|
||||
- DRY_RUN=false
|
||||
|
||||
- LOGGING__LOGLEVEL=Information
|
||||
- LOGGING__FILE__ENABLED=false
|
||||
- LOGGING__FILE__PATH=/var/logs/
|
||||
- LOGGING__ENHANCED=true
|
||||
|
||||
- TRIGGERS__QUEUECLEANER=0 0/5 * * * ?
|
||||
- TRIGGERS__CONTENTBLOCKER=0 0/5 * * * ?
|
||||
- TRIGGERS__DOWNLOADCLEANER=0 0 * * * ?
|
||||
|
||||
- QUEUECLEANER__ENABLED=true
|
||||
- QUEUECLEANER__IGNORED_DOWNLOADS_PATH=/ignored.txt
|
||||
- QUEUECLEANER__RUNSEQUENTIALLY=true
|
||||
- QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES=5
|
||||
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE=false
|
||||
- QUEUECLEANER__IMPORT_FAILED_DELETE_PRIVATE=false
|
||||
# - QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__0=title mismatch
|
||||
# - QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__1=manual import required
|
||||
- QUEUECLEANER__STALLED_MAX_STRIKES=5
|
||||
- QUEUECLEANER__STALLED_RESET_STRIKES_ON_PROGRESS=false
|
||||
- QUEUECLEANER__STALLED_IGNORE_PRIVATE=false
|
||||
- QUEUECLEANER__STALLED_DELETE_PRIVATE=false
|
||||
|
||||
- CONTENTBLOCKER__ENABLED=true
|
||||
- CONTENTBLOCKER__IGNORED_DOWNLOADS_PATH=/ignored.txt
|
||||
- CONTENTBLOCKER__IGNORE_PRIVATE=false
|
||||
- CONTENTBLOCKER__DELETE_PRIVATE=false
|
||||
|
||||
- DOWNLOADCLEANER__ENABLED=true
|
||||
- DOWNLOADCLEANER__IGNORED_DOWNLOADS_PATH=/ignored.txt
|
||||
- DOWNLOADCLEANER__DELETE_PRIVATE=false
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__NAME=tv-sonarr
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MAX_RATIO=-1
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MIN_SEED_TIME=0
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MAX_SEED_TIME=240
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__NAME=radarr
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MAX_RATIO=-1
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MIN_SEED_TIME=0
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MAX_SEED_TIME=240
|
||||
|
||||
- DOWNLOAD_CLIENT=none
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=qBittorrent
|
||||
# - QBITTORRENT__URL=http://localhost:8080
|
||||
# - QBITTORRENT__USERNAME=user
|
||||
# - QBITTORRENT__PASSWORD=pass
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=deluge
|
||||
# - DELUGE__URL=http://localhost:8112
|
||||
# - DELUGE__PASSWORD=testing
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=transmission
|
||||
# - TRANSMISSION__URL=http://localhost:9091
|
||||
# - TRANSMISSION__USERNAME=test
|
||||
# - TRANSMISSION__PASSWORD=testing
|
||||
|
||||
- SONARR__ENABLED=true
|
||||
- SONARR__SEARCHTYPE=Episode
|
||||
- SONARR__BLOCK__TYPE=blacklist
|
||||
- SONARR__BLOCK__PATH=https://example.com/path/to/file.txt
|
||||
- SONARR__INSTANCES__0__URL=http://localhost:8989
|
||||
- SONARR__INSTANCES__0__APIKEY=secret1
|
||||
- SONARR__INSTANCES__1__URL=http://localhost:8990
|
||||
- SONARR__INSTANCES__1__APIKEY=secret2
|
||||
|
||||
- RADARR__ENABLED=true
|
||||
- RADARR__BLOCK__TYPE=blacklist
|
||||
- RADARR__BLOCK__PATH=https://example.com/path/to/file.txt
|
||||
- RADARR__INSTANCES__0__URL=http://localhost:7878
|
||||
- RADARR__INSTANCES__0__APIKEY=secret3
|
||||
- RADARR__INSTANCES__1__URL=http://localhost:7879
|
||||
- RADARR__INSTANCES__1__APIKEY=secret4
|
||||
|
||||
- LIDARR__ENABLED=true
|
||||
- LIDARR__BLOCK__TYPE=blacklist
|
||||
- LIDARR__BLOCK__PATH=https://example.com/path/to/file.txt
|
||||
- LIDARR__INSTANCES__0__URL=http://radarr:8686
|
||||
- LIDARR__INSTANCES__0__APIKEY=secret5
|
||||
- LIDARR__INSTANCES__1__URL=http://radarr:8687
|
||||
- LIDARR__INSTANCES__1__APIKEY=secret6
|
||||
|
||||
- NOTIFIARR__ON_IMPORT_FAILED_STRIKE=true
|
||||
- NOTIFIARR__ON_STALLED_STRIKE=true
|
||||
- NOTIFIARR__ON_QUEUE_ITEM_DELETED=true
|
||||
- NOTIFIARR__ON_DOWNLOAD_CLEANED=true
|
||||
- NOTIFIARR__API_KEY=notifiarr_secret
|
||||
- NOTIFIARR__CHANNEL_ID=discord_channel_id
|
||||
```
|
||||
|
||||
### <img src="https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/brands/windows.svg" height="20" style="vertical-align: middle;"> <span style="vertical-align: middle;">Windows</span>
|
||||
|
||||
1. Download the zip file from [releases](https://github.com/flmorg/cleanuperr/releases).
|
||||
2. Extract the zip file into `C:\example\directory`.
|
||||
3. Edit **appsettings.json**. The paths from this json file correspond with the docker env vars, as described [here](#environment-variables).
|
||||
4. Execute `cleanuperr.exe`.
|
||||
|
||||
> [!TIP]
|
||||
> ### Run as a Windows Service
|
||||
> Check out this stackoverflow answer on how to do it: https://stackoverflow.com/a/15719678
|
||||
|
||||
### <img src="https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/brands/linux.svg" height="20" style="vertical-align: middle;"> <span style="vertical-align: middle;">Linux</span>
|
||||
|
||||
1. Download the zip file from [releases](https://github.com/flmorg/cleanuperr/releases).
|
||||
2. Extract the zip file into `/example/directory`.
|
||||
3. Edit **appsettings.json**. The paths from this json file correspond with the docker env vars, as described [here](#environment-variables).
|
||||
4. Open a terminal and execute these commands:
|
||||
```
|
||||
cd /example/directory
|
||||
chmod +x cleanuperr
|
||||
./cleanuperr
|
||||
```
|
||||
|
||||
### <img src="https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/brands/apple.svg" height="20" style="vertical-align: middle;"> <span style="vertical-align: middle;">MacOS</span>
|
||||
|
||||
1. Download the zip file from [releases](https://github.com/flmorg/cleanuperr/releases).
|
||||
2. Extract the zip file into `/example/directory`.
|
||||
3. Edit **appsettings.json**. The paths from this json file correspond with the docker env vars, as described [here](#environment-variables).
|
||||
4. Open a terminal and execute these commands:
|
||||
```
|
||||
cd /example/directory
|
||||
chmod +x cleanuperr
|
||||
./cleanuperr
|
||||
```
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Some people have experienced problems when trying to execute cleanuperr on MacOS because the system actively blocked the file for not being signed.
|
||||
> As per [this](), you may need to also execute this command:
|
||||
> ```
|
||||
> codesign --sign - --force --preserve-metadata=entitlements,requirements,flags,runtime /example/directory/cleanuperr
|
||||
> ```
|
||||
|
||||
### <img src="https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/brands/freebsd.svg" height="20" style="vertical-align: middle;"> <span style="vertical-align: middle;">FreeBSD</span>
|
||||
|
||||
1. Installation:
|
||||
```
|
||||
# install dependencies
|
||||
pkg install -y git icu libinotify libunwind wget
|
||||
|
||||
# set up the dotnet SDK
|
||||
cd ~
|
||||
wget -q https://github.com/Thefrank/dotnet-freebsd-crossbuild/releases/download/v9.0.104-amd64-freebsd-14/dotnet-sdk-9.0.104-freebsd-x64.tar.gz
|
||||
export DOTNET_ROOT=$(pwd)/.dotnet
|
||||
mkdir -p "$DOTNET_ROOT" && tar zxf dotnet-sdk-9.0.104-freebsd-x64.tar.gz -C "$DOTNET_ROOT"
|
||||
export PATH=$PATH:$DOTNET_ROOT:$DOTNET_ROOT/tools
|
||||
|
||||
# download NuGet dependencies
|
||||
mkdir -p /tmp/nuget
|
||||
wget -q -P /tmp/nuget/ https://github.com/Thefrank/dotnet-freebsd-crossbuild/releases/download/v9.0.104-amd64-freebsd-14/Microsoft.AspNetCore.App.Runtime.freebsd-x64.9.0.3.nupkg
|
||||
wget -q -P /tmp/nuget/ https://github.com/Thefrank/dotnet-freebsd-crossbuild/releases/download/v9.0.104-amd64-freebsd-14/Microsoft.NETCore.App.Host.freebsd-x64.9.0.3.nupkg
|
||||
wget -q -P /tmp/nuget/ https://github.com/Thefrank/dotnet-freebsd-crossbuild/releases/download/v9.0.104-amd64-freebsd-14/Microsoft.NETCore.App.Runtime.freebsd-x64.9.0.3.nupkg
|
||||
|
||||
# add NuGet source
|
||||
dotnet nuget add source /tmp/nuget --name tmp
|
||||
|
||||
# add GitHub NuGet source
|
||||
# a PAT (Personal Access Token) can be generated here https://github.com/settings/tokens
|
||||
dotnet nuget add source --username <YOUR_USERNAME> --password <YOUR_PERSONAL_ACCESS_TOKEN> --store-password-in-clear-text --name flmorg https://nuget.pkg.github.com/flmorg/index.json
|
||||
```
|
||||
2. Building:
|
||||
```
|
||||
# clone the project
|
||||
git clone https://github.com/flmorg/cleanuperr.git
|
||||
cd cleanuperr
|
||||
|
||||
# build and publish the app
|
||||
dotnet publish code/Executable/Executable.csproj -c Release --self-contained -o artifacts /p:PublishSingleFile=true
|
||||
|
||||
# move the files to permanent destination
|
||||
mv artifacts/cleanuperr /example/directory/
|
||||
mv artifacts/appsettings.json /example/directory/
|
||||
```
|
||||
3. Edit **appsettings.json**. The paths from this json file correspond with the docker env vars, as described [here](#environment-variables).
|
||||
4. Run the app:
|
||||
```
|
||||
cd /example/directory
|
||||
chmod +x cleanuperr
|
||||
./cleanuperr
|
||||
```
|
||||
<span style="font-size:24px"> ➡️ [**Huntarr**](https://github.com/plexguide/Huntarr.io) </span>
|
||||
|
||||
# Credits
|
||||
Special thanks for inspiration go to:
|
||||
|
||||
@@ -14,7 +14,7 @@ deployment:
|
||||
value: "false"
|
||||
|
||||
- name: LOGGING__LOGLEVEL
|
||||
value: Debug
|
||||
value: Verbose
|
||||
- name: LOGGING__FILE__ENABLED
|
||||
value: "true"
|
||||
- name: LOGGING__FILE__PATH
|
||||
|
||||
@@ -3,7 +3,7 @@ using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.DownloadCleaner;
|
||||
|
||||
public sealed record Category : IConfig
|
||||
public sealed record CleanCategory : IConfig
|
||||
{
|
||||
public required string Name { get; init; }
|
||||
|
||||
@@ -8,8 +8,8 @@ public sealed record DownloadCleanerConfig : IJobConfig, IIgnoredDownloadsConfig
|
||||
public const string SectionName = "DownloadCleaner";
|
||||
|
||||
public bool Enabled { get; init; }
|
||||
|
||||
public List<Category>? Categories { get; init; }
|
||||
|
||||
public List<CleanCategory>? Categories { get; init; }
|
||||
|
||||
[ConfigurationKeyName("DELETE_PRIVATE")]
|
||||
public bool DeletePrivate { get; init; }
|
||||
@@ -17,6 +17,15 @@ public sealed record DownloadCleanerConfig : IJobConfig, IIgnoredDownloadsConfig
|
||||
[ConfigurationKeyName("IGNORED_DOWNLOADS_PATH")]
|
||||
public string? IgnoredDownloadsPath { get; init; }
|
||||
|
||||
[ConfigurationKeyName("UNLINKED_TARGET_CATEGORY")]
|
||||
public string UnlinkedTargetCategory { get; init; } = "cleanuperr-unlinked";
|
||||
|
||||
[ConfigurationKeyName("UNLINKED_IGNORED_ROOT_DIR")]
|
||||
public string UnlinkedIgnoredRootDir { get; init; } = string.Empty;
|
||||
|
||||
[ConfigurationKeyName("UNLINKED_CATEGORIES")]
|
||||
public List<string>? UnlinkedCategories { get; init; }
|
||||
|
||||
public void Validate()
|
||||
{
|
||||
if (!Enabled)
|
||||
@@ -31,9 +40,34 @@ public sealed record DownloadCleanerConfig : IJobConfig, IIgnoredDownloadsConfig
|
||||
|
||||
if (Categories?.GroupBy(x => x.Name).Any(x => x.Count() > 1) is true)
|
||||
{
|
||||
throw new ValidationException("duplicated categories found");
|
||||
throw new ValidationException("duplicated clean categories found");
|
||||
}
|
||||
|
||||
Categories?.ForEach(x => x.Validate());
|
||||
|
||||
if (string.IsNullOrEmpty(UnlinkedTargetCategory))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (UnlinkedCategories?.Count is null or 0)
|
||||
{
|
||||
throw new ValidationException("no unlinked categories configured");
|
||||
}
|
||||
|
||||
if (UnlinkedCategories.Contains(UnlinkedTargetCategory))
|
||||
{
|
||||
throw new ValidationException($"{SectionName.ToUpperInvariant()}__UNLINKED_TARGET_CATEGORY should not be present in {SectionName.ToUpperInvariant()}__UNLINKED_CATEGORIES");
|
||||
}
|
||||
|
||||
if (UnlinkedCategories.Any(string.IsNullOrEmpty))
|
||||
{
|
||||
throw new ValidationException("empty unlinked category filter found");
|
||||
}
|
||||
|
||||
if (!string.IsNullOrEmpty(UnlinkedIgnoredRootDir) && !Directory.Exists(UnlinkedIgnoredRootDir))
|
||||
{
|
||||
throw new ValidationException($"{UnlinkedIgnoredRootDir} root directory does not exist");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
using Common.Exceptions;
|
||||
using Common.Exceptions;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.DownloadClient;
|
||||
|
||||
@@ -8,6 +9,9 @@ public sealed record DelugeConfig : IConfig
|
||||
|
||||
public Uri? Url { get; init; }
|
||||
|
||||
[ConfigurationKeyName("URL_BASE")]
|
||||
public string UrlBase { get; init; } = string.Empty;
|
||||
|
||||
public string? Password { get; init; }
|
||||
|
||||
public void Validate()
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
using Common.Exceptions;
|
||||
using Common.Exceptions;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.DownloadClient;
|
||||
|
||||
@@ -8,6 +9,9 @@ public sealed class QBitConfig : IConfig
|
||||
|
||||
public Uri? Url { get; init; }
|
||||
|
||||
[ConfigurationKeyName("URL_BASE")]
|
||||
public string UrlBase { get; init; } = string.Empty;
|
||||
|
||||
public string? Username { get; init; }
|
||||
|
||||
public string? Password { get; init; }
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
using Common.Exceptions;
|
||||
using Common.Exceptions;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.DownloadClient;
|
||||
|
||||
@@ -8,6 +9,9 @@ public record TransmissionConfig : IConfig
|
||||
|
||||
public Uri? Url { get; init; }
|
||||
|
||||
[ConfigurationKeyName("URL_BASE")]
|
||||
public string UrlBase { get; init; } = "transmission";
|
||||
|
||||
public string? Username { get; init; }
|
||||
|
||||
public string? Password { get; init; }
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
using Common.Exceptions;
|
||||
using Common.Enums;
|
||||
using Common.Exceptions;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.General;
|
||||
@@ -10,6 +11,9 @@ public sealed record HttpConfig : IConfig
|
||||
|
||||
[ConfigurationKeyName("HTTP_TIMEOUT")]
|
||||
public ushort Timeout { get; init; } = 100;
|
||||
|
||||
[ConfigurationKeyName("HTTP_VALIDATE_CERT")]
|
||||
public CertificateValidationType CertificateValidation { get; init; } = CertificateValidationType.Enabled;
|
||||
|
||||
public void Validate()
|
||||
{
|
||||
|
||||
@@ -10,13 +10,25 @@ public abstract record NotificationConfig
|
||||
[ConfigurationKeyName("ON_STALLED_STRIKE")]
|
||||
public bool OnStalledStrike { get; init; }
|
||||
|
||||
[ConfigurationKeyName("ON_SLOW_STRIKE")]
|
||||
public bool OnSlowStrike { get; init; }
|
||||
|
||||
[ConfigurationKeyName("ON_QUEUE_ITEM_DELETED")]
|
||||
public bool OnQueueItemDeleted { get; init; }
|
||||
|
||||
[ConfigurationKeyName("ON_DOWNLOAD_CLEANED")]
|
||||
public bool OnDownloadCleaned { get; init; }
|
||||
|
||||
[ConfigurationKeyName("ON_CATEGORY_CHANGED")]
|
||||
public bool OnCategoryChanged { get; init; }
|
||||
|
||||
public bool IsEnabled => OnImportFailedStrike || OnStalledStrike || OnQueueItemDeleted || OnDownloadCleaned;
|
||||
public bool IsEnabled =>
|
||||
OnImportFailedStrike ||
|
||||
OnStalledStrike ||
|
||||
OnSlowStrike ||
|
||||
OnQueueItemDeleted ||
|
||||
OnDownloadCleaned ||
|
||||
OnCategoryChanged;
|
||||
|
||||
public abstract bool IsValid();
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
using Common.Exceptions;
|
||||
using Common.CustomDataTypes;
|
||||
using Common.Exceptions;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace Common.Configuration.QueueCleaner;
|
||||
@@ -24,7 +25,7 @@ public sealed record QueueCleanerConfig : IJobConfig, IIgnoredDownloadsConfig
|
||||
public bool ImportFailedDeletePrivate { get; init; }
|
||||
|
||||
[ConfigurationKeyName("IMPORT_FAILED_IGNORE_PATTERNS")]
|
||||
public List<string>? ImportFailedIgnorePatterns { get; init; }
|
||||
public IReadOnlyList<string>? ImportFailedIgnorePatterns { get; init; }
|
||||
|
||||
[ConfigurationKeyName("STALLED_MAX_STRIKES")]
|
||||
public ushort StalledMaxStrikes { get; init; }
|
||||
@@ -37,17 +38,82 @@ public sealed record QueueCleanerConfig : IJobConfig, IIgnoredDownloadsConfig
|
||||
|
||||
[ConfigurationKeyName("STALLED_DELETE_PRIVATE")]
|
||||
public bool StalledDeletePrivate { get; init; }
|
||||
|
||||
[ConfigurationKeyName("DOWNLOADING_METADATA_MAX_STRIKES")]
|
||||
public ushort DownloadingMetadataMaxStrikes { get; init; }
|
||||
|
||||
[ConfigurationKeyName("SLOW_MAX_STRIKES")]
|
||||
public ushort SlowMaxStrikes { get; init; }
|
||||
|
||||
[ConfigurationKeyName("SLOW_RESET_STRIKES_ON_PROGRESS")]
|
||||
public bool SlowResetStrikesOnProgress { get; init; }
|
||||
|
||||
[ConfigurationKeyName("SLOW_IGNORE_PRIVATE")]
|
||||
public bool SlowIgnorePrivate { get; init; }
|
||||
|
||||
[ConfigurationKeyName("SLOW_DELETE_PRIVATE")]
|
||||
public bool SlowDeletePrivate { get; init; }
|
||||
|
||||
[ConfigurationKeyName("SLOW_MIN_SPEED")]
|
||||
public string SlowMinSpeed { get; init; } = string.Empty;
|
||||
|
||||
public ByteSize SlowMinSpeedByteSize => string.IsNullOrEmpty(SlowMinSpeed) ? new ByteSize(0) : ByteSize.Parse(SlowMinSpeed);
|
||||
|
||||
[ConfigurationKeyName("SLOW_MAX_TIME")]
|
||||
public double SlowMaxTime { get; init; }
|
||||
|
||||
[ConfigurationKeyName("SLOW_IGNORE_ABOVE_SIZE")]
|
||||
public string SlowIgnoreAboveSize { get; init; } = string.Empty;
|
||||
|
||||
public ByteSize? SlowIgnoreAboveSizeByteSize => string.IsNullOrEmpty(SlowIgnoreAboveSize) ? null : ByteSize.Parse(SlowIgnoreAboveSize);
|
||||
|
||||
public void Validate()
|
||||
{
|
||||
if (ImportFailedMaxStrikes is > 0 and < 3)
|
||||
{
|
||||
throw new ValidationException("the minimum value for IMPORT_FAILED_MAX_STRIKES must be 3");
|
||||
throw new ValidationException($"the minimum value for {SectionName.ToUpperInvariant()}__IMPORT_FAILED_MAX_STRIKES must be 3");
|
||||
}
|
||||
|
||||
if (StalledMaxStrikes is > 0 and < 3)
|
||||
{
|
||||
throw new ValidationException("the minimum value for STALLED_MAX_STRIKES must be 3");
|
||||
throw new ValidationException($"the minimum value for {SectionName.ToUpperInvariant()}__STALLED_MAX_STRIKES must be 3");
|
||||
}
|
||||
|
||||
if (DownloadingMetadataMaxStrikes is > 0 and < 3)
|
||||
{
|
||||
throw new ValidationException($"the minimum value for {SectionName.ToUpperInvariant()}__DOWNLOADING_METADATA_MAX_STRIKES must be 3");
|
||||
}
|
||||
|
||||
if (SlowMaxStrikes is > 0 and < 3)
|
||||
{
|
||||
throw new ValidationException($"the minimum value for {SectionName.ToUpperInvariant()}__SLOW_MAX_STRIKES must be 3");
|
||||
}
|
||||
|
||||
if (SlowMaxStrikes > 0)
|
||||
{
|
||||
bool isSlowSpeedSet = !string.IsNullOrEmpty(SlowMinSpeed);
|
||||
|
||||
if (isSlowSpeedSet && ByteSize.TryParse(SlowMinSpeed, out _) is false)
|
||||
{
|
||||
throw new ValidationException($"invalid value for {SectionName.ToUpperInvariant()}__SLOW_MIN_SPEED");
|
||||
}
|
||||
|
||||
if (SlowMaxTime < 0)
|
||||
{
|
||||
throw new ValidationException($"invalid value for {SectionName.ToUpperInvariant()}__SLOW_MAX_TIME");
|
||||
}
|
||||
|
||||
if (!isSlowSpeedSet && SlowMaxTime is 0)
|
||||
{
|
||||
throw new ValidationException($"either {SectionName.ToUpperInvariant()}__SLOW_MIN_SPEED or {SectionName.ToUpperInvariant()}__SLOW_MAX_STRIKES must be set");
|
||||
}
|
||||
|
||||
bool isSlowIgnoreAboveSizeSet = !string.IsNullOrEmpty(SlowIgnoreAboveSize);
|
||||
|
||||
if (isSlowIgnoreAboveSizeSet && ByteSize.TryParse(SlowIgnoreAboveSize, out _) is false)
|
||||
{
|
||||
throw new ValidationException($"invalid value for {SectionName.ToUpperInvariant()}__SLOW_IGNORE_ABOVE_SIZE");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
115
code/Common/CustomDataTypes/ByteSize.cs
Normal file
115
code/Common/CustomDataTypes/ByteSize.cs
Normal file
@@ -0,0 +1,115 @@
|
||||
using System.Globalization;
|
||||
|
||||
namespace Common.CustomDataTypes;
|
||||
|
||||
public readonly struct ByteSize : IComparable<ByteSize>, IEquatable<ByteSize>
|
||||
{
|
||||
public long Bytes { get; }
|
||||
|
||||
private const long BytesPerKB = 1024;
|
||||
private const long BytesPerMB = 1024 * 1024;
|
||||
private const long BytesPerGB = 1024 * 1024 * 1024;
|
||||
|
||||
public ByteSize(long bytes)
|
||||
{
|
||||
if (bytes < 0)
|
||||
{
|
||||
throw new ArgumentOutOfRangeException(nameof(bytes), "bytes can not be negative");
|
||||
}
|
||||
|
||||
Bytes = bytes;
|
||||
}
|
||||
|
||||
public static ByteSize FromKilobytes(double kilobytes) => new((long)(kilobytes * BytesPerKB));
|
||||
public static ByteSize FromMegabytes(double megabytes) => new((long)(megabytes * BytesPerMB));
|
||||
public static ByteSize FromGigabytes(double gigabytes) => new((long)(gigabytes * BytesPerGB));
|
||||
|
||||
public static ByteSize Parse(string input)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(input))
|
||||
{
|
||||
throw new ArgumentNullException(nameof(input));
|
||||
}
|
||||
|
||||
input = input.Trim().ToUpperInvariant();
|
||||
double value;
|
||||
if (input.EndsWith("KB", StringComparison.InvariantCultureIgnoreCase))
|
||||
{
|
||||
value = double.Parse(input[..^2], CultureInfo.InvariantCulture);
|
||||
return FromKilobytes(value);
|
||||
}
|
||||
|
||||
if (input.EndsWith("MB", StringComparison.InvariantCultureIgnoreCase))
|
||||
{
|
||||
value = double.Parse(input[..^2], CultureInfo.InvariantCulture);
|
||||
return FromMegabytes(value);
|
||||
}
|
||||
|
||||
if (input.EndsWith("GB", StringComparison.InvariantCultureIgnoreCase))
|
||||
{
|
||||
value = double.Parse(input[..^2], CultureInfo.InvariantCulture);
|
||||
return FromGigabytes(value);
|
||||
}
|
||||
|
||||
throw new FormatException("invalid size format | only KB, MB and GB are supported");
|
||||
}
|
||||
|
||||
public static bool TryParse(string? input, out ByteSize? result)
|
||||
{
|
||||
result = default;
|
||||
|
||||
if (string.IsNullOrWhiteSpace(input))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
input = input.Trim().ToUpperInvariant();
|
||||
|
||||
if (input.EndsWith("KB", StringComparison.InvariantCultureIgnoreCase) &&
|
||||
double.TryParse(input[..^2], NumberStyles.Float, CultureInfo.InvariantCulture, out double kb))
|
||||
{
|
||||
result = FromKilobytes(kb);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (input.EndsWith("MB", StringComparison.InvariantCultureIgnoreCase) &&
|
||||
double.TryParse(input[..^2], NumberStyles.Float, CultureInfo.InvariantCulture, out double mb))
|
||||
{
|
||||
result = FromMegabytes(mb);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (input.EndsWith("GB", StringComparison.InvariantCultureIgnoreCase) &&
|
||||
double.TryParse(input[..^2], NumberStyles.Float, CultureInfo.InvariantCulture, out double gb))
|
||||
{
|
||||
result = FromGigabytes(gb);
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
public override string ToString() =>
|
||||
Bytes switch
|
||||
{
|
||||
>= BytesPerGB => $"{Bytes / (double)BytesPerGB:0.##} GB",
|
||||
>= BytesPerMB => $"{Bytes / (double)BytesPerMB:0.##} MB",
|
||||
_ => $"{Bytes / (double)BytesPerKB:0.##} KB"
|
||||
};
|
||||
|
||||
public int CompareTo(ByteSize other) => Bytes.CompareTo(other.Bytes);
|
||||
public bool Equals(ByteSize other) => Bytes == other.Bytes;
|
||||
|
||||
public override bool Equals(object? obj) => obj is ByteSize other && Equals(other);
|
||||
public override int GetHashCode() => Bytes.GetHashCode();
|
||||
|
||||
public static bool operator ==(ByteSize left, ByteSize right) => left.Equals(right);
|
||||
public static bool operator !=(ByteSize left, ByteSize right) => !(left == right);
|
||||
public static bool operator <(ByteSize left, ByteSize right) => left.Bytes < right.Bytes;
|
||||
public static bool operator >(ByteSize left, ByteSize right) => left.Bytes > right.Bytes;
|
||||
public static bool operator <=(ByteSize left, ByteSize right) => left.Bytes <= right.Bytes;
|
||||
public static bool operator >=(ByteSize left, ByteSize right) => left.Bytes >= right.Bytes;
|
||||
|
||||
public static ByteSize operator +(ByteSize left, ByteSize right) => new(left.Bytes + right.Bytes);
|
||||
public static ByteSize operator -(ByteSize left, ByteSize right) => new(Math.Max(left.Bytes - right.Bytes, 0));
|
||||
}
|
||||
66
code/Common/CustomDataTypes/SmartTimeSpan.cs
Normal file
66
code/Common/CustomDataTypes/SmartTimeSpan.cs
Normal file
@@ -0,0 +1,66 @@
|
||||
using System.Text;
|
||||
|
||||
namespace Common.CustomDataTypes;
|
||||
|
||||
public readonly struct SmartTimeSpan : IComparable<SmartTimeSpan>, IEquatable<SmartTimeSpan>
|
||||
{
|
||||
public TimeSpan Time { get; }
|
||||
|
||||
public SmartTimeSpan(TimeSpan time)
|
||||
{
|
||||
Time = time;
|
||||
}
|
||||
|
||||
public override string ToString()
|
||||
{
|
||||
if (Time == TimeSpan.Zero)
|
||||
{
|
||||
return "0 seconds";
|
||||
}
|
||||
|
||||
StringBuilder sb = new();
|
||||
|
||||
if (Time.Days > 0)
|
||||
{
|
||||
sb.Append($"{Time.Days} day{(Time.Days > 1 ? "s" : "")} ");
|
||||
}
|
||||
|
||||
if (Time.Hours > 0)
|
||||
{
|
||||
sb.Append($"{Time.Hours} hour{(Time.Hours > 1 ? "s" : "")} ");
|
||||
}
|
||||
|
||||
if (Time.Minutes > 0)
|
||||
{
|
||||
sb.Append($"{Time.Minutes} minute{(Time.Minutes > 1 ? "s" : "")} ");
|
||||
}
|
||||
|
||||
if (Time.Seconds > 0)
|
||||
{
|
||||
sb.Append($"{Time.Seconds} second{(Time.Seconds > 1 ? "s" : "")}");
|
||||
}
|
||||
|
||||
return sb.ToString().TrimEnd();
|
||||
}
|
||||
|
||||
public static SmartTimeSpan FromMinutes(double minutes) => new(TimeSpan.FromMinutes(minutes));
|
||||
public static SmartTimeSpan FromSeconds(double seconds) => new(TimeSpan.FromSeconds(seconds));
|
||||
public static SmartTimeSpan FromHours(double hours) => new(TimeSpan.FromHours(hours));
|
||||
public static SmartTimeSpan FromDays(double days) => new(TimeSpan.FromDays(days));
|
||||
|
||||
public int CompareTo(SmartTimeSpan other) => Time.CompareTo(other.Time);
|
||||
public bool Equals(SmartTimeSpan other) => Time.Equals(other.Time);
|
||||
|
||||
public override bool Equals(object? obj) => obj is SmartTimeSpan other && Equals(other);
|
||||
public override int GetHashCode() => Time.GetHashCode();
|
||||
|
||||
public static bool operator ==(SmartTimeSpan left, SmartTimeSpan right) => left.Equals(right);
|
||||
public static bool operator !=(SmartTimeSpan left, SmartTimeSpan right) => !left.Equals(right);
|
||||
public static bool operator <(SmartTimeSpan left, SmartTimeSpan right) => left.Time < right.Time;
|
||||
public static bool operator >(SmartTimeSpan left, SmartTimeSpan right) => left.Time > right.Time;
|
||||
public static bool operator <=(SmartTimeSpan left, SmartTimeSpan right) => left.Time <= right.Time;
|
||||
public static bool operator >=(SmartTimeSpan left, SmartTimeSpan right) => left.Time >= right.Time;
|
||||
|
||||
public static SmartTimeSpan operator +(SmartTimeSpan left, SmartTimeSpan right) => new(left.Time + right.Time);
|
||||
public static SmartTimeSpan operator -(SmartTimeSpan left, SmartTimeSpan right) => new(left.Time - right.Time);
|
||||
}
|
||||
8
code/Common/Enums/CertificateValidationType.cs
Normal file
8
code/Common/Enums/CertificateValidationType.cs
Normal file
@@ -0,0 +1,8 @@
|
||||
namespace Common.Enums;
|
||||
|
||||
public enum CertificateValidationType
|
||||
{
|
||||
Enabled = 0,
|
||||
DisabledForLocalAddresses = 1,
|
||||
Disabled = 2
|
||||
}
|
||||
12
code/Common/Exceptions/FatalException.cs
Normal file
12
code/Common/Exceptions/FatalException.cs
Normal file
@@ -0,0 +1,12 @@
|
||||
namespace Common.Exceptions;
|
||||
|
||||
public class FatalException : Exception
|
||||
{
|
||||
public FatalException()
|
||||
{
|
||||
}
|
||||
|
||||
public FatalException(string message) : base(message)
|
||||
{
|
||||
}
|
||||
}
|
||||
@@ -1,8 +1,14 @@
|
||||
namespace Domain.Enums;
|
||||
namespace Domain.Enums;
|
||||
|
||||
public enum DeleteReason
|
||||
{
|
||||
None,
|
||||
Stalled,
|
||||
ImportFailed,
|
||||
AllFilesBlocked
|
||||
DownloadingMetadata,
|
||||
SlowSpeed,
|
||||
SlowTime,
|
||||
AllFilesSkipped,
|
||||
AllFilesSkippedByQBit,
|
||||
AllFilesBlocked,
|
||||
}
|
||||
@@ -3,5 +3,8 @@
|
||||
public enum StrikeType
|
||||
{
|
||||
Stalled,
|
||||
ImportFailed
|
||||
DownloadingMetadata,
|
||||
ImportFailed,
|
||||
SlowSpeed,
|
||||
SlowTime,
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
namespace Domain.Models.Cache;
|
||||
|
||||
public sealed record CacheItem
|
||||
public sealed record StalledCacheItem
|
||||
{
|
||||
/// <summary>
|
||||
/// The amount of bytes that have been downloaded.
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
namespace Domain.Models.Deluge.Response;
|
||||
|
||||
public sealed record TorrentStatus
|
||||
public sealed record DownloadStatus
|
||||
{
|
||||
public string? Hash { get; init; }
|
||||
|
||||
@@ -12,12 +12,18 @@ public sealed record TorrentStatus
|
||||
|
||||
public ulong Eta { get; init; }
|
||||
|
||||
[JsonProperty("download_payload_rate")]
|
||||
public long DownloadSpeed { get; init; }
|
||||
|
||||
public bool Private { get; init; }
|
||||
|
||||
[JsonProperty("total_size")]
|
||||
public long Size { get; init; }
|
||||
|
||||
[JsonProperty("total_done")]
|
||||
public long TotalDone { get; init; }
|
||||
|
||||
public string? Label { get; init; }
|
||||
public string? Label { get; set; }
|
||||
|
||||
[JsonProperty("seeding_time")]
|
||||
public long SeedingTime { get; init; }
|
||||
@@ -25,9 +31,12 @@ public sealed record TorrentStatus
|
||||
public float Ratio { get; init; }
|
||||
|
||||
public required IReadOnlyList<Tracker> Trackers { get; init; }
|
||||
|
||||
[JsonProperty("download_location")]
|
||||
public required string DownloadLocation { get; init; }
|
||||
}
|
||||
|
||||
public sealed record Tracker
|
||||
{
|
||||
public required Uri Url { get; init; }
|
||||
public required string Url { get; init; }
|
||||
}
|
||||
@@ -1,10 +1,12 @@
|
||||
using System.Net;
|
||||
using Common.Configuration.General;
|
||||
using Common.Helpers;
|
||||
using Infrastructure.Services;
|
||||
using Infrastructure.Verticals.DownloadClient.Deluge;
|
||||
using Infrastructure.Verticals.Notifications.Consumers;
|
||||
using Infrastructure.Verticals.Notifications.Models;
|
||||
using MassTransit;
|
||||
using Microsoft.Extensions.Options;
|
||||
using Polly;
|
||||
using Polly.Extensions.Http;
|
||||
|
||||
@@ -17,7 +19,9 @@ public static class MainDI
|
||||
.AddLogging(builder => builder.ClearProviders().AddConsole())
|
||||
.AddHttpClients(configuration)
|
||||
.AddConfiguration(configuration)
|
||||
.AddMemoryCache()
|
||||
.AddMemoryCache(options => {
|
||||
options.ExpirationScanFrequency = TimeSpan.FromMinutes(1);
|
||||
})
|
||||
.AddServices()
|
||||
.AddQuartzServices(configuration)
|
||||
.AddNotifications(configuration)
|
||||
@@ -25,8 +29,10 @@ public static class MainDI
|
||||
{
|
||||
config.AddConsumer<NotificationConsumer<FailedImportStrikeNotification>>();
|
||||
config.AddConsumer<NotificationConsumer<StalledStrikeNotification>>();
|
||||
config.AddConsumer<NotificationConsumer<SlowStrikeNotification>>();
|
||||
config.AddConsumer<NotificationConsumer<QueueItemDeletedNotification>>();
|
||||
config.AddConsumer<NotificationConsumer<DownloadCleanedNotification>>();
|
||||
config.AddConsumer<NotificationConsumer<CategoryChangedNotification>>();
|
||||
|
||||
config.UsingInMemory((context, cfg) =>
|
||||
{
|
||||
@@ -34,8 +40,10 @@ public static class MainDI
|
||||
{
|
||||
e.ConfigureConsumer<NotificationConsumer<FailedImportStrikeNotification>>(context);
|
||||
e.ConfigureConsumer<NotificationConsumer<StalledStrikeNotification>>(context);
|
||||
e.ConfigureConsumer<NotificationConsumer<SlowStrikeNotification>>(context);
|
||||
e.ConfigureConsumer<NotificationConsumer<QueueItemDeletedNotification>>(context);
|
||||
e.ConfigureConsumer<NotificationConsumer<DownloadCleanedNotification>>(context);
|
||||
e.ConfigureConsumer<NotificationConsumer<CategoryChangedNotification>>(context);
|
||||
e.ConcurrentMessageLimit = 1;
|
||||
e.PrefetchCount = 1;
|
||||
});
|
||||
@@ -56,6 +64,15 @@ public static class MainDI
|
||||
{
|
||||
x.Timeout = TimeSpan.FromSeconds(config.Timeout);
|
||||
})
|
||||
.ConfigurePrimaryHttpMessageHandler(provider =>
|
||||
{
|
||||
CertificateValidationService service = provider.GetRequiredService<CertificateValidationService>();
|
||||
|
||||
return new HttpClientHandler
|
||||
{
|
||||
ServerCertificateCustomValidationCallback = service.ShouldByPassValidationError
|
||||
};
|
||||
})
|
||||
.AddRetryPolicyHandler(config);
|
||||
|
||||
// add Deluge HttpClient
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Infrastructure.Verticals.Notifications.Apprise;
|
||||
using Infrastructure.Verticals.Notifications.Notifiarr;
|
||||
|
||||
namespace Executable.DependencyInjection;
|
||||
@@ -8,8 +9,11 @@ public static class NotificationsDI
|
||||
public static IServiceCollection AddNotifications(this IServiceCollection services, IConfiguration configuration) =>
|
||||
services
|
||||
.Configure<NotifiarrConfig>(configuration.GetSection(NotifiarrConfig.SectionName))
|
||||
.Configure<AppriseConfig>(configuration.GetSection(AppriseConfig.SectionName))
|
||||
.AddTransient<INotifiarrProxy, NotifiarrProxy>()
|
||||
.AddTransient<INotificationProvider, NotifiarrProvider>()
|
||||
.AddTransient<IAppriseProxy, AppriseProxy>()
|
||||
.AddTransient<INotificationProvider, AppriseProvider>()
|
||||
.AddTransient<INotificationPublisher, NotificationPublisher>()
|
||||
.AddTransient<INotificationFactory, NotificationFactory>()
|
||||
.AddTransient<NotificationService>();
|
||||
|
||||
@@ -3,6 +3,7 @@ using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Providers;
|
||||
using Infrastructure.Services;
|
||||
using Infrastructure.Verticals.Arr;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.DownloadCleaner;
|
||||
@@ -10,6 +11,7 @@ using Infrastructure.Verticals.DownloadClient;
|
||||
using Infrastructure.Verticals.DownloadClient.Deluge;
|
||||
using Infrastructure.Verticals.DownloadClient.QBittorrent;
|
||||
using Infrastructure.Verticals.DownloadClient.Transmission;
|
||||
using Infrastructure.Verticals.Files;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.QueueCleaner;
|
||||
|
||||
@@ -20,6 +22,7 @@ public static class ServicesDI
|
||||
public static IServiceCollection AddServices(this IServiceCollection services) =>
|
||||
services
|
||||
.AddTransient<IDryRunInterceptor, DryRunInterceptor>()
|
||||
.AddTransient<CertificateValidationService>()
|
||||
.AddTransient<SonarrClient>()
|
||||
.AddTransient<RadarrClient>()
|
||||
.AddTransient<LidarrClient>()
|
||||
@@ -27,14 +30,17 @@ public static class ServicesDI
|
||||
.AddTransient<ContentBlocker>()
|
||||
.AddTransient<DownloadCleaner>()
|
||||
.AddTransient<IFilenameEvaluator, FilenameEvaluator>()
|
||||
.AddTransient<IHardLinkFileService, HardLinkFileService>()
|
||||
.AddTransient<UnixHardLinkFileService>()
|
||||
.AddTransient<WindowsHardLinkFileService>()
|
||||
.AddTransient<DummyDownloadService>()
|
||||
.AddTransient<QBitService>()
|
||||
.AddTransient<DelugeService>()
|
||||
.AddTransient<TransmissionService>()
|
||||
.AddTransient<ArrQueueIterator>()
|
||||
.AddTransient<DownloadServiceFactory>()
|
||||
.AddTransient<IStriker, Striker>()
|
||||
.AddSingleton<BlocklistProvider>()
|
||||
.AddSingleton<IStriker, Striker>()
|
||||
.AddSingleton<IgnoredDownloadsProvider<QueueCleanerConfig>>()
|
||||
.AddSingleton<IgnoredDownloadsProvider<ContentBlockerConfig>>()
|
||||
.AddSingleton<IgnoredDownloadsProvider<DownloadCleanerConfig>>();
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
{
|
||||
"DRY_RUN": true,
|
||||
"HTTP_MAX_RETRIES": 0,
|
||||
"HTTP_TIMEOUT": 10,
|
||||
"HTTP_TIMEOUT": 100,
|
||||
"HTTP_VALIDATE_CERT": "enabled",
|
||||
"Logging": {
|
||||
"LogLevel": "Debug",
|
||||
"LogLevel": "Verbose",
|
||||
"Enhanced": true,
|
||||
"File": {
|
||||
"Enabled": false,
|
||||
@@ -25,16 +26,24 @@
|
||||
"Enabled": true,
|
||||
"RunSequentially": true,
|
||||
"IGNORED_DOWNLOADS_PATH": "../test/data/cleanuperr/ignored_downloads",
|
||||
"IMPORT_FAILED_MAX_STRIKES": 5,
|
||||
"IMPORT_FAILED_MAX_STRIKES": 3,
|
||||
"IMPORT_FAILED_IGNORE_PRIVATE": true,
|
||||
"IMPORT_FAILED_DELETE_PRIVATE": false,
|
||||
"IMPORT_FAILED_IGNORE_PATTERNS": [
|
||||
"file is a sample"
|
||||
],
|
||||
"STALLED_MAX_STRIKES": 5,
|
||||
"STALLED_MAX_STRIKES": 3,
|
||||
"STALLED_RESET_STRIKES_ON_PROGRESS": true,
|
||||
"STALLED_IGNORE_PRIVATE": true,
|
||||
"STALLED_DELETE_PRIVATE": false
|
||||
"STALLED_DELETE_PRIVATE": false,
|
||||
"DOWNLOADING_METADATA_MAX_STRIKES": 3,
|
||||
"SLOW_MAX_STRIKES": 5,
|
||||
"SLOW_RESET_STRIKES_ON_PROGRESS": true,
|
||||
"SLOW_IGNORE_PRIVATE": false,
|
||||
"SLOW_DELETE_PRIVATE": false,
|
||||
"SLOW_MIN_SPEED": "1MB",
|
||||
"SLOW_MAX_TIME": 20,
|
||||
"SLOW_IGNORE_ABOVE_SIZE": "4GB"
|
||||
},
|
||||
"DownloadCleaner": {
|
||||
"Enabled": false,
|
||||
@@ -44,23 +53,32 @@
|
||||
"Name": "tv-sonarr",
|
||||
"MAX_RATIO": -1,
|
||||
"MIN_SEED_TIME": 0,
|
||||
"MAX_SEED_TIME": -1
|
||||
"MAX_SEED_TIME": 240
|
||||
}
|
||||
],
|
||||
"UNLINKED_TARGET_CATEGORY": "cleanuperr-unlinked",
|
||||
"UNLINKED_IGNORED_ROOT_DIR": "",
|
||||
"UNLINKED_CATEGORIES": [
|
||||
"tv-sonarr",
|
||||
"radarr"
|
||||
],
|
||||
"IGNORED_DOWNLOADS_PATH": "../test/data/cleanuperr/ignored_downloads"
|
||||
},
|
||||
"DOWNLOAD_CLIENT": "qbittorrent",
|
||||
"qBittorrent": {
|
||||
"Url": "http://localhost:8080",
|
||||
"URL_BASE": "",
|
||||
"Username": "test",
|
||||
"Password": "testing"
|
||||
},
|
||||
"Deluge": {
|
||||
"Url": "http://localhost:8112",
|
||||
"URL_BASE": "",
|
||||
"Password": "testing"
|
||||
},
|
||||
"Transmission": {
|
||||
"Url": "http://localhost:9091",
|
||||
"URL_BASE": "transmission",
|
||||
"Username": "test",
|
||||
"Password": "testing"
|
||||
},
|
||||
@@ -107,9 +125,21 @@
|
||||
"Notifiarr": {
|
||||
"ON_IMPORT_FAILED_STRIKE": true,
|
||||
"ON_STALLED_STRIKE": true,
|
||||
"ON_SLOW_STRIKE": true,
|
||||
"ON_QUEUE_ITEM_DELETED": true,
|
||||
"ON_DOWNLOAD_CLEANED": true,
|
||||
"ON_CATEGORY_CHANGED": true,
|
||||
"API_KEY": "",
|
||||
"CHANNEL_ID": ""
|
||||
},
|
||||
"Apprise": {
|
||||
"ON_IMPORT_FAILED_STRIKE": true,
|
||||
"ON_STALLED_STRIKE": true,
|
||||
"ON_SLOW_STRIKE": true,
|
||||
"ON_QUEUE_ITEM_DELETED": true,
|
||||
"ON_DOWNLOAD_CLEANED": true,
|
||||
"ON_CATEGORY_CHANGED": true,
|
||||
"URL": "http://localhost:8000",
|
||||
"KEY": ""
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
"DRY_RUN": false,
|
||||
"HTTP_MAX_RETRIES": 0,
|
||||
"HTTP_TIMEOUT": 100,
|
||||
"HTTP_VALIDATE_CERT": "enabled",
|
||||
"Logging": {
|
||||
"LogLevel": "Information",
|
||||
"Enhanced": true,
|
||||
@@ -21,7 +22,7 @@
|
||||
"IGNORED_DOWNLOADS_PATH": ""
|
||||
},
|
||||
"QueueCleaner": {
|
||||
"Enabled": true,
|
||||
"Enabled": false,
|
||||
"RunSequentially": true,
|
||||
"IGNORED_DOWNLOADS_PATH": "",
|
||||
"IMPORT_FAILED_MAX_STRIKES": 0,
|
||||
@@ -31,26 +32,40 @@
|
||||
"STALLED_MAX_STRIKES": 0,
|
||||
"STALLED_RESET_STRIKES_ON_PROGRESS": false,
|
||||
"STALLED_IGNORE_PRIVATE": false,
|
||||
"STALLED_DELETE_PRIVATE": false
|
||||
"STALLED_DELETE_PRIVATE": false,
|
||||
"DOWNLOADING_METADATA_MAX_STRIKES": 0,
|
||||
"SLOW_MAX_STRIKES": 0,
|
||||
"SLOW_RESET_STRIKES_ON_PROGRESS": true,
|
||||
"SLOW_IGNORE_PRIVATE": false,
|
||||
"SLOW_DELETE_PRIVATE": false,
|
||||
"SLOW_MIN_SPEED": "",
|
||||
"SLOW_MAX_TIME": 0,
|
||||
"SLOW_IGNORE_ABOVE_SIZE": ""
|
||||
},
|
||||
"DownloadCleaner": {
|
||||
"Enabled": false,
|
||||
"DELETE_PRIVATE": false,
|
||||
"CATEGORIES": [],
|
||||
"UNLINKED_TARGET_CATEGORY": "cleanuperr-unlinked",
|
||||
"UNLINKED_IGNORED_ROOT_DIR": "",
|
||||
"UNLINKED_CATEGORIES": [],
|
||||
"IGNORED_DOWNLOADS_PATH": ""
|
||||
},
|
||||
"DOWNLOAD_CLIENT": "none",
|
||||
"qBittorrent": {
|
||||
"Url": "http://localhost:8080",
|
||||
"URL_BASE": "",
|
||||
"Username": "",
|
||||
"Password": ""
|
||||
},
|
||||
"Deluge": {
|
||||
"Url": "http://localhost:8112",
|
||||
"URL_BASE": "",
|
||||
"Password": "testing"
|
||||
},
|
||||
"Transmission": {
|
||||
"Url": "http://localhost:9091",
|
||||
"URL_BASE": "transmission",
|
||||
"Username": "test",
|
||||
"Password": "testing"
|
||||
},
|
||||
@@ -97,9 +112,21 @@
|
||||
"Notifiarr": {
|
||||
"ON_IMPORT_FAILED_STRIKE": false,
|
||||
"ON_STALLED_STRIKE": false,
|
||||
"ON_SLOW_STRIKE": false,
|
||||
"ON_QUEUE_ITEM_DELETED": false,
|
||||
"ON_DOWNLOAD_CLEANED": false,
|
||||
"ON_CATEGORY_CHANGED": false,
|
||||
"API_KEY": "",
|
||||
"CHANNEL_ID": ""
|
||||
},
|
||||
"Apprise": {
|
||||
"ON_IMPORT_FAILED_STRIKE": false,
|
||||
"ON_STALLED_STRIKE": false,
|
||||
"ON_SLOW_STRIKE": false,
|
||||
"ON_QUEUE_ITEM_DELETED": false,
|
||||
"ON_DOWNLOAD_CLEANED": false,
|
||||
"ON_CATEGORY_CHANGED": false,
|
||||
"URL": "",
|
||||
"KEY": ""
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.DownloadClient;
|
||||
using Infrastructure.Verticals.Files;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
@@ -56,6 +57,7 @@ public class DownloadServiceFixture : IDisposable
|
||||
var filenameEvaluator = Substitute.For<IFilenameEvaluator>();
|
||||
var notifier = Substitute.For<INotificationPublisher>();
|
||||
var dryRunInterceptor = Substitute.For<IDryRunInterceptor>();
|
||||
var hardlinkFileService = Substitute.For<IHardLinkFileService>();
|
||||
|
||||
return new TestDownloadService(
|
||||
Logger,
|
||||
@@ -66,7 +68,8 @@ public class DownloadServiceFixture : IDisposable
|
||||
filenameEvaluator,
|
||||
Striker,
|
||||
notifier,
|
||||
dryRunInterceptor
|
||||
dryRunInterceptor,
|
||||
hardlinkFileService
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -39,7 +39,7 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||
});
|
||||
|
||||
// Act
|
||||
sut.ResetStrikesOnProgress("test-hash", 100);
|
||||
sut.ResetStalledStrikesOnProgress("test-hash", 100);
|
||||
|
||||
// Assert
|
||||
_fixture.Cache.ReceivedCalls().ShouldBeEmpty();
|
||||
@@ -50,19 +50,19 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||
{
|
||||
// Arrange
|
||||
const string hash = "test-hash";
|
||||
CacheItem cacheItem = new CacheItem { Downloaded = 100 };
|
||||
StalledCacheItem stalledCacheItem = new StalledCacheItem { Downloaded = 100 };
|
||||
|
||||
_fixture.Cache.TryGetValue(Arg.Any<object>(), out Arg.Any<object?>())
|
||||
.Returns(x =>
|
||||
{
|
||||
x[1] = cacheItem;
|
||||
x[1] = stalledCacheItem;
|
||||
return true;
|
||||
});
|
||||
|
||||
TestDownloadService sut = _fixture.CreateSut();
|
||||
|
||||
// Act
|
||||
sut.ResetStrikesOnProgress(hash, 200);
|
||||
sut.ResetStalledStrikesOnProgress(hash, 200);
|
||||
|
||||
// Assert
|
||||
_fixture.Cache.Received(1).Remove(CacheKeys.Strike(StrikeType.Stalled, hash));
|
||||
@@ -73,20 +73,20 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||
{
|
||||
// Arrange
|
||||
const string hash = "test-hash";
|
||||
CacheItem cacheItem = new CacheItem { Downloaded = 200 };
|
||||
StalledCacheItem stalledCacheItem = new StalledCacheItem { Downloaded = 200 };
|
||||
|
||||
_fixture.Cache
|
||||
.TryGetValue(Arg.Any<object>(), out Arg.Any<object?>())
|
||||
.Returns(x =>
|
||||
{
|
||||
x[1] = cacheItem;
|
||||
x[1] = stalledCacheItem;
|
||||
return true;
|
||||
});
|
||||
|
||||
TestDownloadService sut = _fixture.CreateSut();
|
||||
|
||||
// Act
|
||||
sut.ResetStrikesOnProgress(hash, 100);
|
||||
sut.ResetStalledStrikesOnProgress(hash, 100);
|
||||
|
||||
// Assert
|
||||
_fixture.Cache.DidNotReceive().Remove(Arg.Any<object>());
|
||||
@@ -98,27 +98,6 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||
public StrikeAndCheckLimitTests(DownloadServiceFixture fixture) : base(fixture)
|
||||
{
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ShouldDelegateCallToStriker()
|
||||
{
|
||||
// Arrange
|
||||
const string hash = "test-hash";
|
||||
const string itemName = "test-item";
|
||||
_fixture.Striker.StrikeAndCheckLimit(hash, itemName, 3, StrikeType.Stalled)
|
||||
.Returns(true);
|
||||
|
||||
TestDownloadService sut = _fixture.CreateSut();
|
||||
|
||||
// Act
|
||||
bool result = await sut.StrikeAndCheckLimit(hash, itemName);
|
||||
|
||||
// Assert
|
||||
result.ShouldBeTrue();
|
||||
await _fixture.Striker
|
||||
.Received(1)
|
||||
.StrikeAndCheckLimit(hash, itemName, 3, StrikeType.Stalled);
|
||||
}
|
||||
}
|
||||
|
||||
public class ShouldCleanDownloadTests : DownloadServiceTests
|
||||
@@ -132,7 +111,7 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||
public void WhenRatioAndMinSeedTimeReached_ShouldReturnTrue()
|
||||
{
|
||||
// Arrange
|
||||
Category category = new()
|
||||
CleanCategory category = new()
|
||||
{
|
||||
Name = "test",
|
||||
MaxRatio = 1.0,
|
||||
@@ -158,7 +137,7 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||
public void WhenRatioReachedAndMinSeedTimeNotReached_ShouldReturnFalse()
|
||||
{
|
||||
// Arrange
|
||||
Category category = new()
|
||||
CleanCategory category = new()
|
||||
{
|
||||
Name = "test",
|
||||
MaxRatio = 1.0,
|
||||
@@ -184,7 +163,7 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||
public void WhenMaxSeedTimeReached_ShouldReturnTrue()
|
||||
{
|
||||
// Arrange
|
||||
Category category = new()
|
||||
CleanCategory category = new()
|
||||
{
|
||||
Name = "test",
|
||||
MaxRatio = -1,
|
||||
@@ -210,7 +189,7 @@ public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||
public void WhenNeitherConditionMet_ShouldReturnFalse()
|
||||
{
|
||||
// Arrange
|
||||
Category category = new()
|
||||
CleanCategory category = new()
|
||||
{
|
||||
Name = "test",
|
||||
MaxRatio = 2.0,
|
||||
|
||||
@@ -1,11 +1,13 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Domain.Enums;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.DownloadClient;
|
||||
using Infrastructure.Verticals.Files;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
@@ -25,26 +27,28 @@ public class TestDownloadService : DownloadService
|
||||
IFilenameEvaluator filenameEvaluator,
|
||||
IStriker striker,
|
||||
INotificationPublisher notifier,
|
||||
IDryRunInterceptor dryRunInterceptor
|
||||
IDryRunInterceptor dryRunInterceptor,
|
||||
IHardLinkFileService hardLinkFileService
|
||||
) : base(
|
||||
logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig, cache,
|
||||
filenameEvaluator, striker, notifier, dryRunInterceptor
|
||||
filenameEvaluator, striker, notifier, dryRunInterceptor, hardLinkFileService
|
||||
)
|
||||
{
|
||||
}
|
||||
|
||||
public override void Dispose() { }
|
||||
public override Task LoginAsync() => Task.CompletedTask;
|
||||
public override Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads) => Task.FromResult(new StalledResult());
|
||||
public override Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads) => Task.FromResult(new DownloadCheckResult());
|
||||
public override Task<BlockFilesResult> BlockUnwantedFilesAsync(string hash, BlocklistType blocklistType,
|
||||
ConcurrentBag<string> patterns, ConcurrentBag<Regex> regexes, IReadOnlyList<string> ignoredDownloads) => Task.FromResult(new BlockFilesResult());
|
||||
public override Task DeleteDownload(string hash) => Task.CompletedTask;
|
||||
public override Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories) => Task.FromResult<List<object>?>(null);
|
||||
public override Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes,
|
||||
IReadOnlyList<string> ignoredDownloads) => Task.CompletedTask;
|
||||
|
||||
public override Task CreateCategoryAsync(string name) => Task.CompletedTask;
|
||||
public override Task<List<object>?> GetSeedingDownloads() => Task.FromResult<List<object>?>(null);
|
||||
public override List<object>? FilterDownloadsToBeCleanedAsync(List<object>? downloads, List<CleanCategory> categories) => null;
|
||||
public override List<object>? FilterDownloadsToChangeCategoryAsync(List<object>? downloads, List<string> categories) => null;
|
||||
public override Task CleanDownloadsAsync(List<object>? downloads, List<CleanCategory> categoriesToClean, HashSet<string> excludedHashes, IReadOnlyList<string> ignoredDownloads) => Task.CompletedTask;
|
||||
public override Task ChangeCategoryForNoHardLinksAsync(List<object>? downloads, HashSet<string> excludedHashes, IReadOnlyList<string> ignoredDownloads) => Task.CompletedTask;
|
||||
// Expose protected methods for testing
|
||||
public new void ResetStrikesOnProgress(string hash, long downloaded) => base.ResetStrikesOnProgress(hash, downloaded);
|
||||
public new Task<bool> StrikeAndCheckLimit(string hash, string itemName) => base.StrikeAndCheckLimit(hash, itemName);
|
||||
public new SeedingCheckResult ShouldCleanDownload(double ratio, TimeSpan seedingTime, Category category) => base.ShouldCleanDownload(ratio, seedingTime, category);
|
||||
public new void ResetStalledStrikesOnProgress(string hash, long downloaded) => base.ResetStalledStrikesOnProgress(hash, downloaded);
|
||||
public new SeedingCheckResult ShouldCleanDownload(double ratio, TimeSpan seedingTime, CleanCategory category) => base.ShouldCleanDownload(ratio, seedingTime, category);
|
||||
}
|
||||
@@ -1,10 +1,11 @@
|
||||
using Domain.Models.Deluge.Response;
|
||||
using Infrastructure.Helpers;
|
||||
|
||||
namespace Infrastructure.Extensions;
|
||||
|
||||
public static class DelugeExtensions
|
||||
{
|
||||
public static bool ShouldIgnore(this TorrentStatus download, IReadOnlyList<string> ignoredDownloads)
|
||||
public static bool ShouldIgnore(this DownloadStatus download, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
foreach (string value in ignoredDownloads)
|
||||
{
|
||||
@@ -18,7 +19,7 @@ public static class DelugeExtensions
|
||||
return true;
|
||||
}
|
||||
|
||||
if (download.Trackers.Any(x => x.Url.Host.EndsWith(value, StringComparison.InvariantCultureIgnoreCase)))
|
||||
if (download.Trackers.Any(x => UriService.GetDomain(x.Url)?.EndsWith(value, StringComparison.InvariantCultureIgnoreCase) ?? false))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
55
code/Infrastructure/Extensions/IpAddressExtensions.cs
Normal file
55
code/Infrastructure/Extensions/IpAddressExtensions.cs
Normal file
@@ -0,0 +1,55 @@
|
||||
using System.Net;
|
||||
using System.Net.Sockets;
|
||||
|
||||
namespace Infrastructure.Extensions;
|
||||
|
||||
public static class IpAddressExtensions
|
||||
{
|
||||
public static bool IsLocalAddress(this IPAddress ipAddress)
|
||||
{
|
||||
// Map back to IPv4 if mapped to IPv6, for example "::ffff:1.2.3.4" to "1.2.3.4".
|
||||
if (ipAddress.IsIPv4MappedToIPv6)
|
||||
{
|
||||
ipAddress = ipAddress.MapToIPv4();
|
||||
}
|
||||
|
||||
// Checks loopback ranges for both IPv4 and IPv6.
|
||||
if (IPAddress.IsLoopback(ipAddress))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
// IPv4
|
||||
if (ipAddress.AddressFamily == AddressFamily.InterNetwork)
|
||||
{
|
||||
return IsLocalIPv4(ipAddress.GetAddressBytes());
|
||||
}
|
||||
|
||||
// IPv6
|
||||
if (ipAddress.AddressFamily == AddressFamily.InterNetworkV6)
|
||||
{
|
||||
return ipAddress.IsIPv6LinkLocal ||
|
||||
ipAddress.IsIPv6UniqueLocal ||
|
||||
ipAddress.IsIPv6SiteLocal;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
private static bool IsLocalIPv4(byte[] ipv4Bytes)
|
||||
{
|
||||
// Link local (no IP assigned by DHCP): 169.254.0.0 to 169.254.255.255 (169.254.0.0/16)
|
||||
bool IsLinkLocal() => ipv4Bytes[0] == 169 && ipv4Bytes[1] == 254;
|
||||
|
||||
// Class A private range: 10.0.0.0 – 10.255.255.255 (10.0.0.0/8)
|
||||
bool IsClassA() => ipv4Bytes[0] == 10;
|
||||
|
||||
// Class B private range: 172.16.0.0 – 172.31.255.255 (172.16.0.0/12)
|
||||
bool IsClassB() => ipv4Bytes[0] == 172 && ipv4Bytes[1] >= 16 && ipv4Bytes[1] <= 31;
|
||||
|
||||
// Class C private range: 192.168.0.0 – 192.168.255.255 (192.168.0.0/16)
|
||||
bool IsClassC() => ipv4Bytes[0] == 192 && ipv4Bytes[1] == 168;
|
||||
|
||||
return IsLinkLocal() || IsClassA() || IsClassC() || IsClassB();
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
using QBittorrent.Client;
|
||||
using Infrastructure.Helpers;
|
||||
using QBittorrent.Client;
|
||||
|
||||
namespace Infrastructure.Extensions;
|
||||
|
||||
@@ -29,9 +30,16 @@ public static class QBitExtensions
|
||||
|
||||
public static bool ShouldIgnore(this TorrentTracker tracker, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
string? trackerUrl = UriService.GetDomain(tracker.Url);
|
||||
|
||||
if (trackerUrl is null)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
foreach (string value in ignoredDownloads)
|
||||
{
|
||||
if (tracker.Url.Host.EndsWith(value, StringComparison.InvariantCultureIgnoreCase))
|
||||
if (trackerUrl.EndsWith(value, StringComparison.InvariantCultureIgnoreCase))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
using Transmission.API.RPC.Entity;
|
||||
using Infrastructure.Helpers;
|
||||
using Transmission.API.RPC.Entity;
|
||||
|
||||
namespace Infrastructure.Extensions;
|
||||
|
||||
@@ -19,7 +20,7 @@ public static class TransmissionExtensions
|
||||
}
|
||||
|
||||
bool? hasIgnoredTracker = download.Trackers?
|
||||
.Any(x => new Uri(x.Announce!).Host.EndsWith(value, StringComparison.InvariantCultureIgnoreCase));
|
||||
.Any(x => UriService.GetDomain(x.Announce)?.EndsWith(value, StringComparison.InvariantCultureIgnoreCase) ?? false);
|
||||
|
||||
if (hasIgnoredTracker is true)
|
||||
{
|
||||
|
||||
@@ -10,7 +10,7 @@ public static class CacheKeys
|
||||
public static string BlocklistPatterns(InstanceType instanceType) => $"{instanceType.ToString()}_patterns";
|
||||
public static string BlocklistRegexes(InstanceType instanceType) => $"{instanceType.ToString()}_regexes";
|
||||
|
||||
public static string Item(string hash) => $"item_{hash}";
|
||||
public static string StrikeItem(string hash, StrikeType strikeType) => $"item_{hash}_{strikeType.ToString()}";
|
||||
|
||||
public static string IgnoredDownloads(string name) => $"{name}_ignored";
|
||||
}
|
||||
37
code/Infrastructure/Helpers/UriService.cs
Normal file
37
code/Infrastructure/Helpers/UriService.cs
Normal file
@@ -0,0 +1,37 @@
|
||||
using System.Text.RegularExpressions;
|
||||
|
||||
namespace Infrastructure.Helpers;
|
||||
|
||||
public static class UriService
|
||||
{
|
||||
public static string? GetDomain(string? input)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(input))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
// add "http://" if scheme is missing to help Uri.TryCreate
|
||||
if (!input.StartsWith("http", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
input = "http://" + input;
|
||||
}
|
||||
|
||||
if (Uri.TryCreate(input, UriKind.Absolute, out var uri))
|
||||
{
|
||||
return uri.Host;
|
||||
}
|
||||
|
||||
// url might be malformed
|
||||
var regex = new Regex(@"^(?:https?:\/\/)?([^\/\?:]+)", RegexOptions.IgnoreCase);
|
||||
var match = regex.Match(input);
|
||||
|
||||
if (match.Success)
|
||||
{
|
||||
return match.Groups[1].Value;
|
||||
}
|
||||
|
||||
// could not extract
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -12,12 +12,13 @@
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="FLM.QBittorrent" Version="1.0.0" />
|
||||
<PackageReference Include="FLM.Transmission" Version="1.0.2" />
|
||||
<PackageReference Include="FLM.QBittorrent" Version="1.0.1" />
|
||||
<PackageReference Include="FLM.Transmission" Version="1.0.3" />
|
||||
<PackageReference Include="Mapster" Version="7.4.0" />
|
||||
<PackageReference Include="MassTransit" Version="8.3.6" />
|
||||
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="9.0.2" />
|
||||
<PackageReference Include="Microsoft.Extensions.Http" Version="9.0.2" />
|
||||
<PackageReference Include="Mono.Posix.NETStandard" Version="1.0.0" />
|
||||
<PackageReference Include="Quartz" Version="3.13.1" />
|
||||
<PackageReference Include="Scrutor" Version="6.0.1" />
|
||||
</ItemGroup>
|
||||
|
||||
86
code/Infrastructure/Services/CertificateValidationService.cs
Normal file
86
code/Infrastructure/Services/CertificateValidationService.cs
Normal file
@@ -0,0 +1,86 @@
|
||||
using System.Net;
|
||||
using System.Net.Security;
|
||||
using System.Security.Cryptography.X509Certificates;
|
||||
using Common.Configuration.General;
|
||||
using Common.Enums;
|
||||
using Infrastructure.Extensions;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
namespace Infrastructure.Services;
|
||||
|
||||
public class CertificateValidationService
|
||||
{
|
||||
private readonly ILogger<CertificateValidationService> _logger;
|
||||
private readonly HttpConfig _config;
|
||||
|
||||
public CertificateValidationService(ILogger<CertificateValidationService> logger, IOptions<HttpConfig> config)
|
||||
{
|
||||
_logger = logger;
|
||||
_config = config.Value;
|
||||
}
|
||||
|
||||
public bool ShouldByPassValidationError(object sender, X509Certificate? certificate, X509Chain? chain, SslPolicyErrors sslPolicyErrors)
|
||||
{
|
||||
var targetHostName = string.Empty;
|
||||
|
||||
if (sender is not SslStream && sender is not string)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
if (sender is SslStream request)
|
||||
{
|
||||
targetHostName = request.TargetHostName;
|
||||
}
|
||||
|
||||
// Mailkit passes host in sender as string
|
||||
if (sender is string stringHost)
|
||||
{
|
||||
targetHostName = stringHost;
|
||||
}
|
||||
|
||||
if (certificate is X509Certificate2 cert2 && cert2.SignatureAlgorithm.FriendlyName == "md5RSA")
|
||||
{
|
||||
_logger.LogError(
|
||||
$"https://{targetHostName} uses the obsolete md5 hash in its https certificate, if that is your certificate, please (re)create certificate with better algorithm as soon as possible.");
|
||||
}
|
||||
|
||||
if (sslPolicyErrors == SslPolicyErrors.None)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
if (targetHostName is "localhost" or "127.0.0.1")
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
var ipAddresses = GetIpAddresses(targetHostName);
|
||||
|
||||
if (_config.CertificateValidation == CertificateValidationType.Disabled)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
if (_config.CertificateValidation == CertificateValidationType.DisabledForLocalAddresses &&
|
||||
ipAddresses.All(i => i.IsLocalAddress()))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
_logger.LogError($"certificate validation for {targetHostName} failed. {sslPolicyErrors}");
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
private static IPAddress[] GetIpAddresses(string host)
|
||||
{
|
||||
if (IPAddress.TryParse(host, out var ipAddress))
|
||||
{
|
||||
return [ipAddress];
|
||||
}
|
||||
|
||||
return Dns.GetHostEntry(host).AddressList;
|
||||
}
|
||||
}
|
||||
@@ -43,9 +43,11 @@ public abstract class ArrClient : IArrClient
|
||||
|
||||
public virtual async Task<QueueListResponse> GetQueueItemsAsync(ArrInstance arrInstance, int page)
|
||||
{
|
||||
Uri uri = new(arrInstance.Url, GetQueueUrlPath(page));
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/{GetQueueUrlPath().TrimStart('/')}";
|
||||
uriBuilder.Query = GetQueueUrlQuery(page);
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uri);
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||
@@ -56,7 +58,7 @@ public abstract class ArrClient : IArrClient
|
||||
}
|
||||
catch
|
||||
{
|
||||
_logger.LogError("queue list failed | {uri}", uri);
|
||||
_logger.LogError("queue list failed | {uri}", uriBuilder.Uri);
|
||||
throw;
|
||||
}
|
||||
|
||||
@@ -65,7 +67,7 @@ public abstract class ArrClient : IArrClient
|
||||
|
||||
if (queueResponse is null)
|
||||
{
|
||||
throw new Exception($"unrecognized queue list response | {uri} | {responseBody}");
|
||||
throw new Exception($"unrecognized queue list response | {uriBuilder.Uri} | {responseBody}");
|
||||
}
|
||||
|
||||
return queueResponse;
|
||||
@@ -112,13 +114,20 @@ public abstract class ArrClient : IArrClient
|
||||
return false;
|
||||
}
|
||||
|
||||
public virtual async Task DeleteQueueItemAsync(ArrInstance arrInstance, QueueRecord record, bool removeFromClient)
|
||||
public virtual async Task DeleteQueueItemAsync(
|
||||
ArrInstance arrInstance,
|
||||
QueueRecord record,
|
||||
bool removeFromClient,
|
||||
DeleteReason deleteReason
|
||||
)
|
||||
{
|
||||
Uri uri = new(arrInstance.Url, GetQueueDeleteUrlPath(record.Id, removeFromClient));
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/{GetQueueDeleteUrlPath(record.Id).TrimStart('/')}";
|
||||
uriBuilder.Query = GetQueueDeleteUrlQuery(removeFromClient);
|
||||
|
||||
try
|
||||
{
|
||||
using HttpRequestMessage request = new(HttpMethod.Delete, uri);
|
||||
using HttpRequestMessage request = new(HttpMethod.Delete, uriBuilder.Uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
HttpResponseMessage? response = await _dryRunInterceptor.InterceptAsync<HttpResponseMessage>(SendRequestAsync, request);
|
||||
@@ -126,15 +135,16 @@ public abstract class ArrClient : IArrClient
|
||||
|
||||
_logger.LogInformation(
|
||||
removeFromClient
|
||||
? "queue item deleted | {url} | {title}"
|
||||
: "queue item removed from arr | {url} | {title}",
|
||||
? "queue item deleted with reason {reason} | {url} | {title}"
|
||||
: "queue item removed from arr with reason {reason} | {url} | {title}",
|
||||
deleteReason.ToString(),
|
||||
arrInstance.Url,
|
||||
record.Title
|
||||
);
|
||||
}
|
||||
catch
|
||||
{
|
||||
_logger.LogError("queue delete failed | {uri} | {title}", uri, record.Title);
|
||||
_logger.LogError("queue delete failed | {uri} | {title}", uriBuilder.Uri, record.Title);
|
||||
throw;
|
||||
}
|
||||
}
|
||||
@@ -152,9 +162,13 @@ public abstract class ArrClient : IArrClient
|
||||
return true;
|
||||
}
|
||||
|
||||
protected abstract string GetQueueUrlPath(int page);
|
||||
protected abstract string GetQueueUrlPath();
|
||||
|
||||
protected abstract string GetQueueDeleteUrlPath(long recordId, bool removeFromClient);
|
||||
protected abstract string GetQueueUrlQuery(int page);
|
||||
|
||||
protected abstract string GetQueueDeleteUrlPath(long recordId);
|
||||
|
||||
protected abstract string GetQueueDeleteUrlQuery(bool removeFromClient);
|
||||
|
||||
protected virtual void SetApiKey(HttpRequestMessage request, string apiKey)
|
||||
{
|
||||
|
||||
@@ -11,7 +11,7 @@ public interface IArrClient
|
||||
|
||||
Task<bool> ShouldRemoveFromQueue(InstanceType instanceType, QueueRecord record, bool isPrivateDownload);
|
||||
|
||||
Task DeleteQueueItemAsync(ArrInstance arrInstance, QueueRecord record, bool removeFromClient);
|
||||
Task DeleteQueueItemAsync(ArrInstance arrInstance, QueueRecord record, bool removeFromClient, DeleteReason deleteReason);
|
||||
|
||||
Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items);
|
||||
|
||||
|
||||
@@ -27,29 +27,42 @@ public class LidarrClient : ArrClient, ILidarrClient
|
||||
{
|
||||
}
|
||||
|
||||
protected override string GetQueueUrlPath(int page)
|
||||
protected override string GetQueueUrlPath()
|
||||
{
|
||||
return $"/api/v1/queue?page={page}&pageSize=200&includeUnknownArtistItems=true&includeArtist=true&includeAlbum=true";
|
||||
return "/api/v1/queue";
|
||||
}
|
||||
|
||||
protected override string GetQueueDeleteUrlPath(long recordId, bool removeFromClient)
|
||||
protected override string GetQueueUrlQuery(int page)
|
||||
{
|
||||
string path = $"/api/v1/queue/{recordId}?blocklist=true&skipRedownload=true&changeCategory=false";
|
||||
return $"page={page}&pageSize=200&includeUnknownArtistItems=true&includeArtist=true&includeAlbum=true";
|
||||
}
|
||||
|
||||
path += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
|
||||
protected override string GetQueueDeleteUrlPath(long recordId)
|
||||
{
|
||||
return $"/api/v1/queue/{recordId}";
|
||||
}
|
||||
|
||||
return path;
|
||||
protected override string GetQueueDeleteUrlQuery(bool removeFromClient)
|
||||
{
|
||||
string query = "blocklist=true&skipRedownload=true&changeCategory=false";
|
||||
query += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
|
||||
|
||||
return query;
|
||||
}
|
||||
|
||||
public override async Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items)
|
||||
{
|
||||
if (items?.Count is null or 0) return;
|
||||
if (items?.Count is null or 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
Uri uri = new(arrInstance.Url, "/api/v1/command");
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v1/command";
|
||||
|
||||
foreach (var command in GetSearchCommands(items))
|
||||
{
|
||||
using HttpRequestMessage request = new(HttpMethod.Post, uri);
|
||||
using HttpRequestMessage request = new(HttpMethod.Post, uriBuilder.Uri);
|
||||
request.Content = new StringContent(
|
||||
JsonConvert.SerializeObject(command, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore }),
|
||||
Encoding.UTF8,
|
||||
@@ -132,8 +145,11 @@ public class LidarrClient : ArrClient, ILidarrClient
|
||||
|
||||
private async Task<List<Album>?> GetAlbumsAsync(ArrInstance arrInstance, List<long> albumIds)
|
||||
{
|
||||
Uri uri = new(arrInstance.Url, $"api/v1/album?{string.Join('&', albumIds.Select(x => $"albumIds={x}"))}");
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uri);
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v1/album";
|
||||
uriBuilder.Query = string.Join('&', albumIds.Select(x => $"albumIds={x}"));
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using var response = await _httpClient.SendAsync(request);
|
||||
|
||||
@@ -27,18 +27,27 @@ public class RadarrClient : ArrClient, IRadarrClient
|
||||
{
|
||||
}
|
||||
|
||||
protected override string GetQueueUrlPath(int page)
|
||||
protected override string GetQueueUrlPath()
|
||||
{
|
||||
return $"/api/v3/queue?page={page}&pageSize=200&includeUnknownMovieItems=true&includeMovie=true";
|
||||
return "/api/v3/queue";
|
||||
}
|
||||
|
||||
protected override string GetQueueDeleteUrlPath(long recordId, bool removeFromClient)
|
||||
protected override string GetQueueUrlQuery(int page)
|
||||
{
|
||||
string path = $"/api/v3/queue/{recordId}?blocklist=true&skipRedownload=true&changeCategory=false";
|
||||
|
||||
path += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
|
||||
return $"page={page}&pageSize=200&includeUnknownMovieItems=true&includeMovie=true";
|
||||
}
|
||||
|
||||
return path;
|
||||
protected override string GetQueueDeleteUrlPath(long recordId)
|
||||
{
|
||||
return $"/api/v3/queue/{recordId}";
|
||||
}
|
||||
|
||||
protected override string GetQueueDeleteUrlQuery(bool removeFromClient)
|
||||
{
|
||||
string query = "blocklist=true&skipRedownload=true&changeCategory=false";
|
||||
query += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
|
||||
|
||||
return query;
|
||||
}
|
||||
|
||||
public override async Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items)
|
||||
@@ -50,14 +59,16 @@ public class RadarrClient : ArrClient, IRadarrClient
|
||||
|
||||
List<long> ids = items.Select(item => item.Id).ToList();
|
||||
|
||||
Uri uri = new(arrInstance.Url, "/api/v3/command");
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/command";
|
||||
|
||||
RadarrCommand command = new()
|
||||
{
|
||||
Name = "MoviesSearch",
|
||||
MovieIds = ids,
|
||||
};
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Post, uri);
|
||||
using HttpRequestMessage request = new(HttpMethod.Post, uriBuilder.Uri);
|
||||
request.Content = new StringContent(
|
||||
JsonConvert.SerializeObject(command),
|
||||
Encoding.UTF8,
|
||||
@@ -135,8 +146,10 @@ public class RadarrClient : ArrClient, IRadarrClient
|
||||
|
||||
private async Task<Movie?> GetMovie(ArrInstance arrInstance, long movieId)
|
||||
{
|
||||
Uri uri = new(arrInstance.Url, $"api/v3/movie/{movieId}");
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uri);
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/movie/{movieId}";
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||
|
||||
@@ -28,18 +28,27 @@ public class SonarrClient : ArrClient, ISonarrClient
|
||||
{
|
||||
}
|
||||
|
||||
protected override string GetQueueUrlPath(int page)
|
||||
protected override string GetQueueUrlPath()
|
||||
{
|
||||
return $"/api/v3/queue?page={page}&pageSize=200&includeUnknownSeriesItems=true&includeSeries=true&includeEpisode=true";
|
||||
return "/api/v3/queue";
|
||||
}
|
||||
|
||||
protected override string GetQueueDeleteUrlPath(long recordId, bool removeFromClient)
|
||||
|
||||
protected override string GetQueueUrlQuery(int page)
|
||||
{
|
||||
string path = $"/api/v3/queue/{recordId}?blocklist=true&skipRedownload=true&changeCategory=false";
|
||||
return $"page={page}&pageSize=200&includeUnknownSeriesItems=true&includeSeries=true&includeEpisode=true";
|
||||
}
|
||||
|
||||
path += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
|
||||
protected override string GetQueueDeleteUrlPath(long recordId)
|
||||
{
|
||||
return $"/api/v3/queue/{recordId}";
|
||||
}
|
||||
|
||||
return path;
|
||||
protected override string GetQueueDeleteUrlQuery(bool removeFromClient)
|
||||
{
|
||||
string query = "blocklist=true&skipRedownload=true&changeCategory=false";
|
||||
query += removeFromClient ? "&removeFromClient=true" : "&removeFromClient=false";
|
||||
|
||||
return query;
|
||||
}
|
||||
|
||||
public override async Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items)
|
||||
@@ -49,11 +58,12 @@ public class SonarrClient : ArrClient, ISonarrClient
|
||||
return;
|
||||
}
|
||||
|
||||
Uri uri = new(arrInstance.Url, "/api/v3/command");
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/command";
|
||||
|
||||
foreach (SonarrCommand command in GetSearchCommands(items.Cast<SonarrSearchItem>().ToHashSet()))
|
||||
{
|
||||
using HttpRequestMessage request = new(HttpMethod.Post, uri);
|
||||
using HttpRequestMessage request = new(HttpMethod.Post, uriBuilder.Uri);
|
||||
request.Content = new StringContent(
|
||||
JsonConvert.SerializeObject(command, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore }),
|
||||
Encoding.UTF8,
|
||||
@@ -199,8 +209,11 @@ public class SonarrClient : ArrClient, ISonarrClient
|
||||
|
||||
private async Task<List<Episode>?> GetEpisodesAsync(ArrInstance arrInstance, List<long> episodeIds)
|
||||
{
|
||||
Uri uri = new(arrInstance.Url, $"api/v3/episode?{string.Join('&', episodeIds.Select(x => $"episodeIds={x}"))}");
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uri);
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/episode";
|
||||
uriBuilder.Query = string.Join('&', episodeIds.Select(x => $"episodeIds={x}"));
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||
@@ -212,8 +225,10 @@ public class SonarrClient : ArrClient, ISonarrClient
|
||||
|
||||
private async Task<Series?> GetSeriesAsync(ArrInstance arrInstance, long seriesId)
|
||||
{
|
||||
Uri uri = new(arrInstance.Url, $"api/v3/series/{seriesId}");
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uri);
|
||||
UriBuilder uriBuilder = new(arrInstance.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/api/v3/series/{seriesId}";
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Get, uriBuilder.Uri);
|
||||
SetApiKey(request, arrInstance.ApiKey);
|
||||
|
||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||
|
||||
@@ -43,7 +43,7 @@ public sealed class BlocklistProvider
|
||||
{
|
||||
if (_initialized)
|
||||
{
|
||||
_logger.LogDebug("blocklists already loaded");
|
||||
_logger.LogTrace("blocklists already loaded");
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
@@ -142,7 +142,7 @@ public sealed class ContentBlocker : GenericHandler
|
||||
removeFromClient = false;
|
||||
}
|
||||
|
||||
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient);
|
||||
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient, DeleteReason.AllFilesBlocked);
|
||||
await _notifier.NotifyQueueItemDeleted(removeFromClient, DeleteReason.AllFilesBlocked);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -21,6 +21,8 @@ public sealed class DownloadCleaner : GenericHandler
|
||||
private readonly IgnoredDownloadsProvider<DownloadCleanerConfig> _ignoredDownloadsProvider;
|
||||
private readonly HashSet<string> _excludedHashes = [];
|
||||
|
||||
private static bool _hardLinkCategoryCreated;
|
||||
|
||||
public DownloadCleaner(
|
||||
ILogger<DownloadCleaner> logger,
|
||||
IOptions<DownloadCleanerConfig> config,
|
||||
@@ -65,13 +67,20 @@ public sealed class DownloadCleaner : GenericHandler
|
||||
IReadOnlyList<string> ignoredDownloads = await _ignoredDownloadsProvider.GetIgnoredDownloads();
|
||||
|
||||
await _downloadService.LoginAsync();
|
||||
|
||||
List<object>? downloads = await _downloadService.GetAllDownloadsToBeCleaned(_config.Categories);
|
||||
|
||||
if (downloads?.Count is null or 0)
|
||||
List<object>? downloads = await _downloadService.GetSeedingDownloads();
|
||||
List<object>? downloadsToChangeCategory = null;
|
||||
|
||||
if (!string.IsNullOrEmpty(_config.UnlinkedTargetCategory) && _config.UnlinkedCategories?.Count > 0)
|
||||
{
|
||||
_logger.LogDebug("no downloads found in the download client");
|
||||
return;
|
||||
if (!_hardLinkCategoryCreated)
|
||||
{
|
||||
_logger.LogDebug("creating category {cat}", _config.UnlinkedTargetCategory);
|
||||
|
||||
await _downloadService.CreateCategoryAsync(_config.UnlinkedTargetCategory);
|
||||
_hardLinkCategoryCreated = true;
|
||||
}
|
||||
|
||||
downloadsToChangeCategory = _downloadService.FilterDownloadsToChangeCategoryAsync(downloads, _config.UnlinkedCategories);
|
||||
}
|
||||
|
||||
// wait for the downloads to appear in the arr queue
|
||||
@@ -81,7 +90,16 @@ public sealed class DownloadCleaner : GenericHandler
|
||||
await ProcessArrConfigAsync(_radarrConfig, InstanceType.Radarr, true);
|
||||
await ProcessArrConfigAsync(_lidarrConfig, InstanceType.Lidarr, true);
|
||||
|
||||
await _downloadService.CleanDownloads(downloads, _config.Categories, _excludedHashes, ignoredDownloads);
|
||||
_logger.LogTrace("looking for downloads to change category");
|
||||
await _downloadService.ChangeCategoryForNoHardLinksAsync(downloadsToChangeCategory, _excludedHashes, ignoredDownloads);
|
||||
|
||||
List<object>? downloadsToClean = _downloadService.FilterDownloadsToBeCleanedAsync(downloads, _config.Categories);
|
||||
|
||||
// release unused objects
|
||||
downloads = null;
|
||||
|
||||
_logger.LogTrace("looking for downloads to clean");
|
||||
await _downloadService.CleanDownloadsAsync(downloadsToClean, _config.Categories, _excludedHashes, ignoredDownloads);
|
||||
}
|
||||
|
||||
protected override async Task ProcessInstanceAsync(ArrInstance instance, InstanceType instanceType)
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
using System.Text.Json.Serialization;
|
||||
using Common.Configuration;
|
||||
using Common.Configuration.DownloadClient;
|
||||
using Common.Exceptions;
|
||||
using Domain.Models.Deluge.Exceptions;
|
||||
using Domain.Models.Deluge.Request;
|
||||
using Domain.Models.Deluge.Response;
|
||||
@@ -27,12 +28,16 @@ public sealed class DelugeClient
|
||||
"label",
|
||||
"seeding_time",
|
||||
"ratio",
|
||||
"trackers"
|
||||
"trackers",
|
||||
"download_payload_rate",
|
||||
"total_size",
|
||||
"download_location"
|
||||
];
|
||||
|
||||
public DelugeClient(IOptions<DelugeConfig> config, IHttpClientFactory httpClientFactory)
|
||||
{
|
||||
_config = config.Value;
|
||||
_config.Validate();
|
||||
_httpClient = httpClientFactory.CreateClient(nameof(DelugeService));
|
||||
}
|
||||
|
||||
@@ -41,11 +46,42 @@ public sealed class DelugeClient
|
||||
return await SendRequest<bool>("auth.login", _config.Password);
|
||||
}
|
||||
|
||||
public async Task<bool> IsConnected()
|
||||
{
|
||||
return await SendRequest<bool>("web.connected");
|
||||
}
|
||||
|
||||
public async Task<bool> Connect()
|
||||
{
|
||||
string? firstHost = await GetHost();
|
||||
|
||||
if (string.IsNullOrEmpty(firstHost))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
var result = await SendRequest<List<string>?>("web.connect", firstHost);
|
||||
|
||||
return result?.Count > 0;
|
||||
}
|
||||
|
||||
public async Task<bool> Logout()
|
||||
{
|
||||
return await SendRequest<bool>("auth.delete_session");
|
||||
}
|
||||
|
||||
public async Task<string?> GetHost()
|
||||
{
|
||||
var hosts = await SendRequest<List<List<string>?>?>("web.get_hosts");
|
||||
|
||||
if (hosts?.Count > 1)
|
||||
{
|
||||
throw new FatalException("multiple Deluge hosts found - please connect to only one host");
|
||||
}
|
||||
|
||||
return hosts?.FirstOrDefault()?.FirstOrDefault();
|
||||
}
|
||||
|
||||
public async Task<List<DelugeTorrent>> ListTorrents(Dictionary<string, string>? filters = null)
|
||||
{
|
||||
filters ??= new Dictionary<string, string>();
|
||||
@@ -77,18 +113,31 @@ public sealed class DelugeClient
|
||||
return torrents.FirstOrDefault();
|
||||
}
|
||||
|
||||
public async Task<TorrentStatus?> GetTorrentStatus(string hash)
|
||||
public async Task<DownloadStatus?> GetTorrentStatus(string hash)
|
||||
{
|
||||
return await SendRequest<TorrentStatus?>(
|
||||
"web.get_torrent_status",
|
||||
hash,
|
||||
Fields
|
||||
);
|
||||
try
|
||||
{
|
||||
return await SendRequest<DownloadStatus?>(
|
||||
"web.get_torrent_status",
|
||||
hash,
|
||||
Fields
|
||||
);
|
||||
}
|
||||
catch (DelugeClientException e)
|
||||
{
|
||||
// Deluge returns an error when the torrent is not found
|
||||
if (e.Message == "AttributeError: 'NoneType' object has no attribute 'call'")
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
public async Task<List<TorrentStatus>?> GetStatusForAllTorrents()
|
||||
public async Task<List<DownloadStatus>?> GetStatusForAllTorrents()
|
||||
{
|
||||
Dictionary<string, TorrentStatus>? downloads = await SendRequest<Dictionary<string, TorrentStatus>?>(
|
||||
Dictionary<string, DownloadStatus>? downloads = await SendRequest<Dictionary<string, DownloadStatus>?>(
|
||||
"core.get_torrents_status",
|
||||
"",
|
||||
Fields
|
||||
@@ -121,15 +170,19 @@ public sealed class DelugeClient
|
||||
{
|
||||
StringContent content = new StringContent(json);
|
||||
content.Headers.ContentType = new MediaTypeWithQualityHeaderValue("application/json");
|
||||
|
||||
var responseMessage = await _httpClient.PostAsync(new Uri(_config.Url, "/json"), content);
|
||||
|
||||
UriBuilder uriBuilder = new(_config.Url);
|
||||
uriBuilder.Path = string.IsNullOrEmpty(_config.UrlBase)
|
||||
? $"{uriBuilder.Path.TrimEnd('/')}/json"
|
||||
: $"{uriBuilder.Path.TrimEnd('/')}/{_config.UrlBase.TrimStart('/').TrimEnd('/')}/json";
|
||||
var responseMessage = await _httpClient.PostAsync(uriBuilder.Uri, content);
|
||||
responseMessage.EnsureSuccessStatusCode();
|
||||
|
||||
var responseJson = await responseMessage.Content.ReadAsStringAsync();
|
||||
return responseJson;
|
||||
}
|
||||
|
||||
private DelugeRequest CreateRequest(string method, params object[] parameters)
|
||||
private static DelugeRequest CreateRequest(string method, params object[] parameters)
|
||||
{
|
||||
if (String.IsNullOrWhiteSpace(method))
|
||||
{
|
||||
@@ -175,4 +228,19 @@ public sealed class DelugeClient
|
||||
|
||||
return webResponse.Result;
|
||||
}
|
||||
|
||||
public async Task<IReadOnlyList<string>> GetLabels()
|
||||
{
|
||||
return await SendRequest<IReadOnlyList<string>>("label.get_labels");
|
||||
}
|
||||
|
||||
public async Task CreateLabel(string label)
|
||||
{
|
||||
await SendRequest<DelugeResponse<object>>("label.add", label);
|
||||
}
|
||||
|
||||
public async Task SetTorrentLabel(string hash, string newLabel)
|
||||
{
|
||||
await SendRequest<DelugeResponse<object>>("label.set_torrent", hash, newLabel);
|
||||
}
|
||||
}
|
||||
@@ -1,16 +1,20 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Globalization;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Attributes;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.DownloadClient;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Common.CustomDataTypes;
|
||||
using Common.Exceptions;
|
||||
using Domain.Enums;
|
||||
using Domain.Models.Deluge.Response;
|
||||
using Infrastructure.Extensions;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.Context;
|
||||
using Infrastructure.Verticals.Files;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
@@ -34,10 +38,11 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
IFilenameEvaluator filenameEvaluator,
|
||||
IStriker striker,
|
||||
INotificationPublisher notifier,
|
||||
IDryRunInterceptor dryRunInterceptor
|
||||
IDryRunInterceptor dryRunInterceptor,
|
||||
IHardLinkFileService hardLinkFileService
|
||||
) : base(
|
||||
logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig, cache,
|
||||
filenameEvaluator, striker, notifier, dryRunInterceptor
|
||||
filenameEvaluator, striker, notifier, dryRunInterceptor, hardLinkFileService
|
||||
)
|
||||
{
|
||||
config.Value.Validate();
|
||||
@@ -47,17 +52,22 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
public override async Task LoginAsync()
|
||||
{
|
||||
await _client.LoginAsync();
|
||||
|
||||
if (!await _client.IsConnected() && !await _client.Connect())
|
||||
{
|
||||
throw new FatalException("Deluge WebUI is not connected to the daemon");
|
||||
}
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
public override async Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
hash = hash.ToLowerInvariant();
|
||||
|
||||
DelugeContents? contents = null;
|
||||
StalledResult result = new();
|
||||
DownloadCheckResult result = new();
|
||||
|
||||
TorrentStatus? download = await _client.GetTorrentStatus(hash);
|
||||
DownloadStatus? download = await _client.GetTorrentStatus(hash);
|
||||
|
||||
if (download?.Hash is null)
|
||||
{
|
||||
@@ -65,6 +75,8 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
return result;
|
||||
}
|
||||
|
||||
result.IsPrivate = download.Private;
|
||||
|
||||
if (ignoredDownloads.Count > 0 && download.ShouldIgnore(ignoredDownloads))
|
||||
{
|
||||
_logger.LogInformation("skip | download is ignored | {name}", download.Name);
|
||||
@@ -79,6 +91,7 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
{
|
||||
_logger.LogDebug(exception, "failed to find torrent {hash} in the download client", hash);
|
||||
}
|
||||
|
||||
|
||||
bool shouldRemove = contents?.Contents?.Count > 0;
|
||||
|
||||
@@ -92,17 +105,15 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
|
||||
if (shouldRemove)
|
||||
{
|
||||
result.DeleteReason = DeleteReason.AllFilesBlocked;
|
||||
// remove if all files are unwanted
|
||||
result.ShouldRemove = true;
|
||||
result.DeleteReason = DeleteReason.AllFilesSkipped;
|
||||
return result;
|
||||
}
|
||||
|
||||
result.ShouldRemove = shouldRemove || await IsItemStuckAndShouldRemove(download);
|
||||
result.IsPrivate = download.Private;
|
||||
// remove if download is stuck
|
||||
(result.ShouldRemove, result.DeleteReason) = await EvaluateDownloadRemoval(download);
|
||||
|
||||
if (!shouldRemove && result.ShouldRemove)
|
||||
{
|
||||
result.DeleteReason = DeleteReason.Stalled;
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -114,7 +125,7 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
{
|
||||
hash = hash.ToLowerInvariant();
|
||||
|
||||
TorrentStatus? download = await _client.GetTorrentStatus(hash);
|
||||
DownloadStatus? download = await _client.GetTorrentStatus(hash);
|
||||
BlockFilesResult result = new();
|
||||
|
||||
if (download?.Hash is null)
|
||||
@@ -123,9 +134,6 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
return result;
|
||||
}
|
||||
|
||||
var ceva = await _client.GetTorrentExtended(hash);
|
||||
|
||||
|
||||
if (ignoredDownloads.Count > 0 && download.ShouldIgnore(ignoredDownloads))
|
||||
{
|
||||
_logger.LogInformation("skip | download is ignored | {name}", download.Name);
|
||||
@@ -208,26 +216,51 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
return result;
|
||||
}
|
||||
|
||||
public override async Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories)
|
||||
public override async Task<List<object>?> GetSeedingDownloads()
|
||||
{
|
||||
return (await _client.GetStatusForAllTorrents())
|
||||
?.Where(x => !string.IsNullOrEmpty(x.Hash))
|
||||
.Where(x => x.State?.Equals("seeding", StringComparison.InvariantCultureIgnoreCase) is true)
|
||||
.Where(x => categories.Any(cat => cat.Name.Equals(x.Label, StringComparison.InvariantCultureIgnoreCase)))
|
||||
.Cast<object>()
|
||||
.ToList();
|
||||
}
|
||||
|
||||
public override List<object>? FilterDownloadsToBeCleanedAsync(List<object>? downloads, List<CleanCategory> categories) =>
|
||||
downloads
|
||||
?.Cast<DownloadStatus>()
|
||||
.Where(x => categories.Any(cat => cat.Name.Equals(x.Label, StringComparison.InvariantCultureIgnoreCase)))
|
||||
.Cast<object>()
|
||||
.ToList();
|
||||
|
||||
public override List<object>? FilterDownloadsToChangeCategoryAsync(List<object>? downloads, List<string> categories) =>
|
||||
downloads
|
||||
?.Cast<DownloadStatus>()
|
||||
.Where(x => !string.IsNullOrEmpty(x.Hash))
|
||||
.Where(x => categories.Any(cat => cat.Equals(x.Label, StringComparison.InvariantCultureIgnoreCase)))
|
||||
.Cast<object>()
|
||||
.ToList();
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes,
|
||||
public override async Task CleanDownloadsAsync(List<object>? downloads, List<CleanCategory> categoriesToClean, HashSet<string> excludedHashes,
|
||||
IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
foreach (TorrentStatus download in downloads)
|
||||
if (downloads?.Count is null or 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
foreach (DownloadStatus download in downloads)
|
||||
{
|
||||
if (string.IsNullOrEmpty(download.Hash))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
if (excludedHashes.Any(x => x.Equals(download.Hash, StringComparison.InvariantCultureIgnoreCase)))
|
||||
{
|
||||
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ignoredDownloads.Count > 0 && download.ShouldIgnore(ignoredDownloads))
|
||||
{
|
||||
@@ -235,19 +268,13 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
continue;
|
||||
}
|
||||
|
||||
Category? category = categoriesToClean
|
||||
CleanCategory? category = categoriesToClean
|
||||
.FirstOrDefault(x => x.Name.Equals(download.Label, StringComparison.InvariantCultureIgnoreCase));
|
||||
|
||||
if (category is null)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
if (excludedHashes.Any(x => x.Equals(download.Hash, StringComparison.InvariantCultureIgnoreCase)))
|
||||
{
|
||||
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!_downloadCleanerConfig.DeletePrivate && download.Private)
|
||||
{
|
||||
@@ -279,7 +306,107 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
await _notifier.NotifyDownloadCleaned(download.Ratio, seedingTime, category.Name, result.Reason);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
public override async Task CreateCategoryAsync(string name)
|
||||
{
|
||||
IReadOnlyList<string> existingLabels = await _client.GetLabels();
|
||||
|
||||
if (existingLabels.Contains(name, StringComparer.InvariantCultureIgnoreCase))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
await _dryRunInterceptor.InterceptAsync(CreateLabel, name);
|
||||
}
|
||||
|
||||
public override async Task ChangeCategoryForNoHardLinksAsync(List<object>? downloads, HashSet<string> excludedHashes, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
if (downloads?.Count is null or 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (!string.IsNullOrEmpty(_downloadCleanerConfig.UnlinkedIgnoredRootDir))
|
||||
{
|
||||
_hardLinkFileService.PopulateFileCounts(_downloadCleanerConfig.UnlinkedIgnoredRootDir);
|
||||
}
|
||||
|
||||
foreach (DownloadStatus download in downloads.Cast<DownloadStatus>())
|
||||
{
|
||||
if (string.IsNullOrEmpty(download.Hash) || string.IsNullOrEmpty(download.Name) || string.IsNullOrEmpty(download.Label))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
if (excludedHashes.Any(x => x.Equals(download.Hash, StringComparison.InvariantCultureIgnoreCase)))
|
||||
{
|
||||
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ignoredDownloads.Count > 0 && download.ShouldIgnore(ignoredDownloads))
|
||||
{
|
||||
_logger.LogInformation("skip | download is ignored | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
ContextProvider.Set("downloadName", download.Name);
|
||||
ContextProvider.Set("hash", download.Hash);
|
||||
|
||||
DelugeContents? contents = null;
|
||||
try
|
||||
{
|
||||
contents = await _client.GetTorrentFiles(download.Hash);
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
_logger.LogDebug(exception, "failed to find torrent files for {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
bool hasHardlinks = false;
|
||||
|
||||
ProcessFiles(contents?.Contents, (_, file) =>
|
||||
{
|
||||
string filePath = string.Join(Path.DirectorySeparatorChar, Path.Combine(download.DownloadLocation, file.Path).Split(['\\', '/']));
|
||||
|
||||
if (file.Priority <= 0)
|
||||
{
|
||||
_logger.LogDebug("skip | file is not downloaded | {file}", filePath);
|
||||
return;
|
||||
}
|
||||
|
||||
long hardlinkCount = _hardLinkFileService.GetHardLinkCount(filePath, !string.IsNullOrEmpty(_downloadCleanerConfig.UnlinkedIgnoredRootDir));
|
||||
|
||||
if (hardlinkCount < 0)
|
||||
{
|
||||
_logger.LogDebug("skip | could not get file properties | {file}", filePath);
|
||||
hasHardlinks = true;
|
||||
return;
|
||||
}
|
||||
|
||||
if (hardlinkCount > 0)
|
||||
{
|
||||
hasHardlinks = true;
|
||||
}
|
||||
});
|
||||
|
||||
if (hasHardlinks)
|
||||
{
|
||||
_logger.LogDebug("skip | download has hardlinks | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
await _dryRunInterceptor.InterceptAsync(ChangeLabel, download.Hash, _downloadCleanerConfig.UnlinkedTargetCategory);
|
||||
|
||||
_logger.LogInformation("category changed for {name}", download.Name);
|
||||
|
||||
await _notifier.NotifyCategoryChanged(download.Label, _downloadCleanerConfig.UnlinkedTargetCategory);
|
||||
|
||||
download.Label = _downloadCleanerConfig.UnlinkedTargetCategory;
|
||||
}
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
[DryRunSafeguard]
|
||||
public override async Task DeleteDownload(string hash)
|
||||
@@ -288,6 +415,12 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
|
||||
await _client.DeleteTorrents([hash]);
|
||||
}
|
||||
|
||||
[DryRunSafeguard]
|
||||
protected async Task CreateLabel(string name)
|
||||
{
|
||||
await _client.CreateLabel(name);
|
||||
}
|
||||
|
||||
[DryRunSafeguard]
|
||||
protected virtual async Task ChangeFilesPriority(string hash, List<int> sortedPriorities)
|
||||
@@ -295,33 +428,96 @@ public class DelugeService : DownloadService, IDelugeService
|
||||
await _client.ChangeFilesPriority(hash, sortedPriorities);
|
||||
}
|
||||
|
||||
private async Task<bool> IsItemStuckAndShouldRemove(TorrentStatus status)
|
||||
[DryRunSafeguard]
|
||||
protected virtual async Task ChangeLabel(string hash, string newLabel)
|
||||
{
|
||||
await _client.SetTorrentLabel(hash, newLabel);
|
||||
}
|
||||
|
||||
private async Task<(bool, DeleteReason)> EvaluateDownloadRemoval(DownloadStatus status)
|
||||
{
|
||||
(bool ShouldRemove, DeleteReason Reason) result = await CheckIfSlow(status);
|
||||
|
||||
if (result.ShouldRemove)
|
||||
{
|
||||
return result;
|
||||
}
|
||||
|
||||
return await CheckIfStuck(status);
|
||||
}
|
||||
|
||||
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfSlow(DownloadStatus download)
|
||||
{
|
||||
if (_queueCleanerConfig.SlowMaxStrikes is 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.State is null || !download.State.Equals("Downloading", StringComparison.InvariantCultureIgnoreCase))
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.DownloadSpeed <= 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.SlowIgnorePrivate && download.Private)
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip slow check | download is private | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.Size > (_queueCleanerConfig.SlowIgnoreAboveSizeByteSize?.Bytes ?? long.MaxValue))
|
||||
{
|
||||
_logger.LogDebug("skip slow check | download is too large | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
ByteSize minSpeed = _queueCleanerConfig.SlowMinSpeedByteSize;
|
||||
ByteSize currentSpeed = new ByteSize(download.DownloadSpeed);
|
||||
SmartTimeSpan maxTime = SmartTimeSpan.FromHours(_queueCleanerConfig.SlowMaxTime);
|
||||
SmartTimeSpan currentTime = SmartTimeSpan.FromSeconds(download.Eta);
|
||||
|
||||
return await CheckIfSlow(
|
||||
download.Hash!,
|
||||
download.Name!,
|
||||
minSpeed,
|
||||
currentSpeed,
|
||||
maxTime,
|
||||
currentTime
|
||||
);
|
||||
}
|
||||
|
||||
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfStuck(DownloadStatus status)
|
||||
{
|
||||
if (_queueCleanerConfig.StalledMaxStrikes is 0)
|
||||
{
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.StalledIgnorePrivate && status.Private)
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip stalled check | download is private | {name}", status.Name);
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (status.State is null || !status.State.Equals("Downloading", StringComparison.InvariantCultureIgnoreCase))
|
||||
{
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (status.Eta > 0)
|
||||
{
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
ResetStrikesOnProgress(status.Hash!, status.TotalDone);
|
||||
|
||||
return await StrikeAndCheckLimit(status.Hash!, status.Name!);
|
||||
ResetStalledStrikesOnProgress(status.Hash!, status.TotalDone);
|
||||
|
||||
return (await _striker.StrikeAndCheckLimit(status.Hash!, status.Name!, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled), DeleteReason.Stalled);
|
||||
}
|
||||
|
||||
private static void ProcessFiles(Dictionary<string, DelugeFileOrDirectory>? contents, Action<string, DelugeFileOrDirectory> processFile)
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
namespace Infrastructure.Verticals.DownloadClient;
|
||||
|
||||
public sealed record StalledResult
|
||||
public sealed record DownloadCheckResult
|
||||
{
|
||||
/// <summary>
|
||||
/// True if the download should be removed; otherwise false.
|
||||
@@ -1,8 +1,9 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Common.CustomDataTypes;
|
||||
using Common.Helpers;
|
||||
using Domain.Enums;
|
||||
using Domain.Models.Cache;
|
||||
@@ -10,6 +11,7 @@ using Infrastructure.Helpers;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.Context;
|
||||
using Infrastructure.Verticals.Files;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
@@ -30,6 +32,7 @@ public abstract class DownloadService : IDownloadService
|
||||
protected readonly MemoryCacheEntryOptions _cacheOptions;
|
||||
protected readonly INotificationPublisher _notifier;
|
||||
protected readonly IDryRunInterceptor _dryRunInterceptor;
|
||||
protected readonly IHardLinkFileService _hardLinkFileService;
|
||||
|
||||
protected DownloadService(
|
||||
ILogger<DownloadService> logger,
|
||||
@@ -40,7 +43,8 @@ public abstract class DownloadService : IDownloadService
|
||||
IFilenameEvaluator filenameEvaluator,
|
||||
IStriker striker,
|
||||
INotificationPublisher notifier,
|
||||
IDryRunInterceptor dryRunInterceptor
|
||||
IDryRunInterceptor dryRunInterceptor,
|
||||
IHardLinkFileService hardLinkFileService
|
||||
)
|
||||
{
|
||||
_logger = logger;
|
||||
@@ -52,6 +56,7 @@ public abstract class DownloadService : IDownloadService
|
||||
_striker = striker;
|
||||
_notifier = notifier;
|
||||
_dryRunInterceptor = dryRunInterceptor;
|
||||
_hardLinkFileService = hardLinkFileService;
|
||||
_cacheOptions = new MemoryCacheEntryOptions()
|
||||
.SetSlidingExpiration(StaticConfiguration.TriggerValue + Constants.CacheLimitBuffer);
|
||||
}
|
||||
@@ -60,7 +65,7 @@ public abstract class DownloadService : IDownloadService
|
||||
|
||||
public abstract Task LoginAsync();
|
||||
|
||||
public abstract Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads);
|
||||
public abstract Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads);
|
||||
|
||||
/// <inheritdoc/>
|
||||
public abstract Task<BlockFilesResult> BlockUnwantedFilesAsync(string hash,
|
||||
@@ -72,41 +77,124 @@ public abstract class DownloadService : IDownloadService
|
||||
public abstract Task DeleteDownload(string hash);
|
||||
|
||||
/// <inheritdoc/>
|
||||
public abstract Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories);
|
||||
public abstract Task<List<object>?> GetSeedingDownloads();
|
||||
|
||||
/// <inheritdoc/>
|
||||
public abstract List<object>? FilterDownloadsToBeCleanedAsync(List<object>? downloads, List<CleanCategory> categories);
|
||||
|
||||
/// <inheritdoc/>
|
||||
public abstract Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes,
|
||||
IReadOnlyList<string> ignoredDownloads);
|
||||
public abstract List<object>? FilterDownloadsToChangeCategoryAsync(List<object>? downloads, List<string> categories);
|
||||
|
||||
protected void ResetStrikesOnProgress(string hash, long downloaded)
|
||||
/// <inheritdoc/>
|
||||
public abstract Task CleanDownloadsAsync(List<object>? downloads, List<CleanCategory> categoriesToClean, HashSet<string> excludedHashes, IReadOnlyList<string> ignoredDownloads);
|
||||
|
||||
/// <inheritdoc/>
|
||||
public abstract Task ChangeCategoryForNoHardLinksAsync(List<object>? downloads, HashSet<string> excludedHashes, IReadOnlyList<string> ignoredDownloads);
|
||||
|
||||
/// <inheritdoc/>
|
||||
public abstract Task CreateCategoryAsync(string name);
|
||||
|
||||
protected void ResetStalledStrikesOnProgress(string hash, long downloaded)
|
||||
{
|
||||
if (!_queueCleanerConfig.StalledResetStrikesOnProgress)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (_cache.TryGetValue(CacheKeys.Item(hash), out CacheItem? cachedItem) && cachedItem is not null && downloaded > cachedItem.Downloaded)
|
||||
|
||||
if (_cache.TryGetValue(CacheKeys.StrikeItem(hash, StrikeType.Stalled), out StalledCacheItem? cachedItem) &&
|
||||
cachedItem is not null && downloaded > cachedItem.Downloaded)
|
||||
{
|
||||
// cache item found
|
||||
_cache.Remove(CacheKeys.Strike(StrikeType.Stalled, hash));
|
||||
_logger.LogDebug("resetting strikes for {hash} due to progress", hash);
|
||||
_logger.LogDebug("resetting stalled strikes for {hash} due to progress", hash);
|
||||
}
|
||||
|
||||
_cache.Set(CacheKeys.Item(hash), new CacheItem { Downloaded = downloaded }, _cacheOptions);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Strikes an item and checks if the limit has been reached.
|
||||
/// </summary>
|
||||
/// <param name="hash">The torrent hash.</param>
|
||||
/// <param name="itemName">The name or title of the item.</param>
|
||||
/// <returns>True if the limit has been reached; otherwise, false.</returns>
|
||||
protected async Task<bool> StrikeAndCheckLimit(string hash, string itemName)
|
||||
{
|
||||
return await _striker.StrikeAndCheckLimit(hash, itemName, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled);
|
||||
_cache.Set(CacheKeys.StrikeItem(hash, StrikeType.Stalled), new StalledCacheItem { Downloaded = downloaded }, _cacheOptions);
|
||||
}
|
||||
|
||||
protected SeedingCheckResult ShouldCleanDownload(double ratio, TimeSpan seedingTime, Category category)
|
||||
protected void ResetSlowSpeedStrikesOnProgress(string downloadName, string hash)
|
||||
{
|
||||
if (!_queueCleanerConfig.SlowResetStrikesOnProgress)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
string key = CacheKeys.Strike(StrikeType.SlowSpeed, hash);
|
||||
|
||||
if (!_cache.TryGetValue(key, out object? value) || value is null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
_cache.Remove(key);
|
||||
_logger.LogDebug("resetting slow speed strikes due to progress | {name}", downloadName);
|
||||
}
|
||||
|
||||
protected void ResetSlowTimeStrikesOnProgress(string downloadName, string hash)
|
||||
{
|
||||
if (!_queueCleanerConfig.SlowResetStrikesOnProgress)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
string key = CacheKeys.Strike(StrikeType.SlowTime, hash);
|
||||
|
||||
if (!_cache.TryGetValue(key, out object? value) || value is null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
_cache.Remove(key);
|
||||
_logger.LogDebug("resetting slow time strikes due to progress | {name}", downloadName);
|
||||
}
|
||||
|
||||
protected async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfSlow(
|
||||
string downloadHash,
|
||||
string downloadName,
|
||||
ByteSize minSpeed,
|
||||
ByteSize currentSpeed,
|
||||
SmartTimeSpan maxTime,
|
||||
SmartTimeSpan currentTime
|
||||
)
|
||||
{
|
||||
if (minSpeed.Bytes > 0 && currentSpeed < minSpeed)
|
||||
{
|
||||
_logger.LogTrace("slow speed | {speed}/s | {name}", currentSpeed.ToString(), downloadName);
|
||||
|
||||
bool shouldRemove = await _striker
|
||||
.StrikeAndCheckLimit(downloadHash, downloadName, _queueCleanerConfig.SlowMaxStrikes, StrikeType.SlowSpeed);
|
||||
|
||||
if (shouldRemove)
|
||||
{
|
||||
return (true, DeleteReason.SlowSpeed);
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
ResetSlowSpeedStrikesOnProgress(downloadName, downloadHash);
|
||||
}
|
||||
|
||||
if (maxTime.Time > TimeSpan.Zero && currentTime > maxTime)
|
||||
{
|
||||
_logger.LogTrace("slow estimated time | {time} | {name}", currentTime.ToString(), downloadName);
|
||||
|
||||
bool shouldRemove = await _striker
|
||||
.StrikeAndCheckLimit(downloadHash, downloadName, _queueCleanerConfig.SlowMaxStrikes, StrikeType.SlowTime);
|
||||
|
||||
if (shouldRemove)
|
||||
{
|
||||
return (true, DeleteReason.SlowTime);
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
ResetSlowTimeStrikesOnProgress(downloadName, downloadHash);
|
||||
}
|
||||
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
protected SeedingCheckResult ShouldCleanDownload(double ratio, TimeSpan seedingTime, CleanCategory category)
|
||||
{
|
||||
// check ratio
|
||||
if (DownloadReachedRatio(ratio, seedingTime, category))
|
||||
@@ -130,8 +218,28 @@ public abstract class DownloadService : IDownloadService
|
||||
|
||||
return new();
|
||||
}
|
||||
|
||||
protected string? GetRootWithFirstDirectory(string path)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(path))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
private bool DownloadReachedRatio(double ratio, TimeSpan seedingTime, Category category)
|
||||
string? root = Path.GetPathRoot(path);
|
||||
|
||||
if (root is null)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
string relativePath = path[root.Length..].TrimStart(Path.DirectorySeparatorChar);
|
||||
string[] parts = relativePath.Split(Path.DirectorySeparatorChar, StringSplitOptions.RemoveEmptyEntries);
|
||||
|
||||
return parts.Length > 0 ? Path.Combine(root, parts[0]) : root;
|
||||
}
|
||||
|
||||
private bool DownloadReachedRatio(double ratio, TimeSpan seedingTime, CleanCategory category)
|
||||
{
|
||||
if (category.MaxRatio < 0)
|
||||
{
|
||||
@@ -157,7 +265,7 @@ public abstract class DownloadService : IDownloadService
|
||||
return true;
|
||||
}
|
||||
|
||||
private bool DownloadReachedMaxSeedTime(TimeSpan seedingTime, Category category)
|
||||
private bool DownloadReachedMaxSeedTime(TimeSpan seedingTime, CleanCategory category)
|
||||
{
|
||||
if (category.MaxSeedTime < 0)
|
||||
{
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.RegularExpressions;
|
||||
using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.Files;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
@@ -15,7 +16,21 @@ namespace Infrastructure.Verticals.DownloadClient;
|
||||
|
||||
public class DummyDownloadService : DownloadService
|
||||
{
|
||||
public DummyDownloadService(ILogger<DownloadService> logger, IOptions<QueueCleanerConfig> queueCleanerConfig, IOptions<ContentBlockerConfig> contentBlockerConfig, IOptions<DownloadCleanerConfig> downloadCleanerConfig, IMemoryCache cache, IFilenameEvaluator filenameEvaluator, IStriker striker, INotificationPublisher notifier, IDryRunInterceptor dryRunInterceptor) : base(logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig, cache, filenameEvaluator, striker, notifier, dryRunInterceptor)
|
||||
public DummyDownloadService(
|
||||
ILogger<DownloadService> logger,
|
||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
||||
IOptions<DownloadCleanerConfig> downloadCleanerConfig,
|
||||
IMemoryCache cache,
|
||||
IFilenameEvaluator filenameEvaluator,
|
||||
IStriker striker,
|
||||
INotificationPublisher notifier,
|
||||
IDryRunInterceptor dryRunInterceptor,
|
||||
IHardLinkFileService hardLinkFileService
|
||||
) : base(
|
||||
logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig,
|
||||
cache, filenameEvaluator, striker, notifier, dryRunInterceptor, hardLinkFileService
|
||||
)
|
||||
{
|
||||
}
|
||||
|
||||
@@ -28,7 +43,7 @@ public class DummyDownloadService : DownloadService
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
|
||||
public override Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
public override Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
@@ -39,13 +54,32 @@ public class DummyDownloadService : DownloadService
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
public override Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories)
|
||||
public override Task<List<object>?> GetSeedingDownloads()
|
||||
{
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
public override Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes,
|
||||
IReadOnlyList<string> ignoredDownloads)
|
||||
public override List<object>? FilterDownloadsToBeCleanedAsync(List<object>? downloads, List<CleanCategory> categories)
|
||||
{
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
public override List<object>? FilterDownloadsToChangeCategoryAsync(List<object>? downloads, List<string> categories)
|
||||
{
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
public override Task CleanDownloadsAsync(List<object>? downloads, List<CleanCategory> categoriesToClean, HashSet<string> excludedHashes, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
public override Task ChangeCategoryForNoHardLinksAsync(List<object>? downloads, HashSet<string> excludedHashes, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
public override Task CreateCategoryAsync(string name)
|
||||
{
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
@@ -15,7 +15,7 @@ public interface IDownloadService : IDisposable
|
||||
/// </summary>
|
||||
/// <param name="hash">The download hash.</param>
|
||||
/// <param name="ignoredDownloads">Downloads to ignore from processing.</param>
|
||||
public Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads);
|
||||
public Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads);
|
||||
|
||||
/// <summary>
|
||||
/// Blocks unwanted files from being fully downloaded.
|
||||
@@ -34,24 +34,52 @@ public interface IDownloadService : IDisposable
|
||||
);
|
||||
|
||||
/// <summary>
|
||||
/// Fetches all downloads.
|
||||
/// Fetches all seeding downloads.
|
||||
/// </summary>
|
||||
/// <returns>A list of downloads that are seeding.</returns>
|
||||
Task<List<object>?> GetSeedingDownloads();
|
||||
|
||||
/// <summary>
|
||||
/// Filters downloads that should be cleaned.
|
||||
/// </summary>
|
||||
/// <param name="downloads">The downloads to filter.</param>
|
||||
/// <param name="categories">The categories by which to filter the downloads.</param>
|
||||
/// <returns>A list of downloads for the provided categories.</returns>
|
||||
Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories);
|
||||
List<object>? FilterDownloadsToBeCleanedAsync(List<object>? downloads, List<CleanCategory> categories);
|
||||
|
||||
/// <summary>
|
||||
/// Filters downloads that should have their category changed.
|
||||
/// </summary>
|
||||
/// <param name="downloads">The downloads to filter.</param>
|
||||
/// <param name="categories">The categories by which to filter the downloads.</param>
|
||||
/// <returns>A list of downloads for the provided categories.</returns>
|
||||
List<object>? FilterDownloadsToChangeCategoryAsync(List<object>? downloads, List<string> categories);
|
||||
|
||||
/// <summary>
|
||||
/// Cleans the downloads.
|
||||
/// </summary>
|
||||
/// <param name="downloads"></param>
|
||||
/// <param name="downloads">The downloads to clean.</param>
|
||||
/// <param name="categoriesToClean">The categories that should be cleaned.</param>
|
||||
/// <param name="excludedHashes">The hashes that should not be cleaned.</param>
|
||||
/// <param name="ignoredDownloads">Downloads to ignore from processing.</param>
|
||||
public abstract Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes,
|
||||
IReadOnlyList<string> ignoredDownloads);
|
||||
/// <param name="ignoredDownloads">The downloads to ignore from processing.</param>
|
||||
Task CleanDownloadsAsync(List<object>? downloads, List<CleanCategory> categoriesToClean, HashSet<string> excludedHashes, IReadOnlyList<string> ignoredDownloads);
|
||||
|
||||
/// <summary>
|
||||
/// Changes the category for downloads that have no hardlinks.
|
||||
/// </summary>
|
||||
/// <param name="downloads">The downloads to change.</param>
|
||||
/// <param name="excludedHashes">The hashes that should not be cleaned.</param>
|
||||
/// <param name="ignoredDownloads">The downloads to ignore from processing.</param>
|
||||
Task ChangeCategoryForNoHardLinksAsync(List<object>? downloads, HashSet<string> excludedHashes, IReadOnlyList<string> ignoredDownloads);
|
||||
|
||||
/// <summary>
|
||||
/// Deletes a download item.
|
||||
/// </summary>
|
||||
public Task DeleteDownload(string hash);
|
||||
|
||||
/// <summary>
|
||||
/// Creates a category.
|
||||
/// </summary>
|
||||
/// <param name="name">The category name.</param>
|
||||
public Task CreateCategoryAsync(string name);
|
||||
}
|
||||
@@ -5,19 +5,20 @@ using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.DownloadClient;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Common.CustomDataTypes;
|
||||
using Common.Helpers;
|
||||
using Domain.Enums;
|
||||
using Infrastructure.Extensions;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.Context;
|
||||
using Infrastructure.Verticals.Files;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using QBittorrent.Client;
|
||||
using Category = Common.Configuration.DownloadCleaner.Category;
|
||||
|
||||
namespace Infrastructure.Verticals.DownloadClient.QBittorrent;
|
||||
|
||||
@@ -37,15 +38,20 @@ public class QBitService : DownloadService, IQBitService
|
||||
IFilenameEvaluator filenameEvaluator,
|
||||
IStriker striker,
|
||||
INotificationPublisher notifier,
|
||||
IDryRunInterceptor dryRunInterceptor
|
||||
IDryRunInterceptor dryRunInterceptor,
|
||||
IHardLinkFileService hardLinkFileService
|
||||
) : base(
|
||||
logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig, cache,
|
||||
filenameEvaluator, striker, notifier, dryRunInterceptor
|
||||
filenameEvaluator, striker, notifier, dryRunInterceptor, hardLinkFileService
|
||||
)
|
||||
{
|
||||
_config = config.Value;
|
||||
_config.Validate();
|
||||
_client = new(httpClientFactory.CreateClient(Constants.HttpClientWithRetryName), _config.Url);
|
||||
UriBuilder uriBuilder = new(_config.Url);
|
||||
uriBuilder.Path = string.IsNullOrEmpty(_config.UrlBase)
|
||||
? uriBuilder.Path
|
||||
: $"{uriBuilder.Path.TrimEnd('/')}/{_config.UrlBase.TrimStart('/')}";
|
||||
_client = new(httpClientFactory.CreateClient(Constants.HttpClientWithRetryName), uriBuilder.Uri);
|
||||
}
|
||||
|
||||
public override async Task LoginAsync()
|
||||
@@ -59,9 +65,9 @@ public class QBitService : DownloadService, IQBitService
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
public override async Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
StalledResult result = new();
|
||||
DownloadCheckResult result = new();
|
||||
TorrentInfo? download = (await _client.GetTorrentListAsync(new TorrentListQuery { Hashes = [hash] }))
|
||||
.FirstOrDefault();
|
||||
|
||||
@@ -92,30 +98,25 @@ public class QBitService : DownloadService, IQBitService
|
||||
bool.TryParse(dictValue?.ToString(), out bool boolValue)
|
||||
&& boolValue;
|
||||
|
||||
// if all files were blocked by qBittorrent
|
||||
if (download is { CompletionOn: not null, Downloaded: null or 0 })
|
||||
{
|
||||
result.ShouldRemove = true;
|
||||
result.DeleteReason = DeleteReason.AllFilesBlocked;
|
||||
return result;
|
||||
}
|
||||
|
||||
IReadOnlyList<TorrentContent>? files = await _client.GetTorrentContentsAsync(hash);
|
||||
|
||||
// if all files are marked as skip
|
||||
if (files?.Count is > 0 && files.All(x => x.Priority is TorrentContentPriority.Skip))
|
||||
{
|
||||
result.ShouldRemove = true;
|
||||
result.DeleteReason = DeleteReason.AllFilesBlocked;
|
||||
|
||||
// if all files were blocked by qBittorrent
|
||||
if (download is { CompletionOn: not null, Downloaded: null or 0 })
|
||||
{
|
||||
result.DeleteReason = DeleteReason.AllFilesSkippedByQBit;
|
||||
return result;
|
||||
}
|
||||
|
||||
// remove if all files are unwanted
|
||||
result.DeleteReason = DeleteReason.AllFilesSkipped;
|
||||
return result;
|
||||
}
|
||||
|
||||
result.ShouldRemove = await IsItemStuckAndShouldRemove(download, result.IsPrivate);
|
||||
|
||||
if (result.ShouldRemove)
|
||||
{
|
||||
result.DeleteReason = DeleteReason.Stalled;
|
||||
}
|
||||
(result.ShouldRemove, result.DeleteReason) = await EvaluateDownloadRemoval(download, result.IsPrivate);
|
||||
|
||||
return result;
|
||||
}
|
||||
@@ -226,20 +227,42 @@ public class QBitService : DownloadService, IQBitService
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories) =>
|
||||
public override async Task<List<object>?> GetSeedingDownloads() =>
|
||||
(await _client.GetTorrentListAsync(new()
|
||||
{
|
||||
Filter = TorrentListFilter.Seeding
|
||||
}))
|
||||
?.Where(x => !string.IsNullOrEmpty(x.Hash))
|
||||
.Where(x => categories.Any(cat => cat.Name.Equals(x.Category, StringComparison.InvariantCultureIgnoreCase)))
|
||||
.Cast<object>()
|
||||
.ToList();
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes,
|
||||
IReadOnlyList<string> ignoredDownloads)
|
||||
public override List<object>? FilterDownloadsToBeCleanedAsync(List<object>? downloads, List<CleanCategory> categories) =>
|
||||
downloads
|
||||
?.Cast<TorrentInfo>()
|
||||
.Where(x => !string.IsNullOrEmpty(x.Hash))
|
||||
.Where(x => categories.Any(cat => cat.Name.Equals(x.Category, StringComparison.InvariantCultureIgnoreCase)))
|
||||
.Cast<object>()
|
||||
.ToList();
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override List<object>? FilterDownloadsToChangeCategoryAsync(List<object>? downloads, List<string> categories) =>
|
||||
downloads
|
||||
?.Cast<TorrentInfo>()
|
||||
.Where(x => !string.IsNullOrEmpty(x.Hash))
|
||||
.Where(x => categories.Any(cat => cat.Equals(x.Category, StringComparison.InvariantCultureIgnoreCase)))
|
||||
.Cast<object>()
|
||||
.ToList();
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task CleanDownloadsAsync(List<object>? downloads, List<CleanCategory> categoriesToClean,
|
||||
HashSet<string> excludedHashes, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
if (downloads?.Count is null or 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
foreach (TorrentInfo download in downloads)
|
||||
{
|
||||
if (string.IsNullOrEmpty(download.Hash))
|
||||
@@ -247,16 +270,22 @@ public class QBitService : DownloadService, IQBitService
|
||||
continue;
|
||||
}
|
||||
|
||||
if (excludedHashes.Any(x => x.Equals(download.Hash, StringComparison.InvariantCultureIgnoreCase)))
|
||||
{
|
||||
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
IReadOnlyList<TorrentTracker> trackers = await GetTrackersAsync(download.Hash);
|
||||
|
||||
if (ignoredDownloads.Count > 0 &&
|
||||
(download.ShouldIgnore(ignoredDownloads) || trackers.Any(x => x.ShouldIgnore(ignoredDownloads)) is true))
|
||||
(download.ShouldIgnore(ignoredDownloads) || trackers.Any(x => x.ShouldIgnore(ignoredDownloads))))
|
||||
{
|
||||
_logger.LogInformation("skip | download is ignored | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
Category? category = categoriesToClean
|
||||
CleanCategory? category = categoriesToClean
|
||||
.FirstOrDefault(x => download.Category.Equals(x.Name, StringComparison.InvariantCultureIgnoreCase));
|
||||
|
||||
if (category is null)
|
||||
@@ -264,12 +293,6 @@ public class QBitService : DownloadService, IQBitService
|
||||
continue;
|
||||
}
|
||||
|
||||
if (excludedHashes.Any(x => x.Equals(download.Hash, StringComparison.InvariantCultureIgnoreCase)))
|
||||
{
|
||||
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!_downloadCleanerConfig.DeletePrivate)
|
||||
{
|
||||
TorrentProperties? torrentProperties = await _client.GetTorrentPropertiesAsync(download.Hash);
|
||||
@@ -315,12 +338,125 @@ public class QBitService : DownloadService, IQBitService
|
||||
}
|
||||
}
|
||||
|
||||
public override async Task CreateCategoryAsync(string name)
|
||||
{
|
||||
IReadOnlyDictionary<string, Category>? existingCategories = await _client.GetCategoriesAsync();
|
||||
|
||||
if (existingCategories.Any(x => x.Value.Name.Equals(name, StringComparison.InvariantCultureIgnoreCase)))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
await _dryRunInterceptor.InterceptAsync(CreateCategory, name);
|
||||
}
|
||||
|
||||
public override async Task ChangeCategoryForNoHardLinksAsync(List<object>? downloads, HashSet<string> excludedHashes, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
if (downloads?.Count is null or 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (!string.IsNullOrEmpty(_downloadCleanerConfig.UnlinkedIgnoredRootDir))
|
||||
{
|
||||
_hardLinkFileService.PopulateFileCounts(_downloadCleanerConfig.UnlinkedIgnoredRootDir);
|
||||
}
|
||||
|
||||
foreach (TorrentInfo download in downloads)
|
||||
{
|
||||
if (string.IsNullOrEmpty(download.Hash))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
if (excludedHashes.Any(x => x.Equals(download.Hash, StringComparison.InvariantCultureIgnoreCase)))
|
||||
{
|
||||
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
IReadOnlyList<TorrentTracker> trackers = await GetTrackersAsync(download.Hash);
|
||||
|
||||
if (ignoredDownloads.Count > 0 &&
|
||||
(download.ShouldIgnore(ignoredDownloads) || trackers.Any(x => x.ShouldIgnore(ignoredDownloads))))
|
||||
{
|
||||
_logger.LogInformation("skip | download is ignored | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
IReadOnlyList<TorrentContent>? files = await _client.GetTorrentContentsAsync(download.Hash);
|
||||
|
||||
if (files is null)
|
||||
{
|
||||
_logger.LogDebug("failed to find files for {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
ContextProvider.Set("downloadName", download.Name);
|
||||
ContextProvider.Set("hash", download.Hash);
|
||||
bool hasHardlinks = false;
|
||||
|
||||
foreach (TorrentContent file in files)
|
||||
{
|
||||
if (!file.Index.HasValue)
|
||||
{
|
||||
_logger.LogDebug("skip | file index is null for {name}", download.Name);
|
||||
hasHardlinks = true;
|
||||
break;
|
||||
}
|
||||
|
||||
string filePath = string.Join(Path.DirectorySeparatorChar, Path.Combine(download.SavePath, file.Name).Split(['\\', '/']));
|
||||
|
||||
if (file.Priority is TorrentContentPriority.Skip)
|
||||
{
|
||||
_logger.LogDebug("skip | file is not downloaded | {file}", filePath);
|
||||
continue;
|
||||
}
|
||||
|
||||
long hardlinkCount = _hardLinkFileService.GetHardLinkCount(filePath, !string.IsNullOrEmpty(_downloadCleanerConfig.UnlinkedIgnoredRootDir));
|
||||
|
||||
if (hardlinkCount < 0)
|
||||
{
|
||||
_logger.LogDebug("skip | could not get file properties | {file}", filePath);
|
||||
hasHardlinks = true;
|
||||
break;
|
||||
}
|
||||
|
||||
if (hardlinkCount > 0)
|
||||
{
|
||||
hasHardlinks = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (hasHardlinks)
|
||||
{
|
||||
_logger.LogDebug("skip | download has hardlinks | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
await _dryRunInterceptor.InterceptAsync(ChangeCategory, download.Hash, _downloadCleanerConfig.UnlinkedTargetCategory);
|
||||
|
||||
_logger.LogInformation("category changed for {name}", download.Name);
|
||||
|
||||
await _notifier.NotifyCategoryChanged(download.Category, _downloadCleanerConfig.UnlinkedTargetCategory);
|
||||
|
||||
download.Category = _downloadCleanerConfig.UnlinkedTargetCategory;
|
||||
}
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
[DryRunSafeguard]
|
||||
public override async Task DeleteDownload(string hash)
|
||||
{
|
||||
await _client.DeleteAsync(hash, deleteDownloadedData: true);
|
||||
}
|
||||
|
||||
[DryRunSafeguard]
|
||||
protected async Task CreateCategory(string name)
|
||||
{
|
||||
await _client.AddCategoryAsync(name);
|
||||
}
|
||||
|
||||
[DryRunSafeguard]
|
||||
protected virtual async Task SkipFile(string hash, int fileIndex)
|
||||
@@ -328,41 +464,115 @@ public class QBitService : DownloadService, IQBitService
|
||||
await _client.SetFilePriorityAsync(hash, fileIndex, TorrentContentPriority.Skip);
|
||||
}
|
||||
|
||||
[DryRunSafeguard]
|
||||
protected virtual async Task ChangeCategory(string hash, string newCategory)
|
||||
{
|
||||
await _client.SetTorrentCategoryAsync([hash], newCategory);
|
||||
}
|
||||
|
||||
public override void Dispose()
|
||||
{
|
||||
_client.Dispose();
|
||||
}
|
||||
|
||||
private async Task<bool> IsItemStuckAndShouldRemove(TorrentInfo torrent, bool isPrivate)
|
||||
private async Task<(bool, DeleteReason)> EvaluateDownloadRemoval(TorrentInfo torrent, bool isPrivate)
|
||||
{
|
||||
if (_queueCleanerConfig.StalledMaxStrikes is 0)
|
||||
(bool ShouldRemove, DeleteReason Reason) result = await CheckIfSlow(torrent, isPrivate);
|
||||
|
||||
if (result.ShouldRemove)
|
||||
{
|
||||
return false;
|
||||
return result;
|
||||
}
|
||||
|
||||
return await CheckIfStuck(torrent, isPrivate);
|
||||
}
|
||||
|
||||
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfSlow(TorrentInfo download, bool isPrivate)
|
||||
{
|
||||
if (_queueCleanerConfig.SlowMaxStrikes is 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.StalledIgnorePrivate && isPrivate)
|
||||
if (download.State is not (TorrentState.Downloading or TorrentState.ForcedDownload))
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.DownloadSpeed <= 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.SlowIgnorePrivate && isPrivate)
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip stalled check | download is private | {name}", torrent.Name);
|
||||
return false;
|
||||
_logger.LogDebug("skip slow check | download is private | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.Size > (_queueCleanerConfig.SlowIgnoreAboveSizeByteSize?.Bytes ?? long.MaxValue))
|
||||
{
|
||||
_logger.LogDebug("skip slow check | download is too large | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
ByteSize minSpeed = _queueCleanerConfig.SlowMinSpeedByteSize;
|
||||
ByteSize currentSpeed = new ByteSize(download.DownloadSpeed);
|
||||
SmartTimeSpan maxTime = SmartTimeSpan.FromHours(_queueCleanerConfig.SlowMaxTime);
|
||||
SmartTimeSpan currentTime = new SmartTimeSpan(download.EstimatedTime ?? TimeSpan.Zero);
|
||||
|
||||
return await CheckIfSlow(
|
||||
download.Hash,
|
||||
download.Name,
|
||||
minSpeed,
|
||||
currentSpeed,
|
||||
maxTime,
|
||||
currentTime
|
||||
);
|
||||
}
|
||||
|
||||
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfStuck(TorrentInfo torrent, bool isPrivate)
|
||||
{
|
||||
if (_queueCleanerConfig.StalledMaxStrikes is 0 && _queueCleanerConfig.DownloadingMetadataMaxStrikes is 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (torrent.State is not TorrentState.StalledDownload and not TorrentState.FetchingMetadata
|
||||
and not TorrentState.ForcedFetchingMetadata)
|
||||
{
|
||||
// ignore other states
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
ResetStrikesOnProgress(torrent.Hash, torrent.Downloaded ?? 0);
|
||||
if (_queueCleanerConfig.StalledMaxStrikes > 0 && torrent.State is TorrentState.StalledDownload)
|
||||
{
|
||||
if (_queueCleanerConfig.StalledIgnorePrivate && isPrivate)
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip stalled check | download is private | {name}", torrent.Name);
|
||||
}
|
||||
else
|
||||
{
|
||||
ResetStalledStrikesOnProgress(torrent.Hash, torrent.Downloaded ?? 0);
|
||||
|
||||
return (await _striker.StrikeAndCheckLimit(torrent.Hash, torrent.Name, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled), DeleteReason.Stalled);
|
||||
}
|
||||
}
|
||||
|
||||
return await StrikeAndCheckLimit(torrent.Hash, torrent.Name);
|
||||
if (_queueCleanerConfig.DownloadingMetadataMaxStrikes > 0 && torrent.State is not TorrentState.StalledDownload)
|
||||
{
|
||||
return (await _striker.StrikeAndCheckLimit(torrent.Hash, torrent.Name, _queueCleanerConfig.DownloadingMetadataMaxStrikes, StrikeType.DownloadingMetadata), DeleteReason.DownloadingMetadata);
|
||||
}
|
||||
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
private async Task<IReadOnlyList<TorrentTracker>> GetTrackersAsync(string hash)
|
||||
{
|
||||
return (await _client.GetTorrentTrackersAsync(hash))
|
||||
.Where(x => !x.Url.ToString().Contains("**"))
|
||||
.Where(x => x.Url.Contains("**"))
|
||||
.ToList();
|
||||
}
|
||||
}
|
||||
@@ -5,12 +5,14 @@ using Common.Configuration.ContentBlocker;
|
||||
using Common.Configuration.DownloadCleaner;
|
||||
using Common.Configuration.DownloadClient;
|
||||
using Common.Configuration.QueueCleaner;
|
||||
using Common.CustomDataTypes;
|
||||
using Common.Helpers;
|
||||
using Domain.Enums;
|
||||
using Infrastructure.Extensions;
|
||||
using Infrastructure.Interceptors;
|
||||
using Infrastructure.Verticals.ContentBlocker;
|
||||
using Infrastructure.Verticals.Context;
|
||||
using Infrastructure.Verticals.Files;
|
||||
using Infrastructure.Verticals.ItemStriker;
|
||||
using Infrastructure.Verticals.Notifications;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
@@ -26,7 +28,6 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
{
|
||||
private readonly TransmissionConfig _config;
|
||||
private readonly Client _client;
|
||||
private TorrentInfo[]? _torrentsCache;
|
||||
|
||||
private static readonly string[] Fields =
|
||||
[
|
||||
@@ -42,7 +43,9 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
TorrentFields.DOWNLOAD_DIR,
|
||||
TorrentFields.SECONDS_SEEDING,
|
||||
TorrentFields.UPLOAD_RATIO,
|
||||
TorrentFields.TRACKERS
|
||||
TorrentFields.TRACKERS,
|
||||
TorrentFields.RATE_DOWNLOAD,
|
||||
TorrentFields.TOTAL_SIZE,
|
||||
];
|
||||
|
||||
public TransmissionService(
|
||||
@@ -56,17 +59,22 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
IFilenameEvaluator filenameEvaluator,
|
||||
IStriker striker,
|
||||
INotificationPublisher notifier,
|
||||
IDryRunInterceptor dryRunInterceptor
|
||||
IDryRunInterceptor dryRunInterceptor,
|
||||
IHardLinkFileService hardLinkFileService
|
||||
) : base(
|
||||
logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig, cache,
|
||||
filenameEvaluator, striker, notifier, dryRunInterceptor
|
||||
filenameEvaluator, striker, notifier, dryRunInterceptor, hardLinkFileService
|
||||
)
|
||||
{
|
||||
_config = config.Value;
|
||||
_config.Validate();
|
||||
UriBuilder uriBuilder = new(_config.Url);
|
||||
uriBuilder.Path = string.IsNullOrEmpty(_config.UrlBase)
|
||||
? $"{uriBuilder.Path.TrimEnd('/')}/rpc"
|
||||
: $"{uriBuilder.Path.TrimEnd('/')}/{_config.UrlBase.TrimStart('/').TrimEnd('/')}/rpc";
|
||||
_client = new(
|
||||
httpClientFactory.CreateClient(Constants.HttpClientWithRetryName),
|
||||
new Uri(_config.Url, "/transmission/rpc").ToString(),
|
||||
uriBuilder.Uri.ToString(),
|
||||
login: _config.Username,
|
||||
password: _config.Password
|
||||
);
|
||||
@@ -78,9 +86,9 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
public override async Task<DownloadCheckResult> ShouldRemoveFromArrQueueAsync(string hash, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
StalledResult result = new();
|
||||
DownloadCheckResult result = new();
|
||||
TorrentInfo? download = await GetTorrentAsync(hash);
|
||||
|
||||
if (download is null)
|
||||
@@ -115,17 +123,15 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
|
||||
if (shouldRemove)
|
||||
{
|
||||
// remove if all files are unwanted
|
||||
result.ShouldRemove = true;
|
||||
result.DeleteReason = DeleteReason.AllFilesBlocked;
|
||||
return result;
|
||||
}
|
||||
|
||||
// remove if all files are unwanted or download is stuck
|
||||
result.ShouldRemove = shouldRemove || await IsItemStuckAndShouldRemove(download);
|
||||
// remove if download is stuck
|
||||
(result.ShouldRemove, result.DeleteReason) = await EvaluateDownloadRemoval(download);
|
||||
|
||||
if (!shouldRemove && result.ShouldRemove)
|
||||
{
|
||||
result.DeleteReason = DeleteReason.Stalled;
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -207,40 +213,59 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
public override async Task<List<object>?> GetSeedingDownloads() =>
|
||||
(await _client.TorrentGetAsync(Fields))
|
||||
?.Torrents
|
||||
?.Where(x => !string.IsNullOrEmpty(x.HashString))
|
||||
.Where(x => x.Status is 5 or 6)
|
||||
.Cast<object>()
|
||||
.ToList();
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories)
|
||||
public override List<object>? FilterDownloadsToBeCleanedAsync(List<object>? downloads, List<CleanCategory> categories)
|
||||
{
|
||||
return (await _client.TorrentGetAsync(Fields))
|
||||
?.Torrents
|
||||
?.Where(x => !string.IsNullOrEmpty(x.HashString))
|
||||
.Where(x => x.Status is 5 or 6)
|
||||
return downloads
|
||||
?
|
||||
.Cast<TorrentInfo>()
|
||||
.Where(x => categories
|
||||
.Any(cat =>
|
||||
{
|
||||
if (x.DownloadDir is null)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
return Path.GetFileName(Path.TrimEndingDirectorySeparator(x.DownloadDir))
|
||||
.Equals(cat.Name, StringComparison.InvariantCultureIgnoreCase);
|
||||
})
|
||||
.Any(cat => cat.Name.Equals(x.GetCategory(), StringComparison.InvariantCultureIgnoreCase))
|
||||
)
|
||||
.Cast<object>()
|
||||
.ToList();
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes,
|
||||
IReadOnlyList<string> ignoredDownloads)
|
||||
public override List<object>? FilterDownloadsToChangeCategoryAsync(List<object>? downloads, List<string> categories)
|
||||
{
|
||||
return downloads
|
||||
?.Cast<TorrentInfo>()
|
||||
.Where(x => !string.IsNullOrEmpty(x.HashString))
|
||||
.Where(x => categories.Any(cat => cat.Equals(x.GetCategory(), StringComparison.InvariantCultureIgnoreCase)))
|
||||
.Cast<object>()
|
||||
.ToList();
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task CleanDownloadsAsync(List<object>? downloads, List<CleanCategory> categoriesToClean,
|
||||
HashSet<string> excludedHashes, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
if (downloads?.Count is null or 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
foreach (TorrentInfo download in downloads)
|
||||
{
|
||||
if (string.IsNullOrEmpty(download.HashString))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
if (excludedHashes.Any(x => x.Equals(download.HashString, StringComparison.InvariantCultureIgnoreCase)))
|
||||
{
|
||||
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ignoredDownloads.Count > 0 && download.ShouldIgnore(ignoredDownloads))
|
||||
{
|
||||
@@ -248,7 +273,7 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
continue;
|
||||
}
|
||||
|
||||
Category? category = categoriesToClean
|
||||
CleanCategory? category = categoriesToClean
|
||||
.FirstOrDefault(x =>
|
||||
{
|
||||
if (download.DownloadDir is null)
|
||||
@@ -265,12 +290,6 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
continue;
|
||||
}
|
||||
|
||||
if (excludedHashes.Any(x => x.Equals(download.HashString, StringComparison.InvariantCultureIgnoreCase)))
|
||||
{
|
||||
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!_downloadCleanerConfig.DeletePrivate && download.IsPrivate is true)
|
||||
{
|
||||
_logger.LogDebug("skip | download is private | {name}", download.Name);
|
||||
@@ -302,6 +321,106 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
}
|
||||
}
|
||||
|
||||
public override async Task CreateCategoryAsync(string name)
|
||||
{
|
||||
await Task.CompletedTask;
|
||||
}
|
||||
|
||||
public override async Task ChangeCategoryForNoHardLinksAsync(List<object>? downloads, HashSet<string> excludedHashes, IReadOnlyList<string> ignoredDownloads)
|
||||
{
|
||||
if (downloads?.Count is null or 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (!string.IsNullOrEmpty(_downloadCleanerConfig.UnlinkedIgnoredRootDir))
|
||||
{
|
||||
_hardLinkFileService.PopulateFileCounts(_downloadCleanerConfig.UnlinkedIgnoredRootDir);
|
||||
}
|
||||
|
||||
foreach (TorrentInfo download in downloads.Cast<TorrentInfo>())
|
||||
{
|
||||
if (string.IsNullOrEmpty(download.HashString) || string.IsNullOrEmpty(download.Name) || download.DownloadDir == null)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
if (excludedHashes.Any(x => x.Equals(download.HashString, StringComparison.InvariantCultureIgnoreCase)))
|
||||
{
|
||||
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ignoredDownloads.Count > 0 && download.ShouldIgnore(ignoredDownloads))
|
||||
{
|
||||
_logger.LogDebug("skip | download is ignored | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
ContextProvider.Set("downloadName", download.Name);
|
||||
ContextProvider.Set("hash", download.HashString);
|
||||
|
||||
bool hasHardlinks = false;
|
||||
|
||||
if (download.Files is null || download.FileStats is null)
|
||||
{
|
||||
_logger.LogDebug("skip | download has no files | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
for (int i = 0; i < download.Files.Length; i++)
|
||||
{
|
||||
TransmissionTorrentFiles file = download.Files[i];
|
||||
TransmissionTorrentFileStats stats = download.FileStats[i];
|
||||
|
||||
if (stats.Wanted is null or false || string.IsNullOrEmpty(file.Name))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
string filePath = string.Join(Path.DirectorySeparatorChar, Path.Combine(download.DownloadDir, file.Name).Split(['\\', '/']));
|
||||
|
||||
long hardlinkCount = _hardLinkFileService.GetHardLinkCount(filePath, !string.IsNullOrEmpty(_downloadCleanerConfig.UnlinkedIgnoredRootDir));
|
||||
|
||||
if (hardlinkCount < 0)
|
||||
{
|
||||
_logger.LogDebug("skip | could not get file properties | {file}", filePath);
|
||||
hasHardlinks = true;
|
||||
break;
|
||||
}
|
||||
|
||||
if (hardlinkCount > 0)
|
||||
{
|
||||
hasHardlinks = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (hasHardlinks)
|
||||
{
|
||||
_logger.LogDebug("skip | download has hardlinks | {name}", download.Name);
|
||||
continue;
|
||||
}
|
||||
|
||||
string currentCategory = download.GetCategory();
|
||||
string newLocation = string.Join(Path.DirectorySeparatorChar, Path.Combine(download.DownloadDir, _downloadCleanerConfig.UnlinkedTargetCategory).Split(['\\', '/']));
|
||||
|
||||
await _dryRunInterceptor.InterceptAsync(ChangeDownloadLocation, download.Id, newLocation);
|
||||
|
||||
_logger.LogInformation("category changed for {name}", download.Name);
|
||||
|
||||
await _notifier.NotifyCategoryChanged(currentCategory, _downloadCleanerConfig.UnlinkedTargetCategory);
|
||||
|
||||
download.DownloadDir = newLocation;
|
||||
}
|
||||
}
|
||||
|
||||
[DryRunSafeguard]
|
||||
protected virtual async Task ChangeDownloadLocation(long downloadId, string newLocation)
|
||||
{
|
||||
await _client.TorrentSetLocationAsync([downloadId], newLocation, true);
|
||||
}
|
||||
|
||||
public override async Task DeleteDownload(string hash)
|
||||
{
|
||||
TorrentInfo? torrent = await GetTorrentAsync(hash);
|
||||
@@ -334,60 +453,96 @@ public class TransmissionService : DownloadService, ITransmissionService
|
||||
});
|
||||
}
|
||||
|
||||
private async Task<bool> IsItemStuckAndShouldRemove(TorrentInfo torrent)
|
||||
private async Task<(bool, DeleteReason)> EvaluateDownloadRemoval(TorrentInfo torrent)
|
||||
{
|
||||
(bool ShouldRemove, DeleteReason Reason) result = await CheckIfSlow(torrent);
|
||||
|
||||
if (result.ShouldRemove)
|
||||
{
|
||||
return result;
|
||||
}
|
||||
|
||||
return await CheckIfStuck(torrent);
|
||||
}
|
||||
|
||||
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfSlow(TorrentInfo download)
|
||||
{
|
||||
if (_queueCleanerConfig.SlowMaxStrikes is 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.Status is not 4)
|
||||
{
|
||||
// not in downloading state
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.RateDownload <= 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.SlowIgnorePrivate && download.IsPrivate is true)
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip slow check | download is private | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (download.TotalSize > (_queueCleanerConfig.SlowIgnoreAboveSizeByteSize?.Bytes ?? long.MaxValue))
|
||||
{
|
||||
_logger.LogDebug("skip slow check | download is too large | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
ByteSize minSpeed = _queueCleanerConfig.SlowMinSpeedByteSize;
|
||||
ByteSize currentSpeed = new ByteSize(download.RateDownload ?? long.MaxValue);
|
||||
SmartTimeSpan maxTime = SmartTimeSpan.FromHours(_queueCleanerConfig.SlowMaxTime);
|
||||
SmartTimeSpan currentTime = SmartTimeSpan.FromSeconds(download.Eta ?? 0);
|
||||
|
||||
return await CheckIfSlow(
|
||||
download.HashString!,
|
||||
download.Name!,
|
||||
minSpeed,
|
||||
currentSpeed,
|
||||
maxTime,
|
||||
currentTime
|
||||
);
|
||||
}
|
||||
|
||||
private async Task<(bool ShouldRemove, DeleteReason Reason)> CheckIfStuck(TorrentInfo download)
|
||||
{
|
||||
if (_queueCleanerConfig.StalledMaxStrikes is 0)
|
||||
{
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.StalledIgnorePrivate && (torrent.IsPrivate ?? false))
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip stalled check | download is private | {name}", torrent.Name);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (torrent.Status is not 4)
|
||||
if (download.Status is not 4)
|
||||
{
|
||||
// not in downloading state
|
||||
return false;
|
||||
}
|
||||
|
||||
if (torrent.Eta > 0)
|
||||
{
|
||||
return false;
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
ResetStrikesOnProgress(torrent.HashString!, torrent.DownloadedEver ?? 0);
|
||||
|
||||
return await StrikeAndCheckLimit(torrent.HashString!, torrent.Name!);
|
||||
if (download.RateDownload > 0 || download.Eta > 0)
|
||||
{
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
if (_queueCleanerConfig.StalledIgnorePrivate && (download.IsPrivate ?? false))
|
||||
{
|
||||
// ignore private trackers
|
||||
_logger.LogDebug("skip stalled check | download is private | {name}", download.Name);
|
||||
return (false, DeleteReason.None);
|
||||
}
|
||||
|
||||
ResetStalledStrikesOnProgress(download.HashString!, download.DownloadedEver ?? 0);
|
||||
|
||||
return (await _striker.StrikeAndCheckLimit(download.HashString!, download.Name!, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled), DeleteReason.Stalled);
|
||||
}
|
||||
|
||||
private async Task<TorrentInfo?> GetTorrentAsync(string hash)
|
||||
{
|
||||
TorrentInfo? torrent = _torrentsCache?
|
||||
.FirstOrDefault(x => x.HashString.Equals(hash, StringComparison.InvariantCultureIgnoreCase));
|
||||
|
||||
if (_torrentsCache is null || torrent is null)
|
||||
{
|
||||
// refresh cache
|
||||
_torrentsCache = (await _client.TorrentGetAsync(Fields))
|
||||
?.Torrents;
|
||||
}
|
||||
|
||||
if (_torrentsCache?.Length is null or 0)
|
||||
{
|
||||
_logger.LogDebug("could not list torrents | {url}", _config.Url);
|
||||
}
|
||||
|
||||
torrent = _torrentsCache?.FirstOrDefault(x => x.HashString.Equals(hash, StringComparison.InvariantCultureIgnoreCase));
|
||||
|
||||
if (torrent is null)
|
||||
{
|
||||
_logger.LogDebug("could not find torrent | {hash} | {url}", hash, _config.Url);
|
||||
}
|
||||
|
||||
return torrent;
|
||||
}
|
||||
private async Task<TorrentInfo?> GetTorrentAsync(string hash) =>
|
||||
(await _client.TorrentGetAsync(Fields, hash))
|
||||
?.Torrents
|
||||
?.FirstOrDefault();
|
||||
}
|
||||
51
code/Infrastructure/Verticals/Files/HardLinkFileService.cs
Normal file
51
code/Infrastructure/Verticals/Files/HardLinkFileService.cs
Normal file
@@ -0,0 +1,51 @@
|
||||
using System.Runtime.InteropServices;
|
||||
using Microsoft.Extensions.Logging;
|
||||
|
||||
namespace Infrastructure.Verticals.Files;
|
||||
|
||||
public class HardLinkFileService : IHardLinkFileService
|
||||
{
|
||||
private readonly ILogger<HardLinkFileService> _logger;
|
||||
private readonly UnixHardLinkFileService _unixHardLinkFileService;
|
||||
private readonly WindowsHardLinkFileService _windowsHardLinkFileService;
|
||||
|
||||
public HardLinkFileService(
|
||||
ILogger<HardLinkFileService> logger,
|
||||
UnixHardLinkFileService unixHardLinkFileService,
|
||||
WindowsHardLinkFileService windowsHardLinkFileService
|
||||
)
|
||||
{
|
||||
_logger = logger;
|
||||
_unixHardLinkFileService = unixHardLinkFileService;
|
||||
_windowsHardLinkFileService = windowsHardLinkFileService;
|
||||
}
|
||||
|
||||
public void PopulateFileCounts(string directoryPath)
|
||||
{
|
||||
_logger.LogTrace("populating file counts from {dir}", directoryPath);
|
||||
|
||||
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
|
||||
{
|
||||
_windowsHardLinkFileService.PopulateFileCounts(directoryPath);
|
||||
return;
|
||||
}
|
||||
|
||||
_unixHardLinkFileService.PopulateFileCounts(directoryPath);
|
||||
}
|
||||
|
||||
public long GetHardLinkCount(string filePath, bool ignoreRootDir)
|
||||
{
|
||||
if (!File.Exists(filePath))
|
||||
{
|
||||
_logger.LogDebug("file {file} does not exist", filePath);
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
|
||||
{
|
||||
return _windowsHardLinkFileService.GetHardLinkCount(filePath, ignoreRootDir);
|
||||
}
|
||||
|
||||
return _unixHardLinkFileService.GetHardLinkCount(filePath, ignoreRootDir);
|
||||
}
|
||||
}
|
||||
19
code/Infrastructure/Verticals/Files/IHardLinkFileService.cs
Normal file
19
code/Infrastructure/Verticals/Files/IHardLinkFileService.cs
Normal file
@@ -0,0 +1,19 @@
|
||||
namespace Infrastructure.Verticals.Files;
|
||||
|
||||
public interface IHardLinkFileService
|
||||
{
|
||||
/// <summary>
|
||||
/// Populates the inode counts for Unix and the file index counts for Windows.
|
||||
/// Needs to be called before <see cref="GetHardLinkCount"/> to populate the inode counts.
|
||||
/// </summary>
|
||||
/// <param name="directoryPath">The root directory where to search for hardlinks.</param>
|
||||
void PopulateFileCounts(string directoryPath);
|
||||
|
||||
/// <summary>
|
||||
/// Get the hardlink count of a file.
|
||||
/// </summary>
|
||||
/// <param name="filePath">File path.</param>
|
||||
/// <param name="ignoreRootDir">Whether to ignore hardlinks found in the same root dir.</param>
|
||||
/// <returns>-1 on error, 0 if there are no hardlinks and 1 otherwise.</returns>
|
||||
long GetHardLinkCount(string filePath, bool ignoreRootDir);
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
using System.Collections.Concurrent;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Mono.Unix.Native;
|
||||
|
||||
namespace Infrastructure.Verticals.Files;
|
||||
|
||||
public class UnixHardLinkFileService : IHardLinkFileService, IDisposable
|
||||
{
|
||||
private readonly ILogger<UnixHardLinkFileService> _logger;
|
||||
private readonly ConcurrentDictionary<ulong, int> _inodeCounts = new();
|
||||
|
||||
public UnixHardLinkFileService(ILogger<UnixHardLinkFileService> logger)
|
||||
{
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public long GetHardLinkCount(string filePath, bool ignoreRootDir)
|
||||
{
|
||||
try
|
||||
{
|
||||
if (Syscall.stat(filePath, out Stat stat) != 0)
|
||||
{
|
||||
_logger.LogDebug("failed to stat file {file}", filePath);
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (!ignoreRootDir)
|
||||
{
|
||||
_logger.LogDebug("stat file | hardlinks: {nlink} | {file}", stat.st_nlink, filePath);
|
||||
return (long)stat.st_nlink == 1 ? 0 : 1;
|
||||
}
|
||||
|
||||
// get the number of hardlinks in the same root directory
|
||||
int linksInIgnoredDir = _inodeCounts.TryGetValue(stat.st_ino, out int count)
|
||||
? count
|
||||
: 1; // default to 1 if not found
|
||||
|
||||
_logger.LogDebug("stat file | hardlinks: {nlink} | ignored: {ignored} | {file}", stat.st_nlink, linksInIgnoredDir, filePath);
|
||||
return (long)stat.st_nlink - linksInIgnoredDir;
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
_logger.LogError(exception, "failed to stat file {file}", filePath);
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public void PopulateFileCounts(string directoryPath)
|
||||
{
|
||||
try
|
||||
{
|
||||
// traverse all files in the ignored path and subdirectories
|
||||
foreach (string file in Directory.EnumerateFiles(directoryPath, "*", SearchOption.AllDirectories))
|
||||
{
|
||||
AddInodeToCount(file);
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "failed to populate inode counts from {dir}", directoryPath);
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
private void AddInodeToCount(string path)
|
||||
{
|
||||
try
|
||||
{
|
||||
if (Syscall.stat(path, out Stat stat) == 0)
|
||||
{
|
||||
_inodeCounts.AddOrUpdate(stat.st_ino, 1, (_, count) => count + 1);
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogWarning(ex, "could not stat {path} during inode counting", path);
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_inodeCounts.Clear();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Runtime.InteropServices;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Win32.SafeHandles;
|
||||
|
||||
namespace Infrastructure.Verticals.Files;
|
||||
|
||||
public class WindowsHardLinkFileService : IHardLinkFileService, IDisposable
|
||||
{
|
||||
private readonly ILogger<WindowsHardLinkFileService> _logger;
|
||||
private readonly ConcurrentDictionary<ulong, int> _fileIndexCounts = new();
|
||||
|
||||
public WindowsHardLinkFileService(ILogger<WindowsHardLinkFileService> logger)
|
||||
{
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public long GetHardLinkCount(string filePath, bool ignoreRootDir)
|
||||
{
|
||||
try
|
||||
{
|
||||
using SafeFileHandle fileStream = File.OpenHandle(filePath);
|
||||
|
||||
if (!GetFileInformationByHandle(fileStream, out var file))
|
||||
{
|
||||
_logger.LogDebug("failed to get file handle {file}", filePath);
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (!ignoreRootDir)
|
||||
{
|
||||
_logger.LogDebug("stat file | hardlinks: {nlink} | {file}", file.NumberOfLinks, filePath);
|
||||
return file.NumberOfLinks == 1 ? 0 : 1;
|
||||
}
|
||||
|
||||
// Get unique file ID (combination of high and low indices)
|
||||
ulong fileIndex = ((ulong)file.FileIndexHigh << 32) | file.FileIndexLow;
|
||||
|
||||
// get the number of hardlinks in the same root directory
|
||||
int linksInIgnoredDir = _fileIndexCounts.TryGetValue(fileIndex, out int count)
|
||||
? count
|
||||
: 1; // default to 1 if not found
|
||||
|
||||
_logger.LogDebug("stat file | hardlinks: {links} | ignored: {ignored} | {file}", file.NumberOfLinks, linksInIgnoredDir, filePath);
|
||||
return file.NumberOfLinks - linksInIgnoredDir;
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
_logger.LogError(exception, "failed to stat file {file}", filePath);
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public void PopulateFileCounts(string directoryPath)
|
||||
{
|
||||
try
|
||||
{
|
||||
// traverse all files in the ignored path and subdirectories
|
||||
foreach (string file in Directory.EnumerateFiles(directoryPath, "*", SearchOption.AllDirectories))
|
||||
{
|
||||
AddFileIndexToCount(file);
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Failed to populate file index counts from {dir}", directoryPath);
|
||||
}
|
||||
}
|
||||
|
||||
private void AddFileIndexToCount(string path)
|
||||
{
|
||||
try
|
||||
{
|
||||
using SafeFileHandle fileStream = File.OpenHandle(path);
|
||||
if (GetFileInformationByHandle(fileStream, out var file))
|
||||
{
|
||||
ulong fileIndex = ((ulong)file.FileIndexHigh << 32) | file.FileIndexLow;
|
||||
_fileIndexCounts.AddOrUpdate(fileIndex, 1, (_, count) => count + 1);
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogWarning(ex, "Couldn't stat {path} during file index counting", path);
|
||||
}
|
||||
}
|
||||
|
||||
[DllImport("kernel32.dll", SetLastError = true)]
|
||||
private static extern bool GetFileInformationByHandle(
|
||||
SafeFileHandle hFile,
|
||||
out BY_HANDLE_FILE_INFORMATION lpFileInformation
|
||||
);
|
||||
|
||||
private struct BY_HANDLE_FILE_INFORMATION
|
||||
{
|
||||
public uint FileAttributes;
|
||||
public System.Runtime.InteropServices.ComTypes.FILETIME CreationTime;
|
||||
public System.Runtime.InteropServices.ComTypes.FILETIME LastAccessTime;
|
||||
public System.Runtime.InteropServices.ComTypes.FILETIME LastWriteTime;
|
||||
public uint VolumeSerialNumber;
|
||||
public uint FileSizeHigh;
|
||||
public uint FileSizeLow;
|
||||
public uint NumberOfLinks;
|
||||
public uint FileIndexHigh;
|
||||
public uint FileIndexLow;
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_fileIndexCounts.Clear();
|
||||
}
|
||||
}
|
||||
@@ -15,14 +15,12 @@ public sealed class Striker : IStriker
|
||||
private readonly IMemoryCache _cache;
|
||||
private readonly MemoryCacheEntryOptions _cacheOptions;
|
||||
private readonly INotificationPublisher _notifier;
|
||||
private readonly IDryRunInterceptor _dryRunInterceptor;
|
||||
|
||||
public Striker(ILogger<Striker> logger, IMemoryCache cache, INotificationPublisher notifier, IDryRunInterceptor dryRunInterceptor)
|
||||
public Striker(ILogger<Striker> logger, IMemoryCache cache, INotificationPublisher notifier)
|
||||
{
|
||||
_logger = logger;
|
||||
_cache = cache;
|
||||
_notifier = notifier;
|
||||
_dryRunInterceptor = dryRunInterceptor;
|
||||
_cacheOptions = new MemoryCacheEntryOptions()
|
||||
.SetSlidingExpiration(StaticConfiguration.TriggerValue + Constants.CacheLimitBuffer);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
using Common.Configuration.Notification;
|
||||
|
||||
namespace Infrastructure.Verticals.Notifications.Apprise;
|
||||
|
||||
public sealed record AppriseConfig : NotificationConfig
|
||||
{
|
||||
public const string SectionName = "Apprise";
|
||||
|
||||
public Uri? Url { get; init; }
|
||||
|
||||
public string? Key { get; init; }
|
||||
|
||||
public override bool IsValid()
|
||||
{
|
||||
if (Url is null)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
if (string.IsNullOrEmpty(Key?.Trim()))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
using System.ComponentModel.DataAnnotations;
|
||||
|
||||
namespace Infrastructure.Verticals.Notifications.Apprise;
|
||||
|
||||
public sealed record ApprisePayload
|
||||
{
|
||||
[Required]
|
||||
public string Title { get; init; }
|
||||
|
||||
[Required]
|
||||
public string Body { get; init; }
|
||||
|
||||
public string Type { get; init; } = NotificationType.Info.ToString().ToLowerInvariant();
|
||||
|
||||
public string Format { get; init; } = FormatType.Text.ToString().ToLowerInvariant();
|
||||
}
|
||||
|
||||
public enum NotificationType
|
||||
{
|
||||
Info,
|
||||
Success,
|
||||
Warning,
|
||||
Failure
|
||||
}
|
||||
|
||||
public enum FormatType
|
||||
{
|
||||
Text,
|
||||
Markdown,
|
||||
Html
|
||||
}
|
||||
@@ -0,0 +1,95 @@
|
||||
using System.Text;
|
||||
using Infrastructure.Verticals.Notifications.Models;
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
namespace Infrastructure.Verticals.Notifications.Apprise;
|
||||
|
||||
public sealed class AppriseProvider : NotificationProvider
|
||||
{
|
||||
private readonly AppriseConfig _config;
|
||||
private readonly IAppriseProxy _proxy;
|
||||
|
||||
public AppriseProvider(IOptions<AppriseConfig> config, IAppriseProxy proxy)
|
||||
: base(config)
|
||||
{
|
||||
_config = config.Value;
|
||||
_proxy = proxy;
|
||||
}
|
||||
|
||||
public override string Name => "Apprise";
|
||||
|
||||
public override async Task OnFailedImportStrike(FailedImportStrikeNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification, NotificationType.Warning), _config);
|
||||
}
|
||||
|
||||
public override async Task OnStalledStrike(StalledStrikeNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification, NotificationType.Warning), _config);
|
||||
}
|
||||
|
||||
public override async Task OnSlowStrike(SlowStrikeNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification, NotificationType.Warning), _config);
|
||||
}
|
||||
|
||||
public override async Task OnQueueItemDeleted(QueueItemDeletedNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification, NotificationType.Warning), _config);
|
||||
}
|
||||
|
||||
public override async Task OnDownloadCleaned(DownloadCleanedNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification, NotificationType.Warning), _config);
|
||||
}
|
||||
|
||||
public override async Task OnCategoryChanged(CategoryChangedNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification, NotificationType.Warning), _config);
|
||||
}
|
||||
|
||||
private static ApprisePayload BuildPayload(ArrNotification notification, NotificationType notificationType)
|
||||
{
|
||||
StringBuilder body = new();
|
||||
body.AppendLine(notification.Description);
|
||||
body.AppendLine();
|
||||
body.AppendLine($"Instance type: {notification.InstanceType.ToString()}");
|
||||
body.AppendLine($"Url: {notification.InstanceUrl}");
|
||||
body.AppendLine($"Download hash: {notification.Hash}");
|
||||
|
||||
foreach (NotificationField field in notification.Fields ?? [])
|
||||
{
|
||||
body.AppendLine($"{field.Title}: {field.Text}");
|
||||
}
|
||||
|
||||
ApprisePayload payload = new()
|
||||
{
|
||||
Title = notification.Title,
|
||||
Body = body.ToString(),
|
||||
Type = notificationType.ToString().ToLowerInvariant(),
|
||||
};
|
||||
|
||||
return payload;
|
||||
}
|
||||
|
||||
private static ApprisePayload BuildPayload(Notification notification, NotificationType notificationType)
|
||||
{
|
||||
StringBuilder body = new();
|
||||
body.AppendLine(notification.Description);
|
||||
body.AppendLine();
|
||||
|
||||
foreach (NotificationField field in notification.Fields ?? [])
|
||||
{
|
||||
body.AppendLine($"{field.Title}: {field.Text}");
|
||||
}
|
||||
|
||||
ApprisePayload payload = new()
|
||||
{
|
||||
Title = notification.Title,
|
||||
Body = body.ToString(),
|
||||
Type = notificationType.ToString().ToLowerInvariant(),
|
||||
};
|
||||
|
||||
return payload;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
using System.Text;
|
||||
using Common.Helpers;
|
||||
using Newtonsoft.Json;
|
||||
using Newtonsoft.Json.Serialization;
|
||||
|
||||
namespace Infrastructure.Verticals.Notifications.Apprise;
|
||||
|
||||
public sealed class AppriseProxy : IAppriseProxy
|
||||
{
|
||||
private readonly HttpClient _httpClient;
|
||||
|
||||
public AppriseProxy(IHttpClientFactory httpClientFactory)
|
||||
{
|
||||
_httpClient = httpClientFactory.CreateClient(Constants.HttpClientWithRetryName);
|
||||
}
|
||||
|
||||
public async Task SendNotification(ApprisePayload payload, AppriseConfig config)
|
||||
{
|
||||
string content = JsonConvert.SerializeObject(payload, new JsonSerializerSettings
|
||||
{
|
||||
ContractResolver = new CamelCasePropertyNamesContractResolver()
|
||||
});
|
||||
|
||||
UriBuilder uriBuilder = new(config.Url);
|
||||
uriBuilder.Path = $"{uriBuilder.Path.TrimEnd('/')}/notify/{config.Key}";
|
||||
|
||||
using HttpRequestMessage request = new(HttpMethod.Post, uriBuilder.Uri);
|
||||
request.Method = HttpMethod.Post;
|
||||
request.Content = new StringContent(content, Encoding.UTF8, "application/json");
|
||||
|
||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||
response.EnsureSuccessStatusCode();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
namespace Infrastructure.Verticals.Notifications.Apprise;
|
||||
|
||||
public interface IAppriseProxy
|
||||
{
|
||||
Task SendNotification(ApprisePayload payload, AppriseConfig config);
|
||||
}
|
||||
@@ -27,12 +27,18 @@ public sealed class NotificationConsumer<T> : IConsumer<T> where T : Notificatio
|
||||
case StalledStrikeNotification stalledMessage:
|
||||
await _notificationService.Notify(stalledMessage);
|
||||
break;
|
||||
case SlowStrikeNotification slowMessage:
|
||||
await _notificationService.Notify(slowMessage);
|
||||
break;
|
||||
case QueueItemDeletedNotification queueItemDeleteMessage:
|
||||
await _notificationService.Notify(queueItemDeleteMessage);
|
||||
break;
|
||||
case DownloadCleanedNotification downloadCleanedNotification:
|
||||
await _notificationService.Notify(downloadCleanedNotification);
|
||||
break;
|
||||
case CategoryChangedNotification categoryChangedNotification:
|
||||
await _notificationService.Notify(categoryChangedNotification);
|
||||
break;
|
||||
default:
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
@@ -6,7 +6,11 @@ public interface INotificationFactory
|
||||
|
||||
List<INotificationProvider> OnStalledStrikeEnabled();
|
||||
|
||||
List<INotificationProvider> OnSlowStrikeEnabled();
|
||||
|
||||
List<INotificationProvider> OnQueueItemDeletedEnabled();
|
||||
|
||||
List<INotificationProvider> OnDownloadCleanedEnabled();
|
||||
|
||||
List<INotificationProvider> OnCategoryChangedEnabled();
|
||||
}
|
||||
@@ -12,8 +12,12 @@ public interface INotificationProvider
|
||||
Task OnFailedImportStrike(FailedImportStrikeNotification notification);
|
||||
|
||||
Task OnStalledStrike(StalledStrikeNotification notification);
|
||||
|
||||
Task OnSlowStrike(SlowStrikeNotification notification);
|
||||
|
||||
Task OnQueueItemDeleted(QueueItemDeletedNotification notification);
|
||||
|
||||
Task OnDownloadCleaned(DownloadCleanedNotification notification);
|
||||
|
||||
Task OnCategoryChanged(CategoryChangedNotification notification);
|
||||
}
|
||||
@@ -9,4 +9,6 @@ public interface INotificationPublisher
|
||||
Task NotifyQueueItemDeleted(bool removeFromClient, DeleteReason reason);
|
||||
|
||||
Task NotifyDownloadCleaned(double ratio, TimeSpan seedingTime, string categoryName, CleanReason reason);
|
||||
|
||||
Task NotifyCategoryChanged(string oldCategory, string newCategory);
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
namespace Infrastructure.Verticals.Notifications.Models;
|
||||
|
||||
public sealed record CategoryChangedNotification : Notification
|
||||
{
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
namespace Infrastructure.Verticals.Notifications.Models;
|
||||
|
||||
public sealed record SlowStrikeNotification : ArrNotification
|
||||
{
|
||||
}
|
||||
@@ -12,6 +12,8 @@ public class NotifiarrProvider : NotificationProvider
|
||||
private const string WarningColor = "f0ad4e";
|
||||
private const string ImportantColor = "bb2124";
|
||||
private const string Logo = "https://github.com/flmorg/cleanuperr/blob/main/Logo/48.png?raw=true";
|
||||
|
||||
public override string Name => "Notifiarr";
|
||||
|
||||
public NotifiarrProvider(IOptions<NotifiarrConfig> config, INotifiarrProxy proxy)
|
||||
: base(config)
|
||||
@@ -20,8 +22,6 @@ public class NotifiarrProvider : NotificationProvider
|
||||
_proxy = proxy;
|
||||
}
|
||||
|
||||
public override string Name => "Notifiarr";
|
||||
|
||||
public override async Task OnFailedImportStrike(FailedImportStrikeNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification, WarningColor), _config);
|
||||
@@ -32,6 +32,11 @@ public class NotifiarrProvider : NotificationProvider
|
||||
await _proxy.SendNotification(BuildPayload(notification, WarningColor), _config);
|
||||
}
|
||||
|
||||
public override async Task OnSlowStrike(SlowStrikeNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification, WarningColor), _config);
|
||||
}
|
||||
|
||||
public override async Task OnQueueItemDeleted(QueueItemDeletedNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification, ImportantColor), _config);
|
||||
@@ -41,6 +46,11 @@ public class NotifiarrProvider : NotificationProvider
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification), _config);
|
||||
}
|
||||
|
||||
public override async Task OnCategoryChanged(CategoryChangedNotification notification)
|
||||
{
|
||||
await _proxy.SendNotification(BuildPayload(notification), _config);
|
||||
}
|
||||
|
||||
private NotifiarrPayload BuildPayload(ArrNotification notification, string color)
|
||||
{
|
||||
@@ -105,4 +115,32 @@ public class NotifiarrProvider : NotificationProvider
|
||||
|
||||
return payload;
|
||||
}
|
||||
|
||||
private NotifiarrPayload BuildPayload(CategoryChangedNotification notification)
|
||||
{
|
||||
NotifiarrPayload payload = new()
|
||||
{
|
||||
Discord = new()
|
||||
{
|
||||
Color = WarningColor,
|
||||
Text = new()
|
||||
{
|
||||
Title = notification.Title,
|
||||
Icon = Logo,
|
||||
Description = notification.Description,
|
||||
Fields = notification.Fields?.Adapt<List<Field>>() ?? []
|
||||
},
|
||||
Ids = new Ids
|
||||
{
|
||||
Channel = _config.ChannelId
|
||||
},
|
||||
Images = new()
|
||||
{
|
||||
Thumbnail = new Uri(Logo)
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
return payload;
|
||||
}
|
||||
}
|
||||
@@ -1,18 +1,21 @@
|
||||
using System.Text;
|
||||
using Common.Helpers;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Newtonsoft.Json;
|
||||
using Newtonsoft.Json.Serialization;
|
||||
|
||||
namespace Infrastructure.Verticals.Notifications.Notifiarr;
|
||||
|
||||
public class NotifiarrProxy : INotifiarrProxy
|
||||
public sealed class NotifiarrProxy : INotifiarrProxy
|
||||
{
|
||||
private readonly ILogger<NotifiarrProxy> _logger;
|
||||
private readonly HttpClient _httpClient;
|
||||
|
||||
private const string Url = "https://notifiarr.com/api/v1/notification/passthrough/";
|
||||
|
||||
public NotifiarrProxy(IHttpClientFactory httpClientFactory)
|
||||
public NotifiarrProxy(ILogger<NotifiarrProxy> logger, IHttpClientFactory httpClientFactory)
|
||||
{
|
||||
_logger = logger;
|
||||
_httpClient = httpClientFactory.CreateClient(Constants.HttpClientWithRetryName);
|
||||
}
|
||||
|
||||
@@ -25,6 +28,8 @@ public class NotifiarrProxy : INotifiarrProxy
|
||||
ContractResolver = new CamelCasePropertyNamesContractResolver()
|
||||
});
|
||||
|
||||
_logger.LogTrace("sending notification to Notifiarr: {content}", content);
|
||||
|
||||
using HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, $"{Url}{config.ApiKey}");
|
||||
request.Method = HttpMethod.Post;
|
||||
request.Content = new StringContent(content, Encoding.UTF8, "application/json");
|
||||
|
||||
@@ -24,6 +24,11 @@ public class NotificationFactory : INotificationFactory
|
||||
ActiveProviders()
|
||||
.Where(n => n.Config.OnStalledStrike)
|
||||
.ToList();
|
||||
|
||||
public List<INotificationProvider> OnSlowStrikeEnabled() =>
|
||||
ActiveProviders()
|
||||
.Where(n => n.Config.OnSlowStrike)
|
||||
.ToList();
|
||||
|
||||
public List<INotificationProvider> OnQueueItemDeletedEnabled() =>
|
||||
ActiveProviders()
|
||||
@@ -34,4 +39,9 @@ public class NotificationFactory : INotificationFactory
|
||||
ActiveProviders()
|
||||
.Where(n => n.Config.OnDownloadCleaned)
|
||||
.ToList();
|
||||
|
||||
public List<INotificationProvider> OnCategoryChangedEnabled() =>
|
||||
ActiveProviders()
|
||||
.Where(n => n.Config.OnCategoryChanged)
|
||||
.ToList();
|
||||
}
|
||||
@@ -18,8 +18,12 @@ public abstract class NotificationProvider : INotificationProvider
|
||||
public abstract Task OnFailedImportStrike(FailedImportStrikeNotification notification);
|
||||
|
||||
public abstract Task OnStalledStrike(StalledStrikeNotification notification);
|
||||
|
||||
public abstract Task OnSlowStrike(SlowStrikeNotification notification);
|
||||
|
||||
public abstract Task OnQueueItemDeleted(QueueItemDeletedNotification notification);
|
||||
|
||||
public abstract Task OnDownloadCleaned(DownloadCleanedNotification notification);
|
||||
|
||||
public abstract Task OnCategoryChanged(CategoryChangedNotification notification);
|
||||
}
|
||||
@@ -48,10 +48,15 @@ public class NotificationPublisher : INotificationPublisher
|
||||
switch (strikeType)
|
||||
{
|
||||
case StrikeType.Stalled:
|
||||
await _dryRunInterceptor.InterceptAsync(Notify<StalledStrikeNotification>, notification.Adapt<StalledStrikeNotification>());
|
||||
case StrikeType.DownloadingMetadata:
|
||||
await NotifyInternal(notification.Adapt<StalledStrikeNotification>());
|
||||
break;
|
||||
case StrikeType.ImportFailed:
|
||||
await _dryRunInterceptor.InterceptAsync(Notify<FailedImportStrikeNotification>, notification.Adapt<FailedImportStrikeNotification>());
|
||||
await NotifyInternal(notification.Adapt<FailedImportStrikeNotification>());
|
||||
break;
|
||||
case StrikeType.SlowSpeed:
|
||||
case StrikeType.SlowTime:
|
||||
await NotifyInternal(notification.Adapt<SlowStrikeNotification>());
|
||||
break;
|
||||
}
|
||||
}
|
||||
@@ -81,7 +86,7 @@ public class NotificationPublisher : INotificationPublisher
|
||||
Fields = [new() { Title = "Removed from download client?", Text = removeFromClient ? "Yes" : "No" }]
|
||||
};
|
||||
|
||||
await _dryRunInterceptor.InterceptAsync(Notify<QueueItemDeletedNotification>, notification);
|
||||
await NotifyInternal(notification);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
@@ -110,13 +115,36 @@ public class NotificationPublisher : INotificationPublisher
|
||||
Level = NotificationLevel.Important
|
||||
};
|
||||
|
||||
await _dryRunInterceptor.InterceptAsync(Notify<DownloadCleanedNotification>, notification);
|
||||
await NotifyInternal(notification);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "failed to notify download cleaned");
|
||||
}
|
||||
}
|
||||
|
||||
public virtual async Task NotifyCategoryChanged(string oldCategory, string newCategory)
|
||||
{
|
||||
CategoryChangedNotification notification = new()
|
||||
{
|
||||
Title = "Category changed",
|
||||
Description = ContextProvider.Get<string>("downloadName"),
|
||||
Fields =
|
||||
[
|
||||
new() { Title = "Hash", Text = ContextProvider.Get<string>("hash").ToLowerInvariant() },
|
||||
new() { Title = "Old category", Text = oldCategory },
|
||||
new() { Title = "New category", Text = newCategory }
|
||||
],
|
||||
Level = NotificationLevel.Important
|
||||
};
|
||||
|
||||
await NotifyInternal(notification);
|
||||
}
|
||||
|
||||
private Task NotifyInternal<T>(T message) where T: notnull
|
||||
{
|
||||
return _dryRunInterceptor.InterceptAsync(Notify<T>, message);
|
||||
}
|
||||
|
||||
[DryRunSafeguard]
|
||||
private Task Notify<T>(T message) where T: notnull
|
||||
|
||||
@@ -44,6 +44,21 @@ public class NotificationService
|
||||
}
|
||||
}
|
||||
|
||||
public async Task Notify(SlowStrikeNotification notification)
|
||||
{
|
||||
foreach (INotificationProvider provider in _notificationFactory.OnSlowStrikeEnabled())
|
||||
{
|
||||
try
|
||||
{
|
||||
await provider.OnSlowStrike(notification);
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
_logger.LogWarning(exception, "failed to send notification | provider {provider}", provider.Name);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public async Task Notify(QueueItemDeletedNotification notification)
|
||||
{
|
||||
foreach (INotificationProvider provider in _notificationFactory.OnQueueItemDeletedEnabled())
|
||||
@@ -73,4 +88,19 @@ public class NotificationService
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public async Task Notify(CategoryChangedNotification notification)
|
||||
{
|
||||
foreach (INotificationProvider provider in _notificationFactory.OnCategoryChangedEnabled())
|
||||
{
|
||||
try
|
||||
{
|
||||
await provider.OnCategoryChanged(notification);
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
_logger.LogWarning(exception, "failed to send notification | provider {provider}", provider.Name);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -77,6 +77,8 @@ public sealed class QueueCleaner : GenericHandler
|
||||
|
||||
QueueRecord record = group.First();
|
||||
|
||||
_logger.LogTrace("processing | {title} | {id}", record.Title, record.DownloadId);
|
||||
|
||||
if (!arrClient.IsRecordValid(record))
|
||||
{
|
||||
continue;
|
||||
@@ -91,7 +93,7 @@ public sealed class QueueCleaner : GenericHandler
|
||||
// push record to context
|
||||
ContextProvider.Set(nameof(QueueRecord), record);
|
||||
|
||||
StalledResult stalledCheckResult = new();
|
||||
DownloadCheckResult downloadCheckResult = new();
|
||||
|
||||
if (record.Protocol is "torrent" && _downloadClientConfig.DownloadClient is not Common.Enums.DownloadClient.Disabled)
|
||||
{
|
||||
@@ -102,14 +104,14 @@ public sealed class QueueCleaner : GenericHandler
|
||||
}
|
||||
|
||||
// stalled download check
|
||||
stalledCheckResult = await _downloadService.ShouldRemoveFromArrQueueAsync(record.DownloadId, ignoredDownloads);
|
||||
downloadCheckResult = await _downloadService.ShouldRemoveFromArrQueueAsync(record.DownloadId, ignoredDownloads);
|
||||
}
|
||||
|
||||
// failed import check
|
||||
bool shouldRemoveFromArr = await arrClient.ShouldRemoveFromQueue(instanceType, record, stalledCheckResult.IsPrivate);
|
||||
DeleteReason deleteReason = stalledCheckResult.ShouldRemove ? stalledCheckResult.DeleteReason : DeleteReason.ImportFailed;
|
||||
bool shouldRemoveFromArr = await arrClient.ShouldRemoveFromQueue(instanceType, record, downloadCheckResult.IsPrivate);
|
||||
DeleteReason deleteReason = downloadCheckResult.ShouldRemove ? downloadCheckResult.DeleteReason : DeleteReason.ImportFailed;
|
||||
|
||||
if (!shouldRemoveFromArr && !stalledCheckResult.ShouldRemove)
|
||||
if (!shouldRemoveFromArr && !downloadCheckResult.ShouldRemove)
|
||||
{
|
||||
_logger.LogInformation("skip | {title}", record.Title);
|
||||
continue;
|
||||
@@ -119,20 +121,26 @@ public sealed class QueueCleaner : GenericHandler
|
||||
|
||||
bool removeFromClient = true;
|
||||
|
||||
if (stalledCheckResult.IsPrivate)
|
||||
if (downloadCheckResult.IsPrivate)
|
||||
{
|
||||
if (stalledCheckResult.ShouldRemove && !_config.StalledDeletePrivate)
|
||||
{
|
||||
removeFromClient = false;
|
||||
}
|
||||
bool isStalledWithoutPruneFlag =
|
||||
downloadCheckResult.DeleteReason is DeleteReason.Stalled &&
|
||||
!_config.StalledDeletePrivate;
|
||||
|
||||
bool isSlowWithoutPruneFlag =
|
||||
downloadCheckResult.DeleteReason is DeleteReason.SlowSpeed or DeleteReason.SlowTime &&
|
||||
!_config.SlowDeletePrivate;
|
||||
|
||||
bool shouldKeepDueToDeleteRules = downloadCheckResult.ShouldRemove && (isStalledWithoutPruneFlag || isSlowWithoutPruneFlag);
|
||||
bool shouldKeepDueToImportRules = shouldRemoveFromArr && !_config.ImportFailedDeletePrivate;
|
||||
|
||||
if (shouldRemoveFromArr && !_config.ImportFailedDeletePrivate)
|
||||
if (shouldKeepDueToDeleteRules || shouldKeepDueToImportRules)
|
||||
{
|
||||
removeFromClient = false;
|
||||
}
|
||||
}
|
||||
|
||||
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient);
|
||||
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient, deleteReason);
|
||||
await _notifier.NotifyQueueItemDeleted(removeFromClient, deleteReason);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -178,7 +178,7 @@ services:
|
||||
- TZ=Europe/Bucharest
|
||||
- DRY_RUN=false
|
||||
|
||||
- LOGGING__LOGLEVEL=Debug
|
||||
- LOGGING__LOGLEVEL=Verbose
|
||||
- LOGGING__FILE__ENABLED=true
|
||||
- LOGGING__FILE__PATH=/var/logs
|
||||
- LOGGING__ENHANCED=true
|
||||
@@ -193,13 +193,25 @@ services:
|
||||
- QUEUECLEANER__ENABLED=true
|
||||
- QUEUECLEANER__IGNORED_DOWNLOADS_PATH=/ignored
|
||||
- QUEUECLEANER__RUNSEQUENTIALLY=true
|
||||
- QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES=5
|
||||
|
||||
- QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES=3
|
||||
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE=true
|
||||
- QUEUECLEANER__IMPORT_FAILED_DELETE_PRIVATE=false
|
||||
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__0=file is a sample
|
||||
- QUEUECLEANER__STALLED_MAX_STRIKES=5
|
||||
|
||||
- QUEUECLEANER__STALLED_MAX_STRIKES=3
|
||||
- QUEUECLEANER__STALLED_RESET_STRIKES_ON_PROGRESS=true
|
||||
- QUEUECLEANER__STALLED_IGNORE_PRIVATE=true
|
||||
- QUEUECLEANER__STALLED_DELETE_PRIVATE=false
|
||||
- QUEUECLEANER__DOWNLOADING_METADATA_MAX_STRIKES=3
|
||||
|
||||
- QUEUECLEANER__SLOW_MAX_STRIKES=5
|
||||
- QUEUECLEANER__SLOW_RESET_STRIKES_ON_PROGRESS=true
|
||||
- QUEUECLEANER__SLOW_IGNORE_PRIVATE=false
|
||||
- QUEUECLEANER__SLOW_DELETE_PRIVATE=false
|
||||
- QUEUECLEANER__SLOW_MIN_SPEED=1MB
|
||||
- QUEUECLEANER__SLOW_MAX_TIME=20
|
||||
- QUEUECLEANER__SLOW_IGNORE_ABOVE_SIZE=1KB
|
||||
|
||||
- CONTENTBLOCKER__ENABLED=true
|
||||
- CONTENTBLOCKER__IGNORED_DOWNLOADS_PATH=/ignored
|
||||
@@ -212,11 +224,15 @@ services:
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__NAME=tv-sonarr
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MAX_RATIO=-1
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MIN_SEED_TIME=0
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MAX_SEED_TIME=0.01
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__NAME=radarr
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MAX_SEED_TIME=99999
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__NAME=nohardlink
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MAX_RATIO=-1
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MIN_SEED_TIME=0
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MAX_SEED_TIME=0.01
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MAX_SEED_TIME=99999
|
||||
- DOWNLOADCLEANER__UNLINKED_TARGET_CATEGORY=cleanuperr-unlinked
|
||||
- DOWNLOADCLEANER__UNLINKED_IGNORED_ROOT_DIR=/downloads
|
||||
- DOWNLOADCLEANER__UNLINKED_CATEGORIES__0=tv-sonarr
|
||||
- DOWNLOADCLEANER__UNLINKED_CATEGORIES__1=radarr
|
||||
|
||||
- DOWNLOAD_CLIENT=qbittorrent
|
||||
- QBITTORRENT__URL=http://qbittorrent:8080
|
||||
@@ -253,13 +269,24 @@ services:
|
||||
|
||||
# - NOTIFIARR__ON_IMPORT_FAILED_STRIKE=true
|
||||
# - NOTIFIARR__ON_STALLED_STRIKE=true
|
||||
# - NOTIFIARR__ON_SLOW_STRIKE=true
|
||||
# - NOTIFIARR__ON_QUEUE_ITEM_DELETED=true
|
||||
# - NOTIFIARR__ON_DOWNLOAD_CLEANED=true
|
||||
# - NOTIFIARR__ON_CATEGORY_CHANGED=true
|
||||
# - NOTIFIARR__API_KEY=notifiarr_secret
|
||||
# - NOTIFIARR__CHANNEL_ID=discord_channel_id
|
||||
|
||||
# - APPRISE__ON_IMPORT_FAILED_STRIKE=true
|
||||
# - APPRISE__ON_STALLED_STRIKE=true
|
||||
# - APPRISE__ON_SLOW_STRIKE=true
|
||||
# - APPRISE__ON_QUEUE_ITEM_DELETED=true
|
||||
# - APPRISE__ON_DOWNLOAD_CLEANED=true
|
||||
# - APPRISE__URL=http://localhost:8000
|
||||
# - APPRISE__KEY=mykey
|
||||
volumes:
|
||||
- ./data/cleanuperr/logs:/var/logs
|
||||
- ./data/cleanuperr/ignored_downloads:/ignored
|
||||
- ./data/qbittorrent/downloads:/downloads
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
- qbittorrent
|
||||
|
||||
20
docs/.gitignore
vendored
Normal file
20
docs/.gitignore
vendored
Normal file
@@ -0,0 +1,20 @@
|
||||
# Dependencies
|
||||
/node_modules
|
||||
|
||||
# Production
|
||||
/build
|
||||
|
||||
# Generated files
|
||||
.docusaurus
|
||||
.cache-loader
|
||||
|
||||
# Misc
|
||||
.DS_Store
|
||||
.env.local
|
||||
.env.development.local
|
||||
.env.test.local
|
||||
.env.production.local
|
||||
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
41
docs/README.md
Normal file
41
docs/README.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Website
|
||||
|
||||
This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.
|
||||
|
||||
### Installation
|
||||
|
||||
```
|
||||
$ yarn
|
||||
```
|
||||
|
||||
### Local Development
|
||||
|
||||
```
|
||||
$ yarn start
|
||||
```
|
||||
|
||||
This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
|
||||
|
||||
### Build
|
||||
|
||||
```
|
||||
$ yarn build
|
||||
```
|
||||
|
||||
This command generates static content into the `build` directory and can be served using any static contents hosting service.
|
||||
|
||||
### Deployment
|
||||
|
||||
Using SSH:
|
||||
|
||||
```
|
||||
$ USE_SSH=true yarn deploy
|
||||
```
|
||||
|
||||
Not using SSH:
|
||||
|
||||
```
|
||||
$ GIT_USER=<Your GitHub username> yarn deploy
|
||||
```
|
||||
|
||||
If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.
|
||||
24
docs/docs/1_cleanuperr.mdx
Normal file
24
docs/docs/1_cleanuperr.mdx
Normal file
@@ -0,0 +1,24 @@
|
||||
---
|
||||
sidebar_position: 1
|
||||
---
|
||||
|
||||
import { Warning } from '@site/src/components/Admonition';
|
||||
|
||||
# Cleanuperr
|
||||
|
||||
Cleanuperr is a tool for automating the cleanup of unwanted or blocked files in Sonarr, Radarr, and supported download clients like qBittorrent. It removes incomplete or blocked downloads, updates queues, and enforces blacklists or whitelists to manage file selection. After removing blocked content, cleanuperr can also trigger a search to replace the deleted shows/movies.
|
||||
|
||||
Cleanuperr was created primarily to address malicious files, such as `*.lnk` or `*.zipx`, that were getting stuck in Sonarr/Radarr and required manual intervention. Some of the reddit posts that made cleanuperr come to life can be found [here](https://www.reddit.com/r/sonarr/comments/1gqnx16/psa_sonarr_downloaded_a_virus/), [here](https://www.reddit.com/r/sonarr/comments/1gqwklr/sonar_downloaded_a_mkv_file_which_looked_like_a/), [here](https://www.reddit.com/r/sonarr/comments/1gpw2wa/downloaded_waiting_to_import/) and [here](https://www.reddit.com/r/sonarr/comments/1gpi344/downloads_not_importing_no_files_found/).
|
||||
|
||||
<Warning>
|
||||
Because this tool is actively developed and still a work in progress, using the `latest` Docker tag may result in breaking changes.
|
||||
|
||||
Join the Discord server if you want to reach out to me quickly (or just stay updated on new releases) so we can squash those pesky bugs together: https://discord.gg/sWggpnmGNY
|
||||
</Warning>
|
||||
|
||||
|
||||
## Naming choice
|
||||
|
||||
I've had people asking why it's `cleanuperr` and not `cleanuparr` and that I should change it. This name was intentional.
|
||||
|
||||
I've seen a few discussions on this type of naming and I've decided that I didn't deserve the `arr` moniker since `cleanuperr` is not a fork of `NZB.Drone` and it does not have any affiliation with the arrs. I still wanted to keep the naming style close enough though, to suggest a correlation between them.
|
||||
13
docs/docs/2_supported-apps.mdx
Normal file
13
docs/docs/2_supported-apps.mdx
Normal file
@@ -0,0 +1,13 @@
|
||||
---
|
||||
sidebar_position: 2
|
||||
---
|
||||
|
||||
# Supported apps
|
||||
|
||||
Only the **latest versions** of the following apps are supported, or earlier versions that have the same API as the latest version:
|
||||
- qBittorrent
|
||||
- Deluge
|
||||
- Transmission
|
||||
- Sonarr
|
||||
- Radarr
|
||||
- Lidarr
|
||||
36
docs/docs/4_how_it_works.mdx
Normal file
36
docs/docs/4_how_it_works.mdx
Normal file
@@ -0,0 +1,36 @@
|
||||
---
|
||||
sidebar_position: 4
|
||||
---
|
||||
|
||||
# How it works
|
||||
|
||||
This is a detailed explanation of how the recurring cleanup jobs work.
|
||||
|
||||
#### 1. **Content blocker** will:
|
||||
- Run every 5 minutes (or configured cron).
|
||||
- Process all items in the *arr queue.
|
||||
- Find the corresponding item from the download client for each queue item.
|
||||
- Mark the files that were found in the queue as **unwanted/skipped** if:
|
||||
- They **are listed in the blacklist**, or
|
||||
- They **are not included in the whitelist**.
|
||||
- If **all files** of a download **are unwanted**:
|
||||
- It will be removed from the *arr's queue and blocked.
|
||||
- It will be deleted from the download client.
|
||||
- A new search will be triggered for the *arr item.
|
||||
#### 2. **Queue cleaner** will:
|
||||
- Run every 5 minutes (or configured cron, or right after `Content Blocker`).
|
||||
- Process all items in the *arr queue.
|
||||
- Check each queue item if it is **stalled (download speed is 0)**, **stuck in metadata downloading**, **failed to be imported** or **slow**.
|
||||
- If it is, the item receives a **strike** and will continue to accumulate strikes every time it meets any of these conditions.
|
||||
- Check each queue item if it meets one of the following condition in the download client:
|
||||
- **Marked as completed, but 0 bytes have been downloaded** (due to files being blocked by qBittorrent or the **content blocker**).
|
||||
- All associated files are marked as **unwanted/skipped/do not download**.
|
||||
- If the item **DOES NOT** match the above criteria, it will be skipped.
|
||||
- If the item **DOES** match the criteria or has received the **maximum number of strikes**:
|
||||
- It will be removed from the *arr's queue and blocked.
|
||||
- It will be deleted from the download client.
|
||||
- A new search will be triggered for the *arr item.
|
||||
#### 3. **Download cleaner** will:
|
||||
- Run every hour (or configured cron).
|
||||
- Automatically clean up downloads that have been seeding for a certain amount of time.
|
||||
- Automatically changes the category of downloads that have no hardlinks.
|
||||
11
docs/docs/configuration/1_general.mdx
Normal file
11
docs/docs/configuration/1_general.mdx
Normal file
@@ -0,0 +1,11 @@
|
||||
---
|
||||
sidebar_position: 1
|
||||
---
|
||||
|
||||
import GeneralSettings from '@site/src/components/configuration/GeneralSettings';
|
||||
|
||||
# General Settings
|
||||
|
||||
These are the general configuration settings that apply to the entire application.
|
||||
|
||||
<GeneralSettings/>
|
||||
8
docs/docs/configuration/_category_.json
Normal file
8
docs/docs/configuration/_category_.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"label": "Configuration",
|
||||
"position": 6,
|
||||
"link": {
|
||||
"type": "generated-index",
|
||||
"description": "This page provides documentation for all the environment variables and settings used in the application."
|
||||
}
|
||||
}
|
||||
18
docs/docs/configuration/arrs/1_sonarr.mdx
Normal file
18
docs/docs/configuration/arrs/1_sonarr.mdx
Normal file
@@ -0,0 +1,18 @@
|
||||
---
|
||||
sidebar_position: 1
|
||||
---
|
||||
|
||||
import SonarrSettings from '@site/src/components/configuration/arrs/SonarrSettings';
|
||||
import { Note } from '@site/src/components/Admonition';
|
||||
|
||||
# Sonarr Settings
|
||||
|
||||
<Note>
|
||||
Multiple instances can be specified for each *arr using this format, where `<NUMBER>` starts from 0:
|
||||
```yaml
|
||||
<ARR>__INSTANCES__<NUMBER>__URL
|
||||
<ARR>__INSTANCES__<NUMBER>__APIKEY
|
||||
```
|
||||
</Note>
|
||||
|
||||
<SonarrSettings/>
|
||||
18
docs/docs/configuration/arrs/2_radarr.mdx
Normal file
18
docs/docs/configuration/arrs/2_radarr.mdx
Normal file
@@ -0,0 +1,18 @@
|
||||
---
|
||||
sidebar_position: 2
|
||||
---
|
||||
|
||||
import RadarrSettings from '@site/src/components/configuration/arrs/RadarrSettings';
|
||||
import { Note } from '@site/src/components/Admonition';
|
||||
|
||||
# Radarr Settings
|
||||
|
||||
<Note>
|
||||
Multiple instances can be specified for each *arr using this format, where `<NUMBER>` starts from 0:
|
||||
```yaml
|
||||
<ARR>__INSTANCES__<NUMBER>__URL
|
||||
<ARR>__INSTANCES__<NUMBER>__APIKEY
|
||||
```
|
||||
</Note>
|
||||
|
||||
<RadarrSettings/>
|
||||
18
docs/docs/configuration/arrs/3_lidarr.mdx
Normal file
18
docs/docs/configuration/arrs/3_lidarr.mdx
Normal file
@@ -0,0 +1,18 @@
|
||||
---
|
||||
sidebar_position: 3
|
||||
---
|
||||
|
||||
import LidarrSettings from '@site/src/components/configuration/arrs/LidarrSettings';
|
||||
import { Note } from '@site/src/components/Admonition';
|
||||
|
||||
# Lidarr Settings
|
||||
|
||||
<Note>
|
||||
Multiple instances can be specified for each *arr using this format, where `<NUMBER>` starts from 0:
|
||||
```yaml
|
||||
<ARR>__INSTANCES__<NUMBER>__URL
|
||||
<ARR>__INSTANCES__<NUMBER>__APIKEY
|
||||
```
|
||||
</Note>
|
||||
|
||||
<LidarrSettings/>
|
||||
8
docs/docs/configuration/arrs/_category_.json
Normal file
8
docs/docs/configuration/arrs/_category_.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"label": "Arrs settings",
|
||||
"position": 6,
|
||||
"link": {
|
||||
"type": "generated-index",
|
||||
"description": "Servarr settings."
|
||||
}
|
||||
}
|
||||
19
docs/docs/configuration/content-blocker/1_general.mdx
Normal file
19
docs/docs/configuration/content-blocker/1_general.mdx
Normal file
@@ -0,0 +1,19 @@
|
||||
---
|
||||
sidebar_position: 1
|
||||
---
|
||||
|
||||
import ContentBlockerGeneralSettings from '@site/src/components/configuration/content-blocker/ContentBlockerGeneralSettings';
|
||||
|
||||
# General Settings
|
||||
|
||||
These settings control the general behavior of the Content Blocker functionality.
|
||||
|
||||
These environment variables are needed to enable the Content Blocker functionality:
|
||||
- [SONARR__BLOCK__TYPE](/docs/configuration/arrs/sonarr?SONARR__BLOCK__TYPE) (if Sonarr is enabled)
|
||||
- [SONARR__BLOCK__PATH](/docs/configuration/arrs/sonarr?SONARR__BLOCK__PATH) (if Sonarr is enabled)
|
||||
- [RADARR__BLOCK__TYPE](/docs/configuration/arrs/radarr?RADARR__BLOCK__TYPE) (if Radarr is enabled)
|
||||
- [RADARR__BLOCK__PATH](/docs/configuration/arrs/radarr?RADARR__BLOCK__PATH) (if Radarr is enabled)
|
||||
- [LIDARR__BLOCK__TYPE](/docs/configuration/arrs/lidarr?LIDARR__BLOCK__TYPE) (if Lidarr is enabled)
|
||||
- [LIDARR__BLOCK__PATH](/docs/configuration/arrs/lidarr?LIDARR__BLOCK__PATH) (if Lidarr is enabled)
|
||||
|
||||
<ContentBlockerGeneralSettings/>
|
||||
8
docs/docs/configuration/content-blocker/_category_.json
Normal file
8
docs/docs/configuration/content-blocker/_category_.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"label": "Content Blocker",
|
||||
"position": 2,
|
||||
"link": {
|
||||
"type": "generated-index",
|
||||
"description": "Settings for the Content Blocker functionality."
|
||||
}
|
||||
}
|
||||
11
docs/docs/configuration/download-cleaner/1_general.mdx
Normal file
11
docs/docs/configuration/download-cleaner/1_general.mdx
Normal file
@@ -0,0 +1,11 @@
|
||||
---
|
||||
sidebar_position: 1
|
||||
---
|
||||
|
||||
import DownloadCleanerGeneralSettings from '@site/src/components/configuration/download-cleaner/DownloadCleanerGeneralSettings';
|
||||
|
||||
# General Settings
|
||||
|
||||
These settings control the basic functionality of the Download Cleaner.
|
||||
|
||||
<DownloadCleanerGeneralSettings/>
|
||||
26
docs/docs/configuration/download-cleaner/2_categories.mdx
Normal file
26
docs/docs/configuration/download-cleaner/2_categories.mdx
Normal file
@@ -0,0 +1,26 @@
|
||||
---
|
||||
sidebar_position: 2
|
||||
---
|
||||
|
||||
import DownloadCleanerCleanupSettings from '@site/src/components/configuration/download-cleaner/DownloadCleanerCleanupSettings';
|
||||
import { Note } from '@site/src/components/Admonition';
|
||||
|
||||
# Cleanup Settings
|
||||
|
||||
These settings control how the Download Cleaner handles different categories of downloads that need to be removed.
|
||||
|
||||
<Note>
|
||||
A download is cleaned when both `MAX_RATIO` and `MIN_SEED_TIME` or just `MAX_SEED_TIME` is reached.
|
||||
</Note>
|
||||
|
||||
<Note>
|
||||
Multiple categories can be specified using this format, where `<NUMBER>` starts from `0`:
|
||||
```yaml
|
||||
DOWNLOADCLEANER__CATEGORIES__<NUMBER>__NAME
|
||||
DOWNLOADCLEANER__CATEGORIES__<NUMBER>__MAX_RATIO
|
||||
DOWNLOADCLEANER__CATEGORIES__<NUMBER>__MIN_SEED_TIME
|
||||
DOWNLOADCLEANER__CATEGORIES__<NUMBER>__MAX_SEED_TIME
|
||||
```
|
||||
</Note>
|
||||
|
||||
<DownloadCleanerCleanupSettings/>
|
||||
23
docs/docs/configuration/download-cleaner/3_hardlinks.mdx
Normal file
23
docs/docs/configuration/download-cleaner/3_hardlinks.mdx
Normal file
@@ -0,0 +1,23 @@
|
||||
---
|
||||
sidebar_position: 3
|
||||
---
|
||||
|
||||
import DownloadCleanerHardlinksSettings from '@site/src/components/configuration/download-cleaner/DownloadCleanerHardlinksSettings';
|
||||
import { Important, Warning } from '@site/src/components/Admonition';
|
||||
|
||||
# Hardlinks Settings
|
||||
|
||||
These settings control how the Download Cleaner handles downloads with no hardlinks remaining (they are not available in the arrs anymore).
|
||||
|
||||
The Download Cleaner will change the category of a download that has no hardlinks and the new category can be cleaned based on the rules configured [here](/docs/configuration/download-cleaner/categories).
|
||||
|
||||
<Important>
|
||||
If you are using Docker, make sure to mount the downloads directory the same way it is mounted for the download client.
|
||||
If your download client's download directory is `/downloads`, it should be the same for Cleanuperr.
|
||||
</Important>
|
||||
|
||||
<Warning>
|
||||
While it is not needed to configure the arrs for this feature, it is recommended you do. If the arrs are not configured, downloads that are waiting to be imported might be affected by it.
|
||||
</Warning>
|
||||
|
||||
<DownloadCleanerHardlinksSettings/>
|
||||
8
docs/docs/configuration/download-cleaner/_category_.json
Normal file
8
docs/docs/configuration/download-cleaner/_category_.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"label": "Download Cleaner",
|
||||
"position": 3,
|
||||
"link": {
|
||||
"type": "generated-index",
|
||||
"description": "Configure the Download Cleaner to automatically clean up downloads that have been seeding for a certain amount of time."
|
||||
}
|
||||
}
|
||||
11
docs/docs/configuration/download-client/1_general.mdx
Normal file
11
docs/docs/configuration/download-client/1_general.mdx
Normal file
@@ -0,0 +1,11 @@
|
||||
---
|
||||
sidebar_position: 1
|
||||
---
|
||||
|
||||
import DownloadClientSettings from '@site/src/components/configuration/download-client/DownloadClientSettings';
|
||||
|
||||
# Download Client Settings
|
||||
|
||||
These settings control how Cleanuperr interacts with your download client.
|
||||
|
||||
<DownloadClientSettings/>
|
||||
8
docs/docs/configuration/download-client/_category_.json
Normal file
8
docs/docs/configuration/download-client/_category_.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"label": "Download Client",
|
||||
"position": 4,
|
||||
"link": {
|
||||
"type": "generated-index",
|
||||
"description": "Configure the download client settings for Cleanuperr."
|
||||
}
|
||||
}
|
||||
157
docs/docs/configuration/examples/1_docker.mdx
Normal file
157
docs/docs/configuration/examples/1_docker.mdx
Normal file
@@ -0,0 +1,157 @@
|
||||
import { Note } from '@site/src/components/Admonition';
|
||||
|
||||
# Docker compose
|
||||
|
||||
<Note>
|
||||
**This example contains all settings and should be modified to fit your needs.**
|
||||
</Note>
|
||||
|
||||
```
|
||||
services:
|
||||
cleanuperr:
|
||||
image: ghcr.io/flmorg/cleanuperr:latest
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ./cleanuperr/logs:/var/logs
|
||||
- ./cleanuperr/ignored.txt:/ignored.txt
|
||||
environment:
|
||||
# general settings
|
||||
- TZ=America/New_York
|
||||
- DRY_RUN=false
|
||||
- HTTP_MAX_RETRIES=0
|
||||
- HTTP_TIMEOUT=100
|
||||
|
||||
# logging
|
||||
- LOGGING__LOGLEVEL=Information
|
||||
- LOGGING__FILE__ENABLED=false
|
||||
- LOGGING__FILE__PATH=/var/logs/
|
||||
- LOGGING__ENHANCED=true
|
||||
|
||||
# job triggers
|
||||
- TRIGGERS__QUEUECLEANER=0 0/5 * * * ?
|
||||
- TRIGGERS__CONTENTBLOCKER=0 0/5 * * * ?
|
||||
- TRIGGERS__DOWNLOADCLEANER=0 0 * * * ?
|
||||
|
||||
# queue cleaner
|
||||
- QUEUECLEANER__ENABLED=true
|
||||
- QUEUECLEANER__IGNORED_DOWNLOADS_PATH=/ignored.txt
|
||||
- QUEUECLEANER__RUNSEQUENTIALLY=true
|
||||
|
||||
# failed imports
|
||||
- QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES=5
|
||||
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE=false
|
||||
- QUEUECLEANER__IMPORT_FAILED_DELETE_PRIVATE=false
|
||||
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__0=title mismatch
|
||||
- QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__1=manual import required
|
||||
|
||||
# stalled downloads
|
||||
- QUEUECLEANER__STALLED_MAX_STRIKES=5
|
||||
- QUEUECLEANER__STALLED_RESET_STRIKES_ON_PROGRESS=false
|
||||
- QUEUECLEANER__STALLED_IGNORE_PRIVATE=false
|
||||
- QUEUECLEANER__STALLED_DELETE_PRIVATE=false
|
||||
|
||||
# slow downloads
|
||||
- QUEUECLEANER__SLOW_MAX_STRIKES=5
|
||||
- QUEUECLEANER__SLOW_RESET_STRIKES_ON_PROGRESS=true
|
||||
- QUEUECLEANER__SLOW_IGNORE_PRIVATE=false
|
||||
- QUEUECLEANER__SLOW_DELETE_PRIVATE=false
|
||||
- QUEUECLEANER__SLOW_MIN_SPEED=1MB
|
||||
- QUEUECLEANER__SLOW_MAX_TIME=20
|
||||
- QUEUECLEANER__SLOW_IGNORE_ABOVE_SIZE=60GB
|
||||
|
||||
# content blocker
|
||||
- CONTENTBLOCKER__ENABLED=true
|
||||
- CONTENTBLOCKER__IGNORED_DOWNLOADS_PATH=/ignored.txt
|
||||
- CONTENTBLOCKER__IGNORE_PRIVATE=false
|
||||
- CONTENTBLOCKER__DELETE_PRIVATE=false
|
||||
|
||||
# download cleaner
|
||||
- DOWNLOADCLEANER__ENABLED=true
|
||||
- DOWNLOADCLEANER__IGNORED_DOWNLOADS_PATH=/ignored.txt
|
||||
- DOWNLOADCLEANER__DELETE_PRIVATE=false
|
||||
|
||||
# categories to seed until max ratio or min seed time has been reached
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__NAME=tv-sonarr
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MAX_RATIO=-1
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MIN_SEED_TIME=0
|
||||
- DOWNLOADCLEANER__CATEGORIES__0__MAX_SEED_TIME=240
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__NAME=radarr
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MAX_RATIO=-1
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MIN_SEED_TIME=0
|
||||
- DOWNLOADCLEANER__CATEGORIES__1__MAX_SEED_TIME=240
|
||||
# remove downloads with no hardlinks
|
||||
- DOWNLOADCLEANER__CATEGORIES__2__NAME=cleanuperr-unlinked
|
||||
- DOWNLOADCLEANER__CATEGORIES__2__MAX_RATIO=-1
|
||||
- DOWNLOADCLEANER__CATEGORIES__2__MIN_SEED_TIME=0
|
||||
- DOWNLOADCLEANER__CATEGORIES__2__MAX_SEED_TIME=0
|
||||
|
||||
# change category for downloads with no hardlinks
|
||||
- DOWNLOADCLEANER__UNLINKED_TARGET_CATEGORY=cleanuperr-unlinked
|
||||
- DOWNLOADCLEANER__UNLINKED_IGNORED_ROOT_DIR=/downloads
|
||||
- DOWNLOADCLEANER__UNLINKED_CATEGORIES__0=tv-sonarr
|
||||
- DOWNLOADCLEANER__UNLINKED_CATEGORIES__1=radarr
|
||||
|
||||
- DOWNLOAD_CLIENT=none
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=disabled
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=qBittorrent
|
||||
# - QBITTORRENT__URL=http://localhost:8080
|
||||
# - QBITTORRENT__URL_BASE=myCustomPath
|
||||
# - QBITTORRENT__USERNAME=user
|
||||
# - QBITTORRENT__PASSWORD=pass
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=deluge
|
||||
# - DELUGE__URL_BASE=myCustomPath
|
||||
# - DELUGE__URL=http://localhost:8112
|
||||
# - DELUGE__PASSWORD=testing
|
||||
# OR
|
||||
# - DOWNLOAD_CLIENT=transmission
|
||||
# - TRANSMISSION__URL=http://localhost:9091
|
||||
# - TRANSMISSION__URL_BASE=myCustomPath
|
||||
# - TRANSMISSION__USERNAME=test
|
||||
# - TRANSMISSION__PASSWORD=testing
|
||||
|
||||
- SONARR__ENABLED=true
|
||||
- SONARR__SEARCHTYPE=Episode
|
||||
- SONARR__BLOCK__TYPE=blacklist
|
||||
- SONARR__BLOCK__PATH=https://example.com/path/to/file.txt
|
||||
- SONARR__INSTANCES__0__URL=http://localhost:8989
|
||||
- SONARR__INSTANCES__0__APIKEY=secret1
|
||||
- SONARR__INSTANCES__1__URL=http://localhost:8990
|
||||
- SONARR__INSTANCES__1__APIKEY=secret2
|
||||
|
||||
- RADARR__ENABLED=true
|
||||
- RADARR__BLOCK__TYPE=blacklist
|
||||
- RADARR__BLOCK__PATH=https://example.com/path/to/file.txt
|
||||
- RADARR__INSTANCES__0__URL=http://localhost:7878
|
||||
- RADARR__INSTANCES__0__APIKEY=secret3
|
||||
- RADARR__INSTANCES__1__URL=http://localhost:7879
|
||||
- RADARR__INSTANCES__1__APIKEY=secret4
|
||||
|
||||
- LIDARR__ENABLED=true
|
||||
- LIDARR__BLOCK__TYPE=blacklist
|
||||
- LIDARR__BLOCK__PATH=https://example.com/path/to/file.txt
|
||||
- LIDARR__INSTANCES__0__URL=http://radarr:8686
|
||||
- LIDARR__INSTANCES__0__APIKEY=secret5
|
||||
- LIDARR__INSTANCES__1__URL=http://radarr:8687
|
||||
- LIDARR__INSTANCES__1__APIKEY=secret6
|
||||
|
||||
- NOTIFIARR__ON_IMPORT_FAILED_STRIKE=true
|
||||
- NOTIFIARR__ON_STALLED_STRIKE=true
|
||||
- NOTIFIARR__ON_SLOW_STRIKE=true
|
||||
- NOTIFIARR__ON_QUEUE_ITEM_DELETED=true
|
||||
- NOTIFIARR__ON_DOWNLOAD_CLEANED=true
|
||||
- NOTIFIARR__ON_CATEGORY_CHANGED=true
|
||||
- NOTIFIARR__API_KEY=notifiarr_secret
|
||||
- NOTIFIARR__CHANNEL_ID=discord_channel_id
|
||||
|
||||
- APPRISE__ON_IMPORT_FAILED_STRIKE=true
|
||||
- APPRISE__ON_STALLED_STRIKE=true
|
||||
- APPRISE__ON_SLOW_STRIKE=true
|
||||
- APPRISE__ON_QUEUE_ITEM_DELETED=true
|
||||
- APPRISE__ON_DOWNLOAD_CLEANED=true
|
||||
- NOTIFIARR__ON_CATEGORY_CHANGED=true
|
||||
- APPRISE__URL=http://apprise:8000
|
||||
- APPRISE__KEY=myConfigKey
|
||||
```
|
||||
180
docs/docs/configuration/examples/2_config-file.mdx
Normal file
180
docs/docs/configuration/examples/2_config-file.mdx
Normal file
@@ -0,0 +1,180 @@
|
||||
import { Note } from '@site/src/components/Admonition';
|
||||
|
||||
# Configuration file example (when not using Docker)
|
||||
|
||||
<Note>
|
||||
**This example contains all settings and should be modified to fit your needs.**
|
||||
</Note>
|
||||
|
||||
```
|
||||
{
|
||||
"TZ": "America/New_York",
|
||||
"DRY_RUN": true,
|
||||
"HTTP_MAX_RETRIES": 0,
|
||||
"HTTP_TIMEOUT": 10,
|
||||
"Logging": {
|
||||
"LogLevel": "Information",
|
||||
"Enhanced": true,
|
||||
"File": {
|
||||
"Enabled": false,
|
||||
"Path": "/var/logs"
|
||||
}
|
||||
},
|
||||
"Triggers": {
|
||||
"QueueCleaner": "0 0/5 * * * ?",
|
||||
"ContentBlocker": "0 0/5 * * * ?",
|
||||
"DownloadCleaner": "0 0 * * * ?"
|
||||
},
|
||||
"QueueCleaner": {
|
||||
"Enabled": true,
|
||||
"RunSequentially": true,
|
||||
"IGNORED_DOWNLOADS_PATH": "/ignored.txt",
|
||||
"IMPORT_FAILED_MAX_STRIKES": 5,
|
||||
"IMPORT_FAILED_IGNORE_PRIVATE": false,
|
||||
"IMPORT_FAILED_DELETE_PRIVATE": false,
|
||||
"IMPORT_FAILED_IGNORE_PATTERNS": [
|
||||
"title mismatch",
|
||||
"manual import required"
|
||||
],
|
||||
"STALLED_MAX_STRIKES": 5,
|
||||
"STALLED_RESET_STRIKES_ON_PROGRESS": true,
|
||||
"STALLED_IGNORE_PRIVATE": false,
|
||||
"STALLED_DELETE_PRIVATE": false,
|
||||
"SLOW_MAX_STRIKES": 5,
|
||||
"SLOW_RESET_STRIKES_ON_PROGRESS": true,
|
||||
"SLOW_IGNORE_PRIVATE": false,
|
||||
"SLOW_DELETE_PRIVATE": false,
|
||||
"SLOW_MIN_SPEED": "1MB",
|
||||
"SLOW_MAX_TIME": 20,
|
||||
"SLOW_IGNORE_ABOVE_SIZE": "60GB"
|
||||
},
|
||||
"ContentBlocker": {
|
||||
"Enabled": true,
|
||||
"IGNORE_PRIVATE": false,
|
||||
"DELETE_PRIVATE": false,
|
||||
"IGNORED_DOWNLOADS_PATH": "/ignored.txt"
|
||||
},
|
||||
"DownloadCleaner": {
|
||||
"Enabled": false,
|
||||
"DELETE_PRIVATE": false,
|
||||
"CATEGORIES": [
|
||||
{
|
||||
"Name": "tv-sonarr",
|
||||
"MAX_RATIO": 1,
|
||||
"MIN_SEED_TIME": 0,
|
||||
"MAX_SEED_TIME": 240
|
||||
},
|
||||
{
|
||||
"Name": "radarr",
|
||||
"MAX_RATIO": 1,
|
||||
"MIN_SEED_TIME": 0,
|
||||
"MAX_SEED_TIME": 240
|
||||
},
|
||||
{
|
||||
"Name": "cleanuperr-unlinked",
|
||||
"MAX_RATIO": 1,
|
||||
"MIN_SEED_TIME": 0,
|
||||
"MAX_SEED_TIME": 240
|
||||
}
|
||||
],
|
||||
"UNLINKED_TARGET_CATEGORY": "cleanuperr-unlinked",
|
||||
"UNLINKED_IGNORED_ROOT_DIR": "/downloads",
|
||||
"UNLINKED_CATEGORIES": [
|
||||
"tv-sonarr",
|
||||
"radarr"
|
||||
],
|
||||
"IGNORED_DOWNLOADS_PATH": "/ignored.txt"
|
||||
},
|
||||
"DOWNLOAD_CLIENT": "none",
|
||||
"qBittorrent": {
|
||||
"Url": "http://localhost:8080",
|
||||
"URL_BASE": "myCustomPath",
|
||||
"Username": "user",
|
||||
"Password": "pass"
|
||||
},
|
||||
"Deluge": {
|
||||
"Url": "http://localhost:8112",
|
||||
"URL_BASE": "myCustomPath",
|
||||
"Password": "pass"
|
||||
},
|
||||
"Transmission": {
|
||||
"Url": "http://localhost:9091",
|
||||
"URL_BASE": "myCustomPath",
|
||||
"Username": "user",
|
||||
"Password": "pass"
|
||||
},
|
||||
"Sonarr": {
|
||||
"Enabled": true,
|
||||
"SearchType": "Episode",
|
||||
"Block": {
|
||||
"Type": "blacklist",
|
||||
"Path": "https://example.com/path/to/file.txt"
|
||||
},
|
||||
"Instances": [
|
||||
{
|
||||
"Url": "http://localhost:8989",
|
||||
"ApiKey": "sonarrSecret1"
|
||||
},
|
||||
{
|
||||
"Url": "http://localhost:8990",
|
||||
"ApiKey": "sonarrSecret2"
|
||||
},
|
||||
]
|
||||
},
|
||||
"Radarr": {
|
||||
"Enabled": true,
|
||||
"Block": {
|
||||
"Type": "blacklist",
|
||||
"Path": "https://example.com/path/to/file.txt"
|
||||
},
|
||||
"Instances": [
|
||||
{
|
||||
"Url": "http://localhost:7878",
|
||||
"ApiKey": "sonarrSecret1"
|
||||
},
|
||||
{
|
||||
"Url": "http://localhost:7879",
|
||||
"ApiKey": "sonarrSecret2"
|
||||
}
|
||||
]
|
||||
},
|
||||
"Lidarr": {
|
||||
"Enabled": true,
|
||||
"Block": {
|
||||
"Type": "blacklist",
|
||||
"Path": "https://example.com/path/to/file.txt"
|
||||
},
|
||||
"Instances": [
|
||||
{
|
||||
"Url": "http://localhost:8686",
|
||||
"ApiKey": "lidarrSecret1"
|
||||
},
|
||||
{
|
||||
"Url": "http://localhost:8687",
|
||||
"ApiKey": "lidarrSecret2"
|
||||
}
|
||||
]
|
||||
},
|
||||
"Notifiarr": {
|
||||
"ON_IMPORT_FAILED_STRIKE": true,
|
||||
"ON_STALLED_STRIKE": true,
|
||||
"ON_SLOW_STRIKE": true,
|
||||
"ON_QUEUE_ITEM_DELETED": true,
|
||||
"ON_DOWNLOAD_CLEANED": true,
|
||||
"ON_CATEGORY_CHANGED": true,
|
||||
"API_KEY": "notifiarr_secret",
|
||||
"CHANNEL_ID": "discord_channel_id"
|
||||
},
|
||||
"Apprise": {
|
||||
"ON_IMPORT_FAILED_STRIKE": true,
|
||||
"ON_STALLED_STRIKE": true,
|
||||
"ON_SLOW_STRIKE": true,
|
||||
"ON_QUEUE_ITEM_DELETED": true,
|
||||
"ON_DOWNLOAD_CLEANED": true,
|
||||
"ON_CATEGORY_CHANGED": true,
|
||||
"URL": "http://localhost:8000",
|
||||
"KEY": "myConfigKey"
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
7
docs/docs/configuration/examples/_category_.json
Normal file
7
docs/docs/configuration/examples/_category_.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"label": "Configuration examples",
|
||||
"position": 8,
|
||||
"link": {
|
||||
"type": "generated-index"
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user