mirror of
https://github.com/calibrain/shelfmark.git
synced 2026-02-20 07:46:18 -05:00
5e04b6bfb8
## Summary
This PR completely overhauls the download system to support concurrent
downloads, cancellation, and advanced queue
management. No more waiting for stuck downloads to block the entire
queue!
## New Features
### **Concurrent Downloads**
- **3 simultaneous downloads** by default (configurable)
- ThreadPoolExecutor-based architecture replaces single-threaded
bottleneck
- Downloads no longer block each other
### **Download Cancellation**
- **Cancel button** for active/queued downloads
- Clean cancellation with partial file cleanup
- Thread-safe cancellation flags
### **Queue Priority & Reordering**
- **Priority-based queue** (lower number = higher priority)
- **Editable priority inputs** in UI for queued items
- **Bulk reordering** API support
## UI Enhancements
- **Enhanced status table** with Priority, Progress, and Actions columns
- **Progress bars** for active downloads
- **Cancel buttons** for downloads
- **Queue management controls**: Refresh, Clear Completed, Active
Counter
- **Real-time updates** every 60 seconds
## Implementation
### New Architecture
- `queue.PriorityQueue` replaces simple `set()` for proper ordering
- `concurrent.futures.ThreadPoolExecutor` handles worker threads
- `threading.Event` flags enable clean cancellation
- Thread-safe operations with proper locking
### New API Endpoints
DELETE /api/download/{id}/cancel # Cancel downloads
PUT /api/queue/{id}/priority # Set priorityPOST /api/queue/reorder #
Bulk reorder
GET /api/downloads/active # Active tracking
DELETE /api/queue/clear # Clear completed
### New Environment Variables
```bash
MAX_CONCURRENT_DOWNLOADS=3 # Worker thread count
DOWNLOAD_PROGRESS_UPDATE_INTERVAL=5 # Progress frequency
```
### Fully backward compatible - existing functionality unchanged
### Fixes
- Resolves: Downloads blocking when one gets stuck
- Resolves: No way to cancel problematic downloads
- Resolves: No queue management or prioritization
#183
---------
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
31 lines
1.1 KiB
YAML
31 lines
1.1 KiB
YAML
services:
|
|
calibre-web-automated-book-downloader:
|
|
image: ghcr.io/calibrain/calibre-web-automated-book-downloader:latest
|
|
# Uncomment to build the image from the Dockerfile for local testing changes.
|
|
# Remember to comment out the image line above.
|
|
#build: .
|
|
container_name: calibre-web-automated-book-downloader
|
|
environment:
|
|
FLASK_PORT: 8084
|
|
LOG_LEVEL: info
|
|
BOOK_LANGUAGE: en
|
|
USE_BOOK_TITLE: true
|
|
TZ: America/New_York
|
|
APP_ENV: prod
|
|
UID: 1000
|
|
GID: 100
|
|
# CWA_DB_PATH: /auth/app.db # Comment out to disable authentication
|
|
# Queue management settings
|
|
MAX_CONCURRENT_DOWNLOADS: 3
|
|
DOWNLOAD_PROGRESS_UPDATE_INTERVAL: 5
|
|
ports:
|
|
- 8084:8084
|
|
restart: unless-stopped
|
|
volumes:
|
|
# This is where the books will be downloaded to, usually it would be
|
|
# the same as whatever you gave in "calibre-web-automated"
|
|
- /tmp/data/calibre-web/ingest:/cwa-book-ingest
|
|
# This is the location of CWA's app.db, which contains authentication
|
|
# details. Comment out to disable authentication
|
|
#- /cwa/config/path/app.db:/auth/app.db:ro
|