Andrey Prygunkov
f5daf39bb8
fixed #67 : mark as bad may return items to queue
...
… if multiple duplicate-items were marked at once
2015-08-20 21:23:37 +02:00
Andrey Prygunkov
c6656cffbf
#64 : indication of par-repair using dupe sources
...
in history details dialog:
- status “REPAIR: RECIPIENT” or “STATUS:DONOR” with hint details;
- in statistics details (via click on “Total downloaded ->”);
- new field “ExtraParBlocks” returned by API-method “history”: positive
numbers for recipient, negative numbers for donor.
2015-08-13 19:38:28 +02:00
Andrey Prygunkov
6e0a3e0ceb
fixed #69 : total articles wasn't reset when downloading again
2015-08-11 22:34:22 +02:00
Andrey Prygunkov
a334a32b35
ec47da608f: fixed: crash if an nzb contains only par2-files
2015-08-06 17:52:37 +02:00
Andrey Prygunkov
6c3f2a9871
#21 : refactor: new modules Service and DiskService
...
Extracted secondary functions from module PrePostProcessor into new
modules Service and DiskService.
2015-08-03 23:27:02 +02:00
Andrey Prygunkov
c208eec5c3
#26 : removed code "SUCCESS" from DeleteStatus
...
A more general code “GOOD” is used instead.
2015-07-24 19:29:41 +02:00
Andrey Prygunkov
659ed48652
#26 : ignored nzbs are now added to history
...
When an nzb-file isn’t added to queue for some reason, the file is now
also added to history (in addition to messages printed to log):
- for malformed nzb-files which cannot be parsed the status in history
“DELETE: SCAN”;
- for duplicate files with exactly same content status “DELETE: COPY”;
- for duplicate files having history items with status “GOOD” - status
“DELETE: GOOD”;
- for duplicate files having history items with status “SUCCESS” -
status “DELETE: SUCCESS”;
- history items have log-entries with explanation;
- new values for field “DeleteStatus” of RPC-Method “history”: GOOD,
SUCCESS, COPY, SCAN;
- new values for field “Status” of RPC-Method “history”:
"FAILURE/SCAN”, ”DELETED/COPY”, "DELETED/GOOD”, "DELETED/SUCCESS”;
- one exception: for files added from RSS-feeds no history items are
created, the files are ignored as if they were filtered.
2015-07-23 23:47:59 +02:00
Andrey Prygunkov
ec47da608f
refactor: reducing usage of "float"-type where possible
2015-06-05 11:24:05 +00:00
Andrey Prygunkov
5d24697b0c
refactor: reworked declaration of global objects (singletones)
2015-05-22 20:28:05 +00:00
Andrey Prygunkov
2de44bfd99
new action "Mark as success" on history page and in history details dialog; items marked as success are considered successfully downloaded and processed, which is important for duplicate check; new action "HistoryMarkSuccess" in RPC-method "editqueue"; new subcommand "S" of command "-E H" (command line interface); new status "SUCCESS/MARK" can be returned by RPC-method "history"
2015-03-26 22:28:30 +00:00
Andrey Prygunkov
7e6f8f19eb
each nzb now has its own individual log, where messages printed during download or post-processing are saved; the messages can be retrieved later at any time; new button "Log" in the history details dialog; button "Log" in the download details dialog is now active during download too (not only during post-processing); the log contains all nzb-related messages except detail-messages and errors printed during retrieving of articles (they would produce way too many messages and are not that useful anyway); new option "NzbLog" to deactivate per-nzb logging if necessary; per-nzb logs are saved in the queue-directory (option "QueueDir"); new RPC-method "loadlog" returns the previously saved messages for a given nzb-file; new field "MessageCount" is returned by RPC-methods "listgroups" and "history" and indicates if there are any messages saved for the item; parameter "NumberOfLogEntries" of RPC-method "listgroups" and the field "Log" returned by the method are now deprecated, use method "loadlag" instead; field "PostInfoText" returned by RPC-method "listgroups" is now automatically filled with the latest message printed by a pp-script eliminating the need to access deprecated field "Log"
2015-02-26 20:57:38 +00:00
Andrey Prygunkov
11bfb57809
added support for password list file; new option "UnpackPassFile" to set the location of the file; during unpack the passwords are tried from the file until unpack succeeds or all passwords were tried; implemented different strategies for rar4 and rar5-archives taking into account the features of formats; for rar5-archives a wrong password is reported by unrar unambiguously and the program can immediately try other passwords from the password list; for rar4-archives and for 7z-archives it is not possible to differentiate between damaged archive and wrong password; for those archives if the first unpack attempt (without password) fails the program executes par-check (preferably quick par-check if enabled via option "ParQuick) before trying the passwords from the list; another optimization is that the password list is tried only when the first unpack attempt (without password) reports a password error or decryption errors; this saves unnecessary unpack attempts for damaged unencrypted archives
2015-01-22 20:57:39 +00:00
Andrey Prygunkov
9269f69a38
improvement in quick par-verification: if unpack fails (excluding invalid password errors) and quick par-check does not find any errors or quick par-check was already performed the full par-check is performed; this helps in rare situations when files were correctly downloaded (and therefore assumed correct by quick par-check) but incorrectly written into disk due to abnormal program termination (caused by bugs or hardware crashes)
2014-10-09 21:11:42 +00:00
Andrey Prygunkov
95b76bc586
when option "ContinuePartial" is active the current state is saved not more often than once per second instead of after every downloaded article; this significantly reduce the amount of disk writings on high download speeds
2014-09-16 20:54:50 +00:00
Andrey Prygunkov
74a1f6301a
added option "EventInterval" allowing to reduce the number of calls of queue-scripts, which can be useful on slow systems
2014-08-19 19:45:30 +00:00
Andrey Prygunkov
f439f09c2e
improvement in support for detection of bad downloads (fakes, etc.): queue-scripts are now called after every downloaded file included in nzb; new event "FILE_DOWNLOADED" of parameter "NZBNA_EVENT"; event "UNPACK" removed; instead added event "NZB_DOWNLOADED" which is similar to "UNPACK" but is called for every download even not having archive files and even if unpack is disabled; the execution of queue-scripts is serialized - only one script is executed at a time and other scripts wait in script-queue; the script-queue is compressed so that the same script for the same event is not queued more than once; this reduces the number of calls of scripts if files are downloaded faster than queue-scripts can work up them; a call for event "NZB_DOWNLOADED" is always performed even if the previous calls for events "FILE_DOWNLOADED" were skipped; when a script marks nzb as bad the nzb is deleted from queue, no further internal post-processing (par, unrar, etc.) is made for the nzb but all post-processing scripts are executed; if option "DeleteCleanupDisk" is active the already downloaded files are deleted; new status "BAD" for field "DeleteStatus" of nzb-item in RPC-method "history"; queue-scripts can set post-processing parameters by printing special command, just like post-processing-scripts can do that; this simplifies transferring (of small amount) of information between queue-scripts and post-processing-scripts
2014-08-15 22:24:53 +00:00
Andrey Prygunkov
40de60dd8b
added quick par verification for damaged (partially downloaded) files
2014-08-06 00:11:07 +00:00
Andrey Prygunkov
c9981472a8
refactor: disk state now holds info about failed files: their IDs, CRCs of download articles and full intitial article information; these data can be used later to retry download of failed articles and for quick par-verification of damaged files
2014-08-05 23:45:28 +00:00
Andrey Prygunkov
a62966227a
added quick file verification during par-check/repair; if par-repair is required for download the files downloaded without errors are verified quickly by comparing their checksums against the checksums stored in the par2-file; this makes the verification of undamaged files almost instant; damaged files are verified as usual; new option "ParQuick" (active by default); added support for block-by-block scan of files during verification, which improves scan speed of damaged files; the quick par-verification requires a patch for libpar2 (see http://nzbget.net/libpar2 for details)
2014-07-27 21:59:00 +00:00
Andrey Prygunkov
3074ea62dc
added per-nzb time and size statistics: total time, download, verify, repair and unpack times, downloaded size and average speed, shown in history details dialog via click on the row with total size in statistics block; RPC-methods "listgroups" and "history" return new fields: "DownloadedSizeLo", "DownloadedSizeHi", "DownloadedSizeMB", "DownloadTimeSec", "PostTotalTimeSec", "ParTimeSec", "RepairTimeSec", "UnpackTimeSec"
2014-07-19 00:06:28 +00:00
Andrey Prygunkov
ba9efe43be
added article cache: new option "ArticleCache" defines memory limit to use for cache; when cache is active the articles are written into cache first and then all flushed to disk into the destination file; article cache reduces disk IO and may reduce file fragmentation improving post-processing speed (unpack); it works with both writing modes (direct write on and off); when option "DirectWrite" is disabled the cache should be big enough (for best performance) to accommodate all articles of one file (sometimes up to 500 MB) in order to avoid writing articles into temporary files, otherwise temporary files are used for articles which do not fill into cache; when used in combination with DirectWrite there is no such limitation and even a small cache (100 MB or even less) can be used effectively; when the cache becomes full it is flushed automatically (directly into destination file) providing room for new articles; new row in the "statistics and status dialog" in web-interface indicates the amount of memory used for cache; new fields "ArticleCacheLo", "ArticleCacheHi" and "ArticleCacheMB" returned by RPC-method "status"; refactor: parts of unit "ArticleDownloader" responsible for writing into disk were moved into new unit "ArticleWriter"
2014-07-18 22:48:35 +00:00
Andrey Prygunkov
702b635826
improved RPC-API: history items now preserve "NZBID" from queue items; that makes the tracking of items across queue and history easier for third-party apps; field "NZBID" returned by RPC-method "history" is now available for history items of all kinds (NZB, URL, DUP); field "ID" is deprecated and should not be used
2014-07-04 19:07:51 +00:00
Andrey Prygunkov
48446367f4
post-processing scripts now have two new parameters: env. var "NZBPP_STATUS" indicates the status of download including the total status (SUCCESS, FAILURE, etc.) and the detail field (for example in case of failures: PAR, UNPACK, etc.); env. var "NZBPP_TOTALSTATUS" is equal to the total status of parameter "NZBPP_STATUS" and is provided for convenience (to avoid parsing of "NZBPP_STATUS"); the new parameters provide a simple way for pp-scripts to determine download status without a guess work needed in previous versions; parameters "NZBPP_PARSTATUS" and "NZBPP_UNPACKSTATUS" are now considered deprecated (still passed for compatibility); updated script "EMail.py" to use new parameters "NZBPP_TOTALSTATUS" and "NZBPP_STATUS" instead of "NZBPP_PARSTATUS" and "NZBPP_UNPACKSTATUS"
2014-05-28 22:19:39 +00:00
Andrey Prygunkov
9112d2277e
fixed: incorrect number of paused files were shown in curses output mode
2014-05-24 12:25:24 +00:00
Andrey Prygunkov
cb13d00844
added force-priorities; downloads with priorities equal to or greater than 900 are downloaded and post-processed even if the program is in paused state (force mode); in web-interface the combo for choosing priority has new entry "force" (priority value 900); new fields "ForcedSizeLo", "ForcedSizeHi" and "ForcedSizeMB" returned by RPC-method "status";
2014-04-22 20:26:29 +00:00
Andrey Prygunkov
482af25c90
fixed: field "STATUS" was not set correctly for par-checked downloads without unpack (bug introduced in r992)
2014-04-16 17:51:28 +00:00
Andrey Prygunkov
80ceca6e28
new field "STATUS" in RPC-method "history" to allow third-party apps easier determine the status of an item without inspecting status-fields of every processing step; web-interface uses new field "STATUS"
2014-04-14 22:06:23 +00:00
Andrey Prygunkov
a83dbccc6c
changed the way option "ContinuePartial" works: now the information about completed articles is stored in a special file in QueueDir; when option "DirectWrite" is active no separate flag-files per article are created in TempDir; the file contains additional information, which were not stored/available before; fixed: per-server/per-nzb article completion statistics could be inaccurate for nzb-files whose download were interrupted by reload/restart; per-server/per-nzb article completion statistics are now available via RPC-method "listgroups" for active downloads (not only for "history")
2014-04-10 20:06:55 +00:00
Andrey Prygunkov
4c2a8c2892
refactor: moved speed meter code from "QueueCoordinator" into new module "StatMeter"
2014-03-20 21:37:32 +00:00
Andrey Prygunkov
3fd7bbc0a3
refactor: reducing dependencies between modules
2014-03-20 21:14:39 +00:00
Andrey Prygunkov
bf66500aac
reworking queue (continued): merged url queue into main download queue: urls added to queue are now immediately shown in web-interface; urls can be reordered and deleted; when urls are fetched the downloaded nzb-files are put into queue at the positions of their urls; this solves the problem with fetched nzb-files ordered differently than the urls if the fetching of upper (position wise) urls were completed after of the lower urls; removed options "ReloadUrlQueue" and "ReloadPostQueue" since there are no separate url- and post-queues anymore; nzb-files added via urls have new field "URL" which can be accessed via RPC-methods "listgroups" and "history"; new env. var. "NZBNP_URL", "NZBNA_URL" and "NZBPP_URL" passed to NzbProcess, NzbAddedProcess and PostProcess-scripts; removed remote command "--list U", urls are now shown as groups by command "--list G"; RPC-method "urlqueue" is still supported for compatibility but should not be used since the urls are now returned by method "listgroups", the entries have new field "Kind" which can be "NZB" or "URL"
2014-03-18 22:35:58 +00:00
Andrey Prygunkov
30e6131cd7
improved par-check for damaged collections with multiple par-sets and having missing files: only orphaned files (not belonging to any par-set) are scanned when looking for missing files; this greatly decrease the par-check time for big collections
2014-03-05 23:46:29 +00:00
Andrey Prygunkov
8168804f05
reorganized source code directory structure: created directory 'daemon' with several subdirectories and put all source code files there
2014-02-24 22:11:14 +00:00
