Andrey Prygunkov
|
7ecb968e23
|
if download was deleted by duplicate check its status in the history is now shown as "DUPE" instead of just "DELETED"
|
2013-09-18 19:49:59 +00:00 |
|
Andrey Prygunkov
|
c4adc8d9be
|
improved detection of same nzb-files acquired from different sources (nzb-sites): 1) the order of individual files as well as the order of articles in nzb-files do not matter; 2) individual files having extensions listed in option "ExtCleanupDisk" are now excluded from content comparison (unless these are par2-files, which are never excluded)
|
2013-09-14 21:20:31 +00:00 |
|
Andrey Prygunkov
|
169719c62d
|
improved duplicate check: when history item expires (as defined by option "KeepHistory") and the duplicate check is active (option "DupeCheck") the item is not completely deleted from history; instead the amount of stored data reduces to minimum required for duplicate check (about 150 bytes vs 2000 bytes for full history item); such old history items are not shown in web-interface by default (to avoid transferring of large amount of history items); new button "Old" in web-interface to show old history items; the items are marked with badge "DUP";
|
2013-09-13 20:13:09 +00:00 |
|
Andrey Prygunkov
|
a509a491af
|
improved duplicate check: 1) added check for nzb-file content to avoid queueing of exactly same files (even with different names); 2) when nzb-file is added with option "Disable duplicate check" the option now works only during adding, it does not exclude the file from later checks (when adding other files)
|
2013-09-12 15:54:14 +00:00 |
|
Andrey Prygunkov
|
cec414126d
|
improved smart duplicates: nzb-files now have field "DupeScore" which can be set from rss filter using command "Options"; items with higher duplicate scores are downloaded even if the history already has successfully downloaded item (with lower score); changed the syntax of rss filter command "Options" to allow use of more options (and easy add new options in the future); new options "DupeScore", "DupeKey" and "NoDupe" to fine tune duplicate handling; updated description of option "DupeCheck"
|
2013-09-11 20:18:52 +00:00 |
|
Andrey Prygunkov
|
d8d9f72985
|
added smart duplicates feature: similar nzb-files are automatically marked as duplicates; queue items can be also manually marked as duplicates using new commands in multi-edit-dialog (action menu); duplicate-mark be manually removed using using new command in multi-edit-dialog and edit-dialog (action menu); duplicates are added in paused state; if download of first duplicates fail, another duplicate is unpaused; if download succeeds all remaining duplicates are removed from queue; an item marked as duplicate has field "DupeKey" which has the same value for all duplicates of the title; this field is shown in web-interface near nzb-name (in a short form to save screen place); new actions "GroupMarkDupe" and "GroupUnMarkDupe" of RPC-command "editqueue" to manually mark/unmark duplicates; new subcommands "DM" and "DU" of command "--edit/-E" to manually mark/unmark duplicates;;; if url-download results in a file without nzb-extension a history item with status "Scan: skipped" is created to inform user about this fact; RPC-commands "listgroups", "postqueue" and "history" now return more info about nzb-item (many new fields); removed option "MergeNzb" because it conflicts with duplicate handling, items can be merged manually if necessary
|
2013-09-09 20:31:17 +00:00 |
|
Andrey Prygunkov
|
1f53d32a62
|
new option "ParRename" to force par-renaming as a first post-processing step (active by default); this saves an unpack attempt and is even more useful if unpack is disabled
|
2013-08-28 15:08:37 +00:00 |
|
Andrey Prygunkov
|
38a273b195
|
added collecting of server usage statistical data for each download: number of successful and failed article downloads per news server; new page in history dialog shows collected statistics; new fields in RPC-method "history": ServerStats (array), TotalArticles, SuccessArticles, FailedArticles; new env. vars passed to pp-scripts: NZBPP_TOTALARTICLES, NZBPP_SUCCESSARTICLES, NZBPP_FAILEDARTICLES and per used news server: NZBPP_SERVERX_SUCCESSARTICLES, NZBPP_SERVERX_FAILEDARTICLES; also new env.vars: DELETED, HEALTHDELETED
|
2013-08-16 21:53:32 +00:00 |
|
Andrey Prygunkov
|
9ba10446e9
|
added download health monitoring: health indicates download status, whether the file is damaged and how much; new option "HealthCheck" to define what to do with bad downloads (pause, delete, none); par-check is now automatically started for downloads having health below 100%, this works independently of unpack (even if unpack is disabled); for downloads having health less than critical health no par-check is performed (it would fail); new fields "Health" and "CriticalHealth" are returned by RPC-Method "listgroups"; new fields "Health", "CriticalHealth", "Deleted" and "HealthDeleted" are returned by RPC-Method "history"; new parameters "NZBPP_HEALTH" and "NZBPP_CRITICALHEALTH" are passed to pp-scripts; manually deleted downloads now have history status "deleted" (instead of "unknown")
|
2013-08-08 21:09:36 +00:00 |
|
Andrey Prygunkov
|
1ad8bd212c
|
refactor: small rework of NZBParameterList-class
|
2013-07-24 21:09:56 +00:00 |
|
Andrey Prygunkov
|
ec48959600
|
changed the way how option "Unpack" works: instead of enabling/disabling the unpacker as a whole, it now defines the initial value of post-processing parameter "Unpack" for nzb-file when it is added to queue; this makes it now possible to disable Unpack globally but still enable it for selected nzb-files; new option "CategoryX.Unpack" to set unpack on a per category basis
|
2013-07-21 20:44:13 +00:00 |
|
Andrey Prygunkov
|
5b3113d96b
|
1) when a nzb-file is added via web-interface or via remote call the file is now put into incoming nzb-directory (option "NzbDir") and then scanned; this has two advantages over the old behavior when the file was parsed directly in memory: the file serves as a backup for troubleshootings and the file is processed by nzbprocess-script (if defined in option "NzbProcess") making the pre-processing much easier; 2) new env-var parameters are passed to NzbProcess-script: NZBNP_NZBNAME, NZBNP_CATEGORY, NZBNP_PRIORITY, NZBNP_TOP, NZBNP_PAUSED; 3) new commands for use in NzbProcess-scripts: "[NZB] TOP=1" to add nzb to the top of queue and "[NZB] PAUSED=1" to add nzb-file in paused state
|
2013-06-17 20:39:46 +00:00 |
|
Andrey Prygunkov
|
9051a4df4d
|
fixed: if the last file of collection was detected as duplicate after the download of the first article the file was deleted from queue (that's OK) but the post-processing was not triggered (that's a bug)
|
2013-06-13 20:51:38 +00:00 |
|
Andrey Prygunkov
|
606021fb8a
|
removed option "AppendNzbDir"; if it was disabled that caused problems in par-checker and unpacker; the option is now assumed always active
|
2013-06-04 20:56:17 +00:00 |
|
Andrey Prygunkov
|
1348ac86f7
|
added setting of post-processing parameters for history items; pp-parameters can now be viewed and changed in history dialog in web-interface; useful before post-processing again; new action <HistorySetParameter> in RPC-method <editqueue>; new action <O> in remote command <--edit/-E> for history items (subcommand <H>)
|
2013-05-16 20:54:13 +00:00 |
|
Andrey Prygunkov
|
340a8130e9
|
addition to r677: added missing headers, causing compilation error in newer gcc versions
|
2013-05-15 16:45:36 +00:00 |
|
Andrey Prygunkov
|
9ab955d026
|
refactor: more consistent using of c-headers
|
2013-05-14 20:20:52 +00:00 |
|
Andrey Prygunkov
|
ed36feeb0a
|
fixed: by deleting of a partially downloaded nzb-file from queue, when the option <DeleteCleanupDisk> was active, the file <_brokenlog.txt> was not deleted preventing the directory from automatic deletion
|
2013-05-13 20:13:28 +00:00 |
|
Andrey Prygunkov
|
26361630c2
|
addition to r660: fixed: downloads were always checked when option <ParCheck> was set to <Auto>
|
2013-05-04 07:45:00 +00:00 |
|
Andrey Prygunkov
|
d9b9786486
|
improved par-check: added support for manual par-check; if option <ParCheck> is set to <Manual> and a damaged download is detected the program downloads all par2-files but doesn't perform par-check; the user must perform par-check/repair manually then (possibly on another, faster computer); old values <yes/no> of option <ParCheck> renamed to <Force> and <Auto> respectively; when set to <Force> all par2-files are always downloaded; removed option <LoadPars> since its functionality is now covered by option <ParCheck>; Result of par-check can now have new value <Manual repair necessary>; field <ParStatus> in RPC-method <history> can have new value <MANUAL>; parameter <NZBPP_PARSTATUS> for pp-script can have new value <4 = manual repair necessary>; extended pp-script <EMail.py> to handle ParStatus=4 (manual)
|
2013-05-02 20:40:36 +00:00 |
|
Andrey Prygunkov
|
6bb760375e
|
added option <ExtCleanupDisk> to automatically delete unwanted files (with specified extensions) after successful par-check or unpack
|
2013-04-24 20:16:04 +00:00 |
|
Andrey Prygunkov
|
025cd043d3
|
history dialog now shows status of every script
|
2013-04-23 18:20:52 +00:00 |
|
Andrey Prygunkov
|
33cb2d108e
|
fixed: if a download didn't have any par-files and the option <ParCheck> was active, the par-check was started anyway and then failed
|
2013-04-18 21:04:48 +00:00 |
|
Andrey Prygunkov
|
f28b35bd28
|
reworked concept of post-processing scripts: multiple scripts can be assigned to each nzb-file; all assigned scripts are executed after the nzb-file is downloaded and internally processed (unpack, repair); option <PostProcess> is obsolete; new option <ScriptDir> sets directory where all pp-scripts must be stored; new option <DefScript> sets the default list of pp-scripts to be assigned to nzb-file when it's added to queue; new option <CategoryX.DefScript> to set the default list of pp-scripts on a category basis; the execution order of pp-scripts can be set using new option <ScriptOrder>; there are no separate configuration files for pp-scripts; configuration options and pp-parameters are defined in the pp-scripts; script configuration options are saved in nzbget configuration file (nzbget.conf); changed parameters list of RPC-methods <loadconfig> and <saveconfig>; new RPC-method <configtemplates> returns configuration descriptions for the program and for all pp-scripts; configuration of all scripts can be done in web-interface; the pp-scripts assigned to a particular nzb-file can be viewed and changed in web-interface on page <pp-parameters> in the edit download dialog; option <PostPauseQueue> renamed to <ScriptPauseQueue> (the old name is still recognized); new option <ConfigTemplate> to define the location of template configuration file (in previous versions it must be always stored in <WebDir>)
|
2013-04-15 20:06:05 +00:00 |
|
Andrey Prygunkov
|
58b0a17986
|
reworked post-processor queue: 1) only one job is created for each nzb-file; no more separate jobs are created for par-collections within one nzb-file; 2) option <AllowReProcess> removed; a post-processing script is called only once per nzb-file, this behavior cannot be altered anymore; 3) with a new feature <Split> (see next commits) individual par-collections can be processed separately in a more effective way than before
|
2013-04-06 20:25:07 +00:00 |
|
Andrey Prygunkov
|
1230d9cdd4
|
added fast renaming of intentionally misnamed (rar-) files; the new renaming algorithm doesn't require full par-scan and restores original filenames in just a few seconds, even on very slow computers (NAS, media players, etc.); the fast renaming is performed automatically when requested by the built-in unpacker (option <Unpack> must be active)
|
2013-03-04 19:55:36 +00:00 |
|
Andrey Prygunkov
|
382faa49cb
|
added new option <InterDir> to put intermediate files during download into a separate directory (instead of storing them directly in destination directory (option <DestDir>); when nzb-file is completely (successfully) downloaded, repaired (if neccessary) and unpacked the files are moved to destination directory (option <DestDir> or <CategoryX.DestDir>); intermediate directory can significantly improve unpack performance if it is located on a separate physical hard drive
|
2013-03-01 20:32:17 +00:00 |
|
Andrey Prygunkov
|
749b4d3083
|
when a history item is post-processed again and the archive files were previously deleted because of option <UnpackCleanupDisk> the post-processing goes directly to script stage; if the archive files were kept, the full post-processing including unpack is performed instead
|
2013-02-28 20:23:50 +00:00 |
|
Andrey Prygunkov
|
940448ffae
|
added built-in unpack: 1) rar and 7-zip formats are supported (via external Unrar and 7-Zip executables); 2) new options <Unpack>, <UnpackPauseQueue>, <UnpackCleanupDisk>, <UnrarCmd>, <SevenZipCmd>; 3) web-interface now shows progress and estimated time during unpack (rar only; for 7-Zip progress is not available due to limitations of 7-Zip) 4) when built-in unpack is enabled, the post-processing script is called after unpack and possibly par-check/repair (if needed); 5) for nzb-files containing multiple collections (par-sets) the post-processing script is called only once, after the last par-set; 6) new parameter <NZBPP_UNPACKSTATUS> passed to post-processing script; 7) if the option <AllowReProcess> is enabled the post-processing-script is called after each par-set (as in previous versions); 8) example post-processing script updated: removed unrar-code, added check for unpack status; 9) new field <UnpackStatus> in result of RPC-method <history>; 10) history-dialog in web-interface shows three status: par-status, unpack-status, script-status; 11) with two built-in special post-processing parameters <*Unpack:> and <*Unpack:Password> the unpack can be disabled for individual nzb-file or the password can be set; 12) built-in special post-processing parameters can be set via web-interface on page <PP-Parameters> (when built-in unpack is enabled).
|
2013-02-06 22:04:50 +00:00 |
|
Andrey Prygunkov
|
3b08abca10
|
refactor: extracted par-related code from module <PrePostProcessor> into new module <ParCoordinator>
|
2013-01-23 21:32:36 +00:00 |
|
Andrey Prygunkov
|
60feae7e5b
|
new feature <Pause for X Minutes> in web-interface; new XML-/JSON-RPC method <scheduleresume>
|
2013-01-21 21:19:04 +00:00 |
|
Andrey Prygunkov
|
11c0563fe5
|
refactor: download speed and speed limit are now internally integers (Bytes) instead of floats (KB)
|
2013-01-18 21:36:17 +00:00 |
|
Andrey Prygunkov
|
2fcfbc2e1a
|
added new option <NzbAddedProcess> to setup a script called after a nzb-file is added to queue
|
2012-11-16 20:50:56 +00:00 |
|
Andrey Prygunkov
|
fdc9464576
|
added subcommand <W> to remote command <-S/--scan> to scan syncronously (wait until scan completed); added parameter <SyncMode> to XML/JSON-RPC method <scan>; the command <Scan> in web-interface now waits for completing of scan before reporting the status
|
2012-09-19 18:42:13 +00:00 |
|
Andrey Prygunkov
|
dc6c1a0fe1
|
with active option <AllowReProcess> the NZB considered completed even if there are paused non-par-files (the paused non-par-files are treated the same way as paused par-files): as a result the reprocessable script is called
|
2012-09-18 02:47:44 +00:00 |
|
Andrey Prygunkov
|
a384f0e6e9
|
prevent duplicate nzb-entries in the history
|
2012-07-03 20:21:11 +00:00 |
|
Andrey Prygunkov
|
3e81a03087
|
refactor: corrected inconsistent include of <config.h>
|
2012-06-23 18:58:56 +00:00 |
|
Andrey Prygunkov
|
4a069266d8
|
added new field <name> to nzb-info-object. It is initially set to the cleaned up name of the nzb-file. The renaming of the group changes this field. All RPC-methods related to nzb-object return the new field, the old field <NZBNicename> is now deprecated. The option <MergeNZB> now checks the <name>-field instead of <nzbfilename> (the latter is not changed when the nzb is renamed). New env-var-parameter <NZBPP_NZBNAME> for post-processing script.
|
2012-06-11 15:09:03 +00:00 |
|
Andrey Prygunkov
|
ab75a8b3e5
|
added the ability to queue URLs. The program automatically downloads nzb-files from given URLs and put them to download queue. When multiple URLs are added in a short time, they are put into a special URL-queue. The number of simultaneous URL-downloads are controlled via new option UrlConnections. With the new option ReloadUrlQueue can be controlled if the URL-queue should be reloaded after the program is restarted (if the URL-queue was not empty). New switch <-U> for remote-command <--append/-A> to queue an URL. New subcommand <-U> in the remote command <--list/-L> prints the current URL-queue. If URL-download fails, the URL is moved into history. With subcommand <-R> of command <--edit> the failed URL can be returned to URL-queue for redownload. The remote command <--list/-L> for history can now print the infos for URL history items. New XML/JSON-RPC command <appendurl> to add an URL or multiple URLs for download. New XML/JSON-RPC command <urlqueue> returns the items from the URL-queue. The XML/JSON-RPC command <history> was exteneded to provide infos about URL history items. The URL-queue obeys the pause-state of download queue. The URL-downloads support HTTP and HTTPS protocols.
|
2012-05-03 13:47:44 +00:00 |
|
Andrey Prygunkov
|
4856503a33
|
fixed: par-repair could fail when the filenames were not correctly parsed from article subjects
|
2011-12-29 21:30:25 +00:00 |
|
Andrey Prygunkov
|
2631550c2f
|
corrected the address of Free Software Foundation in copyright notice; corrected the spelling of authors name (caused by new rules for translating of cyrillic names to latin alphabet / english spelling)
|
2011-05-24 12:52:41 +00:00 |
|
Andrey Prygunkov
|
646ddb4ddb
|
fixed: elapsed time for active post-processing job was calculated incorrectly if the post-processor queue was paused; that could cause the interruption of active par-job when using the option <ParTimeLimit>
|
2010-02-08 16:08:10 +00:00 |
|
Andrey Prygunkov
|
8578078f7c
|
added second pause register, which is independent of main pause-state and therfore is intended for usage from external scripts; that allows to pause download without interferring with options <ParPauseQueue> and <PostPauseQueue> and scheduler tasks <PauseDownload> and <UnpauseDownload> - they all work with first (default) pause register; new subcommand <D2> for commands <--pause/-P> and <--unpause/-U>; new RPC-command <pausedownload2> and <resumedownload2>; existing RPC-commands <pause> und <resume> renamed to <pausedownload> and <resumedownload>; new field <Download2Paused> in result struct for RPC-command <status>; existing fields <ServerPaused> and <ParJobCount> renamed to <DownloadPaused> and <PostJobCount>; old RPC-commands and fields still exist for compatibility; the status output of command <--list/-L> indicates the state of second pause register; key <P> in curses-frontend can unpause second pause-register
|
2010-01-30 14:43:58 +00:00 |
|
Andrey Prygunkov
|
ab204281ed
|
fixed: script-status was not reset by <post-process again>
|
2010-01-29 09:53:57 +00:00 |
|
Andrey Prygunkov
|
1a9451fe61
|
command <pause post-processing> now not only pauses the post-processing queue but also pauses the current post-processing job (par-job or script-job); however the script-job can be paused only after the next line printed to screen
|
2009-12-08 21:50:53 +00:00 |
|
Andrey Prygunkov
|
5567c5abf6
|
fixed: reprocessing of a history item, which doesn't have any files left, caused a seg. fault
|
2009-09-26 19:54:53 +00:00 |
|
Andrey Prygunkov
|
b2e2c5c173
|
clearing the list of completeded files when adding a nzb-file to history if the cleaning up of queue (option <ParCleanupQueue>) is enabled (since there no more files in the nzb-file left, it cannot be returned to download queue; the list of completed files is not needed anymore)
|
2009-09-26 16:01:44 +00:00 |
|
Andrey Prygunkov
|
80116b6687
|
renamed ParStatus constant <FAILED> to <FAILURE> for a consistence with ScriptStatus constant <FAILURE>, that also affects the results of RPC-command <history>
|
2009-08-23 12:32:46 +00:00 |
|
Andrey Prygunkov
|
7d5225b2ba
|
added field <RemainingFileCount> to history items returned via RPC-command <history>
|
2009-08-23 12:18:44 +00:00 |
|
Andrey Prygunkov
|
f2f318b11e
|
fixed: returning the postprocessing exit code <92 - par-check all collections> when there were no par-files results in endless calling of postprocessing script
|
2009-08-22 13:31:03 +00:00 |
|