Commit Graph

73 Commits

Author SHA1 Message Date
Andrey Prygunkov
7ecb968e23 if download was deleted by duplicate check its status in the history is now shown as "DUPE" instead of just "DELETED" 2013-09-18 19:49:59 +00:00
Andrey Prygunkov
f66c012df6 pp-scripts can now set post-processing parameters by printing command "[NZB] NZBPR_varname=value"; this allows scripts which are executed sooner to pass data for scripts executed later 2013-08-28 22:27:50 +00:00
Andrey Prygunkov
baf996fc06 new env-var "NZBPP_FINALDIR" passed to pp-scripts 2013-08-28 21:59:02 +00:00
Andrey Prygunkov
e56a1d3274 refactor: small code rework in passing parameters to post-processing scripts 2013-08-28 20:11:05 +00:00
Andrey Prygunkov
b04af33fb5 addition to r794: removed mistakenly added pp-parameter "NZBPP_DELETED"; since post-processing is not performed for deleted downloads, this parameter has no use 2013-08-28 19:17:41 +00:00
Andrey Prygunkov
38a273b195 added collecting of server usage statistical data for each download: number of successful and failed article downloads per news server; new page in history dialog shows collected statistics; new fields in RPC-method "history": ServerStats (array), TotalArticles, SuccessArticles, FailedArticles; new env. vars passed to pp-scripts: NZBPP_TOTALARTICLES, NZBPP_SUCCESSARTICLES, NZBPP_FAILEDARTICLES and per used news server: NZBPP_SERVERX_SUCCESSARTICLES, NZBPP_SERVERX_FAILEDARTICLES; also new env.vars: DELETED, HEALTHDELETED 2013-08-16 21:53:32 +00:00
Andrey Prygunkov
9ba10446e9 added download health monitoring: health indicates download status, whether the file is damaged and how much; new option "HealthCheck" to define what to do with bad downloads (pause, delete, none); par-check is now automatically started for downloads having health below 100%, this works independently of unpack (even if unpack is disabled); for downloads having health less than critical health no par-check is performed (it would fail); new fields "Health" and "CriticalHealth" are returned by RPC-Method "listgroups"; new fields "Health", "CriticalHealth", "Deleted" and "HealthDeleted" are returned by RPC-Method "history"; new parameters "NZBPP_HEALTH" and "NZBPP_CRITICALHEALTH" are passed to pp-scripts; manually deleted downloads now have history status "deleted" (instead of "unknown") 2013-08-08 21:09:36 +00:00
Andrey Prygunkov
c938714b70 pp-scripts which move files can now inform the program about new location by printing text "[NZB] FINALDIR=/path/to/files"; the final path is then shown in history dialog instead of download path 2013-07-28 21:27:12 +00:00
Andrey Prygunkov
ec48959600 changed the way how option "Unpack" works: instead of enabling/disabling the unpacker as a whole, it now defines the initial value of post-processing parameter "Unpack" for nzb-file when it is added to queue; this makes it now possible to disable Unpack globally but still enable it for selected nzb-files; new option "CategoryX.Unpack" to set unpack on a per category basis 2013-07-21 20:44:13 +00:00
Andrey Prygunkov
c31d38a562 fixed: certain characters printed by pp-scripts could crash the program 2013-07-20 14:02:26 +00:00
Andrey Prygunkov
5a815592dc added new option "CategoryX.Aliases" to configure category name matching with nzb-sites; especially useful with rss feeds 2013-07-15 21:28:55 +00:00
Andrey Prygunkov
5b3113d96b 1) when a nzb-file is added via web-interface or via remote call the file is now put into incoming nzb-directory (option "NzbDir") and then scanned; this has two advantages over the old behavior when the file was parsed directly in memory: the file serves as a backup for troubleshootings and the file is processed by nzbprocess-script (if defined in option "NzbProcess") making the pre-processing much easier; 2) new env-var parameters are passed to NzbProcess-script: NZBNP_NZBNAME, NZBNP_CATEGORY, NZBNP_PRIORITY, NZBNP_TOP, NZBNP_PAUSED; 3) new commands for use in NzbProcess-scripts: "[NZB] TOP=1" to add nzb to the top of queue and "[NZB] PAUSED=1" to add nzb-file in paused state 2013-06-17 20:39:46 +00:00
Andrey Prygunkov
b15499c1dd removed option "ProcessLogKind"; scripts should use prefixes ([INFO], [DETAIL], etc); messages printed without prefixes are added as [INFO] 2013-06-03 20:56:56 +00:00
Andrey Prygunkov
7991c06543 fixed: in the option "NzbAddedProcess" the env-var parameter with nzb-name was passed in "NZBNA_NAME", should be "NZBNA_NZBNAME"; the old parameter name "NZBNA_NAME" is still supported for compatibility 2013-06-02 21:11:09 +00:00
Andrey Prygunkov
36814514b7 new parameter (env. var) <NZBPP_NZBID> is passed to pp_scripts and contains an internal ID of NZB-file 2013-05-21 20:34:42 +00:00
Andrey Prygunkov
381a9a28b0 pp-scripts terminated with unknown status are now considered failed (status=FAILURE instead of status=UNKNOWN) 2013-05-21 20:32:41 +00:00
Andrey Prygunkov
d9b9786486 improved par-check: added support for manual par-check; if option <ParCheck> is set to <Manual> and a damaged download is detected the program downloads all par2-files but doesn't perform par-check; the user must perform par-check/repair manually then (possibly on another, faster computer); old values <yes/no> of option <ParCheck> renamed to <Force> and <Auto> respectively; when set to <Force> all par2-files are always downloaded; removed option <LoadPars> since its functionality is now covered by option <ParCheck>; Result of par-check can now have new value <Manual repair necessary>; field <ParStatus> in RPC-method <history> can have new value <MANUAL>; parameter <NZBPP_PARSTATUS> for pp-script can have new value <4 = manual repair necessary>; extended pp-script <EMail.py> to handle ParStatus=4 (manual) 2013-05-02 20:40:36 +00:00
Andrey Prygunkov
cb90c5e616 when logging messages from a post-processing script, a short name of the script is now used as prefix if possible; a short name doesn't include subdirectory name or file extension; RPC-method <configtemplates> returns new field <DisplayName> representing the short name of the script which is recommended for using in UI 2013-04-26 20:01:53 +00:00
Andrey Prygunkov
025cd043d3 history dialog now shows status of every script 2013-04-23 18:20:52 +00:00
Andrey Prygunkov
c9b7a11a89 fixed: by adding nzb-files with assigned category and empty option <CategoryX.DefScript> the global option <DefScript> should be used but it wasn't 2013-04-19 17:34:17 +00:00
Andrey Prygunkov
e053e74b58 fixed: scripts containg spaces in their names were not assigned to nzb-files by adding to queue (when defined in option <DefScript>) 2013-04-18 20:28:31 +00:00
Andrey Prygunkov
f28b35bd28 reworked concept of post-processing scripts: multiple scripts can be assigned to each nzb-file; all assigned scripts are executed after the nzb-file is downloaded and internally processed (unpack, repair); option <PostProcess> is obsolete; new option <ScriptDir> sets directory where all pp-scripts must be stored; new option <DefScript> sets the default list of pp-scripts to be assigned to nzb-file when it's added to queue; new option <CategoryX.DefScript> to set the default list of pp-scripts on a category basis; the execution order of pp-scripts can be set using new option <ScriptOrder>; there are no separate configuration files for pp-scripts; configuration options and pp-parameters are defined in the pp-scripts; script configuration options are saved in nzbget configuration file (nzbget.conf); changed parameters list of RPC-methods <loadconfig> and <saveconfig>; new RPC-method <configtemplates> returns configuration descriptions for the program and for all pp-scripts; configuration of all scripts can be done in web-interface; the pp-scripts assigned to a particular nzb-file can be viewed and changed in web-interface on page <pp-parameters> in the edit download dialog; option <PostPauseQueue> renamed to <ScriptPauseQueue> (the old name is still recognized); new option <ConfigTemplate> to define the location of template configuration file (in previous versions it must be always stored in <WebDir>) 2013-04-15 20:06:05 +00:00
Andrey Prygunkov
58b0a17986 reworked post-processor queue: 1) only one job is created for each nzb-file; no more separate jobs are created for par-collections within one nzb-file; 2) option <AllowReProcess> removed; a post-processing script is called only once per nzb-file, this behavior cannot be altered anymore; 3) with a new feature <Split> (see next commits) individual par-collections can be processed separately in a more effective way than before 2013-04-06 20:25:07 +00:00
Andrey Prygunkov
5473e57b10 fixed: if an external program (unrar, pp-script, etc.) could not be started, the execute-function has returned code 255 although the code -1 were expected in this case; this could break designed post-processing flow 2013-03-10 22:19:46 +00:00
Andrey Prygunkov
57b5d40851 when post-processing-parameters are passed to the post-processing script a second version of each parameter with a normalized parameter-name is passed in addition to the original parameter name; in the normalized name the special characters <*> and <:> are replaced with <_> and all characters are passed in upper case; this is important for internal post-processing-parameters (*Unpack:=yes/no) which includes special characters 2013-03-07 16:45:41 +00:00
Andrey Prygunkov
d3dd8dc686 fixed a compilation warning 2013-03-03 20:48:14 +00:00
Andrey Prygunkov
382faa49cb added new option <InterDir> to put intermediate files during download into a separate directory (instead of storing them directly in destination directory (option <DestDir>); when nzb-file is completely (successfully) downloaded, repaired (if neccessary) and unpacked the files are moved to destination directory (option <DestDir> or <CategoryX.DestDir>); intermediate directory can significantly improve unpack performance if it is located on a separate physical hard drive 2013-03-01 20:32:17 +00:00
Andrey Prygunkov
749b4d3083 when a history item is post-processed again and the archive files were previously deleted because of option <UnpackCleanupDisk> the post-processing goes directly to script stage; if the archive files were kept, the full post-processing including unpack is performed instead 2013-02-28 20:23:50 +00:00
Andrey Prygunkov
30d792b35b when running external programs (such as unrar or post-processing script) the full path to the program is not neccessary since the search in system PATH is now performed 2013-02-09 10:58:14 +00:00
Andrey Prygunkov
940448ffae added built-in unpack: 1) rar and 7-zip formats are supported (via external Unrar and 7-Zip executables); 2) new options <Unpack>, <UnpackPauseQueue>, <UnpackCleanupDisk>, <UnrarCmd>, <SevenZipCmd>; 3) web-interface now shows progress and estimated time during unpack (rar only; for 7-Zip progress is not available due to limitations of 7-Zip) 4) when built-in unpack is enabled, the post-processing script is called after unpack and possibly par-check/repair (if needed); 5) for nzb-files containing multiple collections (par-sets) the post-processing script is called only once, after the last par-set; 6) new parameter <NZBPP_UNPACKSTATUS> passed to post-processing script; 7) if the option <AllowReProcess> is enabled the post-processing-script is called after each par-set (as in previous versions); 8) example post-processing script updated: removed unrar-code, added check for unpack status; 9) new field <UnpackStatus> in result of RPC-method <history>; 10) history-dialog in web-interface shows three status: par-status, unpack-status, script-status; 11) with two built-in special post-processing parameters <*Unpack:> and <*Unpack:Password> the unpack can be disabled for individual nzb-file or the password can be set; 12) built-in special post-processing parameters can be set via web-interface on page <PP-Parameters> (when built-in unpack is enabled). 2013-02-06 22:04:50 +00:00
Andrey Prygunkov
2fcfbc2e1a added new option <NzbAddedProcess> to setup a script called after a nzb-file is added to queue 2012-11-16 20:50:56 +00:00
Andrey Prygunkov
8b1aff33fe added remote command <--reload/-O> and JSON/XML-RPC method <reload> to reload configuration from disk and reintialize the program; the reload can be performed from web-interface 2012-09-27 20:13:25 +00:00
Andrey Prygunkov
3e81a03087 refactor: corrected inconsistent include of <config.h> 2012-06-23 18:58:56 +00:00
Andrey Prygunkov
4a069266d8 added new field <name> to nzb-info-object. It is initially set to the cleaned up name of the nzb-file. The renaming of the group changes this field. All RPC-methods related to nzb-object return the new field, the old field <NZBNicename> is now deprecated. The option <MergeNZB> now checks the <name>-field instead of <nzbfilename> (the latter is not changed when the nzb is renamed). New env-var-parameter <NZBPP_NZBNAME> for post-processing script. 2012-06-11 15:09:03 +00:00
Andrey Prygunkov
2631550c2f corrected the address of Free Software Foundation in copyright notice; corrected the spelling of authors name (caused by new rules for translating of cyrillic names to latin alphabet / english spelling) 2011-05-24 12:52:41 +00:00
Andrey Prygunkov
08f86cbcf9 added priorities; new action <I> for remote command <--edit/-E> to set priorities for groups or individual files; new actions <SetGroupPriority> and <SetFilePriority> of RPC-command <EditQueue>; remote command <--list/-L> prints priorities and indicates files or groups being downloaded; ncurses-frontend prints priorities and indicates files or groups being download; new command <PRIORITY> to set priority of nzb-file from nzbprocess-script; RPC-commands <ListGroups> and <ListFiles> return priorities and indicate files or groups being downloaded 2011-03-12 11:48:13 +00:00
Andrey Prygunkov
646ddb4ddb fixed: elapsed time for active post-processing job was calculated incorrectly if the post-processor queue was paused; that could cause the interruption of active par-job when using the option <ParTimeLimit> 2010-02-08 16:08:10 +00:00
Andrey Prygunkov
31940d8f58 nzbprocess-script (option <NZBProcess>) can now set category and post-processing parameters for nzb-file 2010-01-29 09:34:44 +00:00
Andrey Prygunkov
1a9451fe61 command <pause post-processing> now not only pauses the post-processing queue but also pauses the current post-processing job (par-job or script-job); however the script-job can be paused only after the next line printed to screen 2009-12-08 21:50:53 +00:00
Andrey Prygunkov
0722417e0b added a new return code <95/POSTPROCESS_NONE> for post-processing scripts for cases when pp-script skips all post-processing work (typically upon a user's request via a pp-parameter); modified the example post-processing script to return the new code instead of a error code when a pp-parameter <PostProcess> was set to <no> 2009-09-20 15:24:19 +00:00
Andrey Prygunkov
dc9e938510 Added message log to history items; new field <Log> is returned by RPC-command <history> for each item 2009-09-19 15:49:08 +00:00
Andrey Prygunkov
feadf59fa0 added history: new option <KeepHistory>, new remote subcommand <H> for commands <L> (list history entries) and <E> (delete history entries), new RPC-command <History> and subcommand <HistoryDelete> for command <EditQueue>; changed the result code returning status <ERROR> for postprocessing script from <1> to <94> 2009-06-30 16:08:41 +00:00
Andrey Prygunkov
4b538b419a refactoring: extended the meaning of DowloadQueue-object, which now contains file queue, post-job queue, list of completed post-jobs and list of nzb-infos; post-jobs now have references to nzb-infos, what eiliminates duplicate infos; since file queue and post-job queue are now accessed via DownloadQueue only one locking mechanism for both queues is needed 2009-05-24 14:26:53 +00:00
Andrey Prygunkov
57b5afa676 fixed: environment block was not passed correctly to child process, what could result in seg faults (windows only) 2009-05-16 18:46:51 +00:00
Andrey Prygunkov
191b8531be option <TaskX.Process> now can contain parameters which must be passed to the script 2009-05-08 16:57:12 +00:00
Andrey Prygunkov
bbdfb2b4f2 added modifier <O> to command <-E/--edit> for editing of post-processor-queue; following subcommands are supported: <+/-offset>, <T>, <B>, <D>; subcommand <D> supports deletion of queued post-jobs and active job as well; deletion of active job means the cancelling of par-check/repair or terminating of post-processing-script (including child processes of the script); updated remote-server to support new edit-subcommands in XML/JSON-RPC 2009-05-03 17:14:52 +00:00
Andrey Prygunkov
10269dc779 added new return code (93) for post-processing script to indicate successful processing; that results in cleaning up of download queue if option <ParCleanupQueue> is active 2009-04-23 20:39:07 +00:00
Andrey Prygunkov
8c4b5c7f6b fixed small memory leak appeared if process-script could not be started 2009-01-15 17:45:21 +00:00
Andrey Prygunkov
4f4e7f8b61 fixed: the output of process-scripts was not always received (bug introduced in r250) 2008-12-02 22:32:02 +00:00
Andrey Prygunkov
5993408eff added command <Process> to scheduler option <TaskX.Command>; replaced options <PostLogKind> and <NzbLogKind> with new option <ProcessLogKind> which is now used by all scripts (PostProcess, NzbProcess, TaskX.Process) 2008-12-01 22:50:35 +00:00