Commit Graph

114 Commits

Author SHA1 Message Date
Andrey Prygunkov
ec48959600 changed the way how option "Unpack" works: instead of enabling/disabling the unpacker as a whole, it now defines the initial value of post-processing parameter "Unpack" for nzb-file when it is added to queue; this makes it now possible to disable Unpack globally but still enable it for selected nzb-files; new option "CategoryX.Unpack" to set unpack on a per category basis 2013-07-21 20:44:13 +00:00
Andrey Prygunkov
5b3113d96b 1) when a nzb-file is added via web-interface or via remote call the file is now put into incoming nzb-directory (option "NzbDir") and then scanned; this has two advantages over the old behavior when the file was parsed directly in memory: the file serves as a backup for troubleshootings and the file is processed by nzbprocess-script (if defined in option "NzbProcess") making the pre-processing much easier; 2) new env-var parameters are passed to NzbProcess-script: NZBNP_NZBNAME, NZBNP_CATEGORY, NZBNP_PRIORITY, NZBNP_TOP, NZBNP_PAUSED; 3) new commands for use in NzbProcess-scripts: "[NZB] TOP=1" to add nzb to the top of queue and "[NZB] PAUSED=1" to add nzb-file in paused state 2013-06-17 20:39:46 +00:00
Andrey Prygunkov
9051a4df4d fixed: if the last file of collection was detected as duplicate after the download of the first article the file was deleted from queue (that's OK) but the post-processing was not triggered (that's a bug) 2013-06-13 20:51:38 +00:00
Andrey Prygunkov
606021fb8a removed option "AppendNzbDir"; if it was disabled that caused problems in par-checker and unpacker; the option is now assumed always active 2013-06-04 20:56:17 +00:00
Andrey Prygunkov
1348ac86f7 added setting of post-processing parameters for history items; pp-parameters can now be viewed and changed in history dialog in web-interface; useful before post-processing again; new action <HistorySetParameter> in RPC-method <editqueue>; new action <O> in remote command <--edit/-E> for history items (subcommand <H>) 2013-05-16 20:54:13 +00:00
Andrey Prygunkov
340a8130e9 addition to r677: added missing headers, causing compilation error in newer gcc versions 2013-05-15 16:45:36 +00:00
Andrey Prygunkov
9ab955d026 refactor: more consistent using of c-headers 2013-05-14 20:20:52 +00:00
Andrey Prygunkov
ed36feeb0a fixed: by deleting of a partially downloaded nzb-file from queue, when the option <DeleteCleanupDisk> was active, the file <_brokenlog.txt> was not deleted preventing the directory from automatic deletion 2013-05-13 20:13:28 +00:00
Andrey Prygunkov
26361630c2 addition to r660: fixed: downloads were always checked when option <ParCheck> was set to <Auto> 2013-05-04 07:45:00 +00:00
Andrey Prygunkov
d9b9786486 improved par-check: added support for manual par-check; if option <ParCheck> is set to <Manual> and a damaged download is detected the program downloads all par2-files but doesn't perform par-check; the user must perform par-check/repair manually then (possibly on another, faster computer); old values <yes/no> of option <ParCheck> renamed to <Force> and <Auto> respectively; when set to <Force> all par2-files are always downloaded; removed option <LoadPars> since its functionality is now covered by option <ParCheck>; Result of par-check can now have new value <Manual repair necessary>; field <ParStatus> in RPC-method <history> can have new value <MANUAL>; parameter <NZBPP_PARSTATUS> for pp-script can have new value <4 = manual repair necessary>; extended pp-script <EMail.py> to handle ParStatus=4 (manual) 2013-05-02 20:40:36 +00:00
Andrey Prygunkov
6bb760375e added option <ExtCleanupDisk> to automatically delete unwanted files (with specified extensions) after successful par-check or unpack 2013-04-24 20:16:04 +00:00
Andrey Prygunkov
025cd043d3 history dialog now shows status of every script 2013-04-23 18:20:52 +00:00
Andrey Prygunkov
33cb2d108e fixed: if a download didn't have any par-files and the option <ParCheck> was active, the par-check was started anyway and then failed 2013-04-18 21:04:48 +00:00
Andrey Prygunkov
f28b35bd28 reworked concept of post-processing scripts: multiple scripts can be assigned to each nzb-file; all assigned scripts are executed after the nzb-file is downloaded and internally processed (unpack, repair); option <PostProcess> is obsolete; new option <ScriptDir> sets directory where all pp-scripts must be stored; new option <DefScript> sets the default list of pp-scripts to be assigned to nzb-file when it's added to queue; new option <CategoryX.DefScript> to set the default list of pp-scripts on a category basis; the execution order of pp-scripts can be set using new option <ScriptOrder>; there are no separate configuration files for pp-scripts; configuration options and pp-parameters are defined in the pp-scripts; script configuration options are saved in nzbget configuration file (nzbget.conf); changed parameters list of RPC-methods <loadconfig> and <saveconfig>; new RPC-method <configtemplates> returns configuration descriptions for the program and for all pp-scripts; configuration of all scripts can be done in web-interface; the pp-scripts assigned to a particular nzb-file can be viewed and changed in web-interface on page <pp-parameters> in the edit download dialog; option <PostPauseQueue> renamed to <ScriptPauseQueue> (the old name is still recognized); new option <ConfigTemplate> to define the location of template configuration file (in previous versions it must be always stored in <WebDir>) 2013-04-15 20:06:05 +00:00
Andrey Prygunkov
58b0a17986 reworked post-processor queue: 1) only one job is created for each nzb-file; no more separate jobs are created for par-collections within one nzb-file; 2) option <AllowReProcess> removed; a post-processing script is called only once per nzb-file, this behavior cannot be altered anymore; 3) with a new feature <Split> (see next commits) individual par-collections can be processed separately in a more effective way than before 2013-04-06 20:25:07 +00:00
Andrey Prygunkov
1230d9cdd4 added fast renaming of intentionally misnamed (rar-) files; the new renaming algorithm doesn't require full par-scan and restores original filenames in just a few seconds, even on very slow computers (NAS, media players, etc.); the fast renaming is performed automatically when requested by the built-in unpacker (option <Unpack> must be active) 2013-03-04 19:55:36 +00:00
Andrey Prygunkov
382faa49cb added new option <InterDir> to put intermediate files during download into a separate directory (instead of storing them directly in destination directory (option <DestDir>); when nzb-file is completely (successfully) downloaded, repaired (if neccessary) and unpacked the files are moved to destination directory (option <DestDir> or <CategoryX.DestDir>); intermediate directory can significantly improve unpack performance if it is located on a separate physical hard drive 2013-03-01 20:32:17 +00:00
Andrey Prygunkov
749b4d3083 when a history item is post-processed again and the archive files were previously deleted because of option <UnpackCleanupDisk> the post-processing goes directly to script stage; if the archive files were kept, the full post-processing including unpack is performed instead 2013-02-28 20:23:50 +00:00
Andrey Prygunkov
940448ffae added built-in unpack: 1) rar and 7-zip formats are supported (via external Unrar and 7-Zip executables); 2) new options <Unpack>, <UnpackPauseQueue>, <UnpackCleanupDisk>, <UnrarCmd>, <SevenZipCmd>; 3) web-interface now shows progress and estimated time during unpack (rar only; for 7-Zip progress is not available due to limitations of 7-Zip) 4) when built-in unpack is enabled, the post-processing script is called after unpack and possibly par-check/repair (if needed); 5) for nzb-files containing multiple collections (par-sets) the post-processing script is called only once, after the last par-set; 6) new parameter <NZBPP_UNPACKSTATUS> passed to post-processing script; 7) if the option <AllowReProcess> is enabled the post-processing-script is called after each par-set (as in previous versions); 8) example post-processing script updated: removed unrar-code, added check for unpack status; 9) new field <UnpackStatus> in result of RPC-method <history>; 10) history-dialog in web-interface shows three status: par-status, unpack-status, script-status; 11) with two built-in special post-processing parameters <*Unpack:> and <*Unpack:Password> the unpack can be disabled for individual nzb-file or the password can be set; 12) built-in special post-processing parameters can be set via web-interface on page <PP-Parameters> (when built-in unpack is enabled). 2013-02-06 22:04:50 +00:00
Andrey Prygunkov
3b08abca10 refactor: extracted par-related code from module <PrePostProcessor> into new module <ParCoordinator> 2013-01-23 21:32:36 +00:00
Andrey Prygunkov
60feae7e5b new feature <Pause for X Minutes> in web-interface; new XML-/JSON-RPC method <scheduleresume> 2013-01-21 21:19:04 +00:00
Andrey Prygunkov
11c0563fe5 refactor: download speed and speed limit are now internally integers (Bytes) instead of floats (KB) 2013-01-18 21:36:17 +00:00
Andrey Prygunkov
2fcfbc2e1a added new option <NzbAddedProcess> to setup a script called after a nzb-file is added to queue 2012-11-16 20:50:56 +00:00
Andrey Prygunkov
fdc9464576 added subcommand <W> to remote command <-S/--scan> to scan syncronously (wait until scan completed); added parameter <SyncMode> to XML/JSON-RPC method <scan>; the command <Scan> in web-interface now waits for completing of scan before reporting the status 2012-09-19 18:42:13 +00:00
Andrey Prygunkov
dc6c1a0fe1 with active option <AllowReProcess> the NZB considered completed even if there are paused non-par-files (the paused non-par-files are treated the same way as paused par-files): as a result the reprocessable script is called 2012-09-18 02:47:44 +00:00
Andrey Prygunkov
a384f0e6e9 prevent duplicate nzb-entries in the history 2012-07-03 20:21:11 +00:00
Andrey Prygunkov
3e81a03087 refactor: corrected inconsistent include of <config.h> 2012-06-23 18:58:56 +00:00
Andrey Prygunkov
4a069266d8 added new field <name> to nzb-info-object. It is initially set to the cleaned up name of the nzb-file. The renaming of the group changes this field. All RPC-methods related to nzb-object return the new field, the old field <NZBNicename> is now deprecated. The option <MergeNZB> now checks the <name>-field instead of <nzbfilename> (the latter is not changed when the nzb is renamed). New env-var-parameter <NZBPP_NZBNAME> for post-processing script. 2012-06-11 15:09:03 +00:00
Andrey Prygunkov
ab75a8b3e5 added the ability to queue URLs. The program automatically downloads nzb-files from given URLs and put them to download queue. When multiple URLs are added in a short time, they are put into a special URL-queue. The number of simultaneous URL-downloads are controlled via new option UrlConnections. With the new option ReloadUrlQueue can be controlled if the URL-queue should be reloaded after the program is restarted (if the URL-queue was not empty). New switch <-U> for remote-command <--append/-A> to queue an URL. New subcommand <-U> in the remote command <--list/-L> prints the current URL-queue. If URL-download fails, the URL is moved into history. With subcommand <-R> of command <--edit> the failed URL can be returned to URL-queue for redownload. The remote command <--list/-L> for history can now print the infos for URL history items. New XML/JSON-RPC command <appendurl> to add an URL or multiple URLs for download. New XML/JSON-RPC command <urlqueue> returns the items from the URL-queue. The XML/JSON-RPC command <history> was exteneded to provide infos about URL history items. The URL-queue obeys the pause-state of download queue. The URL-downloads support HTTP and HTTPS protocols. 2012-05-03 13:47:44 +00:00
Andrey Prygunkov
4856503a33 fixed: par-repair could fail when the filenames were not correctly parsed from article subjects 2011-12-29 21:30:25 +00:00
Andrey Prygunkov
2631550c2f corrected the address of Free Software Foundation in copyright notice; corrected the spelling of authors name (caused by new rules for translating of cyrillic names to latin alphabet / english spelling) 2011-05-24 12:52:41 +00:00
Andrey Prygunkov
646ddb4ddb fixed: elapsed time for active post-processing job was calculated incorrectly if the post-processor queue was paused; that could cause the interruption of active par-job when using the option <ParTimeLimit> 2010-02-08 16:08:10 +00:00
Andrey Prygunkov
8578078f7c added second pause register, which is independent of main pause-state and therfore is intended for usage from external scripts; that allows to pause download without interferring with options <ParPauseQueue> and <PostPauseQueue> and scheduler tasks <PauseDownload> and <UnpauseDownload> - they all work with first (default) pause register; new subcommand <D2> for commands <--pause/-P> and <--unpause/-U>; new RPC-command <pausedownload2> and <resumedownload2>; existing RPC-commands <pause> und <resume> renamed to <pausedownload> and <resumedownload>; new field <Download2Paused> in result struct for RPC-command <status>; existing fields <ServerPaused> and <ParJobCount> renamed to <DownloadPaused> and <PostJobCount>; old RPC-commands and fields still exist for compatibility; the status output of command <--list/-L> indicates the state of second pause register; key <P> in curses-frontend can unpause second pause-register 2010-01-30 14:43:58 +00:00
Andrey Prygunkov
ab204281ed fixed: script-status was not reset by <post-process again> 2010-01-29 09:53:57 +00:00
Andrey Prygunkov
1a9451fe61 command <pause post-processing> now not only pauses the post-processing queue but also pauses the current post-processing job (par-job or script-job); however the script-job can be paused only after the next line printed to screen 2009-12-08 21:50:53 +00:00
Andrey Prygunkov
5567c5abf6 fixed: reprocessing of a history item, which doesn't have any files left, caused a seg. fault 2009-09-26 19:54:53 +00:00
Andrey Prygunkov
b2e2c5c173 clearing the list of completeded files when adding a nzb-file to history if the cleaning up of queue (option <ParCleanupQueue>) is enabled (since there no more files in the nzb-file left, it cannot be returned to download queue; the list of completed files is not needed anymore) 2009-09-26 16:01:44 +00:00
Andrey Prygunkov
80116b6687 renamed ParStatus constant <FAILED> to <FAILURE> for a consistence with ScriptStatus constant <FAILURE>, that also affects the results of RPC-command <history> 2009-08-23 12:32:46 +00:00
Andrey Prygunkov
7d5225b2ba added field <RemainingFileCount> to history items returned via RPC-command <history> 2009-08-23 12:18:44 +00:00
Andrey Prygunkov
f2f318b11e fixed: returning the postprocessing exit code <92 - par-check all collections> when there were no par-files results in endless calling of postprocessing script 2009-08-22 13:31:03 +00:00
Andrey Prygunkov
6bfe1b0cfd added actions <R> (return history item) and <P> (postprocess history item) for subcommand <H> of command <-E/--edit>; added actions <HistoryReturn> and <HistoryProcess> for remote RPC-command <editqueue> 2009-08-22 12:05:25 +00:00
Andrey Prygunkov
feadf59fa0 added history: new option <KeepHistory>, new remote subcommand <H> for commands <L> (list history entries) and <E> (delete history entries), new RPC-command <History> and subcommand <HistoryDelete> for command <EditQueue>; changed the result code returning status <ERROR> for postprocessing script from <1> to <94> 2009-06-30 16:08:41 +00:00
Andrey Prygunkov
123cfe6a38 added <binary> flag by opening of all files to explicitly set binary mode for compatibility with OSes which do not do that by default (like Posix does) 2009-06-11 21:13:37 +00:00
Andrey Prygunkov
34fe0f1077 refactoring: NZBInfo-objects now holds the status of completed post-process jobs; that made the list of completed post jobs unnecessary 2009-06-03 16:00:35 +00:00
Andrey Prygunkov
419b7fcb2c refactoring: moved the code for checking the incoming nzb directory from <PrePostProcessor> into the new module <Scanner> 2009-05-29 22:37:51 +00:00
Andrey Prygunkov
6471928f91 added subcommand <S> to remote commands <--pause/-P> and <--unpause/-U> to pause/unpause the scanning of incoming nzb-directory; added commands <PauseScan> and <UnpauseScan> for scheduler option <TaskX.Command>; added remote commands <PauseScan> and <ResumeScan> for XML-/JSON-RPC 2009-05-25 20:18:20 +00:00
Andrey Prygunkov
4b538b419a refactoring: extended the meaning of DowloadQueue-object, which now contains file queue, post-job queue, list of completed post-jobs and list of nzb-infos; post-jobs now have references to nzb-infos, what eiliminates duplicate infos; since file queue and post-job queue are now accessed via DownloadQueue only one locking mechanism for both queues is needed 2009-05-24 14:26:53 +00:00
Andrey Prygunkov
b1f6735e87 added pausing/resuming for post-processor queue; added new modifier <O> to remote commands <--pause/-P> and <--unpause/-U>; added new commands <postpause> and <postresume> to XML-/JSON-RPC; extended output of remote command <--list/-L> to indicate paused state of post-processor queue; extended command <status> of XML-/JSON-RPC with field <PostPause> 2009-05-06 19:28:54 +00:00
Andrey Prygunkov
bbdfb2b4f2 added modifier <O> to command <-E/--edit> for editing of post-processor-queue; following subcommands are supported: <+/-offset>, <T>, <B>, <D>; subcommand <D> supports deletion of queued post-jobs and active job as well; deletion of active job means the cancelling of par-check/repair or terminating of post-processing-script (including child processes of the script); updated remote-server to support new edit-subcommands in XML/JSON-RPC 2009-05-03 17:14:52 +00:00
Andrey Prygunkov
5d5c6b06b9 commited changes missed in previous revision 2009-04-24 05:48:55 +00:00