30 Commits

Author SHA1 Message Date
Andrey Prygunkov
491d816bff #591: save only changed queue data during downloading
This is a speed optimisation for large queue.
2019-01-14 17:53:51 +01:00
Andrey Prygunkov
57f932cfab #438: hibernating all file infos in one large disk state file
to greatly improve startup time
2017-09-02 11:36:38 +02:00
Andrey Prygunkov
18d89435e4 #205: fixed: nzb-log was deleted
… just before adding to history
2016-05-16 12:41:57 +02:00
Andrey Prygunkov
a1e945661b #205: improved handling of queue cleanup
Also removed option “ParCleanupQueue” as it is not needed anymore with
reworked state handling.
2016-05-15 19:42:54 +02:00
Andrey Prygunkov
43f25587d2 #205: improved file state handling
Better handling of all cases: manual delete, health delete, retry
failed, download remaining with ContinuePartial=on/off and
DirectWrite=on/off.
2016-05-12 23:22:06 +02:00
Andrey Prygunkov
8c4d8cef1a #121: queue directory cleanup on program start
Automatically removing orphaned diskstate files from QueueDir:
- file info state;
- file progress state;
- file completion state;
- nzb-log.
2016-04-28 22:09:56 +02:00
Andrey Prygunkov
8de5461759 #207: separate disk state files for queue and history 2016-04-23 21:33:06 +02:00
Andrey Prygunkov
b82cc06789 #205: retry failed articles
- new action in history dialog: retry failed articles;
- new command “HistoryRetryFailed” of RPC-method “editqueue”;
- new subcommand “F” of command line switch “-E/--edit” for history.
2016-04-21 22:14:12 +02:00
Andrey Prygunkov
857ada54ea #121: unified disk state management
… to use StateDiskFile for all operations.
2016-03-23 19:29:57 +01:00
Andrey Prygunkov
35049d436e #183: dropped support for old queue disk state formats
(v12 or older)
2016-03-10 21:47:38 +01:00
Andrey Prygunkov
f3f7fbd0de #176: updated copyright notice in source files
- added link to http://nzbget.net;
- replaced FSF Post address with a web link;
- removed unusable subversion-tags;
- updated year.
2016-03-01 19:45:07 +01:00
Andrey Prygunkov
8a344c97c9 #176: changed order of member declarations 2016-03-01 00:20:14 +01:00
Andrey Prygunkov
831c73d28d #176: removed aligning of class member names 2016-02-29 19:31:40 +01:00
Andrey Prygunkov
dce6f0c9a3 #131: new class DiskFile to wrap "FILE*"-functions 2015-12-25 18:11:50 +01:00
Andrey Prygunkov
70ccfd9802 normalized whitespace formatting
1) removed trailing spaces and tabs;
2) replaced occasional leading spaces with tabs.
2015-11-05 23:45:19 +01:00
Andrey Prygunkov
a9a6f1e2d4 #103: manual corrections of variable names 2015-11-03 23:33:21 +01:00
Andrey Prygunkov
bf49f16d7c #103: renamed global variables 2015-11-01 21:42:35 +01:00
Andrey Prygunkov
1fb21b330e #103: normalized (renamed) acronyms 2015-10-30 23:54:37 +01:00
Andrey Prygunkov
5adb50274e #103: renamed local, member variables and function parameters 2015-10-27 22:37:23 +01:00
Andrey Prygunkov
d7ab37ad31 #28: disk state handling for disk flush
Reworked disk state handling to use disk flush function. The function
itself is not implemented yet.
2015-07-15 23:17:17 +02:00
Andrey Prygunkov
5d24697b0c refactor: reworked declaration of global objects (singletones) 2015-05-22 20:28:05 +00:00
Andrey Prygunkov
7e6f8f19eb each nzb now has its own individual log, where messages printed during download or post-processing are saved; the messages can be retrieved later at any time; new button "Log" in the history details dialog; button "Log" in the download details dialog is now active during download too (not only during post-processing); the log contains all nzb-related messages except detail-messages and errors printed during retrieving of articles (they would produce way too many messages and are not that useful anyway); new option "NzbLog" to deactivate per-nzb logging if necessary; per-nzb logs are saved in the queue-directory (option "QueueDir"); new RPC-method "loadlog" returns the previously saved messages for a given nzb-file; new field "MessageCount" is returned by RPC-methods "listgroups" and "history" and indicates if there are any messages saved for the item; parameter "NumberOfLogEntries" of RPC-method "listgroups" and the field "Log" returned by the method are now deprecated, use method "loadlag" instead; field "PostInfoText" returned by RPC-method "listgroups" is now automatically filled with the latest message printed by a pp-script eliminating the need to access deprecated field "Log" 2015-02-26 20:57:38 +00:00
Andrey Prygunkov
c9981472a8 refactor: disk state now holds info about failed files: their IDs, CRCs of download articles and full intitial article information; these data can be used later to retry download of failed articles and for quick par-verification of damaged files 2014-08-05 23:45:28 +00:00
Andrey Prygunkov
ba9efe43be added article cache: new option "ArticleCache" defines memory limit to use for cache; when cache is active the articles are written into cache first and then all flushed to disk into the destination file; article cache reduces disk IO and may reduce file fragmentation improving post-processing speed (unpack); it works with both writing modes (direct write on and off); when option "DirectWrite" is disabled the cache should be big enough (for best performance) to accommodate all articles of one file (sometimes up to 500 MB) in order to avoid writing articles into temporary files, otherwise temporary files are used for articles which do not fill into cache; when used in combination with DirectWrite there is no such limitation and even a small cache (100 MB or even less) can be used effectively; when the cache becomes full it is flushed automatically (directly into destination file) providing room for new articles; new row in the "statistics and status dialog" in web-interface indicates the amount of memory used for cache; new fields "ArticleCacheLo", "ArticleCacheHi" and "ArticleCacheMB" returned by RPC-method "status"; refactor: parts of unit "ArticleDownloader" responsible for writing into disk were moved into new unit "ArticleWriter" 2014-07-18 22:48:35 +00:00
Andrey Prygunkov
702b635826 improved RPC-API: history items now preserve "NZBID" from queue items; that makes the tracking of items across queue and history easier for third-party apps; field "NZBID" returned by RPC-method "history" is now available for history items of all kinds (NZB, URL, DUP); field "ID" is deprecated and should not be used 2014-07-04 19:07:51 +00:00
Andrey Prygunkov
f85533d608 fixed: some nzb-file data were not calculated for history items loaded from disk state; this may cause problems for commands "Post-process again" and "Download remaining files" (bug introduced in v13) 2014-05-28 21:37:44 +00:00
Andrey Prygunkov
a83dbccc6c changed the way option "ContinuePartial" works: now the information about completed articles is stored in a special file in QueueDir; when option "DirectWrite" is active no separate flag-files per article are created in TempDir; the file contains additional information, which were not stored/available before; fixed: per-server/per-nzb article completion statistics could be inaccurate for nzb-files whose download were interrupted by reload/restart; per-server/per-nzb article completion statistics are now available via RPC-method "listgroups" for active downloads (not only for "history") 2014-04-10 20:06:55 +00:00
Andrey Prygunkov
47fbe6423e added collecting of download volume statistics data per news server; in web-interface the data is shown as chart in "Statistics and Status" dialog; new RPC-method "servervolumes" returns the collected data 2014-04-01 21:06:31 +00:00
Andrey Prygunkov
bf66500aac reworking queue (continued): merged url queue into main download queue: urls added to queue are now immediately shown in web-interface; urls can be reordered and deleted; when urls are fetched the downloaded nzb-files are put into queue at the positions of their urls; this solves the problem with fetched nzb-files ordered differently than the urls if the fetching of upper (position wise) urls were completed after of the lower urls; removed options "ReloadUrlQueue" and "ReloadPostQueue" since there are no separate url- and post-queues anymore; nzb-files added via urls have new field "URL" which can be accessed via RPC-methods "listgroups" and "history"; new env. var. "NZBNP_URL", "NZBNA_URL" and "NZBPP_URL" passed to NzbProcess, NzbAddedProcess and PostProcess-scripts; removed remote command "--list U", urls are now shown as groups by command "--list G"; RPC-method "urlqueue" is still supported for compatibility but should not be used since the urls are now returned by method "listgroups", the entries have new field "Kind" which can be "NZB" or "URL" 2014-03-18 22:35:58 +00:00
Andrey Prygunkov
8168804f05 reorganized source code directory structure: created directory 'daemon' with several subdirectories and put all source code files there 2014-02-24 22:11:14 +00:00