Compare commits

..

1 Commits
v16.3 ... v13.0

Author SHA1 Message Date
Andrey Prygunkov
491fc1456e version 13.0 2014-07-14 20:51:33 +00:00
256 changed files with 13130 additions and 48995 deletions

25
.gitattributes vendored
View File

@@ -1,25 +0,0 @@
* text=auto
# Use CRLF for certain Windows files.
*.vcproj eol=crlf
*.sln eol=crlf
*.bat eol=crlf
README-WINDOWS.txt eol=crlf
nzbget-setup.nsi eol=crlf
windows/package-info.json eol=crlf
windows/resources/resource.h eol=crlf
windows/resources/nzbget.rc eol=crlf
# Configure GitHub's language detector
lib/* linguist-vendored linguist-language=C++
webui/lib/* linguist-vendored
Makefile.in linguist-vendored
configure linguist-vendored
config.sub linguist-vendored
aclocal.m4 linguist-vendored
config.guess linguist-vendored
depcomp linguist-vendored
install-sh linguist-vendored
missing linguist-vendored
configure.ac linguist-vendored=false
Makefile.am linguist-vendored=false

63
.gitignore vendored
View File

@@ -1,63 +0,0 @@
# Compiled Object files
*.slo
*.lo
*.o
*.obj
# Precompiled Headers
*.gch
*.pch
# Compiled Dynamic libraries
*.so
*.dylib
*.dll
# Compiled Static libraries
*.lai
*.la
*.a
*.lib
# Executables
*.exe
*.out
# GNU Autotools
.deps/
config.h
config.log
config.status
Makefile
stamp-h1
autom4te.cache/
# Visual Studio User-specific files
*.suo
*.user
*.userosscache
*.sln.docstates
.vs/
ipch/
*.aps
*.ncb
*.opensdf
*.sdf
*.cachefile
*.ilk
*.pch
*.pdb
*.pgc
*.pgd
*.rsp
*.sbr
*.vspscc
*.vssscc
*.pidb
*.svclog
*.scc
*.sln
# NZBGet specific
nzbget
code_revision.cpp

4
AUTHORS Normal file
View File

@@ -0,0 +1,4 @@
nzbget:
Sven Henkel <sidddy@users.sourceforge.net> (versions 0.1.0 - ?)
Bo Cordes Petersen <placebodk@users.sourceforge.net> (versions ? - 0.2.3)
Andrey Prygunkov <hugbug@users.sourceforge.net> (versions 0.3.0 and later)

1521
ChangeLog
View File

File diff suppressed because it is too large Load Diff

167
INSTALL Normal file
View File

@@ -0,0 +1,167 @@
Basic Installation
==================
These are generic installation instructions.
The `configure' shell script attempts to guess correct values for
various system-dependent variables used during compilation. It uses
those values to create a `Makefile' in each directory of the package.
It may also create one or more `.h' files containing system-dependent
definitions. Finally, it creates a shell script `config.status' that
you can run in the future to recreate the current configuration, a file
`config.cache' that saves the results of its tests to speed up
reconfiguring, and a file `config.log' containing compiler output
(useful mainly for debugging `configure').
If you need to do unusual things to compile the package, please try
to figure out how `configure' could check whether to do them, and mail
diffs or instructions to the address given in the `README' so they can
be considered for the next release. If at some point `config.cache'
contains results you don't want to keep, you may remove or edit it.
The file `configure.in' is used to create `configure' by a program
called `autoconf'. You only need `configure.in' if you want to change
it or regenerate `configure' using a newer version of `autoconf'.
The simplest way to compile this package is:
1. `cd' to the directory containing the package's source code and type
`./configure' to configure the package for your system. If you're
using `csh' on an old version of System V, you might need to type
`sh ./configure' instead to prevent `csh' from trying to execute
`configure' itself.
Running `configure' takes a while. While running, it prints some
messages telling which features it is checking for.
2. Type `make' to compile the package.
3. Type `make install' to install the programs and any data files and
documentation.
4. You can remove the program binaries and object files from the
source code directory by typing `make clean'.
Compilers and Options
=====================
Some systems require unusual options for compilation or linking that
the `configure' script does not know about. You can give `configure'
initial values for variables by setting them in the environment. Using
a Bourne-compatible shell, you can do that on the command line like
this:
CC=c89 CFLAGS=-O2 LIBS=-lposix ./configure
Or on systems that have the `env' program, you can do it like this:
env CPPFLAGS=-I/usr/local/include LDFLAGS=-s ./configure
Compiling For Multiple Architectures
====================================
You can compile the package for more than one kind of computer at the
same time, by placing the object files for each architecture in their
own directory. To do this, you must use a version of `make' that
supports the `VPATH' variable, such as GNU `make'. `cd' to the
directory where you want the object files and executables to go and run
the `configure' script. `configure' automatically checks for the
source code in the directory that `configure' is in and in `..'.
If you have to use a `make' that does not supports the `VPATH'
variable, you have to compile the package for one architecture at a time
in the source code directory. After you have installed the package for
one architecture, use `make distclean' before reconfiguring for another
architecture.
Installation Names
==================
By default, `make install' will install the package's files in
`/usr/local/bin', `/usr/local/man', etc. You can specify an
installation prefix other than `/usr/local' by giving `configure' the
option `--prefix=PATH'.
You can specify separate installation prefixes for
architecture-specific files and architecture-independent files. If you
give `configure' the option `--exec-prefix=PATH', the package will use
PATH as the prefix for installing programs and libraries.
Documentation and other data files will still use the regular prefix.
If the package supports it, you can cause programs to be installed
with an extra prefix or suffix on their names by giving `configure' the
option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'.
Optional Features
=================
Some packages pay attention to `--enable-FEATURE' options to
`configure', where FEATURE indicates an optional part of the package.
They may also pay attention to `--with-PACKAGE' options, where PACKAGE
is something like `gnu-as' or `x' (for the X Window System). The
`README' should mention any `--enable-' and `--with-' options that the
package recognizes.
For packages that use the X Window System, `configure' can usually
find the X include and library files automatically, but if it doesn't,
you can use the `configure' options `--x-includes=DIR' and
`--x-libraries=DIR' to specify their locations.
Specifying the System Type
==========================
There may be some features `configure' can not figure out
automatically, but needs to determine by the type of host the package
will run on. Usually `configure' can figure that out, but if it prints
a message saying it can not guess the host type, give it the
`--host=TYPE' option. TYPE can either be a short name for the system
type, such as `sun4', or a canonical name with three fields:
CPU-COMPANY-SYSTEM
See the file `config.sub' for the possible values of each field. If
`config.sub' isn't included in this package, then this package doesn't
need to know the host type.
If you are building compiler tools for cross-compiling, you can also
use the `--target=TYPE' option to select the type of system they will
produce code for and the `--build=TYPE' option to select the type of
system on which you are compiling the package.
Sharing Defaults
================
If you want to set default values for `configure' scripts to share,
you can create a site shell script called `config.site' that gives
default values for variables like `CC', `cache_file', and `prefix'.
`configure' looks for `PREFIX/share/config.site' if it exists, then
`PREFIX/etc/config.site' if it exists. Or, you can set the
`CONFIG_SITE' environment variable to the location of the site script.
A warning: not all `configure' scripts look for a site script.
Operation Controls
==================
`configure' recognizes the following options to control how it
operates.
`--cache-file=FILE'
Use and save the results of the tests in FILE instead of
`./config.cache'. Set FILE to `/dev/null' to disable caching, for
debugging `configure'.
`--help'
Print a summary of the options to `configure', and exit.
`--quiet'
`--silent'
`-q'
Do not print messages saying which checks are being made.
`--srcdir=DIR'
Look for the package's source code in directory DIR. Usually
`configure' can determine that directory automatically.
`--version'
Print the version of Autoconf used to generate the `configure'
script, and exit.
`configure' also accepts some other, not widely useful, options.

View File

@@ -1,7 +1,7 @@
#
# This file is part of nzbget
# This file if part of nzbget
#
# Copyright (C) 2008-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
# Copyright (C) 2008-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -28,20 +28,6 @@ nzbget_SOURCES = \
daemon/connect/TLS.h \
daemon/connect/WebDownloader.cpp \
daemon/connect/WebDownloader.h \
daemon/extension/FeedScript.cpp \
daemon/extension/FeedScript.h \
daemon/extension/NzbScript.cpp \
daemon/extension/NzbScript.h \
daemon/extension/PostScript.cpp \
daemon/extension/PostScript.h \
daemon/extension/QueueScript.cpp \
daemon/extension/QueueScript.h \
daemon/extension/ScanScript.cpp \
daemon/extension/ScanScript.h \
daemon/extension/SchedulerScript.cpp \
daemon/extension/SchedulerScript.h \
daemon/extension/ScriptConfig.cpp \
daemon/extension/ScriptConfig.h \
daemon/feed/FeedCoordinator.cpp \
daemon/feed/FeedCoordinator.h \
daemon/feed/FeedFile.cpp \
@@ -58,10 +44,6 @@ nzbget_SOURCES = \
daemon/frontend/LoggableFrontend.h \
daemon/frontend/NCursesFrontend.cpp \
daemon/frontend/NCursesFrontend.h \
daemon/main/CommandLineParser.cpp \
daemon/main/CommandLineParser.h \
daemon/main/DiskService.cpp \
daemon/main/DiskService.h \
daemon/main/Maintenance.cpp \
daemon/main/Maintenance.h \
daemon/main/nzbget.cpp \
@@ -70,12 +52,8 @@ nzbget_SOURCES = \
daemon/main/Options.h \
daemon/main/Scheduler.cpp \
daemon/main/Scheduler.h \
daemon/main/StackTrace.cpp \
daemon/main/StackTrace.h \
daemon/nntp/ArticleDownloader.cpp \
daemon/nntp/ArticleDownloader.h \
daemon/nntp/ArticleWriter.cpp \
daemon/nntp/ArticleWriter.h \
daemon/nntp/Decoder.cpp \
daemon/nntp/Decoder.h \
daemon/nntp/NewsServer.cpp \
@@ -86,18 +64,14 @@ nzbget_SOURCES = \
daemon/nntp/ServerPool.h \
daemon/nntp/StatMeter.cpp \
daemon/nntp/StatMeter.h \
daemon/postprocess/Cleanup.cpp \
daemon/postprocess/Cleanup.h \
daemon/postprocess/DupeMatcher.cpp \
daemon/postprocess/DupeMatcher.h \
daemon/postprocess/ParChecker.cpp \
daemon/postprocess/ParChecker.h \
daemon/postprocess/ParCoordinator.cpp \
daemon/postprocess/ParCoordinator.h \
daemon/postprocess/ParParser.cpp \
daemon/postprocess/ParParser.h \
daemon/postprocess/ParRenamer.cpp \
daemon/postprocess/ParRenamer.h \
daemon/postprocess/PostScript.cpp \
daemon/postprocess/PostScript.h \
daemon/postprocess/PrePostProcessor.cpp \
daemon/postprocess/PrePostProcessor.h \
daemon/postprocess/Unpack.cpp \
@@ -116,6 +90,8 @@ nzbget_SOURCES = \
daemon/queue/QueueCoordinator.h \
daemon/queue/QueueEditor.cpp \
daemon/queue/QueueEditor.h \
daemon/queue/QueueScript.cpp \
daemon/queue/QueueScript.h \
daemon/queue/Scanner.cpp \
daemon/queue/Scanner.h \
daemon/queue/UrlCoordinator.cpp \
@@ -139,61 +115,12 @@ nzbget_SOURCES = \
daemon/util/Script.h \
daemon/util/Thread.cpp \
daemon/util/Thread.h \
daemon/util/Service.cpp \
daemon/util/Service.h \
daemon/util/Util.cpp \
daemon/util/Util.h \
code_revision.cpp
if WITH_PAR2
nzbget_SOURCES += \
lib/par2/commandline.cpp \
lib/par2/commandline.h \
lib/par2/crc.cpp \
lib/par2/crc.h \
lib/par2/creatorpacket.cpp \
lib/par2/creatorpacket.h \
lib/par2/criticalpacket.cpp \
lib/par2/criticalpacket.h \
lib/par2/datablock.cpp \
lib/par2/datablock.h \
lib/par2/descriptionpacket.cpp \
lib/par2/descriptionpacket.h \
lib/par2/diskfile.cpp \
lib/par2/diskfile.h \
lib/par2/filechecksummer.cpp \
lib/par2/filechecksummer.h \
lib/par2/galois.cpp \
lib/par2/galois.h \
lib/par2/letype.h \
lib/par2/mainpacket.cpp \
lib/par2/mainpacket.h \
lib/par2/md5.cpp \
lib/par2/md5.h \
lib/par2/par2cmdline.h \
lib/par2/par2creatorsourcefile.cpp \
lib/par2/par2creatorsourcefile.h \
lib/par2/par2fileformat.cpp \
lib/par2/par2fileformat.h \
lib/par2/par2repairer.cpp \
lib/par2/par2repairer.h \
lib/par2/par2repairersourcefile.cpp \
lib/par2/par2repairersourcefile.h \
lib/par2/parheaders.cpp \
lib/par2/parheaders.h \
lib/par2/recoverypacket.cpp \
lib/par2/recoverypacket.h \
lib/par2/reedsolomon.cpp \
lib/par2/reedsolomon.h \
lib/par2/verificationhashtable.cpp \
lib/par2/verificationhashtable.h \
lib/par2/verificationpacket.cpp \
lib/par2/verificationpacket.h
endif
svn_version.cpp
AM_CPPFLAGS = \
-I$(srcdir)/daemon/connect \
-I$(srcdir)/daemon/extension \
-I$(srcdir)/daemon/feed \
-I$(srcdir)/daemon/frontend \
-I$(srcdir)/daemon/main \
@@ -201,55 +128,21 @@ AM_CPPFLAGS = \
-I$(srcdir)/daemon/postprocess \
-I$(srcdir)/daemon/queue \
-I$(srcdir)/daemon/remote \
-I$(srcdir)/daemon/util \
-I$(srcdir)/lib/par2
if WITH_TESTS
nzbget_SOURCES += \
lib/catch/catch.h \
tests/suite/TestMain.cpp \
tests/suite/TestMain.h \
tests/suite/TestUtil.cpp \
tests/suite/TestUtil.h \
tests/main/CommandLineParserTest.cpp \
tests/main/OptionsTest.cpp \
tests/feed/FeedFilterTest.cpp \
tests/postprocess/ParCheckerTest.cpp \
tests/postprocess/ParRenamerTest.cpp \
tests/queue/NZBFileTest.cpp \
tests/util/UtilTest.cpp
AM_CPPFLAGS += \
-I$(srcdir)/lib/catch \
-I$(srcdir)/tests/suite
endif
-I$(srcdir)/daemon/util
EXTRA_DIST = \
Makefile.cvs \
nzbgetd \
$(windows_FILES) \
$(osx_FILES) \
$(linux_FILES) \
$(testdata_FILES)
$(osx_FILES)
windows_FILES = \
daemon/windows/NTService.cpp \
daemon/windows/NTService.h \
daemon/windows/win32.h \
daemon/windows/WinConsole.cpp \
daemon/windows/WinConsole.h \
nzbget.sln \
nzbget.vcproj \
windows/nzbget-command-shell.bat \
windows/install-update.bat \
windows/README-WINDOWS.txt \
windows/package-info.json \
windows/resources/mainicon.ico \
windows/resources/nzbget.rc \
windows/resources/resource.h \
windows/resources/trayicon_idle.ico \
windows/resources/trayicon_paused.ico \
windows/resources/trayicon_working.ico \
windows/setup/nzbget-setup.nsi \
windows/setup/install.bmp \
windows/setup/uninstall.bmp
nzbget-shell.bat
osx_FILES = \
osx/App_Prefix.pch \
@@ -275,6 +168,8 @@ osx_FILES = \
osx/Resources/Images/mainicon.icns \
osx/Resources/Images/statusicon.png \
osx/Resources/Images/statusicon@2x.png \
osx/Resources/Images/statusicon-inv.png \
osx/Resources/Images/statusicon-inv@2x.png \
osx/Resources/licenses/license-bootstrap.txt \
osx/Resources/licenses/license-jquery-GPL.txt \
osx/Resources/licenses/license-jquery-MIT.txt \
@@ -282,17 +177,7 @@ osx_FILES = \
osx/Resources/Localizable.strings \
osx/Resources/Welcome.rtf
linux_FILES = \
linux/installer.sh \
linux/install-update.sh \
linux/package-info.json \
linux/build-info.txt \
linux/build-nzbget \
linux/build-unpack
doc_FILES = \
lib/par2/AUTHORS \
lib/par2/README \
README \
ChangeLog \
COPYING
@@ -336,24 +221,8 @@ scripts_FILES = \
scripts/EMail.py \
scripts/Logger.py
testdata_FILES = \
tests/testdata/dupematcher1/testfile.part01.rar \
tests/testdata/dupematcher1/testfile.part24.rar \
tests/testdata/dupematcher2/testfile.part04.rar \
tests/testdata/dupematcher2/testfile.part43.rar \
tests/testdata/nzbfile/dotless.nzb \
tests/testdata/nzbfile/dotless.txt \
tests/testdata/nzbfile/plain.nzb \
tests/testdata/nzbfile/plain.txt \
tests/testdata/parchecker/crc.txt \
tests/testdata/parchecker/testfile.dat \
tests/testdata/parchecker/testfile.nfo \
tests/testdata/parchecker/testfile.par2 \
tests/testdata/parchecker/testfile.vol00+1.PAR2 \
tests/testdata/parchecker/testfile.vol01+2.PAR2 \
tests/testdata/parchecker/testfile.vol03+3.PAR2
# Install
sbin_SCRIPTS = nzbgetd
dist_doc_DATA = $(doc_FILES)
exampleconfdir = $(datadir)/nzbget
dist_exampleconf_DATA = $(exampleconf_FILES)
@@ -371,6 +240,13 @@ nobase_dist_scripts_SCRIPTS = $(scripts_FILES)
# 3) delete original.temp
# These steps ensure that the output file has the same permissions as the original file.
# Configure installed script
install-exec-hook:
rm -f "$(DESTDIR)$(sbindir)/nzbgetd.temp"
cp "$(DESTDIR)$(sbindir)/nzbgetd" "$(DESTDIR)$(sbindir)/nzbgetd.temp"
sed 's?/usr/local/bin?$(bindir)?' < "$(DESTDIR)$(sbindir)/nzbgetd.temp" > "$(DESTDIR)$(sbindir)/nzbgetd"
rm "$(DESTDIR)$(sbindir)/nzbgetd.temp"
# Prepare example configuration file
install-data-hook:
rm -f "$(DESTDIR)$(exampleconfdir)/nzbget.conf.temp"
@@ -393,57 +269,44 @@ install-conf:
uninstall-conf:
rm -f "$(DESTDIR)$(sysconfdir)/nzbget.conf"
# Determining git revision:
# 1) If directory ".git" exists we take revision from git log.
# Determining subversion revision:
# 1) If directory ".svn" exists we take revision from it using program svnversion (part of subversion package)
# File is recreated only if revision number was changed.
# 2) If directory ".git" doesn't exists we keep and reuse file "code_revision.cpp",
# 2) If directory ".svn" doesn't exists we keep and reuse file "svn_version.cpp",
# which was possibly created early.
# 3) If neither directory ".git" nor file "code_revision.cpp" are available
# we create new file "code_revision.c" with empty revision number.
code_revision.cpp: FORCE
@ if test -d ./.git ; then \
B="$(shell git branch | sed -n -e 's/^\* \(.*\)/\1/p')"; \
M="$(shell git status --porcelain)" ; \
if test "$$M" != "" ; then \
M="M" ; \
fi ; \
if test "$$B" = "master" ; then \
V="$$M" ; \
elif test "$$B" = "develop" ; then \
V="$(shell git rev-list HEAD | wc -l | xargs)" ; \
V="$${V}$$M" ; \
else \
V="$(shell git rev-list HEAD | wc -l | xargs)" ; \
V="$${V}$$M ($$B)" ; \
fi ; \
H="$(shell test -f ./code_revision.cpp && head -n 1 code_revision.cpp)"; \
# 3) If neither directory ".svn" nor file "svn_version.cpp" are available
# we create new file "svn_version.c" with empty revision number.
svn_version.cpp: FORCE
@ if test -d ./.svn ; then \
V="$(shell svnversion -n .)"; \
H="$(shell test -f ./svn_version.cpp && head -n 1 svn_version.cpp)"; \
if test "/* $$V */" != "$$H" ; then \
( \
echo "/* $$V */" ;\
echo "/* This file is automatically regenerated on each build. Do not edit it. */" ;\
echo "const char* code_revision(void)" ;\
echo "const char* svn_version(void)" ;\
echo "{" ;\
echo " const char* revision = \"$$V\";" ;\
echo " return revision;" ;\
echo " const char* SVN_Version = \"$$V\";" ;\
echo " return SVN_Version;" ;\
echo "}" ;\
) > code_revision.cpp ; \
) > svn_version.cpp ; \
fi \
elif test -f ./code_revision.cpp ; then \
elif test -f ./svn_version.cpp ; then \
test "ok, reuse existing file"; \
else \
( \
echo "/* */" ;\
echo "/* This file is automatically regenerated on each build. Do not edit it. */" ;\
echo "const char* code_revision(void)" ;\
echo "const char* svn_version(void)" ;\
echo "{" ;\
echo " const char* revision = \"\";" ;\
echo " return revision;" ;\
echo " const char* SVN_Version = \"\";" ;\
echo " return SVN_Version;" ;\
echo "}" ;\
) > code_revision.cpp ; \
) > svn_version.cpp ; \
fi
FORCE:
# Ignore "code_revision.cpp" in distcleancheck
# Ignore "svn_version.cpp" in distcleancheck
distcleancheck_listfiles = \
find . -type f -exec sh -c 'test -f $(srcdir)/$$1 || echo $$1' \
sh '{}' ';'
@@ -454,6 +317,4 @@ clean-bak: rm *~
dist-hook:
find $(distdir)/daemon -type f -print -exec chmod -x {} \;
find $(distdir)/webui -type f -print -exec chmod -x {} \;
find $(distdir)/lib -type f -print -exec chmod -x {} \;
find $(distdir)/tests -type f -print -exec chmod -x {} \;

8
Makefile.cvs Normal file
View File

@@ -0,0 +1,8 @@
default: all
all:
aclocal
autoheader
automake
autoconf

View File

File diff suppressed because it is too large Load Diff

0
NEWS Normal file
View File

126
README
View File

@@ -44,16 +44,25 @@ depends on command-line parameters passed to the program.
2. Supported OS
=====================================
NZBGet is written in C++ and works on Windows, OS X, Linux and
most POSIX-conform OS'es.
NZBGet is written in C++ and was initialy developed on Linux.
It was ported to Windows later and tested for compatibility with
several POSIX-OS'es.
It should run at least on:
- Linux Debian 5.0 on x86;
- Linux with uClibc on MIPSEL and ARM;
- OpenBSD 5.0 on x86;
- Mac OS X 10.7 Lion on x64;
- Windows XP SP3 on x86 and Windows 7 on x64.
Clients and servers running on different OS'es may communicate with
each other. For example, you can use NZBGet as client on Windows to
control your NZBGet-server running on Linux.
The download-section of NZBGet web-site provides binary files
for Windows, OS X and Linux. For most POSIX-systems you need to compile
the program yourself.
for Windows. The binary packages for many routers and NAS devices are
also available in OPTWARE repository (http://www.nslu2-linux.org),
but for most POSIX-systems you need to compile the program yourself.
If you have downloaded binaries you can just jump to section
"Configuration".
@@ -62,8 +71,8 @@ If you have downloaded binaries you can just jump to section
3. Prerequisites on POSIX
=====================================
NZBGet is developed on a linux-system, but it runs on other
POSIX platforms.
NZBGet is developed on a linux-system, but it should run on other
POSIX platforms (see the list of tested platforms above).
NZBGet absolutely needs the following libraries:
@@ -76,7 +85,12 @@ And the following libraries are optional:
- libcurses (usually part of commercial systems)
or (better)
- libncurses (http://invisible-island.net/ncurses)
- for par-check and -repair (enabled by default):
- libpar2 (http://launchpad.net/libpar2,
http://parchive.sourceforge.net)
- libsigc++ (http://libsigc.sourceforge.net)
- for encrypted connections (TLS/SSL):
- OpenSSL (http://www.openssl.org)
or
@@ -85,7 +99,7 @@ And the following libraries are optional:
- for gzip support in web-server and web-client (enabled by default):
- zlib (http://www.zlib.net)
All these libraries are included in modern POSIX distributions and
All these libraries are included in modern Linux distributions and
should be available as installable packages. Please note that you also
need the developer packages for these libraries too, they package names
have often suffix "dev" or "devel". On other systems you may need to
@@ -138,7 +152,7 @@ You may run configure with additional arguments:
if you can not use curses/ncurses.
--disable-parcheck - to make without parcheck-support. Use this option
if you have troubles when compiling par2-module.
if you can not use libpar2 or libsigc++.
--with-tlslib=(OpenSSL, GnuTLS) - to select which TLS/SSL library
should be used for encrypted server connections.
@@ -155,13 +169,34 @@ You may run configure with additional arguments:
Optional package: par-check
---------------------------
NZBGet can check and repair downloaded files for you. For this purpose
it uses library par2.
it uses library par2 (libpar2), which needs sigc++ on its part.
For your convenience the source code of libpar2 is integrated into
NZBGets source tree and is compiled automatically when you make NZBGet.
The libpar2 and libsigc++ (version 2 or later) must be installed on your
system. On most linux distributions these libraries are available as packages.
If you do not have these packages you can compile them yourself.
Following configure-parameters may be usefull:
In a case errors occur during this process the inclusion of par2-module
can be disabled using configure option "--disable-parcheck":
--with-libpar2-includes
--with-libpar2-libraries
--with-libsigc-includes
--with-libsigc-libraries
The library libsigc++ must be installed first, since libpar2 requires it.
Official project libpar2 (http://parchive.sourceforge.net) has not been
updated for many years. The last official version of libpar 0.2 contains
known bugs which may crash the program during par-check. These bugs
have been fixed in the semi-official fork maintained by Debian/Ubuntu
team (http://launchpad.net/libpar2). It is highly recommended to use this
version of libpar2 (version 0.4 or newer) instead of the last official
version 0.2. Recent releases of Debian/Ubuntu include this libpar2 version.
NZBGets configure script checks the installed version of libpar2 and prints
an error if it is older than 0.4. The check can be suppressed if the update
of libpar2 is not possible.
If you are not able to use libpar2 or libsigc++ or do not want them you can
make nzbget without support for par-check using option "--disable-parcheck":
./configure --disable-parcheck
@@ -169,10 +204,10 @@ Optional package: curses
-------------------------
For curses-outputmode you need ncurses or curses on your system.
If you do not have one of them you can download and compile ncurses yourself.
Following configure-parameters may be useful:
Following configure-parameters may be usefull:
--with-libcurses-includes=/path/to/curses/includes
--with-libcurses-libraries=/path/to/curses/libraries
--with-libcurses-includes
--with-libcurses-libraries
If you are not able to use curses or ncurses or do not want them you can
make the program without support for curses using option "--disable-curses":
@@ -191,11 +226,11 @@ the option --with-tlslib=(OpenSSL, GnuTLS). For example to build with GnuTLS:
Following configure-parameters may be useful:
--with-libtls-includess=/path/to/gnutls/includes
--with-libtls-libraries=/path/to/gnutls/libraries
--with-libtls-includes
--with-libtls-libraries
--with-openssl-includess=/path/to/openssl/includes
--with-openssl-libraries=/path/to/openssl/libraries
--with-openssl-includes
--with-openssl-libraries
If none of these libraries is available you can make the program without
TLS/SSL support using option "--disable-tls":
@@ -210,15 +245,24 @@ NZBGet is developed using MS Visual C++ 2005. The project file and solution
are provided. If you use MS Visual C++ 2005 Express you need to download
and install Platform SDK.
To compile the program with TLS/SSL support you need either OpenSSL or GnuTLS:
To compile the program with par-check-support you also need the following
libraries:
- libsigc++ (http://libsigc.sourceforge.net)
- libpar2 (http://launchpad.net/libpar2,
http://parchive.sourceforge.net)
Download these libaries, then use patch-files provided with NZBGet to create
preconfigured project files and solutions for each library.
Look at http://gnuwin32.sourceforge.net/packages/patch.htm for info on how
to use patch-files, if you do not familiar with this technique.
To compile the program with TLS/SSL support you also need the library:
- OpenSSL (http://www.openssl.org)
or
- GnuTLS (http://www.gnu.org/software/gnutls)
Also required are:
- Regex (http://gnuwin32.sourceforge.net/packages/regex.htm)
- Zlib (http://gnuwin32.sourceforge.net/packages/zlib.htm)
=====================================
6. Configuration
=====================================
@@ -244,7 +288,6 @@ The program looks for configuration file in following standard
locations (in this order):
On POSIX systems:
<EXE-DIR>/nzbget.conf
~/.nzbget
/etc/nzbget.conf
/usr/etc/nzbget.conf
@@ -454,35 +497,6 @@ Bo Cordes Petersen (placebodk@users.sourceforge.net) until 2005.
In 2007 the abandoned project was overtaken by Andrey Prygunkov.
Since then the program has been completely rewritten.
NZBGet distribution archive includes additional components
written by other authors:
Par2:
Peter Brian Clements <peterbclements@users.sourceforge.net>
Par2 library API:
Francois Lesueur <flesueur@users.sourceforge.net>
Catch:
Two Blue Cubes Ltd <https://github.com/philsquared/Catch>
jQuery:
John Resig <http://jquery.com>
The Dojo Foundation <http://sizzlejs.com>
Bootstrap:
Twitter, Inc <http://twitter.github.com/bootstrap>
Raphaël:
Dmitry Baranovskiy <http://raphaeljs.com>
Sencha Labs <http://sencha.com>
Elycharts:
Void Labs s.n.c. <http://void.it>
iconSweets:
Yummygum <http://yummygum.com>
=====================================
9. Copyright
=====================================

View File

@@ -1,15 +0,0 @@
# NZBGet #
NZBGet is a binary downloader, which downloads files from Usenet
based on information given in nzb-files.
NZBGet is written in C++ and is known for its extraordinary performance and efficiency.
NZBGet can be run at almost every platform - classic PCs, NAS, media players, SAT-receivers, WLAN-routers, etc.
The download area provides precompiled binaries
for Windows, Mac OS X and Linux (compatible with many CPUs and platform variants). For other platforms
the program can be compiled from sources.
- [Home page (nzbget.net)](http://nzbget.net) - for first time visitors, learn more about NZBGet;
- [Downloads](http://nzbget.net/download) - get the binaries and sources;
- [Documentation](https://github.com/nzbget/nzbget/wiki) - installation manuals, HOW-TOs, API;
- [Forum](http://forum.nzbget.net) - get support, share your ideas, scripts, add-ons.

1500
config.guess vendored Executable file
View File

File diff suppressed because it is too large Load Diff

View File

@@ -3,25 +3,18 @@
/* Define to 1 to include debug-code */
#undef DEBUG
/* Define to 1 if deleting of files during reading of directory is not
properly supported by OS */
#undef DIRBROWSER_SNAPSHOT
/* Define to 1 to not use curses */
#undef DISABLE_CURSES
/* Define to 1 to disable gzip-support */
#undef DISABLE_GZIP
/* Define to 1 to disable par-verification and repair */
/* Define to 1 to disable smart par-verification and restoration */
#undef DISABLE_PARCHECK
/* Define to 1 to not use TLS/SSL */
#undef DISABLE_TLS
/* Define to 1 to enable unit and integration tests */
#undef ENABLE_TESTS
/* Define to the name of macro which returns the name of function being
compiled */
#undef FUNCTION_MACRO_NAME
@@ -38,22 +31,6 @@
/* Define to 1 if you have the <curses.h> header file. */
#undef HAVE_CURSES_H
/* Define to 1 if you have the <dirent.h> header file, and it defines `DIR'.
*/
#undef HAVE_DIRENT_H
/* Define to 1 if you have the <endian.h> header file. */
#undef HAVE_ENDIAN_H
/* Define to 1 if fdatasync is supported */
#undef HAVE_FDATASYNC
/* Define to 1 if fseeko (and presumably ftello) exists and is declared. */
#undef HAVE_FSEEKO
/* Define to 1 if F_FULLFSYNC is supported */
#undef HAVE_FULLFSYNC
/* Define to 1 if getaddrinfo is supported */
#undef HAVE_GETADDRINFO
@@ -69,12 +46,6 @@
/* Define to 1 if gethostbyname_r takes 6 arguments */
#undef HAVE_GETHOSTBYNAME_R_6
/* Define to 1 if you have the `getopt' function. */
#undef HAVE_GETOPT
/* Define to 1 if you have the <getopt.h> header file. */
#undef HAVE_GETOPT_H
/* Define to 1 if getopt_long is supported */
#undef HAVE_GETOPT_LONG
@@ -84,9 +55,6 @@
/* Define to 1 to use GnuTLS library for TLS/SSL-support. */
#undef HAVE_LIBGNUTLS
/* Define to 1 if you have the `memcpy' function. */
#undef HAVE_MEMCPY
/* Define to 1 if you have the <memory.h> header file. */
#undef HAVE_MEMORY_H
@@ -96,53 +64,33 @@
/* Define to 1 if you have the <ncurses/ncurses.h> header file. */
#undef HAVE_NCURSES_NCURSES_H
/* Define to 1 if you have the <ndir.h> header file, and it defines `DIR'. */
#undef HAVE_NDIR_H
/* Define to 1 to use OpenSSL library for TLS/SSL-support. */
#undef HAVE_OPENSSL
/* Define to 1 if libpar2 has recent bugfixes-patch (version 2) */
#undef HAVE_PAR2_BUGFIXES_V2
/* Define to 1 if libpar2 supports cancelling (needs a special patch) */
#undef HAVE_PAR2_CANCEL
/* Define to 1 if you have the <regex.h> header file. */
#undef HAVE_REGEX_H
/* Define to 1 if _SC_NPROCESSORS_ONLN is present in unistd.h */
#undef HAVE_SC_NPROCESSORS_ONLN
/* Define to 1 if stdbool.h conforms to C99. */
#undef HAVE_STDBOOL_H
/* Define to 1 if spinlocks are supported */
#undef HAVE_SPINLOCK
/* Define to 1 if you have the <stdint.h> header file. */
#undef HAVE_STDINT_H
/* Define to 1 if you have the <stdio.h> header file. */
#undef HAVE_STDIO_H
/* Define to 1 if you have the <stdlib.h> header file. */
#undef HAVE_STDLIB_H
/* Define to 1 if you have the `strcasecmp' function. */
#undef HAVE_STRCASECMP
/* Define to 1 if you have the `strchr' function. */
#undef HAVE_STRCHR
/* Define to 1 if you have the `stricmp' function. */
#undef HAVE_STRICMP
/* Define to 1 if you have the <strings.h> header file. */
#undef HAVE_STRINGS_H
/* Define to 1 if you have the <string.h> header file. */
#undef HAVE_STRING_H
/* Define to 1 if you have the <sys/dir.h> header file, and it defines `DIR'.
*/
#undef HAVE_SYS_DIR_H
/* Define to 1 if you have the <sys/ndir.h> header file, and it defines `DIR'.
*/
#undef HAVE_SYS_NDIR_H
/* Define to 1 if you have the <sys/prctl.h> header file. */
#undef HAVE_SYS_PRCTL_H
@@ -158,12 +106,6 @@
/* Define to 1 if variadic macros are supported */
#undef HAVE_VARIADIC_MACROS
/* Define to 1 if the system has the type `_Bool'. */
#undef HAVE__BOOL
/* Define to 1 to exclude debug-code */
#undef NDEBUG
/* Name of package */
#undef PACKAGE
@@ -194,27 +136,8 @@
/* Version number of package */
#undef VERSION
/* Define to 1 if your processor stores words with the most significant byte
first (like Motorola and SPARC, unlike Intel and VAX). */
#undef WORDS_BIGENDIAN
/* Number of bits in a file offset, on hosts where this is settable. */
#undef _FILE_OFFSET_BITS
/* Define to 1 to make fseeko visible on some hosts (e.g. glibc 2.2). */
#undef _LARGEFILE_SOURCE
/* Define for large files, on AIX-style hosts. */
#undef _LARGE_FILES
/* Define to empty if `const' does not conform to ANSI C. */
#undef const
/* Define to `__inline__' or `__inline' if that's what the C compiler
calls it, or to nothing if 'inline' is not supported under any name. */
#ifndef __cplusplus
#undef inline
#endif
/* Define to `unsigned int' if <sys/types.h> does not define. */
#undef size_t

1616
config.sub vendored Executable file
View File

File diff suppressed because it is too large Load Diff

2885
configure vendored
View File

File diff suppressed because it is too large Load Diff

View File

@@ -1,31 +1,10 @@
#
# This file is part of nzbget
#
# Copyright (C) 2008-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
#
#
# -*- Autoconf -*-
# Process this file with autoconf to produce a configure script.
AC_PREREQ(2.59)
AC_INIT(nzbget, 16.3, hugbug@users.sourceforge.net)
AC_CONFIG_AUX_DIR(posix)
AM_INIT_AUTOMAKE([foreign])
AC_INIT(nzbget, 13.0, hugbug@users.sourceforge.net)
AC_CANONICAL_SYSTEM
AM_INIT_AUTOMAKE(nzbget, 13.0)
AC_CONFIG_SRCDIR([daemon/main/nzbget.cpp])
AC_CONFIG_HEADERS([config.h])
@@ -76,14 +55,6 @@ AC_CHECK_FUNC(getopt_long,
[AC_DEFINE([HAVE_GETOPT_LONG], 1, [Define to 1 if getopt_long is supported])],)
dnl
dnl fsync
dnl
AC_CHECK_FUNC(fdatasync,
[AC_DEFINE([HAVE_FDATASYNC], 1, [Define to 1 if fdatasync is supported])],)
AC_CHECK_DECL(F_FULLFSYNC,
[AC_DEFINE([HAVE_FULLFSYNC], 1, [Define to 1 if F_FULLFSYNC is supported])],,[#include <fcntl.h>])
dnl
dnl use 64-Bits for file sizes
dnl
@@ -171,6 +142,14 @@ if test "$FOUND" = "no"; then
fi
dnl
dnl cCheck if spinlocks are available
dnl
AC_CHECK_FUNC(pthread_spin_init,
[AC_DEFINE([HAVE_SPINLOCK], 1, [Define to 1 if spinlocks are supported])]
AC_SEARCH_LIBS([pthread_spin_init], [pthread]),)
dnl
dnl Determine what socket length (socklen_t) data type is
dnl
@@ -201,31 +180,6 @@ AC_TRY_COMPILE([
AC_DEFINE_UNQUOTED(SOCKLEN_T, $SOCKLEN_T, [Determine what socket length (socklen_t) data type is])
dnl
dnl Dir-browser's snapshot
dnl
AC_MSG_CHECKING(whether dir-browser snapshot workaround is needed)
if test "$target_vendor" == "apple"; then
AC_MSG_RESULT([[yes]])
AC_DEFINE([DIRBROWSER_SNAPSHOT], 1, [Define to 1 if deleting of files during reading of directory is not properly supported by OS])
else
AC_MSG_RESULT([[no]])
fi
dnl
dnl check cpu cores via sysconf
dnl
AC_MSG_CHECKING(for cpu cores via sysconf)
AC_TRY_COMPILE(
[#include <unistd.h>],
[ int a = _SC_NPROCESSORS_ONLN; ],
FOUND="yes"
AC_MSG_RESULT([[yes]])
AC_DEFINE([HAVE_SC_NPROCESSORS_ONLN], 1, [Define to 1 if _SC_NPROCESSORS_ONLN is present in unistd.h]),
FOUND="no")
dnl
dnl checks for libxml2 includes and libraries.
dnl
@@ -242,8 +196,7 @@ AC_ARG_WITH(libxml2_libraries,
if test "$INCVAL" = "no" -o "$LIBVAL" = "no"; then
PKG_CHECK_MODULES(libxml2, libxml-2.0,
[LIBS="${LIBS} $libxml2_LIBS"]
[CPPFLAGS="${CPPFLAGS} $libxml2_CFLAGS"],
AC_MSG_ERROR("libxml2 library not found"))
[CPPFLAGS="${CPPFLAGS} $libxml2_CFLAGS"])
fi
AC_CHECK_HEADER(libxml/tree.h,,
AC_MSG_ERROR("libxml2 header files not found"))
@@ -299,37 +252,116 @@ fi
dnl
dnl Use par-checking. Deafult: yes.
dnl Use libpar2 for par-checking. Deafult: no
dnl
AC_MSG_CHECKING(whether to include code for par-checking)
AC_ARG_ENABLE(parcheck,
[AS_HELP_STRING([--disable-parcheck], [do not include par-check/-repair-support])],
[AS_HELP_STRING([--disable-parcheck], [do not include par-check/-repair-support (removes dependency from libpar2- and libsigc-libraries)])],
[ ENABLEPARCHECK=$enableval ],
[ ENABLEPARCHECK=yes] )
AC_MSG_RESULT($ENABLEPARCHECK)
if test "$ENABLEPARCHECK" = "yes"; then
dnl PAR2 checks.
dnl
dnl Checks for header files.
AC_HEADER_DIRENT
AC_HEADER_STDBOOL
AC_HEADER_STDC
AC_CHECK_HEADERS([stdio.h] [endian.h] [getopt.h])
dnl Checks for typedefs, structures, and compiler characteristics.
AC_TYPE_SIZE_T
AC_C_BIGENDIAN
AC_C_CONST
AC_C_INLINE
AC_FUNC_FSEEKO
dnl Checks for library functions.
AC_FUNC_MEMCMP
AC_CHECK_FUNCS([stricmp] [strcasecmp])
AC_CHECK_FUNCS([strchr] [memcpy])
AC_CHECK_FUNCS([getopt])
AM_CONDITIONAL(WITH_PAR2, true)
dnl checks for libsigc++ includes and libraries (required for libpar2).
dnl
AC_ARG_WITH(libsigc_includes,
[AS_HELP_STRING([--with-libsigc-includes=DIR], [libsigc++-2.0 include directory])],
[CPPFLAGS="${CPPFLAGS} -I${withval}"]
[INCVAL="yes"],
[INCVAL="no"])
AC_ARG_WITH(libsigc_libraries,
[AS_HELP_STRING([--with-libsigc-libraries=DIR], [libsigc++-2.0 library directory])],
[LDFLAGS="${LDFLAGS} -L${withval}"]
[CPPFLAGS="${CPPFLAGS} -I${withval}/sigc++-2.0/include"]
[LIBVAL="yes"],
[LIBVAL="no"])
if test "$INCVAL" = "no" -o "$LIBVAL" = "no"; then
PKG_CHECK_MODULES(libsigc, sigc++-2.0,
[LIBS="${LIBS} $libsigc_LIBS"]
[CPPFLAGS="${CPPFLAGS} $libsigc_CFLAGS"])
fi
AC_CHECK_HEADER(sigc++/type_traits.h,,
AC_MSG_ERROR("libsigc++-2.0 header files not found"))
dnl
dnl checks for libpar2 includes and libraries.
dnl
INCVAL="${LIBPREF}/include"
LIBVAL="${LIBPREF}/lib"
AC_ARG_WITH(libpar2_includes,
[AS_HELP_STRING([--with-libpar2-includes=DIR], [libpar2 include directory])],
[INCVAL="$withval"])
CPPFLAGS="${CPPFLAGS} -I${INCVAL}"
AC_CHECK_HEADER(libpar2/libpar2.h,,
AC_MSG_ERROR("libpar2 header files not found"))
AC_ARG_WITH(libpar2_libraries,
[AS_HELP_STRING([--with-libpar2-libraries=DIR], [libpar2 library directory])],
[LIBVAL="$withval"])
LDFLAGS="${LDFLAGS} -L${LIBVAL}"
AC_SEARCH_LIBS([_ZN12Par2RepairerC1Ev], [par2], ,
AC_MSG_ERROR("libpar2 library not found"))
dnl
dnl check if libpar2 library is linkable
dnl
AC_MSG_CHECKING(for libpar2 linking)
AC_TRY_LINK(
[#include <libpar2/par2cmdline.h>]
[#include <libpar2/par2repairer.h>]
[ class Repairer : public Par2Repairer { }; ],
[ Repairer* p = new Repairer(); ],
AC_MSG_RESULT([[yes]]),
AC_MSG_RESULT([[no]])
AC_MSG_ERROR("libpar2 library not found"))
dnl
dnl check if libpar2 has support for cancelling
dnl
AC_MSG_CHECKING(whether libpar2 supports cancelling)
AC_TRY_COMPILE(
[#include <libpar2/par2cmdline.h>]
[#include <libpar2/par2repairer.h>]
[ class Repairer : public Par2Repairer { void test() { cancelled = true; } }; ],
[],
AC_MSG_RESULT([[yes]])
AC_DEFINE([HAVE_PAR2_CANCEL], 1, [Define to 1 if libpar2 supports cancelling (needs a special patch)]),
AC_MSG_RESULT([[no]]))
dnl
dnl check if libpar2 has recent bugfixes-patch
dnl
AC_MSG_CHECKING(whether libpar2 has recent bugfixes)
AC_TRY_COMPILE(
[#include <libpar2/par2cmdline.h>]
[#include <libpar2/par2repairer.h>]
[ class Repairer : public Par2Repairer { void test() { BugfixesPatchVersion2(); } }; ],
[],
AC_MSG_RESULT([[yes]])
PAR2PATCHV2=yes
AC_DEFINE([HAVE_PAR2_BUGFIXES_V2], 1, [Define to 1 if libpar2 has recent bugfixes-patch (version 2)]),
AC_MSG_RESULT([[no]])
PAR2PATCHV2=no)
if test "$PAR2PATCHV2" = "no" ; then
AC_ARG_ENABLE(libpar2-bugfixes-check,
[AS_HELP_STRING([--disable-libpar2-bugfixes-check], [do not check libpar2 version])],
[ PAR2PATCHCHECK=$enableval ],
[ PAR2PATCHCHECK=yes] )
if test "$PAR2PATCHCHECK" = "yes"; then
AC_ERROR([Your version of libpar2 doesn't include the recent bugfixes. Please update libpar2 to version 0.4 or newer (http://launchpad.net/libpar2). If you cannot install a newer version of libpar2, you can use configure parameter --disable-libpar2-bugfixes-check to suppress the check. Please note however that in this case the program may crash during par-check/repair. The update is highly recommended!])
fi
fi
else
AC_DEFINE([DISABLE_PARCHECK],1,[Define to 1 to disable par-verification and repair])
AM_CONDITIONAL(WITH_PAR2, false)
AC_DEFINE([DISABLE_PARCHECK],1,[Define to 1 to disable smart par-verification and restoration])
fi
@@ -362,9 +394,10 @@ if test "$USETLS" = "yes"; then
[LIBVAL="yes"],
[LIBVAL="no"])
if test "$INCVAL" = "no" -o "$LIBVAL" = "no"; then
PKG_CHECK_MODULES([openssl], [openssl],
PKG_CHECK_MODULES(openssl, openssl,
[LIBS="${LIBS} $openssl_LIBS"]
[CPPFLAGS="${CPPFLAGS} $openssl_CFLAGS"],
FOUND=yes
FOUND=no)
fi
@@ -412,29 +445,10 @@ if test "$USETLS" = "yes"; then
fi
if test "$FOUND" = "yes"; then
AC_SEARCH_LIBS([gnutls_global_init], [gnutls],
FOUND=yes,
AC_SEARCH_LIBS([gcry_control], [gnutls gcrypt],
FOUND=yes,
FOUND=no),
FOUND=no)
if test "$FOUND" = "yes"; then
dnl gcrypt is optional
AC_MSG_CHECKING([whether gcrypt is needed])
AC_TRY_COMPILE(
[#include <gnutls/gnutls.h>]
[#if GNUTLS_VERSION_NUMBER <= 0x020b00]
[compile error]
[#endif],
[int a;],
AC_MSG_RESULT([no])
GCRYPT=no,
AC_MSG_RESULT([yes])
GCRYPT=yes)
if test "$GCRYPT" = "yes"; then
AC_CHECK_HEADER([gcrypt.h],
AC_SEARCH_LIBS([gcry_control], [gnutls gcrypt],
FOUND=yes,
FOUND=no),
FOUND=yes)
fi
fi
if test "$FOUND" = "no" -a "$TLSLIB" = "GnuTLS"; then
AC_MSG_ERROR([Couldn't find GnuTLS library])
fi
@@ -524,6 +538,16 @@ dnl
AC_DEFINE([DEBUG],1,Define to 1 to include debug-code)
dnl
dnl Set debug flags for gcc (if gcc is used)
dnl
if test "$CC" = "gcc"; then
CXXFLAGS="-g -Wall"
else
CXXFLAGS=""
fi
dnl
dnl check for __FUNCTION__ or __func__ macro
dnl
@@ -587,26 +611,8 @@ AC_MSG_CHECKING(for rdynamic linker flag)
dnl
dnl End of debugging code
dnl
else
AC_DEFINE([NDEBUG],1,Define to 1 to exclude debug-code)
fi
dnl
dnl Enable test suite. Deafult: no.
dnl
AC_MSG_CHECKING(whether to enable unit and integration tests)
AC_ARG_ENABLE(tests,
[AS_HELP_STRING([--enable-tests], [enable unit and integration tests])],
[ ENABLETESTS=$enableval ],
[ ENABLETESTS=no] )
AC_MSG_RESULT($ENABLETESTS)
if test "$ENABLETESTS" = "yes"; then
AC_DEFINE([ENABLE_TESTS],1,[Define to 1 to enable unit and integration tests])
AM_CONDITIONAL(WITH_TESTS, true)
else
AM_CONDITIONAL(WITH_TESTS, false)
fi
AC_CONFIG_FILES([Makefile])
AC_OUTPUT

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -49,12 +49,8 @@
#include <sys/time.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <fcntl.h>
#endif
#include <vector>
#include <algorithm>
#include "nzbget.h"
#include "Connection.h"
#include "Log.h"
@@ -66,43 +62,6 @@ Mutex* Connection::m_pMutexGetHostByName = NULL;
#endif
#endif
void closesocket_gracefully(SOCKET iSocket)
{
char buf[1024];
struct linger linger;
// Set linger option to avoid socket hanging out after close. This prevent
// ephemeral port exhaust problem under high QPS.
linger.l_onoff = 1;
linger.l_linger = 1;
setsockopt(iSocket, SOL_SOCKET, SO_LINGER, (char *) &linger, sizeof(linger));
// Send FIN to the client
shutdown(iSocket, SHUT_WR);
// Set non-blocking mode
#ifdef WIN32
unsigned long on = 1;
ioctlsocket(iSocket, FIONBIO, &on);
#else
int flags;
flags = fcntl(iSocket, F_GETFL, 0);
fcntl(iSocket, F_SETFL, flags | O_NONBLOCK);
#endif
// Read and discard pending incoming data. If we do not do that and close the
// socket, the data in the send buffer may be discarded. This
// behaviour is seen on Windows, when client keeps sending data
// when server decides to close the connection; then when client
// does recv() it gets no data back.
int n;
do {
n = recv(iSocket, buf, sizeof(buf), 0);
} while (n > 0);
// Now we know that our FIN is ACK-ed, safe to close
closesocket(iSocket);
}
void Connection::Init()
{
@@ -169,8 +128,6 @@ Connection::Connection(const char* szHost, int iPort, bool bTLS)
m_bSuppressErrors = true;
m_szReadBuf = (char*)malloc(CONNECTION_READBUFFER_SIZE + 1);
m_iTotalBytesRead = 0;
m_bBroken = false;
m_bGracefull = false;
#ifndef DISABLE_TLS
m_pTLSSocket = NULL;
m_bTLSError = false;
@@ -297,18 +254,14 @@ bool Connection::Bind()
int res = getaddrinfo(m_szHost, iPortStr, &addr_hints, &addr_list);
if (res != 0)
{
ReportError("Could not resolve hostname %s", m_szHost, false, 0);
error("Could not resolve hostname %s", m_szHost);
return false;
}
m_bBroken = false;
m_iSocket = INVALID_SOCKET;
for (addr = addr_list; addr != NULL; addr = addr->ai_next)
{
m_iSocket = socket(addr->ai_family, addr->ai_socktype, addr->ai_protocol);
#ifdef WIN32
SetHandleInformation((HANDLE)m_iSocket, HANDLE_FLAG_INHERIT, 0);
#endif
if (m_iSocket != INVALID_SOCKET)
{
int opt = 1;
@@ -392,10 +345,6 @@ int Connection::WriteLine(const char* pBuffer)
}
int iRes = send(m_iSocket, pBuffer, strlen(pBuffer), 0);
if (iRes <= 0)
{
m_bBroken = true;
}
return iRes;
}
@@ -415,7 +364,6 @@ bool Connection::Send(const char* pBuffer, int iSize)
int iRes = send(m_iSocket, pBuffer + iBytesSent, iSize-iBytesSent, 0);
if (iRes <= 0)
{
m_bBroken = true;
return false;
}
iBytesSent += iRes;
@@ -444,7 +392,6 @@ char* Connection::ReadLine(char* pBuffer, int iSize, int* pBytesRead)
if (iBufAvail < 0)
{
ReportError("Could not receive data on socket", NULL, true, 0);
m_bBroken = true;
break;
}
else if (iBufAvail == 0)
@@ -583,7 +530,6 @@ bool Connection::DoConnect()
debug("Do connecting");
m_iSocket = INVALID_SOCKET;
m_bBroken = false;
#ifdef HAVE_GETADDRINFO
struct addrinfo addr_hints, *addr_list, *addr;
@@ -602,49 +548,32 @@ bool Connection::DoConnect()
return false;
}
std::vector<SockAddr> triedAddr;
bool bConnected = false;
for (addr = addr_list; addr != NULL; addr = addr->ai_next)
{
// don't try the same combinations of ai_family, ai_socktype, ai_protocol multiple times
SockAddr sa = { addr->ai_family, addr->ai_socktype, addr->ai_protocol };
if (std::find(triedAddr.begin(), triedAddr.end(), sa) != triedAddr.end())
{
continue;
}
triedAddr.push_back(sa);
bool bLastAddr = !addr->ai_next;
m_iSocket = socket(addr->ai_family, addr->ai_socktype, addr->ai_protocol);
#ifdef WIN32
SetHandleInformation((HANDLE)m_iSocket, HANDLE_FLAG_INHERIT, 0);
#endif
if (m_iSocket == INVALID_SOCKET)
if (m_iSocket != INVALID_SOCKET)
{
// try another addr/family/protocol
continue;
res = connect(m_iSocket , addr->ai_addr, addr->ai_addrlen);
if (res != -1)
{
// Connection established
break;
}
// Connection failed
if (bLastAddr)
{
ReportError("Connection to %s failed", m_szHost, true, 0);
}
closesocket(m_iSocket);
m_iSocket = INVALID_SOCKET;
}
if (ConnectWithTimeout(addr->ai_addr, addr->ai_addrlen))
else if (bLastAddr)
{
// Connection established
bConnected = true;
break;
ReportError("Socket creation failed for %s", m_szHost, true, 0);
}
}
if (m_iSocket == INVALID_SOCKET && addr_list)
{
ReportError("Socket creation failed for %s", m_szHost, true, 0);
}
if (!bConnected && m_iSocket != INVALID_SOCKET)
{
ReportError("Connection to %s failed", m_szHost, true, 0);
closesocket(m_iSocket);
m_iSocket = INVALID_SOCKET;
}
freeaddrinfo(addr_list);
if (m_iSocket == INVALID_SOCKET)
@@ -671,7 +600,8 @@ bool Connection::DoConnect()
return false;
}
if (!ConnectWithTimeout(&sSocketAddress, sizeof(sSocketAddress)))
int res = connect(m_iSocket , (struct sockaddr *) & sSocketAddress, sizeof(sSocketAddress));
if (res == -1)
{
ReportError("Connection to %s failed", m_szHost, true, 0);
closesocket(m_iSocket);
@@ -680,153 +610,22 @@ bool Connection::DoConnect()
}
#endif
if (!InitSocketOpts())
{
return false;
}
#ifndef DISABLE_TLS
if (m_bTLS && !StartTLS(true, NULL, NULL))
{
return false;
}
#endif
return true;
}
bool Connection::InitSocketOpts()
{
char* optbuf = NULL;
int optsize = 0;
#ifdef WIN32
int MSecVal = m_iTimeout * 1000;
optbuf = (char*)&MSecVal;
optsize = sizeof(MSecVal);
int err = setsockopt(m_iSocket, SOL_SOCKET, SO_RCVTIMEO, (char*)&MSecVal, sizeof(MSecVal));
#else
struct timeval TimeVal;
TimeVal.tv_sec = m_iTimeout;
TimeVal.tv_usec = 0;
optbuf = (char*)&TimeVal;
optsize = sizeof(TimeVal);
int err = setsockopt(m_iSocket, SOL_SOCKET, SO_RCVTIMEO, (char*)&TimeVal, sizeof(TimeVal));
#endif
int err = setsockopt(m_iSocket, SOL_SOCKET, SO_RCVTIMEO, optbuf, optsize);
if (err != 0)
{
ReportError("Socket initialization failed for %s", m_szHost, true, 0);
return false;
}
err = setsockopt(m_iSocket, SOL_SOCKET, SO_SNDTIMEO, optbuf, optsize);
if (err != 0)
{
ReportError("Socket initialization failed for %s", m_szHost, true, 0);
return false;
}
return true;
}
bool Connection::ConnectWithTimeout(void* address, int address_len)
{
int flags = 0, error = 0, ret = 0;
fd_set rset, wset;
socklen_t len = sizeof(error);
struct timeval ts;
ts.tv_sec = m_iTimeout;
ts.tv_usec = 0;
//clear out descriptor sets for select
//add socket to the descriptor sets
FD_ZERO(&rset);
FD_SET(m_iSocket, &rset);
wset = rset; //structure assignment ok
//set socket nonblocking flag
#ifdef WIN32
u_long mode = 1;
if (ioctlsocket(m_iSocket, FIONBIO, &mode) != 0)
{
return false;
}
#else
flags = fcntl(m_iSocket, F_GETFL, 0);
if (flags < 0)
{
return false;
}
if (fcntl(m_iSocket, F_SETFL, flags | O_NONBLOCK) < 0)
{
return false;
}
#endif
//initiate non-blocking connect
ret = connect(m_iSocket, (struct sockaddr*)address, address_len);
if (ret < 0)
{
#ifdef WIN32
int err = WSAGetLastError();
if (err != WSAEWOULDBLOCK)
{
return false;
}
#else
if (errno != EINPROGRESS)
{
return false;
}
#endif
}
//connect succeeded right away?
if (ret != 0)
{
ret = select(m_iSocket + 1, &rset, &wset, NULL, m_iTimeout ? &ts : NULL);
//we are waiting for connect to complete now
if (ret < 0)
{
return false;
}
if (ret == 0)
{
//we had a timeout
#ifdef WIN32
WSASetLastError(WSAETIMEDOUT);
#else
errno = ETIMEDOUT;
#endif
return false;
}
if (!(FD_ISSET(m_iSocket, &rset) || FD_ISSET(m_iSocket, &wset)))
{
return false;
}
//we had a positivite return so a descriptor is ready
if (getsockopt(m_iSocket, SOL_SOCKET, SO_ERROR, (char*)&error, &len) < 0)
{
return false;
}
//check if we had a socket error
if (error)
{
errno = error;
return false;
}
}
//put socket back in blocking mode
#ifdef WIN32
mode = 0;
if (ioctlsocket(m_iSocket, FIONBIO, &mode) != 0)
{
return false;
}
#else
if (fcntl(m_iSocket, F_SETFL, flags) < 0)
#ifndef DISABLE_TLS
if (m_bTLS && !StartTLS(true, NULL, NULL))
{
return false;
}
@@ -844,14 +643,7 @@ bool Connection::DoDisconnect()
#ifndef DISABLE_TLS
CloseTLS();
#endif
if (m_bGracefull)
{
closesocket_gracefully(m_iSocket);
}
else
{
closesocket(m_iSocket);
}
closesocket(m_iSocket);
m_iSocket = INVALID_SOCKET;
}
@@ -894,15 +686,12 @@ void Connection::ReportError(const char* szMsgPrefix, const char* szMsgArg, bool
char szErrPrefix[1024];
snprintf(szErrPrefix, 1024, szMsgPrefix, szMsgArg);
szErrPrefix[1024-1] = '\0';
char szMessage[1024];
if (PrintErrCode)
{
#ifdef WIN32
int ErrCode = WSAGetLastError();
char szErrMsg[1024];
szErrMsg[0] = '\0';
FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM, NULL, ErrCode, 0, szErrMsg, 1024, NULL);
szErrMsg[1024-1] = '\0';
#else
@@ -924,9 +713,7 @@ void Connection::ReportError(const char* szMsgPrefix, const char* szMsgArg, bool
}
else
{
snprintf(szMessage, sizeof(szMessage), "%s: ErrNo %i, %s", szErrPrefix, ErrCode, szErrMsg);
szMessage[sizeof(szMessage) - 1] = '\0';
PrintError(szMessage);
error("%s: ErrNo %i, %s", szErrPrefix, ErrCode, szErrMsg);
}
}
else
@@ -937,23 +724,18 @@ void Connection::ReportError(const char* szMsgPrefix, const char* szMsgArg, bool
}
else
{
PrintError(szErrPrefix);
error(szErrPrefix);
}
}
}
void Connection::PrintError(const char* szErrMsg)
{
error("%s", szErrMsg);
}
#ifndef DISABLE_TLS
bool Connection::StartTLS(bool bIsClient, const char* szCertFile, const char* szKeyFile)
{
debug("Starting TLS");
delete m_pTLSSocket;
m_pTLSSocket = new ConTLSSocket(m_iSocket, bIsClient, szCertFile, szKeyFile, m_szCipher, this);
m_pTLSSocket = new TLSSocket(m_iSocket, bIsClient, szCertFile, szKeyFile, m_szCipher);
m_pTLSSocket->SetSuppressErrors(m_bSuppressErrors);
return m_pTLSSocket->Start();

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -61,32 +61,8 @@ protected:
bool m_bSuppressErrors;
char m_szRemoteAddr[20];
int m_iTotalBytesRead;
bool m_bBroken;
bool m_bGracefull;
struct SockAddr
{
int ai_family;
int ai_socktype;
int ai_protocol;
bool operator==(const SockAddr& rhs) const
{ return memcmp(this, &rhs, sizeof(SockAddr)) == 0; }
};
#ifndef DISABLE_TLS
class ConTLSSocket: public TLSSocket
{
private:
Connection* m_pOwner;
protected:
virtual void PrintError(const char* szErrMsg) { m_pOwner->PrintError(szErrMsg); }
public:
ConTLSSocket(SOCKET iSocket, bool bIsClient, const char* szCertFile,
const char* szKeyFile, const char* szCipher, Connection* pOwner):
TLSSocket(iSocket, bIsClient, szCertFile, szKeyFile, szCipher), m_pOwner(pOwner) {}
};
ConTLSSocket* m_pTLSSocket;
TLSSocket* m_pTLSSocket;
bool m_bTLSError;
#endif
#ifndef HAVE_GETADDRINFO
@@ -97,11 +73,8 @@ protected:
Connection(SOCKET iSocket, bool bTLS);
void ReportError(const char* szMsgPrefix, const char* szMsgArg, bool PrintErrCode, int herrno);
virtual void PrintError(const char* szErrMsg);
bool DoConnect();
bool DoDisconnect();
bool InitSocketOpts();
bool ConnectWithTimeout(void* address, int address_len);
#ifndef HAVE_GETADDRINFO
unsigned int ResolveHostAddr(const char* szHost);
#endif
@@ -137,8 +110,6 @@ public:
void SetSuppressErrors(bool bSuppressErrors);
bool GetSuppressErrors() { return m_bSuppressErrors; }
const char* GetRemoteAddr();
bool GetGracefull() { return m_bGracefull; }
void SetGracefull(bool bGracefull) { m_bGracefull = bGracefull; }
#ifndef DISABLE_TLS
bool StartTLS(bool bIsClient, const char* szCertFile, const char* szKeyFile);
#endif

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2008-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2008-2013 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -35,7 +35,6 @@
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#ifdef WIN32
#include <winsock2.h>
#include <ws2tcpip.h>
@@ -284,8 +283,6 @@ TLSSocket::~TLSSocket()
void TLSSocket::ReportError(const char* szErrMsg)
{
char szMessage[1024];
#ifdef HAVE_LIBGNUTLS
const char* errstr = gnutls_strerror(m_iRetCode);
if (m_bSuppressErrors)
@@ -294,9 +291,7 @@ void TLSSocket::ReportError(const char* szErrMsg)
}
else
{
snprintf(szMessage, sizeof(szMessage), "%s: %s", szErrMsg, errstr);
szMessage[sizeof(szMessage) - 1] = '\0';
PrintError(szMessage);
error("%s: %s", szErrMsg, errstr);
}
#endif /* HAVE_LIBGNUTLS */
@@ -316,23 +311,16 @@ void TLSSocket::ReportError(const char* szErrMsg)
}
else if (errcode != 0)
{
snprintf(szMessage, sizeof(szMessage), "%s: %s", szErrMsg, errstr);
szMessage[sizeof(szMessage) - 1] = '\0';
PrintError(szMessage);
error("%s: %s", szErrMsg, errstr);
}
else
{
PrintError(szErrMsg);
error("%s", szErrMsg);
}
} while (errcode);
#endif /* HAVE_OPENSSL */
}
void TLSSocket::PrintError(const char* szErrMsg)
{
error("%s", szErrMsg);
}
bool TLSSocket::Start()
{
#ifdef HAVE_LIBGNUTLS

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2008-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2008-2013 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -46,12 +46,9 @@ private:
void ReportError(const char* szErrMsg);
protected:
virtual void PrintError(const char* szErrMsg);
public:
TLSSocket(SOCKET iSocket, bool bIsClient, const char* szCertFile, const char* szKeyFile, const char* szCipher);
virtual ~TLSSocket();
~TLSSocket();
static void Init();
static void Final();
bool Start();

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2012-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2012-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -49,6 +49,8 @@
#include "Options.h"
#include "Util.h"
extern Options* g_pOptions;
WebDownloader::WebDownloader()
{
debug("Creating WebDownloader");
@@ -88,7 +90,7 @@ void WebDownloader::SetInfoName(const char* v)
void WebDownloader::SetURL(const char * szURL)
{
free(m_szURL);
m_szURL = WebUtil::URLEncode(szURL);
m_szURL = strdup(szURL);
}
void WebDownloader::SetStatus(EStatus eStatus)
@@ -111,13 +113,14 @@ void WebDownloader::Run()
iRemainedConnectRetries = 1;
}
m_iRedirects = 0;
EStatus Status = adFailed;
while (!IsStopped() && iRemainedDownloadRetries > 0 && iRemainedConnectRetries > 0)
{
SetLastUpdateTimeNow();
Status = DownloadWithRedirects(5);
Status = Download();
if ((((Status == adFailed) && (iRemainedDownloadRetries > 1)) ||
((Status == adConnectError) && (iRemainedConnectRetries > 1)))
@@ -144,6 +147,17 @@ void WebDownloader::Run()
break;
}
if (Status == adRedirect)
{
m_iRedirects++;
if (m_iRedirects > 5)
{
warn("Too many redirects for %s", m_szInfoName);
Status = adFailed;
break;
}
}
if (Status != adConnectError)
{
iRemainedDownloadRetries--;
@@ -193,7 +207,6 @@ WebDownloader::EStatus WebDownloader::Download()
return Status;
}
m_pConnection->SetTimeout(g_pOptions->GetUrlTimeout());
m_pConnection->SetSuppressErrors(false);
// connection
@@ -232,24 +245,6 @@ WebDownloader::EStatus WebDownloader::Download()
return Status;
}
WebDownloader::EStatus WebDownloader::DownloadWithRedirects(int iMaxRedirects)
{
// do sync download, following redirects
EStatus eStatus = adRedirect;
while (eStatus == adRedirect && iMaxRedirects >= 0)
{
iMaxRedirects--;
eStatus = Download();
}
if (eStatus == adRedirect && iMaxRedirects < 0)
{
warn("Too many redirects for %s", m_szInfoName);
eStatus = adFailed;
}
return eStatus;
}
WebDownloader::EStatus WebDownloader::CreateConnection(URL *pUrl)
{
@@ -370,6 +365,10 @@ WebDownloader::EStatus WebDownloader::DownloadHeaders()
// detect body of response
if (*line == '\r' || *line == '\n')
{
if (m_iContentLen == -1 && !m_bGZip)
{
warn("URL %s: Content-Length is not submitted by server, cannot verify whether the file is complete", m_szInfoName);
}
break;
}
@@ -420,10 +419,10 @@ WebDownloader::EStatus WebDownloader::DownloadBody()
szBuffer = szLineBuf;
}
// Connection closed or timeout?
// Have we encountered a timeout?
if (iLen <= 0)
{
if (iLen == 0 && m_iContentLen == -1 && iWrittenLen > 0)
if (m_iContentLen == -1 && iWrittenLen > 0)
{
bEnd = true;
break;
@@ -595,46 +594,15 @@ void WebDownloader::ParseRedirect(const char* szLocation)
URL newUrl(szNewURL);
if (!newUrl.IsValid())
{
// redirect within host
char szResource[1024];
// relative address
URL oldUrl(m_szURL);
if (*szLocation == '/')
{
// absolute path within host
strncpy(szResource, szLocation, 1024);
szResource[1024-1] = '\0';
}
else
{
// relative path within host
strncpy(szResource, oldUrl.GetResource(), 1024);
szResource[1024-1] = '\0';
char* p = strchr(szResource, '?');
if (p)
{
*p = '\0';
}
p = strrchr(szResource, '/');
if (p)
{
p[1] = '\0';
}
strncat(szResource, szLocation, 1024 - strlen(szResource));
szResource[1024-1] = '\0';
}
if (oldUrl.GetPort() > 0)
{
snprintf(szUrlBuf, 1024, "%s://%s:%i%s", oldUrl.GetProtocol(), oldUrl.GetHost(), oldUrl.GetPort(), szResource);
snprintf(szUrlBuf, 1024, "%s://%s:%i%s", oldUrl.GetProtocol(), oldUrl.GetHost(), oldUrl.GetPort(), szNewURL);
}
else
{
snprintf(szUrlBuf, 1024, "%s://%s%s", oldUrl.GetProtocol(), oldUrl.GetHost(), szResource);
snprintf(szUrlBuf, 1024, "%s://%s%s", oldUrl.GetProtocol(), oldUrl.GetHost(), szNewURL);
}
szUrlBuf[1024-1] = '\0';
szNewURL = szUrlBuf;
@@ -696,9 +664,9 @@ bool WebDownloader::PrepareFile()
error("Could not %s file %s", "create", szFilename);
return false;
}
if (g_pOptions->GetWriteBuffer() > 0)
if (g_pOptions->GetWriteBufferSize() > 0)
{
setvbuf(m_pOutFile, NULL, _IOFBF, g_pOptions->GetWriteBuffer() * 1024);
setvbuf(m_pOutFile, (char *)NULL, _IOFBF, g_pOptions->GetWriteBufferSize());
}
return true;

View File

@@ -64,6 +64,7 @@ private:
bool m_bForce;
bool m_bRedirecting;
bool m_bRedirected;
int m_iRedirects;
bool m_bGZip;
bool m_bRetry;
#ifndef DISABLE_GZIP
@@ -92,7 +93,6 @@ public:
virtual void Run();
virtual void Stop();
EStatus Download();
EStatus DownloadWithRedirects(int iMaxRedirects);
bool Terminate();
void SetInfoName(const char* v);
const char* GetInfoName() { return m_szInfoName; }

View File

@@ -1,95 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef WIN32
#include "win32.h"
#endif
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#ifndef WIN32
#include <unistd.h>
#endif
#include <sys/stat.h>
#include <stdio.h>
#include "nzbget.h"
#include "FeedScript.h"
#include "Options.h"
#include "Log.h"
#include "Util.h"
void FeedScriptController::ExecuteScripts(const char* szFeedScript, const char* szFeedFile, int iFeedID)
{
FeedScriptController* pScriptController = new FeedScriptController();
pScriptController->m_szFeedFile = szFeedFile;
pScriptController->m_iFeedID = iFeedID;
pScriptController->ExecuteScriptList(szFeedScript);
delete pScriptController;
}
void FeedScriptController::ExecuteScript(ScriptConfig::Script* pScript)
{
if (!pScript->GetFeedScript())
{
return;
}
PrintMessage(Message::mkInfo, "Executing feed-script %s for Feed%i", pScript->GetName(), m_iFeedID);
SetScript(pScript->GetLocation());
SetArgs(NULL, false);
char szInfoName[1024];
snprintf(szInfoName, 1024, "feed-script %s for Feed%i", pScript->GetName(), m_iFeedID);
szInfoName[1024-1] = '\0';
SetInfoName(szInfoName);
SetLogPrefix(pScript->GetDisplayName());
PrepareParams(pScript->GetName());
Execute();
SetLogPrefix(NULL);
}
void FeedScriptController::PrepareParams(const char* szScriptName)
{
ResetEnv();
SetEnvVar("NZBFP_FILENAME", m_szFeedFile);
SetIntEnvVar("NZBFP_FEEDID", m_iFeedID);
PrepareEnvScript(NULL, szScriptName);
}

View File

@@ -1,46 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifndef FEEDSCRIPT_H
#define FEEDSCRIPT_H
#include "NzbScript.h"
class FeedScriptController : public NZBScriptController
{
private:
const char* m_szFeedFile;
int m_iFeedID;
void PrepareParams(const char* szScriptName);
protected:
virtual void ExecuteScript(ScriptConfig::Script* pScript);
public:
static void ExecuteScripts(const char* szFeedScript, const char* szFeedFile, int iFeedID);
};
#endif

View File

@@ -1,125 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef WIN32
#include "win32.h"
#endif
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#ifndef WIN32
#include <unistd.h>
#endif
#include <sys/stat.h>
#include <stdio.h>
#include <algorithm>
#include "nzbget.h"
#include "NzbScript.h"
#include "Options.h"
#include "Log.h"
#include "Util.h"
/**
* If szStripPrefix is not NULL, only pp-parameters, whose names start with the prefix
* are processed. The prefix is then stripped from the names.
* If szStripPrefix is NULL, all pp-parameters are processed; without stripping.
*/
void NZBScriptController::PrepareEnvParameters(NZBParameterList* pParameters, const char* szStripPrefix)
{
int iPrefixLen = szStripPrefix ? strlen(szStripPrefix) : 0;
for (NZBParameterList::iterator it = pParameters->begin(); it != pParameters->end(); it++)
{
NZBParameter* pParameter = *it;
const char* szValue = pParameter->GetValue();
#ifdef WIN32
char* szAnsiValue = strdup(szValue);
WebUtil::Utf8ToAnsi(szAnsiValue, strlen(szAnsiValue) + 1);
szValue = szAnsiValue;
#endif
if (szStripPrefix && !strncmp(pParameter->GetName(), szStripPrefix, iPrefixLen) && (int)strlen(pParameter->GetName()) > iPrefixLen)
{
SetEnvVarSpecial("NZBPR", pParameter->GetName() + iPrefixLen, szValue);
}
else if (!szStripPrefix)
{
SetEnvVarSpecial("NZBPR", pParameter->GetName(), szValue);
}
#ifdef WIN32
free(szAnsiValue);
#endif
}
}
void NZBScriptController::PrepareEnvScript(NZBParameterList* pParameters, const char* szScriptName)
{
if (pParameters)
{
PrepareEnvParameters(pParameters, NULL);
}
char szParamPrefix[1024];
snprintf(szParamPrefix, 1024, "%s:", szScriptName);
szParamPrefix[1024-1] = '\0';
if (pParameters)
{
PrepareEnvParameters(pParameters, szParamPrefix);
}
PrepareEnvOptions(szParamPrefix);
}
void NZBScriptController::ExecuteScriptList(const char* szScriptList)
{
for (ScriptConfig::Scripts::iterator it = g_pScriptConfig->GetScripts()->begin(); it != g_pScriptConfig->GetScripts()->end(); it++)
{
ScriptConfig::Script* pScript = *it;
if (szScriptList && *szScriptList)
{
// split szScriptList into tokens
Tokenizer tok(szScriptList, ",;");
while (const char* szScriptName = tok.Next())
{
if (Util::SameFilename(szScriptName, pScript->GetName()))
{
ExecuteScript(pScript);
break;
}
}
}
}
}

View File

@@ -1,42 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifndef NZBSCRIPT_H
#define NZBSCRIPT_H
#include "Script.h"
#include "DownloadInfo.h"
#include "ScriptConfig.h"
class NZBScriptController : public ScriptController
{
protected:
void PrepareEnvParameters(NZBParameterList* pParameters, const char* szStripPrefix);
void PrepareEnvScript(NZBParameterList* pParameters, const char* szScriptName);
void ExecuteScriptList(const char* szScriptList);
virtual void ExecuteScript(ScriptConfig::Script* pScript) = 0;
};
#endif

View File

@@ -1,525 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef WIN32
#include "win32.h"
#endif
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#ifndef WIN32
#include <unistd.h>
#endif
#include <sys/stat.h>
#include <stdio.h>
#include "nzbget.h"
#include "QueueScript.h"
#include "NzbScript.h"
#include "Options.h"
#include "Log.h"
#include "Util.h"
static const char* QUEUE_EVENT_NAMES[] = { "FILE_DOWNLOADED", "URL_COMPLETED", "NZB_ADDED", "NZB_DOWNLOADED", "NZB_DELETED" };
class QueueScriptController : public Thread, public NZBScriptController
{
private:
char* m_szNZBName;
char* m_szNZBFilename;
char* m_szUrl;
char* m_szCategory;
char* m_szDestDir;
int m_iID;
int m_iPriority;
char* m_szDupeKey;
EDupeMode m_eDupeMode;
int m_iDupeScore;
NZBParameterList m_Parameters;
int m_iPrefixLen;
ScriptConfig::Script* m_pScript;
QueueScriptCoordinator::EEvent m_eEvent;
bool m_bMarkBad;
NZBInfo::EDeleteStatus m_eDeleteStatus;
NZBInfo::EUrlStatus m_eUrlStatus;
void PrepareParams(const char* szScriptName);
protected:
virtual void ExecuteScript(ScriptConfig::Script* pScript);
virtual void AddMessage(Message::EKind eKind, const char* szText);
public:
virtual ~QueueScriptController();
virtual void Run();
static void StartScript(NZBInfo* pNZBInfo, ScriptConfig::Script* pScript, QueueScriptCoordinator::EEvent eEvent);
};
QueueScriptController::~QueueScriptController()
{
free(m_szNZBName);
free(m_szNZBFilename);
free(m_szUrl);
free(m_szCategory);
free(m_szDestDir);
free(m_szDupeKey);
}
void QueueScriptController::StartScript(NZBInfo* pNZBInfo, ScriptConfig::Script* pScript, QueueScriptCoordinator::EEvent eEvent)
{
QueueScriptController* pScriptController = new QueueScriptController();
pScriptController->m_szNZBName = strdup(pNZBInfo->GetName());
pScriptController->m_szNZBFilename = strdup(pNZBInfo->GetFilename());
pScriptController->m_szUrl = strdup(pNZBInfo->GetURL());
pScriptController->m_szCategory = strdup(pNZBInfo->GetCategory());
pScriptController->m_szDestDir = strdup(pNZBInfo->GetDestDir());
pScriptController->m_iID = pNZBInfo->GetID();
pScriptController->m_iPriority = pNZBInfo->GetPriority();
pScriptController->m_szDupeKey = strdup(pNZBInfo->GetDupeKey());
pScriptController->m_eDupeMode = pNZBInfo->GetDupeMode();
pScriptController->m_iDupeScore = pNZBInfo->GetDupeScore();
pScriptController->m_Parameters.CopyFrom(pNZBInfo->GetParameters());
pScriptController->m_pScript = pScript;
pScriptController->m_eEvent = eEvent;
pScriptController->m_iPrefixLen = 0;
pScriptController->m_bMarkBad = false;
pScriptController->m_eDeleteStatus = pNZBInfo->GetDeleteStatus();
pScriptController->m_eUrlStatus = pNZBInfo->GetUrlStatus();
pScriptController->SetAutoDestroy(true);
pScriptController->Start();
}
void QueueScriptController::Run()
{
ExecuteScript(m_pScript);
SetLogPrefix(NULL);
if (m_bMarkBad)
{
DownloadQueue* pDownloadQueue = DownloadQueue::Lock();
NZBInfo* pNZBInfo = pDownloadQueue->GetQueue()->Find(m_iID);
if (pNZBInfo)
{
PrintMessage(Message::mkWarning, "Cancelling download and deleting %s", m_szNZBName);
pNZBInfo->SetDeleteStatus(NZBInfo::dsBad);
pDownloadQueue->EditEntry(m_iID, DownloadQueue::eaGroupDelete, 0, NULL);
}
DownloadQueue::Unlock();
}
g_pQueueScriptCoordinator->CheckQueue();
}
void QueueScriptController::ExecuteScript(ScriptConfig::Script* pScript)
{
PrintMessage(m_eEvent == QueueScriptCoordinator::qeFileDownloaded ? Message::mkDetail : Message::mkInfo,
"Executing queue-script %s for %s", pScript->GetName(), Util::BaseFileName(m_szNZBName));
SetScript(pScript->GetLocation());
SetArgs(NULL, false);
char szInfoName[1024];
snprintf(szInfoName, 1024, "queue-script %s for %s", pScript->GetName(), Util::BaseFileName(m_szNZBName));
szInfoName[1024-1] = '\0';
SetInfoName(szInfoName);
SetLogPrefix(pScript->GetDisplayName());
m_iPrefixLen = strlen(pScript->GetDisplayName()) + 2; // 2 = strlen(": ");
PrepareParams(pScript->GetName());
Execute();
SetLogPrefix(NULL);
}
void QueueScriptController::PrepareParams(const char* szScriptName)
{
ResetEnv();
SetEnvVar("NZBNA_NZBNAME", m_szNZBName);
SetIntEnvVar("NZBNA_NZBID", m_iID);
SetEnvVar("NZBNA_FILENAME", m_szNZBFilename);
SetEnvVar("NZBNA_DIRECTORY", m_szDestDir);
SetEnvVar("NZBNA_URL", m_szUrl);
SetEnvVar("NZBNA_CATEGORY", m_szCategory);
SetIntEnvVar("NZBNA_PRIORITY", m_iPriority);
SetIntEnvVar("NZBNA_LASTID", m_iID); // deprecated
SetEnvVar("NZBNA_DUPEKEY", m_szDupeKey);
SetIntEnvVar("NZBNA_DUPESCORE", m_iDupeScore);
const char* szDupeModeName[] = { "SCORE", "ALL", "FORCE" };
SetEnvVar("NZBNA_DUPEMODE", szDupeModeName[m_eDupeMode]);
SetEnvVar("NZBNA_EVENT", QUEUE_EVENT_NAMES[m_eEvent]);
const char* szDeleteStatusName[] = { "NONE", "MANUAL", "HEALTH", "DUPE", "BAD", "GOOD", "COPY", "SCAN" };
SetEnvVar("NZBNA_DELETESTATUS", szDeleteStatusName[m_eDeleteStatus]);
const char* szUrlStatusName[] = { "NONE", "UNKNOWN", "SUCCESS", "FAILURE", "UNKNOWN", "SCAN_SKIPPED", "SCAN_FAILURE" };
SetEnvVar("NZBNA_URLSTATUS", szUrlStatusName[m_eUrlStatus]);
PrepareEnvScript(&m_Parameters, szScriptName);
}
void QueueScriptController::AddMessage(Message::EKind eKind, const char* szText)
{
const char* szMsgText = szText + m_iPrefixLen;
if (!strncmp(szMsgText, "[NZB] ", 6))
{
debug("Command %s detected", szMsgText + 6);
if (!strncmp(szMsgText + 6, "NZBPR_", 6))
{
char* szParam = strdup(szMsgText + 6 + 6);
char* szValue = strchr(szParam, '=');
if (szValue)
{
*szValue = '\0';
DownloadQueue* pDownloadQueue = DownloadQueue::Lock();
NZBInfo* pNZBInfo = pDownloadQueue->GetQueue()->Find(m_iID);
if (pNZBInfo)
{
pNZBInfo->GetParameters()->SetParameter(szParam, szValue + 1);
}
DownloadQueue::Unlock();
}
else
{
error("Invalid command \"%s\" received from %s", szMsgText, GetInfoName());
}
free(szParam);
}
else if (!strncmp(szMsgText + 6, "MARK=BAD", 8))
{
m_bMarkBad = true;
DownloadQueue* pDownloadQueue = DownloadQueue::Lock();
NZBInfo* pNZBInfo = pDownloadQueue->GetQueue()->Find(m_iID);
if (pNZBInfo)
{
SetLogPrefix(NULL);
PrintMessage(Message::mkWarning, "Marking %s as bad", m_szNZBName);
SetLogPrefix(m_pScript->GetDisplayName());
pNZBInfo->SetMarkStatus(NZBInfo::ksBad);
}
DownloadQueue::Unlock();
}
else
{
error("Invalid command \"%s\" received from %s", szMsgText, GetInfoName());
}
}
else
{
ScriptController::AddMessage(eKind, szText);
}
}
QueueScriptCoordinator::QueueItem::QueueItem(int iNZBID, ScriptConfig::Script* pScript, EEvent eEvent)
{
m_iNZBID = iNZBID;
m_pScript = pScript;
m_eEvent = eEvent;
}
QueueScriptCoordinator::QueueScriptCoordinator()
{
m_pCurItem = NULL;
m_bStopped = false;
}
QueueScriptCoordinator::~QueueScriptCoordinator()
{
delete m_pCurItem;
for (Queue::iterator it = m_Queue.begin(); it != m_Queue.end(); it++ )
{
delete *it;
}
}
void QueueScriptCoordinator::InitOptions()
{
m_bHasQueueScripts = false;
for (ScriptConfig::Scripts::iterator it = g_pScriptConfig->GetScripts()->begin(); it != g_pScriptConfig->GetScripts()->end(); it++)
{
ScriptConfig::Script* pScript = *it;
if (pScript->GetQueueScript())
{
m_bHasQueueScripts = true;
break;
}
}
}
void QueueScriptCoordinator::EnqueueScript(NZBInfo* pNZBInfo, EEvent eEvent)
{
if (!m_bHasQueueScripts)
{
return;
}
m_mutexQueue.Lock();
if (eEvent == qeNzbDownloaded)
{
// delete all other queued scripts for this nzb
for (Queue::iterator it = m_Queue.begin(); it != m_Queue.end(); )
{
QueueItem* pQueueItem = *it;
if (pQueueItem->GetNZBID() == pNZBInfo->GetID())
{
delete pQueueItem;
it = m_Queue.erase(it);
continue;
}
it++;
}
}
// respect option "EventInterval"
time_t tCurTime = time(NULL);
if (eEvent == qeFileDownloaded &&
(g_pOptions->GetEventInterval() == -1 ||
(g_pOptions->GetEventInterval() > 0 && tCurTime - pNZBInfo->GetQueueScriptTime() > 0 &&
(int)(tCurTime - pNZBInfo->GetQueueScriptTime()) < g_pOptions->GetEventInterval())))
{
m_mutexQueue.Unlock();
return;
}
for (ScriptConfig::Scripts::iterator it = g_pScriptConfig->GetScripts()->begin(); it != g_pScriptConfig->GetScripts()->end(); it++)
{
ScriptConfig::Script* pScript = *it;
if (!pScript->GetQueueScript())
{
continue;
}
bool bUseScript = false;
// check queue-scripts
const char* szQueueScript = g_pOptions->GetQueueScript();
if (!Util::EmptyStr(szQueueScript))
{
// split szQueueScript into tokens
Tokenizer tok(szQueueScript, ",;");
while (const char* szScriptName = tok.Next())
{
if (Util::SameFilename(szScriptName, pScript->GetName()))
{
bUseScript = true;
break;
}
}
}
// check post-processing-scripts
if (!bUseScript)
{
for (NZBParameterList::iterator it = pNZBInfo->GetParameters()->begin(); it != pNZBInfo->GetParameters()->end(); it++)
{
NZBParameter* pParameter = *it;
const char* szVarname = pParameter->GetName();
if (strlen(szVarname) > 0 && szVarname[0] != '*' && szVarname[strlen(szVarname)-1] == ':' &&
(!strcasecmp(pParameter->GetValue(), "yes") ||
!strcasecmp(pParameter->GetValue(), "on") ||
!strcasecmp(pParameter->GetValue(), "1")))
{
char szScriptName[1024];
strncpy(szScriptName, szVarname, 1024);
szScriptName[1024-1] = '\0';
szScriptName[strlen(szScriptName)-1] = '\0'; // remove trailing ':'
if (Util::SameFilename(szScriptName, pScript->GetName()))
{
bUseScript = true;
break;
}
}
}
}
bUseScript &= Util::EmptyStr(pScript->GetQueueEvents()) || strstr(pScript->GetQueueEvents(), QUEUE_EVENT_NAMES[eEvent]);
if (bUseScript)
{
bool bAlreadyQueued = false;
if (eEvent == qeFileDownloaded)
{
// check if this script is already queued for this nzb
for (Queue::iterator it2 = m_Queue.begin(); it2 != m_Queue.end(); it2++)
{
QueueItem* pQueueItem = *it2;
if (pQueueItem->GetNZBID() == pNZBInfo->GetID() && pQueueItem->GetScript() == pScript)
{
bAlreadyQueued = true;
break;
}
}
}
if (!bAlreadyQueued)
{
QueueItem* pQueueItem = new QueueItem(pNZBInfo->GetID(), pScript, eEvent);
if (m_pCurItem)
{
m_Queue.push_back(pQueueItem);
}
else
{
StartScript(pNZBInfo, pQueueItem);
}
}
pNZBInfo->SetQueueScriptTime(time(NULL));
}
}
m_mutexQueue.Unlock();
}
NZBInfo* QueueScriptCoordinator::FindNZBInfo(DownloadQueue* pDownloadQueue, int iNZBID)
{
NZBInfo* pNZBInfo = pDownloadQueue->GetQueue()->Find(iNZBID);
if (!pNZBInfo)
{
for (HistoryList::iterator it = pDownloadQueue->GetHistory()->begin(); it != pDownloadQueue->GetHistory()->end(); it++)
{
HistoryInfo* pHistoryInfo = *it;
if (pHistoryInfo->GetNZBInfo() && pHistoryInfo->GetNZBInfo()->GetID() == iNZBID)
{
pNZBInfo = pHistoryInfo->GetNZBInfo();
break;
}
}
}
return pNZBInfo;
}
void QueueScriptCoordinator::CheckQueue()
{
if (m_bStopped)
{
return;
}
DownloadQueue* pDownloadQueue = DownloadQueue::Lock();
m_mutexQueue.Lock();
delete m_pCurItem;
m_pCurItem = NULL;
NZBInfo* pCurNZBInfo = NULL;
Queue::iterator itCurItem = m_Queue.end();
for (Queue::iterator it = m_Queue.begin(); it != m_Queue.end(); )
{
QueueItem* pQueueItem = *it;
NZBInfo* pNZBInfo = FindNZBInfo(pDownloadQueue, pQueueItem->GetNZBID());
// in a case this nzb must not be processed further - delete queue script from queue
if (!pNZBInfo ||
(pNZBInfo->GetDeleteStatus() != NZBInfo::dsNone && pQueueItem->GetEvent() != qeNzbDeleted) ||
pNZBInfo->GetMarkStatus() == NZBInfo::ksBad)
{
delete pQueueItem;
it = m_Queue.erase(it);
continue;
}
if (!m_pCurItem || pQueueItem->GetEvent() > m_pCurItem->GetEvent())
{
m_pCurItem = pQueueItem;
itCurItem = it;
pCurNZBInfo = pNZBInfo;
}
it++;
}
if (m_pCurItem)
{
m_Queue.erase(itCurItem);
StartScript(pCurNZBInfo, m_pCurItem);
}
m_mutexQueue.Unlock();
DownloadQueue::Unlock();
}
void QueueScriptCoordinator::StartScript(NZBInfo* pNZBInfo, QueueItem* pQueueItem)
{
m_pCurItem = pQueueItem;
QueueScriptController::StartScript(pNZBInfo, pQueueItem->GetScript(), pQueueItem->GetEvent());
}
bool QueueScriptCoordinator::HasJob(int iNZBID, bool* pActive)
{
m_mutexQueue.Lock();
bool bWorking = m_pCurItem && m_pCurItem->GetNZBID() == iNZBID;
if (pActive)
{
*pActive = bWorking;
}
if (!bWorking)
{
for (Queue::iterator it = m_Queue.begin(); it != m_Queue.end(); it++)
{
QueueItem* pQueueItem = *it;
bWorking = pQueueItem->GetNZBID() == iNZBID;
if (bWorking)
{
break;
}
}
}
m_mutexQueue.Unlock();
return bWorking;
}
int QueueScriptCoordinator::GetQueueSize()
{
m_mutexQueue.Lock();
int iQueuedCount = m_Queue.size();
if (m_pCurItem)
{
iQueuedCount++;
}
m_mutexQueue.Unlock();
return iQueuedCount;
}

View File

@@ -1,84 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifndef QUEUESCRIPT_H
#define QUEUESCRIPT_H
#include <list>
#include "DownloadInfo.h"
#include "ScriptConfig.h"
class QueueScriptCoordinator
{
public:
enum EEvent
{
qeFileDownloaded, // lowest priority
qeUrlCompleted,
qeNzbAdded,
qeNzbDownloaded,
qeNzbDeleted // highest priority
};
private:
class QueueItem
{
private:
int m_iNZBID;
ScriptConfig::Script* m_pScript;
EEvent m_eEvent;
public:
QueueItem(int iNZBID, ScriptConfig::Script* pScript, EEvent eEvent);
int GetNZBID() { return m_iNZBID; }
ScriptConfig::Script* GetScript() { return m_pScript; }
EEvent GetEvent() { return m_eEvent; }
};
typedef std::list<QueueItem*> Queue;
Queue m_Queue;
Mutex m_mutexQueue;
QueueItem* m_pCurItem;
bool m_bHasQueueScripts;
bool m_bStopped;
void StartScript(NZBInfo* pNZBInfo, QueueItem* pQueueItem);
NZBInfo* FindNZBInfo(DownloadQueue* pDownloadQueue, int iNZBID);
public:
QueueScriptCoordinator();
~QueueScriptCoordinator();
void Stop() { m_bStopped = true; }
void InitOptions();
void EnqueueScript(NZBInfo* pNZBInfo, EEvent eEvent);
void CheckQueue();
bool HasJob(int iNZBID, bool* pActive);
int GetQueueSize();
};
extern QueueScriptCoordinator* g_pQueueScriptCoordinator;
#endif

View File

@@ -1,143 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef WIN32
#include "win32.h"
#endif
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#ifndef WIN32
#include <unistd.h>
#endif
#include <sys/stat.h>
#include <stdio.h>
#include "nzbget.h"
#include "SchedulerScript.h"
#include "Options.h"
#include "Log.h"
#include "Util.h"
SchedulerScriptController::~SchedulerScriptController()
{
free(m_szScript);
}
void SchedulerScriptController::StartScript(const char* szParam, bool bExternalProcess, int iTaskID)
{
char** argv = NULL;
if (bExternalProcess && !Util::SplitCommandLine(szParam, &argv))
{
error("Could not execute scheduled process-script, failed to parse command line: %s", szParam);
return;
}
SchedulerScriptController* pScriptController = new SchedulerScriptController();
pScriptController->m_bExternalProcess = bExternalProcess;
pScriptController->m_szScript = strdup(szParam);
pScriptController->m_iTaskID = iTaskID;
if (bExternalProcess)
{
pScriptController->SetScript(argv[0]);
pScriptController->SetArgs((const char**)argv, true);
}
pScriptController->SetAutoDestroy(true);
pScriptController->Start();
}
void SchedulerScriptController::Run()
{
if (m_bExternalProcess)
{
ExecuteExternalProcess();
}
else
{
ExecuteScriptList(m_szScript);
}
}
void SchedulerScriptController::ExecuteScript(ScriptConfig::Script* pScript)
{
if (!pScript->GetSchedulerScript())
{
return;
}
PrintMessage(Message::mkInfo, "Executing scheduler-script %s for Task%i", pScript->GetName(), m_iTaskID);
SetScript(pScript->GetLocation());
SetArgs(NULL, false);
char szInfoName[1024];
snprintf(szInfoName, 1024, "scheduler-script %s for Task%i", pScript->GetName(), m_iTaskID);
szInfoName[1024-1] = '\0';
SetInfoName(szInfoName);
SetLogPrefix(pScript->GetDisplayName());
PrepareParams(pScript->GetName());
Execute();
SetLogPrefix(NULL);
}
void SchedulerScriptController::PrepareParams(const char* szScriptName)
{
ResetEnv();
SetIntEnvVar("NZBSP_TASKID", m_iTaskID);
PrepareEnvScript(NULL, szScriptName);
}
void SchedulerScriptController::ExecuteExternalProcess()
{
info("Executing scheduled process-script %s for Task%i", Util::BaseFileName(GetScript()), m_iTaskID);
char szInfoName[1024];
snprintf(szInfoName, 1024, "scheduled process-script %s for Task%i", Util::BaseFileName(GetScript()), m_iTaskID);
szInfoName[1024-1] = '\0';
SetInfoName(szInfoName);
char szLogPrefix[1024];
strncpy(szLogPrefix, Util::BaseFileName(GetScript()), 1024);
szLogPrefix[1024-1] = '\0';
if (char* ext = strrchr(szLogPrefix, '.')) *ext = '\0'; // strip file extension
SetLogPrefix(szLogPrefix);
Execute();
}

View File

@@ -1,50 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifndef SCHEDULERSCRIPT_H
#define SCHEDULERSCRIPT_H
#include "NzbScript.h"
class SchedulerScriptController : public Thread, public NZBScriptController
{
private:
char* m_szScript;
bool m_bExternalProcess;
int m_iTaskID;
void PrepareParams(const char* szScriptName);
void ExecuteExternalProcess();
protected:
virtual void ExecuteScript(ScriptConfig::Script* pScript);
public:
virtual ~SchedulerScriptController();
virtual void Run();
static void StartScript(const char* szParam, bool bExternalProcess, int iTaskID);
};
#endif

View File

@@ -1,559 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef WIN32
#include "win32.h"
#endif
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <ctype.h>
#include <sys/stat.h>
#include <set>
#include "nzbget.h"
#include "Util.h"
#include "Options.h"
#include "Log.h"
#include "ScriptConfig.h"
static const char* BEGIN_SCRIPT_SIGNATURE = "### NZBGET ";
static const char* POST_SCRIPT_SIGNATURE = "POST-PROCESSING";
static const char* SCAN_SCRIPT_SIGNATURE = "SCAN";
static const char* QUEUE_SCRIPT_SIGNATURE = "QUEUE";
static const char* SCHEDULER_SCRIPT_SIGNATURE = "SCHEDULER";
static const char* FEED_SCRIPT_SIGNATURE = "FEED";
static const char* END_SCRIPT_SIGNATURE = " SCRIPT";
static const char* QUEUE_EVENTS_SIGNATURE = "### QUEUE EVENTS:";
ScriptConfig* g_pScriptConfig = NULL;
ScriptConfig::ConfigTemplate::ConfigTemplate(Script* pScript, const char* szTemplate)
{
m_pScript = pScript;
m_szTemplate = strdup(szTemplate ? szTemplate : "");
}
ScriptConfig::ConfigTemplate::~ConfigTemplate()
{
delete m_pScript;
free(m_szTemplate);
}
ScriptConfig::ConfigTemplates::~ConfigTemplates()
{
for (iterator it = begin(); it != end(); it++)
{
delete *it;
}
}
ScriptConfig::Script::Script(const char* szName, const char* szLocation)
{
m_szName = strdup(szName);
m_szLocation = strdup(szLocation);
m_szDisplayName = strdup(szName);
m_bPostScript = false;
m_bScanScript = false;
m_bQueueScript = false;
m_bSchedulerScript = false;
m_bFeedScript = false;
m_szQueueEvents = NULL;
}
ScriptConfig::Script::~Script()
{
free(m_szName);
free(m_szLocation);
free(m_szDisplayName);
free(m_szQueueEvents);
}
void ScriptConfig::Script::SetDisplayName(const char* szDisplayName)
{
free(m_szDisplayName);
m_szDisplayName = strdup(szDisplayName);
}
void ScriptConfig::Script::SetQueueEvents(const char* szQueueEvents)
{
free(m_szQueueEvents);
m_szQueueEvents = szQueueEvents ? strdup(szQueueEvents) : NULL;
}
ScriptConfig::Scripts::~Scripts()
{
Clear();
}
void ScriptConfig::Scripts::Clear()
{
for (iterator it = begin(); it != end(); it++)
{
delete *it;
}
clear();
}
ScriptConfig::Script* ScriptConfig::Scripts::Find(const char* szName)
{
for (iterator it = begin(); it != end(); it++)
{
Script* pScript = *it;
if (!strcmp(pScript->GetName(), szName))
{
return pScript;
}
}
return NULL;
}
ScriptConfig::ScriptConfig()
{
InitScripts();
InitConfigTemplates();
}
ScriptConfig::~ScriptConfig()
{
}
bool ScriptConfig::LoadConfig(Options::OptEntries* pOptEntries)
{
// read config file
FILE* infile = fopen(g_pOptions->GetConfigFilename(), FOPEN_RB);
if (!infile)
{
return false;
}
int iBufLen = (int)Util::FileSize(g_pOptions->GetConfigFilename()) + 1;
char* buf = (char*)malloc(iBufLen);
while (fgets(buf, iBufLen - 1, infile))
{
// remove trailing '\n' and '\r' and spaces
Util::TrimRight(buf);
// skip comments and empty lines
if (buf[0] == 0 || buf[0] == '#' || strspn(buf, " ") == strlen(buf))
{
continue;
}
char* optname;
char* optvalue;
if (g_pOptions->SplitOptionString(buf, &optname, &optvalue))
{
Options::OptEntry* pOptEntry = new Options::OptEntry();
pOptEntry->SetName(optname);
pOptEntry->SetValue(optvalue);
pOptEntries->push_back(pOptEntry);
free(optname);
free(optvalue);
}
}
fclose(infile);
free(buf);
return true;
}
bool ScriptConfig::SaveConfig(Options::OptEntries* pOptEntries)
{
// save to config file
FILE* infile = fopen(g_pOptions->GetConfigFilename(), FOPEN_RBP);
if (!infile)
{
return false;
}
std::vector<char*> config;
std::set<Options::OptEntry*> writtenOptions;
// read config file into memory array
int iBufLen = (int)Util::FileSize(g_pOptions->GetConfigFilename()) + 1;
char* buf = (char*)malloc(iBufLen);
while (fgets(buf, iBufLen - 1, infile))
{
config.push_back(strdup(buf));
}
free(buf);
// write config file back to disk, replace old values of existing options with new values
rewind(infile);
for (std::vector<char*>::iterator it = config.begin(); it != config.end(); it++)
{
char* buf = *it;
const char* eq = strchr(buf, '=');
if (eq && buf[0] != '#')
{
// remove trailing '\n' and '\r' and spaces
Util::TrimRight(buf);
char* optname;
char* optvalue;
if (g_pOptions->SplitOptionString(buf, &optname, &optvalue))
{
Options::OptEntry *pOptEntry = pOptEntries->FindOption(optname);
if (pOptEntry)
{
fputs(pOptEntry->GetName(), infile);
fputs("=", infile);
fputs(pOptEntry->GetValue(), infile);
fputs("\n", infile);
writtenOptions.insert(pOptEntry);
}
free(optname);
free(optvalue);
}
}
else
{
fputs(buf, infile);
}
free(buf);
}
// write new options
for (Options::OptEntries::iterator it = pOptEntries->begin(); it != pOptEntries->end(); it++)
{
Options::OptEntry* pOptEntry = *it;
std::set<Options::OptEntry*>::iterator fit = writtenOptions.find(pOptEntry);
if (fit == writtenOptions.end())
{
fputs(pOptEntry->GetName(), infile);
fputs("=", infile);
fputs(pOptEntry->GetValue(), infile);
fputs("\n", infile);
}
}
// close and truncate the file
int pos = (int)ftell(infile);
fclose(infile);
Util::TruncateFile(g_pOptions->GetConfigFilename(), pos);
return true;
}
bool ScriptConfig::LoadConfigTemplates(ConfigTemplates* pConfigTemplates)
{
char* szBuffer;
int iLength;
if (!Util::LoadFileIntoBuffer(g_pOptions->GetConfigTemplate(), &szBuffer, &iLength))
{
return false;
}
ConfigTemplate* pConfigTemplate = new ConfigTemplate(NULL, szBuffer);
pConfigTemplates->push_back(pConfigTemplate);
free(szBuffer);
if (!g_pOptions->GetScriptDir())
{
return true;
}
Scripts scriptList;
LoadScripts(&scriptList);
const int iBeginSignatureLen = strlen(BEGIN_SCRIPT_SIGNATURE);
const int iQueueEventsSignatureLen = strlen(QUEUE_EVENTS_SIGNATURE);
for (Scripts::iterator it = scriptList.begin(); it != scriptList.end(); it++)
{
Script* pScript = *it;
FILE* infile = fopen(pScript->GetLocation(), FOPEN_RB);
if (!infile)
{
ConfigTemplate* pConfigTemplate = new ConfigTemplate(pScript, "");
pConfigTemplates->push_back(pConfigTemplate);
continue;
}
StringBuilder stringBuilder;
char buf[1024];
bool bInConfig = false;
while (fgets(buf, sizeof(buf) - 1, infile))
{
if (!strncmp(buf, BEGIN_SCRIPT_SIGNATURE, iBeginSignatureLen) &&
strstr(buf, END_SCRIPT_SIGNATURE) &&
(strstr(buf, POST_SCRIPT_SIGNATURE) ||
strstr(buf, SCAN_SCRIPT_SIGNATURE) ||
strstr(buf, QUEUE_SCRIPT_SIGNATURE) ||
strstr(buf, SCHEDULER_SCRIPT_SIGNATURE) ||
strstr(buf, FEED_SCRIPT_SIGNATURE)))
{
if (bInConfig)
{
break;
}
bInConfig = true;
continue;
}
bool bSkip = !strncmp(buf, QUEUE_EVENTS_SIGNATURE, iQueueEventsSignatureLen);
if (bInConfig && !bSkip)
{
stringBuilder.Append(buf);
}
}
fclose(infile);
ConfigTemplate* pConfigTemplate = new ConfigTemplate(pScript, stringBuilder.GetBuffer());
pConfigTemplates->push_back(pConfigTemplate);
}
// clearing the list without deleting of objects, which are in pConfigTemplates now
scriptList.clear();
return true;
}
void ScriptConfig::InitConfigTemplates()
{
if (!LoadConfigTemplates(&m_ConfigTemplates))
{
error("Could not read configuration templates");
}
}
void ScriptConfig::InitScripts()
{
LoadScripts(&m_Scripts);
}
void ScriptConfig::LoadScripts(Scripts* pScripts)
{
if (strlen(g_pOptions->GetScriptDir()) == 0)
{
return;
}
Scripts tmpScripts;
LoadScriptDir(&tmpScripts, g_pOptions->GetScriptDir(), false);
tmpScripts.sort(CompareScripts);
// first add all scripts from m_szScriptOrder
Tokenizer tok(g_pOptions->GetScriptOrder(), ",;");
while (const char* szScriptName = tok.Next())
{
Script* pScript = tmpScripts.Find(szScriptName);
if (pScript)
{
tmpScripts.remove(pScript);
pScripts->push_back(pScript);
}
}
// second add all other scripts from scripts directory
for (Scripts::iterator it = tmpScripts.begin(); it != tmpScripts.end(); it++)
{
Script* pScript = *it;
if (!pScripts->Find(pScript->GetName()))
{
pScripts->push_back(pScript);
}
}
tmpScripts.clear();
BuildScriptDisplayNames(pScripts);
}
void ScriptConfig::LoadScriptDir(Scripts* pScripts, const char* szDirectory, bool bIsSubDir)
{
int iBufSize = 1024*10;
char* szBuffer = (char*)malloc(iBufSize+1);
const int iBeginSignatureLen = strlen(BEGIN_SCRIPT_SIGNATURE);
const int iQueueEventsSignatureLen = strlen(QUEUE_EVENTS_SIGNATURE);
DirBrowser dir(szDirectory);
while (const char* szFilename = dir.Next())
{
if (szFilename[0] != '.' && szFilename[0] != '_')
{
char szFullFilename[1024];
snprintf(szFullFilename, 1024, "%s%s", szDirectory, szFilename);
szFullFilename[1024-1] = '\0';
if (!Util::DirectoryExists(szFullFilename))
{
// check if the file contains pp-script-signature
FILE* infile = fopen(szFullFilename, FOPEN_RB);
if (infile)
{
// read first 10KB of the file and look for signature
int iReadBytes = fread(szBuffer, 1, iBufSize, infile);
fclose(infile);
szBuffer[iReadBytes] = 0;
// split buffer into lines
Tokenizer tok(szBuffer, "\n\r", true);
while (char* szLine = tok.Next())
{
if (!strncmp(szLine, BEGIN_SCRIPT_SIGNATURE, iBeginSignatureLen) &&
strstr(szLine, END_SCRIPT_SIGNATURE))
{
bool bPostScript = strstr(szLine, POST_SCRIPT_SIGNATURE);
bool bScanScript = strstr(szLine, SCAN_SCRIPT_SIGNATURE);
bool bQueueScript = strstr(szLine, QUEUE_SCRIPT_SIGNATURE);
bool bSchedulerScript = strstr(szLine, SCHEDULER_SCRIPT_SIGNATURE);
bool bFeedScript = strstr(szLine, FEED_SCRIPT_SIGNATURE);
if (bPostScript || bScanScript || bQueueScript || bSchedulerScript || bFeedScript)
{
char szScriptName[1024];
if (bIsSubDir)
{
char szDirectory2[1024];
snprintf(szDirectory2, 1024, "%s", szDirectory);
szDirectory2[1024-1] = '\0';
int iLen = strlen(szDirectory2);
if (szDirectory2[iLen-1] == PATH_SEPARATOR || szDirectory2[iLen-1] == ALT_PATH_SEPARATOR)
{
// trim last path-separator
szDirectory2[iLen-1] = '\0';
}
snprintf(szScriptName, 1024, "%s%c%s", Util::BaseFileName(szDirectory2), PATH_SEPARATOR, szFilename);
}
else
{
snprintf(szScriptName, 1024, "%s", szFilename);
}
szScriptName[1024-1] = '\0';
char* szQueueEvents = NULL;
if (bQueueScript)
{
while (char* szLine = tok.Next())
{
if (!strncmp(szLine, QUEUE_EVENTS_SIGNATURE, iQueueEventsSignatureLen))
{
szQueueEvents = szLine + iQueueEventsSignatureLen;
break;
}
}
}
Script* pScript = new Script(szScriptName, szFullFilename);
pScript->SetPostScript(bPostScript);
pScript->SetScanScript(bScanScript);
pScript->SetQueueScript(bQueueScript);
pScript->SetSchedulerScript(bSchedulerScript);
pScript->SetFeedScript(bFeedScript);
pScript->SetQueueEvents(szQueueEvents);
pScripts->push_back(pScript);
break;
}
}
}
}
}
else if (!bIsSubDir)
{
snprintf(szFullFilename, 1024, "%s%s%c", szDirectory, szFilename, PATH_SEPARATOR);
szFullFilename[1024-1] = '\0';
LoadScriptDir(pScripts, szFullFilename, true);
}
}
}
free(szBuffer);
}
bool ScriptConfig::CompareScripts(Script* pScript1, Script* pScript2)
{
return strcmp(pScript1->GetName(), pScript2->GetName()) < 0;
}
void ScriptConfig::BuildScriptDisplayNames(Scripts* pScripts)
{
// trying to use short name without path and extension.
// if there are other scripts with the same short name - using a longer name instead (with ot without extension)
for (Scripts::iterator it = pScripts->begin(); it != pScripts->end(); it++)
{
Script* pScript = *it;
char szShortName[256];
strncpy(szShortName, pScript->GetName(), 256);
szShortName[256-1] = '\0';
if (char* ext = strrchr(szShortName, '.')) *ext = '\0'; // strip file extension
const char* szDisplayName = Util::BaseFileName(szShortName);
for (Scripts::iterator it2 = pScripts->begin(); it2 != pScripts->end(); it2++)
{
Script* pScript2 = *it2;
char szShortName2[256];
strncpy(szShortName2, pScript2->GetName(), 256);
szShortName2[256-1] = '\0';
if (char* ext = strrchr(szShortName2, '.')) *ext = '\0'; // strip file extension
const char* szDisplayName2 = Util::BaseFileName(szShortName2);
if (!strcmp(szDisplayName, szDisplayName2) && pScript->GetName() != pScript2->GetName())
{
if (!strcmp(szShortName, szShortName2))
{
szDisplayName = pScript->GetName();
}
else
{
szDisplayName = szShortName;
}
break;
}
}
pScript->SetDisplayName(szDisplayName);
}
}

View File

@@ -1,128 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifndef SCRIPTCONFIG_H
#define SCRIPTCONFIG_H
#include <vector>
#include <list>
#include <time.h>
#include "Options.h"
class ScriptConfig
{
public:
class Script
{
private:
char* m_szName;
char* m_szLocation;
char* m_szDisplayName;
bool m_bPostScript;
bool m_bScanScript;
bool m_bQueueScript;
bool m_bSchedulerScript;
bool m_bFeedScript;
char* m_szQueueEvents;
public:
Script(const char* szName, const char* szLocation);
~Script();
const char* GetName() { return m_szName; }
const char* GetLocation() { return m_szLocation; }
void SetDisplayName(const char* szDisplayName);
const char* GetDisplayName() { return m_szDisplayName; }
bool GetPostScript() { return m_bPostScript; }
void SetPostScript(bool bPostScript) { m_bPostScript = bPostScript; }
bool GetScanScript() { return m_bScanScript; }
void SetScanScript(bool bScanScript) { m_bScanScript = bScanScript; }
bool GetQueueScript() { return m_bQueueScript; }
void SetQueueScript(bool bQueueScript) { m_bQueueScript = bQueueScript; }
bool GetSchedulerScript() { return m_bSchedulerScript; }
void SetSchedulerScript(bool bSchedulerScript) { m_bSchedulerScript = bSchedulerScript; }
bool GetFeedScript() { return m_bFeedScript; }
void SetFeedScript(bool bFeedScript) { m_bFeedScript = bFeedScript; }
void SetQueueEvents(const char* szQueueEvents);
const char* GetQueueEvents() { return m_szQueueEvents; }
};
typedef std::list<Script*> ScriptsBase;
class Scripts: public ScriptsBase
{
public:
~Scripts();
void Clear();
Script* Find(const char* szName);
};
class ConfigTemplate
{
private:
Script* m_pScript;
char* m_szTemplate;
friend class Options;
public:
ConfigTemplate(Script* pScript, const char* szTemplate);
~ConfigTemplate();
Script* GetScript() { return m_pScript; }
const char* GetTemplate() { return m_szTemplate; }
};
typedef std::vector<ConfigTemplate*> ConfigTemplatesBase;
class ConfigTemplates: public ConfigTemplatesBase
{
public:
~ConfigTemplates();
};
private:
Scripts m_Scripts;
ConfigTemplates m_ConfigTemplates;
void InitScripts();
void InitConfigTemplates();
static bool CompareScripts(Script* pScript1, Script* pScript2);
void LoadScriptDir(Scripts* pScripts, const char* szDirectory, bool bIsSubDir);
void BuildScriptDisplayNames(Scripts* pScripts);
void LoadScripts(Scripts* pScripts);
public:
ScriptConfig();
~ScriptConfig();
Scripts* GetScripts() { return &m_Scripts; }
bool LoadConfig(Options::OptEntries* pOptEntries);
bool SaveConfig(Options::OptEntries* pOptEntries);
bool LoadConfigTemplates(ConfigTemplates* pConfigTemplates);
ConfigTemplates* GetConfigTemplates() { return &m_ConfigTemplates; }
};
extern ScriptConfig* g_pScriptConfig;
#endif

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2013-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -47,9 +47,10 @@
#include "Util.h"
#include "FeedFile.h"
#include "FeedFilter.h"
#include "FeedScript.h"
#include "DiskState.h"
#include "DupeCoordinator.h"
extern Options* g_pOptions;
extern DiskState* g_pDiskState;
FeedCoordinator::FeedCacheItem::FeedCacheItem(const char* szUrl, int iCacheTimeSec,const char* szCacheId,
time_t tLastUsage, FeedItemInfos* pFeedItemInfos)
@@ -69,42 +70,6 @@ FeedCoordinator::FeedCacheItem::~FeedCacheItem()
m_pFeedItemInfos->Release();
}
FeedCoordinator::FilterHelper::FilterHelper()
{
m_pSeasonEpisodeRegEx = NULL;
}
FeedCoordinator::FilterHelper::~FilterHelper()
{
delete m_pSeasonEpisodeRegEx;
}
void FeedCoordinator::FilterHelper::CalcDupeStatus(const char* szTitle, const char* szDupeKey, char* szStatusBuf, int iBufLen)
{
const char* szDupeStatusName[] = { "", "QUEUED", "DOWNLOADING", "3", "SUCCESS", "5", "6", "7", "WARNING",
"9", "10", "11", "12", "13", "14", "15", "FAILURE" };
char szStatuses[200];
szStatuses[0] = '\0';
DownloadQueue* pDownloadQueue = DownloadQueue::Lock();
DupeCoordinator::EDupeStatus eDupeStatus = g_pDupeCoordinator->GetDupeStatus(pDownloadQueue, szTitle, szDupeKey);
DownloadQueue::Unlock();
for (int i = 1; i <= (int)DupeCoordinator::dsFailure; i = i << 1)
{
if (eDupeStatus & i)
{
if (*szStatuses)
{
strcat(szStatuses, ",");
}
strcat(szStatuses, szDupeStatusName[i]);
}
}
strncpy(szStatusBuf, szStatuses, iBufLen);
}
FeedCoordinator::FeedCoordinator()
{
debug("Creating FeedCoordinator");
@@ -396,9 +361,6 @@ void FeedCoordinator::FeedCompleted(FeedDownloader* pFeedDownloader)
{
if (!pFeedInfo->GetPreview())
{
FeedScriptController::ExecuteScripts(
!Util::EmptyStr(pFeedInfo->GetFeedScript()) ? pFeedInfo->GetFeedScript(): g_pOptions->GetFeedScript(),
pFeedInfo->GetOutputFilename(), pFeedInfo->GetID());
FeedFile* pFeedFile = FeedFile::Create(pFeedInfo->GetOutputFilename());
remove(pFeedInfo->GetOutputFilename());
@@ -452,7 +414,6 @@ void FeedCoordinator::FilterFeed(FeedInfo* pFeedInfo, FeedItemInfos* pFeedItemIn
pFeedItemInfo->SetAddCategory(pFeedInfo->GetCategory());
pFeedItemInfo->SetDupeScore(0);
pFeedItemInfo->SetDupeMode(dmScore);
pFeedItemInfo->SetFeedFilterHelper(&m_FilterHelper);
pFeedItemInfo->BuildDupeKey(NULL, NULL);
if (pFeedFilter)
{
@@ -479,7 +440,7 @@ void FeedCoordinator::ProcessFeed(FeedInfo* pFeedInfo, FeedItemInfos* pFeedItemI
{
FeedHistoryInfo* pFeedHistoryInfo = m_FeedHistory.Find(pFeedItemInfo->GetUrl());
FeedHistoryInfo::EStatus eStatus = FeedHistoryInfo::hsUnknown;
if (bFirstFetch && pFeedInfo->GetBacklog())
if (bFirstFetch)
{
eStatus = FeedHistoryInfo::hsBacklog;
}
@@ -518,7 +479,6 @@ NZBInfo* FeedCoordinator::CreateNZBInfo(FeedInfo* pFeedInfo, FeedItemInfo* pFeed
NZBInfo* pNZBInfo = new NZBInfo();
pNZBInfo->SetKind(NZBInfo::nkUrl);
pNZBInfo->SetFeedID(pFeedInfo->GetID());
pNZBInfo->SetURL(pFeedItemInfo->GetUrl());
// add .nzb-extension if not present
@@ -557,19 +517,18 @@ bool FeedCoordinator::ViewFeed(int iID, FeedItemInfos** ppFeedItemInfos)
FeedInfo* pFeedInfo = m_Feeds.at(iID - 1);
return PreviewFeed(pFeedInfo->GetID(), pFeedInfo->GetName(), pFeedInfo->GetUrl(), pFeedInfo->GetFilter(),
pFeedInfo->GetBacklog(), pFeedInfo->GetPauseNzb(), pFeedInfo->GetCategory(),
pFeedInfo->GetPriority(), pFeedInfo->GetInterval(), pFeedInfo->GetFeedScript(), 0, NULL, ppFeedItemInfos);
return PreviewFeed(pFeedInfo->GetName(), pFeedInfo->GetUrl(), pFeedInfo->GetFilter(),
pFeedInfo->GetPauseNzb(), pFeedInfo->GetCategory(), pFeedInfo->GetPriority(),
0, NULL, ppFeedItemInfos);
}
bool FeedCoordinator::PreviewFeed(int iID, const char* szName, const char* szUrl, const char* szFilter,
bool bBacklog, bool bPauseNzb, const char* szCategory, int iPriority, int iInterval, const char* szFeedScript,
bool FeedCoordinator::PreviewFeed(const char* szName, const char* szUrl, const char* szFilter,
bool bPauseNzb, const char* szCategory, int iPriority,
int iCacheTimeSec, const char* szCacheId, FeedItemInfos** ppFeedItemInfos)
{
debug("Preview feed %s", szName);
FeedInfo* pFeedInfo = new FeedInfo(iID, szName, szUrl, bBacklog, iInterval,
szFilter, bPauseNzb, szCategory, iPriority, szFeedScript);
FeedInfo* pFeedInfo = new FeedInfo(0, szName, szUrl, 0, szFilter, bPauseNzb, szCategory, iPriority);
pFeedInfo->SetPreview(true);
FeedItemInfos* pFeedItemInfos = NULL;
@@ -624,9 +583,6 @@ bool FeedCoordinator::PreviewFeed(int iID, const char* szName, const char* szUrl
if (pFeedInfo->GetStatus() == FeedInfo::fsFinished)
{
FeedScriptController::ExecuteScripts(
!Util::EmptyStr(pFeedInfo->GetFeedScript()) ? pFeedInfo->GetFeedScript(): g_pOptions->GetFeedScript(),
pFeedInfo->GetOutputFilename(), pFeedInfo->GetID());
pFeedFile = FeedFile::Create(pFeedInfo->GetOutputFilename());
}
@@ -645,7 +601,7 @@ bool FeedCoordinator::PreviewFeed(int iID, const char* szName, const char* szUrl
for (FeedItemInfos::iterator it = pFeedItemInfos->begin(); it != pFeedItemInfos->end(); it++)
{
FeedItemInfo* pFeedItemInfo = *it;
pFeedItemInfo->SetStatus(bFirstFetch && pFeedInfo->GetBacklog() ? FeedItemInfo::isBacklog : FeedItemInfo::isNew);
pFeedItemInfo->SetStatus(bFirstFetch ? FeedItemInfo::isBacklog : FeedItemInfo::isNew);
FeedHistoryInfo* pFeedHistoryInfo = m_FeedHistory.Find(pFeedItemInfo->GetUrl());
if (pFeedHistoryInfo)
{

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2013-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -36,7 +36,6 @@
#include "DownloadInfo.h"
#include "FeedInfo.h"
#include "Observer.h"
#include "Util.h"
class FeedDownloader;
@@ -72,17 +71,6 @@ private:
FeedItemInfos* GetFeedItemInfos() { return m_pFeedItemInfos; }
};
class FilterHelper : public FeedFilterHelper
{
private:
RegEx* m_pSeasonEpisodeRegEx;
public:
FilterHelper();
~FilterHelper();
virtual RegEx** GetSeasonEpisodeRegEx() { return &m_pSeasonEpisodeRegEx; };
virtual void CalcDupeStatus(const char* szTitle, const char* szDupeKey, char* szStatusBuf, int iBufLen);
};
typedef std::deque<FeedCacheItem*> FeedCache;
typedef std::list<FeedDownloader*> ActiveDownloads;
@@ -95,7 +83,6 @@ private:
bool m_bForce;
bool m_bSave;
FeedCache m_FeedCache;
FilterHelper m_FilterHelper;
void StartFeedDownload(FeedInfo* pFeedInfo, bool bForce);
void FeedCompleted(FeedDownloader* pFeedDownloader);
@@ -118,8 +105,8 @@ public:
virtual void Stop();
void Update(Subject* pCaller, void* pAspect);
void AddFeed(FeedInfo* pFeedInfo);
bool PreviewFeed(int iID, const char* szName, const char* szUrl, const char* szFilter, bool bBacklog,
bool bPauseNzb, const char* szCategory, int iPriority, int iInterval, const char* szFeedScript,
bool PreviewFeed(const char* szName, const char* szUrl, const char* szFilter,
bool bPauseNzb, const char* szCategory, int iPriority,
int iCacheTimeSec, const char* szCacheId, FeedItemInfos** ppFeedItemInfos);
bool ViewFeed(int iID, FeedItemInfos** ppFeedItemInfos);
void FetchFeed(int iID);
@@ -127,8 +114,6 @@ public:
Feeds* GetFeeds() { return &m_Feeds; }
};
extern FeedCoordinator* g_pFeedCoordinator;
class FeedDownloader : public WebDownloader
{
private:

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2013-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -51,6 +51,8 @@ using namespace MSXML;
#include "Options.h"
#include "Util.h"
extern Options* g_pOptions;
FeedFile::FeedFile(const char* szFileName)
{
debug("Creating FeedFile");
@@ -244,13 +246,7 @@ bool FeedFile::ParseFeed(IUnknown* nzb)
if (tag)
{
_bstr_t description(tag->Gettext());
// cleanup CDATA
char* szDescription = strdup((const char*)description);
WebUtil::XmlStripTags(szDescription);
WebUtil::XmlDecode(szDescription);
WebUtil::XmlRemoveEntities(szDescription);
pFeedItemInfo->SetDescription(szDescription);
free(szDescription);
pFeedItemInfo->SetDescription(description);
}
//<enclosure url="http://myindexer.com/fetch/9eeb264aecce961a6e0d" length="150263340" type="application/x-nzb" />
@@ -498,13 +494,7 @@ void FeedFile::Parse_EndElement(const char *name)
}
else if (!strcmp("description", name) && m_pFeedItemInfo)
{
// cleanup CDATA
char* szDescription = strdup(m_szTagContent);
WebUtil::XmlStripTags(szDescription);
WebUtil::XmlDecode(szDescription);
WebUtil::XmlRemoveEntities(szDescription);
m_pFeedItemInfo->SetDescription(szDescription);
free(szDescription);
m_pFeedItemInfo->SetDescription(m_szTagContent);
ResetTagContent();
}
else if (!strcmp("pubDate", name) && m_pFeedItemInfo)

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2013 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -330,7 +330,7 @@ bool FeedFilter::Term::Compile(char* szToken)
}
/*
* If pFeedItemInfo is NULL, only field name is validated
* If pFeedItemInfo is NULL, only field type info is returned
*/
bool FeedFilter::Term::GetFieldData(const char* szField, FeedItemInfo* pFeedItemInfo,
const char** StrValue, long long* IntValue)
@@ -408,11 +408,6 @@ bool FeedFilter::Term::GetFieldData(const char* szField, FeedItemInfo* pFeedItem
*IntValue = pFeedItemInfo ? pFeedItemInfo->GetDupeScore() : 0;
return true;
}
else if (!strcasecmp(szField, "dupestatus"))
{
*StrValue = pFeedItemInfo ? pFeedItemInfo->GetDupeStatus() : NULL;
return true;
}
else if (!strncasecmp(szField, "attr-", 5))
{
if (pFeedItemInfo)
@@ -521,7 +516,7 @@ bool FeedFilter::Term::ParseNumericParam(const char* szParam)
m_bFloat = strchr(szParam, '.');
const char* p;
for (p = szParam; *p && ((*p >= '0' && *p <='9') || *p == '.' || *p == '-') ; p++) ;
for (p = szParam; *p && ((*p >= '0' && *p <='9') || *p == '.') ; p++) ;
if (*p)
{
return false;

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2013 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -41,18 +41,17 @@
#include "FeedInfo.h"
#include "Util.h"
FeedInfo::FeedInfo(int iID, const char* szName, const char* szUrl, bool bBacklog, int iInterval,
const char* szFilter, bool bPauseNzb, const char* szCategory, int iPriority, const char* szFeedScript)
FeedInfo::FeedInfo(int iID, const char* szName, const char* szUrl, int iInterval,
const char* szFilter, bool bPauseNzb, const char* szCategory, int iPriority)
{
m_iID = iID;
m_szName = strdup(szName ? szName : "");
m_szUrl = strdup(szUrl ? szUrl : "");
m_szFilter = strdup(szFilter ? szFilter : "");
m_bBacklog = bBacklog;
m_iFilterHash = Util::HashBJ96(m_szFilter, strlen(m_szFilter), 0);
m_iFilterHash = Util::HashBJ96(szFilter, strlen(szFilter), 0);
m_szCategory = strdup(szCategory ? szCategory : "");
m_iInterval = iInterval;
m_szFeedScript = strdup(szFeedScript ? szFeedScript : "");
m_bPauseNzb = bPauseNzb;
m_iPriority = iPriority;
m_tLastUpdate = 0;
@@ -70,7 +69,6 @@ FeedInfo::~FeedInfo()
free(m_szFilter);
free(m_szCategory);
free(m_szOutputFilename);
free(m_szFeedScript);
}
void FeedInfo::SetOutputFilename(const char* szOutputFilename)
@@ -123,7 +121,7 @@ FeedItemInfo::Attr* FeedItemInfo::Attributes::Find(const char* szName)
FeedItemInfo::FeedItemInfo()
{
m_pFeedFilterHelper = NULL;
m_pSharedFeedData = NULL;
m_szTitle = NULL;
m_szFilename = NULL;
m_szUrl = NULL;
@@ -147,7 +145,6 @@ FeedItemInfo::FeedItemInfo()
m_szDupeKey = NULL;
m_iDupeScore = 0;
m_eDupeMode = dmScore;
m_szDupeStatus = NULL;
}
FeedItemInfo::~FeedItemInfo()
@@ -161,7 +158,6 @@ FeedItemInfo::~FeedItemInfo()
free(m_szEpisode);
free(m_szAddCategory);
free(m_szDupeKey);
free(m_szDupeStatus);
}
void FeedItemInfo::SetTitle(const char* szTitle)
@@ -300,24 +296,20 @@ void FeedItemInfo::ParseSeasonEpisode()
{
m_bSeasonEpisodeParsed = true;
RegEx** ppRegEx = m_pFeedFilterHelper->GetSeasonEpisodeRegEx();
if (!*ppRegEx)
{
*ppRegEx = new RegEx("[^[:alnum:]]s?([0-9]+)[ex]([0-9]+(-?e[0-9]+)?)[^[:alnum:]]", 10);
}
RegEx* pRegEx = m_pSharedFeedData->GetSeasonEpisodeRegEx();
if ((*ppRegEx)->Match(m_szTitle))
if (pRegEx->Match(m_szTitle))
{
char szRegValue[100];
char szValue[100];
snprintf(szValue, 100, "S%02d", atoi(m_szTitle + (*ppRegEx)->GetMatchStart(1)));
snprintf(szValue, 100, "S%02d", atoi(m_szTitle + pRegEx->GetMatchStart(1)));
szValue[100-1] = '\0';
SetSeason(szValue);
int iLen = (*ppRegEx)->GetMatchLen(2);
int iLen = pRegEx->GetMatchLen(2);
iLen = iLen < 99 ? iLen : 99;
strncpy(szRegValue, m_szTitle + (*ppRegEx)->GetMatchStart(2), (*ppRegEx)->GetMatchLen(2));
strncpy(szRegValue, m_szTitle + pRegEx->GetMatchStart(2), pRegEx->GetMatchLen(2));
szRegValue[iLen] = '\0';
snprintf(szValue, 100, "E%s", szRegValue);
szValue[100-1] = '\0';
@@ -327,19 +319,6 @@ void FeedItemInfo::ParseSeasonEpisode()
}
}
const char* FeedItemInfo::GetDupeStatus()
{
if (!m_szDupeStatus)
{
char szStatuses[200];
szStatuses[0] = '\0';
m_pFeedFilterHelper->CalcDupeStatus(m_szTitle, m_szDupeKey, szStatuses, sizeof(szStatuses));
m_szDupeStatus = strdup(szStatuses);
}
return m_szDupeStatus;
}
FeedHistoryInfo::FeedHistoryInfo(const char* szUrl, FeedHistoryInfo::EStatus eStatus, time_t tLastSeen)
{
@@ -436,4 +415,26 @@ void FeedItemInfos::Release()
void FeedItemInfos::Add(FeedItemInfo* pFeedItemInfo)
{
push_back(pFeedItemInfo);
pFeedItemInfo->SetSharedFeedData(&m_SharedFeedData);
}
SharedFeedData::SharedFeedData()
{
m_pSeasonEpisodeRegEx = NULL;
}
SharedFeedData::~SharedFeedData()
{
delete m_pSeasonEpisodeRegEx;
}
RegEx* SharedFeedData::GetSeasonEpisodeRegEx()
{
if (!m_pSeasonEpisodeRegEx)
{
m_pSeasonEpisodeRegEx = new RegEx("[^[:alnum:]]s?([0-9]+)[ex]([0-9]+(-?e[0-9]+)?)[^[:alnum:]]", 10);
}
return m_pSeasonEpisodeRegEx;
}

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2013 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -53,7 +53,6 @@ private:
unsigned int m_iFilterHash;
bool m_bPauseNzb;
char* m_szCategory;
char* m_szFeedScript;
int m_iPriority;
time_t m_tLastUpdate;
bool m_bPreview;
@@ -61,12 +60,10 @@ private:
char* m_szOutputFilename;
bool m_bFetch;
bool m_bForce;
bool m_bBacklog;
public:
FeedInfo(int iID, const char* szName, const char* szUrl, bool bBacklog, int iInterval,
const char* szFilter, bool bPauseNzb, const char* szCategory, int iPriority,
const char* szFeedScript);
FeedInfo(int iID, const char* szName, const char* szUrl, int iInterval,
const char* szFilter, bool bPauseNzb, const char* szCategory, int iPriority);
~FeedInfo();
int GetID() { return m_iID; }
const char* GetName() { return m_szName; }
@@ -77,7 +74,6 @@ public:
bool GetPauseNzb() { return m_bPauseNzb; }
const char* GetCategory() { return m_szCategory; }
int GetPriority() { return m_iPriority; }
const char* GetFeedScript() { return m_szFeedScript; }
time_t GetLastUpdate() { return m_tLastUpdate; }
void SetLastUpdate(time_t tLastUpdate) { m_tLastUpdate = tLastUpdate; }
bool GetPreview() { return m_bPreview; }
@@ -90,17 +86,19 @@ public:
void SetFetch(bool bFetch) { m_bFetch = bFetch; }
bool GetForce() { return m_bForce; }
void SetForce(bool bForce) { m_bForce = bForce; }
bool GetBacklog() { return m_bBacklog; }
void SetBacklog(bool bBacklog) { m_bBacklog = bBacklog; }
};
typedef std::deque<FeedInfo*> Feeds;
class FeedFilterHelper
class SharedFeedData
{
private:
RegEx* m_pSeasonEpisodeRegEx;
public:
virtual RegEx** GetSeasonEpisodeRegEx() = 0;
virtual void CalcDupeStatus(const char* szTitle, const char* szDupeKey, char* szStatusBuf, int iBufLen) = 0;
SharedFeedData();
~SharedFeedData();
RegEx* GetSeasonEpisodeRegEx();
};
class FeedItemInfo
@@ -167,8 +165,7 @@ private:
char* m_szDupeKey;
int m_iDupeScore;
EDupeMode m_eDupeMode;
char* m_szDupeStatus;
FeedFilterHelper* m_pFeedFilterHelper;
SharedFeedData* m_pSharedFeedData;
Attributes m_Attributes;
int ParsePrefixedInt(const char *szValue);
@@ -177,7 +174,7 @@ private:
public:
FeedItemInfo();
~FeedItemInfo();
void SetFeedFilterHelper(FeedFilterHelper* pFeedFilterHelper) { m_pFeedFilterHelper = pFeedFilterHelper; }
void SetSharedFeedData(SharedFeedData* pSharedFeedData) { m_pSharedFeedData = pSharedFeedData; }
const char* GetTitle() { return m_szTitle; }
void SetTitle(const char* szTitle);
const char* GetFilename() { return m_szFilename; }
@@ -222,7 +219,6 @@ public:
void SetDupeScore(int iDupeScore) { m_iDupeScore = iDupeScore; }
EDupeMode GetDupeMode() { return m_eDupeMode; }
void SetDupeMode(EDupeMode eDupeMode) { m_eDupeMode = eDupeMode; }
const char* GetDupeStatus();
Attributes* GetAttributes() { return &m_Attributes; }
};
@@ -232,6 +228,7 @@ class FeedItemInfos : public FeedItemInfosBase
{
private:
int m_iRefCount;
SharedFeedData m_SharedFeedData;
public:
FeedItemInfos();

View File

@@ -88,7 +88,7 @@ void ColoredFrontend::PrintStatus()
char szDownloadLimit[128];
if (m_iDownloadLimit > 0)
{
sprintf(szDownloadLimit, ", Limit %i KB/s", m_iDownloadLimit / 1024);
sprintf(szDownloadLimit, ", Limit %.0f KB/s", (float)m_iDownloadLimit / 1024.0);
}
else
{
@@ -112,12 +112,10 @@ void ColoredFrontend::PrintStatus()
const char* szControlSeq = "\033[K";
#endif
char szFileSize[20];
char szCurrendSpeed[20];
snprintf(tmp, 1024, " %d threads, %s, %s remaining%s%s%s%s%s\n",
m_iThreadCount, Util::FormatSpeed(szCurrendSpeed, sizeof(szCurrendSpeed), iCurrentDownloadSpeed),
Util::FormatSize(szFileSize, sizeof(szFileSize), m_lRemainingSize),
timeString, szPostStatus, m_bPauseDownload ? (m_bStandBy ? ", Paused" : ", Pausing") : "",
snprintf(tmp, 1024, " %d threads, %.*f KB/s, %.2f MB remaining%s%s%s%s%s\n",
m_iThreadCount, (iCurrentDownloadSpeed >= 10*1024 ? 0 : 1), (float)iCurrentDownloadSpeed / 1024.0,
(float)(Util::Int64ToFloat(m_lRemainingSize) / 1024.0 / 1024.0), timeString, szPostStatus,
m_bPauseDownload ? (m_bStandBy ? ", Paused" : ", Pausing") : "",
szDownloadLimit, szControlSeq);
tmp[1024-1] = '\0';
printf("%s", tmp);

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -51,6 +51,9 @@
#include "Util.h"
#include "StatMeter.h"
extern Options* g_pOptions;
extern StatMeter* g_pStatMeter;
Frontend::Frontend()
{
debug("Creating Frontend");
@@ -82,8 +85,7 @@ bool Frontend::PrepareData()
}
if (!RequestMessages() || ((m_bSummary || m_bFileList) && !RequestFileList()))
{
const char* szControlIP = !strcmp(g_pOptions->GetControlIP(), "0.0.0.0") ? "127.0.0.1" : g_pOptions->GetControlIP();
printf("\nUnable to send request to nzbget-server at %s (port %i) \n", szControlIP, g_pOptions->GetControlPort());
printf("\nUnable to send request to nzbget-server at %s (port %i) \n", g_pOptions->GetControlIP(), g_pOptions->GetControlPort());
Stop();
return false;
}
@@ -117,7 +119,11 @@ void Frontend::FreeData()
{
if (IsRemoteMode())
{
m_RemoteMessages.Clear();
for (Log::Messages::iterator it = m_RemoteMessages.begin(); it != m_RemoteMessages.end(); it++)
{
delete *it;
}
m_RemoteMessages.clear();
DownloadQueue* pDownloadQueue = DownloadQueue::Lock();
pDownloadQueue->GetQueue()->Clear();
@@ -125,7 +131,7 @@ void Frontend::FreeData()
}
}
MessageList* Frontend::LockMessages()
Log::Messages* Frontend::LockMessages()
{
if (IsRemoteMode())
{
@@ -216,8 +222,7 @@ void Frontend::InitMessageBase(SNZBRequestBase* pMessageBase, int iRequest, int
bool Frontend::RequestMessages()
{
const char* szControlIP = !strcmp(g_pOptions->GetControlIP(), "0.0.0.0") ? "127.0.0.1" : g_pOptions->GetControlIP();
Connection connection(szControlIP, g_pOptions->GetControlPort(), false);
Connection connection(g_pOptions->GetControlIP(), g_pOptions->GetControlPort(), false);
bool OK = connection.Connect();
if (!OK)
@@ -288,8 +293,7 @@ bool Frontend::RequestMessages()
bool Frontend::RequestFileList()
{
const char* szControlIP = !strcmp(g_pOptions->GetControlIP(), "0.0.0.0") ? "127.0.0.1" : g_pOptions->GetControlIP();
Connection connection(szControlIP, g_pOptions->GetControlPort(), false);
Connection connection(g_pOptions->GetControlIP(), g_pOptions->GetControlPort(), false);
bool OK = connection.Connect();
if (!OK)

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -36,7 +36,7 @@
class Frontend : public Thread
{
private:
MessageList m_RemoteMessages;
Log::Messages m_RemoteMessages;
bool RequestMessages();
bool RequestFileList();
@@ -62,7 +62,7 @@ protected:
bool PrepareData();
void FreeData();
MessageList* LockMessages();
Log::Messages* LockMessages();
void UnlockMessages();
DownloadQueue* LockQueue();
void UnlockQueue();

View File

@@ -2,7 +2,7 @@
* This file if part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -79,7 +79,7 @@ void LoggableFrontend::Update()
BeforePrint();
MessageList* pMessages = LockMessages();
Log::Messages* pMessages = LockMessages();
if (!pMessages->empty())
{
Message* pFirstMessage = pMessages->front();

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -43,7 +43,6 @@
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#ifndef WIN32
#include <unistd.h>
#endif
@@ -73,6 +72,7 @@ void curses_clear()
#undef clear
#endif
extern Options* g_pOptions;
extern void ExitProc();
static const int NCURSES_COLORPAIR_TEXT = 1;
@@ -136,7 +136,7 @@ NCursesFrontend::NCursesFrontend()
m_bShowNZBname = g_pOptions->GetCursesNZBName();
m_bShowTimestamp = g_pOptions->GetCursesTime();
m_bGroupFiles = g_pOptions->GetCursesGroup();
m_QueueWindowPercentage = 50;
m_QueueWindowPercentage = 0.5f;
m_iDataUpdatePos = 0;
m_bUpdateNextTime = false;
m_iLastEditEntry = -1;
@@ -362,7 +362,7 @@ void NCursesFrontend::CalcWindowSizes()
int iQueueSize = CalcQueueSize();
m_iQueueWinTop = 0;
m_iQueueWinHeight = (m_iScreenHeight - 2) * m_QueueWindowPercentage / 100;
m_iQueueWinHeight = (int)((float) (m_iScreenHeight - 2) * m_QueueWindowPercentage);
if (m_iQueueWinHeight - 1 > iQueueSize)
{
m_iQueueWinHeight = iQueueSize > 0 ? iQueueSize + 1 : 1 + 1;
@@ -503,7 +503,7 @@ void NCursesFrontend::PrintMessages()
int iLine = iLineNr + m_iMessagesWinClientHeight - 1;
int iLinesToPrint = m_iMessagesWinClientHeight;
MessageList* pMessages = LockMessages();
Log::Messages* pMessages = LockMessages();
// print messages from bottom
for (int i = (int)pMessages->size() - 1; i >= 0 && iLinesToPrint > 0; i--)
@@ -625,7 +625,7 @@ void NCursesFrontend::PrintStatus()
char szDownloadLimit[128];
if (m_iDownloadLimit > 0)
{
sprintf(szDownloadLimit, ", Limit %i KB/s", m_iDownloadLimit / 1024);
sprintf(szDownloadLimit, ", Limit %.0f KB/s", (float)m_iDownloadLimit / 1024.0);
}
else
{
@@ -642,16 +642,13 @@ void NCursesFrontend::PrintStatus()
szPostStatus[0] = 0;
}
char szCurrentSpeed[20];
char szAverageSpeed[20];
char szRemainingSize[20];
int iAverageSpeed = (int)(m_iDnTimeSec > 0 ? m_iAllBytes / m_iDnTimeSec : 0);
float fAverageSpeed = (float)(Util::Int64ToFloat(m_iDnTimeSec > 0 ? m_iAllBytes / m_iDnTimeSec : 0) / 1024.0);
snprintf(tmp, MAX_SCREEN_WIDTH, " %d threads, %s, %s remaining%s%s%s%s, Avg. %s",
m_iThreadCount, Util::FormatSpeed(szCurrentSpeed, sizeof(szCurrentSpeed), iCurrentDownloadSpeed),
Util::FormatSize(szRemainingSize, sizeof(szRemainingSize), m_lRemainingSize),
timeString, szPostStatus, m_bPauseDownload ? (m_bStandBy ? ", Paused" : ", Pausing") : "",
szDownloadLimit, Util::FormatSpeed(szAverageSpeed, sizeof(szAverageSpeed), iAverageSpeed));
snprintf(tmp, MAX_SCREEN_WIDTH, " %d threads, %.*f KB/s, %.2f MB remaining%s%s%s%s, Avg. %.*f KB/s",
m_iThreadCount, (iCurrentDownloadSpeed >= 10*1024 ? 0 : 1), (float)iCurrentDownloadSpeed / 1024.0,
(float)(Util::Int64ToFloat(m_lRemainingSize) / 1024.0 / 1024.0), timeString, szPostStatus,
m_bPauseDownload ? (m_bStandBy ? ", Paused" : ", Pausing") : "",
szDownloadLimit, (fAverageSpeed >= 10 ? 0 : 1), fAverageSpeed);
tmp[MAX_SCREEN_WIDTH - 1] = '\0';
PlotLine(tmp, iStatusRow, 0, NCURSES_COLORPAIR_STATUS);
}
@@ -778,13 +775,15 @@ void NCursesFrontend::PrintFileQueue()
if (iFileNum > 0)
{
char szRemaining[20];
Util::FormatFileSize(szRemaining, sizeof(szRemaining), lRemaining);
char szUnpaused[20];
Util::FormatFileSize(szUnpaused, sizeof(szUnpaused), lRemaining - lPaused);
char szBuffer[MAX_SCREEN_WIDTH];
snprintf(szBuffer, sizeof(szBuffer), " %sFiles for downloading - %i / %i files in queue - %s / %s",
m_bUseColor ? "" : "*** ", iFileNum,
iFileNum - iPausedFiles,
Util::FormatSize(szRemaining, sizeof(szRemaining), lRemaining),
Util::FormatSize(szUnpaused, sizeof(szUnpaused), lRemaining - lPaused));
iFileNum - iPausedFiles, szRemaining, szUnpaused);
szBuffer[MAX_SCREEN_WIDTH - 1] = '\0';
PrintTopHeader(szBuffer, m_iQueueWinTop, true);
}
@@ -837,7 +836,7 @@ void NCursesFrontend::PrintFilename(FileInfo * pFileInfo, int iRow, bool bSelect
szCompleted[0] = '\0';
if (pFileInfo->GetRemainingSize() < pFileInfo->GetSize())
{
sprintf(szCompleted, ", %i%%", (int)(100 - pFileInfo->GetRemainingSize() * 100 / pFileInfo->GetSize()));
sprintf(szCompleted, ", %i%%", (int)(100 - Util::Int64ToFloat(pFileInfo->GetRemainingSize()) * 100.0 / Util::Int64ToFloat(pFileInfo->GetSize())));
}
char szNZBNiceName[1024];
@@ -853,11 +852,10 @@ void NCursesFrontend::PrintFilename(FileInfo * pFileInfo, int iRow, bool bSelect
szNZBNiceName[0] = '\0';
}
char szSize[20];
char szBuffer[MAX_SCREEN_WIDTH];
snprintf(szBuffer, MAX_SCREEN_WIDTH, "%s%i%s%s%s %s%s (%s%s)%s", Brace1, pFileInfo->GetID(),
Brace2, szPriority, szDownloading, szNZBNiceName, pFileInfo->GetFilename(),
Util::FormatSize(szSize, sizeof(szSize), pFileInfo->GetSize()),
snprintf(szBuffer, MAX_SCREEN_WIDTH, "%s%i%s%s%s %s%s (%.2f MB%s)%s", Brace1, pFileInfo->GetID(),
Brace2, szPriority, szDownloading, szNZBNiceName, pFileInfo->GetFilename(),
(float)(Util::Int64ToFloat(pFileInfo->GetSize()) / 1024.0 / 1024.0),
szCompleted, pFileInfo->GetPaused() ? " (paused)" : "");
szBuffer[MAX_SCREEN_WIDTH - 1] = '\0';
@@ -968,10 +966,10 @@ void NCursesFrontend::PrintGroupQueue()
}
char szRemaining[20];
Util::FormatSize(szRemaining, sizeof(szRemaining), lRemaining);
Util::FormatFileSize(szRemaining, sizeof(szRemaining), lRemaining);
char szUnpaused[20];
Util::FormatSize(szUnpaused, sizeof(szUnpaused), lRemaining - lPaused);
Util::FormatFileSize(szUnpaused, sizeof(szUnpaused), lRemaining - lPaused);
char szBuffer[MAX_SCREEN_WIDTH];
snprintf(szBuffer, sizeof(szBuffer), " %sNZBs for downloading - %i NZBs in queue - %s / %s",
@@ -1013,7 +1011,7 @@ void NCursesFrontend::PrintGroupname(NZBInfo* pNZBInfo, int iRow, bool bSelected
long long lUnpausedRemainingSize = pNZBInfo->GetRemainingSize() - pNZBInfo->GetPausedSize();
char szRemaining[20];
Util::FormatSize(szRemaining, sizeof(szRemaining), lUnpausedRemainingSize);
Util::FormatFileSize(szRemaining, sizeof(szRemaining), lUnpausedRemainingSize);
char szPriority[100];
szPriority[0] = '\0';
@@ -1047,7 +1045,7 @@ void NCursesFrontend::PrintGroupname(NZBInfo* pNZBInfo, int iRow, bool bSelected
szFiles[20-1] = '\0';
char szTotal[20];
Util::FormatSize(szTotal, sizeof(szTotal), pNZBInfo->GetSize());
Util::FormatFileSize(szTotal, sizeof(szTotal), pNZBInfo->GetSize());
char szNameWithIds[1024];
snprintf(szNameWithIds, 1024, "%c%i%c%s%s %s", chBrace1, pNZBInfo->GetID(), chBrace2,
@@ -1060,7 +1058,7 @@ void NCursesFrontend::PrintGroupname(NZBInfo* pNZBInfo, int iRow, bool bSelected
if (pNZBInfo->GetPausedSize() > 0 && lUnpausedRemainingSize == 0)
{
snprintf(szTime, 100, "[paused]");
Util::FormatSize(szRemaining, sizeof(szRemaining), pNZBInfo->GetRemainingSize());
Util::FormatFileSize(szRemaining, sizeof(szRemaining), pNZBInfo->GetRemainingSize());
}
else if (iCurrentDownloadSpeed > 0 && !m_bPauseDownload)
{
@@ -1258,17 +1256,17 @@ void NCursesFrontend::UpdateInput(int initialKey)
break;
case 'w':
// swicth window sizes
if (m_QueueWindowPercentage == 50)
if (m_QueueWindowPercentage == 0.5)
{
m_QueueWindowPercentage = 100;
m_QueueWindowPercentage = 1;
}
else if (m_QueueWindowPercentage == 100 && m_eInputMode != eEditQueue)
else if (m_QueueWindowPercentage == 1 && m_eInputMode != eEditQueue)
{
m_QueueWindowPercentage = 0;
}
else
{
m_QueueWindowPercentage = 50;
m_QueueWindowPercentage = 0.5;
}
CalcWindowSizes();
SetCurrentQueueEntry(m_iSelectedQueueEntry);
@@ -1303,7 +1301,7 @@ void NCursesFrontend::UpdateInput(int initialKey)
m_eInputMode = eEditQueue;
if (m_QueueWindowPercentage == 0)
{
m_QueueWindowPercentage = 50;
m_QueueWindowPercentage = 0.5;
}
return;
}

View File

@@ -85,7 +85,7 @@ private:
bool m_bShowNZBname;
bool m_bShowTimestamp;
bool m_bGroupFiles;
int m_QueueWindowPercentage;
float m_QueueWindowPercentage;
#ifdef WIN32
void init_pair(int iColorNumber, WORD wForeColor, WORD wBackColor);

View File

File diff suppressed because it is too large Load Diff

View File

@@ -1,169 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifndef COMMANDLINEPARSER_H
#define COMMANDLINEPARSER_H
#include <vector>
#include <list>
#include <time.h>
class CommandLineParser
{
public:
enum EClientOperation
{
opClientNoOperation,
opClientRequestDownload,
opClientRequestListFiles,
opClientRequestListGroups,
opClientRequestListStatus,
opClientRequestSetRate,
opClientRequestDumpDebug,
opClientRequestEditQueue,
opClientRequestLog,
opClientRequestShutdown,
opClientRequestReload,
opClientRequestVersion,
opClientRequestPostQueue,
opClientRequestWriteLog,
opClientRequestScanSync,
opClientRequestScanAsync,
opClientRequestDownloadPause,
opClientRequestDownloadUnpause,
opClientRequestPostPause,
opClientRequestPostUnpause,
opClientRequestScanPause,
opClientRequestScanUnpause,
opClientRequestHistory,
opClientRequestHistoryAll
};
enum EMatchMode
{
mmID = 1,
mmName,
mmRegEx
};
typedef std::vector<char*> NameList;
private:
bool m_bNoConfig;
char* m_szConfigFilename;
// Parsed command-line parameters
bool m_bErrors;
bool m_bPrintVersion;
bool m_bPrintUsage;
bool m_bServerMode;
bool m_bDaemonMode;
bool m_bRemoteClientMode;
EClientOperation m_eClientOperation;
NameList m_OptionList;
int m_iEditQueueAction;
int m_iEditQueueOffset;
int* m_pEditQueueIDList;
int m_iEditQueueIDCount;
NameList m_EditQueueNameList;
EMatchMode m_EMatchMode;
char* m_szEditQueueText;
char* m_szArgFilename;
char* m_szAddCategory;
int m_iAddPriority;
bool m_bAddPaused;
char* m_szAddNZBFilename;
char* m_szLastArg;
bool m_bPrintOptions;
bool m_bAddTop;
char* m_szAddDupeKey;
int m_iAddDupeScore;
int m_iAddDupeMode;
int m_iSetRate;
int m_iLogLines;
int m_iWriteLogKind;
bool m_bTestBacktrace;
bool m_bWebGet;
char* m_szWebGetFilename;
bool m_bSigVerify;
char* m_szPubKeyFilename;
char* m_szSigFilename;
bool m_bPauseDownload;
void InitCommandLine(int argc, const char* argv[]);
void InitFileArg(int argc, const char* argv[]);
void ParseFileIDList(int argc, const char* argv[], int optind);
void ParseFileNameList(int argc, const char* argv[], int optind);
bool ParseTime(const char* szTime, int* pHours, int* pMinutes);
void ReportError(const char* szErrMessage);
public:
CommandLineParser(int argc, const char* argv[]);
~CommandLineParser();
void PrintUsage(const char* com);
bool GetErrors() { return m_bErrors; }
bool GetNoConfig() { return m_bNoConfig; }
const char* GetConfigFilename() { return m_szConfigFilename; }
bool GetServerMode() { return m_bServerMode; }
bool GetDaemonMode() { return m_bDaemonMode; }
bool GetRemoteClientMode() { return m_bRemoteClientMode; }
EClientOperation GetClientOperation() { return m_eClientOperation; }
NameList* GetOptionList() { return &m_OptionList; }
int GetEditQueueAction() { return m_iEditQueueAction; }
int GetEditQueueOffset() { return m_iEditQueueOffset; }
int* GetEditQueueIDList() { return m_pEditQueueIDList; }
int GetEditQueueIDCount() { return m_iEditQueueIDCount; }
NameList* GetEditQueueNameList() { return &m_EditQueueNameList; }
EMatchMode GetMatchMode() { return m_EMatchMode; }
const char* GetEditQueueText() { return m_szEditQueueText; }
const char* GetArgFilename() { return m_szArgFilename; }
const char* GetAddCategory() { return m_szAddCategory; }
bool GetAddPaused() { return m_bAddPaused; }
const char* GetLastArg() { return m_szLastArg; }
int GetAddPriority() { return m_iAddPriority; }
char* GetAddNZBFilename() { return m_szAddNZBFilename; }
bool GetAddTop() { return m_bAddTop; }
const char* GetAddDupeKey() { return m_szAddDupeKey; }
int GetAddDupeScore() { return m_iAddDupeScore; }
int GetAddDupeMode() { return m_iAddDupeMode; }
int GetSetRate() { return m_iSetRate; }
int GetLogLines() { return m_iLogLines; }
int GetWriteLogKind() { return m_iWriteLogKind; }
bool GetTestBacktrace() { return m_bTestBacktrace; }
bool GetWebGet() { return m_bWebGet; }
const char* GetWebGetFilename() { return m_szWebGetFilename; }
bool GetSigVerify() { return m_bSigVerify; }
const char* GetPubKeyFilename() { return m_szPubKeyFilename; }
const char* GetSigFilename() { return m_szSigFilename; }
bool GetPrintOptions() { return m_bPrintOptions; }
bool GetPrintVersion() { return m_bPrintVersion; }
bool GetPrintUsage() { return m_bPrintUsage; }
bool GetPauseDownload() const { return m_bPauseDownload; }
};
extern CommandLineParser* g_pCommandLineParser;
#endif

View File

@@ -1,131 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef WIN32
#include "win32.h"
#endif
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#ifdef WIN32
#include <direct.h>
#else
#include <unistd.h>
#endif
#include "nzbget.h"
#include "DiskService.h"
#include "Options.h"
#include "StatMeter.h"
#include "Log.h"
#include "Util.h"
DiskService::DiskService()
{
m_iInterval = 0;
m_bWaitingReported = false;
m_bWaitingRequiredDir = true;
}
void DiskService::ServiceWork()
{
m_iInterval++;
if (m_iInterval == 5)
{
if (!g_pOptions->GetPauseDownload() &&
g_pOptions->GetDiskSpace() > 0 && !g_pStatMeter->GetStandBy())
{
// check free disk space every 1 second
CheckDiskSpace();
}
m_iInterval = 0;
}
if (m_bWaitingRequiredDir)
{
CheckRequiredDir();
}
}
void DiskService::CheckDiskSpace()
{
long long lFreeSpace = Util::FreeDiskSize(g_pOptions->GetDestDir());
if (lFreeSpace > -1 && lFreeSpace / 1024 / 1024 < g_pOptions->GetDiskSpace())
{
warn("Low disk space on %s. Pausing download", g_pOptions->GetDestDir());
g_pOptions->SetPauseDownload(true);
}
if (!Util::EmptyStr(g_pOptions->GetInterDir()))
{
lFreeSpace = Util::FreeDiskSize(g_pOptions->GetInterDir());
if (lFreeSpace > -1 && lFreeSpace / 1024 / 1024 < g_pOptions->GetDiskSpace())
{
warn("Low disk space on %s. Pausing download", g_pOptions->GetInterDir());
g_pOptions->SetPauseDownload(true);
}
}
}
void DiskService::CheckRequiredDir()
{
if (!Util::EmptyStr(g_pOptions->GetRequiredDir()))
{
bool bAllExist = true;
bool bWasWaitingReported = m_bWaitingReported;
// split RequiredDir into tokens
Tokenizer tok(g_pOptions->GetRequiredDir(), ",;");
while (const char* szDir = tok.Next())
{
if (!Util::FileExists(szDir) && !Util::DirectoryExists(szDir))
{
if (!bWasWaitingReported)
{
info("Waiting for required directory %s", szDir);
m_bWaitingReported = true;
}
bAllExist = false;
}
}
if (!bAllExist)
{
return;
}
}
if (m_bWaitingReported)
{
info("All required directories available");
}
g_pOptions->SetTempPauseDownload(false);
g_pOptions->SetTempPausePostprocess(false);
m_bWaitingRequiredDir = false;
}

View File

@@ -1,49 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifndef DISKSERVICE_H
#define DISKSERVICE_H
#include "Service.h"
class DiskService : public Service
{
private:
int m_iInterval;
bool m_bWaitingRequiredDir;
bool m_bWaitingReported;
void CheckDiskSpace();
void CheckRequiredDir();
protected:
virtual int ServiceInterval() { return 200; }
virtual void ServiceWork();
public:
DiskService();
};
#endif

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2013-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -40,46 +40,14 @@
#endif
#include <errno.h>
#ifdef HAVE_OPENSSL
#include <openssl/rsa.h>
#include <openssl/sha.h>
#include <openssl/pem.h>
#endif /* HAVE_OPENSSL */
#include "nzbget.h"
#include "Log.h"
#include "Util.h"
#include "Maintenance.h"
#include "Options.h"
#include "CommandLineParser.h"
extern void ExitProc();
extern int g_iArgumentCount;
extern char* (*g_szArguments)[];
#ifdef HAVE_OPENSSL
class Signature
{
private:
const char* m_szInFilename;
const char* m_szSigFilename;
const char* m_szPubKeyFilename;
unsigned char m_InHash[SHA256_DIGEST_LENGTH];
unsigned char m_Signature[256];
RSA* m_pPubKey;
bool ReadSignature();
bool ComputeInHash();
bool ReadPubKey();
public:
Signature(const char* szInFilename, const char* szSigFilename, const char* szPubKeyFilename);
~Signature();
bool Verify();
};
#endif
extern Options* g_pOptions;
extern Maintenance* g_pMaintenance;
Maintenance::Maintenance()
{
@@ -101,7 +69,7 @@ Maintenance::~Maintenance()
}
}
m_Messages.Clear();
ClearMessages();
free(m_szUpdateScript);
}
@@ -113,7 +81,16 @@ void Maintenance::ResetUpdateController()
m_mutexController.Unlock();
}
MessageList* Maintenance::LockMessages()
void Maintenance::ClearMessages()
{
for (Log::Messages::iterator it = m_Messages.begin(); it != m_Messages.end(); it++)
{
delete *it;
}
m_Messages.clear();
}
Log::Messages* Maintenance::LockMessages()
{
m_mutexLog.Lock();
return &m_Messages;
@@ -124,7 +101,7 @@ void Maintenance::UnlockMessages()
m_mutexLog.Unlock();
}
void Maintenance::AddMessage(Message::EKind eKind, time_t tTime, const char * szText)
void Maintenance::AppendMessage(Message::EKind eKind, time_t tTime, const char * szText)
{
if (tTime == 0)
{
@@ -160,20 +137,7 @@ bool Maintenance::StartUpdate(EBranch eBranch)
return false;
}
// make absolute path
if (m_szUpdateScript[0] != PATH_SEPARATOR
#ifdef WIN32
&& !(strlen(m_szUpdateScript) > 2 && m_szUpdateScript[1] == ':')
#endif
)
{
char szFilename[MAX_PATH + 100];
snprintf(szFilename, sizeof(szFilename), "%s%c%s", g_pOptions->GetAppDir(), PATH_SEPARATOR, m_szUpdateScript);
free(m_szUpdateScript);
m_szUpdateScript = strdup(szFilename);
}
m_Messages.Clear();
ClearMessages();
m_UpdateScriptController = new UpdateScriptController();
m_UpdateScriptController->SetScript(m_szUpdateScript);
@@ -263,21 +227,8 @@ bool Maintenance::ReadPackageInfoStr(const char* szKey, char** pValue)
return true;
}
bool Maintenance::VerifySignature(const char* szInFilename, const char* szSigFilename, const char* szPubKeyFilename)
{
#ifdef HAVE_OPENSSL
Signature signature(szInFilename, szSigFilename, szPubKeyFilename);
return signature.Verify();
#else
return false;
#endif
}
void UpdateScriptController::Run()
{
// the update-script should not be automatically terminated when the program quits
UnregisterRunningScript();
m_iPrefixLen = 0;
PrintMessage(Message::mkInfo, "Executing update-script %s", GetScript());
@@ -289,21 +240,11 @@ void UpdateScriptController::Run()
const char* szBranchName[] = { "STABLE", "TESTING", "DEVEL" };
SetEnvVar("NZBUP_BRANCH", szBranchName[m_eBranch]);
SetEnvVar("NZBUP_RUNMODE", g_pCommandLineParser->GetDaemonMode() ? "DAEMON" : "SERVER");
for (int i = 0; i < g_iArgumentCount; i++)
{
char szEnvName[40];
snprintf(szEnvName, 40, "NZBUP_CMDLINE%i", i);
szInfoName[40-1] = '\0';
SetEnvVar(szEnvName, (*g_szArguments)[i]);
}
char szProcessID[20];
#ifdef WIN32
int pid = (int)GetCurrentProcessId();
#else
int pid = (int)getpid();
int pid = (int)getppid();
#endif
snprintf(szProcessID, 20, "%i", pid);
szProcessID[20-1] = '\0';
@@ -325,24 +266,8 @@ void UpdateScriptController::AddMessage(Message::EKind eKind, const char* szText
{
szText = szText + m_iPrefixLen;
if (!strncmp(szText, "[NZB] ", 6))
{
debug("Command %s detected", szText + 6);
if (!strcmp(szText + 6, "QUIT"))
{
Detach();
ExitProc();
}
else
{
error("Invalid command \"%s\" received", szText);
}
}
else
{
g_pMaintenance->AddMessage(eKind, time(NULL), szText);
ScriptController::AddMessage(eKind, szText);
}
g_pMaintenance->AppendMessage(eKind, time(NULL), szText);
ScriptController::AddMessage(eKind, szText);
}
void UpdateInfoScriptController::ExecuteScript(const char* szScript, char** pUpdateInfo)
@@ -398,110 +323,3 @@ void UpdateInfoScriptController::AddMessage(Message::EKind eKind, const char* sz
ScriptController::AddMessage(eKind, szText);
}
}
#ifdef HAVE_OPENSSL
Signature::Signature(const char *szInFilename, const char *szSigFilename, const char *szPubKeyFilename)
{
m_szInFilename = szInFilename;
m_szSigFilename = szSigFilename;
m_szPubKeyFilename = szPubKeyFilename;
m_pPubKey = NULL;
}
Signature::~Signature()
{
RSA_free(m_pPubKey);
}
// Calculate SHA-256 for input file (m_szInFilename)
bool Signature::ComputeInHash()
{
FILE* infile = fopen(m_szInFilename, FOPEN_RB);
if (!infile)
{
return false;
}
SHA256_CTX sha256;
SHA256_Init(&sha256);
const int bufSize = 32*1024;
char* buffer = (char*)malloc(bufSize);
while(int bytesRead = fread(buffer, 1, bufSize, infile))
{
SHA256_Update(&sha256, buffer, bytesRead);
}
SHA256_Final(m_InHash, &sha256);
free(buffer);
fclose(infile);
return true;
}
// Read signature from file (m_szSigFilename) into memory
bool Signature::ReadSignature()
{
char szSigTitle[256];
snprintf(szSigTitle, sizeof(szSigTitle), "\"RSA-SHA256(%s)\" : \"", Util::BaseFileName(m_szInFilename));
szSigTitle[256-1] = '\0';
FILE* infile = fopen(m_szSigFilename, FOPEN_RB);
if (!infile)
{
return false;
}
bool bOK = false;
int iTitLen = strlen(szSigTitle);
char buf[1024];
unsigned char* output = m_Signature;
while (fgets(buf, sizeof(buf) - 1, infile))
{
if (!strncmp(buf, szSigTitle, iTitLen))
{
char* szHexSig = buf + iTitLen;
int iSigLen = strlen(szHexSig);
if (iSigLen > 2)
{
szHexSig[iSigLen - 2] = '\0'; // trim trailing ",
}
for (; *szHexSig && *(szHexSig+1);)
{
unsigned char c1 = *szHexSig++;
unsigned char c2 = *szHexSig++;
c1 = '0' <= c1 && c1 <= '9' ? c1 - '0' : 'A' <= c1 && c1 <= 'F' ? c1 - 'A' + 10 :
'a' <= c1 && c1 <= 'f' ? c1 - 'a' + 10 : 0;
c2 = '0' <= c2 && c2 <= '9' ? c2 - '0' : 'A' <= c2 && c2 <= 'F' ? c2 - 'A' + 10 :
'a' <= c2 && c2 <= 'f' ? c2 - 'a' + 10 : 0;
unsigned char ch = (c1 << 4) + c2;
*output++ = (char)ch;
}
bOK = output == m_Signature + sizeof(m_Signature);
break;
}
}
fclose(infile);
return bOK;
}
// Read public key from file (m_szPubKeyFilename) into memory
bool Signature::ReadPubKey()
{
char* keybuf;
int keybuflen;
if (!Util::LoadFileIntoBuffer(m_szPubKeyFilename, &keybuf, &keybuflen))
{
return false;
}
BIO* mem = BIO_new_mem_buf(keybuf, keybuflen);
m_pPubKey = PEM_read_bio_RSA_PUBKEY(mem, NULL, NULL, NULL);
BIO_free(mem);
free(keybuf);
return m_pPubKey != NULL;
}
bool Signature::Verify()
{
return ComputeInHash() && ReadSignature() && ReadPubKey() &&
RSA_verify(NID_sha256, m_InHash, sizeof(m_InHash), m_Signature, sizeof(m_Signature), m_pPubKey) == 1;
}
#endif /* HAVE_OPENSSL */

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2013-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -35,7 +35,7 @@ class UpdateScriptController;
class Maintenance
{
private:
MessageList m_Messages;
Log::Messages m_Messages;
Mutex m_mutexLog;
Mutex m_mutexController;
int m_iIDMessageGen;
@@ -54,17 +54,15 @@ public:
Maintenance();
~Maintenance();
void AddMessage(Message::EKind eKind, time_t tTime, const char* szText);
MessageList* LockMessages();
void ClearMessages();
void AppendMessage(Message::EKind eKind, time_t tTime, const char* szText);
Log::Messages* LockMessages();
void UnlockMessages();
bool StartUpdate(EBranch eBranch);
void ResetUpdateController();
bool CheckUpdates(char** pUpdateInfo);
static bool VerifySignature(const char* szInFilename, const char* szSigFilename, const char* szPubKeyFilename);
};
extern Maintenance* g_pMaintenance;
class UpdateScriptController : public Thread, public ScriptController
{
private:

View File

File diff suppressed because it is too large Load Diff

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -37,12 +37,32 @@
class Options
{
public:
enum EWriteLog
enum EClientOperation
{
wlNone,
wlAppend,
wlReset,
wlRotate
opClientNoOperation,
opClientRequestDownload,
opClientRequestListFiles,
opClientRequestListGroups,
opClientRequestListStatus,
opClientRequestSetRate,
opClientRequestDumpDebug,
opClientRequestEditQueue,
opClientRequestLog,
opClientRequestShutdown,
opClientRequestReload,
opClientRequestVersion,
opClientRequestPostQueue,
opClientRequestWriteLog,
opClientRequestScanSync,
opClientRequestScanAsync,
opClientRequestDownloadPause,
opClientRequestDownloadUnpause,
opClientRequestPostPause,
opClientRequestPostUnpause,
opClientRequestScanPause,
opClientRequestScanUnpause,
opClientRequestHistory,
opClientRequestDownloadUrl
};
enum EMessageTarget
{
@@ -67,9 +87,8 @@ public:
enum EParScan
{
psLimited,
psExtended,
psFull,
psDupe
psAuto
};
enum EHealthCheck
{
@@ -77,20 +96,11 @@ public:
hcDelete,
hcNone
};
enum ESchedulerCommand
enum EMatchMode
{
scPauseDownload,
scUnpauseDownload,
scPausePostProcess,
scUnpausePostProcess,
scDownloadRate,
scScript,
scProcess,
scPauseScan,
scUnpauseScan,
scActivateServer,
scDeactivateServer,
scFetchFeed
mmID = 1,
mmName,
mmRegEx
};
class OptEntry
@@ -101,6 +111,8 @@ public:
char* m_szDefValue;
int m_iLineNo;
void SetName(const char* szName);
void SetValue(const char* szValue);
void SetLineNo(int iLineNo) { m_iLineNo = iLineNo; }
friend class Options;
@@ -109,13 +121,10 @@ public:
OptEntry();
OptEntry(const char* szName, const char* szValue);
~OptEntry();
void SetName(const char* szName);
const char* GetName() { return m_szName; }
void SetValue(const char* szValue);
const char* GetValue() { return m_szValue; }
const char* GetDefValue() { return m_szDefValue; }
int GetLineNo() { return m_iLineNo; }
bool Restricted();
};
typedef std::vector<OptEntry*> OptEntriesBase;
@@ -128,7 +137,6 @@ public:
};
typedef std::vector<char*> NameList;
typedef std::vector<const char*> CmdOptList;
class Category
{
@@ -158,33 +166,75 @@ public:
Category* FindCategory(const char* szName, bool bSearchAliases);
};
class Extender
class Script
{
private:
char* m_szName;
char* m_szLocation;
char* m_szDisplayName;
bool m_bPostScript;
bool m_bScanScript;
bool m_bQueueScript;
bool m_bSchedulerScript;
public:
Script(const char* szName, const char* szLocation);
~Script();
const char* GetName() { return m_szName; }
const char* GetLocation() { return m_szLocation; }
void SetDisplayName(const char* szDisplayName);
const char* GetDisplayName() { return m_szDisplayName; }
bool GetPostScript() { return m_bPostScript; }
void SetPostScript(bool bPostScript) { m_bPostScript = bPostScript; }
bool GetScanScript() { return m_bScanScript; }
void SetScanScript(bool bScanScript) { m_bScanScript = bScanScript; }
bool GetQueueScript() { return m_bQueueScript; }
void SetQueueScript(bool bQueueScript) { m_bQueueScript = bQueueScript; }
bool GetSchedulerScript() { return m_bSchedulerScript; }
void SetSchedulerScript(bool bSchedulerScript) { m_bSchedulerScript = bSchedulerScript; }
};
typedef std::list<Script*> ScriptListBase;
class ScriptList: public ScriptListBase
{
public:
virtual void AddNewsServer(int iID, bool bActive, const char* szName, const char* szHost,
int iPort, const char* szUser, const char* szPass, bool bJoinGroup,
bool bTLS, const char* szCipher, int iMaxConnections, int iRetention,
int iLevel, int iGroup) = 0;
virtual void AddFeed(int iID, const char* szName, const char* szUrl, int iInterval,
const char* szFilter, bool bBacklog, bool bPauseNzb, const char* szCategory,
int iPriority, const char* szFeedScript) {}
virtual void AddTask(int iID, int iHours, int iMinutes, int iWeekDaysBits, ESchedulerCommand eCommand,
const char* szParam) {}
virtual void SetupFirstStart() {}
~ScriptList();
Script* Find(const char* szName);
};
class ConfigTemplate
{
private:
Script* m_pScript;
char* m_szTemplate;
friend class Options;
public:
ConfigTemplate(Script* pScript, const char* szTemplate);
~ConfigTemplate();
Script* GetScript() { return m_pScript; }
const char* GetTemplate() { return m_szTemplate; }
};
typedef std::vector<ConfigTemplate*> ConfigTemplatesBase;
class ConfigTemplates: public ConfigTemplatesBase
{
public:
~ConfigTemplates();
};
private:
OptEntries m_OptEntries;
bool m_bConfigInitialized;
Mutex m_mutexOptEntries;
Categories m_Categories;
bool m_bNoDiskAccess;
bool m_bFatalError;
Extender* m_pExtender;
// Options
bool m_bConfigErrors;
int m_iConfigLine;
char* m_szAppDir;
char* m_szConfigFilename;
char* m_szDestDir;
char* m_szInterDir;
@@ -194,32 +244,25 @@ private:
char* m_szWebDir;
char* m_szConfigTemplate;
char* m_szScriptDir;
char* m_szRequiredDir;
EMessageTarget m_eInfoTarget;
EMessageTarget m_eWarningTarget;
EMessageTarget m_eErrorTarget;
EMessageTarget m_eDebugTarget;
EMessageTarget m_eDetailTarget;
bool m_bDecode;
bool m_bBrokenLog;
bool m_bNzbLog;
int m_iArticleTimeout;
int m_iUrlTimeout;
bool m_bCreateBrokenLog;
bool m_bResetLog;
int m_iConnectionTimeout;
int m_iTerminateTimeout;
bool m_bAppendCategoryDir;
bool m_bContinuePartial;
int m_iRetries;
int m_iRetryInterval;
bool m_bSaveQueue;
bool m_bFlushQueue;
bool m_bDupeCheck;
char* m_szControlIP;
char* m_szControlUsername;
char* m_szControlPassword;
char* m_szRestrictedUsername;
char* m_szRestrictedPassword;
char* m_szAddUsername;
char* m_szAddPassword;
int m_iControlPort;
bool m_bSecureControl;
int m_iSecurePort;
@@ -232,22 +275,17 @@ private:
bool m_bReloadQueue;
int m_iUrlConnections;
int m_iLogBufferSize;
EWriteLog m_eWriteLog;
int m_iRotateLog;
bool m_bCreateLog;
char* m_szLogFile;
EParCheck m_eParCheck;
bool m_bParRepair;
EParScan m_eParScan;
bool m_bParQuick;
bool m_bParRename;
int m_iParBuffer;
int m_iParThreads;
EHealthCheck m_eHealthCheck;
char* m_szPostScript;
char* m_szScriptOrder;
char* m_szScanScript;
char* m_szQueueScript;
char* m_szFeedScript;
bool m_bNoConfig;
int m_iUMask;
int m_iUpdateInterval;
@@ -256,7 +294,7 @@ private:
bool m_bCursesGroup;
bool m_bCrcCheck;
bool m_bDirectWrite;
int m_iWriteBuffer;
int m_iWriteBufferSize;
int m_iNzbDirInterval;
int m_iNzbDirFileAge;
bool m_bParCleanupQueue;
@@ -274,7 +312,6 @@ private:
bool m_bUnpackCleanupDisk;
char* m_szUnrarCmd;
char* m_szSevenZipCmd;
char* m_szUnpackPassFile;
bool m_bUnpackPauseQueue;
char* m_szExtCleanupDisk;
char* m_szParIgnoreExt;
@@ -282,65 +319,89 @@ private:
bool m_bUrlForce;
int m_iTimeCorrection;
int m_iPropagationDelay;
int m_iArticleCache;
int m_iEventInterval;
// Parsed command-line parameters
bool m_bServerMode;
bool m_bDaemonMode;
bool m_bRemoteClientMode;
int m_iEditQueueAction;
int m_iEditQueueOffset;
int* m_pEditQueueIDList;
int m_iEditQueueIDCount;
NameList m_EditQueueNameList;
EMatchMode m_EMatchMode;
char* m_szEditQueueText;
char* m_szArgFilename;
char* m_szAddCategory;
int m_iAddPriority;
bool m_bAddPaused;
char* m_szAddNZBFilename;
char* m_szLastArg;
bool m_bPrintOptions;
bool m_bAddTop;
int m_iSetRate;
int m_iLogLines;
int m_iWriteLogKind;
bool m_bTestBacktrace;
// Current state
bool m_bServerMode;
bool m_bRemoteClientMode;
bool m_bPauseDownload;
bool m_bPausePostProcess;
bool m_bPauseScan;
bool m_bTempPauseDownload;
int m_iDownloadRate;
EClientOperation m_eClientOperation;
time_t m_tResumeTime;
int m_iLocalTimeOffset;
bool m_bTempPausePostprocess;
void Init(const char* szExeName, const char* szConfigFilename, bool bNoConfig,
CmdOptList* pCommandLineOptions, bool bNoDiskAccess, Extender* pExtender);
void InitDefaults();
void InitOptions();
void InitDefault();
void InitOptFile();
void InitCommandLine(int argc, char* argv[]);
void InitOptions();
void InitFileArg(int argc, char* argv[]);
void InitServers();
void InitCategories();
void InitScheduler();
void InitFeeds();
void InitCommandLineOptions(CmdOptList* pCommandLineOptions);
void CheckOptions();
void PrintUsage(char* com);
void Dump();
int ParseEnumValue(const char* OptName, int argc, const char* argn[], const int argv[]);
int ParseIntValue(const char* OptName, int iBase);
float ParseFloatValue(const char* OptName);
OptEntry* FindOption(const char* optname);
const char* GetOption(const char* optname);
void SetOption(const char* optname, const char* value);
bool SetOptionString(const char* option);
bool SplitOptionString(const char* option, char** pOptName, char** pOptValue);
bool ValidateOptionName(const char* optname, const char* optvalue);
void LoadConfigFile();
void CheckDir(char** dir, const char* szOptionName, const char* szParentDir,
bool bAllowEmpty, bool bCreate);
bool ParseTime(const char* szTime, int* pHours, int* pMinutes);
void CheckDir(char** dir, const char* szOptionName, bool bAllowEmpty, bool bCreate);
void ParseFileIDList(int argc, char* argv[], int optind);
void ParseFileNameList(int argc, char* argv[], int optind);
bool ParseTime(const char** pTime, int* pHours, int* pMinutes);
bool ParseWeekDays(const char* szWeekDays, int* pWeekDaysBits);
void ConfigError(const char* msg, ...);
void ConfigWarn(const char* msg, ...);
void LocateOptionSrcPos(const char *szOptionName);
void ConvertOldOption(char *szOption, int iOptionBufLen, char *szValue, int iValueBufLen);
static bool CompareScripts(Script* pScript1, Script* pScript2);
void LoadScriptDir(ScriptList* pScriptList, const char* szDirectory, bool bIsSubDir);
void BuildScriptDisplayNames(ScriptList* pScriptList);
public:
Options(const char* szExeName, const char* szConfigFilename, bool bNoConfig,
CmdOptList* pCommandLineOptions, Extender* pExtender);
Options(CmdOptList* pCommandLineOptions, Extender* pExtender);
Options(int argc, char* argv[]);
~Options();
bool SplitOptionString(const char* option, char** pOptName, char** pOptValue);
bool GetFatalError() { return m_bFatalError; }
OptEntries* LockOptEntries();
void UnlockOptEntries();
bool LoadConfig(OptEntries* pOptEntries);
bool SaveConfig(OptEntries* pOptEntries);
bool LoadConfigTemplates(ConfigTemplates* pConfigTemplates);
void LoadScriptList(ScriptList* pScriptList);
// Options
OptEntries* LockOptEntries();
void UnlockOptEntries();
const char* GetConfigFilename() { return m_szConfigFilename; }
bool GetConfigErrors() { return m_bConfigErrors; }
const char* GetAppDir() { return m_szAppDir; }
const char* GetDestDir() { return m_szDestDir; }
const char* GetInterDir() { return m_szInterDir; }
const char* GetTempDir() { return m_szTempDir; }
@@ -349,16 +410,14 @@ public:
const char* GetWebDir() { return m_szWebDir; }
const char* GetConfigTemplate() { return m_szConfigTemplate; }
const char* GetScriptDir() { return m_szScriptDir; }
const char* GetRequiredDir() { return m_szRequiredDir; }
bool GetBrokenLog() const { return m_bBrokenLog; }
bool GetNzbLog() const { return m_bNzbLog; }
bool GetCreateBrokenLog() const { return m_bCreateBrokenLog; }
bool GetResetLog() const { return m_bResetLog; }
EMessageTarget GetInfoTarget() const { return m_eInfoTarget; }
EMessageTarget GetWarningTarget() const { return m_eWarningTarget; }
EMessageTarget GetErrorTarget() const { return m_eErrorTarget; }
EMessageTarget GetDebugTarget() const { return m_eDebugTarget; }
EMessageTarget GetDetailTarget() const { return m_eDetailTarget; }
int GetArticleTimeout() { return m_iArticleTimeout; }
int GetUrlTimeout() { return m_iUrlTimeout; }
int GetConnectionTimeout() { return m_iConnectionTimeout; }
int GetTerminateTimeout() { return m_iTerminateTimeout; }
bool GetDecode() { return m_bDecode; };
bool GetAppendCategoryDir() { return m_bAppendCategoryDir; }
@@ -366,15 +425,10 @@ public:
int GetRetries() { return m_iRetries; }
int GetRetryInterval() { return m_iRetryInterval; }
bool GetSaveQueue() { return m_bSaveQueue; }
bool GetFlushQueue() { return m_bFlushQueue; }
bool GetDupeCheck() { return m_bDupeCheck; }
const char* GetControlIP() { return m_szControlIP; }
const char* GetControlUsername() { return m_szControlUsername; }
const char* GetControlPassword() { return m_szControlPassword; }
const char* GetRestrictedUsername() { return m_szRestrictedUsername; }
const char* GetRestrictedPassword() { return m_szRestrictedPassword; }
const char* GetAddUsername() { return m_szAddUsername; }
const char* GetAddPassword() { return m_szAddPassword; }
int GetControlPort() { return m_iControlPort; }
bool GetSecureControl() { return m_bSecureControl; }
int GetSecurePort() { return m_iSecurePort; }
@@ -387,22 +441,17 @@ public:
bool GetReloadQueue() { return m_bReloadQueue; }
int GetUrlConnections() { return m_iUrlConnections; }
int GetLogBufferSize() { return m_iLogBufferSize; }
EWriteLog GetWriteLog() { return m_eWriteLog; }
bool GetCreateLog() { return m_bCreateLog; }
const char* GetLogFile() { return m_szLogFile; }
int GetRotateLog() { return m_iRotateLog; }
EParCheck GetParCheck() { return m_eParCheck; }
bool GetParRepair() { return m_bParRepair; }
EParScan GetParScan() { return m_eParScan; }
bool GetParQuick() { return m_bParQuick; }
bool GetParRename() { return m_bParRename; }
int GetParBuffer() { return m_iParBuffer; }
int GetParThreads() { return m_iParThreads; }
EHealthCheck GetHealthCheck() { return m_eHealthCheck; }
const char* GetScriptOrder() { return m_szScriptOrder; }
const char* GetPostScript() { return m_szPostScript; }
const char* GetScanScript() { return m_szScanScript; }
const char* GetQueueScript() { return m_szQueueScript; }
const char* GetFeedScript() { return m_szFeedScript; }
int GetUMask() { return m_iUMask; }
int GetUpdateInterval() {return m_iUpdateInterval; }
bool GetCursesNZBName() { return m_bCursesNZBName; }
@@ -410,7 +459,7 @@ public:
bool GetCursesGroup() { return m_bCursesGroup; }
bool GetCrcCheck() { return m_bCrcCheck; }
bool GetDirectWrite() { return m_bDirectWrite; }
int GetWriteBuffer() { return m_iWriteBuffer; }
int GetWriteBufferSize() { return m_iWriteBufferSize; }
int GetNzbDirInterval() { return m_iNzbDirInterval; }
int GetNzbDirFileAge() { return m_iNzbDirFileAge; }
bool GetParCleanupQueue() { return m_bParCleanupQueue; }
@@ -428,7 +477,6 @@ public:
bool GetUnpackCleanupDisk() { return m_bUnpackCleanupDisk; }
const char* GetUnrarCmd() { return m_szUnrarCmd; }
const char* GetSevenZipCmd() { return m_szSevenZipCmd; }
const char* GetUnpackPassFile() { return m_szUnpackPassFile; }
bool GetUnpackPauseQueue() { return m_bUnpackPauseQueue; }
const char* GetExtCleanupDisk() { return m_szExtCleanupDisk; }
const char* GetParIgnoreExt() { return m_szParIgnoreExt; }
@@ -436,17 +484,34 @@ public:
bool GetUrlForce() { return m_bUrlForce; }
int GetTimeCorrection() { return m_iTimeCorrection; }
int GetPropagationDelay() { return m_iPropagationDelay; }
int GetArticleCache() { return m_iArticleCache; }
int GetEventInterval() { return m_iEventInterval; }
Categories* GetCategories() { return &m_Categories; }
Category* FindCategory(const char* szName, bool bSearchAliases) { return m_Categories.FindCategory(szName, bSearchAliases); }
// Current state
void SetServerMode(bool bServerMode) { m_bServerMode = bServerMode; }
// Parsed command-line parameters
bool GetServerMode() { return m_bServerMode; }
void SetRemoteClientMode(bool bRemoteClientMode) { m_bRemoteClientMode = bRemoteClientMode; }
bool GetDaemonMode() { return m_bDaemonMode; }
bool GetRemoteClientMode() { return m_bRemoteClientMode; }
EClientOperation GetClientOperation() { return m_eClientOperation; }
int GetEditQueueAction() { return m_iEditQueueAction; }
int GetEditQueueOffset() { return m_iEditQueueOffset; }
int* GetEditQueueIDList() { return m_pEditQueueIDList; }
int GetEditQueueIDCount() { return m_iEditQueueIDCount; }
NameList* GetEditQueueNameList() { return &m_EditQueueNameList; }
EMatchMode GetMatchMode() { return m_EMatchMode; }
const char* GetEditQueueText() { return m_szEditQueueText; }
const char* GetArgFilename() { return m_szArgFilename; }
const char* GetAddCategory() { return m_szAddCategory; }
bool GetAddPaused() { return m_bAddPaused; }
const char* GetLastArg() { return m_szLastArg; }
int GetAddPriority() { return m_iAddPriority; }
char* GetAddNZBFilename() { return m_szAddNZBFilename; }
bool GetAddTop() { return m_bAddTop; }
int GetSetRate() { return m_iSetRate; }
int GetLogLines() { return m_iLogLines; }
int GetWriteLogKind() { return m_iWriteLogKind; }
bool GetTestBacktrace() { return m_bTestBacktrace; }
// Current state
void SetPauseDownload(bool bPauseDownload) { m_bPauseDownload = bPauseDownload; }
bool GetPauseDownload() const { return m_bPauseDownload; }
void SetPausePostProcess(bool bPausePostProcess) { m_bPausePostProcess = bPausePostProcess; }
@@ -455,8 +520,6 @@ public:
bool GetPauseScan() const { return m_bPauseScan; }
void SetTempPauseDownload(bool bTempPauseDownload) { m_bTempPauseDownload = bTempPauseDownload; }
bool GetTempPauseDownload() const { return m_bTempPauseDownload; }
bool GetTempPausePostprocess() const { return m_bTempPausePostprocess; }
void SetTempPausePostprocess(bool bTempPausePostprocess) { m_bTempPausePostprocess = bTempPausePostprocess; }
void SetDownloadRate(int iRate) { m_iDownloadRate = iRate; }
int GetDownloadRate() const { return m_iDownloadRate; }
void SetResumeTime(time_t tResumeTime) { m_tResumeTime = tResumeTime; }
@@ -465,6 +528,4 @@ public:
int GetLocalTimeOffset() { return m_iLocalTimeOffset; }
};
extern Options* g_pOptions;
#endif

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2008-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2008-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -45,7 +45,30 @@
#include "ServerPool.h"
#include "FeedInfo.h"
#include "FeedCoordinator.h"
#include "SchedulerScript.h"
#include "QueueScript.h"
extern Options* g_pOptions;
extern ServerPool* g_pServerPool;
extern FeedCoordinator* g_pFeedCoordinator;
class SchedulerScriptController : public Thread, public NZBScriptController
{
private:
char* m_szScript;
bool m_bExternalProcess;
int m_iTaskID;
void PrepareParams(const char* szScriptName);
void ExecuteExternalProcess();
protected:
virtual void ExecuteScript(Options::Script* pScript);
public:
virtual ~SchedulerScriptController();
virtual void Run();
static void StartScript(const char* szParam, bool bExternalProcess, int iTaskID);
};
Scheduler::Task::Task(int iID, int iHours, int iMinutes, int iWeekDaysBits, ECommand eCommand, const char* szParam)
{
@@ -68,7 +91,6 @@ Scheduler::Scheduler()
{
debug("Creating Scheduler");
m_bFirstChecked = false;
m_tLastCheck = 0;
m_TaskList.clear();
}
@@ -103,23 +125,16 @@ void Scheduler::FirstCheck()
m_mutexTaskList.Unlock();
// check all tasks for the last week
time_t tCurrent = time(NULL);
m_tLastCheck = tCurrent - 60*60*24*7;
m_bDetectClockChanges = false;
m_bExecuteProcess = false;
CheckTasks();
}
void Scheduler::ServiceWork()
void Scheduler::IntervalCheck()
{
if (!DownloadQueue::IsLoaded())
{
return;
}
if (!m_bFirstChecked)
{
FirstCheck();
m_bFirstChecked = true;
return;
}
m_bDetectClockChanges = true;
m_bExecuteProcess = true;
CheckTasks();
CheckScheduledResume();
@@ -135,20 +150,21 @@ void Scheduler::CheckTasks()
if (!m_TaskList.empty())
{
// Detect large step changes of system time
time_t tDiff = tCurrent - m_tLastCheck;
if (tDiff > 60*90 || tDiff < 0)
if (m_bDetectClockChanges)
{
debug("Reset scheduled tasks (detected clock change greater than 90 minutes or negative)");
// check all tasks for the last week
m_tLastCheck = tCurrent - 60*60*24*7;
m_bExecuteProcess = false;
for (TaskList::iterator it = m_TaskList.begin(); it != m_TaskList.end(); it++)
// Detect large step changes of system time
time_t tDiff = tCurrent - m_tLastCheck;
if (tDiff > 60*90 || tDiff < -60*90)
{
Task* pTask = *it;
pTask->m_tLastExecuted = 0;
debug("Reset scheduled tasks (detected clock adjustment greater than 90 minutes)");
m_bExecuteProcess = false;
m_tLastCheck = tCurrent;
for (TaskList::iterator it = m_TaskList.begin(); it != m_TaskList.end(); it++)
{
Task* pTask = *it;
pTask->m_tLastExecuted = 0;
}
}
}
@@ -214,8 +230,7 @@ void Scheduler::CheckTasks()
void Scheduler::ExecuteTask(Task* pTask)
{
const char* szCommandName[] = { "Pause", "Unpause", "Pause Post-processing", "Unpause Post-processing",
"Set download rate", "Execute process", "Execute script",
const char* szCommandName[] = { "Pause", "Unpause", "Set download rate", "Execute process", "Execute script",
"Pause Scan", "Unpause Scan", "Enable Server", "Disable Server", "Fetch Feed" };
debug("Executing scheduled command: %s", szCommandName[pTask->m_eCommand]);
@@ -235,18 +250,6 @@ void Scheduler::ExecuteTask(Task* pTask)
m_bPauseDownloadChanged = true;
break;
case scPausePostProcess:
case scUnpausePostProcess:
g_pOptions->SetPausePostProcess(pTask->m_eCommand == scPausePostProcess);
m_bPausePostProcessChanged = true;
break;
case scPauseScan:
case scUnpauseScan:
g_pOptions->SetPauseScan(pTask->m_eCommand == scPauseScan);
m_bPauseScanChanged = true;
break;
case scScript:
case scProcess:
if (m_bExecuteProcess)
@@ -255,6 +258,12 @@ void Scheduler::ExecuteTask(Task* pTask)
}
break;
case scPauseScan:
case scUnpauseScan:
g_pOptions->SetPauseScan(pTask->m_eCommand == scPauseScan);
m_bPauseScanChanged = true;
break;
case scActivateServer:
case scDeactivateServer:
EditServer(pTask->m_eCommand == scActivateServer, pTask->m_szParam);
@@ -273,7 +282,6 @@ void Scheduler::PrepareLog()
{
m_bDownloadRateChanged = false;
m_bPauseDownloadChanged = false;
m_bPausePostProcessChanged = false;
m_bPauseScanChanged = false;
m_bServerChanged = false;
}
@@ -288,10 +296,6 @@ void Scheduler::PrintLog()
{
info("Scheduler: %s download", g_pOptions->GetPauseDownload() ? "pausing" : "unpausing");
}
if (m_bPausePostProcessChanged)
{
info("Scheduler: %s post-processing", g_pOptions->GetPausePostProcess() ? "pausing" : "unpausing");
}
if (m_bPauseScanChanged)
{
info("Scheduler: %s scan", g_pOptions->GetPauseScan() ? "pausing" : "unpausing");
@@ -375,3 +379,99 @@ void Scheduler::CheckScheduledResume()
g_pOptions->SetPauseScan(false);
}
}
SchedulerScriptController::~SchedulerScriptController()
{
free(m_szScript);
}
void SchedulerScriptController::StartScript(const char* szParam, bool bExternalProcess, int iTaskID)
{
char** argv = NULL;
if (bExternalProcess && !Util::SplitCommandLine(szParam, &argv))
{
error("Could not execute scheduled process-script, failed to parse command line: %s", szParam);
return;
}
SchedulerScriptController* pScriptController = new SchedulerScriptController();
pScriptController->m_bExternalProcess = bExternalProcess;
pScriptController->m_szScript = strdup(szParam);
pScriptController->m_iTaskID = iTaskID;
if (bExternalProcess)
{
pScriptController->SetScript(argv[0]);
pScriptController->SetArgs((const char**)argv, true);
}
pScriptController->SetAutoDestroy(true);
pScriptController->Start();
}
void SchedulerScriptController::Run()
{
if (m_bExternalProcess)
{
ExecuteExternalProcess();
}
else
{
ExecuteScriptList(m_szScript);
}
}
void SchedulerScriptController::ExecuteScript(Options::Script* pScript)
{
if (!pScript->GetSchedulerScript())
{
return;
}
PrintMessage(Message::mkInfo, "Executing scheduler-script %s for Task%i", pScript->GetName(), m_iTaskID);
SetScript(pScript->GetLocation());
SetArgs(NULL, false);
char szInfoName[1024];
snprintf(szInfoName, 1024, "scheduler-script %s for Task%i", pScript->GetName(), m_iTaskID);
szInfoName[1024-1] = '\0';
SetInfoName(szInfoName);
SetLogPrefix(pScript->GetDisplayName());
PrepareParams(pScript->GetName());
Execute();
SetLogPrefix(NULL);
}
void SchedulerScriptController::PrepareParams(const char* szScriptName)
{
ResetEnv();
SetIntEnvVar("NZBSP_TASKID", m_iTaskID);
PrepareEnvScript(NULL, szScriptName);
}
void SchedulerScriptController::ExecuteExternalProcess()
{
info("Executing scheduled process-script %s for Task%i", Util::BaseFileName(GetScript()), m_iTaskID);
char szInfoName[1024];
snprintf(szInfoName, 1024, "scheduled process-script %s for Task%i", Util::BaseFileName(GetScript()), m_iTaskID);
szInfoName[1024-1] = '\0';
SetInfoName(szInfoName);
char szLogPrefix[1024];
strncpy(szLogPrefix, Util::BaseFileName(GetScript()), 1024);
szLogPrefix[1024-1] = '\0';
if (char* ext = strrchr(szLogPrefix, '.')) *ext = '\0'; // strip file extension
SetLogPrefix(szLogPrefix);
Execute();
}

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2008-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2008-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -31,17 +31,14 @@
#include <time.h>
#include "Thread.h"
#include "Service.h"
class Scheduler : public Service
class Scheduler
{
public:
enum ECommand
{
scPauseDownload,
scUnpauseDownload,
scPausePostProcess,
scUnpausePostProcess,
scDownloadRate,
scScript,
scProcess,
@@ -78,15 +75,13 @@ private:
TaskList m_TaskList;
Mutex m_mutexTaskList;
time_t m_tLastCheck;
bool m_bDetectClockChanges;
bool m_bDownloadRateChanged;
bool m_bExecuteProcess;
bool m_bPauseDownloadChanged;
bool m_bPausePostProcessChanged;
bool m_bPauseScanChanged;
bool m_bServerChanged;
ServerStatusList m_ServerStatusList;
bool m_bFirstChecked;
void ExecuteTask(Task* pTask);
void CheckTasks();
static bool CompareTasks(Scheduler::Task* pTask1, Scheduler::Task* pTask2);
@@ -95,18 +90,13 @@ private:
void EditServer(bool bActive, const char* szServerList);
void FetchFeed(const char* szFeedList);
void CheckScheduledResume();
void FirstCheck();
protected:
virtual int ServiceInterval() { return 1000; }
virtual void ServiceWork();
public:
Scheduler();
~Scheduler();
void AddTask(Task* pTask);
void FirstCheck();
void IntervalCheck();
};
extern Scheduler* g_pScheduler;
#endif

View File

@@ -1,320 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef WIN32
#include "win32.h"
#endif
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#ifdef WIN32
#include <dbghelp.h>
#else
#include <unistd.h>
#include <sys/resource.h>
#include <signal.h>
#endif
#ifdef HAVE_SYS_PRCTL_H
#include <sys/prctl.h>
#endif
#ifdef HAVE_BACKTRACE
#include <execinfo.h>
#endif
#include "nzbget.h"
#include "Log.h"
#include "Options.h"
#include "StackTrace.h"
extern void ExitProc();
#ifdef WIN32
#ifdef DEBUG
void PrintBacktrace(PCONTEXT pContext)
{
HANDLE hProcess = GetCurrentProcess();
HANDLE hThread = GetCurrentThread();
char szAppDir[MAX_PATH + 1];
GetModuleFileName(NULL, szAppDir, sizeof(szAppDir));
char* end = strrchr(szAppDir, PATH_SEPARATOR);
if (end) *end = '\0';
SymSetOptions(SymGetOptions() | SYMOPT_LOAD_LINES | SYMOPT_FAIL_CRITICAL_ERRORS);
if (!SymInitialize(hProcess, szAppDir, TRUE))
{
warn("Could not obtain detailed exception information: SymInitialize failed");
return;
}
const int MAX_NAMELEN = 1024;
IMAGEHLP_SYMBOL64* pSym = (IMAGEHLP_SYMBOL64 *) malloc(sizeof(IMAGEHLP_SYMBOL64) + MAX_NAMELEN);
memset(pSym, 0, sizeof(IMAGEHLP_SYMBOL64) + MAX_NAMELEN);
pSym->SizeOfStruct = sizeof(IMAGEHLP_SYMBOL64);
pSym->MaxNameLength = MAX_NAMELEN;
IMAGEHLP_LINE64 ilLine;
memset(&ilLine, 0, sizeof(ilLine));
ilLine.SizeOfStruct = sizeof(ilLine);
STACKFRAME64 sfStackFrame;
memset(&sfStackFrame, 0, sizeof(sfStackFrame));
DWORD imageType;
#ifdef _M_IX86
imageType = IMAGE_FILE_MACHINE_I386;
sfStackFrame.AddrPC.Offset = pContext->Eip;
sfStackFrame.AddrPC.Mode = AddrModeFlat;
sfStackFrame.AddrFrame.Offset = pContext->Ebp;
sfStackFrame.AddrFrame.Mode = AddrModeFlat;
sfStackFrame.AddrStack.Offset = pContext->Esp;
sfStackFrame.AddrStack.Mode = AddrModeFlat;
#elif _M_X64
imageType = IMAGE_FILE_MACHINE_AMD64;
sfStackFrame.AddrPC.Offset = pContext->Rip;
sfStackFrame.AddrPC.Mode = AddrModeFlat;
sfStackFrame.AddrFrame.Offset = pContext->Rsp;
sfStackFrame.AddrFrame.Mode = AddrModeFlat;
sfStackFrame.AddrStack.Offset = pContext->Rsp;
sfStackFrame.AddrStack.Mode = AddrModeFlat;
#else
warn("Could not obtain detailed exception information: platform not supported");
return;
#endif
for (int frameNum = 0; ; frameNum++)
{
if (frameNum > 1000)
{
warn("Endless stack, abort tracing");
return;
}
if (!StackWalk64(imageType, hProcess, hThread, &sfStackFrame, pContext, NULL, SymFunctionTableAccess64, SymGetModuleBase64, NULL))
{
warn("Could not obtain detailed exception information: StackWalk64 failed");
return;
}
DWORD64 dwAddr = sfStackFrame.AddrPC.Offset;
char szSymName[1024];
char szSrcFileName[1024];
int iLineNumber = 0;
DWORD64 dwSymbolDisplacement;
if (SymGetSymFromAddr64(hProcess, dwAddr, &dwSymbolDisplacement, pSym))
{
UnDecorateSymbolName(pSym->Name, szSymName, sizeof(szSymName), UNDNAME_COMPLETE);
szSymName[sizeof(szSymName) - 1] = '\0';
}
else
{
strncpy(szSymName, "<symbol not available>", sizeof(szSymName));
}
DWORD dwLineDisplacement;
if (SymGetLineFromAddr64(hProcess, dwAddr, &dwLineDisplacement, &ilLine))
{
iLineNumber = ilLine.LineNumber;
char* szUseFileName = ilLine.FileName;
char* szRoot = strstr(szUseFileName, "\\daemon\\");
if (szRoot)
{
szUseFileName = szRoot;
}
strncpy(szSrcFileName, szUseFileName, sizeof(szSrcFileName));
szSrcFileName[sizeof(szSrcFileName) - 1] = '\0';
}
else
{
strncpy(szSrcFileName, "<filename not available>", sizeof(szSymName));
}
info("%s (%i) : %s", szSrcFileName, iLineNumber, szSymName);
if (sfStackFrame.AddrReturn.Offset == 0)
{
break;
}
}
}
#endif
LONG __stdcall ExceptionFilter(EXCEPTION_POINTERS* pExPtrs)
{
error("Unhandled Exception: code: 0x%8.8X, flags: %d, address: 0x%8.8X",
pExPtrs->ExceptionRecord->ExceptionCode,
pExPtrs->ExceptionRecord->ExceptionFlags,
pExPtrs->ExceptionRecord->ExceptionAddress);
#ifdef DEBUG
PrintBacktrace(pExPtrs->ContextRecord);
#else
info("Detailed exception information can be printed by debug version of NZBGet (available from download page)");
#endif
ExitProcess(-1);
return EXCEPTION_CONTINUE_SEARCH;
}
void InstallErrorHandler()
{
SetUnhandledExceptionFilter(ExceptionFilter);
}
#else
#ifdef DEBUG
typedef void(*sighandler)(int);
std::vector<sighandler> SignalProcList;
#endif
#ifdef HAVE_SYS_PRCTL_H
/**
* activates the creation of core-files
*/
void EnableDumpCore()
{
rlimit rlim;
rlim.rlim_cur= RLIM_INFINITY;
rlim.rlim_max= RLIM_INFINITY;
setrlimit(RLIMIT_CORE, &rlim);
prctl(PR_SET_DUMPABLE, 1);
}
#endif
void PrintBacktrace()
{
#ifdef HAVE_BACKTRACE
printf("Segmentation fault, tracing...\n");
void *array[100];
size_t size;
char **strings;
size_t i;
size = backtrace(array, 100);
strings = backtrace_symbols(array, size);
// first trace to screen
printf("Obtained %zd stack frames\n", size);
for (i = 0; i < size; i++)
{
printf("%s\n", strings[i]);
}
// then trace to log
error("Segmentation fault, tracing...");
error("Obtained %zd stack frames", size);
for (i = 0; i < size; i++)
{
error("%s", strings[i]);
}
free(strings);
#else
error("Segmentation fault");
#endif
}
/*
* Signal handler
*/
void SignalProc(int iSignal)
{
switch (iSignal)
{
case SIGINT:
signal(SIGINT, SIG_DFL); // Reset the signal handler
ExitProc();
break;
case SIGTERM:
signal(SIGTERM, SIG_DFL); // Reset the signal handler
ExitProc();
break;
case SIGCHLD:
// ignoring
break;
#ifdef DEBUG
case SIGSEGV:
signal(SIGSEGV, SIG_DFL); // Reset the signal handler
PrintBacktrace();
break;
#endif
}
}
void InstallErrorHandler()
{
#ifdef HAVE_SYS_PRCTL_H
if (g_pOptions->GetDumpCore())
{
EnableDumpCore();
}
#endif
signal(SIGINT, SignalProc);
signal(SIGTERM, SignalProc);
signal(SIGPIPE, SIG_IGN);
#ifdef DEBUG
signal(SIGSEGV, SignalProc);
#endif
#ifdef SIGCHLD_HANDLER
// it could be necessary on some systems to activate a handler for SIGCHLD
// however it make troubles on other systems and is deactivated by default
signal(SIGCHLD, SignalProc);
#endif
}
#endif
#ifdef DEBUG
class SegFault
{
public:
void DoSegFault()
{
char* N = NULL;
strcpy(N, "");
}
};
void TestSegFault()
{
SegFault s;
s.DoSegFault();
}
#endif

View File

@@ -1,35 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifndef STACKTRACE_H
#define STACKTRACE_H
void InstallErrorHandler();
#ifdef DEBUG
void TestSegFault();
#endif
#endif

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -40,6 +40,7 @@
#include <unistd.h>
#include <pwd.h>
#include <grp.h>
#include <sys/resource.h>
#ifdef HAVE_SYS_PRCTL_H
#include <sys/prctl.h>
#endif
@@ -52,21 +53,21 @@
#ifndef DISABLE_PARCHECK
#include <iostream>
#endif
#ifdef HAVE_BACKTRACE
#include <execinfo.h>
#endif
#include "nzbget.h"
#include "ServerPool.h"
#include "Log.h"
#include "NZBFile.h"
#include "Options.h"
#include "CommandLineParser.h"
#include "ScriptConfig.h"
#include "Thread.h"
#include "ColoredFrontend.h"
#include "NCursesFrontend.h"
#include "QueueCoordinator.h"
#include "UrlCoordinator.h"
#include "RemoteServer.h"
#include "WebServer.h"
#include "RemoteClient.h"
#include "MessageBase.h"
#include "DiskState.h"
@@ -77,21 +78,11 @@
#include "Scheduler.h"
#include "Scanner.h"
#include "FeedCoordinator.h"
#include "Service.h"
#include "DiskService.h"
#include "Maintenance.h"
#include "ArticleWriter.h"
#include "StatMeter.h"
#include "QueueScript.h"
#include "Util.h"
#include "StackTrace.h"
#ifdef WIN32
#include "NTService.h"
#include "WinConsole.h"
#include "WebDownloader.h"
#endif
#ifdef ENABLE_TESTS
#include "TestMain.h"
#endif
// Prototypes
@@ -100,24 +91,30 @@ void Run(bool bReload);
void Reload();
void Cleanup();
void ProcessClientRequest();
void ProcessWebGet();
void ProcessSigVerify();
#ifndef WIN32
void InstallSignalHandlers();
void Daemonize();
void PrintBacktrace();
#ifdef HAVE_SYS_PRCTL_H
void EnableDumpCore();
#endif
#ifdef DEBUG
void MakeSegFault();
#endif
#endif
#ifndef DISABLE_PARCHECK
void DisableCout();
#endif
void BootConfig();
Thread* g_pFrontend = NULL;
CommandLineParser* g_pCommandLineParser = NULL;
Options* g_pOptions = NULL;
ServerPool* g_pServerPool = NULL;
QueueCoordinator* g_pQueueCoordinator = NULL;
UrlCoordinator* g_pUrlCoordinator = NULL;
RemoteServer* g_pRemoteServer = NULL;
RemoteServer* g_pRemoteSecureServer = NULL;
StatMeter* g_pStatMeter = NULL;
Log* g_pLog = NULL;
PrePostProcessor* g_pPrePostProcessor = NULL;
HistoryCoordinator* g_pHistoryCoordinator = NULL;
DupeCoordinator* g_pDupeCoordinator = NULL;
@@ -126,17 +123,10 @@ Scheduler* g_pScheduler = NULL;
Scanner* g_pScanner = NULL;
FeedCoordinator* g_pFeedCoordinator = NULL;
Maintenance* g_pMaintenance = NULL;
ArticleCache* g_pArticleCache = NULL;
QueueScriptCoordinator* g_pQueueScriptCoordinator = NULL;
ServiceCoordinator* g_pServiceCoordinator = NULL;
DiskService* g_pDiskService = NULL;
int g_iArgumentCount;
char* (*g_szEnvironmentVariables)[] = NULL;
char* (*g_szArguments)[] = NULL;
bool g_bReloading = true;
#ifdef WIN32
WinConsole* g_pWinConsole = NULL;
#endif
/*
* Main loop
@@ -147,7 +137,7 @@ int main(int argc, char *argv[], char *argp[])
#ifdef _DEBUG
_CrtSetReportMode(_CRT_WARN, _CRTDBG_MODE_FILE | _CRTDBG_MODE_DEBUG);
_CrtSetReportFile(_CRT_WARN, _CRTDBG_FILE_STDERR);
_CrtSetDbgFlag(_CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF
_CrtSetDbgFlag(_CRTDBG_ALLOC_MEM_DF
#ifdef DEBUG_CRTMEMLEAKS
| _CRTDBG_CHECK_CRT_DF | _CRTDBG_CHECK_ALWAYS_DF
#endif
@@ -155,26 +145,8 @@ int main(int argc, char *argv[], char *argp[])
#endif
#endif
Util::Init();
g_iArgumentCount = argc;
g_szArguments = (char*(*)[])argv;
g_szEnvironmentVariables = (char*(*)[])argp;
if (argc > 1 && (!strcmp(argv[1], "-tests") || !strcmp(argv[1], "--tests")))
{
#ifdef ENABLE_TESTS
return TestMain(argc, argv);
#else
printf("ERROR: Could not start tests, the program was compiled without tests\n");
return 1;
#endif
}
#ifdef ENABLE_TESTS
TestCleanup();
#endif
Util::InitVersionRevision();
#ifdef WIN32
InstallUninstallServiceCheck(argc, argv);
#endif
@@ -183,7 +155,11 @@ int main(int argc, char *argv[], char *argp[])
DisableCout();
#endif
srand(time(NULL));
srand (time(NULL));
g_iArgumentCount = argc;
g_szArguments = (char*(*)[])argv;
g_szEnvironmentVariables = (char*(*)[])argp;
#ifdef WIN32
for (int i=0; i < argc; i++)
@@ -198,6 +174,12 @@ int main(int argc, char *argv[], char *argp[])
RunMain();
#ifdef WIN32
#ifdef _DEBUG
_CrtDumpMemoryLeaks();
#endif
#endif
return 0;
}
@@ -222,7 +204,7 @@ void RunMain()
void Run(bool bReload)
{
Log::Init();
g_pLog = new Log();
debug("nzbget %s", Util::VersionRevision());
@@ -231,12 +213,6 @@ void Run(bool bReload)
Thread::Init();
}
#ifdef WIN32
g_pWinConsole = new WinConsole();
g_pWinConsole->InitAppMode();
#endif
g_pServiceCoordinator = new ServiceCoordinator();
g_pServerPool = new ServerPool();
g_pScheduler = new Scheduler();
g_pQueueCoordinator = new QueueCoordinator();
@@ -247,12 +223,10 @@ void Run(bool bReload)
g_pDupeCoordinator = new DupeCoordinator();
g_pUrlCoordinator = new UrlCoordinator();
g_pFeedCoordinator = new FeedCoordinator();
g_pArticleCache = new ArticleCache();
g_pMaintenance = new Maintenance();
g_pQueueScriptCoordinator = new QueueScriptCoordinator();
g_pDiskService = new DiskService();
BootConfig();
debug("Reading options");
g_pOptions = new Options(g_iArgumentCount, *g_szArguments);
#ifndef WIN32
if (g_pOptions->GetUMask() < 01000)
@@ -262,10 +236,16 @@ void Run(bool bReload)
}
#endif
g_pScanner->InitOptions();
g_pQueueScriptCoordinator->InitOptions();
if (g_pOptions->GetServerMode() && g_pOptions->GetCreateLog() && g_pOptions->GetResetLog())
{
debug("Deleting old log-file");
g_pLog->ResetLog();
}
if (g_pCommandLineParser->GetDaemonMode())
g_pLog->InitOptions();
g_pScanner->InitOptions();
if (g_pOptions->GetDaemonMode())
{
#ifdef WIN32
info("nzbget %s service-mode", Util::VersionRevision());
@@ -281,7 +261,7 @@ void Run(bool bReload)
{
info("nzbget %s server-mode", Util::VersionRevision());
}
else if (g_pCommandLineParser->GetRemoteClientMode())
else if (g_pOptions->GetRemoteClientMode())
{
info("nzbget %s remote-mode", Util::VersionRevision());
}
@@ -291,35 +271,33 @@ void Run(bool bReload)
Connection::Init();
}
if (!g_pCommandLineParser->GetRemoteClientMode())
if (!g_pOptions->GetRemoteClientMode())
{
g_pServerPool->InitConnections();
g_pStatMeter->Init();
}
InstallErrorHandler();
#ifdef DEBUG
if (g_pCommandLineParser->GetTestBacktrace())
#ifndef WIN32
#ifdef HAVE_SYS_PRCTL_H
if (g_pOptions->GetDumpCore())
{
TestSegFault();
EnableDumpCore();
}
#endif
#endif
if (g_pCommandLineParser->GetWebGet())
#ifndef WIN32
InstallSignalHandlers();
#ifdef DEBUG
if (g_pOptions->GetTestBacktrace())
{
ProcessWebGet();
return;
}
if (g_pCommandLineParser->GetSigVerify())
{
ProcessSigVerify();
return;
MakeSegFault();
}
#endif
#endif
// client request
if (g_pCommandLineParser->GetClientOperation() != CommandLineParser::opClientNoOperation)
if (g_pOptions->GetClientOperation() != Options::opClientNoOperation)
{
ProcessClientRequest();
Cleanup();
@@ -329,7 +307,6 @@ void Run(bool bReload)
// Setup the network-server
if (g_pOptions->GetServerMode())
{
WebProcessor::Init();
g_pRemoteServer = new RemoteServer(false);
g_pRemoteServer->Start();
@@ -341,7 +318,7 @@ void Run(bool bReload)
}
// Create the frontend
if (!g_pCommandLineParser->GetDaemonMode())
if (!g_pOptions->GetDaemonMode())
{
switch (g_pOptions->GetOutputMode())
{
@@ -366,20 +343,19 @@ void Run(bool bReload)
}
// Starting QueueCoordinator and PrePostProcessor
if (!g_pCommandLineParser->GetRemoteClientMode())
if (!g_pOptions->GetRemoteClientMode())
{
// Standalone-mode
if (!g_pCommandLineParser->GetServerMode())
if (!g_pOptions->GetServerMode())
{
const char* szCategory = g_pCommandLineParser->GetAddCategory() ? g_pCommandLineParser->GetAddCategory() : "";
NZBFile* pNZBFile = new NZBFile(g_pCommandLineParser->GetArgFilename(), szCategory);
if (!pNZBFile->Parse())
const char* szCategory = g_pOptions->GetAddCategory() ? g_pOptions->GetAddCategory() : "";
NZBFile* pNZBFile = NZBFile::Create(g_pOptions->GetArgFilename(), szCategory);
if (!pNZBFile)
{
printf("Parsing NZB-document %s failed\n\n", g_pCommandLineParser->GetArgFilename() ? g_pCommandLineParser->GetArgFilename() : "N/A");
delete pNZBFile;
abort("FATAL ERROR: Parsing NZB-document %s failed\n\n", g_pOptions->GetArgFilename() ? g_pOptions->GetArgFilename() : "N/A");
return;
}
g_pScanner->InitPPParameters(szCategory, pNZBFile->GetNZBInfo()->GetParameters(), false);
g_pScanner->InitPPParameters(szCategory, pNZBFile->GetNZBInfo()->GetParameters());
g_pQueueCoordinator->AddNZBFileToQueue(pNZBFile, NULL, false);
delete pNZBFile;
}
@@ -389,29 +365,16 @@ void Run(bool bReload)
g_pDiskState = new DiskState();
}
#ifdef WIN32
g_pWinConsole->Start();
#endif
g_pQueueCoordinator->Start();
g_pUrlCoordinator->Start();
g_pPrePostProcessor->Start();
g_pFeedCoordinator->Start();
g_pServiceCoordinator->Start();
if (g_pOptions->GetArticleCache() > 0)
{
g_pArticleCache->Start();
}
// enter main program-loop
while (g_pQueueCoordinator->IsRunning() ||
g_pUrlCoordinator->IsRunning() ||
g_pPrePostProcessor->IsRunning() ||
g_pFeedCoordinator->IsRunning() ||
g_pServiceCoordinator->IsRunning() ||
#ifdef WIN32
g_pWinConsole->IsRunning() ||
#endif
g_pArticleCache->IsRunning())
g_pFeedCoordinator->IsRunning())
{
if (!g_pOptions->GetServerMode() &&
!g_pQueueCoordinator->HasMoreJobs() &&
@@ -435,14 +398,6 @@ void Run(bool bReload)
{
g_pFeedCoordinator->Stop();
}
if (!g_pArticleCache->IsStopped())
{
g_pArticleCache->Stop();
}
if (!g_pServiceCoordinator->IsStopped())
{
g_pServiceCoordinator->Stop();
}
}
usleep(100 * 1000);
}
@@ -452,12 +407,8 @@ void Run(bool bReload)
debug("UrlCoordinator stopped");
debug("PrePostProcessor stopped");
debug("FeedCoordinator stopped");
debug("ServiceCoordinator stopped");
debug("ArticleCache stopped");
}
ScriptController::TerminateAll();
// Stop network-server
if (g_pRemoteServer)
{
@@ -498,7 +449,7 @@ void Run(bool bReload)
// Stop Frontend
if (g_pFrontend)
{
if (!g_pCommandLineParser->GetRemoteClientMode())
if (!g_pOptions->GetRemoteClientMode())
{
debug("Stopping Frontend");
g_pFrontend->Stop();
@@ -513,224 +464,121 @@ void Run(bool bReload)
Cleanup();
}
class OptionsExtender : public Options::Extender
{
protected:
#ifdef WIN32
virtual void SetupFirstStart()
{
g_pWinConsole->SetupFirstStart();
}
#endif
virtual void AddNewsServer(int iID, bool bActive, const char* szName, const char* szHost,
int iPort, const char* szUser, const char* szPass, bool bJoinGroup,
bool bTLS, const char* szCipher, int iMaxConnections, int iRetention,
int iLevel, int iGroup)
{
g_pServerPool->AddServer(new NewsServer(iID, bActive, szName, szHost, iPort, szUser, szPass, bJoinGroup,
bTLS, szCipher, iMaxConnections, iRetention, iLevel, iGroup));
}
virtual void AddFeed(int iID, const char* szName, const char* szUrl, int iInterval,
const char* szFilter, bool bBacklog, bool bPauseNzb, const char* szCategory,
int iPriority, const char* szFeedScript)
{
g_pFeedCoordinator->AddFeed(new FeedInfo(iID, szName, szUrl, bBacklog, iInterval, szFilter, bPauseNzb, szCategory, iPriority, szFeedScript));
}
virtual void AddTask(int iID, int iHours, int iMinutes, int iWeekDaysBits,
Options::ESchedulerCommand eCommand, const char* szParam)
{
g_pScheduler->AddTask(new Scheduler::Task(iID, iHours, iMinutes, iWeekDaysBits, (Scheduler::ECommand)eCommand, szParam));
}
} g_OptionsExtender;
void BootConfig()
{
debug("Parsing command line");
g_pCommandLineParser = new CommandLineParser(g_iArgumentCount, (const char**)(*g_szArguments));
if (g_pCommandLineParser->GetPrintVersion())
{
printf("nzbget version: %s\n", Util::VersionRevision());
exit(0);
}
if (g_pCommandLineParser->GetPrintUsage() || g_pCommandLineParser->GetErrors() || g_iArgumentCount <= 1)
{
g_pCommandLineParser->PrintUsage(((const char**)(*g_szArguments))[0]);
exit(0);
}
debug("Reading options");
g_pOptions = new Options((*g_szArguments)[0], g_pCommandLineParser->GetConfigFilename(),
g_pCommandLineParser->GetNoConfig(), (Options::CmdOptList*)g_pCommandLineParser->GetOptionList(),
&g_OptionsExtender);
g_pOptions->SetRemoteClientMode(g_pCommandLineParser->GetRemoteClientMode());
g_pOptions->SetServerMode(g_pCommandLineParser->GetServerMode());
g_pOptions->SetPauseDownload(g_pCommandLineParser->GetPauseDownload());
g_pLog->InitOptions();
if (g_pOptions->GetFatalError())
{
exit(1);
}
else if (g_pOptions->GetConfigErrors() &&
g_pCommandLineParser->GetClientOperation() == CommandLineParser::opClientNoOperation)
{
info("Pausing all activities due to errors in configuration");
g_pOptions->SetPauseDownload(true);
g_pOptions->SetPausePostProcess(true);
g_pOptions->SetPauseScan(true);
}
g_pServerPool->SetTimeout(g_pOptions->GetArticleTimeout());
g_pServerPool->SetRetryInterval(g_pOptions->GetRetryInterval());
g_pScriptConfig = new ScriptConfig();
}
void ProcessClientRequest()
{
RemoteClient* Client = new RemoteClient();
switch (g_pCommandLineParser->GetClientOperation())
switch (g_pOptions->GetClientOperation())
{
case CommandLineParser::opClientRequestListFiles:
Client->RequestServerList(true, false, g_pCommandLineParser->GetMatchMode() == CommandLineParser::mmRegEx ? g_pCommandLineParser->GetEditQueueText() : NULL);
case Options::opClientRequestListFiles:
Client->RequestServerList(true, false, g_pOptions->GetMatchMode() == Options::mmRegEx ? g_pOptions->GetEditQueueText() : NULL);
break;
case CommandLineParser::opClientRequestListGroups:
Client->RequestServerList(false, true, g_pCommandLineParser->GetMatchMode() == CommandLineParser::mmRegEx ? g_pCommandLineParser->GetEditQueueText() : NULL);
case Options::opClientRequestListGroups:
Client->RequestServerList(false, true, g_pOptions->GetMatchMode() == Options::mmRegEx ? g_pOptions->GetEditQueueText() : NULL);
break;
case CommandLineParser::opClientRequestListStatus:
case Options::opClientRequestListStatus:
Client->RequestServerList(false, false, NULL);
break;
case CommandLineParser::opClientRequestDownloadPause:
case Options::opClientRequestDownloadPause:
Client->RequestServerPauseUnpause(true, eRemotePauseUnpauseActionDownload);
break;
case CommandLineParser::opClientRequestDownloadUnpause:
case Options::opClientRequestDownloadUnpause:
Client->RequestServerPauseUnpause(false, eRemotePauseUnpauseActionDownload);
break;
case CommandLineParser::opClientRequestSetRate:
Client->RequestServerSetDownloadRate(g_pCommandLineParser->GetSetRate());
case Options::opClientRequestSetRate:
Client->RequestServerSetDownloadRate(g_pOptions->GetSetRate());
break;
case CommandLineParser::opClientRequestDumpDebug:
case Options::opClientRequestDumpDebug:
Client->RequestServerDumpDebug();
break;
case CommandLineParser::opClientRequestEditQueue:
Client->RequestServerEditQueue((DownloadQueue::EEditAction)g_pCommandLineParser->GetEditQueueAction(),
g_pCommandLineParser->GetEditQueueOffset(), g_pCommandLineParser->GetEditQueueText(),
g_pCommandLineParser->GetEditQueueIDList(), g_pCommandLineParser->GetEditQueueIDCount(),
g_pCommandLineParser->GetEditQueueNameList(), (eRemoteMatchMode)g_pCommandLineParser->GetMatchMode());
case Options::opClientRequestEditQueue:
Client->RequestServerEditQueue((DownloadQueue::EEditAction)g_pOptions->GetEditQueueAction(),
g_pOptions->GetEditQueueOffset(), g_pOptions->GetEditQueueText(),
g_pOptions->GetEditQueueIDList(), g_pOptions->GetEditQueueIDCount(),
g_pOptions->GetEditQueueNameList(), (eRemoteMatchMode)g_pOptions->GetMatchMode());
break;
case CommandLineParser::opClientRequestLog:
Client->RequestServerLog(g_pCommandLineParser->GetLogLines());
case Options::opClientRequestLog:
Client->RequestServerLog(g_pOptions->GetLogLines());
break;
case CommandLineParser::opClientRequestShutdown:
case Options::opClientRequestShutdown:
Client->RequestServerShutdown();
break;
case CommandLineParser::opClientRequestReload:
case Options::opClientRequestReload:
Client->RequestServerReload();
break;
case CommandLineParser::opClientRequestDownload:
Client->RequestServerDownload(g_pCommandLineParser->GetAddNZBFilename(), g_pCommandLineParser->GetArgFilename(),
g_pCommandLineParser->GetAddCategory(), g_pCommandLineParser->GetAddTop(), g_pCommandLineParser->GetAddPaused(), g_pCommandLineParser->GetAddPriority(),
g_pCommandLineParser->GetAddDupeKey(), g_pCommandLineParser->GetAddDupeMode(), g_pCommandLineParser->GetAddDupeScore());
case Options::opClientRequestDownload:
Client->RequestServerDownload(g_pOptions->GetArgFilename(), g_pOptions->GetAddCategory(), g_pOptions->GetAddTop(), g_pOptions->GetAddPaused(), g_pOptions->GetAddPriority());
break;
case CommandLineParser::opClientRequestVersion:
case Options::opClientRequestVersion:
Client->RequestServerVersion();
break;
case CommandLineParser::opClientRequestPostQueue:
case Options::opClientRequestPostQueue:
Client->RequestPostQueue();
break;
case CommandLineParser::opClientRequestWriteLog:
Client->RequestWriteLog(g_pCommandLineParser->GetWriteLogKind(), g_pCommandLineParser->GetLastArg());
case Options::opClientRequestWriteLog:
Client->RequestWriteLog(g_pOptions->GetWriteLogKind(), g_pOptions->GetLastArg());
break;
case CommandLineParser::opClientRequestScanAsync:
case Options::opClientRequestScanAsync:
Client->RequestScan(false);
break;
case CommandLineParser::opClientRequestScanSync:
case Options::opClientRequestScanSync:
Client->RequestScan(true);
break;
case CommandLineParser::opClientRequestPostPause:
case Options::opClientRequestPostPause:
Client->RequestServerPauseUnpause(true, eRemotePauseUnpauseActionPostProcess);
break;
case CommandLineParser::opClientRequestPostUnpause:
case Options::opClientRequestPostUnpause:
Client->RequestServerPauseUnpause(false, eRemotePauseUnpauseActionPostProcess);
break;
case CommandLineParser::opClientRequestScanPause:
case Options::opClientRequestScanPause:
Client->RequestServerPauseUnpause(true, eRemotePauseUnpauseActionScan);
break;
case CommandLineParser::opClientRequestScanUnpause:
case Options::opClientRequestScanUnpause:
Client->RequestServerPauseUnpause(false, eRemotePauseUnpauseActionScan);
break;
case CommandLineParser::opClientRequestHistory:
case CommandLineParser::opClientRequestHistoryAll:
Client->RequestHistory(g_pCommandLineParser->GetClientOperation() == CommandLineParser::opClientRequestHistoryAll);
case Options::opClientRequestHistory:
Client->RequestHistory();
break;
case CommandLineParser::opClientNoOperation:
case Options::opClientRequestDownloadUrl:
Client->RequestServerDownloadUrl(g_pOptions->GetLastArg(), g_pOptions->GetAddNZBFilename(), g_pOptions->GetAddCategory(), g_pOptions->GetAddTop(), g_pOptions->GetAddPaused(), g_pOptions->GetAddPriority());
break;
case Options::opClientNoOperation:
break;
}
delete Client;
}
void ProcessWebGet()
{
WebDownloader downloader;
downloader.SetURL(g_pCommandLineParser->GetLastArg());
downloader.SetForce(true);
downloader.SetRetry(false);
downloader.SetOutputFilename(g_pCommandLineParser->GetWebGetFilename());
downloader.SetInfoName("WebGet");
WebDownloader::EStatus eStatus = downloader.DownloadWithRedirects(5);
bool bOK = eStatus == WebDownloader::adFinished;
exit(bOK ? 0 : 1);
}
void ProcessSigVerify()
{
#ifdef HAVE_OPENSSL
bool bOK = Maintenance::VerifySignature(g_pCommandLineParser->GetLastArg(),
g_pCommandLineParser->GetSigFilename(), g_pCommandLineParser->GetPubKeyFilename());
exit(bOK ? 93 : 1);
#else
printf("ERROR: Could not verify signature, the program was compiled without OpenSSL support\n");
exit(1);
#endif
}
void ExitProc()
{
if (!g_bReloading)
{
info("Stopping, please wait...");
}
if (g_pCommandLineParser->GetRemoteClientMode())
if (g_pOptions->GetRemoteClientMode())
{
if (g_pFrontend)
{
@@ -743,16 +591,10 @@ void ExitProc()
if (g_pQueueCoordinator)
{
debug("Stopping QueueCoordinator");
g_pServiceCoordinator->Stop();
g_pQueueCoordinator->Stop();
g_pUrlCoordinator->Stop();
g_pPrePostProcessor->Stop();
g_pFeedCoordinator->Stop();
g_pArticleCache->Stop();
g_pQueueScriptCoordinator->Stop();
#ifdef WIN32
g_pWinConsole->Stop();
#endif
}
}
}
@@ -764,6 +606,114 @@ void Reload()
ExitProc();
}
#ifndef WIN32
#ifdef DEBUG
typedef void(*sighandler)(int);
std::vector<sighandler> SignalProcList;
#endif
/*
* Signal handler
*/
void SignalProc(int iSignal)
{
switch (iSignal)
{
case SIGINT:
signal(SIGINT, SIG_DFL); // Reset the signal handler
ExitProc();
break;
case SIGTERM:
signal(SIGTERM, SIG_DFL); // Reset the signal handler
ExitProc();
break;
case SIGCHLD:
// ignoring
break;
#ifdef DEBUG
case SIGSEGV:
signal(SIGSEGV, SIG_DFL); // Reset the signal handler
PrintBacktrace();
break;
#endif
}
}
void InstallSignalHandlers()
{
signal(SIGINT, SignalProc);
signal(SIGTERM, SignalProc);
signal(SIGPIPE, SIG_IGN);
#ifdef DEBUG
signal(SIGSEGV, SignalProc);
#endif
#ifdef SIGCHLD_HANDLER
// it could be necessary on some systems to activate a handler for SIGCHLD
// however it make troubles on other systems and is deactivated by default
signal(SIGCHLD, SignalProc);
#endif
}
void PrintBacktrace()
{
#ifdef HAVE_BACKTRACE
printf("Segmentation fault, tracing...\n");
void *array[100];
size_t size;
char **strings;
size_t i;
size = backtrace(array, 100);
strings = backtrace_symbols(array, size);
// first trace to screen
printf("Obtained %zd stack frames\n", size);
for (i = 0; i < size; i++)
{
printf("%s\n", strings[i]);
}
// then trace to log
error("Segmentation fault, tracing...");
error("Obtained %zd stack frames", size);
for (i = 0; i < size; i++)
{
error("%s", strings[i]);
}
free(strings);
#else
error("Segmentation fault");
#endif
}
#ifdef DEBUG
void MakeSegFault()
{
char* N = NULL;
strcpy(N, "");
}
#endif
#ifdef HAVE_SYS_PRCTL_H
/**
* activates the creation of core-files
*/
void EnableDumpCore()
{
rlimit rlim;
rlim.rlim_cur= RLIM_INFINITY;
rlim.rlim_max= RLIM_INFINITY;
setrlimit(RLIMIT_CORE, &rlim);
prctl(PR_SET_DUMPABLE, 1);
}
#endif
#endif
void Cleanup()
{
debug("Cleaning up global objects");
@@ -818,31 +768,16 @@ void Cleanup()
debug("Deleting Options");
if (g_pOptions)
{
if (g_pCommandLineParser->GetDaemonMode() && !g_bReloading)
if (g_pOptions->GetDaemonMode() && !g_bReloading)
{
info("Deleting lock file");
remove(g_pOptions->GetLockFile());
}
delete g_pOptions;
g_pOptions = NULL;
}
debug("Options deleted");
debug("Deleting CommandLineParser");
if (g_pCommandLineParser)
{
delete g_pCommandLineParser;
g_pCommandLineParser = NULL;
}
debug("CommandLineParser deleted");
debug("Deleting ScriptConfig");
if (g_pScriptConfig)
{
delete g_pScriptConfig;
g_pScriptConfig = NULL;
}
debug("ScriptConfig deleted");
debug("Deleting ServerPool");
delete g_pServerPool;
g_pServerPool = NULL;
@@ -858,16 +793,6 @@ void Cleanup()
g_pFeedCoordinator = NULL;
debug("FeedCoordinator deleted");
debug("Deleting ArticleCache");
delete g_pArticleCache;
g_pArticleCache = NULL;
debug("ArticleCache deleted");
debug("Deleting QueueScriptCoordinator");
delete g_pQueueScriptCoordinator;
g_pQueueScriptCoordinator = NULL;
debug("QueueScriptCoordinator deleted");
debug("Deleting Maintenance");
delete g_pMaintenance;
g_pMaintenance = NULL;
@@ -878,105 +803,57 @@ void Cleanup()
g_pStatMeter = NULL;
debug("StatMeter deleted");
debug("Deleting ServiceCoordinator");
delete g_pServiceCoordinator;
g_pServiceCoordinator = NULL;
debug("ServiceCoordinator deleted");
debug("Deleting DiskService");
delete g_pDiskService;
g_pDiskService = NULL;
debug("DiskService deleted");
if (!g_bReloading)
{
Connection::Final();
Thread::Final();
}
#ifdef WIN32
delete g_pWinConsole;
g_pWinConsole = NULL;
#endif
debug("Global objects cleaned up");
Log::Final();
delete g_pLog;
g_pLog = NULL;
}
#ifndef WIN32
void Daemonize()
{
int f = fork();
if (f < 0) exit(1); /* fork error */
if (f > 0) exit(0); /* parent exits */
int i, lfp;
char str[10];
if (getppid() == 1) return; /* already a daemon */
i = fork();
if (i < 0) exit(1); /* fork error */
if (i > 0) exit(0); /* parent exits */
/* child (daemon) continues */
// obtain a new process group
setsid();
// close all descriptors
for (int i = getdtablesize(); i >= 0; --i)
{
close(i);
}
// handle standart I/O
int d = open("/dev/null", O_RDWR);
dup(d);
dup(d);
// change running directory
chdir(g_pOptions->GetDestDir());
// set up lock-file
int lfp = -1;
if (!Util::EmptyStr(g_pOptions->GetLockFile()))
{
lfp = open(g_pOptions->GetLockFile(), O_RDWR | O_CREAT, 0640);
if (lfp < 0)
{
error("Starting daemon failed: could not create lock-file %s", g_pOptions->GetLockFile());
exit(1);
}
if (lockf(lfp, F_TLOCK, 0) < 0)
{
error("Starting daemon failed: could not acquire lock on lock-file %s", g_pOptions->GetLockFile());
exit(1);
}
}
setsid(); /* obtain a new process group */
for (i = getdtablesize();i >= 0;--i) close(i); /* close all descriptors */
i = open("/dev/null", O_RDWR); dup(i); dup(i); /* handle standart I/O */
chdir(g_pOptions->GetDestDir()); /* change running directory */
lfp = open(g_pOptions->GetLockFile(), O_RDWR | O_CREAT, 0640);
if (lfp < 0) exit(1); /* can not open */
if (lockf(lfp, F_TLOCK, 0) < 0) exit(0); /* can not lock */
/* Drop user if there is one, and we were run as root */
if (getuid() == 0 || geteuid() == 0)
if ( getuid() == 0 || geteuid() == 0 )
{
struct passwd *pw = getpwnam(g_pOptions->GetDaemonUsername());
if (pw)
{
// Change owner of lock file
fchown(lfp, pw->pw_uid, pw->pw_gid);
// Set aux groups to null.
setgroups(0, (const gid_t*)0);
// Set primary group.
setgid(pw->pw_gid);
// Try setting aux groups correctly - not critical if this fails.
initgroups(g_pOptions->GetDaemonUsername(), pw->pw_gid);
// Finally, set uid.
fchown(lfp, pw->pw_uid, pw->pw_gid); /* change owner of lock file */
setgroups( 0, (const gid_t*) 0 ); /* Set aux groups to null. */
setgid(pw->pw_gid); /* Set primary group. */
/* Try setting aux groups correctly - not critical if this fails. */
initgroups( g_pOptions->GetDaemonUsername(),pw->pw_gid);
/* Finally, set uid. */
setuid(pw->pw_uid);
}
}
// record pid to lockfile
if (lfp > -1)
{
char str[10];
sprintf(str, "%d\n", getpid());
write(lfp, str, strlen(str));
}
// ignore unwanted signals
signal(SIGCHLD, SIG_IGN);
signal(SIGTSTP, SIG_IGN);
/* first instance continues */
sprintf(str, "%d\n", getpid());
write(lfp, str, strlen(str)); /* record pid to lockfile */
signal(SIGCHLD, SIG_IGN); /* ignore child */
signal(SIGTSTP, SIG_IGN); /* ignore tty signals */
signal(SIGTTOU, SIG_IGN);
signal(SIGTTIN, SIG_IGN);
}
@@ -997,4 +874,3 @@ void DisableCout()
std::cout.rdbuf(&NullStreamBufInstance);
}
#endif

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2013 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -39,9 +39,7 @@
#define gmtime_r(time, tm) gmtime_s(tm, time)
#define strtok_r(str, delim, saveptr) strtok_s(str, delim, saveptr)
#define strerror_r(errnum, buffer, size) strerror_s(buffer, size, errnum)
#if (_MSC_VER < 1600)
#define int32_t __int32
#endif
#define mkdir(dir, flags) _mkdir(dir)
#define rmdir _rmdir
#define strcasecmp(a, b) _stricmp(a, b)
@@ -52,6 +50,7 @@
#define S_ISREG(mode) __S_ISTYPE((mode), _S_IFREG)
#define S_DIRMODE NULL
#define usleep(usec) Sleep((usec) / 1000)
#define gettimeofday(tm, ignore) _ftime(tm)
#define socklen_t int
#define SHUT_WR 0x01
#define SHUT_RDWR 0x02
@@ -60,11 +59,6 @@
#define LINE_ENDING "\r\n"
#define pid_t int
#define atoll _atoi64
#define fseek _fseeki64
#define ftell _ftelli64
#if _MSC_VER < 1800 // va_copy is available in vc2013 and onwards
#define va_copy(d,s) ((d) = (s))
#endif
#ifndef FSCTL_SET_SPARSE
#define FSCTL_SET_SPARSE 590020
#endif
@@ -75,11 +69,6 @@
#define FOPEN_AB "abN"
#define FOPEN_ABP "ab+N"
#ifdef DEBUG
// redefine "exit" to avoid printing memory leaks report when terminated because of wrong command line switches
#define exit(code) ExitProcess(code)
#endif
#pragma warning(disable:4800) // 'type' : forcing value to bool 'true' or 'false' (performance warning)
#pragma warning(disable:4267) // 'var' : conversion from 'size_t' to 'type', possible loss of data

View File

File diff suppressed because it is too large Load Diff

View File

@@ -34,7 +34,6 @@
#include "Thread.h"
#include "NNTPConnection.h"
#include "Decoder.h"
#include "ArticleWriter.h"
class ArticleDownloader : public Thread, public Subject
{
@@ -48,20 +47,13 @@ public:
adFailed,
adRetry,
adCrcError,
adDecoding,
adJoining,
adJoined,
adNotFound,
adConnectError,
adFatalError
};
class ArticleWriterImpl : public ArticleWriter
{
private:
ArticleDownloader* m_pOwner;
protected:
virtual void SetLastUpdateTimeNow() { m_pOwner->SetLastUpdateTimeNow(); }
public:
void SetOwner(ArticleDownloader* pOwner) { m_pOwner = pOwner; }
};
private:
FileInfo* m_pFileInfo;
@@ -69,25 +61,31 @@ private:
NNTPConnection* m_pConnection;
EStatus m_eStatus;
Mutex m_mutexConnection;
char* m_szInfoName;
char m_szConnectionName[250];
const char* m_szResultFilename;
char* m_szTempFilename;
char* m_szArticleFilename;
char* m_szInfoName;
char* m_szOutputFilename;
time_t m_tLastUpdateTime;
Decoder::EFormat m_eFormat;
YDecoder m_YDecoder;
UDecoder m_UDecoder;
ArticleWriterImpl m_ArticleWriter;
FILE* m_pOutFile;
bool m_bDuplicate;
ServerStatList m_ServerStats;
bool m_bWritingStarted;
int m_iDownloadedSize;
EStatus Download();
bool Write(char* szLine, int iLen);
bool PrepareFile(char* szLine);
bool CreateOutputFile(int iSize);
void BuildOutputFilename();
EStatus DecodeCheck();
void FreeConnection(bool bKeepConnected);
EStatus CheckResponse(const char* szResponse, const char* szComment);
void SetStatus(EStatus eStatus) { m_eStatus = eStatus; }
bool Write(char* szLine, int iLen);
void AddServerData();
const char* GetTempFilename() { return m_szTempFilename; }
void SetTempFilename(const char* v);
void SetOutputFilename(const char* v);
public:
ArticleDownloader();
@@ -104,12 +102,11 @@ public:
time_t GetLastUpdateTime() { return m_tLastUpdateTime; }
void SetLastUpdateTimeNow() { m_tLastUpdateTime = ::time(NULL); }
const char* GetArticleFilename() { return m_szArticleFilename; }
void SetInfoName(const char* szInfoName);
void SetInfoName(const char* v);
const char* GetInfoName() { return m_szInfoName; }
const char* GetConnectionName() { return m_szConnectionName; }
void CompleteFileParts();
static bool MoveCompletedFiles(NZBInfo* pNZBInfo, const char* szOldDestDir);
void SetConnection(NNTPConnection* pConnection) { m_pConnection = pConnection; }
void CompleteFileParts() { m_ArticleWriter.CompleteFileParts(); }
int GetDownloadedSize() { return m_iDownloadedSize; }
void LogDebugInfo();
};

View File

File diff suppressed because it is too large Load Diff

View File

@@ -1,104 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2014-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifndef ARTICLEWRITER_H
#define ARTICLEWRITER_H
#include "DownloadInfo.h"
#include "Decoder.h"
class ArticleWriter
{
private:
FileInfo* m_pFileInfo;
ArticleInfo* m_pArticleInfo;
FILE* m_pOutFile;
char* m_szTempFilename;
char* m_szOutputFilename;
const char* m_szResultFilename;
Decoder::EFormat m_eFormat;
char* m_pArticleData;
long long m_iArticleOffset;
int m_iArticleSize;
int m_iArticlePtr;
bool m_bFlushing;
bool m_bDuplicate;
char* m_szInfoName;
bool PrepareFile(char* szLine);
bool CreateOutputFile(long long iSize);
void BuildOutputFilename();
bool IsFileCached();
void SetWriteBuffer(FILE* pOutFile, int iRecSize);
protected:
virtual void SetLastUpdateTimeNow() {}
public:
ArticleWriter();
~ArticleWriter();
void SetInfoName(const char* szInfoName);
void SetFileInfo(FileInfo* pFileInfo) { m_pFileInfo = pFileInfo; }
void SetArticleInfo(ArticleInfo* pArticleInfo) { m_pArticleInfo = pArticleInfo; }
void Prepare();
bool Start(Decoder::EFormat eFormat, const char* szFilename, long long iFileSize, long long iArticleOffset, int iArticleSize);
bool Write(char* szBufffer, int iLen);
void Finish(bool bSuccess);
bool GetDuplicate() { return m_bDuplicate; }
void CompleteFileParts();
static bool MoveCompletedFiles(NZBInfo* pNZBInfo, const char* szOldDestDir);
void FlushCache();
};
class ArticleCache : public Thread
{
private:
size_t m_iAllocated;
bool m_bFlushing;
Mutex m_mutexAlloc;
Mutex m_mutexFlush;
Mutex m_mutexContent;
FileInfo* m_pFileInfo;
bool CheckFlush(bool bFlushEverything);
public:
ArticleCache();
virtual void Run();
void* Alloc(int iSize);
void* Realloc(void* buf, int iOldSize, int iNewSize);
void Free(int iSize);
void LockFlush();
void UnlockFlush();
void LockContent() { m_mutexContent.Lock(); }
void UnlockContent() { m_mutexContent.Unlock(); }
bool GetFlushing() { return m_bFlushing; }
size_t GetAllocated() { return m_iAllocated; }
bool FileBusy(FileInfo* pFileInfo) { return pFileInfo == m_pFileInfo; }
};
extern ArticleCache* g_pArticleCache;
#endif

View File

@@ -1,7 +1,8 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -44,11 +45,14 @@
#include "Util.h"
const char* Decoder::FormatNames[] = { "Unknown", "yEnc", "UU" };
unsigned int YDecoder::crc_tab[256];
Decoder::Decoder()
{
debug("Creating Decoder");
m_szSrcFilename = NULL;
m_szDestFilename = NULL;
m_szArticleFilename = NULL;
}
@@ -65,14 +69,14 @@ void Decoder::Clear()
m_szArticleFilename = NULL;
}
Decoder::EFormat Decoder::DetectFormat(const char* buffer, int len, bool inBody)
Decoder::EFormat Decoder::DetectFormat(const char* buffer, int len)
{
if (!strncmp(buffer, "=ybegin ", 8))
{
return efYenc;
}
if (inBody && (len == 62 || len == 63) && (buffer[62] == '\n' || buffer[62] == '\r') && *buffer == 'M')
if ((len == 62 || len == 63) && (buffer[62] == '\n' || buffer[62] == '\r') && *buffer == 'M')
{
return efUx;
}
@@ -103,6 +107,17 @@ Decoder::EFormat Decoder::DetectFormat(const char* buffer, int len, bool inBody)
* YDecoder: fast implementation of yEnc-Decoder
*/
void YDecoder::Init()
{
debug("Initializing global decoder");
crc32gentab();
}
void YDecoder::Final()
{
debug("Finalizing global Decoder");
}
YDecoder::YDecoder()
{
Clear();
@@ -120,13 +135,69 @@ void YDecoder::Clear()
m_lExpectedCRC = 0;
m_lCalculatedCRC = 0xFFFFFFFF;
m_iBegin = 0;
m_iEnd = 0;
m_iEnd = 0xFFFFFFFF;
m_iSize = 0;
m_iEndSize = 0;
m_bAutoSeek = false;
m_bNeedSetPos = false;
m_bCrcCheck = false;
}
int YDecoder::DecodeBuffer(char* buffer, int len)
/* from crc32.c (http://www.koders.com/c/fid699AFE0A656F0022C9D6B9D1743E697B69CE5815.aspx)
*
* (c) 1999,2000 Krzysztof Dabrowski
* (c) 1999,2000 ElysiuM deeZine
* Released under GPL (thanks)
*
* chksum_crc32gentab() -- to a global crc_tab[256], this one will
* calculate the crcTable for crc32-checksums.
* it is generated to the polynom [..]
*/
void YDecoder::crc32gentab()
{
unsigned long crc, poly;
int i, j;
poly = 0xEDB88320L;
for (i = 0; i < 256; i++)
{
crc = i;
for (j = 8; j > 0; j--)
{
if (crc & 1)
{
crc = (crc >> 1) ^ poly;
}
else
{
crc >>= 1;
}
}
crc_tab[i] = crc;
}
}
/* This is modified version of chksum_crc() from
* crc32.c (http://www.koders.com/c/fid699AFE0A656F0022C9D6B9D1743E697B69CE5815.aspx)
* (c) 1999,2000 Krzysztof Dabrowski
* (c) 1999,2000 ElysiuM deeZine
*
* chksum_crc() -- to a given block, this one calculates the
* crc32-checksum until the length is
* reached. the crc32-checksum will be
* the result.
*/
unsigned long YDecoder::crc32m(unsigned long startCrc, unsigned char *block, unsigned int length)
{
register unsigned long crc = startCrc;
for (unsigned long i = 0; i < length; i++)
{
crc = ((crc >> 8) & 0x00FFFFFF) ^ crc_tab[(crc ^ *block++) & 0xFF];
}
return crc;
}
unsigned int YDecoder::DecodeBuffer(char* buffer)
{
if (m_bBody && !m_bEnd)
{
@@ -144,7 +215,7 @@ int YDecoder::DecodeBuffer(char* buffer, int len)
if (pb)
{
pb += 6; //=strlen(" size=")
m_iEndSize = (long long)atoll(pb);
m_iEndSize = (int)atoi(pb);
}
return 0;
}
@@ -176,9 +247,9 @@ BreakLoop:
if (m_bCrcCheck)
{
m_lCalculatedCRC = Util::Crc32m(m_lCalculatedCRC, (unsigned char *)buffer, (unsigned int)(optr - buffer));
m_lCalculatedCRC = crc32m(m_lCalculatedCRC, (unsigned char *)buffer, (unsigned int)(optr - buffer));
}
return optr - buffer;
return (unsigned int)(optr - buffer);
}
else
{
@@ -200,7 +271,7 @@ BreakLoop:
if (pb)
{
pb += 6; //=strlen(" size=")
m_iSize = (long long)atoll(pb);
m_iSize = (int)atoi(pb);
}
m_bPart = strstr(buffer, " part=");
if (!m_bPart)
@@ -218,13 +289,13 @@ BreakLoop:
if (pb)
{
pb += 7; //=strlen(" begin=")
m_iBegin = (long long)atoll(pb);
m_iBegin = (int)atoi(pb);
}
pb = strstr(buffer, " end=");
if (pb)
{
pb += 5; //=strlen(" end=")
m_iEnd = (long long)atoll(pb);
m_iEnd = (int)atoi(pb);
}
}
}
@@ -232,6 +303,28 @@ BreakLoop:
return 0;
}
bool YDecoder::Write(char* buffer, int len, FILE* outfile)
{
unsigned int wcnt = DecodeBuffer(buffer);
if (wcnt > 0)
{
if (m_bNeedSetPos)
{
if (m_iBegin == 0 || m_iEnd == 0xFFFFFFFF || !outfile)
{
return false;
}
if (fseek(outfile, m_iBegin - 1, SEEK_SET))
{
return false;
}
m_bNeedSetPos = false;
}
fwrite(buffer, 1, wcnt, outfile);
}
return true;
}
Decoder::EStatus YDecoder::Check()
{
m_lCalculatedCRC ^= 0xFFFFFFFF;
@@ -286,7 +379,7 @@ void UDecoder::Clear()
#define UU_DECODE_CHAR(c) (c == '`' ? 0 : (((c) - ' ') & 077))
int UDecoder::DecodeBuffer(char* buffer, int len)
unsigned int UDecoder::DecodeBuffer(char* buffer, int len)
{
if (!m_bBody)
{
@@ -353,12 +446,22 @@ int UDecoder::DecodeBuffer(char* buffer, int len)
}
}
return optr - buffer;
return (unsigned int)(optr - buffer);
}
return 0;
}
bool UDecoder::Write(char* buffer, int len, FILE* outfile)
{
unsigned int wcnt = DecodeBuffer(buffer, len);
if (wcnt > 0)
{
fwrite(buffer, 1, wcnt, outfile);
}
return true;
}
Decoder::EStatus UDecoder::Check()
{
if (!m_bBody)

View File

@@ -1,7 +1,8 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2008 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -49,6 +50,8 @@ public:
static const char* FormatNames[];
protected:
const char* m_szSrcFilename;
const char* m_szDestFilename;
char* m_szArticleFilename;
public:
@@ -56,14 +59,17 @@ public:
virtual ~Decoder();
virtual EStatus Check() = 0;
virtual void Clear();
virtual int DecodeBuffer(char* buffer, int len) = 0;
virtual bool Write(char* buffer, int len, FILE* outfile) = 0;
void SetSrcFilename(const char* szSrcFilename) { m_szSrcFilename = szSrcFilename; }
void SetDestFilename(const char* szDestFilename) { m_szDestFilename = szDestFilename; }
const char* GetArticleFilename() { return m_szArticleFilename; }
static EFormat DetectFormat(const char* buffer, int len, bool inBody);
static EFormat DetectFormat(const char* buffer, int len);
};
class YDecoder: public Decoder
{
protected:
static unsigned int crc_tab[256];
bool m_bBegin;
bool m_bPart;
bool m_bBody;
@@ -71,23 +77,28 @@ protected:
bool m_bCrc;
unsigned long m_lExpectedCRC;
unsigned long m_lCalculatedCRC;
long long m_iBegin;
long long m_iEnd;
long long m_iSize;
long long m_iEndSize;
unsigned long m_iBegin;
unsigned long m_iEnd;
unsigned long m_iSize;
unsigned long m_iEndSize;
bool m_bAutoSeek;
bool m_bNeedSetPos;
bool m_bCrcCheck;
unsigned int DecodeBuffer(char* buffer);
static void crc32gentab();
unsigned long crc32m(unsigned long startCrc, unsigned char *block, unsigned int length);
public:
YDecoder();
virtual EStatus Check();
virtual void Clear();
virtual int DecodeBuffer(char* buffer, int len);
virtual bool Write(char* buffer, int len, FILE* outfile);
void SetAutoSeek(bool bAutoSeek) { m_bAutoSeek = m_bNeedSetPos = bAutoSeek; }
void SetCrcCheck(bool bCrcCheck) { m_bCrcCheck = bCrcCheck; }
long long GetBegin() { return m_iBegin; }
long long GetEnd() { return m_iEnd; }
long long GetSize() { return m_iSize; }
unsigned long GetExpectedCrc() { return m_lExpectedCRC; }
unsigned long GetCalculatedCrc() { return m_lCalculatedCRC; }
static void Init();
static void Final();
};
class UDecoder: public Decoder
@@ -96,11 +107,13 @@ private:
bool m_bBody;
bool m_bEnd;
unsigned int DecodeBuffer(char* buffer, int len);
public:
UDecoder();
virtual EStatus Check();
virtual void Clear();
virtual int DecodeBuffer(char* buffer, int len);
virtual bool Write(char* buffer, int len, FILE* outfile);
};
#endif

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2013 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -99,8 +99,8 @@ bool NNTPConnection::Authenticate()
{
if (strlen(m_pNewsServer->GetUser()) == 0 || strlen(m_pNewsServer->GetPassword()) == 0)
{
ReportError("Could not connect to %s: server requested authorization but username/password are not set in settings",
m_pNewsServer->GetHost(), false, 0);
error("%c%s (%s) requested authorization but username/password are not set in settings",
toupper(m_pNewsServer->GetName()[0]), m_pNewsServer->GetName() + 1, m_pNewsServer->GetHost());
m_bAuthError = true;
return false;
}
@@ -125,7 +125,7 @@ bool NNTPConnection::AuthInfoUser(int iRecur)
char* answer = ReadLine(m_szLineBuf, CONNECTION_LINEBUFFER_SIZE, NULL);
if (!answer)
{
ReportErrorAnswer("Authorization for %s (%s) failed: Connection closed by remote host", NULL);
ReportErrorAnswer("Authorization for server%i (%s) failed: Connection closed by remote host", NULL);
return false;
}
@@ -147,7 +147,7 @@ bool NNTPConnection::AuthInfoUser(int iRecur)
if (GetStatus() != csCancelled)
{
ReportErrorAnswer("Authorization for %s (%s) failed: %s", answer);
ReportErrorAnswer("Authorization for server%i (%s) failed (Answer: %s)", answer);
}
return false;
}
@@ -168,7 +168,7 @@ bool NNTPConnection::AuthInfoPass(int iRecur)
char* answer = ReadLine(m_szLineBuf, CONNECTION_LINEBUFFER_SIZE, NULL);
if (!answer)
{
ReportErrorAnswer("Authorization failed for %s (%s): Connection closed by remote host", NULL);
ReportErrorAnswer("Authorization for server%i (%s) failed: Connection closed by remote host", NULL);
return false;
}
else if (!strncmp(answer, "2", 1))
@@ -185,7 +185,7 @@ bool NNTPConnection::AuthInfoPass(int iRecur)
if (GetStatus() != csCancelled)
{
ReportErrorAnswer("Authorization for %s (%s) failed: %s", answer);
ReportErrorAnswer("Authorization for server%i (%s) failed (Answer: %s)", answer);
}
return false;
}
@@ -237,14 +237,14 @@ bool NNTPConnection::Connect()
if (!answer)
{
ReportErrorAnswer("Connection to %s (%s) failed: Connection closed by remote host", NULL);
ReportErrorAnswer("Connection to server%i (%s) failed: Connection closed by remote host", NULL);
Disconnect();
return false;
}
if (strncmp(answer, "2", 1))
{
ReportErrorAnswer("Connection to %s (%s) failed: %s", answer);
ReportErrorAnswer("Connection to server%i (%s) failed (Answer: %s)", answer);
Disconnect();
return false;
}
@@ -264,10 +264,7 @@ bool NNTPConnection::Disconnect()
{
if (m_eStatus == csConnected)
{
if (!m_bBroken)
{
Request("quit\r\n");
}
Request("quit\r\n");
free(m_szActiveGroup);
m_szActiveGroup = NULL;
}
@@ -277,7 +274,7 @@ bool NNTPConnection::Disconnect()
void NNTPConnection::ReportErrorAnswer(const char* szMsgPrefix, const char* szAnswer)
{
char szErrStr[1024];
snprintf(szErrStr, 1024, szMsgPrefix, m_pNewsServer->GetName(), m_pNewsServer->GetHost(), szAnswer);
snprintf(szErrStr, 1024, szMsgPrefix, m_pNewsServer->GetID(), m_pNewsServer->GetHost(), szAnswer);
szErrStr[1024-1] = '\0';
ReportError(szErrStr, NULL, false, 0);

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2008 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -53,7 +53,6 @@ public:
const char* Request(const char* req);
const char* JoinGroup(const char* grp);
bool GetAuthError() { return m_bAuthError; }
};
#endif

View File

@@ -2,7 +2,7 @@
* This file if part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2013 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -41,7 +41,7 @@
NewsServer::NewsServer(int iID, bool bActive, const char* szName, const char* szHost, int iPort,
const char* szUser, const char* szPass, bool bJoinGroup, bool bTLS,
const char* szCipher, int iMaxConnections, int iRetention, int iLevel, int iGroup)
const char* szCipher, int iMaxConnections, int iLevel, int iGroup)
{
m_iID = iID;
m_iStateID = 0;
@@ -57,8 +57,6 @@ NewsServer::NewsServer(int iID, bool bActive, const char* szName, const char* sz
m_szUser = strdup(szUser ? szUser : "");
m_szPassword = strdup(szPass ? szPass : "");
m_szCipher = strdup(szCipher ? szCipher : "");
m_iRetention = iRetention;
m_tBlockTime = 0;
if (szName && strlen(szName) > 0)
{

View File

@@ -2,7 +2,7 @@
* This file if part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2013 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -28,7 +28,6 @@
#define NEWSSERVER_H
#include <vector>
#include <time.h>
class NewsServer
{
@@ -48,14 +47,11 @@ private:
bool m_bJoinGroup;
bool m_bTLS;
char* m_szCipher;
int m_iRetention;
time_t m_tBlockTime;
public:
NewsServer(int iID, bool bActive, const char* szName, const char* szHost, int iPort,
const char* szUser, const char* szPass, bool bJoinGroup,
bool bTLS, const char* szCipher, int iMaxConnections, int iRetention,
int iLevel, int iGroup);
bool bTLS, const char* szCipher, int iMaxConnections, int iLevel, int iGroup);
~NewsServer();
int GetID() { return m_iID; }
int GetStateID() { return m_iStateID; }
@@ -75,9 +71,6 @@ public:
int GetJoinGroup() { return m_bJoinGroup; }
bool GetTLS() { return m_bTLS; }
const char* GetCipher() { return m_szCipher; }
int GetRetention() { return m_iRetention; }
time_t GetBlockTime() { return m_tBlockTime; }
void SetBlockTime(time_t tBlockTime) { m_tBlockTime = tBlockTime; }
};
typedef std::vector<NewsServer*> Servers;

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -58,7 +58,6 @@ ServerPool::ServerPool()
m_iMaxNormLevel = 0;
m_iTimeout = 60;
m_iGeneration = 0;
m_iRetryInterval = 0;
g_pLog->RegisterDebuggable(this);
}
@@ -158,7 +157,6 @@ void ServerPool::InitConnections()
for (Servers::iterator it = m_SortedServers.begin(); it != m_SortedServers.end(); it++)
{
NewsServer* pNewsServer = *it;
pNewsServer->SetBlockTime(0);
int iNormLevel = pNewsServer->GetNormLevel();
if (pNewsServer->GetNormLevel() > -1)
{
@@ -201,15 +199,11 @@ void ServerPool::InitConnections()
NNTPConnection* ServerPool::GetConnection(int iLevel, NewsServer* pWantServer, Servers* pIgnoreServers)
{
PooledConnection* pConnection = NULL;
m_mutexConnections.Lock();
time_t tCurTime = time(NULL);
m_mutexConnections.Lock();
if (iLevel < (int)m_Levels.size() && m_Levels[iLevel] > 0)
{
Connections candidates;
candidates.reserve(m_Connections.size());
for (Connections::iterator it = m_Connections.begin(); it != m_Connections.end(); it++)
{
PooledConnection* pCandidateConnection = *it;
@@ -217,11 +211,7 @@ NNTPConnection* ServerPool::GetConnection(int iLevel, NewsServer* pWantServer, S
if (!pCandidateConnection->GetInUse() && pCandidateServer->GetActive() &&
pCandidateServer->GetNormLevel() == iLevel &&
(!pWantServer || pCandidateServer == pWantServer ||
(pWantServer->GetGroup() > 0 && pWantServer->GetGroup() == pCandidateServer->GetGroup())) &&
(pCandidateConnection->GetStatus() == Connection::csConnected ||
!pCandidateServer->GetBlockTime() ||
pCandidateServer->GetBlockTime() + m_iRetryInterval <= tCurTime ||
pCandidateServer->GetBlockTime() > tCurTime))
(pWantServer->GetGroup() > 0 && pWantServer->GetGroup() == pCandidateServer->GetGroup())))
{
// free connection found, check if it's not from the server which should be ignored
bool bUseConnection = true;
@@ -240,25 +230,15 @@ NNTPConnection* ServerPool::GetConnection(int iLevel, NewsServer* pWantServer, S
}
}
pCandidateServer->SetBlockTime(0);
if (bUseConnection)
{
candidates.push_back(pCandidateConnection);
pConnection = pCandidateConnection;
pConnection->SetInUse(true);
break;
}
}
}
if (!candidates.empty())
{
// Peeking a random free connection. This is better than taking the first
// available connection because provides better distribution across news servers,
// especially when one of servers becomes unavailable or doesn't have requested articles.
int iRandomIndex = rand() % candidates.size();
pConnection = candidates[iRandomIndex];
pConnection->SetInUse(true);
}
if (pConnection)
{
m_Levels[iLevel]--;
@@ -293,27 +273,12 @@ void ServerPool::FreeConnection(NNTPConnection* pConnection, bool bUsed)
m_mutexConnections.Unlock();
}
void ServerPool::BlockServer(NewsServer* pNewsServer)
{
m_mutexConnections.Lock();
time_t tCurTime = time(NULL);
bool bNewBlock = pNewsServer->GetBlockTime() != tCurTime;
pNewsServer->SetBlockTime(tCurTime);
m_mutexConnections.Unlock();
if (bNewBlock && m_iRetryInterval > 0)
{
warn("Blocking %s (%s) for %i sec", pNewsServer->GetName(), pNewsServer->GetHost(), m_iRetryInterval);
}
}
void ServerPool::CloseUnusedConnections()
{
m_mutexConnections.Lock();
time_t curtime = ::time(NULL);
// close and free all connections of servers which were disabled since the last check
int i = 0;
for (Connections::iterator it = m_Connections.begin(); it != m_Connections.end(); )
{
@@ -335,6 +300,16 @@ void ServerPool::CloseUnusedConnections()
bDeleted = true;
}
if (!bDeleted && !pConnection->GetInUse() && pConnection->GetStatus() == Connection::csConnected)
{
int tdiff = (int)(curtime - pConnection->GetFreeTime());
if (tdiff > CONNECTION_HOLD_SECODNS)
{
debug("Closing (and keeping) unused connection to server%i", pConnection->GetNewsServer()->GetID());
pConnection->Disconnect();
}
}
if (!bDeleted)
{
it++;
@@ -342,50 +317,6 @@ void ServerPool::CloseUnusedConnections()
}
}
// close all opened connections on levels not having any in-use connections
for (int iLevel = 0; iLevel <= m_iMaxNormLevel; iLevel++)
{
// check if we have in-use connections on the level
bool bHasInUseConnections = false;
int iInactiveTime = 0;
for (Connections::iterator it = m_Connections.begin(); it != m_Connections.end(); it++)
{
PooledConnection* pConnection = *it;
if (pConnection->GetNewsServer()->GetNormLevel() == iLevel)
{
if (pConnection->GetInUse())
{
bHasInUseConnections = true;
break;
}
else
{
int tdiff = (int)(curtime - pConnection->GetFreeTime());
if (tdiff > iInactiveTime)
{
iInactiveTime = tdiff;
}
}
}
}
// if there are no in-use connections on the level and the hold time out has
// expired - close all connections of the level.
if (!bHasInUseConnections && iInactiveTime > CONNECTION_HOLD_SECODNS)
{
for (Connections::iterator it = m_Connections.begin(); it != m_Connections.end(); it++)
{
PooledConnection* pConnection = *it;
if (pConnection->GetNewsServer()->GetNormLevel() == iLevel &&
pConnection->GetStatus() == Connection::csConnected)
{
debug("Closing (and keeping) unused connection to server%i", pConnection->GetNewsServer()->GetID());
pConnection->Disconnect();
}
}
}
}
m_mutexConnections.Unlock();
}
@@ -405,16 +336,12 @@ void ServerPool::LogDebugInfo()
m_mutexConnections.Lock();
time_t tCurTime = time(NULL);
info(" Servers: %i", m_Servers.size());
for (Servers::iterator it = m_Servers.begin(); it != m_Servers.end(); it++)
{
NewsServer* pNewsServer = *it;
info(" %i) %s (%s): Level=%i, NormLevel=%i, BlockSec=%i", pNewsServer->GetID(), pNewsServer->GetName(),
pNewsServer->GetHost(), pNewsServer->GetLevel(), pNewsServer->GetNormLevel(),
pNewsServer->GetBlockTime() && pNewsServer->GetBlockTime() + m_iRetryInterval > tCurTime ?
pNewsServer->GetBlockTime() + m_iRetryInterval - tCurTime : 0);
info(" %i) %s (%s): Level=%i, NormLevel=%i", pNewsServer->GetID(), pNewsServer->GetName(),
pNewsServer->GetHost(), pNewsServer->GetLevel(), pNewsServer->GetNormLevel());
}
info(" Levels: %i", m_Levels.size());

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2013 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -61,20 +61,18 @@ private:
int m_iMaxNormLevel;
Mutex m_mutexConnections;
int m_iTimeout;
int m_iRetryInterval;
int m_iGeneration;
void NormalizeLevels();
static bool CompareServers(NewsServer* pServer1, NewsServer* pServer2);
protected:
virtual void LogDebugInfo();
virtual void LogDebugInfo();
public:
ServerPool();
~ServerPool();
void SetTimeout(int iTimeout) { m_iTimeout = iTimeout; }
void SetRetryInterval(int iRetryInterval) { m_iRetryInterval = iRetryInterval; }
void AddServer(NewsServer* pNewsServer);
void InitConnections();
int GetMaxNormLevel() { return m_iMaxNormLevel; }
@@ -84,9 +82,6 @@ public:
void CloseUnusedConnections();
void Changed();
int GetGeneration() { return m_iGeneration; }
void BlockServer(NewsServer* pNewsServer);
};
extern ServerPool* g_pServerPool;
#endif

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2014-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -40,7 +40,10 @@
#include "Options.h"
#include "ServerPool.h"
#include "DiskState.h"
#include "Util.h"
extern ServerPool* g_pServerPool;
extern Options* g_pOptions;
extern DiskState* g_pDiskState;
static const int DAYS_UP_TO_2013_JAN_1 = 15706;
static const int DAYS_IN_TWENTY_YEARS = 366*20;
@@ -168,42 +171,43 @@ void ServerVolume::LogDebugInfo()
{
info(" ---------- ServerVolume");
StringBuilder msg;
char szSec[4000];
szSec[0] = '\0';
for (int i = 0; i < 60; i++)
{
char szNum[30];
snprintf(szNum, 30, "[%i]=%lli ", i, m_BytesPerSeconds[i]);
msg.Append(szNum);
char szNum[20];
snprintf(szNum, 20, "[%i]=%lli ", i, m_BytesPerSeconds[i]);
strncat(szSec, szNum, 4000);
}
info("Secs: %s", msg.GetBuffer());
info("Secs: %s", szSec);
msg.Clear();
szSec[0] = '\0';
for (int i = 0; i < 60; i++)
{
char szNum[30];
snprintf(szNum, 30, "[%i]=%lli ", i, m_BytesPerMinutes[i]);
msg.Append(szNum);
char szNum[20];
snprintf(szNum, 20, "[%i]=%lli ", i, m_BytesPerMinutes[i]);
strncat(szSec, szNum, 4000);
}
info("Mins: %s", msg.GetBuffer());
info("Mins: %s", szSec);
msg.Clear();
szSec[0] = '\0';
for (int i = 0; i < 24; i++)
{
char szNum[30];
snprintf(szNum, 30, "[%i]=%lli ", i, m_BytesPerHours[i]);
msg.Append(szNum);
char szNum[20];
snprintf(szNum, 20, "[%i]=%lli ", i, m_BytesPerHours[i]);
strncat(szSec, szNum, 4000);
}
info("Hours: %s", msg.GetBuffer());
info("Hours: %s", szSec);
msg.Clear();
szSec[0] = '\0';
for (int i = 0; i < (int)m_BytesPerDays.size(); i++)
{
char szNum[30];
snprintf(szNum, 30, "[%i]=%lli ", m_iFirstDay + i, m_BytesPerDays[i]);
msg.Append(szNum);
char szNum[20];
snprintf(szNum, 20, "[%i]=%lli ", m_iFirstDay + i, m_BytesPerDays[i]);
strncat(szSec, szNum, 4000);
}
info("Days: %s", msg.GetBuffer());
info("Days: %s", szSec);
}
StatMeter::StatMeter()
@@ -352,7 +356,9 @@ void StatMeter::CalcTotalStat(int* iUpTimeSec, int* iDnTimeSec, long long* iAllB
m_mutexStat.Unlock();
}
// Average speed in last 30 seconds
/*
* NOTE: see note to "AddSpeedReading"
*/
int StatMeter::CalcCurrentDownloadSpeed()
{
if (m_bStandBy)
@@ -369,14 +375,6 @@ int StatMeter::CalcCurrentDownloadSpeed()
return (int)(m_iSpeedTotalBytes / iTimeDiff);
}
// Amount of data downloaded in current second
int StatMeter::CalcMomentaryDownloadSpeed()
{
time_t tCurTime = time(NULL);
int iSpeed = tCurTime == m_tCurSecTime ? m_iCurSecBytes : 0;
return iSpeed;
}
void StatMeter::AddSpeedReading(int iBytes)
{
time_t tCurTime = time(NULL);
@@ -384,16 +382,13 @@ void StatMeter::AddSpeedReading(int iBytes)
if (g_pOptions->GetAccurateRate())
{
#ifdef HAVE_SPINLOCK
m_spinlockSpeed.Lock();
#else
m_mutexSpeed.Lock();
#endif
}
if (tCurTime != m_tCurSecTime)
{
m_tCurSecTime = tCurTime;
m_iCurSecBytes = 0;
}
m_iCurSecBytes += iBytes;
while (iNowSlot > m_iSpeedTime[m_iSpeedBytesIndex])
{
//record bytes in next slot
@@ -437,7 +432,11 @@ void StatMeter::AddSpeedReading(int iBytes)
if (g_pOptions->GetAccurateRate())
{
#ifdef HAVE_SPINLOCK
m_spinlockSpeed.Unlock();
#else
m_mutexSpeed.Unlock();
#endif
}
}
@@ -453,16 +452,14 @@ void StatMeter::ResetSpeedStat()
m_iSpeedBytesIndex = 0;
m_iSpeedTotalBytes = 0;
m_tSpeedCorrection = tCurTime;
m_tCurSecTime = 0;
m_iCurSecBytes = 0;
}
void StatMeter::LogDebugInfo()
{
info(" ---------- SpeedMeter");
int iSpeed = CalcCurrentDownloadSpeed() / 1024;
float fSpeed = (float)(CalcCurrentDownloadSpeed() / 1024.0);
int iTimeDiff = (int)time(NULL) - m_iSpeedStartTime * SPEEDMETER_SLOTSIZE;
info(" Speed: %i", iSpeed);
info(" Speed: %f", fSpeed);
info(" SpeedStartTime: %i", m_iSpeedStartTime);
info(" SpeedTotalBytes: %i", m_iSpeedTotalBytes);
info(" SpeedBytesIndex: %i", m_iSpeedBytesIndex);

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2014-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -93,9 +93,11 @@ private:
int m_iSpeedStartTime;
time_t m_tSpeedCorrection;
int m_iSpeedBytesIndex;
int m_iCurSecBytes;
time_t m_tCurSecTime;
#ifdef HAVE_SPINLOCK
SpinLock m_spinlockSpeed;
#else
Mutex m_mutexSpeed;
#endif
// time
long long m_iAllBytes;
@@ -123,7 +125,6 @@ public:
~StatMeter();
void Init();
int CalcCurrentDownloadSpeed();
int CalcMomentaryDownloadSpeed();
void AddSpeedReading(int iBytes);
void AddServerData(int iBytes, int iServerID);
void CalcTotalStat(int* iUpTimeSec, int* iDnTimeSec, long long* iAllBytes, bool* bStandBy);
@@ -136,6 +137,4 @@ public:
bool Load(bool* pPerfectServerMatch);
};
extern StatMeter* g_pStatMeter;
#endif

View File

@@ -1,278 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef WIN32
#include "win32.h"
#endif
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <ctype.h>
#ifndef WIN32
#include <unistd.h>
#endif
#include <errno.h>
#include "nzbget.h"
#include "Cleanup.h"
#include "Log.h"
#include "Util.h"
#include "ParParser.h"
#include "Options.h"
void MoveController::StartJob(PostInfo* pPostInfo)
{
MoveController* pMoveController = new MoveController();
pMoveController->m_pPostInfo = pPostInfo;
pMoveController->SetAutoDestroy(false);
pPostInfo->SetPostThread(pMoveController);
pMoveController->Start();
}
void MoveController::Run()
{
// the locking is needed for accessing the members of NZBInfo
DownloadQueue::Lock();
char szNZBName[1024];
strncpy(szNZBName, m_pPostInfo->GetNZBInfo()->GetName(), 1024);
szNZBName[1024-1] = '\0';
char szInfoName[1024];
snprintf(szInfoName, 1024, "move for %s", m_pPostInfo->GetNZBInfo()->GetName());
szInfoName[1024-1] = '\0';
SetInfoName(szInfoName);
strncpy(m_szInterDir, m_pPostInfo->GetNZBInfo()->GetDestDir(), 1024);
m_szInterDir[1024-1] = '\0';
m_pPostInfo->GetNZBInfo()->BuildFinalDirName(m_szDestDir, 1024);
m_szDestDir[1024-1] = '\0';
DownloadQueue::Unlock();
PrintMessage(Message::mkInfo, "Moving completed files for %s", szNZBName);
bool bOK = MoveFiles();
szInfoName[0] = 'M'; // uppercase
if (bOK)
{
PrintMessage(Message::mkInfo, "%s successful", szInfoName);
// save new dest dir
DownloadQueue::Lock();
m_pPostInfo->GetNZBInfo()->SetDestDir(m_szDestDir);
m_pPostInfo->GetNZBInfo()->SetMoveStatus(NZBInfo::msSuccess);
DownloadQueue::Unlock();
}
else
{
PrintMessage(Message::mkError, "%s failed", szInfoName);
m_pPostInfo->GetNZBInfo()->SetMoveStatus(NZBInfo::msFailure);
}
m_pPostInfo->SetStage(PostInfo::ptQueued);
m_pPostInfo->SetWorking(false);
}
bool MoveController::MoveFiles()
{
char szErrBuf[1024];
if (!Util::ForceDirectories(m_szDestDir, szErrBuf, sizeof(szErrBuf)))
{
PrintMessage(Message::mkError, "Could not create directory %s: %s", m_szDestDir, szErrBuf);
return false;
}
bool bOK = true;
DirBrowser dir(m_szInterDir);
while (const char* filename = dir.Next())
{
if (strcmp(filename, ".") && strcmp(filename, ".."))
{
char szSrcFile[1024];
snprintf(szSrcFile, 1024, "%s%c%s", m_szInterDir, PATH_SEPARATOR, filename);
szSrcFile[1024-1] = '\0';
char szDstFile[1024];
Util::MakeUniqueFilename(szDstFile, 1024, m_szDestDir, filename);
bool bHiddenFile = filename[0] == '.';
if (!bHiddenFile)
{
PrintMessage(Message::mkInfo, "Moving file %s to %s", Util::BaseFileName(szSrcFile), m_szDestDir);
}
if (!Util::MoveFile(szSrcFile, szDstFile) && !bHiddenFile)
{
char szErrBuf[256];
PrintMessage(Message::mkError, "Could not move file %s to %s: %s", szSrcFile, szDstFile,
Util::GetLastErrorMessage(szErrBuf, sizeof(szErrBuf)));
bOK = false;
}
}
}
if (bOK && !Util::DeleteDirectoryWithContent(m_szInterDir, szErrBuf, sizeof(szErrBuf)))
{
PrintMessage(Message::mkWarning, "Could not delete intermediate directory %s: %s", m_szInterDir, szErrBuf);
}
return bOK;
}
void MoveController::AddMessage(Message::EKind eKind, const char* szText)
{
m_pPostInfo->GetNZBInfo()->AddMessage(eKind, szText);
}
void CleanupController::StartJob(PostInfo* pPostInfo)
{
CleanupController* pCleanupController = new CleanupController();
pCleanupController->m_pPostInfo = pPostInfo;
pCleanupController->SetAutoDestroy(false);
pPostInfo->SetPostThread(pCleanupController);
pCleanupController->Start();
}
void CleanupController::Run()
{
// the locking is needed for accessing the members of NZBInfo
DownloadQueue::Lock();
char szNZBName[1024];
strncpy(szNZBName, m_pPostInfo->GetNZBInfo()->GetName(), 1024);
szNZBName[1024-1] = '\0';
char szInfoName[1024];
snprintf(szInfoName, 1024, "cleanup for %s", m_pPostInfo->GetNZBInfo()->GetName());
szInfoName[1024-1] = '\0';
SetInfoName(szInfoName);
strncpy(m_szDestDir, m_pPostInfo->GetNZBInfo()->GetDestDir(), 1024);
m_szDestDir[1024-1] = '\0';
bool bInterDir = strlen(g_pOptions->GetInterDir()) > 0 &&
!strncmp(m_szDestDir, g_pOptions->GetInterDir(), strlen(g_pOptions->GetInterDir()));
if (bInterDir)
{
m_pPostInfo->GetNZBInfo()->BuildFinalDirName(m_szFinalDir, 1024);
m_szFinalDir[1024-1] = '\0';
}
else
{
m_szFinalDir[0] = '\0';
}
DownloadQueue::Unlock();
PrintMessage(Message::mkInfo, "Cleaning up %s", szNZBName);
bool bDeleted = false;
bool bOK = Cleanup(m_szDestDir, &bDeleted);
if (bOK && m_szFinalDir[0] != '\0')
{
bool bDeleted2 = false;
bOK = Cleanup(m_szFinalDir, &bDeleted2);
bDeleted = bDeleted || bDeleted2;
}
szInfoName[0] = 'C'; // uppercase
if (bOK && bDeleted)
{
PrintMessage(Message::mkInfo, "%s successful", szInfoName);
m_pPostInfo->GetNZBInfo()->SetCleanupStatus(NZBInfo::csSuccess);
}
else if (bOK)
{
PrintMessage(Message::mkInfo, "Nothing to cleanup for %s", szNZBName);
m_pPostInfo->GetNZBInfo()->SetCleanupStatus(NZBInfo::csSuccess);
}
else
{
PrintMessage(Message::mkError, "%s failed", szInfoName);
m_pPostInfo->GetNZBInfo()->SetCleanupStatus(NZBInfo::csFailure);
}
m_pPostInfo->SetStage(PostInfo::ptQueued);
m_pPostInfo->SetWorking(false);
}
bool CleanupController::Cleanup(const char* szDestDir, bool *bDeleted)
{
*bDeleted = false;
bool bOK = true;
DirBrowser dir(szDestDir);
while (const char* filename = dir.Next())
{
char szFullFilename[1024];
snprintf(szFullFilename, 1024, "%s%c%s", szDestDir, PATH_SEPARATOR, filename);
szFullFilename[1024-1] = '\0';
bool bIsDir = Util::DirectoryExists(szFullFilename);
if (strcmp(filename, ".") && strcmp(filename, "..") && bIsDir)
{
bOK &= Cleanup(szFullFilename, bDeleted);
}
// check file extension
bool bDeleteIt = Util::MatchFileExt(filename, g_pOptions->GetExtCleanupDisk(), ",;") && !bIsDir;
if (bDeleteIt)
{
PrintMessage(Message::mkInfo, "Deleting file %s", filename);
if (remove(szFullFilename) != 0)
{
char szErrBuf[256];
PrintMessage(Message::mkError, "Could not delete file %s: %s", szFullFilename, Util::GetLastErrorMessage(szErrBuf, sizeof(szErrBuf)));
bOK = false;
}
*bDeleted = true;
}
}
return bOK;
}
void CleanupController::AddMessage(Message::EKind eKind, const char* szText)
{
m_pPostInfo->GetNZBInfo()->AddMessage(eKind, szText);
}

View File

@@ -1,68 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifndef CLEANUP_H
#define CLEANUP_H
#include "Log.h"
#include "Thread.h"
#include "DownloadInfo.h"
#include "Script.h"
class MoveController : public Thread, public ScriptController
{
private:
PostInfo* m_pPostInfo;
char m_szInterDir[1024];
char m_szDestDir[1024];
bool MoveFiles();
protected:
virtual void AddMessage(Message::EKind eKind, const char* szText);
public:
virtual void Run();
static void StartJob(PostInfo* pPostInfo);
};
class CleanupController : public Thread, public ScriptController
{
private:
PostInfo* m_pPostInfo;
char m_szDestDir[1024];
char m_szFinalDir[1024];
bool Cleanup(const char* szDestDir, bool *bDeleted);
protected:
virtual void AddMessage(Message::EKind eKind, const char* szText);
public:
virtual void Run();
static void StartJob(PostInfo* pPostInfo);
};
#endif

View File

@@ -1,260 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef WIN32
#include "win32.h"
#endif
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <ctype.h>
#ifndef WIN32
#include <unistd.h>
#endif
#include <errno.h>
#include "nzbget.h"
#include "DupeMatcher.h"
#include "Log.h"
#include "Util.h"
#include "Options.h"
#include "Script.h"
#include "Thread.h"
class RarLister : public Thread, public ScriptController
{
private:
DupeMatcher* m_pOwner;
long long m_lMaxSize;
bool m_bCompressed;
bool m_bLastSizeMax;
long long m_lExpectedSize;
char* m_szFilenameBuf;
int m_iFilenameBufLen;
char m_szLastFilename[1024];
protected:
virtual void AddMessage(Message::EKind eKind, const char* szText);
public:
virtual void Run();
static bool FindLargestFile(DupeMatcher* pOwner, const char* szDirectory,
char* szFilenameBuf, int iFilenameBufLen, long long lExpectedSize,
int iTimeoutSec, long long* pMaxSize, bool* pCompressed);
};
bool RarLister::FindLargestFile(DupeMatcher* pOwner, const char* szDirectory,
char* szFilenameBuf, int iFilenameBufLen, long long lExpectedSize,
int iTimeoutSec, long long* pMaxSize, bool* pCompressed)
{
RarLister unrar;
unrar.m_pOwner = pOwner;
unrar.m_lExpectedSize = lExpectedSize;
unrar.m_lMaxSize = -1;
unrar.m_bCompressed = false;
unrar.m_bLastSizeMax = false;
unrar.m_szFilenameBuf = szFilenameBuf;
unrar.m_iFilenameBufLen = iFilenameBufLen;
char** pCmdArgs = NULL;
if (!Util::SplitCommandLine(g_pOptions->GetUnrarCmd(), &pCmdArgs))
{
return false;
}
const char* szUnrarPath = *pCmdArgs;
unrar.SetScript(szUnrarPath);
const char* szArgs[4];
szArgs[0] = szUnrarPath;
szArgs[1] = "lt";
szArgs[2] = "*.rar";
szArgs[3] = NULL;
unrar.SetArgs(szArgs, false);
unrar.SetWorkingDir(szDirectory);
time_t curTime = time(NULL);
unrar.Start();
// wait up to iTimeoutSec for unrar output
while (unrar.IsRunning() &&
curTime + iTimeoutSec > time(NULL) &&
curTime >= time(NULL)) // in a case clock was changed
{
usleep(200 * 1000);
}
if (unrar.IsRunning())
{
unrar.Terminate();
}
// wait until terminated or killed
while (unrar.IsRunning())
{
usleep(200 * 1000);
}
for (char** szArgPtr = pCmdArgs; *szArgPtr; szArgPtr++)
{
free(*szArgPtr);
}
free(pCmdArgs);
*pMaxSize = unrar.m_lMaxSize;
*pCompressed = unrar.m_bCompressed;
return true;
}
void RarLister::Run()
{
Execute();
}
void RarLister::AddMessage(Message::EKind eKind, const char* szText)
{
if (!strncasecmp(szText, "Archive: ", 9))
{
m_pOwner->PrintMessage(Message::mkDetail, "Reading file %s", szText + 9);
}
else if (!strncasecmp(szText, " Name: ", 14))
{
strncpy(m_szLastFilename, szText + 14, sizeof(m_szLastFilename));
m_szLastFilename[sizeof(m_szLastFilename)-1] = '\0';
}
else if (!strncasecmp(szText, " Size: ", 14))
{
m_bLastSizeMax = false;
long long lSize = atoll(szText + 14);
if (lSize > m_lMaxSize)
{
m_lMaxSize = lSize;
m_bLastSizeMax = true;
strncpy(m_szFilenameBuf, m_szLastFilename, m_iFilenameBufLen);
m_szFilenameBuf[m_iFilenameBufLen-1] = '\0';
}
return;
}
if (m_bLastSizeMax && !strncasecmp(szText, " Compression: ", 14))
{
m_bCompressed = !strstr(szText, " -m0");
if (m_lMaxSize > m_lExpectedSize ||
DupeMatcher::SizeDiffOK(m_lMaxSize, m_lExpectedSize, 20))
{
// alread found the largest file, aborting unrar
Terminate();
}
}
}
DupeMatcher::DupeMatcher(const char* szDestDir, long long lExpectedSize)
{
m_szDestDir = strdup(szDestDir);
m_lExpectedSize = lExpectedSize;
m_lMaxSize = -1;
m_bCompressed = false;
}
DupeMatcher::~DupeMatcher()
{
free(m_szDestDir);
}
bool DupeMatcher::SizeDiffOK(long long lSize1, long long lSize2, int iMaxDiffPercent)
{
if (lSize1 == 0 || lSize2 == 0)
{
return false;
}
long long lDiff = lSize1 - lSize2;
lDiff = lDiff > 0 ? lDiff : -lDiff;
long long lMax = lSize1 > lSize2 ? lSize1 : lSize2;
int lDiffPercent = (int)(lDiff * 100 / lMax);
return lDiffPercent < iMaxDiffPercent;
}
bool DupeMatcher::Prepare()
{
char szFilename[1024];
FindLargestFile(m_szDestDir, szFilename, sizeof(szFilename), &m_lMaxSize, &m_bCompressed);
bool bSizeOK = SizeDiffOK(m_lMaxSize, m_lExpectedSize, 20);
PrintMessage(Message::mkDetail, "Found main file %s with size %lli bytes%s",
szFilename, m_lMaxSize, bSizeOK ? "" : ", size mismatch");
return bSizeOK;
}
bool DupeMatcher::MatchDupeContent(const char* szDupeDir)
{
long long lDupeMaxSize = 0;
bool lDupeCompressed = false;
char szFilename[1024];
FindLargestFile(szDupeDir, szFilename, sizeof(szFilename), &lDupeMaxSize, &lDupeCompressed);
bool bOK = lDupeMaxSize == m_lMaxSize && lDupeCompressed == m_bCompressed;
PrintMessage(Message::mkDetail, "Found main file %s with size %lli bytes%s",
szFilename, m_lMaxSize, bOK ? "" : ", size mismatch");
return bOK;
}
void DupeMatcher::FindLargestFile(const char* szDirectory, char* szFilenameBuf, int iBufLen,
long long* pMaxSize, bool* pCompressed)
{
*pMaxSize = 0;
*pCompressed = false;
DirBrowser dir(szDirectory);
while (const char* filename = dir.Next())
{
if (strcmp(filename, ".") && strcmp(filename, ".."))
{
char szFullFilename[1024];
snprintf(szFullFilename, 1024, "%s%c%s", szDirectory, PATH_SEPARATOR, filename);
szFullFilename[1024-1] = '\0';
long long lFileSize = Util::FileSize(szFullFilename);
if (lFileSize > *pMaxSize)
{
*pMaxSize = lFileSize;
strncpy(szFilenameBuf, filename, iBufLen);
szFilenameBuf[iBufLen-1] = '\0';
}
if (Util::MatchFileExt(filename, ".rar", ","))
{
RarLister::FindLargestFile(this, szDirectory, szFilenameBuf, iBufLen,
m_lMaxSize, 60, pMaxSize, pCompressed);
return;
}
}
}
}

View File

@@ -1,55 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifndef DUPEMATCHER_H
#define DUPEMATCHER_H
#include "Log.h"
class DupeMatcher
{
private:
char* m_szDestDir;
long long m_lExpectedSize;
long long m_lMaxSize;
bool m_bCompressed;
void FindLargestFile(const char* szDirectory, char* szFilenameBuf, int iBufLen,
long long* pMaxSize, bool* pCompressed);
friend class RarLister;
protected:
virtual void PrintMessage(Message::EKind eKind, const char* szFormat, ...) {}
public:
DupeMatcher(const char* szDestDir, long long lExpectedSize);
~DupeMatcher();
bool Prepare();
bool MatchDupeContent(const char* szDupeDir);
static bool SizeDiffOK(long long lSize1, long long lSize2, int iMaxDiffPercent);
};
#endif

View File

File diff suppressed because it is too large Load Diff

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2013 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -29,7 +29,6 @@
#ifndef DISABLE_PARCHECK
#include <deque>
#include <vector>
#include <string>
#include "Thread.h"
@@ -54,61 +53,8 @@ public:
ptVerifyingRepaired,
};
enum EFileStatus
{
fsUnknown,
fsSuccess,
fsPartial,
fsFailure
};
class Segment
{
private:
bool m_bSuccess;
long long m_iOffset;
int m_iSize;
unsigned long m_lCrc;
public:
Segment(bool bSuccess, long long iOffset, int iSize, unsigned long lCrc);
bool GetSuccess() { return m_bSuccess; }
long long GetOffset() { return m_iOffset; }
int GetSize() { return m_iSize; }
unsigned long GetCrc() { return m_lCrc; }
};
typedef std::deque<Segment*> SegmentListBase;
class SegmentList : public SegmentListBase
{
public:
~SegmentList();
};
class DupeSource
{
private:
int m_iID;
char* m_szDirectory;
int m_iUsedBlocks;
public:
DupeSource(int iID, const char* szDirectory);
~DupeSource();
int GetID() { return m_iID; }
const char* GetDirectory() { return m_szDirectory; }
int GetUsedBlocks() { return m_iUsedBlocks; }
void SetUsedBlocks(int iUsedBlocks) { m_iUsedBlocks = iUsedBlocks; }
};
typedef std::deque<DupeSource*> DupeSourceList;
typedef std::deque<char*> FileList;
typedef std::deque<void*> SourceList;
typedef std::vector<bool> ValidBlocks;
friend class Repairer;
private:
char* m_szInfoName;
@@ -117,8 +63,7 @@ private:
const char* m_szParFilename;
EStatus m_eStatus;
EStage m_eStage;
// declared as void* to prevent the including of libpar2-headers into this header-file
void* m_pRepairer;
void* m_pRepairer; // declared as void* to prevent the including of libpar2-headers into this header-file
char* m_szErrMsg;
FileList m_QueuedParFiles;
Mutex m_mutexQueuedParFiles;
@@ -127,22 +72,14 @@ private:
int m_iProcessedFiles;
int m_iFilesToRepair;
int m_iExtraFiles;
int m_iQuickFiles;
bool m_bVerifyingExtraFiles;
char* m_szProgressLabel;
int m_iFileProgress;
int m_iStageProgress;
bool m_bCancelled;
SourceList m_sourceFiles;
std::string m_lastFilename;
bool m_bHasDamagedFiles;
bool m_bParQuick;
bool m_bForceRepair;
bool m_bParFull;
DupeSourceList m_DupeSources;
void Cleanup();
EStatus RunParCheckAll();
EStatus RunParCheck(const char* szParFilename);
int PreProcessPar();
bool LoadMainParBak();
@@ -150,24 +87,13 @@ private:
bool LoadMorePars();
bool AddSplittedFragments();
bool AddMissingFiles();
bool AddDupeFiles();
bool AddExtraFiles(bool bOnlyMissing, bool bExternalDir, const char* szDirectory);
bool IsProcessedFile(const char* szFilename);
void WriteBrokenLog(EStatus eStatus);
void SaveSourceList();
void DeleteLeftovers();
void signal_filename(std::string str);
void signal_progress(int progress);
void signal_progress(double progress);
void signal_done(std::string str, int available, int total);
// declared as void* to prevent the including of libpar2-headers into this header-file
// DiskFile* pDiskfile, Par2RepairerSourceFile* pSourcefile
EFileStatus VerifyDataFile(void* pDiskfile, void* pSourcefile, int* pAvailableBlocks);
bool VerifySuccessDataFile(void* pDiskfile, void* pSourcefile, unsigned long lDownloadCrc);
bool VerifyPartialDataFile(void* pDiskfile, void* pSourcefile, SegmentList* pSegments, ValidBlocks* pValidBlocks);
bool SmartCalcFileRangeCrc(FILE* pFile, long long lStart, long long lEnd, SegmentList* pSegments,
unsigned long* pDownloadCrc);
bool DumbCalcFileRangeCrc(FILE* pFile, long long lStart, long long lEnd, unsigned long* pDownloadCrc);
void CheckEmptyFiles();
protected:
/**
@@ -181,9 +107,6 @@ protected:
virtual void PrintMessage(Message::EKind eKind, const char* szFormat, ...) {}
virtual void RegisterParredFile(const char* szFilename) {}
virtual bool IsParredFile(const char* szFilename) { return false; }
virtual EFileStatus FindFileCrc(const char* szFilename, unsigned long* lCrc, SegmentList* pSegments) { return fsUnknown; }
virtual void RequestDupeSources(DupeSourceList* pDupeSourceList) {}
virtual void StatDupeSources(DupeSourceList* pDupeSourceList) {}
EStage GetStage() { return m_eStage; }
const char* GetProgressLabel() { return m_szProgressLabel; }
int GetFileProgress() { return m_iFileProgress; }
@@ -198,12 +121,6 @@ public:
const char* GetInfoName() { return m_szInfoName; }
void SetInfoName(const char* szInfoName);
void SetNZBName(const char* szNZBName);
void SetParQuick(bool bParQuick) { m_bParQuick = bParQuick; }
bool GetParQuick() { return m_bParQuick; }
void SetForceRepair(bool bForceRepair) { m_bForceRepair = bForceRepair; }
bool GetForceRepair() { return m_bForceRepair; }
void SetParFull(bool bParFull) { m_bParFull = bParFull; }
bool GetParFull() { return m_bParFull; }
EStatus GetStatus() { return m_eStatus; }
void AddParFile(const char* szParFilename);
void QueueChanged();

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -44,13 +44,12 @@
#include "nzbget.h"
#include "ParCoordinator.h"
#include "DupeCoordinator.h"
#include "ParParser.h"
#include "Options.h"
#include "DiskState.h"
#include "Log.h"
#include "Util.h"
extern Options* g_pOptions;
#ifndef DISABLE_PARCHECK
bool ParCoordinator::PostParChecker::RequestMorePars(int iBlockNeeded, int* pBlockFound)
{
@@ -71,7 +70,7 @@ void ParCoordinator::PostParChecker::PrintMessage(Message::EKind eKind, const ch
va_end(args);
szText[1024-1] = '\0';
m_pPostInfo->GetNZBInfo()->AddMessage(eKind, szText);
m_pOwner->PrintMessage(m_pPostInfo, eKind, "%s", szText);
}
void ParCoordinator::PostParChecker::RegisterParredFile(const char* szFilename)
@@ -92,122 +91,6 @@ bool ParCoordinator::PostParChecker::IsParredFile(const char* szFilename)
return false;
}
ParChecker::EFileStatus ParCoordinator::PostParChecker::FindFileCrc(const char* szFilename,
unsigned long* lCrc, SegmentList* pSegments)
{
CompletedFile* pCompletedFile = NULL;
for (CompletedFiles::iterator it = m_pPostInfo->GetNZBInfo()->GetCompletedFiles()->begin(); it != m_pPostInfo->GetNZBInfo()->GetCompletedFiles()->end(); it++)
{
CompletedFile* pCompletedFile2 = *it;
if (!strcasecmp(pCompletedFile2->GetFileName(), szFilename))
{
pCompletedFile = pCompletedFile2;
break;
}
}
if (!pCompletedFile)
{
return ParChecker::fsUnknown;
}
debug("Found completed file: %s, CRC: %.8x, Status: %i", Util::BaseFileName(pCompletedFile->GetFileName()), pCompletedFile->GetCrc(), (int)pCompletedFile->GetStatus());
*lCrc = pCompletedFile->GetCrc();
if (pCompletedFile->GetStatus() == CompletedFile::cfPartial && pCompletedFile->GetID() > 0 &&
!m_pPostInfo->GetNZBInfo()->GetReprocess())
{
FileInfo* pTmpFileInfo = new FileInfo(pCompletedFile->GetID());
if (!g_pDiskState->LoadFileState(pTmpFileInfo, NULL, true))
{
delete pTmpFileInfo;
return ParChecker::fsUnknown;
}
for (FileInfo::Articles::iterator it = pTmpFileInfo->GetArticles()->begin(); it != pTmpFileInfo->GetArticles()->end(); it++)
{
ArticleInfo* pa = *it;
ParChecker::Segment* pSegment = new Segment(pa->GetStatus() == ArticleInfo::aiFinished,
pa->GetSegmentOffset(), pa->GetSegmentSize(), pa->GetCrc());
pSegments->push_back(pSegment);
}
delete pTmpFileInfo;
}
return pCompletedFile->GetStatus() == CompletedFile::cfSuccess ? ParChecker::fsSuccess :
pCompletedFile->GetStatus() == CompletedFile::cfFailure &&
!m_pPostInfo->GetNZBInfo()->GetReprocess() ? ParChecker::fsFailure :
pCompletedFile->GetStatus() == CompletedFile::cfPartial && pSegments->size() > 0 &&
!m_pPostInfo->GetNZBInfo()->GetReprocess()? ParChecker::fsPartial :
ParChecker::fsUnknown;
}
void ParCoordinator::PostParChecker::RequestDupeSources(DupeSourceList* pDupeSourceList)
{
DownloadQueue* pDownloadQueue = DownloadQueue::Lock();
NZBList dupeList;
g_pDupeCoordinator->ListHistoryDupes(pDownloadQueue, m_pPostInfo->GetNZBInfo(), &dupeList);
if (!dupeList.empty())
{
PostDupeMatcher dupeMatcher(m_pPostInfo);
PrintMessage(Message::mkInfo, "Checking %s for dupe scan usability", m_pPostInfo->GetNZBInfo()->GetName());
bool bSizeComparisonPossible = dupeMatcher.Prepare();
for (NZBList::iterator it = dupeList.begin(); it != dupeList.end(); it++)
{
NZBInfo* pDupeNZBInfo = *it;
if (bSizeComparisonPossible)
{
PrintMessage(Message::mkInfo, "Checking %s for dupe scan usability", Util::BaseFileName(pDupeNZBInfo->GetDestDir()));
}
bool bUseDupe = !bSizeComparisonPossible || dupeMatcher.MatchDupeContent(pDupeNZBInfo->GetDestDir());
if (bUseDupe)
{
PrintMessage(Message::mkInfo, "Adding %s to dupe scan sources", Util::BaseFileName(pDupeNZBInfo->GetDestDir()));
pDupeSourceList->push_back(new ParChecker::DupeSource(pDupeNZBInfo->GetID(), pDupeNZBInfo->GetDestDir()));
}
}
if (pDupeSourceList->empty())
{
PrintMessage(Message::mkInfo, "No usable dupe scan sources found");
}
}
DownloadQueue::Unlock();
}
void ParCoordinator::PostParChecker::StatDupeSources(DupeSourceList* pDupeSourceList)
{
DownloadQueue* pDownloadQueue = DownloadQueue::Lock();
int iTotalExtraParBlocks = 0;
for (DupeSourceList::iterator it = pDupeSourceList->begin(); it != pDupeSourceList->end(); it++)
{
DupeSource* pDupeSource = *it;
if (pDupeSource->GetUsedBlocks() > 0)
{
for (HistoryList::iterator it = pDownloadQueue->GetHistory()->begin(); it != pDownloadQueue->GetHistory()->end(); it++)
{
HistoryInfo* pHistoryInfo = *it;
if (pHistoryInfo->GetKind() == HistoryInfo::hkNzb &&
pHistoryInfo->GetNZBInfo()->GetID() == pDupeSource->GetID())
{
pHistoryInfo->GetNZBInfo()->SetExtraParBlocks(pHistoryInfo->GetNZBInfo()->GetExtraParBlocks() - pDupeSource->GetUsedBlocks());
}
}
}
iTotalExtraParBlocks += pDupeSource->GetUsedBlocks();
}
m_pPostInfo->GetNZBInfo()->SetExtraParBlocks(m_pPostInfo->GetNZBInfo()->GetExtraParBlocks() + iTotalExtraParBlocks);
DownloadQueue::Unlock();
}
void ParCoordinator::PostParRenamer::UpdateProgress()
{
m_pOwner->UpdateParRenameProgress();
@@ -222,7 +105,7 @@ void ParCoordinator::PostParRenamer::PrintMessage(Message::EKind eKind, const ch
va_end(args);
szText[1024-1] = '\0';
m_pPostInfo->GetNZBInfo()->AddMessage(eKind, szText);
m_pOwner->PrintMessage(m_pPostInfo, eKind, "%s", szText);
}
void ParCoordinator::PostParRenamer::RegisterParredFile(const char* szFilename)
@@ -230,34 +113,6 @@ void ParCoordinator::PostParRenamer::RegisterParredFile(const char* szFilename)
m_pPostInfo->GetParredFiles()->push_back(strdup(szFilename));
}
/**
* Update file name in the CompletedFiles-list of NZBInfo
*/
void ParCoordinator::PostParRenamer::RegisterRenamedFile(const char* szOldFilename, const char* szNewFileName)
{
for (CompletedFiles::iterator it = m_pPostInfo->GetNZBInfo()->GetCompletedFiles()->begin(); it != m_pPostInfo->GetNZBInfo()->GetCompletedFiles()->end(); it++)
{
CompletedFile* pCompletedFile = *it;
if (!strcasecmp(pCompletedFile->GetFileName(), szOldFilename))
{
pCompletedFile->SetFileName(szNewFileName);
break;
}
}
}
void ParCoordinator::PostDupeMatcher::PrintMessage(Message::EKind eKind, const char* szFormat, ...)
{
char szText[1024];
va_list args;
va_start(args, szFormat);
vsnprintf(szText, 1024, szFormat, args);
va_end(args);
szText[1024-1] = '\0';
m_pPostInfo->GetNZBInfo()->AddMessage(eKind, szText);
}
#endif
ParCoordinator::ParCoordinator()
@@ -309,6 +164,111 @@ void ParCoordinator::PausePars(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo)
DownloadQueue::eaGroupPauseExtraPars, 0, NULL);
}
bool ParCoordinator::FindMainPars(const char* szPath, ParFileList* pFileList)
{
if (pFileList)
{
pFileList->clear();
}
DirBrowser dir(szPath);
while (const char* filename = dir.Next())
{
int iBaseLen = 0;
if (ParseParFilename(filename, &iBaseLen, NULL))
{
if (!pFileList)
{
return true;
}
// check if the base file already added to list
bool exists = false;
for (ParFileList::iterator it = pFileList->begin(); it != pFileList->end(); it++)
{
const char* filename2 = *it;
exists = SameParCollection(filename, filename2);
if (exists)
{
break;
}
}
if (!exists)
{
pFileList->push_back(strdup(filename));
}
}
}
return pFileList && !pFileList->empty();
}
bool ParCoordinator::SameParCollection(const char* szFilename1, const char* szFilename2)
{
int iBaseLen1 = 0, iBaseLen2 = 0;
return ParseParFilename(szFilename1, &iBaseLen1, NULL) &&
ParseParFilename(szFilename2, &iBaseLen2, NULL) &&
iBaseLen1 == iBaseLen2 &&
!strncasecmp(szFilename1, szFilename2, iBaseLen1);
}
bool ParCoordinator::ParseParFilename(const char* szParFilename, int* iBaseNameLen, int* iBlocks)
{
char szFilename[1024];
strncpy(szFilename, szParFilename, 1024);
szFilename[1024-1] = '\0';
for (char* p = szFilename; *p; p++) *p = tolower(*p); // convert string to lowercase
int iLen = strlen(szFilename);
if (iLen < 6)
{
return false;
}
// find last occurence of ".par2" and trim filename after it
char* szEnd = szFilename;
while (char* p = strstr(szEnd, ".par2")) szEnd = p + 5;
*szEnd = '\0';
iLen = strlen(szFilename);
if (iLen < 6)
{
return false;
}
if (strcasecmp(szFilename + iLen - 5, ".par2"))
{
return false;
}
*(szFilename + iLen - 5) = '\0';
int blockcnt = 0;
char* p = strrchr(szFilename, '.');
if (p && !strncasecmp(p, ".vol", 4))
{
char* b = strchr(p, '+');
if (!b)
{
b = strchr(p, '-');
}
if (b)
{
blockcnt = atoi(b+1);
*p = '\0';
}
}
if (iBaseNameLen)
{
*iBaseNameLen = strlen(szFilename);
}
if (iBlocks)
{
*iBlocks = blockcnt;
}
return true;
}
#ifndef DISABLE_PARCHECK
/**
@@ -320,10 +280,6 @@ void ParCoordinator::StartParCheckJob(PostInfo* pPostInfo)
m_ParChecker.SetPostInfo(pPostInfo);
m_ParChecker.SetDestDir(pPostInfo->GetNZBInfo()->GetDestDir());
m_ParChecker.SetNZBName(pPostInfo->GetNZBInfo()->GetName());
m_ParChecker.SetParTime(time(NULL));
m_ParChecker.SetDownloadSec(pPostInfo->GetNZBInfo()->GetDownloadSec());
m_ParChecker.SetParQuick(g_pOptions->GetParQuick() && !pPostInfo->GetForceParFull());
m_ParChecker.SetForceRepair(pPostInfo->GetForceRepair());
m_ParChecker.PrintMessage(Message::mkInfo, "Checking pars for %s", pPostInfo->GetNZBInfo()->GetName());
pPostInfo->SetWorking(true);
m_ParChecker.Start();
@@ -358,12 +314,16 @@ bool ParCoordinator::Cancel()
{
if (m_eCurrentJob == jkParCheck)
{
#ifdef HAVE_PAR2_CANCEL
if (!m_ParChecker.GetCancelled())
{
debug("Cancelling par-repair for %s", m_ParChecker.GetInfoName());
m_ParChecker.Cancel();
return true;
}
#else
warn("Cannot cancel par-repair for %s, used version of libpar2 does not support cancelling", m_ParChecker.GetInfoName());
#endif
}
else if (m_eCurrentJob == jkParRename)
{
@@ -410,7 +370,6 @@ void ParCoordinator::ParCheckCompleted()
pPostInfo->GetNZBInfo()->GetParStatus() <= NZBInfo::psSkipped)
{
pPostInfo->GetNZBInfo()->SetParStatus(NZBInfo::psSuccess);
pPostInfo->SetParRepaired(m_ParChecker.GetStatus() == ParChecker::psRepaired);
}
else if (m_ParChecker.GetStatus() == ParChecker::psRepairPossible &&
pPostInfo->GetNZBInfo()->GetParStatus() != NZBInfo::psFailure)
@@ -422,13 +381,6 @@ void ParCoordinator::ParCheckCompleted()
pPostInfo->GetNZBInfo()->SetParStatus(NZBInfo::psFailure);
}
int iWaitTime = pPostInfo->GetNZBInfo()->GetDownloadSec() - m_ParChecker.GetDownloadSec();
pPostInfo->SetStartTime(pPostInfo->GetStartTime() + (time_t)iWaitTime);
int iParSec = (int)(time(NULL) - m_ParChecker.GetParTime()) - iWaitTime;
pPostInfo->GetNZBInfo()->SetParSec(pPostInfo->GetNZBInfo()->GetParSec() + iParSec);
pPostInfo->GetNZBInfo()->SetParFull(m_ParChecker.GetParFull());
pPostInfo->SetWorking(false);
pPostInfo->SetStage(PostInfo::ptQueued);
@@ -440,9 +392,7 @@ void ParCoordinator::ParCheckCompleted()
/**
* Unpause par2-files
* returns true, if the files with required number of blocks were unpaused,
* or false if there are no more files in queue for this collection or not enough blocks.
* special case: returns true if there are any unpaused par2-files in the queue regardless
* of the amount of blocks; this is to keep par-checker wait for download completion.
* or false if there are no more files in queue for this collection or not enough blocks
*/
bool ParCoordinator::RequestMorePars(NZBInfo* pNZBInfo, const char* szParFilename, int iBlockNeeded, int* pBlockFound)
{
@@ -520,17 +470,6 @@ bool ParCoordinator::RequestMorePars(NZBInfo* pNZBInfo, const char* szParFilenam
}
}
bool bHasUnpausedParFiles = false;
for (FileList::iterator it = pNZBInfo->GetFileList()->begin(); it != pNZBInfo->GetFileList()->end(); it++)
{
FileInfo* pFileInfo = *it;
if (pFileInfo->GetParFile() && !pFileInfo->GetPaused())
{
bHasUnpausedParFiles = true;
break;
}
}
DownloadQueue::Unlock();
if (pBlockFound)
@@ -544,7 +483,7 @@ bool ParCoordinator::RequestMorePars(NZBInfo* pNZBInfo, const char* szParFilenam
}
blocks.clear();
bool bOK = iBlockNeeded <= 0 || bHasUnpausedParFiles;
bool bOK = iBlockNeeded <= 0;
return bOK;
}
@@ -558,10 +497,10 @@ void ParCoordinator::FindPars(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo,
char* szBaseParFilename = Util::BaseFileName(szParFilename);
char szMainBaseFilename[1024];
int iMainBaseLen = 0;
if (!ParParser::ParseParFilename(szBaseParFilename, &iMainBaseLen, NULL))
if (!ParseParFilename(szBaseParFilename, &iMainBaseLen, NULL))
{
// should not happen
pNZBInfo->PrintMessage(Message::mkError, "Internal error: could not parse filename %s", szBaseParFilename);
error("Internal error: could not parse filename %s", szBaseParFilename);
return;
}
int maxlen = iMainBaseLen < 1024 ? iMainBaseLen : 1024 - 1;
@@ -573,14 +512,14 @@ void ParCoordinator::FindPars(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo,
{
FileInfo* pFileInfo = *it;
int iBlocks = 0;
if (ParParser::ParseParFilename(pFileInfo->GetFilename(), NULL, &iBlocks) &&
if (ParseParFilename(pFileInfo->GetFilename(), NULL, &iBlocks) &&
iBlocks > 0)
{
bool bUseFile = true;
if (bExactParName)
{
bUseFile = ParParser::SameParCollection(pFileInfo->GetFilename(), Util::BaseFileName(szParFilename));
bUseFile = SameParCollection(pFileInfo->GetFilename(), Util::BaseFileName(szParFilename));
}
else if (bStrictParName)
{
@@ -648,22 +587,19 @@ void ParCoordinator::UpdateParCheckProgress()
PostInfo::EStage eStage = StageKind[m_ParChecker.GetStage()];
time_t tCurrent = time(NULL);
if (!pPostInfo->GetStartTime())
{
pPostInfo->SetStartTime(tCurrent);
}
if (pPostInfo->GetStage() != eStage)
{
pPostInfo->SetStage(eStage);
pPostInfo->SetStageTime(tCurrent);
if (pPostInfo->GetStage() == PostInfo::ptRepairing)
{
m_ParChecker.SetRepairTime(tCurrent);
}
else if (pPostInfo->GetStage() == PostInfo::ptVerifyingRepaired)
{
int iRepairSec = (int)(tCurrent - m_ParChecker.GetRepairTime());
pPostInfo->GetNZBInfo()->SetRepairSec(pPostInfo->GetNZBInfo()->GetRepairSec() + iRepairSec);
}
}
bool bParCancel = false;
#ifdef HAVE_PAR2_CANCEL
if (!m_ParChecker.GetCancelled())
{
if ((g_pOptions->GetParTimeLimit() > 0) &&
@@ -682,6 +618,7 @@ void ParCoordinator::UpdateParCheckProgress()
}
}
}
#endif
if (bParCancel)
{
@@ -699,14 +636,12 @@ void ParCoordinator::CheckPauseState(PostInfo* pPostInfo)
{
time_t tStageTime = pPostInfo->GetStageTime();
time_t tStartTime = pPostInfo->GetStartTime();
time_t tParTime = m_ParChecker.GetParTime();
time_t tRepairTime = m_ParChecker.GetRepairTime();
time_t tWaitTime = time(NULL);
// wait until Post-processor is unpaused
while (g_pOptions->GetPausePostProcess() && !pPostInfo->GetNZBInfo()->GetForcePriority() && !m_bStopped)
{
usleep(50 * 1000);
usleep(100 * 1000);
// update time stamps
@@ -716,18 +651,11 @@ void ParCoordinator::CheckPauseState(PostInfo* pPostInfo)
{
pPostInfo->SetStageTime(tStageTime + tDelta);
}
if (tStartTime > 0)
{
pPostInfo->SetStartTime(tStartTime + tDelta);
}
if (tParTime > 0)
{
m_ParChecker.SetParTime(tParTime + tDelta);
}
if (tRepairTime > 0)
{
m_ParChecker.SetRepairTime(tRepairTime + tDelta);
}
}
}
}
@@ -741,7 +669,12 @@ void ParCoordinator::ParRenameCompleted()
if (m_ParRenamer.HasMissedFiles() && pPostInfo->GetNZBInfo()->GetParStatus() <= NZBInfo::psSkipped)
{
m_ParRenamer.PrintMessage(Message::mkInfo, "Requesting par-check/repair for %s to restore missing files ", m_ParRenamer.GetInfoName());
PrintMessage(pPostInfo, Message::mkInfo, "Requesting par-check/repair for %s to restore missing files ", m_ParRenamer.GetInfoName());
pPostInfo->SetRequestParCheck(true);
}
else if (m_ParRenamer.HasSplittedFragments() && pPostInfo->GetNZBInfo()->GetParStatus() <= NZBInfo::psSkipped)
{
PrintMessage(pPostInfo, Message::mkInfo, "Requesting par-check/repair for %s to join splitted fragments", m_ParRenamer.GetInfoName());
pPostInfo->SetRequestParCheck(true);
}
@@ -762,6 +695,11 @@ void ParCoordinator::UpdateParRenameProgress()
pPostInfo->SetStageProgress(m_ParRenamer.GetStageProgress());
time_t tCurrent = time(NULL);
if (!pPostInfo->GetStartTime())
{
pPostInfo->SetStartTime(tCurrent);
}
if (pPostInfo->GetStage() != PostInfo::ptRenaming)
{
pPostInfo->SetStage(PostInfo::ptRenaming);
@@ -773,4 +711,39 @@ void ParCoordinator::UpdateParRenameProgress()
CheckPauseState(pPostInfo);
}
void ParCoordinator::PrintMessage(PostInfo* pPostInfo, Message::EKind eKind, const char* szFormat, ...)
{
char szText[1024];
va_list args;
va_start(args, szFormat);
vsnprintf(szText, 1024, szFormat, args);
va_end(args);
szText[1024-1] = '\0';
pPostInfo->AppendMessage(eKind, szText);
switch (eKind)
{
case Message::mkDetail:
detail("%s", szText);
break;
case Message::mkInfo:
info("%s", szText);
break;
case Message::mkWarning:
warn("%s", szText);
break;
case Message::mkError:
error("%s", szText);
break;
case Message::mkDebug:
debug("%s", szText);
break;
}
}
#endif

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -34,7 +34,6 @@
#ifndef DISABLE_PARCHECK
#include "ParChecker.h"
#include "ParRenamer.h"
#include "DupeMatcher.h"
#endif
class ParCoordinator
@@ -46,9 +45,6 @@ private:
private:
ParCoordinator* m_pOwner;
PostInfo* m_pPostInfo;
time_t m_tParTime;
time_t m_tRepairTime;
int m_iDownloadSec;
protected:
virtual bool RequestMorePars(int iBlockNeeded, int* pBlockFound);
virtual void UpdateProgress();
@@ -56,18 +52,9 @@ private:
virtual void PrintMessage(Message::EKind eKind, const char* szFormat, ...);
virtual void RegisterParredFile(const char* szFilename);
virtual bool IsParredFile(const char* szFilename);
virtual EFileStatus FindFileCrc(const char* szFilename, unsigned long* lCrc, SegmentList* pSegments);
virtual void RequestDupeSources(DupeSourceList* pDupeSourceList);
virtual void StatDupeSources(DupeSourceList* pDupeSourceList);
public:
PostInfo* GetPostInfo() { return m_pPostInfo; }
void SetPostInfo(PostInfo* pPostInfo) { m_pPostInfo = pPostInfo; }
time_t GetParTime() { return m_tParTime; }
void SetParTime(time_t tParTime) { m_tParTime = tParTime; }
time_t GetRepairTime() { return m_tRepairTime; }
void SetRepairTime(time_t tRepairTime) { m_tRepairTime = tRepairTime; }
int GetDownloadSec() { return m_iDownloadSec; }
void SetDownloadSec(int iDownloadSec) { m_iDownloadSec = iDownloadSec; }
friend class ParCoordinator;
};
@@ -82,27 +69,13 @@ private:
virtual void Completed() { m_pOwner->ParRenameCompleted(); }
virtual void PrintMessage(Message::EKind eKind, const char* szFormat, ...);
virtual void RegisterParredFile(const char* szFilename);
virtual void RegisterRenamedFile(const char* szOldFilename, const char* szNewFileName);
public:
PostInfo* GetPostInfo() { return m_pPostInfo; }
void SetPostInfo(PostInfo* pPostInfo) { m_pPostInfo = pPostInfo; }
friend class ParCoordinator;
};
class PostDupeMatcher: public DupeMatcher
{
private:
PostInfo* m_pPostInfo;
protected:
virtual void PrintMessage(Message::EKind eKind, const char* szFormat, ...);
public:
PostDupeMatcher(PostInfo* pPostInfo):
DupeMatcher(pPostInfo->GetNZBInfo()->GetDestDir(),
pPostInfo->GetNZBInfo()->GetSize() - pPostInfo->GetNZBInfo()->GetParSize()),
m_pPostInfo(pPostInfo) {}
};
struct BlockInfo
{
FileInfo* m_pFileInfo;
@@ -130,11 +103,18 @@ protected:
void ParRenameCompleted();
void CheckPauseState(PostInfo* pPostInfo);
bool RequestMorePars(NZBInfo* pNZBInfo, const char* szParFilename, int iBlockNeeded, int* pBlockFound);
void PrintMessage(PostInfo* pPostInfo, Message::EKind eKind, const char* szFormat, ...);
#endif
public:
typedef std::deque<char*> ParFileList;
public:
ParCoordinator();
virtual ~ParCoordinator();
static bool FindMainPars(const char* szPath, ParFileList* pFileList);
static bool ParseParFilename(const char* szParFilename, int* iBaseNameLen, int* iBlocks);
static bool SameParCollection(const char* szFilename1, const char* szFilename2);
void PausePars(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo);
#ifndef DISABLE_PARCHECK

View File

@@ -1,152 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#ifdef WIN32
#include "win32.h"
#endif
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <stdarg.h>
#include <ctype.h>
#ifdef WIN32
#include <direct.h>
#else
#include <unistd.h>
#endif
#include "nzbget.h"
#include "Util.h"
#include "ParParser.h"
bool ParParser::FindMainPars(const char* szPath, ParFileList* pFileList)
{
if (pFileList)
{
pFileList->clear();
}
DirBrowser dir(szPath);
while (const char* filename = dir.Next())
{
int iBaseLen = 0;
if (ParseParFilename(filename, &iBaseLen, NULL))
{
if (!pFileList)
{
return true;
}
// check if the base file already added to list
bool exists = false;
for (ParFileList::iterator it = pFileList->begin(); it != pFileList->end(); it++)
{
const char* filename2 = *it;
exists = SameParCollection(filename, filename2);
if (exists)
{
break;
}
}
if (!exists)
{
pFileList->push_back(strdup(filename));
}
}
}
return pFileList && !pFileList->empty();
}
bool ParParser::SameParCollection(const char* szFilename1, const char* szFilename2)
{
int iBaseLen1 = 0, iBaseLen2 = 0;
return ParseParFilename(szFilename1, &iBaseLen1, NULL) &&
ParseParFilename(szFilename2, &iBaseLen2, NULL) &&
iBaseLen1 == iBaseLen2 &&
!strncasecmp(szFilename1, szFilename2, iBaseLen1);
}
bool ParParser::ParseParFilename(const char* szParFilename, int* iBaseNameLen, int* iBlocks)
{
char szFilename[1024];
strncpy(szFilename, szParFilename, 1024);
szFilename[1024-1] = '\0';
for (char* p = szFilename; *p; p++) *p = tolower(*p); // convert string to lowercase
int iLen = strlen(szFilename);
if (iLen < 6)
{
return false;
}
// find last occurence of ".par2" and trim filename after it
char* szEnd = szFilename;
while (char* p = strstr(szEnd, ".par2")) szEnd = p + 5;
*szEnd = '\0';
iLen = strlen(szFilename);
if (iLen < 6)
{
return false;
}
if (strcasecmp(szFilename + iLen - 5, ".par2"))
{
return false;
}
*(szFilename + iLen - 5) = '\0';
int blockcnt = 0;
char* p = strrchr(szFilename, '.');
if (p && !strncasecmp(p, ".vol", 4))
{
char* b = strchr(p, '+');
if (!b)
{
b = strchr(p, '-');
}
if (b)
{
blockcnt = atoi(b+1);
*p = '\0';
}
}
if (iBaseNameLen)
{
*iBaseNameLen = strlen(szFilename);
}
if (iBlocks)
{
*iBlocks = blockcnt;
}
return true;
}

View File

@@ -1,41 +0,0 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* $Revision$
* $Date$
*
*/
#ifndef PARPARSER_H
#define PARPARSER_H
#include <deque>
class ParParser
{
public:
typedef std::deque<char*> ParFileList;
static bool FindMainPars(const char* szPath, ParFileList* pFileList);
static bool ParseParFilename(const char* szParFilename, int* iBaseNameLen, int* iBlocks);
static bool SameParCollection(const char* szFilename1, const char* szFilename2);
};
#endif

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2013-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -37,21 +37,26 @@
#include <string.h>
#include <stdio.h>
#include <ctype.h>
#ifndef WIN32
#ifdef WIN32
#include <par2cmdline.h>
#include <par2repairer.h>
#include <md5.h>
#else
#include <unistd.h>
#include <libpar2/par2cmdline.h>
#include <libpar2/par2repairer.h>
#include <libpar2/md5.h>
#endif
#include "par2cmdline.h"
#include "par2repairer.h"
#include "md5.h"
#include "nzbget.h"
#include "ParRenamer.h"
#include "ParParser.h"
#include "ParCoordinator.h"
#include "Log.h"
#include "Options.h"
#include "Util.h"
extern Options* g_pOptions;
class ParRenamerRepairer : public Par2Repairer
{
public:
@@ -81,6 +86,7 @@ ParRenamer::ParRenamer()
m_szProgressLabel = (char*)malloc(1024);
m_iStageProgress = 0;
m_bCancelled = false;
m_bHasSplittedFragments = false;
m_bHasMissedFiles = false;
m_bDetectMissing = false;
}
@@ -140,6 +146,7 @@ void ParRenamer::Run()
m_iFileCount = 0;
m_iCurFile = 0;
m_iRenamedCount = 0;
m_bHasSplittedFragments = false;
m_bHasMissedFiles = false;
m_eStatus = psFailed;
@@ -222,10 +229,10 @@ void ParRenamer::BuildDirList(const char* szDestDir)
void ParRenamer::LoadParFiles(const char* szDestDir)
{
ParParser::ParFileList parFileList;
ParParser::FindMainPars(szDestDir, &parFileList);
ParCoordinator::ParFileList parFileList;
ParCoordinator::FindMainPars(szDestDir, &parFileList);
for (ParParser::ParFileList::iterator it = parFileList.begin(); it != parFileList.end(); it++)
for (ParCoordinator::ParFileList::iterator it = parFileList.begin(); it != parFileList.end(); it++)
{
char* szParFilename = *it;
@@ -260,7 +267,7 @@ void ParRenamer::LoadParFile(const char* szParFilename)
Par2RepairerSourceFile* sourceFile = (*it).second;
if (!sourceFile || !sourceFile->GetDescriptionPacket())
{
PrintMessage(Message::mkWarning, "Damaged par2-file detected: %s", szParFilename);
warn("Damaged par2-file detected: %s", szParFilename);
continue;
}
m_FileHashList.push_back(new FileHash(sourceFile->GetDescriptionPacket()->FileName().c_str(),
@@ -313,11 +320,11 @@ void ParRenamer::CheckMissing()
if (Util::MatchFileExt(pFileHash->GetFilename(), g_pOptions->GetParIgnoreExt(), ",;") ||
Util::MatchFileExt(pFileHash->GetFilename(), g_pOptions->GetExtCleanupDisk(), ",;"))
{
PrintMessage(Message::mkInfo, "File %s is missing, ignoring", pFileHash->GetFilename());
info("File %s is missing, ignoring", pFileHash->GetFilename());
}
else
{
PrintMessage(Message::mkInfo, "File %s is missing", pFileHash->GetFilename());
info("File %s is missing", pFileHash->GetFilename());
m_bHasMissedFiles = true;
}
}
@@ -337,10 +344,12 @@ bool ParRenamer::IsSplittedFragment(const char* szFilename, const char* szCorrec
{
for (p++; *p && strchr("0123456789", *p); p++) ;
bSplittedFragement = !*p;
bSplittedFragement = bSplittedFragement && atoi(szDiskBasename + iBaseLen + 1) <= 1; // .000 or .001
bSplittedFragement = bSplittedFragement && atoi(szDiskBasename + iBaseLen + 1) == 1;
}
}
m_bHasSplittedFragments = m_bHasSplittedFragments || bSplittedFragement;
return bSplittedFragement;
}
@@ -394,7 +403,17 @@ void ParRenamer::CheckRegularFile(const char* szDestDir, const char* szFilename)
if (!Util::FileExists(szDstFilename) && !IsSplittedFragment(szFilename, pFileHash->GetFilename()))
{
RenameFile(szFilename, szDstFilename);
PrintMessage(Message::mkInfo, "Renaming %s to %s", Util::BaseFileName(szFilename), pFileHash->GetFilename());
if (Util::MoveFile(szFilename, szDstFilename))
{
m_iRenamedCount++;
}
else
{
char szErrBuf[256];
PrintMessage(Message::mkError, "Could not rename %s to %s: %s", szFilename, szDstFilename,
Util::GetLastErrorMessage(szErrBuf, sizeof(szErrBuf)));
}
}
break;
@@ -464,24 +483,17 @@ void ParRenamer::CheckParFile(const char* szDestDir, const char* szFilename)
iNum++;
}
RenameFile(szFilename, szDestFileName);
}
void ParRenamer::RenameFile(const char* szSrcFilename, const char* szDestFileName)
{
PrintMessage(Message::mkInfo, "Renaming %s to %s", Util::BaseFileName(szSrcFilename), Util::BaseFileName(szDestFileName));
if (!Util::MoveFile(szSrcFilename, szDestFileName))
PrintMessage(Message::mkInfo, "Renaming %s to %s", Util::BaseFileName(szFilename), Util::BaseFileName(szDestFileName));
if (Util::MoveFile(szFilename, szDestFileName))
{
m_iRenamedCount++;
}
else
{
char szErrBuf[256];
PrintMessage(Message::mkError, "Could not rename %s to %s: %s", szSrcFilename, szDestFileName,
PrintMessage(Message::mkError, "Could not rename %s to %s: %s", szFilename, szDestFileName,
Util::GetLastErrorMessage(szErrBuf, sizeof(szErrBuf)));
return;
}
m_iRenamedCount++;
// notify about new file name
RegisterRenamedFile(Util::BaseFileName(szSrcFilename), Util::BaseFileName(szDestFileName));
}
#endif

View File

@@ -59,6 +59,7 @@ public:
};
typedef std::deque<FileHash*> FileHashList;
typedef std::deque<char*> DirList;
private:
@@ -73,6 +74,7 @@ private:
int m_iFileCount;
int m_iCurFile;
int m_iRenamedCount;
bool m_bHasSplittedFragments;
bool m_bHasMissedFiles;
bool m_bDetectMissing;
@@ -87,14 +89,12 @@ private:
void CheckParFile(const char* szDestDir, const char* szFilename);
bool IsSplittedFragment(const char* szFilename, const char* szCorrectName);
void CheckMissing();
void RenameFile(const char* szSrcFilename, const char* szDestFileName);
protected:
virtual void UpdateProgress() {}
virtual void Completed() {}
virtual void PrintMessage(Message::EKind eKind, const char* szFormat, ...) {}
virtual void RegisterParredFile(const char* szFilename) {}
virtual void RegisterRenamedFile(const char* szOldFilename, const char* szNewFileName) {}
const char* GetProgressLabel() { return m_szProgressLabel; }
int GetStageProgress() { return m_iStageProgress; }
@@ -109,6 +109,7 @@ public:
EStatus GetStatus() { return m_eStatus; }
void Cancel();
bool GetCancelled() { return m_bCancelled; }
bool HasSplittedFragments() { return m_bHasSplittedFragments; }
bool HasMissedFiles() { return m_bHasMissedFiles; }
void SetDetectMissing(bool bDetectMissing) { m_bDetectMissing = bDetectMissing; }
};

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -45,6 +45,8 @@
#include "Util.h"
#include "Options.h"
extern Options* g_pOptions;
static const int POSTPROCESS_PARCHECK = 92;
static const int POSTPROCESS_SUCCESS = 93;
static const int POSTPROCESS_ERROR = 94;
@@ -92,7 +94,7 @@ void PostScriptController::Run()
m_pPostInfo->SetWorking(false);
}
void PostScriptController::ExecuteScript(ScriptConfig::Script* pScript)
void PostScriptController::ExecuteScript(Options::Script* pScript)
{
// if any script has requested par-check, do not execute other scripts
if (!pScript->GetPostScript() || m_pPostInfo->GetRequestParCheck())
@@ -102,14 +104,6 @@ void PostScriptController::ExecuteScript(ScriptConfig::Script* pScript)
PrintMessage(Message::mkInfo, "Executing post-process-script %s for %s", pScript->GetName(), m_pPostInfo->GetNZBInfo()->GetName());
char szProgressLabel[1024];
snprintf(szProgressLabel, 1024, "Executing post-process-script %s", pScript->GetName());
szProgressLabel[1024-1] = '\0';
DownloadQueue::Lock();
m_pPostInfo->SetProgressLabel(szProgressLabel);
DownloadQueue::Unlock();
SetScript(pScript->GetLocation());
SetArgs(NULL, false);
@@ -118,7 +112,6 @@ void PostScriptController::ExecuteScript(ScriptConfig::Script* pScript)
szInfoName[1024-1] = '\0';
SetInfoName(szInfoName);
m_pScript = pScript;
SetLogPrefix(pScript->GetDisplayName());
m_iPrefixLen = strlen(pScript->GetDisplayName()) + 2; // 2 = strlen(": ");
PrepareParams(pScript->GetName());
@@ -153,12 +146,6 @@ void PostScriptController::PrepareParams(const char* szScriptName)
SetIntEnvVar("NZBPP_HEALTH", m_pPostInfo->GetNZBInfo()->CalcHealth());
SetIntEnvVar("NZBPP_CRITICALHEALTH", m_pPostInfo->GetNZBInfo()->CalcCriticalHealth(false));
SetEnvVar("NZBPP_DUPEKEY", m_pPostInfo->GetNZBInfo()->GetDupeKey());
SetIntEnvVar("NZBPP_DUPESCORE", m_pPostInfo->GetNZBInfo()->GetDupeScore());
const char* szDupeModeName[] = { "SCORE", "ALL", "FORCE" };
SetEnvVar("NZBPP_DUPEMODE", szDupeModeName[m_pPostInfo->GetNZBInfo()->GetDupeMode()]);
char szStatus[256];
strncpy(szStatus, m_pPostInfo->GetNZBInfo()->MakeTextStatus(true), sizeof(szStatus));
szStatus[256-1] = '\0';
@@ -173,15 +160,7 @@ void PostScriptController::PrepareParams(const char* szScriptName)
// deprecated
int iParStatus[] = { 0, 0, 1, 2, 3, 4 };
NZBInfo::EParStatus eParStatus = m_pPostInfo->GetNZBInfo()->GetParStatus();
// for downloads marked as bad and for deleted downloads pass par status "Failure"
// for compatibility with older scripts which don't check "NZBPP_TOTALSTATUS"
if (m_pPostInfo->GetNZBInfo()->GetDeleteStatus() != NZBInfo::dsNone ||
m_pPostInfo->GetNZBInfo()->GetMarkStatus() == NZBInfo::ksBad)
{
eParStatus = NZBInfo::psFailure;
}
SetIntEnvVar("NZBPP_PARSTATUS", iParStatus[eParStatus]);
SetIntEnvVar("NZBPP_PARSTATUS", iParStatus[m_pPostInfo->GetNZBInfo()->GetParStatus()]);
// deprecated
int iUnpackStatus[] = { 0, 0, 1, 2, 3, 4 };
@@ -245,7 +224,6 @@ ScriptStatus::EStatus PostScriptController::AnalyseExitCode(int iExitCode)
{
PrintMessage(Message::mkInfo, "%s requested par-check/repair", GetInfoName());
m_pPostInfo->SetRequestParCheck(true);
m_pPostInfo->SetForceRepair(true);
return ScriptStatus::srSuccess;
}
break;
@@ -289,30 +267,23 @@ void PostScriptController::AddMessage(Message::EKind eKind, const char* szText)
}
else
{
m_pPostInfo->GetNZBInfo()->PrintMessage(Message::mkError,
"Invalid command \"%s\" received from %s", szMsgText, GetInfoName());
error("Invalid command \"%s\" received from %s", szMsgText, GetInfoName());
}
free(szParam);
}
else if (!strncmp(szMsgText + 6, "MARK=BAD", 8))
{
SetLogPrefix(NULL);
PrintMessage(Message::mkWarning, "Marking %s as bad", m_pPostInfo->GetNZBInfo()->GetName());
SetLogPrefix(m_pScript->GetDisplayName());
m_pPostInfo->GetNZBInfo()->SetMarkStatus(NZBInfo::ksBad);
}
else
{
m_pPostInfo->GetNZBInfo()->PrintMessage(Message::mkError,
"Invalid command \"%s\" received from %s", szMsgText, GetInfoName());
error("Invalid command \"%s\" received from %s", szMsgText, GetInfoName());
}
}
else if (!strncmp(szMsgText, "[HISTORY] ", 10))
{
m_pPostInfo->GetNZBInfo()->AppendMessage(eKind, 0, szMsgText);
}
else
{
m_pPostInfo->GetNZBInfo()->AddMessage(eKind, szText);
DownloadQueue::Lock();
m_pPostInfo->SetProgressLabel(szText);
DownloadQueue::Unlock();
ScriptController::AddMessage(eKind, szText);
m_pPostInfo->AppendMessage(eKind, szText);
}
if (g_pOptions->GetPausePostProcess() && !m_pPostInfo->GetNZBInfo()->GetForcePriority())

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -27,20 +27,23 @@
#define POSTSCRIPT_H
#include "Thread.h"
#include "NzbScript.h"
#include "Log.h"
#include "QueueScript.h"
#include "DownloadInfo.h"
#include "Options.h"
class PostScriptController : public Thread, public NZBScriptController
{
private:
PostInfo* m_pPostInfo;
char m_szNZBName[1024];
int m_iPrefixLen;
ScriptConfig::Script* m_pScript;
void PrepareParams(const char* szScriptName);
ScriptStatus::EStatus AnalyseExitCode(int iExitCode);
protected:
virtual void ExecuteScript(ScriptConfig::Script* pScript);
virtual void ExecuteScript(Options::Script* pScript);
virtual void AddMessage(Message::EKind eKind, const char* szText);
public:

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -50,11 +50,18 @@
#include "DupeCoordinator.h"
#include "PostScript.h"
#include "Util.h"
#include "Scheduler.h"
#include "Scanner.h"
#include "Unpack.h"
#include "Cleanup.h"
#include "NZBFile.h"
#include "QueueScript.h"
#include "ParParser.h"
#include "StatMeter.h"
extern HistoryCoordinator* g_pHistoryCoordinator;
extern DupeCoordinator* g_pDupeCoordinator;
extern Options* g_pOptions;
extern Scheduler* g_pScheduler;
extern Scanner* g_pScanner;
extern StatMeter* g_pStatMeter;
PrePostProcessor::PrePostProcessor()
{
@@ -91,19 +98,54 @@ void PrePostProcessor::Run()
DownloadQueue::Unlock();
}
g_pScheduler->FirstCheck();
int iDiskSpaceInterval = 1000;
int iSchedulerInterval = 1000;
int iHistoryInterval = 600000;
const int iStepMSec = 200;
while (!IsStopped())
{
if (!g_pOptions->GetTempPausePostprocess())
// check incoming nzb directory
g_pScanner->Check();
if (!g_pOptions->GetPauseDownload() &&
g_pOptions->GetDiskSpace() > 0 && !g_pStatMeter->GetStandBy() &&
iDiskSpaceInterval >= 1000)
{
// check post-queue every 200 msec
CheckPostQueue();
// check free disk space every 1 second
CheckDiskSpace();
iDiskSpaceInterval = 0;
}
iDiskSpaceInterval += iStepMSec;
// check post-queue every 200 msec
CheckPostQueue();
if (iSchedulerInterval >= 1000)
{
// check scheduler tasks every 1 second
g_pScheduler->IntervalCheck();
iSchedulerInterval = 0;
}
iSchedulerInterval += iStepMSec;
if (iHistoryInterval >= 600000)
{
// check history (remove old entries) every 10 minutes
g_pHistoryCoordinator->IntervalCheck();
iHistoryInterval = 0;
}
iHistoryInterval += iStepMSec;
Util::SetStandByMode(!m_pCurJob);
usleep(200 * 1000);
usleep(iStepMSec * 1000);
}
g_pHistoryCoordinator->Cleanup();
debug("Exiting PrePostProcessor-loop");
}
@@ -146,26 +188,9 @@ void PrePostProcessor::DownloadQueueUpdate(Subject* Caller, void* Aspect)
{
NZBAdded(pQueueAspect->pDownloadQueue, pQueueAspect->pNZBInfo);
}
else if (pQueueAspect->eAction == DownloadQueue::eaNzbDeleted &&
pQueueAspect->pNZBInfo->GetDeleting() &&
!pQueueAspect->pNZBInfo->GetPostInfo() &&
!pQueueAspect->pNZBInfo->GetParCleanup() &&
pQueueAspect->pNZBInfo->GetFileList()->empty())
{
// the deleting of nzbs is usually handled via eaFileDeleted-event, but when deleting nzb without
// any files left the eaFileDeleted-event is not fired and we need to process eaNzbDeleted-event instead
pQueueAspect->pNZBInfo->PrintMessage(Message::mkInfo,
"Collection %s deleted from queue", pQueueAspect->pNZBInfo->GetName());
NZBDeleted(pQueueAspect->pDownloadQueue, pQueueAspect->pNZBInfo);
}
else if ((pQueueAspect->eAction == DownloadQueue::eaFileCompleted ||
pQueueAspect->eAction == DownloadQueue::eaFileDeleted))
{
if (pQueueAspect->eAction == DownloadQueue::eaFileCompleted && !pQueueAspect->pNZBInfo->GetPostInfo())
{
g_pQueueScriptCoordinator->EnqueueScript(pQueueAspect->pNZBInfo, QueueScriptCoordinator::qeFileDownloaded);
}
if (
#ifndef DISABLE_PARCHECK
!m_ParCoordinator.AddPar(pQueueAspect->pFileInfo, pQueueAspect->eAction == DownloadQueue::eaFileDeleted) &&
@@ -179,9 +204,7 @@ void PrePostProcessor::DownloadQueueUpdate(Subject* Caller, void* Aspect)
IsNZBFileCompleted(pQueueAspect->pNZBInfo, false, true))) &&
pQueueAspect->pFileInfo->GetNZBInfo()->GetDeleteStatus() != NZBInfo::dsHealth)
{
pQueueAspect->pNZBInfo->PrintMessage(Message::mkInfo,
"Collection %s completely downloaded", pQueueAspect->pNZBInfo->GetName());
g_pQueueScriptCoordinator->EnqueueScript(pQueueAspect->pNZBInfo, QueueScriptCoordinator::qeNzbDownloaded);
info("Collection %s completely downloaded", pQueueAspect->pNZBInfo->GetName());
NZBDownloaded(pQueueAspect->pDownloadQueue, pQueueAspect->pNZBInfo);
}
else if ((pQueueAspect->eAction == DownloadQueue::eaFileDeleted ||
@@ -190,8 +213,7 @@ void PrePostProcessor::DownloadQueueUpdate(Subject* Caller, void* Aspect)
!pQueueAspect->pNZBInfo->GetParCleanup() &&
IsNZBFileCompleted(pQueueAspect->pNZBInfo, false, true))
{
pQueueAspect->pNZBInfo->PrintMessage(Message::mkInfo,
"Collection %s deleted from queue", pQueueAspect->pNZBInfo->GetName());
info("Collection %s deleted from queue", pQueueAspect->pNZBInfo->GetName());
NZBDeleted(pQueueAspect->pDownloadQueue, pQueueAspect->pNZBInfo);
}
}
@@ -213,30 +235,22 @@ void PrePostProcessor::NZBAdded(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo
m_ParCoordinator.PausePars(pDownloadQueue, pNZBInfo);
}
if (pNZBInfo->GetDeleteStatus() == NZBInfo::dsDupe ||
pNZBInfo->GetDeleteStatus() == NZBInfo::dsCopy ||
pNZBInfo->GetDeleteStatus() == NZBInfo::dsGood ||
pNZBInfo->GetDeleteStatus() == NZBInfo::dsScan)
if (g_pOptions->GetDupeCheck() && pNZBInfo->GetDupeMode() != dmForce &&
pNZBInfo->GetDeleteStatus() == NZBInfo::dsDupe)
{
NZBCompleted(pDownloadQueue, pNZBInfo, false);
}
else
else if (!Util::EmptyStr(g_pOptions->GetQueueScript()))
{
g_pQueueScriptCoordinator->EnqueueScript(pNZBInfo, QueueScriptCoordinator::qeNzbAdded);
QueueScriptController::StartScripts(pDownloadQueue, pNZBInfo);
}
}
void PrePostProcessor::NZBDownloaded(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo)
{
if (pNZBInfo->GetDeleteStatus() == NZBInfo::dsHealth ||
pNZBInfo->GetDeleteStatus() == NZBInfo::dsBad)
{
g_pQueueScriptCoordinator->EnqueueScript(pNZBInfo, QueueScriptCoordinator::qeNzbDeleted);
}
if (!pNZBInfo->GetPostInfo() && g_pOptions->GetDecode())
{
pNZBInfo->PrintMessage(Message::mkInfo, "Queueing %s for post-processing", pNZBInfo->GetName());
info("Queueing %s for post-processing", pNZBInfo->GetName());
pNZBInfo->EnterPostProcess();
m_iJobCount++;
@@ -253,6 +267,14 @@ void PrePostProcessor::NZBDownloaded(DownloadQueue* pDownloadQueue, NZBInfo* pNZ
pNZBInfo->SetRenameStatus(NZBInfo::rsSkipped);
}
if (pNZBInfo->GetDeleteStatus() != NZBInfo::dsNone)
{
pNZBInfo->SetParStatus(NZBInfo::psFailure);
pNZBInfo->SetUnpackStatus(NZBInfo::usFailure);
pNZBInfo->SetRenameStatus(NZBInfo::rsFailure);
pNZBInfo->SetMoveStatus(NZBInfo::msFailure);
}
pDownloadQueue->Save();
}
else
@@ -269,79 +291,17 @@ void PrePostProcessor::NZBDeleted(DownloadQueue* pDownloadQueue, NZBInfo* pNZBIn
}
pNZBInfo->SetDeleting(false);
DeleteCleanup(pNZBInfo);
if (pNZBInfo->GetDeleteStatus() == NZBInfo::dsHealth ||
pNZBInfo->GetDeleteStatus() == NZBInfo::dsBad)
{
NZBDownloaded(pDownloadQueue, pNZBInfo);
}
else
{
NZBCompleted(pDownloadQueue, pNZBInfo, true);
}
}
void PrePostProcessor::NZBCompleted(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo, bool bSaveQueue)
{
bool bAddToHistory = g_pOptions->GetKeepHistory() > 0 && !pNZBInfo->GetAvoidHistory();
if (bAddToHistory)
{
g_pHistoryCoordinator->AddToHistory(pDownloadQueue, pNZBInfo);
}
pNZBInfo->SetAvoidHistory(false);
bool bNeedSave = bAddToHistory;
if (g_pOptions->GetDupeCheck() && pNZBInfo->GetDupeMode() != dmForce &&
(pNZBInfo->GetDeleteStatus() == NZBInfo::dsNone ||
pNZBInfo->GetDeleteStatus() == NZBInfo::dsHealth ||
pNZBInfo->GetDeleteStatus() == NZBInfo::dsBad ||
pNZBInfo->GetDeleteStatus() == NZBInfo::dsScan))
{
g_pDupeCoordinator->NZBCompleted(pDownloadQueue, pNZBInfo);
bNeedSave = true;
}
if (pNZBInfo->GetDeleteStatus() > NZBInfo::dsNone &&
pNZBInfo->GetDeleteStatus() != NZBInfo::dsHealth &&
pNZBInfo->GetDeleteStatus() != NZBInfo::dsBad)
// nzbs deleted by health check or marked as bad are processed as downloaded with failure status
{
g_pQueueScriptCoordinator->EnqueueScript(pNZBInfo, QueueScriptCoordinator::qeNzbDeleted);
}
if (!bAddToHistory)
{
g_pHistoryCoordinator->DeleteDiskFiles(pNZBInfo);
pDownloadQueue->GetQueue()->Remove(pNZBInfo);
delete pNZBInfo;
}
if (bSaveQueue && bNeedSave)
{
pDownloadQueue->Save();
}
}
void PrePostProcessor::DeleteCleanup(NZBInfo* pNZBInfo)
{
if ((g_pOptions->GetDeleteCleanupDisk() && pNZBInfo->GetCleanupDisk()) ||
pNZBInfo->GetDeleteStatus() == NZBInfo::dsDupe)
{
// download was cancelled, deleting already downloaded files from disk
for (CompletedFiles::reverse_iterator it = pNZBInfo->GetCompletedFiles()->rbegin(); it != pNZBInfo->GetCompletedFiles()->rend(); it++)
for (NZBInfo::Files::reverse_iterator it = pNZBInfo->GetCompletedFiles()->rbegin(); it != pNZBInfo->GetCompletedFiles()->rend(); it++)
{
CompletedFile* pCompletedFile = *it;
char szFullFileName[1024];
snprintf(szFullFileName, 1024, "%s%c%s", pNZBInfo->GetDestDir(), (int)PATH_SEPARATOR, pCompletedFile->GetFileName());
szFullFileName[1024-1] = '\0';
if (Util::FileExists(szFullFileName))
char* szFilename = *it;
if (Util::FileExists(szFilename))
{
detail("Deleting file %s", pCompletedFile->GetFileName());
remove(szFullFileName);
detail("Deleting file %s", Util::BaseFileName(szFilename));
remove(szFilename);
}
}
@@ -367,6 +327,67 @@ void PrePostProcessor::DeleteCleanup(NZBInfo* pNZBInfo)
rmdir(pNZBInfo->GetDestDir());
}
}
if (pNZBInfo->GetDeleteStatus() == NZBInfo::dsHealth)
{
NZBDownloaded(pDownloadQueue, pNZBInfo);
}
else
{
NZBCompleted(pDownloadQueue, pNZBInfo, true);
}
}
void PrePostProcessor::NZBCompleted(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo, bool bSaveQueue)
{
bool bAddToHistory = g_pOptions->GetKeepHistory() > 0 && !pNZBInfo->GetAvoidHistory();
if (bAddToHistory)
{
g_pHistoryCoordinator->AddToHistory(pDownloadQueue, pNZBInfo);
}
pNZBInfo->SetAvoidHistory(false);
bool bNeedSave = bAddToHistory;
if (g_pOptions->GetDupeCheck() && pNZBInfo->GetDupeMode() != dmForce &&
(pNZBInfo->GetDeleteStatus() == NZBInfo::dsNone ||
pNZBInfo->GetDeleteStatus() == NZBInfo::dsHealth))
{
g_pDupeCoordinator->NZBCompleted(pDownloadQueue, pNZBInfo);
bNeedSave = true;
}
if (!bAddToHistory)
{
g_pHistoryCoordinator->DeleteQueuedFile(pNZBInfo->GetQueuedFilename());
pDownloadQueue->GetQueue()->Remove(pNZBInfo);
delete pNZBInfo;
}
if (bSaveQueue && bNeedSave)
{
pDownloadQueue->Save();
}
}
void PrePostProcessor::CheckDiskSpace()
{
long long lFreeSpace = Util::FreeDiskSize(g_pOptions->GetDestDir());
if (lFreeSpace > -1 && lFreeSpace / 1024 / 1024 < g_pOptions->GetDiskSpace())
{
warn("Low disk space on %s. Pausing download", g_pOptions->GetDestDir());
g_pOptions->SetPauseDownload(true);
}
if (!Util::EmptyStr(g_pOptions->GetInterDir()))
{
lFreeSpace = Util::FreeDiskSize(g_pOptions->GetInterDir());
if (lFreeSpace > -1 && lFreeSpace / 1024 / 1024 < g_pOptions->GetDiskSpace())
{
warn("Low disk space on %s. Pausing download", g_pOptions->GetInterDir());
g_pOptions->SetPauseDownload(true);
}
}
}
void PrePostProcessor::CheckPostQueue()
@@ -376,6 +397,11 @@ void PrePostProcessor::CheckPostQueue()
if (!m_pCurJob && m_iJobCount > 0)
{
m_pCurJob = GetNextJob(pDownloadQueue);
if (!m_pCurJob && !g_pOptions->GetPausePostProcess())
{
error("Internal error: no jobs found in queue");
m_iJobCount = 0;
}
}
if (m_pCurJob)
@@ -384,12 +410,9 @@ void PrePostProcessor::CheckPostQueue()
if (!pPostInfo->GetWorking() && !IsNZBFileDownloading(m_pCurJob))
{
#ifndef DISABLE_PARCHECK
if (pPostInfo->GetRequestParCheck() &&
(pPostInfo->GetNZBInfo()->GetParStatus() <= NZBInfo::psSkipped ||
(pPostInfo->GetForceRepair() && !pPostInfo->GetNZBInfo()->GetParFull())) &&
if (pPostInfo->GetRequestParCheck() && pPostInfo->GetNZBInfo()->GetParStatus() <= NZBInfo::psSkipped &&
g_pOptions->GetParCheck() != Options::pcManual)
{
pPostInfo->SetForceParFull(pPostInfo->GetNZBInfo()->GetParStatus() > NZBInfo::psSkipped);
pPostInfo->GetNZBInfo()->SetParStatus(NZBInfo::psNone);
pPostInfo->SetRequestParCheck(false);
pPostInfo->SetStage(PostInfo::ptQueued);
@@ -405,15 +428,13 @@ void PrePostProcessor::CheckPostQueue()
if (!pPostInfo->GetNZBInfo()->GetFileList()->empty())
{
pPostInfo->GetNZBInfo()->PrintMessage(Message::mkInfo,
"Downloading all remaining files for manual par-check for %s", pPostInfo->GetNZBInfo()->GetName());
info("Downloading all remaining files for manual par-check for %s", pPostInfo->GetNZBInfo()->GetName());
pDownloadQueue->EditEntry(pPostInfo->GetNZBInfo()->GetID(), DownloadQueue::eaGroupResume, 0, NULL);
pPostInfo->SetStage(PostInfo::ptFinished);
}
else
{
pPostInfo->GetNZBInfo()->PrintMessage(Message::mkInfo,
"There are no par-files remain for download for %s", pPostInfo->GetNZBInfo()->GetName());
info("There are no par-files remain for download for %s", pPostInfo->GetNZBInfo()->GetName());
pPostInfo->SetStage(PostInfo::ptQueued);
}
}
@@ -453,8 +474,7 @@ NZBInfo* PrePostProcessor::GetNextJob(DownloadQueue* pDownloadQueue)
for (NZBList::iterator it = pDownloadQueue->GetQueue()->begin(); it != pDownloadQueue->GetQueue()->end(); it++)
{
NZBInfo* pNZBInfo1 = *it;
if (pNZBInfo1->GetPostInfo() && !g_pQueueScriptCoordinator->HasJob(pNZBInfo1->GetID(), NULL) &&
(!pNZBInfo || pNZBInfo1->GetPriority() > pNZBInfo->GetPriority()) &&
if (pNZBInfo1->GetPostInfo() && (!pNZBInfo || pNZBInfo1->GetPriority() > pNZBInfo->GetPriority()) &&
(!g_pOptions->GetPausePostProcess() || pNZBInfo1->GetForcePriority()))
{
pNZBInfo = pNZBInfo1;
@@ -499,31 +519,23 @@ void PrePostProcessor::DeletePostThread(PostInfo* pPostInfo)
void PrePostProcessor::StartJob(DownloadQueue* pDownloadQueue, PostInfo* pPostInfo)
{
if (!pPostInfo->GetStartTime())
{
pPostInfo->SetStartTime(time(NULL));
}
#ifndef DISABLE_PARCHECK
if (pPostInfo->GetNZBInfo()->GetRenameStatus() == NZBInfo::rsNone &&
pPostInfo->GetNZBInfo()->GetDeleteStatus() == NZBInfo::dsNone)
if (pPostInfo->GetNZBInfo()->GetRenameStatus() == NZBInfo::rsNone)
{
UpdatePauseState(g_pOptions->GetParPauseQueue(), "par-rename");
m_ParCoordinator.StartParRenameJob(pPostInfo);
return;
}
else if (pPostInfo->GetNZBInfo()->GetParStatus() == NZBInfo::psNone &&
pPostInfo->GetNZBInfo()->GetDeleteStatus() == NZBInfo::dsNone)
else if (pPostInfo->GetNZBInfo()->GetParStatus() == NZBInfo::psNone)
{
if (ParParser::FindMainPars(pPostInfo->GetNZBInfo()->GetDestDir(), NULL))
if (m_ParCoordinator.FindMainPars(pPostInfo->GetNZBInfo()->GetDestDir(), NULL))
{
UpdatePauseState(g_pOptions->GetParPauseQueue(), "par-check");
m_ParCoordinator.StartParCheckJob(pPostInfo);
}
else
{
pPostInfo->GetNZBInfo()->PrintMessage(Message::mkInfo,
"Nothing to par-check for %s", pPostInfo->GetNZBInfo()->GetName());
info("Nothing to par-check for %s", pPostInfo->GetNZBInfo()->GetName());
pPostInfo->GetNZBInfo()->SetParStatus(NZBInfo::psSkipped);
pPostInfo->SetWorking(false);
pPostInfo->SetStage(PostInfo::ptQueued);
@@ -531,27 +543,20 @@ void PrePostProcessor::StartJob(DownloadQueue* pDownloadQueue, PostInfo* pPostIn
return;
}
else if (pPostInfo->GetNZBInfo()->GetParStatus() == NZBInfo::psSkipped &&
((g_pOptions->GetParScan() != Options::psDupe &&
pPostInfo->GetNZBInfo()->CalcHealth() < pPostInfo->GetNZBInfo()->CalcCriticalHealth(false) &&
pPostInfo->GetNZBInfo()->CalcCriticalHealth(false) < 1000) ||
pPostInfo->GetNZBInfo()->CalcHealth() == 0) &&
ParParser::FindMainPars(pPostInfo->GetNZBInfo()->GetDestDir(), NULL))
pPostInfo->GetNZBInfo()->CalcHealth() < pPostInfo->GetNZBInfo()->CalcCriticalHealth(false) &&
pPostInfo->GetNZBInfo()->CalcCriticalHealth(false) < 1000 &&
m_ParCoordinator.FindMainPars(pPostInfo->GetNZBInfo()->GetDestDir(), NULL))
{
pPostInfo->GetNZBInfo()->PrintMessage(Message::mkWarning,
pPostInfo->GetNZBInfo()->CalcHealth() == 0 ?
"Skipping par-check for %s due to health 0%%" :
"Skipping par-check for %s due to health %.1f%% below critical %.1f%%",
pPostInfo->GetNZBInfo()->GetName(),
warn("Skipping par-check for %s due to health %.1f%% below critical %.1f%%", pPostInfo->GetNZBInfo()->GetName(),
pPostInfo->GetNZBInfo()->CalcHealth() / 10.0, pPostInfo->GetNZBInfo()->CalcCriticalHealth(false) / 10.0);
pPostInfo->GetNZBInfo()->SetParStatus(NZBInfo::psFailure);
return;
}
else if (pPostInfo->GetNZBInfo()->GetParStatus() == NZBInfo::psSkipped &&
pPostInfo->GetNZBInfo()->GetFailedSize() - pPostInfo->GetNZBInfo()->GetParFailedSize() > 0 &&
ParParser::FindMainPars(pPostInfo->GetNZBInfo()->GetDestDir(), NULL))
m_ParCoordinator.FindMainPars(pPostInfo->GetNZBInfo()->GetDestDir(), NULL))
{
pPostInfo->GetNZBInfo()->PrintMessage(Message::mkInfo,
"Collection %s with health %.1f%% needs par-check",
info("Collection %s with health %.1f%% needs par-check",
pPostInfo->GetNZBInfo()->GetName(), pPostInfo->GetNZBInfo()->CalcHealth() / 10.0);
pPostInfo->SetRequestParCheck(true);
return;
@@ -560,8 +565,7 @@ void PrePostProcessor::StartJob(DownloadQueue* pDownloadQueue, PostInfo* pPostIn
NZBParameter* pUnpackParameter = pPostInfo->GetNZBInfo()->GetParameters()->Find("*Unpack:", false);
bool bUnpackParam = !(pUnpackParameter && !strcasecmp(pUnpackParameter->GetValue(), "no"));
bool bUnpack = bUnpackParam && pPostInfo->GetNZBInfo()->GetUnpackStatus() == NZBInfo::usNone &&
pPostInfo->GetNZBInfo()->GetDeleteStatus() == NZBInfo::dsNone;
bool bUnpack = bUnpackParam && (pPostInfo->GetNZBInfo()->GetUnpackStatus() == NZBInfo::usNone);
bool bParFailed = pPostInfo->GetNZBInfo()->GetParStatus() == NZBInfo::psFailure ||
pPostInfo->GetNZBInfo()->GetParStatus() == NZBInfo::psRepairPossible ||
@@ -569,18 +573,10 @@ void PrePostProcessor::StartJob(DownloadQueue* pDownloadQueue, PostInfo* pPostIn
bool bCleanup = !bUnpack &&
pPostInfo->GetNZBInfo()->GetCleanupStatus() == NZBInfo::csNone &&
!Util::EmptyStr(g_pOptions->GetExtCleanupDisk()) &&
((pPostInfo->GetNZBInfo()->GetParStatus() == NZBInfo::psSuccess &&
pPostInfo->GetNZBInfo()->GetUnpackStatus() != NZBInfo::usFailure &&
pPostInfo->GetNZBInfo()->GetUnpackStatus() != NZBInfo::usSpace &&
pPostInfo->GetNZBInfo()->GetUnpackStatus() != NZBInfo::usPassword) ||
(pPostInfo->GetNZBInfo()->GetParStatus() == NZBInfo::psSuccess ||
(pPostInfo->GetNZBInfo()->GetUnpackStatus() == NZBInfo::usSuccess &&
pPostInfo->GetNZBInfo()->GetParStatus() != NZBInfo::psFailure) ||
((pPostInfo->GetNZBInfo()->GetUnpackStatus() == NZBInfo::usNone ||
pPostInfo->GetNZBInfo()->GetUnpackStatus() == NZBInfo::usSkipped) &&
(pPostInfo->GetNZBInfo()->GetParStatus() == NZBInfo::psNone ||
pPostInfo->GetNZBInfo()->GetParStatus() == NZBInfo::psSkipped) &&
pPostInfo->GetNZBInfo()->CalcHealth() == 1000));
pPostInfo->GetNZBInfo()->GetParStatus() != NZBInfo::psFailure)) &&
strlen(g_pOptions->GetExtCleanupDisk()) > 0;
bool bMoveInter = !bUnpack &&
pPostInfo->GetNZBInfo()->GetMoveStatus() == NZBInfo::msNone &&
@@ -589,16 +585,14 @@ void PrePostProcessor::StartJob(DownloadQueue* pDownloadQueue, PostInfo* pPostIn
pPostInfo->GetNZBInfo()->GetUnpackStatus() != NZBInfo::usPassword &&
pPostInfo->GetNZBInfo()->GetParStatus() != NZBInfo::psFailure &&
pPostInfo->GetNZBInfo()->GetParStatus() != NZBInfo::psManual &&
pPostInfo->GetNZBInfo()->GetDeleteStatus() == NZBInfo::dsNone &&
!Util::EmptyStr(g_pOptions->GetInterDir()) &&
strlen(g_pOptions->GetInterDir()) > 0 &&
!strncmp(pPostInfo->GetNZBInfo()->GetDestDir(), g_pOptions->GetInterDir(), strlen(g_pOptions->GetInterDir()));
bool bPostScript = true;
if (bUnpack && bParFailed)
{
pPostInfo->GetNZBInfo()->PrintMessage(Message::mkWarning,
"Skipping unpack for %s due to %s", pPostInfo->GetNZBInfo()->GetName(),
warn("Skipping unpack for %s due to %s", pPostInfo->GetNZBInfo()->GetName(),
pPostInfo->GetNZBInfo()->GetParStatus() == NZBInfo::psManual ? "required par-repair" : "par-failure");
pPostInfo->GetNZBInfo()->SetUnpackStatus(NZBInfo::usSkipped);
bUnpack = false;
@@ -618,6 +612,10 @@ void PrePostProcessor::StartJob(DownloadQueue* pDownloadQueue, PostInfo* pPostIn
pDownloadQueue->Save();
if (!pPostInfo->GetStartTime())
{
pPostInfo->SetStartTime(time(NULL));
}
pPostInfo->SetStageTime(time(NULL));
if (bUnpack)
@@ -645,44 +643,27 @@ void PrePostProcessor::StartJob(DownloadQueue* pDownloadQueue, PostInfo* pPostIn
void PrePostProcessor::JobCompleted(DownloadQueue* pDownloadQueue, PostInfo* pPostInfo)
{
NZBInfo* pNZBInfo = pPostInfo->GetNZBInfo();
if (pPostInfo->GetStartTime() > 0)
{
pNZBInfo->SetPostTotalSec((int)(time(NULL) - pPostInfo->GetStartTime()));
pPostInfo->SetStartTime(0);
}
DeletePostThread(pPostInfo);
pNZBInfo->LeavePostProcess();
if (IsNZBFileCompleted(pNZBInfo, true, false))
{
// Cleaning up queue if par-check was successful or unpack was successful or
// health is 100% (if unpack and par-check were not performed)
// or health is below critical health
bool bCanCleanupQueue =
((pNZBInfo->GetParStatus() == NZBInfo::psSuccess ||
pNZBInfo->GetParStatus() == NZBInfo::psRepairPossible) &&
pNZBInfo->GetUnpackStatus() != NZBInfo::usFailure &&
pNZBInfo->GetUnpackStatus() != NZBInfo::usSpace &&
pNZBInfo->GetUnpackStatus() != NZBInfo::usPassword) ||
(pNZBInfo->GetUnpackStatus() == NZBInfo::usSuccess &&
pNZBInfo->GetParStatus() != NZBInfo::psFailure) ||
(pNZBInfo->GetUnpackStatus() <= NZBInfo::usSkipped &&
pNZBInfo->GetParStatus() != NZBInfo::psFailure &&
pNZBInfo->GetFailedSize() - pNZBInfo->GetParFailedSize() == 0) ||
(pNZBInfo->CalcHealth() < pNZBInfo->CalcCriticalHealth(false) &&
pNZBInfo->CalcCriticalHealth(false) < 1000);
if (g_pOptions->GetParCleanupQueue() && bCanCleanupQueue && !pNZBInfo->GetFileList()->empty())
// script was successful (if unpack was not performed)
bool bCanCleanupQueue = pNZBInfo->GetParStatus() == NZBInfo::psSuccess ||
pNZBInfo->GetParStatus() == NZBInfo::psRepairPossible ||
pNZBInfo->GetUnpackStatus() == NZBInfo::usSuccess ||
(pNZBInfo->GetUnpackStatus() == NZBInfo::usNone &&
pNZBInfo->GetScriptStatuses()->CalcTotalStatus() == ScriptStatus::srSuccess);
if (g_pOptions->GetParCleanupQueue() && bCanCleanupQueue)
{
pNZBInfo->PrintMessage(Message::mkInfo, "Cleaning up download queue for %s", pNZBInfo->GetName());
pNZBInfo->SetParCleanup(true);
pDownloadQueue->EditEntry(pNZBInfo->GetID(), DownloadQueue::eaGroupDelete, 0, NULL);
}
if (pNZBInfo->GetUnpackCleanedUpDisk())
{
pNZBInfo->ClearCompletedFiles();
if (!pNZBInfo->GetFileList()->empty())
{
info("Cleaning up download queue for %s", pNZBInfo->GetName());
pNZBInfo->ClearCompletedFiles();
pNZBInfo->SetParCleanup(true);
pDownloadQueue->EditEntry(pNZBInfo->GetID(), DownloadQueue::eaGroupDelete, 0, NULL);
}
}
NZBCompleted(pDownloadQueue, pNZBInfo, false);
@@ -781,8 +762,7 @@ bool PrePostProcessor::PostQueueDelete(DownloadQueue* pDownloadQueue, IDList* pI
{
if (pPostInfo->GetWorking())
{
pPostInfo->GetNZBInfo()->PrintMessage(Message::mkInfo,
"Deleting active post-job %s", pPostInfo->GetNZBInfo()->GetName());
info("Deleting active post-job %s", pPostInfo->GetNZBInfo()->GetName());
pPostInfo->SetDeleted(true);
#ifndef DISABLE_PARCHECK
if (PostInfo::ptLoadingPars <= pPostInfo->GetStage() && pPostInfo->GetStage() <= PostInfo::ptRenaming)
@@ -807,8 +787,7 @@ bool PrePostProcessor::PostQueueDelete(DownloadQueue* pDownloadQueue, IDList* pI
}
else
{
pPostInfo->GetNZBInfo()->PrintMessage(Message::mkInfo,
"Deleting queued post-job %s", pPostInfo->GetNZBInfo()->GetName());
info("Deleting queued post-job %s", pPostInfo->GetNZBInfo()->GetName());
JobCompleted(pDownloadQueue, pPostInfo);
bOK = true;
}

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -57,15 +57,16 @@ private:
void StartJob(DownloadQueue* pDownloadQueue, PostInfo* pPostInfo);
void SaveQueue(DownloadQueue* pDownloadQueue);
void SanitisePostQueue(DownloadQueue* pDownloadQueue);
void CheckDiskSpace();
void UpdatePauseState(bool bNeedPause, const char* szReason);
void NZBFound(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo);
void NZBAdded(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo);
void NZBDeleted(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo);
void NZBCompleted(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo, bool bSaveQueue);
bool PostQueueDelete(DownloadQueue* pDownloadQueue, IDList* pIDList);
void DeletePostThread(PostInfo* pPostInfo);
NZBInfo* GetNextJob(DownloadQueue* pDownloadQueue);
void DownloadQueueUpdate(Subject* Caller, void* Aspect);
void DeleteCleanup(NZBInfo* pNZBInfo);
public:
PrePostProcessor();
@@ -75,10 +76,7 @@ public:
bool HasMoreJobs() { return m_iJobCount > 0; }
int GetJobCount() { return m_iJobCount; }
bool EditList(DownloadQueue* pDownloadQueue, IDList* pIDList, DownloadQueue::EEditAction eAction, int iOffset, const char* szText);
void NZBAdded(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo);
void NZBDownloaded(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo);
};
extern PrePostProcessor* g_pPrePostProcessor;
#endif

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2013-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -44,9 +44,11 @@
#include "Unpack.h"
#include "Log.h"
#include "Util.h"
#include "ParParser.h"
#include "ParCoordinator.h"
#include "Options.h"
extern Options* g_pOptions;
void UnpackController::FileList::Clear()
{
for (iterator it = begin(); it != end(); it++)
@@ -70,28 +72,6 @@ bool UnpackController::FileList::Exists(const char* szFilename)
return false;
}
UnpackController::ParamList::~ParamList()
{
for (iterator it = begin(); it != end(); it++)
{
free(*it);
}
}
bool UnpackController::ParamList::Exists(const char* szParam)
{
for (iterator it = begin(); it != end(); it++)
{
char* szParam1 = *it;
if (!strcmp(szParam1, szParam))
{
return true;
}
}
return false;
}
void UnpackController::StartJob(PostInfo* pPostInfo)
{
UnpackController* pUnpackController = new UnpackController();
@@ -105,8 +85,6 @@ void UnpackController::StartJob(PostInfo* pPostInfo)
void UnpackController::Run()
{
time_t tStart = time(NULL);
// the locking is needed for accessing the members of NZBInfo
DownloadQueue::Lock();
@@ -120,14 +98,7 @@ void UnpackController::Run()
m_szPassword[0] = '\0';
m_szFinalDir[0] = '\0';
m_bFinalDirCreated = false;
m_bUnpackOK = true;
m_bUnpackStartError = false;
m_bUnpackSpaceError = false;
m_bUnpackDecryptError = false;
m_bUnpackPasswordError = false;
m_bAutoTerminated = false;
m_bPassListTried = false;
NZBParameter* pParameter = m_pPostInfo->GetNZBInfo()->GetParameters()->Find("*Unpack:", false);
bool bUnpack = !(pParameter && !strcasecmp(pParameter->GetValue(), "no"));
@@ -146,7 +117,7 @@ void UnpackController::Run()
snprintf(m_szInfoNameUp, 1024, "Unpack for %s", m_szName); // first letter in upper case
m_szInfoNameUp[1024-1] = '\0';
m_bHasParFiles = ParParser::FindMainPars(m_szDestDir, NULL);
m_bHasParFiles = ParCoordinator::FindMainPars(m_szDestDir, NULL);
if (bUnpack)
{
@@ -156,51 +127,36 @@ void UnpackController::Run()
CheckArchiveFiles(bScanNonStdFiles);
}
SetInfoName(m_szInfoName);
SetWorkingDir(m_szDestDir);
bool bHasFiles = m_bHasRarFiles || m_bHasNonStdRarFiles || m_bHasSevenZipFiles || m_bHasSevenZipMultiFiles || m_bHasSplittedFiles;
if (m_pPostInfo->GetUnpackTried() && !m_pPostInfo->GetParRepaired() &&
(!Util::EmptyStr(m_szPassword) || Util::EmptyStr(g_pOptions->GetUnpackPassFile()) || m_pPostInfo->GetPassListTried()))
{
PrintMessage(Message::mkInfo, "Second unpack attempt skipped for %s due to par-check not repaired anything", m_szName);
PrintMessage(Message::mkError,
m_pPostInfo->GetLastUnpackStatus() == (int)NZBInfo::usPassword ?
"%s failed: checksum error in the encrypted file. Corrupt file or wrong password." : "%s failed.",
m_szInfoNameUp);
m_pPostInfo->GetNZBInfo()->SetUnpackStatus((NZBInfo::EUnpackStatus)m_pPostInfo->GetLastUnpackStatus());
m_pPostInfo->SetStage(PostInfo::ptQueued);
}
else if (bUnpack && bHasFiles)
if (bUnpack && (m_bHasRarFiles || m_bHasNonStdRarFiles || m_bHasSevenZipFiles || m_bHasSevenZipMultiFiles))
{
SetInfoName(m_szInfoName);
SetWorkingDir(m_szDestDir);
PrintMessage(Message::mkInfo, "Unpacking %s", m_szName);
CreateUnpackDir();
m_bUnpackOK = true;
m_bUnpackStartError = false;
m_bUnpackSpaceError = false;
m_bUnpackPasswordError = false;
if (m_bHasRarFiles || m_bHasNonStdRarFiles)
{
UnpackArchives(upUnrar, false);
ExecuteUnrar();
}
if (m_bHasSevenZipFiles && m_bUnpackOK)
{
UnpackArchives(upSevenZip, false);
ExecuteSevenZip(false);
}
if (m_bHasSevenZipMultiFiles && m_bUnpackOK)
{
UnpackArchives(upSevenZip, true);
}
if (m_bHasSplittedFiles && m_bUnpackOK)
{
JoinSplittedFiles();
ExecuteSevenZip(true);
}
Completed();
m_JoinedFiles.Clear();
}
else
{
@@ -210,7 +166,7 @@ void UnpackController::Run()
if (bUnpack && m_pPostInfo->GetNZBInfo()->GetParStatus() <= NZBInfo::psSkipped &&
m_pPostInfo->GetNZBInfo()->GetRenameStatus() <= NZBInfo::rsSkipped && m_bHasParFiles)
{
RequestParCheck(false);
RequestParCheck();
}
else
#endif
@@ -220,132 +176,33 @@ void UnpackController::Run()
}
}
int iUnpackSec = (int)(time(NULL) - tStart);
m_pPostInfo->GetNZBInfo()->SetUnpackSec(m_pPostInfo->GetNZBInfo()->GetUnpackSec() + iUnpackSec);
m_pPostInfo->SetWorking(false);
}
void UnpackController::UnpackArchives(EUnpacker eUnpacker, bool bMultiVolumes)
{
if (!m_pPostInfo->GetUnpackTried() || m_pPostInfo->GetParRepaired())
{
ExecuteUnpack(eUnpacker, m_szPassword, bMultiVolumes);
if (!m_bUnpackOK && m_bHasParFiles && !m_bUnpackPasswordError &&
m_pPostInfo->GetNZBInfo()->GetParStatus() <= NZBInfo::psSkipped)
{
// for rar4- or 7z-archives try par-check first, before trying password file
return;
}
}
else
{
m_bUnpackOK = false;
m_bUnpackDecryptError = m_pPostInfo->GetLastUnpackStatus() == (int)NZBInfo::usPassword;
}
if (!m_bUnpackOK && !m_bUnpackStartError && !m_bUnpackSpaceError &&
(m_bUnpackDecryptError || m_bUnpackPasswordError) &&
(!GetTerminated() || m_bAutoTerminated) &&
Util::EmptyStr(m_szPassword) && !Util::EmptyStr(g_pOptions->GetUnpackPassFile()))
{
FILE* infile = fopen(g_pOptions->GetUnpackPassFile(), FOPEN_RB);
if (!infile)
{
PrintMessage(Message::mkError, "Could not open file %s", g_pOptions->GetUnpackPassFile());
return;
}
char szPassword[512];
while (!m_bUnpackOK && !m_bUnpackStartError && !m_bUnpackSpaceError &&
(m_bUnpackDecryptError || m_bUnpackPasswordError) &&
fgets(szPassword, sizeof(szPassword) - 1, infile))
{
// trim trailing <CR> and <LF>
char* szEnd = szPassword + strlen(szPassword) - 1;
while (szEnd >= szPassword && (*szEnd == '\n' || *szEnd == '\r')) *szEnd-- = '\0';
if (!Util::EmptyStr(szPassword))
{
if (IsStopped() && m_bAutoTerminated)
{
ScriptController::Resume();
Thread::Resume();
}
m_bUnpackDecryptError = false;
m_bUnpackPasswordError = false;
m_bAutoTerminated = false;
PrintMessage(Message::mkInfo, "Trying password %s for %s", szPassword, m_szName);
ExecuteUnpack(eUnpacker, szPassword, bMultiVolumes);
}
}
fclose(infile);
m_bPassListTried = !IsStopped() || m_bAutoTerminated;
}
}
void UnpackController::ExecuteUnpack(EUnpacker eUnpacker, const char* szPassword, bool bMultiVolumes)
{
switch (eUnpacker)
{
case upUnrar:
ExecuteUnrar(szPassword);
break;
case upSevenZip:
ExecuteSevenZip(szPassword, bMultiVolumes);
break;
}
}
void UnpackController::ExecuteUnrar(const char* szPassword)
void UnpackController::ExecuteUnrar()
{
// Format:
// unrar x -y -p- -o+ *.rar ./_unpack/
// unrar x -y -p- -o+ *.rar ./_unpack
ParamList params;
if (!PrepareCmdParams(g_pOptions->GetUnrarCmd(), &params, "unrar"))
char szPasswordParam[1024];
const char* szArgs[8];
szArgs[0] = g_pOptions->GetUnrarCmd();
szArgs[1] = "x";
szArgs[2] = "-y";
szArgs[3] = "-p-";
if (strlen(m_szPassword) > 0)
{
return;
snprintf(szPasswordParam, 1024, "-p%s", m_szPassword);
szArgs[3] = szPasswordParam;
}
szArgs[4] = "-o+";
szArgs[5] = m_bHasNonStdRarFiles ? "*.*" : "*.rar";
szArgs[6] = m_szUnpackDir;
szArgs[7] = NULL;
SetArgs(szArgs, false);
if (!params.Exists("x") && !params.Exists("e"))
{
params.push_back(strdup("x"));
}
params.push_back(strdup("-y"));
if (!Util::EmptyStr(szPassword))
{
char szPasswordParam[1024];
snprintf(szPasswordParam, 1024, "-p%s", szPassword);
szPasswordParam[1024-1] = '\0';
params.push_back(strdup(szPasswordParam));
}
else
{
params.push_back(strdup("-p-"));
}
if (!params.Exists("-o+") && !params.Exists("-o-"))
{
params.push_back(strdup("-o+"));
}
params.push_back(strdup(m_bHasNonStdRarFiles ? "*.*" : "*.rar"));
char szUnpackDirParam[1024];
snprintf(szUnpackDirParam, 1024, "%s%c", m_szUnpackDir, PATH_SEPARATOR);
szUnpackDirParam[1024-1] = '\0';
params.push_back(strdup(szUnpackDirParam));
params.push_back(NULL);
SetArgs((const char**)&params.front(), false);
SetScript(params.at(0));
SetScript(g_pOptions->GetUnrarCmd());
SetLogPrefix("Unrar");
ResetEnv();
m_bAllOKMessageReceived = false;
m_eUnpacker = upUnrar;
@@ -358,7 +215,7 @@ void UnpackController::ExecuteUnrar(const char* szPassword)
m_bUnpackOK = iExitCode == 0 && m_bAllOKMessageReceived && !GetTerminated();
m_bUnpackStartError = iExitCode == -1;
m_bUnpackSpaceError = iExitCode == 5;
m_bUnpackPasswordError |= iExitCode == 11; // only for rar5-archives
m_bUnpackPasswordError = iExitCode == 11; // only for rar5-archives
if (!m_bUnpackOK && iExitCode > 0)
{
@@ -366,49 +223,35 @@ void UnpackController::ExecuteUnrar(const char* szPassword)
}
}
void UnpackController::ExecuteSevenZip(const char* szPassword, bool bMultiVolumes)
void UnpackController::ExecuteSevenZip(bool bMultiVolumes)
{
// Format:
// 7z x -y -p- -o./_unpack *.7z
// OR
// 7z x -y -p- -o./_unpack *.7z.001
ParamList params;
if (!PrepareCmdParams(g_pOptions->GetSevenZipCmd(), &params, "7-Zip"))
{
return;
}
char szPasswordParam[1024];
const char* szArgs[7];
szArgs[0] = g_pOptions->GetSevenZipCmd();
szArgs[1] = "x";
szArgs[2] = "-y";
if (!params.Exists("x") && !params.Exists("e"))
szArgs[3] = "-p-";
if (strlen(m_szPassword) > 0)
{
params.push_back(strdup("x"));
}
params.push_back(strdup("-y"));
if (!Util::EmptyStr(szPassword))
{
char szPasswordParam[1024];
snprintf(szPasswordParam, 1024, "-p%s", szPassword);
szPasswordParam[1024-1] = '\0';
params.push_back(strdup(szPasswordParam));
}
else
{
params.push_back(strdup("-p-"));
snprintf(szPasswordParam, 1024, "-p%s", m_szPassword);
szArgs[3] = szPasswordParam;
}
char szUnpackDirParam[1024];
snprintf(szUnpackDirParam, 1024, "-o%s", m_szUnpackDir);
szUnpackDirParam[1024-1] = '\0';
params.push_back(strdup(szUnpackDirParam));
szArgs[4] = szUnpackDirParam;
params.push_back(strdup(bMultiVolumes ? "*.7z.001" : "*.7z"));
szArgs[5] = bMultiVolumes ? "*.7z.001" : "*.7z";
szArgs[6] = NULL;
SetArgs(szArgs, false);
params.push_back(NULL);
SetArgs((const char**)&params.front(), false);
SetScript(params.at(0));
ResetEnv();
SetScript(g_pOptions->GetSevenZipCmd());
m_bAllOKMessageReceived = false;
m_eUnpacker = upSevenZip;
@@ -429,205 +272,6 @@ void UnpackController::ExecuteSevenZip(const char* szPassword, bool bMultiVolume
}
}
bool UnpackController::PrepareCmdParams(const char* szCommand, ParamList* pParams, const char* szInfoName)
{
if (Util::FileExists(szCommand))
{
pParams->push_back(strdup(szCommand));
return true;
}
char** pCmdArgs = NULL;
if (!Util::SplitCommandLine(szCommand, &pCmdArgs))
{
PrintMessage(Message::mkError, "Could not start %s, failed to parse command line: %s", szInfoName, szCommand);
m_bUnpackOK = false;
m_bUnpackStartError = true;
return false;
}
for (char** szArgPtr = pCmdArgs; *szArgPtr; szArgPtr++)
{
pParams->push_back(*szArgPtr);
}
free(pCmdArgs);
return true;
}
void UnpackController::JoinSplittedFiles()
{
SetLogPrefix("Join");
SetProgressLabel("");
m_pPostInfo->SetStageProgress(0);
// determine groups
FileList groups;
RegEx regExSplitExt(".*\\.[a-z,0-9]{3}\\.001$");
DirBrowser dir(m_szDestDir);
while (const char* filename = dir.Next())
{
char szFullFilename[1024];
snprintf(szFullFilename, 1024, "%s%c%s", m_szDestDir, PATH_SEPARATOR, filename);
szFullFilename[1024-1] = '\0';
if (strcmp(filename, ".") && strcmp(filename, "..") && !Util::DirectoryExists(szFullFilename))
{
if (regExSplitExt.Match(filename) && !FileHasRarSignature(szFullFilename))
{
if (!JoinFile(filename))
{
m_bUnpackOK = false;
break;
}
}
}
}
SetLogPrefix(NULL);
SetProgressLabel("");
}
bool UnpackController::JoinFile(const char* szFragBaseName)
{
char szDestBaseName[1024];
strncpy(szDestBaseName, szFragBaseName, 1024);
szDestBaseName[1024-1] = '\0';
// trim extension
char* szExtension = strrchr(szDestBaseName, '.');
*szExtension = '\0';
char szFullFilename[1024];
snprintf(szFullFilename, 1024, "%s%c%s", m_szDestDir, PATH_SEPARATOR, szFragBaseName);
szFullFilename[1024-1] = '\0';
long long lFirstSegmentSize = Util::FileSize(szFullFilename);
long long lDifSegmentSize = 0;
// Validate joinable file:
// - fragments have continuous numbers (no holes);
// - fragments have the same size (except of the last fragment);
// - the last fragment must be smaller than other fragments,
// if it has the same size it is probably not the last and there are missing fragments.
RegEx regExSplitExt(".*\\.[a-z,0-9]{3}\\.[0-9]{3}$");
int iCount = 0;
int iMin = -1;
int iMax = -1;
int iDifSizeCount = 0;
int iDifSizeMin = 999999;
DirBrowser dir(m_szDestDir);
while (const char* filename = dir.Next())
{
snprintf(szFullFilename, 1024, "%s%c%s", m_szDestDir, PATH_SEPARATOR, filename);
szFullFilename[1024-1] = '\0';
if (strcmp(filename, ".") && strcmp(filename, "..") && !Util::DirectoryExists(szFullFilename) &&
regExSplitExt.Match(filename))
{
const char* szSegExt = strrchr(filename, '.');
int iSegNum = atoi(szSegExt + 1);
iCount++;
iMin = iSegNum < iMin || iMin == -1 ? iSegNum : iMin;
iMax = iSegNum > iMax ? iSegNum : iMax;
long long lSegmentSize = Util::FileSize(szFullFilename);
if (lSegmentSize != lFirstSegmentSize)
{
iDifSizeCount++;
iDifSizeMin = iSegNum < iDifSizeMin ? iSegNum : iDifSizeMin;
lDifSegmentSize = lSegmentSize;
}
}
}
int iCorrectedCount = iCount - (iMin == 0 ? 1 : 0);
if ((iMin > 1) || iCorrectedCount != iMax ||
((iDifSizeMin != iCorrectedCount || iDifSizeMin > iMax) &&
m_pPostInfo->GetNZBInfo()->GetParStatus() != NZBInfo::psSuccess))
{
PrintMessage(Message::mkWarning, "Could not join splitted file %s: missing fragments detected", szDestBaseName);
return false;
}
// Now can join
PrintMessage(Message::mkInfo, "Joining splitted file %s", szDestBaseName);
m_pPostInfo->SetStageProgress(0);
char szErrBuf[256];
char szDestFilename[1024];
snprintf(szDestFilename, 1024, "%s%c%s", m_szUnpackDir, PATH_SEPARATOR, szDestBaseName);
szDestFilename[1024-1] = '\0';
FILE* pOutFile = fopen(szDestFilename, FOPEN_WBP);
if (!pOutFile)
{
PrintMessage(Message::mkError, "Could not create file %s: %s", szDestFilename, Util::GetLastErrorMessage(szErrBuf, sizeof(szErrBuf)));
return false;
}
if (g_pOptions->GetWriteBuffer() > 0)
{
setvbuf(pOutFile, NULL, _IOFBF, g_pOptions->GetWriteBuffer() * 1024);
}
long long lTotalSize = lFirstSegmentSize * (iCount - 1) + lDifSegmentSize;
long long lWritten = 0;
static const int BUFFER_SIZE = 1024 * 50;
char* buffer = (char*)malloc(BUFFER_SIZE);
bool bOK = true;
for (int i = iMin; i <= iMax; i++)
{
PrintMessage(Message::mkInfo, "Joining from %s.%.3i", szDestBaseName, i);
char szMessage[1024];
snprintf(szMessage, 1024, "Joining from %s.%.3i", szDestBaseName, i);
szMessage[1024-1] = '\0';
SetProgressLabel(szMessage);
char szFragFilename[1024];
snprintf(szFragFilename, 1024, "%s%c%s.%.3i", m_szDestDir, PATH_SEPARATOR, szDestBaseName, i);
szFragFilename[1024-1] = '\0';
if (!Util::FileExists(szFragFilename))
{
break;
}
FILE* pInFile = fopen(szFragFilename, FOPEN_RB);
if (pInFile)
{
int cnt = BUFFER_SIZE;
while (cnt == BUFFER_SIZE)
{
cnt = (int)fread(buffer, 1, BUFFER_SIZE, pInFile);
fwrite(buffer, 1, cnt, pOutFile);
lWritten += cnt;
m_pPostInfo->SetStageProgress(int(lWritten * 1000 / lTotalSize));
}
fclose(pInFile);
char szFragFilename[1024];
snprintf(szFragFilename, 1024, "%s.%.3i", szDestBaseName, i);
szFragFilename[1024-1] = '\0';
m_JoinedFiles.push_back(strdup(szFragFilename));
}
else
{
PrintMessage(Message::mkError, "Could not open file %s", szFragFilename);
bOK = false;
break;
}
}
fclose(pOutFile);
free(buffer);
return bOK;
}
void UnpackController::Completed()
{
bool bCleanupSuccess = Cleanup();
@@ -647,16 +291,11 @@ void UnpackController::Completed()
else
{
#ifndef DISABLE_PARCHECK
if (!m_bUnpackOK &&
(m_pPostInfo->GetNZBInfo()->GetParStatus() <= NZBInfo::psSkipped ||
!m_pPostInfo->GetNZBInfo()->GetParFull()) &&
if (!m_bUnpackOK && m_pPostInfo->GetNZBInfo()->GetParStatus() <= NZBInfo::psSkipped &&
!m_bUnpackStartError && !m_bUnpackSpaceError && !m_bUnpackPasswordError &&
(!GetTerminated() || m_bAutoTerminated) && m_bHasParFiles)
!GetTerminated() && m_bHasParFiles)
{
RequestParCheck(!Util::EmptyStr(m_szPassword) ||
Util::EmptyStr(g_pOptions->GetUnpackPassFile()) || m_bPassListTried ||
!(m_bUnpackDecryptError || m_bUnpackPasswordError) ||
m_pPostInfo->GetNZBInfo()->GetParStatus() > NZBInfo::psSkipped);
RequestParCheck();
}
else
#endif
@@ -664,7 +303,7 @@ void UnpackController::Completed()
PrintMessage(Message::mkError, "%s failed", m_szInfoNameUp);
m_pPostInfo->GetNZBInfo()->SetUnpackStatus(
m_bUnpackSpaceError ? NZBInfo::usSpace :
m_bUnpackPasswordError || m_bUnpackDecryptError ? NZBInfo::usPassword :
m_bUnpackPasswordError ? NZBInfo::usPassword :
NZBInfo::usFailure);
m_pPostInfo->SetStage(PostInfo::ptQueued);
}
@@ -672,17 +311,11 @@ void UnpackController::Completed()
}
#ifndef DISABLE_PARCHECK
void UnpackController::RequestParCheck(bool bForceRepair)
void UnpackController::RequestParCheck()
{
PrintMessage(Message::mkInfo, "%s requested %s", m_szInfoNameUp, bForceRepair ? "par-check with forced repair" : "par-check/repair");
PrintMessage(Message::mkInfo, "%s requested par-check/repair", m_szInfoNameUp);
m_pPostInfo->SetRequestParCheck(true);
m_pPostInfo->SetForceRepair(bForceRepair);
m_pPostInfo->SetStage(PostInfo::ptFinished);
m_pPostInfo->SetUnpackTried(true);
m_pPostInfo->SetPassListTried(m_bPassListTried);
m_pPostInfo->SetLastUnpackStatus((int)(m_bUnpackSpaceError ? NZBInfo::usSpace :
m_bUnpackPasswordError || m_bUnpackDecryptError ? NZBInfo::usPassword :
NZBInfo::usFailure));
}
#endif
@@ -706,7 +339,7 @@ void UnpackController::CreateUnpackDir()
char szErrBuf[1024];
if (!Util::ForceDirectories(m_szUnpackDir, szErrBuf, sizeof(szErrBuf)))
{
PrintMessage(Message::mkError, "Could not create directory %s: %s", m_szUnpackDir, szErrBuf);
error("Could not create directory %s: %s", m_szUnpackDir, szErrBuf);
}
}
@@ -717,14 +350,12 @@ void UnpackController::CheckArchiveFiles(bool bScanNonStdFiles)
m_bHasNonStdRarFiles = false;
m_bHasSevenZipFiles = false;
m_bHasSevenZipMultiFiles = false;
m_bHasSplittedFiles = false;
RegEx regExRar(".*\\.rar$");
RegEx regExRarMultiSeq(".*\\.(r|s)[0-9][0-9]$");
RegEx regExSevenZip(".*\\.7z$");
RegEx regExSevenZipMulti(".*\\.7z\\.[0-9]+$");
RegEx regExNumExt(".*\\.[0-9]+$");
RegEx regExSplitExt(".*\\.[a-z,0-9]{3}\\.[0-9]{3}$");
DirBrowser dir(m_szDestDir);
while (const char* filename = dir.Next())
@@ -735,9 +366,6 @@ void UnpackController::CheckArchiveFiles(bool bScanNonStdFiles)
if (strcmp(filename, ".") && strcmp(filename, "..") && !Util::DirectoryExists(szFullFilename))
{
const char* szExt = strchr(filename, '.');
int iExtNum = szExt ? atoi(szExt + 1) : -1;
if (regExRar.Match(filename))
{
m_bHasRarFiles = true;
@@ -750,16 +378,12 @@ void UnpackController::CheckArchiveFiles(bool bScanNonStdFiles)
{
m_bHasSevenZipMultiFiles = true;
}
else if (bScanNonStdFiles && !m_bHasNonStdRarFiles && iExtNum > 1 &&
else if (bScanNonStdFiles && !m_bHasNonStdRarFiles &&
!regExRarMultiSeq.Match(filename) && regExNumExt.Match(filename) &&
FileHasRarSignature(szFullFilename))
{
m_bHasNonStdRarFiles = true;
}
else if (regExSplitExt.Match(filename) && (iExtNum == 0 || iExtNum == 1))
{
m_bHasSplittedFiles = true;
}
}
}
}
@@ -804,7 +428,8 @@ bool UnpackController::Cleanup()
DirBrowser dir(m_szUnpackDir);
while (const char* filename = dir.Next())
{
if (strcmp(filename, ".") && strcmp(filename, ".."))
if (strcmp(filename, ".") && strcmp(filename, "..") &&
strcmp(filename, ".AppleDouble") && strcmp(filename, ".DS_Store"))
{
char szSrcFile[1024];
snprintf(szSrcFile, 1024, "%s%c%s", m_szUnpackDir, PATH_SEPARATOR, filename);
@@ -817,13 +442,10 @@ bool UnpackController::Cleanup()
// silently overwrite existing files
remove(szDstFile);
bool bHiddenFile = filename[0] == '.';
if (!Util::MoveFile(szSrcFile, szDstFile) && !bHiddenFile)
if (!Util::MoveFile(szSrcFile, szDstFile))
{
char szErrBuf[256];
PrintMessage(Message::mkError, "Could not move file %s to %s: %s", szSrcFile, szDstFile,
Util::GetLastErrorMessage(szErrBuf, sizeof(szErrBuf)));
PrintMessage(Message::mkError, "Could not move file %s to %s: %s", szSrcFile, szDstFile, Util::GetLastErrorMessage(szErrBuf, sizeof(szErrBuf)));
bOK = false;
}
@@ -851,7 +473,6 @@ bool UnpackController::Cleanup()
RegEx regExRarMultiSeq(".*\\.[r-z][0-9][0-9]$");
RegEx regExSevenZip(".*\\.7z$|.*\\.7z\\.[0-9]+$");
RegEx regExNumExt(".*\\.[0-9]+$");
RegEx regExSplitExt(".*\\.[a-z,0-9]{3}\\.[0-9]{3}$");
DirBrowser dir(m_szDestDir);
while (const char* filename = dir.Next())
@@ -864,9 +485,8 @@ bool UnpackController::Cleanup()
!Util::DirectoryExists(szFullFilename) &&
(m_bInterDir || !extractedFiles.Exists(filename)) &&
(regExRar.Match(filename) || regExSevenZip.Match(filename) ||
(regExRarMultiSeq.Match(filename) && FileHasRarSignature(szFullFilename)) ||
(m_bHasNonStdRarFiles && regExNumExt.Match(filename) && FileHasRarSignature(szFullFilename)) ||
(m_bHasSplittedFiles && regExSplitExt.Match(filename) && m_JoinedFiles.Exists(filename))))
(regExRarMultiSeq.Match(filename) && FileHasRarSignature(szFullFilename)) ||
(m_bHasNonStdRarFiles && regExNumExt.Match(filename) && FileHasRarSignature(szFullFilename))))
{
PrintMessage(Message::mkInfo, "Deleting file %s", filename);
@@ -954,7 +574,6 @@ void UnpackController::AddMessage(Message::EKind eKind, const char* szText)
char szMsgText[1024];
strncpy(szMsgText, szText, 1024);
szMsgText[1024-1] = '\0';
int iLen = strlen(szText);
// Modify unrar messages for better readability:
// remove the destination path part from message "Extracting file.xxx"
@@ -965,7 +584,8 @@ void UnpackController::AddMessage(Message::EKind eKind, const char* szText)
szMsgText[1024-1] = '\0';
}
m_pPostInfo->GetNZBInfo()->AddMessage(eKind, szMsgText);
ScriptController::AddMessage(eKind, szMsgText);
m_pPostInfo->AppendMessage(eKind, szMsgText);
if (m_eUnpacker == upUnrar && !strncmp(szMsgText, "Unrar: UNRAR ", 6) &&
strstr(szMsgText, " Copyright ") && strstr(szMsgText, " Alexander Roshal"))
@@ -987,39 +607,6 @@ void UnpackController::AddMessage(Message::EKind eKind, const char* szText)
SetProgressLabel(szText + 7);
}
if (m_eUnpacker == upUnrar &&
(!strncmp(szText, "Unrar: Checksum error in the encrypted file", 42) ||
!strncmp(szText, "Unrar: CRC failed in the encrypted file", 39)))
{
m_bUnpackDecryptError = true;
}
if (m_eUnpacker == upUnrar && !strncmp(szText, "Unrar: The specified password is incorrect.'", 43))
{
m_bUnpackPasswordError = true;
}
if (m_eUnpacker == upSevenZip &&
(iLen > 18 && !strncmp(szText + iLen - 45, "Data Error in encrypted file. Wrong password?", 45)))
{
m_bUnpackDecryptError = true;
}
if (!IsStopped() && (m_bUnpackDecryptError || m_bUnpackPasswordError ||
strstr(szText, " : packed data CRC failed in volume") ||
strstr(szText, " : packed data checksum error in volume") ||
(iLen > 13 && !strncmp(szText + iLen - 13, " - CRC failed", 13)) ||
(iLen > 18 && !strncmp(szText + iLen - 18, " - checksum failed", 18)) ||
!strncmp(szText, "Unrar: WARNING: You need to start extraction from a previous volume", 67)))
{
char szMsgText[1024];
snprintf(szMsgText, 1024, "Cancelling %s due to errors", m_szInfoName);
szMsgText[1024-1] = '\0';
m_pPostInfo->GetNZBInfo()->AddMessage(Message::mkWarning, szMsgText);
m_bAutoTerminated = true;
Stop();
}
if ((m_eUnpacker == upUnrar && !strncmp(szText, "Unrar: All OK", 13)) ||
(m_eUnpacker == upSevenZip && !strncmp(szText, "7-Zip: Everything is Ok", 23)))
{
@@ -1040,3 +627,211 @@ void UnpackController::SetProgressLabel(const char* szProgressLabel)
m_pPostInfo->SetProgressLabel(szProgressLabel);
DownloadQueue::Unlock();
}
void MoveController::StartJob(PostInfo* pPostInfo)
{
MoveController* pMoveController = new MoveController();
pMoveController->m_pPostInfo = pPostInfo;
pMoveController->SetAutoDestroy(false);
pPostInfo->SetPostThread(pMoveController);
pMoveController->Start();
}
void MoveController::Run()
{
// the locking is needed for accessing the members of NZBInfo
DownloadQueue::Lock();
char szNZBName[1024];
strncpy(szNZBName, m_pPostInfo->GetNZBInfo()->GetName(), 1024);
szNZBName[1024-1] = '\0';
char szInfoName[1024];
snprintf(szInfoName, 1024, "move for %s", m_pPostInfo->GetNZBInfo()->GetName());
szInfoName[1024-1] = '\0';
SetInfoName(szInfoName);
strncpy(m_szInterDir, m_pPostInfo->GetNZBInfo()->GetDestDir(), 1024);
m_szInterDir[1024-1] = '\0';
m_pPostInfo->GetNZBInfo()->BuildFinalDirName(m_szDestDir, 1024);
m_szDestDir[1024-1] = '\0';
DownloadQueue::Unlock();
info("Moving completed files for %s", szNZBName);
bool bOK = MoveFiles();
szInfoName[0] = 'M'; // uppercase
if (bOK)
{
info("%s successful", szInfoName);
// save new dest dir
DownloadQueue::Lock();
m_pPostInfo->GetNZBInfo()->SetDestDir(m_szDestDir);
m_pPostInfo->GetNZBInfo()->SetMoveStatus(NZBInfo::msSuccess);
DownloadQueue::Unlock();
}
else
{
error("%s failed", szInfoName);
m_pPostInfo->GetNZBInfo()->SetMoveStatus(NZBInfo::msFailure);
}
m_pPostInfo->SetStage(PostInfo::ptQueued);
m_pPostInfo->SetWorking(false);
}
bool MoveController::MoveFiles()
{
char szErrBuf[1024];
if (!Util::ForceDirectories(m_szDestDir, szErrBuf, sizeof(szErrBuf)))
{
error("Could not create directory %s: %s", m_szDestDir, szErrBuf);
return false;
}
bool bOK = true;
DirBrowser dir(m_szInterDir);
while (const char* filename = dir.Next())
{
if (strcmp(filename, ".") && strcmp(filename, "..") &&
strcmp(filename, ".AppleDouble") && strcmp(filename, ".DS_Store"))
{
char szSrcFile[1024];
snprintf(szSrcFile, 1024, "%s%c%s", m_szInterDir, PATH_SEPARATOR, filename);
szSrcFile[1024-1] = '\0';
char szDstFile[1024];
Util::MakeUniqueFilename(szDstFile, 1024, m_szDestDir, filename);
PrintMessage(Message::mkInfo, "Moving file %s to %s", Util::BaseFileName(szSrcFile), m_szDestDir);
if (!Util::MoveFile(szSrcFile, szDstFile))
{
PrintMessage(Message::mkError, "Could not move file %s to %s: %s", szSrcFile, szDstFile, Util::GetLastErrorMessage(szErrBuf, sizeof(szErrBuf)));
bOK = false;
}
}
}
if (bOK && !Util::DeleteDirectoryWithContent(m_szInterDir, szErrBuf, sizeof(szErrBuf)))
{
PrintMessage(Message::mkError, "Could not delete intermediate directory %s: %s", m_szInterDir, szErrBuf);
}
return bOK;
}
void CleanupController::StartJob(PostInfo* pPostInfo)
{
CleanupController* pCleanupController = new CleanupController();
pCleanupController->m_pPostInfo = pPostInfo;
pCleanupController->SetAutoDestroy(false);
pPostInfo->SetPostThread(pCleanupController);
pCleanupController->Start();
}
void CleanupController::Run()
{
// the locking is needed for accessing the members of NZBInfo
DownloadQueue::Lock();
char szNZBName[1024];
strncpy(szNZBName, m_pPostInfo->GetNZBInfo()->GetName(), 1024);
szNZBName[1024-1] = '\0';
char szInfoName[1024];
snprintf(szInfoName, 1024, "cleanup for %s", m_pPostInfo->GetNZBInfo()->GetName());
szInfoName[1024-1] = '\0';
SetInfoName(szInfoName);
strncpy(m_szDestDir, m_pPostInfo->GetNZBInfo()->GetDestDir(), 1024);
m_szDestDir[1024-1] = '\0';
bool bInterDir = strlen(g_pOptions->GetInterDir()) > 0 &&
!strncmp(m_szDestDir, g_pOptions->GetInterDir(), strlen(g_pOptions->GetInterDir()));
if (bInterDir)
{
m_pPostInfo->GetNZBInfo()->BuildFinalDirName(m_szFinalDir, 1024);
m_szFinalDir[1024-1] = '\0';
}
else
{
m_szFinalDir[0] = '\0';
}
DownloadQueue::Unlock();
info("Cleaning up %s", szNZBName);
bool bDeleted = false;
bool bOK = Cleanup(m_szDestDir, &bDeleted);
if (bOK && m_szFinalDir[0] != '\0')
{
bool bDeleted2 = false;
bOK = Cleanup(m_szFinalDir, &bDeleted2);
bDeleted = bDeleted || bDeleted2;
}
szInfoName[0] = 'C'; // uppercase
if (bOK && bDeleted)
{
info("%s successful", szInfoName);
m_pPostInfo->GetNZBInfo()->SetCleanupStatus(NZBInfo::csSuccess);
}
else if (bOK)
{
info("Nothing to cleanup for %s", szNZBName);
m_pPostInfo->GetNZBInfo()->SetCleanupStatus(NZBInfo::csSuccess);
}
else
{
error("%s failed", szInfoName);
m_pPostInfo->GetNZBInfo()->SetCleanupStatus(NZBInfo::csFailure);
}
m_pPostInfo->SetStage(PostInfo::ptQueued);
m_pPostInfo->SetWorking(false);
}
bool CleanupController::Cleanup(const char* szDestDir, bool *bDeleted)
{
*bDeleted = false;
bool bOK = true;
DirBrowser dir(szDestDir);
while (const char* filename = dir.Next())
{
// check file extension
bool bDeleteIt = Util::MatchFileExt(filename, g_pOptions->GetExtCleanupDisk(), ",;");
if (bDeleteIt)
{
char szFullFilename[1024];
snprintf(szFullFilename, 1024, "%s%c%s", szDestDir, PATH_SEPARATOR, filename);
szFullFilename[1024-1] = '\0';
PrintMessage(Message::mkInfo, "Deleting file %s", filename);
if (remove(szFullFilename) != 0)
{
char szErrBuf[256];
PrintMessage(Message::mkError, "Could not delete file %s: %s", szFullFilename, Util::GetLastErrorMessage(szErrBuf, sizeof(szErrBuf)));
bOK = false;
}
*bDeleted = true;
}
}
return bOK;
}

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2013-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2013 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -27,7 +27,6 @@
#define UNPACK_H
#include <deque>
#include <vector>
#include "Log.h"
#include "Thread.h"
@@ -51,14 +50,6 @@ private:
bool Exists(const char* szFilename);
};
typedef std::vector<char*> ParamListBase;
class ParamList : public ParamListBase
{
public:
~ParamList();
bool Exists(const char* szParam);
};
private:
PostInfo* m_pPostInfo;
char m_szName[1024];
@@ -76,38 +67,28 @@ private:
bool m_bHasNonStdRarFiles;
bool m_bHasSevenZipFiles;
bool m_bHasSevenZipMultiFiles;
bool m_bHasSplittedFiles;
bool m_bUnpackOK;
bool m_bUnpackStartError;
bool m_bUnpackSpaceError;
bool m_bUnpackDecryptError;
bool m_bUnpackPasswordError;
bool m_bCleanedUpDisk;
bool m_bAutoTerminated;
EUnpacker m_eUnpacker;
bool m_bFinalDirCreated;
FileList m_JoinedFiles;
bool m_bPassListTried;
protected:
virtual bool ReadLine(char* szBuf, int iBufSize, FILE* pStream);
virtual void AddMessage(Message::EKind eKind, const char* szText);
void ExecuteUnpack(EUnpacker eUnpacker, const char* szPassword, bool bMultiVolumes);
void ExecuteUnrar(const char* szPassword);
void ExecuteSevenZip(const char* szPassword, bool bMultiVolumes);
void UnpackArchives(EUnpacker eUnpacker, bool bMultiVolumes);
void JoinSplittedFiles();
bool JoinFile(const char* szFragBaseName);
void ExecuteUnrar();
void ExecuteSevenZip(bool bMultiVolumes);
void Completed();
void CreateUnpackDir();
bool Cleanup();
void CheckArchiveFiles(bool bScanNonStdFiles);
void SetProgressLabel(const char* szProgressLabel);
#ifndef DISABLE_PARCHECK
void RequestParCheck(bool bForceRepair);
void RequestParCheck();
#endif
bool FileHasRarSignature(const char* szFilename);
bool PrepareCmdParams(const char* szCommand, ParamList* pParams, const char* szInfoName);
public:
virtual void Run();
@@ -115,4 +96,32 @@ public:
static void StartJob(PostInfo* pPostInfo);
};
class MoveController : public Thread, public ScriptController
{
private:
PostInfo* m_pPostInfo;
char m_szInterDir[1024];
char m_szDestDir[1024];
bool MoveFiles();
public:
virtual void Run();
static void StartJob(PostInfo* pPostInfo);
};
class CleanupController : public Thread, public ScriptController
{
private:
PostInfo* m_pPostInfo;
char m_szDestDir[1024];
char m_szFinalDir[1024];
bool Cleanup(const char* szDestDir, bool *bDeleted);
public:
virtual void Run();
static void StartJob(PostInfo* pPostInfo);
};
#endif

View File

File diff suppressed because it is too large Load Diff

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -30,12 +30,12 @@
#include "FeedInfo.h"
#include "NewsServer.h"
#include "StatMeter.h"
#include "Log.h"
class DiskState
{
private:
int fscanf(FILE* infile, const char* Format, ...);
int ParseFormatVersion(const char* szFormatSignature);
bool SaveFileInfo(FileInfo* pFileInfo, const char* szFilename);
bool LoadFileInfo(FileInfo* pFileInfo, const char* szFilename, bool bFileSummary, bool bArticles);
void SaveNZBQueue(DownloadQueue* pDownloadQueue, FILE* outfile);
@@ -58,10 +58,10 @@ private:
bool LoadVolumeStat(Servers* pServers, ServerVolumes* pServerVolumes, FILE* infile, int iFormatVersion);
void CalcFileStats(DownloadQueue* pDownloadQueue, int iFormatVersion);
void CalcNZBFileStats(NZBInfo* pNZBInfo, int iFormatVersion);
bool LoadFileState(FileInfo* pFileInfo, Servers* pServers);
bool LoadAllFileStates(DownloadQueue* pDownloadQueue, Servers* pServers);
void SaveServerStats(ServerStatList* pServerStatList, FILE* outfile);
bool LoadServerStats(ServerStatList* pServerStatList, Servers* pServers, FILE* infile);
bool FinishWriteTransaction(const char* szNewFileName, const char* szDestFileName);
// backward compatibility functions (conversions from older formats)
bool LoadPostQueue12(DownloadQueue* pDownloadQueue, NZBList* pNZBList, FILE* infile, int iFormatVersion);
@@ -80,23 +80,15 @@ public:
bool SaveDownloadQueue(DownloadQueue* pDownloadQueue);
bool LoadDownloadQueue(DownloadQueue* pDownloadQueue, Servers* pServers);
bool SaveFile(FileInfo* pFileInfo);
bool SaveFileState(FileInfo* pFileInfo, bool bCompleted);
bool LoadFileState(FileInfo* pFileInfo, Servers* pServers, bool bCompleted);
bool SaveFileState(FileInfo* pFileInfo);
bool LoadArticles(FileInfo* pFileInfo);
void DiscardDownloadQueue();
void DiscardFile(FileInfo* pFileInfo, bool bDeleteData, bool bDeletePartialState, bool bDeleteCompletedState);
void DiscardFiles(NZBInfo* pNZBInfo);
bool DiscardFile(FileInfo* pFileInfo);
bool SaveFeeds(Feeds* pFeeds, FeedHistory* pFeedHistory);
bool LoadFeeds(Feeds* pFeeds, FeedHistory* pFeedHistory);
bool SaveStats(Servers* pServers, ServerVolumes* pServerVolumes);
bool LoadStats(Servers* pServers, ServerVolumes* pServerVolumes, bool* pPerfectMatch);
void CleanupTempDir(DownloadQueue* pDownloadQueue);
void WriteCacheFlag();
void DeleteCacheFlag();
void AppendNZBMessage(int iNZBID, Message::EKind eKind, const char* szText);
void LoadNZBMessages(int iNZBID, MessageList* pMessages);
};
extern DiskState* g_pDiskState;
#endif

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -35,18 +35,17 @@
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <stdarg.h>
#include <ctype.h>
#include <sys/stat.h>
#include <algorithm>
#include "nzbget.h"
#include "DownloadInfo.h"
#include "ArticleWriter.h"
#include "DiskState.h"
#include "Options.h"
#include "Util.h"
extern Options* g_pOptions;
int FileInfo::m_iIDGen = 0;
int FileInfo::m_iIDMax = 0;
int NZBInfo::m_iIDGen = 0;
@@ -54,6 +53,7 @@ int NZBInfo::m_iIDMax = 0;
DownloadQueue* DownloadQueue::g_pDownloadQueue = NULL;
bool DownloadQueue::g_bLoaded = false;
NZBParameter::NZBParameter(const char* szName)
{
m_szName = strdup(szName);
@@ -303,7 +303,6 @@ NZBInfo::NZBInfo() : m_FileList(true)
m_eDeleteStatus = dsNone;
m_eMarkStatus = ksNone;
m_eUrlStatus = lsNone;
m_iExtraParBlocks = 0;
m_bAddUrlPaused = false;
m_bDeleting = false;
m_bDeletePaused = false;
@@ -331,19 +330,6 @@ NZBInfo::NZBInfo() : m_FileList(true)
m_pPostInfo = NULL;
m_iIDMessageGen = 0;
m_iID = ++m_iIDGen;
m_lDownloadedSize = 0;
m_iDownloadSec = 0;
m_iPostTotalSec = 0;
m_iParSec = 0;
m_iRepairSec = 0;
m_iUnpackSec = 0;
m_tDownloadStartTime = 0;
m_bReprocess = false;
m_tQueueScriptTime = 0;
m_bParFull = false;
m_iMessageCount = 0;
m_iCachedMessageCount = 0;
m_iFeedID = 0;
}
NZBInfo::~NZBInfo()
@@ -362,6 +348,12 @@ NZBInfo::~NZBInfo()
ClearCompletedFiles();
for (Messages::iterator it = m_Messages.begin(); it != m_Messages.end(); it++)
{
delete *it;
}
m_Messages.clear();
m_FileList.Clear();
}
@@ -394,9 +386,9 @@ int NZBInfo::GenerateID()
void NZBInfo::ClearCompletedFiles()
{
for (CompletedFiles::iterator it = m_completedFiles.begin(); it != m_completedFiles.end(); it++)
for (Files::iterator it = m_completedFiles.begin(); it != m_completedFiles.end(); it++)
{
delete *it;
free(*it);
}
m_completedFiles.clear();
}
@@ -584,8 +576,9 @@ int NZBInfo::CalcHealth()
return 1000;
}
int iHealth = (int)((m_lSize - m_lParSize -
(m_lCurrentFailedSize - m_lParCurrentFailedSize)) * 1000 / (m_lSize - m_lParSize));
int iHealth = (int)(Util::Int64ToFloat(m_lSize - m_lParSize -
(m_lCurrentFailedSize - m_lParCurrentFailedSize)) * 1000.0 /
Util::Int64ToFloat(m_lSize - m_lParSize));
if (iHealth == 1000 && m_lCurrentFailedSize - m_lParCurrentFailedSize > 0)
{
@@ -602,13 +595,9 @@ int NZBInfo::CalcCriticalHealth(bool bAllowEstimation)
return 1000;
}
if (m_lSize == m_lParSize)
{
return 0;
}
long long lGoodParSize = m_lParSize - m_lParCurrentFailedSize;
int iCriticalHealth = (int)((m_lSize - lGoodParSize*2) * 1000 / (m_lSize - lGoodParSize));
int iCriticalHealth = (int)(Util::Int64ToFloat(m_lSize - lGoodParSize*2) * 1000.0 /
Util::Int64ToFloat(m_lSize - lGoodParSize));
if (lGoodParSize*2 > m_lSize)
{
@@ -657,81 +646,27 @@ void NZBInfo::UpdateMinMaxTime()
}
}
MessageList* NZBInfo::LockCachedMessages()
NZBInfo::Messages* NZBInfo::LockMessages()
{
m_mutexLog.Lock();
return &m_Messages;
}
void NZBInfo::UnlockCachedMessages()
void NZBInfo::UnlockMessages()
{
m_mutexLog.Unlock();
}
void NZBInfo::AddMessage(Message::EKind eKind, const char * szText)
void NZBInfo::AppendMessage(Message::EKind eKind, time_t tTime, const char * szText)
{
switch (eKind)
if (tTime == 0)
{
case Message::mkDetail:
detail("%s", szText);
break;
case Message::mkInfo:
info("%s", szText);
break;
case Message::mkWarning:
warn("%s", szText);
break;
case Message::mkError:
error("%s", szText);
break;
case Message::mkDebug:
debug("%s", szText);
break;
tTime = time(NULL);
}
m_mutexLog.Lock();
Message* pMessage = new Message(++m_iIDMessageGen, eKind, time(NULL), szText);
Message* pMessage = new Message(++m_iIDMessageGen, eKind, tTime, szText);
m_Messages.push_back(pMessage);
if (g_pOptions->GetSaveQueue() && g_pOptions->GetServerMode() && g_pOptions->GetNzbLog())
{
g_pDiskState->AppendNZBMessage(m_iID, eKind, szText);
m_iMessageCount++;
}
while (m_Messages.size() > (unsigned int)g_pOptions->GetLogBufferSize())
{
Message* pMessage = m_Messages.front();
delete pMessage;
m_Messages.pop_front();
}
m_iCachedMessageCount = m_Messages.size();
m_mutexLog.Unlock();
}
void NZBInfo::PrintMessage(Message::EKind eKind, const char* szFormat, ...)
{
char tmp2[1024];
va_list ap;
va_start(ap, szFormat);
vsnprintf(tmp2, 1024, szFormat, ap);
tmp2[1024-1] = '\0';
va_end(ap);
AddMessage(eKind, tmp2);
}
void NZBInfo::ClearMessages()
{
m_mutexLog.Lock();
m_Messages.Clear();
m_iCachedMessageCount = 0;
m_mutexLog.Unlock();
}
@@ -769,7 +704,6 @@ void NZBInfo::CopyFileList(NZBInfo* pSrcNZBInfo)
SetParFailedSize(pSrcNZBInfo->GetParFailedSize());
SetParCurrentFailedSize(pSrcNZBInfo->GetParCurrentFailedSize());
SetTotalArticles(pSrcNZBInfo->GetTotalArticles());
SetSuccessArticles(pSrcNZBInfo->GetSuccessArticles());
SetFailedArticles(pSrcNZBInfo->GetFailedArticles());
SetCurrentSuccessArticles(pSrcNZBInfo->GetSuccessArticles());
@@ -789,41 +723,19 @@ void NZBInfo::LeavePostProcess()
{
delete m_pPostInfo;
m_pPostInfo = NULL;
ClearMessages();
}
void NZBInfo::SetActiveDownloads(int iActiveDownloads)
{
if (((m_iActiveDownloads == 0 && iActiveDownloads > 0) ||
(m_iActiveDownloads > 0 && iActiveDownloads == 0)) &&
m_eKind == NZBInfo::nkNzb)
{
if (iActiveDownloads > 0)
{
m_tDownloadStartTime = time(NULL);
}
else
{
m_iDownloadSec += time(NULL) - m_tDownloadStartTime;
m_tDownloadStartTime = 0;
}
}
m_iActiveDownloads = iActiveDownloads;
}
bool NZBInfo::IsDupeSuccess()
{
bool bFailure =
m_eMarkStatus != NZBInfo::ksSuccess &&
m_eMarkStatus != NZBInfo::ksGood &&
(m_eDeleteStatus != NZBInfo::dsNone ||
m_eDeleteStatus != NZBInfo::dsNone ||
m_eMarkStatus == NZBInfo::ksBad ||
m_eParStatus == NZBInfo::psFailure ||
m_eUnpackStatus == NZBInfo::usFailure ||
m_eUnpackStatus == NZBInfo::usPassword ||
(m_eParStatus == NZBInfo::psSkipped &&
m_eUnpackStatus == NZBInfo::usSkipped &&
CalcHealth() < CalcCriticalHealth(true)));
CalcHealth() < CalcCriticalHealth(true));
return !bFailure;
}
@@ -845,10 +757,6 @@ const char* NZBInfo::MakeTextStatus(bool bIgnoreScriptStatus)
{
szStatus = "SUCCESS/GOOD";
}
else if (m_eMarkStatus == NZBInfo::ksSuccess)
{
szStatus = "SUCCESS/MARK";
}
else if (m_eDeleteStatus == NZBInfo::dsHealth)
{
szStatus = "FAILURE/HEALTH";
@@ -861,22 +769,6 @@ const char* NZBInfo::MakeTextStatus(bool bIgnoreScriptStatus)
{
szStatus = "DELETED/DUPE";
}
else if (m_eDeleteStatus == NZBInfo::dsBad)
{
szStatus = "FAILURE/BAD";
}
else if (m_eDeleteStatus == NZBInfo::dsGood)
{
szStatus = "DELETED/GOOD";
}
else if (m_eDeleteStatus == NZBInfo::dsCopy)
{
szStatus = "DELETED/COPY";
}
else if (m_eDeleteStatus == NZBInfo::dsScan)
{
szStatus = "FAILURE/SCAN";
}
else if (m_eParStatus == NZBInfo::psFailure)
{
szStatus = "FAILURE/PAR";
@@ -1003,38 +895,20 @@ void NZBList::Remove(NZBInfo* pNZBInfo)
}
}
NZBInfo* NZBList::Find(int iID)
{
for (iterator it = begin(); it != end(); it++)
{
NZBInfo* pNZBInfo = *it;
if (pNZBInfo->GetID() == iID)
{
return pNZBInfo;
}
}
return NULL;
}
ArticleInfo::ArticleInfo()
{
//debug("Creating ArticleInfo");
m_szMessageID = NULL;
m_iSize = 0;
m_pSegmentContent = NULL;
m_iSegmentOffset = 0;
m_iSegmentSize = 0;
m_eStatus = aiUndefined;
m_szResultFilename = NULL;
m_lCrc = 0;
m_szMessageID = NULL;
m_iSize = 0;
m_eStatus = aiUndefined;
m_szResultFilename = NULL;
}
ArticleInfo::~ ArticleInfo()
{
//debug("Destroying ArticleInfo");
DiscardSegment();
free(m_szMessageID);
free(m_szResultFilename);
}
@@ -1051,26 +925,8 @@ void ArticleInfo::SetResultFilename(const char * v)
m_szResultFilename = strdup(v);
}
void ArticleInfo::AttachSegment(char* pContent, long long iOffset, int iSize)
{
DiscardSegment();
m_pSegmentContent = pContent;
m_iSegmentOffset = iOffset;
m_iSegmentSize = iSize;
}
void ArticleInfo::DiscardSegment()
{
if (m_pSegmentContent)
{
free(m_pSegmentContent);
m_pSegmentContent = NULL;
g_pArticleCache->Free(m_iSegmentSize);
}
}
FileInfo::FileInfo(int iID)
FileInfo::FileInfo()
{
debug("Creating FileInfo");
@@ -1100,9 +956,7 @@ FileInfo::FileInfo(int iID)
m_bExtraPriority = false;
m_iActiveDownloads = 0;
m_bAutoDeleted = false;
m_iCachedArticles = 0;
m_bPartialChanged = false;
m_iID = iID ? iID : ++m_iIDGen;
m_iID = ++m_iIDGen;
}
FileInfo::~ FileInfo()
@@ -1234,30 +1088,6 @@ void FileList::Remove(FileInfo* pFileInfo)
erase(std::find(begin(), end(), pFileInfo));
}
CompletedFile::CompletedFile(int iID, const char* szFileName, EStatus eStatus, unsigned long lCrc)
{
m_iID = iID;
if (FileInfo::m_iIDMax < m_iID)
{
FileInfo::m_iIDMax = m_iID;
}
m_szFileName = strdup(szFileName);
m_eStatus = eStatus;
m_lCrc = lCrc;
}
void CompletedFile::SetFileName(const char* szFileName)
{
free(m_szFileName);
m_szFileName = strdup(szFileName);
}
CompletedFile::~CompletedFile()
{
free(m_szFileName);
}
PostInfo::PostInfo()
{
@@ -1267,12 +1097,6 @@ PostInfo::PostInfo()
m_bWorking = false;
m_bDeleted = false;
m_bRequestParCheck = false;
m_bForceParFull = false;
m_bForceRepair = false;
m_bParRepaired = false;
m_bUnpackTried = false;
m_bPassListTried = false;
m_eLastUnpackStatus = 0;
m_szProgressLabel = strdup("");
m_iFileProgress = 0;
m_iStageProgress = 0;
@@ -1280,6 +1104,7 @@ PostInfo::PostInfo()
m_tStageTime = 0;
m_eStage = ptQueued;
m_pPostThread = NULL;
m_iIDMessageGen = 0;
}
PostInfo::~ PostInfo()
@@ -1288,6 +1113,11 @@ PostInfo::~ PostInfo()
free(m_szProgressLabel);
for (Messages::iterator it = m_Messages.begin(); it != m_Messages.end(); it++)
{
delete *it;
}
for (ParredFiles::iterator it = m_ParredFiles.begin(); it != m_ParredFiles.end(); it++)
{
free(*it);
@@ -1300,6 +1130,32 @@ void PostInfo::SetProgressLabel(const char* szProgressLabel)
m_szProgressLabel = strdup(szProgressLabel);
}
PostInfo::Messages* PostInfo::LockMessages()
{
m_mutexLog.Lock();
return &m_Messages;
}
void PostInfo::UnlockMessages()
{
m_mutexLog.Unlock();
}
void PostInfo::AppendMessage(Message::EKind eKind, const char * szText)
{
m_mutexLog.Lock();
Message* pMessage = new Message(++m_iIDMessageGen, eKind, time(NULL), szText);
m_Messages.push_back(pMessage);
while (m_Messages.size() > (unsigned int)g_pOptions->GetLogBufferSize())
{
Message* pMessage = m_Messages.front();
delete pMessage;
m_Messages.pop_front();
}
m_mutexLog.Unlock();
}
DupInfo::DupInfo()
{
@@ -1399,29 +1255,6 @@ void HistoryInfo::GetName(char* szBuffer, int iSize)
}
HistoryList::~HistoryList()
{
for (iterator it = begin(); it != end(); it++)
{
delete *it;
}
}
HistoryInfo* HistoryList::Find(int iID)
{
for (iterator it = begin(); it != end(); it++)
{
HistoryInfo* pHistoryInfo = *it;
if (pHistoryInfo->GetID() == iID)
{
return pHistoryInfo;
}
}
return NULL;
}
DownloadQueue* DownloadQueue::Lock()
{
g_pDownloadQueue->m_LockMutex.Lock();

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -89,12 +89,8 @@ private:
int m_iPartNumber;
char* m_szMessageID;
int m_iSize;
char* m_pSegmentContent;
long long m_iSegmentOffset;
int m_iSegmentSize;
EStatus m_eStatus;
char* m_szResultFilename;
unsigned long m_lCrc;
public:
ArticleInfo();
@@ -103,21 +99,12 @@ public:
int GetPartNumber() { return m_iPartNumber; }
const char* GetMessageID() { return m_szMessageID; }
void SetMessageID(const char* szMessageID);
void SetSize(int iSize) { m_iSize = iSize; }
void SetSize(int s) { m_iSize = s; }
int GetSize() { return m_iSize; }
void AttachSegment(char* pContent, long long iOffset, int iSize);
void DiscardSegment();
const char* GetSegmentContent() { return m_pSegmentContent; }
void SetSegmentOffset(long long iSegmentOffset) { m_iSegmentOffset = iSegmentOffset; }
long long GetSegmentOffset() { return m_iSegmentOffset; }
void SetSegmentSize(int iSegmentSize) { m_iSegmentSize = iSegmentSize; }
int GetSegmentSize() { return m_iSegmentSize; }
EStatus GetStatus() { return m_eStatus; }
void SetStatus(EStatus Status) { m_eStatus = Status; }
const char* GetResultFilename() { return m_szResultFilename; }
void SetResultFilename(const char* v);
unsigned long GetCrc() { return m_lCrc; }
void SetCrc(unsigned long lCrc) { m_lCrc = lCrc; }
};
class FileInfo
@@ -155,16 +142,12 @@ private:
bool m_bExtraPriority;
int m_iActiveDownloads;
bool m_bAutoDeleted;
int m_iCachedArticles;
bool m_bPartialChanged;
static int m_iIDGen;
static int m_iIDMax;
friend class CompletedFile;
public:
FileInfo(int iID = 0);
FileInfo();
~FileInfo();
int GetID() { return m_iID; }
void SetID(int iID);
@@ -216,15 +199,11 @@ public:
bool GetOutputInitialized() { return m_bOutputInitialized; }
void SetOutputInitialized(bool bOutputInitialized) { m_bOutputInitialized = bOutputInitialized; }
bool GetExtraPriority() { return m_bExtraPriority; }
void SetExtraPriority(bool bExtraPriority) { m_bExtraPriority = bExtraPriority; }
void SetExtraPriority(bool bExtraPriority) { m_bExtraPriority = bExtraPriority; };
int GetActiveDownloads() { return m_iActiveDownloads; }
void SetActiveDownloads(int iActiveDownloads);
bool GetAutoDeleted() { return m_bAutoDeleted; }
void SetAutoDeleted(bool bAutoDeleted) { m_bAutoDeleted = bAutoDeleted; }
int GetCachedArticles() { return m_iCachedArticles; }
void SetCachedArticles(int iCachedArticles) { m_iCachedArticles = iCachedArticles; }
bool GetPartialChanged() { return m_bPartialChanged; }
void SetPartialChanged(bool bPartialChanged) { m_bPartialChanged = bPartialChanged; }
ServerStatList* GetServerStats() { return &m_ServerStats; }
};
@@ -241,34 +220,6 @@ public:
void Remove(FileInfo* pFileInfo);
};
class CompletedFile
{
public:
enum EStatus
{
cfUnknown,
cfSuccess,
cfPartial,
cfFailure
};
private:
int m_iID;
char* m_szFileName;
EStatus m_eStatus;
unsigned long m_lCrc;
public:
CompletedFile(int iID, const char* szFileName, EStatus eStatus, unsigned long lCrc);
~CompletedFile();
int GetID() { return m_iID; }
void SetFileName(const char* szFileName);
const char* GetFileName() { return m_szFileName; }
EStatus GetStatus() { return m_eStatus; }
unsigned long GetCrc() { return m_lCrc; }
};
typedef std::deque<CompletedFile*> CompletedFiles;
class NZBParameter
{
@@ -312,7 +263,7 @@ public:
private:
char* m_szName;
EStatus m_eStatus;
friend class ScriptStatusList;
public:
@@ -390,19 +341,14 @@ public:
dsNone,
dsManual,
dsHealth,
dsDupe,
dsBad,
dsGood,
dsCopy,
dsScan
dsDupe
};
enum EMarkStatus
{
ksNone,
ksBad,
ksGood,
ksSuccess
ksGood
};
enum EUrlStatus
@@ -422,6 +368,9 @@ public:
nkUrl
};
typedef std::vector<char*> Files;
typedef std::deque<Message*> Messages;
static const int FORCE_PRIORITY = 900;
friend class DupInfo;
@@ -460,7 +409,7 @@ private:
time_t m_tMinTime;
time_t m_tMaxTime;
int m_iPriority;
CompletedFiles m_completedFiles;
Files m_completedFiles;
ERenameStatus m_eRenameStatus;
EParStatus m_eParStatus;
EUnpackStatus m_eUnpackStatus;
@@ -469,7 +418,6 @@ private:
EDeleteStatus m_eDeleteStatus;
EMarkStatus m_eMarkStatus;
EUrlStatus m_eUrlStatus;
int m_iExtraParBlocks;
bool m_bAddUrlPaused;
bool m_bDeletePaused;
bool m_bManyDupeFiles;
@@ -492,28 +440,13 @@ private:
ServerStatList m_ServerStats;
ServerStatList m_CurrentServerStats;
Mutex m_mutexLog;
MessageList m_Messages;
Messages m_Messages;
int m_iIDMessageGen;
PostInfo* m_pPostInfo;
long long m_lDownloadedSize;
time_t m_tDownloadStartTime;
int m_iDownloadSec;
int m_iPostTotalSec;
int m_iParSec;
int m_iRepairSec;
int m_iUnpackSec;
bool m_bReprocess;
time_t m_tQueueScriptTime;
bool m_bParFull;
int m_iMessageCount;
int m_iCachedMessageCount;
int m_iFeedID;
static int m_iIDGen;
static int m_iIDMax;
void ClearMessages();
public:
NZBInfo();
~NZBInfo();
@@ -552,7 +485,7 @@ public:
int GetRemainingParCount() { return m_iRemainingParCount; }
void SetRemainingParCount(int iRemainingParCount) { m_iRemainingParCount = iRemainingParCount; }
int GetActiveDownloads() { return m_iActiveDownloads; }
void SetActiveDownloads(int iActiveDownloads);
void SetActiveDownloads(int iActiveDownloads) { m_iActiveDownloads = iActiveDownloads; }
long long GetSuccessSize() { return m_lSuccessSize; }
void SetSuccessSize(long long lSuccessSize) { m_lSuccessSize = lSuccessSize; }
long long GetFailedSize() { return m_lFailedSize; }
@@ -590,7 +523,7 @@ public:
void SetMaxTime(time_t tMaxTime) { m_tMaxTime = tMaxTime; }
void BuildDestDirName();
void BuildFinalDirName(char* szFinalDirBuf, int iBufSize);
CompletedFiles* GetCompletedFiles() { return &m_completedFiles; } // needs locking (for shared objects)
Files* GetCompletedFiles() { return &m_completedFiles; } // needs locking (for shared objects)
void ClearCompletedFiles();
ERenameStatus GetRenameStatus() { return m_eRenameStatus; }
void SetRenameStatus(ERenameStatus eRenameStatus) { m_eRenameStatus = eRenameStatus; }
@@ -607,8 +540,6 @@ public:
EMarkStatus GetMarkStatus() { return m_eMarkStatus; }
void SetMarkStatus(EMarkStatus eMarkStatus) { m_eMarkStatus = eMarkStatus; }
EUrlStatus GetUrlStatus() { return m_eUrlStatus; }
int GetExtraParBlocks() { return m_iExtraParBlocks; }
void SetExtraParBlocks(int iExtraParBlocks) { m_iExtraParBlocks = iExtraParBlocks; }
void SetUrlStatus(EUrlStatus eUrlStatus) { m_eUrlStatus = eUrlStatus; }
const char* GetQueuedFilename() { return m_szQueuedFilename; }
void SetQueuedFilename(const char* szQueuedFilename);
@@ -647,29 +578,6 @@ public:
void SetFullContentHash(unsigned int iFullContentHash) { m_iFullContentHash = iFullContentHash; }
unsigned int GetFilteredContentHash() { return m_iFilteredContentHash; }
void SetFilteredContentHash(unsigned int iFilteredContentHash) { m_iFilteredContentHash = iFilteredContentHash; }
long long GetDownloadedSize() { return m_lDownloadedSize; }
void SetDownloadedSize(long long lDownloadedSize) { m_lDownloadedSize = lDownloadedSize; }
int GetDownloadSec() { return m_iDownloadSec; }
void SetDownloadSec(int iDownloadSec) { m_iDownloadSec = iDownloadSec; }
int GetPostTotalSec() { return m_iPostTotalSec; }
void SetPostTotalSec(int iPostTotalSec) { m_iPostTotalSec = iPostTotalSec; }
int GetParSec() { return m_iParSec; }
void SetParSec(int iParSec) { m_iParSec = iParSec; }
int GetRepairSec() { return m_iRepairSec; }
void SetRepairSec(int iRepairSec) { m_iRepairSec = iRepairSec; }
int GetUnpackSec() { return m_iUnpackSec; }
void SetUnpackSec(int iUnpackSec) { m_iUnpackSec = iUnpackSec; }
time_t GetDownloadStartTime() { return m_tDownloadStartTime; }
void SetDownloadStartTime(time_t tDownloadStartTime) { m_tDownloadStartTime = tDownloadStartTime; }
void SetReprocess(bool bReprocess) { m_bReprocess = bReprocess; }
bool GetReprocess() { return m_bReprocess; }
time_t GetQueueScriptTime() { return m_tQueueScriptTime; }
void SetQueueScriptTime(time_t tQueueScriptTime) { m_tQueueScriptTime = tQueueScriptTime; }
void SetParFull(bool bParFull) { m_bParFull = bParFull; }
bool GetParFull() { return m_bParFull; }
int GetFeedID() { return m_iFeedID; }
void SetFeedID(int iFeedID) { m_iFeedID = iFeedID; }
void CopyFileList(NZBInfo* pSrcNZBInfo);
void UpdateMinMaxTime();
PostInfo* GetPostInfo() { return m_pPostInfo; }
@@ -678,13 +586,9 @@ public:
bool IsDupeSuccess();
const char* MakeTextStatus(bool bIgnoreScriptStatus);
void AddMessage(Message::EKind eKind, const char* szText);
void PrintMessage(Message::EKind eKind, const char* szFormat, ...);
int GetMessageCount() { return m_iMessageCount; }
void SetMessageCount(int iMessageCount) { m_iMessageCount = iMessageCount; }
int GetCachedMessageCount() { return m_iCachedMessageCount; }
MessageList* LockCachedMessages();
void UnlockCachedMessages();
void AppendMessage(Message::EKind eKind, time_t tTime, const char* szText);
Messages* LockMessages();
void UnlockMessages();
};
typedef std::deque<NZBInfo*> NZBQueueBase;
@@ -699,7 +603,6 @@ public:
void Clear();
void Add(NZBInfo* pNZBInfo, bool bAddTop);
void Remove(NZBInfo* pNZBInfo);
NZBInfo* Find(int iID);
};
class PostInfo
@@ -719,6 +622,7 @@ public:
ptFinished
};
typedef std::deque<Message*> Messages;
typedef std::vector<char*> ParredFiles;
private:
@@ -726,12 +630,6 @@ private:
bool m_bWorking;
bool m_bDeleted;
bool m_bRequestParCheck;
bool m_bForceParFull;
bool m_bForceRepair;
bool m_bParRepaired;
bool m_bUnpackTried;
bool m_bPassListTried;
int m_eLastUnpackStatus;
EStage m_eStage;
char* m_szProgressLabel;
int m_iFileProgress;
@@ -740,6 +638,9 @@ private:
time_t m_tStageTime;
Thread* m_pPostThread;
Mutex m_mutexLog;
Messages m_Messages;
int m_iIDMessageGen;
ParredFiles m_ParredFiles;
public:
@@ -765,20 +666,11 @@ public:
void SetDeleted(bool bDeleted) { m_bDeleted = bDeleted; }
bool GetRequestParCheck() { return m_bRequestParCheck; }
void SetRequestParCheck(bool bRequestParCheck) { m_bRequestParCheck = bRequestParCheck; }
bool GetForceParFull() { return m_bForceParFull; }
void SetForceParFull(bool bForceParFull) { m_bForceParFull = bForceParFull; }
bool GetForceRepair() { return m_bForceRepair; }
void SetForceRepair(bool bForceRepair) { m_bForceRepair = bForceRepair; }
bool GetParRepaired() { return m_bParRepaired; }
void SetParRepaired(bool bParRepaired) { m_bParRepaired = bParRepaired; }
bool GetUnpackTried() { return m_bUnpackTried; }
void SetUnpackTried(bool bUnpackTried) { m_bUnpackTried = bUnpackTried; }
bool GetPassListTried() { return m_bPassListTried; }
void SetPassListTried(bool bPassListTried) { m_bPassListTried = bPassListTried; }
int GetLastUnpackStatus() { return m_eLastUnpackStatus; }
void SetLastUnpackStatus(int eUnpackStatus) { m_eLastUnpackStatus = eUnpackStatus; }
Thread* GetPostThread() { return m_pPostThread; }
void SetPostThread(Thread* pPostThread) { m_pPostThread = pPostThread; }
void AppendMessage(Message::EKind eKind, const char* szText);
Messages* LockMessages();
void UnlockMessages();
ParredFiles* GetParredFiles() { return &m_ParredFiles; }
};
@@ -864,14 +756,7 @@ public:
void GetName(char* szBuffer, int iSize); // needs locking (for shared objects)
};
typedef std::deque<HistoryInfo*> HistoryListBase;
class HistoryList : public HistoryListBase
{
public:
~HistoryList();
HistoryInfo* Find(int iID);
};
typedef std::deque<HistoryInfo*> HistoryList;
class DownloadQueue : public Subject
{
@@ -880,7 +765,6 @@ public:
{
eaNzbFound,
eaNzbAdded,
eaNzbDeleted,
eaFileCompleted,
eaFileDeleted,
eaUrlCompleted
@@ -925,7 +809,6 @@ public:
eaGroupSetDupeKey, // set duplicate key
eaGroupSetDupeScore, // set duplicate score
eaGroupSetDupeMode, // set duplicate mode
eaGroupSort, // sort groups
eaPostDelete, // cancel post-processing
eaHistoryDelete, // hide history-item
eaHistoryFinalDelete, // delete history-item
@@ -938,10 +821,7 @@ public:
eaHistorySetDupeMode, // set duplicate mode
eaHistorySetDupeBackup, // set duplicate backup flag
eaHistoryMarkBad, // mark history-item as bad (and download other duplicate)
eaHistoryMarkGood, // mark history-item as good (and push it into dup-history)
eaHistoryMarkSuccess, // mark history-item as success (and do nothing more)
eaHistorySetCategory, // set or change category for history-item
eaHistorySetName // set history-item name (rename)
eaHistoryMarkGood // mark history-item as good (and push it into dup-history)
};
enum EMatchMode
@@ -966,6 +846,7 @@ protected:
static void Loaded() { g_bLoaded = true; }
public:
virtual ~DownloadQueue() {};
static bool IsLoaded() { return g_bLoaded; }
static DownloadQueue* Lock();
static void Unlock();

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -50,6 +50,9 @@
#include "HistoryCoordinator.h"
#include "DupeCoordinator.h"
extern HistoryCoordinator* g_pHistoryCoordinator;
extern Options* g_pOptions;
bool DupeCoordinator::SameNameOrKey(const char* szName1, const char* szDupeKey1,
const char* szName2, const char* szDupeKey2)
{
@@ -89,82 +92,22 @@ void DupeCoordinator::NZBFound(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo)
// in queue - the new item is skipped
if (pQueuedNZBInfo != pNZBInfo && bSameContent && pNZBInfo->GetKind() == NZBInfo::nkNzb)
{
char szMessage[1024];
if (!strcmp(pNZBInfo->GetName(), pQueuedNZBInfo->GetName()))
{
snprintf(szMessage, 1024, "Skipping duplicate %s, already queued", pNZBInfo->GetName());
warn("Skipping duplicate %s, already queued", pNZBInfo->GetName());
}
else
{
snprintf(szMessage, 1024, "Skipping duplicate %s, already queued as %s",
warn("Skipping duplicate %s, already queued as %s",
pNZBInfo->GetName(), pQueuedNZBInfo->GetName());
}
szMessage[1024-1] = '\0';
if (pNZBInfo->GetFeedID())
{
warn("%s", szMessage);
// Flag saying QueueCoordinator to skip nzb-file
pNZBInfo->SetDeleteStatus(NZBInfo::dsManual);
g_pHistoryCoordinator->DeleteDiskFiles(pNZBInfo);
}
else
{
pNZBInfo->SetDeleteStatus(NZBInfo::dsCopy);
pNZBInfo->AddMessage(Message::mkWarning, szMessage);
}
// Flag saying QueueCoordinator to skip nzb-file
pNZBInfo->SetDeleteStatus(NZBInfo::dsManual);
g_pHistoryCoordinator->DeleteQueuedFile(pNZBInfo->GetQueuedFilename());
return;
}
}
// if download has empty dupekey and empty dupescore - check if download queue
// or history have an item with the same name and non empty dupekey or dupescore and
// take these properties from this item
if (Util::EmptyStr(pNZBInfo->GetDupeKey()) && pNZBInfo->GetDupeScore() == 0)
{
for (NZBList::iterator it = pDownloadQueue->GetQueue()->begin(); it != pDownloadQueue->GetQueue()->end(); it++)
{
NZBInfo* pQueuedNZBInfo = *it;
if (!strcmp(pQueuedNZBInfo->GetName(), pNZBInfo->GetName()) &&
(!Util::EmptyStr(pQueuedNZBInfo->GetDupeKey()) || pQueuedNZBInfo->GetDupeScore() != 0))
{
pNZBInfo->SetDupeKey(pQueuedNZBInfo->GetDupeKey());
pNZBInfo->SetDupeScore(pQueuedNZBInfo->GetDupeScore());
info("Assigning dupekey %s and dupescore %i to %s from existing queue item with the same name",
pNZBInfo->GetDupeKey(), pNZBInfo->GetDupeScore(), pNZBInfo->GetName());
break;
}
}
}
if (Util::EmptyStr(pNZBInfo->GetDupeKey()) && pNZBInfo->GetDupeScore() == 0)
{
for (HistoryList::iterator it = pDownloadQueue->GetHistory()->begin(); it != pDownloadQueue->GetHistory()->end(); it++)
{
HistoryInfo* pHistoryInfo = *it;
if (pHistoryInfo->GetKind() == HistoryInfo::hkNzb &&
!strcmp(pHistoryInfo->GetNZBInfo()->GetName(), pNZBInfo->GetName()) &&
(!Util::EmptyStr(pHistoryInfo->GetNZBInfo()->GetDupeKey()) || pHistoryInfo->GetNZBInfo()->GetDupeScore() != 0))
{
pNZBInfo->SetDupeKey(pHistoryInfo->GetNZBInfo()->GetDupeKey());
pNZBInfo->SetDupeScore(pHistoryInfo->GetNZBInfo()->GetDupeScore());
info("Assigning dupekey %s and dupescore %i to %s from existing history item with the same name",
pNZBInfo->GetDupeKey(), pNZBInfo->GetDupeScore(), pNZBInfo->GetName());
break;
}
if (pHistoryInfo->GetKind() == HistoryInfo::hkDup &&
!strcmp(pHistoryInfo->GetDupInfo()->GetName(), pNZBInfo->GetName()) &&
(!Util::EmptyStr(pHistoryInfo->GetDupInfo()->GetDupeKey()) || pHistoryInfo->GetDupInfo()->GetDupeScore() != 0))
{
pNZBInfo->SetDupeKey(pHistoryInfo->GetDupInfo()->GetDupeKey());
pNZBInfo->SetDupeScore(pHistoryInfo->GetDupInfo()->GetDupeScore());
info("Assigning dupekey %s and dupescore %i to %s from existing history item with the same name",
pNZBInfo->GetDupeKey(), pNZBInfo->GetDupeScore(), pNZBInfo->GetName());
break;
}
}
}
// find duplicates in history
bool bSkip = false;
@@ -172,9 +115,9 @@ void DupeCoordinator::NZBFound(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo)
bool bSameContent = false;
const char* szDupeName = NULL;
// find duplicates in history having exactly same content
// find duplicates in queue having exactly same content
// also: nzb-files having duplicates marked as good are skipped
// also (only in score mode): nzb-files having success-duplicates in dup-history but not having duplicates in recent history are skipped
// also (only in score mode): nzb-files having success-duplicates in dup-history but don't having duplicates in recent history are skipped
for (HistoryList::iterator it = pDownloadQueue->GetHistory()->begin(); it != pDownloadQueue->GetHistory()->end(); it++)
{
HistoryInfo* pHistoryInfo = *it;
@@ -246,7 +189,7 @@ void DupeCoordinator::NZBFound(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo)
{
// Flag saying QueueCoordinator to skip nzb-file
pNZBInfo->SetDeleteStatus(NZBInfo::dsDupe);
info("Collection %s is a duplicate to %s", pNZBInfo->GetName(), pHistoryInfo->GetNZBInfo()->GetName());
info("Collection %s is duplicate to %s", pNZBInfo->GetName(), pHistoryInfo->GetNZBInfo()->GetName());
return;
}
}
@@ -254,33 +197,21 @@ void DupeCoordinator::NZBFound(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo)
if (bSkip)
{
char szMessage[1024];
if (!strcmp(pNZBInfo->GetName(), szDupeName))
{
snprintf(szMessage, 1024, "Skipping duplicate %s, found in history with %s", pNZBInfo->GetName(),
warn("Skipping duplicate %s, found in history with %s", pNZBInfo->GetName(),
bSameContent ? "exactly same content" : bGood ? "good status" : "success status");
}
else
{
snprintf(szMessage, 1024, "Skipping duplicate %s, found in history %s with %s",
warn("Skipping duplicate %s, found in history %s with %s",
pNZBInfo->GetName(), szDupeName,
bSameContent ? "exactly same content" : bGood ? "good status" : "success status");
}
szMessage[1024-1] = '\0';
if (pNZBInfo->GetFeedID())
{
warn("%s", szMessage);
// Flag saying QueueCoordinator to skip nzb-file
pNZBInfo->SetDeleteStatus(NZBInfo::dsManual);
g_pHistoryCoordinator->DeleteDiskFiles(pNZBInfo);
}
else
{
pNZBInfo->SetDeleteStatus(bSameContent ? NZBInfo::dsCopy : NZBInfo::dsGood);
pNZBInfo->AddMessage(Message::mkWarning, szMessage);
}
// Flag saying QueueCoordinator to skip nzb-file
pNZBInfo->SetDeleteStatus(NZBInfo::dsManual);
g_pHistoryCoordinator->DeleteQueuedFile(pNZBInfo->GetQueuedFilename());
return;
}
@@ -305,7 +236,7 @@ void DupeCoordinator::NZBFound(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo)
{
// Flag saying QueueCoordinator to skip nzb-file
pNZBInfo->SetDeleteStatus(NZBInfo::dsDupe);
info("Collection %s is a duplicate to %s", pNZBInfo->GetName(), pQueuedNZBInfo->GetName());
info("Collection %s is duplicate to %s", pNZBInfo->GetName(), pQueuedNZBInfo->GetName());
return;
}
@@ -356,7 +287,8 @@ void DupeCoordinator::ReturnBestDupe(DownloadQueue* pDownloadQueue, NZBInfo* pNZ
if (pHistoryInfo->GetKind() == HistoryInfo::hkNzb &&
pHistoryInfo->GetNZBInfo()->GetDupeMode() != dmForce &&
pHistoryInfo->GetNZBInfo()->IsDupeSuccess() &&
(pHistoryInfo->GetNZBInfo()->IsDupeSuccess() ||
pHistoryInfo->GetNZBInfo()->GetMarkStatus() == NZBInfo::ksGood) &&
SameNameOrKey(pHistoryInfo->GetNZBInfo()->GetName(), pHistoryInfo->GetNZBInfo()->GetDupeKey(), szNZBName, szDupeKey))
{
if (!bHistoryDupe || pHistoryInfo->GetNZBInfo()->GetDupeScore() > iHistoryScore)
@@ -433,25 +365,20 @@ void DupeCoordinator::ReturnBestDupe(DownloadQueue* pDownloadQueue, NZBInfo* pNZ
}
}
void DupeCoordinator::HistoryMark(DownloadQueue* pDownloadQueue, HistoryInfo* pHistoryInfo, NZBInfo::EMarkStatus eMarkStatus)
void DupeCoordinator::HistoryMark(DownloadQueue* pDownloadQueue, HistoryInfo* pHistoryInfo, bool bGood)
{
char szNZBName[1024];
pHistoryInfo->GetName(szNZBName, 1024);
const char* szMarkStatusName[] = { "NONE", "bad", "good", "success" };
info("Marking %s as %s", szNZBName, szMarkStatusName[eMarkStatus]);
info("Marking %s as %s", szNZBName, (bGood ? "good" : "bad"));
if (pHistoryInfo->GetKind() == HistoryInfo::hkNzb)
{
pHistoryInfo->GetNZBInfo()->SetMarkStatus(eMarkStatus);
pHistoryInfo->GetNZBInfo()->SetMarkStatus(bGood ? NZBInfo::ksGood : NZBInfo::ksBad);
}
else if (pHistoryInfo->GetKind() == HistoryInfo::hkDup)
{
pHistoryInfo->GetDupInfo()->SetStatus(
eMarkStatus == NZBInfo::ksGood ? DupInfo::dsGood :
eMarkStatus == NZBInfo::ksSuccess ? DupInfo::dsSuccess :
DupInfo::dsBad);
pHistoryInfo->GetDupInfo()->SetStatus(bGood ? DupInfo::dsGood : DupInfo::dsBad);
}
else
{
@@ -468,13 +395,13 @@ void DupeCoordinator::HistoryMark(DownloadQueue* pDownloadQueue, HistoryInfo* pH
return;
}
if (eMarkStatus == NZBInfo::ksGood)
if (bGood)
{
// mark as good
// moving all duplicates from history to dup-history
HistoryCleanup(pDownloadQueue, pHistoryInfo);
}
else if (eMarkStatus == NZBInfo::ksBad)
else
{
// mark as bad
const char* szDupeKey = pHistoryInfo->GetKind() == HistoryInfo::hkNzb ? pHistoryInfo->GetNZBInfo()->GetDupeKey() :
@@ -524,89 +451,3 @@ void DupeCoordinator::HistoryCleanup(DownloadQueue* pDownloadQueue, HistoryInfo*
pDownloadQueue->Save();
}
}
DupeCoordinator::EDupeStatus DupeCoordinator::GetDupeStatus(DownloadQueue* pDownloadQueue,
const char* szName, const char* szDupeKey)
{
EDupeStatus eStatuses = dsNone;
// find duplicates in download queue
for (NZBList::iterator it = pDownloadQueue->GetQueue()->begin(); it != pDownloadQueue->GetQueue()->end(); it++)
{
NZBInfo* pNZBInfo = *it;
if (SameNameOrKey(szName, szDupeKey, pNZBInfo->GetName(), pNZBInfo->GetDupeKey()))
{
if (pNZBInfo->GetSuccessArticles() + pNZBInfo->GetFailedArticles() > 0)
{
eStatuses = (EDupeStatus)(eStatuses | dsDownloading);
}
else
{
eStatuses = (EDupeStatus)(eStatuses | dsQueued);
}
}
}
// find duplicates in history
for (HistoryList::iterator it = pDownloadQueue->GetHistory()->begin(); it != pDownloadQueue->GetHistory()->end(); it++)
{
HistoryInfo* pHistoryInfo = *it;
if (pHistoryInfo->GetKind() == HistoryInfo::hkNzb &&
SameNameOrKey(szName, szDupeKey, pHistoryInfo->GetNZBInfo()->GetName(), pHistoryInfo->GetNZBInfo()->GetDupeKey()))
{
const char* szTextStatus = pHistoryInfo->GetNZBInfo()->MakeTextStatus(true);
if (!strncasecmp(szTextStatus, "SUCCESS", 7))
{
eStatuses = (EDupeStatus)(eStatuses | dsSuccess);
}
else if (!strncasecmp(szTextStatus, "FAILURE", 7))
{
eStatuses = (EDupeStatus)(eStatuses | dsFailure);
}
else if (!strncasecmp(szTextStatus, "WARNING", 7))
{
eStatuses = (EDupeStatus)(eStatuses | dsWarning);
}
}
if (pHistoryInfo->GetKind() == HistoryInfo::hkDup &&
SameNameOrKey(szName, szDupeKey, pHistoryInfo->GetDupInfo()->GetName(), pHistoryInfo->GetDupInfo()->GetDupeKey()))
{
if (pHistoryInfo->GetDupInfo()->GetStatus() == DupInfo::dsSuccess ||
pHistoryInfo->GetDupInfo()->GetStatus() == DupInfo::dsGood)
{
eStatuses = (EDupeStatus)(eStatuses | dsSuccess);
}
else if (pHistoryInfo->GetDupInfo()->GetStatus() == DupInfo::dsFailed ||
pHistoryInfo->GetDupInfo()->GetStatus() == DupInfo::dsBad)
{
eStatuses = (EDupeStatus)(eStatuses | dsFailure);
}
}
}
return eStatuses;
}
void DupeCoordinator::ListHistoryDupes(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo, NZBList* pDupeList)
{
if (pNZBInfo->GetDupeMode() == dmForce)
{
return;
}
// find duplicates in history
for (HistoryList::iterator it = pDownloadQueue->GetHistory()->begin(); it != pDownloadQueue->GetHistory()->end(); it++)
{
HistoryInfo* pHistoryInfo = *it;
if (pHistoryInfo->GetKind() == HistoryInfo::hkNzb &&
pHistoryInfo->GetNZBInfo()->GetDupeMode() != dmForce &&
SameNameOrKey(pHistoryInfo->GetNZBInfo()->GetName(), pHistoryInfo->GetNZBInfo()->GetDupeKey(),
pNZBInfo->GetName(), pNZBInfo->GetDupeKey()))
{
pDupeList->push_back(pHistoryInfo->GetNZBInfo());
}
}
}

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -30,17 +30,6 @@
class DupeCoordinator
{
public:
enum EDupeStatus
{
dsNone = 0,
dsQueued = 1,
dsDownloading = 2,
dsSuccess = 4,
dsWarning = 8,
dsFailure = 16
};
private:
void ReturnBestDupe(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo, const char* szNZBName, const char* szDupeKey);
void HistoryCleanup(DownloadQueue* pDownloadQueue, HistoryInfo* pMarkHistoryInfo);
@@ -49,11 +38,7 @@ private:
public:
void NZBCompleted(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo);
void NZBFound(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo);
void HistoryMark(DownloadQueue* pDownloadQueue, HistoryInfo* pHistoryInfo, NZBInfo::EMarkStatus eMarkStatus);
EDupeStatus GetDupeStatus(DownloadQueue* pDownloadQueue, const char* szName, const char* szDupeKey);
void ListHistoryDupes(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo, NZBList* pDupeList);
void HistoryMark(DownloadQueue* pDownloadQueue, HistoryInfo* pHistoryInfo, bool bGood);
};
extern DupeCoordinator* g_pDupeCoordinator;
#endif

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -51,10 +51,16 @@
#include "Util.h"
#include "NZBFile.h"
#include "DupeCoordinator.h"
#include "ParParser.h"
#include "ParCoordinator.h"
#include "PrePostProcessor.h"
#include "DupeCoordinator.h"
extern QueueCoordinator* g_pQueueCoordinator;
extern PrePostProcessor* g_pPrePostProcessor;
extern DupeCoordinator* g_pDupeCoordinator;
extern Options* g_pOptions;
extern DiskState* g_pDiskState;
HistoryCoordinator::HistoryCoordinator()
{
debug("Creating HistoryCoordinator");
@@ -65,10 +71,25 @@ HistoryCoordinator::~HistoryCoordinator()
debug("Destroying HistoryCoordinator");
}
void HistoryCoordinator::Cleanup()
{
debug("Cleaning up HistoryCoordinator");
DownloadQueue* pDownloadQueue = DownloadQueue::Lock();
for (HistoryList::iterator it = pDownloadQueue->GetHistory()->begin(); it != pDownloadQueue->GetHistory()->end(); it++)
{
delete *it;
}
pDownloadQueue->GetHistory()->clear();
DownloadQueue::Unlock();
}
/**
* Removes old entries from (recent) history
*/
void HistoryCoordinator::ServiceWork()
void HistoryCoordinator::IntervalCheck()
{
DownloadQueue* pDownloadQueue = DownloadQueue::Lock();
@@ -98,7 +119,7 @@ void HistoryCoordinator::ServiceWork()
if (pHistoryInfo->GetKind() == HistoryInfo::hkNzb)
{
DeleteDiskFiles(pHistoryInfo->GetNZBInfo());
DeleteQueuedFile(pHistoryInfo->GetNZBInfo()->GetQueuedFilename());
}
info("Collection %s removed from history", szNiceName);
@@ -123,24 +144,16 @@ void HistoryCoordinator::ServiceWork()
DownloadQueue::Unlock();
}
void HistoryCoordinator::DeleteDiskFiles(NZBInfo* pNZBInfo)
void HistoryCoordinator::DeleteQueuedFile(const char* szQueuedFile)
{
if (g_pOptions->GetSaveQueue() && g_pOptions->GetServerMode())
{
// delete parked files
g_pDiskState->DiscardFiles(pNZBInfo);
}
pNZBInfo->GetFileList()->Clear();
// delete nzb-file
if (!g_pOptions->GetNzbCleanupDisk())
{
return;
}
// QueuedFile may contain one filename or several filenames separated
// szQueuedFile may contain one filename or several filenames separated
// with "|"-character (for merged groups)
char* szFilename = strdup(pNZBInfo->GetQueuedFilename());
char* szFilename = strdup(szQueuedFile);
char* szEnd = szFilename - 1;
while (szEnd)
@@ -207,7 +220,7 @@ void HistoryCoordinator::AddToHistory(DownloadQueue* pDownloadQueue, NZBInfo* pN
pNZBInfo->GetFileList()->Clear();
}
pNZBInfo->PrintMessage(Message::mkInfo, "Collection %s added to history", pNZBInfo->GetName());
info("Collection %s added to history", pNZBInfo->GetName());
}
void HistoryCoordinator::HistoryHide(DownloadQueue* pDownloadQueue, HistoryInfo* pHistoryInfo, int rindex)
@@ -229,11 +242,8 @@ void HistoryCoordinator::HistoryHide(DownloadQueue* pDownloadQueue, HistoryInfo*
pDupInfo->SetStatus(
pHistoryInfo->GetNZBInfo()->GetMarkStatus() == NZBInfo::ksGood ? DupInfo::dsGood :
pHistoryInfo->GetNZBInfo()->GetMarkStatus() == NZBInfo::ksBad ? DupInfo::dsBad :
pHistoryInfo->GetNZBInfo()->GetMarkStatus() == NZBInfo::ksSuccess ? DupInfo::dsSuccess :
pHistoryInfo->GetNZBInfo()->GetDeleteStatus() == NZBInfo::dsDupe ? DupInfo::dsDupe :
pHistoryInfo->GetNZBInfo()->GetDeleteStatus() == NZBInfo::dsManual ||
pHistoryInfo->GetNZBInfo()->GetDeleteStatus() == NZBInfo::dsGood ||
pHistoryInfo->GetNZBInfo()->GetDeleteStatus() == NZBInfo::dsCopy ? DupInfo::dsDeleted :
pHistoryInfo->GetNZBInfo()->GetDeleteStatus() == NZBInfo::dsManual ? DupInfo::dsDeleted :
pHistoryInfo->GetNZBInfo()->IsDupeSuccess() ? DupInfo::dsSuccess :
DupInfo::dsFailed);
@@ -241,34 +251,15 @@ void HistoryCoordinator::HistoryHide(DownloadQueue* pDownloadQueue, HistoryInfo*
pNewHistoryInfo->SetTime(pHistoryInfo->GetTime());
(*pDownloadQueue->GetHistory())[pDownloadQueue->GetHistory()->size() - 1 - rindex] = pNewHistoryInfo;
DeleteDiskFiles(pHistoryInfo->GetNZBInfo());
DeleteQueuedFile(pHistoryInfo->GetNZBInfo()->GetQueuedFilename());
delete pHistoryInfo;
info("Collection %s removed from history", szNiceName);
}
void HistoryCoordinator::PrepareEdit(DownloadQueue* pDownloadQueue, IDList* pIDList, DownloadQueue::EEditAction eAction)
{
// First pass: when marking multiple items - mark them bad without performing the mark-logic,
// this will later (on second step) avoid moving other items to download queue, if they are marked bad too.
if (eAction == DownloadQueue::eaHistoryMarkBad)
{
for (IDList::iterator itID = pIDList->begin(); itID != pIDList->end(); itID++)
{
int iID = *itID;
HistoryInfo* pHistoryInfo = pDownloadQueue->GetHistory()->Find(iID);
if (pHistoryInfo && pHistoryInfo->GetKind() == HistoryInfo::hkNzb)
{
pHistoryInfo->GetNZBInfo()->SetMarkStatus(NZBInfo::ksBad);
}
}
}
}
bool HistoryCoordinator::EditList(DownloadQueue* pDownloadQueue, IDList* pIDList, DownloadQueue::EEditAction eAction, int iOffset, const char* szText)
{
bool bOK = false;
PrepareEdit(pDownloadQueue, pIDList, eAction);
for (IDList::iterator itID = pIDList->begin(); itID != pIDList->end(); itID++)
{
@@ -278,8 +269,6 @@ bool HistoryCoordinator::EditList(DownloadQueue* pDownloadQueue, IDList* pIDList
HistoryInfo* pHistoryInfo = *itHistory;
if (pHistoryInfo->GetID() == iID)
{
bOK = true;
switch (eAction)
{
case DownloadQueue::eaHistoryDelete:
@@ -297,15 +286,7 @@ bool HistoryCoordinator::EditList(DownloadQueue* pDownloadQueue, IDList* pIDList
break;
case DownloadQueue::eaHistorySetParameter:
bOK = HistorySetParameter(pHistoryInfo, szText);
break;
case DownloadQueue::eaHistorySetCategory:
bOK = HistorySetCategory(pHistoryInfo, szText);
break;
case DownloadQueue::eaHistorySetName:
bOK = HistorySetName(pHistoryInfo, szText);
HistorySetParameter(pHistoryInfo, szText);
break;
case DownloadQueue::eaHistorySetDupeKey:
@@ -316,15 +297,8 @@ bool HistoryCoordinator::EditList(DownloadQueue* pDownloadQueue, IDList* pIDList
break;
case DownloadQueue::eaHistoryMarkBad:
g_pDupeCoordinator->HistoryMark(pDownloadQueue, pHistoryInfo, NZBInfo::ksBad);
break;
case DownloadQueue::eaHistoryMarkGood:
g_pDupeCoordinator->HistoryMark(pDownloadQueue, pHistoryInfo, NZBInfo::ksGood);
break;
case DownloadQueue::eaHistoryMarkSuccess:
g_pDupeCoordinator->HistoryMark(pDownloadQueue, pHistoryInfo, NZBInfo::ksSuccess);
g_pDupeCoordinator->HistoryMark(pDownloadQueue, pHistoryInfo, eAction == DownloadQueue::eaHistoryMarkGood);
break;
default:
@@ -332,6 +306,7 @@ bool HistoryCoordinator::EditList(DownloadQueue* pDownloadQueue, IDList* pIDList
break;
}
bOK = true;
break;
}
}
@@ -354,7 +329,20 @@ void HistoryCoordinator::HistoryDelete(DownloadQueue* pDownloadQueue, HistoryLis
if (pHistoryInfo->GetKind() == HistoryInfo::hkNzb)
{
DeleteDiskFiles(pHistoryInfo->GetNZBInfo());
NZBInfo* pNZBInfo = pHistoryInfo->GetNZBInfo();
// delete parked files
if (g_pOptions->GetSaveQueue() && g_pOptions->GetServerMode())
{
for (FileList::iterator it = pNZBInfo->GetFileList()->begin(); it != pNZBInfo->GetFileList()->end(); it++)
{
FileInfo* pFileInfo = *it;
g_pDiskState->DiscardFile(pFileInfo);
}
}
pNZBInfo->GetFileList()->Clear();
DeleteQueuedFile(pNZBInfo->GetQueuedFilename());
}
if (pHistoryInfo->GetKind() == HistoryInfo::hkNzb &&
@@ -394,6 +382,7 @@ void HistoryCoordinator::HistoryReturn(DownloadQueue* pDownloadQueue, HistoryLis
char szNiceName[1024];
pHistoryInfo->GetName(szNiceName, 1024);
debug("Returning %s from history back to download queue", szNiceName);
bool bUnparked = false;
NZBInfo* pNZBInfo = NULL;
if (bReprocess && pHistoryInfo->GetKind() != HistoryInfo::hkNzb)
@@ -407,7 +396,6 @@ void HistoryCoordinator::HistoryReturn(DownloadQueue* pDownloadQueue, HistoryLis
pNZBInfo = pHistoryInfo->GetNZBInfo();
// unpark files
bool bUnparked = false;
for (FileList::iterator it = pNZBInfo->GetFileList()->begin(); it != pNZBInfo->GetFileList()->end(); it++)
{
FileInfo* pFileInfo = *it;
@@ -415,12 +403,6 @@ void HistoryCoordinator::HistoryReturn(DownloadQueue* pDownloadQueue, HistoryLis
bUnparked = true;
}
if (!(bUnparked || bReprocess))
{
warn("Could not return %s back from history to download queue: history item does not have any files left for download", szNiceName);
return;
}
pDownloadQueue->GetQueue()->push_front(pNZBInfo);
pHistoryInfo->DiscardNZBInfo();
@@ -431,16 +413,10 @@ void HistoryCoordinator::HistoryReturn(DownloadQueue* pDownloadQueue, HistoryLis
pNZBInfo->SetUnpackStatus(NZBInfo::usNone);
pNZBInfo->SetCleanupStatus(NZBInfo::csNone);
pNZBInfo->SetRenameStatus(NZBInfo::rsNone);
pNZBInfo->SetPostTotalSec(pNZBInfo->GetPostTotalSec() - pNZBInfo->GetUnpackSec());
pNZBInfo->SetUnpackSec(0);
if (ParParser::FindMainPars(pNZBInfo->GetDestDir(), NULL))
if (ParCoordinator::FindMainPars(pNZBInfo->GetDestDir(), NULL))
{
pNZBInfo->SetParStatus(NZBInfo::psNone);
pNZBInfo->SetPostTotalSec(pNZBInfo->GetPostTotalSec() - pNZBInfo->GetParSec());
pNZBInfo->SetParSec(0);
pNZBInfo->SetRepairSec(0);
pNZBInfo->SetParFull(false);
}
}
pNZBInfo->SetDeleteStatus(NZBInfo::dsNone);
@@ -452,21 +428,28 @@ void HistoryCoordinator::HistoryReturn(DownloadQueue* pDownloadQueue, HistoryLis
{
pNZBInfo->SetMoveStatus(NZBInfo::msNone);
}
pNZBInfo->SetReprocess(bReprocess);
}
if (pHistoryInfo->GetKind() == HistoryInfo::hkUrl)
{
pNZBInfo = pHistoryInfo->GetNZBInfo();
NZBInfo* pNZBInfo = pHistoryInfo->GetNZBInfo();
pHistoryInfo->DiscardNZBInfo();
pNZBInfo->SetUrlStatus(NZBInfo::lsNone);
pNZBInfo->SetDeleteStatus(NZBInfo::dsNone);
pDownloadQueue->GetQueue()->push_front(pNZBInfo);
bUnparked = true;
}
pDownloadQueue->GetHistory()->erase(itHistory);
// the object "pHistoryInfo" is released few lines later, after the call to "NZBDownloaded"
pNZBInfo->PrintMessage(Message::mkInfo, "%s returned from history back to download queue", szNiceName);
if (bUnparked || bReprocess)
{
pDownloadQueue->GetHistory()->erase(itHistory);
// the object "pHistoryInfo" is released few lines later, after the call to "NZBDownloaded"
info("%s returned from history back to download queue", szNiceName);
}
else
{
warn("Could not return %s back from history to download queue: history item does not have any files left for download", szNiceName);
}
if (bReprocess)
{
@@ -475,46 +458,34 @@ void HistoryCoordinator::HistoryReturn(DownloadQueue* pDownloadQueue, HistoryLis
g_pPrePostProcessor->NZBDownloaded(pDownloadQueue, pNZBInfo);
}
delete pHistoryInfo;
if (bUnparked || bReprocess)
{
delete pHistoryInfo;
}
}
void HistoryCoordinator::HistoryRedownload(DownloadQueue* pDownloadQueue, HistoryList::iterator itHistory,
HistoryInfo* pHistoryInfo, bool bRestorePauseState)
{
if (pHistoryInfo->GetKind() == HistoryInfo::hkUrl)
{
HistoryReturn(pDownloadQueue, itHistory, pHistoryInfo, false);
return;
}
if (pHistoryInfo->GetKind() != HistoryInfo::hkNzb)
{
char szNiceName[1024];
pHistoryInfo->GetName(szNiceName, 1024);
error("Could not return %s from history back to queue: history item has wrong type", szNiceName);
return;
}
NZBInfo* pNZBInfo = pHistoryInfo->GetNZBInfo();
bool bPaused = bRestorePauseState && pNZBInfo->GetDeletePaused();
if (!Util::FileExists(pNZBInfo->GetQueuedFilename()))
{
error("Could not return %s from history back to queue: could not find source nzb-file %s",
error("Could not return collection %s from history back to queue: could not find source nzb-file %s",
pNZBInfo->GetName(), pNZBInfo->GetQueuedFilename());
return;
}
NZBFile* pNZBFile = new NZBFile(pNZBInfo->GetQueuedFilename(), "");
if (!pNZBFile->Parse())
NZBFile* pNZBFile = NZBFile::Create(pNZBInfo->GetQueuedFilename(), "");
if (pNZBFile == NULL)
{
error("Could not return %s from history back to queue: could not parse nzb-file",
error("Could not return collection %s from history back to queue: could not parse nzb-file",
pNZBInfo->GetName());
delete pNZBFile;
return;
}
info("Returning %s from history back to queue", pNZBInfo->GetName());
info("Returning collection %s from history back to queue", pNZBInfo->GetName());
for (FileList::iterator it = pNZBFile->GetNZBInfo()->GetFileList()->begin(); it != pNZBFile->GetNZBInfo()->GetFileList()->end(); it++)
{
@@ -543,20 +514,11 @@ void HistoryCoordinator::HistoryRedownload(DownloadQueue* pDownloadQueue, Histor
}
}
g_pDiskState->DiscardFiles(pNZBInfo);
// reset status fields (which are not reset by "HistoryReturn")
pNZBInfo->SetMoveStatus(NZBInfo::msNone);
pNZBInfo->SetUnpackCleanedUpDisk(false);
pNZBInfo->SetParStatus(NZBInfo::psNone);
pNZBInfo->SetRenameStatus(NZBInfo::rsNone);
pNZBInfo->SetDownloadedSize(0);
pNZBInfo->SetDownloadSec(0);
pNZBInfo->SetPostTotalSec(0);
pNZBInfo->SetParSec(0);
pNZBInfo->SetRepairSec(0);
pNZBInfo->SetUnpackSec(0);
pNZBInfo->SetExtraParBlocks(0);
pNZBInfo->ClearCompletedFiles();
pNZBInfo->GetServerStats()->Clear();
pNZBInfo->GetCurrentServerStats()->Clear();
@@ -568,10 +530,14 @@ void HistoryCoordinator::HistoryRedownload(DownloadQueue* pDownloadQueue, Histor
HistoryReturn(pDownloadQueue, itHistory, pHistoryInfo, false);
g_pPrePostProcessor->NZBAdded(pDownloadQueue, pNZBInfo);
if (!bPaused && g_pOptions->GetParCheck() != Options::pcForce)
{
pDownloadQueue->EditEntry(pNZBInfo->GetID(),
DownloadQueue::eaGroupPauseExtraPars, 0, NULL);
}
}
bool HistoryCoordinator::HistorySetParameter(HistoryInfo* pHistoryInfo, const char* szText)
void HistoryCoordinator::HistorySetParameter(HistoryInfo* pHistoryInfo, const char* szText)
{
char szNiceName[1024];
pHistoryInfo->GetName(szNiceName, 1024);
@@ -580,7 +546,7 @@ bool HistoryCoordinator::HistorySetParameter(HistoryInfo* pHistoryInfo, const ch
if (!(pHistoryInfo->GetKind() == HistoryInfo::hkNzb || pHistoryInfo->GetKind() == HistoryInfo::hkUrl))
{
error("Could not set post-process-parameter for %s: history item has wrong type", szNiceName);
return false;
return;
}
char* szStr = strdup(szText);
@@ -598,49 +564,6 @@ bool HistoryCoordinator::HistorySetParameter(HistoryInfo* pHistoryInfo, const ch
}
free(szStr);
return true;
}
bool HistoryCoordinator::HistorySetCategory(HistoryInfo* pHistoryInfo, const char* szText)
{
char szNiceName[1024];
pHistoryInfo->GetName(szNiceName, 1024);
debug("Setting category '%s' for '%s'", szText, szNiceName);
if (!(pHistoryInfo->GetKind() == HistoryInfo::hkNzb || pHistoryInfo->GetKind() == HistoryInfo::hkUrl))
{
error("Could not set category for %s: history item has wrong type", szNiceName);
return false;
}
pHistoryInfo->GetNZBInfo()->SetCategory(szText);
return true;
}
bool HistoryCoordinator::HistorySetName(HistoryInfo* pHistoryInfo, const char* szText)
{
char szNiceName[1024];
pHistoryInfo->GetName(szNiceName, 1024);
debug("Setting name '%s' for '%s'", szText, szNiceName);
if (Util::EmptyStr(szText))
{
error("Could not rename %s. The new name cannot be empty", szNiceName);
return false;
}
if (pHistoryInfo->GetKind() == HistoryInfo::hkNzb || pHistoryInfo->GetKind() == HistoryInfo::hkUrl)
{
pHistoryInfo->GetNZBInfo()->SetName(szText);
}
else if (pHistoryInfo->GetKind() == HistoryInfo::hkDup)
{
pHistoryInfo->GetDupInfo()->SetName(szText);
}
return true;
}
void HistoryCoordinator::HistorySetDupeParam(HistoryInfo* pHistoryInfo, DownloadQueue::EEditAction eAction, const char* szText)

View File

@@ -1,7 +1,7 @@
/*
* This file is part of nzbget
*
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -27,36 +27,28 @@
#define HISTORYCOORDINATOR_H
#include "DownloadInfo.h"
#include "Service.h"
class HistoryCoordinator : public Service
class HistoryCoordinator
{
private:
void HistoryDelete(DownloadQueue* pDownloadQueue, HistoryList::iterator itHistory, HistoryInfo* pHistoryInfo, bool bFinal);
void HistoryReturn(DownloadQueue* pDownloadQueue, HistoryList::iterator itHistory, HistoryInfo* pHistoryInfo, bool bReprocess);
void HistoryRedownload(DownloadQueue* pDownloadQueue, HistoryList::iterator itHistory, HistoryInfo* pHistoryInfo, bool bRestorePauseState);
bool HistorySetParameter(HistoryInfo* pHistoryInfo, const char* szText);
void HistorySetParameter(HistoryInfo* pHistoryInfo, const char* szText);
void HistorySetDupeParam(HistoryInfo* pHistoryInfo, DownloadQueue::EEditAction eAction, const char* szText);
bool HistorySetCategory(HistoryInfo* pHistoryInfo, const char* szText);
bool HistorySetName(HistoryInfo* pHistoryInfo, const char* szText);
void HistoryTransformToDup(DownloadQueue* pDownloadQueue, HistoryInfo* pHistoryInfo, int rindex);
void SaveQueue(DownloadQueue* pDownloadQueue);
void PrepareEdit(DownloadQueue* pDownloadQueue, IDList* pIDList, DownloadQueue::EEditAction eAction);
protected:
virtual int ServiceInterval() { return 600000; }
virtual void ServiceWork();
public:
HistoryCoordinator();
virtual ~HistoryCoordinator();
void AddToHistory(DownloadQueue* pDownloadQueue, NZBInfo* pNZBInfo);
bool EditList(DownloadQueue* pDownloadQueue, IDList* pIDList, DownloadQueue::EEditAction eAction, int iOffset, const char* szText);
void DeleteDiskFiles(NZBInfo* pNZBInfo);
void DeleteQueuedFile(const char* szQueuedFile);
void HistoryHide(DownloadQueue* pDownloadQueue, HistoryInfo* pHistoryInfo, int rindex);
void Redownload(DownloadQueue* pDownloadQueue, HistoryInfo* pHistoryInfo);
void IntervalCheck();
void Cleanup();
};
extern HistoryCoordinator* g_pHistoryCoordinator;
#endif

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -54,6 +54,9 @@ using namespace MSXML;
#include "DiskState.h"
#include "Util.h"
extern Options* g_pOptions;
extern DiskState* g_pDiskState;
NZBFile::NZBFile(const char* szFileName, const char* szCategory)
{
debug("Creating NZBFile");
@@ -169,32 +172,10 @@ void NZBFile::AddFileInfo(FileInfo* pFileInfo)
void NZBFile::ParseSubject(FileInfo* pFileInfo, bool TryQuotes)
{
// Example subject: some garbage "title" yEnc (10/99)
// strip the "yEnc (10/99)"-suffix
char szSubject[1024];
strncpy(szSubject, pFileInfo->GetSubject(), sizeof(szSubject));
szSubject[1024-1] = '\0';
char* end = szSubject + strlen(szSubject) - 1;
if (*end == ')')
{
end--;
while (strchr("0123456789", *end) && end > szSubject) end--;
if (*end == '/')
{
end--;
while (strchr("0123456789", *end) && end > szSubject) end--;
if (end - 6 > szSubject && !strncmp(end - 6, " yEnc (", 7))
{
end[-6] = '\0';
}
}
}
if (TryQuotes)
{
// try to use the filename in quatation marks
char* p = szSubject;
char* p = (char*)pFileInfo->GetSubject();
char* start = strchr(p, '\"');
if (start)
{
@@ -226,7 +207,7 @@ void NZBFile::ParseSubject(FileInfo* pFileInfo, bool TryQuotes)
tokens.clear();
// tokenizing
char* p = szSubject;
char* p = (char*)pFileInfo->GetSubject();
char* start = p;
bool quot = false;
while (true)
@@ -480,7 +461,7 @@ void NZBFile::ReadPassword()
// obtain file size.
fseek(pFile , 0 , SEEK_END);
int iSize = (int)ftell(pFile);
int iSize = ftell(pFile);
rewind(pFile);
// reading first 4KB of the file
@@ -515,7 +496,7 @@ void NZBFile::ReadPassword()
}
#ifdef WIN32
bool NZBFile::Parse()
NZBFile* NZBFile::Create(const char* szFileName, const char* szCategory)
{
CoInitialize(NULL);
@@ -525,7 +506,7 @@ bool NZBFile::Parse()
hr = doc.CreateInstance(MSXML::CLSID_DOMDocument);
if (FAILED(hr))
{
return false;
return NULL;
}
// Load the XML document file...
@@ -534,8 +515,8 @@ bool NZBFile::Parse()
doc->put_async(VARIANT_FALSE);
// filename needs to be properly encoded
char* szURL = (char*)malloc(strlen(m_szFileName)*3 + 1);
EncodeURL(m_szFileName, szURL);
char* szURL = (char*)malloc(strlen(szFileName)*3 + 1);
EncodeURL(szFileName, szURL);
debug("url=\"%s\"", szURL);
_variant_t v(szURL);
free(szURL);
@@ -545,33 +526,22 @@ bool NZBFile::Parse()
{
_bstr_t r(doc->GetparseError()->reason);
const char* szErrMsg = r;
char szMessageText[1024];
snprintf(szMessageText, 1024, "Error parsing nzb-file %s: %s", Util::BaseFileName(m_szFileName), szErrMsg);
szMessageText[1024-1] = '\0';
m_pNZBInfo->AddMessage(Message::mkError, szMessageText);
return false;
error("Error parsing nzb-file: %s", szErrMsg);
return NULL;
}
if (!ParseNZB(doc))
NZBFile* pFile = new NZBFile(szFileName, szCategory);
if (pFile->ParseNZB(doc))
{
return false;
pFile->ProcessFiles();
}
if (GetNZBInfo()->GetFileList()->empty())
else
{
char szMessageText[1024];
snprintf(szMessageText, 1024, "Error parsing nzb-file %s: file has no content", Util::BaseFileName(m_szFileName));
szMessageText[1024-1] = '\0';
m_pNZBInfo->AddMessage(Message::mkError, szMessageText);
return false;
delete pFile;
pFile = NULL;
}
ProcessFiles();
return true;
return pFile;
}
void NZBFile::EncodeURL(const char* szFilename, char* szURL)
@@ -669,8 +639,10 @@ bool NZBFile::ParseNZB(IUnknown* nzb)
#else
bool NZBFile::Parse()
NZBFile* NZBFile::Create(const char* szFileName, const char* szCategory)
{
NZBFile* pFile = new NZBFile(szFileName, szCategory);
xmlSAXHandler SAX_handler = {0};
SAX_handler.startElement = reinterpret_cast<startElementSAXFunc>(SAX_StartElement);
SAX_handler.endElement = reinterpret_cast<endElementSAXFunc>(SAX_EndElement);
@@ -678,39 +650,26 @@ bool NZBFile::Parse()
SAX_handler.error = reinterpret_cast<errorSAXFunc>(SAX_error);
SAX_handler.getEntity = reinterpret_cast<getEntitySAXFunc>(SAX_getEntity);
m_bIgnoreNextError = false;
pFile->m_bIgnoreNextError = false;
int ret = xmlSAXUserParseFile(&SAX_handler, this, m_szFileName);
int ret = xmlSAXUserParseFile(&SAX_handler, pFile, szFileName);
if (ret != 0)
if (ret == 0)
{
char szMessageText[1024];
snprintf(szMessageText, 1024, "Error parsing nzb-file %s", Util::BaseFileName(m_szFileName));
szMessageText[1024-1] = '\0';
m_pNZBInfo->AddMessage(Message::mkError, szMessageText);
return false;
pFile->ProcessFiles();
}
if (m_pNZBInfo->GetFileList()->empty())
else
{
char szMessageText[1024];
snprintf(szMessageText, 1024, "Error parsing nzb-file %s: file has no content", Util::BaseFileName(m_szFileName));
szMessageText[1024-1] = '\0';
m_pNZBInfo->AddMessage(Message::mkError, szMessageText);
return false;
error("Failed to parse nzb-file");
delete pFile;
pFile = NULL;
}
ProcessFiles();
return true;
return pFile;
}
void NZBFile::Parse_StartElement(const char *name, const char **atts)
{
char szTagAttrMessage[1024];
snprintf(szTagAttrMessage, 1024, "Malformed nzb-file, tag <%s> must have attributes", name);
szTagAttrMessage[1024-1] = '\0';
if (m_szTagContent)
{
free(m_szTagContent);
@@ -725,7 +684,7 @@ void NZBFile::Parse_StartElement(const char *name, const char **atts)
if (!atts)
{
m_pNZBInfo->AddMessage(Message::mkWarning, szTagAttrMessage);
warn("Malformed nzb-file, tag <%s> must have attributes", name);
return;
}
@@ -747,13 +706,13 @@ void NZBFile::Parse_StartElement(const char *name, const char **atts)
{
if (!m_pFileInfo)
{
m_pNZBInfo->AddMessage(Message::mkWarning, "Malformed nzb-file, tag <segment> without tag <file>");
warn("Malformed nzb-file, tag <segment> without tag <file>");
return;
}
if (!atts)
{
m_pNZBInfo->AddMessage(Message::mkWarning, szTagAttrMessage);
warn("Malformed nzb-file, tag <%s> must have attributes", name);
return;
}
@@ -787,7 +746,7 @@ void NZBFile::Parse_StartElement(const char *name, const char **atts)
{
if (!atts)
{
m_pNZBInfo->AddMessage(Message::mkWarning, szTagAttrMessage);
warn("Malformed nzb-file, tag <%s> must have attributes", name);
return;
}
m_bPassword = atts[0] && atts[1] && !strcmp("type", atts[0]) && !strcmp("password", atts[1]);
@@ -900,7 +859,7 @@ void* NZBFile::SAX_getEntity(NZBFile* pFile, const char * name)
xmlEntityPtr e = xmlGetPredefinedEntity((xmlChar* )name);
if (!e)
{
pFile->GetNZBInfo()->AddMessage(Message::mkWarning, "entity not found");
warn("entity not found");
pFile->m_bIgnoreNextError = true;
}
@@ -924,10 +883,6 @@ void NZBFile::SAX_error(NZBFile* pFile, const char *msg, ...)
// remove trailing CRLF
for (char* pend = szErrMsg + strlen(szErrMsg) - 1; pend >= szErrMsg && (*pend == '\n' || *pend == '\r' || *pend == ' '); pend--) *pend = '\0';
char szTextMessage[1024];
snprintf(szTextMessage, 1024, "Error parsing nzb-file: %s", szErrMsg);
szTextMessage[1024-1] = '\0';
pFile->GetNZBInfo()->AddMessage(Message::mkError, szTextMessage);
error("Error parsing nzb-file: %s", szErrMsg);
}
#endif

View File

@@ -2,7 +2,7 @@
* This file is part of nzbget
*
* Copyright (C) 2004 Sven Henkel <sidddy@users.sourceforge.net>
* Copyright (C) 2007-2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
* Copyright (C) 2007-2013 Andrey Prygunkov <hugbug@users.sourceforge.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -41,6 +41,7 @@ private:
char* m_szFileName;
char* m_szPassword;
NZBFile(const char* szFileName, const char* szCategory);
void AddArticle(FileInfo* pFileInfo, ArticleInfo* pArticleInfo);
void AddFileInfo(FileInfo* pFileInfo);
void ParseSubject(FileInfo* pFileInfo, bool TryQuotes);
@@ -71,9 +72,8 @@ private:
#endif
public:
NZBFile(const char* szFileName, const char* szCategory);
~NZBFile();
bool Parse();
virtual ~NZBFile();
static NZBFile* Create(const char* szFileName, const char* szCategory);
const char* GetFileName() const { return m_szFileName; }
NZBInfo* GetNZBInfo() { return m_pNZBInfo; }
const char* GetPassword() { return m_szPassword; }

Some files were not shown because too many files have changed in this diff Show More