mirror of
https://github.com/kiwix/kiwix-tools.git
synced 2026-01-10 06:58:00 -05:00
Compare commits
47 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cbe8f1df87 | ||
|
|
2fe5ec1d7c | ||
|
|
eba80b92ef | ||
|
|
c1e635bd4e | ||
|
|
c8be8fbad8 | ||
|
|
640f907fb2 | ||
|
|
eb407956b9 | ||
|
|
422dde9ff2 | ||
|
|
f691f85724 | ||
|
|
fd4f228a41 | ||
|
|
bf40d4ff91 | ||
|
|
74fecd34e6 | ||
|
|
881fe9f41d | ||
|
|
c23e0bc38a | ||
|
|
d4d32aa2e8 | ||
|
|
4a88c44626 | ||
|
|
0cc5b18488 | ||
|
|
88a32a152f | ||
|
|
4b8c8c5f6f | ||
|
|
f0568ff4a7 | ||
|
|
ccdfe9839d | ||
|
|
221f3ef340 | ||
|
|
0785636ca5 | ||
|
|
a3429fdc88 | ||
|
|
f99b6965e9 | ||
|
|
228402b505 | ||
|
|
a9f8364333 | ||
|
|
9f06c9c5eb | ||
|
|
0efbb2461d | ||
|
|
b251e18af1 | ||
|
|
5c040d3ee6 | ||
|
|
bb1afb5120 | ||
|
|
8fccbc4c99 | ||
|
|
d0dc9ac81b | ||
|
|
f9edd75f6c | ||
|
|
9571148375 | ||
|
|
282b85c341 | ||
|
|
4c3acd06de | ||
|
|
4cd9d78d21 | ||
|
|
efd4a1434e | ||
|
|
dfc601dacf | ||
|
|
7c254544ca | ||
|
|
b22ee94f10 | ||
|
|
3766c4882b | ||
|
|
26da54f9c3 | ||
|
|
4433421c48 | ||
|
|
c00f0be7ef |
11
.travis.yml
11
.travis.yml
@@ -2,22 +2,11 @@ language: cpp
|
||||
dist: trusty
|
||||
sudo: required
|
||||
cache: ccache
|
||||
before_install:
|
||||
- eval "${MATRIX_EVAL}"
|
||||
- ${CXX} --version
|
||||
install: travis/install_deps.sh
|
||||
script: travis/compile.sh
|
||||
env:
|
||||
global:
|
||||
- MATRIX_EVAL="CC=gcc-5 && CXX=g++-5"
|
||||
matrix:
|
||||
- PLATFORM="native_static"
|
||||
- PLATFORM="native_dyn"
|
||||
- PLATFORM="win32_static"
|
||||
- PLATFORM="win32_dyn"
|
||||
addons:
|
||||
apt:
|
||||
sources:
|
||||
- ubuntu-toolchain-r-test
|
||||
packages:
|
||||
- g++-5
|
||||
|
||||
50
Changelog
50
Changelog
@@ -1,3 +1,51 @@
|
||||
kiwix-tools 0.7.0
|
||||
=================
|
||||
|
||||
* Adapt to kiwix-lib new API
|
||||
|
||||
kiwix-serve
|
||||
-----------
|
||||
|
||||
* Dumps only valid books in the opdsfeed.
|
||||
* Allow the opds feed to be filtered by lang and paginated.
|
||||
|
||||
kiwix-manage
|
||||
------------
|
||||
|
||||
* Add a download command to download a remote book locally
|
||||
* Book are referenced by bookId not index.
|
||||
* No more indexType option as it is always XAPIAN.
|
||||
|
||||
kiwix-tools 0.6.1
|
||||
=================
|
||||
|
||||
kiwix-serve
|
||||
-----------
|
||||
|
||||
* Update README.
|
||||
* Fix crash when `--library` flag is provided without value.
|
||||
* Correctly handle mimetype of file without extension on 64bits.
|
||||
* Minor fixes
|
||||
|
||||
kiwix-tools 0.6.0
|
||||
=================
|
||||
|
||||
* remove kiwix-install tool.
|
||||
|
||||
kiwix-serve
|
||||
-----------
|
||||
|
||||
* Improved taskbar #160
|
||||
* Fix global page when using the option `--nosearchbar`
|
||||
* Return 404 for missing resources
|
||||
* Fix compilation for gcc 4.8.
|
||||
|
||||
kiwix-manage
|
||||
------------
|
||||
|
||||
* Returns proper exit code (not always 0)
|
||||
|
||||
|
||||
kiwix-tools 0.5.0
|
||||
=================
|
||||
|
||||
@@ -5,7 +53,7 @@ kiwix-tools 0.5.0
|
||||
* Compile without warning.
|
||||
|
||||
|
||||
kiwix-server
|
||||
kiwix-serve
|
||||
------------
|
||||
|
||||
* Serve metadata information using the "/meta" url.
|
||||
|
||||
100
README.md
100
README.md
@@ -1,14 +1,14 @@
|
||||
Kiwix tools
|
||||
===========
|
||||
|
||||
The Kiwix tools gathers kiwix command line tools.
|
||||
The Kiwix tools is a collection of Kiwix related command line tools.
|
||||
|
||||
Disclaimer
|
||||
----------
|
||||
|
||||
This document assumes you have a little knowledge about software
|
||||
compilation. If you experience difficulties with the dependencies or
|
||||
with the Kiwix libary compilation itself, we recommend to have a look
|
||||
with the Kiwix tools compilation itself, we recommend to have a look
|
||||
to [kiwix-build](https://github.com/kiwix/kiwix-build).
|
||||
|
||||
Preamble
|
||||
@@ -22,12 +22,12 @@ on recent releases of Ubuntu and Fedora.
|
||||
Dependencies
|
||||
------------
|
||||
|
||||
The Kiwix tools rely on a few third parts software libraries. They
|
||||
are prerequisites to the Kiwix library compilation. Following
|
||||
The Kiwix tools rely on a few third party software libraries. They are
|
||||
prerequisites to the Kiwix tools compilation. Therefore, following
|
||||
libraries need to be available:
|
||||
|
||||
* Kiwix lib ....................... https://github.com/kiwix/kiwix-lib
|
||||
(no package for now)
|
||||
(no package so far)
|
||||
* Libmicrohttpd .......... https://www.gnu.org/software/libmicrohttpd/
|
||||
(package libmicrohttpd-dev on Ubuntu)
|
||||
* CTPP2 ..................................... http://ctpp.havoc.ru/en/
|
||||
@@ -53,47 +53,26 @@ If you compile manually Libmicrohttpd, you might need to compile it
|
||||
without GNU TLS, a bug here will empeach further compilation of Kiwix
|
||||
tools otherwise.
|
||||
|
||||
Environnement
|
||||
Environment
|
||||
-------------
|
||||
|
||||
The Kiwix library builds using [Meson](http://mesonbuild.com/) version
|
||||
0.34 or higher. Meson relies itself on Ninja, pkg-config and few other
|
||||
compilation tools.
|
||||
|
||||
Install first the few common compilation tools:
|
||||
* Automake
|
||||
* Libtool
|
||||
* Virtualenv
|
||||
The Kiwix tools build using [Meson](http://mesonbuild.com/) version
|
||||
0.39 or higher. Meson relies itself on Ninja, pkg-config and few other
|
||||
compilation tools. Install them first:
|
||||
* Meson
|
||||
* Ninja
|
||||
* Pkg-config
|
||||
|
||||
Then install Meson itself:
|
||||
```
|
||||
virtualenv -p python3 ./ # Create virtualenv
|
||||
source bin/activate # Activate the virtualenv
|
||||
pip install meson # Install Meson
|
||||
hash -r # Refresh bash paths
|
||||
```
|
||||
|
||||
Finally download and build Ninja locally:
|
||||
```
|
||||
git clone git://github.com/ninja-build/ninja.git
|
||||
cd ninja
|
||||
git checkout release
|
||||
./configure.py --bootstrap
|
||||
mkdir ../bin
|
||||
cp ninja ../bin
|
||||
cd ..
|
||||
```
|
||||
These tools should be packaged if you use a cutting edge operating
|
||||
system. If not, have a look to the "Troubleshooting" section.
|
||||
|
||||
Compilation
|
||||
-----------
|
||||
|
||||
Once all dependencies are installed, you can compile kiwix-lib with:
|
||||
Once all dependencies are installed, you can compile Kiwix tools with:
|
||||
```
|
||||
mkdir build
|
||||
meson . build
|
||||
cd build
|
||||
ninja
|
||||
ninja -C build
|
||||
```
|
||||
|
||||
By default, it will compile dynamic linked libraries. If you want
|
||||
@@ -105,16 +84,55 @@ Depending of you system, `ninja` may be called `ninja-build`.
|
||||
Installation
|
||||
------------
|
||||
|
||||
If you want to install the Kiwix tools you just have compiled on your
|
||||
system, here we go:
|
||||
If you want to install the Kiwix tools, here we go:
|
||||
|
||||
```
|
||||
ninja install
|
||||
ninja -C build install
|
||||
```
|
||||
|
||||
You might need to run the command as root (or using 'sudo'), depending
|
||||
where you want to install the Kiwix tools. After the installation
|
||||
succeeded, you may need to run ldconfig (as root).
|
||||
|
||||
Uninstallation
|
||||
------------
|
||||
|
||||
If you want to uninstall the Kiwix tools:
|
||||
|
||||
```
|
||||
ninja -C build uninstall
|
||||
```
|
||||
|
||||
Like for the installation, you might need to run the command as root
|
||||
(or using 'sudo').
|
||||
|
||||
Troubleshooting
|
||||
---------------
|
||||
|
||||
If you need to install Meson "manually":
|
||||
```
|
||||
virtualenv -p python3 ./ # Create virtualenv
|
||||
source bin/activate # Activate the virtualenv
|
||||
pip3 install meson # Install Meson
|
||||
hash -r # Refresh bash paths
|
||||
```
|
||||
|
||||
If you need to install Ninja "manually":
|
||||
```
|
||||
git clone git://github.com/ninja-build/ninja.git
|
||||
cd ninja
|
||||
git checkout release
|
||||
./configure.py --bootstrap
|
||||
mkdir ../bin
|
||||
cp ninja ../bin
|
||||
cd ..
|
||||
```
|
||||
|
||||
You might need to run the command as root, depending where you want to
|
||||
install the libraries.
|
||||
If the compilation still fails, you might need to get a more recent
|
||||
version of a dependency than the one packaged by your Linux
|
||||
distribution. Try then with a source tarball distributed by the
|
||||
problematic upstream project or even directly from the source code
|
||||
repository.
|
||||
|
||||
License
|
||||
-------
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
project('kiwix-tools', 'cpp',
|
||||
version : '0.5.0',
|
||||
version : '0.7.0',
|
||||
license : 'GPL',
|
||||
default_options: ['c_std=c11', 'cpp_std=c++11', 'werror=true'])
|
||||
|
||||
@@ -11,7 +11,7 @@ if static_linkage
|
||||
endif
|
||||
|
||||
thread_dep = dependency('threads')
|
||||
kiwixlib_dep = dependency('kiwix', version:'>=2.0.0', static:static_linkage)
|
||||
kiwixlib_dep = dependency('kiwix', version:'>=3.0.0', static:static_linkage)
|
||||
microhttpd_dep = dependency('libmicrohttpd', static:static_linkage)
|
||||
z_dep = dependency('zlib', static:static_linkage)
|
||||
|
||||
|
||||
@@ -1,173 +0,0 @@
|
||||
/*
|
||||
* Copyright 2011-2014 Emmanuel Engelhart <kelson@kiwix.org>
|
||||
*
|
||||
* This program is free software; you can redistribute it and/or modify
|
||||
* it under the terms of the GNU General Public License as published by
|
||||
* the Free Software Foundation; either version 3 of the License, or
|
||||
* any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU General Public License
|
||||
* along with this program; if not, write to the Free Software
|
||||
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
|
||||
* MA 02110-1301, USA.
|
||||
*/
|
||||
|
||||
#include <getopt.h>
|
||||
#include <kiwix/common/pathTools.h>
|
||||
#include <kiwix/manager.h>
|
||||
#include <kiwix/reader.h>
|
||||
|
||||
enum supportedAction { NONE, ADDCONTENT };
|
||||
|
||||
void usage()
|
||||
{
|
||||
cout << "Usage: kiwix-install [--verbose] addcontent ZIM_PATH KIWIX_PATH"
|
||||
<< endl;
|
||||
exit(1);
|
||||
}
|
||||
|
||||
int main(int argc, char** argv)
|
||||
{
|
||||
/* Init the variables */
|
||||
const char* contentPath = NULL;
|
||||
const char* kiwixPath = NULL;
|
||||
supportedAction action = NONE;
|
||||
bool verboseFlag = false;
|
||||
int option_index = 0;
|
||||
int c = 0;
|
||||
|
||||
/* Argument parsing */
|
||||
while (42) {
|
||||
static struct option long_options[]
|
||||
= {{"verbose", no_argument, 0, 'v'}, {0, 0, 0, 0}};
|
||||
|
||||
if (c != -1) {
|
||||
c = getopt_long(argc, argv, "vi", long_options, &option_index);
|
||||
|
||||
switch (c) {
|
||||
case 'v':
|
||||
verboseFlag = true;
|
||||
break;
|
||||
}
|
||||
} else {
|
||||
if (optind < argc) {
|
||||
if (action == NONE) {
|
||||
string actionString = argv[optind++];
|
||||
if (actionString == "addcontent" || actionString == "ADDCONTENT") {
|
||||
action = ADDCONTENT;
|
||||
}
|
||||
} else if (contentPath == NULL) {
|
||||
contentPath = argv[optind++];
|
||||
} else if (kiwixPath == NULL) {
|
||||
kiwixPath = argv[optind++];
|
||||
} else {
|
||||
usage();
|
||||
}
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Check if we have enough arguments */
|
||||
if (contentPath == NULL || kiwixPath == NULL) {
|
||||
usage();
|
||||
}
|
||||
|
||||
/* Make the action */
|
||||
if (action == ADDCONTENT) {
|
||||
/* Check if the content path exists and is readable */
|
||||
if (verboseFlag) {
|
||||
std::cout << "Check if the ZIM file exists..." << std::endl;
|
||||
}
|
||||
if (!fileExists(contentPath)) {
|
||||
cerr << "The content path '" << contentPath
|
||||
<< "' does not exist or is not readable." << endl;
|
||||
exit(1);
|
||||
}
|
||||
|
||||
/* Check if this is a ZIM file */
|
||||
try {
|
||||
if (verboseFlag) {
|
||||
std::cout << "Check if the ZIM file is valid..." << std::endl;
|
||||
}
|
||||
kiwix::Reader* reader = new kiwix::Reader(contentPath);
|
||||
delete reader;
|
||||
} catch (exception& e) {
|
||||
cerr << "The content available at '" << contentPath
|
||||
<< "' is not a ZIM file." << endl;
|
||||
exit(1);
|
||||
}
|
||||
string contentFilename = getLastPathElement(contentPath);
|
||||
|
||||
/* Check if kiwixPath/kiwix/kiwix.exe exists */
|
||||
if (verboseFlag) {
|
||||
std::cout << "Check if the Kiwix path is valid..." << std::endl;
|
||||
}
|
||||
string kiwixBinaryPath = computeAbsolutePath(kiwixPath, "kiwix/kiwix.exe");
|
||||
if (!fileExists(kiwixBinaryPath)) {
|
||||
kiwixBinaryPath = computeAbsolutePath(kiwixPath, "kiwix/kiwix");
|
||||
if (!fileExists(kiwixBinaryPath)) {
|
||||
cerr << "Unable to find the Kiwix binary at '" << kiwixBinaryPath
|
||||
<< "[.exe]'." << endl;
|
||||
exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
/* Check if the directory "data" structure exists */
|
||||
if (verboseFlag) {
|
||||
std::cout << "Check the target data directory structure..." << std::endl;
|
||||
}
|
||||
string dataPath = computeAbsolutePath(kiwixPath, "data/");
|
||||
if (!fileExists(dataPath)) {
|
||||
makeDirectory(dataPath);
|
||||
}
|
||||
|
||||
/* Check if the directory "data/content" structure exists */
|
||||
string dataContentPath = computeAbsolutePath(kiwixPath, "data/content/");
|
||||
if (!fileExists(dataContentPath)) {
|
||||
makeDirectory(dataContentPath);
|
||||
}
|
||||
|
||||
/* Check if the directory "data/library" structure exists */
|
||||
string dataLibraryPath = computeAbsolutePath(kiwixPath, "data/library/");
|
||||
if (!fileExists(dataLibraryPath)) {
|
||||
makeDirectory(dataLibraryPath);
|
||||
}
|
||||
|
||||
/* Copy the file to the data/content directory */
|
||||
if (verboseFlag) {
|
||||
std::cout << "Copy ZIM file to the target directory..." << std::endl;
|
||||
}
|
||||
string newContentPath
|
||||
= computeAbsolutePath(dataContentPath, contentFilename);
|
||||
if (!fileExists(newContentPath)
|
||||
|| getFileSize(contentPath) != getFileSize(newContentPath)) {
|
||||
copyFile(contentPath, newContentPath);
|
||||
}
|
||||
|
||||
/* Add the file to the library.xml */
|
||||
if (verboseFlag) {
|
||||
std::cout << "Create the library XML file..." << std::endl;
|
||||
}
|
||||
kiwix::Manager libraryManager;
|
||||
string libraryPath
|
||||
= computeAbsolutePath(dataLibraryPath, contentFilename + ".xml");
|
||||
string bookId = libraryManager.addBookFromPathAndGetId(
|
||||
newContentPath, "../content/" + contentFilename, "", false);
|
||||
if (!bookId.empty()) {
|
||||
libraryManager.setCurrentBookId(bookId);
|
||||
libraryManager.writeFile(libraryPath);
|
||||
} else {
|
||||
cerr << "Unable to build or save library file '" << libraryPath << "'"
|
||||
<< endl;
|
||||
}
|
||||
}
|
||||
|
||||
exit(0);
|
||||
}
|
||||
@@ -1,4 +0,0 @@
|
||||
executable('kiwix-install', ['kiwix-install.cpp'],
|
||||
dependencies:all_deps,
|
||||
install:true,
|
||||
install_rpath: join_paths(get_option('prefix'), get_option('libdir')))
|
||||
@@ -1,40 +0,0 @@
|
||||
.TH KIWIX 1 "21 May 2012"
|
||||
.SH NAME
|
||||
kiwix\-install \- Installeur de fichiers ZIM
|
||||
.SH SYNOPSIS
|
||||
.IX Header SYNOPSIS
|
||||
kiwix\-install [\-\-verbose|-v] [\-\-backend|\-b=xapian|clucene] [\-\-buildIndex|\-i] addcontent ZIM_PATH KIWIX_PATH
|
||||
.SH DESCRIPTION
|
||||
.PP
|
||||
Créé une arborescence contenant Kiwix et des données pour redistribution.
|
||||
.br
|
||||
Si demandé, créé aussi un index plein texte.
|
||||
|
||||
.TP
|
||||
\fB\-\-verbose\fR
|
||||
Active le mode verbeux de la sortie.
|
||||
|
||||
.TP
|
||||
\fB\-\-backend=xapian|clucene\fR
|
||||
Séléctionne un moteur d'indexation.
|
||||
|
||||
.TP
|
||||
\fB\-\-buildIndex\fR
|
||||
Créer un index plein texte pour le fichier ZIM.
|
||||
|
||||
.TP
|
||||
\fBZIM_PATH\fR
|
||||
Chemin du fichier de contenu ZIM.
|
||||
|
||||
.TP
|
||||
\fBKIWIX_PATH\fR
|
||||
Chemin d'accès dossier kiwix/.
|
||||
.br
|
||||
Le chemin doit contenit le binaire statique de Kiwix.
|
||||
|
||||
.SH SEE ALSO
|
||||
kiwix(1) kiwix\-serve(1) kiwix\-manage(1)
|
||||
.SH AUTHOR
|
||||
Emmanuel Engelhart <kelson@kiwix.org>
|
||||
.br
|
||||
Vasudev Kamath <kamathvasudev@gmail.com> (Manual)
|
||||
@@ -1,52 +0,0 @@
|
||||
.TH KIWIX 1 "12 juin 2012"
|
||||
.SH NAME
|
||||
Kiwix \- Lecteur hors-ligne de fichiers ZIM
|
||||
.SH SYNOPSIS
|
||||
.B kiwix [-jsconsole] [-articleByUrl] [-articleByTitle] [FILE]
|
||||
.SH DESCRIPTION
|
||||
.PP
|
||||
Kiwix est un lecteur de content au format ZIM.
|
||||
.br
|
||||
Les fichiers ZIM sont des archives de contenus compressés (généralement HTML)
|
||||
.br
|
||||
Les fichiers ZIM les plus populaires sont ceux de Wikipédia et Wikileaks.
|
||||
|
||||
.TP
|
||||
\fB\-jsconsole\fR
|
||||
Active la console de debogage JavaScript de Xulrunner.
|
||||
\fB\-articleByUrl url\fR
|
||||
Ouvre un article en particulier en renseignant son url. FILE doit être donné.
|
||||
.TP
|
||||
\fB\-articleByTitle title\fR
|
||||
Ouvre un article en particulier en renseignant son titre. FILE doit être donné.
|
||||
|
||||
.PP
|
||||
Fonctionnalités:
|
||||
* Lecteur de ZIM natif
|
||||
* Moteur de recherche plein texte
|
||||
* Signets et Notes
|
||||
* Serveur Web pour diffuser les fichiers ZIM sur le réseau
|
||||
* Export HTML et PDF
|
||||
* Traduits dans de nombreuses langues
|
||||
* Suggestions de recherche
|
||||
* Indexation des fichiers ZIM
|
||||
* Onglets de navigation
|
||||
* Bibliothèque de contenus intégrée
|
||||
|
||||
.SH SEE ALSO
|
||||
kiwix\-install(1) kiwix\-serve(1)
|
||||
.br
|
||||
kiwix\-manage(1)
|
||||
|
||||
|
||||
.SH TROUBLESHOOTING
|
||||
Aller à http://reportabug.kiwix.org pour plus de détails sur comment rapporter un bogue.
|
||||
|
||||
.SH AUTHORS
|
||||
Emmanuel Engelhart <kelson@kiwix.org>
|
||||
Guillaume Duhamel <gduhamel@linterweb.com>
|
||||
Fabien Coullon <fcoulon@linterweb.com>
|
||||
Renaud Gaudin <rgaudin@gmail.com>
|
||||
Wilfredo Rodriguez <wilfredor@kiwix.org>
|
||||
.br
|
||||
Vasudev Kamath <kamathvasudev@gmail.com> (Manual)
|
||||
@@ -1,4 +1,3 @@
|
||||
install_man('kiwix-install.1',
|
||||
'kiwix-manage.1',
|
||||
install_man('kiwix-manage.1',
|
||||
'kiwix-serve.1',
|
||||
install_dir:get_option('mandir')+'/fr/man1')
|
||||
install_dir:get_option('mandir')+'/fr/man1')
|
||||
|
||||
@@ -1,40 +0,0 @@
|
||||
.TH KIWIX 1 "21 May 2012"
|
||||
.SH NAME
|
||||
kiwix\-install \- Kiwix ZIM Installer
|
||||
.SH SYNOPSIS
|
||||
.IX Header SYNOPSIS
|
||||
kiwix\-install [\-\-verbose|-v] [\-\-backend|\-b=xapian|clucene] [\-\-buildIndex|\-i] addcontent ZIM_PATH KIWIX_PATH
|
||||
.SH DESCRIPTION
|
||||
.PP
|
||||
Creates a standalone Kiwix + data hierarchy.
|
||||
.br
|
||||
If specified, also creates a full\-text index.
|
||||
|
||||
.TP
|
||||
\fB\-\-verbose\fR
|
||||
Enable verbose output.
|
||||
|
||||
.TP
|
||||
\fB\-\-backend=xapian|clucene\fR
|
||||
Select an indexing backend.
|
||||
|
||||
.TP
|
||||
\fB\-\-buildIndex\fR
|
||||
Build an index for the ZIM file.
|
||||
|
||||
.TP
|
||||
\fBZIM_PATH\fR
|
||||
ZIM content file path.
|
||||
|
||||
.TP
|
||||
\fBKIWIX_PATH\fR
|
||||
Path to the kiwix/ folder.
|
||||
.br
|
||||
The path must contain the kiwix standalone binary.
|
||||
|
||||
.SH SEE ALSO
|
||||
kiwix(1) kiwix\-serve(1) kiwix\-manage(1)
|
||||
.SH AUTHOR
|
||||
Emmanuel Engelhart <kelson@kiwix.org>
|
||||
.br
|
||||
Vasudev Kamath <kamathvasudev@gmail.com> (Manual)
|
||||
@@ -1,4 +1,3 @@
|
||||
install_man('kiwix-install.1',
|
||||
'kiwix-manage.1',
|
||||
install_man('kiwix-manage.1',
|
||||
'kiwix-serve.1')
|
||||
subdir('fr')
|
||||
|
||||
@@ -22,33 +22,39 @@
|
||||
#endif
|
||||
#include <getopt.h>
|
||||
#include <kiwix/common/pathTools.h>
|
||||
#include <kiwix/common/stringTools.h>
|
||||
#include <kiwix/manager.h>
|
||||
#include <kiwix/downloader.h>
|
||||
#include <cstdlib>
|
||||
#include <iostream>
|
||||
|
||||
#include <thread>
|
||||
#include <chrono>
|
||||
|
||||
using namespace std;
|
||||
|
||||
enum supportedAction { NONE, ADD, SHOW, REMOVE };
|
||||
enum supportedAction { NONE, ADD, SHOW, REMOVE, DOWNLOAD };
|
||||
|
||||
void show(kiwix::Library library)
|
||||
void show(kiwix::Library* library)
|
||||
{
|
||||
std::vector<kiwix::Book>::iterator itr;
|
||||
auto booksIds = library->getBooksIds();
|
||||
unsigned int inc = 1;
|
||||
for (itr = library.books.begin(); itr != library.books.end(); ++itr) {
|
||||
for(auto& id: booksIds) {
|
||||
auto& book = library->getBookById(id);
|
||||
std::cout << "#" << inc++ << std::endl
|
||||
<< "id:\t\t" << itr->id << std::endl
|
||||
<< "path:\t\t" << itr->path << std::endl
|
||||
<< "indexpath:\t" << itr->indexPath << std::endl
|
||||
<< "url:\t\t" << itr->url << std::endl
|
||||
<< "title:\t\t" << itr->title << std::endl
|
||||
<< "name:\t\t" << itr->name << std::endl
|
||||
<< "tags:\t\t" << itr->tags << std::endl
|
||||
<< "description:\t" << itr->description << std::endl
|
||||
<< "creator:\t" << itr->creator << std::endl
|
||||
<< "date:\t\t" << itr->date << std::endl
|
||||
<< "articleCount:\t" << itr->articleCount << std::endl
|
||||
<< "mediaCount:\t" << itr->mediaCount << std::endl
|
||||
<< "size:\t\t" << itr->size << " KB" << std::endl
|
||||
<< "id:\t\t" << book.getId() << std::endl
|
||||
<< "path:\t\t" << book.getPath() << std::endl
|
||||
<< "indexpath:\t" << book.getIndexPath() << std::endl
|
||||
<< "url:\t\t" << book.getUrl() << std::endl
|
||||
<< "title:\t\t" << book.getTitle() << std::endl
|
||||
<< "name:\t\t" << book.getName() << std::endl
|
||||
<< "tags:\t\t" << book.getTags() << std::endl
|
||||
<< "description:\t" << book.getDescription() << std::endl
|
||||
<< "creator:\t" << book.getCreator() << std::endl
|
||||
<< "date:\t\t" << book.getDate() << std::endl
|
||||
<< "articleCount:\t" << book.getArticleCount() << std::endl
|
||||
<< "mediaCount:\t" << book.getMediaCount() << std::endl
|
||||
<< "size:\t\t" << book.getSize() << " KB" << std::endl
|
||||
<< std::endl;
|
||||
}
|
||||
}
|
||||
@@ -68,24 +74,24 @@ void usage()
|
||||
}
|
||||
|
||||
|
||||
void handle_show(kiwix::Manager* libraryManager, const std::string& libraryPath,
|
||||
bool handle_show(kiwix::Library* library, const std::string& libraryPath,
|
||||
int argc, char* argv[])
|
||||
{
|
||||
show(libraryManager->cloneLibrary());
|
||||
show(library);
|
||||
return(0);
|
||||
}
|
||||
|
||||
void handle_add(kiwix::Manager* libraryManager, const std::string& libraryPath,
|
||||
bool handle_add(kiwix::Library* library, const std::string& libraryPath,
|
||||
int argc, char* argv[])
|
||||
{
|
||||
string zimPath;
|
||||
string zimPathToSave = ".";
|
||||
string indexPath;
|
||||
kiwix::supportedIndexType indexBackend = kiwix::XAPIAN;
|
||||
string url;
|
||||
string origID = "";
|
||||
bool setCurrent = false;
|
||||
int option_index = 0;
|
||||
int c = 0;
|
||||
bool resultCode = 0;
|
||||
|
||||
if (argc > 3) {
|
||||
zimPath = argv[3];
|
||||
@@ -98,12 +104,10 @@ void handle_add(kiwix::Manager* libraryManager, const std::string& libraryPath,
|
||||
= {{"url", required_argument, 0, 'u'},
|
||||
{"origId", required_argument, 0, 'o'},
|
||||
{"indexPath", required_argument, 0, 'i'},
|
||||
{"indexBackend", required_argument, 0, 'b'},
|
||||
{"zimPathToSave", required_argument, 0, 'z'},
|
||||
{"current", no_argument, 0, 'c'},
|
||||
{0, 0, 0, 0}};
|
||||
|
||||
c = getopt_long(argc, argv, "cz:u:i:b:", long_options, &option_index);
|
||||
c = getopt_long(argc, argv, "cz:u:i:", long_options, &option_index);
|
||||
|
||||
if (c != -1) {
|
||||
switch (c) {
|
||||
@@ -115,22 +119,10 @@ void handle_add(kiwix::Manager* libraryManager, const std::string& libraryPath,
|
||||
origID = optarg;
|
||||
break;
|
||||
|
||||
case 'c':
|
||||
setCurrent = true;
|
||||
break;
|
||||
|
||||
case 'i':
|
||||
indexPath = optarg;
|
||||
break;
|
||||
|
||||
case 'b':
|
||||
if (!strcmp(optarg, "xapian")) {
|
||||
indexBackend = kiwix::XAPIAN;
|
||||
} else {
|
||||
usage();
|
||||
}
|
||||
break;
|
||||
|
||||
case 'z':
|
||||
zimPathToSave = optarg;
|
||||
break;
|
||||
@@ -141,54 +133,119 @@ void handle_add(kiwix::Manager* libraryManager, const std::string& libraryPath,
|
||||
}
|
||||
|
||||
if (!zimPath.empty()) {
|
||||
kiwix::Manager manager(library);
|
||||
zimPathToSave = zimPathToSave == "." ? zimPath : zimPathToSave;
|
||||
string bookId = libraryManager->addBookFromPathAndGetId(
|
||||
string bookId = manager.addBookFromPathAndGetId(
|
||||
zimPath, zimPathToSave, url, false);
|
||||
if (!bookId.empty()) {
|
||||
if (setCurrent)
|
||||
libraryManager->setCurrentBookId(bookId);
|
||||
/* Save the index infos if necessary */
|
||||
if (!indexPath.empty())
|
||||
libraryManager->setBookIndex(bookId, indexPath, indexBackend);
|
||||
if (!indexPath.empty()) {
|
||||
if (isRelativePath(indexPath)) {
|
||||
indexPath = computeAbsolutePath(indexPath, getCurrentDirectory());
|
||||
}
|
||||
library->getBookById(bookId).setIndexPath(indexPath);
|
||||
}
|
||||
} else {
|
||||
cerr << "Unable to build or save library file '" << libraryPath << "'"
|
||||
<< endl;
|
||||
resultCode = 1;
|
||||
}
|
||||
} else {
|
||||
std::cerr << "Invalid zim file path" << std::endl;
|
||||
resultCode = 1;
|
||||
}
|
||||
|
||||
return(resultCode);
|
||||
}
|
||||
|
||||
void handle_remove(kiwix::Manager* libraryManager, const std::string& libraryPath,
|
||||
bool handle_remove(kiwix::Library* library, const std::string& libraryPath,
|
||||
int argc, char* argv[])
|
||||
{
|
||||
unsigned int bookIndex = 0;
|
||||
const unsigned int totalBookCount = libraryManager->getBookCount(true, true);
|
||||
std::string bookId;
|
||||
const unsigned int totalBookCount = library->getBookCount(true, true);
|
||||
bool exitCode = 0;
|
||||
|
||||
if (argc > 3) {
|
||||
bookIndex = atoi(argv[3]);
|
||||
bookId = argv[3];
|
||||
}
|
||||
|
||||
if (bookIndex > 0 && bookIndex <= totalBookCount) {
|
||||
libraryManager->removeBookByIndex(bookIndex - 1);
|
||||
} else {
|
||||
if (!library->removeBookById(bookId)) {
|
||||
if (totalBookCount > 0) {
|
||||
std::cerr
|
||||
<< "Invalid book index number. Please give a number between 1 and "
|
||||
<< totalBookCount << std::endl;
|
||||
<< "Invalid book id." << std::endl;
|
||||
exitCode = 1;
|
||||
} else {
|
||||
std::cerr
|
||||
<< "Invalid book index number. Library is empty, no book to delete."
|
||||
<< "Invalid book id. Library is empty, no book to delete."
|
||||
<< std::endl;
|
||||
exitCode = 1;
|
||||
}
|
||||
}
|
||||
|
||||
return(exitCode);
|
||||
}
|
||||
|
||||
bool handle_download(kiwix::Library* library, const std::string& libraryPath,
|
||||
int argc, char* argv[])
|
||||
{
|
||||
std::string bookId;
|
||||
bool exitCode = false;
|
||||
|
||||
if (argc > 3) {
|
||||
bookId = argv[3];
|
||||
}
|
||||
|
||||
auto& book = library->getBookById(bookId);
|
||||
auto did = book.getDownloadId();
|
||||
kiwix::Downloader downloader;
|
||||
kiwix::Download* download = nullptr;
|
||||
if (!did.empty()) {
|
||||
std::cout << "try resume " << did << std::endl;
|
||||
try {
|
||||
download = downloader.getDownload(did);
|
||||
} catch(...) {}
|
||||
}
|
||||
if (nullptr == download || download->getStatus() == kiwix::Download::K_UNKNOWN) {
|
||||
download = downloader.startDownload(book.getUrl());
|
||||
book.setDownloadId(download->getDid());
|
||||
}
|
||||
int step = 60*5;
|
||||
while (step--) {
|
||||
download->updateStatus();
|
||||
if (download->getStatus() == kiwix::Download::K_COMPLETE) {
|
||||
auto followingId = download->getFollowedBy();
|
||||
if (followingId.empty()) {
|
||||
book.setPath(download->getPath());
|
||||
book.setDownloadId("");
|
||||
std::cout << "File downloaded to " << book.getPath() << std::endl;
|
||||
break;
|
||||
} else {
|
||||
download = downloader.getDownload(followingId);
|
||||
}
|
||||
} else if (download->getStatus() == kiwix::Download::K_ACTIVE) {
|
||||
std::cout << download->getDid() << " : "
|
||||
<< kiwix::beautifyFileSize(download->getCompletedLength()) << "/"
|
||||
<< kiwix::beautifyFileSize(download->getTotalLength())
|
||||
<< " (" << kiwix::beautifyFileSize(download->getDownloadSpeed()) << "/s) "
|
||||
<< " [" << kiwix::beautifyFileSize(download->getVerifiedLength()) << "]"
|
||||
<< "[" << step << "] \n" << std::flush;
|
||||
} else if (download->getStatus() == kiwix::Download::K_ERROR) {
|
||||
std::cout << "File Error" << std::endl;
|
||||
exitCode = true;
|
||||
break;
|
||||
}
|
||||
std::this_thread::sleep_for(std::chrono::seconds(1));
|
||||
}
|
||||
|
||||
downloader.close();
|
||||
return exitCode;
|
||||
}
|
||||
|
||||
int main(int argc, char** argv)
|
||||
{
|
||||
string libraryPath = "";
|
||||
supportedAction action = NONE;
|
||||
kiwix::Manager libraryManager;
|
||||
kiwix::Library library;
|
||||
|
||||
/* Argument parsing */
|
||||
if (argc > 2) {
|
||||
@@ -201,6 +258,8 @@ int main(int argc, char** argv)
|
||||
action = SHOW;
|
||||
else if (actionString == "remove" || actionString == "delete")
|
||||
action = REMOVE;
|
||||
else if (actionString == "download")
|
||||
action = DOWNLOAD;
|
||||
}
|
||||
|
||||
/* Print usage)) if necessary */
|
||||
@@ -213,21 +272,32 @@ int main(int argc, char** argv)
|
||||
libraryPath = isRelativePath(libraryPath)
|
||||
? computeAbsolutePath(getCurrentDirectory(), libraryPath)
|
||||
: libraryPath;
|
||||
libraryManager.readFile(libraryPath, false);
|
||||
kiwix::Manager manager(&library);
|
||||
manager.readFile(libraryPath, false);
|
||||
|
||||
/* SHOW */
|
||||
if (action == SHOW) {
|
||||
handle_show(&libraryManager, libraryPath, argc, argv);
|
||||
} else if (action == ADD) {
|
||||
handle_add(&libraryManager, libraryPath, argc, argv);
|
||||
} else if (action == REMOVE) {
|
||||
handle_remove(&libraryManager, libraryPath, argc, argv);
|
||||
bool exitCode = 0;
|
||||
switch (action) {
|
||||
case SHOW:
|
||||
exitCode = handle_show(&library, libraryPath, argc, argv);
|
||||
break;
|
||||
case ADD:
|
||||
exitCode = handle_add(&library, libraryPath, argc, argv);
|
||||
break;
|
||||
case REMOVE:
|
||||
exitCode = handle_remove(&library, libraryPath, argc, argv);
|
||||
break;
|
||||
case DOWNLOAD:
|
||||
exitCode = handle_download(&library, libraryPath, argc, argv);
|
||||
break;
|
||||
case NONE:
|
||||
break;
|
||||
}
|
||||
|
||||
/* Rewrite the library file */
|
||||
if (action == REMOVE || action == ADD) {
|
||||
libraryManager.writeFile(libraryPath);
|
||||
if (action == REMOVE || action == ADD || action == DOWNLOAD) {
|
||||
library.writeToFile(libraryPath);
|
||||
}
|
||||
|
||||
exit(0);
|
||||
exit(exitCode);
|
||||
}
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
|
||||
subdir('installer')
|
||||
subdir('manager')
|
||||
subdir('reader')
|
||||
subdir('searcher')
|
||||
|
||||
@@ -79,6 +79,7 @@ extern "C" {
|
||||
#include <netdb.h>
|
||||
#include <stdint.h>
|
||||
#include <sys/socket.h>
|
||||
#include <netinet/in.h>
|
||||
#include <unistd.h>
|
||||
#endif
|
||||
|
||||
@@ -100,16 +101,28 @@ static std::map<std::string, std::string> extMimeTypes;
|
||||
static std::map<std::string, kiwix::Reader*> readers;
|
||||
static std::map<std::string, kiwix::Searcher*> searchers;
|
||||
static kiwix::Searcher* globalSearcher = nullptr;
|
||||
static kiwix::Manager libraryManager;
|
||||
static kiwix::Library library;
|
||||
static pthread_mutex_t searchLock = PTHREAD_MUTEX_INITIALIZER;
|
||||
static pthread_mutex_t compressorLock = PTHREAD_MUTEX_INITIALIZER;
|
||||
static pthread_mutex_t regexLock = PTHREAD_MUTEX_INITIALIZER;
|
||||
|
||||
std::pair<kiwix::Reader*, kiwix::Searcher*>
|
||||
get_from_humanReadableBookId(const std::string& humanReadableBookId) {
|
||||
kiwix::Searcher* searcher
|
||||
= searchers.find(humanReadableBookId) != searchers.end()
|
||||
? searchers.find(humanReadableBookId)->second
|
||||
: globalSearcher;
|
||||
kiwix::Reader* reader = readers.find(humanReadableBookId) != readers.end()
|
||||
? readers.find(humanReadableBookId)->second
|
||||
: NULL;
|
||||
return std::pair<kiwix::Reader*, kiwix::Searcher*>(reader, searcher);
|
||||
}
|
||||
|
||||
/* Try to get the mimeType from the file extension */
|
||||
static std::string getMimeTypeForFile(const std::string& filename)
|
||||
{
|
||||
std::string mimeType = "text/plain";
|
||||
unsigned int pos = filename.find_last_of(".");
|
||||
auto pos = filename.find_last_of(".");
|
||||
|
||||
if (pos != std::string::npos) {
|
||||
std::string extension = filename.substr(pos + 1);
|
||||
@@ -138,7 +151,10 @@ static bool startswith(const std::string& base, const std::string& start)
|
||||
|
||||
void introduceTaskbar(string& content, const string& humanReadableBookId)
|
||||
{
|
||||
string zimTitle;
|
||||
|
||||
pthread_mutex_lock(®exLock);
|
||||
|
||||
if (!noSearchBarFlag) {
|
||||
content = appendToFirstOccurence(
|
||||
content,
|
||||
@@ -160,10 +176,18 @@ void introduceTaskbar(string& content, const string& humanReadableBookId)
|
||||
RESOURCE::taskbar_html_part,
|
||||
humanReadableBookId,
|
||||
"__CONTENT__"));
|
||||
|
||||
auto reader = get_from_humanReadableBookId(humanReadableBookId).first;
|
||||
if (reader != nullptr) {
|
||||
zimTitle = reader->getTitle();
|
||||
}
|
||||
}
|
||||
content = replaceRegex(content, rootLocation, "__ROOT_LOCATION__");
|
||||
content = replaceRegex(content, replaceRegex(humanReadableBookId, "%26", "&"), "__CONTENT_ESCAPED__");
|
||||
}
|
||||
|
||||
content = replaceRegex(content, rootLocation, "__ROOT_LOCATION__");
|
||||
content = replaceRegex(content, replaceRegex(zimTitle, "%26", "&"), "__ZIM_TITLE__");
|
||||
content = replaceRegex(content, replaceRegex(humanReadableBookId, "%26", "&"), "__CONTENT_ESCAPED__");
|
||||
|
||||
pthread_mutex_unlock(®exLock);
|
||||
}
|
||||
|
||||
@@ -352,7 +376,7 @@ static struct MHD_Response* build_callback_response_from_entry(
|
||||
MHD_add_response_header(response, MHD_HTTP_HEADER_CONTENT_RANGE, oss.str().c_str());
|
||||
|
||||
MHD_add_response_header(response, MHD_HTTP_HEADER_CONTENT_LENGTH,
|
||||
std::to_string(range_len).c_str());
|
||||
kiwix::to_string(range_len).c_str());
|
||||
|
||||
/* Specify the mime type */
|
||||
MHD_add_response_header(
|
||||
@@ -369,18 +393,6 @@ static struct MHD_Response* build_callback_response_from_entry(
|
||||
return response;
|
||||
}
|
||||
|
||||
std::pair<kiwix::Reader*, kiwix::Searcher*>
|
||||
get_from_humanReadableBookId(const std::string& humanReadableBookId) {
|
||||
kiwix::Searcher* searcher
|
||||
= searchers.find(humanReadableBookId) != searchers.end()
|
||||
? searchers.find(humanReadableBookId)->second
|
||||
: globalSearcher;
|
||||
kiwix::Reader* reader = readers.find(humanReadableBookId) != readers.end()
|
||||
? readers.find(humanReadableBookId)->second
|
||||
: NULL;
|
||||
return std::pair<kiwix::Reader*, kiwix::Searcher*>(reader, searcher);
|
||||
}
|
||||
|
||||
static struct MHD_Response* handle_meta(RequestContext* request)
|
||||
{
|
||||
std::string humanReadableBookId;
|
||||
@@ -443,7 +455,7 @@ static struct MHD_Response* handle_suggest(RequestContext* request)
|
||||
humanReadableBookId = request->get_argument("content");
|
||||
term = request->get_argument("term");
|
||||
} catch (const std::out_of_range&) {
|
||||
return build_homepage(request);
|
||||
return build_404(request, "");
|
||||
}
|
||||
|
||||
if (isVerbose.load()) {
|
||||
@@ -628,12 +640,12 @@ static struct MHD_Response* handle_random(RequestContext* request)
|
||||
try {
|
||||
humanReadableBookId = request->get_argument("content");
|
||||
} catch (const std::out_of_range&) {
|
||||
return build_homepage(request);
|
||||
return build_404(request, "");
|
||||
}
|
||||
|
||||
auto reader = get_from_humanReadableBookId(humanReadableBookId).first;
|
||||
if (reader == nullptr) {
|
||||
return build_homepage(request);
|
||||
return build_404(request, "");
|
||||
}
|
||||
|
||||
try {
|
||||
@@ -673,33 +685,51 @@ static struct MHD_Response* handle_catalog(RequestContext* request)
|
||||
kiwix::OPDSDumper opdsDumper;
|
||||
opdsDumper.setRootLocation(rootLocation);
|
||||
opdsDumper.setSearchDescriptionUrl("catalog/searchdescription.xml");
|
||||
opdsDumper.setId(kiwix::to_string(uuid));
|
||||
opdsDumper.setLibrary(&library);
|
||||
mimeType = "application/atom+xml;profile=opds-catalog;kind=acquisition; charset=utf-8";
|
||||
kiwix::Library library_to_dump;
|
||||
std::vector<std::string> bookIdsToDump;
|
||||
if (url == "root.xml") {
|
||||
opdsDumper.setTitle("All zims");
|
||||
uuid = zim::Uuid::generate(host);
|
||||
library_to_dump = libraryManager.cloneLibrary();
|
||||
|
||||
bookIdsToDump = library.listBooksIds(
|
||||
kiwix::VALID|kiwix::LOCAL|kiwix::REMOTE);
|
||||
} else if (url == "search") {
|
||||
std::string query;
|
||||
std::string language;
|
||||
size_t count(10);
|
||||
size_t startIndex(0);
|
||||
try {
|
||||
query = request->get_argument("q");
|
||||
} catch (const std::out_of_range&) {
|
||||
return build_404(request, "");
|
||||
}
|
||||
} catch (const std::out_of_range&) {}
|
||||
try {
|
||||
language = request->get_argument("lang");
|
||||
} catch (const std::out_of_range&) {}
|
||||
try {
|
||||
count = stoul(request->get_argument("count"));
|
||||
} catch (...) {}
|
||||
try {
|
||||
startIndex = stoul(request->get_argument("start"));
|
||||
} catch (...) {}
|
||||
opdsDumper.setTitle("Search result for " + query);
|
||||
uuid = zim::Uuid::generate();
|
||||
library_to_dump = libraryManager.filter(query);
|
||||
bookIdsToDump = library.listBooksIds(
|
||||
kiwix::VALID|kiwix::LOCAL|kiwix::REMOTE,
|
||||
kiwix::UNSORTED,
|
||||
query,
|
||||
language);
|
||||
auto totalResults = bookIdsToDump.size();
|
||||
bookIdsToDump.erase(bookIdsToDump.begin(), bookIdsToDump.begin()+startIndex);
|
||||
if (count>0 && bookIdsToDump.size() > count) {
|
||||
bookIdsToDump.resize(count);
|
||||
}
|
||||
opdsDumper.setOpenSearchInfo(totalResults, startIndex, bookIdsToDump.size());
|
||||
} else {
|
||||
return build_404(request, "");
|
||||
}
|
||||
|
||||
{
|
||||
std::stringstream ss;
|
||||
ss << uuid;
|
||||
opdsDumper.setId(ss.str());
|
||||
}
|
||||
opdsDumper.setLibrary(library_to_dump);
|
||||
content = opdsDumper.dumpOPDSFeed();
|
||||
content = opdsDumper.dumpOPDSFeed(bookIdsToDump);
|
||||
}
|
||||
|
||||
bool deflated = request->can_compress() && compress_content(content, mimeType);
|
||||
@@ -728,7 +758,11 @@ static struct MHD_Response* handle_content(RequestContext* request)
|
||||
|
||||
auto reader = get_from_humanReadableBookId(humanReadableBookId).first;
|
||||
if (reader == nullptr) {
|
||||
return build_homepage(request);
|
||||
if (humanReadableBookId.size() == 0){
|
||||
return build_homepage(request);
|
||||
} else {
|
||||
return build_404(request, "");
|
||||
}
|
||||
}
|
||||
|
||||
auto urlStr = request->get_url().substr(humanReadableBookId.size()+1);
|
||||
@@ -842,7 +876,7 @@ static int accessHandlerCallback(void* cls,
|
||||
request.httpResponseCode = request.has_range() ? MHD_HTTP_PARTIAL_CONTENT : MHD_HTTP_OK;
|
||||
|
||||
if (! request.is_valid_url()) {
|
||||
response = build_homepage(&request);
|
||||
response = build_404(&request, "");
|
||||
} else {
|
||||
if (startswith(request.get_url(), "/skin/")) {
|
||||
response = handle_skin(&request);
|
||||
@@ -959,15 +993,15 @@ int main(int argc, char** argv)
|
||||
break;
|
||||
}
|
||||
} else {
|
||||
if (optind <= argc) {
|
||||
if (optind < argc) {
|
||||
if (libraryFlag) {
|
||||
libraryPath = argv[optind++];
|
||||
} else {
|
||||
while (optind < argc)
|
||||
zimPathes.push_back(std::string(argv[optind++]));
|
||||
}
|
||||
break;
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -996,6 +1030,7 @@ int main(int argc, char** argv)
|
||||
}
|
||||
|
||||
/* Setup the library manager and get the list of books */
|
||||
kiwix::Manager manager(&library);
|
||||
if (libraryFlag) {
|
||||
vector<string> libraryPaths = kiwix::split(libraryPath, ";");
|
||||
vector<string>::iterator itr;
|
||||
@@ -1009,7 +1044,7 @@ int main(int argc, char** argv)
|
||||
= isRelativePath(*itr)
|
||||
? computeAbsolutePath(getCurrentDirectory(), *itr)
|
||||
: *itr;
|
||||
retVal = libraryManager.readFile(libraryPath, true);
|
||||
retVal = manager.readFile(libraryPath, true);
|
||||
} catch (...) {
|
||||
retVal = false;
|
||||
}
|
||||
@@ -1023,74 +1058,69 @@ int main(int argc, char** argv)
|
||||
}
|
||||
|
||||
/* Check if the library is not empty (or only remote books)*/
|
||||
if (libraryManager.getBookCount(true, false) == 0) {
|
||||
if (library.getBookCount(true, false) == 0) {
|
||||
cerr << "The XML library file '" << libraryPath
|
||||
<< "' is empty (or has only remote books)." << endl;
|
||||
}
|
||||
} else {
|
||||
std::vector<std::string>::iterator it;
|
||||
for (it = zimPathes.begin(); it != zimPathes.end(); it++) {
|
||||
if (!libraryManager.addBookFromPath(*it, *it, "", false)) {
|
||||
if (!manager.addBookFromPath(*it, *it, "", false)) {
|
||||
cerr << "Unable to add the ZIM file '" << *it
|
||||
<< "' to the internal library." << endl;
|
||||
exit(1);
|
||||
}
|
||||
}
|
||||
if (!indexPath.empty()) {
|
||||
libraryManager.setBookIndex(libraryManager.getBooksIds()[0], indexPath);
|
||||
if (isRelativePath(indexPath)) {
|
||||
indexPath = computeAbsolutePath(indexPath, getCurrentDirectory());
|
||||
}
|
||||
library.getBookById(library.getBooksIds()[0]).setIndexPath(indexPath);
|
||||
}
|
||||
}
|
||||
|
||||
/* Instance the readers and searcher and build the corresponding maps */
|
||||
vector<string> booksIds = libraryManager.getBooksIds();
|
||||
vector<string>::iterator itr;
|
||||
kiwix::Book currentBook;
|
||||
vector<string> booksIds = library.listBooksIds(kiwix::LOCAL);
|
||||
globalSearcher = new kiwix::Searcher();
|
||||
globalSearcher->setProtocolPrefix(rootLocation + "/");
|
||||
globalSearcher->setSearchProtocolPrefix(rootLocation + "/" + "search?");
|
||||
for (itr = booksIds.begin(); itr != booksIds.end(); ++itr) {
|
||||
bool zimFileOk = false;
|
||||
libraryManager.getBookById(*itr, currentBook);
|
||||
std::string zimPath = currentBook.pathAbsolute;
|
||||
for (auto& bookId: booksIds) {
|
||||
auto& currentBook = library.getBookById(bookId);
|
||||
auto zimPath = currentBook.getPath();
|
||||
auto indexPath = currentBook.getIndexPath();
|
||||
|
||||
if (!zimPath.empty()) {
|
||||
indexPath = currentBook.indexPathAbsolute;
|
||||
/* Instanciate the ZIM file handler */
|
||||
kiwix::Reader* reader = NULL;
|
||||
try {
|
||||
reader = new kiwix::Reader(zimPath);
|
||||
} catch (...) {
|
||||
cerr << "Unable to open the ZIM file '" << zimPath << "'." << endl;
|
||||
continue;
|
||||
}
|
||||
|
||||
/* Instanciate the ZIM file handler */
|
||||
kiwix::Reader* reader = NULL;
|
||||
auto humanReadableId = currentBook.getHumanReadableIdFromPath();
|
||||
readers[humanReadableId] = reader;
|
||||
|
||||
if (reader->hasFulltextIndex()) {
|
||||
kiwix::Searcher* searcher = new kiwix::Searcher(humanReadableId);
|
||||
searcher->setProtocolPrefix(rootLocation + "/");
|
||||
searcher->setSearchProtocolPrefix(rootLocation + "/" + "search?");
|
||||
searcher->add_reader(reader, humanReadableId);
|
||||
globalSearcher->add_reader(reader, humanReadableId);
|
||||
searchers[humanReadableId] = searcher;
|
||||
} else if ( !indexPath.empty() ) {
|
||||
try {
|
||||
reader = new kiwix::Reader(zimPath);
|
||||
zimFileOk = true;
|
||||
kiwix::Searcher* searcher = new kiwix::Searcher(indexPath, reader, humanReadableId);
|
||||
searcher->setProtocolPrefix(rootLocation + "/");
|
||||
searcher->setSearchProtocolPrefix(rootLocation + "/" + "search?");
|
||||
searchers[humanReadableId] = searcher;
|
||||
} catch (...) {
|
||||
cerr << "Unable to open the ZIM file '" << zimPath << "'." << endl;
|
||||
}
|
||||
|
||||
if (zimFileOk) {
|
||||
string humanReadableId = currentBook.getHumanReadableIdFromPath();
|
||||
readers[humanReadableId] = reader;
|
||||
|
||||
if ( reader->hasFulltextIndex()) {
|
||||
kiwix::Searcher* searcher = new kiwix::Searcher(humanReadableId);
|
||||
searcher->setProtocolPrefix(rootLocation + "/");
|
||||
searcher->setSearchProtocolPrefix(rootLocation + "/" + "search?");
|
||||
searcher->add_reader(reader, humanReadableId);
|
||||
globalSearcher->add_reader(reader, humanReadableId);
|
||||
searchers[humanReadableId] = searcher;
|
||||
} else if ( !indexPath.empty() ) {
|
||||
try {
|
||||
kiwix::Searcher* searcher = new kiwix::Searcher(indexPath, reader, humanReadableId);
|
||||
searcher->setProtocolPrefix(rootLocation + "/");
|
||||
searcher->setSearchProtocolPrefix(rootLocation + "/" + "search?");
|
||||
searchers[humanReadableId] = searcher;
|
||||
} catch (...) {
|
||||
cerr << "Unable to open the search index '" << indexPath << "'."
|
||||
<< endl;
|
||||
searchers[humanReadableId] = nullptr;
|
||||
}
|
||||
} else {
|
||||
searchers[humanReadableId] = nullptr;
|
||||
}
|
||||
cerr << "Unable to open the search index '" << indexPath << "'."
|
||||
<< endl;
|
||||
searchers[humanReadableId] = nullptr;
|
||||
}
|
||||
} else {
|
||||
searchers[humanReadableId] = nullptr;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1098,24 +1128,25 @@ int main(int argc, char** argv)
|
||||
string welcomeBooksHtml
|
||||
= ""
|
||||
"<div class='book__list'>";
|
||||
for (itr = booksIds.begin(); itr != booksIds.end(); ++itr) {
|
||||
libraryManager.getBookById(*itr, currentBook);
|
||||
for (auto& bookId: booksIds) {
|
||||
auto& currentBook = library.getBookById(bookId);
|
||||
|
||||
if (!currentBook.path.empty()
|
||||
if (!currentBook.getPath().empty()
|
||||
&& readers.find(currentBook.getHumanReadableIdFromPath())
|
||||
!= readers.end()) {
|
||||
welcomeBooksHtml += ""
|
||||
"<a href='" + rootLocation + "/" + currentBook.getHumanReadableIdFromPath() + "/'>"
|
||||
"<div class='book'>"
|
||||
"<div class='book__background' style='background-image: url(data:" + currentBook.faviconMimeType+ ";base64," + currentBook.favicon + ");'>"
|
||||
"<div class='book__title' title='" + currentBook.title + "'>" + currentBook.title + "</div>"
|
||||
"<div class='book__description' title='" + currentBook.description + "'>" + currentBook.description + "</div>"
|
||||
"<div class='book__background' style=\"background-image: url('/meta?content="
|
||||
+ currentBook.getHumanReadableIdFromPath() + "&name=favicon');\">"
|
||||
"<div class='book__title' title='" + currentBook.getTitle() + "'>" + currentBook.getTitle() + "</div>"
|
||||
"<div class='book__description' title='" + currentBook.getDescription() + "'>" + currentBook.getDescription() + "</div>"
|
||||
"<div class='book__info'>"
|
||||
"" + kiwix::beautifyInteger(atoi(currentBook.articleCount.c_str())) + " articles, " + kiwix::beautifyInteger(atoi(currentBook.mediaCount.c_str())) + " medias"
|
||||
"" + kiwix::beautifyInteger(currentBook.getArticleCount()) + " articles, " + kiwix::beautifyInteger(currentBook.getMediaCount()) + " medias"
|
||||
"</div>"
|
||||
"</div>"
|
||||
"</div>"
|
||||
"</a>";
|
||||
"</a>\n";
|
||||
}
|
||||
}
|
||||
welcomeBooksHtml += ""
|
||||
|
||||
@@ -23,13 +23,13 @@
|
||||
#include <string.h>
|
||||
#include <stdexcept>
|
||||
#include <sstream>
|
||||
#include <cstdio>
|
||||
|
||||
RequestContext::RequestContext(struct MHD_Connection* connection,
|
||||
std::string rootLocation,
|
||||
const std::string& _url,
|
||||
const std::string& method,
|
||||
const std::string& version) :
|
||||
connection(connection),
|
||||
full_url(_url),
|
||||
url(_url),
|
||||
valid_url(true),
|
||||
@@ -196,16 +196,6 @@ std::string RequestContext::get_argument(const std::string& name) {
|
||||
return arguments.at(name);
|
||||
}
|
||||
|
||||
template<>
|
||||
unsigned int RequestContext::get_argument(const std::string& name) {
|
||||
return std::stoi(arguments.at(name).c_str());
|
||||
}
|
||||
|
||||
template<>
|
||||
float RequestContext::get_argument(const std::string& name) {
|
||||
return std::stof(arguments.at(name).c_str());
|
||||
}
|
||||
|
||||
std::string RequestContext::get_header(const std::string& name) {
|
||||
return headers.at(name);
|
||||
}
|
||||
|
||||
@@ -23,6 +23,7 @@
|
||||
#define REQUEST_CONTEXT_H
|
||||
|
||||
#include <string>
|
||||
#include <sstream>
|
||||
#include <map>
|
||||
#include <stdexcept>
|
||||
|
||||
@@ -62,7 +63,13 @@ class RequestContext {
|
||||
|
||||
std::string get_header(const std::string& name);
|
||||
template<typename T=std::string>
|
||||
T get_argument(const std::string& name);
|
||||
T get_argument(const std::string& name) {
|
||||
std::istringstream stream(arguments.at(name));
|
||||
T v;
|
||||
stream >> v;
|
||||
return v;
|
||||
}
|
||||
|
||||
|
||||
RequestMethod get_method();
|
||||
std::string get_url();
|
||||
@@ -78,7 +85,6 @@ class RequestContext {
|
||||
int httpResponseCode;
|
||||
|
||||
private:
|
||||
struct MHD_Connection* connection;
|
||||
std::string full_url;
|
||||
std::string url;
|
||||
bool valid_url;
|
||||
@@ -96,5 +102,7 @@ class RequestContext {
|
||||
static int fill_argument(void *, enum MHD_ValueKind, const char*, const char*);
|
||||
};
|
||||
|
||||
template<> std::string RequestContext::get_argument(const std::string& name);
|
||||
|
||||
|
||||
#endif //REQUEST_CONTEXT_H
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
<div class="kiwix_searchform">
|
||||
<form class="kiwixsearch" method="GET" action="__ROOT_LOCATION__/search" id="kiwixsearchform">
|
||||
<input autocomplete="off" class="ui-autocomplete-input" id="kiwixsearchbox" name="pattern" type="text">
|
||||
<input type="submit" value="Search">
|
||||
<input type="submit" value="🔍">
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -4,5 +4,6 @@
|
||||
<Description>Search zim files in the catalog.</Description>
|
||||
<Url type="application/atom+xml;profile=opds-catalog"
|
||||
xmlns:atom="http://www.w3.org/2005/Atom"
|
||||
template="/__ROOT_LOCATION__/catalog/search?q={searchTerms}"/>
|
||||
indexOffset="0"
|
||||
template="/__ROOT_LOCATION__/catalog/search?q={searchTerms}&lang={language}&count={count}&start={startIndex}"/>
|
||||
</OpenSearchDescription>
|
||||
|
||||
@@ -2,15 +2,15 @@
|
||||
<span id="kiwixtoolbar" class="ui-widget-header">
|
||||
<div class="kiwix_centered">
|
||||
<div class="kiwix_button_wrapper">
|
||||
<a id="kiwix_serve_taskbar_library_button" href="__ROOT_LOCATION__/"><button>Library</button></a>
|
||||
<a id="kiwix_serve_taskbar_home_button" href="__ROOT_LOCATION__/__CONTENT__/"><button>Home</button></a>
|
||||
<a id="kiwix_serve_taskbar_random_button" href="__ROOT_LOCATION__/random?content=__CONTENT_ESCAPED__"><button>Random</button></a>
|
||||
<a id="kiwix_serve_taskbar_library_button" href="__ROOT_LOCATION__/"><button>🏠</button></a>
|
||||
<a id="kiwix_serve_taskbar_home_button" href="__ROOT_LOCATION__/__CONTENT__/"><button>__ZIM_TITLE__</button></a>
|
||||
<a id="kiwix_serve_taskbar_random_button" href="__ROOT_LOCATION__/random?content=__CONTENT_ESCAPED__"><button>🎲</button></a>
|
||||
</div>
|
||||
<div class="kiwix_searchform">
|
||||
<form class="kiwixsearch" method="GET" action="__ROOT_LOCATION__/search" id="kiwixsearchform">
|
||||
<input type="hidden" name="content" value="__CONTENT__" />
|
||||
<input autocomplete="off" class="ui-autocomplete-input" id="kiwixsearchbox" name="pattern" type="text">
|
||||
<input type="submit" value="Search">
|
||||
<input type="submit" value="🔍">
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -42,6 +42,4 @@ sudo cp ninja /bin
|
||||
# Dependencies comming from kiwix-build.
|
||||
cd ${HOME}
|
||||
wget http://tmp.kiwix.org/ci/${ARCHIVE_NAME}
|
||||
mkdir -p BUILD_${PLATFORM}
|
||||
cd BUILD_${PLATFORM}
|
||||
tar xf ${HOME}/${ARCHIVE_NAME}
|
||||
|
||||
Reference in New Issue
Block a user