Merge pull request #276 from kiwix/new_version

New version 1.2.0

Changelog

@@ -1,3 +1,96 @@
+kiwix-tools 1.2.0
+=================
+
+ * Remove rpath for installed binaries.
+
+kiwix-serve
+-----------
+
+ * New Dockerfile of kiwix-serve
+ * New --nodatealiases option
+ * Do not use POLL on windows
+
+kiwix-manage
+------------
+
+ * Do not show all books if book ids has been provided.
+ * Be able to add several zim files in the same time in a library.
+
+kiwix-tools 1.1.0
+=================
+
+kiwix-serve
+-----------
+
+ * Fix bug about handling of absolute url in old zim file.
+ * All the catalog to be searched by tags.
+
+kiwix-tools 1.0.0
+=================
+
+ * [CI] Use the new deps archive xz
+ * Move version 1.0.0. There is no need to stay in pre 1.0 version.
+
+kiwix-serve
+-----------
+
+ * Correctly implement redirection.
+   kiwix-serve now return a 302 http status code instead of resolving the
+   redirection internally and return the content.
+
+
+kiwix-tools 0.9.0
+=================
+
+ * Update README
+ * Update man pages
+ * Remove support of external indexes (manage, search, serve)
+ * Update build system as we don't use ctpp2 anymore
+ * Update to last kiwix-lib API.
+
+kiwix-manage
+------------
+
+ * Update usage.
+
+
+kiwix-tools 0.8.0
+=================
+
+kiwix-manage
+------------
+
+ * Be able to remove several books from the library in one command.
+
+kiwix-tools 0.7.0
+=================
+
+ * Adapt to kiwix-lib new API
+
+kiwix-serve
+-----------
+
+ * Dumps only valid books in the opdsfeed.
+ * Allow the opds feed to be filtered by lang and paginated.
+
+kiwix-manage
+------------
+
+ * Add a download command to download a remote book locally
+ * Book are referenced by bookId not index.
+ * No more indexType option as it is always XAPIAN.
+
+kiwix-tools 0.6.1
+=================
+
+kiwix-serve
+-----------
+
+ * Update README.
+ * Fix crash when `--library` flag is provided without value.
+ * Correctly handle mimetype of file without extension on 64bits.
+ * Minor fixes
+
 kiwix-tools 0.6.0
 =================
 

README.md

@@ -1,14 +1,14 @@
 Kiwix tools
 ===========
 
-The Kiwix tools gathers kiwix command line tools.
+The Kiwix tools is a collection of Kiwix related command line tools.
 
 Disclaimer
 ----------
 
 This document assumes you have a little knowledge about software
 compilation. If you experience difficulties with the dependencies or
-with the Kiwix libary compilation itself, we recommend to have a look
+with the Kiwix tools compilation itself, we recommend to have a look
 to [kiwix-build](https://github.com/kiwix/kiwix-build).
 
 Preamble
@@ -16,18 +16,18 @@ Preamble
 
 Although the Kiwix tools can be compiled/cross-compiled on/for many
 sytems, the following documentation explains how to do it on POSIX
-ones. It is primarly though for GNU/Linux systems and has been tested
+ones. It is primarly thought for GNU/Linux systems and has been tested
 on recent releases of Ubuntu and Fedora.
 
 Dependencies
 ------------
 
-The Kiwix tools rely on a few third parts software libraries. They
-are prerequisites to the Kiwix library compilation. Following
+The Kiwix tools rely on a few third party software libraries. They are
+prerequisites to the Kiwix tools compilation. Therefore, following
 libraries need to be available:
 
 * Kiwix lib ....................... https://github.com/kiwix/kiwix-lib
-(no package for now)
+(no package so far)
 * Libmicrohttpd .......... https://www.gnu.org/software/libmicrohttpd/
 (package libmicrohttpd-dev on Ubuntu)
 * CTPP2 ..................................... http://ctpp.havoc.ru/en/
@@ -53,47 +53,26 @@ If you compile manually Libmicrohttpd, you might need to compile it
 without GNU TLS, a bug here will empeach further compilation of Kiwix
 tools otherwise.
 
-Environnement
+Environment
 -------------
 
-The Kiwix library builds using [Meson](http://mesonbuild.com/) version
-0.34 or higher. Meson relies itself on Ninja, pkg-config and few other
-compilation tools.
-
-Install first the few common compilation tools:
-* Automake
-* Libtool
-* Virtualenv
+The Kiwix tools build using [Meson](http://mesonbuild.com/) version
+0.39 or higher. Meson relies itself on Ninja, pkg-config and few other
+compilation tools. Install them first:
+* Meson
+* Ninja
 * Pkg-config
 
-Then install Meson itself:
-```
-virtualenv -p python3 ./ # Create virtualenv
-source bin/activate      # Activate the virtualenv
-pip install meson        # Install Meson
-hash -r                  # Refresh bash paths
-```
-
-Finally download and build Ninja locally:
-```
-git clone git://github.com/ninja-build/ninja.git
-cd ninja
-git checkout release
-./configure.py --bootstrap
-mkdir ../bin                      
-cp ninja ../bin
-cd ..
-```
+These tools should be packaged if you use a cutting edge operating
+system. If not, have a look to the "Troubleshooting" section.
 
 Compilation
 -----------
 
-Once all dependencies are installed, you can compile kiwix-lib with:
+Once all dependencies are installed, you can compile Kiwix tools with:
 ```
-mkdir build
 meson . build
-cd build
-ninja
+ninja -C build
 ```
 
 By default, it will compile dynamic linked libraries. If you want
@@ -105,16 +84,61 @@ Depending of you system, `ninja` may be called `ninja-build`.
 Installation
 ------------
 
-If you want to install the Kiwix tools you just have compiled on your
-system, here we go:
+If you want to install the Kiwix tools, here we go:
 
 ```
-ninja install
+ninja -C build install
+```
+
+You might need to run the command as root (or using 'sudo'), depending
+where you want to install the Kiwix tools. After the installation
+succeeded, you may need to run ldconfig (as root).
+
+Uninstallation
+------------
+
+If you want to uninstall the Kiwix tools:
+
+```
+ninja -C build uninstall
+```
+
+Like for the installation, you might need to run the command as root
+(or using 'sudo').
+
+Docker
+------
+
+An official Docker image of `kiwix-serve` can be found at
+https://hub.docker.com/r/kiwix/kiwix-serve.
+
+Troubleshooting
+---------------
+
+If you need to install Meson "manually":
+```
+virtualenv -p python3 ./ # Create virtualenv
+source bin/activate      # Activate the virtualenv
+pip3 install meson       # Install Meson
+hash -r                  # Refresh bash paths
+```
+
+If you need to install Ninja "manually":
+```
+git clone git://github.com/ninja-build/ninja.git
+cd ninja
+git checkout release
+./configure.py --bootstrap
+mkdir ../bin
+cp ninja ../bin
 cd ..
 ```
 
-You might need to run the command as root, depending where you want to
-install the libraries.
+If the compilation still fails, you might need to get a more recent
+version of a dependency than the one packaged by your Linux
+distribution. Try then with a source tarball distributed by the
+problematic upstream project or even directly from the source code
+repository.
 
 License
 -------

docker/server/Dockerfile

@@ -0,0 +1,15 @@
+FROM alpine:latest
+LABEL maintainer Emmanuel Engelhart <kelson@kiwix.org>
+
+# Install kiwix-serve
+WORKDIR /
+RUN apk add --no-cache curl bzip2
+RUN curl -kL https://download.kiwix.org/release/kiwix-tools/kiwix-tools_linux-x86_64-1.1.0.tar.gz | tar -xz && \
+    mv kiwix-tools*/kiwix-serve /usr/local/bin && \
+    rm -r kiwix-tools*
+
+# Run kiwix-serve
+EXPOSE 80
+VOLUME /data
+WORKDIR /data
+ENTRYPOINT ["/usr/local/bin/kiwix-serve", "--port", "80"]

docker/server/README.md

@@ -0,0 +1,13 @@
+kiwix-serve Docker image
+========================
+
+* Download a ZIM file from <https://wiki.kiwix.org/wiki/Content>
+
+* Given `wikipedia.zim` resides in `/tmp/zim/`, execute the following:
+
+```
+docker run -v /tmp/zim:/data -p 8080:80 kiwix/kiwix-serve wikipedia.zim
+```
+
+![screenshot_1.png](https://github.com/kiwix/kiwix-tools/raw/master/docker/server/pictures/screenshot_1.png)
+![screenshot_2.png](https://github.com/kiwix/kiwix-tools/raw/master/docker/server/pictures/screenshot_2.png)

meson.build

@@ -1,5 +1,5 @@
 project('kiwix-tools', 'cpp',
-  version : '0.6.0',
+  version : '1.2.0',
   license : 'GPL',
   default_options: ['c_std=c11', 'cpp_std=c++11', 'werror=true'])
 
@@ -11,7 +11,7 @@ if static_linkage
 endif
 
 thread_dep = dependency('threads')
-kiwixlib_dep = dependency('kiwix', version:'>=2.0.1', static:static_linkage)
+kiwixlib_dep = dependency('kiwix', version:'>=4.1.0', static:static_linkage)
 microhttpd_dep = dependency('libmicrohttpd', static:static_linkage)
 z_dep = dependency('zlib', static:static_linkage)
 

src/man/kiwix-manage.1

@@ -15,12 +15,12 @@ kiwix\-manage LIBRARY_PATH add ZIM_PATH ...
 .PP
 \fBkiwix\-manage\fP tool for managing a kiwix library.
 .PP
-Allows one to manage the content of the kiwix library. Library file is an XML flat file
-listing ZIM files with all necessary information like favicon, date, creator,
-description, indexpath, filepath, title, source/metalink etc.
+Allows one to manage the content of the kiwix library. The library file is a
+flat XML file listing ZIM files with all necessary information like favicon,
+date, creator, description, indexpath, filepath, title, source/metalink, etc.
 .
 .PP
- Example library file can be found at http://www.kiwix.org/library\.xml
+ Example library file can be found at https://www.kiwix.org/library\.xml
 .br
 .TP
 \fBadd\fR
@@ -55,7 +55,7 @@ Path to full-text index for that ZIM file.
 Set the content location of the ZIM file over the network for in\-kiwix download.
 
 .SH SEE ALSO
-kiwix(1) kiwix\-install(1) kiwix\-serve(1)
+kiwix(1) kiwix\-serve(1)
 .SH AUTHOR
 Emmanuel Engelhart <kelson@kiwix.org>
 .br

src/man/kiwix-serve.1

@@ -9,7 +9,7 @@ kiwix\-serve [\-\-index=INDEX_PATH] [\-\-port=PORT] [\-\-verbose] [\-\-daemon] [
 kiwix\-serve \-\-library [\-\-port=PORT] [\-\-verbose] [\-\-daemon] [\-\-attachToProcess=PID] LIBRARY_PATH
 .SH DESCRIPTION
 .PP
-Stand\-alone HTTP server for serving ZIM content over the network.
+Stand\-alone HTTP server for serving ZIM contents over the network.
 
 .TP
 \fB\-\-index=INDEX_PATH\fR
@@ -27,7 +27,7 @@ Enable verbose output.
 
 .TP
 \fB\-\-daemon\fR
-Run the server as a daemon
+Run the server as a daemon.
 
 .TP
 \fB\-\-attachToProcess=PID\fR
@@ -49,12 +49,11 @@ Serves the contents of a library file instead of a single ZIM file.
 \fBLIBRARY_PATH\fR
 Kiwix library file path.
 .br
-Library is XML file created using \fBkiwix-manage\fB.
+Library is an XML file created using \fBkiwix-manage\fB.
 
 .SH SEE ALSO
 kiwix(1) kiwix\-manage(1)
-.br
-kiwix\-install(1)
+
 .SH AUTHOR
 Emmanuel Engelhart <kelson@kiwix.org>
 .br

src/manager/kiwix-manage.cpp

@@ -21,36 +21,40 @@
 #include <unistd.h>
 #endif
 #include <getopt.h>
-#include <kiwix/common/pathTools.h>
+#include <kiwix/tools/pathTools.h>
+#include <kiwix/tools/stringTools.h>
 #include <kiwix/manager.h>
+#include <kiwix/downloader.h>
 #include <cstdlib>
 #include <iostream>
 
+#include <thread>
+#include <chrono>
+
 using namespace std;
 
-enum supportedAction { NONE, ADD, SHOW, REMOVE };
+enum supportedAction { NONE, ADD, SHOW, REMOVE, DOWNLOAD };
 
-void show(kiwix::Library library)
+void show(kiwix::Library* library, const std::string& bookId)
 {
-  std::vector<kiwix::Book>::iterator itr;
-  unsigned int inc = 1;
-  for (itr = library.books.begin(); itr != library.books.end(); ++itr) {
-    std::cout << "#" << inc++ << std::endl
-              << "id:\t\t" << itr->id << std::endl
-              << "path:\t\t" << itr->path << std::endl
-              << "indexpath:\t" << itr->indexPath << std::endl
-              << "url:\t\t" << itr->url << std::endl
-              << "title:\t\t" << itr->title << std::endl
-              << "name:\t\t" << itr->name << std::endl
-              << "tags:\t\t" << itr->tags << std::endl
-              << "description:\t" << itr->description << std::endl
-              << "creator:\t" << itr->creator << std::endl
-              << "date:\t\t" << itr->date << std::endl
-              << "articleCount:\t" << itr->articleCount << std::endl
-              << "mediaCount:\t" << itr->mediaCount << std::endl
-              << "size:\t\t" << itr->size << " KB" << std::endl
-              << std::endl;
+  try {
+    auto& book = library->getBookById(bookId);
+    std::cout << "id:\t\t" << book.getId() << std::endl
+              << "path:\t\t" << book.getPath() << std::endl
+              << "url:\t\t" << book.getUrl() << std::endl
+              << "title:\t\t" << book.getTitle() << std::endl
+              << "name:\t\t" << book.getName() << std::endl
+              << "tags:\t\t" << book.getTags() << std::endl
+              << "description:\t" << book.getDescription() << std::endl
+              << "creator:\t" << book.getCreator() << std::endl
+              << "date:\t\t" << book.getDate() << std::endl
+              << "articleCount:\t" << book.getArticleCount() << std::endl
+              << "mediaCount:\t" << book.getMediaCount() << std::endl
+              << "size:\t\t" << book.getSize() << " KB" << std::endl;
+  } catch (std::out_of_range&) {
+     std::cout << "No book " << bookId << " in the library" << std::endl;
   }
+  std::cout << std::endl;
 }
 
 void usage()
@@ -58,7 +62,6 @@ void usage()
   cerr << "Usage:" << endl;
   cerr << "\tkiwix-manage LIBRARY_PATH add ZIM_PATH "
           "[--zimPathToSave=../content/foobar.zim] [--current] "
-          "[--indexBackend=xapian] [--indexPath=FULLTEXT_IDX_PATH] "
           "[--url=http://...metalink]"
        << endl;
   cerr << "\tkiwix-manage LIBRARY_PATH show [CONTENTID1] [CONTENTID2] ... "
@@ -68,126 +71,119 @@ void usage()
 }
 
 
-bool handle_show(kiwix::Manager* libraryManager, const std::string& libraryPath,
+bool handle_show(kiwix::Library* library, const std::string& libraryPath,
                  int argc, char* argv[])
 {
-  show(libraryManager->cloneLibrary());
+  if (argc > 3 ) {
+    for(auto i=3; i<argc; i++) {
+       std::string bookId = argv[i];
+       show(library, bookId);
+    }
+  } else {
+    auto booksIds = library->getBooksIds();
+    for(auto& bookId: booksIds) {
+      show(library, bookId);
+    }
+  }
   return(0);
 }
 
-bool handle_add(kiwix::Manager* libraryManager, const std::string& libraryPath,
+bool handle_add(kiwix::Library* library, const std::string& libraryPath,
                 int argc, char* argv[])
 {
   string zimPath;
   string zimPathToSave = ".";
-  string indexPath;
-  kiwix::supportedIndexType indexBackend = kiwix::XAPIAN;
   string url;
   string origID = "";
-  bool setCurrent = false;
   int option_index = 0;
   int c = 0;
   bool resultCode = 0;
 
-  if (argc > 3) {
-    zimPath = argv[3];
+  if (argc <= 3) {
+    std::cerr << "Path to zim file to add is missing in the command line" << std::endl;
+    return (-1);
   }
 
   /* Options parsing */
-  optind = 2;
-  while (42) {
-    static struct option long_options[]
-        = {{"url", required_argument, 0, 'u'},
-           {"origId", required_argument, 0, 'o'},
-           {"indexPath", required_argument, 0, 'i'},
-           {"indexBackend", required_argument, 0, 'b'},
-           {"zimPathToSave", required_argument, 0, 'z'},
-           {"current", no_argument, 0, 'c'},
-           {0, 0, 0, 0}};
+  optind = 3;
+  static struct option long_options[] = {
+    {"url", required_argument, 0, 'u'},
+    {"origId", required_argument, 0, 'o'},
+    {"zimPathToSave", required_argument, 0, 'z'},
+    {0, 0, 0, 0}
+  };
 
-    c = getopt_long(argc, argv, "cz:u:i:b:", long_options, &option_index);
-
-    if (c != -1) {
-      switch (c) {
-        case 'u':
-          url = optarg;
-          break;
-
-        case 'o':
-          origID = optarg;
-          break;
-
-        case 'c':
-          setCurrent = true;
-          break;
-
-        case 'i':
-          indexPath = optarg;
-          break;
-
-        case 'b':
-          if (!strcmp(optarg, "xapian")) {
-            indexBackend = kiwix::XAPIAN;
-          } else {
-            usage();
-          }
-          break;
-
-        case 'z':
-          zimPathToSave = optarg;
-          break;
-      }
-    } else {
+  bool has_option = false;
+  while (true) {
+    c = getopt_long(argc, argv, "cz:u:", long_options, &option_index);
+    if (c == -1)
       break;
+
+    has_option = true;
+    switch (c) {
+      case 'u':
+        url = optarg;
+        break;
+
+      case 'o':
+        origID = optarg;
+        break;
+
+      case 'z':
+        zimPathToSave = optarg;
+        break;
     }
   }
 
-  if (!zimPath.empty()) {
-    zimPathToSave = zimPathToSave == "." ? zimPath : zimPathToSave;
-    string bookId = libraryManager->addBookFromPathAndGetId(
-        zimPath, zimPathToSave, url, false);
-     if (!bookId.empty()) {
-      if (setCurrent)
-        libraryManager->setCurrentBookId(bookId);
-      /* Save the index infos if necessary */
-      if (!indexPath.empty())
-        libraryManager->setBookIndex(bookId, indexPath, indexBackend);
-     } else {
-      cerr << "Unable to build or save library file '" << libraryPath << "'"
-           << endl;
+  if (optind >= argc) {
+    std::cerr << "Path to zim file to add is missing in the command line" << std::endl;
+    return (-1);
+  }
+
+  if (has_option && argc-optind > 1) {
+    std::cerr << "You cannot give option and several zim files to add" << std::endl;
+    return (-1);
+  }
+
+  kiwix::Manager manager(library);
+
+  for(auto i=optind; i<argc; i++) {
+    std::string zimPath = argv[i];
+    if (!zimPath.empty()) {
+      zimPathToSave = zimPathToSave == "." ? zimPath : zimPathToSave;
+      manager.addBookFromPathAndGetId(zimPath, zimPathToSave, url, false);
+    } else {
+      std::cerr << "Invalid zim file path" << std::endl;
       resultCode = 1;
     }
-  } else {
-    std::cerr << "Invalid zim file path" << std::endl;
-    resultCode = 1;
   }
 
   return(resultCode);
 }
 
-bool handle_remove(kiwix::Manager* libraryManager, const std::string& libraryPath,
+bool handle_remove(kiwix::Library* library, const std::string& libraryPath,
                    int argc, char* argv[])
 {
-  unsigned int bookIndex = 0;
-  const unsigned int totalBookCount = libraryManager->getBookCount(true, true);
+  std::string bookId;
+  const unsigned int totalBookCount = library->getBookCount(true, true);
   bool exitCode = 0;
 
-  if (argc > 3) {
-    bookIndex = atoi(argv[3]);
+  if (argc <= 3) {
+    std::cerr << "BookId to remove missing in the command line" << std::endl;
+    return (-1);
   }
 
-  if (bookIndex > 0 && bookIndex <= totalBookCount) {
-    libraryManager->removeBookByIndex(bookIndex - 1);
-  } else {
-    if (totalBookCount > 0) {
-      std::cerr
-          << "Invalid book index number. Please give a number between 1 and "
-          << totalBookCount << std::endl;
-      exitCode = 1;
-    } else {
-      std::cerr
-          << "Invalid book index number. Library is empty, no book to delete."
-          << std::endl;
+  if (!totalBookCount) {
+    std::cerr << "Library is empty, no book to delete."
+              << std::endl;
+    return 1;
+  }
+
+  for (int i = 3; i<argc; i++) {
+    bookId = argv[i];
+
+    if (!library->removeBookById(bookId)) {
+      std::cerr << "Invalid book id '" << bookId << "'." << std::endl;
       exitCode = 1;
     }
   }
@@ -195,11 +191,67 @@ bool handle_remove(kiwix::Manager* libraryManager, const std::string& libraryPat
   return(exitCode);
 }
 
+bool handle_download(kiwix::Library* library, const std::string& libraryPath,
+                     int argc, char* argv[])
+{
+  std::string bookId;
+  bool exitCode = false;
+
+  if (argc > 3) {
+     bookId = argv[3];
+  }
+
+  auto& book = library->getBookById(bookId);
+  auto did = book.getDownloadId();
+  kiwix::Downloader downloader;
+  kiwix::Download* download = nullptr;
+  if (!did.empty()) {
+    std::cout << "try resume " << did << std::endl;
+    try {
+      download = downloader.getDownload(did);
+    } catch(...) {}
+  }
+  if (nullptr == download || download->getStatus() == kiwix::Download::K_UNKNOWN) {
+    download = downloader.startDownload(book.getUrl());
+    book.setDownloadId(download->getDid());
+  }
+  int step = 60*5;
+  while (step--) {
+    download->updateStatus();
+    if (download->getStatus() == kiwix::Download::K_COMPLETE) {
+      auto followingId = download->getFollowedBy();
+      if (followingId.empty()) {
+        book.setPath(download->getPath());
+        book.setDownloadId("");
+        std::cout << "File downloaded to " << book.getPath() << std::endl;
+        break;
+      } else {
+        download = downloader.getDownload(followingId);
+      }
+    } else if (download->getStatus() == kiwix::Download::K_ACTIVE) {
+      std::cout << download->getDid() << " : "
+                << kiwix::beautifyFileSize(download->getCompletedLength()) << "/"
+                << kiwix::beautifyFileSize(download->getTotalLength())
+                << " (" << kiwix::beautifyFileSize(download->getDownloadSpeed()) << "/s) "
+                << " [" << kiwix::beautifyFileSize(download->getVerifiedLength()) << "]"
+                << "[" << step << "]    \n" << std::flush;
+    } else if (download->getStatus() == kiwix::Download::K_ERROR) {
+      std::cout << "File Error" << std::endl;
+      exitCode = true;
+      break;
+    }
+    std::this_thread::sleep_for(std::chrono::seconds(1));
+  }
+
+  downloader.close();
+  return exitCode;
+}
+
 int main(int argc, char** argv)
 {
   string libraryPath = "";
   supportedAction action = NONE;
-  kiwix::Manager libraryManager;
+  kiwix::Library library;
 
   /* Argument parsing */
   if (argc > 2) {
@@ -212,6 +264,8 @@ int main(int argc, char** argv)
       action = SHOW;
     else if (actionString == "remove" || actionString == "delete")
       action = REMOVE;
+    else if (actionString == "download")
+      action = DOWNLOAD;
   }
 
   /* Print usage)) if necessary */
@@ -224,21 +278,31 @@ int main(int argc, char** argv)
   libraryPath = isRelativePath(libraryPath)
                     ? computeAbsolutePath(getCurrentDirectory(), libraryPath)
                     : libraryPath;
-  libraryManager.readFile(libraryPath, false);
+  kiwix::Manager manager(&library);
+  manager.readFile(libraryPath, false);
 
   /* SHOW */
   bool exitCode = 0;
-  if (action == SHOW) {
-    exitCode = handle_show(&libraryManager, libraryPath, argc, argv);
-  } else if (action == ADD) {
-    exitCode = handle_add(&libraryManager, libraryPath, argc, argv);
-  } else if (action == REMOVE) {
-    exitCode = handle_remove(&libraryManager, libraryPath, argc, argv);
+  switch (action) {
+    case SHOW:
+      exitCode = handle_show(&library, libraryPath, argc, argv);
+      break;
+    case ADD:
+      exitCode = handle_add(&library, libraryPath, argc, argv);
+      break;
+    case REMOVE:
+      exitCode = handle_remove(&library, libraryPath, argc, argv);
+      break;
+    case DOWNLOAD:
+      exitCode = handle_download(&library, libraryPath, argc, argv);
+      break;
+    case NONE:
+      break;
   }
 
   /* Rewrite the library file */
-  if (action == REMOVE || action == ADD) {
-    libraryManager.writeFile(libraryPath);
+  if (action == REMOVE || action == ADD || action == DOWNLOAD) {
+    library.writeToFile(libraryPath);
   }
 
   exit(exitCode);

src/manager/meson.build

@@ -1,4 +1,3 @@
 executable('kiwix-manage', ['kiwix-manage.cpp'],
   dependencies:all_deps,
-  install:true,
-  install_rpath: join_paths(get_option('prefix'), get_option('libdir')))
+  install:true)

src/reader/meson.build

@@ -1,4 +1,3 @@
 executable('kiwix-read', ['kiwix-read.cpp'],
   dependencies:all_deps,
-  install:true,
-  install_rpath: join_paths(get_option('prefix'), get_option('libdir')))
+  install:true)

src/searcher/kiwix-search.cpp

@@ -27,8 +27,7 @@ void usage()
 {
   cout << "Usage: kiwix-search [OPTIONS] ZIM PATTERN" << endl << endl
        << "  kiwix-search allows to find articles based on the a fulltext search pattern." << endl << endl
-       << "  ZIM is the full path of the ZIM file. This can also be a disctinct fulltext" << endl
-       << "  index directory (usually distributed with the *.idx extension)." << endl
+       << "  ZIM is the full path of the ZIM file." << endl
        << "  PATTERN is/are word(s) - or part of - to search in the ZIM." << endl << endl
        << "  -s, --suggestion\tSuggest article titles based on the few letters of the PATTERN instead of making a fulltext search. Work a bit like a completion solution." << endl
        << "  -v, --verbose\t\tGive details about the search process" << endl;
@@ -102,12 +101,8 @@ int main(int argc, char** argv)
     searcher = new kiwix::Searcher();
     searcher->add_reader(reader, "");
   } else {
-    try {
-      searcher = new kiwix::Searcher(zimPath, NULL, "");
-    } catch (...) {
-      cerr << "Unable to search through zim '" << zimPath << "'." << endl;
-      exit(1);
-    }
+    cerr << "Unable to search through zim '" << zimPath << "'." << endl;
+    exit(1);
   }
 
   /* Start the indexing */

src/searcher/meson.build

@@ -1,4 +1,3 @@
 executable('kiwix-search', ['kiwix-search.cpp'],
   dependencies:all_deps,
-  install:true,
-  install_rpath: join_paths(get_option('prefix'), get_option('libdir')))
+  install:true)

src/server/kiwix-serve.cpp

@@ -44,10 +44,10 @@ extern "C" {
 #endif
 
 #include <getopt.h>
-#include <kiwix/common/otherTools.h>
-#include <kiwix/common/pathTools.h>
-#include <kiwix/common/regexTools.h>
-#include <kiwix/common/stringTools.h>
+#include <kiwix/tools/otherTools.h>
+#include <kiwix/tools/pathTools.h>
+#include <kiwix/tools/regexTools.h>
+#include <kiwix/tools/stringTools.h>
 #include <kiwix/manager.h>
 #include <kiwix/reader.h>
 #include <kiwix/searcher.h>
@@ -79,6 +79,7 @@ extern "C" {
 #include <netdb.h>
 #include <stdint.h>
 #include <sys/socket.h>
+#include <netinet/in.h>
 #include <unistd.h>
 #endif
 
@@ -92,6 +93,7 @@ using namespace std;
 
 static bool noLibraryButtonFlag = false;
 static bool noSearchBarFlag = false;
+static bool noDateAliasesFlag = false;
 static string welcomeHTML;
 static string catalogOpenSearchDescription;
 static std::atomic_bool isVerbose(false);
@@ -100,20 +102,11 @@ static std::map<std::string, std::string> extMimeTypes;
 static std::map<std::string, kiwix::Reader*> readers;
 static std::map<std::string, kiwix::Searcher*> searchers;
 static kiwix::Searcher* globalSearcher = nullptr;
-static kiwix::Manager libraryManager;
+static kiwix::Library library;
 static pthread_mutex_t searchLock = PTHREAD_MUTEX_INITIALIZER;
 static pthread_mutex_t compressorLock = PTHREAD_MUTEX_INITIALIZER;
 static pthread_mutex_t regexLock = PTHREAD_MUTEX_INITIALIZER;
 
-template<typename T>
-inline std::string _tostring(const T& value)
-{
-    std::ostringstream stream;
-    stream << value;
-    return stream.str();
-}
-
-
 std::pair<kiwix::Reader*, kiwix::Searcher*>
 get_from_humanReadableBookId(const std::string& humanReadableBookId) {
   kiwix::Searcher* searcher
@@ -130,7 +123,7 @@ get_from_humanReadableBookId(const std::string& humanReadableBookId) {
 static std::string getMimeTypeForFile(const std::string& filename)
 {
   std::string mimeType = "text/plain";
-  unsigned int pos = filename.find_last_of(".");
+  auto pos = filename.find_last_of(".");
 
   if (pos != std::string::npos) {
     std::string extension = filename.substr(pos + 1);
@@ -384,7 +377,7 @@ static struct MHD_Response* build_callback_response_from_entry(
   MHD_add_response_header(response, MHD_HTTP_HEADER_CONTENT_RANGE, oss.str().c_str());
 
   MHD_add_response_header(response, MHD_HTTP_HEADER_CONTENT_LENGTH,
-    _tostring(range_len).c_str());
+    kiwix::to_string(range_len).c_str());
 
   /* Specify the mime type */
   MHD_add_response_header(
@@ -546,9 +539,9 @@ static struct MHD_Response* handle_search(RequestContext* request)
 
   /* Retrive geo search */
   bool has_geo_query = false;
-  float latitude;
-  float longitude;
-  float distance;
+  float latitude = 0;
+  float longitude = 0;
+  float distance = 0;
   try {
     latitude = request->get_argument<float>("latitude");
     longitude = request->get_argument<float>("longitude");
@@ -693,33 +686,58 @@ static struct MHD_Response* handle_catalog(RequestContext* request)
     kiwix::OPDSDumper opdsDumper;
     opdsDumper.setRootLocation(rootLocation);
     opdsDumper.setSearchDescriptionUrl("catalog/searchdescription.xml");
+    opdsDumper.setId(kiwix::to_string(uuid));
+    opdsDumper.setLibrary(&library);
     mimeType = "application/atom+xml;profile=opds-catalog;kind=acquisition; charset=utf-8";
-    kiwix::Library library_to_dump;
+    std::vector<std::string> bookIdsToDump;
     if (url == "root.xml") {
       opdsDumper.setTitle("All zims");
       uuid = zim::Uuid::generate(host);
-      library_to_dump = libraryManager.cloneLibrary();
+
+      bookIdsToDump = library.listBooksIds(
+        kiwix::VALID|kiwix::LOCAL|kiwix::REMOTE);
     } else if (url == "search") {
       std::string query;
+      std::string language;
+      std::vector<std::string> tags;
+      size_t count(10);
+      size_t startIndex(0);
       try {
         query = request->get_argument("q");
-      } catch (const std::out_of_range&) {
-        return build_404(request, "");
-      }
+      } catch (const std::out_of_range&) {}
+      try {
+        language = request->get_argument("lang");
+      } catch (const std::out_of_range&) {}
+      try {
+        count = stoul(request->get_argument("count"));
+      } catch (...) {}
+      try {
+        startIndex = stoul(request->get_argument("start"));
+      } catch (...) {}
+      try {
+        tags.push_back(request->get_argument("tag"));
+      } catch (...) {}
       opdsDumper.setTitle("Search result for " + query);
       uuid = zim::Uuid::generate();
-      library_to_dump = libraryManager.filter(query);
+      bookIdsToDump = library.listBooksIds(
+        kiwix::VALID|kiwix::LOCAL|kiwix::REMOTE,
+        kiwix::UNSORTED,
+        query,
+        language,
+        "", // creator
+        "", // publisher
+        tags);
+      auto totalResults = bookIdsToDump.size();
+      bookIdsToDump.erase(bookIdsToDump.begin(), bookIdsToDump.begin()+startIndex);
+      if (count>0 && bookIdsToDump.size() > count) {
+        bookIdsToDump.resize(count);
+      }
+      opdsDumper.setOpenSearchInfo(totalResults, startIndex, bookIdsToDump.size());
     } else {
       return build_404(request, "");
     }
 
-    {
-      std::stringstream ss;
-      ss << uuid;
-      opdsDumper.setId(ss.str());
-    }
-    opdsDumper.setLibrary(library_to_dump);
-    content = opdsDumper.dumpOPDSFeed();
+    content = opdsDumper.dumpOPDSFeed(bookIdsToDump);
   }
 
   bool deflated = request->can_compress() && compress_content(content, mimeType);
@@ -756,10 +774,22 @@ static struct MHD_Response* handle_content(RequestContext* request)
   }
 
   auto urlStr = request->get_url().substr(humanReadableBookId.size()+1);
+  if (urlStr[0] == '/') {
+    urlStr = urlStr.substr(1);
+  }
 
   try {
     entry = reader->getEntryFromPath(urlStr);
-    entry = entry.getFinalEntry();
+    if (entry.isRedirect() || urlStr.empty()) {
+      // If urlStr is empty, we want to mainPage.
+      // We must do a redirection to the real page.
+      entry = entry.getFinalEntry();
+      std::string httpRedirection = (
+        rootLocation + "/" + humanReadableBookId + "/" +
+        kiwix::urlEncode(entry.getPath()));
+      request->httpResponseCode = MHD_HTTP_FOUND;
+      return build_response("", 0, httpRedirection, "", false, false);
+    }
   } catch(kiwix::NoEntry& e) {
     if (isVerbose.load())
       printf("Failed to find %s\n", urlStr.c_str());
@@ -787,7 +817,6 @@ static struct MHD_Response* handle_content(RequestContext* request)
     /* Special rewrite URL in case of ZIM file use intern *asbolute* url like
      * /A/Kiwix */
     if (mimeType.find("text/html") != string::npos) {
-      baseUrl = "/" + entry.getPath();
       pthread_mutex_lock(&regexLock);
       content = replaceRegex(content,
                              "$1$2" + rootLocation + "/" + humanReadableBookId + "/$3/",
@@ -795,10 +824,6 @@ static struct MHD_Response* handle_content(RequestContext* request)
       content = replaceRegex(content,
                              "$1$2" + rootLocation + "/" + humanReadableBookId + "/$3/",
                              "(@import[ ]+)([\"|\']{0,1})/([A-Z|\\-])/");
-      content = replaceRegex(
-          content,
-          "<head><base href=\"" + rootLocation + "/" + humanReadableBookId + baseUrl + "\" />",
-          "<head>");
       pthread_mutex_unlock(&regexLock);
       introduceTaskbar(content, humanReadableBookId);
     } else if (mimeType.find("text/css") != string::npos) {
@@ -899,12 +924,24 @@ static int accessHandlerCallback(void* cls,
   return ret;
 }
 
+bool hasHumanReadableIdCollision(const string &humanReadableId,
+                                 const string &zimPath)
+{
+  if (readers.find(humanReadableId) != readers.end()) {
+    cerr << "Path collision: " << readers[humanReadableId]->getZimFilePath()
+         << " and " << zimPath << " can't share the same URL path /"
+         << humanReadableId << "/. Therefore, only "
+         << zimPath << " will be served." << endl;
+    return true;
+  }
+  return false;
+}
+
 int main(int argc, char** argv)
 {
   struct MHD_Daemon* daemon;
   vector<string> zimPathes;
   string libraryPath;
-  string indexPath;
   string rootPath;
   string interface;
   int serverPort = 80;
@@ -919,8 +956,8 @@ int main(int argc, char** argv)
          {"verbose", no_argument, 0, 'v'},
          {"library", no_argument, 0, 'l'},
          {"nolibrarybutton", no_argument, 0, 'm'},
+         {"nodatealiases", no_argument, 0, 'z'},
          {"nosearchbar", no_argument, 0, 'n'},
-         {"index", required_argument, 0, 'i'},
          {"attachToProcess", required_argument, 0, 'a'},
          {"port", required_argument, 0, 'p'},
          {"interface", required_argument, 0, 'f'},
@@ -932,7 +969,7 @@ int main(int argc, char** argv)
   while (true) {
     int option_index = 0;
     int c
-        = getopt_long(argc, argv, "mndvli:a:p:f:t:r:", long_options, &option_index);
+        = getopt_long(argc, argv, "mndvla:p:f:t:r:", long_options, &option_index);
 
     if (c != -1) {
       switch (c) {
@@ -948,12 +985,12 @@ int main(int argc, char** argv)
         case 'n':
           noSearchBarFlag = true;
           break;
+        case 'z':
+          noDateAliasesFlag = true;
+          break;
         case 'm':
           noLibraryButtonFlag = true;
           break;
-        case 'i':
-          indexPath = optarg;
-          break;
         case 'p':
           serverPort = atoi(optarg);
           break;
@@ -983,28 +1020,28 @@ int main(int argc, char** argv)
           break;
       }
     } else {
-      if (optind <= argc) {
+      if (optind < argc) {
         if (libraryFlag) {
           libraryPath = argv[optind++];
         } else {
           while (optind < argc)
             zimPathes.push_back(std::string(argv[optind++]));
         }
-        break;
       }
+      break;
     }
   }
 
   /* Print usage)) if necessary */
   if (zimPathes.empty() && libraryPath.empty()) {
     cerr << "Usage: kiwix-serve [--index=INDEX_PATH] [--port=PORT] [--verbose] "
-            "[--nosearchbar] [--nolibrarybutton] [--daemon] "
+            "[--nosearchbar] [--nolibrarybutton] [--nodatealiases] [--daemon] "
             "[--attachToProcess=PID] [--interface=IF_NAME] "
             "[--urlRootLocation=/URL_ROOT] "
             "[--threads=NB_THREAD(" << nb_threads << ")] ZIM_PATH+"
          << endl;
     cerr << "       kiwix-serve --library [--port=PORT] [--verbose] [--daemon] "
-            "[--nosearchbar] [--nolibrarybutton] [--attachToProcess=PID] "
+            "[--nosearchbar] [--nolibrarybutton] [--nodatealiases] [--attachToProcess=PID] "
             "[--interface=IF_NAME] [--urlRootLocation=/URL_ROOT] "
             "[--threads=NB_THREAD(" << nb_threads << ")] LIBRARY_PATH "
          << endl;
@@ -1014,12 +1051,8 @@ int main(int argc, char** argv)
     exit(1);
   }
 
-  if ((zimPathes.size() > 1) && !indexPath.empty()) {
-    cerr << "You cannot set a indexPath if you also set several zimPathes";
-    exit(1);
-  }
-
   /* Setup the library manager and get the list of books */
+  kiwix::Manager manager(&library);
   if (libraryFlag) {
     vector<string> libraryPaths = kiwix::split(libraryPath, ";");
     vector<string>::iterator itr;
@@ -1033,7 +1066,7 @@ int main(int argc, char** argv)
               = isRelativePath(*itr)
                     ? computeAbsolutePath(getCurrentDirectory(), *itr)
                     : *itr;
-          retVal = libraryManager.readFile(libraryPath, true);
+          retVal = manager.readFile(libraryPath, true);
         } catch (...) {
           retVal = false;
         }
@@ -1047,74 +1080,60 @@ int main(int argc, char** argv)
     }
 
     /* Check if the library is not empty (or only remote books)*/
-    if (libraryManager.getBookCount(true, false) == 0) {
+    if (library.getBookCount(true, false) == 0) {
       cerr << "The XML library file '" << libraryPath
            << "' is empty (or has only remote books)." << endl;
     }
   } else {
     std::vector<std::string>::iterator it;
     for (it = zimPathes.begin(); it != zimPathes.end(); it++) {
-      if (!libraryManager.addBookFromPath(*it, *it, "", false)) {
+      if (!manager.addBookFromPath(*it, *it, "", false)) {
         cerr << "Unable to add the ZIM file '" << *it
              << "' to the internal library." << endl;
         exit(1);
       }
     }
-    if (!indexPath.empty()) {
-      libraryManager.setBookIndex(libraryManager.getBooksIds()[0], indexPath);
-    }
   }
 
   /* Instance the readers and searcher and build the corresponding maps */
-  vector<string> booksIds = libraryManager.getBooksIds();
-  vector<string>::iterator itr;
-  kiwix::Book currentBook;
+  vector<string> booksIds = library.listBooksIds(kiwix::LOCAL);
   globalSearcher = new kiwix::Searcher();
   globalSearcher->setProtocolPrefix(rootLocation + "/");
   globalSearcher->setSearchProtocolPrefix(rootLocation + "/" + "search?");
-  for (itr = booksIds.begin(); itr != booksIds.end(); ++itr) {
-    bool zimFileOk = false;
-    libraryManager.getBookById(*itr, currentBook);
-    std::string zimPath = currentBook.pathAbsolute;
+  for (auto& bookId: booksIds) {
+    auto& currentBook = library.getBookById(bookId);
+    auto zimPath = currentBook.getPath();
 
-    if (!zimPath.empty()) {
-      indexPath = currentBook.indexPathAbsolute;
+    /* Instanciate the ZIM file handler */
+    kiwix::Reader* reader = NULL;
+    try {
+      reader = new kiwix::Reader(zimPath);
+    } catch (...) {
+      cerr << "Unable to open the ZIM file '" << zimPath << "'." << endl;
+      continue;
+    }
 
-      /* Instanciate the ZIM file handler */
-      kiwix::Reader* reader = NULL;
-      try {
-        reader = new kiwix::Reader(zimPath);
-        zimFileOk = true;
-      } catch (...) {
-        cerr << "Unable to open the ZIM file '" << zimPath << "'." << endl;
-      }
+    auto humanReadableId = currentBook.getHumanReadableIdFromPath();
+    hasHumanReadableIdCollision(humanReadableId, currentBook.getPath());
+    readers[humanReadableId] = reader;
 
-      if (zimFileOk) {
-        string humanReadableId = currentBook.getHumanReadableIdFromPath();
-        readers[humanReadableId] = reader;
+    if (reader->hasFulltextIndex()) {
+      kiwix::Searcher* searcher = new kiwix::Searcher(humanReadableId);
+      searcher->setProtocolPrefix(rootLocation + "/");
+      searcher->setSearchProtocolPrefix(rootLocation + "/" + "search?");
+      searcher->add_reader(reader, humanReadableId);
+      globalSearcher->add_reader(reader, humanReadableId);
+      searchers[humanReadableId] = searcher;
+    } else {
+      searchers[humanReadableId] = nullptr;
+    }
 
-        if ( reader->hasFulltextIndex()) {
-          kiwix::Searcher* searcher = new kiwix::Searcher(humanReadableId);
-          searcher->setProtocolPrefix(rootLocation + "/");
-          searcher->setSearchProtocolPrefix(rootLocation + "/" + "search?");
-          searcher->add_reader(reader, humanReadableId);
-          globalSearcher->add_reader(reader, humanReadableId);
-          searchers[humanReadableId] = searcher;
-        } else if ( !indexPath.empty() ) {
-          try {
-            kiwix::Searcher* searcher = new kiwix::Searcher(indexPath, reader, humanReadableId);
-            searcher->setProtocolPrefix(rootLocation + "/");
-            searcher->setSearchProtocolPrefix(rootLocation + "/" + "search?");
-            searchers[humanReadableId] = searcher;
-          } catch (...) {
-            cerr << "Unable to open the search index '" << indexPath << "'."
-                 << endl;
-            searchers[humanReadableId] = nullptr;
-          }
-        } else {
-            searchers[humanReadableId] = nullptr;
-        }
-      }
+    /* Deal with noDateAliases */
+    if (noDateAliasesFlag) {
+      string alias = replaceRegex(humanReadableId, "", "_[[:digit:]]{4}-[[:digit:]]{2}$");
+      hasHumanReadableIdCollision(alias, currentBook.getPath());
+      readers[alias] = readers[humanReadableId];
+      searchers[alias] = searchers[humanReadableId];
     }
   }
 
@@ -1122,24 +1141,25 @@ int main(int argc, char** argv)
   string welcomeBooksHtml
       = ""
         "<div class='book__list'>";
-  for (itr = booksIds.begin(); itr != booksIds.end(); ++itr) {
-    libraryManager.getBookById(*itr, currentBook);
+  for (auto& bookId: booksIds) {
+    auto& currentBook = library.getBookById(bookId);
 
-    if (!currentBook.path.empty()
+    if (!currentBook.getPath().empty()
         && readers.find(currentBook.getHumanReadableIdFromPath())
                != readers.end()) {
       welcomeBooksHtml += ""
 "<a href='" + rootLocation + "/" + currentBook.getHumanReadableIdFromPath() + "/'>"
     "<div class='book'>"
-        "<div class='book__background' style='background-image: url(data:" + currentBook.faviconMimeType+ ";base64," + currentBook.favicon + ");'>"
-            "<div class='book__title' title='" + currentBook.title + "'>" + currentBook.title + "</div>"
-            "<div class='book__description' title='" + currentBook.description + "'>" + currentBook.description + "</div>"
+        "<div class='book__background' style=\"background-image: url('/meta?content="
+             + currentBook.getHumanReadableIdFromPath() + "&name=favicon');\">"
+            "<div class='book__title' title='" + currentBook.getTitle() + "'>" + currentBook.getTitle() + "</div>"
+            "<div class='book__description' title='" + currentBook.getDescription() + "'>" + currentBook.getDescription() + "</div>"
             "<div class='book__info'>"
-                "" + kiwix::beautifyInteger(atoi(currentBook.articleCount.c_str())) + " articles, " + kiwix::beautifyInteger(atoi(currentBook.mediaCount.c_str())) + " medias"
+                "" + kiwix::beautifyInteger(currentBook.getArticleCount()) + " articles, " + kiwix::beautifyInteger(currentBook.getMediaCount()) + " medias"
             "</div>"
         "</div>"
     "</div>"
-"</a>";
+"</a>\n";
     }
   }
   welcomeBooksHtml += ""
@@ -1265,7 +1285,7 @@ int main(int argc, char** argv)
 #endif
 
   } else {
-    daemon = MHD_start_daemon(MHD_USE_POLL_INTERNALLY,
+    daemon = MHD_start_daemon(MHD_USE_SELECT_INTERNALLY,
                               serverPort,
                               NULL,
                               NULL,

src/server/meson.build

@@ -4,5 +4,4 @@ sources += server_resources
 
 executable('kiwix-serve', sources,
   dependencies:all_deps,
-  install:true,
-  install_rpath: join_paths(get_option('prefix'), get_option('libdir')))
+  install:true)

src/server/request_context.cpp

@@ -30,7 +30,6 @@ RequestContext::RequestContext(struct MHD_Connection* connection,
                                const std::string& _url,
                                const std::string& method,
                                const std::string& version) :
-  connection(connection),
   full_url(_url),
   url(_url),
   valid_url(true),

src/server/request_context.h

@@ -23,6 +23,7 @@
 #define REQUEST_CONTEXT_H
 
 #include <string>
+#include <sstream>
 #include <map>
 #include <stdexcept>
 
@@ -84,7 +85,6 @@ class RequestContext {
     int httpResponseCode;
 
   private:
-    struct MHD_Connection* connection;
     std::string full_url;
     std::string url;
     bool valid_url;

static/server/opensearchdescription.xml

@@ -4,5 +4,6 @@
   <Description>Search zim files in the catalog.</Description>
   <Url type="application/atom+xml;profile=opds-catalog" 
        xmlns:atom="http://www.w3.org/2005/Atom"
-       template="/__ROOT_LOCATION__/catalog/search?q={searchTerms}"/>
+       indexOffset="0"
+       template="/__ROOT_LOCATION__/catalog/search?q={searchTerms}&lang={language}&count={count}&start={startIndex}"/>
 </OpenSearchDescription>

travis/compile.sh

@@ -24,6 +24,6 @@ esac
 cd ${TRAVIS_BUILD_DIR}
 export PKG_CONFIG_PATH=${INSTALL_DIR}/lib/x86_64-linux-gnu/pkgconfig
 export PATH=${INSTALL_DIR}/bin:$PATH
-meson . build -Dctpp2-install-prefix=${INSTALL_DIR} ${MESON_OPTION}
+meson . build ${MESON_OPTION}
 cd build
 ninja

travis/install_deps.sh

@@ -3,33 +3,36 @@
 set -e
 
 REPO_NAME=${TRAVIS_REPO_SLUG#*/}
-ARCHIVE_NAME=deps_${TRAVIS_OS_NAME}_${PLATFORM}_${REPO_NAME}.tar.gz
+ARCHIVE_NAME=deps_${TRAVIS_OS_NAME}_${PLATFORM}_${REPO_NAME}.tar.xz
 
 # Packages.
 case ${PLATFORM} in
      "native_static")
-         PACKAGES="gcc cmake libbz2-dev ccache zlib1g-dev uuid-dev libctpp2-dev"
+         PACKAGES="gcc python3.5 cmake libbz2-dev ccache zlib1g-dev uuid-dev"
          ;;
      "native_dyn")
-         PACKAGES="gcc cmake libbz2-dev ccache zlib1g-dev uuid-dev libctpp2-dev libmicrohttpd-dev"
+         PACKAGES="gcc python3.5 cmake libbz2-dev ccache zlib1g-dev uuid-dev libmicrohttpd-dev"
          ;;
      "win32_static")
-         PACKAGES="g++-mingw-w64-i686 gcc-mingw-w64-i686 gcc-mingw-w64-base mingw-w64-tools ccache"
+         PACKAGES="g++-mingw-w64-i686 gcc-mingw-w64-i686 gcc-mingw-w64-base mingw-w64-tools ccache python3.5"
          ;;
      "win32_dyn")
-         PACKAGES="g++-mingw-w64-i686 gcc-mingw-w64-i686 gcc-mingw-w64-base mingw-w64-tools ccache"
+         PACKAGES="g++-mingw-w64-i686 gcc-mingw-w64-i686 gcc-mingw-w64-base mingw-w64-tools ccache python3.5"
          ;;
      "android_arm")
-         PACKAGES="gcc cmake ccache"
+         PACKAGES="gcc python3.5 cmake ccache"
          ;;
      "android_arm64")
-         PACKAGES="gcc cmake ccache"
+         PACKAGES="gcc python3.5 cmake ccache"
          ;;
 esac
 
 sudo apt-get update -qq
 sudo apt-get install -qq python3-pip ${PACKAGES}
-sudo pip3 install meson==0.43.0
+wget https://bootstrap.pypa.io/get-pip.py
+python3.5 get-pip.py --user
+python3.5 -m pip install --user --upgrade pip
+python3.5 -m pip install --user meson==0.49.2
 
 # Ninja
 cd $HOME
@@ -42,6 +45,4 @@ sudo cp ninja /bin
 # Dependencies comming from kiwix-build.
 cd ${HOME}
 wget http://tmp.kiwix.org/ci/${ARCHIVE_NAME}
-mkdir -p BUILD_${PLATFORM}
-cd BUILD_${PLATFORM}
 tar xf ${HOME}/${ARCHIVE_NAME}