115 Commits
1.2.0 ... 3.0.3

Author SHA1 Message Date
Kelson
d6914eb34f Bump-up version to 3.0.3 2020-03-01 15:50:32 +01:00
Kelson
b65212fab7 Merge pull request #364 from kiwix/fix-argument-parsing
Fix argument parsing
2020-03-01 15:49:17 +01:00
Kelson
83427af3d2 Update Changelog 2020-03-01 15:45:31 +01:00
Kelson
d572bf875d Better --version argument parsing 2020-03-01 15:42:44 +01:00
Kelson
833c997782 Remove usell origId option parsing 2020-03-01 15:36:31 +01:00
Kelson
44613aae5d Print information about /data if error 2020-02-18 08:38:18 +01:00
Matthieu Gautier
62e6c7ffb0 Merge pull request #359 from kiwix/new_version
New version 3.0.2
2020-02-13 11:44:29 +01:00
Matthieu Gautier
1df899ea87 New version 3.0.2 2020-02-12 15:23:29 +01:00
Kelson
3dd357e8ac Merge pull request #356 from kiwix/missing-options
added shortcut for options --address and --nodatealiases
2020-02-10 16:42:24 +01:00
renaud gaudin
2744879df0 added shortcut for options --address and --nodatealiases
* allowed shortcut `-i` for `--address`
* allowed shortcut `-z` for `--nodatealiases`
2020-02-10 15:30:31 +00:00
Matthieu Gautier
19c3ebe629 Merge pull request #353 from kiwix/trust_library
[serve] Trust the library by default.
2020-02-05 11:06:05 +01:00
Kelson
cccd17a394 Github Kiwix Sponsoring page link 2020-02-01 17:56:27 +01:00
Matthieu Gautier
e9676ae66f [serve] Trust the library by default.
Now `kiwix-serve` trust the information in the library.xml.
We do not open all zim file at startup and so kiwix-serve starts faster.

Need kiwix/kiwix-lib#314
Fix #319
2020-01-30 17:58:41 +01:00
Matthieu Gautier
2fd393285f Use github action for the CI. (#352)
Use github action for the CI.
2020-01-21 18:52:26 +01:00
Matthieu Gautier
a44fd1a4d0 Use github action for the CI. 2020-01-21 18:50:58 +01:00
Kelson
a9ff2aebee Simpliciation of README 2020-01-16 16:02:59 +01:00
Kelson
a2b944618f More debug messages at kiwix-serve start 2020-01-16 14:57:53 +01:00
Kelson
d27abb41b7 Fix wrong URL in Kiwix serve Docker documentation 2020-01-16 14:49:17 +01:00
Kelson
5f12d0349f Use latest version of kiwix-serve in Dockerfile 2020-01-16 14:19:46 +01:00
Kelson
7ec2308053 Add ability to download a file at container start 2020-01-16 14:04:03 +01:00
Kelson
e8b33a7136 Merge pull request #326 from kiwix/version
Version
2019-12-05 02:06:42 -05:00
Kelson
ca943b9d0b Rename VERSION to KIWIX_TOOLS_VERSION to avoid collision 2019-12-05 07:59:57 +01:00
Kelson
c6757dd309 Various README.md improvements 2019-12-05 07:59:07 +01:00
Kelson
23b0ff3d01 Update changelog 2019-12-05 07:57:52 +01:00
Kelson
93116c76b3 Add --version option to kiwix-read 2019-12-05 07:57:52 +01:00
Kelson
4aca5ff551 Add --version option to kiwix-search 2019-12-05 07:57:52 +01:00
Kelson
4eb38869c1 New --version argument to kiwix-manage 2019-12-05 07:57:52 +01:00
Kelson
d17fd2af94 Add --version argument to kiwix-serve 2019-12-05 07:57:52 +01:00
Kelson
48db92af09 Move version.h to src/ 2019-12-05 07:57:52 +01:00
Kelson
612081ec51 Compile with Meson version #defined 2019-12-05 07:57:52 +01:00
Kelson
4b5eb482b5 New version() function 2019-12-05 07:57:52 +01:00
Kelson
28997ccd58 Bump-up version to 3.0.2 2019-12-05 07:57:52 +01:00
Kelson
9dd042d813 travis-ci.org -> travis-ci.com 2019-12-05 06:56:43 +01:00
Kelson
61580bb738 Add 'Distribution' section in README.md 2019-11-23 17:57:12 +01:00
Kelson
dbae6439b2 Use latest version of kiwix-serve 2019-09-29 01:56:45 +02:00
Kelson
d6094b4112 Merge pull request #334 from kiwix/docker-badge-fix
Fix Docker badge
2019-09-08 20:07:33 +02:00
Kelson
64f1aa2408 Fix Docker badge 2019-09-08 20:06:51 +02:00
Kelson
ed1bae76c3 Merge pull request #333 from kiwix/aur-badges
Add AUR badge
2019-09-08 17:10:27 +02:00
Kelson
7418702a07 Add AUR badge 2019-09-08 17:08:09 +02:00
Kelson
796614fd7d New kiwix-serve screenshots 2019-08-27 16:37:22 +02:00
Kelson
2ba9037687 Merge pull request #328 from kiwix/better-readme
Better README.md
2019-08-27 14:59:51 +02:00
Kelson
22be7c0dcf Better README.md 2019-08-27 14:59:03 +02:00
Kelson
3fc9525754 Remove benchmark.sh 2019-08-27 14:46:10 +02:00
Kelson
a9d8f77526 Merge pull request #327 from kiwix/remove-backticks
Remove backticks from benchmark.sh
2019-08-27 14:42:38 +02:00
Kelson
1109b62570 Remove backticks from benchmark.sh 2019-08-27 14:27:44 +02:00
Kelson
3248c0f233 Merge pull request #325 from kiwix/last-kiwix-serve
Use kiwix-serve 3.0.1
2019-08-22 20:49:26 +02:00
Kelson
f8f66288ea Use kiwix-serve 3.0.1 2019-08-22 20:42:49 +02:00
Matthieu Gautier
98c810eb2a Nodatealiases regression fix (#324)
Nodatealiases regression fix
2019-08-22 17:31:51 +02:00
Kelson
50ca36417f Remove [[gnu::unused]] for noDateAliasesFlag variable 2019-08-22 17:28:25 +02:00
Kelson
5a2714c9c9 Fix --nodatealiases inverted logic regression 2019-08-22 17:23:18 +02:00
Kelson
63cc513974 Bump-up version to 3.0.1 2019-08-22 17:23:11 +02:00
Matthieu Gautier
63ec03a3d4 New version (#320)
New version
2019-08-20 14:32:54 +02:00
Matthieu Gautier
e2f7c7a0ac New version 3.0.0 2019-08-19 17:26:37 +02:00
Matthieu Gautier
a14cd5de23 Fix kiwix-serve usage message. 2019-08-19 17:26:28 +02:00
Matthieu Gautier
06c31430c5 Port kiwix-serve to new api in kiwix-lib to serve a content. (#315)
Port kiwix-serve to new api in kiwix-lib to serve a content.
2019-08-12 15:25:35 +02:00
Matthieu Gautier
8ffff9cd47 Port kiwix-serve to new api in kiwix-lib to serve a content.
A lot of code removed from this repository !!
2019-08-11 11:59:21 +02:00
Kelson
ef0fc514ad Configure the Move bot 2019-08-05 10:32:37 +02:00
Kelson
6c1739d4cd Merge pull request #314 from kiwix/nolibrarybutton-fix
Fix buggy --nolibrarybutton (regression)
2019-08-04 22:15:40 +02:00
Kelson
cd44ba53a5 Fix buggy --nolibrarybutton (regression) 2019-08-04 20:19:47 +02:00
Matthieu Gautier
d82f9e7d5e Merge pull request #312 from kiwix/new_version
New version 2.1.0
2019-07-31 14:33:54 +02:00
Matthieu Gautier
720a319d53 New version 2.1.0 2019-07-30 17:46:38 +02:00
Matthieu Gautier
2426c35547 Merge pull request #306 from kiwix/multiple-tags-api
search api accept multiple tags parameter
2019-07-30 17:32:27 +02:00
luddens
794f63e22b search api accept multiple tags parameter 2019-07-30 16:56:48 +02:00
Matthieu Gautier
79a64d5af3 Merge pull request #305 from kiwix/notag-parameter
add notag parameter to the api
2019-07-29 14:51:31 +02:00
luddens
2ef160bcad add notag parameter to the api
this parameter get a string that contains all tags to exclude separated by
a ";". The kiwix::Filter() is set with this rejected tags.
2019-07-25 15:17:42 +02:00
Kelson
817a5d4847 Merge pull request #311 from kiwix/tools-details
Infos about the tools
2019-07-21 15:58:08 +02:00
Kelson
2c3882671b Infos about the tools 2019-07-21 15:36:27 +02:00
Kelson
7e2c5bda03 Merge pull request #310 from kiwix/better-taskbar-padding
Increase a bit the taskbar padding
2019-07-13 07:19:40 +02:00
Kelson
59281bf496 Increase a bit the taskbar padding 2019-07-12 21:33:37 +02:00
Kelson
ee9ae0e81f Merge pull request #309 from kiwix/better-taskbar-padding
Better kiwix-serve content (because of taskbar) top-padding
2019-07-12 20:18:25 +02:00
Kelson
2dbe0a0016 Better kiwix-serve content (because of taskbar) top-padding 2019-07-12 20:14:00 +02:00
Kelson
64717c2124 Merge pull request #300 from kiwix/ISNIT0/server-toolbar
Make server toolbar auto-hide and collapse buttons into dropdown on small screens
2019-07-09 12:36:23 +02:00
Kelson
fc1eb3c282 Bump-up kiwix-tools version to 2.1.0 2019-07-09 12:31:25 +02:00
Kelson
e31c1cd9a3 Update Changelog 2019-07-09 12:31:22 +02:00
Joe Reeve
e897dd4e59 Update classes and ids to have Kiwix namespace 2019-07-09 12:24:57 +02:00
Joe Reeve
7db7d2a2ab Switch caret to image and move JS to separate file 2019-07-09 12:24:57 +02:00
Joe Reeve
5b612e67bd Fix a few bugs found in code review 2019-07-09 12:24:57 +02:00
Joe Reeve
df047588a9 fix kiwix-serve toolbar search icon on article pages 2019-07-09 12:24:57 +02:00
Joe Reeve
9dede26717 Search icon positioning fix for Firefox 2019-07-09 12:24:57 +02:00
Joe Reeve
a22ae3d467 👌 updates from pr review 2019-07-09 12:24:57 +02:00
Joe Reeve
ea93a8a195 Fix padding and search bug, update button toggle icon 2019-07-09 12:24:57 +02:00
Joe Reeve
0a7f375f3a Format taskbar.css 2019-07-09 12:24:57 +02:00
Joe Reeve
66c35b7b3e Fix alignment, improve search box appearance 2019-07-09 12:24:57 +02:00
Joe Reeve
bad7f2ddf1 Make server toolbar auto-hide and collapse buttons into dropdown on small screens 2019-07-09 12:24:57 +02:00
Matthieu Gautier
758014a5ba Merge pull request #304 from kiwix/int_retcode
Use integer to store the returnCode.
2019-07-01 11:24:22 +02:00
Matthieu Gautier
4fd961a761 Use integer to store the returnCode.
If we want to return `-1` we must use a integer, not a bool.
2019-06-27 19:15:41 +02:00
Matthieu Gautier
c063587a69 Merge pull request #303 from kiwix/no_unistd
Remove include of unistd.h
2019-06-27 18:49:13 +02:00
Matthieu Gautier
9b24676b02 Remove include of unistd.h
unistd.h is useless.
2019-06-27 18:20:22 +02:00
Matthieu Gautier
fc5572f495 Merge pull request #302 from kiwix/adapt_to_new_book_filter
Adapt kiwix-serve to the new api to filter library.
2019-06-27 09:55:50 +02:00
Matthieu Gautier
a1b3f46e4e Adapt kiwix-serve to the new api to filter library.
See kiwix/kiwix-lib#234
2019-06-26 17:38:35 +02:00
Kelson
e0177c4923 Create FUNDING.yml 2019-06-22 07:58:19 +02:00
Matthieu Gautier
a365c1e720 Merge pull request #297 from kiwix/new_version
New version 2.0.0
2019-05-29 12:00:23 +02:00
Matthieu Gautier
3e97352128 [CI] Build on xenial. 2019-05-29 11:52:03 +02:00
Matthieu Gautier
78d2530878 New version 2.0.0 2019-05-29 11:52:03 +02:00
Kelson
d09f481b47 Merge pull request #298 from kiwix/better_usage
Small updates of the README
2019-05-29 10:37:27 +02:00
Emmanuel Engelhart
e0f07086d5 Small updates of the README 2019-05-29 10:29:48 +02:00
Kelson
bbff57e6aa Merge pull request #290 from kiwix/add_footer
Add footer
2019-05-14 13:29:47 +02:00
Kelson
3c24d0a026 Add kiwix-serve welcome page footer 2019-05-04 14:09:26 +02:00
Kelson
6107bad17e Remove deprecated js code 2019-05-04 13:44:06 +02:00
Kelson
5931ac45c4 Merge pull request #289 from kiwix/fix_multiple_add
Correctly set the zimPathToSave if there are several zim file to add.
2019-05-03 12:50:19 +02:00
Matthieu Gautier
4de8623a52 Correctly set the zimPathToSave if there are several zim file to add.
The variable `zimPathToSave` is used to store the option passed by the
user.
If we reuse this variable to store the real path to use for the first
zim file, then we will mess the path to save of other zim files.

Let's use a local variable here.

Fix #288
2019-05-03 12:09:23 +02:00
Kelson
6eace310a2 Merge pull request #284 from kiwix/better_kiwix-serve_usage
Better kiwix-serve usage()
2019-05-03 11:49:58 +02:00
Kelson
c6499ad5c6 Fix typo in kiwix-manage usage() 2019-05-03 11:41:18 +02:00
Emmanuel Engelhart
34ceaf37f2 Better kiwix-serve usage() 2019-05-03 11:38:35 +02:00
Kelson
f05096e2ce Merge pull request #282 from kiwix/better_usage
Better kiwix-manage usage()
2019-05-03 11:31:48 +02:00
Emmanuel Engelhart
3864b2404c Better kiwix-manage usage() 2019-05-03 11:23:53 +02:00
Kelson
4cd74dca4b Merge pull request #283 from kiwix/remove_download_action
Remove kiwix-manage download action
2019-05-03 10:50:11 +02:00
Kelson
88cea54520 Revert "[kiwix-manage] Add a download command to kiwix-manage."
This reverts commit c1e635bd4e.
2019-05-03 10:42:14 +02:00
Matthieu Gautier
3061c8bba3 Merge pull request #281 from kiwix/thread_win
Do not use this_thread::sleep or thread::hardware_concurrency.
2019-04-17 17:57:03 +02:00
Matthieu Gautier
2774eb9529 Do not use this_thread::sleep or thread::hardware_concurrency.
Mingw doesn't implement them. So, we should not use it.
I suppose that it was working before because mingw package for debian trusty
simply no provides a "thread" header.
We may face to include the native "thread" header.
2019-04-17 15:48:33 +00:00
Kelson
77dc2ef2ed Merge pull request #280 from kiwix/3_columns_fix
Display cards in 3 columns properly
2019-04-17 16:44:18 +02:00
Emmanuel Engelhart
17d11f9772 Display cards in 3 columns properly 2019-04-17 09:12:23 +02:00
Matthieu Gautier
68ac06c397 Merge pull request #278 from kiwix/fix_server
Fix server
2019-04-16 14:37:58 +02:00
Matthieu Gautier
99f6941614 New version 1.2.1 2019-04-16 14:30:54 +02:00
Matthieu Gautier
848d058610 Always use POLL wen possible.
SELECT works only with fd < 1024. If we've got "too many" zim files,
the fd of the socket is > 1024 and we cannot create the daemon.

Also add the MHD_USE_DEBUG flag when we are verbose.

Fix #277
2019-04-16 14:29:06 +02:00
51 changed files with 482 additions and 11816 deletions

12
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,12 @@
# These are supported funding model platforms
github: kiwix # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username
custom: # https://kiwix.org/support-us/

27
.github/move.yml vendored Normal file
View File

@@ -0,0 +1,27 @@
# Configuration for Move Issues - https://github.com/dessant/move-issues
# Delete the command comment when it contains no other content
deleteCommand: true
# Close the source issue after moving
closeSourceIssue: true
# Lock the source issue after moving
lockSourceIssue: false
# Mention issue and comment authors
mentionAuthors: true
# Preserve mentions in the issue content
keepContentMentions: true
# Move labels that also exist on the target repository
moveLabels: true
# Set custom aliases for targets
# aliases:
# r: repo
# or: owner/repo
# Repository to extend settings from
# _extends: repo

77
.github/workflows/pr.yml vendored Normal file
View File

@@ -0,0 +1,77 @@
name: CI
on: [pull_request]
jobs:
Linux:
strategy:
fail-fast: false
matrix:
target:
- native_static
- native_dyn
- win32_static
- win32_dyn
include:
- target: native_static
image_variant: xenial
lib_postfix: '/x86_64-linux-gnu'
- target: native_dyn
image_variant: xenial
lib_postfix: '/x86_64-linux-gnu'
- target: win32_static
image_variant: f31
lib_postfix: '64'
- target: win32_dyn
image_variant: f31
lib_postfix: '64'
env:
HOME: /home/runner
runs-on: ubuntu-latest
container:
image: "kiwix/kiwix-build_ci:${{matrix.image_variant}}-26"
steps:
- name: Checkout code
shell: python
run: |
import json
from subprocess import check_call
from os import environ
with open(environ['GITHUB_EVENT_PATH'], 'r') as f:
content = f.read()
event_data = json.loads(content)
try:
branch_ref = event_data['ref'].split('/')[-1]
except KeyError:
branch_ref = event_data['pull_request']['head']['ref']
print("Cloning branch", branch_ref)
command = [
'git', 'clone',
'https://github.com/${{github.repository}}',
'--depth=1',
'--branch', branch_ref
]
check_call(command, cwd=environ['HOME'])
- name: Install deps
shell: bash
run: |
ARCHIVE_NAME=deps2_${OS_NAME}_${{matrix.target}}_kiwix-tools.tar.xz
wget -O- http://tmp.kiwix.org/ci/${ARCHIVE_NAME} | tar -xJ -C /home/runner
- name: Compile
shell: bash
run: |
if [[ "${{matrix.target}}" =~ .*_static ]]; then
MESON_OPTION="-Dstatic-linkage=true"
else
MESON_OPTION=""
fi
if [[ ! "${{matrix.target}}" =~ native_.* ]]; then
MESON_OPTION="$MESON_OPTION --cross-file $HOME/BUILD_${{matrix.target}}/meson_cross_file.txt"
fi
cd $HOME/kiwix-tools
meson . build ${MESON_OPTION}
cd build
ninja
env:
PKG_CONFIG_PATH: "/home/runner/BUILD_${{matrix.target}}/INSTALL/lib/pkgconfig:/home/runner/BUILD_${{matrix.target}}/INSTALL/lib${{matrix.lib_postfix}}/pkgconfig"
CPPFLAGS: "-I/home/runner/BUILD_${{matrix.target}}/INSTALL/include"

View File

@@ -1,12 +0,0 @@
language: cpp
dist: trusty
sudo: required
cache: ccache
install: travis/install_deps.sh
script: travis/compile.sh
env:
matrix:
- PLATFORM="native_static"
- PLATFORM="native_dyn"
- PLATFORM="win32_static"
- PLATFORM="win32_dyn"

View File

@@ -1,3 +1,66 @@
kiwix-tools 3.0.3
=================
* [MANAGER] Fix broken --version argument parsin
kiwix-tools 3.0.2
=================
* New option --version for all tools
* Remove benchmark.sh file.
* [DOCKER] Add ability to download a file at container start.
* [CI] Move to github actions instead of travis.
* [SERVER] Trust the given library by default.
* [SERVER] Add shortcut alias for option `--address` and `--nodatealias`
kiwix-tools 3.0.1
=================
* Fix --nodatealiases inverted logic regression
kiwix-tools 3.0.0
=================
* Move kiwix-serve implementation in kiwix-lib.
kiwix-tools 2.1.0
=================
* Fix few compilation errors.
kiwix-serve
-----------
* Use new api to filter the library.
* Mobile friendly top bar.
* Add notag parameter to be able to exclude tags from the zim search.
kiwix-tools 2.0.0
=================
kiwix-manage
-----------
* Better usage()
* Adding multiple files bug fix
* Remove download command.
kiwix-serve
-----------
* Better usage()
* Display properly welcome page on 3 columns
* New welcome page footer "Powered by Kiwix"
kiwix-tools 1.2.1
=================
kiwix-serve
-----------
* Always use POLL when avaible.
kiwix-tools 1.2.0
=================

View File

@@ -1,7 +1,29 @@
Kiwix tools
===========
The Kiwix tools is a collection of Kiwix related command line tools.
The Kiwix tools is a collection of [Kiwix](https://kiwix.org) related
command line tools:
* kiwix-manage: Manage XML based library of ZIM files
* kiwix-read: Read ZIM file content
* kiwix-search: Fulltext search in ZIM files
* kiwix-serve: HTTP daemon serving ZIM files
[![Build Status](https://travis-ci.com/kiwix/kiwix-tools.svg?branch=master)](https://travis-ci.com/kiwix/kiwix-tools)
[![CodeFactor](https://www.codefactor.io/repository/github/kiwix/kiwix-tools/badge)](https://www.codefactor.io/repository/github/kiwix/kiwix-tools)
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
Distribution
------------
#### Official
[![latest release](https://img.shields.io/github/v/tag/kiwix/kiwix-tools?label=latest%20release&sort=semver)](https://download.kiwix.org/release/kiwix-tools/)
[![Docker](https://img.shields.io/docker/cloud/build/kiwix/kiwix-serve)](https://hub.docker.com/r/kiwix/kiwix-serve)
#### Contribution
[![AUR](https://img.shields.io/aur/version/kiwix-tools)](https://aur.archlinux.org/packages/kiwix-tools/)
[![Sandstorm](https://img.shields.io/badge/Sandstorm-kiwix-blue)](https://apps.sandstorm.io/app/5uh349d0kky2zp5whrh2znahn27gwha876xze3864n0fu9e5220h)
Disclaimer
----------
@@ -17,7 +39,9 @@ Preamble
Although the Kiwix tools can be compiled/cross-compiled on/for many
sytems, the following documentation explains how to do it on POSIX
ones. It is primarly thought for GNU/Linux systems and has been tested
on recent releases of Ubuntu and Fedora.
on recent releases of
[Debian](https://debian.org)/[Ubuntu](https://ubuntu.com) and
[Fedora](https://getfedora.org).
Dependencies
------------
@@ -25,15 +49,10 @@ Dependencies
The Kiwix tools rely on a few third party software libraries. They are
prerequisites to the Kiwix tools compilation. Therefore, following
libraries need to be available:
* Kiwix lib ....................... https://github.com/kiwix/kiwix-lib
(no package so far)
* Libmicrohttpd .......... https://www.gnu.org/software/libmicrohttpd/
(package libmicrohttpd-dev on Ubuntu)
* CTPP2 ..................................... http://ctpp.havoc.ru/en/
(package libctpp2-dev on Ubuntu)
* Zlib .......................................... http://www.zlib.net/
(package zlib1g-dev on Ubuntu)
* [Kiwix lib](https://github.com/kiwix/kiwix-lib) (no package so far)
* [Libmicrohttpd](https://www.gnu.org/software/libmicrohttpd/)
(package `libmicrohttpd-dev` on Debian/Ubuntu)
* [Zlib](https://www.zlib.net/) (package `zlib1g-dev` on Debian/Ubuntu)
These dependencies may or may not be packaged by your operating
system. They may also be packaged but only in an older version. They
@@ -46,8 +65,8 @@ If you want to install these dependencies locally, then use the
kiwix-tools directory as install prefix.
If you want to compile Kiwix tools statically, the dependencies should
be compile statically (provide a lib...a library), for example by
using "--enable-static" with "./configure".
be compiled statically (provide a `lib...a` library), for example by
using `--enable-static` with `./configure`.
If you compile manually Libmicrohttpd, you might need to compile it
without GNU TLS, a bug here will empeach further compilation of Kiwix
@@ -57,20 +76,21 @@ Environment
-------------
The Kiwix tools build using [Meson](http://mesonbuild.com/) version
0.39 or higher. Meson relies itself on Ninja, pkg-config and few other
0.43 or higher. Meson relies itself on Ninja, pkg-config and few other
compilation tools. Install them first:
* Meson
* Ninja
* Pkg-config
* [Meson](http://mesonbuild.com/)
* [Ninja](https://ninja-build.org/)
* [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/)
These tools should be packaged if you use a cutting edge operating
system. If not, have a look to the "Troubleshooting" section.
system. If not, have a look to the [Troubleshooting](#Troubleshooting)
section.
Compilation
-----------
Once all dependencies are installed, you can compile Kiwix tools with:
```
```bash
meson . build
ninja -C build
```
@@ -85,38 +105,35 @@ Installation
------------
If you want to install the Kiwix tools, here we go:
```
```bash
ninja -C build install
```
You might need to run the command as root (or using 'sudo'), depending
where you want to install the Kiwix tools. After the installation
succeeded, you may need to run ldconfig (as root).
You might need to run the command as `root` (or using `sudo`),
depending where you want to install the Kiwix tools. After the
installation succeeded, you may need to run ldconfig (as `root`).
Uninstallation
------------
If you want to uninstall the Kiwix tools:
```
```bash
ninja -C build uninstall
```
Like for the installation, you might need to run the command as root
(or using 'sudo').
Like for the installation, you might need to run the command as `root`
(or using `sudo`).
Docker
------
An official Docker image of `kiwix-serve` can be found at
https://hub.docker.com/r/kiwix/kiwix-serve.
An official Docker image of `kiwix-serve` can be found in the [Docker Hub](https://hub.docker.com/r/kiwix/kiwix-serve).
Troubleshooting
---------------
If you need to install Meson "manually":
```
```bash
virtualenv -p python3 ./ # Create virtualenv
source bin/activate # Activate the virtualenv
pip3 install meson # Install Meson
@@ -124,7 +141,7 @@ hash -r # Refresh bash paths
```
If you need to install Ninja "manually":
```
```bash
git clone git://github.com/ninja-build/ninja.git
cd ninja
git checkout release
@@ -143,4 +160,5 @@ repository.
License
-------
GPLv3 or later, see COPYING for more details.
[GPLv3](https://www.gnu.org/licenses/gpl-3.0) or later, see
[COPYING](COPYING) for more details.

View File

@@ -4,7 +4,7 @@ LABEL maintainer Emmanuel Engelhart <kelson@kiwix.org>
# Install kiwix-serve
WORKDIR /
RUN apk add --no-cache curl bzip2
RUN curl -kL https://download.kiwix.org/release/kiwix-tools/kiwix-tools_linux-x86_64-1.1.0.tar.gz | tar -xz && \
RUN curl -kL https://download.kiwix.org/release/kiwix-tools/kiwix-tools_linux-x86_64.tar.gz | tar -xz && \
mv kiwix-tools*/kiwix-serve /usr/local/bin && \
rm -r kiwix-tools*
@@ -12,4 +12,7 @@ RUN curl -kL https://download.kiwix.org/release/kiwix-tools/kiwix-tools_linux-x8
EXPOSE 80
VOLUME /data
WORKDIR /data
ENTRYPOINT ["/usr/local/bin/kiwix-serve", "--port", "80"]
COPY ./start.sh /usr/local/bin/
ENTRYPOINT ["/usr/local/bin/start.sh"]

View File

@@ -1,13 +1,26 @@
kiwix-serve Docker image
Kiwix serve Docker image
========================
With local ZIM file
-------------------
* Download a ZIM file from <https://wiki.kiwix.org/wiki/Content>
* Given `wikipedia.zim` resides in `/tmp/zim/`, execute the following:
```
```bash
docker run -v /tmp/zim:/data -p 8080:80 kiwix/kiwix-serve wikipedia.zim
```
With remote ZIM file
--------------------
```bash
docker run -e "DOWNLOAD=https://download.kiwix.org/zim/wikipedia_bm_all.zim" -p 8080:80 kiwix/kiwix-serve
```
Screenshots
-----------
![screenshot_1.png](https://github.com/kiwix/kiwix-tools/raw/master/docker/server/pictures/screenshot_1.png)
![screenshot_2.png](https://github.com/kiwix/kiwix-tools/raw/master/docker/server/pictures/screenshot_2.png)

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 19 KiB

After

Width:  |  Height:  |  Size: 187 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 199 KiB

After

Width:  |  Height:  |  Size: 302 KiB

25
docker/server/start.sh Executable file
View File

@@ -0,0 +1,25 @@
#!/bin/sh
# Download if necessary a file
if [ ! -z "$DOWNLOAD" ]
then
ZIM=`basename $DOWNLOAD`
wget $DOWNLOAD -O "$ZIM"
# Set arguments
if [ "$#" -eq "0" ]
then
set -- "$@" $ZIM
fi
fi
CMD="/usr/local/bin/kiwix-serve --port=80 $@"
echo $CMD
$CMD
# If error, print the content of /data
if [ $? -ne 0 ]
then
echo "Here is the content of /data:"
find /data -type f
fi

View File

@@ -1,17 +1,19 @@
project('kiwix-tools', 'cpp',
version : '1.2.0',
version : '3.0.3',
license : 'GPL',
default_options: ['c_std=c11', 'cpp_std=c++11', 'werror=true'])
compiler = meson.get_compiler('cpp')
add_global_arguments('-DKIWIX_TOOLS_VERSION="@0@"'.format(meson.project_version()), language : 'cpp')
static_linkage = get_option('static-linkage')
if static_linkage
add_global_link_arguments('-static-libstdc++', '--static', language:'cpp')
endif
thread_dep = dependency('threads')
kiwixlib_dep = dependency('kiwix', version:'>=4.1.0', static:static_linkage)
kiwixlib_dep = dependency('kiwix', version:'>=9.0.0', static:static_linkage)
microhttpd_dep = dependency('libmicrohttpd', static:static_linkage)
z_dep = dependency('zlib', static:static_linkage)
@@ -24,6 +26,4 @@ if static_linkage
endif
endif
#subdir('include')
subdir('static')
subdir('src')

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2011 Emmanuel Engelhart <kelson@kiwix.org>
* Copyright 2011-2019 Emmanuel Engelhart <kelson@kiwix.org>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -17,23 +17,17 @@
* MA 02110-1301, USA.
*/
#ifndef _WIN32
#include <unistd.h>
#endif
#include <getopt.h>
#include <kiwix/tools/pathTools.h>
#include <kiwix/tools/stringTools.h>
#include <kiwix/manager.h>
#include <kiwix/downloader.h>
#include <cstdlib>
#include <iostream>
#include <thread>
#include <chrono>
#include "../version.h"
using namespace std;
enum supportedAction { NONE, ADD, SHOW, REMOVE, DOWNLOAD };
enum supportedAction { NONE, ADD, SHOW, REMOVE };
void show(kiwix::Library* library, const std::string& bookId)
{
@@ -57,21 +51,52 @@ void show(kiwix::Library* library, const std::string& bookId)
std::cout << std::endl;
}
/* Print correct console usage options */
void usage()
{
cerr << "Usage:" << endl;
cerr << "\tkiwix-manage LIBRARY_PATH add ZIM_PATH "
"[--zimPathToSave=../content/foobar.zim] [--current] "
"[--url=http://...metalink]"
<< endl;
cerr << "\tkiwix-manage LIBRARY_PATH show [CONTENTID1] [CONTENTID2] ... "
"(show everything if no param.)"
<< endl;
cerr << "\tkiwix-manage LIBRARY_PATH remove CONTENTID1 [CONTENTID2]" << endl;
std::cout << "Usage:" << std::endl
<< "\tkiwix-manage LIBRARY_PATH add ZIM_PATH [OPTIONS]" << std::endl
<< "\tkiwix-manage LIBRARY_PATH remove ZIM_ID [ZIM_ID]..." << std::endl
<< "\tkiwix-manage LIBRARY_PATH show [ZIM_ID]..." << std::endl
<< std::endl
<< "Purpose:" << std::endl
<< "\tManipulates the Kiwix library XML file"
<< std::endl << std::endl
<< "Arguments:" << std::endl
<< "\tLIBRARY_PATH\tis the XML library file path."
<< std::endl << std::endl
<< "\tACTION\t\tis the pre-defined string to specify the action to run on the XML library file."
<< std::endl << std::endl
<< "\t\t\tMust be one of the following values:" << std::endl
<< "\t\t\t* add: add a ZIM file to the library" << std::endl
<< "\t\t\t* remove: remove a ZIM file from the library" << std::endl
<< "\t\t\t* show: show the content of the library"
<< std::endl << std::endl
<< "\tZIM_ID\t\tZIM file unique ID"
<< std::endl << std::endl
<< "\tOPTIONS\t\tCustom options for \"add\" action:" << std::endl
<< "\t\t\t--zimPathToSave=CUSTOM_ZIM_PATH to replace the current ZIM file path" << std::endl
<< "\t\t\t--url=HTTP_ZIM_URL to create an \"url\" attribute for the online version of the ZIM file" << std::endl
<< std::endl
<< "\t\t\tOther options:" << std::endl
<< "\t\t\t-v, --version to print the software version" << std::endl
<< std::endl
<< "Examples:" << std::endl
<< "\tAdd ZIM files to library: kiwix-manage my_library.xml add first.zim second.zim" << std::endl
<< "\tRemove ZIM files from library: kiwix-manage my_library.xml remove e5c2c003-b49e-2756-5176-5d9c86393dd9" << std::endl
<< "\tShow all library ZIM files: kiwix-manage my_library.xml show" << std::endl
<< std::endl
<< "Documentation:" << std::endl
<< "\tSource code\thttps://github.com/kiwix/kiwix-tools" << std::endl
<< "\tMore info\thttps://wiki.kiwix.org/wiki/Kiwix-manage" << std::endl
<< std::endl;
}
bool handle_show(kiwix::Library* library, const std::string& libraryPath,
int handle_show(kiwix::Library* library, const std::string& libraryPath,
int argc, char* argv[])
{
if (argc > 3 ) {
@@ -88,16 +113,15 @@ bool handle_show(kiwix::Library* library, const std::string& libraryPath,
return(0);
}
bool handle_add(kiwix::Library* library, const std::string& libraryPath,
int handle_add(kiwix::Library* library, const std::string& libraryPath,
int argc, char* argv[])
{
string zimPath;
string zimPathToSave = ".";
string url;
string origID = "";
int option_index = 0;
int c = 0;
bool resultCode = 0;
int resultCode = 0;
if (argc <= 3) {
std::cerr << "Path to zim file to add is missing in the command line" << std::endl;
@@ -108,7 +132,6 @@ bool handle_add(kiwix::Library* library, const std::string& libraryPath,
optind = 3;
static struct option long_options[] = {
{"url", required_argument, 0, 'u'},
{"origId", required_argument, 0, 'o'},
{"zimPathToSave", required_argument, 0, 'z'},
{0, 0, 0, 0}
};
@@ -124,11 +147,6 @@ bool handle_add(kiwix::Library* library, const std::string& libraryPath,
case 'u':
url = optarg;
break;
case 'o':
origID = optarg;
break;
case 'z':
zimPathToSave = optarg;
break;
@@ -150,8 +168,8 @@ bool handle_add(kiwix::Library* library, const std::string& libraryPath,
for(auto i=optind; i<argc; i++) {
std::string zimPath = argv[i];
if (!zimPath.empty()) {
zimPathToSave = zimPathToSave == "." ? zimPath : zimPathToSave;
manager.addBookFromPathAndGetId(zimPath, zimPathToSave, url, false);
auto _zimPathToSave = zimPathToSave == "." ? zimPath : zimPathToSave;
manager.addBookFromPathAndGetId(zimPath, _zimPathToSave, url, false);
} else {
std::cerr << "Invalid zim file path" << std::endl;
resultCode = 1;
@@ -161,12 +179,12 @@ bool handle_add(kiwix::Library* library, const std::string& libraryPath,
return(resultCode);
}
bool handle_remove(kiwix::Library* library, const std::string& libraryPath,
int handle_remove(kiwix::Library* library, const std::string& libraryPath,
int argc, char* argv[])
{
std::string bookId;
const unsigned int totalBookCount = library->getBookCount(true, true);
bool exitCode = 0;
int exitCode = 0;
if (argc <= 3) {
std::cerr << "BookId to remove missing in the command line" << std::endl;
@@ -191,69 +209,33 @@ bool handle_remove(kiwix::Library* library, const std::string& libraryPath,
return(exitCode);
}
bool handle_download(kiwix::Library* library, const std::string& libraryPath,
int argc, char* argv[])
{
std::string bookId;
bool exitCode = false;
if (argc > 3) {
bookId = argv[3];
}
auto& book = library->getBookById(bookId);
auto did = book.getDownloadId();
kiwix::Downloader downloader;
kiwix::Download* download = nullptr;
if (!did.empty()) {
std::cout << "try resume " << did << std::endl;
try {
download = downloader.getDownload(did);
} catch(...) {}
}
if (nullptr == download || download->getStatus() == kiwix::Download::K_UNKNOWN) {
download = downloader.startDownload(book.getUrl());
book.setDownloadId(download->getDid());
}
int step = 60*5;
while (step--) {
download->updateStatus();
if (download->getStatus() == kiwix::Download::K_COMPLETE) {
auto followingId = download->getFollowedBy();
if (followingId.empty()) {
book.setPath(download->getPath());
book.setDownloadId("");
std::cout << "File downloaded to " << book.getPath() << std::endl;
break;
} else {
download = downloader.getDownload(followingId);
}
} else if (download->getStatus() == kiwix::Download::K_ACTIVE) {
std::cout << download->getDid() << " : "
<< kiwix::beautifyFileSize(download->getCompletedLength()) << "/"
<< kiwix::beautifyFileSize(download->getTotalLength())
<< " (" << kiwix::beautifyFileSize(download->getDownloadSpeed()) << "/s) "
<< " [" << kiwix::beautifyFileSize(download->getVerifiedLength()) << "]"
<< "[" << step << "] \n" << std::flush;
} else if (download->getStatus() == kiwix::Download::K_ERROR) {
std::cout << "File Error" << std::endl;
exitCode = true;
break;
}
std::this_thread::sleep_for(std::chrono::seconds(1));
}
downloader.close();
return exitCode;
}
int main(int argc, char** argv)
{
string libraryPath = "";
supportedAction action = NONE;
kiwix::Library library;
/* Argument parsing */
/* General argument parsing */
static struct option long_options[] = {
{"version", no_argument, 0, 'v'},
{0, 0, 0, 0}
};
int option_index = 0;
int c;
while (true && argc == 2) {
c = getopt_long(argc, argv, "v", long_options, &option_index);
if (c == -1)
break;
switch (c) {
case 'v':
version();
return 0;
}
}
/* Action related argument parsing */
if (argc > 2) {
libraryPath = argv[1];
string actionString = argv[2];
@@ -264,8 +246,6 @@ int main(int argc, char** argv)
action = SHOW;
else if (actionString == "remove" || actionString == "delete")
action = REMOVE;
else if (actionString == "download")
action = DOWNLOAD;
}
/* Print usage)) if necessary */
@@ -282,7 +262,7 @@ int main(int argc, char** argv)
manager.readFile(libraryPath, false);
/* SHOW */
bool exitCode = 0;
int exitCode = 0;
switch (action) {
case SHOW:
exitCode = handle_show(&library, libraryPath, argc, argv);
@@ -293,15 +273,12 @@ int main(int argc, char** argv)
case REMOVE:
exitCode = handle_remove(&library, libraryPath, argc, argv);
break;
case DOWNLOAD:
exitCode = handle_download(&library, libraryPath, argc, argv);
break;
case NONE:
break;
}
/* Rewrite the library file */
if (action == REMOVE || action == ADD || action == DOWNLOAD) {
if (action == REMOVE || action == ADD) {
library.writeToFile(libraryPath);
}

View File

@@ -19,13 +19,14 @@
#include <getopt.h>
#include <kiwix/reader.h>
#include <unistd.h>
#include <map>
#include <string>
#include "../version.h"
void usage()
{
cout << "Usage: kiwix-read --suggest=<PATTERN> ZIM_FILE_PATH" << endl;
cout << "Usage: kiwix-read [--verbose] [--version] --suggest=<PATTERN> ZIM_FILE_PATH" << endl;
exit(1);
}
@@ -43,15 +44,19 @@ int main(int argc, char** argv)
while (42) {
static struct option long_options[]
= {{"verbose", no_argument, 0, 'v'},
{"version", no_argument, 0, 'V'},
{"suggest", required_argument, 0, 's'},
{0, 0, 0, 0}};
if (c != -1) {
c = getopt_long(argc, argv, "vs:", long_options, &option_index);
c = getopt_long(argc, argv, "Vvs:", long_options, &option_index);
switch (c) {
case 'v':
break;
case 'V':
version();
return 0;
case 's':
pattern = optarg;
break;

View File

@@ -20,8 +20,8 @@
#include <getopt.h>
#include <kiwix/reader.h>
#include <kiwix/searcher.h>
#include <unistd.h>
#include "../version.h"
void usage()
{
@@ -30,7 +30,8 @@ void usage()
<< " ZIM is the full path of the ZIM file." << endl
<< " PATTERN is/are word(s) - or part of - to search in the ZIM." << endl << endl
<< " -s, --suggestion\tSuggest article titles based on the few letters of the PATTERN instead of making a fulltext search. Work a bit like a completion solution." << endl
<< " -v, --verbose\t\tGive details about the search process" << endl;
<< " -v, --verbose\t\tGive details about the search process" << endl
<< " -V, --version\t\tPrint software version" << endl;
exit(1);
}
@@ -55,15 +56,19 @@ int main(int argc, char** argv)
static struct option long_options[]
= {{"verbose", no_argument, 0, 'v'},
{"suggestion", no_argument, 0, 's'},
{"version", no_argument, 0, 'V'},
{0, 0, 0, 0}};
if (c != -1) {
c = getopt_long(argc, argv, "vsb:", long_options, &option_index);
c = getopt_long(argc, argv, "Vvsb:", long_options, &option_index);
switch (c) {
case 'v':
verboseFlag = true;
break;
case 'V':
version();
return 0;
case 's':
suggestionFlag = true;
break;
@@ -99,7 +104,7 @@ int main(int argc, char** argv)
if (reader) {
searcher = new kiwix::Searcher();
searcher->add_reader(reader, "");
searcher->add_reader(reader);
} else {
cerr << "Unable to search through zim '" << zimPath << "'." << endl;
exit(1);

View File

@@ -1,72 +0,0 @@
#!/bin/bash
# Parse command line
zim=$1
serve=$2
if (( $# != 2 ))
then
echo "Usage: ./benchmark ZIM_PATH KIWIX-SERVE_PATH"
exit 1
fi
# Constants
delay=0.01
top_log=/tmp/top.log
top_calc=/tmp/top.calc
# Available memory
memory_kb=`cat /proc/meminfo | grep "MemTotal" | tr -d " " | cut -f2 -d: | sed -e 's/[^0123456789]//g'`
memory_mb=`echo "scale=2;$memory_kb/1024" | bc -l`
# Start Kiwix-serve
echo "Starting kiwix-serve to be tested..."
$serve --port=8080 "$zim" &
serve_pid=$!
# Start top to monitor resource usage
top -d $delay -b -p $serve_pid | grep $serve_pid > "$top_log" &
top_pid=$!
# Print environnement informations
echo "Process to monitor: $serve_pid"
echo "Monitoring process: $top_pid"
echo "Monitoring delay: $delay s"
echo "ZIM file path: $zim"
echo "Kiwix-serve path: $serve"
echo "Total memory: $memory_mb MB"
echo
# Compute artice list
echo "Computing article list snippet..."
articles=`zimdump -l $zim | grep ".html" | shuf | head -n 1000`
# Run wget against kiwix-serve
start_date=`date +%s`
for LINE in $articles
do
echo "Scrapping $LINE..."
wget --quiet p -P /dev/shm/tmp "http://localhost:8080/wikipedia_en_medicine_2016-09/A/$LINE"
rm -rf /dev/shm/tmp
done
end_date=`date +%s`
# Kill top instance
kill -s STOP $top_pid 2>&1 > /dev/null
sed -i '$ d' "$top_log"
times=`cat "$top_log" | wc -l`
# Compute KPI
duration=`echo "$end_date-$start_date" | bc -l`
cpu_percent=`cat "$top_log" | sed -r -e "s;\s\s*; ;g" -e "s;^ *;;" | cut -d' ' -f9 | tr '\n' '+0' | sed -r -e "s;(.*)[+]$;\1;" -e "s/.*/scale=2\n(&)\/$times\nquit\n/" > "$top_calc" ; bc -q "$top_calc"`
memory_percent=`cat "$top_log" | sed -r -e "s;\s\s*; ;g" -e "s;^ *;;" | cut -d' ' -f10 | tr '\n' '+' | sed -r -e "s;(.*)[+]$;\1;" -e "s/.*/scale=2\n(&)\/$times\nquit\n/" > "$top_calc" ; bc -q "$top_calc"`
memory_absolut=`echo "scale=2;$memory_mb/100*$memory_percent" | bc -l`
echo
echo "Measure count: $times"
echo "Duration: $duration s"
echo "CPU (average): $cpu_percent %"
echo "Memory (average): $memory_absolut MB"
# Kill kiwix-serve & top instances
kill -s STOP $serve_pid 2>&1 > /dev/null
#rm "$top_log" "$top_calc"

View File

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,5 @@
sources = ['kiwix-serve.cpp', 'request_context.cpp']
sources += server_resources
sources = ['kiwix-serve.cpp']
executable('kiwix-serve', sources,
dependencies:all_deps,

View File

@@ -1,201 +0,0 @@
/*
* Copyright 2009-2016 Emmanuel Engelhart <kelson@kiwix.org>
* Copyright 2017 Matthieu Gautier<mgautier@kymeria.fr>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
* MA 02110-1301, USA.
*/
#include "request_context.h"
#include <string.h>
#include <stdexcept>
#include <sstream>
#include <cstdio>
RequestContext::RequestContext(struct MHD_Connection* connection,
std::string rootLocation,
const std::string& _url,
const std::string& method,
const std::string& version) :
full_url(_url),
url(_url),
valid_url(true),
version(version),
acceptEncodingDeflate(false),
accept_range(false),
range_pair(0, -1)
{
if (method == "GET") {
this->method = RequestMethod::GET;
} else if (method == "HEAD") {
this->method = RequestMethod::HEAD;
} else if (method == "POST") {
this->method = RequestMethod::POST;
} else if (method == "PUT") {
this->method = RequestMethod::PUT;
} else if (method == "DELETE") {
this->method = RequestMethod::DELETE_;
} else if (method == "CONNECT") {
this->method = RequestMethod::CONNECT;
} else if (method == "OPTIONS") {
this->method = RequestMethod::OPTIONS;
} else if (method == "TRACE") {
this->method = RequestMethod::TRACE;
} else if (method == "PATCH") {
this->method = RequestMethod::PATCH;
} else {
this->method = RequestMethod::OTHER;
}
MHD_get_connection_values(connection, MHD_HEADER_KIND, &RequestContext::fill_header, this);
MHD_get_connection_values(connection, MHD_GET_ARGUMENT_KIND, &RequestContext::fill_argument, this);
valid_url = true;
if (rootLocation.empty()) {
// nothing special to handle.
url = full_url;
} else {
if (full_url.size() > rootLocation.size() &&
full_url.substr(0, rootLocation.size()+1) == rootLocation + "/") {
url = full_url.substr(rootLocation.size());
} else {
valid_url = false;
}
}
try {
acceptEncodingDeflate =
(get_header(MHD_HTTP_HEADER_ACCEPT_ENCODING).find("deflate") != std::string::npos);
} catch (const std::out_of_range&) {}
/*Check if range is requested. */
try {
auto range = get_header(MHD_HTTP_HEADER_RANGE);
int start = 0;
int end = -1;
std::istringstream iss(range);
char c;
iss >> start >> c;
if (iss.good() && c=='-') {
iss >> end;
if (iss.fail()) {
// Something went wrong will extracting.
end = -1;
}
if (iss.eof()) {
accept_range = true;
range_pair = std::pair<int, int>(start, end);
}
}
} catch (const std::out_of_range&) {}
}
RequestContext::~RequestContext()
{}
int RequestContext::fill_header(void *__this, enum MHD_ValueKind kind,
const char *key, const char *value)
{
RequestContext *_this = static_cast<RequestContext*>(__this);
_this->headers[key] = value;
return MHD_YES;
}
int RequestContext::fill_argument(void *__this, enum MHD_ValueKind kind,
const char *key, const char* value)
{
RequestContext *_this = static_cast<RequestContext*>(__this);
_this->arguments[key] = value == nullptr ? "" : value;
return MHD_YES;
}
void RequestContext::print_debug_info() {
printf("method : %s (%d)\n", method==RequestMethod::GET ? "GET" :
method==RequestMethod::POST ? "POST" :
"OTHER", (int)method);
printf("version : %s\n", version.c_str());
printf("headers :\n");
for (auto it=headers.begin(); it!=headers.end(); it++) {
printf(" - %s : '%s'\n", it->first.c_str(), it->second.c_str());
}
printf("arguments :\n");
for (auto it=arguments.begin(); it!=arguments.end(); it++) {
printf(" - %s : '%s'\n", it->first.c_str(), it->second.c_str());
}
printf("Parsed : \n");
printf("url : %s\n", url.c_str());
printf("acceptEncodingDeflate : %d\n", acceptEncodingDeflate);
printf("has_range : %d\n", accept_range);
printf(".............\n");
}
RequestMethod RequestContext::get_method() {
return method;
}
std::string RequestContext::get_url() {
return url;
}
std::string RequestContext::get_url_part(int number) {
size_t start = 1;
while(true) {
auto found = url.find('/', start);
if (number == 0) {
if (found == std::string::npos) {
return url.substr(start);
} else {
return url.substr(start, found-start);
}
} else {
if (found == std::string::npos) {
throw std::out_of_range("No parts");
}
start = found + 1;
number -= 1;
}
}
}
std::string RequestContext::get_full_url() {
return full_url;
}
bool RequestContext::is_valid_url() {
return valid_url;
}
bool RequestContext::has_range() {
return accept_range;
}
std::pair<int, int> RequestContext::get_range() {
return range_pair;
}
template<>
std::string RequestContext::get_argument(const std::string& name) {
return arguments.at(name);
}
std::string RequestContext::get_header(const std::string& name) {
return headers.at(name);
}

View File

@@ -1,108 +0,0 @@
/*
* Copyright 2009-2016 Emmanuel Engelhart <kelson@kiwix.org>
* Copyright 2017 Matthieu Gautier<mgautier@kymeria.fr>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
* MA 02110-1301, USA.
*/
#ifndef REQUEST_CONTEXT_H
#define REQUEST_CONTEXT_H
#include <string>
#include <sstream>
#include <map>
#include <stdexcept>
extern "C" {
#include <microhttpd.h>
}
enum class RequestMethod {
GET,
HEAD,
POST,
PUT,
DELETE_,
CONNECT,
OPTIONS,
TRACE,
PATCH,
OTHER
};
class KeyError : public std::runtime_error {};
class IndexError: public std::runtime_error {};
class RequestContext {
public:
RequestContext(struct MHD_Connection* connection,
std::string rootLocation,
const std::string& url,
const std::string& method,
const std::string& version);
~RequestContext();
void print_debug_info();
bool is_valid_url();
std::string get_header(const std::string& name);
template<typename T=std::string>
T get_argument(const std::string& name) {
std::istringstream stream(arguments.at(name));
T v;
stream >> v;
return v;
}
RequestMethod get_method();
std::string get_url();
std::string get_url_part(int part);
std::string get_full_url();
bool has_range();
std::pair<int, int> get_range();
bool can_compress() { return acceptEncodingDeflate; }
// [TODO] Move this to the response builder
int httpResponseCode;
private:
std::string full_url;
std::string url;
bool valid_url;
RequestMethod method;
std::string version;
bool acceptEncodingDeflate;
bool accept_range;
std::pair<int, int> range_pair;
std::map<std::string, std::string> headers;
std::map<std::string, std::string> arguments;
static int fill_header(void *, enum MHD_ValueKind, const char*, const char*);
static int fill_argument(void *, enum MHD_ValueKind, const char*, const char*);
};
template<> std::string RequestContext::get_argument(const std::string& name);
#endif //REQUEST_CONTEXT_H

32
src/version.h Normal file
View File

@@ -0,0 +1,32 @@
/*
* Copyright 2009-2016 Emmanuel Engelhart <kelson@kiwix.org>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
* MA 02110-1301, USA.
*/
#ifndef _KIWIX_TOOLS_VERSION_H_
#define _KIWIX_TOOLS_VERSION_H_
#ifndef KIWIX_TOOLS_VERSION
#define KIWIX_TOOLS_VERSION "undefined"
#endif
void version()
{
std::cout << KIWIX_TOOLS_VERSION << std::endl;
}
#endif //_KIWIX_TOOLs_VERSION_H_

View File

@@ -1,4 +0,0 @@
res_compiler = find_program('kiwix-compile-resources', 'compile_resources.py')
subdir('server')

View File

@@ -1,13 +0,0 @@
<span class="kiwix">
<span id="kiwixtoolbar" class="ui-widget-header">
<div class="kiwix_centered">
<div class="kiwix_searchform">
<form class="kiwixsearch" method="GET" action="__ROOT_LOCATION__/search" id="kiwixsearchform">
<input autocomplete="off" class="ui-autocomplete-input" id="kiwixsearchbox" name="pattern" type="text">
<input type="submit" value="&#x1f50d;">
</form>
</div>
</div>
</span>
</span>
<div style="display: block; height: 5em;"></div>

View File

@@ -1,55 +0,0 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta charset="UTF-8" />
<title>Welcome to Kiwix Server</title>
<script type="text/javascript" src="__ROOT_LOCATION__/skin/jquery-ui/external/jquery/jquery.js"></script>
<script type="text/javascript" src="__ROOT_LOCATION__/skin/jquery-ui/jquery-ui.min.js"></script>
<link type="text/css" href="__ROOT_LOCATION__/skin/jquery-ui/jquery-ui.min.css" rel="Stylesheet" />
<link type="text/css" href="__ROOT_LOCATION__/skin/jquery-ui/jquery-ui.theme.min.css" rel="Stylesheet" />
<style>
body {
background:
radial-gradient(#EEEEEE 15%, transparent 16%) 0 0,
radial-gradient(#EEEEEE 15%, transparent 16%) 8px 8px,
radial-gradient(rgba(255,255,255,.1) 15%, transparent 20%) 0 1px,
radial-gradient(rgba(255,255,255,.1) 15%, transparent 20%) 8px 9px;
background-color:#E8E8E8;
background-size:16px 16px;
margin-left: auto;
margin-right: auto;
max-width: 65.8em;
}
.book__list { text-align: center; }
.book {
display: inline-block; vertical-align: bottom; margin: 8px; padding: 12px 15px; width: 300px;
border: 1px solid #ccc; border-radius: 8px;
text-align: left; color: #000; font-family: sans-serif; font-size: 13px;
background-color:#F1F1F1;
box-shadow: 2px 2px 5px 0px #ccc;
}
.book:hover { background-color: #F9F9F9; box-shadow: none;}
.book__background { background-repeat: no-repeat; background-size: auto; background-position: top right; }
.book__title {
padding: 0 55px 0 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
font-size: 18px; color: #0645ad;
}
.book__description {
padding: 5px 55px 5px 0px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
font-size: 15px;
}
.book__info { line-height: 18px; color: #777; font-weight: bold; font-size: 13px; }
</style>
<script>
$(function() {
$( "#accordion" ).accordion();
});
</script>
</head>
<body class="kiwix">
<div class="kiwix">
__BOOKS__
</div>
</body>
</html>

View File

@@ -1,27 +0,0 @@
<link type="text/css" href="__ROOT_LOCATION__/skin/jquery-ui/jquery-ui.min.css" rel="Stylesheet" />
<link type="text/css" href="__ROOT_LOCATION__/skin/jquery-ui/jquery-ui.theme.min.css" rel="Stylesheet" />
<link type="text/css" href="__ROOT_LOCATION__/skin/taskbar.css" rel="Stylesheet" />
<script type="text/javascript" src="__ROOT_LOCATION__/skin/jquery-ui/external/jquery/jquery.js"></script>
<script type="text/javascript" src="__ROOT_LOCATION__/skin/jquery-ui/jquery-ui.min.js"></script>
<script>
var jk = jQuery.noConflict();
jk(function() {
jk( "#kiwixsearchbox" ).autocomplete({
source: "__ROOT_LOCATION__/suggest?content=__CONTENT_ESCAPED__",
dataType: "json",
cache: false,
select: function(event, ui) {
jk( "#kiwixsearchbox" ).val(ui.item.value);
jk( "#kiwixsearchform" ).submit();
},
});
});
/* cybook hack */
if (navigator.userAgent.indexOf("bookeen/cybook") != -1) {
jk("html").addClass("cybook");
}
</script>

View File

File diff suppressed because it is too large Load Diff

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.7 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 212 B

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 208 B

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 335 B

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 207 B

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 262 B

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 262 B

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 332 B

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 280 B

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.8 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.4 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.8 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.8 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.4 KiB

View File

File diff suppressed because one or more lines are too long

View File

File diff suppressed because one or more lines are too long

View File

File diff suppressed because one or more lines are too long

View File

File diff suppressed because one or more lines are too long

View File

@@ -1,8 +0,0 @@
server_resources = custom_target('resources',
input: 'resources_list.txt',
output: ['server-resources.cpp', 'server-resources.h'],
command:[res_compiler, '--cxxfile', '@OUTPUT0@', '--hfile', '@OUTPUT1@', '@INPUT@'],
build_always: true
)

View File

@@ -1,9 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
<ShortName>Zim catalog search</ShortName>
<Description>Search zim files in the catalog.</Description>
<Url type="application/atom+xml;profile=opds-catalog"
xmlns:atom="http://www.w3.org/2005/Atom"
indexOffset="0"
template="/__ROOT_LOCATION__/catalog/search?q={searchTerms}&lang={language}&count={count}&start={startIndex}"/>
</OpenSearchDescription>

View File

@@ -1,25 +0,0 @@
jquery-ui/jquery-ui.structure.min.css
jquery-ui/jquery-ui.min.js
jquery-ui/external/jquery/jquery.js
jquery-ui/images/ui-bg_flat_0_aaaaaa_40x100.png
jquery-ui/images/ui-bg_flat_75_ffffff_40x100.png
jquery-ui/images/ui-icons_222222_256x240.png
jquery-ui/images/ui-bg_glass_55_fbf9ee_1x400.png
jquery-ui/images/ui-bg_highlight-soft_75_cccccc_1x100.png
jquery-ui/images/ui-bg_glass_65_ffffff_1x400.png
jquery-ui/images/ui-icons_2e83ff_256x240.png
jquery-ui/images/ui-icons_cd0a0a_256x240.png
jquery-ui/images/ui-icons_888888_256x240.png
jquery-ui/images/ui-bg_glass_75_e6e6e6_1x400.png
jquery-ui/images/animated-overlay.gif
jquery-ui/images/ui-bg_glass_75_dadada_1x400.png
jquery-ui/images/ui-icons_454545_256x240.png
jquery-ui/images/ui-bg_glass_95_fef1ec_1x400.png
jquery-ui/jquery-ui.theme.min.css
jquery-ui/jquery-ui.min.css
home.html.tmpl
include.html.part
taskbar.css
taskbar.html.part
global_taskbar.html.part
opensearchdescription.xml

View File

@@ -1,88 +0,0 @@
#kiwixtoolbar {
position: fixed;
padding: .5em;
left:0;
right:0;
top: 0;
z-index:100;
}
#kiwixtoolbar > a {
float: left;
}
.height_separator {
height: 3em;
}
#kiwixsearchbox {
width: 20em;
}
.kiwix .kiwix_centered {
max-width: 720px;
margin: 0 auto;
}
.kiwix .kiwix_button_wrapper {
float: left;
}
.kiwix .kiwix_searchform {
float: right;
}
.kiwix #kiwixtoolbar button,
.kiwix #kiwixtoolbar input[type="submit"] {
box-sizing: border-box !important;
height: 26px !important;
line-height: 20px !important;
margin-right: 5px !important;
padding: 2px 6px !important;
border: 1px solid #999 !important;
border-radius: 3px !important;
background-color: #ededed !important;
font-weight: normal !important;
cursor: pointer !important;
font-size: 16px !important;
}
.kiwix #kiwixtoolbar a {
margin-bottom: 5px;
}
.kiwix #kiwixtoolbar #kiwixsearchform input[type='text'] {
box-sizing: border-box !important;
height: 26px !important;
line-height: 20px !important;
border: 1px solid #999 !important;
border-radius: 3px !important;
background-color: #fff !important;
padding: 2px 2px 2px 3px !important;
font-size: 16px !important;
}
/* Try to fix buggy stuff in jquery-ui autocomplete */
#ui-id-1, .ui-autocomplete {
background: white !important;
border: solid 1px grey !important;
}
li.ui-state-focus {
font-weight: bold;
}
@media (max-width: 520px) {
.kiwixsearch {
margin-top: 5px;
}
}
@media (max-width: 645px) {
.kiwixsearch {
float: none;
}
#kiwixsearchbox {
width: 65%;
}
#kiwixsearchform input[type="submit"] {
width: 28%;
}
.height_separator {
height: 6em;
}
}

View File

@@ -1,19 +0,0 @@
<span class="kiwix">
<span id="kiwixtoolbar" class="ui-widget-header">
<div class="kiwix_centered">
<div class="kiwix_button_wrapper">
<a id="kiwix_serve_taskbar_library_button" href="__ROOT_LOCATION__/"><button>&#x1f3e0;</button></a>
<a id="kiwix_serve_taskbar_home_button" href="__ROOT_LOCATION__/__CONTENT__/"><button>__ZIM_TITLE__</button></a>
<a id="kiwix_serve_taskbar_random_button" href="__ROOT_LOCATION__/random?content=__CONTENT_ESCAPED__"><button>&#x1F3B2;</button></a>
</div>
<div class="kiwix_searchform">
<form class="kiwixsearch" method="GET" action="__ROOT_LOCATION__/search" id="kiwixsearchform">
<input type="hidden" name="content" value="__CONTENT__" />
<input autocomplete="off" class="ui-autocomplete-input" id="kiwixsearchbox" name="pattern" type="text">
<input type="submit" value="&#x1f50d;">
</form>
</div>
</div>
</span>
</span>
<div style="display: block; height: 5em;"></div>

View File

@@ -1,29 +0,0 @@
#!/usr/bin/env bash
set -e
BUILD_DIR=${HOME}/BUILD_${PLATFORM}
INSTALL_DIR=${BUILD_DIR}/INSTALL
case ${PLATFORM} in
"native_static")
MESON_OPTION="-Dstatic-linkage=true"
;;
"native_dyn")
MESON_OPTION=""
;;
"win32_static")
MESON_OPTION="-Dstatic-linkage=true --cross-file ${BUILD_DIR}/meson_cross_file.txt"
;;
"win32_dyn")
MESON_OPTION="--cross-file ${BUILD_DIR}/meson_cross_file.txt"
;;
esac
cd ${TRAVIS_BUILD_DIR}
export PKG_CONFIG_PATH=${INSTALL_DIR}/lib/x86_64-linux-gnu/pkgconfig
export PATH=${INSTALL_DIR}/bin:$PATH
meson . build ${MESON_OPTION}
cd build
ninja

View File

@@ -1,48 +0,0 @@
#!/usr/bin/env bash
set -e
REPO_NAME=${TRAVIS_REPO_SLUG#*/}
ARCHIVE_NAME=deps_${TRAVIS_OS_NAME}_${PLATFORM}_${REPO_NAME}.tar.xz
# Packages.
case ${PLATFORM} in
"native_static")
PACKAGES="gcc python3.5 cmake libbz2-dev ccache zlib1g-dev uuid-dev"
;;
"native_dyn")
PACKAGES="gcc python3.5 cmake libbz2-dev ccache zlib1g-dev uuid-dev libmicrohttpd-dev"
;;
"win32_static")
PACKAGES="g++-mingw-w64-i686 gcc-mingw-w64-i686 gcc-mingw-w64-base mingw-w64-tools ccache python3.5"
;;
"win32_dyn")
PACKAGES="g++-mingw-w64-i686 gcc-mingw-w64-i686 gcc-mingw-w64-base mingw-w64-tools ccache python3.5"
;;
"android_arm")
PACKAGES="gcc python3.5 cmake ccache"
;;
"android_arm64")
PACKAGES="gcc python3.5 cmake ccache"
;;
esac
sudo apt-get update -qq
sudo apt-get install -qq python3-pip ${PACKAGES}
wget https://bootstrap.pypa.io/get-pip.py
python3.5 get-pip.py --user
python3.5 -m pip install --user --upgrade pip
python3.5 -m pip install --user meson==0.49.2
# Ninja
cd $HOME
git clone git://github.com/ninja-build/ninja.git
cd ninja
git checkout release
./configure.py --bootstrap
sudo cp ninja /bin
# Dependencies comming from kiwix-build.
cd ${HOME}
wget http://tmp.kiwix.org/ci/${ARCHIVE_NAME}
tar xf ${HOME}/${ARCHIVE_NAME}