Compare commits

...

102 Commits

Author SHA1 Message Date
luddens
6b04eb3214 wip 2020-03-11 18:12:40 +01:00
Matthieu Gautier
3ab3ffe3ea Merge pull request #329 from kiwix/getBookByPath
Get book by path
2020-03-06 12:12:43 +01:00
Matthieu Gautier
064d5f3fa6 Make the search argument constant. 2020-03-06 12:08:05 +01:00
Matthieu Gautier
46626a3f98 Add the method get bookByPath in library. 2020-03-06 12:08:05 +01:00
Matthieu Gautier
90ba27fbab Merge pull request #328 from kiwix/java_article_size
[JAVA] Add a method to get the size of an article.
2020-03-04 17:10:18 +01:00
Matthieu Gautier
76c293e403 [JAVA] Add a method to get the size of an article.
Fix #327
2020-03-04 16:40:03 +01:00
Matthieu Gautier
78e57c1a51 New version 9.0.1 2020-02-21 14:58:04 +01:00
Matthieu Gautier
7c49dc6af9 Merge pull request #326 from kiwix/fix_pair_size
[JAVA] Use a long to store the offset of a article in the zim file.
2020-02-19 14:58:43 +01:00
Matthieu Gautier
2e60a088ab [JAVA] Use a long to store the offset of a article in the zim file.
Fixes kiwix/kiwix-android#1769
2020-02-19 14:27:51 +01:00
Matthieu Gautier
9ab0f825f4 Merge pull request #325 from kiwix/new_version
New version 9.0.0
2020-02-12 14:54:12 +01:00
Matthieu Gautier
411ca28598 New version 9.0.0 2020-02-12 14:48:03 +01:00
Matthieu Gautier
cc2b5e63ca Merge pull request #323 from kiwix/fix_wrapper
Fix wrapper
2020-02-12 10:47:29 +01:00
Matthieu Gautier
ea29557a33 [Java] Add a wrapper on method to update book from another book or reader. 2020-02-11 17:44:14 +01:00
Matthieu Gautier
b53f531f2b Fix typo getTagStr in the wrapper 2020-02-10 17:25:31 +01:00
Matthieu Gautier
bd74ebed1e Merge pull request #320 from kiwix/path_win
Fix launching command with path containing spaces.
2020-02-07 13:15:24 +01:00
Matthieu Gautier
1632e9c55b Fix launching command with path containing spaces.
We must correctly quote path with space on windows.
This is needed as we can't launch command using a array of string on
windows but by giving only one string using space as separator.

Fix kiwix/kiwix-desktop#268
2020-02-06 11:46:33 +01:00
Kelson
6dd6a5dd80 Github Kiwix Sponsoring page link 2020-02-01 17:44:42 +01:00
Kelson
b65b90d78c Merge pull request #319 from kiwix/fix-missing-stdexcept-include
Include stdexcept to fix GCC v10 compilation
2020-02-01 17:20:53 +01:00
Kelson
6a975994cc Include stdexcept to fix GCC v10 compilation 2020-02-01 13:52:27 +01:00
Matthieu Gautier
3ae596783d Name size filtering (#315)
Name size filtering
2020-01-30 19:13:55 +01:00
Matthieu Gautier
ce6e956434 [OPDS] Add the url argument to filter by size and name.
Fix kiwix/kiwix-tools#231
2020-01-30 19:02:33 +01:00
Matthieu Gautier
f560a1f815 Be able to filter the books by name. 2020-01-30 19:02:33 +01:00
Matthieu Gautier
d14ba0c2e8 Merge pull request #314 from kiwix/trust_library
Trust the library.xml information by default.
2020-01-30 19:02:13 +01:00
Matthieu Gautier
34257cfc1f Trust the library.xml information by default.
Do not try to read the zim file and update the book when parsing a
library.xml.
Needed by kiwix/kiwix-tools#319
2020-01-30 18:22:07 +01:00
Matthieu Gautier
8f990feabb Merge pull request #316 from kiwix/tagging_system
Tagging system
2020-01-30 18:21:28 +01:00
Matthieu Gautier
fff2524ee2 Fix opensearch description. 2020-01-30 17:48:56 +01:00
Matthieu Gautier
a756e7f8f3 Add flavour attribute to book.
Fix #259
Fix kiwix/kiwix-tools#316
2020-01-30 17:48:56 +01:00
Matthieu Gautier
bc257d2d6d Add method to get value of tag from a book.
Only the `getTagStr` method is available on android because we need a
proper exception handling on wrapping side.

Fix #298
2020-01-30 17:48:56 +01:00
Matthieu Gautier
7275f9b8e3 Move function to convert and use tags inside otherTools. 2020-01-30 17:48:56 +01:00
Matthieu Gautier
b63827196c Small fixes (#313)
Small fixes
2020-01-30 17:23:30 +01:00
Matthieu Gautier
2881face70 Add missing setting of attribute. 2020-01-30 15:42:54 +01:00
Matthieu Gautier
77ba09c310 Reorder setting and dumper of book attribute.
No real change. Reordering setting and dumping of attribute in the same
order (mostly) they are declared in book.h make it easier to detect missing
attribute.
2020-01-30 15:42:54 +01:00
Matthieu Gautier
49aa0fbb9f Use a macro to write the filters. 2020-01-30 15:42:54 +01:00
Matthieu Gautier
7846b45bef Fix opds filtering by tag. 2020-01-30 15:42:54 +01:00
Matthieu Gautier
7e26f3502d Fix opds dumper/parser.
Add missing attributes.
2020-01-30 15:42:54 +01:00
Matthieu Gautier
47a866400c Add parameter options to start a download with aria2 (#310)
Add parameter options to start a download with aria2
2020-01-30 15:42:40 +01:00
luddens
4b6c26bd0b add parameter option to start a download with aria2
Downloader::startDownload has a new parameter option which is a vector of
pair that represents the options that can be set for adding a uri with aria2
with the function Aria2::addUri.

Aria2::addUri uses this parameter to set the struct of parameters for the
aria2 command
2020-01-29 15:38:09 +01:00
Matthieu Gautier
94c47383a9 Remove useless Aria2c error handling functions (#306)
Remove useless Aria2c error handling functions
2020-01-28 17:36:54 +01:00
luddens
6bcecc2677 remove useless error handling functions
Some of the changes from https://github.com/kiwix/kiwix-lib/pull/292
that doesn't work properly are removed
2020-01-28 16:58:55 +01:00
Matthieu Gautier
e20c3520dc Android manager (#311)
Android manager
2020-01-28 15:48:38 +01:00
Matthieu Gautier
24fb1868d7 Fix typo in metadata. 2020-01-28 12:08:18 +01:00
Matthieu Gautier
82afb804e1 Add a small java test on the kiwix-lib.
Will the compilation should be made by meson.
It seems it is not possible to specify a existing jar
to link with. Use a custom script for now.
2020-01-28 12:08:18 +01:00
Matthieu Gautier
0951546356 Create the jar library when creating the java wrapper. 2020-01-28 12:08:18 +01:00
Matthieu Gautier
f09c739c1f Be able to create a wrapper for java.
Android is a specific wrapper.
Java is another one.
2020-01-28 12:08:18 +01:00
Matthieu Gautier
df9ddd5451 Use correct mutex on android and java. 2020-01-28 12:08:18 +01:00
Matthieu Gautier
fe513951d3 Use a macro to print error log.
This allow use to compile the JNI wrapper not for
android.
2020-01-28 12:08:18 +01:00
Matthieu Gautier
7f0d509a88 Add the filter functionality. 2020-01-28 12:08:18 +01:00
Matthieu Gautier
54f671b2f1 Add some methods to get information from the library.
Else, the library is useless.
2020-01-28 12:08:18 +01:00
Matthieu Gautier
c2c89c6c86 Rename the JNIKiwixLibrary class to Library.
This mainly use the "new memory system". No need to call dispose function.
Rename the class to Library to conform with the naming semantics
(JNIKiwix* use old memory system)
2020-01-28 12:08:18 +01:00
Matthieu Gautier
75652d0e9f WIP Add a wrapper around the kiwix::manager class.
The JNIKiwixManager is used to manage (insertion of book in) the library.

It is created, as needed, using an existing Library as input.
It is then used to add books, parse library.xml or opds content.
Then it can be destruct (and must be) with the `dispose` method.

```java
library = JNILibrary(...);
manager = JNIManager(library);
manager.parseOpds(opdscontent);
manager.dispose();
// library contains the books declared in the opds content.
// Use the library methods to get the books' info.
```
2020-01-28 12:08:18 +01:00
Matthieu Gautier
6535dc2e38 Add a wrapper for the Book class. 2020-01-28 12:08:18 +01:00
Matthieu Gautier
6b2f768c8f Use template function c2jni to convert c++ type to jni. 2020-01-28 12:08:18 +01:00
Matthieu Gautier
6099c3113f Readd code coverage (#312)
Readd code coverage
2020-01-28 10:53:36 +01:00
Matthieu Gautier
21d8a6952f Readd code coverage 2020-01-27 18:05:12 +01:00
Matthieu Gautier
b84f4e03ea update book even if the members aren't empty (#302)
update book even if the members aren't empty
2020-01-27 17:58:41 +01:00
luddens
ff21a095cb update book even if the members aren't empty
remove the conditions to always update the book
2020-01-27 17:50:44 +01:00
Matthieu Gautier
fa091a19c6 Merge pull request #309 from kiwix/github-action
Use github actions to run the CI.
2020-01-21 18:18:56 +01:00
Matthieu Gautier
e3ba9fa5cc Use github actions to run the CI. 2020-01-21 18:10:33 +01:00
Matthieu Gautier
560f67522f Do not build tests if we are cross-compiling (#308)
Do not build tests if we are cross-compiling
2020-01-21 11:11:22 +01:00
Matthieu Gautier
11118efd84 Do not build tests if we are cross-compiling 2020-01-21 10:58:43 +01:00
Matthieu Gautier
9c2bc6affc Fix tests. (#307)
Fix tests.
2020-01-20 17:12:58 +01:00
Matthieu Gautier
4fe31a20e3 Fix tests.
As we use the main library of gtest (if already installed)
we don't need to (and must not) define a main function for the tests.

Does the same (use main library) if we use the subproject.
2020-01-20 17:00:18 +01:00
Matthieu Gautier
0f99c1ad20 Win library path (#305)
Win library path
2020-01-13 17:12:16 +01:00
Matthieu Gautier
91db055d86 Remove function to read file using a native path.
All path must be utf8. This is already the case in all our project.
(If this not the case, this is a bug)

So we don't need to have a version with a native and utf8 path.
2020-01-13 16:59:58 +01:00
Matthieu Gautier
5540149e2b Correctly open the library path on windows.
We need to convert the path to wstring on Windows to handle directory
with accented characters.

Fix kiwix-desktop#269
2020-01-13 16:54:09 +01:00
Matthieu Gautier
7c7e351d34 Add missing function's declarations to convert path for windows. 2020-01-13 16:51:48 +01:00
Matthieu Gautier
c5051b343e Catalog empty search (#304)
Catalog empty search
2020-01-13 14:09:33 +01:00
Matthieu Gautier
02a0d592f9 Install python3 in Travis MacOS
For some reason, `homebrew install gcovr` now install python3 as
dependency.
This is a dependency since a long time, I don't know why it was not
installing it before.

But the installation fails because it cannot link correctly python3
because python2 is already installed.

Then meson, when it tries to run the conversion script, fails because
the script cannot find python3.

The solution I've found is to unlink python2 and relink python3 to have
a correct installation of python3.
2020-01-13 13:29:18 +01:00
Matthieu Gautier
071e9e3fec Correctly filter the catalog when we don't what to filter.
Set the different filter's fields only when we are requested to filter
them. Else, we ends to requests that some fields are empty.

If the request has no argument, we raise an exception (catched) and so
we don't set the corresponding field in the filter.

Fix #303
2020-01-07 17:29:43 +01:00
Matthieu Gautier
4a01303438 Correctly set the id of the opds stream.
Set the id of the stream *after* the uuid is generated.
2020-01-07 17:29:29 +01:00
Kelson
8095a87bf1 Bump-up version to 8.2.2 and update Changelog 2019-12-08 12:12:39 +01:00
Kelson
bb55527508 Merge pull request #301 from kiwix/remove-trailing-spaces
Remove trailing spaces
2019-12-08 12:07:21 +01:00
Kelson
b7c5e5f339 Remove trailing spaces 2019-12-08 11:52:16 +01:00
Kelson
316deeb485 Merge pull request #300 from kiwix/exec-perm
Add execution permission to a few scripts
2019-12-08 10:03:57 +01:00
Kelson
721d981825 Add execution permission to a few scripts 2019-12-07 14:00:22 +01:00
Kelson
2244074f3c travis-ci.org -> travis-ci.com 2019-12-05 08:10:56 +01:00
Kelson
0a1e01eb2b Update install_deps.sh 2019-11-29 17:28:25 +01:00
Kelson
abd2fa3bf3 Bump-up to 8.2.1 in build.gradle 2019-11-27 19:15:24 +01:00
Kelson
ea3349f37c Merge pull request #297 from kiwix/reintroduce-taskbar
Reintroduce taskbar
2019-11-26 08:55:19 -05:00
Kelson
e3c6ca0d1b Bump-up version to 8.2.1 2019-11-26 11:54:39 +01:00
Kelson
52e165cf78 Reintroduce kiwix-serve taskbar 2019-11-26 11:54:00 +01:00
Kelson
3b7c805183 Bump-up version to 8.2.0 2019-11-20 13:07:28 +01:00
Kelson
9c4867a95a Update Changelog 2019-11-20 13:06:24 +01:00
Kelson
223f7ee78a Add default Github configuration 2019-11-20 06:12:08 +01:00
Kelson
20690bd5f5 Merge pull request #294 from kiwix/remove-absolute-url-support
Remove absolute internal URL support
2019-11-08 15:29:40 +01:00
Emmanuel Engelhart
de7b7c34b5 Remove absolute internal URL support 2019-11-07 18:05:58 +01:00
Kelson
f0ac66aea1 Merge pull request #292 from kiwix/aria2-comment
More error handling for aria2 cmd
2019-11-01 15:40:51 +01:00
luddens
20a2c78733 add get aria2 launch cmd method 2019-11-01 15:27:21 +01:00
luddens
9850be7267 add Curl error message 2019-11-01 15:27:21 +01:00
luddens
0dd996c6a3 add try catch around aria2 first commands 2019-11-01 15:27:21 +01:00
Kelson
2500cc8e63 Merge pull request #288 from kiwix/get-bookmarks-api
Add a parameter to getBookmarks fct to get valid bookmarks only
2019-10-31 14:38:50 +01:00
luddens
bd6797143c add getBookmarks test 2019-10-31 15:01:52 +02:00
luddens
c9a15c9961 Add a parameter to getBookmarks fct to get valid bookmarks only
The default value of this parameter is false, in this case all the bookmarks
are returned, otherwise only those who are related to books of the library.
2019-10-31 14:05:21 +02:00
Kelson
f1d55f8e86 Merge pull request #290 from kiwix/set-port-kiwixserve
add setPort() method
2019-10-28 16:41:19 +01:00
luddens
a2c2955f41 add kiwixserve unit test 2019-10-28 16:00:26 +01:00
luddens
9975e0b369 add setPort() method 2019-10-28 15:56:49 +01:00
Kelson
efe1c2dea3 Merge pull request #289 from Aditya-Sood/kiwix-android/issue/765
Fixes kiwix-android issue#765
2019-10-03 11:25:18 +01:00
Aditya-Sood
2af9ba4eab Readd original getNextSuggestion() 2019-10-01 13:30:35 +05:30
Aditya-Sood
c007373b46 Re-add comment 2019-10-01 13:30:34 +05:30
Aditya-Sood
e1acf9acff Code & local repository cleanup 2019-10-01 13:30:34 +05:30
Aditya-Sood
daaadf3e1c Comment out previous definitions 2019-10-01 13:30:34 +05:30
Aditya-Sood
74bd482335 Preliminary work 2019-10-01 13:30:34 +05:30
87 changed files with 1663 additions and 616 deletions

12
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,12 @@
# These are supported funding model platforms
github: kiwix
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username
custom: # https://kiwix.org/support-us/

27
.github/move.yml vendored Normal file
View File

@@ -0,0 +1,27 @@
# Configuration for Move Issues - https://github.com/dessant/move-issues
# Delete the command comment when it contains no other content
deleteCommand: true
# Close the source issue after moving
closeSourceIssue: true
# Lock the source issue after moving
lockSourceIssue: false
# Mention issue and comment authors
mentionAuthors: true
# Preserve mentions in the issue content
keepContentMentions: true
# Move labels that also exist on the target repository
moveLabels: true
# Set custom aliases for targets
# aliases:
# r: repo
# or: owner/repo
# Repository to extend settings from
# _extends: repo

15
.github/stale.yml vendored Normal file
View File

@@ -0,0 +1,15 @@
daysUntilClose: false
staleLabel: stale
issues:
daysUntilStale: 60
markComment: >
This issue has been automatically marked as stale because it has not had
recent activity. It will be now be reviewed manually. Thank you
for your contributions.
pulls:
daysUntilStale: 7
markComment: >
This pull request has been automatically marked as stale because it has not had
recent activity. It will be now be reviewed manually. Thank you
for your contributions.

157
.github/workflows/pr.yml vendored Normal file
View File

@@ -0,0 +1,157 @@
name: CI
on: [pull_request]
jobs:
Macos:
runs-on: macos-latest
steps:
- name: Checkout code
uses: actions/checkout@v1
- name: Setup python 3.5
uses: actions/setup-python@v1
with:
python-version: '3.5'
- name: Install packages
uses: mstksg/get-package@v1
with:
brew: gcovr pkg-config ninja
- name: Install python modules
run: pip3 install meson==0.49.2 pytest
- name: Install deps
shell: bash
run: |
ARCHIVE_NAME=deps2_osx_native_dyn_kiwix-lib.tar.xz
wget -O- http://tmp.kiwix.org/ci/${ARCHIVE_NAME} | tar -xJ -C $HOME
- name: Compile
shell: bash
run: |
export PKG_CONFIG_PATH=$HOME/BUILD_native_dyn/INSTALL/lib/pkgconfig
export CPPFLAGS="-I$HOME/BUILD_native_dyn/INSTALL/include"
meson . build --default-library=shared -Db_coverage=true
cd build
ninja
- name: Test
shell: bash
run: |
export LD_LIBRARY_PATH=$HOME/BUILD_native_dyn/INSTALL/lib:$HOME/BUILD_native_dyn/INSTALL/lib64
cd build
meson test --verbose
ninja coverage
env:
SKIP_BIG_MEMORY_TEST: 1
- name: Publish coverage
shell: bash
run: |
curl https://codecov.io/bash -o codecov.sh
bash codecov.sh -n osx_native_dyn -Z
rm codecov.sh
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
Linux:
strategy:
fail-fast: false
matrix:
target:
- native_static
- native_dyn
- android_arm
- android_arm64
- win32_static
- win32_dyn
include:
- target: native_static
image_variant: xenial
lib_postfix: '/x86_64-linux-gnu'
- target: native_dyn
image_variant: xenial
lib_postfix: '/x86_64-linux-gnu'
- target: android_arm
image_variant: xenial
lib_postfix: '/x86_64-linux-gnu'
- target: android_arm64
image_variant: xenial
lib_postfix: '/x86_64-linux-gnu'
- target: win32_static
image_variant: f31
lib_postfix: '64'
- target: win32_dyn
image_variant: f31
lib_postfix: '64'
env:
HOME: /home/runner
runs-on: ubuntu-latest
container:
image: "kiwix/kiwix-build_ci:${{matrix.image_variant}}-26"
steps:
- name: Checkout code
shell: python
run: |
import json
from subprocess import check_call
from os import environ
with open(environ['GITHUB_EVENT_PATH'], 'r') as f:
content = f.read()
event_data = json.loads(content)
try:
branch_ref = event_data['ref'].split('/')[-1]
except KeyError:
branch_ref = event_data['pull_request']['head']['ref']
print("Cloning branch", branch_ref)
command = [
'git', 'clone',
'https://github.com/${{github.repository}}',
'--depth=1',
'--branch', branch_ref
]
check_call(command, cwd=environ['HOME'])
- name: Install deps
shell: bash
run: |
ARCHIVE_NAME=deps2_${OS_NAME}_${{matrix.target}}_kiwix-lib.tar.xz
wget -O- http://tmp.kiwix.org/ci/${ARCHIVE_NAME} | tar -xJ -C /home/runner
- name: Compile
shell: bash
run: |
meson --version
if [[ "${{matrix.target}}" =~ .*_dyn ]]; then
MESON_OPTION="--default-library=shared"
else
MESON_OPTION="--default-library=static"
fi
if [[ "${{matrix.target}}" =~ native_.* ]]; then
MESON_OPTION="$MESON_OPTION -Db_coverage=true"
else
MESON_OPTION="$MESON_OPTION --cross-file $HOME/BUILD_${{matrix.target}}/meson_cross_file.txt"
fi
if [[ "${{matrix.target}}" =~ android_.* ]]; then
MESON_OPTION="$MESON_OPTION -Dandroid=true"
fi
cd $HOME/kiwix-lib
meson . build ${MESON_OPTION}
cd build
ninja
env:
PKG_CONFIG_PATH: "/home/runner/BUILD_${{matrix.target}}/INSTALL/lib/pkgconfig:/home/runner/BUILD_${{matrix.target}}/INSTALL/lib${{matrix.lib_postfix}}/pkgconfig"
CPPFLAGS: "-I/home/runner/BUILD_${{matrix.target}}/INSTALL/include"
- name: Test
if: startsWith(matrix.target, 'native_')
shell: bash
run: |
cd $HOME/kiwix-lib/build
meson test --verbose
ninja coverage
env:
LD_LIBRARY_PATH: "/home/runner/BUILD_${{matrix.target}}/INSTALL/lib:/home/runner/BUILD_${{matrix.target}}/INSTALL/lib${{matrix.lib_postfix}}"
SKIP_BIG_MEMORY_TEST: 1
- name: Publish coverage
shell: bash
run: |
cd $HOME/kiwix-lib
curl https://codecov.io/bash -o codecov.sh
bash codecov.sh -n "${OS_NAME}_${{matrix.target}}" -Z
rm codecov.sh
if: startsWith(matrix.target, 'native_')
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

1
.gitignore vendored
View File

@@ -1,3 +1,4 @@
.idea/
*.swp
subprojects/googletest-release*
*.class

View File

@@ -1,42 +0,0 @@
language: cpp
dist: xenial
sudo: true
cache: ccache
before_install:
- PATH=$PATH:$HOME/bin
install: travis/install_deps.sh
script: travis/compile.sh
env:
matrix:
- PLATFORM="native_static"
- PLATFORM="native_dyn"
- PLATFORM="win32_static"
- PLATFORM="win32_dyn"
- PLATFORM="android_arm"
- PLATFORM="android_arm64"
addons:
apt:
packages:
- cmake
- python3.5
- python3-pip
- libbz2-dev
- ccache
- zlib1g-dev
- uuid-dev
- libctpp2-dev
- ctpp2-utils
- libmicrohttpd-dev
- g++-mingw-w64-i686
- gcc-mingw-w64-i686
- gcc-mingw-w64-base
- mingw-w64-tools
- gcovr
homebrew:
update: true
packages:
- gcovr
matrix:
include:
- env: PLATFORM="native_dyn"
os: osx

View File

@@ -77,7 +77,7 @@ modification follow.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
@@ -510,7 +510,7 @@ actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties

View File

@@ -1,3 +1,51 @@
kiwix-lib 9.0.1
===============
* [JAVA] Use a long to store the offset of an article in the zim file instead
of an int.
kiwix-lib 9.0.0
===============
* [OPDS] Correctly set the id of the OPDS stream.
* [OPDS] Do not try to filter the catalog if no filter field is given in the
request.
* [WINDOWS] Correctly convert path to wide chars when opening the library.xml
* [LIBRARY] Remove the function the read file using a native path.
All path must be utf8, no need to pass a native path along the utf8 path.
* [TEST] Fix tests using the main function of gtest instead of custom one.
* [CI] Move to github CI instead of Travis.
* The `Book::update` method always update the book's fields. Even if they are
not empty.
* [JAVA] Add wrapping around the library manager (opds parsing)
* [ARIA2] Add api option to start download with option (destination folder)
* [OPDS] Fixes about opds parsing, generation (missing attributes)
and requesting (server)
* Add methods on `Book` to get specific tag values (was on `Reader` only)
* Add flavour attribute to `Book`
* Fix opensearch description.
* Trust the given library.xml (by default) instead of reading the value from
the zim files.
* [OPDS] Be able to filter the content by name or size.
* [WINDOWS] Fix launching subcommand when there is spaces in the path.
kiwix-lib 8.2.2
===============
* Improve a few compilation scripts
kiwix-lib 8.2.1
===============
* Reintroduce kiwix-serve taskbar
kiwix-lib 8.2.0
===============
* More debug information if aria2c command fails
* Allow to set kiwix-serve port
* Better (dead) bookmarks mgmt
kiwix-lib 8.1.0
===============

View File

@@ -7,7 +7,7 @@ GNU/Linux, macOS, Android, iOS, ...).
[![Download](https://api.bintray.com/packages/kiwix/kiwix/kiwixlib/images/download.svg)](https://bintray.com/kiwix/kiwix/kiwixlib/_latestVersion)
[![AUR version](https://img.shields.io/aur/version/kiwix-lib)](https://aur.archlinux.org/packages/kiwix-lib/)
[![Build Status](https://travis-ci.org/kiwix/kiwix-lib.svg?branch=master)](https://travis-ci.org/kiwix/kiwix-lib)
[![Build Status](https://travis-ci.com/kiwix/kiwix-lib.svg?branch=master)](https://travis-ci.com/kiwix/kiwix-lib)
[![CodeFactor](https://www.codefactor.io/repository/github/kiwix/kiwix-lib/badge)](https://www.codefactor.io/repository/github/kiwix/kiwix-lib)
[![Codecov](https://codecov.io/gh/kiwix/kiwix-lib/branch/master/graph/badge.svg)](https://codecov.io/gh/kiwix/kiwix-lib)
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)

0
android-kiwix-lib-publisher/gradlew vendored Normal file → Executable file
View File

0
android-kiwix-lib-publisher/gradlew.bat vendored Normal file → Executable file
View File

View File

@@ -26,7 +26,7 @@ task writePom {
project {
groupId 'org.kiwix.kiwixlib'
artifactId 'kiwixlib'
version '8.1.0' + (System.env.KIWIXLIB_BUILDVERSION == null ? '' : '-'+System.env.KIWIXLIB_BUILDVERSION)
version '9.0.1' + (System.env.KIWIXLIB_BUILDVERSION == null ? '' : '-'+System.env.KIWIXLIB_BUILDVERSION)
packaging 'aar'
name 'kiwixlib'
url 'https://github.com/kiwix/kiwix-lib'

View File

@@ -1,5 +1,5 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="kiwix.org.kiwixlib">
package="org.kiwix.kiwixlib">
<application
android:allowBackup="true"

View File

@@ -60,6 +60,9 @@ class Book
const std::string& getUrl() const { return m_url; }
const std::string& getName() const { return m_name; }
const std::string& getTags() const { return m_tags; }
std::string getTagStr(const std::string& tagName) const;
bool getTagBool(const std::string& tagName) const;
const std::string& getFlavour() const { return m_flavour; }
const std::string& getOrigId() const { return m_origId; }
const uint64_t& getArticleCount() const { return m_articleCount; }
const uint64_t& getMediaCount() const { return m_mediaCount; }
@@ -81,6 +84,7 @@ class Book
void setDate(const std::string& date) { m_date = date; }
void setUrl(const std::string& url) { m_url = url; }
void setName(const std::string& name) { m_name = name; }
void setFlavour(const std::string& flavour) { m_flavour = flavour; }
void setTags(const std::string& tags) { m_tags = tags; }
void setOrigId(const std::string& origId) { m_origId = origId; }
void setArticleCount(uint64_t articleCount) { m_articleCount = articleCount; }
@@ -103,6 +107,7 @@ class Book
std::string m_date;
std::string m_url;
std::string m_name;
std::string m_flavour;
std::string m_tags;
std::string m_origId;
uint64_t m_articleCount;

View File

@@ -25,6 +25,7 @@
#include <map>
#include <pthread.h>
#include <memory>
#include <stdexcept>
namespace kiwix
{
@@ -87,12 +88,12 @@ class Download {
class Downloader
{
public:
Downloader();
Downloader(std::string sessionFileDir = "");
virtual ~Downloader();
void close();
Download* startDownload(const std::string& uri);
Download* startDownload(const std::string& uri, const std::vector<std::pair<std::string, std::string>>& options = {});
Download* getDownload(const std::string& did);
size_t getNbDownload() { return m_knownDownloads.size(); }

View File

@@ -64,14 +64,14 @@ class Entry
* @return the path of the entry.
*/
std::string getPath() const;
/**
* Get the title of the entry.
*
* @return the title of the entry.
*/
std::string getTitle() const;
/**
* Get the content of the entry.
*
@@ -81,7 +81,7 @@ class Entry
* @return the content of the entry.
*/
std::string getContent() const;
/**
* Get the blob of the entry.
*
@@ -91,7 +91,7 @@ class Entry
* @return the blob of the entry.
*/
zim::Blob getBlob(offset_type offset = 0) const;
/**
* Get the blob of the entry.
*
@@ -102,7 +102,7 @@ class Entry
* @return the blob of the entry.
*/
zim::Blob getBlob(offset_type offset, size_type size) const;
/**
* Get the info for direct access to the content of the entry.
*
@@ -118,7 +118,7 @@ class Entry
* Return <"",0> if is not possible to read directly.
*/
std::pair<std::string, offset_type> getDirectAccessInfo() const;
/**
* Get the size of the entry.
*
@@ -132,8 +132,8 @@ class Entry
* @return the mime_type of the entry.
*/
std::string getMimetype() const;
/**
* Get if the entry is a redirect entry.
*

View File

@@ -17,6 +17,8 @@ class KiwixServe
void shutDown();
bool isRunning();
int getPort() { return m_port; }
int setPort(int port);
void setLibraryPath(std::string path) { m_libraryPath = path; }
private:
std::unique_ptr<Subprocess> mp_kiwixServe;

View File

@@ -57,6 +57,7 @@ class Filter {
std::string _creator;
size_t _maxSize;
std::string _query;
std::string _name;
public:
Filter();
@@ -100,6 +101,7 @@ class Filter {
Filter& creator(std::string creator);
Filter& maxSize(size_t size);
Filter& query(std::string query);
Filter& name(std::string name);
bool accept(const Book& book) const;
};
@@ -147,6 +149,7 @@ class Library
bool removeBookmark(const std::string& zimId, const std::string& url);
Book& getBookById(const std::string& id);
Book& getBookByPath(const std::string& path);
std::shared_ptr<Reader> getReaderById(const std::string& id);
/**
@@ -208,7 +211,7 @@ class Library
*
* @return A list of bookmarks
*/
const std::vector<kiwix::Bookmark>& getBookmarks() { return m_bookmarks; }
const std::vector<kiwix::Bookmark> getBookmarks(bool onlyValidBookmarks = true);
/**
* Get all book ids of the books in the library.

View File

@@ -69,26 +69,12 @@ class Manager
/**
* Read a `library.xml` and add book in the file to the library.
*
* @param path The path to the `library.xml`.
* @param path The (utf8) path to the `library.xml`.
* @param readOnly Set if the libray path could be overwritten latter with
* updated content.
* @return True if file has been properly parsed.
*/
bool readFile(const std::string& path, const bool readOnly = true);
/**
* Read a `library.xml` and add book in the file to the library.
*
* @param nativePath The path of the `library.xml`
* @param UTF8Path The utf8 version (?) of the path. Also the path where the
* library will be writen i readOnly is False.
* @param readOnly Set if the libray path could be overwritten latter with
* updated content.
* @return True if file has been properly parsed.
*/
bool readFile(const std::string& nativePath,
const std::string& UTF8Path,
const bool readOnly = true);
bool readFile(const std::string& path, bool readOnly = true, bool trustLibrary = true);
/**
* Load a library content store in the string.
@@ -101,7 +87,8 @@ class Manager
*/
bool readXml(const std::string& xml,
const bool readOnly = true,
const std::string& libraryPath = "");
const std::string& libraryPath = "",
bool trustLibrary = true);
/**
* Load a library content stored in a OPDS stream.
@@ -168,8 +155,9 @@ class Manager
bool readBookFromPath(const std::string& path, Book* book);
bool parseXmlDom(const pugi::xml_document& doc,
const bool readOnly,
const std::string& libraryPath);
bool readOnly,
const std::string& libraryPath,
bool trustLibrary);
bool parseOpdsDom(const pugi::xml_document& doc,
const std::string& urlHost);

View File

@@ -89,7 +89,7 @@ class Searcher
* @param resultEnd the end offset of the search results (used for pagination).
* @param verbose print some info on stdout if true.
*/
void search(std::string& search,
void search(const std::string& search,
unsigned int resultStart,
unsigned int resultEnd,
const bool verbose = false);

View File

@@ -21,6 +21,7 @@
#define KIWIX_OTHERTOOLS_H
#include <string>
#include <vector>
namespace pugi {
class xml_node;
@@ -28,9 +29,18 @@ namespace pugi {
namespace kiwix
{
void sleep(unsigned int milliseconds);
std::string nodeToString(const pugi::xml_node& node);
std::string converta2toa3(const std::string& a2code);
void sleep(unsigned int milliseconds);
std::string nodeToString(const pugi::xml_node& node);
std::string converta2toa3(const std::string& a2code);
/*
* Convert all format tag string to new format
*/
std::vector<std::string> convertTags(const std::string& tags_str);
std::string getTagValueFromTagList(const std::vector<std::string>& tagList,
const std::string& tagName);
bool convertStrToBool(const std::string& value);
}
#endif

View File

@@ -22,6 +22,10 @@
#include <string>
#ifdef _WIN32
std::string WideToUtf8(const std::wstring& wstr);
std::wstring Utf8ToWide(const std::string& str);
#endif
bool isRelativePath(const std::string& path);
std::string computeAbsolutePath(const std::string& path, const std::string& relativePath);
std::string computeRelativePath(const std::string& path, const std::string& absolutePath);

View File

@@ -1,17 +1,23 @@
project('kiwix-lib', 'cpp',
version : '8.1.0', # Also change this in android-kiwix-lib-publisher/kiwixLibAndroid/build.gradle
version : '9.0.1', # Also change this in android-kiwix-lib-publisher/kiwixLibAndroid/build.gradle
license : 'GPL',
default_options : ['c_std=c11', 'cpp_std=c++11', 'werror=true'])
compiler = meson.get_compiler('cpp')
static_deps = get_option('android') or get_option('default_library') == 'static'
if get_option('android')
wrapper = get_option('wrapper')
static_deps = 'android' in wrapper or 'java' in wrapper or get_option('default_library') == 'static'
if 'android' in wrapper
extra_libs = ['-llog']
else
extra_libs = []
endif
if 'java' in wrapper
add_languages('java')
endif
thread_dep = dependency('threads')
libicu_dep = dependency('icu-i18n', static:static_deps)
libzim_dep = dependency('libzim', version : '>=5.0.0', static:static_deps)

View File

@@ -1,2 +1,2 @@
option('android', type : 'boolean', value : false,
description : 'Do we make a kiwix-lib for android')
option('wrapper', type:'array', choices:['java', 'android'], value:[],
description: 'The wrapper to generate.')

View File

@@ -81,12 +81,12 @@ class Resource:
data_identifier="_".join([""]+self.identifier),
resource_content=",\n ".join(", ".join("{:#04x}".format(i) for i in r) for r in sliced),
resource_len=len(self.data),
namespaces_open=" ".join("namespace {} {{".format(id) for id in self.identifier[:-1]),
namespaces_open=" ".join("namespace {} {{".format(id) for id in self.identifier[:-1]),
namespaces_close=" ".join(["}"]*(len(self.identifier)-1)),
identifier=self.identifier[-1],
env_identifier="RES_"+"_".join(self.identifier)+"_PATH"
)
def dump_getter(self):
return resource_getter_template.format(
common_name=self.filename,
@@ -95,11 +95,11 @@ class Resource:
def dump_decl(self):
return resource_decl_template.format(
namespaces_open=" ".join("namespace {} {{".format(id) for id in self.identifier[:-1]),
namespaces_open=" ".join("namespace {} {{".format(id) for id in self.identifier[:-1]),
namespaces_close=" ".join(["}"]*(len(self.identifier)-1)),
identifier=self.identifier[-1]
)
master_c_template = """//This file is automaically generated. Do not modify it.
@@ -113,7 +113,7 @@ static std::string init_resource(const char* name, const unsigned char* content,
char * resPath = getenv(name);
if (NULL == resPath)
return std::string(reinterpret_cast<const char*>(content), len);
std::ifstream ifs(resPath);
if (!ifs.good())
return std::string(reinterpret_cast<const char*>(content), len);
@@ -137,7 +137,7 @@ def gen_c_file(resources, basename):
include_file=basename,
basename=to_identifier(basename)
)
master_h_template = """//This file is automaically generated. Do not modify it.

View File

@@ -1,72 +0,0 @@
/*
* Copyright (C) 2013 Emmanuel Engelhart <kelson@kiwix.org>
* Copyright (C) 2017 Matthieu Gautier <mgautier@kymeria.fr>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
* MA 02110-1301, USA.
*/
#include <jni.h>
#include <android/log.h>
#include "org_kiwix_kiwixlib_JNIKiwixLibrary.h"
#include "library.h"
#include "reader.h"
#include "utils.h"
/* Kiwix Reader JNI functions */
JNIEXPORT jlong JNICALL Java_org_kiwix_kiwixlib_JNIKiwixLibrary_getNativeLibrary(
JNIEnv* env, jobject obj)
{
__android_log_print(ANDROID_LOG_INFO, "kiwix", "Attempting to create library");
Lock l;
try {
kiwix::Library* library = new kiwix::Library();
return reinterpret_cast<jlong>(new Handle<kiwix::Library>(library));
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_WARN, "kiwix", "Error creating ZIM library");
__android_log_print(ANDROID_LOG_WARN, "kiwix", e.what());
return 0;
}
}
JNIEXPORT void JNICALL
Java_org_kiwix_kiwixlib_JNIKiwixLibrary_dispose(JNIEnv* env, jobject obj)
{
Handle<kiwix::Library>::dispose(env, obj);
}
#define LIBRARY (Handle<kiwix::Library>::getHandle(env, obj))
/* Kiwix library functions */
JNIEXPORT jboolean JNICALL
Java_org_kiwix_kiwixlib_JNIKiwixLibrary_addBook(JNIEnv* env, jobject obj, jstring path)
{
std::string cPath = jni2c(path, env);
bool ret;
try {
kiwix::Reader reader(cPath);
kiwix::Book book;
book.update(reader);
ret = LIBRARY->addBook(book);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to add the book");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
ret = false;
}
return ret;
}

View File

@@ -1,32 +0,0 @@
kiwix_jni = custom_target('jni',
input: ['org/kiwix/kiwixlib/JNIICU.java',
'org/kiwix/kiwixlib/JNIKiwixReader.java',
'org/kiwix/kiwixlib/JNIKiwixLibrary.java',
'org/kiwix/kiwixlib/JNIKiwixSearcher.java',
'org/kiwix/kiwixlib/JNIKiwixServer.java',
'org/kiwix/kiwixlib/JNIKiwixInt.java',
'org/kiwix/kiwixlib/JNIKiwixString.java',
'org/kiwix/kiwixlib/JNIKiwixBool.java',
'org/kiwix/kiwixlib/JNIKiwixException.java',
'org/kiwix/kiwixlib/Pair.java'],
output: ['org_kiwix_kiwixlib_JNIKiwix.h',
'org_kiwix_kiwixlib_JNIKiwixReader.h',
'org_kiwix_kiwixlib_JNIKiwixLibrary.h',
'org_kiwix_kiwixlib_JNIKiwixServer.h',
'org_kiwix_kiwixlib_JNIKiwixSearcher.h',
'org_kiwix_kiwixlib_JNIKiwixSearcher_Result.h'],
command:['javac', '-d', '@OUTDIR@', '-h', '@OUTDIR@', '@INPUT@']
)
kiwix_sources += [
'android/kiwixicu.cpp',
'android/kiwixreader.cpp',
'android/kiwixlibrary.cpp',
'android/kiwixsearcher.cpp',
'android/kiwixserver.cpp',
kiwix_jni]
install_subdir('org', install_dir: 'kiwix-lib/java')
install_subdir('res', install_dir: 'kiwix-lib')
install_data('AndroidManifest.xml', install_dir: 'kiwix-lib')

View File

@@ -22,7 +22,7 @@
namespace kiwix {
Aria2::Aria2():
Aria2::Aria2(std::string sessionFileDir):
mp_aria(nullptr),
m_port(42042),
m_secret("kiwixariarpc"),
@@ -35,7 +35,12 @@ Aria2::Aria2():
std::string rpc_port = "--rpc-listen-port=" + to_string(m_port);
std::string download_dir = "--dir=" + getDataDirectory();
std::string session_file = appendToDirectory(getDataDirectory(), "kiwix.session");
std::string session_file;
if (sessionFileDir.empty()) {
session_file = appendToDirectory(getDataDirectory(), "kiwix.session");
} else {
session_file = appendToDirectory(sessionFileDir, "kiwix.session");
}
std::string session = "--save-session=" + session_file;
std::string inputFile = "--input-file=" + session_file;
// std::string log_dir = "--log=\"" + logDir + "\"";
@@ -78,23 +83,38 @@ Aria2::Aria2():
callCmd.push_back("--max-concurrent-downloads=42");
callCmd.push_back("--rpc-max-request-size=6M");
callCmd.push_back("--file-allocation=none");
std::string launchCmd;
for (auto &cmd : callCmd) {
launchCmd.append(cmd).append(" ");
}
mp_aria = Subprocess::run(callCmd);
mp_curl = curl_easy_init();
char errbuf[CURL_ERROR_SIZE];
curl_easy_setopt(mp_curl, CURLOPT_URL, "http://localhost/rpc");
curl_easy_setopt(mp_curl, CURLOPT_PORT, m_port);
curl_easy_setopt(mp_curl, CURLOPT_POST, 1L);
curl_easy_setopt(mp_curl, CURLOPT_ERRORBUFFER, errbuf);
int watchdog = 50;
while(--watchdog) {
sleep(10);
errbuf[0] = 0;
auto res = curl_easy_perform(mp_curl);
if (res == CURLE_OK) {
break;
} else if (watchdog == 1) {
std::cerr <<" curl_easy_perform() failed." << std::endl;
fprintf(stderr, "\nlibcurl: (%d) ", res);
if (errbuf[0] != 0) {
std::cerr << errbuf << std::endl;
} else {
std::cerr << curl_easy_strerror(res) << std::endl;
}
}
}
if (!watchdog) {
curl_easy_cleanup(mp_curl);
throw std::runtime_error("Cannot connect to aria2c rpc");
throw std::runtime_error("Cannot connect to aria2c rpc. Aria2c launch cmd : " + launchCmd);
}
}
@@ -145,13 +165,16 @@ std::string Aria2::doRequest(const MethodCall& methodCall)
throw std::runtime_error("Cannot perform request");
}
std::string Aria2::addUri(const std::vector<std::string>& uris)
std::string Aria2::addUri(const std::vector<std::string>& uris, const std::vector<std::pair<std::string, std::string>>& options)
{
MethodCall methodCall("aria2.addUri", m_secret);
auto uriParams = methodCall.newParamValue().getArray();
for (auto& uri : uris) {
uriParams.addValue().set(uri);
}
for (auto& option : options) {
methodCall.newParamValue().getStruct().addMember(option.first).getValue().set(option.second);
}
auto ret = doRequest(methodCall);
MethodResponse response(ret);
return response.getParamValue(0).getAsS();

View File

@@ -30,11 +30,11 @@ class Aria2
std::string doRequest(const MethodCall& methodCall);
public:
Aria2();
Aria2(std::string sessionFileDir = "");
virtual ~Aria2();
void close();
std::string addUri(const std::vector<std::string>& uri);
std::string addUri(const std::vector<std::string>& uri, const std::vector<std::pair<std::string, std::string>>& options = {});
std::string tellStatus(const std::string& gid, const std::vector<std::string>& statusKey);
std::vector<std::string> tellActive();
std::vector<std::string> tellWaiting();

View File

@@ -23,6 +23,7 @@
#include "tools/base64.h"
#include "tools/regexTools.h"
#include "tools/networkTools.h"
#include "tools/otherTools.h"
#include <pugixml.hpp>
@@ -44,43 +45,48 @@ bool Book::update(const kiwix::Book& other)
if (m_readOnly)
return false;
if (m_id != other.m_id)
return false;
m_readOnly = other.m_readOnly;
m_path = other.m_path;
m_pathValid = other.m_pathValid;
m_title = other.m_title;
m_description = other.m_description;
m_language = other.m_language;
m_creator = other.m_creator;
m_publisher = other.m_publisher;
m_date = other.m_date;
m_url = other.m_url;
m_name = other.m_name;
m_flavour = other.m_flavour;
m_tags = other.m_tags;
m_origId = other.m_origId;
m_articleCount = other.m_articleCount;
m_mediaCount = other.m_mediaCount;
m_size = other.m_size;
m_favicon = other.m_favicon;
m_faviconMimeType = other.m_faviconMimeType;
m_faviconUrl = other.m_faviconUrl;
if (m_path.empty()) {
m_path = other.m_path;
m_pathValid = other.m_pathValid;
}
m_downloadId = other.m_downloadId;
if (m_url.empty()) {
m_url = other.m_url;
}
if (m_tags.empty()) {
m_tags = other.m_tags;
}
if (m_name.empty()) {
m_name = other.m_name;
}
if (m_faviconMimeType.empty()) {
m_favicon = other.m_favicon;
m_faviconMimeType = other.m_faviconMimeType;
}
return true;
}
void Book::update(const kiwix::Reader& reader)
{
m_path = reader.getZimFilePath();
m_pathValid = true;
m_id = reader.getId();
m_title = reader.getTitle();
m_description = reader.getDescription();
m_language = reader.getLanguage();
m_date = reader.getDate();
m_creator = reader.getCreator();
m_publisher = reader.getPublisher();
m_title = reader.getTitle();
m_date = reader.getDate();
m_name = reader.getName();
m_flavour = reader.getFlavour();
m_tags = reader.getTags();
m_origId = reader.getOrigId();
m_articleCount = reader.getArticleCount();
@@ -100,21 +106,24 @@ void Book::updateFromXml(const pugi::xml_node& node, const std::string& baseDir)
path = computeAbsolutePath(baseDir, path);
}
m_path = path;
m_pathValid = fileExists(path);
m_title = ATTR("title");
m_name = ATTR("name");
m_tags = ATTR("tags");
m_description = ATTR("description");
m_language = ATTR("language");
m_date = ATTR("date");
m_creator = ATTR("creator");
m_publisher = ATTR("publisher");
m_date = ATTR("date");
m_url = ATTR("url");
m_name = ATTR("name");
m_flavour = ATTR("flavour");
m_tags = ATTR("tags");
m_origId = ATTR("origId");
m_articleCount = strtoull(ATTR("articleCount"), 0, 0);
m_mediaCount = strtoull(ATTR("mediaCount"), 0, 0);
m_size = strtoull(ATTR("size"), 0, 0) << 10;
m_favicon = base64_decode(ATTR("favicon"));
m_faviconMimeType = ATTR("faviconMimeType");
m_faviconUrl = ATTR("faviconUrl");
try {
m_downloadId = ATTR("downloadId");
} catch(...) {}
@@ -137,11 +146,15 @@ void Book::updateFromOpds(const pugi::xml_node& node, const std::string& urlHost
if (!m_id.compare(0, 9, "urn:uuid:")) {
m_id.erase(0, 9);
}
// No path on opds.
m_title = VALUE("title");
m_description = VALUE("description");
m_description = VALUE("summary");
m_language = VALUE("language");
m_date = fromOpdsDate(VALUE("updated"));
m_creator = node.child("author").child("name").child_value();
m_publisher = node.child("publisher").child("name").child_value();
m_date = fromOpdsDate(VALUE("updated"));
m_name = VALUE("name");
m_flavour = VALUE("flavour");
m_tags = VALUE("tags");
for(auto linkNode = node.child("link"); linkNode;
linkNode = linkNode.next_sibling("link")) {
@@ -197,4 +210,12 @@ const std::string& Book::getFavicon() const {
return m_favicon;
}
std::string Book::getTagStr(const std::string& tagName) const {
return getTagValueFromTagList(convertTags(m_tags), tagName);
}
bool Book::getTagBool(const std::string& tagName) const {
return convertStrToBool(getTagStr(tagName));
}
}

View File

@@ -124,20 +124,27 @@ void Download::cancelDownload()
}
/* Constructor */
Downloader::Downloader() :
mp_aria(new Aria2())
Downloader::Downloader(std::string sessionFileDir) :
mp_aria(new Aria2(sessionFileDir))
{
for (auto gid : mp_aria->tellActive()) {
m_knownDownloads[gid] = std::unique_ptr<Download>(new Download(mp_aria, gid));
m_knownDownloads[gid]->updateStatus();
try {
for (auto gid : mp_aria->tellActive()) {
m_knownDownloads[gid] = std::unique_ptr<Download>(new Download(mp_aria, gid));
m_knownDownloads[gid]->updateStatus();
}
} catch (std::exception& e) {
std::cerr << "aria2 tellActive failed : " << e.what();
}
for (auto gid : mp_aria->tellWaiting()) {
m_knownDownloads[gid] = std::unique_ptr<Download>(new Download(mp_aria, gid));
m_knownDownloads[gid]->updateStatus();
try {
for (auto gid : mp_aria->tellWaiting()) {
m_knownDownloads[gid] = std::unique_ptr<Download>(new Download(mp_aria, gid));
m_knownDownloads[gid]->updateStatus();
}
} catch (std::exception& e) {
std::cerr << "aria2 tellWaiting failed : " << e.what();
}
}
/* Destructor */
Downloader::~Downloader()
{
@@ -156,7 +163,7 @@ std::vector<std::string> Downloader::getDownloadIds() {
return ret;
}
Download* Downloader::startDownload(const std::string& uri)
Download* Downloader::startDownload(const std::string& uri, const std::vector<std::pair<std::string, std::string>>& options)
{
for (auto& p: m_knownDownloads) {
auto& d = p.second;
@@ -165,7 +172,7 @@ Download* Downloader::startDownload(const std::string& uri)
return d.get();
}
std::vector<std::string> uris = {uri};
auto gid = mp_aria->addUri(uris);
auto gid = mp_aria->addUri(uris, options);
m_knownDownloads[gid] = std::unique_ptr<Download>(new Download(mp_aria, gid));
return m_knownDownloads[gid].get();
}

View File

@@ -68,4 +68,14 @@ bool KiwixServe::isRunning()
return false;
}
int KiwixServe::setPort(int port)
{
if (port >= 1 && port <= 65535) {
m_port = port;
} else {
return -1;
}
return m_port;
}
}

View File

@@ -85,6 +85,18 @@ Book& Library::getBookById(const std::string& id)
return m_books.at(id);
}
Book& Library::getBookByPath(const std::string& path)
{
for(auto& it: m_books) {
auto& book = it.second;
if (book.getPath() == path)
return book;
}
std::ostringstream ss;
ss << "No book with path " << path << " in the library." << std::endl;
throw std::out_of_range(ss.str());
}
std::shared_ptr<Reader> Library::getReaderById(const std::string& id)
{
try {
@@ -184,6 +196,21 @@ std::vector<std::string> Library::getBooksPublishers()
return booksPublishers;
}
const std::vector<kiwix::Bookmark> Library::getBookmarks(bool onlyValidBookmarks)
{
if (!onlyValidBookmarks) {
return m_bookmarks;
}
std::vector<kiwix::Bookmark> validBookmarks;
auto booksId = getBooksIds();
for(auto& bookmark:m_bookmarks) {
if (std::find(booksId.begin(), booksId.end(), bookmark.getBookId()) != booksId.end()) {
validBookmarks.push_back(bookmark);
}
}
return validBookmarks;
}
std::vector<std::string> Library::getBooksIds()
{
std::vector<std::string> bookIds;
@@ -363,6 +390,7 @@ enum filterTypes {
_CREATOR = FLAG(10),
MAXSIZE = FLAG(11),
QUERY = FLAG(12),
NAME = FLAG(13),
};
Filter& Filter::local(bool accept)
@@ -450,24 +478,35 @@ Filter& Filter::query(std::string query)
return *this;
}
Filter& Filter::name(std::string name)
{
_name = name;
activeFilters |= NAME;
return *this;
}
#define ACTIVE(X) (activeFilters & (X))
#define FILTER(TAG, TEST) if (ACTIVE(TAG) && !(TEST)) { return false; }
bool Filter::accept(const Book& book) const
{
auto local = !book.getPath().empty();
if (ACTIVE(_LOCAL) && !local)
return false;
if (ACTIVE(_NOLOCAL) && local)
return false;
FILTER(_LOCAL, local)
FILTER(_NOLOCAL, !local)
auto valid = book.isPathValid();
if (ACTIVE(_VALID) && !valid)
return false;
if (ACTIVE(_NOVALID) && valid)
return false;
FILTER(_VALID, valid)
FILTER(_NOVALID, !valid)
auto remote = !book.getUrl().empty();
if (ACTIVE(_REMOTE) && !remote)
return false;
if (ACTIVE(_NOREMOTE) && remote)
return false;
FILTER(_REMOTE, remote)
FILTER(_NOREMOTE, !remote)
FILTER(MAXSIZE, book.getSize() <= _maxSize)
FILTER(LANG, book.getLanguage() == _lang)
FILTER(_PUBLISHER, book.getPublisher() == _publisher)
FILTER(_CREATOR, book.getCreator() == _creator)
FILTER(NAME, book.getName() == _name)
if (ACTIVE(ACCEPTTAGS)) {
if (!_acceptTags.empty()) {
auto vBookTags = split(book.getTags(), ";");
@@ -490,18 +529,6 @@ bool Filter::accept(const Book& book) const
}
}
}
if (ACTIVE(MAXSIZE) && book.getSize() > _maxSize)
return false;
if (ACTIVE(LANG) && book.getLanguage() != _lang)
return false;
if (ACTIVE(_PUBLISHER) && book.getPublisher() != _publisher)
return false;
if (ACTIVE(_CREATOR) && book.getCreator() != _creator)
return false;
if ( ACTIVE(QUERY)
&& !(matchRegex(book.getTitle(), "\\Q" + _query + "\\E")
|| matchRegex(book.getDescription(), "\\Q" + _query + "\\E")))

View File

@@ -53,13 +53,15 @@ void LibXMLDumper::handleBook(Book book, pugi::xml_node root_node) {
if (book.getOrigId().empty()) {
ADD_ATTR_NOT_EMPTY(entry_node, "title", book.getTitle());
ADD_ATTR_NOT_EMPTY(entry_node, "name", book.getName());
ADD_ATTR_NOT_EMPTY(entry_node, "tags", book.getTags());
ADD_ATTR_NOT_EMPTY(entry_node, "description", book.getDescription());
ADD_ATTR_NOT_EMPTY(entry_node, "language", book.getLanguage());
ADD_ATTR_NOT_EMPTY(entry_node, "creator", book.getCreator());
ADD_ATTR_NOT_EMPTY(entry_node, "publisher", book.getPublisher());
ADD_ATTR_NOT_EMPTY(entry_node, "name", book.getName());
ADD_ATTR_NOT_EMPTY(entry_node, "flavour", book.getFlavour());
ADD_ATTR_NOT_EMPTY(entry_node, "tags", book.getTags());
ADD_ATTR_NOT_EMPTY(entry_node, "faviconMimeType", book.getFaviconMimeType());
ADD_ATTR_NOT_EMPTY(entry_node, "faviconUrl", book.getFaviconUrl());
if (!book.getFavicon().empty())
ADD_ATTRIBUTE(entry_node, "favicon", base64_encode(book.getFavicon()));
} else {

View File

@@ -19,6 +19,8 @@
#include "manager.h"
#include "tools/pathTools.h"
#include <pugixml.hpp>
namespace kiwix
@@ -46,8 +48,9 @@ Manager::~Manager()
}
}
bool Manager::parseXmlDom(const pugi::xml_document& doc,
const bool readOnly,
const std::string& libraryPath)
bool readOnly,
const std::string& libraryPath,
bool trustLibrary)
{
pugi::xml_node libraryNode = doc.child("library");
@@ -61,12 +64,8 @@ bool Manager::parseXmlDom(const pugi::xml_document& doc,
book.updateFromXml(bookNode,
removeLastPathElement(libraryPath));
/* Update the book properties with the new importer */
if (libraryVersion.empty()
|| atoi(libraryVersion.c_str()) <= atoi(KIWIX_LIBRARY_VERSION)) {
if (!book.getPath().empty()) {
this->readBookFromPath(book.getPath(), &book);
}
if (!trustLibrary && !book.getPath().empty()) {
this->readBookFromPath(book.getPath(), &book);
}
manipulator->addBookToLibrary(book);
}
@@ -75,15 +74,16 @@ bool Manager::parseXmlDom(const pugi::xml_document& doc,
}
bool Manager::readXml(const std::string& xml,
const bool readOnly,
const std::string& libraryPath)
bool readOnly,
const std::string& libraryPath,
bool trustLibrary)
{
pugi::xml_document doc;
pugi::xml_parse_result result
= doc.load_buffer_inplace((void*)xml.data(), xml.size());
if (result) {
this->parseXmlDom(doc, readOnly, libraryPath);
this->parseXmlDom(doc, readOnly, libraryPath, trustLibrary);
}
return true;
@@ -134,21 +134,22 @@ bool Manager::readOpds(const std::string& content, const std::string& urlHost)
return false;
}
bool Manager::readFile(const std::string& path, const bool readOnly)
{
return this->readFile(path, path, readOnly);
}
bool Manager::readFile(const std::string& nativePath,
const std::string& UTF8Path,
const bool readOnly)
bool Manager::readFile(
const std::string& path,
bool readOnly,
bool trustLibrary)
{
bool retVal = true;
pugi::xml_document doc;
pugi::xml_parse_result result = doc.load_file(nativePath.c_str());
#ifdef _WIN32
pugi::xml_parse_result result = doc.load_file(Utf8ToWide(path).c_str());
#else
pugi::xml_parse_result result = doc.load_file(path.c_str());
#endif
if (result) {
this->parseXmlDom(doc, readOnly, UTF8Path);
this->parseXmlDom(doc, readOnly, path, trustLibrary);
} else {
retVal = false;
}
@@ -157,7 +158,7 @@ bool Manager::readFile(const std::string& nativePath,
* able to know where to save the library if new content are
* available */
if (!readOnly) {
this->writableLibraryPath = UTF8Path;
this->writableLibraryPath = path;
}
return retVal;

View File

@@ -32,13 +32,16 @@ else
kiwix_sources += 'subprocess_unix.cpp'
endif
if get_option('android')
subdir('android')
if 'android' in wrapper
install_dir = 'kiwix-lib/jniLibs/' + meson.get_cross_property('android_abi')
else
install_dir = get_option('libdir')
endif
if 'android' in wrapper or 'java' in wrapper
subdir('wrapper/java')
endif
config_h = configure_file(output : 'kiwix_config.h',
configuration : conf,
input : 'config.h.in')

View File

@@ -70,12 +70,15 @@ void OPDSDumper::setOpenSearchInfo(int totalResults, int startIndex, int count)
pugi::xml_node OPDSDumper::handleBook(Book book, pugi::xml_node root_node) {
auto entry_node = root_node.append_child("entry");
ADD_TEXT_ENTRY(entry_node, "title", book.getTitle());
ADD_TEXT_ENTRY(entry_node, "id", "urn:uuid:"+book.getId());
ADD_TEXT_ENTRY(entry_node, "icon", rootLocation + "/meta?name=favicon&content=" + book.getHumanReadableIdFromPath());
ADD_TEXT_ENTRY(entry_node, "updated", gen_date_from_yyyy_mm_dd(book.getDate()));
ADD_TEXT_ENTRY(entry_node, "title", book.getTitle());
ADD_TEXT_ENTRY(entry_node, "summary", book.getDescription());
ADD_TEXT_ENTRY(entry_node, "language", book.getLanguage());
ADD_TEXT_ENTRY(entry_node, "updated", gen_date_from_yyyy_mm_dd(book.getDate()));
ADD_TEXT_ENTRY(entry_node, "name", book.getName());
ADD_TEXT_ENTRY(entry_node, "flavour", book.getFlavour());
ADD_TEXT_ENTRY(entry_node, "tags", book.getTags());
ADD_TEXT_ENTRY(entry_node, "icon", rootLocation + "/meta?name=favicon&content=" + book.getHumanReadableIdFromPath());
auto content_node = entry_node.append_child("link");
content_node.append_attribute("type") = "text/html";
@@ -84,6 +87,9 @@ pugi::xml_node OPDSDumper::handleBook(Book book, pugi::xml_node root_node) {
auto author_node = entry_node.append_child("author");
ADD_TEXT_ENTRY(author_node, "name", book.getCreator());
auto publisher_node = entry_node.append_child("publisher");
ADD_TEXT_ENTRY(publisher_node, "name", book.getPublisher());
if (! book.getUrl().empty()) {
auto acquisition_link = entry_node.append_child("link");
acquisition_link.append_attribute("rel") = "http://opds-spec.org/acquisition/open-access";

View File

@@ -22,6 +22,8 @@
#include <zim/search.h>
#include "tools/otherTools.h"
inline char hi(char v)
{
char hex[] = "0123456789abcdef";
@@ -352,43 +354,6 @@ string Reader::getLicense() const
METADATA("License")
}
std::vector<std::string> convertTags(const std::string& tags_str)
{
auto tags = split(tags_str, ";");
std::vector<std::string> tagsList;
bool picSeen(false), vidSeen(false), detSeen(false), indexSeen(false);
for (auto tag: tags) {
picSeen |= (tag == "nopic" || startsWith(tag, "_pictures:"));
vidSeen |= (tag == "novid" || startsWith(tag, "_videos:"));
detSeen |= (tag == "nodet" || startsWith(tag, "_details:"));
indexSeen |= startsWith(tag, "_ftindex");
if (tag == "nopic") {
tagsList.push_back("_pictures:no");
} else if (tag == "novid") {
tagsList.push_back("_videos:no");
} else if (tag == "nodet") {
tagsList.push_back("_details:no");
} else if (tag == "_ftindex") {
tagsList.push_back("_ftindex:yes");
} else {
tagsList.push_back(tag);
}
}
if (!indexSeen) {
tagsList.push_back("_ftindex:no");
}
if (!picSeen) {
tagsList.push_back("_pictures:yes");
}
if (!vidSeen) {
tagsList.push_back("_videos:yes");
}
if (!detSeen) {
tagsList.push_back("_details:yes");
}
return tagsList;
}
string Reader::getTags(bool original) const
{
string tags_str;
@@ -400,26 +365,6 @@ string Reader::getTags(bool original) const
return join(tags, ";");
}
string getTagValueFromTagList(const std::vector<std::string>& tagList, const std::string& tagName)
{
for (auto tag: tagList) {
if (tag[0] == '_') {
auto delimPos = tag.find(':');
if (delimPos == string::npos) {
// No delimiter... what to do ?
continue;
}
auto cTagName = tag.substr(1, delimPos-1);
auto cTagValue = tag.substr(delimPos+1);
if (cTagName == tagName) {
return cTagValue;
}
}
}
std::stringstream ss;
ss << tagName << " cannot be found";
throw std::out_of_range(ss.str());
}
string Reader::getTagStr(const std::string& tagName) const
{
@@ -430,16 +375,7 @@ string Reader::getTagStr(const std::string& tagName) const
bool Reader::getTagBool(const std::string& tagName) const
{
auto tagValue = getTagStr(tagName);
if (tagValue == "yes") {
return true;
} else if (tagValue == "no") {
return false;
} else {
std::stringstream ss;
ss << "Tag value '" << tagValue << "' for " << tagName << " cannot be converted to bool.";
throw std::domain_error(ss.str());
}
return convertStrToBool(getTagStr(tagName));
}
string Reader::getRelation() const

View File

@@ -97,7 +97,7 @@ Reader* Searcher::get_reader(int readerIndex)
}
/* Search strings in the database */
void Searcher::search(std::string& search,
void Searcher::search(const std::string& search,
unsigned int resultStart,
unsigned int resultEnd,
const bool verbose)

View File

@@ -742,7 +742,6 @@ Response InternalServer::handle_catalog(const RequestContext& request)
kiwix::OPDSDumper opdsDumper;
opdsDumper.setRootLocation(m_root);
opdsDumper.setSearchDescriptionUrl("catalog/searchdescription.xml");
opdsDumper.setId(kiwix::to_string(uuid));
opdsDumper.setLibrary(mp_library);
response.set_mimeType("application/atom+xml;profile=opds-catalog;kind=acquisition; charset=utf-8");
std::vector<std::string> bookIdsToDump;
@@ -751,17 +750,22 @@ Response InternalServer::handle_catalog(const RequestContext& request)
uuid = zim::Uuid::generate(host);
bookIdsToDump = mp_library->filter(kiwix::Filter().valid(true).local(true).remote(true));
} else if (url == "search") {
std::string query;
std::string language;
std::vector<std::string> tags;
std::vector<std::string> noTags;
auto filter = kiwix::Filter().valid(true).local(true).remote(true);
string query("<Empty query>");
size_t count(10);
size_t startIndex(0);
try {
query = request.get_argument("q");
filter.query(query);
} catch (const std::out_of_range&) {}
try {
language = request.get_argument("lang");
filter.maxSize(extractFromString<unsigned long>(request.get_argument("maxsize")));
} catch (...) {}
try {
filter.name(request.get_argument("name"));
} catch (const std::out_of_range&) {}
try {
filter.lang(request.get_argument("lang"));
} catch (const std::out_of_range&) {}
try {
count = extractFromString<unsigned long>(request.get_argument("count"));
@@ -770,20 +774,14 @@ Response InternalServer::handle_catalog(const RequestContext& request)
startIndex = extractFromString<unsigned long>(request.get_argument("start"));
} catch (...) {}
try {
tags = kiwix::split(request.get_argument("notag"), ";");
filter.acceptTags(kiwix::split(request.get_argument("tag"), ";"));
} catch (...) {}
try {
noTags = kiwix::split(request.get_argument("notag"), ";");
filter.rejectTags(kiwix::split(request.get_argument("notag"), ";"));
} catch (...) {}
opdsDumper.setTitle("Search result for " + query);
uuid = zim::Uuid::generate();
bookIdsToDump = mp_library->filter(
kiwix::Filter().valid(true).local(true).remote(true)
.query(query)
.lang(language)
.acceptTags(tags)
.rejectTags(noTags)
);
bookIdsToDump = mp_library->filter(filter);
auto totalResults = bookIdsToDump.size();
bookIdsToDump.erase(bookIdsToDump.begin(), bookIdsToDump.begin()+startIndex);
if (count>0 && bookIdsToDump.size() > count) {
@@ -792,6 +790,7 @@ Response InternalServer::handle_catalog(const RequestContext& request)
opdsDumper.setOpenSearchInfo(totalResults, startIndex, bookIdsToDump.size());
}
opdsDumper.setId(kiwix::to_string(uuid));
response.set_content(opdsDumper.dumpOPDSFeed(bookIdsToDump));
return response;
}
@@ -870,23 +869,11 @@ Response InternalServer::handle_content(const RequestContext& request)
zim::Blob raw_content = entry.getBlob();
content = string(raw_content.data(), raw_content.size());
auto response = get_default_response();
response.set_mimeType(mimeType);
/* Special rewrite URL in case of ZIM file use intern *asbolute* url like
* /A/Kiwix */
if (mimeType.find("text/html") != string::npos) {
content = replaceRegex(content,
"$1$2" + m_root + "/" + bookName + "/$3/",
"(href|src)(=[\"|\']{0,1})/([A-Z|\\-])/");
content = replaceRegex(content,
"$1$2" + m_root + "/" + bookName + "/$3/",
"(@import[ ]+)([\"|\']{0,1})/([A-Z|\\-])/");
if (mimeType.find("text/html") != string::npos)
response.set_taskbar(bookName, reader->getTitle());
} else if (mimeType.find("text/css") != string::npos) {
content = replaceRegex(content,
"$1$2" + m_root + "/" + bookName + "/$3/",
"(url|URL)(\\([\"|\']{0,1})/([A-Z|\\-])/");
}
response.set_mimeType(mimeType);
response.set_content(content);
response.set_compress(true);
response.set_cache(true);

View File

@@ -140,7 +140,7 @@ int Response::send(const RequestContext& request, MHD_Connection* connection)
bool shouldCompress = m_compress && request.can_compress();
shouldCompress &= m_mimeType.find("text/") != string::npos
|| m_mimeType.find("application/javascript") != string::npos
|| m_mimeType.find("application/json") != string::npos;
|| m_mimeType.find("application/json") != string::npos;
shouldCompress &= (m_content.size() > KIWIX_MIN_CONTENT_SIZE_TO_DEFLATE);

View File

@@ -4,6 +4,7 @@
#include <windows.h>
#include <winbase.h>
#include <shlwapi.h>
#include <iostream>
#include <sstream>
@@ -34,11 +35,11 @@ DWORD WINAPI WinImpl::waitForPID(void* _self)
return 0;
}
std::unique_ptr<wchar_t[]> toWideChar(const std::string& value)
std::unique_ptr<wchar_t[]> toWideChar(const std::string& value, size_t min_size = 0)
{
auto size = MultiByteToWideChar(CP_UTF8, 0,
size_t size = MultiByteToWideChar(CP_UTF8, 0,
value.c_str(), -1, nullptr, 0);
auto wdata = std::unique_ptr<wchar_t[]>(new wchar_t[size]);
auto wdata = std::unique_ptr<wchar_t[]>(new wchar_t[size>min_size?size:min_size]);
auto ret = MultiByteToWideChar(CP_UTF8, 0,
value.c_str(), -1, wdata.get(), size);
if (0 == ret) {
@@ -46,6 +47,9 @@ std::unique_ptr<wchar_t[]> toWideChar(const std::string& value)
oss << "Cannot convert to wchar : " << GetLastError();
throw std::runtime_error(oss.str());
}
if (size < min_size) {
memset(wdata.get() + size, 0, min_size-size);
}
return wdata;
}
@@ -55,14 +59,16 @@ void WinImpl::run(commandLine_t& commandLine)
STARTUPINFOW startInfo = {0};
PROCESS_INFORMATION procInfo;
startInfo.cb = sizeof(startInfo);
std::ostringstream oss;
std::wostringstream oss;
for(auto& item: commandLine) {
oss << item << " ";
auto witem = toWideChar(item, MAX_PATH);
PathQuoteSpacesW(witem.get());
oss << witem.get() << " ";
}
auto wCommandLine = toWideChar(oss.str());
auto wCommandLine = oss.str();
if (CreateProcessW(
NULL,
wCommandLine.get(),
const_cast<wchar_t*>(wCommandLine.c_str()),
NULL,
NULL,
false,
@@ -70,7 +76,8 @@ void WinImpl::run(commandLine_t& commandLine)
NULL,
NULL,
&startInfo,
&procInfo)) {
&procInfo))
{
m_pid = procInfo.dwProcessId;
m_handle = procInfo.hProcess;
CloseHandle(procInfo.hThread);

View File

@@ -1,4 +1,4 @@
/*
/*
base64.cpp and base64.h
Copyright (C) 2004-2008 René Nyffenegger
@@ -27,7 +27,7 @@
#include <tools/base64.h>
#include <iostream>
static const std::string base64_chars =
static const std::string base64_chars =
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"abcdefghijklmnopqrstuvwxyz"
"0123456789+/";

View File

@@ -25,7 +25,10 @@
#include <unistd.h>
#endif
#include "tools/stringTools.h"
#include <map>
#include <sstream>
#include <pugixml.hpp>
@@ -204,3 +207,76 @@ std::string kiwix::nodeToString(const pugi::xml_node& node)
std::string kiwix::converta2toa3(const std::string& a2code){
return codeisomapping.at(a2code);
}
std::vector<std::string> kiwix::convertTags(const std::string& tags_str)
{
auto tags = kiwix::split(tags_str, ";");
std::vector<std::string> tagsList;
bool picSeen(false), vidSeen(false), detSeen(false), indexSeen(false);
for (auto tag: tags) {
picSeen |= (tag == "nopic" || startsWith(tag, "_pictures:"));
vidSeen |= (tag == "novid" || startsWith(tag, "_videos:"));
detSeen |= (tag == "nodet" || startsWith(tag, "_details:"));
indexSeen |= kiwix::startsWith(tag, "_ftindex");
if (tag == "nopic") {
tagsList.push_back("_pictures:no");
} else if (tag == "novid") {
tagsList.push_back("_videos:no");
} else if (tag == "nodet") {
tagsList.push_back("_details:no");
} else if (tag == "_ftindex") {
tagsList.push_back("_ftindex:yes");
} else {
tagsList.push_back(tag);
}
}
if (!indexSeen) {
tagsList.push_back("_ftindex:no");
}
if (!picSeen) {
tagsList.push_back("_pictures:yes");
}
if (!vidSeen) {
tagsList.push_back("_videos:yes");
}
if (!detSeen) {
tagsList.push_back("_details:yes");
}
return tagsList;
}
std::string kiwix::getTagValueFromTagList(
const std::vector<std::string>& tagList, const std::string& tagName)
{
for (auto tag: tagList) {
if (tag[0] == '_') {
auto delimPos = tag.find(':');
if (delimPos == std::string::npos) {
// No delimiter... what to do ?
continue;
}
auto cTagName = tag.substr(1, delimPos-1);
auto cTagValue = tag.substr(delimPos+1);
if (cTagName == tagName) {
return cTagValue;
}
}
}
std::stringstream ss;
ss << tagName << " cannot be found";
throw std::out_of_range(ss.str());
}
bool kiwix::convertStrToBool(const std::string& value)
{
if (value == "yes") {
return true;
} else if (value == "no") {
return false;
}
std::stringstream ss;
ss << "Tag value '" << value << "' cannot be converted to bool.";
throw std::domain_error(ss.str());
}

View File

@@ -25,6 +25,7 @@
#include <memory>
#include <map>
#include <stdexcept>
#include <pthread.h>
std::map<std::string, std::shared_ptr<icu::RegexPattern>> regexCache;

87
src/wrapper/java/book.cpp Normal file
View File

@@ -0,0 +1,87 @@
/*
* Copyright (C) 2020 Matthieu Gautier <mgautier@kymeria.fr>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
* MA 02110-1301, USA.
*/
#include <jni.h>
#include "org_kiwix_kiwixlib_Book.h"
#include "utils.h"
#include "book.h"
JNIEXPORT void JNICALL
Java_org_kiwix_kiwixlib_Book_allocate(
JNIEnv* env, jobject thisObj)
{
allocate<kiwix::Book>(env, thisObj);
}
JNIEXPORT void JNICALL
Java_org_kiwix_kiwixlib_Book_dispose(JNIEnv* env, jobject thisObj)
{
dispose<kiwix::Book>(env, thisObj);
}
#define BOOK (getPtr<kiwix::Book>(env, thisObj))
METHOD(void, Book, update__Lorg_kiwix_kiwixlib_Book_2, jobject otherBook)
{
BOOK->update(*getPtr<kiwix::Book>(env, otherBook));
}
METHOD(void, Book, update__Lorg_kiwix_kiwixlib_JNIKiwixReader_2, jobject reader)
{
BOOK->update(**Handle<kiwix::Reader>::getHandle(env, reader));
}
#define GETTER(retType, name) JNIEXPORT retType JNICALL \
Java_org_kiwix_kiwixlib_Book_##name (JNIEnv* env, jobject thisObj) \
{ \
auto cRet = BOOK->name(); \
retType ret = c2jni(cRet, env); \
return ret; \
}
GETTER(jstring, getId)
GETTER(jstring, getPath)
GETTER(jboolean, isPathValid)
GETTER(jstring, getTitle)
GETTER(jstring, getDescription)
GETTER(jstring, getLanguage)
GETTER(jstring, getCreator)
GETTER(jstring, getPublisher)
GETTER(jstring, getDate)
GETTER(jstring, getUrl)
GETTER(jstring, getName)
GETTER(jstring, getFlavour)
GETTER(jstring, getTags)
GETTER(jlong, getArticleCount)
GETTER(jlong, getMediaCount)
GETTER(jlong, getSize)
GETTER(jstring, getFavicon)
GETTER(jstring, getFaviconUrl)
GETTER(jstring, getFaviconMimeType)
METHOD(jstring, Book, getTagStr, jstring tagName) try {
auto cRet = BOOK->getTagStr(jni2c(tagName, env));
return c2jni(cRet, env);
} catch(...) {
return c2jni<std::string>("", env);
}
#undef GETTER

View File

@@ -0,0 +1,63 @@
/*
* Copyright (C) 2019-2020 Matthieu Gautier <mgautier@kymeria.fr>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
* MA 02110-1301, USA.
*/
#include <jni.h>
#include "org_kiwix_kiwixlib_Filter.h"
#include "library.h"
#include "utils.h"
/* Kiwix Reader JNI functions */
METHOD0(void, Filter, allocate) {
allocate<kiwix::Filter>(env, thisObj);
}
METHOD0(void, Filter, dispose) {
dispose<kiwix::Library>(env, thisObj);
}
#define FILTER (getPtr<kiwix::Filter>(env, thisObj))
#define FORWARD(name, args_type) \
METHOD(jobject, Filter, name, args_type value) { \
FILTER->name(jni2c(value, env)); \
return thisObj; \
}
#define FORWARDA(name, args_type) \
METHOD(jobject, Filter, name, jobjectArray value) { \
FILTER->name(jni2c<args_type>(value, env)); \
return thisObj; \
}
FORWARD(local, jboolean)
FORWARD(remote, jboolean)
FORWARD(valid, jboolean)
FORWARDA(acceptTags, jstring)
FORWARDA(rejectTags, jstring)
FORWARD(lang, jstring)
FORWARD(publisher, jstring)
FORWARD(creator, jstring)
FORWARD(maxSize, jlong)
FORWARD(query, jstring)

View File

@@ -28,7 +28,11 @@
#include "utils.h"
#if __ANDROID__
pthread_mutex_t globalLock = PTHREAD_RECURSIVE_MUTEX_INITIALIZER;
#else
pthread_mutex_t globalLock = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;
#endif
JNIEXPORT void JNICALL Java_org_kiwix_kiwixlib_JNIICU_setDataDirectory(
JNIEnv* env, jclass kclass, jstring dirStr)

View File

@@ -21,7 +21,7 @@
#include <jni.h>
#include <zim/file.h>
#include <android/log.h>
#include <exception>
#include "org_kiwix_kiwixlib_JNIKiwixReader.h"
#include "tools/base64.h"
@@ -34,14 +34,14 @@ JNIEXPORT jlong JNICALL Java_org_kiwix_kiwixlib_JNIKiwixReader_getNativeReader(
{
std::string cPath = jni2c(filename, env);
__android_log_print(ANDROID_LOG_INFO, "kiwix", "Attempting to create reader with: %s", cPath.c_str());
LOG("Attempting to create reader with: %s", cPath.c_str());
Lock l;
try {
kiwix::Reader* reader = new kiwix::Reader(cPath);
return reinterpret_cast<jlong>(new Handle<kiwix::Reader>(reader));
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_WARN, "kiwix", "Error opening ZIM file");
__android_log_print(ANDROID_LOG_WARN, "kiwix", e.what());
LOG("Error opening ZIM file");
LOG(e.what());
return 0;
}
}
@@ -64,8 +64,8 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getMainPage(JNIEnv* env, jobject obj)
std::string cUrl = READER->getMainPage().getPath();
url = c2jni(cUrl, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get ZIM main page");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get ZIM main page");
LOG(e.what());
url = NULL;
}
return url;
@@ -80,8 +80,8 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getId(JNIEnv* env, jobject obj)
std::string cId = READER->getId();
id = c2jni(cId, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get ZIM id");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get ZIM id");
LOG(e.what());
id = NULL;
}
@@ -95,10 +95,10 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getFileSize(JNIEnv* env, jobject obj)
try {
int cSize = READER->getFileSize();
size = c2jni(cSize);
size = c2jni(cSize, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get ZIM file size");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get ZIM file size");
LOG(e.what());
}
return size;
@@ -113,8 +113,8 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getCreator(JNIEnv* env, jobject obj)
std::string cCreator = READER->getCreator();
creator = c2jni(cCreator, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get ZIM creator");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get ZIM creator");
LOG(e.what());
creator = NULL;
}
@@ -130,8 +130,8 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getPublisher(JNIEnv* env, jobject obj)
std::string cPublisher = READER->getPublisher();
publisher = c2jni(cPublisher, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get ZIM publish");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get ZIM publish");
LOG(e.what());
publisher = NULL;
}
return publisher;
@@ -146,8 +146,8 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getName(JNIEnv* env, jobject obj)
std::string cName = READER->getName();
name = c2jni(cName, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get ZIM name");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get ZIM name");
LOG(e.what());
name = NULL;
}
return name;
@@ -166,8 +166,8 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getFavicon(JNIEnv* env, jobject obj)
base64_encode(cContent),
env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get ZIM favicon");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get ZIM favicon");
LOG(e.what());
favicon = NULL;
}
return favicon;
@@ -182,8 +182,8 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getDate(JNIEnv* env, jobject obj)
std::string cDate = READER->getDate();
date = c2jni(cDate, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get ZIM date");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get ZIM date");
LOG(e.what());
date = NULL;
}
return date;
@@ -198,8 +198,8 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getLanguage(JNIEnv* env, jobject obj)
std::string cLanguage = READER->getLanguage();
language = c2jni(cLanguage, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get ZIM language");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get ZIM language");
LOG(e.what());
language = NULL;
}
@@ -217,8 +217,8 @@ JNIEXPORT jstring JNICALL Java_org_kiwix_kiwixlib_JNIKiwixReader_getMimeType(
auto cMimeType = entry.getMimetype();
mimeType = c2jni(cMimeType, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get mime-type for url: %s", cUrl.c_str());
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get mime-type for url: %s", cUrl.c_str());
LOG(e.what());
mimeType = NULL;
}
return mimeType;
@@ -234,7 +234,7 @@ JNIEXPORT jstring JNICALL Java_org_kiwix_kiwixlib_JNIKiwixReader_checkUrl(
entry = entry.getFinalEntry();
finalUrl = c2jni(entry.getPath(), env);
} catch (std::exception& e) {
finalUrl = c2jni("", env);
finalUrl = c2jni(std::string(), env);
}
return finalUrl;
}
@@ -268,8 +268,8 @@ JNIEXPORT jbyteArray JNICALL Java_org_kiwix_kiwixlib_JNIKiwixReader_getContent(
data, 0, cSize, reinterpret_cast<const jbyte*>(entry.getBlob().data()));
}
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get content for url: %s", cUrl.c_str());
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get content for url: %s", cUrl.c_str());
LOG(e.what());
}
return data;
@@ -284,8 +284,8 @@ JNIEXPORT jbyteArray JNICALL Java_org_kiwix_kiwixlib_JNIKiwixReader_getContentPa
/* Default values */
/* Retrieve the content */
std::string cUrl = jni2c(url, env);
unsigned int cOffset = jni2c(offset);
unsigned int cLen = jni2c(len);
unsigned int cOffset = jni2c(offset, env);
unsigned int cLen = jni2c(len, env);
try {
auto entry = READER->getEntryFromEncodedPath(cUrl);
entry = entry.getFinalEntry();
@@ -300,12 +300,28 @@ JNIEXPORT jbyteArray JNICALL Java_org_kiwix_kiwixlib_JNIKiwixReader_getContentPa
setIntObjValue(cLen, sizeObj, env);
}
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get partial content for url: %s (%u : %u)", cUrl.c_str(), cOffset, cLen);
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get partial content for url: %s (%u : %u)", cUrl.c_str(), cOffset, cLen);
LOG(e.what());
}
return data;
}
JNIEXPORT jlong JNICALL
Java_org_kiwix_kiwixlib_JNIKiwixReader_getArticleSize(
JNIEnv* env, jobject obj, jstring url)
{
std::string cUrl = jni2c(url, env);
try {
auto entry = READER->getEntryFromEncodedPath(cUrl);
entry = entry.getFinalEntry();
return c2jni(entry.getSize(), env);
} catch(std::exception& e) {
LOG("Unable to get size for url : %s", cUrl.c_str());
LOG(e.what());
}
return c2jni(0, env);
}
JNIEXPORT jobject JNICALL
Java_org_kiwix_kiwixlib_JNIKiwixReader_getDirectAccessInformation(
JNIEnv* env, jobject obj, jstring url)
@@ -322,8 +338,8 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getDirectAccessInformation(
auto part_info = entry.getDirectAccessInfo();
setPairObjValue(part_info.first, part_info.second, pair, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get direct access info for url: %s", cUrl.c_str());
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get direct access info for url: %s", cUrl.c_str());
LOG(e.what());
}
return pair;
}
@@ -336,15 +352,15 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_searchSuggestions(JNIEnv* env,
{
jboolean retVal = JNI_FALSE;
std::string cPrefix = jni2c(prefix, env);
unsigned int cCount = jni2c(count);
unsigned int cCount = jni2c(count, env);
try {
if (READER->searchSuggestionsSmart(cPrefix, cCount)) {
retVal = JNI_TRUE;
}
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_WARN, "kiwix", "Unable to get search results for pattern: %s", cPrefix.c_str());
__android_log_print(ANDROID_LOG_WARN, "kiwix", e.what());
LOG("Unable to get search results for pattern: %s", cPrefix.c_str());
LOG(e.what());
}
return retVal;
@@ -353,19 +369,22 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_searchSuggestions(JNIEnv* env,
JNIEXPORT jboolean JNICALL
Java_org_kiwix_kiwixlib_JNIKiwixReader_getNextSuggestion(JNIEnv* env,
jobject obj,
jobject titleObj)
jobject titleObj,
jobject urlObj)
{
jboolean retVal = JNI_FALSE;
std::string cTitle;
std::string cUrl;
try {
if (READER->getNextSuggestion(cTitle)) {
if (READER->getNextSuggestion(cTitle, cUrl)) {
setStringObjValue(cTitle, titleObj, env);
setStringObjValue(cUrl, urlObj, env);
retVal = JNI_TRUE;
}
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_WARN, "kiwix", "Unable to get next suggestion");
__android_log_print(ANDROID_LOG_WARN, "kiwix", e.what());
LOG("Unable to get next suggestion");
LOG(e.what());
}
return retVal;
@@ -385,8 +404,8 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getPageUrlFromTitle(JNIEnv* env,
setStringObjValue(entry.getPath(), urlObj, env);
return JNI_TRUE;
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_WARN, "kiwix", "Unable to get url for title %s: ", cTitle.c_str());
__android_log_print(ANDROID_LOG_WARN, "kiwix", e.what());
LOG("Unable to get url for title %s: ", cTitle.c_str());
LOG(e.what());
}
return JNI_FALSE;
@@ -401,8 +420,8 @@ JNIEXPORT jstring JNICALL Java_org_kiwix_kiwixlib_JNIKiwixReader_getTitle(
std::string cTitle = READER->getTitle();
title = c2jni(cTitle, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get zim title");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get zim title");
LOG(e.what());
title = NULL;
}
return title;
@@ -417,8 +436,8 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getDescription(JNIEnv* env, jobject obj)
std::string cDescription = READER->getDescription();
description = c2jni(cDescription, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get zim description");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get zim description");
LOG(e.what());
description = NULL;
}
return description;
@@ -430,10 +449,10 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getArticleCount(JNIEnv* env, jobject obj)
jint articleCount = 0;
try {
auto cArticleCount = READER->getArticleCount();
articleCount = c2jni(cArticleCount);
articleCount = c2jni(cArticleCount, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get article count.");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get article count.");
LOG(e.what());
}
return articleCount;
}
@@ -444,10 +463,10 @@ Java_org_kiwix_kiwixlib_JNIKiwixReader_getMediaCount(JNIEnv* env, jobject obj)
jint mediaCount = 0;
try {
auto cMediaCount = READER->getMediaCount();
mediaCount = c2jni(cMediaCount);
mediaCount = c2jni(cMediaCount, env);
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get media count.");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get media count.");
LOG(e.what());
}
return mediaCount;
}
@@ -464,8 +483,8 @@ JNIEXPORT jboolean JNICALL Java_org_kiwix_kiwixlib_JNIKiwixReader_getRandomPage(
setStringObjValue(cUrl, urlObj, env);
retVal = JNI_TRUE;
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_ERROR, "kiwix", "Unable to get random page");
__android_log_print(ANDROID_LOG_ERROR, "kiwix", e.what());
LOG("Unable to get random page");
LOG(e.what());
}
return retVal;
}

View File

@@ -59,7 +59,7 @@ JNIEXPORT void JNICALL Java_org_kiwix_kiwixlib_JNIKiwixSearcher_search(
JNIEnv* env, jobject obj, jstring query, jint count)
{
std::string cquery = jni2c(query, env);
unsigned int ccount = jni2c(count);
unsigned int ccount = jni2c(count, env);
SEARCHER->search(cquery, 0, ccount);
}

View File

@@ -21,7 +21,6 @@
#include <jni.h>
#include <zim/file.h>
#include <android/log.h>
#include "org_kiwix_kiwixlib_JNIKiwixServer.h"
#include "tools/base64.h"
@@ -32,15 +31,15 @@
JNIEXPORT jlong JNICALL Java_org_kiwix_kiwixlib_JNIKiwixServer_getNativeServer(
JNIEnv* env, jobject obj, jobject jLibrary)
{
__android_log_print(ANDROID_LOG_INFO, "kiwix", "Attempting to create server");
LOG("Attempting to create server");
Lock l;
try {
auto library = Handle<kiwix::Library>::getHandle(env, jLibrary);
kiwix::Server* server = new kiwix::Server(*library);
auto library = getPtr<kiwix::Library>(env, jLibrary);
kiwix::Server* server = new kiwix::Server(library);
return reinterpret_cast<jlong>(new Handle<kiwix::Server>(server));
} catch (std::exception& e) {
__android_log_print(ANDROID_LOG_WARN, "kiwix", "Error creating the server");
__android_log_print(ANDROID_LOG_WARN, "kiwix", e.what());
LOG("Error creating the server");
LOG(e.what());
return 0;
}
}

View File

@@ -0,0 +1,96 @@
/*
* Copyright (C) 2019-2020 Matthieu Gautier <mgautier@kymeria.fr>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
* MA 02110-1301, USA.
*/
#include <jni.h>
#include "org_kiwix_kiwixlib_Library.h"
#include "library.h"
#include "reader.h"
#include "utils.h"
/* Kiwix Reader JNI functions */
JNIEXPORT void JNICALL
Java_org_kiwix_kiwixlib_Library_allocate(
JNIEnv* env, jobject thisObj)
{
allocate<kiwix::Library>(env, thisObj);
}
JNIEXPORT void JNICALL
Java_org_kiwix_kiwixlib_Library_dispose(JNIEnv* env, jobject thisObj)
{
dispose<kiwix::Library>(env, thisObj);
}
#define LIBRARY (getPtr<kiwix::Library>(env, thisObj))
/* Kiwix library functions */
JNIEXPORT jboolean JNICALL
Java_org_kiwix_kiwixlib_Library_addBook(
JNIEnv* env, jobject thisObj, jstring path)
{
auto cPath = jni2c(path, env);
try {
kiwix::Reader reader(cPath);
kiwix::Book book;
book.update(reader);
return LIBRARY->addBook(book);
} catch (std::exception& e) {
LOG("Unable to add the book");
LOG(e.what()); }
return false;
}
METHOD(jobject, Library, getBookById, jstring id) {
auto cId = jni2c(id, env);
auto cBook = new kiwix::Book(LIBRARY->getBookById(cId));
jclass cls = env->FindClass("org/kiwix/kiwixlib/Book");
jmethodID constructorId = env->GetMethodID(cls, "<init>", "()V");
jobject book = env->NewObject(cls, constructorId);
setPtr(env, book, cBook);
return book;
}
METHOD(jint, Library, getBookCount, jboolean localBooks, jboolean remoteBooks) {
return LIBRARY->getBookCount(localBooks, remoteBooks);
}
METHOD0(jobjectArray, Library, getBooksIds) {
return c2jni(LIBRARY->getBooksIds(), env);
}
METHOD(jobjectArray, Library, filter, jobject filterObj) {
auto filter = getPtr<kiwix::Filter>(env, filterObj);
return c2jni(LIBRARY->filter(*filter), env);
}
METHOD0(jobjectArray, Library, getBooksLanguages) {
return c2jni(LIBRARY->getBooksLanguages(), env);
}
METHOD0(jobjectArray, Library, getBooksCreators) {
return c2jni(LIBRARY->getBooksCreators(), env);
}
METHOD0(jobjectArray, Library, getBooksPublisher) {
return c2jni(LIBRARY->getBooksPublishers(), env);
}

View File

@@ -0,0 +1,132 @@
/*
* Copyright (C) 2020 Matthieu Gautier <mgautier@kymeria.fr>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
* MA 02110-1301, USA.
*/
#include <jni.h>
#include <zim/file.h>
#include "org_kiwix_kiwixlib_Manager.h"
#include "manager.h"
#include "utils.h"
JNIEXPORT void JNICALL
Java_org_kiwix_kiwixlib_Manager_allocate(
JNIEnv* env, jobject thisObj, jobject libraryObj)
{
auto lib = getPtr<kiwix::Library>(env, libraryObj);
allocate<kiwix::Manager>(env, thisObj, lib);
}
JNIEXPORT void JNICALL
Java_org_kiwix_kiwixlib_Manager_dispose(JNIEnv* env, jobject thisObj)
{
dispose<kiwix::Manager>(env, thisObj);
}
#define MANAGER (getPtr<kiwix::Manager>(env, thisObj))
/* Kiwix manager functions */
JNIEXPORT jboolean JNICALL
Java_org_kiwix_kiwixlib_Manager_readFile(
JNIEnv* env, jobject thisObj, jstring path)
{
auto cPath = jni2c(path, env);
try {
return MANAGER->readFile(cPath);
} catch (std::exception& e) {
LOG("Unable to get readFile");
LOG(e.what());
}
return false;
}
JNIEXPORT jboolean JNICALL
Java_org_kiwix_kiwixlib_Manager_readXml(
JNIEnv* env, jobject thisObj, jstring content, jstring libraryPath)
{
auto cContent = jni2c(content, env);
auto cPath = jni2c(libraryPath, env);
try {
return MANAGER->readXml(cContent, false, cPath);
} catch (std::exception& e) {
LOG("Unable to get ZIM id");
LOG(e.what());
}
return false;
}
JNIEXPORT jboolean JNICALL
Java_org_kiwix_kiwixlib_Manager_readOpds(
JNIEnv* env, jobject thisObj, jstring content, jstring urlHost)
{
auto cContent = jni2c(content, env);
auto cUrl = jni2c(urlHost, env);
try {
return MANAGER->readOpds(cContent, cUrl);
} catch (std::exception& e) {
LOG("Unable to get ZIM id");
LOG(e.what());
}
return false;
}
JNIEXPORT jboolean JNICALL
Java_org_kiwix_kiwixlib_Manager_readBookmarkFile(
JNIEnv* env, jobject thisObj, jstring path)
{
auto cPath = jni2c(path, env);
try {
return MANAGER->readBookmarkFile(cPath);
} catch (std::exception& e) {
LOG("Unable to get ZIM id");
LOG(e.what());
}
return false;
}
JNIEXPORT jstring JNICALL
Java_org_kiwix_kiwixlib_Manager_addBookFromPath(
JNIEnv* env, jobject thisObj,
jstring pathToOpen, jstring pathToSave, jstring url, jboolean checkMetaData)
{
auto cPathToOpen = jni2c(pathToOpen, env);
auto cPathToSave = jni2c(pathToSave, env);
auto cUrl = jni2c(url, env);
jstring id = NULL;
try {
auto cId = MANAGER->addBookFromPathAndGetId(cPathToOpen, cPathToSave, cUrl, checkMetaData);
if ( !cId.empty() ) {
id = c2jni(cId, env);
}
} catch (std::exception& e) {
LOG("Unable to get ZIM file size");
LOG(e.what());
}
return id;
}

View File

@@ -0,0 +1,55 @@
java_sources = files([
'org/kiwix/kiwixlib/JNIICU.java',
'org/kiwix/kiwixlib/Book.java',
'org/kiwix/kiwixlib/JNIKiwixReader.java',
'org/kiwix/kiwixlib/Library.java',
'org/kiwix/kiwixlib/Manager.java',
'org/kiwix/kiwixlib/Filter.java',
'org/kiwix/kiwixlib/JNIKiwixSearcher.java',
'org/kiwix/kiwixlib/JNIKiwixServer.java',
'org/kiwix/kiwixlib/JNIKiwixInt.java',
'org/kiwix/kiwixlib/JNIKiwixString.java',
'org/kiwix/kiwixlib/JNIKiwixBool.java',
'org/kiwix/kiwixlib/JNIKiwixException.java',
'org/kiwix/kiwixlib/Pair.java'
])
kiwix_jni = custom_target('jni',
input: java_sources,
output: ['org_kiwix_kiwixlib_JNIKiwix.h',
'org_kiwix_kiwixlib_Book.h',
'org_kiwix_kiwixlib_JNIKiwixReader.h',
'org_kiwix_kiwixlib_Library.h',
'org_kiwix_kiwixlib_Manager.h',
'org_kiwix_kiwixlib_Filter.h',
'org_kiwix_kiwixlib_JNIKiwixServer.h',
'org_kiwix_kiwixlib_JNIKiwixSearcher.h',
'org_kiwix_kiwixlib_JNIKiwixSearcher_Result.h'],
command:['javac', '-d', '@OUTDIR@', '-h', '@OUTDIR@', '@INPUT@']
)
jni_sources = files([
'kiwixicu.cpp',
'book.cpp',
'kiwixreader.cpp',
'library.cpp',
'manager.cpp',
'filter.cpp',
'kiwixsearcher.cpp',
'kiwixserver.cpp',
])
kiwix_sources += jni_sources + [kiwix_jni]
if 'java' in wrapper
kiwix_jar = jar('kiwixlib', java_sources)
#junit_jar = files('org/kiwix/testing/junit-4.13.jar')
#test_jar = jar('testing', 'org/kiwix/testing/test.java',
# link_with: [kiwix_jar, junit_jar])
#test('javatest', test_jar)
endif
install_subdir('org', install_dir: 'kiwix-lib/java', exclude_directories: ['kiwix/testing'])
install_subdir('res', install_dir: 'kiwix-lib')
install_data('AndroidManifest.xml', install_dir: 'kiwix-lib')

View File

@@ -0,0 +1,47 @@
package org.kiwix.kiwixlib;
public class Book
{
public Book() { allocate(); }
public native void update(Book book);
public native void update(JNIKiwixReader reader);
@Override
protected void finalize() { dispose(); }
public native String getId();
public native String getPath();
public native boolean isPathValid();
public native String getTitle();
public native String getDescription();
public native String getLanguage();
public native String getCreator();
public native String getPublisher();
public native String getDate();
public native String getUrl();
public native String getName();
public native String getFlavour();
public native String getTags();
/**
* Return the value associated to the tag tagName
*
* @param tagName the tag name to search for.
* @return The value of the tag. If the tag is not found, return empty string.
*/
public native String getTagStr(String tagName);
public native long getArticleCount();
public native long getMediaCount();
public native long getSize();
public native String getFavicon();
public native String getFaviconUrl();
public native String getFaviconMimeType();
private native void allocate();
private native void dispose();
private long nativeHandle;
}

View File

@@ -0,0 +1,44 @@
/*
* Copyright (C) 2019-2020 Matthieu Gautier <mgautier@kymeria.fr>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
* MA 02110-1301, USA.
*/
package org.kiwix.kiwixlib;
public class Filter
{
public native Filter local(boolean accept);
public native Filter remote(boolean accept);
public native Filter valid(boolean accept);
public native Filter acceptTags(String[] tags);
public native Filter rejectTags(String[] tags);
public native Filter lang(String lang);
public native Filter publisher(String publisher);
public native Filter creator(String creator);
public native Filter maxSize(long size);
public native Filter query(String query);
public Filter() { allocate(); }
@Override
protected void finalize() { dispose(); }
private native void allocate();
private native void dispose();
private long nativeHandle;
}

View File

@@ -82,6 +82,16 @@ public class JNIKiwixReader
int len,
JNIKiwixInt size);
/**
*
* Get the size of an article.
*
* @param url The url of the article.
* @return The size of the final (redirections are resolved) article (in byte).
* Return 0 if the article is not found.
*/
public native long getArticleSize(String url);
/**
* getDirectAccessInformation.
*
@@ -102,7 +112,7 @@ public class JNIKiwixReader
public native boolean searchSuggestions(String prefix, int count);
public native boolean getNextSuggestion(JNIKiwixString title);
public native boolean getNextSuggestion(JNIKiwixString title, JNIKiwixString url);
public native boolean getPageUrlFromTitle(String title, JNIKiwixString url);

View File

@@ -20,7 +20,7 @@
package org.kiwix.kiwixlib;
import org.kiwix.kiwixlib.JNIKiwixException;
import org.kiwix.kiwixlib.JNIKiwixLibrary;
import org.kiwix.kiwixlib.Library;
public class JNIKiwixServer
{
@@ -38,11 +38,11 @@ public class JNIKiwixServer
public native void stop();
public JNIKiwixServer(JNIKiwixLibrary library)
public JNIKiwixServer(Library library)
{
nativeHandle = getNativeServer(library);
}
private native long getNativeServer(JNIKiwixLibrary library);
private native long getNativeServer(Library library);
private long nativeHandle;
}

View File

@@ -1,6 +1,5 @@
/*
* Copyright (C) 2013 Emmanuel Engelhart <kelson@kiwix.org>
* Copyright (C) 2017 Matthieu Gautier <mgautier@kymeria.fr>
* Copyright (C) 2019-2020 Matthieu Gautier <mgautier@kymeria.fr>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -20,19 +19,31 @@
package org.kiwix.kiwixlib;
import org.kiwix.kiwixlib.Book;
import org.kiwix.kiwixlib.JNIKiwixException;
public class JNIKiwixLibrary
public class Library
{
public native boolean addBook(String path) throws JNIKiwixException;
public JNIKiwixLibrary()
public native Book getBookById(String id);
public native int getBookCount(boolean localBooks, boolean remoteBooks);
public native String[] getBooksIds();
public native String[] filter(Filter filter);
public native String[] getBooksLanguages();
public native String[] getBooksCreators();
public native String[] getBooksPublishers();
public Library()
{
nativeHandle = getNativeLibrary();
allocate();
}
public native void dispose();
private native long getNativeLibrary();
@Override
protected void finalize() { dispose(); }
private native void allocate();
private native void dispose();
private long nativeHandle;
}

View File

@@ -0,0 +1,90 @@
/*
* Copyright (C) 2020 Matthieu Gautier <mgautier@kymeria.fr>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
* MA 02110-1301, USA.
*/
package org.kiwix.kiwixlib;
import org.kiwix.kiwixlib.Library;
public class Manager
{
/**
* Read a `library.xml` file and add books in the library.
*
* @param path The (utf8) path to the `library.xml` file.
* @return True if the file has been properly parsed.
*/
public native boolean readFile(String path);
/**
* Load a library content stored in a string (at `library.xml` format).
*
* @param content The content corresponding of the library xml.
* @param libraryPath The library path (used to resolve relative paths)
* @return True if the content has been properly parsed.
*/
public native boolean readXml(String content, String libraryPath);
/**
* Load a library content stored in a string (at OPDS stream format)
*
* @param content the content of the OPDS stream.
* @param urlHost the url of the stream (used to resolve relative url)
* @return True if the content has been properly parsed.
*/
public native boolean readOpds(String content, String urlHost);
/**
* Load a bookmark file
*
* @param path The path of the file to read.
* @return True if the content has been properly parsed
*/
public native boolean readBookmarkFile(String path);
/**
* Add a book to the library.
*
* @param pathToOpen The path of the zim file to add.
* @param pathToSave The path to store in the library in place of
* pathToOpen.
* @param url The url of the book to store in the library
* (useful for kiiwix-serve catalog)
* @param checkMetaData Tell if we check metadata before adding a book to the
* library.
* @return The id of te book if the book has been added to the library.
* Empty string if not.
*/
public native String addBookFromPath(String pathToOpen,
String pathToSave,
String url,
boolean checkMetaData);
public Manager(Library library) {
allocate(library);
_library = library;
}
private Library _library;
@Override
protected void finalize() { dispose(); }
private native void allocate(Library library);
private native void dispose();
private long nativeHandle;
}

View File

@@ -22,5 +22,5 @@ package org.kiwix.kiwixlib;
public class Pair
{
public String filename;
public int offset;
public long offset;
}

View File

@@ -0,0 +1,26 @@
#!/usr/bin/bash
# This script compile the unit test to test the java wrapper.
# This is not integrated in meson because ... this is not so easy.
KIWIX_LIB_JAR=$1
if [ -z $KIWIX_LIB_JAR ]
then
echo "You must give the path to the kiwixlib.jar as first argument"
exit 1
fi
KIWIX_LIB_DIR=$2
if [ -z $KIWIX_LIB_DIR ]
then
echo "You must give the path to directory containing libkiwix.so as second argument"
exit 1
fi
TEST_SOURCE_DIR=$(dirname $(readlink -f $0))
javac -g -d . -s . -cp $TEST_SOURCE_DIR/junit-4.13.jar:$KIWIX_LIB_JAR $TEST_SOURCE_DIR/test.java
java -Djava.library.path=$KIWIX_LIB_DIR -cp $TEST_SOURCE_DIR/junit-4.13.jar:$TEST_SOURCE_DIR/hamcrest-core-1.3.jar:$KIWIX_LIB_JAR:. org.junit.runner.JUnitCore test

View File

Binary file not shown.

View File

Binary file not shown.

View File

@@ -0,0 +1,53 @@
import java.io.*;
import java.util.*;
import org.junit.Test;
import static org.junit.Assert.*;
import org.kiwix.kiwixlib.*;
public class test {
static {
System.loadLibrary("kiwix");
}
private static String getCatalogContent()
throws IOException
{
BufferedReader reader = new BufferedReader(new FileReader("catalog.xml"));
String line;
StringBuilder sb = new StringBuilder();
while ((line = reader.readLine()) != null)
{
sb.append(line + "\n");
}
reader.close();
return sb.toString();
}
@Test
public void testSome()
throws IOException
{
Library lib = new Library();
Manager manager = new Manager(lib);
String content = getCatalogContent();
manager.readOpds(content, "https://library.kiwix.org");
assertEquals(lib.getBookCount(true, true), 10);
String[] bookIds = lib.getBooksIds();
assertEquals(bookIds.length, 10);
Book book = lib.getBookById(bookIds[0]);
assertEquals(book.getTitle(), "Wikisource");
assertEquals(book.getTags(), "wikisource;_category:wikisource;_pictures:no;_videos:no;_details:yes;_ftindex:yes");
assertEquals(book.getFaviconUrl(), "https://library.kiwix.org/meta?name=favicon&content=wikisource_fr_all_nopic_2020-01");
assertEquals(book.getUrl(), "http://download.kiwix.org/zim/wikisource/wikisource_fr_all_nopic_2020-01.zim.meta4");
}
static
public void main(String[] args) {
Library lib = new Library();
lib.getBookCount(true, true);
}
}

View File

@@ -26,9 +26,55 @@
#include <pthread.h>
#include <string>
#include <vector>
#include <iostream>
#if __ANDROID__
#include <android/log.h>
#define LOG(...) __android_log_print(ANDROID_LOG_ERROR, "kiwix", __VA_ARGS__)
#else
#define LOG(...)
#endif
extern pthread_mutex_t globalLock;
template<typename T>
void setPtr(JNIEnv* env, jobject thisObj, T* ptr)
{
jclass thisClass = env->GetObjectClass(thisObj);
jfieldID fieldId = env->GetFieldID(thisClass, "nativeHandle", "J");
env->SetLongField(thisObj, fieldId, reinterpret_cast<jlong>(ptr));
}
template<typename T, typename ...Args>
void allocate(JNIEnv* env, jobject thisObj, Args && ...args)
{
T* ptr = new T(std::forward<Args>(args)...);
setPtr(env, thisObj, ptr);
}
template<typename T>
T* getPtr(JNIEnv* env, jobject thisObj)
{
jclass thisClass = env->GetObjectClass(thisObj);
jfieldID fidNumber = env->GetFieldID(thisClass, "nativeHandle", "J");
return reinterpret_cast<T*>(env->GetLongField(thisObj, fidNumber));
}
template<typename T>
void dispose(JNIEnv* env, jobject thisObj)
{
delete getPtr<T>(env, thisObj);
}
#define METHOD0(retType, class, name) \
JNIEXPORT retType JNICALL Java_org_kiwix_kiwixlib_##class##_##name( \
JNIEnv* env, jobject thisObj)
#define METHOD(retType, class, name, ...) \
JNIEXPORT retType JNICALL Java_org_kiwix_kiwixlib_##class##_##name( \
JNIEnv* env, jobject thisObj, __VA_ARGS__)
inline jfieldID getHandleField(JNIEnv* env, jobject obj)
{
jclass c = env->GetObjectClass(obj);
@@ -36,6 +82,12 @@ inline jfieldID getHandleField(JNIEnv* env, jobject obj)
return env->GetFieldID(c, "nativeHandle", "J");
}
inline jobjectArray createArray(JNIEnv* env, size_t length, const std::string& type_sig)
{
jclass c = env->FindClass(type_sig.c_str());
return env->NewObjectArray(length, c, NULL);
}
class Lock
{
protected:
@@ -91,19 +143,67 @@ struct LockedHandle : public Lock {
T* operator->() { return h->h; }
T* operator*() { return h->h; }
operator bool() const { return (h->h != nullptr); }
operator T*() const { return h->h; }
};
/* c2jni type conversion functions */
inline jboolean c2jni(const bool& val) { return val ? JNI_TRUE : JNI_FALSE; }
template<typename T>
struct JType { };
template<> struct JType<bool>{ typedef jboolean type_t; };
template<> struct JType<int>{ typedef jint type_t; };
template<> struct JType<long>{ typedef jlong type_t; };
template<> struct JType<uint64_t> { typedef jlong type_t; };
template<> struct JType<uint32_t> { typedef jlong type_t; };
template<> struct JType<std::string>{ typedef jstring type_t; };
template<> struct JType<std::vector<std::string>>{ typedef jobjectArray type_t; };
template<typename T>
inline typename JType<T>::type_t c2jni(const T& val, JNIEnv* env) {
return static_cast<typename JType<T>::type_t>(val);
}
template<>
inline jboolean c2jni(const bool& val, JNIEnv* env) { return val ? JNI_TRUE : JNI_FALSE; }
template<>
inline jstring c2jni(const std::string& val, JNIEnv* env)
{
return env->NewStringUTF(val.c_str());
}
inline jint c2jni(const int val) { return (jint)val; }
inline jint c2jni(const unsigned val) { return (unsigned)val; }
template<>
inline jobjectArray c2jni(const std::vector<std::string>& val, JNIEnv* env)
{
auto array = createArray(env, val.size(), "java/lang/String");
size_t index = 0;
for (auto& elem: val) {
auto jElem = c2jni(elem, env);
env->SetObjectArrayElement(array, index++, jElem);
}
return array;
}
template<typename T, typename U=T>
struct CType { };
template<> struct CType<jboolean>{ typedef bool type_t; };
template<> struct CType<jint>{ typedef int type_t; };
template<> struct CType<jlong>{ typedef long type_t; };
template<> struct CType<jstring>{ typedef std::string type_t; };
template<typename U>
struct CType<jobjectArray, U>{ typedef std::vector<typename CType<U>::type_t> type_t; };
/* jni2c type conversion functions */
inline bool jni2c(const jboolean& val) { return val == JNI_TRUE; }
template<typename T, typename U=T>
inline typename CType<T>::type_t jni2c(const T& val, JNIEnv* env) {
return static_cast<typename CType<T>::type_t>(val);
}
template<>
inline bool jni2c(const jboolean& val, JNIEnv* env) { return val == JNI_TRUE; }
template<>
inline std::string jni2c(const jstring& val, JNIEnv* env)
{
const char* chars = env->GetStringUTFChars(val, 0);
@@ -112,7 +212,21 @@ inline std::string jni2c(const jstring& val, JNIEnv* env)
return ret;
}
inline int jni2c(const jint val) { return (int)val; }
template<typename U>
inline typename CType<jobjectArray, U>::type_t jni2c(const jobjectArray& val, JNIEnv* env)
{
jsize length = env->GetArrayLength(val);
typename CType<jobjectArray, U>::type_t v(length);
int i;
for (i = 0; i < length; i++) {
U obj = (U) env->GetObjectArrayElement(val, i);
auto cobj = jni2c<U>(obj, env);
v.push_back(cobj);
}
return v;
}
/* Method to deal with variable passed by reference */
inline std::string getStringObjValue(const jobject obj, JNIEnv* env)
{
@@ -141,17 +255,17 @@ inline void setBoolObjValue(const bool value, const jobject obj, JNIEnv* env)
{
jclass objClass = env->GetObjectClass(obj);
jfieldID objFid = env->GetFieldID(objClass, "value", "Z");
env->SetIntField(obj, objFid, c2jni(value));
env->SetIntField(obj, objFid, c2jni(value, env));
}
inline void setPairObjValue(const std::string& filename, const int offset,
inline void setPairObjValue(const std::string& filename, const long offset,
const jobject obj, JNIEnv* env)
{
jclass objClass = env->GetObjectClass(obj);
jfieldID filenameFid = env->GetFieldID(objClass, "filename", "Ljava/lang/String;");
env->SetObjectField(obj, filenameFid, c2jni(filename, env));
jfieldID offsetFid = env->GetFieldID(objClass, "offset", "I");
env->SetIntField(obj, offsetFid, offset);
jfieldID offsetFid = env->GetFieldID(objClass, "offset", "J");
env->SetLongField(obj, offsetFid, offset);
}
#endif // _ANDROID_JNI_UTILS_H

View File

@@ -2,8 +2,9 @@
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
<ShortName>Zim catalog search</ShortName>
<Description>Search zim files in the catalog.</Description>
<Url type="application/atom+xml;profile=opds-catalog"
<Url type="application/atom+xml;profile=opds-catalog"
xmlns:atom="http://www.w3.org/2005/Atom"
xmlns:k="http://kiwix.org/opensearchextension/1.0"
indexOffset="0"
template="/{{root}}/catalog/search?q={searchTerms}&lang={language}&count={count}&start={startIndex}"/>
template="/{{root}}/catalog/search?q={searchTerms?}&lang={language?}&name={k:name?}&tag={k:tag?}&notag={k:notag?}&maxsize={k:maxsize?}&count={count?}&start={startIndex?}"/>
</OpenSearchDescription>

33
test/book.cpp Normal file
View File

@@ -0,0 +1,33 @@
#include "gtest/gtest.h"
#include "../include/book.h"
TEST(BookTest, updateTest)
{
kiwix::Book book;
book.setReadOnly(false);
book.setPath("/home/user/Downloads/skin-of-color-society_en_all_2019-11.zim");
book.setPathValid(true);
book.setUrl("book-url");
book.setTags("youtube;_videos:yes;_ftindex:yes;_ftindex:yes;_pictures:yes;_details:yes");
book.setName("skin-of-color-society_en_all");
book.setFavicon("book-favicon");
book.setFaviconMimeType("book-favicon-mimetype");
kiwix::Book newBook;
newBook.setReadOnly(true);
EXPECT_FALSE(newBook.update(book));
newBook.setReadOnly(false);
EXPECT_TRUE(newBook.update(book));
EXPECT_EQ(newBook.readOnly(), book.readOnly());
EXPECT_EQ(newBook.getPath(), book.getPath());
EXPECT_EQ(newBook.isPathValid(), book.isPathValid());
EXPECT_EQ(newBook.getUrl(), book.getUrl());
EXPECT_EQ(newBook.getTags(), book.getTags());
EXPECT_EQ(newBook.getName(), book.getName());
EXPECT_EQ(newBook.getFavicon(), book.getFavicon());
EXPECT_EQ(newBook.getFaviconMimeType(), book.getFaviconMimeType());
}

12
test/kiwixserve.cpp Normal file
View File

@@ -0,0 +1,12 @@
#include "gtest/gtest.h"
#include "../include/kiwixserve.h"
TEST(KiwixServeTest, PortTest)
{
kiwix::KiwixServe kiwixServe("libraryPath", 8181);
EXPECT_EQ(kiwixServe.getPort(), 8181);
kiwixServe.setPort(8484);
EXPECT_EQ(kiwixServe.getPort(), 8484);
EXPECT_EQ(kiwixServe.setPort(0), -1);
EXPECT_EQ(kiwixServe.setPort(3456789), -1);
}

View File

@@ -180,6 +180,7 @@ const char * sampleOpdsStream = R"(
#include "../include/library.h"
#include "../include/manager.h"
#include "../include/bookmark.h"
namespace
{
@@ -191,9 +192,34 @@ class LibraryTest : public ::testing::Test {
manager.readOpds(sampleOpdsStream, "foo.urlHost");
}
kiwix::Bookmark createBookmark(const std::string &id) {
kiwix::Bookmark bookmark;
bookmark.setBookId(id);
return bookmark;
};
kiwix::Library lib;
};
TEST_F(LibraryTest, getBookMarksTest)
{
auto bookId1 = lib.getBooksIds()[0];
auto bookId2 = lib.getBooksIds()[1];
lib.addBookmark(createBookmark(bookId1));
lib.addBookmark(createBookmark("invalid-bookmark-id"));
lib.addBookmark(createBookmark(bookId2));
auto onlyValidBookmarks = lib.getBookmarks();
auto allBookmarks = lib.getBookmarks(false);
EXPECT_EQ(onlyValidBookmarks[0].getBookId(), bookId1);
EXPECT_EQ(onlyValidBookmarks[1].getBookId(), bookId2);
EXPECT_EQ(allBookmarks[0].getBookId(), bookId1);
EXPECT_EQ(allBookmarks[1].getBookId(), "invalid-bookmark-id");
EXPECT_EQ(allBookmarks[2].getBookId(), bookId2);
}
TEST_F(LibraryTest, sanityCheck)
{
EXPECT_EQ(lib.getBookCount(true, true), 10U);
@@ -226,16 +252,18 @@ TEST_F(LibraryTest, filterCheck)
EXPECT_EQ(bookIds.size(), 1U);
bookIds = lib.filter(kiwix::Filter().query("Wiki"));
EXPECT_EQ(bookIds.size(), 3U);
EXPECT_EQ(bookIds.size(), 4U);
bookIds = lib.filter(kiwix::Filter().query("Wiki").creator("Wiki"));
EXPECT_EQ(bookIds.size(), 1U);
}
};
int main(int argc, char** argv)
TEST_F(LibraryTest, getBookByPath)
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
auto& book = lib.getBookById(lib.getBooksIds()[0]);
book.setPath("/some/abs/path.zim");
EXPECT_EQ(lib.getBookByPath("/some/abs/path.zim").getId(), book.getId());
EXPECT_THROW(lib.getBookByPath("non/existant/path.zim"), std::out_of_range);
}
};

View File

@@ -6,16 +6,18 @@ tests = [
'regex',
'tagParsing',
'stringTools',
'pathTools'
'pathTools',
'kiwixserve',
'book'
]
gtest_dep = dependency('gtest',
main:true,
fallback: ['gtest', 'gtest_dep'],
fallback: ['gtest', 'gtest_main_dep'],
required:false)
if gtest_dep.found()
if gtest_dep.found() and not meson.is_cross_build()
foreach test_name : tests
test_exe = executable(test_name, [test_name+'.cpp'],
link_with : kiwixlib,

View File

@@ -65,9 +65,3 @@ TEST(ParseUrlTest, valid)
ASSERT_EQ(title, "/title");
}
};
int main(int argc, char** argv)
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}

View File

@@ -227,8 +227,3 @@ TEST(pathTools, WideToUtf8)
};
int main(int argc, char** argv)
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}

View File

@@ -101,9 +101,3 @@ TEST(append, middle)
}
};
int main(int argc, char** argv)
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}

View File

@@ -54,8 +54,3 @@ TEST(stringTools, split)
}
};
int main(int argc, char** argv)
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}

View File

@@ -95,8 +95,3 @@ TEST(ParseTagTest, invalid)
}
};
int main(int argc, char** argv)
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}

View File

@@ -1,57 +0,0 @@
#!/usr/bin/env bash
set -e
BUILD_DIR=${HOME}/BUILD_${PLATFORM}
INSTALL_DIR=${BUILD_DIR}/INSTALL
TEST=0
case ${PLATFORM} in
"native_static")
MESON_OPTION="--default-library=static"
TEST=1
;;
"native_dyn")
MESON_OPTION="--default-library=shared"
TEST=1
;;
"win32_static")
MESON_OPTION="--default-library=static --cross-file ${BUILD_DIR}/meson_cross_file.txt"
;;
"win32_dyn")
MESON_OPTION="--default-library=shared --cross-file ${BUILD_DIR}/meson_cross_file.txt"
;;
"android_arm")
MESON_OPTION="-Dandroid=true --default-library=shared --cross-file ${BUILD_DIR}/meson_cross_file.txt"
;;
"android_arm64")
MESON_OPTION="-Dandroid=true --default-library=shared --cross-file ${BUILD_DIR}/meson_cross_file.txt"
;;
esac
cd ${TRAVIS_BUILD_DIR}
if [[ "${TRAVIS_OS_NAME}" == "osx" ]]
then
export PKG_CONFIG_PATH=${INSTALL_DIR}/lib/pkgconfig
else
export PKG_CONFIG_PATH=${INSTALL_DIR}/lib/x86_64-linux-gnu/pkgconfig
fi
export CPPFLAGS="-I${INSTALL_DIR}/include"
if [[ "$TEST" == "1" ]]
then
MESON_OPTION="${MESON_OPTION} -Db_coverage=true"
fi
meson . build ${MESON_OPTION}
cd build
ninja
if [[ "$TEST" == "1" ]]
then
echo "Running test"
export LD_LIBRARY_PATH=${INSTALL_DIR}/lib:${INSTALL_DIR}/lib64:${INSTALL_DIR}/lib/x86_64-linux-gnu
meson test --verbose
ninja coverage
cd ..
echo "*** Publish code coverage"
bash <(curl -s https://codecov.io/bash)
fi

View File

@@ -1,35 +0,0 @@
#!/usr/bin/env bash
set -e
REPO_NAME=${TRAVIS_REPO_SLUG#*/}
# Ninja
cd $HOME
if [[ "$TRAVIS_OS_NAME" == "osx" ]]
then
pip3 install meson==0.49.2
wget https://github.com/ninja-build/ninja/releases/download/v1.8.2/ninja-mac.zip
unzip ninja-mac.zip ninja
ARCHIVE_NAME=deps_osx_${PLATFORM}_${REPO_NAME}.tar.xz
else
wget https://bootstrap.pypa.io/get-pip.py
python3.5 get-pip.py --user
python3.5 -m pip install --user --upgrade pip
python3.5 -m pip install --user meson==0.49.2
wget https://github.com/ninja-build/ninja/releases/download/v1.8.2/ninja-linux.zip
unzip ninja-linux.zip ninja
ARCHIVE_NAME=deps_linux_xenial_${PLATFORM}_${REPO_NAME}.tar.xz
fi
mkdir -p $HOME/bin
cp ninja $HOME/bin
# Dependencies comming from kiwix-build.
cd ${HOME}
wget http://tmp.kiwix.org/ci/${ARCHIVE_NAME}
tar xf ${HOME}/${ARCHIVE_NAME}
sudo ln -s travis ../ci_builder