Compare commits

..

82 Commits

Author SHA1 Message Date
Matthieu Gautier
e2e42ac65c Patch httplib.h for gcc maybe-uninitialized warning.
Recent version of gcc warn about the use `buf` where it may be
uninitialized.
But here, buf is used as a output buffer to initialize it.

I'm not sure we can fix this differently than ignoring the warning.
2021-06-30 17:30:30 +02:00
Kelson
4124ad30d5 Merge pull request #561 from kiwix/issue/557
Keep Kiwix Serve filter values over time
2021-06-25 08:05:21 +02:00
Manan Jethwani
3c5d73027d created separate variable for time delta 2021-06-24 15:41:28 +05:30
Manan Jethwani
d88bdd3ebf corrected filter on no results 2021-06-23 14:15:22 +05:30
Manan Jethwani
5cfe34a5c2 corrected filter working 2021-06-22 19:36:22 +05:30
Manan Jethwani
ad133bc9a3 added cookies for filter effect 2021-06-21 19:57:52 +05:30
Kelson
8eabae6286 Merge pull request #560 from kiwix/fix_compilation_illustration 2021-06-19 13:15:44 +02:00
Maneesh P M
f3c96b23fd Use getIllustrationItem instead of getFaviconEntry method
With openzim/libzim#540 we now have a new function to get
illustration(previously favicon in 48x48 size and unity scale) in
multiple sizes. We need to replace getFaviconEntry with this new
getIllustrationItem method.
2021-06-19 10:23:24 +05:30
Kelson
a92e9d8756 Merge pull request #490 from soumyankar/404ContentHomeButton
404 content home button
2021-06-17 09:59:45 +02:00
Vertigo
8d39b2c4c1 Added content ZIM home button on 404 2021-06-17 12:51:27 +05:30
Kelson
6d237ff1d5 Merge pull request #492 from kiwix/opds_categories_feed 2021-06-08 20:01:37 +02:00
Veloman Yunkan
78083f1f4a Moved OPDS templates under static/templates 2021-06-08 20:37:00 +04:00
Veloman Yunkan
dd60235010 Fixed the self link in the output of /catalog/v2/entries 2021-06-08 20:37:00 +04:00
Veloman Yunkan
e799f2ff1e OPDSDumper::dumpOPDSFeed() works via mustache
This changes the output of `/catalog/search` as follows:

- Entire search query (rather than only the value of the `q` parameter)
  is put in the <title> node.

- Search performed with an empty query presents itself as "All zims".

- The feed id remains stable for identical searches on the same
  library.
2021-06-08 20:37:00 +04:00
Veloman Yunkan
312f2cb560 Moved handle_catalog_v2*() methods into a new file 2021-06-08 20:37:00 +04:00
Veloman Yunkan
fa42cbc48f Pagination info in /catalog/v2/entries 2021-06-08 20:37:00 +04:00
Veloman Yunkan
f1797993af Reused InternalServer::search_catalog() 2021-06-08 20:37:00 +04:00
Veloman Yunkan
f886c8c07b Root url is normalized once in the constructor 2021-06-08 20:37:00 +04:00
Veloman Yunkan
9ca6bd006f /catalog/v2/categories goes through OPDSDumper too 2021-06-08 20:37:00 +04:00
Veloman Yunkan
cdacc0caf1 /catalog/v2/entries going through OPDSDumper
OPDSDumper sensed threats to its job security, so it lobbied to be
involved in handling the /catalog/v2 endpoints, too.
2021-06-08 20:37:00 +04:00
Veloman Yunkan
dfad1c3815 /catalog/v2/searchdescription.xml 2021-06-08 20:37:00 +04:00
Veloman Yunkan
07252a127a /catalog/v2/entries is also a search endpoint 2021-06-08 20:37:00 +04:00
Veloman Yunkan
b60e3ffb26 RequestContext::get_optional_param() 2021-06-08 20:37:00 +04:00
Veloman Yunkan
70d42aec98 A small simplification 2021-06-08 20:37:00 +04:00
Veloman Yunkan
4aa3c792aa Extracted get_search_filter() 2021-06-08 20:37:00 +04:00
Veloman Yunkan
208dece7e3 Reordered several statements
Reordered several statements so that the next couple of commits are a
little simpler.
2021-06-08 20:37:00 +04:00
Veloman Yunkan
19b59fd72f Serving /catalog/v2/entries
/catalog/v2/entries is intended to play the combined role of
/catalog/root.xml and /catalog/search of the old OPDS API. Currently,
the latter role is not yet implemented.

Implementation note: instead of tweaking and reusing
`OPDSDumper::dumpOPDSFeed()`, the generation of the OPDS feed is done via `mustache`
and a new template `static/catalog_v2_entries.xml`.
2021-06-08 20:37:00 +04:00
Veloman Yunkan
92c2de8d46 Enter InternalServer::m_library_id
The new field is intended to serve as a seed for generating semi-stable
OPDS feed ids that only need to change when the library is updated.
2021-06-08 20:37:00 +04:00
Veloman Yunkan
feeb9f206e /catalog/v2/* XMLs are OPDS 1.2 2021-06-08 20:37:00 +04:00
Veloman Yunkan
a1520ce7f1 Fixing the xenial build
Under Ubuntu 16.04/xenial, ccache seems to have issues with multiline
raw string literals used inside macros.
2021-06-08 20:37:00 +04:00
Veloman Yunkan
2e53b51696 Serving /catalog/v2/categories 2021-06-08 20:37:00 +04:00
Veloman Yunkan
b259afa408 Library::getBooksCategories()
Note: no unit test added
2021-06-08 20:37:00 +04:00
Veloman Yunkan
3c3cf08a1a Serving /catalog/v2/root.xml
Note: This commit somewhat relaxes validation of non variable
`<updated>` elements in the OPDS feed - the contents of any `<updated>`
element is replaced with the YYYY-MM-DDThh:mm:ssZ placeholder.
2021-06-08 16:03:43 +04:00
Veloman Yunkan
54b78eaf56 Moved gen_date_str() to tools/otherTools.cpp 2021-06-08 16:03:43 +04:00
Veloman Yunkan
1e0ff1fbb0 Fixed the double colon in OPDS date string 2021-06-08 16:03:43 +04:00
Veloman Yunkan
5b272ac49c Fixed handling of /catalogBLABLA/root.xml & alike
Also removed an unneeded namespace qualifier.
2021-06-08 16:03:43 +04:00
Veloman Yunkan
0a3d293ae0 Broke Server.404 with /catalogBLABLABLA/root.xml
The new negative test-point demonstrates that Kiwix server doesn't
distinguish /catalogBLABLABLA from /catalog.
2021-06-08 16:03:43 +04:00
Kelson
86ef2e2199 Merge pull request #550 from kiwix/remove-bintray
Remove Bintray badge
2021-06-07 16:01:37 +02:00
Emmanuel Engelhart
a0332e7599 Remove Bintray badge 2021-06-07 15:55:19 +02:00
Kelson
2ef488816c Merge pull request #534 from kiwix/filter_library
Add filters to kiwix-serve welcome page
2021-06-07 15:46:37 +02:00
Manan Jethwani
1ccafe2d97 minor changes in fadeout effect 2021-06-07 15:38:31 +02:00
Manan Jethwani
d6c62b3cd3 corrected spinner and fadeout effect 2021-06-07 15:37:20 +02:00
Manan Jethwani
f39c558d2a added fade out 2021-06-07 15:37:20 +02:00
Manan Jethwani
5b46ad5934 added spinned 2021-06-07 15:37:20 +02:00
Manan Jethwani
49dbd0aa52 fixed reset filters link 2021-06-07 15:37:20 +02:00
Manan Jethwani
179f0faeb1 added minor features 2021-06-07 15:37:20 +02:00
Manan Jethwani
bb92f26b60 added filter functionality 2021-06-07 15:37:20 +02:00
Matthieu Gautier
3a4e8303a0 Merge pull request #541 from kiwix/adding_dynamic_and_subset_loading
Dynamic and subset loading of catalogue in kiwix-serve
2021-06-07 15:35:13 +02:00
Manan Jethwani
063bb8cd65 added dynamic and subset loading of zim-files in kiwix-serve 2021-06-01 19:33:42 +05:30
Kelson
b54e5ab969 Merge pull request #543 from kiwix/add-libmicrohttpd-compilation-hint
Add libmicrohttpd compilation hint
2021-05-30 15:51:40 +02:00
Emmanuel Engelhart
2632a21d24 Move Repology to wiki 2021-05-30 15:46:45 +02:00
Emmanuel Engelhart
5c97b1fff9 gtest is need for testing 2021-05-30 15:43:21 +02:00
Emmanuel Engelhart
4f7175ad59 Libkiwix, not Kiwix library 2021-05-30 15:42:28 +02:00
Emmanuel Engelhart
f4b8d0c303 Add libmicrohttpd compilation hint 2021-05-30 15:35:40 +02:00
Matthieu Gautier
188694f2a1 Merge pull request #510 from kiwix/add_function_zimId 2021-05-26 15:15:13 +02:00
Maneesh P M
e2f6d91d51 Remove get_readerIndex in favor of get_zimId
The function get_readerIndex was used to get the zimId using an ordered
vector of readers. Now we can use get_zimId directly.
2021-05-26 14:45:25 +02:00
Maneesh P M
c35f6f9142 Add get_zimId method to Result
get_zimId method allows the user to get the uuid of the archive from
which a result is retrieved directly from the search result itself.
2021-05-26 14:45:25 +02:00
Matthieu Gautier
7f0d3004c9 Merge pull request #505 from kiwix/suggestion_snippets 2021-05-26 11:04:52 +02:00
Maneesh P M
5567d8ca49 Replace std::vector<std::string> with SuggestionItem
Each sugestions used to be stored as vector of strings to hold various values
such as title, path etc inside them. With this commit, we use the new
dedicated class `SuggestionItem` to do the same.
2021-05-26 10:53:39 +02:00
Maneesh P M
5315034afe Introduce SuggestionItem class
This is a helper class that allows to create and manage individual
suggestion item and their data.
2021-05-26 10:53:00 +02:00
Maneesh P M
3288cd80e5 Render suggestion snippet properly
To render the snippets properly, we need to use the _renderItem property
of the autocomple ui.
2021-05-26 10:53:00 +02:00
Maneesh P M
56434de79e Set label to title snippet if present
With openzim/libzim#545 we now support snippet generation of titles
which can be used as the display label on the ui for highlighted titles
via the "label" field.
The old version used plain title which is still available in the value
field.
2021-05-26 10:52:58 +02:00
Matthieu Gautier
e9ba151e6f Merge pull request #539 from kiwix/fix_windows_build
Avoid windows header to define min/max macros.
2021-05-26 09:52:43 +02:00
Matthieu Gautier
5f83944699 Avoid windows header to define min/max macros.
PR #507 use std::min.
But on windows, the header define min and max macros and so the
compilation is broken.
Add `-DNOMINMAX` define to avoid that.
2021-05-26 09:20:17 +02:00
Kelson
9c0ae835e2 Merge pull request #537 from kiwix/search_iterator_api_rename
Update libkiwix with search iterator rename in libzim
2021-05-26 08:49:26 +02:00
Maneesh P M
e5fac30cee Update libkiwix with search iterator rename in libzim
Search iterator API in libzim has been shifted to use camel case naming.
This has to be accomodated in libkiwix as well.
2021-05-26 08:39:13 +02:00
Matthieu Gautier
7ef08b670b Merge pull request #538 from kiwix/revert-530-adding_dynamic_and_subset_loading
Revert "Kiwix Serve welcome page dynamic and subset loading (OPDS based)"
2021-05-25 17:33:23 +02:00
Matthieu Gautier
2736a46cfe Revert "Kiwix Serve welcome page dynamic and subset loading (OPDS based)" 2021-05-25 17:30:05 +02:00
Kelson
672b4fc907 Merge pull request #530 from kiwix/adding_dynamic_and_subset_loading
Kiwix Serve welcome page dynamic and subset loading (OPDS based)
2021-05-25 16:22:54 +02:00
Manan Jethwani
012973d14a added dynamic and subset loading of zim-files in kiwix-serve 2021-05-25 02:41:12 +05:30
Kelson
67984cca5b Merge pull request #535 from kiwix/further-kiwix-lib-renaming
Rename kiwix-lib in libkiwix
2021-05-23 21:57:03 +02:00
Emmanuel Engelhart
d4e35c7067 Rename kiwix-lib in libkiwix 2021-05-23 21:46:52 +02:00
Matthieu Gautier
6e37cabaea Merge pull request #529 from kiwix/libkiwix-github-url-fix
Fix Libkiwix Github repository URLS
2021-05-21 16:09:15 +02:00
Emmanuel Engelhart
c8b7f8772a Fix Libkiwix Github repository URLS 2021-05-20 08:56:44 +02:00
Veloman Yunkan
fc7484ac86 Merge pull request #528 from kiwix/fix_suggestion_search_result_page
Check if bookName is available in url parameters
2021-05-19 00:36:15 +04:00
Maneesh P M
c236f3a32b Check if bookName is available in url parameters
In certain pages like the search result page, bookName is not of the
form `/bookName/endpoint?parameters`. Rather it is available as a query
parameter. From these pages bookName should be assigned from parameters.
2021-05-19 01:12:29 +05:30
Matthieu Gautier
3c7faddb6e Merge pull request #526 from kiwix/lizim_search_api_change
Fixed the libkiwix build broken by the changed libzim search API
2021-05-17 15:06:15 +02:00
Veloman Yunkan
cd02b4de3b Dummy application of new libzim search API
Didn't take any advantage of the new libzim search API. Just fixed the
libkiwix build in the most straightforward way.
2021-05-15 23:34:51 +04:00
Kelson
5188355878 Merge pull request #525 from kiwix/html-only-root-link
Insert root link only if html content
2021-05-14 16:51:33 +02:00
Emmanuel Engelhart
05cc3d015f Insert root link only if html content 2021-05-14 14:49:28 +02:00
Kelson
39b62c6108 Merge pull request #522 from kiwix/libkiwix_ci_fix
Fixed CI after repository rename
2021-05-13 11:09:57 +02:00
Veloman Yunkan
9c43353b72 Fixed CI after repository rename 2021-05-13 12:49:44 +04:00
47 changed files with 1715 additions and 527 deletions

View File

@@ -134,7 +134,7 @@ jobs:
if [[ "${{matrix.target}}" =~ android_.* ]]; then
MESON_OPTION="$MESON_OPTION -Dandroid=true"
fi
cd $HOME/kiwix-lib
cd $HOME/libkiwix
meson . build ${MESON_OPTION}
cd build
ninja
@@ -145,7 +145,7 @@ jobs:
if: startsWith(matrix.target, 'native_')
shell: bash
run: |
cd $HOME/kiwix-lib/build
cd $HOME/libkiwix/build
meson test --verbose
ninja coverage
env:
@@ -154,7 +154,7 @@ jobs:
- name: Publish coverage
shell: bash
run: |
cd $HOME/kiwix-lib
cd $HOME/libkiwix
curl https://codecov.io/bash -o codecov.sh
bash codecov.sh -n "${OS_NAME}_${{matrix.target}}" -Z
rm codecov.sh

1
.gitignore vendored
View File

@@ -4,3 +4,4 @@ subprojects/googletest-release*
*.class
build/
.vscode/
builddir/

View File

@@ -1,3 +1,8 @@
libkiwix 10.0.0
===============
* ...
kiwix-lib 9.4.1
===============

View File

@@ -1,50 +1,50 @@
Kiwix library
=============
Libkiwix
========
The Kiwix library provides the [Kiwix](https://kiwix.org) software
suite core. It contains the code shared by all Kiwix ports (Windows,
The Libkiwix provides the [Kiwix](https://kiwix.org) software suite
core. It contains the code shared by all Kiwix ports (Windows,
GNU/Linux, macOS, Android, iOS, ...).
[![Download](https://api.bintray.com/packages/kiwix/kiwix/kiwixlib/images/download.svg)](https://bintray.com/kiwix/kiwix/kiwixlib/_latestVersion)
[![Build Status](https://github.com/kiwix/kiwix-lib/workflows/CI/badge.svg?query=branch%3Amaster)](https://github.com/kiwix/kiwix-lib/actions?query=branch%3Amaster)
[![CodeFactor](https://www.codefactor.io/repository/github/kiwix/kiwix-lib/badge)](https://www.codefactor.io/repository/github/kiwix/kiwix-lib)
[![Codecov](https://codecov.io/gh/kiwix/kiwix-lib/branch/master/graph/badge.svg)](https://codecov.io/gh/kiwix/kiwix-lib)
[![Repositories](https://img.shields.io/repology/repositories/libkiwix?label=repositories)](https://github.com/kiwix/libkiwix/wiki/Repology)
[![Build Status](https://github.com/kiwix/libkiwix/workflows/CI/badge.svg?query=branch%3Amaster)](https://github.com/kiwix/libkiwix/actions?query=branch%3Amaster)
[![CodeFactor](https://www.codefactor.io/repository/github/kiwix/libkiwix/badge)](https://www.codefactor.io/repository/github/kiwix/libkiwix)
[![Codecov](https://codecov.io/gh/kiwix/libkiwix/branch/master/graph/badge.svg)](https://codecov.io/gh/kiwix/libkiwix)
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
[![Packaging status](https://repology.org/badge/vertical-allrepos/kiwix-lib.svg)](https://repology.org/project/kiwix-lib/versions)
Disclaimer
----------
This document assumes you have a little knowledge about software
compilation. If you experience difficulties with the dependencies or
with the Kiwix libary compilation itself, we recommend to have a look
to [kiwix-build](https://github.com/kiwix/kiwix-build).
with the Libkiwix compilation itself, we recommend to have a look to
[kiwix-build](https://github.com/kiwix/kiwix-build).
Preamble
--------
Although the Kiwix library can be (cross-)compiled on/for many
sytems, the following documentation explains how to do it on POSIX
ones. It is primarly thought for GNU/Linux systems and has been tested
on recent releases of Ubuntu and Fedora.
Although the Libkiwix can be (cross-)compiled on/for many sytems, the
following documentation explains how to do it on POSIX ones. It is
primarly thought for GNU/Linux systems and has been tested on recent
releases of Ubuntu and Fedora.
Dependencies
------------
The Kiwix library relies on many third parts software libraries. They
are prerequisites to the Kiwix library compilation. Following
libraries need to be available:
The Libkiwix relies on many third party software libraries. They are
prerequisites to the Libkiwix compilation. Following libraries need to
be available:
* [ICU](https://site.icu-project.org/) (package `libicu-dev` on Ubuntu)
* [ZIM](https://openzim.org/) (package `libzim-dev` on Ubuntu)
* [Pugixml](https://pugixml.org/) (package `libpugixml-dev` on Ubuntu)
* [Mustache](https://github.com/kainjow/Mustache) (Just copy the
header `mustache.hpp` somewhere it can be found by the compiler and/or
set CPPFLAGS with correct `-I` option). Use Mustache version 4.1 or above.
* [libcurl](https://curl.se/libcurl) (`libcurl4-gnutls-dev`, `libcurl4-nss-dev` or `libcurl4-openssl-dev` on Ubuntu)
* [microhttpd](https://www.gnu.org/software/libmicrohttpd) (package `libmicrohttpd-dev` on Ubuntu)
* [zlib](https://zlib.net/) (package `zlib1g-dev` on Ubuntu)
* [Libcurl](https://curl.se/libcurl) (`libcurl4-gnutls-dev`, `libcurl4-nss-dev` or `libcurl4-openssl-dev` on Ubuntu)
* [Microhttpd](https://www.gnu.org/software/libmicrohttpd) (package `libmicrohttpd-dev` on Ubuntu)
* [Zlib](https://zlib.net/) (package `zlib1g-dev` on Ubuntu)
To test the code:
* [Google Test](https://github.com/google/googletest) (package `googletest` on Ubuntu)
The following dependency needs to be available at runtime:
* [Aria2](https://aria2.github.io/) (package `aria2` on Ubuntu)
@@ -56,12 +56,12 @@ In the worse case, you will have to download and compile bleeding edge
version by hand.
If you want to install these dependencies locally, then use the
`kiwix-lib` directory as install prefix.
`libkiwix` directory as install prefix.
Environment
-------------
The Kiwix library builds using [Meson](https://mesonbuild.com/) version
The Libkiwix builds using [Meson](https://mesonbuild.com/) version
0.45 or higher. Meson relies itself on Ninja, pkg-config and few other
compilation tools.
@@ -77,7 +77,7 @@ section.
Compilation
-----------
Once all dependencies are installed, you can compile the Kiwix library
Once all dependencies are installed, you can compile the Libkiwix
with:
```bash
meson . build
@@ -85,7 +85,7 @@ ninja -C build
```
By default, it will compile dynamic linked libraries. All binary files
will be created in the "build" directory created automatically by
will be created in the `build` directory created automatically by
Meson. If you want statically linked libraries, you can add
`--default-library=static` option to the Meson command.
@@ -103,7 +103,7 @@ meson test
Installation
------------
If you want to install the Kiwix library and the headers you just have
If you want to install the Libkiwix and the headers you just have
compiled on your system, here we go:
```bash
ninja -C build install
@@ -146,6 +146,10 @@ cp ninja ../bin
cd ..
```
If you compile manually Libmicrohttpd, you might need to compile it
without GNU TLS, a bug here will empeach further compilation
otherwise.
If the compilation still fails, you might need to get a more recent
version of a dependency than the one packaged by your Linux
distribution. Try then with a source tarball distributed by the

View File

@@ -29,7 +29,7 @@ task writePom {
version '10.0.0' + (System.env.KIWIXLIB_BUILDVERSION == null ? '' : '-'+System.env.KIWIXLIB_BUILDVERSION)
packaging 'aar'
name 'kiwixlib'
url 'https://github.com/kiwix/kiwix-lib'
url 'https://github.com/kiwix/libkiwix'
licenses {
license {
name 'GPLv3'
@@ -44,9 +44,9 @@ task writePom {
}
}
scm {
connection 'https://github.com/kiwix/kiwix-lib.git'
developerConnection 'https://github.com/kiwix/kiwix-lib.git'
url 'https://github.com/kiwix/kiwix-lib'
connection 'https://github.com/kiwix/libkiwix.git'
developerConnection 'https://github.com/kiwix/libkiwix.git'
url 'https://github.com/kiwix/libkiwix'
}
}
}.withXml {

2
debian/control vendored
View File

@@ -15,7 +15,7 @@ Build-Depends: debhelper-compat (= 13),
zlib1g-dev
Standards-Version: 4.5.0
Section: libs
Homepage: https://github.com/kiwix/kiwix-lib
Homepage: https://github.com/kiwix/libkiwix
Rules-Requires-Root: no
Package: libkiwix-dev

View File

@@ -103,7 +103,7 @@ class Entry
* Some entry (ie binary ones) have their content plain stored
* in the zim file. Knowing the offset where the content is stored
* an user can directly read the content in the zim file bypassing the
* kiwix-lib/libzim.
* libkiwix/libzim.
*
* @return A pair specifying where to read the content.
* The string is the real file to read (may be different that .zim

View File

@@ -233,12 +233,19 @@ class Library
unsigned int getBookCount(const bool localBooks, const bool remoteBooks) const;
/**
* Get all langagues of the books in the library.
* Get all languagues of the books in the library.
*
* @return A list of languages.
*/
std::vector<std::string> getBooksLanguages() const;
/**
* Get all categories of the books in the library.
*
* @return A list of categories.
*/
std::vector<std::string> getBooksCategories() const;
/**
* Get all book creators of the books in the library.
*

View File

@@ -51,24 +51,35 @@ class OPDSDumper
/**
* Dump the OPDS feed.
*
* @param id The id of the library.
* @param bookIds the ids of the books to include in the feed
* @param query the query used to obtain the list of book ids
* @return The OPDS feed.
*/
std::string dumpOPDSFeed(const std::vector<std::string>& bookIds);
std::string dumpOPDSFeed(const std::vector<std::string>& bookIds, const std::string& query) const;
/**
* Set the id of the opds stream.
* Dump the OPDS feed.
*
* @param bookIds the ids of the books to include in the feed
* @param query the query used to obtain the list of book ids
* @return The OPDS feed.
*/
std::string dumpOPDSFeedV2(const std::vector<std::string>& bookIds, const std::string& query) const;
/**
* Dump the categories OPDS feed.
*
* @param categories list of category names
* @return The OPDS feed.
*/
std::string categoriesOPDSFeed(const std::vector<std::string>& categories) const;
/**
* Set the id of the library.
*
* @param id the id to use.
*/
void setId(const std::string& id) { this->id = id;}
/**
* Set the title oft the opds stream.
*
* @param title the title to use.
*/
void setTitle(const std::string& title) { this->title = title; }
void setLibraryId(const std::string& id) { this->libraryId = id;}
/**
* Set the root location used when generating url.
@@ -77,13 +88,6 @@ class OPDSDumper
*/
void setRootLocation(const std::string& rootLocation) { this->rootLocation = rootLocation; }
/**
* Set the search url.
*
* @param searchUrl the search url to use.
*/
void setSearchDescriptionUrl(const std::string& searchDescriptionUrl) { this->searchDescriptionUrl = searchDescriptionUrl; }
/**
* Set some informations about the search results.
*
@@ -93,27 +97,13 @@ class OPDSDumper
*/
void setOpenSearchInfo(int totalResult, int startIndex, int count);
/**
* Set the library to dump.
*
* @param library The library to dump.
*/
void setLibrary(Library* library) { this->library = library; }
protected:
kiwix::Library* library;
std::string id;
std::string title;
std::string date;
std::string libraryId;
std::string rootLocation;
std::string searchDescriptionUrl;
int m_totalResults;
int m_startIndex;
int m_count;
bool m_isSearchResult = false;
private:
pugi::xml_node handleBook(Book book, pugi::xml_node root_node);
};
}

View File

@@ -37,12 +37,47 @@ using namespace std;
namespace kiwix
{
/**
* The SuggestionItem is a helper class that contains the info about a single
* suggestion item.
*/
class SuggestionItem
{
// Functions
private:
// Create a sugggestion item.
explicit SuggestionItem(std::string title, std::string normalizedTitle,
std::string path, std::string snippet = "") :
title(title),
normalizedTitle(normalizedTitle),
path(path),
snippet(snippet) {}
public:
const std::string getTitle() {return title;}
const std::string getNormalizedTitle() {return normalizedTitle;}
const std::string getPath() {return path;}
const std::string getSnippet() {return snippet;}
const bool hasSnippet() {return !snippet.empty();}
// Data
private:
std::string title;
std::string normalizedTitle;
std::string path;
std::string snippet;
friend class Reader;
};
/**
* The Reader class is the class who allow to get an entry content from a zim
* file.
*/
using SuggestionsList_t = std::vector<std::vector<std::string>>;
using SuggestionsList_t = std::vector<SuggestionItem>;
class Reader
{
public:

View File

@@ -27,7 +27,7 @@
#include <cctype>
#include <locale>
#include <string>
#include <vector>
#include <memory>
#include <vector>
#include "tools/pathTools.h"
#include "tools/stringTools.h"
@@ -48,7 +48,7 @@ class Result
virtual std::string get_content() = 0;
virtual int get_wordCount() = 0;
virtual int get_size() = 0;
virtual int get_readerIndex() = 0;
virtual std::string get_zimId() = 0;
};
struct SearcherInternal;
@@ -154,7 +154,7 @@ class Searcher
const bool verbose = false);
std::vector<Reader*> readers;
SearcherInternal* internal;
std::unique_ptr<SearcherInternal> internal;
std::string searchPattern;
unsigned int estimatedResultCount;
unsigned int resultStart;

View File

@@ -24,6 +24,7 @@
#include <vector>
#include <map>
#include <zim/zim.h>
#include <mustache.hpp>
namespace pugi {
class xml_node;
@@ -45,6 +46,11 @@ namespace kiwix
using MimeCounterType = std::map<const std::string, zim::entry_index_type>;
MimeCounterType parseMimetypeCounter(const std::string& counterData);
std::string gen_date_str();
std::string gen_uuid(const std::string& s);
std::string render_template(const std::string& template_str, kainjow::mustache::data data);
}
#endif

View File

@@ -1,4 +1,4 @@
project('kiwix-lib', 'cpp',
project('libkiwix', 'cpp',
version : '10.0.0', # Also change this in android-kiwix-lib-publisher/kiwixLibAndroid/build.gradle
license : 'GPLv3+',
default_options : ['c_std=c11', 'cpp_std=c++11', 'werror=true'])
@@ -18,7 +18,7 @@ if wrapper.contains('java')
add_languages('java')
endif
# See https://github.com/kiwix/kiwix-lib/issues/371
# See https://github.com/kiwix/libkiwix/issues/371
if ['arm', 'mips', 'm68k', 'ppc', 'sh4'].contains(target_machine.cpu_family())
extra_libs += '-latomic'
endif
@@ -56,6 +56,10 @@ if target_machine.system() == 'windows' and static_deps
extra_cflags += '-DCURL_STATICLIB'
endif
if target_machine.system() == 'windows'
add_project_arguments('-DNOMINMAX', language: 'cpp')
endif
all_deps = [thread_dep, libicu_dep, libzim_dep, pugixml_dep, libcurl_dep, microhttpd_dep, zlib_dep, xapian_dep]
inc = include_directories('include', extra_include)

View File

@@ -201,6 +201,21 @@ std::vector<std::string> Library::getBooksLanguages() const
return booksLanguages;
}
std::vector<std::string> Library::getBooksCategories() const
{
std::set<std::string> categories;
for (const auto& pair: m_books) {
const auto& book = pair.second;
const auto& c = book.getCategory();
if ( !c.empty() ) {
categories.insert(c);
}
}
return std::vector<std::string>(categories.begin(), categories.end());
}
std::vector<std::string> Library::getBooksCreators() const
{
std::vector<std::string> booksCreators;

View File

@@ -25,7 +25,8 @@ kiwix_sources = [
'server/etag.cpp',
'server/request_context.cpp',
'server/response.cpp',
'server/internalServer.cpp'
'server/internalServer.cpp',
'server/internalServer_catalog_v2.cpp'
]
kiwix_sources += lib_resources

View File

@@ -21,10 +21,13 @@
#include "book.h"
#include "tools/otherTools.h"
#include <iomanip>
#include "kiwixlib-resources.h"
#include <mustache.hpp>
namespace kiwix
{
/* Constructor */
OPDSDumper::OPDSDumper(Library* library)
: library(library)
@@ -35,121 +38,111 @@ OPDSDumper::~OPDSDumper()
{
}
std::string gen_date_str()
{
auto now = time(0);
auto tm = localtime(&now);
std::stringstream is;
is << std::setw(2) << std::setfill('0')
<< 1900+tm->tm_year << "-"
<< std::setw(2) << std::setfill('0') << tm->tm_mon+1 << "-"
<< std::setw(2) << std::setfill('0') << tm->tm_mday << "T"
<< std::setw(2) << std::setfill('0') << tm->tm_hour << ":"
<< std::setw(2) << std::setfill('0') << tm->tm_min << ":"
<< std::setw(2) << std::setfill('0') << tm->tm_sec << "Z";
return is.str();
}
static std::string gen_date_from_yyyy_mm_dd(const std::string& date)
{
std::stringstream is;
is << date << "T00:00::00Z";
return is.str();
}
void OPDSDumper::setOpenSearchInfo(int totalResults, int startIndex, int count)
{
m_totalResults = totalResults;
m_startIndex = startIndex,
m_count = count;
m_isSearchResult = true;
}
#define ADD_TEXT_ENTRY(node, child, value) (node).append_child((child)).append_child(pugi::node_pcdata).set_value((value).c_str())
pugi::xml_node OPDSDumper::handleBook(Book book, pugi::xml_node root_node) {
auto entry_node = root_node.append_child("entry");
ADD_TEXT_ENTRY(entry_node, "id", "urn:uuid:"+book.getId());
ADD_TEXT_ENTRY(entry_node, "title", book.getTitle());
ADD_TEXT_ENTRY(entry_node, "summary", book.getDescription());
ADD_TEXT_ENTRY(entry_node, "language", book.getLanguage());
ADD_TEXT_ENTRY(entry_node, "updated", gen_date_from_yyyy_mm_dd(book.getDate()));
ADD_TEXT_ENTRY(entry_node, "name", book.getName());
ADD_TEXT_ENTRY(entry_node, "flavour", book.getFlavour());
ADD_TEXT_ENTRY(entry_node, "category", book.getCategory());
ADD_TEXT_ENTRY(entry_node, "tags", book.getTags());
ADD_TEXT_ENTRY(entry_node, "articleCount", to_string(book.getArticleCount()));
ADD_TEXT_ENTRY(entry_node, "mediaCount", to_string(book.getMediaCount()));
ADD_TEXT_ENTRY(entry_node, "icon", rootLocation + "/meta?name=favicon&content=" + book.getHumanReadableIdFromPath());
auto content_node = entry_node.append_child("link");
content_node.append_attribute("type") = "text/html";
content_node.append_attribute("href") = (rootLocation + "/" + book.getHumanReadableIdFromPath()).c_str();
auto author_node = entry_node.append_child("author");
ADD_TEXT_ENTRY(author_node, "name", book.getCreator());
auto publisher_node = entry_node.append_child("publisher");
ADD_TEXT_ENTRY(publisher_node, "name", book.getPublisher());
if (! book.getUrl().empty()) {
auto acquisition_link = entry_node.append_child("link");
acquisition_link.append_attribute("rel") = "http://opds-spec.org/acquisition/open-access";
acquisition_link.append_attribute("type") = "application/x-zim";
acquisition_link.append_attribute("href") = book.getUrl().c_str();
acquisition_link.append_attribute("length") = to_string(book.getSize()).c_str();
}
if (! book.getFaviconMimeType().empty() ) {
auto image_link = entry_node.append_child("link");
image_link.append_attribute("rel") = "http://opds-spec.org/image/thumbnail";
image_link.append_attribute("type") = book.getFaviconMimeType().c_str();
image_link.append_attribute("href") = (rootLocation + "/meta?name=favicon&content=" + book.getHumanReadableIdFromPath()).c_str();
}
return entry_node;
}
string OPDSDumper::dumpOPDSFeed(const std::vector<std::string>& bookIds)
namespace
{
date = gen_date_str();
pugi::xml_document doc;
auto root_node = doc.append_child("feed");
root_node.append_attribute("xmlns") = "http://www.w3.org/2005/Atom";
root_node.append_attribute("xmlns:opds") = "http://opds-spec.org/2010/catalog";
typedef kainjow::mustache::data MustacheData;
typedef kainjow::mustache::list BookData;
ADD_TEXT_ENTRY(root_node, "id", id);
ADD_TEXT_ENTRY(root_node, "title", title);
ADD_TEXT_ENTRY(root_node, "updated", date);
if (m_isSearchResult) {
ADD_TEXT_ENTRY(root_node, "totalResults", to_string(m_totalResults));
ADD_TEXT_ENTRY(root_node, "startIndex", to_string(m_startIndex));
ADD_TEXT_ENTRY(root_node, "itemsPerPage", to_string(m_count));
BookData getBookData(const Library* library, const std::vector<std::string>& bookIds)
{
BookData bookData;
for ( const auto& bookId : bookIds ) {
const Book& book = library->getBookById(bookId);
const MustacheData bookUrl = book.getUrl().empty()
? MustacheData(false)
: MustacheData(book.getUrl());
bookData.push_back(kainjow::mustache::object{
{"id", "urn:uuid:"+book.getId()},
{"name", book.getName()},
{"title", book.getTitle()},
{"description", book.getDescription()},
{"language", book.getLanguage()},
{"content_id", book.getHumanReadableIdFromPath()},
{"updated", book.getDate() + "T00:00:00Z"},
{"category", book.getCategory()},
{"flavour", book.getFlavour()},
{"tags", book.getTags()},
{"article_count", to_string(book.getArticleCount())},
{"media_count", to_string(book.getMediaCount())},
{"author_name", book.getCreator()},
{"publisher_name", book.getPublisher()},
{"url", bookUrl},
{"size", to_string(book.getSize())},
});
}
auto self_link_node = root_node.append_child("link");
self_link_node.append_attribute("rel") = "self";
self_link_node.append_attribute("href") = "";
self_link_node.append_attribute("type") = "application/atom+xml";
return bookData;
}
} // unnamed namespace
if (!searchDescriptionUrl.empty() ) {
auto search_link = root_node.append_child("link");
search_link.append_attribute("rel") = "search";
search_link.append_attribute("type") = "application/opensearchdescription+xml";
search_link.append_attribute("href") = searchDescriptionUrl.c_str();
string OPDSDumper::dumpOPDSFeed(const std::vector<std::string>& bookIds, const std::string& query) const
{
const auto bookData = getBookData(library, bookIds);
const kainjow::mustache::object template_data{
{"date", gen_date_str()},
{"root", rootLocation},
{"feed_id", gen_uuid(libraryId + "/catalog/search?"+query)},
{"filter", query.empty() ? MustacheData(false) : MustacheData(query)},
{"totalResults", to_string(m_totalResults)},
{"startIndex", to_string(m_startIndex)},
{"itemsPerPage", to_string(m_count)},
{"books", bookData }
};
return render_template(RESOURCE::templates::catalog_entries_xml, template_data);
}
string OPDSDumper::dumpOPDSFeedV2(const std::vector<std::string>& bookIds, const std::string& query) const
{
const auto bookData = getBookData(library, bookIds);
const kainjow::mustache::object template_data{
{"date", gen_date_str()},
{"endpoint_root", rootLocation + "/catalog/v2"},
{"feed_id", gen_uuid(libraryId + "/entries?"+query)},
{"filter", query.empty() ? MustacheData(false) : MustacheData(query)},
{"query", query.empty() ? "" : "?" + urlEncode(query)},
{"totalResults", to_string(m_totalResults)},
{"startIndex", to_string(m_startIndex)},
{"itemsPerPage", to_string(m_count)},
{"books", bookData }
};
return render_template(RESOURCE::templates::catalog_v2_entries_xml, template_data);
}
std::string OPDSDumper::categoriesOPDSFeed(const std::vector<std::string>& categories) const
{
const auto now = gen_date_str();
kainjow::mustache::list categoryData;
for ( const auto& category : categories ) {
const auto urlencodedCategoryName = urlEncode(category);
categoryData.push_back(kainjow::mustache::object{
{"name", category},
{"urlencoded_name", urlencodedCategoryName},
{"updated", now},
{"id", gen_uuid(libraryId + "/categories/" + urlencodedCategoryName)}
});
}
if (library) {
for (auto& bookId: bookIds) {
handleBook(library->getBookById(bookId), root_node);
}
}
return nodeToString(root_node);
return render_template(
RESOURCE::templates::catalog_v2_categories_xml,
kainjow::mustache::object{
{"date", now},
{"endpoint_root", rootLocation + "/catalog/v2"},
{"feed_id", gen_uuid(libraryId + "/categories")},
{"categories", categoryData }
}
);
}
}

View File

@@ -184,8 +184,7 @@ Entry Reader::getMainPage() const
bool Reader::getFavicon(string& content, string& mimeType) const
{
try {
auto entry = zimArchive->getFaviconEntry();
auto item = entry.getItem(true);
auto item = zimArchive->getIllustrationItem();
content = item.getData();
mimeType = item.getMimetype();
return true;
@@ -432,12 +431,12 @@ bool Reader::searchSuggestions(const string& prefix,
article is already in the suggestions list (with an other
title) */
bool insert = true;
std::vector<std::vector<std::string>>::iterator suggestionItr;
std::vector<SuggestionItem>::iterator suggestionItr;
for (suggestionItr = results.begin();
suggestionItr != results.end();
suggestionItr++) {
int result = normalizedArticleTitle.compare((*suggestionItr)[2]);
if (result == 0 && articleFinalUrl.compare((*suggestionItr)[1]) == 0) {
int result = normalizedArticleTitle.compare((*suggestionItr).getNormalizedTitle());
if (result == 0 && articleFinalUrl.compare((*suggestionItr).getPath()) == 0) {
insert = false;
break;
} else if (result < 0) {
@@ -447,10 +446,7 @@ bool Reader::searchSuggestions(const string& prefix,
/* Insert if possible */
if (insert) {
std::vector<std::string> suggestion;
suggestion.push_back(entry.getTitle());
suggestion.push_back(articleFinalUrl);
suggestion.push_back(normalizedArticleTitle);
SuggestionItem suggestion(entry.getTitle(), normalizedArticleTitle, articleFinalUrl);
results.insert(suggestionItr, suggestion);
}
@@ -495,19 +491,19 @@ bool Reader::searchSuggestionsSmart(const string& prefix,
bool retVal = false;
/* Try to search in the title using fulltext search database */
auto suggestionSearch = zim::Search(*zimArchive);
suggestionSearch.set_query(prefix);
suggestionSearch.set_range(0, suggestionsCount);
suggestionSearch.set_suggestion_mode(true);
if (suggestionSearch.get_matches_estimated()) {
for (auto current = suggestionSearch.begin();
current != suggestionSearch.end();
auto suggestionSearcher = zim::Searcher(*zimArchive);
zim::Query suggestionQuery;
suggestionQuery.setQuery(prefix, true);
auto suggestionSearch = suggestionSearcher.search(suggestionQuery);
if (suggestionSearch.getEstimatedMatches()) {
const auto suggestions = suggestionSearch.getResults(0, suggestionsCount);
for (auto current = suggestions.begin();
current != suggestions.end();
current++) {
std::vector<std::string> suggestion;
suggestion.push_back(current->getTitle());
suggestion.push_back(current->getPath());
suggestion.push_back(kiwix::normalize(current->getTitle()));
SuggestionItem suggestion(current.getTitle(), kiwix::normalize(current.getTitle()),
current.getPath(), current.getSnippet());
results.push_back(suggestion);
}
retVal = true;
@@ -528,7 +524,7 @@ bool Reader::getNextSuggestion(string& title)
{
if (this->suggestionsOffset != this->suggestions.end()) {
/* title */
title = (*(this->suggestionsOffset))[0];
title = (*(this->suggestionsOffset)).getTitle();
/* increment the cursor for the next call */
this->suggestionsOffset++;
@@ -543,8 +539,8 @@ bool Reader::getNextSuggestion(string& title, string& url)
{
if (this->suggestionsOffset != this->suggestions.end()) {
/* title */
title = (*(this->suggestionsOffset))[0];
url = (*(this->suggestionsOffset))[1];
title = (*(this->suggestionsOffset)).getTitle();
url = (*(this->suggestionsOffset)).getPath();
/* increment the cursor for the next call */
this->suggestionsOffset++;

View File

@@ -77,9 +77,7 @@ std::string SearchRenderer::getHtml()
result.set("title", p_result->get_title());
result.set("url", p_result->get_url());
result.set("snippet", p_result->get_snippet());
auto readerIndex = p_result->get_readerIndex();
auto reader = mp_searcher->get_reader(readerIndex);
result.set("resultContentId", mp_nameMapper->getNameForId(reader->getId()));
result.set("resultContentId", mp_nameMapper->getNameForId(p_result->get_zimId()));
if (p_result->get_wordCount() >= 0) {
result.set("wordCount", kiwix::beautifyInteger(p_result->get_wordCount()));

View File

@@ -35,7 +35,7 @@ namespace kiwix
class _Result : public Result
{
public:
_Result(zim::Search::iterator& iterator);
_Result(zim::SearchResultSet::iterator iterator);
virtual ~_Result(){};
virtual std::string get_url();
@@ -45,29 +45,25 @@ class _Result : public Result
virtual std::string get_content();
virtual int get_wordCount();
virtual int get_size();
virtual int get_readerIndex();
virtual std::string get_zimId();
private:
zim::Search::iterator iterator;
zim::SearchResultSet::iterator iterator;
};
struct SearcherInternal {
const zim::Search* _search;
zim::Search::iterator current_iterator;
SearcherInternal() : _search(NULL) {}
~SearcherInternal()
struct SearcherInternal : zim::SearchResultSet {
explicit SearcherInternal(const zim::SearchResultSet& srs)
: zim::SearchResultSet(srs)
, current_iterator(srs.begin())
{
if (_search != NULL) {
delete _search;
}
}
zim::SearchResultSet::iterator current_iterator;
};
/* Constructor */
Searcher::Searcher()
: internal(new SearcherInternal()),
searchPattern(""),
: searchPattern(""),
estimatedResultCount(0),
resultStart(0),
resultEnd(0)
@@ -78,7 +74,6 @@ Searcher::Searcher()
/* Destructor */
Searcher::~Searcher()
{
delete internal;
}
bool Searcher::add_reader(Reader* reader)
@@ -122,13 +117,13 @@ void Searcher::search(const std::string& search,
archives.push_back(*(*current)->getZimArchive());
}
}
zim::Search* search = new zim::Search(archives);
search->set_verbose(verbose);
search->set_query(unaccentedSearch);
search->set_range(resultStart, resultEnd);
internal->_search = search;
internal->current_iterator = internal->_search->begin();
this->estimatedResultCount = internal->_search->get_matches_estimated();
zim::Searcher searcher(archives);
zim::Query query;
query.setQuery(unaccentedSearch, false);
query.setVerbose(verbose);
zim::Search search = searcher.search(query);
internal.reset(new SearcherInternal(search.getResults(resultStart, resultEnd)));
this->estimatedResultCount = search.getEstimatedMatches();
}
return;
@@ -163,28 +158,28 @@ void Searcher::geo_search(float latitude, float longitude, float distance,
current++) {
archives.push_back(*(*current)->getZimArchive());
}
zim::Search* search = new zim::Search(archives);
search->set_verbose(verbose);
search->set_query("");
search->set_georange(latitude, longitude, distance);
search->set_range(resultStart, resultEnd);
internal->_search = search;
internal->current_iterator = internal->_search->begin();
this->estimatedResultCount = internal->_search->get_matches_estimated();
zim::Searcher searcher(archives);
zim::Query query;
query.setVerbose(verbose);
query.setQuery("", false);
query.setGeorange(latitude, longitude, distance);
zim::Search search = searcher.search(query);
internal.reset(new SearcherInternal(search.getResults(resultStart, resultEnd)));
this->estimatedResultCount = search.getEstimatedMatches();
}
void Searcher::restart_search()
{
if (internal->_search) {
internal->current_iterator = internal->_search->begin();
if (internal.get()) {
internal->current_iterator = internal->begin();
}
}
Result* Searcher::getNextResult()
{
if (internal->_search &&
internal->current_iterator != internal->_search->end()) {
if (internal.get() &&
internal->current_iterator != internal->end()) {
Result* result = new _Result(internal->current_iterator);
internal->current_iterator++;
return result;
@@ -218,14 +213,13 @@ void Searcher::suggestions(std::string& searchPattern, const bool verbose)
current++) {
archives.push_back(*(*current)->getZimArchive());
}
zim::Search* search = new zim::Search(archives);
search->set_verbose(verbose);
search->set_query(unaccentedSearch);
search->set_range(resultStart, resultEnd);
search->set_suggestion_mode(true);
internal->_search = search;
internal->current_iterator = internal->_search->begin();
this->estimatedResultCount = internal->_search->get_matches_estimated();
zim::Searcher searcher(archives);
zim::Query query;
query.setVerbose(verbose);
query.setQuery(unaccentedSearch, true);
zim::Search search = searcher.search(query);
internal.reset(new SearcherInternal(search.getResults(resultStart, resultEnd)));
this->estimatedResultCount = search.getEstimatedMatches();
}
/* Return the result count estimation */
@@ -234,26 +228,26 @@ unsigned int Searcher::getEstimatedResultCount()
return this->estimatedResultCount;
}
_Result::_Result(zim::Search::iterator& iterator)
_Result::_Result(zim::SearchResultSet::iterator iterator)
: iterator(iterator)
{
}
std::string _Result::get_url()
{
return iterator.get_path();
return iterator.getPath();
}
std::string _Result::get_title()
{
return iterator.get_title();
return iterator.getTitle();
}
int _Result::get_score()
{
return iterator.get_score();
return iterator.getScore();
}
std::string _Result::get_snippet()
{
return iterator.get_snippet();
return iterator.getSnippet();
}
std::string _Result::get_content()
{
@@ -261,15 +255,17 @@ std::string _Result::get_content()
}
int _Result::get_size()
{
return iterator.get_size();
return iterator.getSize();
}
int _Result::get_wordCount()
{
return iterator.get_wordCount();
return iterator.getWordCount();
}
int _Result::get_readerIndex()
std::string _Result::get_zimId()
{
return iterator.get_fileIndex();
std::ostringstream s;
s << iterator.getZimId();
return s.str();
}

View File

@@ -76,6 +76,21 @@ extern "C" {
namespace kiwix {
namespace
{
inline std::string normalizeRootUrl(std::string rootUrl)
{
while ( !rootUrl.empty() && rootUrl.back() == '/' )
rootUrl.pop_back();
while ( !rootUrl.empty() && rootUrl.front() == '/' )
rootUrl = rootUrl.substr(1);
return rootUrl.empty() ? rootUrl : "/" + rootUrl;
}
} // unnamed namespace
static IdNameMapper defaultNameMapper;
static MHD_Result staticHandlerCallback(void* cls,
@@ -100,7 +115,7 @@ InternalServer::InternalServer(Library* library,
bool blockExternalLinks) :
m_addr(addr),
m_port(port),
m_root(root),
m_root(normalizeRootUrl(root)),
m_nbThreads(nbThreads),
m_verbose(verbose),
m_withTaskbar(withTaskbar),
@@ -153,6 +168,7 @@ bool InternalServer::start() {
}
auto server_start_time = std::chrono::system_clock::now().time_since_epoch();
m_server_id = kiwix::to_string(server_start_time.count());
m_library_id = m_server_id;
return true;
}
@@ -237,16 +253,16 @@ std::unique_ptr<Response> InternalServer::handle_request(const RequestContext& r
{
try {
if (! request.is_valid_url())
return Response::build_404(*this, request, "");
return Response::build_404(*this, request, "", "");
const ETag etag = get_matching_if_none_match_etag(request);
if ( etag )
return Response::build_304(*this, etag);
if (kiwix::startsWith(request.get_url(), "/skin/"))
if (startsWith(request.get_url(), "/skin/"))
return handle_skin(request);
if (startsWith(request.get_url(), "/catalog"))
if (startsWith(request.get_url(), "/catalog/"))
return handle_catalog(request);
if (request.get_url() == "/meta")
@@ -281,27 +297,6 @@ MustacheData InternalServer::get_default_data() const
return data;
}
MustacheData InternalServer::homepage_data() const
{
auto data = get_default_data();
MustacheData books{MustacheData::type::list};
for (auto& bookId: mp_library->filter(kiwix::Filter().local(true).valid(true))) {
auto& currentBook = mp_library->getBookById(bookId);
MustacheData book;
book.set("name", mp_nameMapper->getNameForId(bookId));
book.set("title", currentBook.getTitle());
book.set("description", currentBook.getDescription());
book.set("articleCount", beautifyInteger(currentBook.getArticleCount()));
book.set("mediaCount", beautifyInteger(currentBook.getMediaCount()));
books.push_back(book);
}
data.set("books", books);
return data;
}
bool InternalServer::etag_not_needed(const RequestContext& request) const
{
const std::string url = request.get_url();
@@ -325,7 +320,7 @@ InternalServer::get_matching_if_none_match_etag(const RequestContext& r) const
std::unique_ptr<Response> InternalServer::build_homepage(const RequestContext& request)
{
return ContentResponse::build(*this, RESOURCE::templates::index_html, homepage_data(), "text/html; charset=utf-8", true);
return ContentResponse::build(*this, RESOURCE::templates::index_html, get_default_data(), "text/html; charset=utf-8", true);
}
std::unique_ptr<Response> InternalServer::handle_meta(const RequestContext& request)
@@ -340,11 +335,11 @@ std::unique_ptr<Response> InternalServer::handle_meta(const RequestContext& requ
meta_name = request.get_argument("name");
reader = mp_library->getReaderById(bookId);
} catch (const std::out_of_range& e) {
return Response::build_404(*this, request, bookName);
return Response::build_404(*this, request, bookName, "");
}
if (reader == nullptr) {
return Response::build_404(*this, request, bookName);
return Response::build_404(*this, request, bookName, "");
}
std::string content;
@@ -369,7 +364,7 @@ std::unique_ptr<Response> InternalServer::handle_meta(const RequestContext& requ
} else if (meta_name == "favicon") {
reader->getFavicon(content, mimeType);
} else {
return Response::build_404(*this, request, bookName);
return Response::build_404(*this, request, bookName, "");
}
auto response = ContentResponse::build(*this, content, mimeType);
@@ -398,7 +393,7 @@ std::unique_ptr<Response> InternalServer::handle_suggest(const RequestContext& r
term = request.get_argument("term");
reader = mp_library->getReaderById(bookId);
} catch (const std::out_of_range&) {
return Response::build_404(*this, request, bookName);
return Response::build_404(*this, request, bookName, "");
}
if (m_verbose.load()) {
@@ -414,10 +409,15 @@ std::unique_ptr<Response> InternalServer::handle_suggest(const RequestContext& r
reader->searchSuggestionsSmart(term, maxSuggestionCount, suggestions);
for(auto& suggestion:suggestions) {
MustacheData result;
result.set("label", suggestion[0]);
result.set("value", suggestion[0]);
result.set("label", suggestion.getTitle());
if (suggestion.hasSnippet()) {
result.set("label", suggestion.getSnippet());
}
result.set("value", suggestion.getTitle());
result.set("kind", "path");
result.set("path", suggestion[1]);
result.set("path", suggestion.getPath());
result.set("first", first);
first = false;
results.push_back(result);
@@ -457,7 +457,7 @@ std::unique_ptr<Response> InternalServer::handle_skin(const RequestContext& requ
response->set_cacheable();
return std::move(response);
} catch (const ResourceNotFound& e) {
return Response::build_404(*this, request, "");
return Response::build_404(*this, request, "", "");
}
}
@@ -577,18 +577,18 @@ std::unique_ptr<Response> InternalServer::handle_random(const RequestContext& re
bookId = mp_nameMapper->getIdForName(bookName);
reader = mp_library->getReaderById(bookId);
} catch (const std::out_of_range&) {
return Response::build_404(*this, request, bookName);
return Response::build_404(*this, request, bookName, "");
}
if (reader == nullptr) {
return Response::build_404(*this, request, bookName);
return Response::build_404(*this, request, bookName, "");
}
try {
auto entry = reader->getRandomPage();
return build_redirect(bookName, entry.getFinalEntry());
} catch(kiwix::NoEntry& e) {
return Response::build_404(*this, request, bookName);
return Response::build_404(*this, request, bookName, "");
}
}
@@ -600,7 +600,7 @@ std::unique_ptr<Response> InternalServer::handle_captured_external(const Request
} catch (const std::out_of_range& e) {}
if (source.empty())
return Response::build_404(*this, request, "");
return Response::build_404(*this, request, "", "");
auto data = get_default_data();
data.set("source", source);
@@ -619,11 +619,15 @@ std::unique_ptr<Response> InternalServer::handle_catalog(const RequestContext& r
host = request.get_header("Host");
url = request.get_url_part(1);
} catch (const std::out_of_range&) {
return Response::build_404(*this, request, "");
return Response::build_404(*this, request, "", "");
}
if (url == "v2") {
return handle_catalog_v2(request);
}
if (url != "searchdescription.xml" && url != "root.xml" && url != "search") {
return Response::build_404(*this, request, "");
return Response::build_404(*this, request, "", "");
}
if (url == "searchdescription.xml") {
@@ -632,13 +636,11 @@ std::unique_ptr<Response> InternalServer::handle_catalog(const RequestContext& r
}
zim::Uuid uuid;
kiwix::OPDSDumper opdsDumper;
kiwix::OPDSDumper opdsDumper(mp_library);
opdsDumper.setRootLocation(m_root);
opdsDumper.setSearchDescriptionUrl("catalog/searchdescription.xml");
opdsDumper.setLibrary(mp_library);
opdsDumper.setLibraryId(m_library_id);
std::vector<std::string> bookIdsToDump;
if (url == "root.xml") {
opdsDumper.setTitle("All zims");
uuid = zim::Uuid::generate(host);
bookIdsToDump = mp_library->filter(kiwix::Filter().valid(true).local(true).remote(true));
} else if (url == "search") {
@@ -646,28 +648,24 @@ std::unique_ptr<Response> InternalServer::handle_catalog(const RequestContext& r
uuid = zim::Uuid::generate();
}
opdsDumper.setId(kiwix::to_string(uuid));
auto response = ContentResponse::build(
*this,
opdsDumper.dumpOPDSFeed(bookIdsToDump),
opdsDumper.dumpOPDSFeed(bookIdsToDump, request.get_query()),
"application/atom+xml; profile=opds-catalog; kind=acquisition; charset=utf-8");
return std::move(response);
}
std::vector<std::string>
InternalServer::search_catalog(const RequestContext& request,
kiwix::OPDSDumper& opdsDumper)
namespace
{
auto filter = kiwix::Filter().valid(true).local(true).remote(true);
string query("<Empty query>");
size_t count(10);
size_t startIndex(0);
Filter get_search_filter(const RequestContext& request)
{
auto filter = kiwix::Filter().valid(true).local(true);
try {
query = request.get_argument("q");
filter.query(query);
filter.query(request.get_argument("q"));
} catch (const std::out_of_range&) {}
try {
filter.maxSize(extractFromString<unsigned long>(request.get_argument("maxsize")));
filter.maxSize(request.get_argument<unsigned long>("maxsize"));
} catch (...) {}
try {
filter.name(request.get_argument("name"));
@@ -678,26 +676,37 @@ InternalServer::search_catalog(const RequestContext& request,
try {
filter.lang(request.get_argument("lang"));
} catch (const std::out_of_range&) {}
try {
count = extractFromString<unsigned long>(request.get_argument("count"));
} catch (...) {}
try {
startIndex = extractFromString<unsigned long>(request.get_argument("start"));
} catch (...) {}
try {
filter.acceptTags(kiwix::split(request.get_argument("tag"), ";"));
} catch (...) {}
try {
filter.rejectTags(kiwix::split(request.get_argument("notag"), ";"));
} catch (...) {}
opdsDumper.setTitle("Search result for " + query);
return filter;
}
template<class T>
std::vector<T> subrange(const std::vector<T>& v, size_t s, size_t n)
{
const size_t e = std::min(v.size(), s+n);
return std::vector<T>(v.begin()+std::min(v.size(), s), v.begin()+e);
}
} // unnamed namespace
std::vector<std::string>
InternalServer::search_catalog(const RequestContext& request,
kiwix::OPDSDumper& opdsDumper)
{
const auto filter = get_search_filter(request);
const std::string q = filter.hasQuery()
? filter.getQuery()
: "<Empty query>";
std::vector<std::string> bookIdsToDump = mp_library->filter(filter);
const auto totalResults = bookIdsToDump.size();
const auto s = std::min(startIndex, totalResults);
bookIdsToDump.erase(bookIdsToDump.begin(), bookIdsToDump.begin()+s);
if (count>0 && bookIdsToDump.size() > count) {
bookIdsToDump.resize(count);
}
const size_t count = request.get_optional_param("count", 10UL);
const size_t startIndex = request.get_optional_param("start", 0UL);
bookIdsToDump = subrange(bookIdsToDump, startIndex, count);
opdsDumper.setOpenSearchInfo(totalResults, startIndex, bookIdsToDump.size());
return bookIdsToDump;
}
@@ -761,7 +770,7 @@ std::unique_ptr<Response> InternalServer::handle_content(const RequestContext& r
std::string searchURL = m_root+"/search?pattern="+pattern; // Make a full search on the entire library.
const std::string details = searchSuggestionHTML(searchURL, kiwix::urlDecode(pattern));
return Response::build_404(*this, request, bookName, details);
return Response::build_404(*this, request, bookName, "", details);
}
auto urlStr = request.get_url().substr(bookName.size()+1);
@@ -794,7 +803,7 @@ std::unique_ptr<Response> InternalServer::handle_content(const RequestContext& r
std::string searchURL = m_root+"/search?content="+bookName+"&pattern="+pattern; // Make a search on this specific book only.
const std::string details = searchSuggestionHTML(searchURL, kiwix::urlDecode(pattern));
return Response::build_404(*this, request, bookName, details);
return Response::build_404(*this, request, bookName, reader->getTitle(), details);
}
}

View File

@@ -73,6 +73,10 @@ class InternalServer {
std::unique_ptr<Response> build_homepage(const RequestContext& request);
std::unique_ptr<Response> handle_skin(const RequestContext& request);
std::unique_ptr<Response> handle_catalog(const RequestContext& request);
std::unique_ptr<Response> handle_catalog_v2(const RequestContext& request);
std::unique_ptr<Response> handle_catalog_v2_root(const RequestContext& request);
std::unique_ptr<Response> handle_catalog_v2_entries(const RequestContext& request);
std::unique_ptr<Response> handle_catalog_v2_categories(const RequestContext& request);
std::unique_ptr<Response> handle_meta(const RequestContext& request);
std::unique_ptr<Response> handle_search(const RequestContext& request);
std::unique_ptr<Response> handle_suggest(const RequestContext& request);
@@ -84,7 +88,6 @@ class InternalServer {
kiwix::OPDSDumper& opdsDumper);
MustacheData get_default_data() const;
MustacheData homepage_data() const;
std::shared_ptr<Reader> get_reader(const std::string& bookName) const;
bool etag_not_needed(const RequestContext& r) const;
@@ -105,6 +108,7 @@ class InternalServer {
NameMapper* mp_nameMapper;
std::string m_server_id;
std::string m_library_id;
friend std::unique_ptr<Response> Response::build(const InternalServer& server);
friend std::unique_ptr<ContentResponse> ContentResponse::build(const InternalServer& server, const std::string& content, const std::string& mimetype, bool isHomePage);

View File

@@ -0,0 +1,109 @@
/*
* Copyright 2021 Veloman Yunkan <veloman.yunkan@gmail.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
* MA 02110-1301, USA.
*/
#include "internalServer.h"
#include "library.h"
#include "opds_dumper.h"
#include "request_context.h"
#include "response.h"
#include "tools/otherTools.h"
#include "kiwixlib-resources.h"
#include <mustache.hpp>
#include <string>
#include <vector>
namespace kiwix {
std::unique_ptr<Response> InternalServer::handle_catalog_v2(const RequestContext& request)
{
if (m_verbose.load()) {
printf("** running handle_catalog_v2");
}
std::string url;
try {
url = request.get_url_part(2);
} catch (const std::out_of_range&) {
return Response::build_404(*this, request, "", "");
}
if (url == "root.xml") {
return handle_catalog_v2_root(request);
} else if (url == "searchdescription.xml") {
const std::string endpoint_root = m_root + "/catalog/v2";
return ContentResponse::build(*this,
RESOURCE::catalog_v2_searchdescription_xml,
kainjow::mustache::object({{"endpoint_root", endpoint_root}}),
"application/opensearchdescription+xml"
);
} else if (url == "entries") {
return handle_catalog_v2_entries(request);
} else if (url == "categories") {
return handle_catalog_v2_categories(request);
} else {
return Response::build_404(*this, request, "", "");
}
}
std::unique_ptr<Response> InternalServer::handle_catalog_v2_root(const RequestContext& request)
{
return ContentResponse::build(
*this,
RESOURCE::templates::catalog_v2_root_xml,
kainjow::mustache::object{
{"date", gen_date_str()},
{"endpoint_root", m_root + "/catalog/v2"},
{"feed_id", gen_uuid(m_library_id)},
{"all_entries_feed_id", gen_uuid(m_library_id + "/entries")},
{"category_list_feed_id", gen_uuid(m_library_id + "/categories")}
},
"application/atom+xml;profile=opds-catalog;kind=navigation"
);
}
std::unique_ptr<Response> InternalServer::handle_catalog_v2_entries(const RequestContext& request)
{
OPDSDumper opdsDumper(mp_library);
opdsDumper.setRootLocation(m_root);
opdsDumper.setLibraryId(m_library_id);
const auto bookIds = search_catalog(request, opdsDumper);
const auto opdsFeed = opdsDumper.dumpOPDSFeedV2(bookIds, request.get_query());
return ContentResponse::build(
*this,
opdsFeed,
"application/atom+xml;profile=opds-catalog;kind=acquisition"
);
}
std::unique_ptr<Response> InternalServer::handle_catalog_v2_categories(const RequestContext& request)
{
OPDSDumper opdsDumper(mp_library);
opdsDumper.setRootLocation(m_root);
opdsDumper.setLibraryId(m_library_id);
return ContentResponse::build(
*this,
opdsDumper.categoriesOPDSFeed(mp_library->getBooksCategories()),
"application/atom+xml;profile=opds-catalog;kind=navigation"
);
}
} // namespace kiwix

View File

@@ -183,4 +183,14 @@ std::string RequestContext::get_header(const std::string& name) const {
return headers.at(lcAll(name));
}
std::string RequestContext::get_query() const {
std::string q;
const char* sep = "";
for ( const auto& a : arguments ) {
q += sep + a.first + '=' + a.second;
sep = "&";
}
return q;
}
}

View File

@@ -74,11 +74,21 @@ class RequestContext {
return v;
}
template<class T>
T get_optional_param(const std::string& name, T default_value) const
{
try {
return get_argument<T>(name);
} catch (...) {}
return default_value;
}
RequestMethod get_method() const;
std::string get_url() const;
std::string get_url_part(int part) const;
std::string get_full_url() const;
std::string get_query() const;
ByteRange get_range() const;

View File

@@ -24,6 +24,7 @@
#include "tools/regexTools.h"
#include "tools/stringTools.h"
#include "tools/otherTools.h"
#include "string.h"
#include <mustache.hpp>
@@ -38,17 +39,6 @@ namespace
{
// some utilities
std::string render_template(const std::string& template_str, kainjow::mustache::data data)
{
kainjow::mustache::mustache tmpl(template_str);
kainjow::mustache::data urlencode{kainjow::mustache::lambda2{
[](const std::string& str,const kainjow::mustache::renderer& r) { return urlEncode(r(str), true); }}};
data.set("urlencoded", urlencode);
std::stringstream ss;
tmpl.render(data, [&ss](const std::string& str) { ss << str; });
return ss.str();
}
std::string get_mime_type(const zim::Item& item)
{
try {
@@ -93,7 +83,7 @@ std::unique_ptr<Response> Response::build_304(const InternalServer& server, cons
return response;
}
std::unique_ptr<Response> Response::build_404(const InternalServer& server, const RequestContext& request, const std::string& bookName, const std::string& details)
std::unique_ptr<Response> Response::build_404(const InternalServer& server, const RequestContext& request, const std::string& bookName, const std::string& bookTitle, const std::string& details)
{
MustacheData results;
results.set("url", request.get_full_url());
@@ -101,7 +91,7 @@ std::unique_ptr<Response> Response::build_404(const InternalServer& server, cons
auto response = ContentResponse::build(server, RESOURCE::templates::_404_html, results, "text/html");
response->set_code(MHD_HTTP_NOT_FOUND);
response->set_taskbar(bookName, "");
response->set_taskbar(bookName, bookTitle);
return std::move(response);
}
@@ -200,7 +190,7 @@ void ContentResponse::introduce_taskbar()
kainjow::mustache::data data;
data.set("root", m_root);
data.set("content", m_bookName);
data.set("hascontent", !m_bookName.empty());
data.set("hascontent", (!m_bookName.empty() && !m_bookTitle.empty()));
data.set("title", m_bookTitle);
data.set("withlibrarybutton", m_withLibraryButton);
auto head_content = render_template(RESOURCE::templates::head_taskbar_html, data);
@@ -260,9 +250,9 @@ Response::create_mhd_response(const RequestContext& request)
MHD_Response*
ContentResponse::create_mhd_response(const RequestContext& request)
{
inject_root_link();
if (contentDecorationAllowed()) {
inject_root_link();
if (m_withTaskbar) {
introduce_taskbar();
}

View File

@@ -47,7 +47,7 @@ class Response {
static std::unique_ptr<Response> build(const InternalServer& server);
static std::unique_ptr<Response> build_304(const InternalServer& server, const ETag& etag);
static std::unique_ptr<Response> build_404(const InternalServer& server, const RequestContext& request, const std::string& bookName, const std::string& details="");
static std::unique_ptr<Response> build_404(const InternalServer& server, const RequestContext& request, const std::string& bookName, const std::string& bookTitle, const std::string& details="");
static std::unique_ptr<Response> build_416(const InternalServer& server, size_t resourceLength);
static std::unique_ptr<Response> build_500(const InternalServer& server, const std::string& msg);
static std::unique_ptr<Response> build_redirect(const InternalServer& server, const std::string& redirectUrl);

View File

@@ -19,6 +19,7 @@
#include "tools/otherTools.h"
#include <algorithm>
#include <iomanip>
#ifdef _WIN32
#include <windows.h>
@@ -32,6 +33,8 @@
#include <sstream>
#include <pugixml.hpp>
#include <zim/uuid.h>
static std::map<std::string, std::string> codeisomapping {
{ "aa", "aar" },
@@ -341,3 +344,35 @@ kiwix::MimeCounterType kiwix::parseMimetypeCounter(const std::string& counterDat
return counters;
}
std::string kiwix::gen_date_str()
{
auto now = std::time(0);
auto tm = std::localtime(&now);
std::stringstream is;
is << std::setw(2) << std::setfill('0')
<< 1900+tm->tm_year << "-"
<< std::setw(2) << std::setfill('0') << tm->tm_mon+1 << "-"
<< std::setw(2) << std::setfill('0') << tm->tm_mday << "T"
<< std::setw(2) << std::setfill('0') << tm->tm_hour << ":"
<< std::setw(2) << std::setfill('0') << tm->tm_min << ":"
<< std::setw(2) << std::setfill('0') << tm->tm_sec << "Z";
return is.str();
}
std::string kiwix::gen_uuid(const std::string& s)
{
return kiwix::to_string(zim::Uuid::generate(s));
}
std::string kiwix::render_template(const std::string& template_str, kainjow::mustache::data data)
{
kainjow::mustache::mustache tmpl(template_str);
kainjow::mustache::data urlencode{kainjow::mustache::lambda2{
[](const std::string& str,const kainjow::mustache::renderer& r) { return urlEncode(r(str), true); }}};
data.set("urlencoded", urlencode);
std::stringstream ss;
tmpl.render(data, [&ss](const std::string& str) { ss << str; });
return ss.str();
}

View File

@@ -339,7 +339,7 @@ std::string makeTmpDirectory()
_wmkdir(ctmp);
return WideToUtf8(ctmp);
#else
char _template_array[] = {"/tmp/kiwix-lib_XXXXXX"};
char _template_array[] = {"/tmp/libkiwix_XXXXXX"};
std::string dir = mkdtemp(_template_array);
return dir;
#endif

View File

@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
<ShortName>Zim catalog search</ShortName>
<Description>Search zim files in the catalog.</Description>
<Url type="application/atom+xml;profile=opds-catalog;kind=acquisition"
xmlns:atom="http://www.w3.org/2005/Atom"
xmlns:k="http://kiwix.org/opensearchextension/1.0"
indexOffset="0"
template="{{endpoint_root}}/entries?q={searchTerms?}&lang={language?}&name={k:name?}&tag={k:tag?}&maxsize={k:maxsize?}&count={count?}&start={startIndex?}"/>
</OpenSearchDescription>

View File

@@ -6,5 +6,5 @@
xmlns:atom="http://www.w3.org/2005/Atom"
xmlns:k="http://kiwix.org/opensearchextension/1.0"
indexOffset="0"
template="/{{root}}/catalog/search?q={searchTerms?}&lang={language?}&name={k:name?}&tag={k:tag?}&notag={k:notag?}&maxsize={k:maxsize?}&count={count?}&start={startIndex?}"/>
template="{{root}}/catalog/search?q={searchTerms?}&lang={language?}&name={k:name?}&tag={k:tag?}&notag={k:notag?}&maxsize={k:maxsize?}&count={count?}&start={startIndex?}"/>
</OpenSearchDescription>

View File

@@ -19,6 +19,10 @@ skin/jquery-ui/jquery-ui.theme.min.css
skin/jquery-ui/jquery-ui.min.css
skin/caret.png
skin/taskbar.js
skin/langList.js
skin/categoryList.js
skin/iso6391To3.js
skin/isotope.pkgd.min.js
skin/index.js
skin/taskbar.css
skin/block_external.js
@@ -32,4 +36,9 @@ templates/head_taskbar.html
templates/taskbar_part.html
templates/external_blocker_part.html
templates/captured_external.html
templates/catalog_entries.xml
templates/catalog_v2_root.xml
templates/catalog_v2_entries.xml
templates/catalog_v2_categories.xml
opensearchdescription.xml
catalog_v2_searchdescription.xml

View File

@@ -0,0 +1,19 @@
// eslint-disable-next-line no-unused-vars
const categoryList = {
"other": "Other",
"gutenberg": "Gutenberg",
"mooc": "Mooc",
"phet": "Phet",
"psiram": "Psiram",
"stack_exchange": "Stack Exchange",
"ted": "Ted",
"vikidia": "Vikidia",
"wikibooks": "Wikibooks",
"wikinews": "Wikinews",
"wikipedia": "Wikipedia",
"wikiquote": "Wikiquote",
"wikisource": "Wikisource",
"wikiversity": "Wikiversity",
"wikivoyage": "Wikivoyage",
"wiktionary": "Wiktionary"
}

View File

@@ -1,95 +1,250 @@
const root = $( `link[type='root']` ).attr("href");
let viewPortHeight = window.innerHeight;
let isFetching = false;
let isEnd = false;
const params = new URLSearchParams(window.location.search);
const basicConfig = {
'start': 0,
'count': Math.floor(viewPortHeight/100 + 1) * 3
};
(function() {
const root = $(`link[type='root']`).attr('href');
const incrementalLoadingParams = {
start: 0,
count: viewPortToCount()
};
const filterTypes = ['lang', 'category', 'q'];
const bookOrderMap = new Map();
const filterCookieName = 'filters';
const oneDayDelta = 86400000;
let footer;
let fadeOutDiv;
let iso;
let isFetching = false;
let noResultInjected = false;
let filters = getCookie(filterCookieName);
let params = new URLSearchParams(window.location.search || filters || '');
let timer;
function queryUrlBuilder() {
let url = `${root}/catalog/search?`
Object.keys(basicConfig).forEach((key, idx) => {
url += `${key}=${basicConfig[key]}${idx !== Object.keys(basicConfig).length - 1 ? '&' : ''}`;
});
return url + (params.toString() ? `&${params.toString()}` : '');
}
function queryUrlBuilder() {
let url = `${root}/catalog/search?`;
url += Object.keys(incrementalLoadingParams).map(key => `${key}=${incrementalLoadingParams[key]}`).join("&");
params.forEach((value, key) => {url+= value ? `&${key}=${value}` : ''});
return (url);
}
async function loadAndDisplayBooks(append = false) {
isFetching = true;
await fetch(queryUrlBuilder())
.then(async (resp) => {
const data = new window.DOMParser().parseFromString(await resp.text(), 'application/xml');
const books = data.querySelectorAll("entry");
let bookHtml = '';
books.forEach((book) => {
const link = book.querySelector('link').getAttribute('href');
const title = getInnerHtml(book, 'title');
const description = getInnerHtml(book, 'summary');
bookHtml += `<a href='${link}'><div class='book'>
<div class='book__background' style="background-image: url('${getInnerHtml(book, 'icon')}');">
<div class='book__title' title='${title}'>${title}</div>
<div class='book__description' title='${description}'>${description}</div>
<div class='book__info'>${getInnerHtml(book, 'articleCount')} articles, ${getInnerHtml(book, 'mediaCount')} medias</div>
</div>
</div></a>`;
function setCookie(cookieName, cookieValue) {
const date = new Date();
date.setTime(date.getTime() + oneDayDelta);
document.cookie = `${cookieName}=${cookieValue};expires=${date.toUTCString()};sameSite=Strict`;
}
function getCookie(cookieName) {
const name = cookieName + "=";
let result;
decodeURIComponent(document.cookie).split('; ').forEach(val => {
if (val.indexOf(name) === 0) {
result = val.substring(name.length);
}
});
document.querySelector('.book__list').innerHTML = (append ? document.querySelector('.book__list').innerHTML : '') + bookHtml;
return result;
}
function htmlEncode(str) {
return str.replace(/[\u00A0-\u9999<>\&]/gim, (i) => `&#${i.charCodeAt(0)};`);
}
function viewPortToCount(){
return Math.floor(window.innerHeight/100 + 1)*(window.innerWidth>1000 ? 3 : 2);
}
function getInnerHtml(node, query) {
return node.querySelector(query).innerHTML;
}
function generateBookHtml(book, sort = false) {
const link = book.querySelector('link').getAttribute('href');
const title = getInnerHtml(book, 'title');
const description = getInnerHtml(book, 'summary');
const id = getInnerHtml(book, 'id');
const iconUrl = getInnerHtml(book, 'icon');
const articleCount = getInnerHtml(book, 'articleCount');
const mediaCount = getInnerHtml(book, 'mediaCount');
const linkTag = document.createElement('a');
linkTag.setAttribute('class', 'book');
linkTag.setAttribute('data-id', id);
linkTag.setAttribute('href', link);
if (sort) {
linkTag.setAttribute('data-idx', bookOrderMap.get(id));
}
linkTag.innerHTML = `<div class='book__background' style="background-image: url('${iconUrl}');">
<div class='book__title' title='${title}'>${title}</div>
<div class='book__description' title='${description}'>${description}</div>
<div class='book__info'>${articleCount} articles, ${mediaCount} medias</div>
</div>`;
return linkTag;
}
function toggleFooter(show=false) {
if (show) {
footer.style.display = 'block';
} else {
footer.style.display = 'none';
fadeOutDiv.style.display = 'block';
}
}
async function loadBooks() {
const loader = document.querySelector('.loader');
loader.style.display = 'block';
return await fetch(queryUrlBuilder()).then(async (resp) => {
const data = new window.DOMParser().parseFromString(await resp.text(), 'application/xml');
const books = data.querySelectorAll('entry');
books.forEach((book, idx) => {
bookOrderMap.set(getInnerHtml(book, 'id'), idx);
});
incrementalLoadingParams.start += books.length;
if (parseInt(data.querySelector('totalResults').innerHTML) === bookOrderMap.size) {
incrementalLoadingParams.count = 0;
toggleFooter(true);
} else {
toggleFooter();
}
loader.style.display = 'none';
return books;
});
}
async function loadAndDisplayOptions(nodeQuery, query) {
// currently taking an object in place of query, will replace it with query while fetching data from backend later on.
document.querySelector(nodeQuery).innerHTML += Object.keys(query)
.map((option) => {return `<option value='${option}'>${htmlEncode(query[option])}</option>`})
.join('');
}
function checkAndInjectEmptyMessage() {
if (!bookOrderMap.size) {
if (!noResultInjected) {
noResultInjected = true;
iso.remove(document.getElementsByClassName('book__list')[0].getElementsByTagName('a'));
iso.layout();
const spanTag = document.createElement('span');
spanTag.setAttribute('class', 'noResults');
spanTag.innerHTML = `No result. Would you like to <a href="/?lang=">reset filter?</a>`;
document.querySelector('body').append(spanTag);
spanTag.getElementsByTagName('a')[0].onclick = (event) => {
event.preventDefault();
window.history.pushState({}, null, `${window.location.href.split('?')[0]}?lang=`);
setCookie(filterCookieName, 'lang=');
resetAndFilter();
filterTypes.forEach(key => {document.getElementsByName(key)[0].value = params.get(key) || ''});
};
}
return true;
} else if (noResultInjected) {
noResultInjected = false;
document.getElementsByClassName('noResults')[0].remove();
}
return false;
}
async function loadAndDisplayBooks(sort = false) {
if (isFetching) return;
isFetching = true;
await loadAndDisplayBooksUnguarded(sort);
isFetching = false;
isEnd = !books.length;
}
async function loadAndDisplayBooksUnguarded(sort) {
let books = await loadBooks();
if (checkAndInjectEmptyMessage()) {return}
const booksToFilter = new Set();
const booksToDelete = new Set();
iso.arrange({
filter: function (idx, elem) {
const id = elem.getAttribute('data-id');
const retVal = bookOrderMap.has(id);
if (retVal) {
booksToFilter.add(id);
if (sort) {
elem.setAttribute('data-idx', bookOrderMap.get(id));
iso.updateSortData(elem);
}
} else {
booksToDelete.add(elem);
}
return retVal;
}
});
books = [...books].filter((book) => {return !booksToFilter.has(getInnerHtml(book, 'id'))});
booksToDelete.forEach(book => {iso.remove(book);});
books.forEach((book) => {iso.insert(generateBookHtml(book, sort))});
}
async function resetAndFilter(filterType = '', filterValue = '') {
isFetching = false;
incrementalLoadingParams.start = 0;
incrementalLoadingParams.count = viewPortToCount();
fadeOutDiv.style.display = 'none';
bookOrderMap.clear();
params = new URLSearchParams(window.location.search);
if (filterType) {
params.set(filterType, filterValue);
window.history.pushState({}, null, `${window.location.href.split('?')[0]}?${params.toString()}`);
setCookie(filterCookieName, params.toString());
}
await loadAndDisplayBooks(true);
}
window.addEventListener('popstate', async () => {
await resetAndFilter();
filterTypes.forEach(key => {document.getElementsByName(key)[0].value = params.get(key) || ''});
});
}
async function loadAndDisplayOptions(nodeQuery, query) {
// currently taking an array in place of query, will replace it with query while fetching data from backend later on.
query.forEach((option) => {
let value = Object.keys(option)[0];
let label = option[value];
document.querySelector(nodeQuery).innerHTML += `<option value='${value}'>${label}</option>`;
})
}
async function filterBooks(filterType, filterValue) {
isEnd = false;
isFetching = false;
basicConfig['start'] = 0;
const params = new URLSearchParams(window.location.search);
if (!filterValue) {
params.delete(filterType);
} else {
params.set(filterType, filterValue);
async function loadSubset() {
if (window.innerHeight + window.scrollY >= document.body.offsetHeight) {
if (incrementalLoadingParams.count) {
loadAndDisplayBooks();
}
else {
fadeOutDiv.style.display = 'none';
}
}
}
window.location.search = params.toString();
loadAndDisplayBooks();
}
function getInnerHtml(node, query) {
return node.querySelector(query).innerHTML;
}
window.addEventListener('resize', (event) => {
if (timer) {clearTimeout(timer)}
timer = setTimeout(() => {
incrementalLoadingParams.count = incrementalLoadingParams.count && viewPortToCount();
loadSubset();
}, 100, event);
});
window.addEventListener('resize', async () => {
if (isFetching || isEnd) return;
viewPortHeight = window.innerHeight;
basicConfig['count'] = Math.floor(viewPortHeight/100 + 1) * 3;
basicConfig['start'] = basicConfig['start'] + basicConfig['count'];
loadAndDisplayBooks(true);
});
window.addEventListener('scroll', loadSubset);
window.addEventListener('scroll', async () => {
if (isFetching || isEnd) return;
if (viewPortHeight + window.scrollY >= document.body.offsetHeight) {
basicConfig['start'] = basicConfig['start'] + basicConfig['count'];
loadAndDisplayBooks(true);
window.onload = async () => {
iso = new Isotope( '.book__list', {
itemSelector: '.book',
getSortData:{
weight: function( itemElem ) {
const index = itemElem.getAttribute('data-idx');
return index ? parseInt(index) : Infinity;
}
},
sortBy: 'weight'
});
footer = document.getElementById('kiwixfooter');
fadeOutDiv = document.getElementById('fadeOut');
await loadAndDisplayBooks();
await loadAndDisplayOptions('#languageFilter', langList);
await loadAndDisplayOptions('#categoryFilter', categoryList);
filterTypes.forEach((filter) => {
const filterTag = document.getElementsByName(filter)[0];
filterTag.addEventListener('change', () => {resetAndFilter(filterTag.name, filterTag.value)});
});
if (filters) {
window.history.pushState({}, null, `${window.location.href.split('?')[0]}?${params.toString()}`);
}
params.forEach((value, key) => {document.getElementsByName(key)[0].value = value});
document.getElementById('kiwixSearchForm').onsubmit = (event) => {event.preventDefault()};
if (!window.location.search) {
const browserLang = navigator.language.split('-')[0];
const langFilter = document.getElementById('languageFilter');
langFilter.value = browserLang.length === 3 ? browserLang : iso6391To3[browserLang];
langFilter.dispatchEvent(new Event('change'));
}
setCookie(filterCookieName, params.toString());
}
});
window.onload = async (event) => {
loadAndDisplayBooks();
loadAndDisplayOptions('#languageFilter', [{'eng': 'English'}, {'fra': 'french'}, {'ara': 'arab'}, {'hin': 'Hindi'}]);
loadAndDisplayOptions('#categoryFilter', [{'stack_exchange': 'stack exchange'}, {'wikipedia': 'wikipedia'}, {'phet': 'phet'}, {'youtube': 'youtube'}]);
for (const key of params.keys()) {
document.getElementsByName(key)[0].value = params.get(key);
}
}
})();

124
static/skin/iso6391To3.js Normal file
View File

@@ -0,0 +1,124 @@
// eslint-disable-next-line no-unused-vars
const iso6391To3 = {
"aa": "aar",
"af": "afr",
"ak": "aka",
"am": "amh",
"ar": "ara",
"as": "asm",
"az": "aze",
"ba": "bak",
"be": "bel",
"bg": "bul",
"bm": "bam",
"bn": "ben",
"bo": "bod",
"br": "bre",
"bs": "bos",
"ca": "cat",
"ce": "che",
"co": "cos",
"cs": "ces",
"cv": "chv",
"cy": "cym",
"da": "dan",
"de": "deu",
"dz": "dzo",
"ee": "ewe",
"en": "eng",
"es": "spa",
"et": "est",
"eu": "eus",
"fa": "fas",
"ff": "ful",
"fi": "fin",
"fo": "fao",
"fr": "fra",
"ga": "gle",
"gl": "glg",
"gn": "grn",
"gu": "guj",
"gv": "glv",
"ha": "hau",
"he": "heb",
"hi": "hin",
"hr": "hrv",
"hu": "hun",
"hy": "hye",
"id": "ind",
"ig": "ibo",
"is": "isl",
"it": "ita",
"iu": "iku",
"ja": "jpn",
"jv": "jav",
"ka": "kat",
"ki": "kik",
"kk": "kaz",
"km": "khm",
"kn": "kan",
"ko": "kor",
"ks": "kas",
"ku": "kur",
"kw": "cor",
"ky": "kir",
"lb": "ltz",
"lg": "lug",
"ln": "lin",
"lo": "lao",
"lt": "lit",
"lv": "lav",
"mg": "mlg",
"mi": "mri",
"mk": "mkd",
"ml": "mal",
"mn": "mon",
"mr": "mar",
"mt": "mlt",
"my": "mya",
"nl": "nld",
"ny": "nya",
"om": "orm",
"pl": "pol",
"pt": "por",
"qu": "que",
"rm": "roh",
"rn": "run",
"ro": "ron",
"ru": "rus",
"rw": "kin",
"sa": "san",
"sd": "snd",
"sg": "sag",
"si": "sin",
"sk": "slk",
"sl": "slv",
"sn": "sna",
"so": "som",
"sq": "sqi",
"sr": "srp",
"ss": "ssw",
"sv": "swe",
"ta": "tam",
"te": "tel",
"tg": "tgk",
"th": "tha",
"ti": "tir",
"tk": "tuk",
"tn": "tsn",
"tr": "tur",
"ts": "tso",
"tt": "tat",
"ug": "uig",
"uk": "ukr",
"ur": "urd",
"uz": "uzb",
"ve": "ven",
"vi": "vie",
"wa": "wln",
"wo": "wol",
"xh": "xho",
"yo": "yor",
"zh": "zho",
"zu": "zul"
}

12
static/skin/isotope.pkgd.min.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

124
static/skin/langList.js Normal file
View File

@@ -0,0 +1,124 @@
// eslint-disable-next-line no-unused-vars
const langList = {
"aar": "Afaraf",
"afr": "Afrikaans",
"aka": "Akan",
"amh": "አማርኛ",
"ara": "اللغة العربية",
"asm": "অসমীয়া",
"aze": "azərbaycan dili",
"bak": "башҡорт теле",
"bel": "беларуская мова",
"bul": "български език",
"bam": "bamanankan",
"ben": "বাংলা",
"bod": "བོད་ཡིག",
"bre": "brezhoneg",
"bos": "bosanski jezik",
"cat": "Català",
"che": "нохчийн мотт",
"cos": "corsu",
"ces": "čeština",
"chv": "чӑваш чӗлхи",
"cym": "Cymraeg",
"dan": "dansk",
"deu": "Deutsch",
"dzo": "རྫོང་ཁ",
"ewe": "Eʋegbe",
"eng": "English",
"spa": "Español",
"est": "eesti",
"eus": "euskara",
"fas": "فارسی",
"ful": "Fulfulde",
"fin": "suomi",
"fao": "føroyskt",
"fra": "Français",
"gle": "Gaeilge",
"glg": "galego",
"grn": "Avañe'ẽ",
"guj": "ગુજરાતી",
"glv": "Gaelg",
"hau": "هَوُسَ",
"heb": "עברית",
"hin": "हिन्दी",
"hrv": "hrvatski jezik",
"hun": "magyar",
"hye": "Հայերեն",
"ind": "Bahasa Indonesia",
"ibo": "Asụsụ Igbo",
"isl": "Íslenska",
"ita": "Italiano",
"iku": "ᐃᓄᒃᑎᑐᑦ",
"jpn": "日本語",
"jav": "basa Jawa",
"kat": "ქართული",
"kik": "Gĩkũyũ",
"kaz": "қазақ тілі",
"khm": "ខេមរភាសា",
"kan": "ಕನ್ನಡ",
"kor": "한국어",
"kas": "कश्मीरी",
"kur": "Kurdî",
"cor": "Kernewek",
"kir": "Кыргызча",
"ltz": "Lëtzebuergesch",
"lug": "Luganda",
"lin": "Lingála",
"lao": "ພາສາ",
"lit": "lietuvių kalba",
"lav": "latviešu valoda",
"mlg": "fiteny malagasy",
"mri": "te reo Māori",
"mkd": "македонски јазик",
"mal": "മലയാളം",
"mon": "Монгол хэл",
"mar": "मराठी",
"mlt": "Malti",
"mya": "ဗမာစာ",
"nld": "Nederlands",
"nya": "chiCheŵa",
"orm": "Afaan Oromoo",
"pol": "język polski",
"por": "Português",
"que": "Runa Simi",
"roh": "rumantsch grischun",
"run": "Ikirundi",
"ron": "Română",
"rus": "Русский",
"kin": "Ikinyarwanda",
"san": "संस्कृतम्",
"snd": "सिन्धी",
"sag": "yângâ tî sängö",
"sin": "සිංහල",
"slk": "slovenčina",
"slv": "slovenski jezik",
"sna": "chiShona",
"som": "Soomaaliga",
"sqi": "Shqip",
"srp": "српски језик",
"ssw": "SiSwati",
"swe": "svenska",
"tam": "தமிழ்",
"tel": "తెలుగు",
"tgk": "тоҷикӣ",
"tha": "ไทย",
"tir": "ትግርኛ",
"tuk": "Türkmen",
"tsn": "Setswana",
"tur": "Türkçe",
"tso": "Xitsonga",
"tat": "татар теле",
"uig": "ئۇيغۇرچە‎",
"ukr": "Українська",
"urd": "اردو",
"uzb": "Ўзбек",
"ven": "Tshivenḓa",
"vie": "Tiếng Việt",
"wln": "walon",
"wol": "Wollof",
"xho": "isiXhosa",
"yor": "Yorùbá",
"zho": "中文",
"zul": "isiZulu"
}

View File

@@ -7,7 +7,10 @@ const jq = jQuery.noConflict(true);
jq(document).ready(() => {
(function ($) {
const root = $( `link[type='root']` ).attr("href");
const bookName = window.location.pathname.split(`${root}/`)[1].split('/')[0];
const bookName = (window.location.pathname == `${root}/search`)
? (new URLSearchParams(window.location.search)).get('content')
: window.location.pathname.split(`${root}/`)[1].split('/')[0];
$( "#kiwixsearchbox" ).autocomplete({
@@ -31,7 +34,12 @@ jq(document).ready(() => {
$( "#kiwixsearchform" ).submit();
}
},
});
}).data( "ui-autocomplete" )._renderItem = function( ul, item ) {
return $( "<li>" )
.data( "ui-autocomplete-item", item )
.append( item.label )
.appendTo( ul );
};
/* cybook hack */
if (navigator.userAgent.indexOf("bookeen/cybook") != -1) {
@@ -85,4 +93,4 @@ jq(document).ready(() => {
}
});
})(jq);
})
})

View File

@@ -0,0 +1,38 @@
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:opds="http://opds-spec.org/2010/catalog">
<id>{{feed_id}}</id>
<title>{{^filter}}All zims{{/filter}}{{#filter}}Filtered zims ({{filter}}){{/filter}}</title>
<updated>{{date}}</updated>
{{#filter}}
<totalResults>{{totalResults}}</totalResults>
<startIndex>{{startIndex}}</startIndex>
<itemsPerPage>{{itemsPerPage}}</itemsPerPage>
{{/filter}}
<link rel="self" href="" type="application/atom+xml" />
<link rel="search" type="application/opensearchdescription+xml" href="{{root}}/catalog/searchdescription.xml" />
{{#books}}
<entry>
<id>{{id}}</id>
<title>{{title}}</title>
<summary>{{description}}</summary>
<language>{{language}}</language>
<updated>{{updated}}</updated>
<name>{{name}}</name>
<flavour>{{flavour}}</flavour>
<category>{{category}}</category>
<tags>{{tags}}</tags>
<articleCount>{{article_count}}</articleCount>
<mediaCount>{{media_count}}</mediaCount>
<icon>/meta?name=favicon&amp;content={{{content_id}}}</icon>
<link type="text/html" href="/{{{content_id}}}" />
<author>
<name>{{author_name}}</name>
</author>
<publisher>
<name>{{publisher_name}}</name>
</publisher>
{{#url}}
<link rel="http://opds-spec.org/acquisition/open-access" type="application/x-zim" href="{{{url}}}" length="{{{size}}}" />
{{/url}}
</entry>
{{/books}}
</feed>

View File

@@ -0,0 +1,25 @@
<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:opds="https://specs.opds.io/opds-1.2">
<id>{{feed_id}}</id>
<link rel="self"
href="{{endpoint_root}}/categories"
type="application/atom+xml;profile=opds-catalog;kind=navigation"/>
<link rel="start"
href="{{endpoint_root}}/root.xml"
type="application/atom+xml;profile=opds-catalog;kind=navigation"/>
<title>List of categories</title>
<updated>{{date}}</updated>
{{#categories}}
<entry>
<title>{{name}}</title>
<link rel="subsection"
href="{{endpoint_root}}/entries?category={{{urlencoded_name}}}"
type="application/atom+xml;profile=opds-catalog;kind=acquisition"/>
<updated>{{updated}}</updated>
<id>{{id}}</id>
<content type="text">All entries with category of '{{name}}'.</content>
</entry>
{{/categories}}
</feed>

View File

@@ -0,0 +1,50 @@
<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:opds="https://specs.opds.io/opds-1.2"
xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">
<id>{{feed_id}}</id>
<link rel="self"
href="{{endpoint_root}}/entries{{{query}}}"
type="application/atom+xml;profile=opds-catalog;kind=acquisition"/>
<link rel="start"
href="{{endpoint_root}}/root.xml"
type="application/atom+xml;profile=opds-catalog;kind=navigation"/>
<link rel="up"
href="{{endpoint_root}}/root.xml"
type="application/atom+xml;profile=opds-catalog;kind=navigation"/>
<title>{{^filter}}All Entries{{/filter}}{{#filter}}Filtered Entries ({{filter}}){{/filter}}</title>
<updated>{{date}}</updated>
{{#filter}}
<totalResults>{{totalResults}}</totalResults>
<startIndex>{{startIndex}}</startIndex>
<itemsPerPage>{{itemsPerPage}}</itemsPerPage>
{{/filter}}
{{#books}}
<entry>
<id>{{id}}</id>
<title>{{title}}</title>
<summary>{{description}}</summary>
<language>{{language}}</language>
<updated>{{updated}}</updated>
<name>{{name}}</name>
<flavour>{{flavour}}</flavour>
<category>{{category}}</category>
<tags>{{tags}}</tags>
<articleCount>{{article_count}}</articleCount>
<mediaCount>{{media_count}}</mediaCount>
<icon>/meta?name=favicon&amp;content={{{content_id}}}</icon>
<link type="text/html" href="/{{{content_id}}}" />
<author>
<name>{{author_name}}</name>
</author>
<publisher>
<name>{{publisher_name}}</name>
</publisher>
{{#url}}
<link rel="http://opds-spec.org/acquisition/open-access" type="application/x-zim" href="{{{url}}}" length="{{{size}}}" />
{{/url}}
</entry>
{{/books}}
</feed>

View File

@@ -0,0 +1,35 @@
<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:opds="https://specs.opds.io/opds-1.2">
<id>{{feed_id}}</id>
<link rel="self"
href="{{endpoint_root}}/root.xml"
type="application/atom+xml;profile=opds-catalog;kind=navigation"/>
<link rel="start"
href="{{endpoint_root}}/root.xml"
type="application/atom+xml;profile=opds-catalog;kind=navigation"/>
<link rel="search"
href="{{endpoint_root}}/searchdescription.xml"
type="application/opensearchdescription+xml"/>
<title>OPDS Catalog Root</title>
<updated>{{date}}</updated>
<entry>
<title>All entries</title>
<link rel="subsection"
href="{{endpoint_root}}/entries"
type="application/atom+xml;profile=opds-catalog;kind=acquisition"/>
<updated>{{date}}</updated>
<id>{{all_entries_feed_id}}</id>
<content type="text">All entries from this catalog.</content>
</entry>
<entry>
<title>List of categories</title>
<link rel="subsection"
href="{{endpoint_root}}/categories"
type="application/atom+xml;profile=opds-catalog;kind=navigation"/>
<updated>{{date}}</updated>
<id>{{category_list_feed_id}}</id>
<content type="text">List of all categories in this catalog.</content>
</entry>
</feed>

View File

@@ -1,63 +1,206 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width">
<title>Welcome to Kiwix Server</title>
<script type="text/javascript" src="{{root}}/skin/jquery-ui/external/jquery/jquery.js"></script>
<script type="text/javascript" src="{{root}}/skin/jquery-ui/jquery-ui.min.js"></script>
<link type="text/css" href="{{root}}/skin/jquery-ui/jquery-ui.min.css" rel="Stylesheet" />
<link type="text/css" href="{{root}}/skin/jquery-ui/jquery-ui.theme.min.css" rel="Stylesheet" />
<style>
body {
background:
radial-gradient(#EEEEEE 15%, transparent 16%) 0 0,
radial-gradient(#EEEEEE 15%, transparent 16%) 8px 8px,
radial-gradient(rgba(255,255,255,.1) 15%, transparent 20%) 0 1px,
radial-gradient(rgba(255,255,255,.1) 15%, transparent 20%) 8px 9px;
background-color:#E8E8E8;
background-size:16px 16px;
margin-left: auto;
margin-right: auto;
max-width: 1100px;
}
.book__list { text-align: center; }
.book {
display: inline-block; vertical-align: bottom; margin: 8px; padding: 12px 15px; width: 300px;
border: 1px solid #ccc; border-radius: 8px;
text-align: left; color: #000; font-family: sans-serif; font-size: 13px;
background-color:#F1F1F1;
box-shadow: 2px 2px 5px 0px #ccc;
}
.book:hover { background-color: #F9F9F9; box-shadow: none;}
.book__background { background-repeat: no-repeat; background-size: 48px 48px; background-position: top right; }
.book__title {
padding: 0 55px 0 0;overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
font-size: 18px; color: #0645ad; line-height: 1em;
}
.book__description {
padding: 5px 55px 5px 0px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
font-size: 15px; line-height: 1em;
}
.book__info { color: #777; font-weight: bold; font-size: 13px; line-height: 1em; }
</style>
<script type="text/javascript" src="{{root}}/skin/index.js" defer></script>
</head>
<body class="kiwix">
<div class='kiwixHomeNavbar'>
<select name="lang" id="languageFilter" onchange="filterBooks('lang', this.value)">
<option value="" disabled selected>Languages</option>
</select>
<select name="category" id="categoryFilter" onchange="filterBooks('category', this.value)">
<option value="" disabled selected>Categories</option>
</select>
<input type="text" name="q" id="searchFilter" onchange="filterBooks('q', this.value)">
</div>
<div class="kiwix">
<div class='book__list'></div>
</div>
<div id="kiwixfooter">
Powered by <a href="https://kiwix.org">Kiwix</a>
</div>
</body>
<head>
<meta charset="UTF-8" />
<title>Welcome to Kiwix Server</title>
<script
type="text/javascript"
src="{{root}}/skin/jquery-ui/external/jquery/jquery.js"
></script>
<script
type="text/javascript"
src="{{root}}/skin/jquery-ui/jquery-ui.min.js"
></script>
<link
type="text/css"
href="{{root}}/skin/jquery-ui/jquery-ui.min.css"
rel="Stylesheet"
/>
<link
type="text/css"
href="{{root}}/skin/jquery-ui/jquery-ui.theme.min.css"
rel="Stylesheet"
/>
<style>
html {
min-height: 100%;
position: relative;
}
body {
background: radial-gradient(#eeeeee 15%, transparent 16%) 0 0,
radial-gradient(#eeeeee 15%, transparent 16%) 8px 8px,
radial-gradient(rgba(255, 255, 255, 0.1) 15%, transparent 20%) 0 1px,
radial-gradient(rgba(255, 255, 255, 0.1) 15%, transparent 20%) 8px 9px;
background-color: #e8e8e8;
background-size: 16px 16px;
margin-left: auto;
margin-right: auto;
max-width: 1100px;
min-height: 100%;
}
.book__list {
text-align: center;
}
.kiwixHomeBody {
position: relative;
text-align: center;
min-height: 100%;
margin: 0 0 15px;
}
.book {
display: inline-block;
vertical-align: bottom;
margin: 8px;
padding: 12px 15px;
width: 300px;
border: 1px solid #ccc;
border-radius: 8px;
text-align: left;
color: #000;
font-family: sans-serif;
font-size: 13px;
background-color: #f1f1f1;
box-shadow: 2px 2px 5px 0 #ccc;
}
#kiwixfooter {
text-align: center;
margin: 0.5em;
position: absolute;
bottom: 0;
left: 46%;
}
.kiwixHomeNavbar {
display: flex;
justify-content: center;
}
.kiwixFilter {
margin: 8px 10px;
}
.kiwixSearch, .searchButton {
margin: 0 13px 0 0;
}
.kiwixSearchForm {
margin: 8px 10px;
float: right;
}
@media (max-width: 1100px) {
.kiwixHomeBody {
padding: 0 125px;
}
}
.book:hover {
background-color: #f9f9f9;
box-shadow: none;
}
.book__background {
background-repeat: no-repeat;
background-size: 48px 48px;
background-position: top right;
}
.book__title {
padding: 0 55px 0 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
font-size: 18px;
color: #0645ad;
line-height: 1em;
}
.book__description {
padding: 5px 55px 5px 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
font-size: 15px;
line-height: 1em;
}
.book__info {
color: #777;
font-weight: bold;
font-size: 13px;
line-height: 1em;
}
a:link {
text-decoration: none;
}
a:visited {
text-decoration: none;
}
.noResults {
position: absolute;
top: 48%;
left: 42%;
}
.loader-spinner {
position: absolute;
top: -50%;
left: 50%;
border: 5px solid #f3f3f3;
border-radius: 50%;
border-top: 5px solid #3498db;
width: 40px;
height: 40px;
margin: auto;
-webkit-animation: spin 1s linear infinite; /* Safari */
animation: spin 1s linear infinite;
margin-top: 35px;
margin-bottom: -35px;
z-index: 2;
}
/* Safari */
@-webkit-keyframes spin {
0% { -webkit-transform: rotate(0deg); }
100% { -webkit-transform: rotate(360deg); }
}
@keyframes spin {
0% { transform: rotate(0deg); }
100% { transform: rotate(360deg); }
}
.loader {
position: relative;
height: 70px;
width: 100%;
}
.fadeOut {
position: fixed;
display: none;
bottom: 0;
left: 0;
z-index: 1;
background: linear-gradient(180deg, rgba(232, 232, 232, 0) 0%, rgb(232, 232, 232) 100%);
height: 80px;
width: 100%;
}
.spacer {
height: 20px;
background: transparent;
}
</style>
<script src="{{root}}/skin/isotope.pkgd.min.js" defer></script>
<script src="{{root}}/skin/categoryList.js"></script>
<script src="{{root}}/skin/langList.js"></script>
<script src="{{root}}/skin/iso6391To3.js"></script>
<script type="text/javascript" src="{{root}}/skin/index.js" defer></script>
</head>
<body class="kiwix">
<div class='kiwixHomeNavbar'>
<select name="lang" id="languageFilter" class='kiwixFilter'>
<option value="" selected>All languages</option>
</select>
<select name="category" id="categoryFilter" class='kiwixFilter'>
<option value="" selected>All categories</option>
</select>
<form id='kiwixSearchForm' class='kiwixSearchForm'>
<input type="text" name="q" id="searchFilter" class='kiwixSearch'>
<input type="submit" class="searchButton" value="Search"/>
</form>
</div>
<div class="kiwixHomeBody">
<div class="book__list"></div>
<div id="fadeOut" class="fadeOut"></div>
</div>
<div class="loader"><div class="loader-spinner"></div></div>
<div class="spacer"></div>
<div id="kiwixfooter">Powered by <a href="https://kiwix.org">Kiwix</a></div>
</body>
</html>

View File

@@ -8,18 +8,18 @@
<input autocomplete="off" class="ui-autocomplete-input" id="kiwixsearchbox" name="pattern" type="text">
</form>
</div>
{{#hascontent}}
<input type="checkbox" id="kiwix_button_show_toggle">
<label for="kiwix_button_show_toggle"><img src="{{root}}/skin/caret.png" alt=""></label>
<div class="kiwix_button_cont">
{{#withlibrarybutton}}
<a id="kiwix_serve_taskbar_library_button" href="{{root}}/"><button>&#x1f3e0;</button></a>
{{/withlibrarybutton}}
{{#hascontent}}
<a id="kiwix_serve_taskbar_home_button" href="{{root}}/{{content}}/"><button>{{title}}</button></a>
<a id="kiwix_serve_taskbar_random_button"
href="{{root}}/random?content={{#urlencoded}}{{{content}}}{{/urlencoded}}"><button>&#x1F3B2;</button></a>
{{/hascontent}}
</div>
{{/hascontent}}
</div>
</span>
</span>

View File

@@ -2,7 +2,7 @@
<book
id="raycharles"
path="./zimfile.zim"
url="https://github.com/kiwix/kiwix-lib/raw/master/test/data/zimfile.zim"
url="https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile.zim"
title="Ray Charles"
description="Wikipedia articles about Ray Charles"
language="eng"
@@ -18,7 +18,7 @@
<book
id="raycharles_uncategorized"
path="./zimfile.zim"
url="https://github.com/kiwix/kiwix-lib/raw/master/test/data/zimfile.zim"
url="https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile.zim"
title="Ray (uncategorized) Charles"
description="No category is assigned to this library entry."
language="eng"
@@ -34,7 +34,7 @@
<book
id="charlesray"
path="./zimfile.zim"
url="https://github.com/kiwix/kiwix-lib/raw/master/test/data/zimfile.zim"
url="https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile.zim"
title="Charles, Ray"
description="Wikipedia articles about Ray Charles"
language="eng"

View File

@@ -2890,12 +2890,15 @@ inline ssize_t Stream::write(const std::string &s) {
template <typename... Args>
inline ssize_t Stream::write_format(const char *fmt, const Args &... args) {
std::array<char, 2048> buf;
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wunknown-warning-option"
#pragma GCC diagnostic ignored "-Wmaybe-uninitialized"
#if defined(_MSC_VER) && _MSC_VER < 1900
auto sn = _snprintf_s(buf, bufsiz, buf.size() - 1, fmt, args...);
#else
auto sn = snprintf(buf.data(), buf.size() - 1, fmt, args...);
#endif
#pragma GCC diagnostic pop
if (sn <= 0) { return sn; }
auto n = static_cast<size_t>(sn);
@@ -3651,7 +3654,11 @@ Server::process_request(Stream &strm, bool last_connection,
const std::function<void(Request &)> &setup_request) {
std::array<char, 2048> buf{};
detail::stream_line_reader line_reader(strm, buf.data(), buf.size());
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wunknown-warning-option"
#pragma GCC diagnostic ignored "-Wmaybe-uninitialized"
detail::stream_line_reader line_reader(strm, buf.data(), buf.size());
#pragma GCC diagnostic pop
// Connection has been closed on client
if (!line_reader.getline()) { return false; }
@@ -3757,9 +3764,11 @@ inline socket_t Client::create_client_socket() const {
inline bool Client::read_response_line(Stream &strm, Response &res) {
std::array<char, 2048> buf;
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wunknown-warning-option"
#pragma GCC diagnostic ignored "-Wmaybe-uninitialized"
detail::stream_line_reader line_reader(strm, buf.data(), buf.size());
#pragma GCC diagnostic pop
if (!line_reader.getline()) { return false; }
const static std::regex re("(HTTP/1\\.[01]) (\\d+?) .*\r\n");

View File

@@ -186,7 +186,7 @@ const char sampleLibraryXML[] = R"(
<book
id="raycharles"
path="./zimfile.zim"
url="https://github.com/kiwix/kiwix-lib/raw/master/test/data/zimfile.zim"
url="https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile.zim"
title="Ray Charles"
description="Wikipedia articles about Ray Charles"
language="eng"

View File

@@ -240,6 +240,7 @@ const char* urls404[] = {
"/skin/non-existent-skin-resource",
"/catalog",
"/catalog/non-existent-item",
"/catalogBLABLABLA/root.xml",
"/meta",
"/meta?content=zimfile",
"/meta?content=zimfile&name=non-existent-item",
@@ -570,18 +571,21 @@ protected:
}
};
// Returns a copy of 'text' with every line that fully matches 'pattern'
// replaced with the fixed string 'replacement'
// Returns a copy of 'text' where every line that fully matches 'pattern'
// preceded by optional whitespace is replaced with the fixed string
// 'replacement' preserving the leading whitespace
std::string replaceLines(const std::string& text,
const std::string& pattern,
const std::string& replacement)
{
std::regex regex("^" + pattern + "$");
std::regex regex("^ *" + pattern + "$");
std::ostringstream oss;
std::istringstream iss(text);
std::string line;
while ( std::getline(iss, line) ) {
if ( std::regex_match(line, regex) ) {
for ( size_t i = 0; i < line.size() && line[i] == ' '; ++i )
oss << ' ';
oss << replacement << "\n";
} else {
oss << line << "\n";
@@ -592,10 +596,10 @@ std::string replaceLines(const std::string& text,
std::string maskVariableOPDSFeedData(std::string s)
{
s = replaceLines(s, " <updated>.+</updated>",
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>");
s = replaceLines(s, " <id>.+</id>",
" <id>12345678-90ab-cdef-1234-567890abcdef</id>");
s = replaceLines(s, R"(<updated>\d\d\d\d-\d\d-\d\dT\d\d:\d\d:\d\dZ</updated>)",
"<updated>YYYY-MM-DDThh:mm:ssZ</updated>");
s = replaceLines(s, "<id>[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}</id>",
"<id>12345678-90ab-cdef-1234-567890abcdef</id>");
return s;
}
@@ -607,7 +611,7 @@ std::string maskVariableOPDSFeedData(std::string s)
" <link rel=\"self\" href=\"\" type=\"application/atom+xml\" />\n" \
" <link rel=\"search\"" \
" type=\"application/opensearchdescription+xml\"" \
" href=\"catalog/searchdescription.xml\" />\n"
" href=\"/catalog/searchdescription.xml\" />\n"
#define CHARLES_RAY_CATALOG_ENTRY \
" <entry>\n" \
@@ -615,7 +619,7 @@ std::string maskVariableOPDSFeedData(std::string s)
" <title>Charles, Ray</title>\n" \
" <summary>Wikipedia articles about Ray Charles</summary>\n" \
" <language>eng</language>\n" \
" <updated>2020-03-31T00:00::00Z</updated>\n" \
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n" \
" <name>wikipedia_en_ray_charles</name>\n" \
" <flavour></flavour>\n" \
" <category>jazz</category>\n" \
@@ -630,7 +634,7 @@ std::string maskVariableOPDSFeedData(std::string s)
" <publisher>\n" \
" <name>Kiwix</name>\n" \
" </publisher>\n" \
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"https://github.com/kiwix/kiwix-lib/raw/master/test/data/zimfile.zim\" length=\"569344\" />\n" \
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile.zim\" length=\"569344\" />\n" \
" </entry>\n"
#define RAY_CHARLES_CATALOG_ENTRY \
@@ -639,7 +643,7 @@ std::string maskVariableOPDSFeedData(std::string s)
" <title>Ray Charles</title>\n" \
" <summary>Wikipedia articles about Ray Charles</summary>\n" \
" <language>eng</language>\n" \
" <updated>2020-03-31T00:00::00Z</updated>\n" \
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n" \
" <name>wikipedia_en_ray_charles</name>\n" \
" <flavour></flavour>\n" \
" <category>wikipedia</category>\n" \
@@ -654,7 +658,7 @@ std::string maskVariableOPDSFeedData(std::string s)
" <publisher>\n" \
" <name>Kiwix</name>\n" \
" </publisher>\n" \
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"https://github.com/kiwix/kiwix-lib/raw/master/test/data/zimfile.zim\" length=\"569344\" />\n" \
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile.zim\" length=\"569344\" />\n" \
" </entry>\n"
#define UNCATEGORIZED_RAY_CHARLES_CATALOG_ENTRY \
@@ -663,7 +667,7 @@ std::string maskVariableOPDSFeedData(std::string s)
" <title>Ray (uncategorized) Charles</title>\n" \
" <summary>No category is assigned to this library entry.</summary>\n" \
" <language>eng</language>\n" \
" <updated>2020-03-31T00:00::00Z</updated>\n" \
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n" \
" <name>wikipedia_en_ray_charles</name>\n" \
" <flavour></flavour>\n" \
" <category></category>\n" \
@@ -678,7 +682,7 @@ std::string maskVariableOPDSFeedData(std::string s)
" <publisher>\n" \
" <name>Kiwix</name>\n" \
" </publisher>\n" \
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"https://github.com/kiwix/kiwix-lib/raw/master/test/data/zimfile.zim\" length=\"125952\" />\n" \
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile.zim\" length=\"125952\" />\n" \
" </entry>\n"
TEST_F(LibraryServerTest, catalog_root_xml)
@@ -690,6 +694,7 @@ TEST_F(LibraryServerTest, catalog_root_xml)
" <id>12345678-90ab-cdef-1234-567890abcdef</id>\n"
" <title>All zims</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
"\n"
CATALOG_LINK_TAGS
CHARLES_RAY_CATALOG_ENTRY
RAY_CHARLES_CATALOG_ENTRY
@@ -711,7 +716,7 @@ TEST_F(LibraryServerTest, catalog_searchdescription_xml)
" xmlns:atom=\"http://www.w3.org/2005/Atom\"\n"
" xmlns:k=\"http://kiwix.org/opensearchextension/1.0\"\n"
" indexOffset=\"0\"\n"
" template=\"//catalog/search?q={searchTerms?}&lang={language?}&name={k:name?}&tag={k:tag?}&notag={k:notag?}&maxsize={k:maxsize?}&count={count?}&start={startIndex?}\"/>\n"
" template=\"/catalog/search?q={searchTerms?}&lang={language?}&name={k:name?}&tag={k:tag?}&notag={k:notag?}&maxsize={k:maxsize?}&count={count?}&start={startIndex?}\"/>\n"
"</OpenSearchDescription>\n"
);
}
@@ -723,7 +728,7 @@ TEST_F(LibraryServerTest, catalog_search_by_phrase)
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
OPDS_FEED_TAG
" <id>12345678-90ab-cdef-1234-567890abcdef</id>\n"
" <title>Search result for \"ray charles\"</title>\n"
" <title>Filtered zims (q=&quot;ray charles&quot;)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>2</totalResults>\n"
" <startIndex>0</startIndex>\n"
@@ -742,7 +747,7 @@ TEST_F(LibraryServerTest, catalog_search_by_words)
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
OPDS_FEED_TAG
" <id>12345678-90ab-cdef-1234-567890abcdef</id>\n"
" <title>Search result for ray charles</title>\n"
" <title>Filtered zims (q=ray charles)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>3</totalResults>\n"
" <startIndex>0</startIndex>\n"
@@ -763,7 +768,7 @@ TEST_F(LibraryServerTest, catalog_prefix_search)
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
OPDS_FEED_TAG
" <id>12345678-90ab-cdef-1234-567890abcdef</id>\n"
" <title>Search result for description:ray description:charles</title>\n"
" <title>Filtered zims (q=description:ray description:charles)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>2</totalResults>\n"
" <startIndex>0</startIndex>\n"
@@ -780,7 +785,7 @@ TEST_F(LibraryServerTest, catalog_prefix_search)
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
OPDS_FEED_TAG
" <id>12345678-90ab-cdef-1234-567890abcdef</id>\n"
" <title>Search result for title:\"ray charles\"</title>\n"
" <title>Filtered zims (q=title:&quot;ray charles&quot;)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>1</totalResults>\n"
" <startIndex>0</startIndex>\n"
@@ -799,7 +804,7 @@ TEST_F(LibraryServerTest, catalog_search_with_word_exclusion)
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
OPDS_FEED_TAG
" <id>12345678-90ab-cdef-1234-567890abcdef</id>\n"
" <title>Search result for ray -uncategorized</title>\n"
" <title>Filtered zims (q=ray -uncategorized)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>2</totalResults>\n"
" <startIndex>0</startIndex>\n"
@@ -818,7 +823,7 @@ TEST_F(LibraryServerTest, catalog_search_by_tag)
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
OPDS_FEED_TAG
" <id>12345678-90ab-cdef-1234-567890abcdef</id>\n"
" <title>Search result for &lt;Empty query&gt;</title>\n"
" <title>Filtered zims (tag=_category:jazz)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>1</totalResults>\n"
" <startIndex>0</startIndex>\n"
@@ -836,7 +841,7 @@ TEST_F(LibraryServerTest, catalog_search_by_category)
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
OPDS_FEED_TAG
" <id>12345678-90ab-cdef-1234-567890abcdef</id>\n"
" <title>Search result for &lt;Empty query&gt;</title>\n"
" <title>Filtered zims (category=jazz)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>1</totalResults>\n"
" <startIndex>0</startIndex>\n"
@@ -855,7 +860,7 @@ TEST_F(LibraryServerTest, catalog_search_results_pagination)
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
OPDS_FEED_TAG
" <id>12345678-90ab-cdef-1234-567890abcdef</id>\n"
" <title>Search result for &lt;Empty query&gt;</title>\n"
" <title>Filtered zims (count=1)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>3</totalResults>\n"
" <startIndex>0</startIndex>\n"
@@ -871,7 +876,7 @@ TEST_F(LibraryServerTest, catalog_search_results_pagination)
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
OPDS_FEED_TAG
" <id>12345678-90ab-cdef-1234-567890abcdef</id>\n"
" <title>Search result for &lt;Empty query&gt;</title>\n"
" <title>Filtered zims (count=1&amp;start=1)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>3</totalResults>\n"
" <startIndex>1</startIndex>\n"
@@ -887,13 +892,217 @@ TEST_F(LibraryServerTest, catalog_search_results_pagination)
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
OPDS_FEED_TAG
" <id>12345678-90ab-cdef-1234-567890abcdef</id>\n"
" <title>Search result for &lt;Empty query&gt;</title>\n"
" <title>Filtered zims (count=10&amp;start=100)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>3</totalResults>\n"
" <startIndex>100</startIndex>\n"
" <itemsPerPage>0</itemsPerPage>\n"
CATALOG_LINK_TAGS
" \n"
"</feed>\n"
);
}
}
TEST_F(LibraryServerTest, catalog_v2_root)
{
const auto r = zfs1_->GET("/catalog/v2/root.xml");
EXPECT_EQ(r->status, 200);
const char expected_output[] = R"(<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:opds="https://specs.opds.io/opds-1.2">
<id>12345678-90ab-cdef-1234-567890abcdef</id>
<link rel="self"
href="/catalog/v2/root.xml"
type="application/atom+xml;profile=opds-catalog;kind=navigation"/>
<link rel="start"
href="/catalog/v2/root.xml"
type="application/atom+xml;profile=opds-catalog;kind=navigation"/>
<link rel="search"
href="/catalog/v2/searchdescription.xml"
type="application/opensearchdescription+xml"/>
<title>OPDS Catalog Root</title>
<updated>YYYY-MM-DDThh:mm:ssZ</updated>
<entry>
<title>All entries</title>
<link rel="subsection"
href="/catalog/v2/entries"
type="application/atom+xml;profile=opds-catalog;kind=acquisition"/>
<updated>YYYY-MM-DDThh:mm:ssZ</updated>
<id>12345678-90ab-cdef-1234-567890abcdef</id>
<content type="text">All entries from this catalog.</content>
</entry>
<entry>
<title>List of categories</title>
<link rel="subsection"
href="/catalog/v2/categories"
type="application/atom+xml;profile=opds-catalog;kind=navigation"/>
<updated>YYYY-MM-DDThh:mm:ssZ</updated>
<id>12345678-90ab-cdef-1234-567890abcdef</id>
<content type="text">List of all categories in this catalog.</content>
</entry>
</feed>
)";
EXPECT_EQ(maskVariableOPDSFeedData(r->body), expected_output);
}
TEST_F(LibraryServerTest, catalog_v2_searchdescription_xml)
{
const auto r = zfs1_->GET("/catalog/v2/searchdescription.xml");
EXPECT_EQ(r->status, 200);
EXPECT_EQ(r->body,
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
"<OpenSearchDescription xmlns=\"http://a9.com/-/spec/opensearch/1.1/\">\n"
" <ShortName>Zim catalog search</ShortName>\n"
" <Description>Search zim files in the catalog.</Description>\n"
" <Url type=\"application/atom+xml;profile=opds-catalog;kind=acquisition\"\n"
" xmlns:atom=\"http://www.w3.org/2005/Atom\"\n"
" xmlns:k=\"http://kiwix.org/opensearchextension/1.0\"\n"
" indexOffset=\"0\"\n"
" template=\"/catalog/v2/entries?q={searchTerms?}&lang={language?}&name={k:name?}&tag={k:tag?}&maxsize={k:maxsize?}&count={count?}&start={startIndex?}\"/>\n"
"</OpenSearchDescription>\n"
);
}
TEST_F(LibraryServerTest, catalog_v2_categories)
{
const auto r = zfs1_->GET("/catalog/v2/categories");
EXPECT_EQ(r->status, 200);
const char expected_output[] = R"(<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:opds="https://specs.opds.io/opds-1.2">
<id>12345678-90ab-cdef-1234-567890abcdef</id>
<link rel="self"
href="/catalog/v2/categories"
type="application/atom+xml;profile=opds-catalog;kind=navigation"/>
<link rel="start"
href="/catalog/v2/root.xml"
type="application/atom+xml;profile=opds-catalog;kind=navigation"/>
<title>List of categories</title>
<updated>YYYY-MM-DDThh:mm:ssZ</updated>
<entry>
<title>jazz</title>
<link rel="subsection"
href="/catalog/v2/entries?category=jazz"
type="application/atom+xml;profile=opds-catalog;kind=acquisition"/>
<updated>YYYY-MM-DDThh:mm:ssZ</updated>
<id>12345678-90ab-cdef-1234-567890abcdef</id>
<content type="text">All entries with category of 'jazz'.</content>
</entry>
<entry>
<title>wikipedia</title>
<link rel="subsection"
href="/catalog/v2/entries?category=wikipedia"
type="application/atom+xml;profile=opds-catalog;kind=acquisition"/>
<updated>YYYY-MM-DDThh:mm:ssZ</updated>
<id>12345678-90ab-cdef-1234-567890abcdef</id>
<content type="text">All entries with category of 'wikipedia'.</content>
</entry>
</feed>
)";
EXPECT_EQ(maskVariableOPDSFeedData(r->body), expected_output);
}
#define CATALOG_V2_ENTRIES_PREAMBLE(q) \
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" \
"<feed xmlns=\"http://www.w3.org/2005/Atom\"\n" \
" xmlns:opds=\"https://specs.opds.io/opds-1.2\"\n" \
" xmlns:opensearch=\"http://a9.com/-/spec/opensearch/1.1/\">\n" \
" <id>12345678-90ab-cdef-1234-567890abcdef</id>\n" \
"\n" \
" <link rel=\"self\"\n" \
" href=\"/catalog/v2/entries" q "\"\n" \
" type=\"application/atom+xml;profile=opds-catalog;kind=acquisition\"/>\n" \
" <link rel=\"start\"\n" \
" href=\"/catalog/v2/root.xml\"\n" \
" type=\"application/atom+xml;profile=opds-catalog;kind=navigation\"/>\n" \
" <link rel=\"up\"\n" \
" href=\"/catalog/v2/root.xml\"\n" \
" type=\"application/atom+xml;profile=opds-catalog;kind=navigation\"/>\n" \
"\n" \
TEST_F(LibraryServerTest, catalog_v2_entries)
{
const auto r = zfs1_->GET("/catalog/v2/entries");
EXPECT_EQ(r->status, 200);
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
CATALOG_V2_ENTRIES_PREAMBLE("")
" <title>All Entries</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
"\n"
CHARLES_RAY_CATALOG_ENTRY
RAY_CHARLES_CATALOG_ENTRY
UNCATEGORIZED_RAY_CHARLES_CATALOG_ENTRY
"</feed>\n"
);
}
TEST_F(LibraryServerTest, catalog_v2_entries_filtered_by_range)
{
{
const auto r = zfs1_->GET("/catalog/v2/entries?start=1");
EXPECT_EQ(r->status, 200);
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
CATALOG_V2_ENTRIES_PREAMBLE("?start=1")
" <title>Filtered Entries (start=1)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>3</totalResults>\n"
" <startIndex>1</startIndex>\n"
" <itemsPerPage>2</itemsPerPage>\n"
RAY_CHARLES_CATALOG_ENTRY
UNCATEGORIZED_RAY_CHARLES_CATALOG_ENTRY
"</feed>\n"
);
}
{
const auto r = zfs1_->GET("/catalog/v2/entries?count=2");
EXPECT_EQ(r->status, 200);
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
CATALOG_V2_ENTRIES_PREAMBLE("?count=2")
" <title>Filtered Entries (count=2)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>3</totalResults>\n"
" <startIndex>0</startIndex>\n"
" <itemsPerPage>2</itemsPerPage>\n"
CHARLES_RAY_CATALOG_ENTRY
RAY_CHARLES_CATALOG_ENTRY
"</feed>\n"
);
}
{
const auto r = zfs1_->GET("/catalog/v2/entries?start=1&count=1");
EXPECT_EQ(r->status, 200);
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
CATALOG_V2_ENTRIES_PREAMBLE("?count=1&start=1")
" <title>Filtered Entries (count=1&amp;start=1)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>3</totalResults>\n"
" <startIndex>1</startIndex>\n"
" <itemsPerPage>1</itemsPerPage>\n"
RAY_CHARLES_CATALOG_ENTRY
"</feed>\n"
);
}
}
TEST_F(LibraryServerTest, catalog_v2_entries_filtered_by_search_terms)
{
const auto r = zfs1_->GET("/catalog/v2/entries?q=\"ray%20charles\"");
EXPECT_EQ(r->status, 200);
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
CATALOG_V2_ENTRIES_PREAMBLE("?q=%22ray%20charles%22")
" <title>Filtered Entries (q=&quot;ray charles&quot;)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>2</totalResults>\n"
" <startIndex>0</startIndex>\n"
" <itemsPerPage>2</itemsPerPage>\n"
RAY_CHARLES_CATALOG_ENTRY
CHARLES_RAY_CATALOG_ENTRY
"</feed>\n"
);
}