Compare commits

...
94 Commits
Author SHA1 Message Date
Kelson 74f664ea5c Merge pull request #1338 from kiwix/version-14.3.0
Bump-up version to 14.3.0
2026-09-03 20:16:49 +02:00
Emmanuel Engelhart 14d297faa1 Bump-up version to 14.3.0 2026-09-03 20:00:14 +02:00
Kelson 4d4ac1f072 Merge pull request #1321 from kiwix/issue-1309-write-opds
Writing and Reading an OPDS library file
2026-09-01 12:44:23 +02:00
Hamazasp Avetisyan af07bf9dbc Warn when two acquisition links disagree on length 2026-09-01 13:00:53 +04:00
Hamazasp Avetisyan 0d459296bd Replace OPDS rel="self" links with acquisition/open-access
rel="self" on a book entry misused the Atom/OPDS convention (which
normally names the feed/entry's own URL, not a local file path).
Local-path links are now written and parsed as
rel="http://opds-spec.org/acquisition/open-access" as well, with the
href's shape (absolute URL vs. path) telling readers which is which -
an entry may carry one of each.
2026-09-01 13:00:53 +04:00
Hamazasp Avetisyan a1c963f413 Archive size support for self links 2026-09-01 12:17:50 +04:00
Hamazasp Avetisyan 694d989797 Get rid of getArchiveFileSize() 2026-09-01 12:17:29 +04:00
Hamazasp Avetisyan 0c6ed03320 Add backward compatibility for readOpds() 2026-09-01 00:20:34 +04:00
Hamazasp Avetisyan 557bbc1ee2 Use a full URL in updateFromOPDSMultipleThumbnailLinksTest
Resolution behavior is already covered elsewhere; this test just needs some URL.
2026-08-30 13:02:01 +04:00
Hamazasp Avetisyan d261a05a2e Rename thumbnail-info builders to reflect the unified <link> output
Both builders produce the same <link> shape now; old names/docs described a distinction that no longer exists. No behavior change.
2026-08-30 13:02:01 +04:00
Hamazasp Avetisyan 9b6b9b9cab Merge linkIcons/base64Icons into a single thumbnailLinks list
Both lists render identical <link> markup now, so the split was redundant. Pure refactor, no behavior change.
2026-08-30 13:02:01 +04:00
Hamazasp Avetisyan 5ac7ceee26 Emit embedded thumbnails as data: URI links per OPDS 1.2 5.2.2
Replaces the non-standard <thumbnails> element with a data: URI href on the thumbnail <link>, matching the external-url case's shape.
2026-08-30 13:02:01 +04:00
Hamazasp Avetisyan de4cf431df Remove legacy <thumbnails><thumbnail> parsing from updateFromOpds()
Superseded by data: URI thumbnail links. Nothing shipped with this non-standard element, so drop it outright.
2026-08-30 13:02:01 +04:00
Hamazasp Avetisyan 1c18154e71 Decode data: URI thumbnail links per OPDS 1.2 5.2.2
Base64-decode a data: href on a thumbnail link into the Illustration's data, per the spec's artwork-relations section. Legacy <thumbnails> parsing stays for now.
2026-08-30 13:02:01 +04:00
Hamazasp Avetisyan a5e005729d Parse width/height/scale out of OPDS thumbnail link's type attribute
getIllustrationMimeTypeStr() encodes these as params in the type attribute; updateFromOpds() now reads them back instead of storing the raw string as mimeType.
2026-08-30 13:02:01 +04:00
Hamazasp Avetisyan 459c2fa0ff Base64 images dumping into OPDS libs 2026-08-30 13:02:01 +04:00
Hamazasp Avetisyan 6ef401ec60 Moving catalog URL generation out of template file, to support link thumbnails for library 2026-08-30 13:02:01 +04:00
Hamazasp Avetisyan ea3f16e268 Distinguishing between file and catalog stream dumps for OPDS 2026-08-30 13:02:01 +04:00
Hamazasp Avetisyan 0125d0f248 Support writing the library as an OPDS document 2026-08-30 13:01:59 +04:00
Veloman Yunkan a64bb9ca7b Library::dumpOpds() 2026-08-30 00:51:41 +04:00
Hamazasp Avetisyan 8bd1bcc4fe Avoid stray blank line after optional contentAccessUrl link in catalog_v2_entry.xml 2026-08-30 00:51:41 +04:00
Hamazasp Avetisyan e18ea62751 Ability to dump OPDS with "path" support 2026-08-30 00:51:41 +04:00
Hamazasp Avetisyan f8394d96bd Parameterize ManagerTest to cover also OPDS file re-loading 2026-08-30 00:51:41 +04:00
Veloman Yunkan ac866cc624 Suppress MIME-type-less illustrations coming from OPDS 2026-08-30 00:51:41 +04:00
Hamazasp Avetisyan 69d7b2d6b4 Parameterize LibraryServerTest to cover also OPDS file loading 2026-08-30 00:51:41 +04:00
Hamazasp Avetisyan 428dae1050 Parameterize LibraryTest to cover also OPDS file loading 2026-08-30 00:51:41 +04:00
Hamazasp Avetisyan 39545433a1 Parameterize LibForServerSearchTest to also search an OPDS library 2026-08-30 00:51:41 +04:00
Veloman Yunkan 80d42bb5f7 Fix for illustration link MIME-type in OPDS output 2026-08-30 00:51:41 +04:00
Hamazasp Avetisyan f67966216a Combine urlHost/libraryPath into contentOriginUri for readOpds()
readOpds() previously required callers to separately derive a host
string and a library path. Replace them with a single contentOriginUri
parameter, letting resolveContentOrigin() fully determine both
urlHost and baseDir from it based on whether it looks like a URL or a
local filesystem path.
2026-08-30 00:51:41 +04:00
Hamazasp Avetisyan 6d37a832c6 Adapt readOpds()'s body to resolveContentOrigin() (not yet fully functional)
Introduce a ContentOrigin struct and a resolveContentOrigin() helper
that will let readOpds() take a single contentOriginUri parameter
instead of separate urlHost/libraryPath ones. As a first step, wire
readOpds()'s body to use resolveContentOrigin() to derive baseDir from
libraryPath (replacing parseOpdsDom()'s internal
removeLastPathElement() call), while urlHost is still passed through
untouched.

This is not yet functional as a single-parameter API: readOpds()'s
public signature and callers are unchanged, and the urlHost side of
resolveContentOrigin() isn't exercised yet. That follows in the next
commit.
2026-08-30 00:51:41 +04:00
Hamazasp Avetisyan 6df1eeead5 Fix Book::updateFromOpds()'s naive urlHost + thumbnailUrl concatenation
favicon->url was built by simply concatenating urlHost and the
thumbnail's href attribute, which can produce a malformed URL (a
missing or doubled slash) depending on whether each side already ends
or starts with one. Add a joinUrl() helper that inserts or omits a
separating slash as needed and use it instead.
2026-08-30 00:51:41 +04:00
Hamazasp Avetisyan e9820154f3 Read base64-encoded OPDS thumbnails
Parse <thumbnails><thumbnail> elements containing base64-encoded
image data, decoding them the same way as updateFromXml()'s
"favicon" attribute.

Also read the element's width, height, and mimetype attributes into
the Illustration, mirroring how the link-based thumbnail's "type"
attribute is read into mimeType. Missing attributes fall back to
Illustration's own defaults (48x48, empty mimeType) rather than
overwriting them.
2026-08-30 00:51:41 +04:00
Hamazasp Avetisyan 455eff617a Avoid mangling absolute thumbnail URLs with urlHost prefix 2026-08-30 00:51:41 +04:00
Hamazasp Avetisyan 866411df15 Thread library path through readOpds()/readFile() as baseDir
Manager::readOpds() gains a libraryPath parameter, resolved to a
baseDir via removeLastPathElement() and passed to
Book::updateFromOpds() so a rel="self" link in a library file read
via readFile() resolves relative to that file's own directory - the
OPDS analogue of readXml()'s existing "path" attribute handling.
2026-08-30 00:51:41 +04:00
Hamazasp Avetisyan cebdf87fb5 Enhance Book::updateFromOpds() to resolve a rel="self" link into a path
Mirrors updateFromXml()'s handling of the "path" attribute: if an OPDS
entry carries a <link rel="self" href="..."/>, resolve it (against
baseDir, when relative) into Book::m_path/m_pathValid. baseDir defaults
to "" so existing callers are unaffected until they start passing a
real value.
2026-08-29 21:32:26 +04:00
Hamazasp Avetisyan efe23e07e7 Detect file format in readFile() to support both XML and OPDS 2026-08-29 21:32:26 +04:00
Hamazasp Avetisyan fe9efe84ec Add readOnly parameter to Manager::readOpds() 2026-08-29 21:32:26 +04:00
Hamazasp Avetisyan a855af4849 Preparation of test cases to support upcoming changes in readOpds interface 2026-08-29 21:32:26 +04:00
Hamazasp Avetisyan 70d751cdec Fix stale doc comment for readOpds()
The doc described params readOnly/libraryPath which do not exist on
readOpds() (leftover from a copy-paste of readXml()'s doc), and never
mentioned the actual urlHost parameter.
Also removed a comment in tets case that was refering to a class that was removed in previous branches.
2026-08-29 21:32:26 +04:00
Hamazasp Avetisyan 1e4873d8fe Moving search specific information out from LibraryDumper to OPDSDumper 2026-08-29 21:32:26 +04:00
Kelson 8920940cf4 Merge pull request #1329 from kiwix/flawfinder-error-suppression
Added Flawfinder error suppression comment.
2026-08-28 08:50:51 +02:00
Hamazasp Avetisyan 105b192ffb Added Flawfinder error suppression comment. 2026-08-28 08:50:39 +02:00
Veloman Yunkan e1b652b7b8 Merge pull request #1332 from kiwix/fix-manager-writable-library-path
Fix Manager::readFile() losing writableLibraryPath for new libraries
2026-08-24 12:09:39 +04:00
Hamazasp AvetisyanandClaude Sonnet 5 bfb148e4c5 Fix Manager::readFile() losing writableLibraryPath for new libraries
The fileExists() early-return skipped setting writableLibraryPath,
so relative --zimPathToSave resolved against CWD instead of the new
library's own directory when creating a library that doesn't exist yet.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-24 11:42:43 +04:00
Hamazasp Avetisyan f5e5f283ab Add resolveAbsPath() test helper
Wrap the recurring kiwix::computeAbsolutePath(kiwix::removeLastPathElement(...), ...)
pattern in a small helper and use it in addBookFromPathAndGetIdTest.
2026-08-24 10:58:57 +04:00
Kelson 689117f351 Merge pull request #1328 from lakshyajn/feat/beautify-filesize-decimal
Rewrite beautifyFileSize to use SI (base-1000) units
2026-08-18 10:02:05 +03:00
lakshyajn 7de90f700f feat: rewrite beautifyFileSize to use SI (base-1000) units
- Replace binary (1024) bit-shift logic with SI/decimal (1000) units
- Support all tiers: B, KB, MB, GB, TB, PB, EB
- Use EXPECT_ONE_OF macro in unit tests to safely handle slight floating-point
  precision differences at rounding-promotion boundaries across platforms
2026-08-18 11:20:11 +05:30
Veloman Yunkan f572ef03f8 Merge pull request #1327 from kiwix/read-file-method-changes
Reimplemented Manager::readFile() via Manager::readXml()
2026-08-15 16:54:43 +04:00
Hamazasp Avetisyan 5eb00fea04 Enhance readOpds() tests
Added couple of test cases to test readOpds() method deeper.
2026-08-15 16:36:44 +04:00
Hamazasp Avetisyan 85352a5aca readXML() usage in readFile() 2026-08-15 16:36:44 +04:00
Hamazasp Avetisyan 01ae801044 Enhance readXml() tests
Check more of the fields populated by readXml(). Also fixed a return value bug of readXml() method.
2026-08-15 16:36:42 +04:00
Veloman Yunkan 636bef7933 Merge pull request #1326 from kiwix/opds-search-metadata-related-bugfix
Replace try/catch with explicit presence checks in parseOpdsDom()
2026-08-15 14:38:30 +04:00
Hamazasp Avetisyan db1320c077 Replace try/catch with explicit presence checks in parseOpdsDom()
totalResults/startIndex/itemsPerPage were parsed inside a try/catch,
but strtoull() never throws on malformed input, so a feed missing
those elements silently fell through with zeroed-out fields and no
way to distinguish "not a search result" from "search result of size
zero". Check each node's presence directly instead, and cover both
paths with new Manager tests.
2026-08-15 13:30:04 +04:00
Veloman Yunkan 4c8147b139 Merge pull request #1325 from kiwix/multiple-thumbnails-support-in-opds
Multiple thumbnails support while reading from OPDS
2026-08-14 11:32:29 +04:00
Hamazasp Avetisyan 48003fa6e8 Multiple thumbnails support while reading from OPDS 2026-08-14 11:09:53 +04:00
Kelson df3699a635 Merge pull request #1304 from kiwix/translatewiki
Localisation updates from https://translatewiki.net.
2026-08-13 16:14:18 +03:00
translatewiki.net 3949aae2d5 Localisation updates from https://translatewiki.net. 2026-08-13 14:09:33 +02:00
Kelson 335eb122d4 Merge pull request #1322 from kiwix/ci_fix
Bump linux builder image version
2026-08-07 10:56:29 +02:00
Veloman Yunkan 21f096b2b3 Bump linux builder image version
kiwix-build's CI/CD which produces the deps archives for this project
now uses a builder image with OpenSSL develepment package installed
which results in the libcurl dependency being configured with ssl
support enabled. That dependency is unusable on the old builder because
of missing OpenSSL dev package.
2026-08-07 12:14:40 +04:00
Kelson 3fe2dffe73 Merge pull request #1317 from hamazaspavetisyan/refactorng-and-new-test-cases
Refactoring and new test cases
2026-08-05 11:19:11 +02:00
Hamazasp Avetisyan 4016d177ee Add unit tests for OPDSDumper
OPDSDumper predates this PR but had no dedicated test suite; its
behavior was only ever exercised indirectly through the HTTP server
tests.
2026-08-03 18:31:17 +04:00
Hamazasp Avetisyan e21e3cf4a3 Add unit tests for LibXMLDumper
LibXMLDumper predates this PR but had no dedicated test suite. Also
default-initialize the library pointer to nullptr: LibXMLDumper's
default constructor left it as an uninitialized (unusable and
non-empty) garbage pointer, which the new
defaultConstructedDumperHasNoLibrary test exercises directly.
2026-08-03 18:31:17 +04:00
Hamazasp Avetisyan 3a40b8339f Add OPDS coverage for Book::updateFromOpds
Book::updateFromOpds predates this PR but its category-resolution and
language-parsing behavior (mirroring the existing XML equivalents) had
no direct test coverage.
2026-08-03 18:31:17 +04:00
Hamazasp Avetisyan fd74d6100f Add unit tests for fullEntryOpds
These helpers (formerly fullEntryXML, moved out of opds_dumper.cpp in
an earlier commit) render the guts of every OPDS entry - live catalog
and local dumps alike - but had no direct test coverage of their own.
Named after what they exercise (rendering an OPDS entry), not the
source file they happen to live in, to avoid implying they test the
LibraryDumper base class.
2026-08-03 18:31:17 +04:00
Hamazasp Avetisyan 6b4e14241e Refactor: move fullEntryOpds/getBookIllustrationInfo into library_dumper
These were private helpers inside opds_dumper.cpp, only reachable by
OPDSDumper and impossible to unit-test directly. Move them (unchanged)
into library_dumper.{h,cpp} as free functions declared in the header,
so they become a shared, independently-testable utility that a future
LibOPDSDumper class can reuse for local/offline library dumps.
2026-08-03 18:31:17 +04:00
Hamazasp Avetisyan 287b004f84 Rename fullEntryXML/partialEntryXML to fullEntryOpds/partialEntryOpds
These functions render OPDS entry XML. Naming them after XML rather than
OPDS was confusing ("XML" is the serialization format, not the catalog
format); rename to match what they actually produce. Renamed ahead of
the upcoming refactor that moves fullEntryOpds out of this file and
exposes it more broadly, so it never goes through a 'public API named
after the wrong thing' phase.
2026-08-03 18:31:17 +04:00
Hamazasp Avetisyan b7801626b1 Fix macOS failure in UnusablePort test 2026-08-03 18:31:17 +04:00
Kelson 0d8dd7bf9d Merge pull request #1308 from kiwix/revert-magnet-link-hack
Rollback magnet link hack
2026-06-20 17:58:16 +03:00
Emmanuel Engelhart 486215bfa2 Rollback magnet link hack 2026-06-20 06:39:07 +02:00
Kelson 096cba9fff Merge pull request #1306 from kiwix/ci-updates
CI updates
2026-06-14 13:52:39 +03:00
Emmanuel Engelhart d1424e76b2 Reintroduce Debian unstable in the CI 2026-06-13 12:25:20 +02:00
Emmanuel Engelhart 120cdebf1e Move to Python 3.13 in the CI 2026-06-13 12:19:11 +02:00
Emmanuel Engelhart 37920a39b5 Update actions/checkout to v6 2026-06-13 12:18:00 +02:00
Emmanuel Engelhart cdf3385e0b Add Ubuntu 24.04 and 26.04 to CI 2026-06-13 12:14:59 +02:00
Emmanuel Engelhart 4250dca28f Update to windows-2025-vs2026 in the CI 2026-06-13 11:57:53 +02:00
Emmanuel Engelhart 60fd21e2ca Update to macos-26 in the CI 2026-06-13 11:57:14 +02:00
Kelson d7657e99a0 Merge pull request #1290 from kiwix/translatewiki
Localisation updates from https://translatewiki.net.
2026-06-06 07:22:16 +02:00
translatewiki.net fccc4e9167 Localisation updates from https://translatewiki.net. 2026-06-04 14:06:37 +02:00
Kelson 46b599fb2c Merge pull request #1297 from kiwix/libkiwix-14.2.1
Libkiwix 14.2.1
2026-05-09 17:31:46 +02:00
Emmanuel Engelhart 940abd2e8f Add 14.2.1 changelog 2026-05-09 17:00:35 +02:00
Emmanuel Engelhart 2f987ee727 Bump-up version to 14.2.1 2026-05-09 17:00:35 +02:00
Kelson d072e57f05 Merge pull request #1298 from kiwix/exact_version_of_libmicrohttpd_url_parsing_change
Exact version of libmicrohttpd affecting the server unit-test
2026-05-09 16:58:37 +02:00
Veloman Yunkan a7df774f54 More accurate value of MHD_VERSION in the server test
Identified the exact version of libmicrohttpd where the change in URL parsing
affects the server unit-test.
2026-05-09 17:37:25 +04:00
Kelson 7693557065 Merge pull request #1277 from BPerlakiH/1210-fix-docs
Update docs
2026-05-02 11:36:54 +02:00
Balazs Perlaki-Horvath 58e458b658 Update docs 2026-05-02 11:36:21 +02:00
Kelson fb1b8b85e0 Merge pull request #1295 from kiwix/fix-ifram-bottom-white-line
Fix white line at bottom of iframe
2026-05-02 11:28:25 +02:00
zeyad elkholy 5cc3ce1895 Fix white line at bottom of iframe 2026-05-02 11:22:01 +02:00
Kelson 7e50bf611b Merge pull request #1294 from kiwix/handling-of-requests-with-wrong-root-url-prefix
Correct error message for requests with wrong root URL prefix
2026-05-01 11:32:55 +02:00
Veloman Yunkan 59ba61fcd2 Fixed the error text for requests with wrong root 2026-04-30 19:08:08 +04:00
Veloman Yunkan 40b9e2b876 New test-points demonstrating a bug in kiwix-serve
When kiwix-serve is started with a non-empty --urlRootLocation parameter and
a request for a URL *not* starting with the specified URL prefix is
received, the message in the error response contains a wrong URL. For
example if the URL root location is set to '/abc', and '/klmn/xyz' is
requested the error message reads:

  The requested URL "/abcINVALID URL" was not found on this server.
2026-04-30 19:02:17 +04:00
Veloman Yunkan 90867968c1 Dropped (unused) RequestContext::get_root_path() 2026-04-30 19:02:17 +04:00
Veloman Yunkan fcebc2d4f6 Exposed MHD_VERSION to test/server.cpp
It turns out that the C preprocessor doesn't complain about undefined
macros used in the arithmetic expressions in `#if` directives (treating
those conditions simply as not satisfied). As a result, PR #1293 had
a slightly different effect from that which was intended. Now taking
the opportunity to correct that subtle mistake.
2026-04-30 18:55:42 +04:00
Kelson d24bd1ea3e Merge pull request #1293 from kiwix/workaround_for_libmicrohttpd_1.x
"Fixed" the server unit-test for libmicrohttpd 1.x
2026-04-30 15:23:56 +02:00
Veloman Yunkan 20b3dd77bc "Fixed" the server unit-test for libmicrohttpd 1.x
libmicrohttpd 1.0.5 (the version used at this point under the debian
testing distribution) seems to return an HTTP 400 (Bad Request) error
if the URL contains an invalid URI-encoding sequence, thus breaking
our unit-tests involving such URLs. Upgrading our libmicrohttpd
dependency is not easy, so this change is an optimistic attempt to fix
the build of the libkiwix package under debian testing.
2026-04-30 17:01:02 +04:00
54 changed files with 3445 additions and 412 deletions

No files matched your search

+38 -15
View File
@@ -29,13 +29,13 @@ jobs:
- target: ios-x86_64-dyn
arch_name: x86-apple-ios-simulator
run_test: false
runs-on: macos-15
runs-on: macos-26
env:
HOME: /Users/runner
steps:
- name: Retrieve source code
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Install packages
run: |
@@ -69,16 +69,16 @@ jobs:
run: meson test -C build --verbose
Windows:
runs-on: windows-2025
runs-on: windows-2025-vs2026
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Setup python 3.10
uses: actions/setup-python@v5
- name: Setup python 3.13
uses: actions/setup-python@v6
with:
python-version: '3.10'
python-version: '3.13'
- name: Install packages
run:
@@ -120,43 +120,66 @@ jobs:
matrix:
target:
- linux-x86_64-static
- linux-x86_64-dyn
- jammy-x86_64-dyn
- noble-x86_64-dyn
- resolute-x86_64-dyn
- android-arm
- android-arm64
image_variant: ['jammy']
include:
- target: linux-x86_64-static
image_variant: jammy
dl_deps_archive_target: linux-x86_64-static
lib_postfix: '/x86_64-linux-gnu'
arch_name: linux-x86_64
run_test: true
coverage: true
- target: linux-x86_64-dyn
- target: jammy-x86_64-dyn
image_variant: jammy
dl_deps_archive_target: linux-x86_64-dyn
lib_postfix: '/x86_64-linux-gnu'
arch_name: linux-x86_64
run_test: true
coverage: true
coverage: false
- target: noble-x86_64-dyn
image_variant: noble
dl_deps_archive_target: linux-x86_64-dyn
lib_postfix: '/x86_64-linux-gnu'
arch_name: linux-x86_64
run_test: true
coverage: false
- target: resolute-x86_64-dyn
image_variant: resolute
dl_deps_archive_target: linux-x86_64-dyn
lib_postfix: '/x86_64-linux-gnu'
arch_name: linux-x86_64
run_test: true
coverage: false
- target: android-arm
image_variant: jammy
dl_deps_archive_target: android-arm
lib_postfix: '/arm-linux-androideabi'
arch_name: arm-linux-androideabi
run_test: false
coverage: false
- target: android-arm64
image_variant: jammy
dl_deps_archive_target: android-arm64
lib_postfix: '/aarch64-linux-android'
arch_name: aarch64-linux-android
run_test: false
coverage: false
env:
HOME: /home/runner
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
container:
image: "ghcr.io/kiwix/kiwix-build_ci_${{matrix.image_variant}}:2025-06-07"
image: "ghcr.io/kiwix/kiwix-build_ci_${{matrix.image_variant}}:2026-07-22"
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Install dependencies
uses: kiwix/kiwix-build/actions/dl_deps_archive@main
with:
target_platform: ${{ matrix.target }}
target_platform: ${{ matrix.dl_deps_archive_target }}
- name: Compile
shell: bash
run: |
+9 -9
View File
@@ -15,7 +15,7 @@ jobs:
fail-fast: false
matrix:
distro:
# - debian-unstable
- debian-unstable
# - debian-trixie
# - debian-bookworm
# - debian-bullseye
@@ -23,7 +23,7 @@ jobs:
- ubuntu-jammy
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
# Determine which PPA we should upload to
- name: PPA
@@ -44,13 +44,13 @@ jobs:
email: release+launchpad@kiwix.org
distro: ${{ matrix.distro }}
# - uses: legoktm/gh-action-build-deb@debian-unstable
# if: matrix.distro == 'debian-unstable'
# name: Build package for debian-unstable
# id: build-debian-unstable
# with:
# args: --no-sign
#
- uses: legoktm/gh-action-build-deb@debian-unstable
if: matrix.distro == 'debian-unstable'
name: Build package for debian-unstable
id: build-debian-unstable
with:
args: --no-sign
# - uses: legoktm/gh-action-build-deb@b47978ba8498dc8b8153cc3b5f99a5fc1afa5de1 # pin@debian-trixie
# if: matrix.distro == 'debian-trixie'
# name: Build package for debian-trixie
+7
View File
@@ -1,3 +1,10 @@
libkiwix 14.2.1
===============
* Documentation fix (@bperlakih #1277)
* Fix kiwix-serve unit-test for libmicrohttpd 1.x (@veloman-yunkan #1293)
* Fix kiwix-server error message for requests with wrong root URL prefix (@veloman-yunkan #1294)
libkiwix 14.2.0
===============
+22
View File
@@ -70,6 +70,28 @@ class Book
bool update(const Book& other);
void update(const zim::Archive& archive);
void updateFromXml(const pugi::xml_node& node, const std::string& baseDir);
/**
* Update the book's metadata from an OPDS entry XML node.
*
* @param node the `<entry>` node of an OPDS feed describing the book.
* @param urlHost host to prepend to relative illustration/thumbnail URLs.
* @param baseDir base directory used to resolve a relative local-path
* acquisition link (`rel="http://opds-spec.org/acquisition/
* open-access"` with a non-URL href) into an absolute book
* path.
*/
void updateFromOpds(const pugi::xml_node& node, const std::string& urlHost, const std::string& baseDir);
/**
* Update the book's metadata from an OPDS entry XML node.
*
* A simple wrapper around the three-parameter updateFromOpds() above, kept
* for backward compatibility. Equivalent to calling
* updateFromOpds(node, urlHost, "").
*
* @param node the `<entry>` node of an OPDS feed describing the book.
* @param urlHost host to prepend to relative illustration/thumbnail URLs.
*/
void updateFromOpds(const pugi::xml_node& node, const std::string& urlHost);
std::string getHumanReadableIdFromPath() const;
+44 -7
View File
@@ -312,7 +312,7 @@ class Library: public std::enable_shared_from_this<Library>
* @return The number of bookmarks updated.
*/
int migrateBookmarks(const std::string& sourceBookId, const std::string& targetBookId);
/**
* Get the best available bookId for a bookmark.
*
@@ -361,12 +361,32 @@ class Library: public std::enable_shared_from_this<Library>
bool removeBookById(const std::string& id);
/**
* Write the library to a file.
* Writes the library to a file as library.xml.
*
* @deprecated Calls writeAsXML directly and is kept for compatibility
* with external usages. This method is subject to removal in the future;
* please use writeAsXML instead.
*
* @param path the path of the file to write to.
* @return True if the library has been correctly saved.
*/
bool writeToFile(const std::string& path) const;
DEPRECATED bool writeToFile(const std::string& path) const;
/**
* Write the library to a file as library.xml.
*
* @param path the path of the file to write to.
* @return True if the library has been correctly saved.
*/
bool writeAsXML(const std::string& path) const;
/**
* Write the library to a file as an OPDS document.
*
* @param path the path of the file to write to.
* @return True if the library has been correctly saved.
*/
bool writeAsOPDS(const std::string& path) const;
/**
* Write the library bookmarks to a file.
@@ -444,11 +464,11 @@ class Library: public std::enable_shared_from_this<Library>
/**
* Sort (in place) bookIds using the given comparator.
* Sort (in place) bookIds using the given arguments.
*
* @param bookIds the list of book Ids to sort
* @param comparator how to sort the books
* @return The sorted list of books
* @param sortBy how to sort the books (UNSORTED, TITLE, SIZE, DATE, CREATOR, PUBLISHER)
* @param ascending ascending or descending
*/
void sort(BookIdCollection& bookIds, supportedListSortBy sortBy, bool ascending) const;
@@ -470,8 +490,25 @@ class Library: public std::enable_shared_from_this<Library>
*/
uint32_t removeBooksNotUpdatedSince(Revision rev);
/**
* Dump the library as an OPDS feed suitable for an offline/local library
* file (no HTTP root; book paths are exposed via acquisition links whose
* href is a local path rather than a URL).
*
* This does not write to outputPath itself - it only returns the OPDS
* XML. outputPath is used solely to resolve each book's local-path
* acquisition link: each book's path is made relative to outputPath's
* parent directory, the same way writeToFile() resolves book paths
* relative to the file it writes.
*
* @param outputPath the path the returned OPDS content is intended to be
* written to; only its parent directory is used, to
* compute book paths relative to it.
* @return The library dumped as an OPDS feed.
*/
std::string dumpOpds(const std::string& outputPath) const;
friend class OPDSDumper;
friend class libXMLDumper;
private: // types
typedef const std::string& (Book::*BookStrPropMemFn)() const;
+34 -9
View File
@@ -68,9 +68,10 @@ class Manager
explicit Manager(LibraryPtr library);
/**
* Read a `library.xml` and add book in the file to the library.
* Read a library XML or an OPDS file and add the books in the file to the
* library.
*
* @param path The (utf8) path to the `library.xml`.
* @param path The (utf8) path to the library file.
* @param readOnly Set if the libray path could be overwritten latter with
* updated content.
* @param trustLibrary use book metadata coming from XML.
@@ -104,17 +105,40 @@ class Manager
bool trustLibrary = true);
/**
* Load a library content stored in a OPDS stream.
* Load a library content stored in a OPDS stream or a local library file.
*
* @param content The content of the OPDS stream.
* @param contentOriginUri Where content was read from: either the URL it
* was fetched from (used to resolve relative acquisition/thumbnail
* links) or the local filesystem path it was read from (used to
* resolve a relative local-path acquisition link). See
* kiwix::resolveContentOrigin() for the exact splitting rules. It is
* assumed that URL links come in the form of an absolute path
* component.
* @param readOnly Set if the library path could be overwritten later with
* updated content.
* @param libraryPath The library path (used to resolve relative path)
* @return True if the content has been properly parsed.
*/
bool readOpds(const std::string& content,
const std::string& contentOriginUri,
bool readOnly);
/**
* Load a library content stored in a OPDS stream or a local library file.
*
* A simple wrapper around the three-parameter readOpds() above, kept for
* backward compatibility. Equivalent to calling
* readOpds(content, urlHost, false).
*
* @param content The content of the OPDS stream.
* @param urlHost The host used to resolve relative acquisition/thumbnail
* links. It is assumed that those links come in the form of an
* absolute path component, so the full URL is obtained by
* prepending the urlHost string to the href value of the link.
* @return True if the content has been properly parsed.
*/
bool readOpds(const std::string& content, const std::string& urlHost);
/**
* Load a bookmark file.
*
@@ -157,7 +181,7 @@ class Manager
/**
* Add all books from the directory tree into the library.
*
*
* @param path The path of the directory to scan.
* @param verboseFlag Verbose logs flag.
*/
@@ -180,9 +204,10 @@ class Manager
const std::string& libraryPath,
bool trustLibrary);
bool parseOpdsDom(const pugi::xml_document& doc,
const std::string& urlHost);
const std::string& urlHost,
const std::string& baseDir,
bool readOnly);
};
}
} // namespace kiwix
#endif
+1 -1
View File
@@ -1,5 +1,5 @@
project('libkiwix', 'cpp',
version : '14.2.0',
version : '14.3.0',
license : 'GPLv3+',
default_options : ['c_std=c11', 'cpp_std=c++17', 'werror=true'])
+179 -12
View File
@@ -25,13 +25,133 @@
#include "tools/networkTools.h"
#include "tools/otherTools.h"
#include "tools/stringTools.h"
#include "tools/pathTools.h"
#include "tools/archiveTools.h"
#include <zim/archive.h>
#include <zim/item.h>
#include <pugixml.hpp>
#include <sstream>
#include <cctype>
namespace
{
/**
* Tells whether a URL string is already absolute (contains a scheme,
* e.g. "https://example.com/x.png") as opposed to being a relative path
* (e.g. "/x.png"). Only a relative URL should be prefixed with a base host
* or URL—doing so unconditionally would garble an already-absolute one.
*/
bool isAbsoluteUrl(const std::string& url)
{
// Find the scheme separator
size_t pos = url.find("://");
if (pos == 0 || pos == std::string::npos) {
return false; // No scheme or empty scheme
}
// RFC 3986: Scheme must begin with a letter, followed by letters, digits, '+', '.', or '-'
if (!std::isalpha(static_cast<unsigned char>(url[0]))) {
return false;
}
// Validate that all remaining characters in the scheme comply with RFC 3986
for (size_t i = 1; i < pos; ++i) {
char c = url[i];
if (!std::isalnum(static_cast<unsigned char>(c)) && c != '+' && c != '-' && c != '.') {
return false;
}
}
return true;
}
/**
* Joins a host and a reference into a single URL without producing a double slash,
* respecting the host's format.
*/
std::string joinUrl(const std::string& host, const std::string& ref)
{
if (host.empty()) {
return ref;
}
if (ref.empty()) {
return host;
}
const bool hostEndsWithSlash = (host.back() == '/');
const bool refStartsWithSlash = (ref.front() == '/');
if (hostEndsWithSlash && refStartsWithSlash) {
// Both have a slash; omit one to avoid a double slash
return host + ref.substr(1);
} else if (!hostEndsWithSlash && !refStartsWithSlash) {
// Neither has a slash; insert one
return host + "/" + ref;
} else {
// Exactly one has a slash; simple concatenation is correct
return host + ref;
}
}
/**
* Splits an OPDS thumbnail link's "type" attribute value into its base MIME
* type and its "width"/"height" parameters, mirroring the
* "<mimetype>;width=<w>;height=<h>;scale=<s>" convention that
* getIllustrationMimeTypeStr() (library_dumper.cpp) writes on the way out.
* "scale" is parsed away but otherwise unused, since Illustration has no
* such field. A width/height left at 0 (i.e. absent from the type string)
* means "unspecified" and must not overwrite the Illustration's own default.
*/
struct ParsedIllustrationType
{
std::string mimeType;
uint16_t width = 0;
uint16_t height = 0;
};
ParsedIllustrationType parseIllustrationType(const std::string& type)
{
ParsedIllustrationType result;
const auto parts = kiwix::split(type, ";");
if (parts.empty()) {
return result;
}
const std::string potentialMime = kiwix::trim(parts[0]);
// A basic check: MIME types typically contain a slash (e.g., "image/jpeg")
if (potentialMime.find('/') != std::string::npos) {
result.mimeType = potentialMime;
}
for (auto it = parts.begin() + 1; it != parts.end(); ++it) {
const auto eqPos = it->find('=');
if (eqPos == std::string::npos) {
continue;
}
// Trim both key and value to handle spaces safely (e.g., " width=100")
const std::string key = kiwix::trim(it->substr(0, eqPos));
const std::string value = kiwix::trim(it->substr(eqPos + 1));
if (!value.empty()) {
const uint16_t numericValue =
static_cast<uint16_t>(strtoul(value.c_str(), nullptr, 10));
if (key == "width") {
result.width = numericValue;
} else if (key == "height") {
result.height = numericValue;
}
}
}
return result;
}
} // anonymous namespace
namespace kiwix
{
/* Constructor */
@@ -79,7 +199,7 @@ void Book::update(const zim::Archive& archive) {
m_category = getCategoryFromTags();
m_articleCount = archive.getArticleCount();
m_mediaCount = archive.getMediaCount();
m_size = static_cast<uint64_t>(getArchiveFileSize(archive)) << 10;
m_size = archive.getFilesize();
m_illustrations.clear();
for ( const auto& illustrationInfo : archive.getIllustrationInfos() ) {
@@ -144,14 +264,18 @@ static std::string fromOpdsDate(const std::string& date)
}
#define VALUE(name) node.child(name).child_value()
void Book::updateFromOpds(const pugi::xml_node& node, const std::string& urlHost)
{
updateFromOpds(node, urlHost, "");
}
#define VALUE(name) node.child(name).child_value()
void Book::updateFromOpds(const pugi::xml_node& node, const std::string& urlHost, const std::string& baseDir)
{
m_id = VALUE("id");
if (!m_id.compare(0, 9, "urn:uuid:")) {
m_id.erase(0, 9);
}
// No path on opds.
m_title = VALUE("title");
m_description = VALUE("summary");
m_language = VALUE("language");
@@ -167,23 +291,66 @@ void Book::updateFromOpds(const pugi::xml_node& node, const std::string& urlHost
m_category = catnode.empty() ? getCategoryFromTags() : catnode.child_value();
m_articleCount = strtoull(VALUE("articleCount"), 0, 0);
m_mediaCount = strtoull(VALUE("mediaCount"), 0, 0);
m_illustrations.clear();
std::string firstAcquisitionHref;
std::string firstLength;
for(auto linkNode = node.child("link"); linkNode;
linkNode = linkNode.next_sibling("link")) {
std::string rel = linkNode.attribute("rel").value();
if (rel == "http://opds-spec.org/acquisition/open-access") {
m_url = linkNode.attribute("href").value();
m_size = strtoull(linkNode.attribute("length").value(), 0, 0);
// The href tells us whether this link points at a remote copy of the
// book (an absolute URL) or a local one (a filesystem path, absolute
// or relative to baseDir) - a single entry may carry one of each.
const std::string href = linkNode.attribute("href").value();
if (isAbsoluteUrl(href)) {
m_url = href;
} else {
m_path = isRelativePath(href)? computeAbsolutePath(baseDir, href): href;
m_pathValid = fileReadable(m_path);
}
const std::string length = linkNode.attribute("length").value();
if (!length.empty()) {
if (!firstLength.empty() && length != firstLength) {
std::cerr << "Book '" << m_id << "': acquisition links '"
<< firstAcquisitionHref << "' (length " << firstLength
<< ") and '" << href << "' (length " << length
<< ") disagree on length." << std::endl;
}
m_size = strtoull(length.c_str(), 0, 0);
firstAcquisitionHref = href;
firstLength = length;
}
}
if (rel == "http://opds-spec.org/image/thumbnail") {
const auto favicon = std::make_shared<Illustration>();
favicon->data.clear();
favicon->url = urlHost + linkNode.attribute("href").value();
favicon->mimeType = linkNode.attribute("type").value();
m_illustrations.assign(1, favicon);
const std::string thumbnailUrl = linkNode.attribute("href").value();
if (startsWith(thumbnailUrl, "data:")) {
// OPDS 1.2's "data" URL scheme (spec 5.2.2): the payload is
// whatever follows the first comma, regardless of what media-type
// text (if any) precedes it - the link's own "type" attribute is
// authoritative for that.
const auto commaPos = thumbnailUrl.find(',');
if (commaPos != std::string::npos) {
favicon->data = base64_decode(thumbnailUrl.substr(commaPos + 1));
}
} else {
// XXX non-absolute URL is expected to be an absolute-path.
favicon->url = isAbsoluteUrl(thumbnailUrl)? thumbnailUrl: joinUrl(urlHost, thumbnailUrl);
}
const auto parsedType = parseIllustrationType(linkNode.attribute("type").value());
favicon->mimeType = parsedType.mimeType;
if (parsedType.width) {
favicon->width = parsedType.width;
}
if (parsedType.height) {
favicon->height = parsedType.height;
}
if (!favicon->mimeType.empty()) {
m_illustrations.push_back(favicon);
}
}
}
}
}
#undef VALUE
+46 -4
View File
@@ -20,10 +20,9 @@
#include "library.h"
#include "book.h"
#include "libxml_dumper.h"
#include "library_dumper.h"
#include "tools.h"
#include "tools/base64.h"
#include "tools/regexTools.h"
#include "tools/pathTools.h"
#include "tools/stringTools.h"
#include "tools/otherTools.h"
@@ -433,6 +432,11 @@ unsigned int Library::getBookCount(const bool localBooks,
}
bool Library::writeToFile(const std::string& path) const
{
return writeAsXML(path);
}
bool Library::writeAsXML(const std::string& path) const
{
const auto allBookIds = getBooksIds();
@@ -447,6 +451,11 @@ bool Library::writeToFile(const std::string& path) const
return writeTextFile(path, xml);
}
bool Library::writeAsOPDS(const std::string& path) const
{
return writeTextFile(path, dumpOpds(path));
}
bool Library::writeBookmarksToFile(const std::string& path) const
{
LibXMLDumper dumper(this);
@@ -729,7 +738,7 @@ Xapian::Query buildXapianQuery(const Filter& filter)
q = Xapian::Query(Xapian::Query::OP_AND, q, nameQuery(filter.getName()));
}
if ( filter.hasFlavour() ) {
q = Xapian::Query(Xapian::Query::OP_AND, q, flavourQuery(filter.getFlavour()));
q = Xapian::Query(Xapian::Query::OP_AND, q, flavourQuery(filter.getFlavour()));
}
if ( filter.hasCategory() ) {
q = Xapian::Query(Xapian::Query::OP_AND, q, categoryQuery(filter.getCategory()));
@@ -998,7 +1007,7 @@ Filter& Filter::name(std::string name)
activeFilters |= NAME;
return *this;
}
Filter& Filter::flavour(std::string flavour)
{
_flavour = flavour;
@@ -1075,4 +1084,37 @@ bool Filter::accept(const Book& book) const
return true;
}
std::string Library::dumpOpds(const std::string& outputPath) const
{
std::lock_guard<std::recursive_mutex> lock(m_mutex);
const auto baseDir = kiwix::removeLastPathElement(outputPath);
std::ostringstream ss;
ss << R"(<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:dc="http://purl.org/dc/terms/"
xmlns:opds="http://opds-spec.org/2010/catalog">
)";
for (auto& pair: m_books) {
const Book& book = pair.second;
// Local/offline dump: the book's own id is used as the content id, there
// is no content-access URL to link to (no server involved), and the
// book's local path is safe to expose as an acquisition link, resolved
// relative to baseDir the same way LibXMLDumper does.
const std::string localPath = book.getPath().empty()
? ""
: computeRelativePath(baseDir, book.getPath());
ss << fullEntryOpds(book,
/*rootLocation=*/"",
/*contentAccessUrl=*/"",
/*contentId=*/book.getId(),
localPath,
/*isLiveCatalog=*/false);
}
ss << "</feed>\n";
return ss.str();
}
}
+140 -9
View File
@@ -1,10 +1,148 @@
#include "library_dumper.h"
#include "tools/stringTools.h"
#include "tools/otherTools.h"
#include <mustache.hpp>
#include "book.h"
#include "libkiwix-resources.h"
#include "tools.h"
#include "tools/base64.h"
#include "tools/otherTools.h"
#include "tools/stringTools.h"
namespace kiwix
{
namespace
{
std::string getIllustrationMimeTypeStr(const Book::Illustration& illustration)
{
std::ostringstream result;
result << illustration.mimeType;
if ( !contains(illustration.mimeType, ";width=") ) {
result << ";width=" << illustration.width;
}
if ( !contains(illustration.mimeType, ";height=") ) {
result << ";height=" << illustration.height;
}
if ( !contains(illustration.mimeType, ";scale=") ) {
result << ";scale=1";
}
return result.str();
}
/**
* Get thumbnail <link> info for a book's illustrations that already have
* (or, for the live catalog, are always given) an external url to fetch
* their data from.
*
* Live catalog: keep every icon (its own endpoint can serve embedded or
* remote data). File dump: keep only icons with an external url, since one
* with no url has no server to serve its data from (see
* getEmbeddedThumbnailLinks() for those instead).
*/
kainjow::mustache::list
getExternalThumbnailLinks(const Book& book, const std::string& rootLocation,
bool isLiveCatalog = true)
{
kainjow::mustache::list thumbnailLinks;
for ( const auto& illustration : book.getIllustrations() ) {
if (!(isLiveCatalog || !illustration->url.empty())) {
continue;
}
const std::string iconSizeWidth = to_string(illustration->width);
// Book IDs do not contain special characters, so they are HTML-safe.
// Therefore, no HTML encoding is required.
const std::string thumbnailUrl = isLiveCatalog
? rootLocation + "/catalog/v2/illustration/" + kiwix::urlEncode(book.getId()) + "/?size=" + iconSizeWidth
: illustration->url;
thumbnailLinks.push_back(kainjow::mustache::object{
{"icon_mimetype", getIllustrationMimeTypeStr(*illustration)},
{"icon_url", thumbnailUrl}
});
}
return thumbnailLinks;
}
/**
* Get thumbnail <link> info for a book's illustrations that have no
* external url, rendering their data as a "data:" URI href
* (OPDS 1.2 5.2.2), same as getExternalThumbnailLinks()'s external-url
* links.
*
* Only applies to file dumps (never the live catalog, which serves embedded
* data from its own endpoint instead) and only for icons without an external
* url, i.e. those that would otherwise have no way to reach the reader.
*/
kainjow::mustache::list
getEmbeddedThumbnailLinks(const Book& book, bool isLiveCatalog = true)
{
kainjow::mustache::list thumbnailLinks;
if (isLiveCatalog) {
return thumbnailLinks;
}
for ( const auto& illustration : book.getIllustrations() ) {
if (!illustration->url.empty()) {
continue;
}
const std::string thumbnailData = illustration->getData();
const std::string dataUri = thumbnailData.empty() ? ""
: "data:" + illustration->mimeType + ";base64," + base64_encode(thumbnailData);
thumbnailLinks.push_back(kainjow::mustache::object{
{"icon_mimetype", getIllustrationMimeTypeStr(*illustration)},
{"icon_url", dataUri}
});
}
return thumbnailLinks;
}
} // namespace
std::string fullEntryOpds(const Book& book,
const std::string& rootLocation,
const std::string& contentAccessUrl,
const std::string& contentId,
const std::string& localPath,
bool isLiveCatalog)
{
const auto bookDate = book.getDate() + "T00:00:00Z";
auto thumbnailLinks
= getExternalThumbnailLinks(book, rootLocation, isLiveCatalog);
auto embeddedThumbnailLinks
= getEmbeddedThumbnailLinks(book, isLiveCatalog);
thumbnailLinks.insert(thumbnailLinks.end(),
std::make_move_iterator(embeddedThumbnailLinks.begin()),
std::make_move_iterator(embeddedThumbnailLinks.end()));
const kainjow::mustache::object data{
{"contentAccessUrl", onlyAsNonEmptyMustacheValue(contentAccessUrl)},
{"id", book.getId()},
{"name", book.getName()},
{"title", book.getTitle()},
{"description", book.getDescription()},
{"language", book.getCommaSeparatedLanguages()},
{"content_id", urlEncode(contentId)},
{"updated", bookDate}, // XXX: this should be the entry update datetime
{"book_date", bookDate},
{"category", book.getCategory()},
{"flavour", book.getFlavour()},
{"tags", book.getTags()},
{"article_count", to_string(book.getArticleCount())},
{"media_count", to_string(book.getMediaCount())},
{"author_name", book.getCreator()},
{"publisher_name", book.getPublisher()},
{"url", onlyAsNonEmptyMustacheValue(book.getUrl())},
{"size", to_string(book.getSize())},
{"thumbnailLinks", thumbnailLinks},
{"local_path", onlyAsNonEmptyMustacheValue(localPath)},
};
return render_template(RESOURCE::templates::catalog_v2_entry_xml, data);
}
/* Constructor */
LibraryDumper::LibraryDumper(const Library* library, const NameMapper* nameMapper)
: library(library),
@@ -16,13 +154,6 @@ LibraryDumper::~LibraryDumper()
{
}
void LibraryDumper::setOpenSearchInfo(int totalResults, int startIndex, int count)
{
m_totalResults = totalResults;
m_startIndex = startIndex,
m_count = count;
}
kainjow::mustache::list LibraryDumper::getCategoryData() const
{
const auto now = gen_date_str();
+35 -12
View File
@@ -31,6 +31,41 @@
namespace kiwix
{
class Book;
/**
* Render the full OPDS entry XML for a book.
*
* @param book The book to render the OPDS entry for.
* @param rootLocation The root URL/path the catalog is served from, used to
* build absolute links within the entry.
* @param contentAccessUrl The URL (or URL prefix) used to access the book's
* content itself (as opposed to the catalog entry).
* @param contentId The identifier of the book's content, used when building
* content access links.
* @param localPath If non-empty, rendered as an additional rel="http://opds-
* spec.org/acquisition/open-access" link carrying the book's
* local file path (as opposed to the book's remote url, if
* any, which is rendered as its own such link). Only meant
* for local/offline dumps - leave empty for the live HTTP
* catalog, which must not leak server-side filesystem paths
* to remote clients.
* @param isLiveCatalog Whether this entry is rendered for the live HTTP
* catalog (OPDSDumper) as opposed to a local/offline file
* dump (Library::dumpOpds). The live catalog can serve any
* illustration - embedded or remote - through its own
* /catalog/v2/illustration endpoint, but an offline dump has
* no server behind that endpoint, so only illustrations
* backed by a real external url can produce a working
* thumbnail link there.
*/
std::string fullEntryOpds(const Book& book,
const std::string& rootLocation,
const std::string& contentAccessUrl,
const std::string& contentId,
const std::string& localPath = "",
bool isLiveCatalog = true);
/**
* A base class to dump Library in various formats.
*
@@ -57,15 +92,6 @@ class LibraryDumper
*/
void setContentAccessUrl(const std::string& url) { this->contentAccessUrl = url; }
/**
* Set some informations about the search results.
*
* @param totalResult the total number of results of the search.
* @param startIndex the start index of the result.
* @param count the number of result of the current set (or page).
*/
void setOpenSearchInfo(int totalResult, int startIndex, int count);
/**
* Sets user default language
*
@@ -90,9 +116,6 @@ class LibraryDumper
std::string rootLocation;
std::string contentAccessUrl;
std::string m_userLang;
int m_totalResults;
int m_startIndex;
int m_count;
};
}
+1 -1
View File
@@ -72,7 +72,7 @@ class LibXMLDumper
void setLibrary(const Library* library) { this->library = library; }
protected:
const kiwix::Library* library;
const kiwix::Library* library = nullptr;
std::string baseDir;
private:
void handleBook(Book book, pugi::xml_node root_node);
+72 -34
View File
@@ -20,7 +20,7 @@
#include "manager.h"
#include "tools.h"
#include "tools/pathTools.h"
#include "tools/otherTools.h"
#include <pugixml.hpp>
#include <filesystem>
@@ -32,6 +32,32 @@
namespace fs = std::filesystem;
namespace
{
/**
* The format of the file passed to readFile().
*/
enum class FileFormat { XML, OPDS };
/**
* Detect the format of a library file based on its content.
*
* @param content The content of the file to inspect.
* @return FileFormat::OPDS if the file content looks like an OPDS feed,
* FileFormat::XML otherwise.
*/
FileFormat detectFormat(const std::string& content)
{
auto format
= (content.find("<feed") != std::string::npos)
? FileFormat::OPDS
: FileFormat::XML;
return format;
}
} // anonymous namespace
namespace kiwix
{
@@ -135,33 +161,46 @@ bool Manager::readXml(const std::string& xml,
= doc.load_buffer((void*)xml.data(), xml.size());
if (result) {
this->parseXmlDom(doc, readOnly, libraryPath, trustLibrary);
return this->parseXmlDom(doc, readOnly, libraryPath, trustLibrary);
}
return true;
return false;
}
bool Manager::parseOpdsDom(const pugi::xml_document& doc, const std::string& urlHost)
bool Manager::parseOpdsDom(const pugi::xml_document& doc,
const std::string& urlHost,
const std::string& baseDir,
bool readOnly)
{
pugi::xml_node libraryNode = doc.child("feed");
try {
m_totalBooks = strtoull(libraryNode.child("totalResults").child_value(), 0, 0);
m_startIndex = strtoull(libraryNode.child("startIndex").child_value(), 0, 0);
m_itemsPerPage = strtoull(libraryNode.child("itemsPerPage").child_value(), 0, 0);
m_hasSearchResult = true;
} catch(...) {
if (!libraryNode) {
m_hasSearchResult = false;
return false;
}
const pugi::xml_node totalResultsNode = libraryNode.child("totalResults");
const pugi::xml_node startIndexNode = libraryNode.child("startIndex");
const pugi::xml_node itemsPerPageNode = libraryNode.child("itemsPerPage");
m_hasSearchResult = totalResultsNode && startIndexNode && itemsPerPageNode;
if (m_hasSearchResult) {
m_totalBooks = strtoull(totalResultsNode.child_value(), 0, 0);
m_startIndex = strtoull(startIndexNode.child_value(), 0, 0);
m_itemsPerPage = strtoull(itemsPerPageNode.child_value(), 0, 0);
} else {
m_totalBooks = 0;
m_startIndex = 0;
m_itemsPerPage = 0;
}
for (pugi::xml_node entryNode = libraryNode.child("entry"); entryNode;
entryNode = entryNode.next_sibling("entry")) {
kiwix::Book book;
book.setReadOnly(false);
book.updateFromOpds(entryNode, urlHost);
book.setReadOnly(readOnly);
book.updateFromOpds(entryNode, urlHost, baseDir);
/* Update the book properties with the new importer */
manipulator.addBookToLibrary(book);
@@ -170,17 +209,23 @@ bool Manager::parseOpdsDom(const pugi::xml_document& doc, const std::string& url
return true;
}
bool Manager::readOpds(const std::string& content,
const std::string& urlHost)
{
return readOpds(content, urlHost,false);
}
bool Manager::readOpds(const std::string& content, const std::string& urlHost)
bool Manager::readOpds(const std::string& content,
const std::string& contentOriginUri,
bool readOnly)
{
pugi::xml_document doc;
pugi::xml_parse_result result
= doc.load_buffer((void*)content.data(), content.size());
if (result) {
this->parseOpdsDom(doc, urlHost);
return true;
const auto origin = resolveContentOrigin(contentOriginUri);
return this->parseOpdsDom(doc, origin.urlHost, origin.baseDir, readOnly);
}
return false;
@@ -191,29 +236,22 @@ bool Manager::readFile(
bool readOnly,
bool trustLibrary)
{
bool retVal = true;
pugi::xml_document doc;
#ifdef _WIN32
pugi::xml_parse_result result = doc.load_file(Utf8ToWide(path).c_str());
#else
pugi::xml_parse_result result = doc.load_file(path.c_str());
#endif
if (result) {
this->parseXmlDom(doc, readOnly, path, trustLibrary);
} else {
retVal = false;
}
/* This has to be set (although if the file does not exists) to be
* able to know where to save the library if new content are
* available */
if (!readOnly) {
if (!readOnly) { // todo XXX, better to introduce setWritableLibraryPath and remove this code from the readFile
this->writableLibraryPath = path;
}
return retVal;
if (!kiwix::fileExists(path)) {
return false;
}
const std::string content = getFileContent(path);
return detectFormat(content) == FileFormat::OPDS
? this->readOpds(content, path, readOnly)
: this->readXml(content, readOnly, path, trustLibrary);
}
+11 -50
View File
@@ -46,54 +46,8 @@ const std::string XML_HEADER(R"(<?xml version="1.0" encoding="UTF-8"?>)");
typedef kainjow::mustache::data MustacheData;
typedef kainjow::mustache::list BooksData;
typedef kainjow::mustache::list IllustrationInfo;
IllustrationInfo getBookIllustrationInfo(const Book& book)
{
kainjow::mustache::list illustrations;
for ( const auto& illustration : book.getIllustrations() ) {
// For now, we are handling only sizexsize@1 illustration.
// So we can simply pass one size to mustache.
illustrations.push_back(kainjow::mustache::object{
{"icon_size", to_string(illustration->width)},
{"icon_mimetype", illustration->mimeType}
});
}
return illustrations;
}
std::string fullEntryXML(const Book& book,
const std::string& rootLocation,
const std::string& contentAccessUrl,
const std::string& contentId)
{
const auto bookDate = book.getDate() + "T00:00:00Z";
const kainjow::mustache::object data{
{"root", rootLocation},
{"contentAccessUrl", onlyAsNonEmptyMustacheValue(contentAccessUrl)},
{"id", book.getId()},
{"name", book.getName()},
{"title", book.getTitle()},
{"description", book.getDescription()},
{"language", book.getCommaSeparatedLanguages()},
{"content_id", urlEncode(contentId)},
{"updated", bookDate}, // XXX: this should be the entry update datetime
{"book_date", bookDate},
{"category", book.getCategory()},
{"flavour", book.getFlavour()},
{"tags", book.getTags()},
{"article_count", to_string(book.getArticleCount())},
{"media_count", to_string(book.getMediaCount())},
{"author_name", book.getCreator()},
{"publisher_name", book.getPublisher()},
{"url", onlyAsNonEmptyMustacheValue(book.getUrl())},
{"size", to_string(book.getSize())},
{"icons", getBookIllustrationInfo(book)},
};
return render_template(RESOURCE::templates::catalog_v2_entry_xml, data);
}
std::string partialEntryXML(const Book& book, const std::string& rootLocation)
std::string partialEntryOpds(const Book& book, const std::string& rootLocation)
{
const auto bookDate = book.getDate() + "T00:00:00Z";
const kainjow::mustache::object data{
@@ -120,8 +74,8 @@ BooksData getBooksData(const Library* library,
const Book book = library->getBookByIdThreadSafe(bookId);
const std::string contentId = nameMapper->getNameForId(bookId);
const auto entryXML = partial
? partialEntryXML(book, rootLocation)
: fullEntryXML(book, rootLocation, contentAccessUrl, contentId);
? partialEntryOpds(book, rootLocation)
: fullEntryOpds(book, rootLocation, contentAccessUrl, contentId);
booksData.push_back(kainjow::mustache::object{ {"entry", entryXML} });
} catch ( const std::out_of_range& ) {
// the book was removed from the library since its id was obtained
@@ -179,7 +133,7 @@ std::string OPDSDumper::dumpOPDSCompleteEntry(const std::string& bookId) const
const std::string contentId = nameMapper->getNameForId(bookId);
return XML_HEADER
+ "\n"
+ fullEntryXML(book, rootLocation, contentAccessUrl, contentId);
+ fullEntryOpds(book, rootLocation, contentAccessUrl, contentId);
}
std::string OPDSDumper::categoriesOPDSFeed() const
@@ -212,4 +166,11 @@ std::string OPDSDumper::languagesOPDSFeed() const
);
}
void OPDSDumper::setOpenSearchInfo(int totalResults, int startIndex, int count)
{
m_totalResults = totalResults;
m_startIndex = startIndex;
m_count = count;
}
}
+15
View File
@@ -85,6 +85,21 @@ class OPDSDumper : public LibraryDumper
* @return The OPDS feed.
*/
std::string languagesOPDSFeed() const;
/**
* Set some information about the search results.
*
* @param totalResult the total number of results of the search.
* @param startIndex the start index of the result.
* @param count the number of result of the current set (or page).
*/
void setOpenSearchInfo(int totalResult, int startIndex, int count);
private:
int m_totalResults = 0;
int m_startIndex = 0;
int m_count = 0;
};
}
+6 -12
View File
@@ -181,16 +181,6 @@ int getMHDFlags(IpMode ipMode, bool verbose)
return flags;
}
std::string
fullURL2LocalURL(const std::string& fullUrl, const std::string& rootLocation)
{
if ( kiwix::startsWith(fullUrl, rootLocation) ) {
return fullUrl.substr(rootLocation.size());
} else {
return "INVALID URL";
}
}
std::string getSearchComponent(const RequestContext& request)
{
const std::string query = request.get_query();
@@ -645,11 +635,15 @@ MHD_Result InternalServer::handlerCallback(struct MHD_Connection* connection,
printf("full_url : %s\n", fullUrl);
}
const auto url = fullURL2LocalURL(fullUrl, m_rootPrefixOfDecodedURL);
RequestContext::NameValuePairs headers, queryArgs;
MHD_get_connection_values(connection, MHD_HEADER_KIND, add_name_value_pair, &headers);
MHD_get_connection_values(connection, MHD_GET_ARGUMENT_KIND, add_name_value_pair, &queryArgs);
RequestContext request(m_root, url, method, version, headers, queryArgs);
const int rootPrefixLen = kiwix::startsWith(fullUrl, m_rootPrefixOfDecodedURL)
? int(m_rootPrefixOfDecodedURL.size())
: -1;
RequestContext request(fullUrl, rootPrefixLen, method, version, headers, queryArgs);
if (m_verbose.load() ) {
request.print_debug_info();
+15 -11
View File
@@ -51,14 +51,14 @@ RequestMethod str2RequestMethod(const std::string& method) {
} // unnamed namespace
RequestContext::RequestContext(const std::string& _rootLocation, // URI-encoded
const std::string& unrootedUrl, // URI-decoded
RequestContext::RequestContext(const std::string& _fullUrl, // URI-decoded
int _rootPrefixLength,
const std::string& _method,
const std::string& version,
const NameValuePairs& headers,
const NameValuePairs& queryArgs) :
rootLocation(_rootLocation),
url(unrootedUrl),
fullUrl(_fullUrl),
rootPrefixLength(_rootPrefixLength),
method(str2RequestMethod(_method)),
version(version),
requestIndex(s_requestIndex++),
@@ -128,7 +128,8 @@ void RequestContext::print_debug_info() const {
printf("\n");
}
printf("Parsed : \n");
printf("url : %s\n", url.c_str());
printf("full url: %s\n", fullUrl.c_str());
printf("derooted url: %s\n", get_url().c_str());
printf("acceptEncodingGzip : %d\n", acceptEncodingGzip);
printf("has_range : %d\n", byteRange_.kind() != ByteRange::NONE);
printf("is_valid_url : %d\n", is_valid_url());
@@ -141,11 +142,14 @@ RequestMethod RequestContext::get_method() const {
}
std::string RequestContext::get_url() const {
return url;
return rootPrefixLength < 0
? ""
: fullUrl.substr(rootPrefixLength);
}
std::string RequestContext::get_url_part(int number) const {
size_t start = 1;
const std::string url = get_url();
while(true) {
auto found = url.find('/', start);
if (number == 0) {
@@ -165,14 +169,14 @@ std::string RequestContext::get_url_part(int number) const {
}
std::string RequestContext::get_full_url() const {
return rootLocation + urlEncode(url);
}
std::string RequestContext::get_root_path() const {
return rootLocation.empty() ? "/" : rootLocation;
return urlEncode(fullUrl);
}
bool RequestContext::is_valid_url() const {
if ( rootPrefixLength < 0 )
return false;
const std::string url = get_url();
return url.empty() || url[0] == '/';
}
+4 -5
View File
@@ -59,8 +59,8 @@ class RequestContext {
typedef std::vector<std::pair<const char*, const char*>> NameValuePairs;
public: // functions
RequestContext(const std::string& rootLocation, // URI-encoded
const std::string& unrootedUrl, // URI-decoded
RequestContext(const std::string& fullUrl, // URI-decoded
int rootPrefixLength,
const std::string& method,
const std::string& version,
const NameValuePairs& headers,
@@ -96,7 +96,6 @@ class RequestContext {
std::string get_url() const;
std::string get_url_part(int part) const;
std::string get_full_url() const;
std::string get_root_path() const;
std::string get_query() const { return queryString; }
@@ -139,8 +138,8 @@ class RequestContext {
};
private: // data
std::string rootLocation;
std::string url;
const std::string fullUrl; // URI-decoded
const int rootPrefixLength;
RequestMethod method;
std::string version;
unsigned long long requestIndex;
-4
View File
@@ -105,10 +105,6 @@ bool getArchiveFavicon(const zim::Archive& archive, unsigned size,
return false;
}
unsigned int getArchiveFileSize(const zim::Archive& archive) {
return archive.getFilesize() / 1024;
}
zim::Item getFinalItem(const zim::Archive& archive, const zim::Entry& entry)
{
return entry.getItem(true);
-1
View File
@@ -46,7 +46,6 @@ namespace kiwix
unsigned int getArchiveMediaCount(const zim::Archive& archive);
unsigned int getArchiveArticleCount(const zim::Archive& archive);
unsigned int getArchiveFileSize(const zim::Archive& archive);
zim::Item getFinalItem(const zim::Archive& archive, const zim::Entry& entry);
+15
View File
@@ -410,3 +410,18 @@ std::string kiwix::Suggestions::getJSON() const
return render_template(RESOURCE::templates::suggestion_json, data);
}
kiwix::ContentOrigin kiwix::resolveContentOrigin(const std::string& contentOriginUri)
{
const auto schemeEnd = contentOriginUri.find("://");
if (schemeEnd == std::string::npos) {
return { /*urlHost=*/"", removeLastPathElement(contentOriginUri) };
}
const auto hostEnd = contentOriginUri.find_first_of("/?#", schemeEnd + 3);
const auto urlHost = hostEnd == std::string::npos
? contentOriginUri
: contentOriginUri.substr(0, hostEnd);
return { urlHost, /*baseDir=*/"" };
}
+28
View File
@@ -37,8 +37,36 @@ namespace zim {
class SuggestionItem;
}
namespace kiwix
{
/**
* The pieces needed to resolve references found in OPDS content, split out
* of the location that content was read from. See resolveContentOrigin().
*/
struct ContentOrigin
{
std::string urlHost; /**< scheme+host[:port], no trailing slash; "" if the origin is a local path */
std::string baseDir; /**< parent directory of a local path; "" if the origin is a URL */
};
/** Split a "content origin" locator into a urlHost/baseDir pair.
*
* contentOriginUri is either a URL that some content was fetched from (e.g.
* "https://library.kiwix.org/catalog/v2/entries") or a local filesystem path
* that content was read from (e.g. "/data/library.opds"). Classification is
* purely syntactic: a "://" substring marks the input as a URL; everything
* else (including a Windows drive-letter path) is treated as a local path.
* Exactly one of the two returned fields is non-empty.
*
* Known limitation: a "file://" URI is classified as a URL, not a local
* path (no current caller produces one).
*
* @param contentOriginUri the URL or local path some content was read from.
* @return the corresponding urlHost/baseDir pair.
*/
ContentOrigin resolveContentOrigin(const std::string& contentOriginUri);
std::string nodeToString(const pugi::xml_node& node);
/*
+5 -1
View File
@@ -386,7 +386,11 @@ std::string kiwix::getExecutablePath(bool realPathOnly)
return std::string(binRootPath);
#else
char binRootPath[PATH_MAX];
ssize_t size = readlink("/proc/self/exe", binRootPath, PATH_MAX);
// /proc/self/exe is a kernel-maintained symlink private to this process
// (not an attacker-controllable filename to race), and the resulting
// string below is bounded by the `size` readlink() returns, not by
// (missing) NUL-termination.
ssize_t size = readlink("/proc/self/exe", binRootPath, PATH_MAX); // Flawfinder: ignoreCollapse comment
if (size != -1) {
return std::string(binRootPath, size);
}
+20 -7
View File
@@ -31,6 +31,7 @@
#include <algorithm>
#include <cctype>
#include <cmath>
#include <iostream>
#include <iomanip>
#include <regex>
@@ -55,6 +56,11 @@ std::string asciiToLower(std::string s)
return s;
}
double roundToTwoDecimalPlaces(double value)
{
return std::round(value * 100.0) / 100.0;
}
} // unnamed namespace
/* tell ICU where to find its dat file (tables) */
@@ -126,16 +132,23 @@ std::string kiwix::beautifyInteger(uint64_t number)
std::string kiwix::beautifyFileSize(uint64_t number)
{
constexpr const char* units[] = {"B", "KB", "MB", "GB", "TB", "PB", "EB"};
constexpr size_t unitCount = sizeof(units) / sizeof(units[0]);
constexpr long double unitSize = 1000.0L;
long double value = static_cast<long double>(number);
size_t unit = 0;
while (roundToTwoDecimalPlaces(value) >= unitSize && unit + 1 < unitCount) {
value /= unitSize;
++unit;
}
std::stringstream ss;
ss << std::fixed << std::setprecision(2);
if (number>>30)
ss << (number/(1024.0*1024*1024)) << " GB";
else if (number>>20)
ss << (number/(1024.0*1024)) << " MB";
else if (number>>10)
ss << (number/1024.0) << " KB";
else
if (unit == 0)
ss << number << " B";
else
ss << value << " " << units[unit];
return ss.str();
}
+8 -2
View File
@@ -69,8 +69,8 @@ std::string lcAll(const std::string& word);
std::string ucFirst(const std::string& word);
std::string lcFirst(const std::string& word);
/* This function is broken, related Github issue
* https://github.com/kiwix/libkiwix/issues/1188 */
/* This function is broken, related Github issue
* https://github.com/kiwix/libkiwix/issues/1188 */
std::string toTitle(const std::string& word);
std::string normalize(const std::string& word);
@@ -98,6 +98,12 @@ std::string extractFromString(const std::string& str);
bool startsWith(const std::string& base, const std::string& start);
inline bool contains(const std::string& str, const std::string& substr)
{
return str.find(substr) != std::string::npos;
}
std::string stripSuffix(const std::string& str, const std::string& suffix);
std::vector<std::string> getTitleVariants(const std::string& title);
+1 -1
View File
@@ -42,7 +42,7 @@
"torrent-download-link-text": "بت تورنت",
"torrent-download-alt-text": "التنزيل بواسطة بت تورنت",
"unknown-error": "خطأ غير معروف",
"book-category.wikibooks": "ويكي الكتب",
"book-category.wikibooks": "ويكي كتب",
"book-category.wikinews": "ويكي الأخبار",
"book-category.wikipedia": "ويكيبيديا",
"book-category.wikiquote": "ويكي الإقتباس",
+8 -7
View File
@@ -2,6 +2,7 @@
"@metadata": {
"authors": [
"MathXplore",
"Tensama0415",
"もなー(偽物)"
]
},
@@ -12,7 +13,7 @@
"404-page-title": "コンテンツが見つかりませんでした",
"404-page-heading": "見つかりません",
"500-page-title": "内部サーバーエラー",
"500-page-heading": "内部サーバーエラー",
"500-page-heading": "おおっと、ページが作動しません。",
"fulltext-search-unavailable": "全文検索は利用できません",
"no-search-results": "このコンテンツでは全文検索エンジンが利用できません",
"library-button-text": "ウェルカムページに移動",
@@ -22,12 +23,12 @@
"book-filtering-all-languages": "すべての言語",
"download": "ダウンロード",
"direct-download-link-text": "直ダウンロードリンク",
"direct-download-alt-text": "直ダウンロード",
"hash-download-link-text": "Sha256 ハッシュ",
"hash-download-alt-text": "ハッシュをダウンロード",
"direct-download-alt-text": "HTTP(S)で直ダウンロード",
"hash-download-link-text": "SHA-256チェックサム",
"hash-download-alt-text": "SHA-256ファイルチェックサムを表示",
"magnet-link-text": "マグネットリンク",
"magnet-alt-text": "マグネットダウンロード",
"torrent-download-link-text": "Torrentファイル",
"torrent-download-alt-text": "Torrentダウンロード",
"magnet-alt-text": "マグネットリンクでダウンロード",
"torrent-download-link-text": "BitTorrent",
"torrent-download-alt-text": "BitTorrentダウンロード",
"preview-book": "プレビュー"
}
+14 -2
View File
@@ -1,6 +1,7 @@
{
"@metadata": {
"authors": [
"Tensama0415",
"YeBoy371",
"Ykhwong"
]
@@ -13,7 +14,7 @@
"url-not-found": "\"{{url}}\" 요청 URL은 이 서버에서 찾을 수 없습니다.",
"suggest-search": "<a href=\"{{{SEARCH_URL}}}\">{{PATTERN}}</a>에 대한 전문 검색을 수행해 보세요",
"random-article-failure": "이런! 임의 문서를 선택하지 못했습니다 :(",
"invalid-raw-data-type": "{{DATATYPE}} 값은 원 콘텐츠에 대한 유효한 요청이 아닙니다.",
"invalid-raw-data-type": "{{DATATYPE}} 값은 원 콘텐츠에 대한 유효한 요청이 아닙니다.",
"invalid-request": "\"{{{url}}}\" 요청 URL은 유효한 요청이 아닙니다.",
"no-value-for-arg": "{{ARGUMENT}} 인수에 지정된 값이 없습니다",
"no-query": "지정된 쿼리가 없습니다.",
@@ -26,13 +27,21 @@
"new-404-page-heading": "이런. 페이지를 찾을 수 없습니다.",
"404-img-text": "찾을 수 없습니다!",
"path-was-not-found": "요청한 경로를 찾을 수 없습니다:",
"404-advice.p1": "찾으시는 콘텐츠는 여전히 존재할 수 있지만, ZIM 파일 내의 다른 위치에 있을 수 있습니다.",
"404-advice.p2": "부탁:",
"404-advice.p3": "원하는 콘텐츠를 찾으려면 검색 기능을 사용해 보세요",
"404-advice.p4": "찾고자 하는 정보와 관련된 키워드나 제목을 찾아보세요.",
"404-advice.p5": "이 방법을 사용하면 원래 링크가 제대로 작동하지 않더라도 원하는 콘텐츠를 찾을 수 있습니다.",
"500-page-title": "내부 서버 오류",
"500-page-heading": "죄송합니다. 문서가 동작하지 않습니다.",
"500-page-text": "요청된 경로를 제대로 전달할 수 없습니다:",
"500-img-text": "문서가 동작하지 않습니다",
"external-link-detected": "외부 링크가 발견되었습니다",
"caution-warning": "경고!",
"external-link-intro": "Kiwix의 ZIM 리더를 종료하고 온라인 페이지로 이동하려고 합니다",
"external-link-advice.p1": "접속하려는 링크는 오프라인 패키지에 포함되어 있지 않으며 인터넷 연결이 필요합니다.",
"external-link-advice.p2": "인터넷에 접속할 수 있다면 링크를 열어보세요.",
"external-link-advice.p3": "브라우저의 뒤로 가기 버튼을 사용하면 ZIM의 오프라인 콘텐츠로 돌아갈 수 있습니다.",
"fulltext-search-unavailable": "전문 검색을 사용할 수 없습니다",
"no-search-results": "이 콘텐츠에는 전문 검색 엔진을 사용할 수 없습니다.",
"search-results-page-title": "검색: {{SEARCH_PATTERN}}",
@@ -40,9 +49,11 @@
"empty-search-results-page-header": "<b>\"{{{SEARCH_PATTERN}}}\"</b>의 결과가 없습니다",
"search-result-book-info": "{{BOOK_TITLE}}에서",
"word-count": "단어 {{COUNT}}개",
"library-button-text": "환영 페이지로 이동",
"home-button-text": "'{{{BOOK_TITLE}}}'의 메인 페이지로 이동",
"random-page-button-text": "무작위로 선택된 문서로 이동",
"searchbox-tooltip": "'{{{BOOK_TITLE}}}' 검색",
"confusion-of-tongues": "서로 다른 언어로 된 두 권 이상의 책이 검색에 나오게 되면 혼란스러울 수 있습니다.",
"welcome-page-overzealous-filter": "결과가 없습니다. <a href=\"{{URL}}\">필터를 재설정</a>하시겠습니까?",
"powered-by-kiwix-html": "<a href=\"https://kiwix.org\">Kiwix</a>에서 제공",
"search": "검색",
@@ -76,5 +87,6 @@
"book-category.wikiversity": "위키배움터",
"book-category.wikivoyage": "위키여행",
"book-category.wiktionary": "위키낱말사전",
"book-category.other": "기타"
"book-category.other": "기타",
"text-loading-content": "내용을 불러오는 중"
}
+3
View File
@@ -13,6 +13,9 @@
"url-not-found": "Déi ugefroten URL „{{url}}“ gouf op dësem Server net fonnt.",
"suggest-search": "Maacht eng Volltext-Sich fir <a href=\"{{{SEARCH_URL}}}\">{{PATTERN}}</a>",
"random-article-failure": "Ups! Et konnt keen zoufällegen Artikel ausgewielt ginn :(",
"no-value-for-arg": "Kee Wäert ugi fir d'Argument {{ARGUMENT}}",
"400-page-title": "Ongülteg Ufro",
"400-page-heading": "Ongülteg Ufro",
"404-page-title": "Inhalt net fonnt",
"404-page-heading": "Net fonnt",
"new-404-page-title": "Säit net fonnt",
+3 -1
View File
@@ -1,6 +1,7 @@
{
"@metadata": {
"authors": [
"Hakimi97",
"Tofeiku"
]
},
@@ -16,5 +17,6 @@
"download": "Muat turun",
"direct-download-link-text": "Langsung",
"direct-download-alt-text": "muat turun langsung",
"download-links-title": "Muat turun buku"
"download-links-title": "Muat turun buku",
"book-category.wikibooks": "Wikibuku"
}
+4 -4
View File
@@ -15,14 +15,14 @@
"no-book-found": "Er zijn geen boeken die overeenkomen met de zoekcriteria",
"url-not-found": "De opgevraagde URL “{{url}}” is niet gevonden op deze server.",
"suggest-search": "In volledige tekst zoeen naar <a href=\"{{{SEARCH_URL}}}\">{{PATTERN}}</a>",
"random-article-failure": "Oeps! Kan geen willekeurig artikel kiezen :(",
"random-article-failure": "Oeps! Kan geen willekeurige pagina kiezen :(",
"invalid-raw-data-type": "{{DATATYPE}} is geen geldig verzoek voor onbewerkte inhoud.",
"invalid-request": "De gevraagde URL “{{{url}}}” is onjuist.",
"invalid-request": "De gevraagde url “{{{url}}}” is onjuist.",
"no-value-for-arg": "Er is geen waarde opgegeven bij {{ARGUMENT}}",
"no-query": "Er is geen zoekterm opgegeven.",
"raw-entry-not-found": "Kan het {{DATATYPE}}-item {{ENTRY}} niet vinden",
"400-page-title": "Ongeldig verzoek",
"400-page-heading": "Ongeldig verzoek",
"400-page-title": "Onjuist verzoek",
"400-page-heading": "Onjuist verzoek",
"404-page-title": "Inhoud niet gevonden",
"404-page-heading": "Niet gevonden",
"new-404-page-title": "Pagina niet gevonden",
+71 -3
View File
@@ -1,6 +1,7 @@
{
"@metadata": {
"authors": [
"Wizzard",
"Yardom78"
]
},
@@ -13,6 +14,7 @@
"suggest-search": "Spustite hľadanie celého textu <a href=\"{{{SEARCH_URL}}}\">{{PATTERN}}</a>",
"random-article-failure": "Nepodarilo sa vybrať náhodný článok :(",
"invalid-raw-data-type": "{{DATATYPE}} nie je platná požiadavka pre surový obsah.",
"invalid-request": "Požadovaná URL „{{{url}}}“ nie je platná požiadavka.",
"no-value-for-arg": "Pre argument {{ARGUMENT}} nebola poskytnutá žiadna hodnota",
"no-query": "Nebol poskytnutý žiadny dopyt.",
"raw-entry-not-found": "Nepodarilo sa nájsť {{DATATYPE}} položka {{ENTRY}}",
@@ -20,12 +22,78 @@
"400-page-heading": "Neplatná požiadavka",
"404-page-title": "Obsah nebol nájdený",
"404-page-heading": "Nenájdené",
"new-404-page-title": "Stránka sa nenašla",
"new-404-page-heading": "Ejha. Stránka sa nenašla.",
"404-img-text": "Nenájdené!",
"path-was-not-found": "Požadovaná cesta sa nenašla:",
"404-advice.p1": "Obsah, ktorý hľadáte, môže byť stále dostupný, ale môže sa nachádzať na inom mieste v súbore ZIM.",
"404-advice.p2": "Prosím:",
"404-advice.p3": "Skúste použiť funkciu vyhľadávania na nájdenie požadovaného obsahu",
"404-advice.p4": "Hľadajte kľúčové slová alebo názvy súvisiace s informáciami, ktoré hľadáte",
"404-advice.p5": "Tento postup by vám mal pomôcť nájsť požadovaný obsah, aj keď pôvodný odkaz nefunguje správne.",
"500-page-title": "Chyba interného servera",
"500-page-heading": "Chyba interného servera",
"500-page-heading": "Ejha. Stránka nefunguje.",
"500-page-text": "Požadovanú cestu nemožno správne doručiť:",
"500-img-text": "Stránka nefunguje",
"external-link-detected": "Zistený externý odkaz",
"caution-warning": "Pozor!",
"external-link-intro": "Chystáte sa opustiť čítačku ZIM od Kiwixu a prejsť online na",
"external-link-advice.p1": "Odkaz, na ktorý sa pokúšate pristúpiť, nie je súčasťou vášho offline balíka a vyžaduje pripojenie na internet.",
"external-link-advice.p2": "Ak sa môžete pripojiť na internet, môžete sa pokúsiť odkaz otvoriť.",
"external-link-advice.p3": "V opačnom prípade sa môžete vrátiť k offline obsahu svojho súboru ZIM pomocou tlačidla Späť vo vašom prehliadači.",
"fulltext-search-unavailable": "Vyhľadávanie celého textu nie je dostupné",
"no-search-results": "Funkcia na vyhľadávanie celého textu nie je pre tento obsah dostupná.",
"search-results-page-title": "Vyhľadávanie: {{SEARCH_PATTERN}}",
"search-results-page-header": "Výsledky <b>{{START}}-{{END}}</b> z <b>{{COUNT}}</b> pre <b>„{{{SEARCH_PATTERN}}}“</b>",
"empty-search-results-page-header": "Pre <b>„{{{SEARCH_PATTERN}}}“</b> sa nenašli žiadne výsledky",
"search-result-book-info": "z {{BOOK_TITLE}}",
"word-count": "{{COUNT}} slov",
"library-button-text": "Prejsť na uvítaciu stránku",
"home-button-text": "Prejsť na hlavnú stránku knihy {{BOOK_TITLE}}",
"home-button-text": "Prejsť na hlavnú stránku „{{{BOOK_TITLE}}}“",
"random-page-button-text": "Prejsť na náhodne vybranú stránku",
"searchbox-tooltip": "Vyhľadať {{BOOK_TITLE}}"
"searchbox-tooltip": "Hľadať v „{{{BOOK_TITLE}}}“",
"confusion-of-tongues": "Vo vyhľadávaní by sa zúčastnili dve alebo viac kníh v rôznych jazykoch, čo môže viesť k mätúcim výsledkom.",
"welcome-page-overzealous-filter": "Žiadny výsledok. Chcete <a href=\"{{URL}}\">obnoviť filter</a>?",
"powered-by-kiwix-html": "Poháňané&nbsp;<a href=\"https://kiwix.org\">Kiwixom</a>",
"search": "Hľadať",
"book-filtering-all-categories": "Všetky kategórie",
"book-filtering-all-languages": "Všetky jazyky",
"count-of-matching-books": "{{COUNT}} kníh",
"download": "Stiahnuť",
"direct-download-link-text": "Priamo",
"direct-download-alt-text": "Stiahnuť priamo cez HTTP(S)",
"hash-download-link-text": "Kontrolný súčet SHA-256",
"hash-download-alt-text": "Zobraziť kontrolný súčet SHA-256 súboru",
"magnet-link-text": "Magnet odkaz",
"magnet-alt-text": "Stiahnuť cez Magnet odkaz",
"torrent-download-link-text": "BitTorrent",
"torrent-download-alt-text": "Stiahnuť cez BitTorrent",
"library-opds-feed-all-entries": "OPDS kanál knižnice všetky položky",
"filter-by-tag": "Filtrovať podľa značky „{{{TAG}}}“",
"stop-filtering-by-tag": "Prestať filtrovať podľa značky „{{{TAG}}}“",
"library-opds-feed-parameterised": "OPDS kanál knižnice položky zodpovedajúce {{#LANG}}\nJazyk: {{LANG}} {{/LANG}}{{#CATEGORY}}\nKategória: {{CATEGORY}} {{/CATEGORY}}{{#TAG}}\nZnačka: {{TAG}} {{/TAG}}{{#Q}}\nDopyt: {{Q}} {{/Q}}",
"welcome-to-kiwix-server": "Vitajte na serveri Kiwix",
"download-links-heading": "Odkazy na stiahnutie pre <b><i>{{BOOK_TITLE}}</i></b>",
"download-links-title": "Stiahnuť knihu",
"preview-book": "Náhľad",
"unknown-error": "Neznáma chyba",
"book-category.gutenberg": "Gutenberg",
"book-category.iFixit": "iFixit",
"book-category.mooc": "MOOC",
"book-category.phet": "Phet",
"book-category.stack_exchange": "Stack Exchange",
"book-category.ted": "Ted",
"book-category.vikidia": "Vikidia",
"book-category.wikibooks": "Wikibooks",
"book-category.wikihow": "wikiHow",
"book-category.wikinews": "Wikinews",
"book-category.wikipedia": "Wikipedia",
"book-category.wikiquote": "Wikiquote",
"book-category.wikisource": "Wikisource",
"book-category.wikispecies": "Wikispecies",
"book-category.wikiversity": "Wikiversity",
"book-category.wikivoyage": "Wikivoyage",
"book-category.wiktionary": "Wiktionary",
"book-category.other": "Ostatné",
"text-loading-content": "Načítava sa obsah"
}
+1
View File
@@ -5,6 +5,7 @@
"GuoPC",
"IceButBin",
"Kichin",
"Peterxy",
"Peterxy12",
"Prmsh",
"StarrySky",
-56
View File
@@ -244,59 +244,6 @@
}
}
function makeURLSearchString(params, keysToURIEncode) {
let output = '';
for (const [key, value] of params.entries()) {
let finalValue = (keysToURIEncode.indexOf(key) >= 0) ? encodeURIComponent(value) : value;
output += `&${key}=${finalValue}`;
}
// exclude first char so the first params are not prefixed with &
return output.substring(1);
}
/* hack for library.kiwix.org magnet links (created by MirrorBrain)
See https://github.com/kiwix/container-images/issues/242 */
async function getFixedMirrorbrainMagnet(magnetLink) {
// parse as query parameters
const params = new URLSearchParams(
magnetLink.replaceAll('&amp;', '&').replace(/^magnet:/, ''));
const zimUrl = params.get('as'); // as= is fallback URL
// download metalink to build list of mirrored URLs
let mirrorUrls = [];
const metalink = await fetch(`${zimUrl}.meta4`).then(response => {
return response.ok ? response.text() : '';
}).catch((_error) => '');
if (metalink) {
try {
const parser = new DOMParser();
const doc = parser.parseFromString(metalink, "application/xml");
doc.querySelectorAll("url").forEach((node) => {
if (node.hasAttribute("priority")) { // ensures its a mirror link
mirrorUrls.push(node.innerHTML);
}
});
} catch (err) {
// not a big deal, magnet will only contain primary URL
console.debug(`Failed to parse mirror links for ${zimUrl}`);
}
}
// set webseed (ws=) URL to primary download URL (redirects to mirror)
params.set('ws', zimUrl);
// if we got metalink mirror URLs, append them all
if (mirrorUrls) {
mirrorUrls.forEach((url) => {
params.append('ws', url);
});
}
params.set('xs', `${zimUrl}.torrent`); // adding xs= to point to torrent URL
return 'magnet:?' + makeURLSearchString(params, ['ws', 'as', 'dn', 'xs', 'tr']);
}
async function getMagnetLink(downloadLink) {
const magnetUrl = downloadLink + '.magnet';
const controller = new AbortController();
@@ -304,9 +251,6 @@
const magnetLink = await fetch(magnetUrl, { signal: controller.signal }).then(response => {
return response.ok ? response.text() : '';
}).catch((_error) => '');
if (magnetLink) {
return await getFixedMirrorbrainMagnet(magnetLink);
}
return magnetLink;
}
-8
View File
@@ -228,12 +228,6 @@ function updateToolbarVisibilityState() {
previousScrollTop = st;
}
function handle_visual_viewport_change() {
const wh = window.visualViewport
? window.visualViewport.height
: window.innerHeight;
contentIframe.height = wh - contentIframe.offsetTop - 4;
}
function setIframeUrl(path) {
try {
@@ -583,9 +577,7 @@ function setupViewer() {
// Defer the call of handle_visual_viewport_change() until after the
// presence or absence of the taskbar as determined by this function
// has been settled.
setTimeout(handle_visual_viewport_change, 0);
window.onresize = handle_visual_viewport_change;
const kiwixToolBarWrapper = document.getElementById('kiwixtoolbarwrapper');
if ( ! viewerSettings.toolbarEnabled ) {
+7 -7
View File
@@ -10,12 +10,12 @@
<tags>{{tags}}</tags>
<articleCount>{{article_count}}</articleCount>
<mediaCount>{{media_count}}</mediaCount>
{{#icons}}<link rel="http://opds-spec.org/image/thumbnail"
href="{{root}}/catalog/v2/illustration/{{{id}}}/?size={{icon_size}}"
type="{{icon_mimetype}};width={{icon_size}};height={{icon_size}};scale=1"/>
{{/icons}}{{#contentAccessUrl}}<link type="text/html" href="{{contentAccessUrl}}/{{{content_id}}}" />
{{/contentAccessUrl}}
<author>
{{#thumbnailLinks}}<link rel="http://opds-spec.org/image/thumbnail"
href="{{icon_url}}"
type="{{icon_mimetype}}"/>
{{/thumbnailLinks}}{{#contentAccessUrl}}
<link type="text/html" href="{{contentAccessUrl}}/{{{content_id}}}" />
{{/contentAccessUrl}}<author>
<name>{{author_name}}</name>
</author>
<publisher>
@@ -24,5 +24,5 @@
<dc:issued>{{book_date}}</dc:issued>
{{#url}}
<link rel="http://opds-spec.org/acquisition/open-access" type="application/x-zim" href="{{{url}}}" length="{{{size}}}" />
{{/url}}
{{/url}}{{#local_path}}<link rel="http://opds-spec.org/acquisition/open-access" href="{{local_path}}" type="application/x-zim" length="{{{size}}}"/>{{/local_path}}
</entry>
+10
View File
@@ -7,6 +7,16 @@
frame-src 'self';
object-src 'none';">
<title>ZIM Viewer</title>
<style>
html {
height: 100%;
}
html, body, #content_iframe {
display: flex;
flex-direction: column;
flex: 1;
}
</style>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link type="text/css" href="./skin/kiwix.css?KIWIXCACHEID" rel="Stylesheet" />
<link type="text/css" href="./skin/taskbar.css?KIWIXCACHEID" rel="Stylesheet" />
+351
View File
@@ -1,6 +1,8 @@
#include "gtest/gtest.h"
#include "../include/book.h"
#include "testing_tools.h"
#include <pugixml.hpp>
#include <zim/archive.h>
namespace
{
@@ -66,6 +68,130 @@ TEST(BookTest, updateFromXMLTest)
EXPECT_EQ(defaultIllustration->url, "http://who.org/zara.fav");
}
// OPDS analogue of updateFromXMLTest above.
TEST(BookTest, updateFromOPDSTest)
{
const XMLDoc opds(R"(
<entry>
<id>urn:uuid:zara</id>
<title>Catch an infection in 24 hours</title>
<summary>Complete guide to contagious diseases</summary>
<name>who_contagious_diseases_en</name>
<tags>unittest;_category:medicine;_pictures:yes</tags>
<category>medicine</category>
<articleCount>123456</articleCount>
<mediaCount>234567</mediaCount>
<link rel="http://opds-spec.org/acquisition/open-access"
type="application/x-zim"
href="https://who.org/zara.zim"
length="345678" />
<link rel="http://opds-spec.org/image/thumbnail"
type="text/plain"
href="/zara.fav" />
</entry>
)");
kiwix::Book book;
book.updateFromOpds(opds.child("entry"), "http://who.org", "");
EXPECT_EQ(book.getPath(), "");
EXPECT_FALSE(book.isPathValid());
// The "urn:uuid:" prefix (as used by <id> in real OPDS feeds) must be
// stripped.
EXPECT_EQ(book.getId(), "zara");
EXPECT_EQ(book.getUrl(), "https://who.org/zara.zim");
EXPECT_EQ(book.getTitle(), "Catch an infection in 24 hours");
EXPECT_EQ(book.getDescription(), "Complete guide to contagious diseases");
EXPECT_EQ(book.getTags(), "unittest;_category:medicine;_pictures:yes");
EXPECT_EQ(book.getName(), "who_contagious_diseases_en");
EXPECT_EQ(book.getCategory(), "medicine");
EXPECT_EQ(book.getArticleCount(), 123456U);
EXPECT_EQ(book.getMediaCount(), 234567U);
// Unlike updateFromXml()'s "size" attribute (interpreted in KiB and
// converted to bytes), the OPDS acquisition link's "length" is taken
// as-is, already in bytes.
EXPECT_EQ(book.getSize(), 345678U);
// Unlike updateFromXml()'s "favicon" attribute (which embeds the
// actual base64-encoded image data directly), OPDS only ever gives us
// a URL to fetch the thumbnail from later, so there's no embedded data
// to check here - just that the URL (prefixed with urlHost) and mime
// type made it through.
auto defaultIllustration = book.getIllustration(48);
EXPECT_EQ(defaultIllustration->mimeType, "text/plain");
EXPECT_EQ(defaultIllustration->url, "http://who.org/zara.fav");
}
TEST(BookTest, updateFromOPDSLocalPathAcquisitionLinkTest)
{
const XMLDoc opds(R"(
<entry>
<id>urn:uuid:zara</id>
<link rel="http://opds-spec.org/acquisition/open-access" href="zara.zim" />
</entry>
)");
kiwix::Book book;
book.updateFromOpds(opds.child("entry"), "http://who.org", DATA_ABS_PATH);
EXPECT_EQ(book.getPath(), ZARA_ABS_PATH);
EXPECT_EQ(book.getUrl(), "");
}
TEST(BookTest, updateFromOPDSTwoAcquisitionLinksTest)
{
const XMLDoc opds(R"(
<entry>
<id>urn:uuid:zara</id>
<link rel="http://opds-spec.org/acquisition/open-access"
type="application/x-zim"
href="zara.zim"
length="111" />
<link rel="http://opds-spec.org/acquisition/open-access"
type="application/x-zim"
href="https://who.org/zara.zim"
length="222" />
</entry>
)");
kiwix::Book book;
book.updateFromOpds(opds.child("entry"), "http://who.org", DATA_ABS_PATH);
EXPECT_EQ(book.getPath(), ZARA_ABS_PATH);
EXPECT_EQ(book.getUrl(), "https://who.org/zara.zim");
EXPECT_EQ(book.getSize(), 222U);
}
TEST(BookTest, updateFromOPDSDuplicateLengthWarnsTest)
{
const XMLDoc opds(R"(
<entry>
<id>urn:uuid:zara</id>
<link rel="http://opds-spec.org/acquisition/open-access"
type="application/x-zim"
href="zara.zim"
length="111" />
<link rel="http://opds-spec.org/acquisition/open-access"
type="application/x-zim"
href="https://who.org/zara.zim"
length="222" />
</entry>
)");
kiwix::Book book;
kiwix::testing::CapturedStderr stderror;
book.updateFromOpds(opds.child("entry"), "http://who.org", DATA_ABS_PATH);
EXPECT_EQ(
"Book 'zara': acquisition links 'zara.zim' (length 111) and "
"'https://who.org/zara.zim' (length 222) disagree on length.\n",
std::string(stderror));
EXPECT_EQ(book.getSize(), 222U);
}
namespace
{
@@ -77,6 +203,14 @@ kiwix::Book makeBook(const std::string& attr, const std::string& baseDir="")
return book;
}
kiwix::Book makeBookFromOpds(const std::string& entryContent, const std::string& urlHost="", const std::string& baseDir="")
{
const XMLDoc opds("<entry>" + entryContent + "</entry>");
kiwix::Book book;
book.updateFromOpds(opds.child("entry"), urlHost, baseDir);
return book;
}
} // unnamed namespace
TEST(BookTest, updateFromXMLCategoryHandlingTest)
@@ -117,6 +251,176 @@ TEST(BookTest, updateFromXMLCategoryHandlingTest)
}
}
TEST(BookTest, updateFromOPDSCategoryHandlingTest)
{
{
const kiwix::Book book = makeBookFromOpds(R"(
<id>abcd</id>
<tags>_category:category_defined_via_tags_only</tags>
)");
EXPECT_EQ(book.getCategory(), "category_defined_via_tags_only");
}
{
const kiwix::Book book = makeBookFromOpds(R"(
<id>abcd</id>
<category>category_defined_via_element_only</category>
)");
EXPECT_EQ(book.getCategory(), "category_defined_via_element_only");
}
{
const kiwix::Book book = makeBookFromOpds(R"(
<id>abcd</id>
<category>category_element_overrides_tags</category>
<tags>_category:tags_override_category_element</tags>
)");
EXPECT_EQ(book.getCategory(), "category_element_overrides_tags");
}
{
const kiwix::Book book = makeBookFromOpds(R"(
<id>abcd</id>
<tags>_category:tags_override_category_element</tags>
<category>category_element_overrides_tags</category>
)");
EXPECT_EQ(book.getCategory(), "category_element_overrides_tags");
}
}
TEST(BookTest, updateFromOPDSThumbnailWithAbsoluteHrefIgnoresUrlHostTest)
{
// An already-absolute href (as real OPDS catalogs commonly send, see
// test/data/library.opds) must be left untouched, not prefixed with
// urlHost - concatenating the two would produce a garbled URL. This is
// the counterpart to updateFromOPDSTest above, which covers the
// relative-href case (where prefixing with urlHost IS expected).
const kiwix::Book book = makeBookFromOpds(R"(
<link rel="http://opds-spec.org/image/thumbnail"
type="image/png"
href="https://example.com/favicon/zara.png" />
)", "http://who.org");
const auto illustration = book.getIllustrations().at(0);
EXPECT_EQ(illustration->url, "https://example.com/favicon/zara.png");
}
TEST(BookTest, updateFromOPDSThumbnailLinkTypeWithSizeSuffixTest)
{
const kiwix::Book book = makeBookFromOpds(R"(
<link rel="http://opds-spec.org/image/thumbnail"
type="image/png;width=96;height=256;scale=1"
href="https://example.com/zara.png" />
)");
const auto illustration = book.getIllustrations().at(0);
EXPECT_EQ(illustration->mimeType, "image/png");
EXPECT_EQ(illustration->width, 96);
EXPECT_EQ(illustration->height, 256);
}
TEST(BookTest, updateFromOPDSThumbnailLinkTypeWithPartialSizeSuffixTest)
{
const kiwix::Book book = makeBookFromOpds(R"(
<link rel="http://opds-spec.org/image/thumbnail"
type="image/png;width=96"
href="https://example.com/zara.png" />
)");
const auto illustration = book.getIllustrations().at(0);
EXPECT_EQ(illustration->mimeType, "image/png");
EXPECT_EQ(illustration->width, 96);
EXPECT_EQ(illustration->height, 48);
}
TEST(BookTest, updateFromOPDSDataUriThumbnailTest)
{
const kiwix::Book book = makeBookFromOpds(R"(
<link rel="http://opds-spec.org/image/thumbnail"
type="image/jpeg;width=96;height=256;scale=1"
href="data:image/jpeg;base64,Zmlyc3QtdGh1bWJuYWls" />
)");
const auto illustration = book.getIllustrations().at(0);
EXPECT_EQ(illustration->getData(), "first-thumbnail");
EXPECT_EQ(illustration->url, "");
EXPECT_EQ(illustration->mimeType, "image/jpeg");
EXPECT_EQ(illustration->width, 96);
EXPECT_EQ(illustration->height, 256);
}
TEST(BookTest, updateFromOPDSMultipleDataUriThumbnailsTest)
{
const kiwix::Book book = makeBookFromOpds(R"(
<link rel="http://opds-spec.org/image/thumbnail"
type="image/png;width=48;height=48;scale=1"
href="data:image/png;base64,Zmlyc3QtdGh1bWJuYWls" />
<link rel="http://opds-spec.org/image/thumbnail"
type="image/jpeg;width=96;height=96;scale=1"
href="data:image/jpeg;base64,c2Vjb25kLXRodW1ibmFpbA==" />
)");
const auto& illustrations = book.getIllustrations();
ASSERT_EQ(illustrations.size(), 2U);
EXPECT_EQ(illustrations[0]->getData(), "first-thumbnail");
EXPECT_EQ(illustrations[0]->width, 48);
EXPECT_EQ(illustrations[0]->height, 48);
EXPECT_EQ(illustrations[1]->getData(), "second-thumbnail");
EXPECT_EQ(illustrations[1]->width, 96);
EXPECT_EQ(illustrations[1]->height, 96);
}
TEST(BookTest, updateFromOPDSMixedDataUriAndExternalThumbnailLinksTest)
{
const kiwix::Book book = makeBookFromOpds(R"(
<link rel="http://opds-spec.org/image/thumbnail"
type="image/png"
href="https://example.com/favicon.png" />
<link rel="http://opds-spec.org/image/thumbnail"
type="image/jpeg"
href="data:image/jpeg;base64,Zmlyc3QtdGh1bWJuYWls" />
)");
const auto& illustrations = book.getIllustrations();
ASSERT_EQ(illustrations.size(), 2U);
EXPECT_EQ(illustrations[0]->url, "https://example.com/favicon.png");
EXPECT_EQ(illustrations[1]->url, "");
EXPECT_EQ(illustrations[1]->getData(), "first-thumbnail");
}
TEST(BookTest, updateFromOPDSDataUriThumbnailWithoutCommaTest)
{
const kiwix::Book book = makeBookFromOpds(R"(
<link rel="http://opds-spec.org/image/thumbnail"
type="image/png"
href="data:image/png;base64" />
)");
const auto illustration = book.getIllustrations().at(0);
EXPECT_EQ(illustration->getData(), "");
EXPECT_EQ(illustration->url, "");
}
TEST(BookTest, updateFromOPDSThumbnailLinkWithoutTypeIsIgnoredTest)
{
const kiwix::Book book = makeBookFromOpds(R"(
<link rel="http://opds-spec.org/image/thumbnail"
href="https://example.com/favicon.png" />
)");
EXPECT_TRUE(book.getIllustrations().empty());
}
TEST(BookTest, updateFromOPDSNoThumbnailsTest)
{
const kiwix::Book book = makeBookFromOpds(R"(
<id>abcd</id>
)");
EXPECT_TRUE(book.getIllustrations().empty());
}
TEST(BookTest, setTagsDoesntAffectCategory)
{
kiwix::Book book;
@@ -175,6 +479,14 @@ TEST(BookTest, updateTest)
EXPECT_EQ(newDefaultIllustration->mimeType, defaultIllustration->mimeType);
}
TEST(BookTest, updateFromArchiveSetsByteExactSize)
{
const zim::Archive archive("./test/zimfile.zim");
kiwix::Book book;
book.update(archive);
EXPECT_EQ(book.getSize(), archive.getFilesize());
}
namespace
{
@@ -222,3 +534,42 @@ TEST(BookTest, getLanguages)
EXPECT_EQ(book.getLanguages(), Langs({ "eng", "ong", "ing" }));
}
}
// OPDS analogue of getLanguages above.
TEST(BookTest, getLanguagesOpds)
{
typedef std::vector<std::string> Langs;
{
const kiwix::Book book = makeBookFromOpds("<id>abcd</id><language>fra</language>");
EXPECT_EQ(book.getCommaSeparatedLanguages(), "fra");
EXPECT_EQ(book.getLanguages(), Langs{ "fra" });
}
{
const kiwix::Book book = makeBookFromOpds("<id>abcd</id><language>eng,ong,ing</language>");
EXPECT_EQ(book.getCommaSeparatedLanguages(), "eng,ong,ing");
EXPECT_EQ(book.getLanguages(), Langs({ "eng", "ong", "ing" }));
}
}
TEST(BookTest, updateFromOPDSMultipleThumbnailLinksTest)
{
// Several rel="...thumbnail" links (e.g. one per size) must all be kept,
// not just the last one seen.
const kiwix::Book book = makeBookFromOpds(R"(
<link rel="http://opds-spec.org/image/thumbnail"
type="image/png"
href="https://example.com/zara-48.png" />
<link rel="http://opds-spec.org/image/thumbnail"
type="image/png"
href="https://example.com/zara-96.png" />
)");
const auto& illustrations = book.getIllustrations();
ASSERT_EQ(illustrations.size(), 2U);
EXPECT_EQ(illustrations[0]->url, "https://example.com/zara-48.png");
EXPECT_EQ(illustrations[1]->url, "https://example.com/zara-96.png");
}
+45
View File
@@ -0,0 +1,45 @@
<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:dc="http://purl.org/dc/terms/"
xmlns:opds="http://opds-spec.org/2010/catalog">
<id>32f4d6e1-1a5b-4c9a-9d4a-0b8f2f7f9c3a</id>
<title>lib_for_server_search_test</title>
<entry>
<id>urn:uuid:5dc0b3af-5df2-0925-f0ca-d2bf75e78af6</id>
<title>Wikibooks</title>
<updated>2021-04-17T00:00:00Z</updated>
<summary>testZim</summary>
<language>eng</language>
<name>bookname_of_example_zim</name>
<tags>_ftindex:yes;_ftindex:yes;_pictures:yes;_videos:yes;_details:yes</tags>
<mediaCount>22</mediaCount>
<author>
<name>test</name>
</author>
<publisher>
<name>test</name>
</publisher>
<dc:issued>2021-04-17T00:00:00Z</dc:issued>
<link rel="http://opds-spec.org/acquisition/open-access" href="./example.zim" type="application/x-zim"/>
</entry>
<entry>
<id>urn:uuid:6f1d19d0-633f-087b-fb55-7ac324ff9baf</id>
<title>Ray Charles</title>
<updated>2020-03-31T00:00:00Z</updated>
<summary>Wikipedia articles about Ray Charles</summary>
<language>eng</language>
<name>wikipedia_en_ray_charles</name>
<flavour>_mini</flavour>
<tags>wikipedia;_category:wikipedia;_pictures:no;_videos:no;_details:no;_ftindex:yes</tags>
<articleCount>129</articleCount>
<mediaCount>45</mediaCount>
<author>
<name>Wikipedia</name>
</author>
<publisher>
<name>Kiwix</name>
</publisher>
<dc:issued>2020-03-31T00:00:00Z</dc:issued>
<link rel="http://opds-spec.org/acquisition/open-access" href="./zimfile.zim" type="application/x-zim"/>
</entry>
</feed>
+111
View File
@@ -0,0 +1,111 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
OPDS analogue of library.xml - same four books, with each XML attribute
mapped to its corresponding OPDS element/link (see Book::updateFromOpds()
in src/book.cpp):
id -> <id>urn:uuid:ID</id> ("urn:uuid:" is
stripped by updateFromOpds())
path -> an acquisition <link>'s href, when that href
is not an absolute URL - resolved against
the library file's own directory just like
XML's "path" attribute (see
Book::updateFromOpds())
url -> an acquisition <link>'s href, when that href
IS an absolute URL
size (KiB, XML) -> an acquisition <link>'s length (bytes, i.e.
size * 1024)
title/description/ -> <title>/<summary>/<language>/<name>/<tags>
language/name/tags
creator/publisher -> <author><name>/<publisher><name>
date -> <updated> (only the first 10 chars are kept
by Book::fromOpdsDate())
articleCount/mediaCount -> <articleCount>/<mediaCount>
favicon/faviconMimeType -> thumbnail <link>'s href/type (OPDS only
carries a URL to the image, never embedded
data, so favicon's placeholder text becomes
a placeholder href instead)
-->
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:opds="https://specs.opds.io/opds-1.2">
<id>7f8a2b1e-0000-4000-8000-000000000000</id>
<title>Test OPDS library</title>
<entry>
<id>urn:uuid:raycharles</id>
<title>Ray Charles</title>
<summary>Wikipedia articles about Ray Charles (not all of them but near to what an average newborn may find more than enough)</summary>
<language>eng</language>
<author><name>Wikipedia</name></author>
<publisher><name>Kiwix</name></publisher>
<updated>2020-03-31T00:00:00Z</updated>
<name>wikipedia_en_ray_charles</name>
<tags>public_tag_without_a_value;_private_tag_without_a_value;wikipedia;_category:wikipedia;_pictures:no;_videos:no;_details:no;_ftindex:yes</tags>
<articleCount>284</articleCount>
<mediaCount>2</mediaCount>
<link rel="http://opds-spec.org/acquisition/open-access" href="./zimfile_raycharles.zim" />
<link rel="http://opds-spec.org/acquisition/open-access"
type="application/x-zim"
href="https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile_raycharles.zim"
length="569344" />
<link rel="http://opds-spec.org/image/thumbnail"
type="image/png;width=48;height=48;scale=1"
href="https://example.com/favicon/raycharles.png" />
</entry>
<entry>
<id>urn:uuid:raycharles_uncategorized</id>
<title>Ray (uncategorized) Charles</title>
<summary>No category is assigned to this library entry (neither adj nor xor was considered a good option)</summary>
<language>rus,eng</language>
<author><name>Wikipedia</name></author>
<publisher><name>Kiwix</name></publisher>
<updated>2020-03-31T00:00:00Z</updated>
<name>wikipedia_ru_ray_charles</name>
<tags>public_tag_with_a_value:value_of_a_public_tag;_private_tag_with_a_value:value_of_a_private_tag;wikipedia;_pictures:no;_videos:no;_details:no</tags>
<articleCount>284</articleCount>
<mediaCount>2</mediaCount>
<link rel="http://opds-spec.org/acquisition/open-access" href="./zimfile_raycharles_uncategorized.zim" />
<link rel="http://opds-spec.org/acquisition/open-access"
type="application/x-zim"
href="https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile_raycharles_uncategorized.zim"
length="125952" />
</entry>
<entry>
<id>urn:uuid:charlesray</id>
<title>Charles, Ray</title>
<summary>Wikipedia articles about Ray Charles or why and when one should go to library</summary>
<language>fra</language>
<author><name>Wikipedia</name></author>
<publisher><name>Kiwix</name></publisher>
<updated>2020-03-31T00:00:00Z</updated>
<name>wikipedia_fr_ray_charles</name>
<tags>unittest;wikipedia;_category:jazz;_pictures:no;_videos:no;_details:no;_ftindex:yes</tags>
<articleCount>284</articleCount>
<mediaCount>2</mediaCount>
<link rel="http://opds-spec.org/acquisition/open-access" href="./zimfile&amp;other.zim" />
<link rel="http://opds-spec.org/acquisition/open-access"
type="application/x-zim"
href="https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile%26other.zim"
length="569344" />
<link rel="http://opds-spec.org/image/thumbnail"
href="https://example.com/favicon/charlesray" />
</entry>
<entry>
<id>urn:uuid:inaccessiblezim</id>
<title>Catalog of all catalogs</title>
<summary>Testing that running kiwix-serve without access to ZIM files doesn't lead to a catastrophe</summary>
<language>cat</language>
<author><name>Catherine of Catalonia</name></author>
<publisher><name>Caterpillar</name></publisher>
<updated>2025-09-04T00:00:00Z</updated>
<name>catalog_of_all_catalogs</name>
<tags>unittest;_category:cats</tags>
<articleCount>12107</articleCount>
<mediaCount>8</mediaCount>
<link rel="http://opds-spec.org/acquisition/open-access" href="./nosuchzimfile.zim" />
<link rel="http://opds-spec.org/acquisition/open-access"
type="application/x-zim"
href="https://github.com/kiwix/libkiwix/raw/master/test/data/nosuchzimfile.zim"
length="20736925696" />
<link rel="http://opds-spec.org/image/thumbnail"
href="https://example.com/favicon/inaccessiblezim" />
</entry>
</feed>
+125 -45
View File
@@ -268,13 +268,15 @@ const char * sampleOpdsStream = R"(
)";
#ifdef _WIN32
# define ZIMFILE_PATH ".\\zimfile.zim"
# define EXAMPLE_PATH ".\\example.zim"
# define LIBRARY_PATH ".\\test\\library.xml"
# define ZIMFILE_PATH "zimfile.zim"
# define EXAMPLE_PATH "example.zim"
# define XML_LIBRARY_PATH "test\\library.xml"
# define OPDS_LIBRARY_PATH "test\\library.opds"
#else
# define ZIMFILE_PATH "./zimfile.zim"
# define EXAMPLE_PATH "./example.zim"
# define LIBRARY_PATH "./test/library.xml"
# define ZIMFILE_PATH "zimfile.zim"
# define EXAMPLE_PATH "example.zim"
# define XML_LIBRARY_PATH "test/library.xml"
# define OPDS_LIBRARY_PATH "test/library.opds"
#endif
const char sampleLibraryXML[] = R"(
@@ -313,10 +315,65 @@ const char sampleLibraryXML[] = R"(
</library>
)";
// The formatting of this OPDS XML (element order, indentation, and
// whitespace) is intentionally chosen to exactly match Library::dumpOpds()'s
// output, so that it stays invariant when read via Manager::readOpds() and
// dumped back via Library::dumpOpds() (see LibraryOpdsExportTest.allInOne
// below). Reformatting this string may break that round-trip test.
const char sampleLibraryOpds[] = R"(<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:dc="http://purl.org/dc/terms/"
xmlns:opds="http://opds-spec.org/2010/catalog">
<entry>
<id>urn:uuid:example</id>
<title>An example ZIM archive</title>
<updated>2021-04-11T00:00:00Z</updated>
<summary>An eXaMpLe book added to the catalog via XML</summary>
<language>deu</language>
<name>wikibooks.de</name>
<flavour>maxi</flavour>
<category>wikibooks</category>
<tags>unittest;wikibooks;_category:wikibooks</tags>
<articleCount>12</articleCount>
<mediaCount>0</mediaCount>
<author>
<name>Wikibooks</name>
</author>
<publisher>
<name>Kiwix &amp; Some Enthusiasts</name>
</publisher>
<dc:issued>2021-04-11T00:00:00Z</dc:issued>
<link rel="http://opds-spec.org/acquisition/open-access" href=")" EXAMPLE_PATH R"(" type="application/x-zim" length="129024"/>
</entry>
<entry>
<id>urn:uuid:raycharles</id>
<title>Ray Charles</title>
<updated>2020-03-31T00:00:00Z</updated>
<summary>Wikipedia articles about Ray Charles</summary>
<language>eng,spa</language>
<name>wikipedia_en_ray_charles</name>
<flavour>mini</flavour>
<category>wikipedia</category>
<tags>wikipedia;_category:wikipedia;_pictures:no</tags>
<articleCount>284</articleCount>
<mediaCount>2</mediaCount>
<author>
<name>Wikipedia</name>
</author>
<publisher>
<name>Kiwix</name>
</publisher>
<dc:issued>2020-03-31T00:00:00Z</dc:issued>
<link rel="http://opds-spec.org/acquisition/open-access" type="application/x-zim" href="https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile.zim" length="569344" />
<link rel="http://opds-spec.org/acquisition/open-access" href=")" ZIMFILE_PATH R"(" type="application/x-zim" length="569344"/>
</entry>
</feed>
)";
#include "../include/library.h"
#include "../include/manager.h"
#include "../include/book.h"
#include "../include/bookmark.h"
#include "../include/tools.h"
namespace
{
@@ -327,7 +384,7 @@ TEST(LibraryOpdsImportTest, allInOne)
{
auto lib = kiwix::Library::create();
kiwix::Manager manager(lib);
manager.readOpds(sampleOpdsStream, "library-opds-import.unittests.dev");
manager.readOpds(sampleOpdsStream, "http://library-opds-import.unittests.dev");
EXPECT_EQ(14U, lib->getBookCount(true, true));
@@ -354,7 +411,7 @@ TEST(LibraryOpdsImportTest, allInOne)
EXPECT_EQ(illustration->width, 48U);
EXPECT_EQ(illustration->height, 48U);
EXPECT_EQ(illustration->mimeType, "image/png");
EXPECT_EQ(illustration->url, "library-opds-import.unittests.dev/meta?name=favicon&content=wikipedia_fr_tunisie_novid_2018-10");
EXPECT_EQ(illustration->url, "http://library-opds-import.unittests.dev/meta?name=favicon&content=wikipedia_fr_tunisie_novid_2018-10");
}
{
@@ -379,21 +436,29 @@ TEST(LibraryOpdsImportTest, allInOne)
EXPECT_EQ(illustration->width, 48U);
EXPECT_EQ(illustration->height, 48U);
EXPECT_EQ(illustration->mimeType, "image/png");
EXPECT_EQ(illustration->url, "library-opds-import.unittests.dev/meta?name=favicon&content=ted_en_business_2018-07");
EXPECT_EQ(illustration->url, "http://library-opds-import.unittests.dev/meta?name=favicon&content=ted_en_business_2018-07");
}
}
class LibraryTest : public ::testing::Test {
class LibraryTest : public ::testing::TestWithParam<std::string> {
protected:
typedef kiwix::Library::BookIdCollection BookIdCollection;
typedef std::vector<std::string> TitleCollection;
LibraryTest(): lib(kiwix::Library::create()) {}
void loadSampleLibrary(kiwix::Manager& manager) {
if (GetParam() == "xml") {
manager.readXml(sampleLibraryXML, false, XML_LIBRARY_PATH, true);
} else {
manager.readOpds(sampleLibraryOpds, OPDS_LIBRARY_PATH, false);
}
}
void SetUp() override {
kiwix::Manager manager(lib);
manager.readOpds(sampleOpdsStream, "foo.urlHost");
manager.readXml(sampleLibraryXML, false, LIBRARY_PATH, true);
manager.readOpds(sampleOpdsStream, "http://foo.urlHost");
loadSampleLibrary(manager);
}
kiwix::Bookmark createBookmark(const std::string &id, const std::string& url="", const std::string& title="") {
@@ -421,7 +486,10 @@ class LibraryTest : public ::testing::Test {
std::shared_ptr<kiwix::Library> lib;
};
TEST_F(LibraryTest, createBookMark)
INSTANTIATE_TEST_CASE_P(XmlAndOpds, LibraryTest,
::testing::Values("xml", "opds"));
TEST_P(LibraryTest, createBookMark)
{
auto bookId = "0c45160e-f917-760a-9159-dfe3c53cdcdd";
auto book = lib->getBookById(bookId);
@@ -439,7 +507,7 @@ TEST_F(LibraryTest, createBookMark)
EXPECT_EQ(bookmark.getLanguage(), book.getCommaSeparatedLanguages());
}
TEST_F(LibraryTest, getBookMarksTest)
TEST_P(LibraryTest, getBookMarksTest)
{
auto bookId1 = "0c45160e-f917-760a-9159-dfe3c53cdcdd";
auto bookId2 = "0189d9be-2fd0-b4b6-7300-20fab0b5cdc8";
@@ -461,7 +529,7 @@ TEST_F(LibraryTest, getBookMarksTest)
EXPECT_EQ(allBookmarks[2].getBookId(), bookId2);
}
TEST_F(LibraryTest, bookmarksSerializationTest)
TEST_P(LibraryTest, bookmarksSerializationTest)
{
auto bookId1 = lib->getBooksIds()[0];
auto bookId2 = lib->getBooksIds()[1];
@@ -480,8 +548,8 @@ TEST_F(LibraryTest, bookmarksSerializationTest)
auto new_lib = kiwix::Library::create();
{
kiwix::Manager manager(new_lib);
manager.readOpds(sampleOpdsStream, "foo.urlHost");
manager.readXml(sampleLibraryXML, false, "./test/library.xml", true);
manager.readOpds(sampleOpdsStream, "http://foo.urlHost");
loadSampleLibrary(manager);
manager.readBookmarkFile("__test__bookmarks.xml");
}
std::remove("__test__bookmarks.xml");
@@ -525,7 +593,7 @@ TEST_F(LibraryTest, bookmarksSerializationTest)
EXPECT_EQ(bookmark3.getDate(), book2.getDate());
}
TEST_F(LibraryTest, MigrateBookmark)
TEST_P(LibraryTest, MigrateBookmark)
{
std::string bookId1 = "0c45160e-f917-760a-9159-dfe3c53cdcdd";
std::string bookId2 = "0189d9be-2fd0-b4b6-7300-20fab0b5cdc8";
@@ -644,7 +712,7 @@ TEST_F(LibraryTest, MigrateBookmark)
EXPECT_EQ(allBookmarks[5].getBookId(), bookId1+"_updated1yearlater_flavour");
}
TEST_F(LibraryTest, GetBestTargetBookIdOlder)
TEST_P(LibraryTest, GetBestTargetBookIdOlder)
{
auto bookId = std::string("0c45160e-f917-760a-9159-dfe3c53cdcdd");
@@ -657,7 +725,7 @@ TEST_F(LibraryTest, GetBestTargetBookIdOlder)
ASSERT_EQ(lib->getBestTargetBookId(validBookmark, kiwix::ALLOW_DOWNGRADE), bookId+"_updated1yearlater");
}
TEST_F(LibraryTest, GetBestTargetBookIdNewer)
TEST_P(LibraryTest, GetBestTargetBookIdNewer)
{
auto bookId = std::string("0c45160e-f917-760a-9159-dfe3c53cdcdd_updated1yearlater");
@@ -678,7 +746,7 @@ TEST_F(LibraryTest, GetBestTargetBookIdNewer)
ASSERT_EQ(lib->getBestTargetBookId(validBookmark, kiwix::ALLOW_DOWNGRADE), bookId);
}
TEST_F(LibraryTest, GetBestTargetBookIdInvalidOlder)
TEST_P(LibraryTest, GetBestTargetBookIdInvalidOlder)
{
auto bookId = std::string("0c45160e-f917-760a-9159-dfe3c53cdcdd");
@@ -692,7 +760,7 @@ TEST_F(LibraryTest, GetBestTargetBookIdInvalidOlder)
ASSERT_EQ(lib->getBestTargetBookId(invalidBookmark, kiwix::ALLOW_DOWNGRADE), bookId+"_updated1yearlater");
}
TEST_F(LibraryTest, GetBestTargetBookIdInvalidNewer)
TEST_P(LibraryTest, GetBestTargetBookIdInvalidNewer)
{
auto bookId = std::string("0c45160e-f917-760a-9159-dfe3c53cdcdd");
@@ -708,7 +776,7 @@ TEST_F(LibraryTest, GetBestTargetBookIdInvalidNewer)
ASSERT_EQ(lib->getBestTargetBookId(invalidBookmark, kiwix::ALLOW_DOWNGRADE), bookId+"_updated1yearlater");
}
TEST_F(LibraryTest, GetBestTargetBookIdFlavour)
TEST_P(LibraryTest, GetBestTargetBookIdFlavour)
{
auto bookId = std::string("0c45160e-f917-760a-9159-dfe3c53cdcdd_flavour");
@@ -724,7 +792,7 @@ TEST_F(LibraryTest, GetBestTargetBookIdFlavour)
ASSERT_EQ(lib->getBestTargetBookId(invalidBookmark, kiwix::ALLOW_DOWNGRADE), "0c45160e-f917-760a-9159-dfe3c53cdcdd_updated1yearlater_flavour");
}
TEST_F(LibraryTest, GetBestTargetBookIdName)
TEST_P(LibraryTest, GetBestTargetBookIdName)
{
ASSERT_EQ(lib->getBestTargetBookId("wikipedia_fr_tunisie"), "0c45160e-f917-760a-9159-dfe3c53cdcdd_updated1yearlater");
ASSERT_EQ(lib->getBestTargetBookId("wikipedia_fr_tunisie", "novid"), "0c45160e-f917-760a-9159-dfe3c53cdcdd_updated1yearlater");
@@ -732,7 +800,7 @@ TEST_F(LibraryTest, GetBestTargetBookIdName)
ASSERT_EQ(lib->getBestTargetBookId("wikipedia_fr_tunisie", "other_flavour", "2020-12-12"), "");
}
TEST_F(LibraryTest, sanityCheck)
TEST_P(LibraryTest, sanityCheck)
{
EXPECT_EQ(lib->getBookCount(true, true), 16U);
EXPECT_EQ(lib->getBooksLanguages(),
@@ -757,7 +825,7 @@ TEST_F(LibraryTest, sanityCheck)
}));
}
TEST_F(LibraryTest, categoryHandling)
TEST_P(LibraryTest, categoryHandling)
{
EXPECT_EQ("", lib->getBookById("0c45160e-f917-760a-9159-dfe3c53cdcdd").getCategory());
EXPECT_EQ("category_defined_via_tags_only", lib->getBookById("0d0bcd57-d3f6-cb22-44cc-a723ccb4e1b2").getCategory());
@@ -766,7 +834,7 @@ TEST_F(LibraryTest, categoryHandling)
EXPECT_EQ("category_element_overrides_tags", lib->getBookById("14829621-c490-c376-0792-9de558b57efa").getCategory());
}
TEST_F(LibraryTest, emptyFilter)
TEST_P(LibraryTest, emptyFilter)
{
const auto bookIds = lib->filter(kiwix::Filter());
EXPECT_EQ(bookIds, lib->getBooksIds());
@@ -778,7 +846,7 @@ TEST_F(LibraryTest, emptyFilter)
TitleCollection({ __VA_ARGS__ }) \
)
TEST_F(LibraryTest, filterLocal)
TEST_P(LibraryTest, filterLocal)
{
EXPECT_FILTER_RESULTS(kiwix::Filter().local(true),
"An example ZIM archive",
@@ -803,7 +871,7 @@ TEST_F(LibraryTest, filterLocal)
);
}
TEST_F(LibraryTest, filterRemote)
TEST_P(LibraryTest, filterRemote)
{
EXPECT_FILTER_RESULTS(kiwix::Filter().remote(true),
"Business talks about TED",
@@ -828,7 +896,7 @@ TEST_F(LibraryTest, filterRemote)
);
}
TEST_F(LibraryTest, filterByLanguage)
TEST_P(LibraryTest, filterByLanguage)
{
EXPECT_FILTER_RESULTS(kiwix::Filter().lang("eng"),
"Business talks about TED",
@@ -855,7 +923,7 @@ TEST_F(LibraryTest, filterByLanguage)
);
}
TEST_F(LibraryTest, filterByFlavour)
TEST_P(LibraryTest, filterByFlavour)
{
EXPECT_FILTER_RESULTS(kiwix::Filter().flavour("full"),
"Géographie par Wikipédia",
@@ -874,7 +942,7 @@ TEST_F(LibraryTest, filterByFlavour)
);
}
TEST_F(LibraryTest, filterByTags)
TEST_P(LibraryTest, filterByTags)
{
EXPECT_FILTER_RESULTS(kiwix::Filter().acceptTags({"stackexchange"}),
"Islam Stack Exchange",
@@ -922,7 +990,7 @@ TEST_F(LibraryTest, filterByTags)
}
TEST_F(LibraryTest, filterByQuery)
TEST_P(LibraryTest, filterByQuery)
{
// filtering by query checks the title
EXPECT_FILTER_RESULTS(kiwix::Filter().query("Exchange"),
@@ -990,7 +1058,7 @@ TEST_F(LibraryTest, filterByQuery)
}
TEST_F(LibraryTest, filteringByEmptyQueryReturnsAllEntries)
TEST_P(LibraryTest, filteringByEmptyQueryReturnsAllEntries)
{
EXPECT_FILTER_RESULTS(kiwix::Filter().query(""),
"An example ZIM archive",
@@ -1012,7 +1080,7 @@ TEST_F(LibraryTest, filteringByEmptyQueryReturnsAllEntries)
);
}
TEST_F(LibraryTest, filterByCreator)
TEST_P(LibraryTest, filterByCreator)
{
EXPECT_FILTER_RESULTS(kiwix::Filter().creator("Wikipedia"),
"Encyclopédie de la Tunisie",
@@ -1070,7 +1138,7 @@ TEST_F(LibraryTest, filterByCreator)
}
TEST_F(LibraryTest, filterByPublisher)
TEST_P(LibraryTest, filterByPublisher)
{
EXPECT_FILTER_RESULTS(kiwix::Filter().publisher("Kiwix"),
"An example ZIM archive",
@@ -1106,7 +1174,7 @@ TEST_F(LibraryTest, filterByPublisher)
);
}
TEST_F(LibraryTest, filterByName)
TEST_P(LibraryTest, filterByName)
{
EXPECT_FILTER_RESULTS(kiwix::Filter().name("wikibooks.de"),
"An example ZIM archive"
@@ -1142,7 +1210,7 @@ TEST_F(LibraryTest, filterByName)
);
}
TEST_F(LibraryTest, filterByCategory)
TEST_P(LibraryTest, filterByCategory)
{
EXPECT_FILTER_RESULTS(kiwix::Filter().category("category_element_overrides_tags"),
"Géographie par Wikipédia",
@@ -1159,14 +1227,14 @@ TEST_F(LibraryTest, filterByCategory)
);
}
TEST_F(LibraryTest, filterByMaxSize)
TEST_P(LibraryTest, filterByMaxSize)
{
EXPECT_FILTER_RESULTS(kiwix::Filter().maxSize(200000),
"An example ZIM archive"
);
}
TEST_F(LibraryTest, filterByMultipleCriteria)
TEST_P(LibraryTest, filterByMultipleCriteria)
{
EXPECT_FILTER_RESULTS(kiwix::Filter().query("Wiki").creator("Wikipedia"),
"Encyclopédie de la Tunisie",
@@ -1194,7 +1262,7 @@ TEST_F(LibraryTest, filterByMultipleCriteria)
);
}
TEST_F(LibraryTest, getBookByPath)
TEST_P(LibraryTest, getBookByPath)
{
kiwix::Book book = lib->getBookById(lib->getBooksIds()[0]);
#ifdef _WIN32
@@ -1208,7 +1276,7 @@ TEST_F(LibraryTest, getBookByPath)
EXPECT_THROW(lib->getBookByPath("non/existant/path.zim"), std::out_of_range);
}
TEST_F(LibraryTest, removeBookByIdRemovesTheBook)
TEST_P(LibraryTest, removeBookByIdRemovesTheBook)
{
const auto initialBookCount = lib->getBookCount(true, true);
ASSERT_GT(initialBookCount, 0U);
@@ -1218,14 +1286,14 @@ TEST_F(LibraryTest, removeBookByIdRemovesTheBook)
EXPECT_THROW(lib->getBookById("raycharles"), std::out_of_range);
};
TEST_F(LibraryTest, removeBookByIdDropsTheReader)
TEST_P(LibraryTest, removeBookByIdDropsTheReader)
{
EXPECT_NE(nullptr, lib->getArchiveById("raycharles"));
lib->removeBookById("raycharles");
EXPECT_THROW(lib->getArchiveById("raycharles"), std::out_of_range);
};
TEST_F(LibraryTest, removeBookByIdUpdatesTheSearchDB)
TEST_P(LibraryTest, removeBookByIdUpdatesTheSearchDB)
{
kiwix::Filter f;
f.local(true).valid(true).query(R"(title:"ray charles")", false);
@@ -1243,7 +1311,7 @@ TEST_F(LibraryTest, removeBookByIdUpdatesTheSearchDB)
EXPECT_THROW(lib->getBookById("raycharles"), std::out_of_range);
};
TEST_F(LibraryTest, removeBooksNotUpdatedSince)
TEST_P(LibraryTest, removeBooksNotUpdatedSince)
{
EXPECT_FILTER_RESULTS(kiwix::Filter(),
"An example ZIM archive",
@@ -1284,4 +1352,16 @@ TEST_F(LibraryTest, removeBooksNotUpdatedSince)
);
};
TEST(LibraryOpdsExportTest, allInOne)
{
auto lib = kiwix::Library::create();
kiwix::Manager manager(lib);
manager.readOpds(sampleLibraryOpds, OPDS_LIBRARY_PATH);
// sampleLibraryOpds is formatted to be round-trip invariant (see comment
// above its definition), so dumping what was just read back out should
// reproduce it exactly.
EXPECT_EQ(lib->dumpOpds(OPDS_LIBRARY_PATH), sampleLibraryOpds);
}
};
+58 -37
View File
@@ -6,11 +6,13 @@
#define SERVER_PORT 8001
#include "server_testing_tools.h"
#include "../src/tools/stringTools.h"
////////////////////////////////////////////////////////////////////////////////
// Testing of the library-related functionality of the server
////////////////////////////////////////////////////////////////////////////////
class LibraryServerTest : public ::testing::Test
class LibraryServerTest : public ::testing::TestWithParam<std::string>
{
protected:
std::unique_ptr<ZimFileServer> zfs1_;
@@ -20,7 +22,7 @@ protected:
protected:
void resetServer(ZimFileServer::Cfg cfg) {
zfs1_.reset();
zfs1_.reset(new ZimFileServer(PORT, cfg, "./test/library.xml"));
zfs1_.reset(new ZimFileServer(PORT, cfg, GetParam()));
}
void resetServer(ZimFileServer::Options options, std::string contentServerUrl="") {
@@ -30,7 +32,7 @@ protected:
}
void SetUp() override {
zfs1_.reset(new ZimFileServer(PORT, ZimFileServer::DEFAULT_OPTIONS, "./test/library.xml"));
zfs1_.reset(new ZimFileServer(PORT, ZimFileServer::DEFAULT_OPTIONS, GetParam()));
}
void TearDown() override {
@@ -38,6 +40,9 @@ protected:
}
};
INSTANTIATE_TEST_CASE_P(XmlAndOpds, LibraryServerTest,
::testing::Values("./test/library.xml", "./test/library.opds"));
// Returns a copy of 'text' where every line that fully matches 'pattern'
// preceded by optional whitespace is replaced with the fixed string
// 'replacement' preserving the leading whitespace
@@ -94,7 +99,7 @@ std::string maskVariableOPDSFeedData(std::string s)
" <tags>" TAGS "</tags>\n" \
" <articleCount>284</articleCount>\n" \
" <mediaCount>2</mediaCount>\n" \
" " EXTRA_LINK "<link type=\"text/html\" href=\"/ROOT%23%3F/content/" CONTENT_NAME "\" />\n" \
EXTRA_LINK " <link type=\"text/html\" href=\"/ROOT%23%3F/content/" CONTENT_NAME "\" />\n" \
" <author>\n" \
" <name>Wikipedia</name>\n" \
" </author>\n" \
@@ -131,9 +136,9 @@ std::string maskVariableOPDSFeedData(std::string s)
"wikipedia_en_ray_charles",\
"wikipedia",\
"public_tag_without_a_value;_private_tag_without_a_value;wikipedia;_category:wikipedia;_pictures:no;_videos:no;_details:no;_ftindex:yes",\
"<link rel=\"http://opds-spec.org/image/thumbnail\"\n" \
" <link rel=\"http://opds-spec.org/image/thumbnail\"\n" \
" href=\"/ROOT%23%3F/catalog/v2/illustration/raycharles/?size=48\"\n" \
" type=\"image/png;width=48;height=48;scale=1\"/>\n ", \
" type=\"image/png;width=48;height=48;scale=1\"/>\n", \
CONTENT_NAME, \
"zimfile_raycharles", \
"569344"\
@@ -180,7 +185,7 @@ std::string maskVariableOPDSFeedData(std::string s)
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"https://github.com/kiwix/libkiwix/raw/master/test/data/nosuchzimfile.zim\" length=\"20736925696\" />\n" \
" </entry>\n"
TEST_F(LibraryServerTest, catalog_root_xml)
TEST_P(LibraryServerTest, catalog_root_xml)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/root.xml");
EXPECT_EQ(r->status, 200);
@@ -198,7 +203,23 @@ TEST_F(LibraryServerTest, catalog_root_xml)
);
}
TEST_F(LibraryServerTest, catalog_searchdescription_xml)
TEST_P(LibraryServerTest, ThumbnailMimeTypeHandling)
{
resetServer(ZimFileServer::CATALOG_ONLY_MODE);
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/entries");
EXPECT_EQ(r->status, 200);
EXPECT_TRUE(kiwix::contains(r->body,
"<link rel=\"http://opds-spec.org/image/thumbnail\"\n"
" href=\"/ROOT%23%3F/catalog/v2/illustration/raycharles/?size=48\"\n"
" type=\"image/png;width=48;height=48;scale=1\"/>"
));
EXPECT_FALSE(kiwix::contains(r->body, "illustration/charlesray/"));
EXPECT_FALSE(kiwix::contains(r->body, "illustration/inaccessiblezim/"));
}
TEST_P(LibraryServerTest, catalog_searchdescription_xml)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/searchdescription.xml");
EXPECT_EQ(r->status, 200);
@@ -216,7 +237,7 @@ TEST_F(LibraryServerTest, catalog_searchdescription_xml)
);
}
TEST_F(LibraryServerTest, catalog_search_by_phrase)
TEST_P(LibraryServerTest, catalog_search_by_phrase)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/search?q=\"ray%20charles\"");
EXPECT_EQ(r->status, 200);
@@ -235,7 +256,7 @@ TEST_F(LibraryServerTest, catalog_search_by_phrase)
);
}
TEST_F(LibraryServerTest, catalog_search_by_words)
TEST_P(LibraryServerTest, catalog_search_by_words)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/search?q=ray%20charles");
EXPECT_EQ(r->status, 200);
@@ -255,7 +276,7 @@ TEST_F(LibraryServerTest, catalog_search_by_words)
);
}
TEST_F(LibraryServerTest, catalog_prefix_search)
TEST_P(LibraryServerTest, catalog_prefix_search)
{
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/search?q=description:ray%20description:charles");
@@ -292,7 +313,7 @@ TEST_F(LibraryServerTest, catalog_prefix_search)
}
}
TEST_F(LibraryServerTest, catalog_search_with_word_exclusion)
TEST_P(LibraryServerTest, catalog_search_with_word_exclusion)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/search?q=ray%20-uncategorized");
EXPECT_EQ(r->status, 200);
@@ -311,7 +332,7 @@ TEST_F(LibraryServerTest, catalog_search_with_word_exclusion)
);
}
TEST_F(LibraryServerTest, catalog_search_by_tag)
TEST_P(LibraryServerTest, catalog_search_by_tag)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/search?tag=_category:jazz");
EXPECT_EQ(r->status, 200);
@@ -329,7 +350,7 @@ TEST_F(LibraryServerTest, catalog_search_by_tag)
);
}
TEST_F(LibraryServerTest, catalog_search_by_category)
TEST_P(LibraryServerTest, catalog_search_by_category)
{
{
@@ -368,7 +389,7 @@ TEST_F(LibraryServerTest, catalog_search_by_category)
}
}
TEST_F(LibraryServerTest, catalog_search_by_language)
TEST_P(LibraryServerTest, catalog_search_by_language)
{
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/search?lang=eng");
@@ -408,7 +429,7 @@ TEST_F(LibraryServerTest, catalog_search_by_language)
}
}
TEST_F(LibraryServerTest, catalog_search_results_pagination)
TEST_P(LibraryServerTest, catalog_search_results_pagination)
{
{
// count=-1 disables the limit on the number of results
@@ -494,7 +515,7 @@ TEST_F(LibraryServerTest, catalog_search_results_pagination)
}
}
TEST_F(LibraryServerTest, catalog_v2_root)
TEST_P(LibraryServerTest, catalog_v2_root)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/root.xml");
EXPECT_EQ(r->status, 200);
@@ -555,7 +576,7 @@ TEST_F(LibraryServerTest, catalog_v2_root)
EXPECT_EQ(maskVariableOPDSFeedData(r->body), expected_output);
}
TEST_F(LibraryServerTest, catalog_v2_searchdescription_xml)
TEST_P(LibraryServerTest, catalog_v2_searchdescription_xml)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/searchdescription.xml");
EXPECT_EQ(r->status, 200);
@@ -573,7 +594,7 @@ TEST_F(LibraryServerTest, catalog_v2_searchdescription_xml)
);
}
TEST_F(LibraryServerTest, catalog_v2_categories)
TEST_P(LibraryServerTest, catalog_v2_categories)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/categories");
EXPECT_EQ(r->status, 200);
@@ -622,7 +643,7 @@ TEST_F(LibraryServerTest, catalog_v2_categories)
EXPECT_EQ(maskVariableOPDSFeedData(r->body), expected_output);
}
TEST_F(LibraryServerTest, catalog_v2_languages)
TEST_P(LibraryServerTest, catalog_v2_languages)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/languages");
EXPECT_EQ(r->status, 200);
@@ -711,7 +732,7 @@ TEST_F(LibraryServerTest, catalog_v2_languages)
#define CATALOG_V2_PARTIAL_ENTRIES_PREAMBLE(q) \
CATALOG_V2_ENTRIES_PREAMBLE0("partial_entries" q)
TEST_F(LibraryServerTest, catalog_v2_entries)
TEST_P(LibraryServerTest, catalog_v2_entries)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/entries");
EXPECT_EQ(r->status, 200);
@@ -727,7 +748,7 @@ TEST_F(LibraryServerTest, catalog_v2_entries)
);
}
TEST_F(LibraryServerTest, catalog_v2_entries_catalog_only_mode)
TEST_P(LibraryServerTest, catalog_v2_entries_catalog_only_mode)
{
const std::string contentServerUrl = "https://demo.kiwix.org";
const auto fixContentLinks = [=](std::string s) -> std::string {
@@ -770,7 +791,7 @@ TEST_F(LibraryServerTest, catalog_v2_entries_catalog_only_mode)
}
}
TEST_F(LibraryServerTest, catalog_v2_entries_filtered_by_range)
TEST_P(LibraryServerTest, catalog_v2_entries_filtered_by_range)
{
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/entries?start=1");
@@ -853,7 +874,7 @@ TEST_F(LibraryServerTest, catalog_v2_entries_filtered_by_range)
}
}
TEST_F(LibraryServerTest, catalog_v2_entries_filtered_by_search_terms)
TEST_P(LibraryServerTest, catalog_v2_entries_filtered_by_search_terms)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/entries?q=\"ray%20charles\"");
EXPECT_EQ(r->status, 200);
@@ -870,7 +891,7 @@ TEST_F(LibraryServerTest, catalog_v2_entries_filtered_by_search_terms)
);
}
TEST_F(LibraryServerTest, catalog_v2_entries_filtering_special_queries)
TEST_P(LibraryServerTest, catalog_v2_entries_filtering_special_queries)
{
{
// 'or' doesn't act as a Xapian boolean operator
@@ -1031,7 +1052,7 @@ TEST_F(LibraryServerTest, catalog_v2_entries_filtering_special_queries)
}
}
TEST_F(LibraryServerTest, catalog_v2_entries_filtered_by_language)
TEST_P(LibraryServerTest, catalog_v2_entries_filtered_by_language)
{
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/entries?lang=eng");
@@ -1067,7 +1088,7 @@ TEST_F(LibraryServerTest, catalog_v2_entries_filtered_by_language)
}
}
TEST_F(LibraryServerTest, catalog_v2_entries_filtered_by_category)
TEST_P(LibraryServerTest, catalog_v2_entries_filtered_by_category)
{
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/entries?category=jazz");
@@ -1101,7 +1122,7 @@ TEST_F(LibraryServerTest, catalog_v2_entries_filtered_by_category)
}
}
TEST_F(LibraryServerTest, catalog_v2_entries_multiple_filters)
TEST_P(LibraryServerTest, catalog_v2_entries_multiple_filters)
{
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/entries?lang=fra&category=jazz");
@@ -1119,7 +1140,7 @@ TEST_F(LibraryServerTest, catalog_v2_entries_multiple_filters)
}
}
TEST_F(LibraryServerTest, catalog_v2_individual_entry_access)
TEST_P(LibraryServerTest, catalog_v2_individual_entry_access)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/entry/raycharles");
EXPECT_EQ(r->status, 200);
@@ -1132,7 +1153,7 @@ TEST_F(LibraryServerTest, catalog_v2_individual_entry_access)
EXPECT_EQ(r1->status, 404);
}
TEST_F(LibraryServerTest, catalog_v2_partial_entries)
TEST_P(LibraryServerTest, catalog_v2_partial_entries)
{
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/partial_entries");
EXPECT_EQ(r->status, 200);
@@ -1189,7 +1210,7 @@ TEST_F(LibraryServerTest, catalog_v2_partial_entries)
); \
}
TEST_F(LibraryServerTest, catalog_search_includes_public_tags)
TEST_P(LibraryServerTest, catalog_search_includes_public_tags)
{
EXPECT_SEARCH_RESULTS("public_tag_without_a_value",
1,
@@ -1222,13 +1243,13 @@ TEST_F(LibraryServerTest, catalog_search_includes_public_tags)
#define EXPECT_ZERO_RESULTS(SEARCH_TERM) EXPECT_SEARCH_RESULTS(SEARCH_TERM, 0, )
TEST_F(LibraryServerTest, catalog_search_on_tags_is_not_an_any_substring_match)
TEST_P(LibraryServerTest, catalog_search_on_tags_is_not_an_any_substring_match)
{
EXPECT_ZERO_RESULTS("tag_with")
EXPECT_ZERO_RESULTS("alue_of_a_public_tag")
}
TEST_F(LibraryServerTest, catalog_search_excludes_hidden_tags)
TEST_P(LibraryServerTest, catalog_search_excludes_hidden_tags)
{
EXPECT_ZERO_RESULTS("_private_tag_without_a_value");
EXPECT_ZERO_RESULTS("private_tag_without_a_value");
@@ -1237,7 +1258,7 @@ TEST_F(LibraryServerTest, catalog_search_excludes_hidden_tags)
#undef EXPECT_ZERO_RESULTS
}
TEST_F(LibraryServerTest, no_name_mapper_returned_catalog_use_uuid_in_link)
TEST_P(LibraryServerTest, no_name_mapper_returned_catalog_use_uuid_in_link)
{
resetServer(ZimFileServer::NO_NAME_MAPPER);
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/search?tag=_category:jazz");
@@ -1257,7 +1278,7 @@ TEST_F(LibraryServerTest, no_name_mapper_returned_catalog_use_uuid_in_link)
}
TEST_F(LibraryServerTest, no_name_mapper_catalog_v2_individual_entry_access)
TEST_P(LibraryServerTest, no_name_mapper_catalog_v2_individual_entry_access)
{
resetServer(ZimFileServer::NO_NAME_MAPPER);
const auto r = zfs1_->GET("/ROOT%23%3F/catalog/v2/entry/raycharles");
@@ -1533,7 +1554,7 @@ TEST_F(LibraryServerTest, no_name_mapper_catalog_v2_individual_entry_access)
"</body>\n" \
"</html>"
TEST_F(LibraryServerTest, noJS) {
TEST_P(LibraryServerTest, noJS) {
// no_js_default
auto r = zfs1_->GET("/ROOT%23%3F/nojs");
EXPECT_EQ(r->status, 200);
@@ -1572,7 +1593,7 @@ TEST_F(LibraryServerTest, noJS) {
EXPECT_EQ(r->body, RAY_CHARLES_UNCTZ_DOWNLOAD);
}
TEST_F(LibraryServerTest, noJS_catalogOnlyMode) {
TEST_P(LibraryServerTest, noJS_catalogOnlyMode) {
const std::string contentServerUrl = "https://demo.kiwix.org";
const auto fixContentLinks = [=](std::string s) -> std::string {
s = replace(s, "/ROOT%23%3F/content", contentServerUrl + "/content");
+289
View File
@@ -0,0 +1,289 @@
/*
* Copyright 2026 Hamazasp Avetisyan <hamik.avetisyan@gmail.com>
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
* published by the Free Software Foundation; either version 2 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* is provided AS IS, WITHOUT ANY WARRANTY; without even the implied
* warranty of MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, and
* NON-INFRINGEMENT. See the GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*
*/
#include "../src/libxml_dumper.h"
#include "../include/book.h"
#include "../include/bookmark.h"
#include "../include/library.h"
#include "../include/tools.h"
#include "gtest/gtest.h"
namespace
{
using namespace kiwix;
Book createBook(const std::string& id, const std::string& title)
{
Book book;
book.setId(id);
// A relative path is resolved by Book::setPath() against the current
// directory; the test dumper's baseDir is set to that same directory
// below, so the dumped "path" attribute is portably just "<id>.zim" on
// every platform (a hardcoded absolute path like "/local/path/..." is
// POSIX-only - Windows treats a leading '/' without a drive letter as
// relative, which silently prefixes it with the build's cwd instead).
book.setPath(id + ".zim");
book.setTitle(title);
book.setDescription("Description of " + title);
book.setLanguage("eng,spa");
book.setCreator(title + " Creator");
book.setPublisher(title + " Publisher");
book.setDate("2021-03-25");
book.setUrl("http://download.kiwix.org/zim/" + id + ".zim");
book.setName("test_" + id);
book.setFlavour("nopic");
book.setTags("tag1;tag2");
book.setArticleCount(42);
book.setMediaCount(7);
book.setSize(122880); // 120 KiB - stored on disk as KiB (>>10)
book.setDownloadId("download-" + id);
return book;
}
class LibXMLDumperTest : public ::testing::Test
{
protected:
void SetUp() override
{
lib = Library::create();
lib->addBook(createBook("book1-id", "First Book"));
lib->addBook(createBook("book2-id", "Second Book"));
dumper.setLibrary(lib.get());
dumper.setBaseDir(getCurrentDirectory());
}
LibraryPtr lib;
LibXMLDumper dumper;
};
TEST_F(LibXMLDumperTest, emptyBookIdListProducesEmptyLibraryElement)
{
EXPECT_EQ(dumper.dumpLibXMLContent({}),
"<library version=\"20110515\" />\n"
);
}
TEST_F(LibXMLDumperTest, dumpsRequestedBookWithAllAttributes)
{
EXPECT_EQ(dumper.dumpLibXMLContent({"book1-id"}),
"<library version=\"20110515\">\n"
// The book's path is stored relative to the dumper's base dir, and
// size on disk in KiB (bytes >> 10).
" <book id=\"book1-id\" path=\"book1-id.zim\" title=\"First Book\""
" description=\"Description of First Book\" language=\"eng,spa\""
" creator=\"First Book Creator\" publisher=\"First Book Publisher\""
" name=\"test_book1-id\" flavour=\"nopic\" tags=\"tag1;tag2\""
" date=\"2021-03-25\" url=\"http://download.kiwix.org/zim/book1-id.zim\""
" articleCount=\"42\" mediaCount=\"7\" size=\"120\""
" downloadId=\"download-book1-id\" />\n"
"</library>\n"
);
}
TEST_F(LibXMLDumperTest, dumpsOnlyRequestedBooksInOrder)
{
EXPECT_EQ(dumper.dumpLibXMLContent({"book2-id", "book1-id"}),
"<library version=\"20110515\">\n"
" <book id=\"book2-id\" path=\"book2-id.zim\" title=\"Second Book\""
" description=\"Description of Second Book\" language=\"eng,spa\""
" creator=\"Second Book Creator\" publisher=\"Second Book Publisher\""
" name=\"test_book2-id\" flavour=\"nopic\" tags=\"tag1;tag2\""
" date=\"2021-03-25\" url=\"http://download.kiwix.org/zim/book2-id.zim\""
" articleCount=\"42\" mediaCount=\"7\" size=\"120\""
" downloadId=\"download-book2-id\" />\n"
" <book id=\"book1-id\" path=\"book1-id.zim\" title=\"First Book\""
" description=\"Description of First Book\" language=\"eng,spa\""
" creator=\"First Book Creator\" publisher=\"First Book Publisher\""
" name=\"test_book1-id\" flavour=\"nopic\" tags=\"tag1;tag2\""
" date=\"2021-03-25\" url=\"http://download.kiwix.org/zim/book1-id.zim\""
" articleCount=\"42\" mediaCount=\"7\" size=\"120\""
" downloadId=\"download-book1-id\" />\n"
"</library>\n"
);
}
TEST_F(LibXMLDumperTest, omitsEmptyOrZeroOptionalAttributes)
{
Book book = createBook("bare-book-id", "Bare Book");
book.setUrl("");
book.setDownloadId("");
book.setArticleCount(0);
book.setMediaCount(0);
book.setSize(0);
lib->addBook(book);
EXPECT_EQ(dumper.dumpLibXMLContent({"bare-book-id"}),
"<library version=\"20110515\">\n"
" <book id=\"bare-book-id\" path=\"bare-book-id.zim\" title=\"Bare Book\""
" description=\"Description of Bare Book\" language=\"eng,spa\""
" creator=\"Bare Book Creator\" publisher=\"Bare Book Publisher\""
" name=\"test_bare-book-id\" flavour=\"nopic\" tags=\"tag1;tag2\""
" date=\"2021-03-25\" />\n"
"</library>\n"
);
}
TEST_F(LibXMLDumperTest, skipsReadOnlyBooks)
{
Book readOnlyBook = createBook("readonly-book-id", "Read Only Book");
readOnlyBook.setReadOnly(true);
lib->addBook(readOnlyBook);
// The read-only book is entirely omitted from the dump, not just stripped
// of its attributes.
EXPECT_EQ(dumper.dumpLibXMLContent({"book1-id", "readonly-book-id"}),
"<library version=\"20110515\">\n"
" <book id=\"book1-id\" path=\"book1-id.zim\" title=\"First Book\""
" description=\"Description of First Book\" language=\"eng,spa\""
" creator=\"First Book Creator\" publisher=\"First Book Publisher\""
" name=\"test_book1-id\" flavour=\"nopic\" tags=\"tag1;tag2\""
" date=\"2021-03-25\" url=\"http://download.kiwix.org/zim/book1-id.zim\""
" articleCount=\"42\" mediaCount=\"7\" size=\"120\""
" downloadId=\"download-book1-id\" />\n"
"</library>\n"
);
}
TEST_F(LibXMLDumperTest, aliasBookWithOrigIdOnlyEmitsIdAndOrigId)
{
Book aliasBook = createBook("alias-book-id", "Alias Book");
aliasBook.setOrigId("book1-id");
lib->addBook(aliasBook);
// Metadata that's only meaningful for the "real" entry is skipped for a
// flavour alias - only path (above the origId/title branch) and
// date/url/counts/size/downloadId (below it) are still emitted.
EXPECT_EQ(dumper.dumpLibXMLContent({"alias-book-id"}),
"<library version=\"20110515\">\n"
" <book id=\"alias-book-id\" path=\"alias-book-id.zim\" origId=\"book1-id\""
" date=\"2021-03-25\" url=\"http://download.kiwix.org/zim/alias-book-id.zim\""
" articleCount=\"42\" mediaCount=\"7\" size=\"120\""
" downloadId=\"download-alias-book-id\" />\n"
"</library>\n"
);
}
TEST_F(LibXMLDumperTest, dumpLibXMLContentThrowsForUnknownBookId)
{
EXPECT_THROW(dumper.dumpLibXMLContent({"no-such-book"}), std::out_of_range);
}
TEST_F(LibXMLDumperTest, defaultConstructedDumperHasNoLibrary)
{
LibXMLDumper freshDumper;
EXPECT_EQ(freshDumper.dumpLibXMLContent({"book1-id"}),
"<library version=\"20110515\" />\n"
);
EXPECT_EQ(freshDumper.dumpLibXMLBookmark(),
"<bookmarks />\n"
);
}
TEST_F(LibXMLDumperTest, setLibraryReplacesTheDumpedLibrary)
{
auto otherLib = Library::create();
otherLib->addBook(createBook("other-book-id", "Other Book"));
dumper.setLibrary(otherLib.get());
EXPECT_EQ(dumper.dumpLibXMLContent({"other-book-id"}),
"<library version=\"20110515\">\n"
" <book id=\"other-book-id\" path=\"other-book-id.zim\" title=\"Other Book\""
" description=\"Description of Other Book\" language=\"eng,spa\""
" creator=\"Other Book Creator\" publisher=\"Other Book Publisher\""
" name=\"test_other-book-id\" flavour=\"nopic\" tags=\"tag1;tag2\""
" date=\"2021-03-25\" url=\"http://download.kiwix.org/zim/other-book-id.zim\""
" articleCount=\"42\" mediaCount=\"7\" size=\"120\""
" downloadId=\"download-other-book-id\" />\n"
"</library>\n"
);
// "book1-id" belonged to the previously set library and is no longer known.
EXPECT_THROW(dumper.dumpLibXMLContent({"book1-id"}), std::out_of_range);
}
TEST_F(LibXMLDumperTest, emptyBookmarksProducesEmptyBookmarksElement)
{
EXPECT_EQ(dumper.dumpLibXMLBookmark(), "<bookmarks />\n");
}
TEST_F(LibXMLDumperTest, dumpsBookmarkUsingLiveBookDataWhenBookIsInLibrary)
{
Bookmark bookmark;
bookmark.setBookId("book1-id");
bookmark.setUrl("/A/Some_Article");
bookmark.setTitle("Some Article Title");
lib->addBookmark(bookmark);
// These values come from the live library entry, not from the bookmark's
// own (unset) book* fields.
EXPECT_EQ(dumper.dumpLibXMLBookmark(),
"<bookmarks>\n"
" <bookmark>\n"
" <book>\n"
" <id>book1-id</id>\n"
" <title>First Book</title>\n"
" <name>test_book1-id</name>\n"
" <flavour>nopic</flavour>\n"
" <language>eng,spa</language>\n"
" <date>2021-03-25</date>\n"
" </book>\n"
" <title>Some Article Title</title>\n"
" <url>/A/Some_Article</url>\n"
" </bookmark>\n"
"</bookmarks>\n"
);
}
TEST_F(LibXMLDumperTest, dumpsBookmarkUsingStoredFieldsWhenBookIsUnknown)
{
Bookmark bookmark;
bookmark.setBookId("no-such-book");
bookmark.setBookTitle("Vanished Book");
bookmark.setBookName("vanished_book");
bookmark.setBookFlavour("full");
bookmark.setLanguage("fra");
bookmark.setDate("2019-01-01");
bookmark.setUrl("/A/Some_Article");
bookmark.setTitle("Some Article Title");
lib->addBookmark(bookmark);
EXPECT_EQ(dumper.dumpLibXMLBookmark(),
"<bookmarks>\n"
" <bookmark>\n"
" <book>\n"
" <id>no-such-book</id>\n"
" <title>Vanished Book</title>\n"
" <name>vanished_book</name>\n"
" <flavour>full</flavour>\n"
" <language>fra</language>\n"
" <date>2019-01-01</date>\n"
" </book>\n"
" <title>Some Article Title</title>\n"
" <url>/A/Some_Article</url>\n"
" </bookmark>\n"
"</bookmarks>\n"
);
}
} // unnamed namespace
+322 -4
View File
@@ -6,6 +6,29 @@
#include <iostream>
#include <fstream>
namespace
{
std::string resolveAbsPath(const std::string& basePath, const std::string& relPath)
{
return kiwix::computeAbsolutePath(kiwix::removeLastPathElement(basePath), relPath);
}
// Absolute path of test/library.opds, computed (rather than hardcoded) so
// it resolves correctly regardless of the checkout location - unlike
// LIB_ABS_PATH below, this one has to point to a real file, since
// (unlike the readXml() tests) readFile()/readOpds() actually open it.
// Built from single, separator-free path segments (rather than a
// "test/library.opds"-style literal) because computeAbsolutePath() splits
// on the OS-native separator only, so a literal using the "wrong" slash
// would silently fail to be split into components on Windows.
const std::string LIB_OPDS_ABS_PATH
= kiwix::computeAbsolutePath(
kiwix::computeAbsolutePath(kiwix::getCurrentDirectory(), "test"),
"library.opds");
} // unnamed namespace
TEST(ManagerTest, addBookFromPathAndGetIdTest)
{
auto lib = kiwix::Library::create();
@@ -20,11 +43,32 @@ TEST(ManagerTest, addBookFromPathAndGetIdTest)
const std::string url = "url";
bookId = manager.addBookFromPathAndGetId("./test/example.zim", pathToSave, url, true);
book = lib->getBookById(bookId);
auto savedPath = kiwix::computeAbsolutePath(kiwix::removeLastPathElement(manager.writableLibraryPath), pathToSave);
auto savedPath = resolveAbsPath(manager.writableLibraryPath, pathToSave);
EXPECT_EQ(book.getPath(), savedPath);
EXPECT_EQ(book.getUrl(), url);
}
TEST(ManagerTest, readFileSetsWritableLibraryPathEvenIfFileDoesNotExist)
{
auto lib = kiwix::Library::create();
kiwix::Manager manager(lib);
const std::string nonExistentPath
= kiwix::computeAbsolutePath(
kiwix::computeAbsolutePath(kiwix::getCurrentDirectory(), "test"),
"does_not_exist.xml");
EXPECT_FALSE(manager.readFile(nonExistentPath, /*readOnly=*/false));
EXPECT_EQ(manager.writableLibraryPath, nonExistentPath);
const std::string pathToSave = "./relative.zim";
auto bookId = manager.addBookFromPathAndGetId("./test/example.zim", pathToSave);
ASSERT_NE(bookId, "");
kiwix::Book book = lib->getBookById(bookId);
auto savedPath = resolveAbsPath(nonExistentPath, pathToSave);
EXPECT_EQ(book.getPath(), savedPath);
}
#if _WIN32
@@ -65,7 +109,14 @@ TEST(ManagerTest, readXml)
EXPECT_EQ(true, manager.readXml(sampleLibraryXML, true, LIB_ABS_PATH, true));
kiwix::Book book = lib->getBookById("0d0bcd57-d3f6-cb22-44cc-a723ccb4e1b2");
// "path" is relative in the XML - readXml() resolves it against the
// directory of LIB_ABS_PATH, yielding ZIM_ABS_PATH.
EXPECT_EQ(ZIM_ABS_PATH, book.getPath());
// ... but ZIM_ABS_PATH doesn't exist on disk, so the resolved path is
// not considered valid.
EXPECT_FALSE(book.isPathValid());
EXPECT_TRUE(book.readOnly());
EXPECT_EQ("https://example.com/zimfiles/unittest.zim", book.getUrl());
EXPECT_EQ("Unit Test", book.getTitle());
EXPECT_EQ("Wikipedia articles about unit testing", book.getDescription());
@@ -80,12 +131,88 @@ TEST(ManagerTest, readXml)
EXPECT_EQ(678U*1024, book.getSize());
}
TEST(Manager, reload)
TEST(ManagerTest, readXmlInvalid)
{
auto lib = kiwix::Library::create();
kiwix::Manager manager = kiwix::Manager(lib);
const std::string invalidXML = R"(
<library version="1.0">
<book
id="0d0bcd57-d3f6-cb22-44cc-a723ccb4e1b2"
path=")" UNITTEST_ZIM_PATH R"("
url="https://example.com/zimfiles/unittest.zim"
title="Unit Test"
description="Wikipedia articles about unit testing"
language="eng"
creator="Wikipedia"
publisher="Kiwix"
date="2020-03-31"
name="wikipedia_en_unit_testing"
tags="unittest;wikipedia"
articleCount="123"
mediaCount="45"
size="678"
></book>
<book
id="1a1bcd57-d3f6-cb22-44cc-a723ccb4e1b3"
url="https://example.com/zimfiles/unittest2.zim"
title="Unit Test 2"
)";
EXPECT_FALSE(manager.readXml(invalidXML, true, LIB_ABS_PATH, true));
EXPECT_TRUE(lib->getBooksIds().empty());
}
TEST(ManagerTest, readXmlNotXml)
{
auto lib = kiwix::Library::create();
kiwix::Manager manager = kiwix::Manager(lib);
const std::string notXML = "this is definitely not xml content";
EXPECT_FALSE(manager.readXml(notXML, true, LIB_ABS_PATH, true));
EXPECT_TRUE(lib->getBooksIds().empty());
}
TEST(ManagerTest, readOpdsWithNoEntriesReturnsTrue)
{
auto lib = kiwix::Library::create();
kiwix::Manager manager(lib);
manager.reload({ "./test/library.xml" });
EXPECT_TRUE(manager.readOpds(R"(<feed xmlns="http://www.w3.org/2005/Atom"></feed>)", "http://example.com"));
EXPECT_TRUE(lib->getBooksIds().empty());
}
TEST(ManagerTest, readOpdsWithMalformedInputAddsNoBooks)
{
auto lib = kiwix::Library::create();
kiwix::Manager manager(lib);
const std::string feed = R"(
<feed xmlns="http://www.w3.org/2005/Atom">
<entry>
<id>urn:uuid:book1</id>
<title>Book One</title>
</entry>
<entry>
<id>urn:uuid:book2</id>
<title>Book Two</title>
</feed>
)";
EXPECT_FALSE(manager.readOpds(feed, "http://example.com"));
EXPECT_TRUE(lib->getBooksIds().empty());
}
class ManagerReloadTest : public ::testing::TestWithParam<std::string> {};
TEST_P(ManagerReloadTest, reload)
{
auto lib = kiwix::Library::create();
kiwix::Manager manager(lib);
manager.reload({GetParam()});
EXPECT_EQ(lib->getBooksIds(), (kiwix::Library::BookIdCollection{
"charlesray",
"inaccessiblezim",
@@ -100,7 +227,7 @@ TEST(Manager, reload)
"raycharles_uncategorized"
}));
manager.reload({ "./test/library.xml" });
manager.reload({GetParam()});
EXPECT_EQ(lib->getBooksIds(), kiwix::Library::BookIdCollection({
"charlesray",
"inaccessiblezim",
@@ -108,3 +235,194 @@ TEST(Manager, reload)
"raycharles_uncategorized"
}));
}
INSTANTIATE_TEST_CASE_P(XmlAndOpds, ManagerReloadTest,
::testing::Values("./test/library.xml", "./test/library.opds"));
const char sampleOpdsFeed[] = R"(
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:opds="https://specs.opds.io/opds-1.2">
<totalResults>9</totalResults>
<startIndex>7</startIndex>
<itemsPerPage>10</itemsPerPage>
<entry>
<id>urn:uuid:book1</id>
<title>Book One</title>
<link rel="http://opds-spec.org/acquisition/open-access"
type="application/x-zim"
href="https://example.com/book1.zim"
length="111" />
</entry>
<entry>
<id>urn:uuid:book2</id>
<title>Book Two</title>
<link rel="http://opds-spec.org/acquisition/open-access"
type="application/x-zim"
href="https://example.com/book2.zim"
length="222" />
</entry>
</feed>
)";
TEST(ManagerTest, readOpdsHonorsReadOnlyTrue)
{
// readOpds() defaults to readOnly=false (see
// ManagerTest.readOpdsAddsEntriesAndParsesSearchMetadata below, which
// covers that case) - this checks that readOnly=true is honored too.
auto lib = kiwix::Library::create();
kiwix::Manager manager(lib);
EXPECT_TRUE(manager.readOpds(sampleOpdsFeed, "http://example.com", /*readOnly=*/true));
EXPECT_TRUE(lib->getBookById("book1").readOnly());
EXPECT_TRUE(lib->getBookById("book2").readOnly());
}
TEST(ManagerTest, readOpdsAddsEntriesAndParsesSearchMetadata)
{
auto lib = kiwix::Library::create();
kiwix::Manager manager(lib);
EXPECT_TRUE(manager.readOpds(sampleOpdsFeed, "http://example.com"));
EXPECT_TRUE(manager.m_hasSearchResult);
EXPECT_EQ(manager.m_totalBooks, 9U);
EXPECT_EQ(manager.m_startIndex, 7U);
EXPECT_EQ(manager.m_itemsPerPage, 10U);
EXPECT_EQ(lib->getBooksIds(), (kiwix::Library::BookIdCollection{"book1", "book2"}));
kiwix::Book book1 = lib->getBookById("book1");
EXPECT_EQ(book1.getTitle(), "Book One");
EXPECT_EQ(book1.getUrl(), "https://example.com/book1.zim");
EXPECT_EQ(book1.getPath(), "");
EXPECT_FALSE(book1.isPathValid());
EXPECT_FALSE(book1.readOnly());
}
TEST(ManagerTest, readOpdsWithInvalidLocalPath)
{
auto lib = kiwix::Library::create();
kiwix::Manager manager(lib);
const std::string feed = R"(
<feed xmlns="http://www.w3.org/2005/Atom">
<entry>
<id>urn:uuid:book1</id>
<title>Book From OPDS</title>
<link rel="http://opds-spec.org/acquisition/open-access" href="does-not-exist.zim" />
</entry>
</feed>
)";
EXPECT_TRUE(manager.readOpds(feed, "./test/library.opds", /*readOnly=*/false));
kiwix::Book book = lib->getBookById("book1");
EXPECT_FALSE(book.isPathValid());
EXPECT_EQ(book.getTitle(), "Book From OPDS");
}
TEST(ManagerTest, readOpdsWithoutSearchMetadata)
{
auto lib = kiwix::Library::create();
kiwix::Manager manager(lib);
const std::string feed = R"(
<feed xmlns="http://www.w3.org/2005/Atom">
<entry>
<id>urn:uuid:book1</id>
<title>Book One</title>
</entry>
</feed>
)";
EXPECT_TRUE(manager.readOpds(feed, "http://example.com"));
// None of <totalResults>/<startIndex>/<itemsPerPage> are present, so
// there's no search result to report.
EXPECT_FALSE(manager.m_hasSearchResult);
EXPECT_EQ(manager.m_totalBooks, 0U);
EXPECT_EQ(manager.m_startIndex, 0U);
EXPECT_EQ(manager.m_itemsPerPage, 0U);
EXPECT_EQ(lib->getBooksIds(), (kiwix::Library::BookIdCollection{"book1"}));
}
TEST(ManagerTest, readFileDetectsXmlFormat)
{
// readFile() sniffs the file content for a "<feed" substring to tell
// OPDS files apart from XML library files (see detectFormat() in
// manager.cpp).
auto lib = kiwix::Library::create();
kiwix::Manager manager(lib);
EXPECT_TRUE(manager.readFile("./test/library.xml"));
EXPECT_EQ(lib->getBooksIds(), (kiwix::Library::BookIdCollection{
"charlesray",
"inaccessiblezim",
"raycharles",
"raycharles_uncategorized"
}));
kiwix::Book book = lib->getBookById("raycharles");
EXPECT_EQ(book.getPath(), resolveAbsPath("./test/library.xml", "./zimfile_raycharles.zim"));
// readOnly defaults to true.
EXPECT_TRUE(book.readOnly());
}
TEST(ManagerTest, readFileDetectsOpdsFormat)
{
auto lib = kiwix::Library::create();
kiwix::Manager manager(lib);
EXPECT_TRUE(manager.readFile(LIB_OPDS_ABS_PATH, /*readOnly=*/false));
EXPECT_EQ(lib->getBooksIds(), (kiwix::Library::BookIdCollection{
"charlesray",
"inaccessiblezim",
"raycharles",
"raycharles_uncategorized"
}));
kiwix::Book book = lib->getBookById("raycharles");
EXPECT_EQ(book.getPath(), resolveAbsPath(LIB_OPDS_ABS_PATH, "./zimfile_raycharles.zim"));
EXPECT_TRUE(book.isPathValid());
EXPECT_EQ(book.getUrl(), "https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile_raycharles.zim");
EXPECT_EQ(book.getTitle(), "Ray Charles");
EXPECT_EQ(book.getDescription(), "Wikipedia articles about Ray Charles (not all of them but near to what an average newborn may find more than enough)");
EXPECT_EQ(book.getCommaSeparatedLanguages(), "eng");
EXPECT_EQ(book.getCreator(), "Wikipedia");
EXPECT_EQ(book.getPublisher(), "Kiwix");
EXPECT_EQ(book.getDate(), "2020-03-31");
EXPECT_EQ(book.getName(), "wikipedia_en_ray_charles");
EXPECT_EQ(book.getTags(), "public_tag_without_a_value;_private_tag_without_a_value;wikipedia;_category:wikipedia;_pictures:no;_videos:no;_details:no;_ftindex:yes");
EXPECT_EQ(book.getArticleCount(), 284U);
EXPECT_EQ(book.getMediaCount(), 2U);
EXPECT_EQ(book.getSize(), 556U*1024U);
auto illustration = book.getIllustration(48);
EXPECT_EQ(illustration->mimeType, "image/png");
EXPECT_EQ(illustration->width, 48);
EXPECT_EQ(illustration->height, 48);
EXPECT_EQ(illustration->url, "https://example.com/favicon/raycharles.png");
EXPECT_FALSE(book.readOnly());
}
TEST(ManagerTest, readFileWithOpdsFormatHonorsReadOnly)
{
auto lib = kiwix::Library::create();
kiwix::Manager manager(lib);
EXPECT_TRUE(manager.readFile("./test/library.opds", /*readOnly=*/true, /*trustLibrary=*/true));
for (const auto& id : lib->getBooksIds()) {
EXPECT_TRUE(lib->getBookById(id).readOnly());
}
}
+5
View File
@@ -12,6 +12,9 @@ tests = [
'manager',
'name_mapper',
'opds_catalog',
'opds_entry_rendering',
'libxml_dumper',
'opds_dumper',
'server_helper',
'lrucache',
'i18n',
@@ -45,7 +48,9 @@ if gtest_dep.found() and not meson.is_cross_build()
'poor.zim',
'spelling_correction_test.zim',
'library.xml',
'library.opds',
'lib_for_server_search_test.xml',
'lib_for_server_search_test.opds',
'customized_resources.txt',
'helloworld.txt',
'welcome.html',
+410
View File
@@ -0,0 +1,410 @@
/*
* Copyright 2026 Hamazasp Avetisyan <hamik.avetisyan@gmail.com>
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
* published by the Free Software Foundation; either version 2 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* is provided AS IS, WITHOUT ANY WARRANTY; without even the implied
* warranty of MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, and
* NON-INFRINGEMENT. See the GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*
*/
#include "gtest/gtest.h"
#include <regex>
#include <sstream>
#include "../include/library.h"
#include "../include/manager.h"
#include "../include/name_mapper.h"
#include "../include/book.h"
#include "../include/tools.h"
#include "../src/opds_dumper.h"
#include "../src/tools/otherTools.h"
namespace
{
using namespace kiwix;
// Two books, deliberately exercising the two different ways a category can
// be attributed to a book (an explicit "category" attribute vs. a
// "_category:" tag), and two different languages.
const char sampleLibraryXML[] = R"(
<library version="1.0">
<book
id="book1-id"
path="/local/path/book1-id.zim"
title="First Book"
description="Description of First Book"
language="eng"
creator="First Book Creator"
publisher="First Book Publisher"
date="2021-03-25"
url="http://download.kiwix.org/zim/book1-id.zim"
name="test_book1-id"
tags="tag1;tag2;_category:wikipedia"
articleCount="42"
mediaCount="7"
size="120"
></book>
<book
id="book2-id"
path="/local/path/book2-id.zim"
title="Second Book"
description="Description of Second Book"
language="fra"
creator="Second Book Creator"
publisher="Second Book Publisher"
date="2021-03-26"
url="http://download.kiwix.org/zim/book2-id.zim"
name="test_book2-id"
category="wiktionary"
tags="tag3"
articleCount="10"
mediaCount="1"
size="45"
></book>
</library>
)";
// Returns a copy of 'text' where every line that fully matches 'pattern'
// (preceded by optional whitespace) is replaced with 'replacement',
// preserving the leading whitespace. Copied from test/library_server.cpp,
// which uses the same technique for the same reason (masking timestamps
// generated from the current wall-clock time).
std::string replaceLines(const std::string& text,
const std::string& pattern,
const std::string& replacement)
{
std::regex regex("^ *" + pattern + "$");
std::ostringstream oss;
std::istringstream iss(text);
std::string line;
while ( std::getline(iss, line) ) {
if ( std::regex_match(line, regex) ) {
for ( size_t i = 0; i < line.size() && line[i] == ' '; ++i )
oss << ' ';
oss << replacement << "\n";
} else {
oss << line << "\n";
}
}
return oss.str();
}
// Masks every <updated>...</updated> timestamp - both the feed-level one
// (always "now", via gen_date_str()) and the per-entry/per-category/
// per-language ones used by categoriesOPDSFeed()/languagesOPDSFeed() (also
// "now" - see LibraryDumper::getCategoryData()/getLanguageData()). Per-book
// entries elsewhere carry a real, deterministic book date, but this matches
// them too, same as library_server.cpp's CATALOG_ENTRY does for the live
// HTTP catalog - the trade-off of a slightly less precise assertion in
// exchange for a much simpler, more robust one.
std::string maskUpdatedTimestamps(std::string s)
{
return replaceLines(s, R"(<updated>\d\d\d\d-\d\d-\d\dT\d\d:\d\d:\d\dZ</updated>)",
"<updated>YYYY-MM-DDThh:mm:ssZ</updated>");
}
// Compares 'actual' (an OPDS feed/entry string, possibly containing
// wall-clock-generated <updated> timestamps) against 'expected' after
// masking those timestamps out of 'actual', so tests don't have to spell
// out maskUpdatedTimestamps(...) at every assertion.
#define EXPECT_OPDS(actual, expected) \
EXPECT_EQ(maskUpdatedTimestamps(actual), (expected))
#define BOOK1_ENTRY \
" <entry>\n" \
" <id>urn:uuid:book1-id</id>\n" \
" <title>First Book</title>\n" \
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n" \
" <summary>Description of First Book</summary>\n" \
" <language>eng</language>\n" \
" <name>test_book1-id</name>\n" \
" <flavour></flavour>\n" \
" <category>wikipedia</category>\n" \
" <tags>tag1;tag2;_category:wikipedia</tags>\n" \
" <articleCount>42</articleCount>\n" \
" <mediaCount>7</mediaCount>\n" \
" <link type=\"text/html\" href=\"http://root.location/content/book1-id\" />\n" \
" <author>\n" \
" <name>First Book Creator</name>\n" \
" </author>\n" \
" <publisher>\n" \
" <name>First Book Publisher</name>\n" \
" </publisher>\n" \
" <dc:issued>2021-03-25T00:00:00Z</dc:issued>\n" \
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"http://download.kiwix.org/zim/book1-id.zim\" length=\"122880\" />\n" \
" </entry>\n"
#define BOOK2_ENTRY \
" <entry>\n" \
" <id>urn:uuid:book2-id</id>\n" \
" <title>Second Book</title>\n" \
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n" \
" <summary>Description of Second Book</summary>\n" \
" <language>fra</language>\n" \
" <name>test_book2-id</name>\n" \
" <flavour></flavour>\n" \
" <category>wiktionary</category>\n" \
" <tags>tag3</tags>\n" \
" <articleCount>10</articleCount>\n" \
" <mediaCount>1</mediaCount>\n" \
" <link type=\"text/html\" href=\"http://root.location/content/book2-id\" />\n" \
" <author>\n" \
" <name>Second Book Creator</name>\n" \
" </author>\n" \
" <publisher>\n" \
" <name>Second Book Publisher</name>\n" \
" </publisher>\n" \
" <dc:issued>2021-03-26T00:00:00Z</dc:issued>\n" \
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"http://download.kiwix.org/zim/book2-id.zim\" length=\"46080\" />\n" \
" </entry>\n"
#define BOOK1_PARTIAL_ENTRY \
" <entry>\n" \
" <id>urn:uuid:book1-id</id>\n" \
" <title>First Book</title>\n" \
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n" \
" <link rel=\"alternate\"\n" \
" href=\"http://root.location/catalog/v2/entry/book1-id\"\n" \
" type=\"application/atom+xml;type=entry;profile=opds-catalog\"/>\n" \
" </entry>\n"
#define BOOK2_PARTIAL_ENTRY \
" <entry>\n" \
" <id>urn:uuid:book2-id</id>\n" \
" <title>Second Book</title>\n" \
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n" \
" <link rel=\"alternate\"\n" \
" href=\"http://root.location/catalog/v2/entry/book2-id\"\n" \
" type=\"application/atom+xml;type=entry;profile=opds-catalog\"/>\n" \
" </entry>\n"
class OPDSDumperTest : public ::testing::Test {
protected:
void SetUp() override {
lib = Library::create();
Manager manager(lib);
manager.readXml(sampleLibraryXML, /*readOnly=*/false, "", /*trustLibrary=*/true);
dumper.reset(new OPDSDumper(lib.get(), &nameMapper));
dumper->setRootLocation("http://root.location");
dumper->setLibraryId("test-library-id");
dumper->setContentAccessUrl("http://root.location/content");
dumper->setOpenSearchInfo(/*totalResults=*/2, /*startIndex=*/0, /*count=*/1);
}
LibraryPtr lib;
IdNameMapper nameMapper;
std::unique_ptr<OPDSDumper> dumper;
};
TEST_F(OPDSDumperTest, dumpOPDSFeedWithoutFilteringInfo)
{
EXPECT_OPDS(dumper->dumpOPDSFeed({"book1-id", "book2-id"}, ""),
"<feed xmlns=\"http://www.w3.org/2005/Atom\"\n"
" xmlns:dc=\"http://purl.org/dc/terms/\"\n"
" xmlns:opds=\"http://opds-spec.org/2010/catalog\">\n"
" <id>" + gen_uuid("test-library-id/catalog/search?") + "</id>\n"
" <title>All zims</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
"\n"
" <link rel=\"self\" href=\"\" type=\"application/atom+xml\" />\n"
" <link rel=\"search\" type=\"application/opensearchdescription+xml\" href=\"http://root.location/catalog/searchdescription.xml\" />\n"
BOOK1_ENTRY
BOOK2_ENTRY
"</feed>\n"
);
}
TEST_F(OPDSDumperTest, dumpOPDSFeedWithFilteringInfo)
{
EXPECT_OPDS(dumper->dumpOPDSFeed({"book1-id"}, "lang=eng"),
"<feed xmlns=\"http://www.w3.org/2005/Atom\"\n"
" xmlns:dc=\"http://purl.org/dc/terms/\"\n"
" xmlns:opds=\"http://opds-spec.org/2010/catalog\">\n"
" <id>" + gen_uuid("test-library-id/catalog/search?lang=eng") + "</id>\n"
" <title>Filtered zims (lang=eng)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>2</totalResults>\n"
" <startIndex>0</startIndex>\n"
" <itemsPerPage>1</itemsPerPage>\n"
" <link rel=\"self\" href=\"\" type=\"application/atom+xml\" />\n"
" <link rel=\"search\" type=\"application/opensearchdescription+xml\" href=\"http://root.location/catalog/searchdescription.xml\" />\n"
BOOK1_ENTRY
"</feed>\n"
);
}
TEST_F(OPDSDumperTest, dumpOPDSFeedV2FullEntries)
{
EXPECT_OPDS(dumper->dumpOPDSFeedV2({"book1-id", "book2-id"}, "", /*partial=*/false),
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
"<feed xmlns=\"http://www.w3.org/2005/Atom\"\n"
" xmlns:dc=\"http://purl.org/dc/terms/\"\n"
" xmlns:opds=\"https://specs.opds.io/opds-1.2\"\n"
" xmlns:opensearch=\"http://a9.com/-/spec/opensearch/1.1/\">\n"
" <id>" + gen_uuid("test-library-id/entries?") + "</id>\n"
"\n"
" <link rel=\"self\"\n"
" href=\"http://root.location/catalog/v2/entries\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=acquisition\"/>\n"
" <link rel=\"start\"\n"
" href=\"http://root.location/catalog/v2/root.xml\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=navigation\"/>\n"
" <link rel=\"up\"\n"
" href=\"http://root.location/catalog/v2/root.xml\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=navigation\"/>\n"
"\n"
" <title>All Entries</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
"\n"
BOOK1_ENTRY
BOOK2_ENTRY
"</feed>\n"
);
}
TEST_F(OPDSDumperTest, dumpOPDSFeedV2PartialEntries)
{
EXPECT_OPDS(dumper->dumpOPDSFeedV2({"book1-id", "book2-id"}, "", /*partial=*/true),
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
"<feed xmlns=\"http://www.w3.org/2005/Atom\"\n"
" xmlns:dc=\"http://purl.org/dc/terms/\"\n"
" xmlns:opds=\"https://specs.opds.io/opds-1.2\"\n"
" xmlns:opensearch=\"http://a9.com/-/spec/opensearch/1.1/\">\n"
" <id>" + gen_uuid("test-library-id/partial_entries?") + "</id>\n"
"\n"
" <link rel=\"self\"\n"
" href=\"http://root.location/catalog/v2/partial_entries\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=acquisition\"/>\n"
" <link rel=\"start\"\n"
" href=\"http://root.location/catalog/v2/root.xml\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=navigation\"/>\n"
" <link rel=\"up\"\n"
" href=\"http://root.location/catalog/v2/root.xml\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=navigation\"/>\n"
"\n"
" <title>All Entries</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
"\n"
BOOK1_PARTIAL_ENTRY
BOOK2_PARTIAL_ENTRY
"</feed>\n"
);
}
TEST_F(OPDSDumperTest, dumpOPDSCompleteEntryProducesStandaloneEntryDocument)
{
EXPECT_OPDS(dumper->dumpOPDSCompleteEntry("book2-id"),
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
BOOK2_ENTRY
);
}
TEST_F(OPDSDumperTest, dumpOPDSCompleteEntryThrowsForUnknownBookId)
{
EXPECT_THROW(dumper->dumpOPDSCompleteEntry("no-such-book"), std::out_of_range);
}
TEST_F(OPDSDumperTest, categoriesOPDSFeedListsDistinctSortedCategories)
{
// Categories are listed in sorted order: "wikipedia" (from the
// "_category:" tag on book1) before "wiktionary" (from book2's explicit
// "category" attribute).
EXPECT_OPDS(dumper->categoriesOPDSFeed(),
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
"<feed xmlns=\"http://www.w3.org/2005/Atom\"\n"
" xmlns:opds=\"https://specs.opds.io/opds-1.2\">\n"
" <id>" + gen_uuid("test-library-id/categories") + "</id>\n"
" <link rel=\"self\"\n"
" href=\"http://root.location/catalog/v2/categories\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=navigation\"/>\n"
" <link rel=\"start\"\n"
" href=\"http://root.location/catalog/v2/root.xml\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=navigation\"/>\n"
" <title>List of categories</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
"\n"
" <entry>\n"
" <title>wikipedia</title>\n"
" <link rel=\"subsection\"\n"
" href=\"http://root.location/catalog/v2/entries?category=wikipedia\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=acquisition\"/>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <id>" + gen_uuid("test-library-id/categories/wikipedia") + "</id>\n"
" <content type=\"text\">All entries with category of 'wikipedia'.</content>\n"
" </entry>\n"
" <entry>\n"
" <title>wiktionary</title>\n"
" <link rel=\"subsection\"\n"
" href=\"http://root.location/catalog/v2/entries?category=wiktionary\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=acquisition\"/>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <id>" + gen_uuid("test-library-id/categories/wiktionary") + "</id>\n"
" <content type=\"text\">All entries with category of 'wiktionary'.</content>\n"
" </entry>\n"
"</feed>\n"
);
}
TEST_F(OPDSDumperTest, languagesOPDSFeedListsLanguagesWithCounts)
{
// Languages are listed in sorted order of their ISO code: "eng" before "fra".
EXPECT_OPDS(dumper->languagesOPDSFeed(),
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
"<feed xmlns=\"http://www.w3.org/2005/Atom\"\n"
" xmlns:dc=\"http://purl.org/dc/terms/\"\n"
" xmlns:opds=\"https://specs.opds.io/opds-1.2\"\n"
" xmlns:thr=\"http://purl.org/syndication/thread/1.0\">\n"
" <id>" + gen_uuid("test-library-id/languages") + "</id>\n"
" <link rel=\"self\"\n"
" href=\"http://root.location/catalog/v2/languages\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=navigation\"/>\n"
" <link rel=\"start\"\n"
" href=\"http://root.location/catalog/v2/root.xml\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=navigation\"/>\n"
" <title>List of languages</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
"\n"
" <entry>\n"
" <title>" + getLanguageSelfName("eng") + "</title>\n"
" <dc:language>eng</dc:language>\n"
" <thr:count>1</thr:count>\n"
" <link rel=\"subsection\"\n"
" href=\"http://root.location/catalog/v2/entries?lang=eng\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=acquisition\"/>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <id>" + gen_uuid("test-library-id/languages/eng") + "</id>\n"
" </entry>\n"
" <entry>\n"
" <title>" + getLanguageSelfName("fra") + "</title>\n"
" <dc:language>fra</dc:language>\n"
" <thr:count>1</thr:count>\n"
" <link rel=\"subsection\"\n"
" href=\"http://root.location/catalog/v2/entries?lang=fra\"\n"
" type=\"application/atom+xml;profile=opds-catalog;kind=acquisition\"/>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <id>" + gen_uuid("test-library-id/languages/fra") + "</id>\n"
" </entry>\n"
"</feed>\n"
);
}
#undef EXPECT_OPDS
#undef BOOK1_ENTRY
#undef BOOK2_ENTRY
#undef BOOK2_PARTIAL_ENTRY
#undef BOOK1_PARTIAL_ENTRY
} // unnamed namespace
+636
View File
@@ -0,0 +1,636 @@
/*
* Copyright 2026 Hamazasp Avetisyan <hamik.avetisyan@gmail.com>
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
* published by the Free Software Foundation; either version 2 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* is provided AS IS, WITHOUT ANY WARRANTY; without even the implied
* warranty of MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, and
* NON-INFRINGEMENT. See the GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*
*/
#include "gtest/gtest.h"
#include "../include/book.h"
#include "../include/library.h"
#include "../include/manager.h"
#include "../src/library_dumper.h"
#include "testing_tools.h"
namespace
{
using namespace kiwix;
Book createBook()
{
Book book;
book.setId("book-id");
book.setTitle("Some Title");
book.setDescription("Some Description");
book.setLanguage("eng");
book.setCreator("Some Creator");
book.setPublisher("Some Publisher");
book.setDate("2021-03-25");
book.setName("some_name");
book.setTags("tag1;tag2;_category:wikipedia");
book.setFlavour("nopic");
book.setArticleCount(42);
book.setMediaCount(7);
return book;
}
#define CORE_ENTRY_BODY \
" <id>urn:uuid:book-id</id>\n" \
" <title>Some Title</title>\n" \
" <updated>2021-03-25T00:00:00Z</updated>\n" \
" <summary>Some Description</summary>\n" \
" <language>eng</language>\n" \
" <name>some_name</name>\n" \
" <flavour>nopic</flavour>\n" \
/* Book::getCategory() only reflects a "_category:" tag/attribute */ \
/* resolved during XML/OPDS parsing - it is not derived on the fly */ \
/* from setTags(), so a manually-constructed Book always reports an */ \
/* empty category here. */ \
" <category></category>\n" \
" <tags>tag1;tag2;_category:wikipedia</tags>\n" \
" <articleCount>42</articleCount>\n" \
" <mediaCount>7</mediaCount>\n"
TEST(FullEntryOpdsTest, rendersCoreBookFields)
{
const Book book = createBook();
// No url and no contentAccessUrl were set, so neither the acquisition nor
// the content <link> is rendered (see rendersAcquisitionLinkWhenUrlIsSet /
// rendersUrlEncodedContentLinkWhenContentAccessUrlIsSet below for when
// they are).
EXPECT_EQ(fullEntryOpds(book, "http://root.location", /*contentAccessUrl=*/"", /*contentId=*/"book-id"),
" <entry>\n"
CORE_ENTRY_BODY
" <author>\n"
" <name>Some Creator</name>\n"
" </author>\n"
" <publisher>\n"
" <name>Some Publisher</name>\n"
" </publisher>\n"
" <dc:issued>2021-03-25T00:00:00Z</dc:issued>\n"
" \n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, omitsAcquisitionLinkWhenUrlIsEmpty)
{
const Book book = createBook();
// url left at its default (empty).
EXPECT_EQ(fullEntryOpds(book, "http://root.location", "", "book-id"),
" <entry>\n"
CORE_ENTRY_BODY
" <author>\n"
" <name>Some Creator</name>\n"
" </author>\n"
" <publisher>\n"
" <name>Some Publisher</name>\n"
" </publisher>\n"
" <dc:issued>2021-03-25T00:00:00Z</dc:issued>\n"
" \n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, rendersAcquisitionLinkWhenUrlIsSet)
{
Book book = createBook();
book.setUrl("http://download.kiwix.org/zim/book.zim");
book.setSize(123456);
EXPECT_EQ(fullEntryOpds(book, "http://root.location", "", "book-id"),
" <entry>\n"
CORE_ENTRY_BODY
" <author>\n"
" <name>Some Creator</name>\n"
" </author>\n"
" <publisher>\n"
" <name>Some Publisher</name>\n"
" </publisher>\n"
" <dc:issued>2021-03-25T00:00:00Z</dc:issued>\n"
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"http://download.kiwix.org/zim/book.zim\" length=\"123456\" />\n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, omitsContentLinkWhenContentAccessUrlIsEmpty)
{
const Book book = createBook();
// contentAccessUrl left at its default (empty).
EXPECT_EQ(fullEntryOpds(book, "http://root.location", /*contentAccessUrl=*/"", "book-id"),
" <entry>\n"
CORE_ENTRY_BODY
" <author>\n"
" <name>Some Creator</name>\n"
" </author>\n"
" <publisher>\n"
" <name>Some Publisher</name>\n"
" </publisher>\n"
" <dc:issued>2021-03-25T00:00:00Z</dc:issued>\n"
" \n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, rendersUrlEncodedContentLinkWhenContentAccessUrlIsSet)
{
const Book book = createBook();
// '/' is one of urlEncode()'s "harmless" characters (paths may legitimately
// contain it) and passes through untouched, while ' ' and '#' do not.
EXPECT_EQ(fullEntryOpds(book, "http://root.location", "http://root.location/content", "a/b c#d"),
" <entry>\n"
CORE_ENTRY_BODY
" <link type=\"text/html\" href=\"http://root.location/content/a/b%20c%23d\" />\n"
" <author>\n"
" <name>Some Creator</name>\n"
" </author>\n"
" <publisher>\n"
" <name>Some Publisher</name>\n"
" </publisher>\n"
" <dc:issued>2021-03-25T00:00:00Z</dc:issued>\n"
" \n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, rendersThumbnailLinkForBookIllustration)
{
auto lib = Library::create();
Manager manager(lib);
const char sampleXML[] = R"(
<library version="1.0">
<book
id="book-with-icon"
path="/local/path/book.zim"
title="Book With Icon"
favicon="AAAA"
faviconMimeType="image/png"
faviconUrl="/favicon.png"
></book>
</library>
)";
manager.readXml(sampleXML, /*readOnly=*/false, "", /*trustLibrary=*/true);
const Book& book = lib->getBookById("book-with-icon");
EXPECT_EQ(fullEntryOpds(book, "http://root.location", "", "book-with-icon"),
" <entry>\n"
" <id>urn:uuid:book-with-icon</id>\n"
" <title>Book With Icon</title>\n"
" <updated>T00:00:00Z</updated>\n"
" <summary></summary>\n"
" <language></language>\n"
" <name></name>\n"
" <flavour></flavour>\n"
" <category></category>\n"
" <tags></tags>\n"
" <articleCount>0</articleCount>\n"
" <mediaCount>0</mediaCount>\n"
" <link rel=\"http://opds-spec.org/image/thumbnail\"\n"
" href=\"http://root.location/catalog/v2/illustration/book-with-icon/?size=48\"\n"
" type=\"image/png;width=48;height=48;scale=1\"/>\n"
" <author>\n"
" <name></name>\n"
" </author>\n"
" <publisher>\n"
" <name></name>\n"
" </publisher>\n"
" <dc:issued>T00:00:00Z</dc:issued>\n"
" \n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, rendersBase64ThumbnailForEmbeddedOnlyIllustrationInFileDump)
{
auto lib = Library::create();
Manager manager(lib);
const char sampleXML[] = R"(
<library version="1.0">
<book
id="book-with-embedded-icon"
path="/local/path/book.zim"
title="Book With Embedded Icon"
favicon="AAAA"
faviconMimeType="image/png"
></book>
</library>
)";
manager.readXml(sampleXML, /*readOnly=*/false, "", /*trustLibrary=*/true);
const Book& book = lib->getBookById("book-with-embedded-icon");
// isLiveCatalog=false (offline file dump): an illustration with no
// external url is only available as embedded data, and there is no server
// behind /catalog/v2/illustration there to serve it from, so it is
// embedded directly as a data: URI in the thumbnail link's href instead
// (OPDS 1.2 5.2.2).
EXPECT_EQ(fullEntryOpds(book, "http://root.location", "", "book-with-embedded-icon",
/*localPath=*/"", /*isLiveCatalog=*/false),
" <entry>\n"
" <id>urn:uuid:book-with-embedded-icon</id>\n"
" <title>Book With Embedded Icon</title>\n"
" <updated>T00:00:00Z</updated>\n"
" <summary></summary>\n"
" <language></language>\n"
" <name></name>\n"
" <flavour></flavour>\n"
" <category></category>\n"
" <tags></tags>\n"
" <articleCount>0</articleCount>\n"
" <mediaCount>0</mediaCount>\n"
" <link rel=\"http://opds-spec.org/image/thumbnail\"\n"
" href=\"data:image/png;base64,AAAA\"\n"
" type=\"image/png;width=48;height=48;scale=1\"/>\n"
" <author>\n"
" <name></name>\n"
" </author>\n"
" <publisher>\n"
" <name></name>\n"
" </publisher>\n"
" <dc:issued>T00:00:00Z</dc:issued>\n"
" \n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, rendersIllustrationUrlDirectlyInFileDump)
{
auto lib = Library::create();
Manager manager(lib);
const char sampleXML[] = R"(
<library version="1.0">
<book
id="book-with-icon-url"
path="/local/path/book.zim"
title="Book With Icon Url"
favicon="https://example.com/favicon/zara.png"
faviconMimeType="image/png"
faviconUrl="/favicon.png"
></book>
</library>
)";
manager.readXml(sampleXML, /*readOnly=*/false, "", /*trustLibrary=*/true);
const Book& book = lib->getBookById("book-with-icon-url");
// isLiveCatalog=false (offline file dump): with an external url available,
// the thumbnail link points straight at that url instead of the live
// /catalog/v2/illustration endpoint, which has no server behind it there.
EXPECT_EQ(fullEntryOpds(book, "http://root.location", "", "book-with-icon-url",
/*localPath=*/"", /*isLiveCatalog=*/false),
" <entry>\n"
" <id>urn:uuid:book-with-icon-url</id>\n"
" <title>Book With Icon Url</title>\n"
" <updated>T00:00:00Z</updated>\n"
" <summary></summary>\n"
" <language></language>\n"
" <name></name>\n"
" <flavour></flavour>\n"
" <category></category>\n"
" <tags></tags>\n"
" <articleCount>0</articleCount>\n"
" <mediaCount>0</mediaCount>\n"
" <link rel=\"http://opds-spec.org/image/thumbnail\"\n"
" href=\"/favicon.png\"\n"
" type=\"image/png;width=48;height=48;scale=1\"/>\n"
" <author>\n"
" <name></name>\n"
" </author>\n"
" <publisher>\n"
" <name></name>\n"
" </publisher>\n"
" <dc:issued>T00:00:00Z</dc:issued>\n"
" \n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, doesNotDownloadIllustrationDataForLinkBasedThumbnail)
{
auto lib = Library::create();
Manager manager(lib);
const char sampleOpds[] = R"(
<feed>
<entry>
<id>book-with-remote-icon</id>
<title>Book With Remote Icon</title>
<link rel="http://opds-spec.org/image/thumbnail"
type="image/png"
href="http://127.0.0.1:1/unreachable-favicon.png" />
</entry>
</feed>
)";
ASSERT_TRUE(manager.readOpds(sampleOpds, "http://root.location"));
const Book& book = lib->getBookById("book-with-remote-icon");
ASSERT_EQ(book.getIllustrations().size(), 1U);
kiwix::testing::CapturedStderr stderror;
fullEntryOpds(book, "http://root.location", "", "book-with-remote-icon");
// XXX This is a fragile way of testing the sought behaviour relying on the fact
// that Book::Illustration::getData() prints to stderr if it tries to
// download the thumbnail data. A more robust test would start an HTTP
// server and check that no request has been made to it.
EXPECT_EQ(std::string(stderror), "");
EXPECT_TRUE(book.getIllustrations().at(0)->getData().empty());
}
TEST(FullEntryOpdsTest, roundTripsBase64ThumbnailDataThroughOPDSReadback)
{
auto lib = Library::create();
Manager manager(lib);
const char sampleXML[] = R"(
<library version="1.0">
<book
id="book-with-roundtrip-icon"
path="/local/path/book.zim"
title="Book With Roundtrip Icon"
favicon="SGVsbG8sIFdvcmxkIQ=="
faviconMimeType="image/png"
></book>
</library>
)";
manager.readXml(sampleXML, /*readOnly=*/false, "", /*trustLibrary=*/true);
const Book& book = lib->getBookById("book-with-roundtrip-icon");
ASSERT_EQ(book.getIllustrations().at(0)->getData(), "Hello, World!");
const std::string rendered = fullEntryOpds(book, "http://root.location", "", "book-with-roundtrip-icon",
/*localPath=*/"", /*isLiveCatalog=*/false);
// Guards against the class of bug where the template puts whitespace (e.g.
// a newline+indentation) around the base64 payload: base64_decode() stops
// at the first non-base64 character rather than skipping it, so such
// whitespace silently turns the decoded data into an empty string. A plain
// golden-string comparison wouldn't catch this if the expected string were
// updated to match a broken rendering, so decode the actual rendered
// output through the real OPDS reader (Manager::readOpds()) instead.
auto roundTripLib = Library::create();
Manager roundTripManager(roundTripLib);
ASSERT_TRUE(roundTripManager.readOpds("<feed>" + rendered + "</feed>", "http://root.location"));
const Book& roundTrippedBook = roundTripLib->getBookById("book-with-roundtrip-icon");
ASSERT_EQ(roundTrippedBook.getIllustrations().size(), 1U);
EXPECT_EQ(roundTrippedBook.getIllustrations().at(0)->getData(), "Hello, World!");
}
TEST(FullEntryOpdsTest, rendersMultipleBase64Thumbnails)
{
auto lib = Library::create();
Manager manager(lib);
const char sampleOpds[] = R"(
<feed>
<entry>
<id>multi-icon-book</id>
<title>Book With Multiple Icons</title>
<link rel="http://opds-spec.org/image/thumbnail"
type="image/png;width=48;height=48;scale=1"
href="data:image/png;base64,Zmlyc3QtdGh1bWJuYWls" />
<link rel="http://opds-spec.org/image/thumbnail"
type="image/jpeg;width=96;height=96;scale=1"
href="data:image/jpeg;base64,c2Vjb25kLXRodW1ibmFpbA==" />
</entry>
</feed>
)";
ASSERT_TRUE(manager.readOpds(sampleOpds, "http://root.location"));
const Book& book = lib->getBookById("multi-icon-book");
EXPECT_EQ(fullEntryOpds(book, "http://root.location", "", "multi-icon-book",
/*localPath=*/"", /*isLiveCatalog=*/false),
" <entry>\n"
" <id>urn:uuid:multi-icon-book</id>\n"
" <title>Book With Multiple Icons</title>\n"
" <updated>T00:00:00Z</updated>\n"
" <summary></summary>\n"
" <language></language>\n"
" <name></name>\n"
" <flavour></flavour>\n"
" <category></category>\n"
" <tags></tags>\n"
" <articleCount>0</articleCount>\n"
" <mediaCount>0</mediaCount>\n"
" <link rel=\"http://opds-spec.org/image/thumbnail\"\n"
" href=\"data:image/png;base64,Zmlyc3QtdGh1bWJuYWls\"\n"
" type=\"image/png;width=48;height=48;scale=1\"/>\n"
" <link rel=\"http://opds-spec.org/image/thumbnail\"\n"
" href=\"data:image/jpeg;base64,c2Vjb25kLXRodW1ibmFpbA==\"\n"
" type=\"image/jpeg;width=96;height=96;scale=1\"/>\n"
" <author>\n"
" <name></name>\n"
" </author>\n"
" <publisher>\n"
" <name></name>\n"
" </publisher>\n"
" <dc:issued>T00:00:00Z</dc:issued>\n"
" \n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, rendersMixedLinkAndBase64ThumbnailsInFileDump)
{
auto lib = Library::create();
Manager manager(lib);
const char sampleOpds[] = R"(
<feed>
<entry>
<id>mixed-icon-book</id>
<title>Book With Mixed Icons</title>
<link rel="http://opds-spec.org/image/thumbnail"
type="image/png"
href="https://example.com/favicon.png" />
<link rel="http://opds-spec.org/image/thumbnail"
type="image/jpeg;width=48;height=48;scale=1"
href="data:image/jpeg;base64,Zmlyc3QtdGh1bWJuYWls" />
</entry>
</feed>
)";
ASSERT_TRUE(manager.readOpds(sampleOpds, "http://root.location"));
const Book& book = lib->getBookById("mixed-icon-book");
EXPECT_EQ(fullEntryOpds(book, "http://root.location", "", "mixed-icon-book",
/*localPath=*/"", /*isLiveCatalog=*/false),
" <entry>\n"
" <id>urn:uuid:mixed-icon-book</id>\n"
" <title>Book With Mixed Icons</title>\n"
" <updated>T00:00:00Z</updated>\n"
" <summary></summary>\n"
" <language></language>\n"
" <name></name>\n"
" <flavour></flavour>\n"
" <category></category>\n"
" <tags></tags>\n"
" <articleCount>0</articleCount>\n"
" <mediaCount>0</mediaCount>\n"
" <link rel=\"http://opds-spec.org/image/thumbnail\"\n"
" href=\"https://example.com/favicon.png\"\n"
" type=\"image/png;width=48;height=48;scale=1\"/>\n"
" <link rel=\"http://opds-spec.org/image/thumbnail\"\n"
" href=\"data:image/jpeg;base64,Zmlyc3QtdGh1bWJuYWls\"\n"
" type=\"image/jpeg;width=48;height=48;scale=1\"/>\n"
" <author>\n"
" <name></name>\n"
" </author>\n"
" <publisher>\n"
" <name></name>\n"
" </publisher>\n"
" <dc:issued>T00:00:00Z</dc:issued>\n"
" \n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, omitsDataUriThumbnailForEmbeddedOnlyIllustrationInLiveCatalog)
{
auto lib = Library::create();
Manager manager(lib);
const char sampleXML[] = R"(
<library version="1.0">
<book
id="book-with-embedded-icon"
path="/local/path/book.zim"
title="Book With Embedded Icon"
favicon="AAAA"
faviconMimeType="image/png"
></book>
</library>
)";
manager.readXml(sampleXML, /*readOnly=*/false, "", /*trustLibrary=*/true);
const Book& book = lib->getBookById("book-with-embedded-icon");
EXPECT_EQ(fullEntryOpds(book, "http://root.location", "", "book-with-embedded-icon"),
" <entry>\n"
" <id>urn:uuid:book-with-embedded-icon</id>\n"
" <title>Book With Embedded Icon</title>\n"
" <updated>T00:00:00Z</updated>\n"
" <summary></summary>\n"
" <language></language>\n"
" <name></name>\n"
" <flavour></flavour>\n"
" <category></category>\n"
" <tags></tags>\n"
" <articleCount>0</articleCount>\n"
" <mediaCount>0</mediaCount>\n"
" <link rel=\"http://opds-spec.org/image/thumbnail\"\n"
" href=\"http://root.location/catalog/v2/illustration/book-with-embedded-icon/?size=48\"\n"
" type=\"image/png;width=48;height=48;scale=1\"/>\n"
" <author>\n"
" <name></name>\n"
" </author>\n"
" <publisher>\n"
" <name></name>\n"
" </publisher>\n"
" <dc:issued>T00:00:00Z</dc:issued>\n"
" \n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, omitsLocalPathAcquisitionLinkWhenEmpty)
{
const Book book = createBook();
// localPath left at its default ("") - mirrors how OPDSDumper calls this
// function for the live HTTP catalog, which must never leak a local path.
EXPECT_EQ(fullEntryOpds(book, "http://root.location", "", "book-id"),
" <entry>\n"
CORE_ENTRY_BODY
" <author>\n"
" <name>Some Creator</name>\n"
" </author>\n"
" <publisher>\n"
" <name>Some Publisher</name>\n"
" </publisher>\n"
" <dc:issued>2021-03-25T00:00:00Z</dc:issued>\n"
" \n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, rendersLocalPathAcquisitionLinkWhenSet)
{
Book book = createBook();
book.setSize(123456);
EXPECT_EQ(fullEntryOpds(book, /*rootLocation=*/"", "", "book-id",
/*localPath=*/"/local/path/book.zim"),
" <entry>\n"
CORE_ENTRY_BODY
" <author>\n"
" <name>Some Creator</name>\n"
" </author>\n"
" <publisher>\n"
" <name>Some Publisher</name>\n"
" </publisher>\n"
" <dc:issued>2021-03-25T00:00:00Z</dc:issued>\n"
" <link rel=\"http://opds-spec.org/acquisition/open-access\" href=\"/local/path/book.zim\" type=\"application/x-zim\" length=\"123456\"/>\n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, rendersBothAcquisitionLinksWhenUrlAndLocalPathAreSet)
{
Book book = createBook();
book.setUrl("http://download.kiwix.org/zim/book.zim");
book.setSize(123456);
EXPECT_EQ(fullEntryOpds(book, /*rootLocation=*/"", "", "book-id",
/*localPath=*/"/local/path/book.zim"),
" <entry>\n"
CORE_ENTRY_BODY
" <author>\n"
" <name>Some Creator</name>\n"
" </author>\n"
" <publisher>\n"
" <name>Some Publisher</name>\n"
" </publisher>\n"
" <dc:issued>2021-03-25T00:00:00Z</dc:issued>\n"
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"http://download.kiwix.org/zim/book.zim\" length=\"123456\" />\n"
" <link rel=\"http://opds-spec.org/acquisition/open-access\" href=\"/local/path/book.zim\" type=\"application/x-zim\" length=\"123456\"/>\n"
" </entry>\n"
);
}
TEST(FullEntryOpdsTest, specialCharactersInBookMetadataAreEscaped)
{
Book book = createBook();
book.setTitle("Q&A <Tricky> \"Title\"");
book.setDescription("Ideas & \"quotes\" <tags>");
EXPECT_EQ(fullEntryOpds(book, "http://root.location", "", "book-id"),
" <entry>\n"
" <id>urn:uuid:book-id</id>\n"
" <title>Q&amp;A &lt;Tricky&gt; &quot;Title&quot;</title>\n"
" <updated>2021-03-25T00:00:00Z</updated>\n"
" <summary>Ideas &amp; &quot;quotes&quot; &lt;tags&gt;</summary>\n"
" <language>eng</language>\n"
" <name>some_name</name>\n"
" <flavour>nopic</flavour>\n"
" <category></category>\n"
" <tags>tag1;tag2;_category:wikipedia</tags>\n"
" <articleCount>42</articleCount>\n"
" <mediaCount>7</mediaCount>\n"
" <author>\n"
" <name>Some Creator</name>\n"
" </author>\n"
" <publisher>\n"
" <name>Some Publisher</name>\n"
" </publisher>\n"
" <dc:issued>2021-03-25T00:00:00Z</dc:issued>\n"
" \n"
" </entry>\n"
);
}
#undef CORE_ENTRY_BODY
} // unnamed namespace
+45
View File
@@ -22,6 +22,7 @@
#include "../src/tools/otherTools.h"
#include "zim/suggestion_iterator.h"
#include "../src/server/i18n_utils.h"
#include "./path_testing_tools.h"
#include <regex>
@@ -258,3 +259,47 @@ TEST(networkTools, getBestPublicIps)
std::cout << "getBestPublicIps(): " << "[" << kiwix::getBestPublicIps().addr << ", " << kiwix::getBestPublicIps().addr6 << "]" << std::endl;
std::cout << "getBestPublicIp(): " << kiwix::getBestPublicIp() << std::endl;
}
TEST(pathTools, resolveContentOrigin)
{
{
const auto origin = kiwix::resolveContentOrigin("https://library.kiwix.org/catalog/v2/entries");
ASSERT_EQ(origin.urlHost, "https://library.kiwix.org");
ASSERT_EQ(origin.baseDir, "");
}
{
const auto origin = kiwix::resolveContentOrigin("http://example.com");
ASSERT_EQ(origin.urlHost, "http://example.com");
ASSERT_EQ(origin.baseDir, "");
}
{
const auto origin = kiwix::resolveContentOrigin("https://library.kiwix.org/");
ASSERT_EQ(origin.urlHost, "https://library.kiwix.org");
ASSERT_EQ(origin.baseDir, "");
}
{
// removeLastPathElement() (which this delegates to for local paths)
// normalizes away the leading "." component - see
// pathTools.normalizePartsRelative above.
const auto origin = kiwix::resolveContentOrigin(P3(".","test","lib.xml"));
ASSERT_EQ(origin.urlHost, "");
ASSERT_EQ(origin.baseDir, "test");
}
{
const auto origin = kiwix::resolveContentOrigin(A2("data","library.opds"));
ASSERT_EQ(origin.urlHost, "");
ASSERT_EQ(origin.baseDir, A1("data"));
}
#ifdef _WIN32
{
const auto origin = kiwix::resolveContentOrigin("C:\\data\\library.opds");
ASSERT_EQ(origin.urlHost, "");
ASSERT_EQ(origin.baseDir, "C:\\data");
}
#endif
{
const auto origin = kiwix::resolveContentOrigin("");
ASSERT_EQ(origin.urlHost, "");
ASSERT_EQ(origin.baseDir, "");
}
}
+1 -20
View File
@@ -26,27 +26,8 @@
#endif
#include "../include/tools.h"
#include "../src/tools/pathTools.h"
#include "./path_testing_tools.h"
#ifdef _WIN32
# define S "\\"
# define AS "c:"
# define A_SAMBA "\\\\sambadir"
#else
# define S "/"
# define AS ""
#endif
#define P2(a, b) a S b
#define P3(a, b, c) P2(P2(a, b), c)
#define P4(a, b, c, d) P2(P3(a, b, c), d)
#define P5(a, b, c, d, e) P2(P4(a, b, c, d), e)
#define P6(a, b, c, d, e, f) P2(P5(a, b, c ,d, e), f)
#define A1(a) P2(AS,a)
#define A2(a, b) A1(P2(a, b))
#define A3(a, b, c) A1(P3(a, b, c))
#define A4(a, b, c, d) A1(P4(a, b, c, d))
#define A5(a, b, c, d, e) A1(P5(a, b, c, d, e))
std::vector<std::string> normalizeParts(std::vector<std::string>& parts, bool absolute);
std::vector<std::string> nParts(std::vector<std::string> parts, bool absolute) {
+44
View File
@@ -0,0 +1,44 @@
/*
* Copyright 2026 Hamazasp Avetisyan <hamik.avetisyan@gmail.com>
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
* published by the Free Software Foundation; either version 2 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* is provided AS IS, WITHOUT ANY WARRANTY; without even the implied
* warranty of MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, and
* NON-INFRINGEMENT. See the GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*
*/
#ifndef LIBKIWIX_PATH_TESTING_TOOLS_H
#define LIBKIWIX_PATH_TESTING_TOOLS_H
#ifdef _WIN32
# define S "\\"
# define AS "c:"
# define A_SAMBA "\\\\sambadir"
#else
# define S "/"
# define AS ""
#endif
#define P2(a, b) a S b
#define P3(a, b, c) P2(P2(a, b), c)
#define P4(a, b, c, d) P2(P3(a, b, c), d)
#define P5(a, b, c, d, e) P2(P4(a, b, c, d), e)
#define P6(a, b, c, d, e, f) P2(P5(a, b, c ,d, e), f)
#define A1(a) P2(AS,a)
#define A2(a, b) A1(P2(a, b))
#define A3(a, b, c) A1(P3(a, b, c))
#define A4(a, b, c, d) A1(P4(a, b, c, d))
#define A5(a, b, c, d, e) A1(P5(a, b, c, d, e))
#endif // LIBKIWIX_PATH_TESTING_TOOLS_H
+1 -1
View File
@@ -12,7 +12,7 @@ RequestContext makeHttpGetRequest(const std::string& url,
const RequestContext::NameValuePairs& headers,
const RequestContext::NameValuePairs& queryArgs)
{
return RequestContext("", url, "GET", "1.1", headers, queryArgs);
return RequestContext(url, 0, "GET", "1.1", headers, queryArgs);
}
std::string getResponseContent(const ContentResponseBlueprint& crb)
+55 -4
View File
@@ -9,6 +9,9 @@
#include "../src/tools/stringTools.h"
#include "testing_tools.h"
#include "../src/server/microhttpd_wrapper.h" // for MHD_VERSION
using namespace kiwix::testing;
const std::string ROOT_PREFIX("/ROOT%23%3F");
@@ -67,7 +70,7 @@ const ResourceCollection resources200Compressible{
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/index.css" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/index.css?cacheid=ae79e41a" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/index.js" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/index.js?cacheid=e3305ca0" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/index.js?cacheid=3bd1263f" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/iso6391To3.js" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/iso6391To3.js?cacheid=ecde2bb3" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/isotope.pkgd.min.js" },
@@ -79,7 +82,7 @@ const ResourceCollection resources200Compressible{
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/taskbar.css" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/taskbar.css?cacheid=42e90cb9" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/viewer.js" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/viewer.js?cacheid=6192cae1" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/viewer.js?cacheid=f78c03d9" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/fonts/Poppins.ttf" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/fonts/Poppins.ttf?cacheid=af705837" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/fonts/Roboto.ttf" },
@@ -306,7 +309,7 @@ R"EXPECTEDRESULT( href="/ROOT%23%3F/skin/kiwix.css?cacheid=b4e29e64"
<script type="text/javascript" src="/ROOT%23%3F/skin/languages.js?cacheid=d2d6933b" defer></script>
<script src="/ROOT%23%3F/skin/isotope.pkgd.min.js?cacheid=2e48d392" defer></script>
<script src="/ROOT%23%3F/skin/iso6391To3.js?cacheid=ecde2bb3"></script>
<script type="text/javascript" src="/ROOT%23%3F/skin/index.js?cacheid=e3305ca0" defer></script>
<script type="text/javascript" src="/ROOT%23%3F/skin/index.js?cacheid=3bd1263f" defer></script>
<img src="/ROOT%23%3F/skin/feed.svg?cacheid=055b333f"
<img src="/ROOT%23%3F/skin/langSelector.svg?cacheid=00b59961"
)EXPECTEDRESULT"
@@ -340,7 +343,7 @@ R"EXPECTEDRESULT( <link type="text/css" href="./skin/kiwix.css?cacheid=b4e29e
<script type="text/javascript" src="./skin/polyfills.js?cacheid=a0e0343d"></script>
<script type="module" src="./skin/i18n.js?cacheid=e9a10ac1" defer></script>
<script type="text/javascript" src="./skin/languages.js?cacheid=d2d6933b" defer></script>
<script type="text/javascript" src="./skin/viewer.js?cacheid=6192cae1" defer></script>
<script type="text/javascript" src="./skin/viewer.js?cacheid=f78c03d9" defer></script>
<script type="text/javascript" src="./skin/autoComplete/autoComplete.min.js?cacheid=1191aaaf"></script>
const blankPageUrl = root + "/skin/blank.html?cacheid=6b1fa032";
<label for="kiwix_button_show_toggle"><img src="./skin/caret.png?cacheid=22b942b4" alt=""></label>
@@ -439,6 +442,10 @@ TEST_F(ServerTest, CacheIdsOfStaticResourcesMatchTheSha1HashOfResourceContent)
}
const char* urls400[] = {
#if MHD_VERSION >= 0x01000300
"/ROOT%23%",
"/ROOT%23%3",
#endif // MHD_VERSION
"/ROOT%23%3F/search",
"/ROOT%23%3F/search?content=non-existing-book&pattern=asdfqwerty",
"/ROOT%23%3F/search?content=non-existing-book&pattern=asd<qwerty",
@@ -461,8 +468,10 @@ const char* urls404[] = {
"/",
"/zimfile",
"/ROOT",
#if MHD_VERSION < 0x01000300
"/ROOT%23%",
"/ROOT%23%3",
#endif // MHD_VERSION
"/ROOT%23%3Fxyz",
"/ROOT%23%3F/skin/non-existent-skin-resource",
"/ROOT%23%3F/skin/autoComplete/autoComplete.min.js?cacheid=wrongcacheid",
@@ -779,6 +788,47 @@ TEST_F(ServerTest, Http404HtmlError)
{
using namespace TestingOfHtmlResponses;
const std::vector<TestContentIn404HtmlResponse> testData{
// wrong root URL (root URL missing completely)
{ /* url */ "/",
expected_kiwix_response_data==R"({ "CSS_URL" : false, "PAGE_HEADING" : { "msgid" : "404-page-heading", "params" : { } }, "PAGE_TITLE" : { "msgid" : "404-page-title", "params" : { } }, "details" : [ { "p" : { "msgid" : "url-not-found", "params" : { "url" : "/" } } } ] })" &&
expected_body==R"(
<h1>Not Found</h1>
<p>
The requested URL "/" was not found on this server.
</p>
)" },
// (conspicuously) wrong root URL
{ /* url */ "/WRONGROOT",
expected_kiwix_response_data==R"({ "CSS_URL" : false, "PAGE_HEADING" : { "msgid" : "404-page-heading", "params" : { } }, "PAGE_TITLE" : { "msgid" : "404-page-title", "params" : { } }, "details" : [ { "p" : { "msgid" : "url-not-found", "params" : { "url" : "/WRONGROOT" } } } ] })" &&
expected_body==R"(
<h1>Not Found</h1>
<p>
The requested URL "/WRONGROOT" was not found on this server.
</p>
)" },
// wrong root URL with the correct root URL appearing as a suffix
{ /* url */ "/WRONGROOT/ROOT%23%3F",
expected_kiwix_response_data==R"({ "CSS_URL" : false, "PAGE_HEADING" : { "msgid" : "404-page-heading", "params" : { } }, "PAGE_TITLE" : { "msgid" : "404-page-title", "params" : { } }, "details" : [ { "p" : { "msgid" : "url-not-found", "params" : { "url" : "/WRONGROOT/ROOT%23%3F" } } } ] })" &&
expected_body==R"(
<h1>Not Found</h1>
<p>
The requested URL "/WRONGROOT/ROOT%23%3F" was not found on this server.
</p>
)" },
// wrong root URL (with the correct root URL appearing as a prefix)
{ /* url */ "/ROOT%23%3FWRONGROOT",
expected_kiwix_response_data==R"({ "CSS_URL" : false, "PAGE_HEADING" : { "msgid" : "404-page-heading", "params" : { } }, "PAGE_TITLE" : { "msgid" : "404-page-title", "params" : { } }, "details" : [ { "p" : { "msgid" : "url-not-found", "params" : { "url" : "/ROOT%23%3FWRONGROOT" } } } ] })" &&
expected_body==R"(
<h1>Not Found</h1>
<p>
The requested URL "/ROOT%23%3FWRONGROOT" was not found on this server.
</p>
)" },
{ /* url */ "/ROOT%23%3F/random?content=non-existent-book",
expected_kiwix_response_data==R"({ "CSS_URL" : false, "PAGE_HEADING" : { "msgid" : "404-page-heading", "params" : { } }, "PAGE_TITLE" : { "msgid" : "404-page-title", "params" : { } }, "details" : [ { "p" : { "msgid" : "no-such-book", "params" : { "BOOK_NAME" : "non-existent-book" } } } ] })" &&
expected_body==R"(
@@ -2376,6 +2426,7 @@ TEST(ServerNegativeTest, UnusablePort)
EXPECT_ERROR("Unable to instantiate the HTTP daemon. The port 8910 is maybe "
"already occupied or need more permissions to be open. Please "
"try as root or with a port number higher or equal to 1024.",
server.setAddress("127.0.0.1");
server.setPort(8910);
);
}
+10 -6
View File
@@ -950,7 +950,9 @@ struct TestData
}
};
TEST(ServerSearchTest, searchResults)
class LibForServerSearchTest : public ::testing::TestWithParam<std::string> {};
TEST_P(LibForServerSearchTest, searchResults)
{
const TestData testData[] = {
{
@@ -1556,8 +1558,7 @@ TEST(ServerSearchTest, searchResults)
},
};
ZimFileServer zfs(SERVER_PORT, ZimFileServer::DEFAULT_OPTIONS,
"./test/lib_for_server_search_test.xml");
ZimFileServer zfs(SERVER_PORT, ZimFileServer::DEFAULT_OPTIONS, GetParam());
for ( const auto& t : testData ) {
const std::string htmlSearchUrl = t.url();
@@ -1675,10 +1676,9 @@ std::string noBookFoundErrorHtml(std::string url)
);
}
TEST(ServerSearchTest, bookSelectionNegativeTests)
TEST_P(LibForServerSearchTest, bookSelectionNegativeTests)
{
ZimFileServer zfs(SERVER_PORT, ZimFileServer::DEFAULT_OPTIONS,
"./test/lib_for_server_search_test.xml");
ZimFileServer zfs(SERVER_PORT, ZimFileServer::DEFAULT_OPTIONS, GetParam());
{
// books.name (unlike books.filter.name) DOESN'T consider the book name
@@ -1703,3 +1703,7 @@ TEST(ServerSearchTest, bookSelectionNegativeTests)
EXPECT_EQ(r->body, noBookFoundErrorHtml(url));
}
}
INSTANTIATE_TEST_CASE_P(XmlAndOpds, LibForServerSearchTest,
::testing::Values("./test/lib_for_server_search_test.xml",
"./test/lib_for_server_search_test.opds"));
+60
View File
@@ -20,6 +20,7 @@
#include "gtest/gtest.h"
#include "../src/tools/stringTools.h"
#include "../include/tools.h"
#include <limits>
#include <string>
#include <vector>
@@ -64,6 +65,65 @@ TEST(stringTools, join)
ASSERT_EQ(join(list, ";"), "a;b;c");
}
#define EXPECT_ONE_OF(val, expected1, expected2) \
do { \
auto res = (val); \
EXPECT_TRUE(res == expected1 || res == expected2) \
<< "Expected one of '" << expected1 << "' or '" << expected2 \
<< "', but got '" << res << "'"; \
} while(0)
TEST(stringTools, beautifyFileSize)
{
// Bytes range (< 1 000 B — displayed without decimal places)
EXPECT_EQ(beautifyFileSize(0), "0 B");
EXPECT_EQ(beautifyFileSize(1), "1 B");
EXPECT_EQ(beautifyFileSize(999), "999 B");
// KB range (1 000 999 994 bytes, base-1000 / SI)
EXPECT_EQ(beautifyFileSize(1000), "1.00 KB");
EXPECT_EQ(beautifyFileSize(1001), "1.00 KB");
EXPECT_EQ(beautifyFileSize(1500), "1.50 KB");
EXPECT_EQ(beautifyFileSize(999994), "999.99 KB");
// Rounding-promotion boundary: >= 999 995 bytes would display as "1000.00 KB",
// so the function promotes to the next unit → 1.00 MB.
// Due to floating point precision, this might be "999.99 KB" or "1.00 MB".
EXPECT_ONE_OF(beautifyFileSize(999995), "999.99 KB", "1.00 MB");
EXPECT_EQ(beautifyFileSize(999999), "1.00 MB");
// MB range
EXPECT_EQ(beautifyFileSize(1000000), "1.00 MB");
// 1 254 786 / 1 000 000 = 1.254786 → rounded to 2 dp → 1.25 MB
EXPECT_EQ(beautifyFileSize(1254786), "1.25 MB");
EXPECT_EQ(beautifyFileSize(999000000), "999.00 MB");
// GB range
EXPECT_EQ(beautifyFileSize(1000000000ULL), "1.00 GB");
EXPECT_EQ(beautifyFileSize(1500000000ULL), "1.50 GB");
// MB→GB promotion boundary
EXPECT_EQ(beautifyFileSize(999994999ULL), "999.99 MB");
EXPECT_ONE_OF(beautifyFileSize(999995000ULL), "999.99 MB", "1.00 GB");
// TB range
EXPECT_EQ(beautifyFileSize(1000000000000ULL), "1.00 TB");
// GB→TB promotion boundary
EXPECT_EQ(beautifyFileSize(999994999999ULL), "999.99 GB");
EXPECT_ONE_OF(beautifyFileSize(999995000000ULL), "999.99 GB", "1.00 TB");
// PB range — also covers TB→PB promotion boundary
EXPECT_EQ(beautifyFileSize(999994999999999ULL), "999.99 TB");
EXPECT_ONE_OF(beautifyFileSize(999995000000000ULL), "999.99 TB", "1.00 PB");
EXPECT_EQ(beautifyFileSize(1000000000000000ULL), "1.00 PB");
// EB range
EXPECT_EQ(beautifyFileSize(1000000000000000000ULL), "1.00 EB");
// uint64_t max: 18 446 744 073 709 551 615 / 1e18 ≈ 18.446 → 18.45 EB
EXPECT_EQ(beautifyFileSize(std::numeric_limits<uint64_t>::max()), "18.45 EB");
}
#undef EXPECT_ONE_OF
TEST(stringTools, split)
{
std::vector<std::string> list1 = { "a", "b", "c" };