Compare commits

...

36 Commits

Author SHA1 Message Date
Veloman Yunkan
34fc831c13 Documentation of a subset of Server API
Documentation of libkiwix API has been largely neglected. This commit
documents the Server API having to do with the functionality fixed in
the previous commit.
2026-02-24 16:10:57 +04:00
Veloman Yunkan
47f1a61443 Made kiwix-serve friendly to IPv4-only systems
kiwix-serve without an explicitly supplied -i option can henceforth run
on IPv4-only systems. Before this fix, an "Unable to instantiate the
HTTP daemon." error was being reported.
2026-02-23 20:04:04 +04:00
Veloman Yunkan
a3b246fd6d Extracted InternalServer::startMHD(flags, sockaddr) 2026-02-23 19:50:38 +04:00
Veloman Yunkan
75ff043476 Moved some code into InSockAddr::setAnyAddress() 2026-02-23 19:22:03 +04:00
Veloman Yunkan
0f7458c693 Extracted getMHDFlags() 2026-02-23 19:11:43 +04:00
Veloman Yunkan
add7c4cc3a Moved some code into InSockAddr::setAddress() 2026-02-23 19:11:43 +04:00
Veloman Yunkan
fcdc3a7eaf Extracted InSockAddr 2026-02-23 19:09:22 +04:00
Veloman Yunkan
db3705dd23 Errors in InternalServer::startMHD() as exceptions 2026-02-23 18:58:55 +04:00
Veloman Yunkan
3b34130cfc Extracted InternalServer::startMHD() 2026-02-23 18:14:09 +04:00
Veloman Yunkan
b773e93a95 Negative unit tests for starting the server 2026-02-23 18:14:09 +04:00
Veloman Yunkan
e481164258 Extracted test/testing_tools.h 2026-02-23 18:09:33 +04:00
Kelson
32cd7661ff Merge pull request #1274 from kiwix/no-pagination-atom-feed
No pagination for Atom feed URL
2026-02-23 10:14:07 +01:00
Emmanuel Engelhart
f29c69aac6 No pagination for Atom feed URL
New cache busting id for skin/index.js
2026-02-23 10:13:10 +01:00
Kelson
3fe9be88d4 Merge pull request #1273 from kiwix/add-10.211-to-known-loca-ip-range
Add 10.211 IPv4 range to known local ranges
2026-02-23 10:11:09 +01:00
Emmanuel Engelhart
910ad614c4 Add 10.211 IPv4 range to known local ranges 2026-02-23 10:10:59 +01:00
Kelson
724b6cc0cc Merge pull request #1220 from kiwix/dirScan
Introduce Manager::addBooksFromDirectory()
2026-02-22 20:01:34 +01:00
Nikhil Tanwar
c5ec7651a6 Introduce Manager::addBooksFromDirectory()
Added a function to load books from a directory. Requires rootPath to iterate over.
2026-02-22 19:35:07 +01:00
Kelson
949a612fd4 Merge pull request #1261 from shbhmexe/fix/stability-and-correctness-fixes
Fix core stability, correctness, and memory issues
2026-02-22 19:01:31 +01:00
shbhmexe
974a5ff87f Fix infinite loop and memory leak in getNetworkInterfacesWin
- Increment the iteration counter to prevent infinite loops when GetAdaptersAddresses returns ERROR_BUFFER_OVERFLOW.
- Free the previously allocated buffer before re-allocating to prevent memory leaks on retry.

Signed-off-by: shbhmexe <shubhushukla586@gmail.com>
2026-02-22 18:37:36 +01:00
shbhmexe
503e55c87e Fix various stability and correctness issues in libkiwix
- Fix infinite loop and memory leak in Windows network tools (networkTools.cpp)
- Improve exception safety in removeAccents using RAII (stringTools.cpp)
- Change getBookById to return by value to avoid dangling references (library.cpp)
- Use proper constant for HTTP 416 instead of magic number (response.cpp)
- Fix exception slicing in getDownload re-throw (downloader.cpp)

Signed-off-by: shbhmexe <shubhushukla586@gmail.com>
2026-02-22 18:37:36 +01:00
Veloman Yunkan
fb6448011b Merge pull request #1256 from kiwix/translatewiki
Localisation updates from https://translatewiki.net.
2026-02-12 17:39:01 +04:00
Veloman Yunkan
9493c092ad Updated/regenerated static/skin/languages.js 2026-02-12 17:32:41 +04:00
translatewiki.net
52b02b964d Localisation updates from https://translatewiki.net. 2026-02-12 13:07:56 +01:00
Kelson
78124833b3 Merge pull request #1269 from kiwix/windows-2025-ci
Windows 2025 ci
2026-02-11 11:12:36 +01:00
Emmanuel Engelhart
0d9bab12b9 use windows-2025 runner for CI 2026-02-11 11:05:22 +01:00
Veloman Yunkan
650f2d0027 Merge pull request #1255 from pippotadde/fix-empty-pattern-737
Fix: avoid error on empty search pattern (#1255)
2026-01-12 12:38:07 +04:00
Veloman Yunkan
b6a13d9b03 Minor cosmetic clean-up
Performed minor cosmetic clean-up in the ServerSearchTest.searchResults
unit-test:

1. Fixed alignment in test data
2. Moved the test point for search query "yellow submarine" out of the
   set of test points for search query "jazz"
2026-01-12 12:21:56 +04:00
pippotadde
86f125f8fb Fix: avoid error on empty search pattern 2026-01-12 12:18:53 +04:00
pippotadde
3cd8554733 Tools: Add kiwix::trim() helper and unit tests 2025-12-28 20:31:51 +01:00
Kelson
a10f0f287f Merge pull request #1262 from kiwix/carriage-return-after-error
Carriage return after error
2025-12-27 11:55:07 +01:00
Emmanuel Engelhart
cdf48fc987 Add carriage return after error 2025-12-27 11:40:28 +01:00
Veloman Yunkan
ba598bda9b Merge pull request #1259 from pippotadde/pr_yellow_tests
Testing of a search pattern containing a space
2025-12-23 21:36:58 +04:00
pippotadde
0ad2710884 Tests: add yellow submarine search case 2025-12-23 17:02:52 +01:00
pippotadde
ab31ed9ca5 Tests: deduplicate yellow search results 2025-12-23 17:01:13 +01:00
Veloman Yunkan
86cbc303cb Merge pull request #1258 from pippotadde/pr_viewerjs
Frontend: guard empty search input
2025-12-23 17:50:25 +04:00
pippotadde
19d9bc36c8 Frontend: guard empty search input 2025-12-23 14:30:42 +01:00
29 changed files with 571 additions and 267 deletions

View File

@@ -69,7 +69,7 @@ jobs:
run: meson test -C build --verbose
Windows:
runs-on: windows-2022
runs-on: windows-2025
steps:
- name: Checkout code

View File

@@ -18,7 +18,7 @@ import os
# -- Project information -----------------------------------------------------
project = 'libkiwix'
copyright = '2022, libkiwix-team'
copyright = '2026, libkiwix-team'
author = 'libkiwix-team'

View File

@@ -14,12 +14,43 @@
#endif
namespace kiwix {
namespace kiwix
{
/**
* `IpMode` is used to [configure](@ref Server::setIpMode()) a `Server` object
* to listen for incoming connections not on a single IP address but on all
* addresses of the specified family.
*/
enum class IpMode
{
/**
* Listen on all IPv4 addresses.
*/
IPV4,
/**
* Listen on all IPv6 addresses.
*/
IPV6,
/**
* Listen on all available addresses.
*/
ALL,
/**
* `IpMode::AUTO` (which is the default) must be used when an explicit
* (non-empty) IP address for listening is provided via
* `Server::setAddress()`. If no such address is enforced, then
* `IpMode::AUTO` is equivalent to `IpMode::ALL`.
*/
AUTO
};
enum class IpMode { IPV4, IPV6, ALL, AUTO }; // AUTO: Server decides the protocol
typedef zim::size_type size_type;
typedef zim::offset_type offset_type;
}
} // namespace kiwix
#endif //_KIWIX_COMMON_H_

View File

@@ -155,6 +155,15 @@ class Manager
const std::string& url = "",
const bool checkMetaData = false);
/**
* Add all books from the directory tree into the library.
*
* @param path The path of the directory to scan.
* @param verboseFlag Verbose logs flag.
*/
void addBooksFromDirectory(const std::string& path,
const bool verboseFlag = false);
std::string writableLibraryPath;
bool m_hasSearchResult = false;

View File

@@ -42,7 +42,7 @@ namespace kiwix
virtual ~Server();
/**
* Serve the content.
* Start serving the content.
*/
bool start();
@@ -51,9 +51,27 @@ namespace kiwix
*/
void stop();
/**
* Set the path of the root URL served by this server instance.
*/
void setRoot(const std::string& root);
/**
* Set the IP address on which to listen for incoming connections.
*
* Specifying a non-empty IP address requires that the IpMode is
* [set](@ref setIpMode()) to `IpMode::AUTO` (which is the default).
* Otherwise, [starting](@ref start()) the server will fail.
*/
void setAddress(const std::string& addr);
/**
* Set the port on which to listen for incoming connections.
*
* Default port is 80, but using it requires special privileges.
*/
void setPort(int port) { m_port = port; }
void setNbThreads(int threads) { m_nbThreads = threads; }
void setMultiZimSearchLimit(unsigned int limit) { m_multizimSearchLimit = limit; }
void setIpConnectionLimit(int limit) { m_ipConnectionLimit = limit; }
@@ -65,10 +83,50 @@ namespace kiwix
{ m_blockExternalLinks = blockExternalLinks; }
void setCatalogOnlyMode(bool enable) { m_catalogOnlyMode = enable; }
void setContentServerUrl(std::string url) { m_contentServerUrl = url; }
/**
* Listen for incoming connections on all IP addresses of the specified
* IP protocol family.
*/
void setIpMode(IpMode mode) { m_ipMode = mode; }
/**
* Get the port on which the server listens for incoming connections
*/
int getPort() const;
/**
* Get the IPv4 and/or IPv6 address(es) on which the server can be
* contacted.
*
* The server may actually be listening on other IP addresses as well
* (see `setIpMode()`). The IP address(es) returned by this method
* represent the best-guess public IP address(es) accessible by the
* broadest set of clients.
*/
IpAddress getAddress() const;
/**
* Get the effective IpMode used by this server.
*
* The returned value may be different from the one configured via
* `setIpMode()`, since the server is expected to adjust to the
* constraints of the environment (e.g. `IpMode::ALL` can be converted
* to `IpMode::IPV4` on a system that does not support IPv6).
*/
IpMode getIpMode() const;
/**
* Get the list of HTTP URLs through which the server can be contacted.
*
* Each URL is composed of an IP address and optional port and includes
* the [root](@ref setRoot()) path component.
*
* In the current implementation at most 2 URLs may be returned - one
* for IPv4 and another for IPv6 protocol (whichever is available), as
* returned by `getAddress()`. Note, however, that the server may be
* also accessible via other IP-addresses/URLs (see `setIpMode()`).
*/
std::vector<std::string> getServerAccessUrls() const;
protected:

View File

@@ -29,10 +29,13 @@
namespace kiwix
{
/**
* An IPv4 and/or IPv6 address.
*/
struct IpAddress
{
std::string addr; // IPv4 address
std::string addr6; // IPv6 address
std::string addr; /**< IPv4 address */
std::string addr6; /**< IPv6 address */
};
typedef std::pair<std::string, std::string> LangNameCodePair;
@@ -263,7 +266,7 @@ std::string getLanguageSelfName(const std::string& lang);
* Slugifies the filename by converting any characters reserved by the operating
* system to '_'. Note filename is only the file name and not a path.
*
* @param filename Valid UTF-8 encoded file name string.
* @param filename Valid UTF-8 encoded file name string.
* @return slugified string.
*/
std::string getSlugifiedFileName(const std::string& filename);

View File

@@ -242,7 +242,7 @@ const std::string& Book::Illustration::getData() const
try {
data = download(url);
} catch(...) {
std::cerr << "Cannot download favicon from " << url;
std::cerr << "Cannot download favicon from " << url << std::endl;
}
}
}

View File

@@ -245,7 +245,7 @@ std::shared_ptr<Download> Downloader::getDownload(const std::string& did)
return m_knownDownloads[gid];
}
}
throw e;
throw;
}
}

View File

@@ -202,7 +202,7 @@ std::string Library::getBestFromBookCollection(BookIdCollection books, const Boo
}
sort(books, DATE, false);
stable_sort(books.begin(), books.end(), [&](const std::string& bookId1, const std::string& bookId2) {
std::stable_sort(books.begin(), books.end(), [&](const std::string& bookId1, const std::string& bookId2) {
const auto& book1 = getBookById(bookId1);
const auto& book2 = getBookById(bookId2);
bool same_flavour1 = book1.getFlavour() == bookmark.getBookFlavour();

View File

@@ -23,6 +23,14 @@
#include "tools/pathTools.h"
#include <pugixml.hpp>
#include <filesystem>
#include <iostream>
#include <set>
#include <queue>
#include <cctype>
#include <algorithm>
namespace fs = std::filesystem;
namespace kiwix
{
@@ -251,6 +259,58 @@ bool Manager::addBookFromPath(const std::string& pathToOpen,
.empty());
}
void Manager::addBooksFromDirectory(const std::string& path,
const bool verboseFlag)
{
std::set<std::string> iteratedDirs;
std::queue<std::string> dirQueue;
dirQueue.push(fs::absolute(path).u8string());
int totalBooksAdded = 0;
if (verboseFlag)
std::cout << "Adding books from the directory tree: " << dirQueue.front() << std::endl;
while (!dirQueue.empty()) {
const auto currentPath = dirQueue.front();
dirQueue.pop();
if (verboseFlag)
std::cout << "Visiting directory: " << currentPath << std::endl;
for (const auto& dirEntry : fs::directory_iterator(currentPath)) {
auto resolvedPath = dirEntry.path();
if (fs::is_symlink(dirEntry)) {
try {
resolvedPath = fs::canonical(dirEntry.path());
} catch (const std::exception& e) {
std::cerr << "Could not resolve symlink " << resolvedPath.u8string() << " to a valid path. Skipping..." << std::endl;
continue;
}
}
const std::string pathString = resolvedPath.u8string();
std::string resolvedPathExtension = resolvedPath.extension().u8string();
std::transform(resolvedPathExtension.begin(), resolvedPathExtension.end(), resolvedPathExtension.begin(),
[](unsigned char c){ return std::tolower(c); });
if (fs::is_directory(resolvedPath)) {
if (iteratedDirs.find(pathString) == iteratedDirs.end())
dirQueue.push(pathString);
else if (verboseFlag)
std::cout << "Already iterated over " << pathString << ". Skipping..." << std::endl;
} else if (resolvedPathExtension == ".zim" || resolvedPathExtension == ".zimaa") {
if (!this->addBookFromPath(pathString, pathString, "", false)) {
std::cerr << "Could not add " << pathString << " into the library." << std::endl;
} else if (verboseFlag) {
std::cout << "Added " << pathString << " into the library." << std::endl;
totalBooksAdded++;
}
} else if (verboseFlag) {
std::cout << "Skipped " << pathString << " - unsupported file type or permission denied." << std::endl;
}
}
iteratedDirs.insert(currentPath);
}
if (verboseFlag)
std::cout << "Traversal completed. Total books added: " << totalBooksAdded << std::endl;
}
bool Manager::readBookFromPath(const std::string& path, kiwix::Book* book)
{
std::string tmp_path = path;

View File

@@ -85,6 +85,10 @@ namespace kiwix {
namespace
{
void error(const std::string& msg) {
throw std::runtime_error(msg);
}
bool ipAvailable(const std::string addr)
{
auto interfaces = kiwix::getNetworkInterfacesIPv4Or6();
@@ -99,6 +103,84 @@ bool ipAvailable(const std::string addr)
return false;
}
class InSockAddr
{
private: // data
struct sockaddr_in v4 = {0};
struct sockaddr_in6 v6 = {0};
public: // functions
explicit InSockAddr(int port)
{
v4.sin_family = AF_INET;
v4.sin_port = htons(port);
v6.sin6_family = AF_INET6;
v6.sin6_port = htons(port);
}
IpAddress setAnyAddress(IpMode ipMode)
{
v6.sin6_addr = in6addr_any;
v4.sin_addr.s_addr = htonl(INADDR_ANY);
IpAddress a = kiwix::getBestPublicIps();
if (ipMode == IpMode::IPV6) {
a.addr = "";
} else if (ipMode == IpMode::IPV4) {
a.addr6 = "";
}
return a;
}
IpMode setAddress(const IpAddress& a)
{
const std::string addrStr = !a.addr.empty() ? a.addr : a.addr6;
const int r1 = inet_pton(AF_INET, a.addr.c_str(), &v4.sin_addr.s_addr);
const int r2 = inet_pton(AF_INET6, a.addr6.c_str(), &v6.sin6_addr.s6_addr);
if ( r1 != 1 && r2 != 1 ) {
error("invalid IP address: " + addrStr);
}
if ( !ipAvailable(addrStr) ) {
error("IP address is not available on this system: " + addrStr);
}
return !a.addr.empty() ? IpMode::IPV4 : IpMode::IPV6;
}
struct sockaddr* sockaddr(IpMode ipMode) const
{
return (ipMode==IpMode::ALL || ipMode==IpMode::IPV6)
? (struct sockaddr*)&v6
: (struct sockaddr*)&v4;
}
};
int getMHDFlags(IpMode ipMode, bool verbose)
{
#ifdef _WIN32
int flags = MHD_USE_SELECT_INTERNALLY;
#else
int flags = MHD_USE_POLL_INTERNALLY;
#endif
if (ipMode == IpMode::ALL) {
flags |= MHD_USE_DUAL_STACK;
} else if (ipMode == IpMode::IPV6) {
flags |= MHD_USE_IPv6;
}
if (verbose) {
flags |= MHD_USE_DEBUG;
}
return flags;
}
std::string
fullURL2LocalURL(const std::string& fullUrl, const std::string& rootLocation)
{
@@ -346,11 +428,6 @@ SearchInfo InternalServer::getSearchInfo(const RequestContext& request) const
geoQuery = GeoQuery(latitude, longitude, distance);
} catch(const std::out_of_range&) {}
catch(const std::invalid_argument&) {}
if (!geoQuery && pattern.empty()) {
throw Error(nonParameterizedMessage("no-query"));
}
return SearchInfo(pattern, geoQuery, bookIds.second, bookIds.first);
}
@@ -366,7 +443,7 @@ zim::Query SearchInfo::getZimQuery(bool verbose) const {
if (verbose) {
std::cout << "Performing query '" << pattern<< "'";
}
query.setQuery(pattern);
query.setQuery(kiwix::trim(pattern));
if (geoQuery) {
if (verbose) {
std::cout << " with geo query '" << geoQuery.distance << "&(" << geoQuery.latitude << ";" << geoQuery.longitude << ")'";
@@ -455,81 +532,61 @@ InternalServer::InternalServer(LibraryPtr library,
InternalServer::~InternalServer() = default;
bool InternalServer::start() {
#ifdef _WIN32
int flags = MHD_USE_SELECT_INTERNALLY;
#else
int flags = MHD_USE_POLL_INTERNALLY;
#endif
if (m_verbose.load())
flags |= MHD_USE_DEBUG;
struct sockaddr_in sockAddr4={0};
sockAddr4.sin_family = AF_INET;
sockAddr4.sin_port = htons(m_port);
struct sockaddr_in6 sockAddr6={0};
sockAddr6.sin6_family = AF_INET6;
sockAddr6.sin6_port = htons(m_port);
struct MHD_Daemon* InternalServer::startMHD(int flags,
struct sockaddr* sockaddr)
{
return MHD_start_daemon(flags,
m_port,
NULL,
NULL,
&staticHandlerCallback,
this,
MHD_OPTION_SOCK_ADDR, sockaddr,
MHD_OPTION_THREAD_POOL_SIZE, m_nbThreads,
MHD_OPTION_PER_IP_CONNECTION_LIMIT, m_ipConnectionLimit,
MHD_OPTION_END);
}
void InternalServer::startMHD() {
InSockAddr inSockAddr(m_port);
if (m_addr.addr.empty() && m_addr.addr6.empty()) { // No ip address provided
if (m_ipMode == IpMode::AUTO) m_ipMode = IpMode::ALL;
sockAddr6.sin6_addr = in6addr_any;
sockAddr4.sin_addr.s_addr = htonl(INADDR_ANY);
IpAddress bestIps = kiwix::getBestPublicIps();
if (m_ipMode == IpMode::IPV4 || m_ipMode == IpMode::ALL) m_addr.addr = bestIps.addr;
if (m_ipMode == IpMode::IPV6 || m_ipMode == IpMode::ALL) m_addr.addr6 = bestIps.addr6;
m_addr = inSockAddr.setAnyAddress(m_ipMode);
} else {
const std::string addr = !m_addr.addr.empty() ? m_addr.addr : m_addr.addr6;
if (m_ipMode != kiwix::IpMode::AUTO) {
std::cerr << "ERROR: When an IP address is provided the IP mode must not be set" << std::endl;
return false;
error("When an IP address is provided the IP mode must not be set");
}
bool validV4 = inet_pton(AF_INET, m_addr.addr.c_str(), &(sockAddr4.sin_addr.s_addr)) == 1;
bool validV6 = inet_pton(AF_INET6, m_addr.addr6.c_str(), &(sockAddr6.sin6_addr.s6_addr)) == 1;
if (!validV4 && !validV6) {
std::cerr << "ERROR: invalid IP address: " << addr << std::endl;
return false;
}
if (!ipAvailable(addr)) {
std::cerr << "ERROR: IP address is not available on this system: " << addr << std::endl;
return false;
}
m_ipMode = !m_addr.addr.empty() ? IpMode::IPV4 : IpMode::IPV6;
m_ipMode = inSockAddr.setAddress(m_addr);
}
if (m_ipMode == IpMode::ALL) {
flags|=MHD_USE_DUAL_STACK;
} else if (m_ipMode == IpMode::IPV6) {
flags|=MHD_USE_IPv6;
const int flags = getMHDFlags(m_ipMode, m_verbose.load());
mp_daemon = startMHD(flags, inSockAddr.sockaddr(m_ipMode));
if (mp_daemon == nullptr && m_ipMode == IpMode::ALL) {
// MHD_USE_DUAL_STACK (set in IpMode::ALL case) fails on systems with IPv6
// disabled. Let's retry in IPv4-only mode.
m_ipMode = IpMode::IPV4;
m_addr.addr6 = "";
const int flags = getMHDFlags(m_ipMode, m_verbose.load());
mp_daemon = startMHD(flags, inSockAddr.sockaddr(m_ipMode));
}
struct sockaddr* sockaddr = (m_ipMode==IpMode::ALL || m_ipMode==IpMode::IPV6)
? (struct sockaddr*)&sockAddr6
: (struct sockaddr*)&sockAddr4;
mp_daemon = MHD_start_daemon(flags,
m_port,
NULL,
NULL,
&staticHandlerCallback,
this,
MHD_OPTION_SOCK_ADDR, sockaddr,
MHD_OPTION_THREAD_POOL_SIZE, m_nbThreads,
MHD_OPTION_PER_IP_CONNECTION_LIMIT, m_ipConnectionLimit,
MHD_OPTION_END);
if (mp_daemon == nullptr) {
std::cerr << "Unable to instantiate the HTTP daemon. The port " << m_port
<< " is maybe already occupied or need more permissions to be open. "
"Please try as root or with a port number higher or equal to 1024."
<< std::endl;
error("Unable to instantiate the HTTP daemon. "
"The port " + kiwix::to_string(m_port) + " is maybe already occupied"
" or need more permissions to be open. "
"Please try as root or with a port number higher or equal to 1024.");
}
}
bool InternalServer::start() {
try {
startMHD();
} catch (const std::runtime_error& err ) {
std::cerr << "ERROR: " << err.what() << std::endl;
return false;
}
auto server_start_time = std::chrono::system_clock::now().time_since_epoch();
m_server_id = kiwix::to_string(server_start_time.count());
return true;

View File

@@ -126,6 +126,8 @@ class InternalServer {
IpMode getIpMode() const { return m_ipMode; }
private: // functions
void startMHD();
struct MHD_Daemon* startMHD(int flags, struct sockaddr* sockaddr);
std::unique_ptr<Response> handle_request(const RequestContext& request);
std::unique_ptr<Response> build_redirect(const std::string& bookName, const zim::Item& item) const;
std::unique_ptr<Response> build_homepage(const RequestContext& request);

View File

@@ -520,9 +520,7 @@ HTTP500Response::HTTP500Response(const RequestContext& request,
std::unique_ptr<Response> Response::build_416(size_t resourceLength)
{
auto response = Response::build();
// [FIXME] (compile with recent enough version of libmicrohttpd)
// response->set_code(MHD_HTTP_RANGE_NOT_SATISFIABLE);
response->set_code(416);
response->set_code(MHD_HTTP_RANGE_NOT_SATISFIABLE);
std::ostringstream oss;
oss << "bytes */" << resourceLength;
response->add_header(MHD_HTTP_HEADER_CONTENT_RANGE, oss.str());

View File

@@ -117,6 +117,10 @@ std::map<std::string, IpAddress> getNetworkInterfacesWin() {
// Successively allocate the required memory until GetAdaptersAddresses does not
// results in ERROR_BUFFER_OVERFLOW for a maximum of max_tries
do{
if (interfacesHead) {
free(interfacesHead);
interfacesHead = NULL;
}
interfacesHead = (IP_ADAPTER_ADDRESSES *) malloc(outBufLen);
if (interfacesHead == NULL) {
std::cerr << "Memory allocation failed for IP_ADAPTER_ADDRESSES struct" << std::endl;
@@ -124,6 +128,7 @@ std::map<std::string, IpAddress> getNetworkInterfacesWin() {
}
dwRetVal = GetAdaptersAddresses(family, flags, NULL, interfacesHead, &outBufLen);
Iterations++;
} while ((dwRetVal == ERROR_BUFFER_OVERFLOW) && (Iterations < max_tries));
if (dwRetVal == NO_ERROR) {
@@ -230,7 +235,7 @@ IpAddress getBestPublicIps() {
}
}
#endif
const char* const v4prefixes[] = { "192.168", "172.16", "10.0" };
const char* const v4prefixes[] = { "192.168", "172.16", "10.211", "10.0" };
for (const auto& prefix : v4prefixes) {
for (const auto& kv : interfaces) {
const auto& interfaceIps = kv.second;

View File

@@ -29,7 +29,8 @@
#include <unicode/uniset.h>
#include <unicode/ustring.h>
#include <algorithm>
#include <cctype>
#include <iostream>
#include <iomanip>
#include <regex>
@@ -73,11 +74,10 @@ std::string kiwix::removeAccents(const std::string& text)
loadICUExternalTables();
ucnv_setDefaultName("UTF-8");
UErrorCode status = U_ZERO_ERROR;
auto removeAccentsTrans = icu::Transliterator::createInstance(
"Lower; NFD; [:M:] remove; NFC", UTRANS_FORWARD, status);
std::unique_ptr<icu::Transliterator> removeAccentsTrans(icu::Transliterator::createInstance(
"Lower; NFD; [:M:] remove; NFC", UTRANS_FORWARD, status));
icu::UnicodeString ustring(text.c_str());
removeAccentsTrans->transliterate(ustring);
delete removeAccentsTrans;
std::string unaccentedText;
ustring.toUTF8String(unaccentedText);
return unaccentedText;
@@ -450,3 +450,11 @@ std::string kiwix::getSlugifiedFileName(const std::string& filename)
#endif
return std::regex_replace(filename, reservedCharsReg, "_");
}
std::string kiwix::trim(const std::string& s)
{
auto is_space = [](unsigned char c) { return std::isspace(c); };
auto start = std::find_if_not(s.begin(), s.end(), is_space);
auto end = std::find_if_not(s.rbegin(), s.rend(), is_space).base();
return (start < end) ? std::string(start, end) : std::string();
}

View File

@@ -60,6 +60,8 @@ std::string escapeForJSON(const std::string& s, bool escapeQuote = true);
std::string urlEncode(const std::string& value);
std::string urlDecode(const std::string& value, bool component = false);
std::string trim(const std::string& s);
std::string join(const std::vector<std::string>& list, const std::string& sep);
std::string ucAll(const std::string& word);

View File

@@ -10,7 +10,7 @@
"no-such-book": "Necun tal libro: {{BOOK_NAME}}",
"too-many-books": "Troppo de libros demandate ({{NB_BOOKS}}); le limite es {{LIMIT}}",
"no-book-found": "Necun libro corresponde al criterios de selection",
"url-not-found": "Le URL reuqestate \"{{url}}\" non ha essite trovate sur iste servitor.",
"url-not-found": "Le URL requestate {{url}} non ha essite trovate sur iste servitor.",
"suggest-search": "Facer un recerca in texto complete de <a href=\"{{{SEARCH_URL}}}\">{{PATTERN}}</a>",
"random-article-failure": "Ups! Non poteva eliger un articulo aleatori :(",
"invalid-raw-data-type": "{{DATATYPE}} non es un requesta valide pro contento brute.",

View File

@@ -1,15 +1,17 @@
{
"@metadata": {
"authors": [
"Akmaie Ajam"
"Akmaie Ajam",
"Bennylin",
"Penyuwangi"
]
},
"name": "Bahasa Inggris",
"suggest-full-text-search": "mengandung '{{{SEARCH_TERMS}}}'...",
"no-such-book": "Tidak ada buku seperti ini: {{BOOK_NAME}}",
"too-many-books": "Terlalu banyak buku yang diminta ({{NB_BOOKS}}) dimana batasnya adalah {{LIMIT}}",
"too-many-books": "Terlalu banyak buku yang diminta ({{NB_BOOKS}}), batasnya adalah {{LIMIT}}",
"no-book-found": "Tidak ada buku yang sesuai kriteria yang dipilih",
"url-not-found": "URL yang diminta \"{{url}}\" tidak ditemukan di server ini.",
"url-not-found": "URL yang diminta \"{{url}}\" tidak ditemukan di peladen ini.",
"suggest-search": "Lakukan pencarian teks lengkap untuk <a href=\"{{{SEARCH_URL}}}\">{{PATTERN}}</a>",
"random-article-failure": "Waduh! Gagal memilih artikel acak :(",
"invalid-raw-data-type": "{{DATATYPE}} bukan permintaan yang sah untuk konten mentah.",
@@ -21,9 +23,9 @@
"400-page-heading": "Permintaan tidak sah",
"404-page-title": "Konten tidak ditemukan",
"404-page-heading": "Tidak Ditemukan",
"500-page-title": "Kesalahan Server Internal",
"500-page-heading": "Kesalahan Server Internal",
"500-page-text": "Terjadi kesalahan server internal. Kami mohon maaf atas hal ini :/",
"500-page-title": "Galat Peladen Internal",
"500-page-heading": "Aduh. Halaman tidak bekerja.",
"500-page-text": "Jalur yang diminta tidak dapat diantar dengan benar:",
"fulltext-search-unavailable": "Pencarian teks lengkap tidak tersedia",
"no-search-results": "Mesin pencari teks lengkap tidak tersedia untuk konten ini.",
"search-results-page-title": "Pencarian: {{SEARCH_PATTERN}}",
@@ -52,10 +54,10 @@
"torrent-download-link-text": "BitTorrent",
"torrent-download-alt-text": "Unduh melalui BitTorrent",
"library-opds-feed-all-entries": "Umpan OPDS Perpustakaan - Semua entri",
"filter-by-tag": "Saring berdasarkan tag \"{{{TAG}}}\"",
"stop-filtering-by-tag": "Berhenti penyaringan berdasarkan tag \"{{{TAG}}}\"",
"library-opds-feed-parameterised": "Umpan OPDS Perpustakaan - entri yang cocok dengan {{#LANG}}\nBahasa: {{LANG}} {{/LANG}}{{#CATEGORY}}\nKategori: {{CATEGORY}} {{/CATEGORY}}{{#TAG}}\nTag: {{TAG}} {{/TAG}}{{#Q}}\nKueri: {{Q}} {{/Q}}",
"welcome-to-kiwix-server": "Selamat datang di Server Kiwix",
"filter-by-tag": "Saring berdasarkan tanda \"{{{TAG}}}\"",
"stop-filtering-by-tag": "Hentikan penyaringan berdasarkan tanda \"{{{TAG}}}\"",
"library-opds-feed-parameterised": "Umpan OPDS Perpustakaan - entri yang cocok dengan {{#LANG}}\nBahasa: {{LANG}} {{/LANG}}{{#CATEGORY}}\nKategori: {{CATEGORY}} {{/CATEGORY}}{{#TAG}}\nTanda: {{TAG}} {{/TAG}}{{#Q}}\nKueri: {{Q}} {{/Q}}",
"welcome-to-kiwix-server": "Selamat datang di Peladen Kiwix",
"download-links-heading": "Tautan unduhan untuk <b><i>{{BOOK_TITLE}}</i></b>",
"download-links-title": "Unduh buku",
"preview-book": "Pratayang",

View File

@@ -67,7 +67,7 @@
"direct-download-alt-text": "Direct downloaden via HTTP(S)",
"hash-download-link-text": "SHA-256-controlesom",
"hash-download-alt-text": "De SHA-256-controlesom van het bestand weergeven",
"magnet-link-text": "Magnet-link",
"magnet-link-text": "Magnet-koppeling",
"magnet-alt-text": "Downloaden via Magnet-link",
"torrent-download-link-text": "BitTorrent",
"torrent-download-alt-text": "Downloaden via BitTorrent",

View File

@@ -6,6 +6,7 @@
"Lutece398",
"Okras",
"Pacha Tchernof",
"Putnik",
"Razno0",
"Rofiatmustapha12",
"Smavrina"
@@ -59,7 +60,7 @@
"torrent-download-link-text": "BitTorrent",
"torrent-download-alt-text": "Скачать через BitTorrent",
"library-opds-feed-all-entries": "Канал библиотеки OPDS  все записи",
"filter-by-tag": "Фильтровать по тегу \"{{{TAG}}}\"",
"filter-by-tag": "Фильтровать по тегу «{{{TAG}}}»",
"stop-filtering-by-tag": "Прекратить фильтрацию по тегу \"{{{TAG}}}\"",
"library-opds-feed-parameterised": "Канал OPDS библиотеки – записи, соответствующие {{#LANG}}\nLanguage: {{LANG}} {{/LANG}}{{#CATEGORY}}\nCategory: {{CATEGORY}} {{/CATEGORY}} {{#TAG}}\nTag: {{TAG}} {{/TAG}}{{#Q}}\nЗапрос: {{Q}} {{/Q}}",
"welcome-to-kiwix-server": "Добро пожаловать на сервер Kiwix",

View File

@@ -36,7 +36,7 @@
filteredParams.set(key, value);
}
}
const feedLink = `${root}/catalog/v2/entries?${filteredParams.toString()}`;
const feedLink = `${root}/catalog/v2/entries?count=-1&${filteredParams.toString()}`;
document.querySelector('#headFeedLink').href = feedLink;
document.querySelector('#feedLink').href = feedLink;
setFeedToolTip();

View File

@@ -32,7 +32,7 @@ const uiLanguages = [
{
"iso_code": "el",
"self_name": "Αγγλικά",
"translation_count": 23
"translation_count": 27
},
{
"iso_code": "en",
@@ -41,13 +41,13 @@ const uiLanguages = [
},
{
"iso_code": "es",
"self_name": "español",
"translation_count": 67
"self_name": "Español",
"translation_count": 92
},
{
"iso_code": "fi",
"self_name": "suomi",
"translation_count": 29
"translation_count": 49
},
{
"iso_code": "fr",
@@ -117,12 +117,12 @@ const uiLanguages = [
{
"iso_code": "lb",
"self_name": "Lëtzebuergesch",
"translation_count": 48
"translation_count": 49
},
{
"iso_code": "mk",
"self_name": "македонски",
"translation_count": 81
"translation_count": 92
},
{
"iso_code": "ms",
@@ -137,7 +137,7 @@ const uiLanguages = [
{
"iso_code": "nl",
"self_name": "Nederlands",
"translation_count": 68
"translation_count": 84
},
{
"iso_code": "nqo",
@@ -192,7 +192,7 @@ const uiLanguages = [
{
"iso_code": "sl",
"self_name": "slovenščina",
"translation_count": 57
"translation_count": 84
},
{
"iso_code": "sq",
@@ -202,7 +202,7 @@ const uiLanguages = [
{
"iso_code": "sv",
"self_name": "Svenska",
"translation_count": 67
"translation_count": 84
},
{
"iso_code": "sw",
@@ -222,7 +222,7 @@ const uiLanguages = [
{
"iso_code": "zh-hans",
"self_name": "简体中文",
"translation_count": 68
"translation_count": 84
},
{
"iso_code": "zh-hant",

View File

@@ -83,6 +83,7 @@ function quasiUriEncode(s, specialSymbols) {
function performSearch() {
const searchbox = document.getElementById('kiwixsearchbox');
if (!searchbox.value.trim()) { return;}
const q = encodeURIComponent(searchbox.value);
gotoUrl(`/search?books.name=${currentBook}&pattern=${q}&userlang=${viewerState.uiLanguage}`);
}

View File

@@ -20,7 +20,7 @@
type="application/atom+xml"
title="Library OPDS Feed"
id="headFeedLink"
href="{{root}}/catalog/v2/entries"
href="{{root}}/catalog/v2/entries?count=-1"
/>
<link rel="search" type="application/opensearchdescription+xml" href="{{root}}/catalog/searchdescription.xml" />
<link rel="apple-touch-icon" sizes="180x180" href="{{root}}/skin/favicon/apple-touch-icon.png?KIWIXCACHEID">
@@ -64,7 +64,7 @@
</div>
</noscript>
<div class='kiwixNav'>
<a href="{{root}}/catalog/v2/entries" id="feedLink">
<a href="{{root}}/catalog/v2/entries?count=-1" id="feedLink">
<img src="{{root}}/skin/feed.svg?KIWIXCACHEID"
class="feedLogo"
id="feedLogo"

View File

@@ -2,6 +2,9 @@
#include "../include/library.h"
#include "../include/manager.h"
#include "testing_tools.h"
using namespace kiwix::testing;
#include "gtest/gtest.h"
namespace
@@ -58,27 +61,6 @@ class NameMapperTest : public ::testing::Test {
std::shared_ptr<kiwix::Library> lib;
};
class CapturedStderr
{
std::ostringstream buffer;
std::streambuf* const sbuf;
public:
CapturedStderr()
: sbuf(std::cerr.rdbuf())
{
std::cerr.rdbuf(buffer.rdbuf());
}
CapturedStderr(const CapturedStderr&) = delete;
~CapturedStderr()
{
std::cerr.rdbuf(sbuf);
}
operator std::string() const { return buffer.str(); }
};
#if _WIN32
const std::string ZERO_FOUR_NAME_CONFLICT_MSG =
"Path collision: 'C:\\data\\zero_four_2021-10.zim' and"

View File

@@ -8,6 +8,8 @@
#include "../src/tools/stringTools.h"
#include "testing_tools.h"
using namespace kiwix::testing;
const std::string ROOT_PREFIX("/ROOT%23%3F");
@@ -65,7 +67,7 @@ const ResourceCollection resources200Compressible{
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/index.css" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/index.css?cacheid=ae79e41a" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/index.js" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/index.js?cacheid=4e232c58" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/index.js?cacheid=e3305ca0" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/iso6391To3.js" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/iso6391To3.js?cacheid=ecde2bb3" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/isotope.pkgd.min.js" },
@@ -77,7 +79,7 @@ const ResourceCollection resources200Compressible{
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/taskbar.css" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/taskbar.css?cacheid=42e90cb9" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/viewer.js" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/viewer.js?cacheid=00e0fdf3" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/viewer.js?cacheid=6192cae1" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/fonts/Poppins.ttf" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/fonts/Poppins.ttf?cacheid=af705837" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/fonts/Roboto.ttf" },
@@ -86,7 +88,7 @@ const ResourceCollection resources200Compressible{
// TODO: implement cache management of i18n resources
//{ STATIC_CONTENT, "/ROOT%23%3F/skin/i18n/test.json?cacheid=unknown" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/skin/languages.js" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/languages.js?cacheid=08955948" },
{ STATIC_CONTENT, "/ROOT%23%3F/skin/languages.js?cacheid=d2d6933b" },
{ DYNAMIC_CONTENT, "/ROOT%23%3F/catalog/search" },
@@ -301,10 +303,10 @@ R"EXPECTEDRESULT( href="/ROOT%23%3F/skin/kiwix.css?cacheid=b4e29e64"
<meta name="msapplication-config" content="/ROOT%23%3F/skin/favicon/browserconfig.xml?cacheid=f29a7c4a">
<script type="text/javascript" src="./skin/polyfills.js?cacheid=a0e0343d"></script>
<script type="module" src="/ROOT%23%3F/skin/i18n.js?cacheid=e9a10ac1" defer></script>
<script type="text/javascript" src="/ROOT%23%3F/skin/languages.js?cacheid=08955948" defer></script>
<script type="text/javascript" src="/ROOT%23%3F/skin/languages.js?cacheid=d2d6933b" defer></script>
<script src="/ROOT%23%3F/skin/isotope.pkgd.min.js?cacheid=2e48d392" defer></script>
<script src="/ROOT%23%3F/skin/iso6391To3.js?cacheid=ecde2bb3"></script>
<script type="text/javascript" src="/ROOT%23%3F/skin/index.js?cacheid=4e232c58" defer></script>
<script type="text/javascript" src="/ROOT%23%3F/skin/index.js?cacheid=e3305ca0" defer></script>
<img src="/ROOT%23%3F/skin/feed.svg?cacheid=055b333f"
<img src="/ROOT%23%3F/skin/langSelector.svg?cacheid=00b59961"
)EXPECTEDRESULT"
@@ -337,8 +339,8 @@ R"EXPECTEDRESULT( <link type="text/css" href="./skin/kiwix.css?cacheid=b4e29e
<link type="text/css" href="./skin/print.css?cacheid=65b1c1d2" media="print" rel="Stylesheet" />
<script type="text/javascript" src="./skin/polyfills.js?cacheid=a0e0343d"></script>
<script type="module" src="./skin/i18n.js?cacheid=e9a10ac1" defer></script>
<script type="text/javascript" src="./skin/languages.js?cacheid=08955948" defer></script>
<script type="text/javascript" src="./skin/viewer.js?cacheid=00e0fdf3" defer></script>
<script type="text/javascript" src="./skin/languages.js?cacheid=d2d6933b" defer></script>
<script type="text/javascript" src="./skin/viewer.js?cacheid=6192cae1" defer></script>
<script type="text/javascript" src="./skin/autoComplete/autoComplete.min.js?cacheid=1191aaaf"></script>
const blankPageUrl = root + "/skin/blank.html?cacheid=6b1fa032";
<label for="kiwix_button_show_toggle"><img src="./skin/caret.png?cacheid=22b942b4" alt=""></label>
@@ -438,7 +440,6 @@ TEST_F(ServerTest, CacheIdsOfStaticResourcesMatchTheSha1HashOfResourceContent)
const char* urls400[] = {
"/ROOT%23%3F/search",
"/ROOT%23%3F/search?content=zimfile",
"/ROOT%23%3F/search?content=non-existing-book&pattern=asdfqwerty",
"/ROOT%23%3F/search?content=non-existing-book&pattern=asd<qwerty",
"/ROOT%23%3F/search?books.name=non-exsitent-book&pattern=asd<qwerty",
@@ -1052,17 +1053,6 @@ TEST_F(ServerTest, Http400HtmlError)
<p>
Too many books requested (4) where limit is 3
</p>
)" },
{ /* url */ "/ROOT%23%3F/search?content=zimfile",
expected_kiwix_response_data==R"({ "CSS_URL" : false, "PAGE_HEADING" : { "msgid" : "400-page-heading", "params" : { } }, "PAGE_TITLE" : { "msgid" : "400-page-title", "params" : { } }, "details" : [ { "p" : { "msgid" : "invalid-request", "params" : { "url" : "/ROOT%23%3F/search?content=zimfile" } } }, { "p" : { "msgid" : "no-query", "params" : { } } } ] })" &&
expected_body==R"(
<h1>Invalid request</h1>
<p>
The requested URL "/ROOT%23%3F/search?content=zimfile" is not a valid request.
</p>
<p>
No query provided.
</p>
)" },
{ /* url */ "/ROOT%23%3F/search?content=non-existing-book&pattern=asdfqwerty",
expected_kiwix_response_data==R"({ "CSS_URL" : false, "PAGE_HEADING" : { "msgid" : "400-page-heading", "params" : { } }, "PAGE_TITLE" : { "msgid" : "400-page-title", "params" : { } }, "details" : [ { "p" : { "msgid" : "invalid-request", "params" : { "url" : "/ROOT%23%3F/search?content=non-existing-book&pattern=asdfqwerty" } } }, { "p" : { "msgid" : "no-such-book", "params" : { "BOOK_NAME" : "non-existing-book" } } } ] })" &&
@@ -1085,19 +1075,6 @@ TEST_F(ServerTest, Http400HtmlError)
<p>
No such book: non-existing-book
</p>
)" },
// There is a flaw in our way to handle query string, we cannot differenciate
// between `pattern` and `pattern=`
{ /* url */ "/ROOT%23%3F/search?books.filter.lang=eng&pattern",
expected_kiwix_response_data==R"({ "CSS_URL" : false, "PAGE_HEADING" : { "msgid" : "400-page-heading", "params" : { } }, "PAGE_TITLE" : { "msgid" : "400-page-title", "params" : { } }, "details" : [ { "p" : { "msgid" : "invalid-request", "params" : { "url" : "/ROOT%23%3F/search?books.filter.lang=eng&pattern" } } }, { "p" : { "msgid" : "no-query", "params" : { } } } ] })" &&
expected_body==R"(
<h1>Invalid request</h1>
<p>
The requested URL "/ROOT%23%3F/search?books.filter.lang=eng&pattern" is not a valid request.
</p>
<p>
No query provided.
</p>
)" },
{ /* url */ "/ROOT%23%3F/search?pattern=foo",
expected_kiwix_response_data==R"({ "CSS_URL" : false, "PAGE_HEADING" : { "msgid" : "400-page-heading", "params" : { } }, "PAGE_TITLE" : { "msgid" : "400-page-title", "params" : { } }, "details" : [ { "p" : { "msgid" : "invalid-request", "params" : { "url" : "/ROOT%23%3F/search?pattern=foo" } } }, { "p" : { "msgid" : "too-many-books", "params" : { "LIMIT" : "3", "NB_BOOKS" : "4" } } } ] })" &&
@@ -1110,20 +1087,6 @@ TEST_F(ServerTest, Http400HtmlError)
Too many books requested (4) where limit is 3
</p>
)" },
// Testing of translation
{ /* url */ "/ROOT%23%3F/search?content=zimfile&userlang=test",
expected_page_title=="[I18N TESTING] Invalid request ($400 fine must be paid)" &&
expected_kiwix_response_data==R"({ "CSS_URL" : false, "PAGE_HEADING" : { "msgid" : "400-page-heading", "params" : { } }, "PAGE_TITLE" : { "msgid" : "400-page-title", "params" : { } }, "details" : [ { "p" : { "msgid" : "invalid-request", "params" : { "url" : "/ROOT%23%3F/search?content=zimfile&userlang=test" } } }, { "p" : { "msgid" : "no-query", "params" : { } } } ] })" &&
expected_body==R"(
<h1>[I18N TESTING] -400 karma for an invalid request</h1>
<p>
[I18N TESTING] Invalid URL: "/ROOT%23%3F/search?content=zimfile&userlang=test"
</p>
<p>
[I18N TESTING] Kiwix can read your thoughts but it is against GDPR. Please provide your query explicitly.
</p>
)" },
};
for ( const auto& t : testData ) {
@@ -1154,20 +1117,6 @@ TEST_F(ServerTest, HttpXmlError)
};
const std::vector<TestData> testData{
{ /* url */ "/ROOT%23%3F/search?format=xml",
/* HTTP status code */ 400,
/* expected response XML */ R"(
<error>Invalid request</error>
<detail>The requested URL "/ROOT%23%3F/search?format=xml" is not a valid request.</detail>
<detail>Too many books requested (4) where limit is 3</detail>
)" },
{ /* url */ "/ROOT%23%3F/search?format=xml&content=zimfile",
/* HTTP status code */ 400,
/* expected response XML */ R"(
<error>Invalid request</error>
<detail>The requested URL "/ROOT%23%3F/search?format=xml&content=zimfile" is not a valid request.</detail>
<detail>No query provided.</detail>
)" },
{ /* url */ "/ROOT%23%3F/search?format=xml&content=non-existing-book&pattern=asdfqwerty",
/* HTTP status code */ 400,
/* expected response XML */ R"(
@@ -1181,15 +1130,6 @@ TEST_F(ServerTest, HttpXmlError)
<error>Invalid request</error>
<detail>The requested URL "/ROOT%23%3F/search?format=xml&content=non-existing-book&pattern=a%22%3Cscript%20foo%3E" is not a valid request.</detail>
<detail>No such book: non-existing-book</detail>
)" },
// There is a flaw in our way to handle query string, we cannot differenciate
// between `pattern` and `pattern=`
{ /* url */ "/ROOT%23%3F/search?format=xml&books.filter.lang=eng&pattern",
/* HTTP status code */ 400,
/* expected response XML */ R"(
<error>Invalid request</error>
<detail>The requested URL "/ROOT%23%3F/search?format=xml&books.filter.lang=eng&pattern" is not a valid request.</detail>
<detail>No query provided.</detail>
)" },
{ /* url */ "/ROOT%23%3F/search?format=xml&pattern=foo",
/* HTTP status code */ 400,
@@ -1346,7 +1286,7 @@ R"EXPECTEDRESPONSE(const uiLanguages = [
{
"iso_code": "el",
"self_name": "Αγγλικά",
"translation_count": 23
"translation_count": 27
},
{
"iso_code": "en",
@@ -1355,13 +1295,13 @@ R"EXPECTEDRESPONSE(const uiLanguages = [
},
{
"iso_code": "es",
"self_name": "español",
"translation_count": 67
"self_name": "Español",
"translation_count": 92
},
{
"iso_code": "fi",
"self_name": "suomi",
"translation_count": 29
"translation_count": 49
},
{
"iso_code": "fr",
@@ -1431,12 +1371,12 @@ R"EXPECTEDRESPONSE(const uiLanguages = [
{
"iso_code": "lb",
"self_name": "Lëtzebuergesch",
"translation_count": 48
"translation_count": 49
},
{
"iso_code": "mk",
"self_name": "македонски",
"translation_count": 81
"translation_count": 92
},
{
"iso_code": "ms",
@@ -1451,7 +1391,7 @@ R"EXPECTEDRESPONSE(const uiLanguages = [
{
"iso_code": "nl",
"self_name": "Nederlands",
"translation_count": 68
"translation_count": 84
},
{
"iso_code": "nqo",
@@ -1506,7 +1446,7 @@ R"EXPECTEDRESPONSE(const uiLanguages = [
{
"iso_code": "sl",
"self_name": "slovenščina",
"translation_count": 57
"translation_count": 84
},
{
"iso_code": "sq",
@@ -1516,7 +1456,7 @@ R"EXPECTEDRESPONSE(const uiLanguages = [
{
"iso_code": "sv",
"self_name": "Svenska",
"translation_count": 67
"translation_count": 84
},
{
"iso_code": "sw",
@@ -1536,7 +1476,7 @@ R"EXPECTEDRESPONSE(const uiLanguages = [
{
"iso_code": "zh-hans",
"self_name": "简体中文",
"translation_count": 68
"translation_count": 84
},
{
"iso_code": "zh-hant",
@@ -2363,3 +2303,79 @@ R"(const viewerSettings = {
)");
}
}
TEST_F(ServerTest, EmptyPatternSearchDoesNotError)
{
EXPECT_EQ(200, zfs1_->GET("/ROOT%23%3F/search?content=zimfile")->status);
}
#define EXPECT_ERROR(MSG, SERVER_SETUP_CODE) \
{ \
kiwix::Server server(kiwix::Library::create()); \
CapturedStderr stderror; \
SERVER_SETUP_CODE; \
EXPECT_FALSE(server.start()); \
EXPECT_EQ(std::string(stderror), std::string("ERROR: ") + MSG + "\n"); \
}
TEST(ServerNegativeTest, IpAddressAndIpModeAreMutuallyExclusive)
{
EXPECT_ERROR("When an IP address is provided the IP mode must not be set",
server.setAddress("127.0.0.1");
server.setIpMode(kiwix::IpMode::IPV4);
);
EXPECT_ERROR("When an IP address is provided the IP mode must not be set",
server.setAddress("[::1]");
server.setIpMode(kiwix::IpMode::IPV6);
);
EXPECT_ERROR("When an IP address is provided the IP mode must not be set",
server.setAddress("localhost");
server.setIpMode(kiwix::IpMode::ALL);
);
}
TEST(ServerNegativeTest, InvalidIpAddressDetection)
{
EXPECT_ERROR("invalid IP address: 1.2.3",
server.setAddress("1.2.3");
);
EXPECT_ERROR("invalid IP address: 127.0.0.256",
server.setAddress("127.0.0.256");
);
EXPECT_ERROR("invalid IP address: localhost",
server.setAddress("localhost");
);
EXPECT_ERROR("invalid IP address: fe80::94d2:16e7:5f3e:89bx",
server.setAddress("[fe80::94d2:16e7:5f3e:89bx]");
);
// We assume that our unit tests won't be run on Google's DNS server
EXPECT_ERROR("IP address is not available on this system: 8.8.8.8",
server.setAddress("8.8.8.8");
);
// According to the spec, IPv6 addresses 2001:db8::/32 are reserved
// for documentation and example source code
EXPECT_ERROR("IP address is not available on this system: 2001:db8::",
server.setAddress("[2001:db8::]");
);
}
TEST(ServerNegativeTest, UnusablePort)
{
// Occupy port 8910
httplib::Server portOccupant;
ASSERT_TRUE(portOccupant.bind_to_port("127.0.0.1", 8910));
// Try to listen on the same port
EXPECT_ERROR("Unable to instantiate the HTTP daemon. The port 8910 is maybe "
"already occupied or need more permissions to be open. Please "
"try as root or with a port number higher or equal to 1024.",
server.setPort(8910);
);
}

View File

@@ -689,6 +689,24 @@ bool isSubSnippet(std::string subSnippet, const std::string& superSnippet)
#define RAYCHARLESZIMID "6f1d19d0-633f-087b-fb55-7ac324ff9baf"
#define EXAMPLEZIMID "5dc0b3af-5df2-0925-f0ca-d2bf75e78af6"
const std::vector<SearchResult> YELLOW_SEARCH_RESULTS = {
SEARCH_RESULT(
/*link*/ "/ROOT%23%3F/content/zimfile/A/Eleanor_Rigby",
/*title*/ "Eleanor Rigby",
/*snippet*/ R"SNIPPET(...-side "<b>Yellow</b> Submarine" (double A-side) Released 5)SNIPPET" "\xC2\xA0" "August" "\xC2\xA0" "1966" "\xC2\xA0" R"SNIPPET((1966-08-05) Format 7-inch single Recorded 2829 April &amp; 6 June 1966 Studio EMI, London Genre Baroque pop, art rock Length 2:08 Label Parlophone (UK), Capitol (US) Songwriter(s) LennonMcCartney Producer(s) George Martin The Beatles singles chronology "Paperback Writer" (1966) "Eleanor Rigby" / "<b>Yellow</b> Submarine" (1966) "Strawberry Fields Forever" / "Penny Lane" (1967) Music video "Eleanor Rigby" on YouTube The song continued the......)SNIPPET",
/*bookTitle*/ "Ray Charles",
/*wordCount*/ "201"
),
SEARCH_RESULT(
/*link*/ "/ROOT%23%3F/content/zimfile/A/If_You_Go_Away",
/*title*/ "If You Go Away",
/*snippet*/ R"SNIPPET(...standard and has been recorded by many artists, including Greta Keller, for whom some say McKuen wrote the lyrics. "If You Go Away" Single by Damita Jo from the album If You Go Away B-side "<b>Yellow</b> Days" Released 1966 Genre Jazz Length 3:49 Label Epic Records Songwriter(s) Jacques Brel, Rod McKuen Producer(s) Bob Morgan Damita Jo singles chronology "Gotta Travel On" (1965) "If You Go Away" (1966) "Walk Away" (1967) Damita Jo reached #10 on the Adult Contemporary chart and #68 on the Billboard Hot 100 in 1966 for her version of the song. Terry Jacks recorded a version of the song which was released as a single in 1974 and reached #29 on the Adult Contemporary chart, #68 on the......)SNIPPET",
/*bookTitle*/ "Ray Charles",
/*wordCount*/ "204"
)
};
struct TestData
{
struct PaginationEntry
@@ -935,6 +953,26 @@ struct TestData
TEST(ServerSearchTest, searchResults)
{
const TestData testData[] = {
{
/* query */ "pattern=&books.id=" RAYCHARLESZIMID,
/* start */ -1,
/* resultsPerPage */ 0,
/* totalResultCount */ 0,
/* firstResultIndex */ 0,
/* results */ {},
/* pagination */ {}
},
{
/* query */ "pattern=%20&books.id=" RAYCHARLESZIMID,
/* start */ -1,
/* resultsPerPage */ 0,
/* totalResultCount */ 0,
/* firstResultIndex */ 0,
/* results */ {},
/* pagination */ {}
},
{
/* query */ "pattern=velomanyunkan&books.id=" RAYCHARLESZIMID,
/* start */ -1,
@@ -980,24 +1018,8 @@ TEST(ServerSearchTest, searchResults)
/* resultsPerPage */ 0,
/* totalResultCount */ 2,
/* firstResultIndex */ 0,
/* results */ {
SEARCH_RESULT(
/*link*/ "/ROOT%23%3F/content/zimfile/A/Eleanor_Rigby",
/*title*/ "Eleanor Rigby",
/*snippet*/ R"SNIPPET(...-side "<b>Yellow</b> Submarine" (double A-side) Released 5 August 1966 (1966-08-05) Format 7-inch single Recorded 2829 April &amp; 6 June 1966 Studio EMI, London Genre Baroque pop, art rock Length 2:08 Label Parlophone (UK), Capitol (US) Songwriter(s) LennonMcCartney Producer(s) George Martin The Beatles singles chronology "Paperback Writer" (1966) "Eleanor Rigby" / "<b>Yellow</b> Submarine" (1966) "Strawberry Fields Forever" / "Penny Lane" (1967) Music video "Eleanor Rigby" on YouTube The song continued the......)SNIPPET",
/*bookTitle*/ "Ray Charles",
/*wordCount*/ "201"
),
SEARCH_RESULT(
/*link*/ "/ROOT%23%3F/content/zimfile/A/If_You_Go_Away",
/*title*/ "If You Go Away",
/*snippet*/ R"SNIPPET(...standard and has been recorded by many artists, including Greta Keller, for whom some say McKuen wrote the lyrics. "If You Go Away" Single by Damita Jo from the album If You Go Away B-side "<b>Yellow</b> Days" Released 1966 Genre Jazz Length 3:49 Label Epic Records Songwriter(s) Jacques Brel, Rod McKuen Producer(s) Bob Morgan Damita Jo singles chronology "Gotta Travel On" (1965) "If You Go Away" (1966) "Walk Away" (1967) Damita Jo reached #10 on the Adult Contemporary chart and #68 on the Billboard Hot 100 in 1966 for her version of the song. Terry Jacks recorded a version of the song which was released as a single in 1974 and reached #29 on the Adult Contemporary chart, #68 on the......)SNIPPET",
/*bookTitle*/ "Ray Charles",
/*wordCount*/ "204"
)
},
/* pagination */ {}
/* results */ YELLOW_SEARCH_RESULTS,
/* pagination */ {}
},
{
@@ -1006,50 +1028,28 @@ TEST(ServerSearchTest, searchResults)
/* resultsPerPage */ 0,
/* totalResultCount */ 2,
/* firstResultIndex */ 0,
/* results */ {
SEARCH_RESULT(
/*link*/ "/ROOT%23%3F/content/zimfile/A/Eleanor_Rigby",
/*title*/ "Eleanor Rigby",
/*snippet*/ R"SNIPPET(...-side "<b>Yellow</b> Submarine" (double A-side) Released 5 August 1966 (1966-08-05) Format 7-inch single Recorded 2829 April &amp; 6 June 1966 Studio EMI, London Genre Baroque pop, art rock Length 2:08 Label Parlophone (UK), Capitol (US) Songwriter(s) LennonMcCartney Producer(s) George Martin The Beatles singles chronology "Paperback Writer" (1966) "Eleanor Rigby" / "<b>Yellow</b> Submarine" (1966) "Strawberry Fields Forever" / "Penny Lane" (1967) Music video "Eleanor Rigby" on YouTube The song continued the......)SNIPPET",
/*bookTitle*/ "Ray Charles",
/*wordCount*/ "201"
),
SEARCH_RESULT(
/*link*/ "/ROOT%23%3F/content/zimfile/A/If_You_Go_Away",
/*title*/ "If You Go Away",
/*snippet*/ R"SNIPPET(...standard and has been recorded by many artists, including Greta Keller, for whom some say McKuen wrote the lyrics. "If You Go Away" Single by Damita Jo from the album If You Go Away B-side "<b>Yellow</b> Days" Released 1966 Genre Jazz Length 3:49 Label Epic Records Songwriter(s) Jacques Brel, Rod McKuen Producer(s) Bob Morgan Damita Jo singles chronology "Gotta Travel On" (1965) "If You Go Away" (1966) "Walk Away" (1967) Damita Jo reached #10 on the Adult Contemporary chart and #68 on the Billboard Hot 100 in 1966 for her version of the song. Terry Jacks recorded a version of the song which was released as a single in 1974 and reached #29 on the Adult Contemporary chart, #68 on the......)SNIPPET",
/*bookTitle*/ "Ray Charles",
/*wordCount*/ "204"
)
},
/* pagination */ {}
/* results */ YELLOW_SEARCH_RESULTS,
/* pagination */ {}
},
{
/* query */ "pattern=yellow&books.id=" RAYCHARLESZIMID,
/* query */ "pattern=%20yellow%20&books.id=" RAYCHARLESZIMID,
/* start */ 0,
/* resultsPerPage */ 0,
/* totalResultCount */ 2,
/* firstResultIndex */ 0,
/* results */ {
SEARCH_RESULT(
/*link*/ "/ROOT%23%3F/content/zimfile/A/Eleanor_Rigby",
/*title*/ "Eleanor Rigby",
/*snippet*/ R"SNIPPET(...-side "<b>Yellow</b> Submarine" (double A-side) Released 5 August 1966 (1966-08-05) Format 7-inch single Recorded 2829 April &amp; 6 June 1966 Studio EMI, London Genre Baroque pop, art rock Length 2:08 Label Parlophone (UK), Capitol (US) Songwriter(s) LennonMcCartney Producer(s) George Martin The Beatles singles chronology "Paperback Writer" (1966) "Eleanor Rigby" / "<b>Yellow</b> Submarine" (1966) "Strawberry Fields Forever" / "Penny Lane" (1967) Music video "Eleanor Rigby" on YouTube The song continued the......)SNIPPET",
/*bookTitle*/ "Ray Charles",
/*wordCount*/ "201"
),
/* results */ YELLOW_SEARCH_RESULTS,
/* pagination */ {}
},
SEARCH_RESULT(
/*link*/ "/ROOT%23%3F/content/zimfile/A/If_You_Go_Away",
/*title*/ "If You Go Away",
/*snippet*/ R"SNIPPET(...standard and has been recorded by many artists, including Greta Keller, for whom some say McKuen wrote the lyrics. "If You Go Away" Single by Damita Jo from the album If You Go Away B-side "<b>Yellow</b> Days" Released 1966 Genre Jazz Length 3:49 Label Epic Records Songwriter(s) Jacques Brel, Rod McKuen Producer(s) Bob Morgan Damita Jo singles chronology "Gotta Travel On" (1965) "If You Go Away" (1966) "Walk Away" (1967) Damita Jo reached #10 on the Adult Contemporary chart and #68 on the Billboard Hot 100 in 1966 for her version of the song. Terry Jacks recorded a version of the song which was released as a single in 1974 and reached #29 on the Adult Contemporary chart, #68 on the......)SNIPPET",
/*bookTitle*/ "Ray Charles",
/*wordCount*/ "204"
)
},
/* pagination */ {}
{
/* query */ "pattern=yellow%20submarine&books.id=" RAYCHARLESZIMID,
/* start */ 0,
/* resultsPerPage */ 0,
/* totalResultCount */ 1,
/* firstResultIndex */ 0,
/* results */ { YELLOW_SEARCH_RESULTS[0] },
/* pagination */ {}
},
{

View File

@@ -184,4 +184,16 @@ TEST(stringTools, getSlugifiedFileName)
#endif
}
TEST(stringTools, Trim)
{
EXPECT_EQ(kiwix::trim(""), "");
EXPECT_EQ(kiwix::trim("abc123"), "abc123");
EXPECT_EQ(kiwix::trim(" abc123"), "abc123");
EXPECT_EQ(kiwix::trim("abc123 "), "abc123");
EXPECT_EQ(kiwix::trim(" abc123 "), "abc123");
EXPECT_EQ(kiwix::trim("abc 123"), "abc 123");
EXPECT_EQ(kiwix::trim(" "), "");
EXPECT_EQ(kiwix::trim("\t abc123 \n"), "abc123");
}
};

57
test/testing_tools.h Normal file
View File

@@ -0,0 +1,57 @@
/*
* Copyright 2026 Veloman Yunkan
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
* MA 02110-1301, USA.
*/
#ifndef LIBKIWIX_TESTING_TOOLS_H
#define LIBKIWIX_TESTING_TOOLS_H
#include <sstream>
#include <iostream>
namespace kiwix
{
namespace testing
{
class CapturedStderr
{
std::ostringstream buffer;
std::streambuf* const sbuf;
public:
CapturedStderr()
: sbuf(std::cerr.rdbuf())
{
std::cerr.rdbuf(buffer.rdbuf());
}
CapturedStderr(const CapturedStderr&) = delete;
~CapturedStderr()
{
std::cerr.rdbuf(sbuf);
}
operator std::string() const { return buffer.str(); }
};
} // namespace testing
} // namespace kiwix
#endif // LIBKIWIX_TESTING_TOOLS_H