Compare commits

..

37 Commits

Author SHA1 Message Date
Matthieu Gautier
d1124704f9 Make kiwix::Server a simple wrapper around kiwix::InternalServer.
`kiwix::Server` is now a simple exposition of a public API on top of the
private `InternalServer`.
2022-10-18 17:00:11 +02:00
Matthieu Gautier
1fcc2ad709 Add a Server::isRunning to know if the server is running or not. 2022-10-18 17:00:11 +02:00
Matthieu Gautier
7d83127d58 Change the default NameMappe to use a HumanReadableNameMapper 2022-10-18 17:00:11 +02:00
Matthieu Gautier
37ccfb54ed Make fullEntryXML use the configuration to get the bookName. 2022-10-18 17:00:11 +02:00
Matthieu Gautier
6815a4c6a9 Make the OPDSDumper use the ServerConfiguration. 2022-10-18 17:00:11 +02:00
Matthieu Gautier
3e54e56291 [NEW] Make InternalServer inherite Server::Configuration.
This avoid a lot of silly `m_configuration.foo`.
2022-10-18 17:00:11 +02:00
Matthieu Gautier
422e71a017 Make InternalServer use the NameMapper of the configuration.
This move the defaulting to IdNameMapper in the configuration instead of
in the InternalServer.

This also create a default shared_ptr per Configuration instead of
using a static default one.
2022-10-18 17:00:11 +02:00
Matthieu Gautier
92c0e145d4 Make internalServer use m_root from configuration instead of using its own. 2022-10-18 17:00:11 +02:00
Matthieu Gautier
3d75f24a29 Do not store raw pointer to NameMapper.
While it was "ok" to store raw pointer as, in our use case, the nameMapper
always live longer than object using it; it is not safe to store
raw pointer on object than may be deleted.

All classes storing a NameMapper now store a shared_ptr.
Functions only using the library (as `HumanReadableNameMapper`) continue
to use a (const) reference.
2022-10-18 17:00:11 +02:00
Matthieu Gautier
4842fa0355 fixup! Do not store raw pointer to Library. 2022-10-18 17:00:11 +02:00
Matthieu Gautier
0bd5a5ec3b Do not store raw pointer to Library.
While it was "ok" to store raw pointer as, in our use case, the library
always live longer than object using it; it is not safe to store
raw pointer on object than may be deleted.

All classes storing a library now store a shared_ptr.
Functionr only using the library (as `HumanReadableNameMapper`) continue
to use a (const) reference.
2022-10-18 17:00:11 +02:00
Matthieu Gautier
5896691b31 fixup! Introduce a ServerConfiguration object.
Move from `ServerConfiguration` to `Server::Configuration`.
2022-10-18 17:00:11 +02:00
Matthieu Gautier
85f58b8e01 Introduce a ServerConfiguration object.
It is used to store the server configuration instead of passing (a lot of)
arguments to functions/creators.

Please note that this remove the thread protected on m_verbose.
m_verbose is initialized once and never modified, be don't need to protect
access.
2022-10-18 17:00:11 +02:00
Matthieu Gautier
24472e03dd fixup! Make the opds_dumper respect the provided nameMapper used in the server. 2022-10-18 17:00:11 +02:00
Matthieu Gautier
96fb4236a6 Make the opds_dumper respect the provided nameMapper used in the server.
Fix #828
2022-10-18 17:00:11 +02:00
Matthieu Gautier
cc849b31da [New] Use a macro to define catalog's entries in test. 2022-10-18 17:00:11 +02:00
Matthieu Gautier
7feef320d9 Merge pull request #834 from kiwix/concurrency_safe_suggestion_endpoint 2022-10-18 17:00:02 +02:00
Veloman Yunkan
73191fb8f8 Made the /suggest endpoint concurrency-safe 2022-10-13 13:39:25 +04:00
Matthieu Gautier
a844bc4000 Merge pull request #829 from kiwix/opds_dumper_cleanup
OPDS dumper cleanup
2022-10-06 14:11:47 +02:00
Veloman Yunkan
f13ca55ef6 Eliminated the endpointRoot parameter 2022-10-06 14:02:50 +04:00
Veloman Yunkan
dc194683bb Split XML generation code for full & partial entries 2022-10-06 13:48:58 +04:00
Veloman Yunkan
0841472004 Separate templates for full & partial OPDS entries 2022-10-06 13:44:39 +04:00
Veloman Yunkan
ebb713cb85 Got rid of an unjustified parameter
The XML header is injected in a more straightforward way in the single
location where it is needed.
2022-10-06 12:49:51 +04:00
Matthieu Gautier
cd6cbe3655 Merge pull request #827 from kiwix/http_redirect_new_logic
New logic for generating HTTP-redirects
2022-10-04 16:04:20 +02:00
Veloman Yunkan
582c8d868a New logic for generating HTTP-redirects
Before this fix the root URL for a book was assumed to resolve to the
main page.  This was not true for ZIM files containing an entry at an
empty path or with a path equal to "/", resulting in issue #826. The
logic behind this behaviour is found in `kiwix::getEntryFromPath()`.

The fix to that issue is a little more general and will result in an
HTTP redirect in any case where `kiwix::getEntryFromPath(zim, path)`
returns an entry with a real path different from the requested one. In
particular, this will affect the behaviour on ZIM files with the old
namespace scheme, where the requested resource - if not found - is also
looked up in the 'A', 'I', 'J', and/or '-' namespaces. Now instead of
returning the contents of that other resource an HTTP redirect response
will be sent.
2022-10-04 14:18:08 +04:00
Kelson
f6ae75e41d Merge pull request #822 from kiwix/update-format-code-script
Update format_code.sh script
2022-10-03 16:20:32 +02:00
Emmanuel Engelhart
ffbda34b75 Fix: improvement to handle dirs with spaces 2022-10-01 21:15:50 +02:00
Emmanuel Engelhart
f61fc07121 Fix: autodetect proper directories to format 2022-09-29 20:37:20 +02:00
Emmanuel Engelhart
de7fa771fc More generic format_code.sh script 2022-09-29 20:23:15 +02:00
Emmanuel Engelhart
24c1ca5a4a Move format_code.sh to script/ folder 2022-09-29 20:23:15 +02:00
Matthieu Gautier
15f5abad3c Merge pull request #821 from kiwix/taskbar_autohiding
Auto-hiding of the iframe-based taskbar
2022-09-28 17:15:59 +02:00
Veloman Yunkan
0a866fa914 Fixed auto-hiding of the toolbar 2022-09-28 17:00:00 +02:00
Veloman Yunkan
ff192cba49 Fixed a misused setInterval()
In the commit that introduced `setInterval()` in `setupViewer()`
actually `setTimeout()` was intended.
2022-09-28 17:00:00 +02:00
Matthieu Gautier
0dd638f261 Merge pull request #825 from kiwix/no_impish
PPA: Drop impish
2022-09-28 16:59:26 +02:00
Kunal Mehta
229c0ceaf9 PPA: Drop impish 2022-09-28 16:31:33 +02:00
Matthieu Gautier
70f7be4202 Merge pull request #823 from kiwix/kiwix-desktop-friendly-search-results
Fixed search results for kiwix-desktop
2022-09-28 15:40:02 +02:00
Veloman Yunkan
60148717e1 Fixed search results for kiwix-desktop 2022-09-26 13:11:25 +04:00
33 changed files with 493 additions and 477 deletions

View File

@@ -9,7 +9,6 @@ jobs:
matrix:
distro:
- ubuntu-jammy
- ubuntu-impish
- ubuntu-focal
- ubuntu-bionic
steps:
@@ -42,14 +41,6 @@ jobs:
args: --no-sign
ppa: ${{ steps.ppa.outputs.ppa }}
- uses: legoktm/gh-action-build-deb@ubuntu-impish
if: matrix.distro == 'ubuntu-impish'
name: Build package for ubuntu-impish
id: build-ubuntu-impish
with:
args: --no-sign
ppa: ${{ steps.ppa.outputs.ppa }}
- uses: legoktm/gh-action-build-deb@ubuntu-focal
if: matrix.distro == 'ubuntu-focal'
name: Build package for ubuntu-focal

View File

@@ -1,38 +0,0 @@
#!/usr/bin/bash
files=(
"include/library.h"
"include/common/stringTools.h"
"include/common/pathTools.h"
"include/common/otherTools.h"
"include/common/regexTools.h"
"include/common/networkTools.h"
"include/common/archiveTools.h"
"include/manager.h"
"include/reader.h"
"include/kiwix.h"
"include/xapianSearcher.h"
"include/searcher.h"
"src/library.cpp"
"src/android/kiwix.cpp"
"src/android/org/kiwix/kiwixlib/JNIKiwixBool.java"
"src/android/org/kiwix/kiwixlib/JNIKiwix.java"
"src/android/org/kiwix/kiwixlib/JNIKiwixString.java"
"src/android/org/kiwix/kiwixlib/JNIKiwixInt.java"
"src/searcher.cpp"
"src/common/pathTools.cpp"
"src/common/regexTools.cpp"
"src/common/otherTools.cpp"
"src/common/archiveTools.cpp"
"src/common/networkTools.cpp"
"src/common/stringTools.cpp"
"src/xapianSearcher.cpp"
"src/manager.cpp"
"src/reader.cpp"
)
for i in "${files[@]}"
do
echo $i
clang-format -i -style=file $i
done

View File

@@ -54,7 +54,6 @@ enum supportedListMode {
class Filter {
public: // types
using Tags = std::vector<std::string>;
using AliasNames = std::vector<std::string>;
private: // data
uint64_t activeFilters;
@@ -68,7 +67,6 @@ class Filter {
std::string _query;
bool _queryIsPartial;
std::string _name;
AliasNames _aliasNames;
public: // functions
Filter();
@@ -114,7 +112,6 @@ class Filter {
Filter& maxSize(size_t size);
Filter& query(std::string query, bool partial=true);
Filter& name(std::string name);
Filter& aliasNames(const AliasNames& aliasNames);
bool hasQuery() const;
const std::string& getQuery() const { return _query; }
@@ -138,8 +135,6 @@ class Filter {
const Tags& getAcceptTags() const { return _acceptTags; }
const Tags& getRejectTags() const { return _rejectTags; }
const AliasNames& getAliasNames() const { return _aliasNames; }
private: // functions
friend class Library;

View File

@@ -37,10 +37,10 @@ namespace kiwix
class LibraryManipulator
{
public: // functions
explicit LibraryManipulator(Library* library);
explicit LibraryManipulator(std::shared_ptr<Library> library);
virtual ~LibraryManipulator();
Library& getLibrary() const { return library; }
Library& getLibrary() const { return *library.get(); }
bool addBookToLibrary(const Book& book);
void addBookmarkToLibrary(const Bookmark& bookmark);
@@ -52,7 +52,7 @@ class LibraryManipulator
virtual void booksWereRemovedFromLibrary();
private: // data
kiwix::Library& library;
std::shared_ptr<kiwix::Library> library;
};
/**
@@ -65,7 +65,7 @@ class Manager
public: // functions
explicit Manager(LibraryManipulator* manipulator);
explicit Manager(Library* library);
explicit Manager(std::shared_ptr<Library> library);
/**
* Read a `library.xml` and add book in the file to the library.

View File

@@ -50,17 +50,16 @@ class HumanReadableNameMapper : public NameMapper {
std::map<std::string, std::string> m_nameToId;
public:
HumanReadableNameMapper(kiwix::Library& library, bool withAlias);
HumanReadableNameMapper(const kiwix::Library& library, bool withAlias);
virtual ~HumanReadableNameMapper() = default;
virtual std::string getNameForId(const std::string& id) const;
virtual std::string getIdForName(const std::string& name) const;
static std::string removeDateFromBookId(const std::string& bookId);
};
class UpdatableNameMapper : public NameMapper {
typedef std::shared_ptr<NameMapper> NameMapperHandle;
public:
UpdatableNameMapper(Library& library, bool withAlias);
UpdatableNameMapper(std::shared_ptr<Library> library, bool withAlias);
virtual std::string getNameForId(const std::string& id) const;
virtual std::string getIdForName(const std::string& name) const;
@@ -72,7 +71,7 @@ class UpdatableNameMapper : public NameMapper {
private:
mutable std::mutex mutex;
Library& library;
std::shared_ptr<Library> library;
NameMapperHandle nameMapper;
const bool withAlias;
};

View File

@@ -23,10 +23,11 @@
#include <time.h>
#include <sstream>
#include <string>
#include <vector>
#include <pugixml.hpp>
#include "library.h"
#include "server.h"
using namespace std;
@@ -40,8 +41,7 @@ namespace kiwix
class OPDSDumper
{
public:
OPDSDumper() = default;
OPDSDumper(Library* library);
OPDSDumper(Server::Configuration configuration);
~OPDSDumper();
/**
@@ -92,13 +92,6 @@ class OPDSDumper
*/
void setLibraryId(const std::string& id) { this->libraryId = id;}
/**
* Set the root location used when generating url.
*
* @param rootLocation the root location to use.
*/
void setRootLocation(const std::string& rootLocation) { this->rootLocation = rootLocation; }
/**
* Set some informations about the search results.
*
@@ -109,9 +102,8 @@ class OPDSDumper
void setOpenSearchInfo(int totalResult, int startIndex, int count);
protected:
kiwix::Library* library;
Server::Configuration m_configuration;
std::string libraryId;
std::string rootLocation;
int m_totalResults;
int m_startIndex;
int m_count;

View File

@@ -46,7 +46,7 @@ class SearchRenderer
* @param start The start offset used for the srs.
* @param estimatedResultCount The estimatedResultCount of the whole search
*/
SearchRenderer(zim::SearchResultSet srs, NameMapper* mapper,
SearchRenderer(zim::SearchResultSet srs, std::shared_ptr<NameMapper> mapper,
unsigned int start, unsigned int estimatedResultCount);
/**
@@ -58,7 +58,7 @@ class SearchRenderer
* @param start The start offset used for the srs.
* @param estimatedResultCount The estimatedResultCount of the whole search
*/
SearchRenderer(zim::SearchResultSet srs, NameMapper* mapper, Library* library,
SearchRenderer(zim::SearchResultSet srs, std::shared_ptr<NameMapper> mapper, std::shared_ptr<Library> library,
unsigned int start, unsigned int estimatedResultCount);
~SearchRenderer();
@@ -106,8 +106,8 @@ class SearchRenderer
protected:
std::string beautifyInteger(const unsigned int number);
zim::SearchResultSet m_srs;
NameMapper* mp_nameMapper;
Library* mp_library;
std::shared_ptr<NameMapper> mp_nameMapper;
std::shared_ptr<Library> mp_library;
std::string searchBookQuery;
std::string searchPattern;
std::string protocolPrefix;

View File

@@ -29,15 +29,81 @@ namespace kiwix
class NameMapper;
class InternalServer;
class Server {
public:
public:
class Configuration {
public:
explicit Configuration(std::shared_ptr<Library> library, std::shared_ptr<NameMapper> nameMapper=nullptr);
Configuration& setRoot(const std::string& root);
Configuration& setAddress(const std::string& addr) {
m_addr = addr;
return *this;
}
Configuration& setPort(int port) {
m_port = port;
return *this;
}
Configuration& setNbThreads(int threads) {
m_nbThreads = threads;
return *this;
}
Configuration& setMultiZimSearchLimit(unsigned int limit) {
m_multizimSearchLimit = limit;
return *this;
}
Configuration& setIpConnectionLimit(int limit) {
m_ipConnectionLimit = limit;
return *this;
}
Configuration& setVerbose(bool verbose) {
m_verbose = verbose;
return *this;
}
Configuration& setIndexTemplateString(const std::string& indexTemplateString) {
m_indexTemplateString = indexTemplateString;
return *this;
}
Configuration& setTaskbar(bool withTaskbar, bool withLibraryButton) {
m_withTaskbar = withTaskbar;
m_withLibraryButton = withLibraryButton;
return *this;
}
Configuration& setBlockExternalLinks(bool blockExternalLinks) {
m_blockExternalLinks = blockExternalLinks;
return *this;
}
std::shared_ptr<Library> mp_library;
std::shared_ptr<NameMapper> mp_nameMapper;
std::string m_root = "";
std::string m_addr = "";
std::string m_indexTemplateString = "";
int m_port = 80;
int m_nbThreads = 1;
unsigned int m_multizimSearchLimit = 0;
bool m_verbose = false;
bool m_withTaskbar = true;
bool m_withLibraryButton = true;
bool m_blockExternalLinks = false;
int m_ipConnectionLimit = 0;
};
/**
* The default constructor.
*
* @param library The library to serve.
*/
Server(Library* library, NameMapper* nameMapper=nullptr);
explicit Server(const Configuration& configuration);
virtual ~Server();
/**
@@ -50,35 +116,22 @@ namespace kiwix
*/
void stop();
void setRoot(const std::string& root);
void setAddress(const std::string& addr) { m_addr = addr; }
void setPort(int port) { m_port = port; }
void setNbThreads(int threads) { m_nbThreads = threads; }
void setMultiZimSearchLimit(unsigned int limit) { m_multizimSearchLimit = limit; }
void setIpConnectionLimit(int limit) { m_ipConnectionLimit = limit; }
void setVerbose(bool verbose) { m_verbose = verbose; }
void setIndexTemplateString(const std::string& indexTemplateString) { m_indexTemplateString = indexTemplateString; }
void setTaskbar(bool withTaskbar, bool withLibraryButton)
{ m_withTaskbar = withTaskbar; m_withLibraryButton = withLibraryButton; }
void setBlockExternalLinks(bool blockExternalLinks)
{ m_blockExternalLinks = blockExternalLinks; }
/**
* Tell if the server is running or not.
*/
bool isRunning();
/**
* Get the port of the server
*/
int getPort();
/**
* Get the ipAddress of the server
*/
std::string getAddress();
protected:
Library* mp_library;
NameMapper* mp_nameMapper;
std::string m_root = "";
std::string m_addr = "";
std::string m_indexTemplateString = "";
int m_port = 80;
int m_nbThreads = 1;
unsigned int m_multizimSearchLimit = 0;
bool m_verbose = false;
bool m_withTaskbar = true;
bool m_withLibraryButton = true;
bool m_blockExternalLinks = false;
int m_ipConnectionLimit = 0;
std::unique_ptr<InternalServer> mp_server;
};
}

14
scripts/format_code.sh Executable file
View File

@@ -0,0 +1,14 @@
#!/usr/bin/bash
# Compute 'src' path
SCRIPT_DIR=$(dirname "$0")
REPO_DIR=$(readlink -f "$SCRIPT_DIR"/..)
DIRS="src include"
# Apply formating to all *.cpp and *.h files
cd "$REPO_DIR"
for FILE in $(find $DIRS -name '*.h' -o -name '*.cpp')
do
echo $FILE
clang-format -i -style=file "$FILE"
done

View File

@@ -28,7 +28,6 @@
#include "tools/stringTools.h"
#include "tools/otherTools.h"
#include "tools/concurrent_cache.h"
#include "name_mapper.h"
#include <pugixml.hpp>
#include <algorithm>
@@ -462,9 +461,6 @@ void Library::updateBookDB(const Book& book)
indexer.index_text(normalizeText(book.getPublisher()), 1, "XP");
indexer.index_text(normalizeText(book.getName()), 1, "XN");
indexer.index_text(normalizeText(book.getCategory()), 1, "XC");
const auto bookName = book.getHumanReadableIdFromPath();
const auto aliasName = HumanReadableNameMapper::removeDateFromBookId(bookName);
indexer.index_text(normalizeText(aliasName), 1, "XF");
for ( const auto& tag : split(normalizeText(book.getTags()), ";") ) {
doc.add_boolean_term("XT" + tag);
@@ -509,7 +505,6 @@ Xapian::Query buildXapianQueryFromFilterQuery(const Filter& filter)
queryParser.add_prefix("publisher", "XP");
queryParser.add_prefix("creator", "A");
queryParser.add_prefix("tag", "XT");
queryParser.add_prefix("filename", "XF");
const auto partialQueryFlag = filter.queryIsPartial()
? Xapian::QueryParser::FLAG_PARTIAL
: 0;
@@ -526,16 +521,6 @@ Xapian::Query buildXapianQueryFromFilterQuery(const Filter& filter)
return queryParser.parse_query(normalizeText(filter.getQuery()), flags);
}
Xapian::Query makePhraseQuery(const std::string& query, const std::string& prefix)
{
Xapian::QueryParser queryParser;
queryParser.set_default_op(Xapian::Query::OP_OR);
queryParser.set_stemming_strategy(Xapian::QueryParser::STEM_NONE);
const auto flags = 0;
const auto q = queryParser.parse_query(normalizeText(query), flags, prefix);
return Xapian::Query(Xapian::Query::OP_PHRASE, q.get_terms_begin(), q.get_terms_end(), q.get_length());
}
Xapian::Query nameQuery(const std::string& name)
{
return Xapian::Query("XN" + normalizeText(name));
@@ -546,18 +531,6 @@ Xapian::Query categoryQuery(const std::string& category)
return Xapian::Query("XC" + normalizeText(category));
}
Xapian::Query aliasNamesQuery(const Filter::AliasNames& aliasNames)
{
Xapian::Query q = Xapian::Query(std::string());
std::vector<Xapian::Query> queryVec;
for (const auto& aliasName : aliasNames) {
queryVec.push_back(makePhraseQuery(aliasName, "XF"));
}
Xapian::Query combinedQuery(Xapian::Query::OP_OR, queryVec.begin(), queryVec.end());
q = Xapian::Query(Xapian::Query::OP_FILTER, q, combinedQuery);
return q;
}
Xapian::Query langQuery(const std::string& lang)
{
return Xapian::Query("L" + normalizeText(lang));
@@ -565,12 +538,22 @@ Xapian::Query langQuery(const std::string& lang)
Xapian::Query publisherQuery(const std::string& publisher)
{
return makePhraseQuery(publisher, "XP");
Xapian::QueryParser queryParser;
queryParser.set_default_op(Xapian::Query::OP_OR);
queryParser.set_stemming_strategy(Xapian::QueryParser::STEM_NONE);
const auto flags = 0;
const auto q = queryParser.parse_query(normalizeText(publisher), flags, "XP");
return Xapian::Query(Xapian::Query::OP_PHRASE, q.get_terms_begin(), q.get_terms_end(), q.get_length());
}
Xapian::Query creatorQuery(const std::string& creator)
{
return makePhraseQuery(creator, "A");
Xapian::QueryParser queryParser;
queryParser.set_default_op(Xapian::Query::OP_OR);
queryParser.set_stemming_strategy(Xapian::QueryParser::STEM_NONE);
const auto flags = 0;
const auto q = queryParser.parse_query(normalizeText(creator), flags, "A");
return Xapian::Query(Xapian::Query::OP_PHRASE, q.get_terms_begin(), q.get_terms_end(), q.get_length());
}
Xapian::Query tagsQuery(const Filter::Tags& acceptTags, const Filter::Tags& rejectTags)
@@ -610,9 +593,6 @@ Xapian::Query buildXapianQuery(const Filter& filter)
const auto tq = tagsQuery(filter.getAcceptTags(), filter.getRejectTags());
q = Xapian::Query(Xapian::Query::OP_AND, q, tq);;
}
if ( !filter.getAliasNames().empty() ) {
q = Xapian::Query(Xapian::Query::OP_AND, q, aliasNamesQuery(filter.getAliasNames()));
}
return q;
}
@@ -762,7 +742,6 @@ enum filterTypes {
QUERY = FLAG(12),
NAME = FLAG(13),
CATEGORY = FLAG(14),
ALIASNAMES = FLAG(15),
};
Filter& Filter::local(bool accept)
@@ -865,13 +844,6 @@ Filter& Filter::name(std::string name)
return *this;
}
Filter& Filter::aliasNames(const AliasNames& aliasNames)
{
_aliasNames = aliasNames;
activeFilters |= ALIASNAMES;
return *this;
}
#define ACTIVE(X) (activeFilters & (X))
#define FILTER(TAG, TEST) if (ACTIVE(TAG) && !(TEST)) { return false; }
bool Filter::hasQuery() const

View File

@@ -41,8 +41,8 @@ struct NoDelete
// LibraryManipulator
////////////////////////////////////////////////////////////////////////////////
LibraryManipulator::LibraryManipulator(Library* library)
: library(*library)
LibraryManipulator::LibraryManipulator(std::shared_ptr<Library> library)
: library(library)
{}
LibraryManipulator::~LibraryManipulator()
@@ -50,7 +50,7 @@ LibraryManipulator::~LibraryManipulator()
bool LibraryManipulator::addBookToLibrary(const Book& book)
{
const auto ret = library.addBook(book);
const auto ret = library->addBook(book);
if ( ret ) {
bookWasAddedToLibrary(book);
}
@@ -59,13 +59,13 @@ bool LibraryManipulator::addBookToLibrary(const Book& book)
void LibraryManipulator::addBookmarkToLibrary(const Bookmark& bookmark)
{
library.addBookmark(bookmark);
library->addBookmark(bookmark);
bookmarkWasAddedToLibrary(bookmark);
}
uint32_t LibraryManipulator::removeBooksNotUpdatedSince(Library::Revision rev)
{
const auto n = library.removeBooksNotUpdatedSince(rev);
const auto n = library->removeBooksNotUpdatedSince(rev);
if ( n != 0 ) {
booksWereRemovedFromLibrary();
}
@@ -95,7 +95,7 @@ Manager::Manager(LibraryManipulator* manipulator):
{
}
Manager::Manager(Library* library) :
Manager::Manager(std::shared_ptr<Library> library) :
writableLibraryPath(""),
manipulator(new LibraryManipulator(library))
{

View File

@@ -24,7 +24,7 @@
namespace kiwix {
HumanReadableNameMapper::HumanReadableNameMapper(kiwix::Library& library, bool withAlias) {
HumanReadableNameMapper::HumanReadableNameMapper(const kiwix::Library& library, bool withAlias) {
for (auto& bookId: library.filter(kiwix::Filter().local(true).valid(true))) {
auto& currentBook = library.getBookById(bookId);
auto bookName = currentBook.getHumanReadableIdFromPath();
@@ -34,7 +34,7 @@ HumanReadableNameMapper::HumanReadableNameMapper(kiwix::Library& library, bool w
if (!withAlias)
continue;
auto aliasName = removeDateFromBookId(bookName);
auto aliasName = replaceRegex(bookName, "", "_[[:digit:]]{4}-[[:digit:]]{2}$");
if (aliasName == bookName) {
continue;
}
@@ -51,10 +51,6 @@ HumanReadableNameMapper::HumanReadableNameMapper(kiwix::Library& library, bool w
}
}
std::string HumanReadableNameMapper::removeDateFromBookId(const std::string& bookId) {
return replaceRegex(bookId, "", "_[[:digit:]]{4}-[[:digit:]]{2}$");
}
std::string HumanReadableNameMapper::getNameForId(const std::string& id) const {
return m_idToName.at(id);
}
@@ -67,7 +63,7 @@ std::string HumanReadableNameMapper::getIdForName(const std::string& name) const
// UpdatableNameMapper
////////////////////////////////////////////////////////////////////////////////
UpdatableNameMapper::UpdatableNameMapper(Library& lib, bool withAlias)
UpdatableNameMapper::UpdatableNameMapper(std::shared_ptr<Library> lib, bool withAlias)
: library(lib)
, withAlias(withAlias)
{
@@ -76,7 +72,7 @@ UpdatableNameMapper::UpdatableNameMapper(Library& lib, bool withAlias)
void UpdatableNameMapper::update()
{
const auto newNameMapper = new HumanReadableNameMapper(library, withAlias);
const auto newNameMapper = new HumanReadableNameMapper(*library, withAlias);
std::lock_guard<std::mutex> lock(mutex);
nameMapper.reset(newNameMapper);
}

View File

@@ -19,6 +19,8 @@
#include "opds_dumper.h"
#include "book.h"
#include "library.h"
#include "name_mapper.h"
#include "libkiwix-resources.h"
#include <mustache.hpp>
@@ -30,8 +32,8 @@ namespace kiwix
{
/* Constructor */
OPDSDumper::OPDSDumper(Library* library)
: library(library)
OPDSDumper::OPDSDumper(Server::Configuration configuration)
: m_configuration(configuration)
{
}
/* Destructor */
@@ -49,6 +51,8 @@ void OPDSDumper::setOpenSearchInfo(int totalResults, int startIndex, int count)
namespace
{
const std::string XML_HEADER(R"(<?xml version="1.0" encoding="UTF-8"?>)");
typedef kainjow::mustache::data MustacheData;
typedef kainjow::mustache::list BooksData;
typedef kainjow::mustache::list IllustrationInfo;
@@ -69,16 +73,17 @@ IllustrationInfo getBookIllustrationInfo(const Book& book)
return illustrations;
}
kainjow::mustache::object getSingleBookData(const Book& book)
std::string fullEntryXML(const Server::Configuration& configuration, const Book& book)
{
const auto bookDate = book.getDate() + "T00:00:00Z";
return kainjow::mustache::object{
const kainjow::mustache::object data{
{"root", configuration.m_root},
{"id", book.getId()},
{"name", book.getName()},
{"title", book.getTitle()},
{"description", book.getDescription()},
{"language", book.getLanguage()},
{"content_id", urlEncode(book.getHumanReadableIdFromPath(), true)},
{"content_id", urlEncode(configuration.mp_nameMapper->getNameForId(book.getId()), true)},
{"updated", bookDate}, // XXX: this should be the entry update datetime
{"book_date", bookDate},
{"category", book.getCategory()},
@@ -92,27 +97,33 @@ kainjow::mustache::object getSingleBookData(const Book& book)
{"size", to_string(book.getSize())},
{"icons", getBookIllustrationInfo(book)},
};
return render_template(RESOURCE::templates::catalog_v2_entry_xml, data);
}
std::string getSingleBookEntryXML(const Book& book, bool withXMLHeader, const std::string& rootLocation, const std::string& endpointRoot, bool partial)
std::string partialEntryXML(const Server::Configuration& configuration, const Book& book)
{
auto data = getSingleBookData(book);
data["with_xml_header"] = MustacheData(withXMLHeader);
data["dump_partial_entries"] = MustacheData(partial);
data["endpoint_root"] = endpointRoot;
data["root"] = rootLocation;
return render_template(RESOURCE::templates::catalog_v2_entry_xml, data);
const auto bookDate = book.getDate() + "T00:00:00Z";
const kainjow::mustache::object data{
{"root", configuration.m_root},
{"endpoint_root", configuration.m_root + "/catalog/v2"},
{"id", book.getId()},
{"title", book.getTitle()},
{"updated", bookDate}, // XXX: this should be the entry update datetime
};
const auto xmlTemplate = RESOURCE::templates::catalog_v2_partial_entry_xml;
return render_template(xmlTemplate, data);
}
BooksData getBooksData(const Library* library, const std::vector<std::string>& bookIds, const std::string& rootLocation, const std::string& endpointRoot, bool partial)
BooksData getBooksData(const Server::Configuration& configuration, const std::vector<std::string>& bookIds, bool partial)
{
BooksData booksData;
for ( const auto& bookId : bookIds ) {
try {
const Book book = library->getBookByIdThreadSafe(bookId);
booksData.push_back(kainjow::mustache::object{
{"entry", getSingleBookEntryXML(book, false, rootLocation, endpointRoot, partial)}
});
const Book book = configuration.mp_library->getBookByIdThreadSafe(bookId);
const auto entryXML = partial
? partialEntryXML(configuration, book)
: fullEntryXML(configuration, book);
booksData.push_back(kainjow::mustache::object{ {"entry", entryXML} });
} catch ( const std::out_of_range& ) {
// the book was removed from the library since its id was obtained
// ignore it
@@ -179,10 +190,10 @@ std::string getLanguageSelfName(const std::string& lang) {
string OPDSDumper::dumpOPDSFeed(const std::vector<std::string>& bookIds, const std::string& query) const
{
const auto booksData = getBooksData(library, bookIds, rootLocation, "", false);
const auto booksData = getBooksData(m_configuration, bookIds, false);
const kainjow::mustache::object template_data{
{"date", gen_date_str()},
{"root", rootLocation},
{"root", m_configuration.m_root},
{"feed_id", gen_uuid(libraryId + "/catalog/search?"+query)},
{"filter", onlyAsNonEmptyMustacheValue(query)},
{"totalResults", to_string(m_totalResults)},
@@ -196,8 +207,8 @@ string OPDSDumper::dumpOPDSFeed(const std::vector<std::string>& bookIds, const s
string OPDSDumper::dumpOPDSFeedV2(const std::vector<std::string>& bookIds, const std::string& query, bool partial) const
{
const auto endpointRoot = rootLocation + "/catalog/v2";
const auto booksData = getBooksData(library, bookIds, rootLocation, endpointRoot, partial);
const auto endpointRoot = m_configuration.m_root + "/catalog/v2";
const auto booksData = getBooksData(m_configuration, bookIds, partial);
const char* const endpoint = partial ? "/partial_entries" : "/entries";
const kainjow::mustache::object template_data{
@@ -218,14 +229,17 @@ string OPDSDumper::dumpOPDSFeedV2(const std::vector<std::string>& bookIds, const
std::string OPDSDumper::dumpOPDSCompleteEntry(const std::string& bookId) const
{
return getSingleBookEntryXML(library->getBookById(bookId), true, rootLocation, "", false);
const auto book = m_configuration.mp_library->getBookById(bookId);
return XML_HEADER
+ "\n"
+ fullEntryXML(m_configuration, book);
}
std::string OPDSDumper::categoriesOPDSFeed() const
{
const auto now = gen_date_str();
kainjow::mustache::list categoryData;
for ( const auto& category : library->getBooksCategories() ) {
for ( const auto& category : m_configuration.mp_library->getBooksCategories() ) {
const auto urlencodedCategoryName = urlEncode(category);
categoryData.push_back(kainjow::mustache::object{
{"name", category},
@@ -239,7 +253,7 @@ std::string OPDSDumper::categoriesOPDSFeed() const
RESOURCE::templates::catalog_v2_categories_xml,
kainjow::mustache::object{
{"date", now},
{"endpoint_root", rootLocation + "/catalog/v2"},
{"endpoint_root", m_configuration.m_root + "/catalog/v2"},
{"feed_id", gen_uuid(libraryId + "/categories")},
{"categories", categoryData }
}
@@ -251,7 +265,7 @@ std::string OPDSDumper::languagesOPDSFeed() const
const auto now = gen_date_str();
kainjow::mustache::list languageData;
std::call_once(fillLanguagesFlag, fillLanguagesMap);
for ( const auto& langAndBookCount : library->getBooksLanguagesWithCounts() ) {
for ( const auto& langAndBookCount : m_configuration.mp_library->getBooksLanguagesWithCounts() ) {
const std::string languageCode = langAndBookCount.first;
const int bookCount = langAndBookCount.second;
const auto languageSelfName = getLanguageSelfName(languageCode);
@@ -268,7 +282,7 @@ std::string OPDSDumper::languagesOPDSFeed() const
RESOURCE::templates::catalog_v2_languages_xml,
kainjow::mustache::object{
{"date", now},
{"endpoint_root", rootLocation + "/catalog/v2"},
{"endpoint_root", m_configuration.m_root + "/catalog/v2"},
{"feed_id", gen_uuid(libraryId + "/languages")},
{"languages", languageData }
}

View File

@@ -36,12 +36,12 @@ namespace kiwix
{
/* Constructor */
SearchRenderer::SearchRenderer(zim::SearchResultSet srs, NameMapper* mapper,
SearchRenderer::SearchRenderer(zim::SearchResultSet srs, std::shared_ptr<NameMapper> mapper,
unsigned int start, unsigned int estimatedResultCount)
: SearchRenderer(srs, mapper, nullptr, start, estimatedResultCount)
{}
SearchRenderer::SearchRenderer(zim::SearchResultSet srs, NameMapper* mapper, Library* library,
SearchRenderer::SearchRenderer(zim::SearchResultSet srs, std::shared_ptr<NameMapper> mapper, std::shared_ptr<Library> library,
unsigned int start, unsigned int estimatedResultCount)
: m_srs(srs),
mp_nameMapper(mapper),
@@ -166,7 +166,7 @@ kainjow::mustache::data buildPagination(
std::string SearchRenderer::renderTemplate(const std::string& tmpl_str)
{
const std::string absPathPrefix = protocolPrefix + "content/";
const std::string absPathPrefix = protocolPrefix;
// Build the results list
kainjow::mustache::data items{kainjow::mustache::data::type::list};
for (auto it = m_srs.begin(); it != m_srs.end(); it++) {
@@ -206,7 +206,7 @@ std::string SearchRenderer::renderTemplate(const std::string& tmpl_str)
kainjow::mustache::data allData;
allData.set("protocolPrefix", protocolPrefix);
allData.set("searchProtocolPrefix", searchProtocolPrefix);
allData.set("results", results);
allData.set("pagination", pagination);
allData.set("query", query);

View File

@@ -26,52 +26,38 @@
#include <zim/item.h>
#include "server/internalServer.h"
#include "tools/otherTools.h"
namespace kiwix {
Server::Server(Library* library, NameMapper* nameMapper) :
Server::Configuration::Configuration(std::shared_ptr<Library> library, std::shared_ptr<NameMapper> nameMapper) :
mp_library(library),
mp_nameMapper(nameMapper),
mp_server(nullptr)
mp_nameMapper(nameMapper ? nameMapper : std::make_shared<HumanReadableNameMapper>(*library, true))
{}
Server::Configuration& Server::Configuration::setRoot(const std::string& root)
{
m_root = normalizeRootUrl(root);
return *this;
}
Server::Server(const Server::Configuration& configuration) :
mp_server(new InternalServer(configuration))
{
}
Server::~Server() = default;
bool Server::start() {
mp_server.reset(new InternalServer(
mp_library,
mp_nameMapper,
m_addr,
m_port,
m_root,
m_nbThreads,
m_multizimSearchLimit,
m_verbose,
m_withTaskbar,
m_withLibraryButton,
m_blockExternalLinks,
m_indexTemplateString,
m_ipConnectionLimit));
return mp_server->start();
}
void Server::stop() {
if (mp_server) {
mp_server->stop();
mp_server.reset(nullptr);
}
mp_server->stop();
}
void Server::setRoot(const std::string& root)
{
m_root = root;
if (m_root[0] != '/') {
m_root = "/" + m_root;
}
if (m_root.back() == '/') {
m_root.erase(m_root.size() - 1);
}
bool Server::isRunning() {
return mp_server->isRunning();
}
int Server::getPort()

View File

@@ -85,16 +85,6 @@ namespace kiwix {
namespace
{
inline std::string normalizeRootUrl(std::string rootUrl)
{
while ( !rootUrl.empty() && rootUrl.back() == '/' )
rootUrl.pop_back();
while ( !rootUrl.empty() && rootUrl.front() == '/' )
rootUrl = rootUrl.substr(1);
return rootUrl.empty() ? rootUrl : "/" + rootUrl;
}
Filter get_search_filter(const RequestContext& request, const std::string& prefix="")
{
auto filter = kiwix::Filter().valid(true).local(true);
@@ -119,9 +109,6 @@ Filter get_search_filter(const RequestContext& request, const std::string& prefi
try {
filter.rejectTags(kiwix::split(request.get_argument(prefix+"notag"), ";"));
} catch (...) {}
try {
filter.aliasNames(request.get_arguments(prefix + "book"));
} catch (...) {}
return filter;
}
@@ -336,8 +323,6 @@ zim::Query SearchInfo::getZimQuery(bool verbose) const {
return query;
}
static IdNameMapper defaultNameMapper;
static MHD_Result staticHandlerCallback(void* cls,
struct MHD_Connection* connection,
const char* url,
@@ -369,33 +354,10 @@ public:
};
InternalServer::InternalServer(Library* library,
NameMapper* nameMapper,
std::string addr,
int port,
std::string root,
int nbThreads,
unsigned int multizimSearchLimit,
bool verbose,
bool withTaskbar,
bool withLibraryButton,
bool blockExternalLinks,
std::string indexTemplateString,
int ipConnectionLimit) :
m_addr(addr),
m_port(port),
m_root(normalizeRootUrl(root)),
m_nbThreads(nbThreads),
m_multizimSearchLimit(multizimSearchLimit),
m_verbose(verbose),
m_withTaskbar(withTaskbar),
m_withLibraryButton(withLibraryButton),
m_blockExternalLinks(blockExternalLinks),
m_indexTemplateString(indexTemplateString.empty() ? RESOURCE::templates::index_html : indexTemplateString),
m_ipConnectionLimit(ipConnectionLimit),
InternalServer::InternalServer(const Server::Configuration& configuration) :
Server::Configuration(configuration),
m_indexTemplateString(configuration.m_indexTemplateString.empty() ? RESOURCE::templates::index_html : configuration.m_indexTemplateString),
mp_daemon(nullptr),
mp_library(library),
mp_nameMapper(nameMapper ? nameMapper : &defaultNameMapper),
searchCache(getEnvVar<int>("KIWIX_SEARCH_CACHE_SIZE", DEFAULT_CACHE_SIZE)),
suggestionSearcherCache(getEnvVar<int>("KIWIX_SUGGESTION_SEARCHER_CACHE_SIZE", std::max((unsigned int) (mp_library->getBookCount(true, true)*0.1), 1U))),
m_customizedResources(new CustomizedResources)
@@ -409,7 +371,7 @@ bool InternalServer::start() {
#else
int flags = MHD_USE_POLL_INTERNALLY;
#endif
if (m_verbose.load())
if (m_verbose)
flags |= MHD_USE_DEBUG;
struct sockaddr_in sockAddr;
@@ -453,8 +415,13 @@ bool InternalServer::start() {
void InternalServer::stop()
{
MHD_stop_daemon(mp_daemon);
mp_daemon = nullptr;
}
bool InternalServer::isRunning() const
{
return mp_daemon != nullptr;
}
static MHD_Result staticHandlerCallback(void* cls,
struct MHD_Connection* connection,
const char* url,
@@ -484,14 +451,14 @@ MHD_Result InternalServer::handlerCallback(struct MHD_Connection* connection,
void** cont_cls)
{
auto start_time = std::chrono::steady_clock::now();
if (m_verbose.load() ) {
if (m_verbose) {
printf("======================\n");
printf("Requesting : \n");
printf("full_url : %s\n", url);
}
RequestContext request(connection, m_root, url, method, version);
if (m_verbose.load() ) {
if (m_verbose) {
request.print_debug_info();
}
/* Unexpected method */
@@ -507,7 +474,7 @@ MHD_Result InternalServer::handlerCallback(struct MHD_Connection* connection,
if (response->getReturnCode() == MHD_HTTP_INTERNAL_SERVER_ERROR) {
printf("========== INTERNAL ERROR !! ============\n");
if (!m_verbose.load()) {
if (!m_verbose) {
printf("Requesting : \n");
printf("full_url : %s\n", url);
request.print_debug_info();
@@ -520,7 +487,7 @@ MHD_Result InternalServer::handlerCallback(struct MHD_Connection* connection,
auto ret = response->send(request, connection);
auto end_time = std::chrono::steady_clock::now();
auto time_span = std::chrono::duration_cast<std::chrono::duration<double>>(end_time - start_time);
if (m_verbose.load()) {
if (m_verbose) {
printf("Request time : %fs\n", time_span.count());
printf("----------------------\n");
}
@@ -636,9 +603,24 @@ std::unique_ptr<Response> InternalServer::build_homepage(const RequestContext& r
* Archive and Zim handlers begin
**/
class InternalServer::LockableSuggestionSearcher : public zim::SuggestionSearcher
{
public:
explicit LockableSuggestionSearcher(const zim::Archive& archive)
: zim::SuggestionSearcher(archive)
{}
std::unique_lock<std::mutex> getLock() {
return std::unique_lock<std::mutex>(m_mutex);
}
virtual ~LockableSuggestionSearcher() = default;
private:
std::mutex m_mutex;
};
std::unique_ptr<Response> InternalServer::handle_suggest(const RequestContext& request)
{
if (m_verbose.load()) {
if (m_verbose) {
printf("** running handle_suggest\n");
}
@@ -669,7 +651,7 @@ std::unique_ptr<Response> InternalServer::handle_suggest(const RequestContext& r
count = 10;
}
if (m_verbose.load()) {
if (m_verbose) {
printf("Searching suggestions for: \"%s\"\n", queryString.c_str());
}
@@ -679,8 +661,9 @@ std::unique_ptr<Response> InternalServer::handle_suggest(const RequestContext& r
/* Get the suggestions */
auto searcher = suggestionSearcherCache.getOrPut(bookId,
[=](){ return make_shared<zim::SuggestionSearcher>(*archive); }
[=](){ return make_shared<LockableSuggestionSearcher>(*archive); }
);
const auto lock(searcher->getLock());
auto search = searcher->suggest(queryString);
auto srs = search.getResults(start, count);
@@ -721,7 +704,7 @@ std::unique_ptr<Response> InternalServer::handle_suggest(const RequestContext& r
std::unique_ptr<Response> InternalServer::handle_viewer_settings(const RequestContext& request)
{
if (m_verbose.load()) {
if (m_verbose) {
printf("** running handle_viewer_settings\n");
}
@@ -735,7 +718,7 @@ std::unique_ptr<Response> InternalServer::handle_viewer_settings(const RequestCo
std::unique_ptr<Response> InternalServer::handle_skin(const RequestContext& request)
{
if (m_verbose.load()) {
if (m_verbose) {
printf("** running handle_skin\n");
}
@@ -758,7 +741,7 @@ std::unique_ptr<Response> InternalServer::handle_skin(const RequestContext& requ
std::unique_ptr<Response> InternalServer::handle_search(const RequestContext& request)
{
if (m_verbose.load()) {
if (m_verbose) {
printf("** running handle_search\n");
}
@@ -787,7 +770,7 @@ std::unique_ptr<Response> InternalServer::handle_search(const RequestContext& re
try {
search = searchCache.getOrPut(searchInfo,
[=](){
return make_shared<zim::Search>(searcher->search(searchInfo.getZimQuery(m_verbose.load())));
return make_shared<zim::Search>(searcher->search(searchInfo.getZimQuery(m_verbose)));
}
);
} catch(std::runtime_error& e) {
@@ -833,7 +816,7 @@ std::unique_ptr<Response> InternalServer::handle_search(const RequestContext& re
search->getEstimatedMatches());
renderer.setSearchPattern(searchInfo.pattern);
renderer.setSearchBookQuery(searchInfo.bookFilterQuery);
renderer.setProtocolPrefix(m_root + "/");
renderer.setProtocolPrefix(m_root + "/content/");
renderer.setSearchProtocolPrefix(m_root + "/search");
renderer.setPageLength(pageLength);
if (request.get_requested_format() == "xml") {
@@ -859,7 +842,7 @@ std::unique_ptr<Response> InternalServer::handle_search(const RequestContext& re
std::unique_ptr<Response> InternalServer::handle_random(const RequestContext& request)
{
if (m_verbose.load()) {
if (m_verbose) {
printf("** running handle_random\n");
}
@@ -911,7 +894,7 @@ std::unique_ptr<Response> InternalServer::handle_captured_external(const Request
std::unique_ptr<Response> InternalServer::handle_catch(const RequestContext& request)
{
if (m_verbose.load()) {
if (m_verbose) {
printf("** running handle_catch\n");
}
@@ -925,7 +908,7 @@ std::unique_ptr<Response> InternalServer::handle_catch(const RequestContext& req
std::unique_ptr<Response> InternalServer::handle_catalog(const RequestContext& request)
{
if (m_verbose.load()) {
if (m_verbose) {
printf("** running handle_catalog");
}
@@ -954,8 +937,7 @@ std::unique_ptr<Response> InternalServer::handle_catalog(const RequestContext& r
}
zim::Uuid uuid;
kiwix::OPDSDumper opdsDumper(mp_library);
opdsDumper.setRootLocation(m_root);
kiwix::OPDSDumper opdsDumper(*this);
opdsDumper.setLibraryId(m_library_id);
std::vector<std::string> bookIdsToDump;
if (url == "root.xml") {
@@ -1017,7 +999,7 @@ std::unique_ptr<Response> InternalServer::handle_content(const RequestContext& r
{
const std::string url = request.get_url();
const std::string pattern = url.substr((url.find_last_of('/'))+1);
if (m_verbose.load()) {
if (m_verbose) {
printf("** running handle_content\n");
}
@@ -1046,21 +1028,26 @@ std::unique_ptr<Response> InternalServer::handle_content(const RequestContext& r
try {
auto entry = getEntryFromPath(*archive, urlStr);
if (entry.isRedirect() || urlStr.empty()) {
// If urlStr is empty, we want to mainPage.
// We must do a redirection to the real page.
if (entry.isRedirect() || urlStr != entry.getPath()) {
// In the condition above, the second case (an entry with a different
// URL was returned) can occur in the following situations:
// 1. urlStr is empty or equal to "/" and the ZIM file doesn't contain
// such an entry, in which case the main entry is returned instead.
// 2. The ZIM file uses old namespace scheme, and the resource at urlStr
// is not present but can be found under one of the 'A', 'I', 'J' or
// '-' namespaces, in which case that resource is returned instead.
return build_redirect(bookName, getFinalItem(*archive, entry));
}
auto response = ItemResponse::build(*this, request, entry.getItem());
if (m_verbose.load()) {
if (m_verbose) {
printf("Found %s\n", entry.getPath().c_str());
printf("mimeType: %s\n", entry.getItem(true).getMimetype().c_str());
}
return response;
} catch(zim::EntryNotFound& e) {
if (m_verbose.load())
if (m_verbose)
printf("Failed to find %s\n", urlStr.c_str());
std::string searchURL = m_root + "/search?content=" + bookName + "&pattern=" + kiwix::urlEncode(pattern, true);
@@ -1073,7 +1060,7 @@ std::unique_ptr<Response> InternalServer::handle_content(const RequestContext& r
std::unique_ptr<Response> InternalServer::handle_raw(const RequestContext& request)
{
if (m_verbose.load()) {
if (m_verbose) {
printf("** running handle_raw\n");
}
@@ -1123,7 +1110,7 @@ std::unique_ptr<Response> InternalServer::handle_raw(const RequestContext& reque
return ItemResponse::build(*this, request, entry.getItem());
}
} catch (zim::EntryNotFound& e ) {
if (m_verbose.load()) {
if (m_verbose) {
printf("Failed to find %s\n", itemPath.c_str());
}
return HTTP404Response(*this, request)
@@ -1139,13 +1126,13 @@ bool InternalServer::isLocallyCustomizedResource(const std::string& url) const
std::unique_ptr<Response> InternalServer::handle_locally_customized_resource(const RequestContext& request)
{
if (m_verbose.load()) {
if (m_verbose) {
printf("** running handle_locally_customized_resource\n");
}
const CustomizedResourceData& crd = m_customizedResources->at(request.get_url());
if (m_verbose.load()) {
if (m_verbose) {
std::cout << "Reading " << crd.resourceFilePath << std::endl;
}
const auto resourceData = getFileContent(crd.resourceFilePath);

View File

@@ -26,7 +26,7 @@ extern "C" {
}
#include "library.h"
#include "name_mapper.h"
#include "server.h"
#include <zim/search.h>
#include <zim/suggestion.h>
@@ -88,26 +88,11 @@ class SearchInfo {
typedef kainjow::mustache::data MustacheData;
typedef ConcurrentCache<SearchInfo, std::shared_ptr<zim::Search>> SearchCache;
typedef ConcurrentCache<std::string, std::shared_ptr<zim::SuggestionSearcher>> SuggestionSearcherCache;
class OPDSDumper;
class InternalServer {
class InternalServer : Server::Configuration {
public:
InternalServer(Library* library,
NameMapper* nameMapper,
std::string addr,
int port,
std::string root,
int nbThreads,
unsigned int multizimSearchLimit,
bool verbose,
bool withTaskbar,
bool withLibraryButton,
bool blockExternalLinks,
std::string indexTemplateString,
int ipConnectionLimit);
InternalServer(const Server::Configuration& configuration);
virtual ~InternalServer();
MHD_Result handlerCallback(struct MHD_Connection* connection,
@@ -119,6 +104,7 @@ class InternalServer {
void** cont_cls);
bool start();
void stop();
bool isRunning() const;
std::string getAddress() { return m_addr; }
int getPort() { return m_port; }
@@ -157,23 +143,16 @@ class InternalServer {
bool isLocallyCustomizedResource(const std::string& url) const;
private: // types
class LockableSuggestionSearcher;
typedef ConcurrentCache<SearchInfo, std::shared_ptr<zim::Search>> SearchCache;
typedef ConcurrentCache<std::string, std::shared_ptr<LockableSuggestionSearcher>> SuggestionSearcherCache;
private: // data
std::string m_addr;
int m_port;
std::string m_root;
int m_nbThreads;
unsigned int m_multizimSearchLimit;
std::atomic_bool m_verbose;
bool m_withTaskbar;
bool m_withLibraryButton;
bool m_blockExternalLinks;
std::string m_indexTemplateString;
int m_ipConnectionLimit;
struct MHD_Daemon* mp_daemon;
Library* mp_library;
NameMapper* mp_nameMapper;
SearchCache searchCache;
SuggestionSearcherCache suggestionSearcherCache;

View File

@@ -35,7 +35,7 @@ namespace kiwix {
std::unique_ptr<Response> InternalServer::handle_catalog_v2(const RequestContext& request)
{
if (m_verbose.load()) {
if (m_verbose) {
printf("** running handle_catalog_v2");
}
@@ -95,8 +95,7 @@ std::unique_ptr<Response> InternalServer::handle_catalog_v2_root(const RequestCo
std::unique_ptr<Response> InternalServer::handle_catalog_v2_entries(const RequestContext& request, bool partial)
{
OPDSDumper opdsDumper(mp_library);
opdsDumper.setRootLocation(m_root);
OPDSDumper opdsDumper(*this);
opdsDumper.setLibraryId(m_library_id);
const auto bookIds = search_catalog(request, opdsDumper);
const auto opdsFeed = opdsDumper.dumpOPDSFeedV2(bookIds, request.get_query(), partial);
@@ -116,8 +115,7 @@ std::unique_ptr<Response> InternalServer::handle_catalog_v2_complete_entry(const
+ urlNotFoundMsg;
}
OPDSDumper opdsDumper(mp_library);
opdsDumper.setRootLocation(m_root);
OPDSDumper opdsDumper(*this);
opdsDumper.setLibraryId(m_library_id);
const auto opdsFeed = opdsDumper.dumpOPDSCompleteEntry(entryId);
return ContentResponse::build(
@@ -129,8 +127,7 @@ std::unique_ptr<Response> InternalServer::handle_catalog_v2_complete_entry(const
std::unique_ptr<Response> InternalServer::handle_catalog_v2_categories(const RequestContext& request)
{
OPDSDumper opdsDumper(mp_library);
opdsDumper.setRootLocation(m_root);
OPDSDumper opdsDumper(*this);
opdsDumper.setLibraryId(m_library_id);
return ContentResponse::build(
*this,
@@ -141,8 +138,7 @@ std::unique_ptr<Response> InternalServer::handle_catalog_v2_categories(const Req
std::unique_ptr<Response> InternalServer::handle_catalog_v2_languages(const RequestContext& request)
{
OPDSDumper opdsDumper(mp_library);
opdsDumper.setRootLocation(m_root);
OPDSDumper opdsDumper(*this);
opdsDumper.setLibraryId(m_library_id);
return ContentResponse::build(
*this,

View File

@@ -114,7 +114,7 @@ Response::Response(bool verbose)
std::unique_ptr<Response> Response::build(const InternalServer& server)
{
return std::unique_ptr<Response>(new Response(server.m_verbose.load()));
return std::unique_ptr<Response>(new Response(server.m_verbose));
}
std::unique_ptr<Response> Response::build_304(const InternalServer& server, const ETag& etag)
@@ -390,7 +390,7 @@ std::unique_ptr<ContentResponse> ContentResponse::build(
{
return std::unique_ptr<ContentResponse>(new ContentResponse(
server.m_root,
server.m_verbose.load(),
server.m_verbose,
content,
mimetype));
}
@@ -435,7 +435,7 @@ std::unique_ptr<Response> ItemResponse::build(const InternalServer& server, cons
}
return std::unique_ptr<Response>(new ItemResponse(
server.m_verbose.load(),
server.m_verbose,
item,
mimetype,
byteRange));

View File

@@ -370,6 +370,16 @@ std::string kiwix::gen_uuid(const std::string& s)
return kiwix::to_string(zim::Uuid::generate(s));
}
std::string kiwix::normalizeRootUrl(std::string rootUrl)
{
while ( !rootUrl.empty() && rootUrl.back() == '/' )
rootUrl.pop_back();
while ( !rootUrl.empty() && rootUrl.front() == '/' )
rootUrl = rootUrl.substr(1);
return rootUrl.empty() ? rootUrl : "/" + rootUrl;
}
kainjow::mustache::data kiwix::onlyAsNonEmptyMustacheValue(const std::string& s)
{
return s.empty()

View File

@@ -51,6 +51,8 @@ namespace kiwix
std::string gen_date_str();
std::string gen_uuid(const std::string& s);
std::string normalizeRootUrl(std::string rootUrl);
// if s is empty then returns kainjow::mustache::data(false)
// otherwise kainjow::mustache::data(value)
kainjow::mustache::data onlyAsNonEmptyMustacheValue(const std::string& s);

View File

@@ -27,6 +27,7 @@ templates/catalog_entries.xml
templates/catalog_v2_root.xml
templates/catalog_v2_entries.xml
templates/catalog_v2_entry.xml
templates/catalog_v2_partial_entry.xml
templates/catalog_v2_categories.xml
templates/catalog_v2_languages.xml
templates/url_of_search_results_css

View File

@@ -152,6 +152,27 @@ function updateSearchBoxForBookChange() {
}
}
let previousScrollTop = Infinity;
function updateToolbarVisibilityState() {
const iframeDoc = contentIframe.contentDocument;
const st = iframeDoc.documentElement.scrollTop || iframeDoc.body.scrollTop;
if ( Math.abs(previousScrollTop - st) <= 5 )
return;
const kiwixToolBar = document.querySelector('#kiwixtoolbar');
if (st > previousScrollTop) {
kiwixToolBar.style.position = 'fixed';
kiwixToolBar.style.top = '-100%';
} else {
kiwixToolBar.style.position = 'static';
kiwixToolBar.style.top = '0';
}
previousScrollTop = st;
}
function handle_visual_viewport_change() {
contentIframe.height = window.visualViewport.height - contentIframe.offsetTop - 4;
}
@@ -165,6 +186,7 @@ function handle_location_hash_change() {
contentIframe.contentWindow.location.replace(iframeContentUrl);
}
updateSearchBoxForLocationChange();
previousScrollTop = Infinity;
}
function handle_content_url_change() {
@@ -276,39 +298,7 @@ function htmlDecode(input) {
}
function setupAutoHidingOfTheToolbar() {
let lastScrollTop = 0;
const delta = 5;
let didScroll = false;
const kiwixToolBar = document.querySelector('#kiwixtoolbar');
contentIframe.contentWindow.addEventListener('scroll', () => {
didScroll = true;
});
setInterval(function() {
if (didScroll) {
hasScrolled();
didScroll = false;
}
}, 250);
function hasScrolled() {
const iframeDoc = contentIframe.contentDocument;
const st = iframeDoc.documentElement.scrollTop || iframeDoc.body.scrollTop;
if (Math.abs(lastScrollTop - st) <= delta)
return;
if (st > lastScrollTop) {
kiwixToolBar.style.position = 'fixed';
kiwixToolBar.style.top = '-100%';
} else {
kiwixToolBar.style.position = 'static';
kiwixToolBar.style.top = '0';
}
lastScrollTop = st;
}
setInterval(updateToolbarVisibilityState, 250);
}
function setupSuggestions() {
@@ -384,7 +374,10 @@ function setupSuggestions() {
}
function setupViewer() {
setInterval(handle_visual_viewport_change, 0);
// Defer the call of handle_visual_viewport_change() until after the
// presence or absence of the taskbar as determined by this function
// has been settled.
setTimeout(handle_visual_viewport_change, 0);
const kiwixToolBarWrapper = document.getElementById('kiwixtoolbarwrapper');
if ( ! viewerSettings.toolbarEnabled ) {

View File

@@ -1,13 +1,8 @@
{{#with_xml_header}}<?xml version="1.0" encoding="UTF-8"?>
{{/with_xml_header}} <entry>
<entry>
<id>urn:uuid:{{id}}</id>
<title>{{title}}</title>
<updated>{{updated}}</updated>
{{#dump_partial_entries}}
<link rel="alternate"
href="{{endpoint_root}}/entry/{{{id}}}"
type="application/atom+xml;type=entry;profile=opds-catalog"/>
{{/dump_partial_entries}}{{^dump_partial_entries}} <summary>{{description}}</summary>
<summary>{{description}}</summary>
<language>{{language}}</language>
<name>{{name}}</name>
<flavour>{{flavour}}</flavour>
@@ -29,5 +24,4 @@
{{#url}}
<link rel="http://opds-spec.org/acquisition/open-access" type="application/x-zim" href="{{{url}}}" length="{{{size}}}" />
{{/url}}
{{/dump_partial_entries}}
</entry>

View File

@@ -0,0 +1,8 @@
<entry>
<id>urn:uuid:{{id}}</id>
<title>{{title}}</title>
<updated>{{updated}}</updated>
<link rel="alternate"
href="{{endpoint_root}}/entry/{{{id}}}"
type="application/atom+xml;type=entry;profile=opds-catalog"/>
</entry>

View File

@@ -9,7 +9,7 @@
<opensearch:totalResults>{{results.count}}</opensearch:totalResults>
<opensearch:startIndex>{{results.start}}</opensearch:startIndex>
<opensearch:itemsPerPage>{{pagination.itemsPerPage}}</opensearch:itemsPerPage>
<atom:link rel="search" type="application/opensearchdescription+xml" href="{{protocolPrefix}}search/searchdescription.xml"/>
<atom:link rel="search" type="application/opensearchdescription+xml" href="{{searchProtocolPrefix}}/searchdescription.xml"/>
<opensearch:Query role="request"
searchTerms="{{query.pattern}}"{{#query.lang}}
language="{{query.lang}}"{{/query.lang}}

View File

@@ -20,6 +20,7 @@
#include "gtest/gtest.h"
#include <string>
#include "testing_tools.h"
const char * sampleOpdsStream = R"(
<feed xmlns="http://www.w3.org/2005/Atom"
@@ -237,7 +238,7 @@ namespace
TEST(LibraryOpdsImportTest, allInOne)
{
kiwix::Library lib;
kiwix::Manager manager(&lib);
kiwix::Manager manager{NotOwned<kiwix::Library>(lib)};
manager.readOpds(sampleOpdsStream, "library-opds-import.unittests.dev");
EXPECT_EQ(10U, lib.getBookCount(true, true));
@@ -297,8 +298,11 @@ class LibraryTest : public ::testing::Test {
typedef kiwix::Library::BookIdCollection BookIdCollection;
typedef std::vector<std::string> TitleCollection;
explicit LibraryTest()
{}
void SetUp() override {
kiwix::Manager manager(&lib);
kiwix::Manager manager{NotOwned<kiwix::Library>(lib)};
manager.readOpds(sampleOpdsStream, "foo.urlHost");
manager.readXml(sampleLibraryXML, false, "./test/library.xml", true);
}
@@ -500,24 +504,6 @@ TEST_F(LibraryTest, filterByTags)
);
}
TEST_F(LibraryTest, filterByAliasNames)
{
// filtering for one book
EXPECT_FILTER_RESULTS(kiwix::Filter().aliasNames({"zimfile"}),
"Ray Charles"
);
// filerting for more than one book
EXPECT_FILTER_RESULTS(kiwix::Filter().aliasNames({"zimfile", "example"}),
"An example ZIM archive",
"Ray Charles"
);
// filtering by alias name requires full text match
EXPECT_FILTER_RESULTS(kiwix::Filter().aliasNames({"wrong_name"}),
/* no results */
);
}
TEST_F(LibraryTest, filterByQuery)
{

View File

@@ -18,8 +18,13 @@ protected:
const int PORT = 8002;
protected:
void resetServer(ZimFileServer::Options options) {
zfs1_.reset();
zfs1_.reset(new ZimFileServer(PORT, options, "./test/library.xml"));
}
void SetUp() override {
zfs1_.reset(new ZimFileServer(PORT, "./test/library.xml"));
zfs1_.reset(new ZimFileServer(PORT, ZimFileServer::DEFAULT_OPTIONS, "./test/library.xml"));
}
void TearDown() override {
@@ -70,20 +75,20 @@ std::string maskVariableOPDSFeedData(std::string s)
" type=\"application/opensearchdescription+xml\"" \
" href=\"/ROOT/catalog/searchdescription.xml\" />\n"
#define CHARLES_RAY_CATALOG_ENTRY \
#define CATALOG_ENTRY(UUID, TITLE, SUMMARY, LANG, NAME, CATEGORY, TAGS, EXTRA_LINK, CONTENT_NAME, FILE_NAME, LENGTH) \
" <entry>\n" \
" <id>urn:uuid:charlesray</id>\n" \
" <title>Charles, Ray</title>\n" \
" <id>urn:uuid:" UUID "</id>\n" \
" <title>" TITLE "</title>\n" \
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n" \
" <summary>Wikipedia articles about Ray Charles</summary>\n" \
" <language>fra</language>\n" \
" <name>wikipedia_fr_ray_charles</name>\n" \
" <summary>" SUMMARY "</summary>\n" \
" <language>" LANG "</language>\n" \
" <name>" NAME "</name>\n" \
" <flavour></flavour>\n" \
" <category>jazz</category>\n" \
" <tags>unittest;wikipedia;_category:jazz;_pictures:no;_videos:no;_details:no;_ftindex:yes</tags>\n" \
" <category>" CATEGORY "</category>\n" \
" <tags>" TAGS "</tags>\n" \
" <articleCount>284</articleCount>\n" \
" <mediaCount>2</mediaCount>\n" \
" <link type=\"text/html\" href=\"/ROOT/content/zimfile%26other\" />\n" \
" " EXTRA_LINK "<link type=\"text/html\" href=\"/ROOT/content/" CONTENT_NAME "\" />\n" \
" <author>\n" \
" <name>Wikipedia</name>\n" \
" </author>\n" \
@@ -91,59 +96,57 @@ std::string maskVariableOPDSFeedData(std::string s)
" <name>Kiwix</name>\n" \
" </publisher>\n" \
" <dc:issued>2020-03-31T00:00:00Z</dc:issued>\n" \
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile%26other.zim\" length=\"569344\" />\n" \
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"https://github.com/kiwix/libkiwix/raw/master/test/data/" FILE_NAME ".zim\" length=\"" LENGTH "\" />\n" \
" </entry>\n"
#define RAY_CHARLES_CATALOG_ENTRY \
" <entry>\n" \
" <id>urn:uuid:raycharles</id>\n" \
" <title>Ray Charles</title>\n" \
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n" \
" <summary>Wikipedia articles about Ray Charles</summary>\n" \
" <language>eng</language>\n" \
" <name>wikipedia_en_ray_charles</name>\n" \
" <flavour></flavour>\n" \
" <category>wikipedia</category>\n" \
" <tags>public_tag_without_a_value;_private_tag_without_a_value;wikipedia;_category:wikipedia;_pictures:no;_videos:no;_details:no;_ftindex:yes</tags>\n" \
" <articleCount>284</articleCount>\n" \
" <mediaCount>2</mediaCount>\n" \
" <link rel=\"http://opds-spec.org/image/thumbnail\"\n" \
" href=\"/ROOT/catalog/v2/illustration/raycharles/?size=48\"\n" \
" type=\"image/png;width=48;height=48;scale=1\"/>\n" \
" <link type=\"text/html\" href=\"/ROOT/content/zimfile\" />\n" \
" <author>\n" \
" <name>Wikipedia</name>\n" \
" </author>\n" \
" <publisher>\n" \
" <name>Kiwix</name>\n" \
" </publisher>\n" \
" <dc:issued>2020-03-31T00:00:00Z</dc:issued>\n" \
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile.zim\" length=\"569344\" />\n" \
" </entry>\n"
#define UNCATEGORIZED_RAY_CHARLES_CATALOG_ENTRY \
" <entry>\n" \
" <id>urn:uuid:raycharles_uncategorized</id>\n" \
" <title>Ray (uncategorized) Charles</title>\n" \
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n" \
" <summary>No category is assigned to this library entry.</summary>\n" \
" <language>rus</language>\n" \
" <name>wikipedia_ru_ray_charles</name>\n" \
" <flavour></flavour>\n" \
" <category></category>\n" \
" <tags>public_tag_with_a_value:value_of_a_public_tag;_private_tag_with_a_value:value_of_a_private_tag;wikipedia;_pictures:no;_videos:no;_details:no</tags>\n" \
" <articleCount>284</articleCount>\n" \
" <mediaCount>2</mediaCount>\n" \
" <link type=\"text/html\" href=\"/ROOT/content/zimfile\" />\n" \
" <author>\n" \
" <name>Wikipedia</name>\n" \
" </author>\n" \
" <publisher>\n" \
" <name>Kiwix</name>\n" \
" </publisher>\n" \
" <dc:issued>2020-03-31T00:00:00Z</dc:issued>\n" \
" <link rel=\"http://opds-spec.org/acquisition/open-access\" type=\"application/x-zim\" href=\"https://github.com/kiwix/libkiwix/raw/master/test/data/zimfile.zim\" length=\"125952\" />\n" \
" </entry>\n"
#define _CHARLES_RAY_CATALOG_ENTRY(CONTENT_NAME) CATALOG_ENTRY( \
"charlesray", \
"Charles, Ray", \
"Wikipedia articles about Ray Charles", \
"fra", \
"wikipedia_fr_ray_charles",\
"jazz",\
"unittest;wikipedia;_category:jazz;_pictures:no;_videos:no;_details:no;_ftindex:yes",\
"", \
CONTENT_NAME, \
"zimfile%26other", \
"569344" \
)
#define CHARLES_RAY_CATALOG_ENTRY _CHARLES_RAY_CATALOG_ENTRY("zimfile%26other")
#define _RAY_CHARLES_CATALOG_ENTRY(CONTENT_NAME) CATALOG_ENTRY(\
"raycharles",\
"Ray Charles",\
"Wikipedia articles about Ray Charles",\
"eng",\
"wikipedia_en_ray_charles",\
"wikipedia",\
"public_tag_without_a_value;_private_tag_without_a_value;wikipedia;_category:wikipedia;_pictures:no;_videos:no;_details:no;_ftindex:yes",\
"<link rel=\"http://opds-spec.org/image/thumbnail\"\n" \
" href=\"/ROOT/catalog/v2/illustration/raycharles/?size=48\"\n" \
" type=\"image/png;width=48;height=48;scale=1\"/>\n ", \
CONTENT_NAME, \
"zimfile", \
"569344"\
)
#define RAY_CHARLES_CATALOG_ENTRY _RAY_CHARLES_CATALOG_ENTRY("zimfile")
#define UNCATEGORIZED_RAY_CHARLES_CATALOG_ENTRY CATALOG_ENTRY(\
"raycharles_uncategorized",\
"Ray (uncategorized) Charles",\
"No category is assigned to this library entry.",\
"rus",\
"wikipedia_ru_ray_charles",\
"",\
"public_tag_with_a_value:value_of_a_public_tag;_private_tag_with_a_value:value_of_a_private_tag;wikipedia;_pictures:no;_videos:no;_details:no",\
"",\
"zimfile", \
"zimfile", \
"125952"\
)
TEST_F(LibraryServerTest, catalog_root_xml)
{
@@ -780,4 +783,42 @@ TEST_F(LibraryServerTest, catalog_search_excludes_hidden_tags)
#undef EXPECT_ZERO_RESULTS
}
#define NO_MAPPER_CHARLES_RAY_CATALOG_ENTRY _CHARLES_RAY_CATALOG_ENTRY("charlesray")
#define NO_MAPPER_RAY_CHARLES_CATALOG_ENTRY _RAY_CHARLES_CATALOG_ENTRY("raycharles")
TEST_F(LibraryServerTest, no_name_mapper_returned_catalog_use_uuid_in_link)
{
resetServer(ZimFileServer::NO_NAME_MAPPER);
const auto r = zfs1_->GET("/ROOT/catalog/search?tag=_category:jazz");
EXPECT_EQ(r->status, 200);
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
OPDS_FEED_TAG
" <id>12345678-90ab-cdef-1234-567890abcdef</id>\n"
" <title>Filtered zims (tag=_category:jazz)</title>\n"
" <updated>YYYY-MM-DDThh:mm:ssZ</updated>\n"
" <totalResults>1</totalResults>\n"
" <startIndex>0</startIndex>\n"
" <itemsPerPage>1</itemsPerPage>\n"
CATALOG_LINK_TAGS
NO_MAPPER_CHARLES_RAY_CATALOG_ENTRY
"</feed>\n"
);
}
TEST_F(LibraryServerTest, no_name_mapper_catalog_v2_individual_entry_access)
{
resetServer(ZimFileServer::NO_NAME_MAPPER);
const auto r = zfs1_->GET("/ROOT/catalog/v2/entry/raycharles");
EXPECT_EQ(r->status, 200);
EXPECT_EQ(maskVariableOPDSFeedData(r->body),
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
NO_MAPPER_RAY_CHARLES_CATALOG_ENTRY
);
const auto r1 = zfs1_->GET("/ROOT/catalog/v2/entry/non-existent-entry");
EXPECT_EQ(r1->status, 404);
}
#undef EXPECT_SEARCH_RESULTS

View File

@@ -6,10 +6,12 @@
#include <iostream>
#include <fstream>
#include "testing_tools.h"
TEST(ManagerTest, addBookFromPathAndGetIdTest)
{
kiwix::Library lib;
kiwix::Manager manager = kiwix::Manager(&lib);
kiwix::Manager manager = kiwix::Manager(NotOwned<kiwix::Library>(lib));
auto bookId = manager.addBookFromPathAndGetId("./test/example.zim");
ASSERT_NE(bookId, "");
@@ -49,7 +51,7 @@ const char sampleLibraryXML[] = R"(
TEST(ManagerTest, readXml)
{
kiwix::Library lib;
kiwix::Manager manager = kiwix::Manager(&lib);
kiwix::Manager manager = kiwix::Manager(NotOwned<kiwix::Library>(lib));
EXPECT_EQ(true, manager.readXml(sampleLibraryXML, true, "/data/lib.xml", true));
kiwix::Book book = lib.getBookById("0d0bcd57-d3f6-cb22-44cc-a723ccb4e1b2");
@@ -71,7 +73,7 @@ TEST(ManagerTest, readXml)
TEST(Manager, reload)
{
kiwix::Library lib;
kiwix::Manager manager(&lib);
kiwix::Manager manager = kiwix::Manager(NotOwned<kiwix::Library>(lib));
manager.reload({ "./test/library.xml" });
EXPECT_EQ(lib.getBooksIds(), (kiwix::Library::BookIdCollection{

View File

@@ -4,9 +4,12 @@
#include "../include/manager.h"
#include "gtest/gtest.h"
#include "testing_tools.h"
namespace
{
const char libraryXML[] = R"(
<library version="1.0">
<book id="01" path="/data/zero_one.zim"> </book>
@@ -18,9 +21,13 @@ const char libraryXML[] = R"(
)";
class NameMapperTest : public ::testing::Test {
public:
explicit NameMapperTest()
{}
protected:
void SetUp() override {
kiwix::Manager manager(&lib);
kiwix::Manager manager{NotOwned<kiwix::Library>(lib)};
manager.readXml(libraryXML, false, "./library.xml", true);
for ( const std::string& id : lib.getBooksIds() ) {
kiwix::Book bookCopy = lib.getBookById(id);
@@ -108,7 +115,7 @@ TEST_F(NameMapperTest, HumanReadableNameMapperWithAliases)
TEST_F(NameMapperTest, UpdatableNameMapperWithoutAliases)
{
CapturedStderr stderror;
kiwix::UpdatableNameMapper nm(lib, false);
kiwix::UpdatableNameMapper nm(NotOwned<kiwix::Library>(lib), false);
EXPECT_EQ("", std::string(stderror));
checkUnaliasedEntriesInNameMapper(nm);
@@ -124,7 +131,7 @@ TEST_F(NameMapperTest, UpdatableNameMapperWithoutAliases)
TEST_F(NameMapperTest, UpdatableNameMapperWithAliases)
{
CapturedStderr stderror;
kiwix::UpdatableNameMapper nm(lib, true);
kiwix::UpdatableNameMapper nm(NotOwned<kiwix::Library>(lib), true);
EXPECT_EQ(
"Path collision: /data/zero_four_2021-10.zim and"
" /data/zero_four_2021-11.zim can't share the same URL path 'zero_four'."

View File

@@ -131,6 +131,13 @@ TEST_F(ServerTest, 200)
EXPECT_EQ(200, zfs1_->GET(res.url)->status) << "res.url: " << res.url;
}
TEST_F(ServerTest, 200_IdNameMapper)
{
resetServer(ZimFileServer::NO_NAME_MAPPER);
EXPECT_EQ(200, zfs1_->GET("/ROOT/content/6f1d19d0-633f-087b-fb55-7ac324ff9baf/A/index")->status);
EXPECT_EQ(404, zfs1_->GET("/ROOT/content/zimfile/A/index")->status);
}
TEST_F(ServerTest, CompressibleContentIsCompressedIfAcceptable)
{
for ( const Resource& res : resources200Compressible ) {
@@ -199,7 +206,7 @@ R"EXPECTEDRESULT( <img src="../skin/download.png?
/* url */ "/ROOT/viewer",
R"EXPECTEDRESULT( <link type="text/css" href="./skin/taskbar.css?cacheid=216d6b5d" rel="Stylesheet" />
<link type="text/css" href="./skin/css/autoComplete.css?cacheid=08951e06" rel="Stylesheet" />
<script type="text/javascript" src="./skin/viewer.js?cacheid=9a336712" defer></script>
<script type="text/javascript" src="./skin/viewer.js?cacheid=51e745c2" defer></script>
<script type="text/javascript" src="./skin/autoComplete.min.js?cacheid=1191aaaf"></script>
const blankPageUrl = `${root}/skin/blank.html`;
<label for="kiwix_button_show_toggle"><img src="./skin/caret.png?cacheid=22b942b4" alt=""></label>

View File

@@ -6,6 +6,8 @@
#include <zim/entry.h>
#include <zim/item.h>
#include "testing_tools.h"
// Output generated via mustache templates sometimes contains end-of-line
// whitespace. This complicates representing the expected output of a unit-test
// as C++ raw strings in editors that are configured to delete EOL whitespace.
@@ -61,6 +63,7 @@ public: // types
WITH_TASKBAR = 1 << 1,
WITH_LIBRARY_BUTTON = 1 << 2,
BLOCK_EXTERNAL_LINKS = 1 << 3,
NO_NAME_MAPPER = 1 << 4,
WITH_TASKBAR_AND_LIBRARY_BUTTON = WITH_TASKBAR | WITH_LIBRARY_BUTTON,
@@ -68,7 +71,7 @@ public: // types
};
public: // functions
ZimFileServer(int serverPort, std::string libraryFilePath);
ZimFileServer(int serverPort, Options options, std::string libraryFilePath);
ZimFileServer(int serverPort,
Options options,
const FilePathCollection& zimpaths,
@@ -91,14 +94,15 @@ private:
private: // data
kiwix::Library library;
kiwix::Manager manager;
std::unique_ptr<kiwix::HumanReadableNameMapper> nameMapper;
std::shared_ptr<kiwix::NameMapper> nameMapper;
std::unique_ptr<kiwix::Server> server;
std::unique_ptr<httplib::Client> client;
const Options options = DEFAULT_OPTIONS;
};
ZimFileServer::ZimFileServer(int serverPort, std::string libraryFilePath)
: manager(&this->library)
ZimFileServer::ZimFileServer(int serverPort, Options _options, std::string libraryFilePath)
: manager(NotOwned<kiwix::Library>(library)),
options(_options)
{
if ( kiwix::isRelativePath(libraryFilePath) )
libraryFilePath = kiwix::computeAbsolutePath(kiwix::getCurrentDirectory(), libraryFilePath);
@@ -110,8 +114,8 @@ ZimFileServer::ZimFileServer(int serverPort,
Options _options,
const FilePathCollection& zimpaths,
std::string indexTemplateString)
: manager(&this->library)
, options(_options)
: manager(NotOwned<kiwix::Library>(library)),
options(_options)
{
for ( const auto& zimpath : zimpaths ) {
if (!manager.addBookFromPath(zimpath, zimpath, "", false))
@@ -123,19 +127,25 @@ ZimFileServer::ZimFileServer(int serverPort,
void ZimFileServer::run(int serverPort, std::string indexTemplateString)
{
const std::string address = "127.0.0.1";
nameMapper.reset(new kiwix::HumanReadableNameMapper(library, false));
server.reset(new kiwix::Server(&library, nameMapper.get()));
server->setRoot("ROOT");
server->setAddress(address);
server->setPort(serverPort);
server->setNbThreads(2);
server->setVerbose(false);
server->setTaskbar(options & WITH_TASKBAR, options & WITH_LIBRARY_BUTTON);
server->setBlockExternalLinks(options & BLOCK_EXTERNAL_LINKS);
server->setMultiZimSearchLimit(3);
if (!indexTemplateString.empty()) {
server->setIndexTemplateString(indexTemplateString);
if (options & NO_NAME_MAPPER) {
nameMapper.reset(new kiwix::IdNameMapper());
} else {
nameMapper.reset(new kiwix::HumanReadableNameMapper(library, false));
}
kiwix::Server::Configuration configuration(NotOwned<kiwix::Library>(library), nameMapper);
configuration.setRoot("ROOT")
.setAddress(address)
.setPort(serverPort)
.setNbThreads(2)
.setVerbose(false)
.setTaskbar(options & WITH_TASKBAR, options & WITH_LIBRARY_BUTTON)
.setBlockExternalLinks(options & BLOCK_EXTERNAL_LINKS)
.setMultiZimSearchLimit(3);
if (!indexTemplateString.empty()) {
configuration.setIndexTemplateString(indexTemplateString);
}
server.reset(new kiwix::Server(configuration));
if ( !server->start() )
throw std::runtime_error("ZimFileServer failed to start");

19
test/testing_tools.h Normal file
View File

@@ -0,0 +1,19 @@
#ifndef TESTING_TOOLS
#define TESTING_TOOLS
#include <memory>
struct NoDelete {
template<class T> void operator()(T*) {}
};
template<class T>
class NotOwned : public std::shared_ptr<T> {
public:
NotOwned(T& object) :
std::shared_ptr<T>(&object, NoDelete()) {};
};
#endif // TESTING_TOOLS