Files
shelfmark/tests/core/test_config_api.py
Andrew Doering 816a735cde Add a {Language} naming template variable, and consolidate language resolution (#1142)
Fixes #1138
Fixes #1141

## Problem

Two language editions of one book resolve to the same canonical title,
so they render to the same path and the second gets a `_1` collision
suffix. Audiobookshelf treats a folder as exactly one library item, so
the pair becomes a single book with both files as tracks and a summed
runtime.

Shelfmark already parses and displays the language. It just never
reached the template engine.

## `{Language}` template variable

A template like `{Author}/{Title}{ (Language)}/{Author} - {Title}` now
yields:

```
/library/J K Rowling/Harry Potter (sv)/J K Rowling - Harry Potter.m4b
/library/J K Rowling/Harry Potter/J K Rowling - Harry Potter.m4b
```

The untagged edition's path is byte-identical to today, so no existing
layout shifts.

Three details worth flagging:

**The value is casefolded.** On a case-insensitive filesystem `(SV)` and
`(sv)` would collapse back into one folder, reintroducing the exact
collision being fixed.

**Values meaning "we don't know" render nothing** rather than producing
`Project Hail Mary (unknown)` folders. Anna's Archive reports that
string literally (`direct_download.py`, `language = detected or
"unknown"`).

**The frontend wasn't sending the release language at all**, so the
token would have stayed empty for exactly the audiobook sources in the
report. Prowlarr and AudiobookBay do not put language in `extra` the way
`direct_download` does, hence the payload plumbing. It reads
`release.language`, never `book.language` — the latter is the provider's
canonical edition and would mislabel a translation, with a regression
test for that specifically.

Not gated to audiobooks: Calibre-Web-Automated stages ingested files by
basename and discards folder structure, so the rename (filename)
template is the only lever those users have. Verified that form works:
`J K Rowling - Harry Potter (sv).epub`.

## Language consolidation (#1141)

Three release sources each carried their own alias map, all resolving to
the same ISO 639-1 codes, alongside a bundled database that only one of
them used. Adding a language meant editing three places.

Aliases now live in `data/book-languages.json` beside the code and name
they belong to, and `shelfmark/core/languages.py` resolves any of them —
two-letter code, ISO 639-2 three-letter in either the bibliographic or
terminological form, or English name. Prowlarr and AudiobookBay drop
their tables. Direct Download keeps its own path-parsing heuristics,
including the ambiguous short codes that collide with English words
(`de`, `en`, `no`, `in`), and takes only the alias data.

This also closes a coverage gap. MyAnonamouse offers 62 languages;
Prowlarr mapped 37, and an unmapped code is *dropped* rather than passed
through, so the other 25 carried no language at all — leaving
`{Language}` empty and the collision unfixed for Latin, Farsi, Tamil,
Urdu and the rest. Seven languages MAM offers had no database entry at
all: Bosnian, Burmese, Estonian, Icelandic, Manx, Scottish Gaelic,
Sanskrit.

Also fixes the Traditional Chinese code, which used a U+2011
non-breaking hyphen. Nothing compares against the ASCII spelling today
so it was latent, but it would silently defeat the first thing that did.

## Validation

Verified end to end against a live Prowlarr and MyAnonamouse, not just
unit tests. A real search returning both an English and a Swedish
edition, through the actual `queue_release` → `DownloadTask` → naming
path:

```
STEP 1  real MAM search        -> 37 releases, languages: ['en', 'sv']
STEP 3  queue_release          -> task.language='sv'
STEP 4  build_metadata_dict    -> metadata['Language']='sv'
STEP 5  build_library_path     -> /library/J K Rowling/Harry Potter (sv)/...
two language editions resolve to DIFFERENT folders: True
```

The refactor is pinned by a snapshot of both per-source maps taken
*before* they were deleted. All 131 aliases are asserted to still
resolve to the same code, one parametrised test each, so a regression
names the specific alias.

Also verified: the filename-only template, the retry round-trip
(`serialize_task_for_retry` → `_restore_task_from_retry_payload`, plus a
legacy payload with no `language` key), and placeholder handling.

Added a `KNOWN_TOKENS` ordering invariant test — `find_placeholder()`
does a substring `.find()` in list order and nothing protected that
contract, so a future token in the wrong position could silently shadow
an existing one. And a lockstep guard on the frontend, since
`KNOWN_TOKENS` is hand-duplicated in TypeScript.

**One caveat worth stating.** Three MAM codes are confirmed by
observation (`ENG`→`en`, `SWE`→`sv`, `MAL`→`ml`, the last from a real
`[MAL / EPUB]` Tagore release). The remaining ~59 are derived from ISO
639-2 rather than observed, because MAM's catalogue is overwhelmingly
English — enabling 27 extra languages still yielded only one non-English
hit across 258 results. Mitigated rather than closed: both 639-2
variants are present for every language where they differ, and a wrong
alias is an unused entry while a missing one loses the language. Happy
to correct any code a maintainer knows differs.

## Test results

2056 Python tests pass (up from 1906). Frontend typecheck, lint, format
and 126 unit tests pass.

Pre-existing failures on my machine, unchanged by this branch and
unrelated: `tests/bypass/` needs `seleniumbase`, and
`tests/config/test_entrypoint_permissions.py` uses bash-4 syntax that
macOS bash 3.2 rejects.

---------

Co-authored-by: delize <4028612+delize@users.noreply.github.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
2026-07-28 15:19:59 -04:00

177 lines
6.9 KiB
Python

"""API tests for the frontend config endpoint."""
from __future__ import annotations
import importlib
from pathlib import Path
from unittest.mock import patch
import pytest
@pytest.fixture(scope="module")
def main_module():
"""Import `shelfmark.main` with background startup disabled."""
with patch("shelfmark.download.orchestrator.start"):
import shelfmark.main as main
importlib.reload(main)
return main
@pytest.fixture
def client(main_module):
return main_module.app.test_client()
def _set_session(client, *, user_id: str, db_user_id: int, is_admin: bool) -> None:
with client.session_transaction() as sess:
sess["user_id"] = user_id
sess["db_user_id"] = db_user_id
sess["is_admin"] = is_admin
def test_config_endpoint_uses_user_scope_and_runtime_flags(main_module, client):
_set_session(client, user_id="reader-1", db_user_id=42, is_admin=False)
calls: list[tuple[str, int | None]] = []
def fake_get(key, default=None, user_id=None):
calls.append((key, user_id))
values = {
"SHOW_RELEASE_SOURCE_LINKS": False,
"SHOW_COMBINED_SELECTOR": False,
"SEARCH_MODE": "universal",
"SEARCH_PAGE_TITLE": "Custom Shelfmark",
"METADATA_PROVIDER": "openlibrary",
"METADATA_PROVIDER_AUDIOBOOK": "",
"DEFAULT_RELEASE_SOURCE": "prowlarr",
"DEFAULT_RELEASE_SOURCE_AUDIOBOOK": "audiobookbay",
"DOWNLOAD_TO_BROWSER_CONTENT_TYPES": ["book", "audiobook"],
"AUTO_OPEN_DOWNLOADS_SIDEBAR": False,
"HARDCOVER_AUTO_REMOVE_ON_DOWNLOAD": True,
"AA_DEFAULT_SORT": "newest",
}
return values.get(key, default)
with (
patch.object(main_module.app_config, "get", side_effect=fake_get),
patch("shelfmark.config.env._is_config_dir_writable", return_value=True),
patch("shelfmark.core.onboarding.is_onboarding_complete", return_value=True),
patch("shelfmark.metadata_providers.get_provider_sort_options", return_value=["sort-a"]),
patch("shelfmark.metadata_providers.get_provider_search_fields", return_value=["field-a"]),
patch("shelfmark.metadata_providers.get_provider_default_sort", return_value="relevance"),
):
resp = client.get("/api/config")
assert resp.status_code == 200
data = resp.get_json()
assert data["show_release_source_links"] is False
assert data["show_combined_selector"] is False
assert data["search_mode"] == "universal"
assert data["search_page_title"] == "Custom Shelfmark"
assert data["metadata_sort_options"] == ["sort-a"]
assert data["metadata_search_fields"] == ["field-a"]
assert data["default_release_source"] == "prowlarr"
assert data["default_release_source_audiobook"] == "audiobookbay"
assert data["download_to_browser_content_types"] == ["book", "audiobook"]
assert data["settings_enabled"] is True
assert data["metadata_default_sort"] == "relevance"
assert ("SHOW_RELEASE_SOURCE_LINKS", None) in calls
assert ("SHOW_COMBINED_SELECTOR", 42) in calls
assert ("DOWNLOAD_TO_BROWSER_CONTENT_TYPES", 42) in calls
def test_config_endpoint_falls_back_to_audiobook_metadata_provider(main_module, client):
_set_session(client, user_id="reader-2", db_user_id=77, is_admin=False)
provider_calls: list[str] = []
def fake_get(key, default=None, user_id=None):
values = {
"METADATA_PROVIDER": "",
"METADATA_PROVIDER_AUDIOBOOK": "audiobook-search",
"SHOW_RELEASE_SOURCE_LINKS": True,
}
return values.get(key, default)
def sort_options(provider: str):
provider_calls.append(provider)
return [f"{provider}-sort"]
def search_fields(provider: str):
provider_calls.append(provider)
return [f"{provider}-field"]
def default_sort(provider: str):
provider_calls.append(provider)
return f"{provider}-default"
with (
patch.object(main_module.app_config, "get", side_effect=fake_get),
patch("shelfmark.config.env._is_config_dir_writable", return_value=True),
patch("shelfmark.core.onboarding.is_onboarding_complete", return_value=True),
patch("shelfmark.metadata_providers.get_provider_sort_options", side_effect=sort_options),
patch("shelfmark.metadata_providers.get_provider_search_fields", side_effect=search_fields),
patch("shelfmark.metadata_providers.get_provider_default_sort", side_effect=default_sort),
):
resp = client.get("/api/config")
assert resp.status_code == 200
data = resp.get_json()
assert data["metadata_sort_options"] == ["audiobook-search-sort"]
assert data["metadata_search_fields"] == ["audiobook-search-field"]
assert data["metadata_default_sort"] == "audiobook-search-default"
assert provider_calls == ["audiobook-search", "audiobook-search", "audiobook-search"]
def test_frontend_dist_resolves_from_repo_root(main_module):
expected_project_root = Path(main_module.__file__).resolve().parent.parent
assert main_module.PROJECT_ROOT == expected_project_root
assert main_module.FRONTEND_DIST == expected_project_root / "frontend-dist"
def test_config_endpoint_serves_languages_without_resolution_aliases(main_module, client):
"""book_languages is a client contract, not a dump of the language data file.
data/book-languages.json also carries the aliases used to resolve a source's
spelling of a language to a code. Those are server-side only: the frontend
Language type is {code, language}, and shipping the aliases inflated every
config response by around 40%.
"""
_set_session(client, user_id="reader-1", db_user_id=1, is_admin=False)
with (
patch("shelfmark.config.env._is_config_dir_writable", return_value=True),
patch("shelfmark.core.onboarding.is_onboarding_complete", return_value=True),
):
resp = client.get("/api/config")
assert resp.status_code == 200
languages = resp.get_json()["book_languages"]
assert languages, "no languages served"
offending = [entry for entry in languages if set(entry) != {"code", "language"}]
assert offending == [], f"unexpected keys leaked to clients: {offending[:3]}"
def test_language_data_file_is_only_read_by_the_shared_module(main_module):
"""Reading data/book-languages.json anywhere else reintroduces the drift the
shared module exists to prevent, and bypasses the alias handling."""
del main_module
repo_root = Path(__file__).resolve().parents[2]
allowed = {Path("shelfmark/core/languages.py")}
offenders = []
for path in (repo_root / "shelfmark").rglob("*.py"):
relative = path.relative_to(repo_root)
if relative in allowed:
continue
if "book-languages" in path.read_text(encoding="utf-8"):
offenders.append(str(relative))
assert offenders == [], f"should use shelfmark.core.languages instead: {offenders}"