mirror of
https://github.com/calibrain/shelfmark.git
synced 2026-05-24 14:05:11 -04:00
Patch: Migrate bypasser to pure CDP + Misc fixes (#575)
Bypasser: - Refactored internal bypasser logic to use SeleniumBase Pure CDP mode, removed chromedriver dependencies and UC code. - Added dedicated threading for internal bypasser functions, fixes any potential asyncio CPU spike behavior - Fixed WebGL issue with Chromium 144. Reverted 1.0.3 hotfix and updated to latest Chromium Misc: - Added M4A color mapping - Fix frontend language filtering with multi-language releases - Added "days" age for usenet/torrent releases - Improved entrypoint chown efficiency - Added `ONBOARDING` env variable, default true
This commit is contained in:
Alex
11
Dockerfile
11
Dockerfile
@@ -130,11 +130,11 @@ RUN apt-get update && \
|
||||
xvfb \
|
||||
# For screen recording
|
||||
ffmpeg \
|
||||
# --- Chromium ---
|
||||
chromium=143.0.7499.169-1~deb13u1 \
|
||||
chromium-common=143.0.7499.169-1~deb13u1 \
|
||||
# --- ChromeDriver ---
|
||||
chromium-driver=143.0.7499.169-1~deb13u1 \
|
||||
# --- Chromium (unpinned - uses latest from Debian repos) ---
|
||||
# Chrome 144+ requires --enable-unsafe-swiftshader for WebGL in Docker.
|
||||
# This flag is set in internal_bypasser.py _get_browser_args()
|
||||
chromium \
|
||||
chromium-common \
|
||||
# For tkinter (pyautogui)
|
||||
python3-tk \
|
||||
# For RAR extraction
|
||||
@@ -152,7 +152,6 @@ RUN --mount=type=cache,target=/root/.cache/pip \
|
||||
|
||||
# Grant read/execute permissions to others
|
||||
RUN chmod -R o+rx /usr/bin/chromium && \
|
||||
chmod -R o+rx /usr/bin/chromedriver && \
|
||||
chmod -R o+rwx /usr/local/lib/python3.10/site-packages/seleniumbase/drivers/
|
||||
|
||||
# Default command to run the application entrypoint script
|
||||
|
||||
@@ -36,12 +36,13 @@ These environment variables are used at startup before the settings system loads
|
||||
| `CONFIG_DIR` | Directory for storing configuration files and plugin settings. | string (path) | `/config` |
|
||||
| `LOG_ROOT` | Root directory for log files. | string (path) | `/var/log/` |
|
||||
| `TMP_DIR` | Staging directory for downloads before moving to destination. | string (path) | `/tmp/shelfmark` |
|
||||
| `ENABLE_LOGGING` | Enable file logging to LOG_ROOT/shelfmark/shelfmark.log. | boolean | `true` |
|
||||
| `ENABLE_LOGGING` | Enable file logging under LOG_ROOT/shelfmark/ (including shelfmark.log and startup logs). | boolean | `true` |
|
||||
| `FLASK_HOST` | Host address for the Flask web server. | string | `0.0.0.0` |
|
||||
| `FLASK_PORT` | Port number for the Flask web server. | number | `8084` |
|
||||
| `SESSION_COOKIE_SECURE` | Enable secure cookies (requires HTTPS). | boolean | `false` |
|
||||
| `CWA_DB_PATH` | Path to the Calibre-Web database for authentication integration. | string (path) | `/auth/app.db` |
|
||||
| `DOCKERMODE` | Indicates the application is running inside a Docker container. | boolean | `false` |
|
||||
| `ONBOARDING` | Show the onboarding wizard on first run. Set to false to skip (useful for ephemeral storage). | boolean | `true` |
|
||||
|
||||
<details>
|
||||
<summary>Detailed descriptions</summary>
|
||||
@@ -69,7 +70,7 @@ Staging directory for downloads before moving to destination.
|
||||
|
||||
#### `ENABLE_LOGGING`
|
||||
|
||||
Enable file logging to LOG_ROOT/shelfmark/shelfmark.log.
|
||||
Enable file logging under LOG_ROOT/shelfmark/ (including shelfmark.log and startup logs).
|
||||
|
||||
- **Type:** boolean
|
||||
- **Default:** `true`
|
||||
@@ -109,6 +110,13 @@ Indicates the application is running inside a Docker container.
|
||||
- **Type:** boolean
|
||||
- **Default:** `false`
|
||||
|
||||
#### `ONBOARDING`
|
||||
|
||||
Show the onboarding wizard on first run. Set to false to skip (useful for ephemeral storage).
|
||||
|
||||
- **Type:** boolean
|
||||
- **Default:** `true`
|
||||
|
||||
</details>
|
||||
|
||||
## General
|
||||
@@ -243,8 +251,8 @@ The release source tab to open by default in the release modal.
|
||||
| `BOOKS_OUTPUT_MODE` | Choose where completed book files are sent. | string (choice) | `folder` |
|
||||
| `INGEST_DIR` | Directory where downloaded files are saved. | string | `/books` |
|
||||
| `FILE_ORGANIZATION` | Choose how downloaded book files are named and organized. | string (choice) | `rename` |
|
||||
| `TEMPLATE_RENAME` | Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Rename templates are filename-only (no '/' or '\'); use Organize for folders. | string | `{Author} - {Title} ({Year})` |
|
||||
| `TEMPLATE_ORGANIZE` | Use / to create folders. Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle} | string | `{Author}/{Title} ({Year})` |
|
||||
| `TEMPLATE_RENAME` | Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders. | string | `{Author} - {Title} ({Year})` |
|
||||
| `TEMPLATE_ORGANIZE` | Use / to create folders. Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. | string | `{Author}/{Title} ({Year})` |
|
||||
| `HARDLINK_TORRENTS` | Create hardlinks instead of copying. Preserves seeding but archives won't be extracted. Don't use if destination is a library ingest folder. | boolean | `false` |
|
||||
| `BOOKLORE_HOST` | Base URL of your Booklore instance | string | _none_ |
|
||||
| `BOOKLORE_USERNAME` | Booklore account username | string | _none_ |
|
||||
@@ -253,8 +261,8 @@ The release source tab to open by default in the release modal.
|
||||
| `BOOKLORE_PATH_ID` | Booklore library path for uploads. | string (choice) | _none_ |
|
||||
| `DESTINATION_AUDIOBOOK` | Leave empty to use Books destination. | string | _none_ |
|
||||
| `FILE_ORGANIZATION_AUDIOBOOK` | Choose how downloaded audiobook files are named and organized. | string (choice) | `rename` |
|
||||
| `TEMPLATE_AUDIOBOOK_RENAME` | Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Rename templates are filename-only (no '/' or '\'); use Organize for folders. | string | `{Author} - {Title}` |
|
||||
| `TEMPLATE_AUDIOBOOK_ORGANIZE` | Use / to create folders. Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber} | string | `{Author}/{Title}` |
|
||||
| `TEMPLATE_AUDIOBOOK_RENAME` | Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders. | string | `{Author} - {Title}` |
|
||||
| `TEMPLATE_AUDIOBOOK_ORGANIZE` | Use / to create folders. Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. | string | `{Author}/{Title}` |
|
||||
| `HARDLINK_TORRENTS_AUDIOBOOK` | Create hardlinks instead of copying. Preserves seeding but archives won't be extracted. Don't use if destination is a library ingest folder. | boolean | `true` |
|
||||
| `AUTO_OPEN_DOWNLOADS_SIDEBAR` | Automatically open the downloads sidebar when a new download is queued. | boolean | `false` |
|
||||
| `DOWNLOAD_TO_BROWSER` | Automatically download completed files to your browser. | boolean | `false` |
|
||||
@@ -298,7 +306,7 @@ Choose how downloaded book files are named and organized.
|
||||
|
||||
**Naming Template**
|
||||
|
||||
Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Rename templates are filename-only (no '/' or '\'); use Organize for folders.
|
||||
Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders.
|
||||
|
||||
- **Type:** string
|
||||
- **Default:** `{Author} - {Title} ({Year})`
|
||||
@@ -307,7 +315,7 @@ Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}
|
||||
|
||||
**Path Template**
|
||||
|
||||
Use / to create folders. Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}
|
||||
Use / to create folders. Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.
|
||||
|
||||
- **Type:** string
|
||||
- **Default:** `{Author}/{Title} ({Year})`
|
||||
@@ -394,7 +402,7 @@ Choose how downloaded audiobook files are named and organized.
|
||||
|
||||
**Naming Template**
|
||||
|
||||
Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Rename templates are filename-only (no '/' or '\'); use Organize for folders.
|
||||
Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders.
|
||||
|
||||
- **Type:** string
|
||||
- **Default:** `{Author} - {Title}`
|
||||
@@ -403,7 +411,7 @@ Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {P
|
||||
|
||||
**Path Template**
|
||||
|
||||
Use / to create folders. Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}
|
||||
Use / to create folders. Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.
|
||||
|
||||
- **Type:** string
|
||||
- **Default:** `{Author}/{Title}`
|
||||
|
||||
@@ -148,6 +148,7 @@ test_write() {
|
||||
|
||||
make_writable() {
|
||||
folder=$1
|
||||
did_full_chown=0
|
||||
set +e
|
||||
test_write $folder
|
||||
is_writable=$?
|
||||
@@ -158,31 +159,36 @@ make_writable() {
|
||||
echo "Folder $folder is not writable, changing ownership"
|
||||
change_ownership $folder
|
||||
chmod -R g+r,g+w $folder || echo "Failed to change group permissions for ${folder}, continuing..."
|
||||
did_full_chown=1
|
||||
fi
|
||||
# Fix any misowned subdirectories/files (e.g., from previous runs as root)
|
||||
if [ -d "$folder" ]; then
|
||||
misowned_count=$(find "$folder" -mindepth 1 \( ! -user "$RUN_UID" -o ! -group "$RUN_GID" \) 2>/dev/null | wc -l)
|
||||
if [ "$misowned_count" -gt 0 ]; then
|
||||
echo "Fixing ownership of $misowned_count files/directories in $folder"
|
||||
find "$folder" -mindepth 1 \( ! -user "$RUN_UID" -o ! -group "$RUN_GID" \) \
|
||||
-exec chown "$RUN_UID:$RUN_GID" {} \; 2>/dev/null || true
|
||||
fi
|
||||
if [ "$did_full_chown" -eq 0 ] && [ -d "$folder" ]; then
|
||||
echo "Checking for misowned files/directories in $folder"
|
||||
find "$folder" -mindepth 1 \( ! -user "$RUN_UID" -o ! -group "$RUN_GID" \) \
|
||||
-exec chown "$RUN_UID:$RUN_GID" {} + 2>/dev/null || true
|
||||
fi
|
||||
test_write $folder || echo "Failed to test write to ${folder}, continuing..."
|
||||
}
|
||||
|
||||
fix_misowned() {
|
||||
folder=$1
|
||||
mkdir -p $folder
|
||||
echo "Checking for misowned files/directories in $folder"
|
||||
find "$folder" \( ! -user "$RUN_UID" -o ! -group "$RUN_GID" \) \
|
||||
-exec chown "$RUN_UID:$RUN_GID" {} + 2>/dev/null || true
|
||||
}
|
||||
|
||||
# Ensure proper ownership of application directories
|
||||
change_ownership() {
|
||||
folder=$1
|
||||
mkdir -p $folder
|
||||
echo "Changing ownership of $folder to $USERNAME:$RUN_GID"
|
||||
chown -R "${RUN_UID}" "${folder}" || echo "Failed to change user ownership for ${folder}, continuing..."
|
||||
chown -R ":${RUN_GID}" "${folder}" || echo "Failed to change group ownership for ${folder}, continuing..."
|
||||
chown -R "${RUN_UID}:${RUN_GID}" "${folder}" || echo "Failed to change ownership for ${folder}, continuing..."
|
||||
}
|
||||
|
||||
change_ownership /app
|
||||
change_ownership /var/log/shelfmark
|
||||
change_ownership /tmp/shelfmark
|
||||
fix_misowned /app
|
||||
fix_misowned /var/log/shelfmark
|
||||
fix_misowned /tmp/shelfmark
|
||||
|
||||
# SeleniumBase (internal bypasser) writes a patched chromedriver binary (uc_driver)
|
||||
# into its own drivers directory. Some NAS/docker setups can apply restrictive ACLs
|
||||
|
||||
@@ -178,6 +178,12 @@ def _generate_bootstrap_env_docs() -> List[str]:
|
||||
"type": "boolean",
|
||||
"default": "false",
|
||||
},
|
||||
{
|
||||
"name": "ONBOARDING",
|
||||
"description": "Show the onboarding wizard on first run. Set to false to skip (useful for ephemeral storage).",
|
||||
"type": "boolean",
|
||||
"default": "true",
|
||||
},
|
||||
]
|
||||
|
||||
lines = [
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -133,6 +133,14 @@ TOR_VARIANT_AVAILABLE = shutil.which("tor") is not None
|
||||
USING_TOR = string_to_bool(os.getenv("USING_TOR", "false"))
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Onboarding
|
||||
# =============================================================================
|
||||
|
||||
# Set to false to skip the onboarding wizard entirely (useful for ephemeral storage)
|
||||
ONBOARDING = string_to_bool(os.getenv("ONBOARDING", "true"))
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Debug/development settings
|
||||
# =============================================================================
|
||||
|
||||
@@ -177,8 +177,8 @@ _FORMAT_OPTIONS = [
|
||||
|
||||
_AUDIOBOOK_FORMAT_OPTIONS = [
|
||||
{"value": "m4b", "label": "M4B"},
|
||||
{"value": "m4a", "label": "M4A"},
|
||||
{"value": "mp3", "label": "MP3"},
|
||||
{"value": "m4a", "label": "M4A"},
|
||||
{"value": "zip", "label": "ZIP"},
|
||||
{"value": "rar", "label": "RAR"},
|
||||
]
|
||||
|
||||
@@ -34,6 +34,12 @@ def _get_config_dir() -> Path:
|
||||
|
||||
def is_onboarding_complete() -> bool:
|
||||
"""Check if onboarding has been completed."""
|
||||
from shelfmark.config.env import ONBOARDING
|
||||
|
||||
# If onboarding is disabled via env var, treat as complete
|
||||
if not ONBOARDING:
|
||||
return True
|
||||
|
||||
config_file = _get_config_dir() / "settings.json"
|
||||
if not config_file.exists():
|
||||
return False
|
||||
|
||||
@@ -1056,6 +1056,7 @@ def _book_info_to_release(book_info: BookInfo) -> Release:
|
||||
source_id=book_info.id,
|
||||
title=book_info.title,
|
||||
format=book_info.format,
|
||||
language=book_info.language, # Top-level language for filtering
|
||||
size=book_info.size,
|
||||
download_url=book_info.download_urls[0] if book_info.download_urls else None,
|
||||
info_url=f"{network.get_aa_base_url()}/md5/{book_info.id}",
|
||||
|
||||
@@ -165,7 +165,7 @@ export const ReleaseCell = ({ column, release, compact = false, onlineServers }:
|
||||
const torznabAttrs = extra?.torznab_attrs as Record<string, string> | undefined;
|
||||
const publishDate = extra?.publish_date as string | undefined;
|
||||
|
||||
// Helper to format relative time
|
||||
// Helper to format age in days
|
||||
const formatRelativeTime = (dateStr: string): string | null => {
|
||||
try {
|
||||
const date = new Date(dateStr);
|
||||
@@ -174,14 +174,8 @@ export const ReleaseCell = ({ column, release, compact = false, onlineServers }:
|
||||
const diffMs = now.getTime() - date.getTime();
|
||||
const diffDays = Math.floor(diffMs / (1000 * 60 * 60 * 24));
|
||||
if (diffDays === 0) return 'Today';
|
||||
if (diffDays === 1) return '1 day ago';
|
||||
if (diffDays < 30) return `${diffDays} days ago`;
|
||||
const diffMonths = Math.floor(diffDays / 30);
|
||||
if (diffMonths === 1) return '1 month ago';
|
||||
if (diffMonths < 12) return `${diffMonths} months ago`;
|
||||
const diffYears = Math.floor(diffDays / 365);
|
||||
if (diffYears === 1) return '1 year ago';
|
||||
return `${diffYears} years ago`;
|
||||
if (diffDays === 1) return '1 day';
|
||||
return `${diffDays} days`;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -9,7 +9,7 @@ import { ReleaseCell } from './ReleaseCell';
|
||||
import { getColorStyleFromHint } from '../utils/colorMaps';
|
||||
import { getNestedValue } from '../utils/objectHelpers';
|
||||
import { LanguageMultiSelect } from './LanguageMultiSelect';
|
||||
import { LANGUAGE_OPTION_ALL, LANGUAGE_OPTION_DEFAULT, getLanguageFilterValues, releaseLanguageMatchesFilter } from '../utils/languageFilters';
|
||||
import { LANGUAGE_OPTION_ALL, LANGUAGE_OPTION_DEFAULT, getLanguageFilterValues, releaseLanguageMatchesFilter, buildLanguageNormalizer } from '../utils/languageFilters';
|
||||
|
||||
// Module-level cache for release search results
|
||||
// Key format: `${provider}:${provider_id}:${source}:${contentType}`
|
||||
@@ -1069,6 +1069,11 @@ export const ReleaseModal = ({
|
||||
return getLanguageFilterValues(languageFilter, bookLanguages, defaultLanguages);
|
||||
}, [languageFilter, bookLanguages, defaultLanguages]);
|
||||
|
||||
// Build language normalizer for release filtering (handles both codes like "en" and names like "English")
|
||||
const languageNormalizer = useMemo(() => {
|
||||
return buildLanguageNormalizer(bookLanguages);
|
||||
}, [bookLanguages]);
|
||||
|
||||
// Get column config from response or use default (moved before filteredReleases for sorting)
|
||||
const columnConfig = useMemo((): ReleaseColumnConfig => {
|
||||
const response = releasesBySource[activeTab];
|
||||
@@ -1164,7 +1169,7 @@ export const ReleaseModal = ({
|
||||
// Language filtering - use r.language when provided by enriched indexers
|
||||
// Releases with no language (null/undefined) always pass
|
||||
const releaseLang = r.language as string | undefined;
|
||||
if (!releaseLanguageMatchesFilter(releaseLang, resolvedLanguageCodes ?? defaultLanguages)) {
|
||||
if (!releaseLanguageMatchesFilter(releaseLang, resolvedLanguageCodes ?? defaultLanguages, languageNormalizer)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
@@ -1184,7 +1189,7 @@ export const ReleaseModal = ({
|
||||
}
|
||||
|
||||
return filtered;
|
||||
}, [releasesBySource, activeTab, formatFilter, resolvedLanguageCodes, effectiveFormats, defaultLanguages, indexerFilter, currentSort, sortableColumns, columnConfig]);
|
||||
}, [releasesBySource, activeTab, formatFilter, resolvedLanguageCodes, effectiveFormats, defaultLanguages, languageNormalizer, indexerFilter, currentSort, sortableColumns, columnConfig]);
|
||||
|
||||
// Pre-compute display field lookups to avoid repeated .find() calls in JSX
|
||||
const displayFields = useMemo(() => {
|
||||
|
||||
@@ -17,6 +17,7 @@ const FORMAT_COLORS: Record<string, ColorStyle> = {
|
||||
cbz: { bg: 'bg-amber-500/20', text: 'text-amber-700 dark:text-amber-300' },
|
||||
// Audiobook formats
|
||||
m4b: { bg: 'bg-violet-500/20', text: 'text-violet-700 dark:text-violet-300' },
|
||||
m4a: { bg: 'bg-violet-500/20', text: 'text-violet-700 dark:text-violet-300' },
|
||||
mp3: { bg: 'bg-rose-500/20', text: 'text-rose-700 dark:text-rose-300' },
|
||||
flac: { bg: 'bg-indigo-500/20', text: 'text-indigo-700 dark:text-indigo-300' },
|
||||
};
|
||||
|
||||
@@ -87,12 +87,33 @@ export const formatDefaultLanguageLabel = (
|
||||
return `Default (${joined}${suffix})`;
|
||||
};
|
||||
|
||||
/**
|
||||
* Build a mapping from language names to codes for normalization.
|
||||
* Handles both directions: "english" -> "en" and "en" -> "en"
|
||||
*/
|
||||
export const buildLanguageNormalizer = (languages: Language[]): Map<string, string> => {
|
||||
const map = new Map<string, string>();
|
||||
for (const lang of languages) {
|
||||
const code = lang.code.toLowerCase();
|
||||
map.set(code, code); // code -> code
|
||||
map.set(lang.language.toLowerCase(), code); // name -> code
|
||||
}
|
||||
return map;
|
||||
};
|
||||
|
||||
/**
|
||||
* Check if ALL languages in a multi-language release match the selected filter.
|
||||
* Multi-language releases use separators like comma, slash, plus, or ampersand
|
||||
* (e.g., "English, Spanish", "English/Spanish", "English + Spanish", "English & Spanish").
|
||||
*
|
||||
* @param releaseLang - Language string from the release (can be code or full name)
|
||||
* @param selectedCodes - Array of selected ISO language codes
|
||||
* @param languageNormalizer - Optional map to normalize language names to codes
|
||||
*/
|
||||
export const releaseLanguageMatchesFilter = (
|
||||
releaseLang: string | undefined,
|
||||
selectedCodes: string[] | null,
|
||||
languageNormalizer?: Map<string, string>,
|
||||
): boolean => {
|
||||
if (!releaseLang || !selectedCodes) {
|
||||
return true;
|
||||
@@ -100,7 +121,18 @@ export const releaseLanguageMatchesFilter = (
|
||||
if (selectedCodes.includes(LANGUAGE_OPTION_ALL)) {
|
||||
return true;
|
||||
}
|
||||
const releaseCodes = releaseLang.split(/[,/]/).map(l => l.trim().toLowerCase()).filter(Boolean);
|
||||
|
||||
// Split by common multi-language separators: comma, slash, plus, ampersand
|
||||
const releaseParts = releaseLang.split(/[,/+&]/).map(l => l.trim().toLowerCase()).filter(Boolean);
|
||||
|
||||
// Normalize release language parts to codes (handles both "en" and "english")
|
||||
const releaseCodes = releaseParts.map(part => {
|
||||
if (languageNormalizer) {
|
||||
return languageNormalizer.get(part) ?? part;
|
||||
}
|
||||
return part;
|
||||
});
|
||||
|
||||
const selectedSet = new Set(selectedCodes.map(c => c.toLowerCase()));
|
||||
return releaseCodes.every(code => selectedSet.has(code));
|
||||
};
|
||||
|
||||
@@ -22,3 +22,70 @@ def test_bypass_tries_all_methods_before_abort(monkeypatch):
|
||||
|
||||
assert internal_bypasser._bypass(object(), max_retries=10) is False
|
||||
assert calls == [f"m{i}" for i in range(6)]
|
||||
|
||||
|
||||
def test_extract_cookies_from_cdp_filters_and_stores_ua():
|
||||
import time
|
||||
import shelfmark.bypass.internal_bypasser as internal_bypasser
|
||||
|
||||
class FakeCookie:
|
||||
def __init__(self, name, value, domain, path, expires, secure=True):
|
||||
self.name = name
|
||||
self.value = value
|
||||
self.domain = domain
|
||||
self.path = path
|
||||
self.expires = expires
|
||||
self.secure = secure
|
||||
|
||||
class FakeSb:
|
||||
def get_all_cookies(self, requests_cookie_format=False):
|
||||
assert requests_cookie_format is True
|
||||
return [
|
||||
FakeCookie("cf_clearance", "abc", "example.com", "/", int(time.time()) + 3600),
|
||||
FakeCookie("sessionid", "zzz", "example.com", "/", int(time.time()) + 3600),
|
||||
]
|
||||
|
||||
def get_user_agent(self):
|
||||
return "TestUA/1.0"
|
||||
|
||||
internal_bypasser.clear_cf_cookies()
|
||||
internal_bypasser._extract_cookies_from_cdp(FakeSb(), "https://www.example.com/path")
|
||||
|
||||
cookies = internal_bypasser.get_cf_cookies_for_domain("example.com")
|
||||
assert cookies == {"cf_clearance": "abc"}
|
||||
assert internal_bypasser.get_cf_user_agent_for_domain("example.com") == "TestUA/1.0"
|
||||
|
||||
|
||||
def test_extract_cookies_from_cdp_normalizes_session_expiry():
|
||||
import time
|
||||
import shelfmark.bypass.internal_bypasser as internal_bypasser
|
||||
|
||||
class FakeCookie:
|
||||
def __init__(self, name, value, domain, path, expires, secure=True):
|
||||
self.name = name
|
||||
self.value = value
|
||||
self.domain = domain
|
||||
self.path = path
|
||||
self.expires = expires
|
||||
self.secure = secure
|
||||
|
||||
class FakeSb:
|
||||
def get_all_cookies(self, requests_cookie_format=False):
|
||||
assert requests_cookie_format is True
|
||||
return [
|
||||
FakeCookie("cf_clearance", "abc", "example.com", "/", 0),
|
||||
]
|
||||
|
||||
def get_user_agent(self):
|
||||
return "TestUA/1.0"
|
||||
|
||||
internal_bypasser.clear_cf_cookies()
|
||||
internal_bypasser._extract_cookies_from_cdp(FakeSb(), "https://example.com")
|
||||
|
||||
stored = internal_bypasser._cf_cookies.get("example.com", {})
|
||||
assert stored["cf_clearance"]["expiry"] is None
|
||||
assert internal_bypasser.get_cf_cookies_for_domain("example.com") == {"cf_clearance": "abc"}
|
||||
|
||||
# Verify fallback to "expires" key for expiry checks
|
||||
internal_bypasser._cf_cookies["example.com"]["cf_clearance"]["expires"] = int(time.time()) - 10
|
||||
assert internal_bypasser.get_cf_cookies_for_domain("example.com") == {}
|
||||
|
||||
Reference in New Issue
Block a user