Compare commits

..

41 Commits

Author SHA1 Message Date
Deluan
c2e8b39392 feat(plugins): update TaskWorker interface to return status messages and refactor task queue service
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-27 20:25:52 -05:00
Deluan
1974d1276e refactor(plugins): simplify goroutine management in task queue service
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-27 19:32:06 -05:00
Deluan
d7ace6f95f feat(plugins): increase maxConcurrency for task queue and handle budget exhaustion
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-27 19:32:06 -05:00
Deluan
a196ec9a59 refactor(plugins): streamline task queue configuration and error handling
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-27 19:32:06 -05:00
Deluan
132928abb6 fix(plugins): use context-aware database execution in TaskQueue host service
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-27 19:32:06 -05:00
Deluan
173aa9b979 refactor(plugins): remove capability check for TaskWorker in TaskQueue host service
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-27 19:32:06 -05:00
Deluan
3b2133c134 fix(plugins): harden TaskQueue host service with validation and safety improvements
Add input validation (queue name length, payload size limits), extract
status string constants to eliminate raw SQL literals, make CreateQueue
idempotent via upsert for crash recovery, fix RetentionMs default check
for negative values, cap exponential backoff at 1 hour to prevent
overflow, and replace manual mutex-based delay enforcement with
rate.Limiter from golang.org/x/time/rate for correct concurrent worker
serialization.
2026-02-27 19:32:06 -05:00
Deluan
74bacf6879 docs: document TaskQueue module for persistent task queues
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-27 19:32:06 -05:00
Deluan
55ef58da83 feat(plugins): add integration tests for TaskQueue host service 2026-02-27 19:32:06 -05:00
Deluan
2bfbe6fde1 feat(plugins): add test-taskqueue plugin for integration testing 2026-02-27 19:32:06 -05:00
Deluan
03cce614fd feat(plugins): register TaskQueue host service in manager 2026-02-27 19:32:06 -05:00
Deluan
36a8cb37ca feat(plugins): require TaskWorker capability for taskqueue permission 2026-02-27 19:32:06 -05:00
Deluan
11d2b3b51c feat(plugins): implement TaskQueue service with SQLite persistence and workers
Per-plugin SQLite database with queues and tasks tables. Worker goroutines
dequeue tasks and invoke nd_task_execute callback. Exponential backoff
retries, rate limiting via delayMs, automatic cleanup of terminal tasks.
2026-02-27 19:32:06 -05:00
Deluan
b308c71f38 feat(plugins): add taskqueue permission to manifest schema
Add TaskQueuePermission with maxConcurrency option.
2026-02-27 19:32:06 -05:00
Deluan
591f3a333b feat(plugins): define TaskWorker capability for task execution callbacks 2026-02-27 19:32:06 -05:00
Deluan
36b58a9a10 feat(plugins): define TaskQueue host service interface
Add the TaskQueueService interface with CreateQueue, Enqueue,
GetTaskStatus, and CancelTask methods plus QueueConfig struct.
2026-02-27 19:32:06 -05:00
Deluan Quintão
bd8032b327 fix(plugins): add base64 handling for []byte and remove raw=true (#5121)
* fix(plugins): add base64 handling for []byte and remove raw=true

Go's json.Marshal automatically base64-encodes []byte fields, but Rust's
serde_json serializes Vec<u8> as a JSON array and Python's json.dumps
raises TypeError on bytes. This fixes both directions of plugin
communication by adding proper base64 encoding/decoding in generated
client code.

For Rust templates (client and capability): adds a base64_bytes serde
helper module with #[serde(with = "base64_bytes")] on all Vec<u8> fields,
and adds base64 as a dependency. For Python templates: wraps bytes params
with base64.b64encode() and responses with base64.b64decode().

Also removes the raw=true binary framing protocol from all templates,
the parser, and the Method type. The raw mechanism added complexity that
is no longer needed once []byte works properly over JSON.

* fix(plugins): update production code and tests for base64 migration

Remove raw=true annotation from SubsonicAPI.CallRaw, delete all raw
test fixtures, remove raw-related test cases from parser, generator, and
integration tests, and add new test cases validating base64 handling
for Rust and Python templates.

* fix(plugins): update golden files and regenerate production code

Update golden test fixtures for codec and comprehensive services to
include base64 handling for []byte fields. Regenerate all production
PDK code (Go, Rust, Python) and host wrappers to use standard JSON
with base64-encoded byte fields instead of binary framing protocol.

* refactor: remove base64 helper duplication from rust template

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): add base64 dependency to capabilities' Cargo.toml

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-27 19:00:19 -05:00
Deluan
582d1b3cd9 refactor(plugins): validate scheduler capability at load time
Move scheduler capability check from runtime (when callback fires) to
load-time validation in ValidateWithCapabilities. This ensures plugins
declaring the scheduler permission must export the nd_scheduler_callback
function, failing fast with a clear error instead of silently skipping
callbacks at runtime.
2026-02-26 16:30:50 -05:00
Deluan
cdd3432788 refactor(http): rename HTTP client files and update struct names for consistency
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-26 16:19:37 -05:00
Deluan Quintão
5bc2bbb70e feat(subsonic): append album version to names in Subsonic API (#5111)
* feat(subsonic): append album version to album names in Subsonic API responses

Add AppendAlbumVersion config option (default: true) that appends the
album version tag to album names in Subsonic API responses, similar to
how AppendSubtitle works for track titles. This affects album names in
childFromAlbum and buildAlbumID3 responses.

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): append album version to media file album names in Subsonic API

Add FullAlbumName() to MediaFile that appends the album version tag,
mirroring the Album.FullName() behavior. Use it in childFromMediaFile
and fakePath to ensure media file responses also show the album version.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic): use len() check for album version tag to prevent panic on empty slice

Use len(tags) > 0 instead of != nil to safely guard against empty
slices when accessing the first element of the album version tag.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic): use FullName in buildAlbumDirectory and deduplicate FullName calls

Apply album.FullName() in buildAlbumDirectory (getMusicDirectory) so
album names are consistent across all Subsonic endpoints. Also compute
al.FullName() once in childFromAlbum to avoid redundant calls.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: use len() check in MediaFile.FullTitle() to prevent panic on empty slice

Apply the same safety improvement as FullAlbumName() and Album.FullName()
for consistency.

Signed-off-by: Deluan <deluan@navidrome.org>

* test: add tests for Album.FullName, MediaFile.FullTitle, and MediaFile.FullAlbumName

Cover all cases: config enabled/disabled, tag present, tag absent, and
empty tag slice.

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-26 10:50:12 -05:00
Deluan
14343d91b0 chore(deps): update goose to 3.27.0
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-24 21:44:04 -05:00
Deluan
fc36f1daa6 chore(deps): update go-taglib dependency to latest version (mka fix)
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-24 21:19:11 -05:00
Deluan Quintão
652c27690b feat(plugins): add HTTP host service (#5095)
* feat(httpclient): implement HttpClient service for outbound HTTP requests in plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(httpclient): enhance SSRF protection by validating host requests against private IPs

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(httpclient): support DELETE requests with body in HttpClient service

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(httpclient): refactor HTTP client initialization and enhance redirect handling

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(http): standardize naming conventions for HTTP types and methods

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor example plugin to use host.HTTPSend for improved error management

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): fix IPv6 SSRF bypass and wildcard host matching

Fix two bugs in the plugin HTTP/WebSocket host validation:

1. extractHostname now strips IPv6 brackets when no port is present
(e.g. "[::1]" → "::1"). Previously, net.SplitHostPort failed for
bracketed IPv6 without a port, leaving brackets intact. This caused
net.ParseIP to return nil, bypassing the private/loopback SSRF guard.

2. matchHostPattern now treats "*" as an allow-all pattern. Previously,
a bare "*" only matched via exact equality, so plugins declaring
requiredHosts: ["*"] (like webhook-rs) had all requests rejected.

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-24 14:28:36 -05:00
Deluan Quintão
2bb13e5ff1 feat(server): add ExtAuth logout URL configuration (#5074)
* feat(server): add ExtAuth logout URL configuration (#4467)

When external authentication (reverse proxy auth) is active, the Logout
button is hidden because authentication is managed externally. Many
external auth services (Authelia, Authentik, Keycloak) provide a logout
URL that can terminate the session.

Add `ExtAuth.LogoutURL` config option that, when set, shows the Logout
button in the UI and redirects the user to the external auth provider's
logout endpoint instead of the Navidrome login page.

* feat(server): add validation for ExtAuth logout URL configuration

* feat(server): refactor ExtAuth logout URL validation to a reusable function

* fix(configuration): rename URL validation functions for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(configuration): rename URL validation functions for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-23 20:28:38 -05:00
dependabot[bot]
d1c5e6a2f2 chore(deps): bump goreleaser/goreleaser-action in /.github/workflows (#5089)
Bumps [goreleaser/goreleaser-action](https://github.com/goreleaser/goreleaser-action) from 6 to 7.
- [Release notes](https://github.com/goreleaser/goreleaser-action/releases)
- [Commits](https://github.com/goreleaser/goreleaser-action/compare/v6...v7)

---
updated-dependencies:
- dependency-name: goreleaser/goreleaser-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-23 19:06:45 -05:00
Deluan
0c3cc86535 fix(subsonic): restore public attribute for playlists in XML responses
This was causing issues with DSub and DSub2000

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-23 18:17:44 -05:00
Deluan Quintão
b59eb32961 feat(subsonic): sort search3 results by relevance (#5086)
* fix(subsonic): optimize search3 for high-cardinality FTS queries

Use a two-phase query strategy for FTS5 searches to avoid the
performance penalty of expensive LEFT JOINs (annotation, bookmark,
library) on high-cardinality results like "the".

Phase 1 runs a lightweight query (main table + FTS index only) to get
sorted, paginated rowids. Phase 2 hydrates only those few rowids with
the full JOINs, making them nearly free.

For queries with complex ORDER BY expressions that reference joined
tables (e.g. artist search sorted by play count), the optimization is
skipped and the original single-query approach is used.

* fix(search): update order by clauses to include 'rank' for FTS queries

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): reintroduce 'rank' in Phase 2 ORDER BY for FTS queries

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): remove 'rank' from ORDER BY in non-FTS queries and adjust two-phase query handling

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): update FTS ranking to use bm25 weights and simplify ORDER BY qualification

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): refine FTS query handling and improve comments for clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): refactor full-text search handling to streamline query strategy selection and improve LIKE fallback logic.

Increase e2e coverage for search3

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance FTS column definitions and relevance weights

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): refactor Search method signatures to remove offset and size parameters, streamline query handling

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): allow single-character queries in search strategies and update related tests

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): make FTS Phase 1 treat Max=0 as no limit, reorganize tests

FTS Phase 1 unconditionally called Limit(uint64(options.Max)), which
produced LIMIT 0 when Max was zero. This diverged from applyOptions
where Max=0 means no limit. Now Phase 1 mirrors applyOptions: only add
LIMIT/OFFSET when the value is positive. Also moved legacy backend
integration tests from sql_search_fts_test.go to sql_search_like_test.go
and added regression tests for the Max=0 behavior on both backends.

* refactor: simplify callSearch function by removing variadic options and directly using QueryOptions

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): implement ftsQueryDegraded function to detect significant content loss in FTS queries

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-23 08:51:54 -05:00
Valeri Sokolov
23bf256a66 feat: make album and artist annotations available to smart playlists (#4927)
* feat(criteria): make album ratings available to smart playlist queries

Expose an "albumrating" field mapping to album annotations.

Signed-off-by: Valeri Sokolov <ulfurinn@ulfurinn.net>

* fix(criteria): use query parameters

Signed-off-by: Valeri Sokolov <ulfurinn@ulfurinn.net>

* feat: add album and artist annotation fields to smart playlists

Extend smart playlists to filter songs by album or artist annotations
(rating, loved, play count, last played, date loved, date rated). This
adds 12 new fields (6 album, 6 artist) with conditional JOINs that are
only added when the criteria or sort references them, avoiding
unnecessary query overhead. The album table JOIN is also removed since
media_file.album_id can be used directly.

---------

Signed-off-by: Valeri Sokolov <ulfurinn@ulfurinn.net>
Co-authored-by: Deluan <deluan@navidrome.org>
2026-02-22 22:05:59 -05:00
Deluan
d02bf9a53d test(e2e): add MusicBrainz ID tests for song and album searches
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-22 00:32:14 -05:00
Deluan
ec75808153 fix(subsonic): handle empty quoted phrases in FTS5 query and search expression
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-21 22:00:00 -05:00
Deluan Quintão
7ad2907719 refactor: move playlist business logic from repositories to service layer (#5027)
* refactor: move playlist business logic from repositories to core.Playlists service

Move authorization, permission checks, and orchestration logic from
playlist repositories to the core.Playlists service, following the
existing pattern used by core.Share and core.Library.

Changes:
- Expand core.Playlists interface with read, mutation, track management,
  and REST adapter methods
- Add playlistRepositoryWrapper for REST Save/Update/Delete with
  permission checks (follows Share/Library pattern)
- Simplify persistence/playlist_repository.go: remove isWritable(),
  auth checks from Delete()/Put()/updatePlaylist()
- Simplify persistence/playlist_track_repository.go: remove
  isTracksEditable() and permission checks from Add/Delete/Reorder
- Update Subsonic API handlers to route through service
- Update Native API handlers to accept core.Playlists instead of
  model.DataStore

* test: add coverage for playlist service methods and REST wrapper

Add 30 new tests covering the service methods added during the playlist
refactoring:

- Delete: owner, admin, denied, not found
- Create: new playlist, replace tracks, admin bypass, denied, not found
- AddTracks: owner, admin, denied, smart playlist, not found
- RemoveTracks: owner, smart playlist denied, non-owner denied
- ReorderTrack: owner, smart playlist denied
- NewRepository wrapper: Save (owner assignment, ID clearing),
  Update (owner, admin, denied, ownership change, not found),
  Delete (delegation with permission checks)

Expand mockedPlaylistRepo with Get, Delete, Tracks, GetWithTracks, and
rest.Persistable methods. Add mockedPlaylistTrackRepo for track
operation verification.

* fix: add authorization check to playlist Update method

Added ownership verification to the Subsonic Update endpoint in the
playlist service layer. The authorization check was present in the old
repository code but was not carried over during the refactoring to the
service layer, allowing any authenticated user to modify playlists they
don't own via the Subsonic API. Also added corresponding tests for the
Update method's permission logic.

* refactor: improve playlist permission checks and error handling, add e2e tests

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename core.Playlists to playlists package and update references

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename playlists_internal_test.go to parse_m3u_test.go and update tests; add new parse_nsp.go and rest_adapter.go files

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: block track mutations on smart playlists in Create and Update

Create now rejects replacing tracks on smart playlists (pre-existing
gap). Update now uses checkTracksEditable instead of checkWritable
when track changes are requested, restoring the protection that was
removed from the repository layer during the refactoring. Metadata-only
updates on smart playlists remain allowed.

* test: add smart playlist protection tests to ensure readonly behavior and mutation restrictions

* refactor: optimize track removal and renumbering in playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: implement track reordering in playlists with SQL updates

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: wrap track deletion and reordering in transactions for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: remove unused getTracks method from playlistTrackRepository

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: optimize playlist track renumbering with CTE-based UPDATE

Replace the DELETE + re-INSERT renumbering strategy with a two-step
UPDATE approach using a materialized CTE and ROW_NUMBER() window
function. The previous approach (SELECT all IDs, DELETE all tracks,
re-INSERT in chunks of 200) required 13 SQL operations for a 2000-track
playlist. The new approach uses just 2 UPDATEs: first negating all IDs
to clear the positive space, then assigning sequential positions via
UPDATE...FROM with a CTE. This avoids the UNIQUE constraint violations
that affected the original correlated subquery while reducing per-delete
request time from ~110ms to ~12ms on a 2000-track playlist.

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename New function to NewPlaylists for clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: update mock playlist repository and tests for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-21 19:57:13 -05:00
Deluan
76c01566a9 test(ui): change datagrid from table to div to fix warning
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-21 18:57:12 -05:00
Deluan
1cf3fd9161 fix(scanner): prevent ScanOnStartup when scanner is disabled
Gate the ScanOnStartup config on Scanner.Enabled so that setting
Scanner.Enabled=false prevents automatic startup scans. Other automatic
scan triggers (interrupted scan resume, PID change, post-migration) are
preserved regardless of the Enabled flag to maintain data integrity.
2026-02-21 18:51:16 -05:00
Deluan Quintão
54de0dbc52 feat(server): implement FTS5-based full-text search (#5079)
* build: add sqlite_fts5 build tag to enable FTS5 support

* feat: add SearchBackend config option (default: fts)

* feat: add buildFTS5Query for safe FTS5 query preprocessing

* feat: add FTS5 search backend with config toggle, refactor legacy search

- Add searchExprFunc type and getSearchExpr() for backend selection
- Rename fullTextExpr to legacySearchExpr
- Add ftsSearchExpr using FTS5 MATCH subquery
- Update fullTextFilter in sql_restful.go to use configured backend

* feat: add FTS5 migration with virtual tables, triggers, and search_participants

Creates FTS5 virtual tables for media_file, album, and artist with
unicode61 tokenizer and diacritic folding. Adds search_participants
column, populates from JSON, and sets up INSERT/UPDATE/DELETE triggers.

* feat: populate search_participants in PostMapArgs for FTS5 indexing

* test: add FTS5 search integration tests

* fix: exclude FTS5 virtual tables from e2e DB restore

The restoreDB function iterates all tables in sqlite_master and
runs DELETE + INSERT to reset state. FTS5 contentless virtual tables
cannot be directly deleted from. Since triggers handle FTS5 sync
automatically, simply skip tables matching *_fts and *_fts_* patterns.

* build: add compile-time guard for sqlite_fts5 build tag

Same pattern as netgo: compilation fails with a clear error if
the sqlite_fts5 build tag is missing.

* build: add sqlite_fts5 tag to reflex dev server config

* build: extract GO_BUILD_TAGS variable in Makefile to avoid duplication

* fix: strip leading * from FTS5 queries to prevent "unknown special query" error

* feat: auto-append prefix wildcard to FTS5 search tokens for broader matching

Every plain search token now gets a trailing * appended (e.g., "love" becomes
"love*"), so searching for "love" also matches "lovelace", "lovely", etc.
Quoted phrases are preserved as exact matches without wildcards. Results are
ordered alphabetically by name/title, so shorter exact matches naturally
appear first.

* fix: clarify comments about FTS5 operator neutralization

The comments said "strip" but the code lowercases operators to
neutralize them (FTS5 operators are case-sensitive). Updated comments
to accurately describe the behavior.

* fix: use fmt.Sprintf for FTS5 phrase placeholders

The previous encoding used rune('0'+index) which silently breaks with
10+ quoted phrases. Use fmt.Sprintf for arbitrary index support.

* fix: validate and normalize SearchBackend config option

Normalize the value to lowercase and fall back to "fts" with a log
warning for unrecognized values. This prevents silent misconfiguration
from typos like "FTS", "Legacy", or "fts5".

* refactor: improve documentation for build tags and FTS5 requirements

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: convert FTS5 query and search backend normalization tests to DescribeTable format

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: add sqlite_fts5 build tag to golangci configuration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add UISearchDebounceMs configuration option and update related components

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: fall back to legacy search when SearchFullString is enabled

FTS5 is token-based and cannot match substrings within words, so
getSearchExpr now returns legacySearchExpr when SearchFullString
is true, regardless of SearchBackend setting.

* fix: add sqlite_fts5 build tag to CI pipeline and Dockerfile

* fix: add WHEN clauses to FTS5 AFTER UPDATE triggers

Added WHEN clauses to the media_file_fts_au, album_fts_au, and
artist_fts_au triggers so they only fire when FTS-indexed columns
actually change. Previously, every row update (e.g., play count, rating,
starred status) triggered an unnecessary delete+insert cycle in the FTS
shadow tables. The WHEN clauses use IS NOT for NULL-safe comparison of
each indexed column, avoiding FTS index churn for non-indexed updates.

* feat: add SearchBackend configuration option to data and insights components

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: enhance input sanitization for FTS5 by stripping additional punctuation and special characters

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add search_normalized column for punctuated name search (R.E.M., AC/DC)

Add index-time normalization and query-time single-letter collapsing to
fix FTS5 search for punctuated names. A new search_normalized column
stores concatenated forms of punctuated words (e.g., "R.E.M." → "REM",
"AC/DC" → "ACDC") and is indexed in FTS5 tables. At query time, runs of
consecutive single letters (from dot-stripping) are collapsed into OR
expressions like ("R E M" OR REM*) to match both the original tokens and
the normalized form. This enables searching by "R.E.M.", "REM", "AC/DC",
"ACDC", "A-ha", or "Aha" and finding the correct results.

* refactor: simplify isSingleUnicodeLetter to avoid []rune allocation

Use utf8.DecodeRuneInString to check for a single Unicode letter
instead of converting the entire string to a []rune slice.

* feat: define ftsSearchColumns for flexible FTS5 search column inclusion

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update collapseSingleLetterRuns to return quoted phrases for abbreviations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement extractPunctuatedWords to handle artist/album names with embedded punctuation

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement extractPunctuatedWords to handle artist/album names with embedded punctuation

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: punctuated word handling to improve processing of artist/album names

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add CJK support for search queries with LIKE filters

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance FTS5 search by adding album version support and CJK handling

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: search configuration to use structured options

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance search functionality to support punctuation-only queries and update related tests

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-21 17:52:42 -05:00
Deluan
6f5f58ae9d chore(deps): update go-taglib to v0.0.0-20260221220301-2fab4903f48e
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-21 17:04:59 -05:00
Deluan Quintão
821f22a86f feat(scanner): upgrade TagLib to 2.2, with MKA/Matroska support (#5071)
* chore(deps): update go-taglib fork with MKA/Matroska support

Bump deluan/go-taglib to cf75207bfff8, which upgrades the underlying
taglib to v2.2 and adds Matroska container format detection and
metadata handling (MKA audio files).

* chore(deps): update cross-taglib version to 2.2.0-1

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(make): rename run-docker target to docker-run for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(go-taglib): update version to 2.2 WASM and add debug logging

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(deps): update go-taglib to v0.0.0-20260220032326 for MKA fixes

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-21 16:52:48 -05:00
Boris Rorsvort
74aa4d6fa5 fix(ui): Search focus after clear (#4932)
* wip

* refactor implem

* fixes
2026-02-21 14:39:38 -05:00
dependabot[bot]
dc4607c657 chore(deps): bump ajv from 6.12.6 to 6.14.0 in /ui (#5080)
Bumps [ajv](https://github.com/ajv-validator/ajv) from 6.12.6 to 6.14.0.
- [Release notes](https://github.com/ajv-validator/ajv/releases)
- [Commits](https://github.com/ajv-validator/ajv/compare/v6.12.6...v6.14.0)

---
updated-dependencies:
- dependency-name: ajv
  dependency-version: 6.14.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-21 12:44:32 -05:00
Deluan
ddab0da207 docs: update commit message format in CONTRIBUTING.md
Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-20 11:00:34 -05:00
Deluan Quintão
08a71320ea fix(ui): make toggle switches visible in Gruvbox Dark theme (#5063) (#5064)
The secondary color (#3c3836) matches the panel/table cell background,
making checked MuiSwitch thumbs invisible. Add MuiSwitch override using
Gruvbox cyan (#458588), consistent with existing interactive elements.
2026-02-18 15:38:20 -05:00
Raphael Catolino
44a5482493 fix(ui): activity Indicator switching constantly between online/offline (#5054)
When using HTTP2, setting the writeTimeout too low causes the channel to
close before the keepAlive event has a chance of beeing sent.

Signed-off-by: rca <raphael.catolino@gmail.com>
Co-authored-by: Deluan Quintão <deluan@navidrome.org>
2026-02-17 14:47:20 -05:00
191 changed files with 9980 additions and 2086 deletions

View File

@@ -14,7 +14,7 @@ RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install --no-install-recommends ffmpeg
# Install TagLib from cross-taglib releases
ARG CROSS_TAGLIB_VERSION="2.1.1-1"
ARG CROSS_TAGLIB_VERSION="2.2.0-1"
ARG TARGETARCH
RUN DOWNLOAD_ARCH="linux-${TARGETARCH}" \
&& wget -q "https://github.com/navidrome/cross-taglib/releases/download/v${CROSS_TAGLIB_VERSION}/taglib-${DOWNLOAD_ARCH}.tar.gz" -O /tmp/cross-taglib.tar.gz \

View File

@@ -8,7 +8,7 @@
// Options
"INSTALL_NODE": "true",
"NODE_VERSION": "v24",
"CROSS_TAGLIB_VERSION": "2.1.1-1"
"CROSS_TAGLIB_VERSION": "2.2.0-1"
}
},
"workspaceMount": "",

View File

@@ -14,7 +14,7 @@ concurrency:
cancel-in-progress: true
env:
CROSS_TAGLIB_VERSION: "2.1.1-2"
CROSS_TAGLIB_VERSION: "2.2.0-1"
CGO_CFLAGS_ALLOW: "--define-prefix"
IS_RELEASE: ${{ startsWith(github.ref, 'refs/tags/') && 'true' || 'false' }}
@@ -117,7 +117,7 @@ jobs:
- name: Test
run: |
pkg-config --define-prefix --cflags --libs taglib # for debugging
go test -shuffle=on -tags netgo -race ./... -v
go test -shuffle=on -tags netgo,sqlite_fts5 -race ./... -v
- name: Test ndpgen
run: |
@@ -424,7 +424,7 @@ jobs:
run: echo 'RELEASE_FLAGS=--skip=publish --snapshot' >> $GITHUB_ENV
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v6
uses: goreleaser/goreleaser-action@v7
with:
version: '~> v2'
args: "release --clean -f release/goreleaser.yml ${{ env.RELEASE_FLAGS }}"

View File

@@ -2,6 +2,7 @@ version: "2"
run:
build-tags:
- netgo
- sqlite_fts5
linters:
enable:
- asasalint

View File

@@ -38,7 +38,7 @@ Before submitting a pull request, ensure that you go through the following:
### Commit Conventions
Each commit message must adhere to the following format:
```
<type>(scope): <description> - <issue number>
<type>(scope): <description>
[optional body]
```

View File

@@ -28,7 +28,7 @@ COPY --from=xx-build /out/ /usr/bin/
### Get TagLib
FROM --platform=$BUILDPLATFORM public.ecr.aws/docker/library/alpine:3.20 AS taglib-build
ARG TARGETPLATFORM
ARG CROSS_TAGLIB_VERSION=2.1.1-2
ARG CROSS_TAGLIB_VERSION=2.2.0-1
ENV CROSS_TAGLIB_RELEASES_URL=https://github.com/navidrome/cross-taglib/releases/download/v${CROSS_TAGLIB_VERSION}/
# wget in busybox can't follow redirects
@@ -109,7 +109,7 @@ RUN --mount=type=bind,source=. \
export EXT=".exe"
fi
go build -tags=netgo -ldflags="${LD_EXTRA} -w -s \
go build -tags=netgo,sqlite_fts5 -ldflags="${LD_EXTRA} -w -s \
-X github.com/navidrome/navidrome/consts.gitSha=${GIT_SHA} \
-X github.com/navidrome/navidrome/consts.gitTag=${GIT_TAG}" \
-o /out/navidrome${EXT} .

View File

@@ -1,5 +1,6 @@
GO_VERSION=$(shell grep "^go " go.mod | cut -f 2 -d ' ')
NODE_VERSION=$(shell cat .nvmrc)
GO_BUILD_TAGS=netgo,sqlite_fts5
# Set global environment variables, required for most targets
export CGO_CFLAGS_ALLOW=--define-prefix
@@ -19,7 +20,7 @@ PLATFORMS ?= $(SUPPORTED_PLATFORMS)
DOCKER_TAG ?= deluan/navidrome:develop
# Taglib version to use in cross-compilation, from https://github.com/navidrome/cross-taglib
CROSS_TAGLIB_VERSION ?= 2.1.1-2
CROSS_TAGLIB_VERSION ?= 2.2.0-1
GOLANGCI_LINT_VERSION ?= v2.10.0
UI_SRC_FILES := $(shell find ui -type f -not -path "ui/build/*" -not -path "ui/node_modules/*")
@@ -46,12 +47,12 @@ stop: ##@Development Stop development servers (UI and backend)
.PHONY: stop
watch: ##@Development Start Go tests in watch mode (re-run when code changes)
go tool ginkgo watch -tags=netgo -notify ./...
go tool ginkgo watch -tags=$(GO_BUILD_TAGS) -notify ./...
.PHONY: watch
PKG ?= ./...
test: ##@Development Run Go tests. Use PKG variable to specify packages to test, e.g. make test PKG=./server
go test -tags netgo $(PKG)
go test -tags $(GO_BUILD_TAGS) $(PKG)
.PHONY: test
test-ndpgen: ##@Development Run tests for ndpgen plugin
@@ -62,7 +63,7 @@ testall: test test-ndpgen test-i18n test-js ##@Development Run Go and JS tests
.PHONY: testall
test-race: ##@Development Run Go tests with race detector
go test -tags netgo -race -shuffle=on $(PKG)
go test -tags $(GO_BUILD_TAGS) -race -shuffle=on $(PKG)
.PHONY: test-race
test-js: ##@Development Run JS tests
@@ -108,7 +109,7 @@ format: ##@Development Format code
.PHONY: format
wire: check_go_env ##@Development Update Dependency Injection
go tool wire gen -tags=netgo ./...
go tool wire gen -tags=$(GO_BUILD_TAGS) ./...
.PHONY: wire
gen: check_go_env ##@Development Run go generate for code generation
@@ -144,14 +145,14 @@ setup-git: ##@Development Setup Git hooks (pre-commit and pre-push)
.PHONY: setup-git
build: check_go_env buildjs ##@Build Build the project
go build -ldflags="-X github.com/navidrome/navidrome/consts.gitSha=$(GIT_SHA) -X github.com/navidrome/navidrome/consts.gitTag=$(GIT_TAG)" -tags=netgo
go build -ldflags="-X github.com/navidrome/navidrome/consts.gitSha=$(GIT_SHA) -X github.com/navidrome/navidrome/consts.gitTag=$(GIT_TAG)" -tags=$(GO_BUILD_TAGS)
.PHONY: build
buildall: deprecated build
.PHONY: buildall
debug-build: check_go_env buildjs ##@Build Build the project (with remote debug on)
go build -gcflags="all=-N -l" -ldflags="-X github.com/navidrome/navidrome/consts.gitSha=$(GIT_SHA) -X github.com/navidrome/navidrome/consts.gitTag=$(GIT_TAG)" -tags=netgo
go build -gcflags="all=-N -l" -ldflags="-X github.com/navidrome/navidrome/consts.gitSha=$(GIT_SHA) -X github.com/navidrome/navidrome/consts.gitTag=$(GIT_TAG)" -tags=$(GO_BUILD_TAGS)
.PHONY: debug-build
buildjs: check_node_env ui/build/index.html ##@Build Build only frontend
@@ -201,8 +202,8 @@ docker-msi: ##@Cross_Compilation Build MSI installer for Windows
@du -h binaries/msi/*.msi
.PHONY: docker-msi
run-docker: ##@Development Run a Navidrome Docker image. Usage: make run-docker tag=<tag>
@if [ -z "$(tag)" ]; then echo "Usage: make run-docker tag=<tag>"; exit 1; fi
docker-run: ##@Development Run a Navidrome Docker image. Usage: make docker-run tag=<tag>
@if [ -z "$(tag)" ]; then echo "Usage: make docker-run tag=<tag>"; exit 1; fi
@TAG_DIR="tmp/$$(echo '$(tag)' | tr '/:' '_')"; mkdir -p "$$TAG_DIR"; \
VOLUMES="-v $(PWD)/$$TAG_DIR:/data"; \
if [ -f navidrome.toml ]; then \
@@ -213,7 +214,7 @@ run-docker: ##@Development Run a Navidrome Docker image. Usage: make run-docker
fi; \
fi; \
echo "Running: docker run --rm -p 4533:4533 $$VOLUMES $(tag)"; docker run --rm -p 4533:4533 $$VOLUMES $(tag)
.PHONY: run-docker
.PHONY: docker-run
package: docker-build ##@Cross_Compilation Create binaries and packages for ALL supported platforms
@if [ -z `which goreleaser` ]; then echo "Please install goreleaser first: https://goreleaser.com/install/"; exit 1; fi

View File

@@ -20,6 +20,7 @@ import (
"strings"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/core/storage/local"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model/metadata"
@@ -43,7 +44,7 @@ func (e extractor) Parse(files ...string) (map[string]metadata.Info, error) {
}
func (e extractor) Version() string {
return "go-taglib (TagLib 2.1.1 WASM)"
return "2.2 WASM"
}
func (e extractor) extractMetadata(filePath string) (*metadata.Info, error) {
@@ -279,4 +280,7 @@ func init() {
local.RegisterExtractor("taglib", func(fsys fs.FS, baseDir string) local.Extractor {
return &extractor{fsys}
})
conf.AddHook(func() {
log.Debug("go-taglib version", "version", extractor{}.Version())
})
}

View File

@@ -196,7 +196,8 @@ func runInitialScan(ctx context.Context) func() error {
if err != nil {
return err
}
scanNeeded := conf.Server.Scanner.ScanOnStartup || inProgress || fullScanRequired == "1" || pidHasChanged
scanOnStartup := conf.Server.Scanner.Enabled && conf.Server.Scanner.ScanOnStartup
scanNeeded := scanOnStartup || inProgress || fullScanRequired == "1" || pidHasChanged
time.Sleep(2 * time.Second) // Wait 2 seconds before the initial scan
if scanNeeded {
s := CreateScanner(ctx)

View File

@@ -8,7 +8,7 @@ import (
"os"
"strings"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/db"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
@@ -74,7 +74,7 @@ func runScanner(ctx context.Context) {
sqlDB := db.Db()
defer db.Db().Close()
ds := persistence.New(sqlDB)
pls := core.NewPlaylists(ds)
pls := playlists.NewPlaylists(ds)
// Parse targets from command line or file
var scanTargets []model.ScanTarget

View File

@@ -18,6 +18,7 @@ import (
"github.com/navidrome/navidrome/core/ffmpeg"
"github.com/navidrome/navidrome/core/metrics"
"github.com/navidrome/navidrome/core/playback"
"github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/core/scrobbler"
"github.com/navidrome/navidrome/db"
"github.com/navidrome/navidrome/model"
@@ -61,7 +62,7 @@ func CreateNativeAPIRouter(ctx context.Context) *nativeapi.Router {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
share := core.NewShare(dataStore)
playlists := core.NewPlaylists(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore)
insights := metrics.GetInstance(dataStore)
fileCache := artwork.GetImageCache()
fFmpeg := ffmpeg.New()
@@ -72,12 +73,12 @@ func CreateNativeAPIRouter(ctx context.Context) *nativeapi.Router {
provider := external.NewProvider(dataStore, agentsAgents)
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, provider)
cacheWarmer := artwork.NewCacheWarmer(artworkArtwork, fileCache)
modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlists, metricsMetrics)
modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlistsPlaylists, metricsMetrics)
watcher := scanner.GetWatcher(dataStore, modelScanner)
library := core.NewLibrary(dataStore, modelScanner, watcher, broker, manager)
user := core.NewUser(dataStore, manager)
maintenance := core.NewMaintenance(dataStore)
router := nativeapi.New(dataStore, share, playlists, insights, library, user, maintenance, manager)
router := nativeapi.New(dataStore, share, playlistsPlaylists, insights, library, user, maintenance, manager)
return router
}
@@ -98,11 +99,11 @@ func CreateSubsonicAPIRouter(ctx context.Context) *subsonic.Router {
archiver := core.NewArchiver(mediaStreamer, dataStore, share)
players := core.NewPlayers(dataStore)
cacheWarmer := artwork.NewCacheWarmer(artworkArtwork, fileCache)
playlists := core.NewPlaylists(dataStore)
modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlists, metricsMetrics)
playlistsPlaylists := playlists.NewPlaylists(dataStore)
modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlistsPlaylists, metricsMetrics)
playTracker := scrobbler.GetPlayTracker(dataStore, broker, manager)
playbackServer := playback.GetInstance(dataStore)
router := subsonic.New(dataStore, artworkArtwork, mediaStreamer, archiver, players, provider, modelScanner, broker, playlists, playTracker, share, playbackServer, metricsMetrics)
router := subsonic.New(dataStore, artworkArtwork, mediaStreamer, archiver, players, provider, modelScanner, broker, playlistsPlaylists, playTracker, share, playbackServer, metricsMetrics)
return router
}
@@ -165,8 +166,8 @@ func CreateScanner(ctx context.Context) model.Scanner {
provider := external.NewProvider(dataStore, agentsAgents)
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, provider)
cacheWarmer := artwork.NewCacheWarmer(artworkArtwork, fileCache)
playlists := core.NewPlaylists(dataStore)
modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlists, metricsMetrics)
playlistsPlaylists := playlists.NewPlaylists(dataStore)
modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlistsPlaylists, metricsMetrics)
return modelScanner
}
@@ -182,8 +183,8 @@ func CreateScanWatcher(ctx context.Context) scanner.Watcher {
provider := external.NewProvider(dataStore, agentsAgents)
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, provider)
cacheWarmer := artwork.NewCacheWarmer(artworkArtwork, fileCache)
playlists := core.NewPlaylists(dataStore)
modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlists, metricsMetrics)
playlistsPlaylists := playlists.NewPlaylists(dataStore)
modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlistsPlaylists, metricsMetrics)
watcher := scanner.GetWatcher(dataStore, modelScanner)
return watcher
}

View File

@@ -1,4 +0,0 @@
package buildtags
// This file is left intentionally empty. It is used to make sure the package is not empty, in the case all
// required build tags are disabled.

6
conf/buildtags/doc.go Normal file
View File

@@ -0,0 +1,6 @@
// Package buildtags provides compile-time enforcement of required build tags.
//
// Each file in this package is guarded by a build constraint and exports a variable
// that main.go references. If a required tag is missing during compilation, the build
// fails with an "undefined" error, directing the developer to use `make build`.
package buildtags

View File

@@ -2,10 +2,6 @@
package buildtags
// NOTICE: This file was created to force the inclusion of the `netgo` tag when compiling the project.
// If the tag is not included, the compilation will fail because this variable won't be defined, and the `main.go`
// file requires it.
// Why this tag is required? See https://github.com/navidrome/navidrome/issues/700
// The `netgo` tag is required when compiling the project. See https://github.com/navidrome/navidrome/issues/700
var NETGO = true

View File

@@ -0,0 +1,8 @@
//go:build sqlite_fts5
package buildtags
// FTS5 is required for full-text search. Without this tag, the SQLite driver
// won't include FTS5 support, causing runtime failures on migrations and search queries.
var SQLITE_FTS5 = true

View File

@@ -58,7 +58,7 @@ type configOptions struct {
SmartPlaylistRefreshDelay time.Duration
AutoTranscodeDownload bool
DefaultDownsamplingFormat string
SearchFullString bool
Search searchOptions `json:",omitzero"`
SimilarSongsMatchThreshold int
RecentlyAddedByModTime bool
PreferSortTags bool
@@ -82,6 +82,7 @@ type configOptions struct {
DefaultTheme string
DefaultLanguage string
DefaultUIVolume int
UISearchDebounceMs int
EnableReplayGain bool
EnableCoverAnimation bool
EnableNowPlaying bool
@@ -154,6 +155,7 @@ type scannerOptions struct {
type subsonicOptions struct {
AppendSubtitle bool
AppendAlbumVersion bool
ArtistParticipations bool
DefaultReportRealPath bool
EnableAverageRating bool
@@ -249,6 +251,12 @@ type pluginsOptions struct {
type extAuthOptions struct {
TrustedSources string
UserHeader string
LogoutURL string
}
type searchOptions struct {
Backend string
FullString bool
}
var (
@@ -339,11 +347,14 @@ func Load(noConfigDump bool) {
validateBackupSchedule,
validatePlaylistsPath,
validatePurgeMissingOption,
validateURL("ExtAuth.LogoutURL", Server.ExtAuth.LogoutURL),
)
if err != nil {
os.Exit(1)
}
Server.Search.Backend = normalizeSearchBackend(Server.Search.Backend)
if Server.BaseURL != "" {
u, err := url.Parse(Server.BaseURL)
if err != nil {
@@ -392,6 +403,7 @@ func Load(noConfigDump bool) {
logDeprecatedOptions("Scanner.GenreSeparators", "")
logDeprecatedOptions("Scanner.GroupAlbumReleases", "")
logDeprecatedOptions("DevEnableBufferedScrobble", "") // Deprecated: Buffered scrobbling is now always enabled and this option is ignored
logDeprecatedOptions("SearchFullString", "Search.FullString")
logDeprecatedOptions("ReverseProxyWhitelist", "ExtAuth.TrustedSources")
logDeprecatedOptions("ReverseProxyUserHeader", "ExtAuth.UserHeader")
logDeprecatedOptions("HTTPSecurityHeaders.CustomFrameOptionsValue", "HTTPHeaders.FrameOptions")
@@ -539,6 +551,44 @@ func validateSchedule(schedule, field string) (string, error) {
return schedule, err
}
// validateURL checks if the provided URL is valid and has either http or https scheme.
// It returns a function that can be used as a hook to validate URLs in the config.
func validateURL(optionName, optionURL string) func() error {
return func() error {
if optionURL == "" {
return nil
}
u, err := url.Parse(optionURL)
if err != nil {
log.Error(fmt.Sprintf("Invalid %s: it could not be parsed", optionName), "url", optionURL, "err", err)
return err
}
if u.Scheme != "http" && u.Scheme != "https" {
err := fmt.Errorf("invalid scheme for %s: '%s'. Only 'http' and 'https' are allowed", optionName, u.Scheme)
log.Error(err.Error())
return err
}
// Require an absolute URL with a non-empty host and no opaque component.
if u.Host == "" || u.Opaque != "" {
err := fmt.Errorf("invalid %s: '%s'. A full http(s) URL with a non-empty host is required", optionName, optionURL)
log.Error(err.Error())
return err
}
return nil
}
}
func normalizeSearchBackend(value string) string {
v := strings.ToLower(strings.TrimSpace(value))
switch v {
case "fts", "legacy":
return v
default:
log.Error("Invalid Search.Backend value, falling back to 'fts'", "value", value)
return "fts"
}
}
// AddHook is used to register initialization code that should run as soon as the config is loaded
func AddHook(hook func()) {
hooks = append(hooks, hook)
@@ -585,7 +635,8 @@ func setViperDefaults() {
viper.SetDefault("enablemediafilecoverart", true)
viper.SetDefault("autotranscodedownload", false)
viper.SetDefault("defaultdownsamplingformat", consts.DefaultDownsamplingFormat)
viper.SetDefault("searchfullstring", false)
viper.SetDefault("search.fullstring", false)
viper.SetDefault("search.backend", "fts")
viper.SetDefault("similarsongsmatchthreshold", 85)
viper.SetDefault("recentlyaddedbymodtime", false)
viper.SetDefault("prefersorttags", false)
@@ -604,6 +655,7 @@ func setViperDefaults() {
viper.SetDefault("defaulttheme", "Dark")
viper.SetDefault("defaultlanguage", "")
viper.SetDefault("defaultuivolume", consts.DefaultUIVolume)
viper.SetDefault("uisearchdebouncems", consts.DefaultUISearchDebounceMs)
viper.SetDefault("enablereplaygain", true)
viper.SetDefault("enablecoveranimation", true)
viper.SetDefault("enablenowplaying", true)
@@ -619,6 +671,7 @@ func setViperDefaults() {
viper.SetDefault("passwordencryptionkey", "")
viper.SetDefault("extauth.userheader", "Remote-User")
viper.SetDefault("extauth.trustedsources", "")
viper.SetDefault("extauth.logouturl", "")
viper.SetDefault("prometheus.enabled", false)
viper.SetDefault("prometheus.metricspath", consts.PrometheusDefaultPath)
viper.SetDefault("prometheus.password", "")
@@ -637,6 +690,7 @@ func setViperDefaults() {
viper.SetDefault("scanner.followsymlinks", true)
viper.SetDefault("scanner.purgemissing", consts.PurgeMissingNever)
viper.SetDefault("subsonic.appendsubtitle", true)
viper.SetDefault("subsonic.appendalbumversion", true)
viper.SetDefault("subsonic.artistparticipations", false)
viper.SetDefault("subsonic.defaultreportrealpath", false)
viper.SetDefault("subsonic.enableaveragerating", true)

View File

@@ -52,6 +52,62 @@ var _ = Describe("Configuration", func() {
})
})
Describe("ValidateURL", func() {
It("accepts a valid http URL", func() {
fn := conf.ValidateURL("TestOption", "http://example.com/path")
Expect(fn()).To(Succeed())
})
It("accepts a valid https URL", func() {
fn := conf.ValidateURL("TestOption", "https://example.com/path")
Expect(fn()).To(Succeed())
})
It("rejects a URL with no scheme", func() {
fn := conf.ValidateURL("TestOption", "example.com/path")
Expect(fn()).To(MatchError(ContainSubstring("invalid scheme")))
})
It("rejects a URL with an unsupported scheme", func() {
fn := conf.ValidateURL("TestOption", "javascript://example.com/path")
Expect(fn()).To(MatchError(ContainSubstring("invalid scheme")))
})
It("accepts an empty URL (optional config)", func() {
fn := conf.ValidateURL("TestOption", "")
Expect(fn()).To(Succeed())
})
It("includes the option name in the error message", func() {
fn := conf.ValidateURL("MyOption", "ftp://example.com")
Expect(fn()).To(MatchError(ContainSubstring("MyOption")))
})
It("rejects a URL that cannot be parsed", func() {
fn := conf.ValidateURL("TestOption", "://invalid")
Expect(fn()).To(HaveOccurred())
})
It("rejects a URL without a host", func() {
fn := conf.ValidateURL("TestOption", "http:///path")
Expect(fn()).To(MatchError(ContainSubstring("non-empty host is required")))
})
})
DescribeTable("NormalizeSearchBackend",
func(input, expected string) {
Expect(conf.NormalizeSearchBackend(input)).To(Equal(expected))
},
Entry("accepts 'fts'", "fts", "fts"),
Entry("accepts 'legacy'", "legacy", "legacy"),
Entry("normalizes 'FTS' to lowercase", "FTS", "fts"),
Entry("normalizes 'Legacy' to lowercase", "Legacy", "legacy"),
Entry("trims whitespace", " fts ", "fts"),
Entry("falls back to 'fts' for 'fts5'", "fts5", "fts"),
Entry("falls back to 'fts' for unrecognized values", "invalid", "fts"),
Entry("falls back to 'fts' for empty string", "", "fts"),
)
DescribeTable("should load configuration from",
func(format string) {
filename := filepath.Join("testdata", "cfg."+format)

View File

@@ -7,3 +7,7 @@ func ResetConf() {
var SetViperDefaults = setViperDefaults
var ParseLanguages = parseLanguages
var ValidateURL = validateURL
var NormalizeSearchBackend = normalizeSearchBackend

View File

@@ -66,11 +66,12 @@ const (
I18nFolder = "i18n"
ScanIgnoreFile = ".ndignore"
PlaceholderArtistArt = "artist-placeholder.webp"
PlaceholderAlbumArt = "album-placeholder.webp"
PlaceholderAvatar = "logo-192x192.png"
UICoverArtSize = 300
DefaultUIVolume = 100
PlaceholderArtistArt = "artist-placeholder.webp"
PlaceholderAlbumArt = "album-placeholder.webp"
PlaceholderAvatar = "logo-192x192.png"
UICoverArtSize = 300
DefaultUIVolume = 100
DefaultUISearchDebounceMs = 200
DefaultHttpClientTimeOut = 10 * time.Second

View File

@@ -208,7 +208,8 @@ var staticData = sync.OnceValue(func() insights.Data {
data.Config.TranscodingCacheSize = conf.Server.TranscodingCacheSize
data.Config.ImageCacheSize = conf.Server.ImageCacheSize
data.Config.SessionTimeout = uint64(math.Trunc(conf.Server.SessionTimeout.Seconds()))
data.Config.SearchFullString = conf.Server.SearchFullString
data.Config.SearchFullString = conf.Server.Search.FullString
data.Config.SearchBackend = conf.Server.Search.Backend
data.Config.RecentlyAddedByModTime = conf.Server.RecentlyAddedByModTime
data.Config.PreferSortTags = conf.Server.PreferSortTags
data.Config.BackupSchedule = conf.Server.Backup.Schedule

View File

@@ -68,6 +68,7 @@ type Data struct {
EnableNowPlaying bool `json:"enableNowPlaying,omitempty"`
SessionTimeout uint64 `json:"sessionTimeout,omitempty"`
SearchFullString bool `json:"searchFullString,omitempty"`
SearchBackend string `json:"searchBackend,omitempty"`
RecentlyAddedByModTime bool `json:"recentlyAddedByModTime,omitempty"`
PreferSortTags bool `json:"preferSortTags,omitempty"`
BackupSchedule string `json:"backupSchedule,omitempty"`

119
core/playlists/import.go Normal file
View File

@@ -0,0 +1,119 @@
package playlists
import (
"context"
"errors"
"io"
"os"
"path/filepath"
"strings"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/utils/ioutils"
"golang.org/x/text/unicode/norm"
)
func (s *playlists) ImportFile(ctx context.Context, folder *model.Folder, filename string) (*model.Playlist, error) {
pls, err := s.parsePlaylist(ctx, filename, folder)
if err != nil {
log.Error(ctx, "Error parsing playlist", "path", filepath.Join(folder.AbsolutePath(), filename), err)
return nil, err
}
log.Debug(ctx, "Found playlist", "name", pls.Name, "lastUpdated", pls.UpdatedAt, "path", pls.Path, "numTracks", len(pls.Tracks))
err = s.updatePlaylist(ctx, pls)
if err != nil {
log.Error(ctx, "Error updating playlist", "path", filepath.Join(folder.AbsolutePath(), filename), err)
}
return pls, err
}
func (s *playlists) ImportM3U(ctx context.Context, reader io.Reader) (*model.Playlist, error) {
owner, _ := request.UserFrom(ctx)
pls := &model.Playlist{
OwnerID: owner.ID,
Public: false,
Sync: false,
}
err := s.parseM3U(ctx, pls, nil, reader)
if err != nil {
log.Error(ctx, "Error parsing playlist", err)
return nil, err
}
err = s.ds.Playlist(ctx).Put(pls)
if err != nil {
log.Error(ctx, "Error saving playlist", err)
return nil, err
}
return pls, nil
}
func (s *playlists) parsePlaylist(ctx context.Context, playlistFile string, folder *model.Folder) (*model.Playlist, error) {
pls, err := s.newSyncedPlaylist(folder.AbsolutePath(), playlistFile)
if err != nil {
return nil, err
}
file, err := os.Open(pls.Path)
if err != nil {
return nil, err
}
defer file.Close()
reader := ioutils.UTF8Reader(file)
extension := strings.ToLower(filepath.Ext(playlistFile))
switch extension {
case ".nsp":
err = s.parseNSP(ctx, pls, reader)
default:
err = s.parseM3U(ctx, pls, folder, reader)
}
return pls, err
}
func (s *playlists) updatePlaylist(ctx context.Context, newPls *model.Playlist) error {
owner, _ := request.UserFrom(ctx)
// Try to find existing playlist by path. Since filesystem normalization differs across
// platforms (macOS uses NFD, Linux/Windows use NFC), we try both forms to match
// playlists that may have been imported on a different platform.
pls, err := s.ds.Playlist(ctx).FindByPath(newPls.Path)
if errors.Is(err, model.ErrNotFound) {
// Try alternate normalization form
altPath := norm.NFD.String(newPls.Path)
if altPath == newPls.Path {
altPath = norm.NFC.String(newPls.Path)
}
if altPath != newPls.Path {
pls, err = s.ds.Playlist(ctx).FindByPath(altPath)
}
}
if err != nil && !errors.Is(err, model.ErrNotFound) {
return err
}
if err == nil && !pls.Sync {
log.Debug(ctx, "Playlist already imported and not synced", "playlist", pls.Name, "path", pls.Path)
return nil
}
if err == nil {
log.Info(ctx, "Updating synced playlist", "playlist", pls.Name, "path", newPls.Path)
newPls.ID = pls.ID
newPls.Name = pls.Name
newPls.Comment = pls.Comment
newPls.OwnerID = pls.OwnerID
newPls.Public = pls.Public
newPls.EvaluatedAt = &time.Time{}
} else {
log.Info(ctx, "Adding synced playlist", "playlist", newPls.Name, "path", newPls.Path, "owner", owner.UserName)
newPls.OwnerID = owner.ID
// For NSP files, Public may already be set from the file; for M3U, use server default
if !newPls.IsSmartPlaylist() {
newPls.Public = conf.Server.DefaultPlaylistPublicVisibility
}
}
return s.ds.Playlist(ctx).Put(newPls)
}

View File

@@ -1,4 +1,4 @@
package core_test
package playlists_test
import (
"context"
@@ -9,7 +9,7 @@ import (
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/criteria"
"github.com/navidrome/navidrome/model/request"
@@ -19,18 +19,18 @@ import (
"golang.org/x/text/unicode/norm"
)
var _ = Describe("Playlists", func() {
var _ = Describe("Playlists - Import", func() {
var ds *tests.MockDataStore
var ps core.Playlists
var mockPlsRepo mockedPlaylistRepo
var ps playlists.Playlists
var mockPlsRepo *tests.MockPlaylistRepo
var mockLibRepo *tests.MockLibraryRepo
ctx := context.Background()
BeforeEach(func() {
mockPlsRepo = mockedPlaylistRepo{}
mockPlsRepo = tests.CreateMockPlaylistRepo()
mockLibRepo = &tests.MockLibraryRepo{}
ds = &tests.MockDataStore{
MockedPlaylist: &mockPlsRepo,
MockedPlaylist: mockPlsRepo,
MockedLibrary: mockLibRepo,
}
ctx = request.WithUser(ctx, model.User{ID: "123"})
@@ -39,7 +39,7 @@ var _ = Describe("Playlists", func() {
Describe("ImportFile", func() {
var folder *model.Folder
BeforeEach(func() {
ps = core.NewPlaylists(ds)
ps = playlists.NewPlaylists(ds)
ds.MockedMediaFile = &mockedMediaFileRepo{}
libPath, _ := os.Getwd()
// Set up library with the actual library path that matches the folder
@@ -61,7 +61,7 @@ var _ = Describe("Playlists", func() {
Expect(pls.Tracks).To(HaveLen(2))
Expect(pls.Tracks[0].Path).To(Equal("tests/fixtures/playlists/test.mp3"))
Expect(pls.Tracks[1].Path).To(Equal("tests/fixtures/playlists/test.ogg"))
Expect(mockPlsRepo.last).To(Equal(pls))
Expect(mockPlsRepo.Last).To(Equal(pls))
})
It("parses playlists using LF ending", func() {
@@ -99,7 +99,7 @@ var _ = Describe("Playlists", func() {
It("parses well-formed playlists", func() {
pls, err := ps.ImportFile(ctx, folder, "recently_played.nsp")
Expect(err).ToNot(HaveOccurred())
Expect(mockPlsRepo.last).To(Equal(pls))
Expect(mockPlsRepo.Last).To(Equal(pls))
Expect(pls.OwnerID).To(Equal("123"))
Expect(pls.Name).To(Equal("Recently Played"))
Expect(pls.Comment).To(Equal("Recently played tracks"))
@@ -149,7 +149,7 @@ var _ = Describe("Playlists", func() {
tmpDir := GinkgoT().TempDir()
mockLibRepo.SetData([]model.Library{{ID: 1, Path: tmpDir}})
ds.MockedMediaFile = &mockedMediaFileFromListRepo{data: []string{}}
ps = core.NewPlaylists(ds)
ps = playlists.NewPlaylists(ds)
// Create the playlist file on disk with the filesystem's normalization form
plsFile := tmpDir + "/" + filesystemName + ".m3u"
@@ -163,7 +163,7 @@ var _ = Describe("Playlists", func() {
Path: storedPath,
Sync: true,
}
mockPlsRepo.data = map[string]*model.Playlist{storedPath: existingPls}
mockPlsRepo.PathMap = map[string]*model.Playlist{storedPath: existingPls}
// Import using the filesystem's normalization form
plsFolder := &model.Folder{
@@ -209,7 +209,7 @@ var _ = Describe("Playlists", func() {
"def.mp3", // This is playlists/def.mp3 relative to plsDir
},
}
ps = core.NewPlaylists(ds)
ps = playlists.NewPlaylists(ds)
})
It("handles relative paths that reference files in other libraries", func() {
@@ -365,7 +365,7 @@ var _ = Describe("Playlists", func() {
},
}
// Recreate playlists service to pick up new mock
ps = core.NewPlaylists(ds)
ps = playlists.NewPlaylists(ds)
// Create playlist in music library that references both tracks
plsContent := "#PLAYLIST:Same Path Test\nalbum/track.mp3\n../classical/album/track.mp3"
@@ -408,7 +408,7 @@ var _ = Describe("Playlists", func() {
BeforeEach(func() {
repo = &mockedMediaFileFromListRepo{}
ds.MockedMediaFile = repo
ps = core.NewPlaylists(ds)
ps = playlists.NewPlaylists(ds)
mockLibRepo.SetData([]model.Library{{ID: 1, Path: "/music"}, {ID: 2, Path: "/new"}})
ctx = request.WithUser(ctx, model.User{ID: "123"})
})
@@ -439,7 +439,7 @@ var _ = Describe("Playlists", func() {
Expect(pls.Tracks[1].Path).To(Equal("tests/test.ogg"))
Expect(pls.Tracks[2].Path).To(Equal("downloads/newfile.flac"))
Expect(pls.Tracks[3].Path).To(Equal("tests/01 Invisible (RED) Edit Version.mp3"))
Expect(mockPlsRepo.last).To(Equal(pls))
Expect(mockPlsRepo.Last).To(Equal(pls))
})
It("sets the playlist name as a timestamp if the #PLAYLIST directive is not present", func() {
@@ -460,7 +460,7 @@ var _ = Describe("Playlists", func() {
Expect(pls.Tracks).To(HaveLen(2))
})
It("returns only tracks that exist in the database and in the same other as the m3u", func() {
It("returns only tracks that exist in the database and in the same order as the m3u", func() {
repo.data = []string{
"album1/test1.mp3",
"album2/test2.mp3",
@@ -570,7 +570,7 @@ var _ = Describe("Playlists", func() {
})
Describe("InPlaylistsPath", func() {
Describe("InPath", func() {
var folder model.Folder
BeforeEach(func() {
@@ -584,27 +584,27 @@ var _ = Describe("Playlists", func() {
It("returns true if PlaylistsPath is empty", func() {
conf.Server.PlaylistsPath = ""
Expect(core.InPlaylistsPath(folder)).To(BeTrue())
Expect(playlists.InPath(folder)).To(BeTrue())
})
It("returns true if PlaylistsPath is any (**/**)", func() {
conf.Server.PlaylistsPath = "**/**"
Expect(core.InPlaylistsPath(folder)).To(BeTrue())
Expect(playlists.InPath(folder)).To(BeTrue())
})
It("returns true if folder is in PlaylistsPath", func() {
conf.Server.PlaylistsPath = "other/**:playlists/**"
Expect(core.InPlaylistsPath(folder)).To(BeTrue())
Expect(playlists.InPath(folder)).To(BeTrue())
})
It("returns false if folder is not in PlaylistsPath", func() {
conf.Server.PlaylistsPath = "other"
Expect(core.InPlaylistsPath(folder)).To(BeFalse())
Expect(playlists.InPath(folder)).To(BeFalse())
})
It("returns true if for a playlist in root of MusicFolder if PlaylistsPath is '.'", func() {
conf.Server.PlaylistsPath = "."
Expect(core.InPlaylistsPath(folder)).To(BeFalse())
Expect(playlists.InPath(folder)).To(BeFalse())
folder2 := model.Folder{
LibraryPath: "/music",
@@ -612,7 +612,7 @@ var _ = Describe("Playlists", func() {
Name: ".",
}
Expect(core.InPlaylistsPath(folder2)).To(BeTrue())
Expect(playlists.InPath(folder2)).To(BeTrue())
})
})
})
@@ -693,23 +693,3 @@ func (r *mockedMediaFileFromListRepo) FindByPaths(paths []string) (model.MediaFi
}
return mfs, nil
}
type mockedPlaylistRepo struct {
last *model.Playlist
data map[string]*model.Playlist // keyed by path
model.PlaylistRepository
}
func (r *mockedPlaylistRepo) FindByPath(path string) (*model.Playlist, error) {
if r.data != nil {
if pls, ok := r.data[path]; ok {
return pls, nil
}
}
return nil, model.ErrNotFound
}
func (r *mockedPlaylistRepo) Put(pls *model.Playlist) error {
r.last = pls
return nil
}

View File

@@ -1,183 +1,28 @@
package core
package playlists
import (
"cmp"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/url"
"os"
"path/filepath"
"slices"
"strings"
"time"
"github.com/RaveNoX/go-jsoncommentstrip"
"github.com/bmatcuk/doublestar/v4"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/criteria"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/utils/ioutils"
"github.com/navidrome/navidrome/utils/slice"
"golang.org/x/text/unicode/norm"
)
type Playlists interface {
ImportFile(ctx context.Context, folder *model.Folder, filename string) (*model.Playlist, error)
Update(ctx context.Context, playlistID string, name *string, comment *string, public *bool, idsToAdd []string, idxToRemove []int) error
ImportM3U(ctx context.Context, reader io.Reader) (*model.Playlist, error)
}
type playlists struct {
ds model.DataStore
}
func NewPlaylists(ds model.DataStore) Playlists {
return &playlists{ds: ds}
}
func InPlaylistsPath(folder model.Folder) bool {
if conf.Server.PlaylistsPath == "" {
return true
}
rel, _ := filepath.Rel(folder.LibraryPath, folder.AbsolutePath())
for path := range strings.SplitSeq(conf.Server.PlaylistsPath, string(filepath.ListSeparator)) {
if match, _ := doublestar.Match(path, rel); match {
return true
}
}
return false
}
func (s *playlists) ImportFile(ctx context.Context, folder *model.Folder, filename string) (*model.Playlist, error) {
pls, err := s.parsePlaylist(ctx, filename, folder)
if err != nil {
log.Error(ctx, "Error parsing playlist", "path", filepath.Join(folder.AbsolutePath(), filename), err)
return nil, err
}
log.Debug("Found playlist", "name", pls.Name, "lastUpdated", pls.UpdatedAt, "path", pls.Path, "numTracks", len(pls.Tracks))
err = s.updatePlaylist(ctx, pls)
if err != nil {
log.Error(ctx, "Error updating playlist", "path", filepath.Join(folder.AbsolutePath(), filename), err)
}
return pls, err
}
func (s *playlists) ImportM3U(ctx context.Context, reader io.Reader) (*model.Playlist, error) {
owner, _ := request.UserFrom(ctx)
pls := &model.Playlist{
OwnerID: owner.ID,
Public: false,
Sync: false,
}
err := s.parseM3U(ctx, pls, nil, reader)
if err != nil {
log.Error(ctx, "Error parsing playlist", err)
return nil, err
}
err = s.ds.Playlist(ctx).Put(pls)
if err != nil {
log.Error(ctx, "Error saving playlist", err)
return nil, err
}
return pls, nil
}
func (s *playlists) parsePlaylist(ctx context.Context, playlistFile string, folder *model.Folder) (*model.Playlist, error) {
pls, err := s.newSyncedPlaylist(folder.AbsolutePath(), playlistFile)
if err != nil {
return nil, err
}
file, err := os.Open(pls.Path)
if err != nil {
return nil, err
}
defer file.Close()
reader := ioutils.UTF8Reader(file)
extension := strings.ToLower(filepath.Ext(playlistFile))
switch extension {
case ".nsp":
err = s.parseNSP(ctx, pls, reader)
default:
err = s.parseM3U(ctx, pls, folder, reader)
}
return pls, err
}
func (s *playlists) newSyncedPlaylist(baseDir string, playlistFile string) (*model.Playlist, error) {
playlistPath := filepath.Join(baseDir, playlistFile)
info, err := os.Stat(playlistPath)
if err != nil {
return nil, err
}
var extension = filepath.Ext(playlistFile)
var name = playlistFile[0 : len(playlistFile)-len(extension)]
pls := &model.Playlist{
Name: name,
Comment: fmt.Sprintf("Auto-imported from '%s'", playlistFile),
Public: false,
Path: playlistPath,
Sync: true,
UpdatedAt: info.ModTime(),
}
return pls, nil
}
func getPositionFromOffset(data []byte, offset int64) (line, column int) {
line = 1
for _, b := range data[:offset] {
if b == '\n' {
line++
column = 1
} else {
column++
}
}
return
}
func (s *playlists) parseNSP(_ context.Context, pls *model.Playlist, reader io.Reader) error {
nsp := &nspFile{}
reader = io.LimitReader(reader, 100*1024) // Limit to 100KB
reader = jsoncommentstrip.NewReader(reader)
input, err := io.ReadAll(reader)
if err != nil {
return fmt.Errorf("reading SmartPlaylist: %w", err)
}
err = json.Unmarshal(input, nsp)
if err != nil {
var syntaxErr *json.SyntaxError
if errors.As(err, &syntaxErr) {
line, col := getPositionFromOffset(input, syntaxErr.Offset)
return fmt.Errorf("JSON syntax error in SmartPlaylist at line %d, column %d: %w", line, col, err)
}
return fmt.Errorf("JSON parsing error in SmartPlaylist: %w", err)
}
pls.Rules = &nsp.Criteria
if nsp.Name != "" {
pls.Name = nsp.Name
}
if nsp.Comment != "" {
pls.Comment = nsp.Comment
}
if nsp.Public != nil {
pls.Public = *nsp.Public
} else {
pls.Public = conf.Server.DefaultPlaylistPublicVisibility
}
return nil
}
func (s *playlists) parseM3U(ctx context.Context, pls *model.Playlist, folder *model.Folder, reader io.Reader) error {
mediaFileRepository := s.ds.MediaFile(ctx)
resolver, err := newPathResolver(ctx, s.ds)
if err != nil {
return err
}
var mfs model.MediaFiles
// Chunk size of 100 lines, as each line can generate up to 4 lookup candidates
// (NFC/NFD × raw/lowercase), and SQLite has a max expression tree depth of 1000.
@@ -202,7 +47,7 @@ func (s *playlists) parseM3U(ctx context.Context, pls *model.Playlist, folder *m
}
filteredLines = append(filteredLines, line)
}
resolvedPaths, err := s.resolvePaths(ctx, folder, filteredLines)
resolvedPaths, err := resolver.resolvePaths(ctx, folder, filteredLines)
if err != nil {
log.Warn(ctx, "Error resolving paths in playlist", "playlist", pls.Name, err)
continue
@@ -258,7 +103,9 @@ func (s *playlists) parseM3U(ctx context.Context, pls *model.Playlist, folder *m
existing[key] = idx
}
// Find media files in the order of the resolved paths, to keep playlist order
// Find media files in the order of the resolved paths, to keep playlist order.
// Both `existing` keys and `resolvedPaths` use the library-qualified format "libraryID:relativePath",
// so normalizing the full string produces matching keys (digits and ':' are ASCII-invariant).
for _, path := range resolvedPaths {
key := strings.ToLower(norm.NFC.String(path))
idx, ok := existing[key]
@@ -398,15 +245,10 @@ func (r *pathResolver) findInLibraries(absolutePath string) pathResolution {
// resolvePaths converts playlist file paths to library-qualified paths (format: "libraryID:relativePath").
// For relative paths, it resolves them to absolute paths first, then determines which
// library they belong to. This allows playlists to reference files across library boundaries.
func (s *playlists) resolvePaths(ctx context.Context, folder *model.Folder, lines []string) ([]string, error) {
resolver, err := newPathResolver(ctx, s.ds)
if err != nil {
return nil, err
}
func (r *pathResolver) resolvePaths(ctx context.Context, folder *model.Folder, lines []string) ([]string, error) {
results := make([]string, 0, len(lines))
for idx, line := range lines {
resolution := resolver.resolvePath(line, folder)
resolution := r.resolvePath(line, folder)
if !resolution.valid {
log.Warn(ctx, "Path in playlist not found in any library", "path", line, "line", idx)
@@ -425,123 +267,3 @@ func (s *playlists) resolvePaths(ctx context.Context, folder *model.Folder, line
return results, nil
}
func (s *playlists) updatePlaylist(ctx context.Context, newPls *model.Playlist) error {
owner, _ := request.UserFrom(ctx)
// Try to find existing playlist by path. Since filesystem normalization differs across
// platforms (macOS uses NFD, Linux/Windows use NFC), we try both forms to match
// playlists that may have been imported on a different platform.
pls, err := s.ds.Playlist(ctx).FindByPath(newPls.Path)
if errors.Is(err, model.ErrNotFound) {
// Try alternate normalization form
altPath := norm.NFD.String(newPls.Path)
if altPath == newPls.Path {
altPath = norm.NFC.String(newPls.Path)
}
if altPath != newPls.Path {
pls, err = s.ds.Playlist(ctx).FindByPath(altPath)
}
}
if err != nil && !errors.Is(err, model.ErrNotFound) {
return err
}
if err == nil && !pls.Sync {
log.Debug(ctx, "Playlist already imported and not synced", "playlist", pls.Name, "path", pls.Path)
return nil
}
if err == nil {
log.Info(ctx, "Updating synced playlist", "playlist", pls.Name, "path", newPls.Path)
newPls.ID = pls.ID
newPls.Name = pls.Name
newPls.Comment = pls.Comment
newPls.OwnerID = pls.OwnerID
newPls.Public = pls.Public
newPls.EvaluatedAt = &time.Time{}
} else {
log.Info(ctx, "Adding synced playlist", "playlist", newPls.Name, "path", newPls.Path, "owner", owner.UserName)
newPls.OwnerID = owner.ID
// For NSP files, Public may already be set from the file; for M3U, use server default
if !newPls.IsSmartPlaylist() {
newPls.Public = conf.Server.DefaultPlaylistPublicVisibility
}
}
return s.ds.Playlist(ctx).Put(newPls)
}
func (s *playlists) Update(ctx context.Context, playlistID string,
name *string, comment *string, public *bool,
idsToAdd []string, idxToRemove []int) error {
needsInfoUpdate := name != nil || comment != nil || public != nil
needsTrackRefresh := len(idxToRemove) > 0
return s.ds.WithTxImmediate(func(tx model.DataStore) error {
var pls *model.Playlist
var err error
repo := tx.Playlist(ctx)
tracks := repo.Tracks(playlistID, true)
if tracks == nil {
return fmt.Errorf("%w: playlist '%s'", model.ErrNotFound, playlistID)
}
if needsTrackRefresh {
pls, err = repo.GetWithTracks(playlistID, true, false)
pls.RemoveTracks(idxToRemove)
pls.AddMediaFilesByID(idsToAdd)
} else {
if len(idsToAdd) > 0 {
_, err = tracks.Add(idsToAdd)
if err != nil {
return err
}
}
if needsInfoUpdate {
pls, err = repo.Get(playlistID)
}
}
if err != nil {
return err
}
if !needsTrackRefresh && !needsInfoUpdate {
return nil
}
if name != nil {
pls.Name = *name
}
if comment != nil {
pls.Comment = *comment
}
if public != nil {
pls.Public = *public
}
// Special case: The playlist is now empty
if len(idxToRemove) > 0 && len(pls.Tracks) == 0 {
if err = tracks.DeleteAll(); err != nil {
return err
}
}
return repo.Put(pls)
})
}
type nspFile struct {
criteria.Criteria
Name string `json:"name"`
Comment string `json:"comment"`
Public *bool `json:"public"`
}
func (i *nspFile) UnmarshalJSON(data []byte) error {
m := map[string]any{}
err := json.Unmarshal(data, &m)
if err != nil {
return err
}
i.Name, _ = m["name"].(string)
i.Comment, _ = m["comment"].(string)
if public, ok := m["public"].(bool); ok {
i.Public = &public
}
return json.Unmarshal(data, &i.Criteria)
}

View File

@@ -1,4 +1,4 @@
package core
package playlists
import (
"context"
@@ -214,38 +214,38 @@ var _ = Describe("pathResolver", func() {
})
Describe("resolvePath", func() {
It("resolves absolute paths", func() {
resolution := resolver.resolvePath("/music/artist/album/track.mp3", nil)
Context("basic", func() {
It("resolves absolute paths", func() {
resolution := resolver.resolvePath("/music/artist/album/track.mp3", nil)
Expect(resolution.valid).To(BeTrue())
Expect(resolution.libraryID).To(Equal(1))
Expect(resolution.libraryPath).To(Equal("/music"))
Expect(resolution.absolutePath).To(Equal("/music/artist/album/track.mp3"))
Expect(resolution.valid).To(BeTrue())
Expect(resolution.libraryID).To(Equal(1))
Expect(resolution.libraryPath).To(Equal("/music"))
Expect(resolution.absolutePath).To(Equal("/music/artist/album/track.mp3"))
})
It("resolves relative paths when folder is provided", func() {
folder := &model.Folder{
Path: "playlists",
LibraryPath: "/music",
LibraryID: 1,
}
resolution := resolver.resolvePath("../artist/album/track.mp3", folder)
Expect(resolution.valid).To(BeTrue())
Expect(resolution.libraryID).To(Equal(1))
Expect(resolution.absolutePath).To(Equal("/music/artist/album/track.mp3"))
})
It("returns invalid resolution for paths outside any library", func() {
resolution := resolver.resolvePath("/outside/library/track.mp3", nil)
Expect(resolution.valid).To(BeFalse())
})
})
It("resolves relative paths when folder is provided", func() {
folder := &model.Folder{
Path: "playlists",
LibraryPath: "/music",
LibraryID: 1,
}
resolution := resolver.resolvePath("../artist/album/track.mp3", folder)
Expect(resolution.valid).To(BeTrue())
Expect(resolution.libraryID).To(Equal(1))
Expect(resolution.absolutePath).To(Equal("/music/artist/album/track.mp3"))
})
It("returns invalid resolution for paths outside any library", func() {
resolution := resolver.resolvePath("/outside/library/track.mp3", nil)
Expect(resolution.valid).To(BeFalse())
})
})
Describe("resolvePath", func() {
Context("With absolute paths", func() {
Context("cross-library", func() {
It("resolves path within a library", func() {
resolution := resolver.resolvePath("/music/track.mp3", nil)

103
core/playlists/parse_nsp.go Normal file
View File

@@ -0,0 +1,103 @@
package playlists
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"github.com/RaveNoX/go-jsoncommentstrip"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/criteria"
)
func (s *playlists) newSyncedPlaylist(baseDir string, playlistFile string) (*model.Playlist, error) {
playlistPath := filepath.Join(baseDir, playlistFile)
info, err := os.Stat(playlistPath)
if err != nil {
return nil, err
}
var extension = filepath.Ext(playlistFile)
var name = playlistFile[0 : len(playlistFile)-len(extension)]
pls := &model.Playlist{
Name: name,
Comment: fmt.Sprintf("Auto-imported from '%s'", playlistFile),
Public: false,
Path: playlistPath,
Sync: true,
UpdatedAt: info.ModTime(),
}
return pls, nil
}
func getPositionFromOffset(data []byte, offset int64) (line, column int) {
line = 1
for _, b := range data[:offset] {
if b == '\n' {
line++
column = 1
} else {
column++
}
}
return
}
func (s *playlists) parseNSP(_ context.Context, pls *model.Playlist, reader io.Reader) error {
nsp := &nspFile{}
reader = io.LimitReader(reader, 100*1024) // Limit to 100KB
reader = jsoncommentstrip.NewReader(reader)
input, err := io.ReadAll(reader)
if err != nil {
return fmt.Errorf("reading SmartPlaylist: %w", err)
}
err = json.Unmarshal(input, nsp)
if err != nil {
var syntaxErr *json.SyntaxError
if errors.As(err, &syntaxErr) {
line, col := getPositionFromOffset(input, syntaxErr.Offset)
return fmt.Errorf("JSON syntax error in SmartPlaylist at line %d, column %d: %w", line, col, err)
}
return fmt.Errorf("JSON parsing error in SmartPlaylist: %w", err)
}
pls.Rules = &nsp.Criteria
if nsp.Name != "" {
pls.Name = nsp.Name
}
if nsp.Comment != "" {
pls.Comment = nsp.Comment
}
if nsp.Public != nil {
pls.Public = *nsp.Public
} else {
pls.Public = conf.Server.DefaultPlaylistPublicVisibility
}
return nil
}
type nspFile struct {
criteria.Criteria
Name string `json:"name"`
Comment string `json:"comment"`
Public *bool `json:"public"`
}
func (i *nspFile) UnmarshalJSON(data []byte) error {
m := map[string]any{}
err := json.Unmarshal(data, &m)
if err != nil {
return err
}
i.Name, _ = m["name"].(string)
i.Comment, _ = m["comment"].(string)
if public, ok := m["public"].(bool); ok {
i.Public = &public
}
return json.Unmarshal(data, &i.Criteria)
}

View File

@@ -0,0 +1,213 @@
package playlists
import (
"context"
"os"
"path/filepath"
"strings"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/criteria"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("parseNSP", func() {
var s *playlists
ctx := context.Background()
BeforeEach(func() {
s = &playlists{}
})
It("parses a well-formed NSP with all fields", func() {
nsp := `{
"name": "My Smart Playlist",
"comment": "A test playlist",
"public": true,
"all": [{"is": {"loved": true}}],
"sort": "title",
"order": "asc",
"limit": 50
}`
pls := &model.Playlist{Name: "default-name"}
err := s.parseNSP(ctx, pls, strings.NewReader(nsp))
Expect(err).ToNot(HaveOccurred())
Expect(pls.Name).To(Equal("My Smart Playlist"))
Expect(pls.Comment).To(Equal("A test playlist"))
Expect(pls.Public).To(BeTrue())
Expect(pls.Rules).ToNot(BeNil())
Expect(pls.Rules.Sort).To(Equal("title"))
Expect(pls.Rules.Order).To(Equal("asc"))
Expect(pls.Rules.Limit).To(Equal(50))
Expect(pls.Rules.Expression).To(BeAssignableToTypeOf(criteria.All{}))
})
It("keeps existing name when NSP has no name field", func() {
nsp := `{"all": [{"is": {"loved": true}}]}`
pls := &model.Playlist{Name: "Original Name"}
err := s.parseNSP(ctx, pls, strings.NewReader(nsp))
Expect(err).ToNot(HaveOccurred())
Expect(pls.Name).To(Equal("Original Name"))
})
It("keeps existing comment when NSP has no comment field", func() {
nsp := `{"all": [{"is": {"loved": true}}]}`
pls := &model.Playlist{Comment: "Original Comment"}
err := s.parseNSP(ctx, pls, strings.NewReader(nsp))
Expect(err).ToNot(HaveOccurred())
Expect(pls.Comment).To(Equal("Original Comment"))
})
It("strips JSON comments before parsing", func() {
nsp := `{
// Line comment
"name": "Commented Playlist",
/* Block comment */
"all": [{"is": {"loved": true}}]
}`
pls := &model.Playlist{}
err := s.parseNSP(ctx, pls, strings.NewReader(nsp))
Expect(err).ToNot(HaveOccurred())
Expect(pls.Name).To(Equal("Commented Playlist"))
})
It("uses server default when public field is absent", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.DefaultPlaylistPublicVisibility = true
nsp := `{"all": [{"is": {"loved": true}}]}`
pls := &model.Playlist{}
err := s.parseNSP(ctx, pls, strings.NewReader(nsp))
Expect(err).ToNot(HaveOccurred())
Expect(pls.Public).To(BeTrue())
})
It("honors explicit public: false over server default", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.DefaultPlaylistPublicVisibility = true
nsp := `{"public": false, "all": [{"is": {"loved": true}}]}`
pls := &model.Playlist{}
err := s.parseNSP(ctx, pls, strings.NewReader(nsp))
Expect(err).ToNot(HaveOccurred())
Expect(pls.Public).To(BeFalse())
})
It("returns a syntax error with line and column info", func() {
nsp := "{\n \"name\": \"Bad\",\n \"all\": [INVALID]\n}"
pls := &model.Playlist{}
err := s.parseNSP(ctx, pls, strings.NewReader(nsp))
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("JSON syntax error in SmartPlaylist"))
Expect(err.Error()).To(MatchRegexp(`line \d+, column \d+`))
})
It("returns a parsing error for completely invalid JSON", func() {
nsp := `not json at all`
pls := &model.Playlist{}
err := s.parseNSP(ctx, pls, strings.NewReader(nsp))
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("SmartPlaylist"))
})
It("gracefully handles non-string name field", func() {
nsp := `{"name": 123, "all": [{"is": {"loved": true}}]}`
pls := &model.Playlist{Name: "Original"}
err := s.parseNSP(ctx, pls, strings.NewReader(nsp))
Expect(err).ToNot(HaveOccurred())
// Type assertion in UnmarshalJSON fails silently; name stays as original
Expect(pls.Name).To(Equal("Original"))
})
It("parses criteria with multiple rules", func() {
nsp := `{
"all": [
{"is": {"loved": true}},
{"contains": {"title": "rock"}}
],
"sort": "lastPlayed",
"order": "desc",
"limit": 100
}`
pls := &model.Playlist{}
err := s.parseNSP(ctx, pls, strings.NewReader(nsp))
Expect(err).ToNot(HaveOccurred())
Expect(pls.Rules).ToNot(BeNil())
Expect(pls.Rules.Sort).To(Equal("lastPlayed"))
Expect(pls.Rules.Order).To(Equal("desc"))
Expect(pls.Rules.Limit).To(Equal(100))
})
})
var _ = Describe("getPositionFromOffset", func() {
It("returns correct position on first line", func() {
data := []byte("hello world")
line, col := getPositionFromOffset(data, 5)
Expect(line).To(Equal(1))
Expect(col).To(Equal(5))
})
It("returns correct position after newlines", func() {
data := []byte("line1\nline2\nline3")
// Offsets: l(0) i(1) n(2) e(3) 1(4) \n(5) l(6) i(7) n(8)
line, col := getPositionFromOffset(data, 8)
Expect(line).To(Equal(2))
Expect(col).To(Equal(3))
})
It("returns correct position at start of new line", func() {
data := []byte("line1\nline2")
// After \n at offset 5, col resets to 1; offset 6 is 'l' -> col=1
line, col := getPositionFromOffset(data, 6)
Expect(line).To(Equal(2))
Expect(col).To(Equal(1))
})
It("handles multiple newlines", func() {
data := []byte("a\nb\nc\nd")
// a(0) \n(1) b(2) \n(3) c(4) \n(5) d(6)
line, col := getPositionFromOffset(data, 6)
Expect(line).To(Equal(4))
Expect(col).To(Equal(1))
})
})
var _ = Describe("newSyncedPlaylist", func() {
var s *playlists
BeforeEach(func() {
s = &playlists{}
})
It("creates a synced playlist with correct attributes", func() {
tmpDir := GinkgoT().TempDir()
Expect(os.WriteFile(filepath.Join(tmpDir, "test.m3u"), []byte("content"), 0600)).To(Succeed())
pls, err := s.newSyncedPlaylist(tmpDir, "test.m3u")
Expect(err).ToNot(HaveOccurred())
Expect(pls.Name).To(Equal("test"))
Expect(pls.Comment).To(Equal("Auto-imported from 'test.m3u'"))
Expect(pls.Public).To(BeFalse())
Expect(pls.Path).To(Equal(filepath.Join(tmpDir, "test.m3u")))
Expect(pls.Sync).To(BeTrue())
Expect(pls.UpdatedAt).ToNot(BeZero())
})
It("strips extension from filename to derive name", func() {
tmpDir := GinkgoT().TempDir()
Expect(os.WriteFile(filepath.Join(tmpDir, "My Favorites.nsp"), []byte("{}"), 0600)).To(Succeed())
pls, err := s.newSyncedPlaylist(tmpDir, "My Favorites.nsp")
Expect(err).ToNot(HaveOccurred())
Expect(pls.Name).To(Equal("My Favorites"))
})
It("returns error for non-existent file", func() {
tmpDir := GinkgoT().TempDir()
_, err := s.newSyncedPlaylist(tmpDir, "nonexistent.m3u")
Expect(err).To(HaveOccurred())
})
})

265
core/playlists/playlists.go Normal file
View File

@@ -0,0 +1,265 @@
package playlists
import (
"context"
"io"
"path/filepath"
"strconv"
"strings"
"github.com/bmatcuk/doublestar/v4"
"github.com/deluan/rest"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
)
type Playlists interface {
// Reads
GetAll(ctx context.Context, options ...model.QueryOptions) (model.Playlists, error)
Get(ctx context.Context, id string) (*model.Playlist, error)
GetWithTracks(ctx context.Context, id string) (*model.Playlist, error)
GetPlaylists(ctx context.Context, mediaFileId string) (model.Playlists, error)
// Mutations
Create(ctx context.Context, playlistId string, name string, ids []string) (string, error)
Delete(ctx context.Context, id string) error
Update(ctx context.Context, playlistID string, name *string, comment *string, public *bool, idsToAdd []string, idxToRemove []int) error
// Track management
AddTracks(ctx context.Context, playlistID string, ids []string) (int, error)
AddAlbums(ctx context.Context, playlistID string, albumIds []string) (int, error)
AddArtists(ctx context.Context, playlistID string, artistIds []string) (int, error)
AddDiscs(ctx context.Context, playlistID string, discs []model.DiscID) (int, error)
RemoveTracks(ctx context.Context, playlistID string, trackIds []string) error
ReorderTrack(ctx context.Context, playlistID string, pos int, newPos int) error
// Import
ImportFile(ctx context.Context, folder *model.Folder, filename string) (*model.Playlist, error)
ImportM3U(ctx context.Context, reader io.Reader) (*model.Playlist, error)
// REST adapters (follows Share/Library pattern)
NewRepository(ctx context.Context) rest.Repository
TracksRepository(ctx context.Context, playlistId string, refreshSmartPlaylist bool) rest.Repository
}
type playlists struct {
ds model.DataStore
}
func NewPlaylists(ds model.DataStore) Playlists {
return &playlists{ds: ds}
}
func InPath(folder model.Folder) bool {
if conf.Server.PlaylistsPath == "" {
return true
}
rel, _ := filepath.Rel(folder.LibraryPath, folder.AbsolutePath())
for path := range strings.SplitSeq(conf.Server.PlaylistsPath, string(filepath.ListSeparator)) {
if match, _ := doublestar.Match(path, rel); match {
return true
}
}
return false
}
// --- Read operations ---
func (s *playlists) GetAll(ctx context.Context, options ...model.QueryOptions) (model.Playlists, error) {
return s.ds.Playlist(ctx).GetAll(options...)
}
func (s *playlists) Get(ctx context.Context, id string) (*model.Playlist, error) {
return s.ds.Playlist(ctx).Get(id)
}
func (s *playlists) GetWithTracks(ctx context.Context, id string) (*model.Playlist, error) {
return s.ds.Playlist(ctx).GetWithTracks(id, true, false)
}
func (s *playlists) GetPlaylists(ctx context.Context, mediaFileId string) (model.Playlists, error) {
return s.ds.Playlist(ctx).GetPlaylists(mediaFileId)
}
// --- Mutation operations ---
// Create creates a new playlist (when name is provided) or replaces tracks on an existing
// playlist (when playlistId is provided). This matches the Subsonic createPlaylist semantics.
func (s *playlists) Create(ctx context.Context, playlistId string, name string, ids []string) (string, error) {
usr, _ := request.UserFrom(ctx)
err := s.ds.WithTxImmediate(func(tx model.DataStore) error {
var pls *model.Playlist
var err error
if playlistId != "" {
pls, err = tx.Playlist(ctx).Get(playlistId)
if err != nil {
return err
}
if pls.IsSmartPlaylist() {
return model.ErrNotAuthorized
}
if !usr.IsAdmin && pls.OwnerID != usr.ID {
return model.ErrNotAuthorized
}
} else {
pls = &model.Playlist{Name: name}
pls.OwnerID = usr.ID
}
pls.Tracks = nil
pls.AddMediaFilesByID(ids)
err = tx.Playlist(ctx).Put(pls)
playlistId = pls.ID
return err
})
return playlistId, err
}
func (s *playlists) Delete(ctx context.Context, id string) error {
if _, err := s.checkWritable(ctx, id); err != nil {
return err
}
return s.ds.Playlist(ctx).Delete(id)
}
func (s *playlists) Update(ctx context.Context, playlistID string,
name *string, comment *string, public *bool,
idsToAdd []string, idxToRemove []int) error {
var pls *model.Playlist
var err error
hasTrackChanges := len(idsToAdd) > 0 || len(idxToRemove) > 0
if hasTrackChanges {
pls, err = s.checkTracksEditable(ctx, playlistID)
} else {
pls, err = s.checkWritable(ctx, playlistID)
}
if err != nil {
return err
}
return s.ds.WithTxImmediate(func(tx model.DataStore) error {
repo := tx.Playlist(ctx)
if len(idxToRemove) > 0 {
tracksRepo := repo.Tracks(playlistID, false)
// Convert 0-based indices to 1-based position IDs and delete them directly,
// avoiding the need to load all tracks into memory.
positions := make([]string, len(idxToRemove))
for i, idx := range idxToRemove {
positions[i] = strconv.Itoa(idx + 1)
}
if err := tracksRepo.Delete(positions...); err != nil {
return err
}
if len(idsToAdd) > 0 {
if _, err := tracksRepo.Add(idsToAdd); err != nil {
return err
}
}
return s.updateMetadata(ctx, tx, pls, name, comment, public)
}
if len(idsToAdd) > 0 {
if _, err := repo.Tracks(playlistID, false).Add(idsToAdd); err != nil {
return err
}
}
if name == nil && comment == nil && public == nil {
return nil
}
// Reuse the playlist from checkWritable (no tracks loaded, so Put only refreshes counters)
return s.updateMetadata(ctx, tx, pls, name, comment, public)
})
}
// --- Permission helpers ---
// checkWritable fetches the playlist and verifies the current user can modify it.
func (s *playlists) checkWritable(ctx context.Context, id string) (*model.Playlist, error) {
pls, err := s.ds.Playlist(ctx).Get(id)
if err != nil {
return nil, err
}
usr, _ := request.UserFrom(ctx)
if !usr.IsAdmin && pls.OwnerID != usr.ID {
return nil, model.ErrNotAuthorized
}
return pls, nil
}
// checkTracksEditable verifies the user can modify tracks (ownership + not smart playlist).
func (s *playlists) checkTracksEditable(ctx context.Context, playlistID string) (*model.Playlist, error) {
pls, err := s.checkWritable(ctx, playlistID)
if err != nil {
return nil, err
}
if pls.IsSmartPlaylist() {
return nil, model.ErrNotAuthorized
}
return pls, nil
}
// updateMetadata applies optional metadata changes to a playlist and persists it.
// Accepts a DataStore parameter so it can be used inside transactions.
// The caller is responsible for permission checks.
func (s *playlists) updateMetadata(ctx context.Context, ds model.DataStore, pls *model.Playlist, name *string, comment *string, public *bool) error {
if name != nil {
pls.Name = *name
}
if comment != nil {
pls.Comment = *comment
}
if public != nil {
pls.Public = *public
}
return ds.Playlist(ctx).Put(pls)
}
// --- Track management operations ---
func (s *playlists) AddTracks(ctx context.Context, playlistID string, ids []string) (int, error) {
if _, err := s.checkTracksEditable(ctx, playlistID); err != nil {
return 0, err
}
return s.ds.Playlist(ctx).Tracks(playlistID, false).Add(ids)
}
func (s *playlists) AddAlbums(ctx context.Context, playlistID string, albumIds []string) (int, error) {
if _, err := s.checkTracksEditable(ctx, playlistID); err != nil {
return 0, err
}
return s.ds.Playlist(ctx).Tracks(playlistID, false).AddAlbums(albumIds)
}
func (s *playlists) AddArtists(ctx context.Context, playlistID string, artistIds []string) (int, error) {
if _, err := s.checkTracksEditable(ctx, playlistID); err != nil {
return 0, err
}
return s.ds.Playlist(ctx).Tracks(playlistID, false).AddArtists(artistIds)
}
func (s *playlists) AddDiscs(ctx context.Context, playlistID string, discs []model.DiscID) (int, error) {
if _, err := s.checkTracksEditable(ctx, playlistID); err != nil {
return 0, err
}
return s.ds.Playlist(ctx).Tracks(playlistID, false).AddDiscs(discs)
}
func (s *playlists) RemoveTracks(ctx context.Context, playlistID string, trackIds []string) error {
if _, err := s.checkTracksEditable(ctx, playlistID); err != nil {
return err
}
return s.ds.WithTx(func(tx model.DataStore) error {
return tx.Playlist(ctx).Tracks(playlistID, false).Delete(trackIds...)
})
}
func (s *playlists) ReorderTrack(ctx context.Context, playlistID string, pos int, newPos int) error {
if _, err := s.checkTracksEditable(ctx, playlistID); err != nil {
return err
}
return s.ds.WithTx(func(tx model.DataStore) error {
return tx.Playlist(ctx).Tracks(playlistID, false).Reorder(pos, newPos)
})
}

View File

@@ -0,0 +1,17 @@
package playlists_test
import (
"testing"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
func TestPlaylists(t *testing.T) {
tests.Init(t, false)
log.SetLevel(log.LevelFatal)
RegisterFailHandler(Fail)
RunSpecs(t, "Playlists Suite")
}

View File

@@ -0,0 +1,297 @@
package playlists_test
import (
"context"
"github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/criteria"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("Playlists", func() {
var ds *tests.MockDataStore
var ps playlists.Playlists
var mockPlsRepo *tests.MockPlaylistRepo
ctx := context.Background()
BeforeEach(func() {
mockPlsRepo = tests.CreateMockPlaylistRepo()
ds = &tests.MockDataStore{
MockedPlaylist: mockPlsRepo,
MockedLibrary: &tests.MockLibraryRepo{},
}
ctx = request.WithUser(ctx, model.User{ID: "123"})
})
Describe("Delete", func() {
var mockTracks *tests.MockPlaylistTrackRepo
BeforeEach(func() {
mockTracks = &tests.MockPlaylistTrackRepo{AddCount: 3}
mockPlsRepo.Data = map[string]*model.Playlist{
"pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1"},
}
mockPlsRepo.TracksRepo = mockTracks
ps = playlists.NewPlaylists(ds)
})
It("allows owner to delete their playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
err := ps.Delete(ctx, "pls-1")
Expect(err).ToNot(HaveOccurred())
Expect(mockPlsRepo.Deleted).To(ContainElement("pls-1"))
})
It("allows admin to delete any playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "admin-1", IsAdmin: true})
err := ps.Delete(ctx, "pls-1")
Expect(err).ToNot(HaveOccurred())
Expect(mockPlsRepo.Deleted).To(ContainElement("pls-1"))
})
It("denies non-owner, non-admin from deleting", func() {
ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false})
err := ps.Delete(ctx, "pls-1")
Expect(err).To(MatchError(model.ErrNotAuthorized))
Expect(mockPlsRepo.Deleted).To(BeEmpty())
})
It("returns error when playlist not found", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
err := ps.Delete(ctx, "nonexistent")
Expect(err).To(Equal(model.ErrNotFound))
})
})
Describe("Create", func() {
BeforeEach(func() {
mockPlsRepo.Data = map[string]*model.Playlist{
"pls-1": {ID: "pls-1", Name: "Existing", OwnerID: "user-1"},
"pls-2": {ID: "pls-2", Name: "Other's", OwnerID: "other-user"},
"pls-smart": {ID: "pls-smart", Name: "Smart", OwnerID: "user-1",
Rules: &criteria.Criteria{Expression: criteria.Contains{"title": "test"}}},
}
ps = playlists.NewPlaylists(ds)
})
It("creates a new playlist with owner set from context", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
id, err := ps.Create(ctx, "", "New Playlist", []string{"song-1", "song-2"})
Expect(err).ToNot(HaveOccurred())
Expect(id).ToNot(BeEmpty())
Expect(mockPlsRepo.Last.Name).To(Equal("New Playlist"))
Expect(mockPlsRepo.Last.OwnerID).To(Equal("user-1"))
})
It("replaces tracks on existing playlist when owner matches", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
id, err := ps.Create(ctx, "pls-1", "", []string{"song-3"})
Expect(err).ToNot(HaveOccurred())
Expect(id).To(Equal("pls-1"))
Expect(mockPlsRepo.Last.Tracks).To(HaveLen(1))
})
It("allows admin to replace tracks on any playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "admin-1", IsAdmin: true})
id, err := ps.Create(ctx, "pls-2", "", []string{"song-3"})
Expect(err).ToNot(HaveOccurred())
Expect(id).To(Equal("pls-2"))
})
It("denies non-owner, non-admin from replacing tracks on existing playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
_, err := ps.Create(ctx, "pls-2", "", []string{"song-3"})
Expect(err).To(MatchError(model.ErrNotAuthorized))
})
It("returns error when existing playlistId not found", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
_, err := ps.Create(ctx, "nonexistent", "", []string{"song-1"})
Expect(err).To(Equal(model.ErrNotFound))
})
It("denies replacing tracks on a smart playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
_, err := ps.Create(ctx, "pls-smart", "", []string{"song-1"})
Expect(err).To(MatchError(model.ErrNotAuthorized))
})
})
Describe("Update", func() {
var mockTracks *tests.MockPlaylistTrackRepo
BeforeEach(func() {
mockTracks = &tests.MockPlaylistTrackRepo{AddCount: 2}
mockPlsRepo.Data = map[string]*model.Playlist{
"pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1"},
"pls-other": {ID: "pls-other", Name: "Other's", OwnerID: "other-user"},
"pls-smart": {ID: "pls-smart", Name: "Smart", OwnerID: "user-1",
Rules: &criteria.Criteria{Expression: criteria.Contains{"title": "test"}}},
}
mockPlsRepo.TracksRepo = mockTracks
ps = playlists.NewPlaylists(ds)
})
It("allows owner to update their playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
newName := "Updated Name"
err := ps.Update(ctx, "pls-1", &newName, nil, nil, nil, nil)
Expect(err).ToNot(HaveOccurred())
})
It("allows admin to update any playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "admin-1", IsAdmin: true})
newName := "Updated Name"
err := ps.Update(ctx, "pls-other", &newName, nil, nil, nil, nil)
Expect(err).ToNot(HaveOccurred())
})
It("denies non-owner, non-admin from updating", func() {
ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false})
newName := "Updated Name"
err := ps.Update(ctx, "pls-1", &newName, nil, nil, nil, nil)
Expect(err).To(MatchError(model.ErrNotAuthorized))
})
It("returns error when playlist not found", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
newName := "Updated Name"
err := ps.Update(ctx, "nonexistent", &newName, nil, nil, nil, nil)
Expect(err).To(Equal(model.ErrNotFound))
})
It("denies adding tracks to a smart playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
err := ps.Update(ctx, "pls-smart", nil, nil, nil, []string{"song-1"}, nil)
Expect(err).To(MatchError(model.ErrNotAuthorized))
})
It("denies removing tracks from a smart playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
err := ps.Update(ctx, "pls-smart", nil, nil, nil, nil, []int{0})
Expect(err).To(MatchError(model.ErrNotAuthorized))
})
It("allows metadata updates on a smart playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
newName := "Updated Smart"
err := ps.Update(ctx, "pls-smart", &newName, nil, nil, nil, nil)
Expect(err).ToNot(HaveOccurred())
})
})
Describe("AddTracks", func() {
var mockTracks *tests.MockPlaylistTrackRepo
BeforeEach(func() {
mockTracks = &tests.MockPlaylistTrackRepo{AddCount: 2}
mockPlsRepo.Data = map[string]*model.Playlist{
"pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1"},
"pls-smart": {ID: "pls-smart", Name: "Smart", OwnerID: "user-1",
Rules: &criteria.Criteria{Expression: criteria.Contains{"title": "test"}}},
"pls-other": {ID: "pls-other", Name: "Other's", OwnerID: "other-user"},
}
mockPlsRepo.TracksRepo = mockTracks
ps = playlists.NewPlaylists(ds)
})
It("allows owner to add tracks", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
count, err := ps.AddTracks(ctx, "pls-1", []string{"song-1", "song-2"})
Expect(err).ToNot(HaveOccurred())
Expect(count).To(Equal(2))
Expect(mockTracks.AddedIds).To(ConsistOf("song-1", "song-2"))
})
It("allows admin to add tracks to any playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "admin-1", IsAdmin: true})
count, err := ps.AddTracks(ctx, "pls-other", []string{"song-1"})
Expect(err).ToNot(HaveOccurred())
Expect(count).To(Equal(2))
})
It("denies non-owner, non-admin", func() {
ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false})
_, err := ps.AddTracks(ctx, "pls-1", []string{"song-1"})
Expect(err).To(MatchError(model.ErrNotAuthorized))
})
It("denies editing smart playlists", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
_, err := ps.AddTracks(ctx, "pls-smart", []string{"song-1"})
Expect(err).To(MatchError(model.ErrNotAuthorized))
})
It("returns error when playlist not found", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
_, err := ps.AddTracks(ctx, "nonexistent", []string{"song-1"})
Expect(err).To(Equal(model.ErrNotFound))
})
})
Describe("RemoveTracks", func() {
var mockTracks *tests.MockPlaylistTrackRepo
BeforeEach(func() {
mockTracks = &tests.MockPlaylistTrackRepo{}
mockPlsRepo.Data = map[string]*model.Playlist{
"pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1"},
"pls-smart": {ID: "pls-smart", Name: "Smart", OwnerID: "user-1",
Rules: &criteria.Criteria{Expression: criteria.Contains{"title": "test"}}},
}
mockPlsRepo.TracksRepo = mockTracks
ps = playlists.NewPlaylists(ds)
})
It("allows owner to remove tracks", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
err := ps.RemoveTracks(ctx, "pls-1", []string{"track-1", "track-2"})
Expect(err).ToNot(HaveOccurred())
Expect(mockTracks.DeletedIds).To(ConsistOf("track-1", "track-2"))
})
It("denies on smart playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
err := ps.RemoveTracks(ctx, "pls-smart", []string{"track-1"})
Expect(err).To(MatchError(model.ErrNotAuthorized))
})
It("denies non-owner", func() {
ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false})
err := ps.RemoveTracks(ctx, "pls-1", []string{"track-1"})
Expect(err).To(MatchError(model.ErrNotAuthorized))
})
})
Describe("ReorderTrack", func() {
var mockTracks *tests.MockPlaylistTrackRepo
BeforeEach(func() {
mockTracks = &tests.MockPlaylistTrackRepo{}
mockPlsRepo.Data = map[string]*model.Playlist{
"pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1"},
"pls-smart": {ID: "pls-smart", Name: "Smart", OwnerID: "user-1",
Rules: &criteria.Criteria{Expression: criteria.Contains{"title": "test"}}},
}
mockPlsRepo.TracksRepo = mockTracks
ps = playlists.NewPlaylists(ds)
})
It("allows owner to reorder", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
err := ps.ReorderTrack(ctx, "pls-1", 1, 3)
Expect(err).ToNot(HaveOccurred())
Expect(mockTracks.Reordered).To(BeTrue())
})
It("denies on smart playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
err := ps.ReorderTrack(ctx, "pls-smart", 1, 3)
Expect(err).To(MatchError(model.ErrNotAuthorized))
})
})
})

View File

@@ -0,0 +1,95 @@
package playlists
import (
"context"
"errors"
"github.com/deluan/rest"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
)
// --- REST adapter (follows Share/Library pattern) ---
func (s *playlists) NewRepository(ctx context.Context) rest.Repository {
return &playlistRepositoryWrapper{
ctx: ctx,
PlaylistRepository: s.ds.Playlist(ctx),
service: s,
}
}
// playlistRepositoryWrapper wraps the playlist repository as a thin REST-to-service adapter.
// It satisfies rest.Repository through the embedded PlaylistRepository (via ResourceRepository),
// and rest.Persistable by delegating to service methods for all mutations.
type playlistRepositoryWrapper struct {
model.PlaylistRepository
ctx context.Context
service *playlists
}
func (r *playlistRepositoryWrapper) Save(entity any) (string, error) {
return r.service.savePlaylist(r.ctx, entity.(*model.Playlist))
}
func (r *playlistRepositoryWrapper) Update(id string, entity any, cols ...string) error {
return r.service.updatePlaylistEntity(r.ctx, id, entity.(*model.Playlist), cols...)
}
func (r *playlistRepositoryWrapper) Delete(id string) error {
err := r.service.Delete(r.ctx, id)
switch {
case errors.Is(err, model.ErrNotFound):
return rest.ErrNotFound
case errors.Is(err, model.ErrNotAuthorized):
return rest.ErrPermissionDenied
default:
return err
}
}
func (s *playlists) TracksRepository(ctx context.Context, playlistId string, refreshSmartPlaylist bool) rest.Repository {
repo := s.ds.Playlist(ctx)
tracks := repo.Tracks(playlistId, refreshSmartPlaylist)
if tracks == nil {
return nil
}
return tracks.(rest.Repository)
}
// savePlaylist creates a new playlist, assigning the owner from context.
func (s *playlists) savePlaylist(ctx context.Context, pls *model.Playlist) (string, error) {
usr, _ := request.UserFrom(ctx)
pls.OwnerID = usr.ID
pls.ID = "" // Force new creation
err := s.ds.Playlist(ctx).Put(pls)
if err != nil {
return "", err
}
return pls.ID, nil
}
// updatePlaylistEntity updates playlist metadata with permission checks.
// Used by the REST API wrapper.
func (s *playlists) updatePlaylistEntity(ctx context.Context, id string, entity *model.Playlist, cols ...string) error {
current, err := s.checkWritable(ctx, id)
if err != nil {
switch {
case errors.Is(err, model.ErrNotFound):
return rest.ErrNotFound
case errors.Is(err, model.ErrNotAuthorized):
return rest.ErrPermissionDenied
default:
return err
}
}
usr, _ := request.UserFrom(ctx)
if !usr.IsAdmin && entity.OwnerID != "" && entity.OwnerID != current.OwnerID {
return rest.ErrPermissionDenied
}
// Apply ownership change (admin only)
if entity.OwnerID != "" {
current.OwnerID = entity.OwnerID
}
return s.updateMetadata(ctx, s.ds, current, &entity.Name, &entity.Comment, &entity.Public)
}

View File

@@ -0,0 +1,120 @@
package playlists_test
import (
"context"
"github.com/deluan/rest"
"github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("REST Adapter", func() {
var ds *tests.MockDataStore
var ps playlists.Playlists
var mockPlsRepo *tests.MockPlaylistRepo
ctx := context.Background()
BeforeEach(func() {
mockPlsRepo = tests.CreateMockPlaylistRepo()
ds = &tests.MockDataStore{
MockedPlaylist: mockPlsRepo,
MockedLibrary: &tests.MockLibraryRepo{},
}
ctx = request.WithUser(ctx, model.User{ID: "123"})
})
Describe("NewRepository", func() {
var repo rest.Persistable
BeforeEach(func() {
mockPlsRepo.Data = map[string]*model.Playlist{
"pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1"},
}
ps = playlists.NewPlaylists(ds)
})
Describe("Save", func() {
It("sets the owner from the context user", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
repo = ps.NewRepository(ctx).(rest.Persistable)
pls := &model.Playlist{Name: "New Playlist"}
id, err := repo.Save(pls)
Expect(err).ToNot(HaveOccurred())
Expect(id).ToNot(BeEmpty())
Expect(pls.OwnerID).To(Equal("user-1"))
})
It("forces a new creation by clearing ID", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
repo = ps.NewRepository(ctx).(rest.Persistable)
pls := &model.Playlist{ID: "should-be-cleared", Name: "New"}
_, err := repo.Save(pls)
Expect(err).ToNot(HaveOccurred())
Expect(pls.ID).ToNot(Equal("should-be-cleared"))
})
})
Describe("Update", func() {
It("allows owner to update their playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
repo = ps.NewRepository(ctx).(rest.Persistable)
pls := &model.Playlist{Name: "Updated"}
err := repo.Update("pls-1", pls)
Expect(err).ToNot(HaveOccurred())
})
It("allows admin to update any playlist", func() {
ctx = request.WithUser(ctx, model.User{ID: "admin-1", IsAdmin: true})
repo = ps.NewRepository(ctx).(rest.Persistable)
pls := &model.Playlist{Name: "Updated"}
err := repo.Update("pls-1", pls)
Expect(err).ToNot(HaveOccurred())
})
It("denies non-owner, non-admin", func() {
ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false})
repo = ps.NewRepository(ctx).(rest.Persistable)
pls := &model.Playlist{Name: "Updated"}
err := repo.Update("pls-1", pls)
Expect(err).To(Equal(rest.ErrPermissionDenied))
})
It("denies regular user from changing ownership", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
repo = ps.NewRepository(ctx).(rest.Persistable)
pls := &model.Playlist{Name: "Updated", OwnerID: "other-user"}
err := repo.Update("pls-1", pls)
Expect(err).To(Equal(rest.ErrPermissionDenied))
})
It("returns rest.ErrNotFound when playlist doesn't exist", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
repo = ps.NewRepository(ctx).(rest.Persistable)
pls := &model.Playlist{Name: "Updated"}
err := repo.Update("nonexistent", pls)
Expect(err).To(Equal(rest.ErrNotFound))
})
})
Describe("Delete", func() {
It("delegates to service Delete with permission checks", func() {
ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false})
repo = ps.NewRepository(ctx).(rest.Persistable)
err := repo.Delete("pls-1")
Expect(err).ToNot(HaveOccurred())
Expect(mockPlsRepo.Deleted).To(ContainElement("pls-1"))
})
It("denies non-owner", func() {
ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false})
repo = ps.NewRepository(ctx).(rest.Persistable)
err := repo.Delete("pls-1")
Expect(err).To(Equal(rest.ErrPermissionDenied))
})
})
})
})

View File

@@ -7,6 +7,7 @@ import (
"github.com/navidrome/navidrome/core/ffmpeg"
"github.com/navidrome/navidrome/core/metrics"
"github.com/navidrome/navidrome/core/playback"
"github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/core/scrobbler"
)
@@ -16,7 +17,7 @@ var Set = wire.NewSet(
NewArchiver,
NewPlayers,
NewShare,
NewPlaylists,
playlists.NewPlaylists,
NewLibrary,
NewUser,
NewMaintenance,

View File

@@ -6,9 +6,7 @@ import (
"embed"
"fmt"
"runtime"
"strings"
"github.com/maruel/natural"
"github.com/mattn/go-sqlite3"
"github.com/navidrome/navidrome/conf"
_ "github.com/navidrome/navidrome/db/migrations"
@@ -33,12 +31,7 @@ func Db() *sql.DB {
return singleton.GetInstance(func() *sql.DB {
sql.Register(Driver, &sqlite3.SQLiteDriver{
ConnectHook: func(conn *sqlite3.SQLiteConn) error {
if err := conn.RegisterFunc("SEEDEDRAND", hasher.HashFunc(), false); err != nil {
return err
}
return conn.RegisterCollation("NATURALSORT", func(a, b string) int {
return natural.Compare(strings.ToLower(a), strings.ToLower(b))
})
return conn.RegisterFunc("SEEDEDRAND", hasher.HashFunc(), false)
},
})
Path = conf.Server.DbPath

View File

@@ -1,152 +0,0 @@
-- +goose Up
-- Change order_*/sort_* column collation from NOCASE to NATURALSORT.
-- This way bare ORDER BY on these columns automatically uses natural sorting,
-- without needing explicit COLLATE NATURALSORT in every query.
PRAGMA writable_schema = ON;
UPDATE sqlite_master
SET sql = replace(sql, 'collate NOCASE', 'collate NATURALSORT')
WHERE type = 'table' AND name IN ('artist', 'album', 'media_file', 'playlist', 'radio');
PRAGMA writable_schema = OFF;
-- Recreate indexes on order_* and sort expression fields to use NATURALSORT collation.
-- This enables natural number ordering (e.g., "Album 2" before "Album 10").
-- Artist indexes
drop index if exists artist_order_artist_name;
create index artist_order_artist_name
on artist (order_artist_name collate NATURALSORT);
drop index if exists artist_sort_name;
create index artist_sort_name
on artist (coalesce(nullif(sort_artist_name,''),order_artist_name) collate NATURALSORT);
-- Album indexes
drop index if exists album_order_album_name;
create index album_order_album_name
on album (order_album_name collate NATURALSORT);
drop index if exists album_order_album_artist_name;
create index album_order_album_artist_name
on album (order_album_artist_name collate NATURALSORT);
drop index if exists album_alphabetical_by_artist;
create index album_alphabetical_by_artist
on album (compilation, order_album_artist_name collate NATURALSORT, order_album_name collate NATURALSORT);
drop index if exists album_sort_name;
create index album_sort_name
on album (coalesce(nullif(sort_album_name,''),order_album_name) collate NATURALSORT);
drop index if exists album_sort_album_artist_name;
create index album_sort_album_artist_name
on album (coalesce(nullif(sort_album_artist_name,''),order_album_artist_name) collate NATURALSORT);
-- Media file indexes
drop index if exists media_file_order_title;
create index media_file_order_title
on media_file (order_title collate NATURALSORT);
drop index if exists media_file_order_album_name;
create index media_file_order_album_name
on media_file (order_album_name collate NATURALSORT);
drop index if exists media_file_order_artist_name;
create index media_file_order_artist_name
on media_file (order_artist_name collate NATURALSORT);
drop index if exists media_file_sort_title;
create index media_file_sort_title
on media_file (coalesce(nullif(sort_title,''),order_title) collate NATURALSORT);
drop index if exists media_file_sort_artist_name;
create index media_file_sort_artist_name
on media_file (coalesce(nullif(sort_artist_name,''),order_artist_name) collate NATURALSORT);
drop index if exists media_file_sort_album_name;
create index media_file_sort_album_name
on media_file (coalesce(nullif(sort_album_name,''),order_album_name) collate NATURALSORT);
-- Playlist and radio indexes: recreate to match new NATURALSORT column collation
drop index if exists playlist_name;
create index playlist_name
on playlist (name collate NATURALSORT);
drop index if exists radio_name;
create index radio_name
on radio (name collate NATURALSORT);
-- +goose Down
-- Restore NOCASE column collation
PRAGMA writable_schema = ON;
UPDATE sqlite_master
SET sql = replace(sql, 'collate NATURALSORT', 'collate NOCASE')
WHERE type = 'table' AND name IN ('artist', 'album', 'media_file', 'playlist', 'radio');
PRAGMA writable_schema = OFF;
-- Restore NOCASE collation indexes
-- Artist indexes
drop index if exists artist_order_artist_name;
create index artist_order_artist_name
on artist (order_artist_name);
drop index if exists artist_sort_name;
create index artist_sort_name
on artist (coalesce(nullif(sort_artist_name,''),order_artist_name) collate NOCASE);
-- Album indexes
drop index if exists album_order_album_name;
create index album_order_album_name
on album (order_album_name);
drop index if exists album_order_album_artist_name;
create index album_order_album_artist_name
on album (order_album_artist_name);
drop index if exists album_alphabetical_by_artist;
create index album_alphabetical_by_artist
on album (compilation, order_album_artist_name, order_album_name);
drop index if exists album_sort_name;
create index album_sort_name
on album (coalesce(nullif(sort_album_name,''),order_album_name) collate NOCASE);
drop index if exists album_sort_album_artist_name;
create index album_sort_album_artist_name
on album (coalesce(nullif(sort_album_artist_name,''),order_album_artist_name) collate NOCASE);
-- Media file indexes
drop index if exists media_file_order_title;
create index media_file_order_title
on media_file (order_title);
drop index if exists media_file_order_album_name;
create index media_file_order_album_name
on media_file (order_album_name);
drop index if exists media_file_order_artist_name;
create index media_file_order_artist_name
on media_file (order_artist_name);
drop index if exists media_file_sort_title;
create index media_file_sort_title
on media_file (coalesce(nullif(sort_title,''),order_title) collate NOCASE);
drop index if exists media_file_sort_artist_name;
create index media_file_sort_artist_name
on media_file (coalesce(nullif(sort_artist_name,''),order_artist_name) collate NOCASE);
drop index if exists media_file_sort_album_name;
create index media_file_sort_album_name
on media_file (coalesce(nullif(sort_album_name,''),order_album_name) collate NOCASE);
-- Restore playlist and radio indexes
drop index if exists playlist_name;
create index playlist_name
on playlist (name);
drop index if exists radio_name;
create index radio_name
on radio (name);

View File

@@ -0,0 +1,391 @@
package migrations
import (
"context"
"database/sql"
"fmt"
"github.com/pressly/goose/v3"
)
func init() {
goose.AddMigrationContext(upAddFts5Search, downAddFts5Search)
}
// stripPunct generates a SQL expression that strips common punctuation from a column or expression.
// Used during migration to approximate the Go normalizeForFTS function for bulk-populating search_normalized.
func stripPunct(col string) string {
return fmt.Sprintf(
`REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(%s, '.', ''), '/', ''), '-', ''), '''', ''), '&', ''), ',', '')`,
col,
)
}
func upAddFts5Search(ctx context.Context, tx *sql.Tx) error {
notice(tx, "Adding FTS5 full-text search indexes. This may take a moment on large libraries.")
// Step 1: Add search_participants and search_normalized columns to media_file, album, and artist
_, err := tx.ExecContext(ctx, `ALTER TABLE media_file ADD COLUMN search_participants TEXT NOT NULL DEFAULT ''`)
if err != nil {
return fmt.Errorf("adding search_participants to media_file: %w", err)
}
_, err = tx.ExecContext(ctx, `ALTER TABLE media_file ADD COLUMN search_normalized TEXT NOT NULL DEFAULT ''`)
if err != nil {
return fmt.Errorf("adding search_normalized to media_file: %w", err)
}
_, err = tx.ExecContext(ctx, `ALTER TABLE album ADD COLUMN search_participants TEXT NOT NULL DEFAULT ''`)
if err != nil {
return fmt.Errorf("adding search_participants to album: %w", err)
}
_, err = tx.ExecContext(ctx, `ALTER TABLE album ADD COLUMN search_normalized TEXT NOT NULL DEFAULT ''`)
if err != nil {
return fmt.Errorf("adding search_normalized to album: %w", err)
}
_, err = tx.ExecContext(ctx, `ALTER TABLE artist ADD COLUMN search_normalized TEXT NOT NULL DEFAULT ''`)
if err != nil {
return fmt.Errorf("adding search_normalized to artist: %w", err)
}
// Step 2: Populate search_participants from participants JSON.
// Extract all "name" values from the participants JSON structure.
// participants is a JSON object like: {"artist":[{"name":"...","id":"..."}],"albumartist":[...]}
// We use json_each + json_extract to flatten all names into a space-separated string.
_, err = tx.ExecContext(ctx, `
UPDATE media_file SET search_participants = COALESCE(
(SELECT group_concat(json_extract(je2.value, '$.name'), ' ')
FROM json_each(media_file.participants) AS je1,
json_each(je1.value) AS je2
WHERE json_extract(je2.value, '$.name') IS NOT NULL),
''
)
WHERE participants IS NOT NULL AND participants != '' AND participants != '{}'
`)
if err != nil {
return fmt.Errorf("populating media_file search_participants: %w", err)
}
_, err = tx.ExecContext(ctx, `
UPDATE album SET search_participants = COALESCE(
(SELECT group_concat(json_extract(je2.value, '$.name'), ' ')
FROM json_each(album.participants) AS je1,
json_each(je1.value) AS je2
WHERE json_extract(je2.value, '$.name') IS NOT NULL),
''
)
WHERE participants IS NOT NULL AND participants != '' AND participants != '{}'
`)
if err != nil {
return fmt.Errorf("populating album search_participants: %w", err)
}
// Step 2b: Populate search_normalized using SQL REPLACE chains for common punctuation.
// The Go code will compute the precise value on next scan; this is a best-effort approximation.
_, err = tx.ExecContext(ctx, fmt.Sprintf(`
UPDATE artist SET search_normalized = %s
WHERE name != %s`,
stripPunct("name"), stripPunct("name")))
if err != nil {
return fmt.Errorf("populating artist search_normalized: %w", err)
}
_, err = tx.ExecContext(ctx, fmt.Sprintf(`
UPDATE album SET search_normalized = TRIM(%s || ' ' || %s)
WHERE name != %s OR COALESCE(album_artist, '') != %s`,
stripPunct("name"), stripPunct("COALESCE(album_artist, '')"),
stripPunct("name"), stripPunct("COALESCE(album_artist, '')")))
if err != nil {
return fmt.Errorf("populating album search_normalized: %w", err)
}
_, err = tx.ExecContext(ctx, fmt.Sprintf(`
UPDATE media_file SET search_normalized =
TRIM(%s || ' ' || %s || ' ' || %s || ' ' || %s)
WHERE title != %s
OR COALESCE(album, '') != %s
OR COALESCE(artist, '') != %s
OR COALESCE(album_artist, '') != %s`,
stripPunct("title"), stripPunct("COALESCE(album, '')"),
stripPunct("COALESCE(artist, '')"), stripPunct("COALESCE(album_artist, '')"),
stripPunct("title"), stripPunct("COALESCE(album, '')"),
stripPunct("COALESCE(artist, '')"), stripPunct("COALESCE(album_artist, '')")))
if err != nil {
return fmt.Errorf("populating media_file search_normalized: %w", err)
}
// Step 3: Create FTS5 virtual tables
_, err = tx.ExecContext(ctx, `
CREATE VIRTUAL TABLE IF NOT EXISTS media_file_fts USING fts5(
title, album, artist, album_artist,
sort_title, sort_album_name, sort_artist_name, sort_album_artist_name,
disc_subtitle, search_participants, search_normalized,
content='', content_rowid='rowid',
tokenize='unicode61 remove_diacritics 2'
)
`)
if err != nil {
return fmt.Errorf("creating media_file_fts: %w", err)
}
_, err = tx.ExecContext(ctx, `
CREATE VIRTUAL TABLE IF NOT EXISTS album_fts USING fts5(
name, sort_album_name, album_artist,
search_participants, discs, catalog_num, album_version, search_normalized,
content='', content_rowid='rowid',
tokenize='unicode61 remove_diacritics 2'
)
`)
if err != nil {
return fmt.Errorf("creating album_fts: %w", err)
}
_, err = tx.ExecContext(ctx, `
CREATE VIRTUAL TABLE IF NOT EXISTS artist_fts USING fts5(
name, sort_artist_name, search_normalized,
content='', content_rowid='rowid',
tokenize='unicode61 remove_diacritics 2'
)
`)
if err != nil {
return fmt.Errorf("creating artist_fts: %w", err)
}
// Step 4: Bulk-populate FTS5 indexes from existing data
_, err = tx.ExecContext(ctx, `
INSERT INTO media_file_fts(rowid, title, album, artist, album_artist,
sort_title, sort_album_name, sort_artist_name, sort_album_artist_name,
disc_subtitle, search_participants, search_normalized)
SELECT rowid, title, album, artist, album_artist,
sort_title, sort_album_name, sort_artist_name, sort_album_artist_name,
COALESCE(disc_subtitle, ''), COALESCE(search_participants, ''),
COALESCE(search_normalized, '')
FROM media_file
`)
if err != nil {
return fmt.Errorf("populating media_file_fts: %w", err)
}
_, err = tx.ExecContext(ctx, `
INSERT INTO album_fts(rowid, name, sort_album_name, album_artist,
search_participants, discs, catalog_num, album_version, search_normalized)
SELECT rowid, name, COALESCE(sort_album_name, ''), COALESCE(album_artist, ''),
COALESCE(search_participants, ''), COALESCE(discs, ''),
COALESCE(catalog_num, ''),
COALESCE((SELECT group_concat(json_extract(je.value, '$.value'), ' ')
FROM json_each(album.tags, '$.albumversion') AS je), ''),
COALESCE(search_normalized, '')
FROM album
`)
if err != nil {
return fmt.Errorf("populating album_fts: %w", err)
}
_, err = tx.ExecContext(ctx, `
INSERT INTO artist_fts(rowid, name, sort_artist_name, search_normalized)
SELECT rowid, name, COALESCE(sort_artist_name, ''), COALESCE(search_normalized, '')
FROM artist
`)
if err != nil {
return fmt.Errorf("populating artist_fts: %w", err)
}
// Step 5: Create triggers for media_file
_, err = tx.ExecContext(ctx, `
CREATE TRIGGER media_file_fts_ai AFTER INSERT ON media_file BEGIN
INSERT INTO media_file_fts(rowid, title, album, artist, album_artist,
sort_title, sort_album_name, sort_artist_name, sort_album_artist_name,
disc_subtitle, search_participants, search_normalized)
VALUES (NEW.rowid, NEW.title, NEW.album, NEW.artist, NEW.album_artist,
NEW.sort_title, NEW.sort_album_name, NEW.sort_artist_name, NEW.sort_album_artist_name,
COALESCE(NEW.disc_subtitle, ''), COALESCE(NEW.search_participants, ''),
COALESCE(NEW.search_normalized, ''));
END
`)
if err != nil {
return fmt.Errorf("creating media_file_fts insert trigger: %w", err)
}
_, err = tx.ExecContext(ctx, `
CREATE TRIGGER media_file_fts_ad AFTER DELETE ON media_file BEGIN
INSERT INTO media_file_fts(media_file_fts, rowid, title, album, artist, album_artist,
sort_title, sort_album_name, sort_artist_name, sort_album_artist_name,
disc_subtitle, search_participants, search_normalized)
VALUES ('delete', OLD.rowid, OLD.title, OLD.album, OLD.artist, OLD.album_artist,
OLD.sort_title, OLD.sort_album_name, OLD.sort_artist_name, OLD.sort_album_artist_name,
COALESCE(OLD.disc_subtitle, ''), COALESCE(OLD.search_participants, ''),
COALESCE(OLD.search_normalized, ''));
END
`)
if err != nil {
return fmt.Errorf("creating media_file_fts delete trigger: %w", err)
}
_, err = tx.ExecContext(ctx, `
CREATE TRIGGER media_file_fts_au AFTER UPDATE ON media_file
WHEN
OLD.title IS NOT NEW.title OR
OLD.album IS NOT NEW.album OR
OLD.artist IS NOT NEW.artist OR
OLD.album_artist IS NOT NEW.album_artist OR
OLD.sort_title IS NOT NEW.sort_title OR
OLD.sort_album_name IS NOT NEW.sort_album_name OR
OLD.sort_artist_name IS NOT NEW.sort_artist_name OR
OLD.sort_album_artist_name IS NOT NEW.sort_album_artist_name OR
OLD.disc_subtitle IS NOT NEW.disc_subtitle OR
OLD.search_participants IS NOT NEW.search_participants OR
OLD.search_normalized IS NOT NEW.search_normalized
BEGIN
INSERT INTO media_file_fts(media_file_fts, rowid, title, album, artist, album_artist,
sort_title, sort_album_name, sort_artist_name, sort_album_artist_name,
disc_subtitle, search_participants, search_normalized)
VALUES ('delete', OLD.rowid, OLD.title, OLD.album, OLD.artist, OLD.album_artist,
OLD.sort_title, OLD.sort_album_name, OLD.sort_artist_name, OLD.sort_album_artist_name,
COALESCE(OLD.disc_subtitle, ''), COALESCE(OLD.search_participants, ''),
COALESCE(OLD.search_normalized, ''));
INSERT INTO media_file_fts(rowid, title, album, artist, album_artist,
sort_title, sort_album_name, sort_artist_name, sort_album_artist_name,
disc_subtitle, search_participants, search_normalized)
VALUES (NEW.rowid, NEW.title, NEW.album, NEW.artist, NEW.album_artist,
NEW.sort_title, NEW.sort_album_name, NEW.sort_artist_name, NEW.sort_album_artist_name,
COALESCE(NEW.disc_subtitle, ''), COALESCE(NEW.search_participants, ''),
COALESCE(NEW.search_normalized, ''));
END
`)
if err != nil {
return fmt.Errorf("creating media_file_fts update trigger: %w", err)
}
// Step 6: Create triggers for album
_, err = tx.ExecContext(ctx, `
CREATE TRIGGER album_fts_ai AFTER INSERT ON album BEGIN
INSERT INTO album_fts(rowid, name, sort_album_name, album_artist,
search_participants, discs, catalog_num, album_version, search_normalized)
VALUES (NEW.rowid, NEW.name, COALESCE(NEW.sort_album_name, ''), COALESCE(NEW.album_artist, ''),
COALESCE(NEW.search_participants, ''), COALESCE(NEW.discs, ''),
COALESCE(NEW.catalog_num, ''),
COALESCE((SELECT group_concat(json_extract(je.value, '$.value'), ' ')
FROM json_each(NEW.tags, '$.albumversion') AS je), ''),
COALESCE(NEW.search_normalized, ''));
END
`)
if err != nil {
return fmt.Errorf("creating album_fts insert trigger: %w", err)
}
_, err = tx.ExecContext(ctx, `
CREATE TRIGGER album_fts_ad AFTER DELETE ON album BEGIN
INSERT INTO album_fts(album_fts, rowid, name, sort_album_name, album_artist,
search_participants, discs, catalog_num, album_version, search_normalized)
VALUES ('delete', OLD.rowid, OLD.name, COALESCE(OLD.sort_album_name, ''), COALESCE(OLD.album_artist, ''),
COALESCE(OLD.search_participants, ''), COALESCE(OLD.discs, ''),
COALESCE(OLD.catalog_num, ''),
COALESCE((SELECT group_concat(json_extract(je.value, '$.value'), ' ')
FROM json_each(OLD.tags, '$.albumversion') AS je), ''),
COALESCE(OLD.search_normalized, ''));
END
`)
if err != nil {
return fmt.Errorf("creating album_fts delete trigger: %w", err)
}
_, err = tx.ExecContext(ctx, `
CREATE TRIGGER album_fts_au AFTER UPDATE ON album
WHEN
OLD.name IS NOT NEW.name OR
OLD.sort_album_name IS NOT NEW.sort_album_name OR
OLD.album_artist IS NOT NEW.album_artist OR
OLD.search_participants IS NOT NEW.search_participants OR
OLD.discs IS NOT NEW.discs OR
OLD.catalog_num IS NOT NEW.catalog_num OR
OLD.tags IS NOT NEW.tags OR
OLD.search_normalized IS NOT NEW.search_normalized
BEGIN
INSERT INTO album_fts(album_fts, rowid, name, sort_album_name, album_artist,
search_participants, discs, catalog_num, album_version, search_normalized)
VALUES ('delete', OLD.rowid, OLD.name, COALESCE(OLD.sort_album_name, ''), COALESCE(OLD.album_artist, ''),
COALESCE(OLD.search_participants, ''), COALESCE(OLD.discs, ''),
COALESCE(OLD.catalog_num, ''),
COALESCE((SELECT group_concat(json_extract(je.value, '$.value'), ' ')
FROM json_each(OLD.tags, '$.albumversion') AS je), ''),
COALESCE(OLD.search_normalized, ''));
INSERT INTO album_fts(rowid, name, sort_album_name, album_artist,
search_participants, discs, catalog_num, album_version, search_normalized)
VALUES (NEW.rowid, NEW.name, COALESCE(NEW.sort_album_name, ''), COALESCE(NEW.album_artist, ''),
COALESCE(NEW.search_participants, ''), COALESCE(NEW.discs, ''),
COALESCE(NEW.catalog_num, ''),
COALESCE((SELECT group_concat(json_extract(je.value, '$.value'), ' ')
FROM json_each(NEW.tags, '$.albumversion') AS je), ''),
COALESCE(NEW.search_normalized, ''));
END
`)
if err != nil {
return fmt.Errorf("creating album_fts update trigger: %w", err)
}
// Step 7: Create triggers for artist
_, err = tx.ExecContext(ctx, `
CREATE TRIGGER artist_fts_ai AFTER INSERT ON artist BEGIN
INSERT INTO artist_fts(rowid, name, sort_artist_name, search_normalized)
VALUES (NEW.rowid, NEW.name, COALESCE(NEW.sort_artist_name, ''),
COALESCE(NEW.search_normalized, ''));
END
`)
if err != nil {
return fmt.Errorf("creating artist_fts insert trigger: %w", err)
}
_, err = tx.ExecContext(ctx, `
CREATE TRIGGER artist_fts_ad AFTER DELETE ON artist BEGIN
INSERT INTO artist_fts(artist_fts, rowid, name, sort_artist_name, search_normalized)
VALUES ('delete', OLD.rowid, OLD.name, COALESCE(OLD.sort_artist_name, ''),
COALESCE(OLD.search_normalized, ''));
END
`)
if err != nil {
return fmt.Errorf("creating artist_fts delete trigger: %w", err)
}
_, err = tx.ExecContext(ctx, `
CREATE TRIGGER artist_fts_au AFTER UPDATE ON artist
WHEN
OLD.name IS NOT NEW.name OR
OLD.sort_artist_name IS NOT NEW.sort_artist_name OR
OLD.search_normalized IS NOT NEW.search_normalized
BEGIN
INSERT INTO artist_fts(artist_fts, rowid, name, sort_artist_name, search_normalized)
VALUES ('delete', OLD.rowid, OLD.name, COALESCE(OLD.sort_artist_name, ''),
COALESCE(OLD.search_normalized, ''));
INSERT INTO artist_fts(rowid, name, sort_artist_name, search_normalized)
VALUES (NEW.rowid, NEW.name, COALESCE(NEW.sort_artist_name, ''),
COALESCE(NEW.search_normalized, ''));
END
`)
if err != nil {
return fmt.Errorf("creating artist_fts update trigger: %w", err)
}
return nil
}
func downAddFts5Search(ctx context.Context, tx *sql.Tx) error {
for _, trigger := range []string{
"media_file_fts_ai", "media_file_fts_ad", "media_file_fts_au",
"album_fts_ai", "album_fts_ad", "album_fts_au",
"artist_fts_ai", "artist_fts_ad", "artist_fts_au",
} {
_, err := tx.ExecContext(ctx, "DROP TRIGGER IF EXISTS "+trigger)
if err != nil {
return fmt.Errorf("dropping trigger %s: %w", trigger, err)
}
}
for _, table := range []string{"media_file_fts", "album_fts", "artist_fts"} {
_, err := tx.ExecContext(ctx, "DROP TABLE IF EXISTS "+table)
if err != nil {
return fmt.Errorf("dropping table %s: %w", table, err)
}
}
// Note: We don't drop search_participants columns because SQLite doesn't support DROP COLUMN
// on older versions, and the column is harmless if left in place.
return nil
}

9
go.mod
View File

@@ -1,13 +1,13 @@
module github.com/navidrome/navidrome
go 1.25
go 1.25.0
replace (
// Fork to fix https://github.com/navidrome/navidrome/issues/3254
github.com/dhowden/tag v0.0.0-20240417053706-3d75831295e8 => github.com/deluan/tag v0.0.0-20241002021117-dfe5e6ea396d
// Fork to implement raw tags support
go.senan.xyz/taglib => github.com/deluan/go-taglib v0.0.0-20260212150743-3f1b97cb0d1e
go.senan.xyz/taglib => github.com/deluan/go-taglib v0.0.0-20260225021432-1699562530f1
)
require (
@@ -53,7 +53,7 @@ require (
github.com/onsi/gomega v1.39.1
github.com/pelletier/go-toml/v2 v2.2.4
github.com/pocketbase/dbx v1.12.0
github.com/pressly/goose/v3 v3.26.0
github.com/pressly/goose/v3 v3.27.0
github.com/prometheus/client_golang v1.23.2
github.com/rjeczalik/notify v0.9.3
github.com/robfig/cron/v3 v3.0.1
@@ -88,7 +88,7 @@ require (
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/creack/pty v1.1.24 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 // indirect
github.com/dylibso/observe-sdk/go v0.0.0-20240828172851-9145d8ad07e1 // indirect
github.com/fsnotify/fsnotify v1.9.0 // indirect
github.com/go-logr/logr v1.4.3 // indirect
@@ -140,7 +140,6 @@ require (
go.yaml.in/yaml/v2 v2.4.3 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto v0.48.0 // indirect
golang.org/x/exp v0.0.0-20260112195511-716be5621a96 // indirect
golang.org/x/mod v0.33.0 // indirect
golang.org/x/telemetry v0.0.0-20260209163413-e7419c687ee4 // indirect
golang.org/x/tools v0.42.0 // indirect

36
go.sum
View File

@@ -1,7 +1,7 @@
dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1Xbatp0=
github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/Masterminds/squirrel v1.5.4 h1:uUcX/aBc8O7Fg9kaISIUsHXdKuqehiXAMQTYX8afzqM=
@@ -34,10 +34,10 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 h1:NMZiJj8QnKe1LgsbDayM4UoHwbvwDRwnI3hwNaAHRnc=
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40=
github.com/deluan/go-taglib v0.0.0-20260212150743-3f1b97cb0d1e h1:pwx3kmHzl1N28coJV2C1zfm2ZF0qkQcGX+Z6BvXteB4=
github.com/deluan/go-taglib v0.0.0-20260212150743-3f1b97cb0d1e/go.mod h1:sKDN0U4qXDlq6LFK+aOAkDH4Me5nDV1V/A4B+B69xBA=
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 h1:5RVFMOWjMyRy8cARdy79nAmgYw3hK/4HUq48LQ6Wwqo=
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40=
github.com/deluan/go-taglib v0.0.0-20260225021432-1699562530f1 h1:seWJmkPAb+M1ysRNGzTGS7FfdrUe9wQTHhB9p2fxDWg=
github.com/deluan/go-taglib v0.0.0-20260225021432-1699562530f1/go.mod h1:sKDN0U4qXDlq6LFK+aOAkDH4Me5nDV1V/A4B+B69xBA=
github.com/deluan/rest v0.0.0-20211102003136-6260bc399cbf h1:tb246l2Zmpt/GpF9EcHCKTtwzrd0HGfEmoODFA/qnk4=
github.com/deluan/rest v0.0.0-20211102003136-6260bc399cbf/go.mod h1:tSgDythFsl0QgS/PFWfIZqcJKnkADWneY80jaVRlqK8=
github.com/deluan/sanitize v0.0.0-20241120162836-fdfd8fdfaa55 h1:wSCnggTs2f2ji6nFwQmfwgINcmSMj0xF0oHnoyRSPe4=
@@ -143,8 +143,8 @@ github.com/kardianos/service v1.2.4 h1:XNlGtZOYNx2u91urOdg/Kfmc+gfmuIo1Dd3rEi2Og
github.com/kardianos/service v1.2.4/go.mod h1:E4V9ufUuY82F7Ztlu1eN9VXWIQxg8NoLQlmFe0MtrXc=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/klauspost/compress v1.18.4 h1:RPhnKRAQ4Fh8zU2FY/6ZFDwTVTxgJ/EMydqSTzE9a2c=
github.com/klauspost/compress v1.18.4/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
@@ -193,8 +193,8 @@ github.com/mitchellh/go-wordwrap v1.0.1 h1:TLuKupo69TCn6TQSyGxwI1EblZZEsQ0vMlAFQ
github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/ogier/pflag v0.0.1 h1:RW6JSWSu/RkSatfcLtogGfFgpim5p7ARQ10ECk5O750=
github.com/ogier/pflag v0.0.1/go.mod h1:zkFki7tvTa0tafRvTBIZTvzYyAu6kQhPZFnshFFPE+g=
github.com/onsi/ginkgo/v2 v2.28.1 h1:S4hj+HbZp40fNKuLUQOYLDgZLwNUVn19N3Atb98NCyI=
@@ -212,8 +212,8 @@ github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRI
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pocketbase/dbx v1.12.0 h1:/oLErM+A0b4xI0PWTGPqSDVjzix48PqI/bng2l0PzoA=
github.com/pocketbase/dbx v1.12.0/go.mod h1:xXRCIAKTHMgUCyCKZm55pUOdvFziJjQfXaWKhu2vhMs=
github.com/pressly/goose/v3 v3.26.0 h1:KJakav68jdH0WDvoAcj8+n61WqOIaPGgH0bJWS6jpmM=
github.com/pressly/goose/v3 v3.26.0/go.mod h1:4hC1KrritdCxtuFsqgs1R4AU5bWtTAf+cnWvfhf2DNY=
github.com/pressly/goose/v3 v3.27.0 h1:/D30gVTuQhu0WsNZYbJi4DMOsx1lNq+6SkLe+Wp59BM=
github.com/pressly/goose/v3 v3.27.0/go.mod h1:3ZBeCXqzkgIRvrEMDkYh1guvtoJTU5oMMuDdkutoM78=
github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=
github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
@@ -321,8 +321,8 @@ golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts=
golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos=
golang.org/x/exp v0.0.0-20260112195511-716be5621a96 h1:Z/6YuSHTLOHfNFdb8zVZomZr7cqNgTJvA8+Qz75D8gU=
golang.org/x/exp v0.0.0-20260112195511-716be5621a96/go.mod h1:nzimsREAkjBCIEFtHiYkrJyT+2uy9YZJB7H1k68CXZU=
golang.org/x/exp v0.0.0-20260218203240-3dfff04db8fa h1:Zt3DZoOFFYkKhDT3v7Lm9FDMEV06GpzjG2jrqW+QTE0=
golang.org/x/exp v0.0.0-20260218203240-3dfff04db8fa/go.mod h1:K79w1Vqn7PoiZn+TkNpx3BUWUQksGO3JcVX6qIjytmA=
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/image v0.36.0 h1:Iknbfm1afbgtwPTmHnS2gTM/6PPZfH+z2EFuOkSbqwc=
golang.org/x/image v0.36.0/go.mod h1:YsWD2TyyGKiIX1kZlu9QfKIsQ4nAAK9bdgdrIsE7xy4=
@@ -423,11 +423,11 @@ gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/libc v1.66.3 h1:cfCbjTUcdsKyyZZfEUKfoHcP3S0Wkvz3jgSzByEWVCQ=
modernc.org/libc v1.66.3/go.mod h1:XD9zO8kt59cANKvHPXpx7yS2ELPheAey0vjIuZOhOU8=
modernc.org/libc v1.68.0 h1:PJ5ikFOV5pwpW+VqCK1hKJuEWsonkIJhhIXyuF/91pQ=
modernc.org/libc v1.68.0/go.mod h1:NnKCYeoYgsEqnY3PgvNgAeaJnso968ygU8Z0DxjoEc0=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/sqlite v1.38.2 h1:Aclu7+tgjgcQVShZqim41Bbw9Cho0y/7WzYptXqkEek=
modernc.org/sqlite v1.38.2/go.mod h1:cPTJYSlgg3Sfg046yBShXENNtPrWrDX8bsbAQBzgQ5E=
modernc.org/sqlite v1.46.1 h1:eFJ2ShBLIEnUWlLy12raN0Z1plqmFX9Qe3rjQTKt6sU=
modernc.org/sqlite v1.46.1/go.mod h1:CzbrU2lSB1DKUusvwGz7rqEKIq+NUd8GWuBBZDs9/nA=

View File

@@ -9,11 +9,12 @@ import (
//goland:noinspection GoBoolExpressions
func main() {
// This import is used to force the inclusion of the `netgo` tag when compiling the project.
// These references force the inclusion of build tags when compiling the project.
// If you get compilation errors like "undefined: buildtags.NETGO", this means you forgot to specify
// the `netgo` build tag when compiling the project.
// the required build tags when compiling the project.
// To avoid these kind of errors, you should use `make build` to compile the project.
_ = buildtags.NETGO
_ = buildtags.SQLITE_FTS5
cmd.Execute()
}

View File

@@ -1,11 +1,14 @@
package model
import (
"fmt"
"iter"
"math"
"sync"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/gohugoio/hashstructure"
)
@@ -70,6 +73,13 @@ func (a Album) CoverArtID() ArtworkID {
return artworkIDFromAlbum(a)
}
func (a Album) FullName() string {
if conf.Server.Subsonic.AppendAlbumVersion && len(a.Tags[TagAlbumVersion]) > 0 {
return fmt.Sprintf("%s (%s)", a.Name, a.Tags[TagAlbumVersion][0])
}
return a.Name
}
// Equals compares two Album structs, ignoring calculated fields
func (a Album) Equals(other Album) bool {
// Normalize float32 values to avoid false negatives

View File

@@ -3,11 +3,30 @@ package model_test
import (
"encoding/json"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
. "github.com/navidrome/navidrome/model"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("Album", func() {
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
})
DescribeTable("FullName",
func(enabled bool, tags Tags, expected string) {
conf.Server.Subsonic.AppendAlbumVersion = enabled
a := Album{Name: "Album", Tags: tags}
Expect(a.FullName()).To(Equal(expected))
},
Entry("appends version when enabled and tag is present", true, Tags{TagAlbumVersion: []string{"Remastered"}}, "Album (Remastered)"),
Entry("returns just name when disabled", false, Tags{TagAlbumVersion: []string{"Remastered"}}, "Album"),
Entry("returns just name when tag is absent", true, Tags{}, "Album"),
Entry("returns just name when tag is an empty slice", true, Tags{TagAlbumVersion: []string{}}, "Album"),
)
})
var _ = Describe("Albums", func() {
var albums Albums

View File

@@ -95,6 +95,25 @@ func (c Criteria) ToSql() (sql string, args []any, err error) {
return c.Expression.ToSql()
}
// RequiredJoins inspects the expression tree and Sort field to determine which
// additional JOINs are needed when evaluating this criteria.
func (c Criteria) RequiredJoins() JoinType {
result := JoinNone
if c.Expression != nil {
result |= extractJoinTypes(c.Expression)
}
// Also check Sort fields
if c.Sort != "" {
for _, p := range strings.Split(c.Sort, ",") {
p = strings.TrimSpace(p)
p = strings.TrimLeft(p, "+-")
p = strings.TrimSpace(p)
result |= fieldJoinType(p)
}
}
return result
}
func (c Criteria) ChildPlaylistIds() []string {
if c.Expression == nil {
return nil

View File

@@ -27,6 +27,7 @@ var _ = Describe("Criteria", func() {
StartsWith{"comment": "this"},
InTheRange{"year": []int{1980, 1990}},
IsNot{"genre": "Rock"},
Gt{"albumrating": 3},
},
},
Sort: "title",
@@ -48,7 +49,8 @@ var _ = Describe("Criteria", func() {
{ "all": [
{ "startsWith": {"comment": "this"} },
{ "inTheRange": {"year":[1980,1990]} },
{ "isNot": { "genre": "Rock" }}
{ "isNot": { "genre": "Rock" }},
{ "gt": { "albumrating": 3 } }
]
}
],
@@ -68,10 +70,10 @@ var _ = Describe("Criteria", func() {
gomega.Expect(err).ToNot(gomega.HaveOccurred())
gomega.Expect(sql).To(gomega.Equal(
`(media_file.title LIKE ? AND media_file.title NOT LIKE ? ` +
`AND (not exists (select 1 from json_tree(participants, '$.artist') where key='name' and value = ?) ` +
`AND (not exists (select 1 from json_tree(media_file.participants, '$.artist') where key='name' and value = ?) ` +
`OR media_file.album = ?) AND (media_file.comment LIKE ? AND (media_file.year >= ? AND media_file.year <= ?) ` +
`AND not exists (select 1 from json_tree(tags, '$.genre') where key='value' and value = ?)))`))
gomega.Expect(args).To(gomega.HaveExactElements("%love%", "%hate%", "u2", "best of", "this%", 1980, 1990, "Rock"))
`AND not exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value = ?) AND COALESCE(album_annotation.rating, 0) > ?))`))
gomega.Expect(args).To(gomega.HaveExactElements("%love%", "%hate%", "u2", "best of", "this%", 1980, 1990, "Rock", 3))
})
It("marshals to JSON", func() {
j, err := json.Marshal(goObj)
@@ -172,13 +174,95 @@ var _ = Describe("Criteria", func() {
sql, args, err := goObj.ToSql()
gomega.Expect(err).ToNot(gomega.HaveOccurred())
gomega.Expect(sql).To(gomega.Equal(
`(exists (select 1 from json_tree(participants, '$.artist') where key='name' and value = ?) AND ` +
`exists (select 1 from json_tree(participants, '$.composer') where key='name' and value LIKE ?))`,
`(exists (select 1 from json_tree(media_file.participants, '$.artist') where key='name' and value = ?) AND ` +
`exists (select 1 from json_tree(media_file.participants, '$.composer') where key='name' and value LIKE ?))`,
))
gomega.Expect(args).To(gomega.HaveExactElements("The Beatles", "%Lennon%"))
})
})
Describe("RequiredJoins", func() {
It("returns JoinNone when no annotation fields are used", func() {
c := Criteria{
Expression: All{
Contains{"title": "love"},
},
}
gomega.Expect(c.RequiredJoins()).To(gomega.Equal(JoinNone))
})
It("returns JoinNone for media_file annotation fields", func() {
c := Criteria{
Expression: All{
Is{"loved": true},
Gt{"playCount": 5},
},
}
gomega.Expect(c.RequiredJoins()).To(gomega.Equal(JoinNone))
})
It("returns JoinAlbumAnnotation for album annotation fields", func() {
c := Criteria{
Expression: All{
Gt{"albumRating": 3},
},
}
gomega.Expect(c.RequiredJoins()).To(gomega.Equal(JoinAlbumAnnotation))
})
It("returns JoinArtistAnnotation for artist annotation fields", func() {
c := Criteria{
Expression: All{
Is{"artistLoved": true},
},
}
gomega.Expect(c.RequiredJoins()).To(gomega.Equal(JoinArtistAnnotation))
})
It("returns both join types when both are used", func() {
c := Criteria{
Expression: All{
Gt{"albumRating": 3},
Is{"artistLoved": true},
},
}
j := c.RequiredJoins()
gomega.Expect(j.Has(JoinAlbumAnnotation)).To(gomega.BeTrue())
gomega.Expect(j.Has(JoinArtistAnnotation)).To(gomega.BeTrue())
})
It("detects join types in nested expressions", func() {
c := Criteria{
Expression: All{
Any{
All{
Is{"albumLoved": true},
},
},
Any{
Gt{"artistPlayCount": 10},
},
},
}
j := c.RequiredJoins()
gomega.Expect(j.Has(JoinAlbumAnnotation)).To(gomega.BeTrue())
gomega.Expect(j.Has(JoinArtistAnnotation)).To(gomega.BeTrue())
})
It("detects join types from Sort field", func() {
c := Criteria{
Expression: All{
Contains{"title": "love"},
},
Sort: "albumRating",
}
gomega.Expect(c.RequiredJoins().Has(JoinAlbumAnnotation)).To(gomega.BeTrue())
})
It("detects join types from Sort field with direction prefix", func() {
c := Criteria{
Expression: All{
Contains{"title": "love"},
},
Sort: "-artistRating",
}
gomega.Expect(c.RequiredJoins().Has(JoinArtistAnnotation)).To(gomega.BeTrue())
})
})
Context("with child playlists", func() {
var (
topLevelInPlaylistID string

View File

@@ -9,45 +9,71 @@ import (
"github.com/navidrome/navidrome/log"
)
// JoinType is a bitmask indicating which additional JOINs are needed by a smart playlist expression.
type JoinType int
const (
JoinNone JoinType = 0
JoinAlbumAnnotation JoinType = 1 << iota
JoinArtistAnnotation
)
// Has returns true if j contains all bits in other.
func (j JoinType) Has(other JoinType) bool { return j&other != 0 }
var fieldMap = map[string]*mappedField{
"title": {field: "media_file.title"},
"album": {field: "media_file.album"},
"hascoverart": {field: "media_file.has_cover_art"},
"tracknumber": {field: "media_file.track_number"},
"discnumber": {field: "media_file.disc_number"},
"year": {field: "media_file.year"},
"date": {field: "media_file.date", alias: "recordingdate"},
"originalyear": {field: "media_file.original_year"},
"originaldate": {field: "media_file.original_date"},
"releaseyear": {field: "media_file.release_year"},
"releasedate": {field: "media_file.release_date"},
"size": {field: "media_file.size"},
"compilation": {field: "media_file.compilation"},
"explicitstatus": {field: "media_file.explicit_status"},
"dateadded": {field: "media_file.created_at"},
"datemodified": {field: "media_file.updated_at"},
"discsubtitle": {field: "media_file.disc_subtitle"},
"comment": {field: "media_file.comment"},
"lyrics": {field: "media_file.lyrics"},
"sorttitle": {field: "media_file.sort_title"},
"sortalbum": {field: "media_file.sort_album_name"},
"sortartist": {field: "media_file.sort_artist_name"},
"sortalbumartist": {field: "media_file.sort_album_artist_name"},
"albumcomment": {field: "media_file.mbz_album_comment"},
"catalognumber": {field: "media_file.catalog_num"},
"filepath": {field: "media_file.path"},
"filetype": {field: "media_file.suffix"},
"duration": {field: "media_file.duration"},
"bitrate": {field: "media_file.bit_rate"},
"bitdepth": {field: "media_file.bit_depth"},
"bpm": {field: "media_file.bpm"},
"channels": {field: "media_file.channels"},
"loved": {field: "COALESCE(annotation.starred, false)"},
"dateloved": {field: "annotation.starred_at"},
"lastplayed": {field: "annotation.play_date"},
"daterated": {field: "annotation.rated_at"},
"playcount": {field: "COALESCE(annotation.play_count, 0)"},
"rating": {field: "COALESCE(annotation.rating, 0)"},
"title": {field: "media_file.title"},
"album": {field: "media_file.album"},
"hascoverart": {field: "media_file.has_cover_art"},
"tracknumber": {field: "media_file.track_number"},
"discnumber": {field: "media_file.disc_number"},
"year": {field: "media_file.year"},
"date": {field: "media_file.date", alias: "recordingdate"},
"originalyear": {field: "media_file.original_year"},
"originaldate": {field: "media_file.original_date"},
"releaseyear": {field: "media_file.release_year"},
"releasedate": {field: "media_file.release_date"},
"size": {field: "media_file.size"},
"compilation": {field: "media_file.compilation"},
"explicitstatus": {field: "media_file.explicit_status"},
"dateadded": {field: "media_file.created_at"},
"datemodified": {field: "media_file.updated_at"},
"discsubtitle": {field: "media_file.disc_subtitle"},
"comment": {field: "media_file.comment"},
"lyrics": {field: "media_file.lyrics"},
"sorttitle": {field: "media_file.sort_title"},
"sortalbum": {field: "media_file.sort_album_name"},
"sortartist": {field: "media_file.sort_artist_name"},
"sortalbumartist": {field: "media_file.sort_album_artist_name"},
"albumcomment": {field: "media_file.mbz_album_comment"},
"catalognumber": {field: "media_file.catalog_num"},
"filepath": {field: "media_file.path"},
"filetype": {field: "media_file.suffix"},
"duration": {field: "media_file.duration"},
"bitrate": {field: "media_file.bit_rate"},
"bitdepth": {field: "media_file.bit_depth"},
"bpm": {field: "media_file.bpm"},
"channels": {field: "media_file.channels"},
"loved": {field: "COALESCE(annotation.starred, false)"},
"dateloved": {field: "annotation.starred_at"},
"lastplayed": {field: "annotation.play_date"},
"daterated": {field: "annotation.rated_at"},
"playcount": {field: "COALESCE(annotation.play_count, 0)"},
"rating": {field: "COALESCE(annotation.rating, 0)"},
"albumrating": {field: "COALESCE(album_annotation.rating, 0)", joinType: JoinAlbumAnnotation},
"albumloved": {field: "COALESCE(album_annotation.starred, false)", joinType: JoinAlbumAnnotation},
"albumplaycount": {field: "COALESCE(album_annotation.play_count, 0)", joinType: JoinAlbumAnnotation},
"albumlastplayed": {field: "album_annotation.play_date", joinType: JoinAlbumAnnotation},
"albumdateloved": {field: "album_annotation.starred_at", joinType: JoinAlbumAnnotation},
"albumdaterated": {field: "album_annotation.rated_at", joinType: JoinAlbumAnnotation},
"artistrating": {field: "COALESCE(artist_annotation.rating, 0)", joinType: JoinArtistAnnotation},
"artistloved": {field: "COALESCE(artist_annotation.starred, false)", joinType: JoinArtistAnnotation},
"artistplaycount": {field: "COALESCE(artist_annotation.play_count, 0)", joinType: JoinArtistAnnotation},
"artistlastplayed": {field: "artist_annotation.play_date", joinType: JoinArtistAnnotation},
"artistdateloved": {field: "artist_annotation.starred_at", joinType: JoinArtistAnnotation},
"artistdaterated": {field: "artist_annotation.rated_at", joinType: JoinArtistAnnotation},
"mbz_album_id": {field: "media_file.mbz_album_id"},
"mbz_album_artist_id": {field: "media_file.mbz_album_artist_id"},
"mbz_artist_id": {field: "media_file.mbz_artist_id"},
@@ -65,12 +91,13 @@ var fieldMap = map[string]*mappedField{
}
type mappedField struct {
field string
order string
isRole bool // true if the field is a role (e.g. "artist", "composer", "conductor", etc.)
isTag bool // true if the field is a tag imported from the file metadata
alias string // name from `mappings.yml` that may differ from the name used in the smart playlist
numeric bool // true if the field/tag should be treated as numeric
field string
order string
isRole bool // true if the field is a role (e.g. "artist", "composer", "conductor", etc.)
isTag bool // true if the field is a tag imported from the file metadata
alias string // name from `mappings.yml` that may differ from the name used in the smart playlist
numeric bool // true if the field/tag should be treated as numeric
joinType JoinType // which additional JOINs this field requires
}
func mapFields(expr map[string]any) map[string]any {
@@ -169,7 +196,7 @@ func (e tagCond) ToSql() (string, []any, error) {
}
}
cond = fmt.Sprintf("exists (select 1 from json_tree(tags, '$.%s') where key='value' and %s)",
cond = fmt.Sprintf("exists (select 1 from json_tree(media_file.tags, '$.%s') where key='value' and %s)",
tagName, cond)
if e.not {
cond = "not " + cond
@@ -189,7 +216,7 @@ type roleCond struct {
func (e roleCond) ToSql() (string, []any, error) {
cond, args, err := e.cond.ToSql()
cond = fmt.Sprintf(`exists (select 1 from json_tree(participants, '$.%s') where key='name' and %s)`,
cond = fmt.Sprintf(`exists (select 1 from json_tree(media_file.participants, '$.%s') where key='name' and %s)`,
e.role, cond)
if e.not {
cond = "not " + cond
@@ -197,6 +224,38 @@ func (e roleCond) ToSql() (string, []any, error) {
return cond, args, err
}
// fieldJoinType returns the JoinType for a given field name (case-insensitive).
func fieldJoinType(name string) JoinType {
if f, ok := fieldMap[strings.ToLower(name)]; ok {
return f.joinType
}
return JoinNone
}
// extractJoinTypes walks an expression tree and collects all required JoinType flags.
func extractJoinTypes(expr any) JoinType {
result := JoinNone
switch e := expr.(type) {
case All:
for _, sub := range e {
result |= extractJoinTypes(sub)
}
case Any:
for _, sub := range e {
result |= extractJoinTypes(sub)
}
default:
// Leaf expression: use reflection to check if it's a map with field names
rv := reflect.ValueOf(expr)
if rv.Kind() == reflect.Map && rv.Type().Key().Kind() == reflect.String {
for _, key := range rv.MapKeys() {
result |= fieldJoinType(key.String())
}
}
}
return result
}
// AddRoles adds roles to the field map. This is used to add all artist roles to the field map, so they can be used in
// smart playlists. If a role already exists in the field map, it is ignored, so calls to this function are idempotent.
func AddRoles(roles []string) {

View File

@@ -54,23 +54,43 @@ var _ = Describe("Operators", func() {
Entry("inTheLast", InTheLast{"lastPlayed": 30}, "annotation.play_date > ?", StartOfPeriod(30, time.Now())),
Entry("notInTheLast", NotInTheLast{"lastPlayed": 30}, "(annotation.play_date < ? OR annotation.play_date IS NULL)", StartOfPeriod(30, time.Now())),
// Album annotation fields
Entry("albumRating", Gt{"albumRating": 3}, "COALESCE(album_annotation.rating, 0) > ?", 3),
Entry("albumLoved", Is{"albumLoved": true}, "COALESCE(album_annotation.starred, false) = ?", true),
Entry("albumPlayCount", Gt{"albumPlayCount": 5}, "COALESCE(album_annotation.play_count, 0) > ?", 5),
Entry("albumLastPlayed", After{"albumLastPlayed": rangeStart}, "album_annotation.play_date > ?", rangeStart),
Entry("albumDateLoved", Before{"albumDateLoved": rangeStart}, "album_annotation.starred_at < ?", rangeStart),
Entry("albumDateRated", After{"albumDateRated": rangeStart}, "album_annotation.rated_at > ?", rangeStart),
Entry("albumLastPlayed inTheLast", InTheLast{"albumLastPlayed": 30}, "album_annotation.play_date > ?", StartOfPeriod(30, time.Now())),
Entry("albumLastPlayed notInTheLast", NotInTheLast{"albumLastPlayed": 30}, "(album_annotation.play_date < ? OR album_annotation.play_date IS NULL)", StartOfPeriod(30, time.Now())),
// Artist annotation fields
Entry("artistRating", Gt{"artistRating": 3}, "COALESCE(artist_annotation.rating, 0) > ?", 3),
Entry("artistLoved", Is{"artistLoved": true}, "COALESCE(artist_annotation.starred, false) = ?", true),
Entry("artistPlayCount", Gt{"artistPlayCount": 5}, "COALESCE(artist_annotation.play_count, 0) > ?", 5),
Entry("artistLastPlayed", After{"artistLastPlayed": rangeStart}, "artist_annotation.play_date > ?", rangeStart),
Entry("artistDateLoved", Before{"artistDateLoved": rangeStart}, "artist_annotation.starred_at < ?", rangeStart),
Entry("artistDateRated", After{"artistDateRated": rangeStart}, "artist_annotation.rated_at > ?", rangeStart),
Entry("artistLastPlayed inTheLast", InTheLast{"artistLastPlayed": 30}, "artist_annotation.play_date > ?", StartOfPeriod(30, time.Now())),
Entry("artistLastPlayed notInTheLast", NotInTheLast{"artistLastPlayed": 30}, "(artist_annotation.play_date < ? OR artist_annotation.play_date IS NULL)", StartOfPeriod(30, time.Now())),
// Tag tests
Entry("tag is [string]", Is{"genre": "Rock"}, "exists (select 1 from json_tree(tags, '$.genre') where key='value' and value = ?)", "Rock"),
Entry("tag isNot [string]", IsNot{"genre": "Rock"}, "not exists (select 1 from json_tree(tags, '$.genre') where key='value' and value = ?)", "Rock"),
Entry("tag gt", Gt{"genre": "A"}, "exists (select 1 from json_tree(tags, '$.genre') where key='value' and value > ?)", "A"),
Entry("tag lt", Lt{"genre": "Z"}, "exists (select 1 from json_tree(tags, '$.genre') where key='value' and value < ?)", "Z"),
Entry("tag contains", Contains{"genre": "Rock"}, "exists (select 1 from json_tree(tags, '$.genre') where key='value' and value LIKE ?)", "%Rock%"),
Entry("tag not contains", NotContains{"genre": "Rock"}, "not exists (select 1 from json_tree(tags, '$.genre') where key='value' and value LIKE ?)", "%Rock%"),
Entry("tag startsWith", StartsWith{"genre": "Soft"}, "exists (select 1 from json_tree(tags, '$.genre') where key='value' and value LIKE ?)", "Soft%"),
Entry("tag endsWith", EndsWith{"genre": "Rock"}, "exists (select 1 from json_tree(tags, '$.genre') where key='value' and value LIKE ?)", "%Rock"),
Entry("tag is [string]", Is{"genre": "Rock"}, "exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value = ?)", "Rock"),
Entry("tag isNot [string]", IsNot{"genre": "Rock"}, "not exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value = ?)", "Rock"),
Entry("tag gt", Gt{"genre": "A"}, "exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value > ?)", "A"),
Entry("tag lt", Lt{"genre": "Z"}, "exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value < ?)", "Z"),
Entry("tag contains", Contains{"genre": "Rock"}, "exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value LIKE ?)", "%Rock%"),
Entry("tag not contains", NotContains{"genre": "Rock"}, "not exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value LIKE ?)", "%Rock%"),
Entry("tag startsWith", StartsWith{"genre": "Soft"}, "exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value LIKE ?)", "Soft%"),
Entry("tag endsWith", EndsWith{"genre": "Rock"}, "exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value LIKE ?)", "%Rock"),
// Artist roles tests
Entry("role is [string]", Is{"artist": "u2"}, "exists (select 1 from json_tree(participants, '$.artist') where key='name' and value = ?)", "u2"),
Entry("role isNot [string]", IsNot{"artist": "u2"}, "not exists (select 1 from json_tree(participants, '$.artist') where key='name' and value = ?)", "u2"),
Entry("role contains [string]", Contains{"artist": "u2"}, "exists (select 1 from json_tree(participants, '$.artist') where key='name' and value LIKE ?)", "%u2%"),
Entry("role not contains [string]", NotContains{"artist": "u2"}, "not exists (select 1 from json_tree(participants, '$.artist') where key='name' and value LIKE ?)", "%u2%"),
Entry("role startsWith [string]", StartsWith{"composer": "John"}, "exists (select 1 from json_tree(participants, '$.composer') where key='name' and value LIKE ?)", "John%"),
Entry("role endsWith [string]", EndsWith{"composer": "Lennon"}, "exists (select 1 from json_tree(participants, '$.composer') where key='name' and value LIKE ?)", "%Lennon"),
Entry("role is [string]", Is{"artist": "u2"}, "exists (select 1 from json_tree(media_file.participants, '$.artist') where key='name' and value = ?)", "u2"),
Entry("role isNot [string]", IsNot{"artist": "u2"}, "not exists (select 1 from json_tree(media_file.participants, '$.artist') where key='name' and value = ?)", "u2"),
Entry("role contains [string]", Contains{"artist": "u2"}, "exists (select 1 from json_tree(media_file.participants, '$.artist') where key='name' and value LIKE ?)", "%u2%"),
Entry("role not contains [string]", NotContains{"artist": "u2"}, "not exists (select 1 from json_tree(media_file.participants, '$.artist') where key='name' and value LIKE ?)", "%u2%"),
Entry("role startsWith [string]", StartsWith{"composer": "John"}, "exists (select 1 from json_tree(media_file.participants, '$.composer') where key='name' and value LIKE ?)", "John%"),
Entry("role endsWith [string]", EndsWith{"composer": "Lennon"}, "exists (select 1 from json_tree(media_file.participants, '$.composer') where key='name' and value LIKE ?)", "%Lennon"),
)
// TODO Validate operators that are not valid for each field type.
@@ -88,7 +108,7 @@ var _ = Describe("Operators", func() {
op := EndsWith{"mood": "Soft"}
sql, args, err := op.ToSql()
gomega.Expect(err).ToNot(gomega.HaveOccurred())
gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(tags, '$.mood') where key='value' and value LIKE ?)"))
gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(media_file.tags, '$.mood') where key='value' and value LIKE ?)"))
gomega.Expect(args).To(gomega.HaveExactElements("%Soft"))
})
It("casts numeric comparisons", func() {
@@ -96,7 +116,7 @@ var _ = Describe("Operators", func() {
op := Lt{"rate": 6}
sql, args, err := op.ToSql()
gomega.Expect(err).ToNot(gomega.HaveOccurred())
gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(tags, '$.rate') where key='value' and CAST(value AS REAL) < ?)"))
gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(media_file.tags, '$.rate') where key='value' and CAST(value AS REAL) < ?)"))
gomega.Expect(args).To(gomega.HaveExactElements(6))
})
It("skips unknown tag names", func() {
@@ -110,7 +130,7 @@ var _ = Describe("Operators", func() {
op := Contains{"releasetype": "soundtrack"}
sql, args, err := op.ToSql()
gomega.Expect(err).ToNot(gomega.HaveOccurred())
gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(tags, '$.releasetype') where key='value' and value LIKE ?)"))
gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(media_file.tags, '$.releasetype') where key='value' and value LIKE ?)"))
gomega.Expect(args).To(gomega.HaveExactElements("%soundtrack%"))
})
It("supports albumtype as alias for releasetype", func() {
@@ -118,7 +138,7 @@ var _ = Describe("Operators", func() {
op := Contains{"albumtype": "live"}
sql, args, err := op.ToSql()
gomega.Expect(err).ToNot(gomega.HaveOccurred())
gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(tags, '$.releasetype') where key='value' and value LIKE ?)"))
gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(media_file.tags, '$.releasetype') where key='value' and value LIKE ?)"))
gomega.Expect(args).To(gomega.HaveExactElements("%live%"))
})
It("supports albumtype alias with Is operator", func() {
@@ -127,7 +147,7 @@ var _ = Describe("Operators", func() {
sql, args, err := op.ToSql()
gomega.Expect(err).ToNot(gomega.HaveOccurred())
// Should query $.releasetype, not $.albumtype
gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(tags, '$.releasetype') where key='value' and value = ?)"))
gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(media_file.tags, '$.releasetype') where key='value' and value = ?)"))
gomega.Expect(args).To(gomega.HaveExactElements("album"))
})
It("supports albumtype alias with IsNot operator", func() {
@@ -136,7 +156,7 @@ var _ = Describe("Operators", func() {
sql, args, err := op.ToSql()
gomega.Expect(err).ToNot(gomega.HaveOccurred())
// Should query $.releasetype, not $.albumtype
gomega.Expect(sql).To(gomega.Equal("not exists (select 1 from json_tree(tags, '$.releasetype') where key='value' and value = ?)"))
gomega.Expect(sql).To(gomega.Equal("not exists (select 1 from json_tree(media_file.tags, '$.releasetype') where key='value' and value = ?)"))
gomega.Expect(args).To(gomega.HaveExactElements("compilation"))
})
})
@@ -147,7 +167,7 @@ var _ = Describe("Operators", func() {
op := EndsWith{"producer": "Eno"}
sql, args, err := op.ToSql()
gomega.Expect(err).ToNot(gomega.HaveOccurred())
gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(participants, '$.producer') where key='name' and value LIKE ?)"))
gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(media_file.participants, '$.producer') where key='name' and value LIKE ?)"))
gomega.Expect(args).To(gomega.HaveExactElements("%Eno"))
})
It("skips unknown roles", func() {

View File

@@ -95,12 +95,19 @@ type MediaFile struct {
}
func (mf MediaFile) FullTitle() string {
if conf.Server.Subsonic.AppendSubtitle && mf.Tags[TagSubtitle] != nil {
if conf.Server.Subsonic.AppendSubtitle && len(mf.Tags[TagSubtitle]) > 0 {
return fmt.Sprintf("%s (%s)", mf.Title, mf.Tags[TagSubtitle][0])
}
return mf.Title
}
func (mf MediaFile) FullAlbumName() string {
if conf.Server.Subsonic.AppendAlbumVersion && len(mf.Tags[TagAlbumVersion]) > 0 {
return fmt.Sprintf("%s (%s)", mf.Album, mf.Tags[TagAlbumVersion][0])
}
return mf.Album
}
func (mf MediaFile) ContentType() string {
return mime.TypeByExtension("." + mf.Suffix)
}

View File

@@ -475,7 +475,29 @@ var _ = Describe("MediaFile", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.EnableMediaFileCoverArt = true
})
Describe(".CoverArtId()", func() {
DescribeTable("FullTitle",
func(enabled bool, tags Tags, expected string) {
conf.Server.Subsonic.AppendSubtitle = enabled
mf := MediaFile{Title: "Song", Tags: tags}
Expect(mf.FullTitle()).To(Equal(expected))
},
Entry("appends subtitle when enabled and tag is present", true, Tags{TagSubtitle: []string{"Live"}}, "Song (Live)"),
Entry("returns just title when disabled", false, Tags{TagSubtitle: []string{"Live"}}, "Song"),
Entry("returns just title when tag is absent", true, Tags{}, "Song"),
Entry("returns just title when tag is an empty slice", true, Tags{TagSubtitle: []string{}}, "Song"),
)
DescribeTable("FullAlbumName",
func(enabled bool, tags Tags, expected string) {
conf.Server.Subsonic.AppendAlbumVersion = enabled
mf := MediaFile{Album: "Album", Tags: tags}
Expect(mf.FullAlbumName()).To(Equal(expected))
},
Entry("appends version when enabled and tag is present", true, Tags{TagAlbumVersion: []string{"Deluxe Edition"}}, "Album (Deluxe Edition)"),
Entry("returns just album name when disabled", false, Tags{TagAlbumVersion: []string{"Deluxe Edition"}}, "Album"),
Entry("returns just album name when tag is absent", true, Tags{}, "Album"),
Entry("returns just album name when tag is an empty slice", true, Tags{TagAlbumVersion: []string{}}, "Album"),
)
Describe("CoverArtId()", func() {
It("returns its own id if it HasCoverArt", func() {
mf := MediaFile{ID: "111", AlbumID: "1", HasCoverArt: true}
id := mf.CoverArtID()

View File

@@ -1,5 +1,5 @@
package model
type SearchableRepository[T any] interface {
Search(q string, offset, size int, options ...QueryOptions) (T, error)
Search(q string, options ...QueryOptions) (T, error)
}

View File

@@ -12,7 +12,6 @@ import (
. "github.com/Masterminds/squirrel"
"github.com/deluan/rest"
"github.com/google/uuid"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
@@ -62,11 +61,14 @@ func (a *dbAlbum) PostScan() error {
func (a *dbAlbum) PostMapArgs(args map[string]any) error {
fullText := []string{a.Name, a.SortAlbumName, a.AlbumArtist}
fullText = append(fullText, a.Album.Participants.AllNames()...)
participantNames := a.Album.Participants.AllNames()
fullText = append(fullText, participantNames...)
fullText = append(fullText, slices.Collect(maps.Values(a.Album.Discs))...)
fullText = append(fullText, a.Album.Tags[model.TagAlbumVersion]...)
fullText = append(fullText, a.Album.Tags[model.TagCatalogNumber]...)
args["full_text"] = formatFullText(fullText...)
args["search_participants"] = strings.Join(participantNames, " ")
args["search_normalized"] = normalizeForFTS(a.Name, a.AlbumArtist)
args["tags"] = marshalTags(a.Album.Tags)
args["participants"] = marshalParticipants(a.Album.Participants)
@@ -350,18 +352,21 @@ func (r *albumRepository) purgeEmpty(libraryIDs ...int) error {
return nil
}
func (r *albumRepository) Search(q string, offset int, size int, options ...model.QueryOptions) (model.Albums, error) {
var albumSearchConfig = searchConfig{
NaturalOrder: "album.rowid",
OrderBy: []string{"name"},
MBIDFields: []string{"mbz_album_id", "mbz_release_group_id"},
}
func (r *albumRepository) Search(q string, options ...model.QueryOptions) (model.Albums, error) {
var opts model.QueryOptions
if len(options) > 0 {
opts = options[0]
}
var res dbAlbums
if uuid.Validate(q) == nil {
err := r.searchByMBID(r.selectAlbum(options...), q, []string{"mbz_album_id", "mbz_release_group_id"}, &res)
if err != nil {
return nil, fmt.Errorf("searching album by MBID %q: %w", q, err)
}
} else {
err := r.doSearch(r.selectAlbum(options...), q, offset, size, &res, "album.rowid", "name")
if err != nil {
return nil, fmt.Errorf("searching album by query %q: %w", q, err)
}
err := r.doSearch(r.selectAlbum(options...), q, &res, albumSearchConfig, opts)
if err != nil {
return nil, fmt.Errorf("searching album %q: %w", q, err)
}
return res.toModels(), nil
}

View File

@@ -56,17 +56,23 @@ var _ = Describe("AlbumRepository", func() {
It("returns all records sorted", func() {
Expect(GetAll(model.QueryOptions{Sort: "name"})).To(Equal(model.Albums{
albumAbbeyRoad,
albumWithVersion,
albumCJK,
albumMultiDisc,
albumRadioactivity,
albumSgtPeppers,
albumPunctuation,
}))
})
It("returns all records sorted desc", func() {
Expect(GetAll(model.QueryOptions{Sort: "name", Order: "desc"})).To(Equal(model.Albums{
albumPunctuation,
albumSgtPeppers,
albumRadioactivity,
albumMultiDisc,
albumCJK,
albumWithVersion,
albumAbbeyRoad,
}))
})

View File

@@ -11,7 +11,6 @@ import (
. "github.com/Masterminds/squirrel"
"github.com/deluan/rest"
"github.com/google/uuid"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
@@ -102,6 +101,7 @@ func (a *dbArtist) PostMapArgs(m map[string]any) error {
similarArtists, _ := json.Marshal(sa)
m["similar_artists"] = string(similarArtists)
m["full_text"] = formatFullText(a.Name, a.SortArtistName)
m["search_normalized"] = normalizeForFTS(a.Name)
// Do not override the sort_artist_name and mbz_artist_id fields if they are empty
// TODO: Better way to handle this?
@@ -512,20 +512,25 @@ func (r *artistRepository) RefreshStats(allArtists bool) (int64, error) {
return totalRowsAffected, nil
}
func (r *artistRepository) Search(q string, offset int, size int, options ...model.QueryOptions) (model.Artists, error) {
var res dbArtists
if uuid.Validate(q) == nil {
err := r.searchByMBID(r.selectArtist(options...), q, []string{"mbz_artist_id"}, &res)
if err != nil {
return nil, fmt.Errorf("searching artist by MBID %q: %w", q, err)
}
} else {
func (r *artistRepository) searchCfg() searchConfig {
return searchConfig{
// Natural order for artists is more performant by ID, due to GROUP BY clause in selectArtist
err := r.doSearch(r.selectArtist(options...), q, offset, size, &res, "artist.id",
"sum(json_extract(stats, '$.total.m')) desc", "name")
if err != nil {
return nil, fmt.Errorf("searching artist by query %q: %w", q, err)
}
NaturalOrder: "artist.id",
OrderBy: []string{"sum(json_extract(stats, '$.total.m')) desc", "name"},
MBIDFields: []string{"mbz_artist_id"},
LibraryFilter: r.applyLibraryFilterToArtistQuery,
}
}
func (r *artistRepository) Search(q string, options ...model.QueryOptions) (model.Artists, error) {
var opts model.QueryOptions
if len(options) > 0 {
opts = options[0]
}
var res dbArtists
err := r.doSearch(r.selectArtist(options...), q, &res, r.searchCfg(), opts)
if err != nil {
return nil, fmt.Errorf("searching artist %q: %w", q, err)
}
return res.toModels(), nil
}

View File

@@ -193,7 +193,7 @@ var _ = Describe("ArtistRepository", func() {
Describe("Basic Operations", func() {
Describe("Count", func() {
It("returns the number of artists in the DB", func() {
Expect(repo.CountAll()).To(Equal(int64(2)))
Expect(repo.CountAll()).To(Equal(int64(4)))
})
})
@@ -228,13 +228,19 @@ var _ = Describe("ArtistRepository", func() {
idx, err := repo.GetIndex(false, []int{1})
Expect(err).ToNot(HaveOccurred())
Expect(idx).To(HaveLen(2))
Expect(idx).To(HaveLen(4))
Expect(idx[0].ID).To(Equal("F"))
Expect(idx[0].Artists).To(HaveLen(1))
Expect(idx[0].Artists[0].Name).To(Equal(artistBeatles.Name))
Expect(idx[1].ID).To(Equal("K"))
Expect(idx[1].Artists).To(HaveLen(1))
Expect(idx[1].Artists[0].Name).To(Equal(artistKraftwerk.Name))
Expect(idx[2].ID).To(Equal("R"))
Expect(idx[2].Artists).To(HaveLen(1))
Expect(idx[2].Artists[0].Name).To(Equal(artistPunctuation.Name))
Expect(idx[3].ID).To(Equal("S"))
Expect(idx[3].Artists).To(HaveLen(1))
Expect(idx[3].Artists[0].Name).To(Equal(artistCJK.Name))
// Restore the original value
artistBeatles.SortArtistName = ""
@@ -246,13 +252,19 @@ var _ = Describe("ArtistRepository", func() {
XIt("returns the index when PreferSortTags is true and SortArtistName is empty", func() {
idx, err := repo.GetIndex(false, []int{1})
Expect(err).ToNot(HaveOccurred())
Expect(idx).To(HaveLen(2))
Expect(idx).To(HaveLen(4))
Expect(idx[0].ID).To(Equal("B"))
Expect(idx[0].Artists).To(HaveLen(1))
Expect(idx[0].Artists[0].Name).To(Equal(artistBeatles.Name))
Expect(idx[1].ID).To(Equal("K"))
Expect(idx[1].Artists).To(HaveLen(1))
Expect(idx[1].Artists[0].Name).To(Equal(artistKraftwerk.Name))
Expect(idx[2].ID).To(Equal("R"))
Expect(idx[2].Artists).To(HaveLen(1))
Expect(idx[2].Artists[0].Name).To(Equal(artistPunctuation.Name))
Expect(idx[3].ID).To(Equal("S"))
Expect(idx[3].Artists).To(HaveLen(1))
Expect(idx[3].Artists[0].Name).To(Equal(artistCJK.Name))
})
})
@@ -268,13 +280,19 @@ var _ = Describe("ArtistRepository", func() {
idx, err := repo.GetIndex(false, []int{1})
Expect(err).ToNot(HaveOccurred())
Expect(idx).To(HaveLen(2))
Expect(idx).To(HaveLen(4))
Expect(idx[0].ID).To(Equal("B"))
Expect(idx[0].Artists).To(HaveLen(1))
Expect(idx[0].Artists[0].Name).To(Equal(artistBeatles.Name))
Expect(idx[1].ID).To(Equal("K"))
Expect(idx[1].Artists).To(HaveLen(1))
Expect(idx[1].Artists[0].Name).To(Equal(artistKraftwerk.Name))
Expect(idx[2].ID).To(Equal("R"))
Expect(idx[2].Artists).To(HaveLen(1))
Expect(idx[2].Artists[0].Name).To(Equal(artistPunctuation.Name))
Expect(idx[3].ID).To(Equal("S"))
Expect(idx[3].Artists).To(HaveLen(1))
Expect(idx[3].Artists[0].Name).To(Equal(artistCJK.Name))
// Restore the original value
artistBeatles.SortArtistName = ""
@@ -285,13 +303,19 @@ var _ = Describe("ArtistRepository", func() {
It("returns the index when SortArtistName is empty", func() {
idx, err := repo.GetIndex(false, []int{1})
Expect(err).ToNot(HaveOccurred())
Expect(idx).To(HaveLen(2))
Expect(idx).To(HaveLen(4))
Expect(idx[0].ID).To(Equal("B"))
Expect(idx[0].Artists).To(HaveLen(1))
Expect(idx[0].Artists[0].Name).To(Equal(artistBeatles.Name))
Expect(idx[1].ID).To(Equal("K"))
Expect(idx[1].Artists).To(HaveLen(1))
Expect(idx[1].Artists[0].Name).To(Equal(artistKraftwerk.Name))
Expect(idx[2].ID).To(Equal("R"))
Expect(idx[2].Artists).To(HaveLen(1))
Expect(idx[2].Artists[0].Name).To(Equal(artistPunctuation.Name))
Expect(idx[3].ID).To(Equal("S"))
Expect(idx[3].Artists).To(HaveLen(1))
Expect(idx[3].Artists[0].Name).To(Equal(artistCJK.Name))
})
})
@@ -377,7 +401,7 @@ var _ = Describe("ArtistRepository", func() {
// Admin users can see all content when valid library IDs are provided
idx, err := repo.GetIndex(false, []int{1})
Expect(err).ToNot(HaveOccurred())
Expect(idx).To(HaveLen(2))
Expect(idx).To(HaveLen(4))
// With non-existent library ID, admin users see no content because no artists are associated with that library
idx, err = repo.GetIndex(false, []int{999})
@@ -488,7 +512,7 @@ var _ = Describe("ArtistRepository", func() {
Expect(err).ToNot(HaveOccurred())
// Test the search
results, err := (*testRepo).Search("550e8400-e29b-41d4-a716-446655440010", 0, 10)
results, err := (*testRepo).Search("550e8400-e29b-41d4-a716-446655440010", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
if shouldFind {
@@ -519,12 +543,12 @@ var _ = Describe("ArtistRepository", func() {
Expect(err).ToNot(HaveOccurred())
// Restricted user should not find this artist
results, err := restrictedRepo.Search("a74b1b7f-71a5-4011-9441-d0b5e4122711", 0, 10)
results, err := restrictedRepo.Search("a74b1b7f-71a5-4011-9441-d0b5e4122711", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())
// But admin should find it
results, err = repo.Search("a74b1b7f-71a5-4011-9441-d0b5e4122711", 0, 10)
results, err = repo.Search("a74b1b7f-71a5-4011-9441-d0b5e4122711", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
@@ -536,7 +560,7 @@ var _ = Describe("ArtistRepository", func() {
Context("Text Search", func() {
It("allows admin to find artists by name regardless of library", func() {
results, err := repo.Search("Beatles", 0, 10)
results, err := repo.Search("Beatles", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Name).To(Equal("The Beatles"))
@@ -556,7 +580,7 @@ var _ = Describe("ArtistRepository", func() {
Expect(err).ToNot(HaveOccurred())
// Restricted user should not find this artist
results, err := restrictedRepo.Search("Unique Search Name", 0, 10)
results, err := restrictedRepo.Search("Unique Search Name", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty(), "Text search should respect library filtering")
@@ -625,11 +649,11 @@ var _ = Describe("ArtistRepository", func() {
It("sees all artists regardless of library permissions", func() {
count, err := repo.CountAll()
Expect(err).ToNot(HaveOccurred())
Expect(count).To(Equal(int64(2)))
Expect(count).To(Equal(int64(4)))
artists, err := repo.GetAll()
Expect(err).ToNot(HaveOccurred())
Expect(artists).To(HaveLen(2))
Expect(artists).To(HaveLen(4))
exists, err := repo.Exists(artistBeatles.ID)
Expect(err).ToNot(HaveOccurred())
@@ -661,10 +685,10 @@ var _ = Describe("ArtistRepository", func() {
// Should see missing artist in GetAll by default for admin users
artists, err := repo.GetAll()
Expect(err).ToNot(HaveOccurred())
Expect(artists).To(HaveLen(3)) // Including the missing artist
Expect(artists).To(HaveLen(5)) // Including the missing artist
// Search never returns missing artists (hardcoded behavior)
results, err := repo.Search("Missing Artist", 0, 10)
results, err := repo.Search("Missing Artist", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())
})
@@ -718,11 +742,11 @@ var _ = Describe("ArtistRepository", func() {
})
It("Search returns empty results for users without library access", func() {
results, err := restrictedRepo.Search("Beatles", 0, 10)
results, err := restrictedRepo.Search("Beatles", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())
results, err = restrictedRepo.Search("Kraftwerk", 0, 10)
results, err = restrictedRepo.Search("Kraftwerk", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())
})
@@ -767,19 +791,19 @@ var _ = Describe("ArtistRepository", func() {
It("CountAll returns correct count after gaining access", func() {
count, err := restrictedRepo.CountAll()
Expect(err).ToNot(HaveOccurred())
Expect(count).To(Equal(int64(2))) // Beatles and Kraftwerk
Expect(count).To(Equal(int64(4))) // Beatles, Kraftwerk, Seatbelts, and The Roots
})
It("GetAll returns artists after gaining access", func() {
artists, err := restrictedRepo.GetAll()
Expect(err).ToNot(HaveOccurred())
Expect(artists).To(HaveLen(2))
Expect(artists).To(HaveLen(4))
var names []string
for _, artist := range artists {
names = append(names, artist.Name)
}
Expect(names).To(ContainElements("The Beatles", "Kraftwerk"))
Expect(names).To(ContainElements("The Beatles", "Kraftwerk", "シートベルツ", "The Roots"))
})
It("Exists returns true for accessible artists", func() {
@@ -796,7 +820,7 @@ var _ = Describe("ArtistRepository", func() {
// With valid library access, should see artists
idx, err := restrictedRepo.GetIndex(false, []int{1})
Expect(err).ToNot(HaveOccurred())
Expect(idx).To(HaveLen(2))
Expect(idx).To(HaveLen(4))
// With non-existent library ID, should see nothing (non-admin user)
idx, err = restrictedRepo.GetIndex(false, []int{999})

View File

@@ -17,45 +17,45 @@ import (
var _ = Describe("Collation", func() {
conn := db.Db()
DescribeTable("Column collation",
func(table, column, expectedCollation string) {
Expect(checkCollation(conn, table, column, expectedCollation)).To(Succeed())
func(table, column string) {
Expect(checkCollation(conn, table, column)).To(Succeed())
},
Entry("artist.order_artist_name", "artist", "order_artist_name", "NATURALSORT"),
Entry("artist.sort_artist_name", "artist", "sort_artist_name", "NATURALSORT"),
Entry("album.order_album_name", "album", "order_album_name", "NATURALSORT"),
Entry("album.order_album_artist_name", "album", "order_album_artist_name", "NATURALSORT"),
Entry("album.sort_album_name", "album", "sort_album_name", "NATURALSORT"),
Entry("album.sort_album_artist_name", "album", "sort_album_artist_name", "NATURALSORT"),
Entry("media_file.order_title", "media_file", "order_title", "NATURALSORT"),
Entry("media_file.order_album_name", "media_file", "order_album_name", "NATURALSORT"),
Entry("media_file.order_artist_name", "media_file", "order_artist_name", "NATURALSORT"),
Entry("media_file.sort_title", "media_file", "sort_title", "NATURALSORT"),
Entry("media_file.sort_album_name", "media_file", "sort_album_name", "NATURALSORT"),
Entry("media_file.sort_artist_name", "media_file", "sort_artist_name", "NATURALSORT"),
Entry("playlist.name", "playlist", "name", "NATURALSORT"),
Entry("radio.name", "radio", "name", "NATURALSORT"),
Entry("user.name", "user", "name", "NOCASE"),
Entry("artist.order_artist_name", "artist", "order_artist_name"),
Entry("artist.sort_artist_name", "artist", "sort_artist_name"),
Entry("album.order_album_name", "album", "order_album_name"),
Entry("album.order_album_artist_name", "album", "order_album_artist_name"),
Entry("album.sort_album_name", "album", "sort_album_name"),
Entry("album.sort_album_artist_name", "album", "sort_album_artist_name"),
Entry("media_file.order_title", "media_file", "order_title"),
Entry("media_file.order_album_name", "media_file", "order_album_name"),
Entry("media_file.order_artist_name", "media_file", "order_artist_name"),
Entry("media_file.sort_title", "media_file", "sort_title"),
Entry("media_file.sort_album_name", "media_file", "sort_album_name"),
Entry("media_file.sort_artist_name", "media_file", "sort_artist_name"),
Entry("playlist.name", "playlist", "name"),
Entry("radio.name", "radio", "name"),
Entry("user.name", "user", "name"),
)
DescribeTable("Index collation",
func(table, column string) {
Expect(checkIndexUsage(conn, table, column)).To(Succeed())
},
Entry("artist.order_artist_name", "artist", "order_artist_name collate NATURALSORT"),
Entry("artist.sort_artist_name", "artist", "coalesce(nullif(sort_artist_name,''),order_artist_name) collate NATURALSORT"),
Entry("album.order_album_name", "album", "order_album_name collate NATURALSORT"),
Entry("album.order_album_artist_name", "album", "order_album_artist_name collate NATURALSORT"),
Entry("album.sort_album_name", "album", "coalesce(nullif(sort_album_name,''),order_album_name) collate NATURALSORT"),
Entry("album.sort_album_artist_name", "album", "coalesce(nullif(sort_album_artist_name,''),order_album_artist_name) collate NATURALSORT"),
Entry("media_file.order_title", "media_file", "order_title collate NATURALSORT"),
Entry("media_file.order_album_name", "media_file", "order_album_name collate NATURALSORT"),
Entry("media_file.order_artist_name", "media_file", "order_artist_name collate NATURALSORT"),
Entry("media_file.sort_title", "media_file", "coalesce(nullif(sort_title,''),order_title) collate NATURALSORT"),
Entry("media_file.sort_album_name", "media_file", "coalesce(nullif(sort_album_name,''),order_album_name) collate NATURALSORT"),
Entry("media_file.sort_artist_name", "media_file", "coalesce(nullif(sort_artist_name,''),order_artist_name) collate NATURALSORT"),
Entry("artist.order_artist_name", "artist", "order_artist_name collate nocase"),
Entry("artist.sort_artist_name", "artist", "coalesce(nullif(sort_artist_name,''),order_artist_name) collate nocase"),
Entry("album.order_album_name", "album", "order_album_name collate nocase"),
Entry("album.order_album_artist_name", "album", "order_album_artist_name collate nocase"),
Entry("album.sort_album_name", "album", "coalesce(nullif(sort_album_name,''),order_album_name) collate nocase"),
Entry("album.sort_album_artist_name", "album", "coalesce(nullif(sort_album_artist_name,''),order_album_artist_name) collate nocase"),
Entry("media_file.order_title", "media_file", "order_title collate nocase"),
Entry("media_file.order_album_name", "media_file", "order_album_name collate nocase"),
Entry("media_file.order_artist_name", "media_file", "order_artist_name collate nocase"),
Entry("media_file.sort_title", "media_file", "coalesce(nullif(sort_title,''),order_title) collate nocase"),
Entry("media_file.sort_album_name", "media_file", "coalesce(nullif(sort_album_name,''),order_album_name) collate nocase"),
Entry("media_file.sort_artist_name", "media_file", "coalesce(nullif(sort_artist_name,''),order_artist_name) collate nocase"),
Entry("media_file.path", "media_file", "path collate nocase"),
Entry("playlist.name", "playlist", "name collate NATURALSORT"),
Entry("radio.name", "radio", "name collate NATURALSORT"),
Entry("playlist.name", "playlist", "name collate nocase"),
Entry("radio.name", "radio", "name collate nocase"),
Entry("user.user_name", "user", "user_name collate nocase"),
)
})
@@ -91,7 +91,7 @@ order by %[2]s`, table, column))
return errors.New("no rows returned")
}
func checkCollation(conn *sql.DB, table, column, expectedCollation string) error {
func checkCollation(conn *sql.DB, table string, column string) error {
rows, err := conn.Query(fmt.Sprintf("SELECT sql FROM sqlite_master WHERE type='table' AND tbl_name='%s'", table))
if err != nil {
return err
@@ -113,12 +113,12 @@ func checkCollation(conn *sql.DB, table, column, expectedCollation string) error
if !re.MatchString(res) {
return fmt.Errorf("column '%s' not found in table '%s'", column, table)
}
re = regexp.MustCompile(fmt.Sprintf(`(?i)\b%s\b.*collate\s+%s`, column, expectedCollation))
re = regexp.MustCompile(fmt.Sprintf(`(?i)\b%s\b.*collate\s+NOCASE`, column))
if re.MatchString(res) {
return nil
}
} else {
return fmt.Errorf("table '%s' not found", table)
}
return fmt.Errorf("column '%s' in table '%s' does not have %s collation", column, table, expectedCollation)
return fmt.Errorf("column '%s' in table '%s' does not have NOCASE collation", column, table)
}

View File

@@ -82,11 +82,11 @@ func (e existsCond) ToSql() (string, []any, error) {
var sortOrderRegex = regexp.MustCompile(`order_([a-z_]+)`)
// mapSortOrder converts order_* columns to an expression using sort_* columns with NATURALSORT collation. Example:
// order_album_name -> (coalesce(nullif(sort_album_name,”),order_album_name) collate NATURALSORT)
// Convert the order_* columns to an expression using sort_* columns. Example:
// sort_album_name -> (coalesce(nullif(sort_album_name,”),order_album_name) collate nocase)
// It finds order column names anywhere in the substring
func mapSortOrder(tableName, order string) string {
order = strings.ToLower(order)
repl := fmt.Sprintf("(coalesce(nullif(%[1]s.sort_$1,''),%[1]s.order_$1) collate NATURALSORT)", tableName)
repl := fmt.Sprintf("(coalesce(nullif(%[1]s.sort_$1,''),%[1]s.order_$1) collate nocase)", tableName)
return sortOrderRegex.ReplaceAllString(order, repl)
}

View File

@@ -94,13 +94,13 @@ var _ = Describe("Helpers", func() {
sort := "ORDER_ALBUM_NAME asc"
mapped := mapSortOrder("album", sort)
Expect(mapped).To(Equal(`(coalesce(nullif(album.sort_album_name,''),album.order_album_name)` +
` collate NATURALSORT) asc`))
` collate nocase) asc`))
})
It("changes multiple order columns to sort expressions", func() {
sort := "compilation, order_title asc, order_album_artist_name desc, year desc"
mapped := mapSortOrder("album", sort)
Expect(mapped).To(Equal(`compilation, (coalesce(nullif(album.sort_title,''),album.order_title) collate NATURALSORT) asc,` +
` (coalesce(nullif(album.sort_album_artist_name,''),album.order_album_artist_name) collate NATURALSORT) desc, year desc`))
Expect(mapped).To(Equal(`compilation, (coalesce(nullif(album.sort_title,''),album.order_title) collate nocase) asc,` +
` (coalesce(nullif(album.sort_album_artist_name,''),album.order_album_artist_name) collate nocase) desc, year desc`))
})
})
})

View File

@@ -11,7 +11,6 @@ import (
. "github.com/Masterminds/squirrel"
"github.com/deluan/rest"
"github.com/google/uuid"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
@@ -58,8 +57,11 @@ func (m *dbMediaFile) PostScan() error {
func (m *dbMediaFile) PostMapArgs(args map[string]any) error {
fullText := []string{m.FullTitle(), m.Album, m.Artist, m.AlbumArtist,
m.SortTitle, m.SortAlbumName, m.SortArtistName, m.SortAlbumArtistName, m.DiscSubtitle}
fullText = append(fullText, m.MediaFile.Participants.AllNames()...)
participantNames := m.MediaFile.Participants.AllNames()
fullText = append(fullText, participantNames...)
args["full_text"] = formatFullText(fullText...)
args["search_participants"] = strings.Join(participantNames, " ")
args["search_normalized"] = normalizeForFTS(m.FullTitle(), m.Album, m.Artist, m.AlbumArtist)
args["tags"] = marshalTags(m.MediaFile.Tags)
args["participants"] = marshalParticipants(m.MediaFile.Participants)
return nil
@@ -425,18 +427,21 @@ func (r *mediaFileRepository) FindRecentFilesByProperties(missing model.MediaFil
return res.toModels(), nil
}
func (r *mediaFileRepository) Search(q string, offset int, size int, options ...model.QueryOptions) (model.MediaFiles, error) {
var mediaFileSearchConfig = searchConfig{
NaturalOrder: "media_file.rowid",
OrderBy: []string{"title"},
MBIDFields: []string{"mbz_recording_id", "mbz_release_track_id"},
}
func (r *mediaFileRepository) Search(q string, options ...model.QueryOptions) (model.MediaFiles, error) {
var opts model.QueryOptions
if len(options) > 0 {
opts = options[0]
}
var res dbMediaFiles
if uuid.Validate(q) == nil {
err := r.searchByMBID(r.selectMediaFile(options...), q, []string{"mbz_recording_id", "mbz_release_track_id"}, &res)
if err != nil {
return nil, fmt.Errorf("searching media_file by MBID %q: %w", q, err)
}
} else {
err := r.doSearch(r.selectMediaFile(options...), q, offset, size, &res, "media_file.rowid", "title")
if err != nil {
return nil, fmt.Errorf("searching media_file by query %q: %w", q, err)
}
err := r.doSearch(r.selectMediaFile(options...), q, &res, mediaFileSearchConfig, opts)
if err != nil {
return nil, fmt.Errorf("searching media_file %q: %w", q, err)
}
return res.toModels(), nil
}

View File

@@ -39,7 +39,7 @@ var _ = Describe("MediaRepository", func() {
})
It("counts the number of mediafiles in the DB", func() {
Expect(mr.CountAll()).To(Equal(int64(10)))
Expect(mr.CountAll()).To(Equal(int64(13)))
})
Describe("CountBySuffix", func() {
@@ -527,7 +527,7 @@ var _ = Describe("MediaRepository", func() {
Describe("Search", func() {
Context("text search", func() {
It("finds media files by title", func() {
results, err := mr.Search("Antenna", 0, 10)
results, err := mr.Search("Antenna", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(3)) // songAntenna, songAntennaWithLyrics, songAntenna2
for _, result := range results {
@@ -536,7 +536,7 @@ var _ = Describe("MediaRepository", func() {
})
It("finds media files case insensitively", func() {
results, err := mr.Search("antenna", 0, 10)
results, err := mr.Search("antenna", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(3))
for _, result := range results {
@@ -545,7 +545,7 @@ var _ = Describe("MediaRepository", func() {
})
It("returns empty result when no matches found", func() {
results, err := mr.Search("nonexistent", 0, 10)
results, err := mr.Search("nonexistent", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())
})
@@ -578,7 +578,7 @@ var _ = Describe("MediaRepository", func() {
})
It("finds media file by mbz_recording_id", func() {
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440020", 0, 10)
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440020", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].ID).To(Equal("test-mbid-mediafile"))
@@ -586,7 +586,7 @@ var _ = Describe("MediaRepository", func() {
})
It("finds media file by mbz_release_track_id", func() {
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440021", 0, 10)
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440021", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].ID).To(Equal("test-mbid-mediafile"))
@@ -594,7 +594,7 @@ var _ = Describe("MediaRepository", func() {
})
It("returns empty result when MBID is not found", func() {
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440099", 0, 10)
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440099", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())
})
@@ -614,7 +614,7 @@ var _ = Describe("MediaRepository", func() {
Expect(err).ToNot(HaveOccurred())
// Search never returns missing media files (hardcoded behavior)
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440022", 0, 10)
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440022", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())

View File

@@ -56,12 +56,22 @@ func al(al model.Album) model.Album {
return al
}
func alWithTags(a model.Album, tags model.Tags) model.Album {
a = al(a)
a.Tags = tags
return a
}
var (
artistKraftwerk = model.Artist{ID: "2", Name: "Kraftwerk", OrderArtistName: "kraftwerk"}
artistBeatles = model.Artist{ID: "3", Name: "The Beatles", OrderArtistName: "beatles"}
testArtists = model.Artists{
artistKraftwerk = model.Artist{ID: "2", Name: "Kraftwerk", OrderArtistName: "kraftwerk"}
artistBeatles = model.Artist{ID: "3", Name: "The Beatles", OrderArtistName: "beatles"}
artistCJK = model.Artist{ID: "4", Name: "シートベルツ", SortArtistName: "Seatbelts", OrderArtistName: "seatbelts"}
artistPunctuation = model.Artist{ID: "5", Name: "The Roots", OrderArtistName: "roots"}
testArtists = model.Artists{
artistKraftwerk,
artistBeatles,
artistCJK,
artistPunctuation,
}
)
@@ -70,11 +80,18 @@ var (
albumAbbeyRoad = al(model.Album{ID: "102", Name: "Abbey Road", AlbumArtist: "The Beatles", OrderAlbumName: "abbey road", AlbumArtistID: "3", EmbedArtPath: p("/beatles/1/come together.mp3"), SongCount: 1, MaxYear: 1969})
albumRadioactivity = al(model.Album{ID: "103", Name: "Radioactivity", AlbumArtist: "Kraftwerk", OrderAlbumName: "radioactivity", AlbumArtistID: "2", EmbedArtPath: p("/kraft/radio/radio.mp3"), SongCount: 2})
albumMultiDisc = al(model.Album{ID: "104", Name: "Multi Disc Album", AlbumArtist: "Test Artist", OrderAlbumName: "multi disc album", AlbumArtistID: "1", EmbedArtPath: p("/test/multi/disc1/track1.mp3"), SongCount: 4})
testAlbums = model.Albums{
albumCJK = al(model.Album{ID: "105", Name: "COWBOY BEBOP", AlbumArtist: "シートベルツ", OrderAlbumName: "cowboy bebop", AlbumArtistID: "4", EmbedArtPath: p("/seatbelts/cowboy-bebop/track1.mp3"), SongCount: 1})
albumWithVersion = alWithTags(model.Album{ID: "106", Name: "Abbey Road", AlbumArtist: "The Beatles", OrderAlbumName: "abbey road", AlbumArtistID: "3", EmbedArtPath: p("/beatles/2/come together.mp3"), SongCount: 1, MaxYear: 2019},
model.Tags{model.TagAlbumVersion: {"Deluxe Edition"}})
albumPunctuation = al(model.Album{ID: "107", Name: "Things Fall Apart", AlbumArtist: "The Roots", OrderAlbumName: "things fall apart", AlbumArtistID: "5", EmbedArtPath: p("/roots/things/track1.mp3"), SongCount: 1})
testAlbums = model.Albums{
albumSgtPeppers,
albumAbbeyRoad,
albumRadioactivity,
albumMultiDisc,
albumCJK,
albumWithVersion,
albumPunctuation,
}
)
@@ -101,6 +118,9 @@ var (
songDisc1Track01 = mf(model.MediaFile{ID: "2002", Title: "Disc 1 Track 1", ArtistID: "1", Artist: "Test Artist", AlbumID: "104", Album: "Multi Disc Album", DiscNumber: 1, TrackNumber: 1, Path: p("/test/multi/disc1/track1.mp3"), OrderAlbumName: "multi disc album", OrderArtistName: "test artist"})
songDisc2Track01 = mf(model.MediaFile{ID: "2003", Title: "Disc 2 Track 1", ArtistID: "1", Artist: "Test Artist", AlbumID: "104", Album: "Multi Disc Album", DiscNumber: 2, TrackNumber: 1, Path: p("/test/multi/disc2/track1.mp3"), OrderAlbumName: "multi disc album", OrderArtistName: "test artist"})
songDisc1Track02 = mf(model.MediaFile{ID: "2004", Title: "Disc 1 Track 2", ArtistID: "1", Artist: "Test Artist", AlbumID: "104", Album: "Multi Disc Album", DiscNumber: 1, TrackNumber: 2, Path: p("/test/multi/disc1/track2.mp3"), OrderAlbumName: "multi disc album", OrderArtistName: "test artist"})
songCJK = mf(model.MediaFile{ID: "3001", Title: "プラチナ・ジェット", ArtistID: "4", Artist: "シートベルツ", AlbumID: "105", Album: "COWBOY BEBOP", Path: p("/seatbelts/cowboy-bebop/track1.mp3")})
songVersioned = mf(model.MediaFile{ID: "3002", Title: "Come Together", ArtistID: "3", Artist: "The Beatles", AlbumID: "106", Album: "Abbey Road", Path: p("/beatles/2/come together.mp3")})
songPunctuation = mf(model.MediaFile{ID: "3003", Title: "!!!!!!!", ArtistID: "5", Artist: "The Roots", AlbumID: "107", Album: "Things Fall Apart", Path: p("/roots/things/track1.mp3")})
testSongs = model.MediaFiles{
songDayInALife,
songComeTogether,
@@ -112,6 +132,9 @@ var (
songDisc1Track01,
songDisc2Track01,
songDisc1Track02,
songCJK,
songVersioned,
songPunctuation,
}
)

View File

@@ -96,16 +96,6 @@ func (r *playlistRepository) Exists(id string) (bool, error) {
}
func (r *playlistRepository) Delete(id string) error {
usr := loggedUser(r.ctx)
if !usr.IsAdmin {
pls, err := r.Get(id)
if err != nil {
return err
}
if pls.OwnerID != usr.ID {
return rest.ErrPermissionDenied
}
}
return r.delete(And{Eq{"id": id}, r.userFilter()})
}
@@ -113,14 +103,6 @@ func (r *playlistRepository) Put(p *model.Playlist) error {
pls := dbPlaylist{Playlist: *p}
if pls.ID == "" {
pls.CreatedAt = time.Now()
} else {
ok, err := r.Exists(pls.ID)
if err != nil {
return err
}
if !ok {
return model.ErrNotAuthorized
}
}
pls.UpdatedAt = time.Now()
@@ -132,7 +114,6 @@ func (r *playlistRepository) Put(p *model.Playlist) error {
if p.IsSmartPlaylist() {
// Do not update tracks at this point, as it may take a long time and lock the DB, breaking the scan process
//r.refreshSmartPlaylist(p)
return nil
}
// Only update tracks if they were specified
@@ -260,10 +241,25 @@ func (r *playlistRepository) refreshSmartPlaylist(pls *model.Playlist) bool {
}
sq := Select("row_number() over (order by "+rules.OrderBy()+") as id", "'"+pls.ID+"' as playlist_id", "media_file.id as media_file_id").
From("media_file").LeftJoin("annotation on (" +
"annotation.item_id = media_file.id" +
" AND annotation.item_type = 'media_file'" +
" AND annotation.user_id = '" + usr.ID + "')")
From("media_file").LeftJoin("annotation on ("+
"annotation.item_id = media_file.id"+
" AND annotation.item_type = 'media_file'"+
" AND annotation.user_id = ?)", usr.ID)
// Conditionally join album/artist annotation tables only when referenced by criteria or sort
requiredJoins := rules.RequiredJoins()
if requiredJoins.Has(criteria.JoinAlbumAnnotation) {
sq = sq.LeftJoin("annotation AS album_annotation ON ("+
"album_annotation.item_id = media_file.album_id"+
" AND album_annotation.item_type = 'album'"+
" AND album_annotation.user_id = ?)", usr.ID)
}
if requiredJoins.Has(criteria.JoinArtistAnnotation) {
sq = sq.LeftJoin("annotation AS artist_annotation ON ("+
"artist_annotation.item_id = media_file.artist_id"+
" AND artist_annotation.item_type = 'artist'"+
" AND artist_annotation.user_id = ?)", usr.ID)
}
// Only include media files from libraries the user has access to
sq = r.applyLibraryFilter(sq, "media_file")
@@ -320,10 +316,6 @@ func (r *playlistRepository) updateTracks(id string, tracks model.MediaFiles) er
}
func (r *playlistRepository) updatePlaylist(playlistId string, mediaFileIds []string) error {
if !r.isWritable(playlistId) {
return rest.ErrPermissionDenied
}
// Remove old tracks
del := Delete("playlist_tracks").Where(Eq{"playlist_id": playlistId})
_, err := r.executeSQL(del)
@@ -439,8 +431,7 @@ func (r *playlistRepository) NewInstance() any {
func (r *playlistRepository) Save(entity any) (string, error) {
pls := entity.(*model.Playlist)
pls.OwnerID = loggedUser(r.ctx).ID
pls.ID = "" // Make sure we don't override an existing playlist
pls.ID = "" // Force new creation
err := r.Put(pls)
if err != nil {
return "", err
@@ -450,24 +441,9 @@ func (r *playlistRepository) Save(entity any) (string, error) {
func (r *playlistRepository) Update(id string, entity any, cols ...string) error {
pls := dbPlaylist{Playlist: *entity.(*model.Playlist)}
current, err := r.Get(id)
if err != nil {
return err
}
usr := loggedUser(r.ctx)
if !usr.IsAdmin {
// Only the owner can update the playlist
if current.OwnerID != usr.ID {
return rest.ErrPermissionDenied
}
// Regular users can't change the ownership of a playlist
if pls.OwnerID != "" && pls.OwnerID != usr.ID {
return rest.ErrPermissionDenied
}
}
pls.ID = id
pls.UpdatedAt = time.Now()
_, err = r.put(id, pls, append(cols, "updatedAt")...)
_, err := r.put(id, pls, append(cols, "updatedAt")...)
if errors.Is(err, model.ErrNotFound) {
return rest.ErrNotFound
}
@@ -507,23 +483,31 @@ func (r *playlistRepository) removeOrphans() error {
return nil
}
// renumber updates the position of all tracks in the playlist to be sequential starting from 1, ordered by their
// current position. This is needed after removing orphan tracks, to ensure there are no gaps in the track numbering.
// The two-step approach (negate then reassign via CTE) avoids UNIQUE constraint violations on (playlist_id, id).
func (r *playlistRepository) renumber(id string) error {
var ids []string
sq := Select("media_file_id").From("playlist_tracks").Where(Eq{"playlist_id": id}).OrderBy("id")
err := r.queryAllSlice(sq, &ids)
// Step 1: Negate all IDs to clear the positive ID space
_, err := r.executeSQL(Expr(
`UPDATE playlist_tracks SET id = -id WHERE playlist_id = ? AND id > 0`, id))
if err != nil {
return err
}
return r.updatePlaylist(id, ids)
}
func (r *playlistRepository) isWritable(playlistId string) bool {
usr := loggedUser(r.ctx)
if usr.IsAdmin {
return true
// Step 2: Assign new sequential positive IDs using UPDATE...FROM with a CTE.
// The CTE is fully materialized before the UPDATE begins, avoiding self-referencing issues.
// ORDER BY id DESC restores original order since IDs are now negative.
_, err = r.executeSQL(Expr(
`WITH new_ids AS (
SELECT rowid as rid, ROW_NUMBER() OVER (ORDER BY id DESC) as new_id
FROM playlist_tracks WHERE playlist_id = ?
)
UPDATE playlist_tracks SET id = new_ids.new_id
FROM new_ids
WHERE playlist_tracks.rowid = new_ids.rid AND playlist_tracks.playlist_id = ?`, id, id))
if err != nil {
return err
}
pls, err := r.Get(playlistId)
return err == nil && pls.OwnerID == usr.ID
return r.refreshCounters(&model.Playlist{ID: id})
}
var _ model.PlaylistRepository = (*playlistRepository)(nil)

View File

@@ -287,6 +287,106 @@ var _ = Describe("PlaylistRepository", func() {
})
})
Describe("Smart Playlists with Album/Artist Annotation Criteria", func() {
var testPlaylistID string
AfterEach(func() {
if testPlaylistID != "" {
_ = repo.Delete(testPlaylistID)
testPlaylistID = ""
}
})
It("matches tracks from starred albums using albumLoved", func() {
// albumRadioactivity (ID "103") is starred in test fixtures
// Songs in album 103: 1003, 1004, 1005, 1006
rules := &criteria.Criteria{
Expression: criteria.All{
criteria.Is{"albumLoved": true},
},
}
newPls := model.Playlist{Name: "Starred Album Songs", OwnerID: "userid", Rules: rules}
Expect(repo.Put(&newPls)).To(Succeed())
testPlaylistID = newPls.ID
conf.Server.SmartPlaylistRefreshDelay = -1 * time.Second
pls, err := repo.GetWithTracks(newPls.ID, true, false)
Expect(err).ToNot(HaveOccurred())
trackIDs := make([]string, len(pls.Tracks))
for i, t := range pls.Tracks {
trackIDs[i] = t.MediaFileID
}
Expect(trackIDs).To(ConsistOf("1003", "1004", "1005", "1006"))
})
It("matches tracks from starred artists using artistLoved", func() {
// artistBeatles (ID "3") is starred in test fixtures
// Songs with ArtistID "3": 1001, 1002, 3002
rules := &criteria.Criteria{
Expression: criteria.All{
criteria.Is{"artistLoved": true},
},
}
newPls := model.Playlist{Name: "Starred Artist Songs", OwnerID: "userid", Rules: rules}
Expect(repo.Put(&newPls)).To(Succeed())
testPlaylistID = newPls.ID
conf.Server.SmartPlaylistRefreshDelay = -1 * time.Second
pls, err := repo.GetWithTracks(newPls.ID, true, false)
Expect(err).ToNot(HaveOccurred())
trackIDs := make([]string, len(pls.Tracks))
for i, t := range pls.Tracks {
trackIDs[i] = t.MediaFileID
}
Expect(trackIDs).To(ConsistOf("1001", "1002", "3002"))
})
It("matches tracks with combined album and artist criteria", func() {
// albumLoved=true → songs from album 103 (1003, 1004, 1005, 1006)
// artistLoved=true → songs with artist 3 (1001, 1002)
// Using Any: union of both sets
rules := &criteria.Criteria{
Expression: criteria.Any{
criteria.Is{"albumLoved": true},
criteria.Is{"artistLoved": true},
},
}
newPls := model.Playlist{Name: "Combined Album+Artist", OwnerID: "userid", Rules: rules}
Expect(repo.Put(&newPls)).To(Succeed())
testPlaylistID = newPls.ID
conf.Server.SmartPlaylistRefreshDelay = -1 * time.Second
pls, err := repo.GetWithTracks(newPls.ID, true, false)
Expect(err).ToNot(HaveOccurred())
trackIDs := make([]string, len(pls.Tracks))
for i, t := range pls.Tracks {
trackIDs[i] = t.MediaFileID
}
Expect(trackIDs).To(ConsistOf("1001", "1002", "1003", "1004", "1005", "1006", "3002"))
})
It("returns no tracks when no albums/artists match", func() {
// No album has rating 5 in fixtures
rules := &criteria.Criteria{
Expression: criteria.All{
criteria.Is{"albumRating": 5},
},
}
newPls := model.Playlist{Name: "No Match", OwnerID: "userid", Rules: rules}
Expect(repo.Put(&newPls)).To(Succeed())
testPlaylistID = newPls.ID
conf.Server.SmartPlaylistRefreshDelay = -1 * time.Second
pls, err := repo.GetWithTracks(newPls.ID, true, false)
Expect(err).ToNot(HaveOccurred())
Expect(pls.Tracks).To(BeEmpty())
})
})
Describe("Smart Playlists with Tag Criteria", func() {
var mfRepo model.MediaFileRepository
var testPlaylistID string
@@ -401,6 +501,79 @@ var _ = Describe("PlaylistRepository", func() {
})
})
Describe("Track Deletion and Renumbering", func() {
var testPlaylistID string
AfterEach(func() {
if testPlaylistID != "" {
Expect(repo.Delete(testPlaylistID)).To(BeNil())
testPlaylistID = ""
}
})
// helper to get track positions and media file IDs
getTrackInfo := func(playlistID string) (ids []string, mediaFileIDs []string) {
pls, err := repo.GetWithTracks(playlistID, false, false)
Expect(err).ToNot(HaveOccurred())
for _, t := range pls.Tracks {
ids = append(ids, t.ID)
mediaFileIDs = append(mediaFileIDs, t.MediaFileID)
}
return
}
It("renumbers correctly after deleting a track from the middle", func() {
By("creating a playlist with 4 tracks")
newPls := model.Playlist{Name: "Renumber Test Middle", OwnerID: "userid"}
newPls.AddMediaFilesByID([]string{"1001", "1002", "1003", "1004"})
Expect(repo.Put(&newPls)).To(Succeed())
testPlaylistID = newPls.ID
By("deleting the second track (position 2)")
tracksRepo := repo.Tracks(newPls.ID, false)
Expect(tracksRepo.Delete("2")).To(Succeed())
By("verifying remaining tracks are renumbered sequentially")
ids, mediaFileIDs := getTrackInfo(newPls.ID)
Expect(ids).To(Equal([]string{"1", "2", "3"}))
Expect(mediaFileIDs).To(Equal([]string{"1001", "1003", "1004"}))
})
It("renumbers correctly after deleting the first track", func() {
By("creating a playlist with 3 tracks")
newPls := model.Playlist{Name: "Renumber Test First", OwnerID: "userid"}
newPls.AddMediaFilesByID([]string{"1001", "1002", "1003"})
Expect(repo.Put(&newPls)).To(Succeed())
testPlaylistID = newPls.ID
By("deleting the first track (position 1)")
tracksRepo := repo.Tracks(newPls.ID, false)
Expect(tracksRepo.Delete("1")).To(Succeed())
By("verifying remaining tracks are renumbered sequentially")
ids, mediaFileIDs := getTrackInfo(newPls.ID)
Expect(ids).To(Equal([]string{"1", "2"}))
Expect(mediaFileIDs).To(Equal([]string{"1002", "1003"}))
})
It("renumbers correctly after deleting the last track", func() {
By("creating a playlist with 3 tracks")
newPls := model.Playlist{Name: "Renumber Test Last", OwnerID: "userid"}
newPls.AddMediaFilesByID([]string{"1001", "1002", "1003"})
Expect(repo.Put(&newPls)).To(Succeed())
testPlaylistID = newPls.ID
By("deleting the last track (position 3)")
tracksRepo := repo.Tracks(newPls.ID, false)
Expect(tracksRepo.Delete("3")).To(Succeed())
By("verifying remaining tracks are renumbered sequentially")
ids, mediaFileIDs := getTrackInfo(newPls.ID)
Expect(ids).To(Equal([]string{"1", "2"}))
Expect(mediaFileIDs).To(Equal([]string{"1001", "1002"}))
})
})
Describe("Smart Playlists Library Filtering", func() {
var mfRepo model.MediaFileRepository
var testPlaylistID string

View File

@@ -140,15 +140,7 @@ func (r *playlistTrackRepository) NewInstance() any {
return &model.PlaylistTrack{}
}
func (r *playlistTrackRepository) isTracksEditable() bool {
return r.playlistRepo.isWritable(r.playlistId) && !r.playlist.IsSmartPlaylist()
}
func (r *playlistTrackRepository) Add(mediaFileIds []string) (int, error) {
if !r.isTracksEditable() {
return 0, rest.ErrPermissionDenied
}
if len(mediaFileIds) > 0 {
log.Debug(r.ctx, "Adding songs to playlist", "playlistId", r.playlistId, "mediaFileIds", mediaFileIds)
} else {
@@ -196,22 +188,7 @@ func (r *playlistTrackRepository) AddDiscs(discs []model.DiscID) (int, error) {
return r.addMediaFileIds(clauses)
}
// Get ids from all current tracks
func (r *playlistTrackRepository) getTracks() ([]string, error) {
all := r.newSelect().Columns("media_file_id").Where(Eq{"playlist_id": r.playlistId}).OrderBy("id")
var ids []string
err := r.queryAllSlice(all, &ids)
if err != nil {
log.Error(r.ctx, "Error querying current tracks from playlist", "playlistId", r.playlistId, err)
return nil, err
}
return ids, nil
}
func (r *playlistTrackRepository) Delete(ids ...string) error {
if !r.isTracksEditable() {
return rest.ErrPermissionDenied
}
err := r.delete(And{Eq{"playlist_id": r.playlistId}, Eq{"id": ids}})
if err != nil {
return err
@@ -221,9 +198,6 @@ func (r *playlistTrackRepository) Delete(ids ...string) error {
}
func (r *playlistTrackRepository) DeleteAll() error {
if !r.isTracksEditable() {
return rest.ErrPermissionDenied
}
err := r.delete(Eq{"playlist_id": r.playlistId})
if err != nil {
return err
@@ -232,16 +206,45 @@ func (r *playlistTrackRepository) DeleteAll() error {
return r.playlistRepo.renumber(r.playlistId)
}
// Reorder moves a track from pos to newPos, shifting other tracks accordingly.
func (r *playlistTrackRepository) Reorder(pos int, newPos int) error {
if !r.isTracksEditable() {
return rest.ErrPermissionDenied
if pos == newPos {
return nil
}
ids, err := r.getTracks()
pid := r.playlistId
// Step 1: Move the source track out of the way (temporary sentinel value)
_, err := r.executeSQL(Expr(
`UPDATE playlist_tracks SET id = -999999 WHERE playlist_id = ? AND id = ?`, pid, pos))
if err != nil {
return err
}
newOrder := slice.Move(ids, pos-1, newPos-1)
return r.playlistRepo.updatePlaylist(r.playlistId, newOrder)
// Step 2: Shift the affected range using negative values to avoid unique constraint violations
if pos < newPos {
_, err = r.executeSQL(Expr(
`UPDATE playlist_tracks SET id = -(id - 1) WHERE playlist_id = ? AND id > ? AND id <= ?`,
pid, pos, newPos))
} else {
_, err = r.executeSQL(Expr(
`UPDATE playlist_tracks SET id = -(id + 1) WHERE playlist_id = ? AND id >= ? AND id < ?`,
pid, newPos, pos))
}
if err != nil {
return err
}
// Step 3: Flip the shifted range back to positive
_, err = r.executeSQL(Expr(
`UPDATE playlist_tracks SET id = -id WHERE playlist_id = ? AND id < 0 AND id != -999999`, pid))
if err != nil {
return err
}
// Step 4: Place the source track at its new position
_, err = r.executeSQL(Expr(
`UPDATE playlist_tracks SET id = ? WHERE playlist_id = ? AND id = -999999`, newPos, pid))
return err
}
var _ model.PlaylistTrackRepository = (*playlistTrackRepository)(nil)

View File

@@ -71,8 +71,8 @@ func (r *sqlRepository) registerModel(instance any, filters map[string]filterFun
//
// If PreferSortTags is enabled, it will map the order fields to the corresponding sort expression,
// which gives precedence to sort tags.
// Ex: order_title => (coalesce(nullif(sort_title,""), order_title) collate NATURALSORT)
// To avoid performance issues, indexes should be created for these sort expressions.
// Ex: order_title => (coalesce(nullif(sort_title,),order_title) collate nocase)
// To avoid performance issues, indexes should be created for these sort expressions
//
// NOTE: if an individual item has spaces, it should be wrapped in parentheses. For example,
// you should write "(lyrics != '[]')". This prevents the item being split unexpectedly.

View File

@@ -109,11 +109,10 @@ func booleanFilter(field string, value any) Sqlizer {
func fullTextFilter(tableName string, mbidFields ...string) func(string, any) Sqlizer {
return func(field string, value any) Sqlizer {
v := strings.ToLower(value.(string))
cond := cmp.Or(
return cmp.Or[Sqlizer](
mbidExpr(tableName, v, mbidFields...),
fullTextExpr(tableName, v),
getSearchStrategy(tableName, v),
)
return cond
}
}

View File

@@ -26,7 +26,9 @@ var _ = Describe("sqlRestful", func() {
Expect(r.parseRestFilters(context.Background(), options)).To(BeNil())
})
It(`returns nil if tries a filter with fullTextExpr("'")`, func() {
It(`returns nil if tries a filter with legacySearchExpr("'")`, func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "legacy"
r.filterMappings = map[string]filterFunc{
"name": fullTextFilter("table"),
}
@@ -77,6 +79,7 @@ var _ = Describe("sqlRestful", func() {
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "legacy"
tableName = "test_table"
mbidFields = []string{"mbid", "artist_mbid"}
filter = fullTextFilter(tableName, mbidFields...)
@@ -99,11 +102,11 @@ var _ = Describe("sqlRestful", func() {
uuid := "550e8400-e29b-41d4-a716-446655440000"
result := noMbidFilter("search", uuid)
// mbidExpr with no fields returns nil, so cmp.Or falls back to fullTextExpr
expected := squirrel.And{
squirrel.Like{"test_table.full_text": "% 550e8400-e29b-41d4-a716-446655440000%"},
}
Expect(result).To(Equal(expected))
// mbidExpr with no fields returns nil, so cmp.Or falls back to search strategy
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(ContainElement("% 550e8400-e29b-41d4-a716-446655440000%"))
})
})
@@ -111,54 +114,75 @@ var _ = Describe("sqlRestful", func() {
It("returns full text search condition only", func() {
result := filter("search", "beatles")
// mbidExpr returns nil for non-UUIDs, so fullTextExpr result is returned directly
expected := squirrel.And{
squirrel.Like{"test_table.full_text": "% beatles%"},
}
Expect(result).To(Equal(expected))
// mbidExpr returns nil for non-UUIDs, so search strategy result is returned directly
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(ContainElement("% beatles%"))
})
It("handles multi-word search terms", func() {
result := filter("search", "the beatles abbey road")
// Should return And condition directly
andCondition, ok := result.(squirrel.And)
Expect(ok).To(BeTrue())
Expect(andCondition).To(HaveLen(4))
// Check that all words are present (order may vary)
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% the%"}))
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% beatles%"}))
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% abbey%"}))
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% road%"}))
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
// All words should be present as LIKE conditions
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(HaveLen(4))
Expect(args).To(ContainElement("% the%"))
Expect(args).To(ContainElement("% beatles%"))
Expect(args).To(ContainElement("% abbey%"))
Expect(args).To(ContainElement("% road%"))
})
})
Context("when SearchFullString config changes behavior", func() {
It("uses different separator with SearchFullString=false", func() {
conf.Server.SearchFullString = false
conf.Server.Search.FullString = false
result := filter("search", "test query")
andCondition, ok := result.(squirrel.And)
Expect(ok).To(BeTrue())
Expect(andCondition).To(HaveLen(2))
// Check that all words are present with leading space (order may vary)
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% test%"}))
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% query%"}))
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(HaveLen(2))
Expect(args).To(ContainElement("% test%"))
Expect(args).To(ContainElement("% query%"))
})
It("uses no separator with SearchFullString=true", func() {
conf.Server.SearchFullString = true
conf.Server.Search.FullString = true
result := filter("search", "test query")
andCondition, ok := result.(squirrel.And)
Expect(ok).To(BeTrue())
Expect(andCondition).To(HaveLen(2))
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(HaveLen(2))
Expect(args).To(ContainElement("%test%"))
Expect(args).To(ContainElement("%query%"))
})
})
// Check that all words are present without leading space (order may vary)
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "%test%"}))
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "%query%"}))
Context("single-character queries (regression: must not be rejected)", func() {
It("returns valid filter for single-char query with legacy backend", func() {
conf.Server.Search.Backend = "legacy"
result := filter("search", "a")
Expect(result).ToNot(BeNil(), "single-char REST filter must not be dropped")
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("LIKE"))
Expect(args).ToNot(BeEmpty())
})
It("returns valid filter for single-char query with FTS backend", func() {
conf.Server.Search.Backend = "fts"
conf.Server.Search.FullString = false
ftsFilter := fullTextFilter(tableName, mbidFields...)
result := ftsFilter("search", "a")
Expect(result).ToNot(BeNil(), "single-char REST filter must not be dropped")
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("MATCH"))
Expect(args).ToNot(BeEmpty())
})
})
@@ -176,10 +200,10 @@ var _ = Describe("sqlRestful", func() {
It("handles special characters that are sanitized", func() {
result := filter("search", "don't")
expected := squirrel.And{
squirrel.Like{"test_table.full_text": "% dont%"}, // str.SanitizeStrings removes quotes
}
Expect(result).To(Equal(expected))
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(ContainElement("% dont%"))
})
It("returns nil for single quote (SQL injection protection)", func() {
@@ -203,31 +227,30 @@ var _ = Describe("sqlRestful", func() {
result := filter("search", "550e8400-invalid-uuid")
// Should return full text filter since UUID is invalid
expected := squirrel.And{
squirrel.Like{"test_table.full_text": "% 550e8400-invalid-uuid%"},
}
Expect(result).To(Equal(expected))
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(ContainElement("% 550e8400-invalid-uuid%"))
})
It("handles empty mbid fields array", func() {
emptyMbidFilter := fullTextFilter(tableName, []string{}...)
result := emptyMbidFilter("search", "test")
// mbidExpr with empty fields returns nil, so cmp.Or falls back to fullTextExpr
expected := squirrel.And{
squirrel.Like{"test_table.full_text": "% test%"},
}
Expect(result).To(Equal(expected))
// mbidExpr with empty fields returns nil, so search strategy result is returned directly
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(ContainElement("% test%"))
})
It("converts value to lowercase before processing", func() {
result := filter("search", "TEST")
// The function converts to lowercase internally
expected := squirrel.And{
squirrel.Like{"test_table.full_text": "% test%"},
}
Expect(result).To(Equal(expected))
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(ContainElement("% test%"))
})
})
})

View File

@@ -15,36 +15,71 @@ func formatFullText(text ...string) string {
return " " + fullText
}
// doSearch performs a full-text search with the specified parameters.
// The naturalOrder is used to sort results when no full-text filter is applied. It is useful for cases like
// OpenSubsonic, where an empty search query should return all results in a natural order. Normally the parameter
// should be `tableName + ".rowid"`, but some repositories (ex: artist) may use a different natural order.
func (r sqlRepository) doSearch(sq SelectBuilder, q string, offset, size int, results any, naturalOrder string, orderBys ...string) error {
// searchConfig holds per-repository constants for doSearch.
type searchConfig struct {
NaturalOrder string // ORDER BY for empty-query results (e.g. "album.rowid")
OrderBy []string // ORDER BY for text search results (e.g. ["name"])
MBIDFields []string // columns to match when query is a UUID
// LibraryFilter overrides the default applyLibraryFilter for FTS Phase 1.
// Needed when library access requires a junction table (e.g. artist → library_artist).
LibraryFilter func(sq SelectBuilder) SelectBuilder
}
// searchStrategy defines how to execute a text search against a repository table.
// options carries filters and pagination that must reach all query phases,
// including FTS Phase 1 which builds its own query outside sq.
type searchStrategy interface {
Sqlizer
execute(r sqlRepository, sq SelectBuilder, dest any, cfg searchConfig, options model.QueryOptions) error
}
// getSearchStrategy returns the appropriate search strategy based on config and query content.
// Returns nil when the query produces no searchable tokens.
func getSearchStrategy(tableName, query string) searchStrategy {
if conf.Server.Search.Backend == "legacy" || conf.Server.Search.FullString {
return newLegacySearch(tableName, query)
}
if containsCJK(query) {
return newLikeSearch(tableName, query)
}
return newFTSSearch(tableName, query)
}
// doSearch dispatches a search query: empty → natural order, UUID → MBID match,
// otherwise delegates to getSearchStrategy. sq must already have LIMIT/OFFSET set
// via newSelect(options...). options is forwarded so FTS Phase 1 can apply the same
// filters and pagination independently.
func (r sqlRepository) doSearch(sq SelectBuilder, q string, results any, cfg searchConfig, options model.QueryOptions) error {
q = strings.TrimSpace(q)
q = strings.TrimSuffix(q, "*")
sq = sq.Where(Eq{r.tableName + ".missing": false})
// Empty query (OpenSubsonic `search3?query=""`) — return all in natural order.
if q == "" || q == `""` {
sq = sq.OrderBy(cfg.NaturalOrder)
return r.queryAll(sq, results, options)
}
// MBID search: if query is a valid UUID, search by MBID fields instead
if uuid.Validate(q) == nil && len(cfg.MBIDFields) > 0 {
sq = sq.Where(mbidExpr(r.tableName, q, cfg.MBIDFields...))
return r.queryAll(sq, results)
}
// Min-length guard: single-character queries are too broad for search3.
// This check lives here (not in the strategies) so that fullTextFilter
// (REST filter path) can still use single-character queries.
if len(q) < 2 {
return nil
}
filter := fullTextExpr(r.tableName, q)
if filter != nil {
sq = sq.Where(filter)
sq = sq.OrderBy(orderBys...)
} else {
// This is to speed up the results of `search3?query=""`, for OpenSubsonic
// If the filter is empty, we sort by the specified natural order.
sq = sq.OrderBy(naturalOrder)
strategy := getSearchStrategy(r.tableName, q)
if strategy == nil {
return nil
}
sq = sq.Where(Eq{r.tableName + ".missing": false})
sq = sq.Limit(uint64(size)).Offset(uint64(offset))
return r.queryAll(sq, results, model.QueryOptions{Offset: offset})
}
func (r sqlRepository) searchByMBID(sq SelectBuilder, mbid string, mbidFields []string, results any) error {
sq = sq.Where(mbidExpr(r.tableName, mbid, mbidFields...))
sq = sq.Where(Eq{r.tableName + ".missing": false})
return r.queryAll(sq, results)
return strategy.execute(r, sq, results, cfg, options)
}
func mbidExpr(tableName, mbid string, mbidFields ...string) Sqlizer {
@@ -58,20 +93,3 @@ func mbidExpr(tableName, mbid string, mbidFields ...string) Sqlizer {
}
return Or(cond)
}
func fullTextExpr(tableName string, s string) Sqlizer {
q := str.SanitizeStrings(s)
if q == "" {
return nil
}
var sep string
if !conf.Server.SearchFullString {
sep = " "
}
parts := strings.Split(q, " ")
filters := And{}
for _, part := range parts {
filters = append(filters, Like{tableName + ".full_text": "%" + sep + part + "%"})
}
return filters
}

View File

@@ -0,0 +1,422 @@
package persistence
import (
"fmt"
"regexp"
"strings"
"unicode"
"unicode/utf8"
. "github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
)
// containsCJK returns true if the string contains any CJK (Chinese/Japanese/Korean) characters.
// CJK text doesn't use spaces between words, so FTS5's unicode61 tokenizer treats entire
// CJK phrases as single tokens, making token-based search ineffective for CJK content.
func containsCJK(s string) bool {
for _, r := range s {
if unicode.Is(unicode.Han, r) ||
unicode.Is(unicode.Hiragana, r) ||
unicode.Is(unicode.Katakana, r) ||
unicode.Is(unicode.Hangul, r) {
return true
}
}
return false
}
// fts5SpecialChars matches characters that should be stripped from user input.
// We keep only Unicode letters, numbers, whitespace, * (prefix wildcard), " (phrase quotes),
// and \x00 (internal placeholder marker). All punctuation is removed because the unicode61
// tokenizer treats it as token separators, and characters like ' can cause FTS5 parse errors
// as unbalanced string delimiters.
var fts5SpecialChars = regexp.MustCompile(`[^\p{L}\p{N}\s*"\x00]`)
// fts5PunctStrip strips everything except letters and numbers (no whitespace, wildcards, or quotes).
// Used for normalizing words at index time to create concatenated forms (e.g., "R.E.M." → "REM").
var fts5PunctStrip = regexp.MustCompile(`[^\p{L}\p{N}]`)
// fts5Operators matches FTS5 boolean operators as whole words (case-insensitive).
var fts5Operators = regexp.MustCompile(`(?i)\b(AND|OR|NOT|NEAR)\b`)
// fts5LeadingStar matches a * at the start of a token. FTS5 only supports * at the end (prefix queries).
var fts5LeadingStar = regexp.MustCompile(`(^|[\s])\*+`)
// normalizeForFTS takes multiple strings, strips non-letter/non-number characters from each word,
// and returns a space-separated string of words that changed after stripping (deduplicated).
// This is used at index time to create concatenated forms: "R.E.M." → "REM", "AC/DC" → "ACDC".
func normalizeForFTS(values ...string) string {
seen := make(map[string]struct{})
var result []string
for _, v := range values {
for _, word := range strings.Fields(v) {
stripped := fts5PunctStrip.ReplaceAllString(word, "")
if stripped == "" || stripped == word {
continue
}
lower := strings.ToLower(stripped)
if _, ok := seen[lower]; ok {
continue
}
seen[lower] = struct{}{}
result = append(result, stripped)
}
}
return strings.Join(result, " ")
}
// isSingleUnicodeLetter returns true if token is exactly one Unicode letter.
func isSingleUnicodeLetter(token string) bool {
r, size := utf8.DecodeRuneInString(token)
return size == len(token) && size > 0 && unicode.IsLetter(r)
}
// namePunctuation is the set of characters commonly used as separators in artist/album
// names (hyphens, slashes, dots, apostrophes). Only words containing these are candidates
// for punctuated-word processing; other special characters (^, :, &) are just stripped.
const namePunctuation = `-/.''`
// processPunctuatedWords handles words with embedded name punctuation before the general
// special-character stripping. For each punctuated word it produces either:
// - A quoted phrase for dotted abbreviations: R.E.M. → "R E M"
// - A phrase+concat OR for other patterns: a-ha → ("a ha" OR aha*)
func processPunctuatedWords(input string, phrases []string) (string, []string) {
words := strings.Fields(input)
var result []string
for _, w := range words {
if strings.HasPrefix(w, "\x00") || strings.ContainsAny(w, `*"`) || !strings.ContainsAny(w, namePunctuation) {
result = append(result, w)
continue
}
concat := fts5PunctStrip.ReplaceAllString(w, "")
if concat == "" || concat == w {
result = append(result, w)
continue
}
subTokens := strings.Fields(fts5SpecialChars.ReplaceAllString(w, " "))
if len(subTokens) < 2 {
// Single sub-token after splitting (e.g., N' → N): just use the stripped form
result = append(result, concat)
continue
}
// Dotted abbreviations (R.E.M., U.K.) — all single letters separated by dots only
if isDottedAbbreviation(w, subTokens) {
phrases = append(phrases, fmt.Sprintf(`"%s"`, strings.Join(subTokens, " ")))
} else {
// Punctuated names (a-ha, AC/DC, Jay-Z) — phrase for adjacency + concat for search_normalized
phrases = append(phrases, fmt.Sprintf(`("%s" OR %s*)`, strings.Join(subTokens, " "), concat))
}
result = append(result, fmt.Sprintf("\x00PHRASE%d\x00", len(phrases)-1))
}
return strings.Join(result, " "), phrases
}
// isDottedAbbreviation returns true if w uses only dots as punctuation and all sub-tokens
// are single letters (e.g., "R.E.M.", "U.K." but not "a-ha" or "AC/DC").
func isDottedAbbreviation(w string, subTokens []string) bool {
for _, r := range w {
if !unicode.IsLetter(r) && !unicode.IsNumber(r) && r != '.' {
return false
}
}
for _, st := range subTokens {
if !isSingleUnicodeLetter(st) {
return false
}
}
return true
}
// buildFTS5Query preprocesses user input into a safe FTS5 MATCH expression.
// It preserves quoted phrases and * prefix wildcards, neutralizes FTS5 operators
// (by lowercasing them, since FTS5 operators are case-sensitive) and strips
// special characters to prevent query injection.
func buildFTS5Query(userInput string) string {
q := strings.TrimSpace(userInput)
if q == "" || q == `""` {
return ""
}
var phrases []string
result := q
for {
start := strings.Index(result, `"`)
if start == -1 {
break
}
end := strings.Index(result[start+1:], `"`)
if end == -1 {
// Unmatched quote — remove it
result = result[:start] + result[start+1:]
break
}
end += start + 1
phrase := result[start : end+1] // includes quotes
phrases = append(phrases, phrase)
result = result[:start] + fmt.Sprintf("\x00PHRASE%d\x00", len(phrases)-1) + result[end+1:]
}
// Neutralize FTS5 operators by lowercasing them (FTS5 operators are case-sensitive:
// AND, OR, NOT, NEAR are operators, but and, or, not, near are plain tokens)
result = fts5Operators.ReplaceAllStringFunc(result, strings.ToLower)
// Handle words with embedded punctuation (a-ha, AC/DC, R.E.M.) before stripping
result, phrases = processPunctuatedWords(result, phrases)
result = fts5SpecialChars.ReplaceAllString(result, " ")
result = fts5LeadingStar.ReplaceAllString(result, "$1")
tokens := strings.Fields(result)
// Append * to plain tokens for prefix matching (e.g., "love" → "love*").
// Skip tokens that are already wildcarded or are quoted phrase placeholders.
for i, t := range tokens {
if strings.HasPrefix(t, "\x00") || strings.HasSuffix(t, "*") {
continue
}
tokens[i] = t + "*"
}
result = strings.Join(tokens, " ")
for i, phrase := range phrases {
placeholder := fmt.Sprintf("\x00PHRASE%d\x00", i)
result = strings.ReplaceAll(result, placeholder, phrase)
}
return result
}
// ftsColumn pairs an FTS5 column name with its BM25 relevance weight.
type ftsColumn struct {
Name string
Weight float64
}
// ftsColumnDefs defines FTS5 columns and their BM25 relevance weights.
// The order MUST match the column order in the FTS5 table definition (see migrations).
// All columns are both searched and ranked. When adding indexed-but-not-searched
// columns in the future, use Weight: 0 to exclude from the search column filter.
var ftsColumnDefs = map[string][]ftsColumn{
"media_file": {
{"title", 10.0},
{"album", 5.0},
{"artist", 3.0},
{"album_artist", 3.0},
{"sort_title", 1.0},
{"sort_album_name", 1.0},
{"sort_artist_name", 1.0},
{"sort_album_artist_name", 1.0},
{"disc_subtitle", 1.0},
{"search_participants", 2.0},
{"search_normalized", 1.0},
},
"album": {
{"name", 10.0},
{"sort_album_name", 1.0},
{"album_artist", 3.0},
{"search_participants", 2.0},
{"discs", 1.0},
{"catalog_num", 1.0},
{"album_version", 1.0},
{"search_normalized", 1.0},
},
"artist": {
{"name", 10.0},
{"sort_artist_name", 1.0},
{"search_normalized", 1.0},
},
}
// ftsColumnFilters and ftsBM25Weights are precomputed from ftsColumnDefs at init time
// to avoid per-query allocations.
var (
ftsColumnFilters = map[string]string{}
ftsBM25Weights = map[string]string{}
)
func init() {
for table, cols := range ftsColumnDefs {
var names []string
weights := make([]string, len(cols))
for i, c := range cols {
if c.Weight > 0 {
names = append(names, c.Name)
}
weights[i] = fmt.Sprintf("%.1f", c.Weight)
}
ftsColumnFilters[table] = "{" + strings.Join(names, " ") + "}"
ftsBM25Weights[table] = strings.Join(weights, ", ")
}
}
// ftsSearch implements searchStrategy using FTS5 full-text search with BM25 ranking.
type ftsSearch struct {
tableName string
ftsTable string
matchExpr string
rankExpr string
}
// ToSql returns a single-query fallback for the REST filter path (no two-phase split).
func (s *ftsSearch) ToSql() (string, []interface{}, error) {
sql := s.tableName + ".rowid IN (SELECT rowid FROM " + s.ftsTable + " WHERE " + s.ftsTable + " MATCH ?)"
return sql, []interface{}{s.matchExpr}, nil
}
// execute runs a two-phase FTS5 search:
// - Phase 1: lightweight rowid query (main table + FTS + library filter) for ranking and pagination.
// - Phase 2: full SELECT with all JOINs, scoped to Phase 1's rowid set.
//
// Complex ORDER BY (function calls, aggregations) are dropped from Phase 1.
func (s *ftsSearch) execute(r sqlRepository, sq SelectBuilder, dest any, cfg searchConfig, options model.QueryOptions) error {
qualifiedOrderBys := []string{s.rankExpr}
for _, ob := range cfg.OrderBy {
if qualified := qualifyOrderBy(s.tableName, ob); qualified != "" {
qualifiedOrderBys = append(qualifiedOrderBys, qualified)
}
}
// Phase 1: fresh query — must set LIMIT/OFFSET from options explicitly.
// Mirror applyOptions behavior: Max=0 means no limit, not LIMIT 0.
rowidQuery := Select(s.tableName+".rowid").
From(s.tableName).
Join(s.ftsTable+" ON "+s.ftsTable+".rowid = "+s.tableName+".rowid AND "+s.ftsTable+" MATCH ?", s.matchExpr).
Where(Eq{s.tableName + ".missing": false}).
OrderBy(qualifiedOrderBys...)
if options.Max > 0 {
rowidQuery = rowidQuery.Limit(uint64(options.Max))
}
if options.Offset > 0 {
rowidQuery = rowidQuery.Offset(uint64(options.Offset))
}
// Library filter + musicFolderId must be applied here, before pagination.
if cfg.LibraryFilter != nil {
rowidQuery = cfg.LibraryFilter(rowidQuery)
} else {
rowidQuery = r.applyLibraryFilter(rowidQuery)
}
if options.Filters != nil {
rowidQuery = rowidQuery.Where(options.Filters)
}
rowidSQL, rowidArgs, err := rowidQuery.ToSql()
if err != nil {
return fmt.Errorf("building FTS rowid query: %w", err)
}
// Phase 2: strip LIMIT/OFFSET from sq (Phase 1 handled pagination),
// join on the ranked rowid set to hydrate with full columns.
sq = sq.RemoveLimit().RemoveOffset()
rankedSubquery := fmt.Sprintf(
"(SELECT rowid as _rid, row_number() OVER () AS _rn FROM (%s)) AS _ranked",
rowidSQL,
)
sq = sq.Join(rankedSubquery+" ON "+s.tableName+".rowid = _ranked._rid", rowidArgs...)
sq = sq.OrderBy("_ranked._rn")
return r.queryAll(sq, dest)
}
// qualifyOrderBy prepends tableName to a simple column name. Returns empty string for
// complex expressions (function calls, aggregations) that can't be used in Phase 1.
func qualifyOrderBy(tableName, orderBy string) string {
orderBy = strings.TrimSpace(orderBy)
if orderBy == "" || strings.ContainsAny(orderBy, "(,") {
return ""
}
parts := strings.Fields(orderBy)
if !strings.Contains(parts[0], ".") {
parts[0] = tableName + "." + parts[0]
}
return strings.Join(parts, " ")
}
// ftsQueryDegraded returns true when the FTS query lost significant discriminating
// content compared to the original input. This happens when special characters that
// are part of the entity name (e.g., "1+", "C++", "!!!", "C#") get stripped by FTS
// tokenization, leaving only very short/broad tokens. Also detects quoted phrases
// that would be degraded by FTS5's unicode61 tokenizer (e.g., "1+" → token "1").
func ftsQueryDegraded(original, ftsQuery string) bool {
original = strings.TrimSpace(original)
if original == "" || ftsQuery == "" {
return false
}
// Strip quotes from original for comparison — we want the raw content
stripped := strings.ReplaceAll(original, `"`, "")
// Extract the alphanumeric content from the original query
alphaNum := fts5PunctStrip.ReplaceAllString(stripped, "")
// If the original is entirely alphanumeric, nothing was stripped — not degraded
if len(alphaNum) == len(stripped) {
return false
}
// Check if all effective FTS tokens are very short (≤2 chars).
// Short tokens with prefix matching are too broad when special chars were stripped.
// For quoted phrases, extract the content and check the tokens inside.
tokens := strings.Fields(ftsQuery)
for _, t := range tokens {
t = strings.TrimSuffix(t, "*")
// Skip internal phrase placeholders
if strings.HasPrefix(t, "\x00") {
return false
}
// For OR groups from processPunctuatedWords (e.g., ("a ha" OR aha*)),
// the punctuated word was already handled meaningfully — not degraded.
if strings.HasPrefix(t, "(") {
return false
}
// For quoted phrases, check the tokens inside as FTS5 will tokenize them
if strings.HasPrefix(t, `"`) {
// Extract content between quotes
inner := strings.Trim(t, `"`)
innerAlpha := fts5PunctStrip.ReplaceAllString(inner, " ")
for _, it := range strings.Fields(innerAlpha) {
if len(it) > 2 {
return false
}
}
continue
}
if len(t) > 2 {
return false
}
}
return true
}
// newFTSSearch creates an FTS5 search strategy. Falls back to LIKE search if the
// query produces no FTS tokens (e.g., punctuation-only like "!!!!!!!") or if FTS
// tokenization stripped significant content from the query (e.g., "1+" → "1*").
// Returns nil when the query produces no searchable tokens at all.
func newFTSSearch(tableName, query string) searchStrategy {
q := buildFTS5Query(query)
if q == "" || ftsQueryDegraded(query, q) {
// Fallback: try LIKE search with the raw query
cleaned := strings.TrimSpace(strings.ReplaceAll(query, `"`, ""))
if cleaned != "" {
log.Trace("Search using LIKE fallback for non-tokenizable query", "table", tableName, "query", cleaned)
return newLikeSearch(tableName, cleaned)
}
return nil
}
ftsTable := tableName + "_fts"
matchExpr := q
if cols, ok := ftsColumnFilters[tableName]; ok {
matchExpr = cols + " : (" + q + ")"
}
rankExpr := ftsTable + ".rank"
if weights, ok := ftsBM25Weights[tableName]; ok {
rankExpr = "bm25(" + ftsTable + ", " + weights + ")"
}
s := &ftsSearch{
tableName: tableName,
ftsTable: ftsTable,
matchExpr: matchExpr,
rankExpr: rankExpr,
}
log.Trace("Search using FTS5 backend", "table", tableName, "query", q, "filter", s)
return s
}

View File

@@ -0,0 +1,435 @@
package persistence
import (
"context"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = DescribeTable("buildFTS5Query",
func(input, expected string) {
Expect(buildFTS5Query(input)).To(Equal(expected))
},
Entry("returns empty string for empty input", "", ""),
Entry("returns empty string for whitespace-only input", " ", ""),
Entry("appends * to a single word for prefix matching", "beatles", "beatles*"),
Entry("appends * to each word for prefix matching", "abbey road", "abbey* road*"),
Entry("preserves quoted phrases without appending *", `"the beatles"`, `"the beatles"`),
Entry("does not double-append * to existing prefix wildcard", "beat*", "beat*"),
Entry("strips FTS5 operators and appends * to lowercased words", "AND OR NOT NEAR", "and* or* not* near*"),
Entry("strips special FTS5 syntax characters and appends *", "test^col:val", "test* col* val*"),
Entry("handles mixed phrases and words", `"the beatles" abbey`, `"the beatles" abbey*`),
Entry("handles prefix with multiple words", "beat* abbey", "beat* abbey*"),
Entry("collapses multiple spaces", "abbey road", "abbey* road*"),
Entry("strips leading * from tokens and appends trailing *", "*livia", "livia*"),
Entry("strips leading * and preserves existing trailing *", "*livia oliv*", "livia* oliv*"),
Entry("strips standalone *", "*", ""),
Entry("strips apostrophe from input", "Guns N' Roses", "Guns* N* Roses*"),
Entry("converts slashed word to phrase+concat OR", "AC/DC", `("AC DC" OR ACDC*)`),
Entry("converts hyphenated word to phrase+concat OR", "a-ha", `("a ha" OR aha*)`),
Entry("converts partial hyphenated word to phrase+concat OR", "a-h", `("a h" OR ah*)`),
Entry("converts hyphenated name to phrase+concat OR", "Jay-Z", `("Jay Z" OR JayZ*)`),
Entry("converts contraction to phrase+concat OR", "it's", `("it s" OR its*)`),
Entry("handles punctuated word mixed with plain words", "best of a-ha", `best* of* ("a ha" OR aha*)`),
Entry("strips miscellaneous punctuation", "rock & roll, vol. 2", "rock* roll* vol* 2*"),
Entry("preserves unicode characters with diacritics", "Björk début", "Björk* début*"),
Entry("collapses dotted abbreviation into phrase", "R.E.M.", `"R E M"`),
Entry("collapses abbreviation without trailing dot", "R.E.M", `"R E M"`),
Entry("collapses abbreviation mixed with words", "best of R.E.M.", `best* of* "R E M"`),
Entry("collapses two-letter abbreviation", "U.K.", `"U K"`),
Entry("does not collapse single letter surrounded by words", "I am fine", "I* am* fine*"),
Entry("does not collapse single standalone letter", "A test", "A* test*"),
Entry("preserves quoted phrase with punctuation verbatim", `"ac/dc"`, `"ac/dc"`),
Entry("preserves quoted abbreviation verbatim", `"R.E.M."`, `"R.E.M."`),
Entry("returns empty string for punctuation-only input", "!!!!!!!", ""),
Entry("returns empty string for mixed punctuation", "!@#$%^&", ""),
Entry("returns empty string for empty quoted phrase", `""`, ""),
)
var _ = DescribeTable("ftsQueryDegraded",
func(original, ftsQuery string, expected bool) {
Expect(ftsQueryDegraded(original, ftsQuery)).To(Equal(expected))
},
Entry("not degraded for empty original", "", "1*", false),
Entry("not degraded for empty ftsQuery", "1+", "", false),
Entry("not degraded for purely alphanumeric query", "beatles", "beatles*", false),
Entry("not degraded when long tokens remain", "test^val", "test* val*", false),
Entry("not degraded for quoted phrase with long tokens", `"the beatles"`, `"the beatles"`, false),
Entry("degraded for quoted phrase with only short tokens after tokenizer strips special chars", `"1+"`, `"1+"`, true),
Entry("not degraded for quoted phrase with meaningful content", `"C++ programming"`, `"C++ programming"`, false),
Entry("degraded when special chars stripped leaving short token", "1+", "1*", true),
Entry("degraded when special chars stripped leaving two short tokens", "C# 1", "C* 1*", true),
Entry("not degraded when at least one long token remains", "1+ beatles", "1* beatles*", false),
Entry("not degraded for OR groups from processPunctuatedWords", "AC/DC", `("AC DC" OR ACDC*)`, false),
)
var _ = DescribeTable("normalizeForFTS",
func(expected string, values ...string) {
Expect(normalizeForFTS(values...)).To(Equal(expected))
},
Entry("strips dots and concatenates", "REM", "R.E.M."),
Entry("strips slash", "ACDC", "AC/DC"),
Entry("strips hyphen", "Aha", "A-ha"),
Entry("skips unchanged words", "", "The Beatles"),
Entry("handles mixed input", "REM", "R.E.M.", "Automatic for the People"),
Entry("deduplicates", "REM", "R.E.M.", "R.E.M."),
Entry("strips apostrophe from word", "N", "Guns N' Roses"),
Entry("handles multiple values with punctuation", "REM ACDC", "R.E.M.", "AC/DC"),
)
var _ = DescribeTable("containsCJK",
func(input string, expected bool) {
Expect(containsCJK(input)).To(Equal(expected))
},
Entry("returns false for empty string", "", false),
Entry("returns false for ASCII text", "hello world", false),
Entry("returns false for Latin with diacritics", "Björk début", false),
Entry("detects Chinese characters (Han)", "周杰伦", true),
Entry("detects Japanese Hiragana", "こんにちは", true),
Entry("detects Japanese Katakana", "カタカナ", true),
Entry("detects Korean Hangul", "한국어", true),
Entry("detects CJK mixed with Latin", "best of 周杰伦", true),
Entry("detects single CJK character", "a曲b", true),
)
var _ = DescribeTable("qualifyOrderBy",
func(tableName, orderBy, expected string) {
Expect(qualifyOrderBy(tableName, orderBy)).To(Equal(expected))
},
Entry("returns empty string for empty input", "artist", "", ""),
Entry("qualifies simple column with table name", "artist", "name", "artist.name"),
Entry("qualifies column with direction", "artist", "name desc", "artist.name desc"),
Entry("preserves already-qualified column", "artist", "artist.name", "artist.name"),
Entry("preserves already-qualified column with direction", "artist", "artist.name desc", "artist.name desc"),
Entry("returns empty for function call expression", "artist", "sum(json_extract(stats, '$.total.m')) desc", ""),
Entry("returns empty for expression with comma", "artist", "a, b", ""),
Entry("qualifies media_file column", "media_file", "title", "media_file.title"),
)
var _ = Describe("ftsColumnDefs helpers", func() {
Describe("ftsColumnFilters", func() {
It("returns column filter for media_file", func() {
Expect(ftsColumnFilters).To(HaveKeyWithValue("media_file",
"{title album artist album_artist sort_title sort_album_name sort_artist_name sort_album_artist_name disc_subtitle search_participants search_normalized}",
))
})
It("returns column filter for album", func() {
Expect(ftsColumnFilters).To(HaveKeyWithValue("album",
"{name sort_album_name album_artist search_participants discs catalog_num album_version search_normalized}",
))
})
It("returns column filter for artist", func() {
Expect(ftsColumnFilters).To(HaveKeyWithValue("artist",
"{name sort_artist_name search_normalized}",
))
})
It("has no entry for unknown table", func() {
Expect(ftsColumnFilters).ToNot(HaveKey("unknown"))
})
})
Describe("ftsBM25Weights", func() {
It("returns weight CSV for media_file", func() {
Expect(ftsBM25Weights).To(HaveKeyWithValue("media_file",
"10.0, 5.0, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 1.0",
))
})
It("returns weight CSV for album", func() {
Expect(ftsBM25Weights).To(HaveKeyWithValue("album",
"10.0, 1.0, 3.0, 2.0, 1.0, 1.0, 1.0, 1.0",
))
})
It("returns weight CSV for artist", func() {
Expect(ftsBM25Weights).To(HaveKeyWithValue("artist",
"10.0, 1.0, 1.0",
))
})
It("has no entry for unknown table", func() {
Expect(ftsBM25Weights).ToNot(HaveKey("unknown"))
})
})
It("has definitions for all known tables", func() {
for _, table := range []string{"media_file", "album", "artist"} {
Expect(ftsColumnDefs).To(HaveKey(table))
Expect(ftsColumnDefs[table]).ToNot(BeEmpty())
}
})
It("has matching column count between filter and weights", func() {
for table, cols := range ftsColumnDefs {
// Column filter only includes Weight > 0 columns
filterCount := 0
for _, c := range cols {
if c.Weight > 0 {
filterCount++
}
}
// For now, all columns have Weight > 0, so filter count == total count
Expect(filterCount).To(Equal(len(cols)), "table %s: all columns should have positive weights", table)
}
})
})
var _ = Describe("newFTSSearch", func() {
It("returns nil for empty query", func() {
Expect(newFTSSearch("media_file", "")).To(BeNil())
})
It("returns non-nil for single-character query", func() {
strategy := newFTSSearch("media_file", "a")
Expect(strategy).ToNot(BeNil(), "single-char queries must not be rejected; min-length is enforced in doSearch, not here")
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("MATCH"))
})
It("returns ftsSearch with correct table names and MATCH expression", func() {
strategy := newFTSSearch("media_file", "beatles")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.tableName).To(Equal("media_file"))
Expect(fts.ftsTable).To(Equal("media_file_fts"))
Expect(fts.matchExpr).To(HavePrefix("{title album artist album_artist"))
Expect(fts.matchExpr).To(ContainSubstring("beatles*"))
})
It("ToSql generates rowid IN subquery with MATCH (fallback path)", func() {
strategy := newFTSSearch("media_file", "beatles")
sql, args, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("media_file.rowid IN"))
Expect(sql).To(ContainSubstring("media_file_fts"))
Expect(sql).To(ContainSubstring("MATCH"))
Expect(args).To(HaveLen(1))
})
It("generates correct FTS table name per entity", func() {
for _, table := range []string{"media_file", "album", "artist"} {
strategy := newFTSSearch(table, "test")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.tableName).To(Equal(table))
Expect(fts.ftsTable).To(Equal(table + "_fts"))
}
})
It("builds bm25() rank expression with column weights", func() {
strategy := newFTSSearch("media_file", "beatles")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.rankExpr).To(HavePrefix("bm25(media_file_fts,"))
Expect(fts.rankExpr).To(ContainSubstring("10.0"))
strategy = newFTSSearch("artist", "beatles")
fts, ok = strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.rankExpr).To(HavePrefix("bm25(artist_fts,"))
})
It("falls back to ftsTable.rank for unknown tables", func() {
strategy := newFTSSearch("unknown_table", "test")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.rankExpr).To(Equal("unknown_table_fts.rank"))
})
It("wraps query with column filter for known tables", func() {
strategy := newFTSSearch("artist", "Beatles")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.matchExpr).To(Equal("{name sort_artist_name search_normalized} : (Beatles*)"))
})
It("passes query without column filter for unknown tables", func() {
strategy := newFTSSearch("unknown_table", "test")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.matchExpr).To(Equal("test*"))
})
It("preserves phrase queries inside column filter", func() {
strategy := newFTSSearch("media_file", `"the beatles"`)
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.matchExpr).To(ContainSubstring(`"the beatles"`))
})
It("preserves prefix queries inside column filter", func() {
strategy := newFTSSearch("media_file", "beat*")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.matchExpr).To(ContainSubstring("beat*"))
})
It("falls back to LIKE search for punctuation-only query", func() {
strategy := newFTSSearch("media_file", "!!!!!!!")
Expect(strategy).ToNot(BeNil())
_, ok := strategy.(*ftsSearch)
Expect(ok).To(BeFalse(), "punctuation-only should fall back to LIKE, not FTS")
sql, args, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("LIKE"))
Expect(args).To(ContainElement("%!!!!!!!%"))
})
It("falls back to LIKE search for degraded query (special chars stripped leaving short tokens)", func() {
strategy := newFTSSearch("album", "1+")
Expect(strategy).ToNot(BeNil())
_, ok := strategy.(*ftsSearch)
Expect(ok).To(BeFalse(), "degraded query should fall back to LIKE, not FTS")
sql, args, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("LIKE"))
Expect(args).To(ContainElement("%1+%"))
})
It("returns nil for empty string even with LIKE fallback", func() {
Expect(newFTSSearch("media_file", "")).To(BeNil())
Expect(newFTSSearch("media_file", " ")).To(BeNil())
})
It("returns nil for empty quoted phrase", func() {
Expect(newFTSSearch("media_file", `""`)).To(BeNil())
})
})
var _ = Describe("FTS5 Integration Search", func() {
var (
mr model.MediaFileRepository
alr model.AlbumRepository
arr model.ArtistRepository
)
BeforeEach(func() {
ctx := log.NewContext(context.TODO())
ctx = request.WithUser(ctx, adminUser)
conn := GetDBXBuilder()
mr = NewMediaFileRepository(ctx, conn)
alr = NewAlbumRepository(ctx, conn)
arr = NewArtistRepository(ctx, conn)
})
Describe("MediaFile search", func() {
It("finds media files by title", func() {
results, err := mr.Search("Radioactivity", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Title).To(Equal("Radioactivity"))
Expect(results[0].ID).To(Equal(songRadioactivity.ID))
})
It("finds media files by artist name", func() {
results, err := mr.Search("Beatles", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(3))
for _, r := range results {
Expect(r.Artist).To(Equal("The Beatles"))
}
})
})
Describe("Album search", func() {
It("finds albums by name", func() {
results, err := alr.Search("Sgt Peppers", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Name).To(Equal("Sgt Peppers"))
Expect(results[0].ID).To(Equal(albumSgtPeppers.ID))
})
It("finds albums with multi-word search", func() {
results, err := alr.Search("Abbey Road", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(2))
})
})
Describe("Artist search", func() {
It("finds artists by name", func() {
results, err := arr.Search("Kraftwerk", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Name).To(Equal("Kraftwerk"))
Expect(results[0].ID).To(Equal(artistKraftwerk.ID))
})
})
Describe("CJK search", func() {
It("finds media files by CJK title", func() {
results, err := mr.Search("プラチナ", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Title).To(Equal("プラチナ・ジェット"))
Expect(results[0].ID).To(Equal(songCJK.ID))
})
It("finds media files by CJK artist name", func() {
results, err := mr.Search("シートベルツ", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Artist).To(Equal("シートベルツ"))
})
It("finds albums by CJK artist name", func() {
results, err := alr.Search("シートベルツ", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Name).To(Equal("COWBOY BEBOP"))
Expect(results[0].ID).To(Equal(albumCJK.ID))
})
It("finds artists by CJK name", func() {
results, err := arr.Search("シートベルツ", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Name).To(Equal("シートベルツ"))
Expect(results[0].ID).To(Equal(artistCJK.ID))
})
})
Describe("Album version search", func() {
It("finds albums by version tag via FTS", func() {
results, err := alr.Search("Deluxe", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].ID).To(Equal(albumWithVersion.ID))
})
})
Describe("Punctuation-only search", func() {
It("finds media files with punctuation-only title", func() {
results, err := mr.Search("!!!!!!!", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Title).To(Equal("!!!!!!!"))
Expect(results[0].ID).To(Equal(songPunctuation.ID))
})
})
Describe("Single-character search (doSearch min-length guard)", func() {
It("returns empty results for single-char query via Search", func() {
results, err := mr.Search("a", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty(), "doSearch should reject single-char queries")
})
})
Describe("Max=0 means no limit (regression: must not produce LIMIT 0)", func() {
It("returns results with Max=0", func() {
results, err := mr.Search("Beatles", model.QueryOptions{Max: 0})
Expect(err).ToNot(HaveOccurred())
Expect(results).ToNot(BeEmpty(), "Max=0 should mean no limit, not LIMIT 0")
})
})
})

View File

@@ -0,0 +1,106 @@
package persistence
import (
"strings"
. "github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/str"
)
// likeSearch implements searchStrategy using LIKE-based SQL filters.
// Used for legacy full_text searches, CJK fallback, and punctuation-only fallback.
type likeSearch struct {
filter Sqlizer
}
func (s *likeSearch) ToSql() (string, []interface{}, error) {
return s.filter.ToSql()
}
func (s *likeSearch) execute(r sqlRepository, sq SelectBuilder, dest any, cfg searchConfig, options model.QueryOptions) error {
sq = sq.Where(s.filter)
sq = sq.OrderBy(cfg.OrderBy...)
return r.queryAll(sq, dest, options)
}
// newLegacySearch creates a LIKE search against the full_text column.
// Returns nil when the query produces no searchable tokens.
func newLegacySearch(tableName, query string) searchStrategy {
filter := legacySearchExpr(tableName, query)
if filter == nil {
return nil
}
return &likeSearch{filter: filter}
}
// newLikeSearch creates a LIKE search against core entity columns (CJK, punctuation fallback).
// No minimum length is enforced, since single CJK characters are meaningful words.
// Returns nil when the query produces no searchable tokens.
func newLikeSearch(tableName, query string) searchStrategy {
filter := likeSearchExpr(tableName, query)
if filter == nil {
return nil
}
return &likeSearch{filter: filter}
}
// legacySearchExpr generates LIKE-based search filters against the full_text column.
// This is the original search implementation, used when Search.Backend="legacy".
func legacySearchExpr(tableName string, s string) Sqlizer {
q := str.SanitizeStrings(s)
if q == "" {
log.Trace("Search using legacy backend, query is empty", "table", tableName)
return nil
}
var sep string
if !conf.Server.Search.FullString {
sep = " "
}
parts := strings.Split(q, " ")
filters := And{}
for _, part := range parts {
filters = append(filters, Like{tableName + ".full_text": "%" + sep + part + "%"})
}
log.Trace("Search using legacy backend", "query", filters, "table", tableName)
return filters
}
// likeSearchColumns defines the core columns to search with LIKE queries.
// These are the primary user-visible fields for each entity type.
// Used as a fallback when FTS5 cannot handle the query (e.g., CJK text, punctuation-only input).
var likeSearchColumns = map[string][]string{
"media_file": {"title", "album", "artist", "album_artist"},
"album": {"name", "album_artist"},
"artist": {"name"},
}
// likeSearchExpr generates LIKE-based search filters against core columns.
// Each word in the query must match at least one column (AND between words),
// and each word can match any column (OR within a word).
// Used as a fallback when FTS5 cannot handle the query (e.g., CJK text, punctuation-only input).
func likeSearchExpr(tableName string, s string) Sqlizer {
s = strings.TrimSpace(s)
if s == "" {
log.Trace("Search using LIKE backend, query is empty", "table", tableName)
return nil
}
columns, ok := likeSearchColumns[tableName]
if !ok {
log.Trace("Search using LIKE backend, couldn't find columns for this table", "table", tableName)
return nil
}
words := strings.Fields(s)
wordFilters := And{}
for _, word := range words {
colFilters := Or{}
for _, col := range columns {
colFilters = append(colFilters, Like{tableName + "." + col: "%" + word + "%"})
}
wordFilters = append(wordFilters, colFilters)
}
log.Trace("Search using LIKE backend", "query", wordFilters, "table", tableName)
return wordFilters
}

View File

@@ -0,0 +1,134 @@
package persistence
import (
"context"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("newLegacySearch", func() {
It("returns non-nil for single-character query", func() {
strategy := newLegacySearch("media_file", "a")
Expect(strategy).ToNot(BeNil(), "single-char queries must not be rejected; min-length is enforced in doSearch, not here")
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("LIKE"))
})
})
var _ = Describe("legacySearchExpr", func() {
It("returns nil for empty query", func() {
Expect(legacySearchExpr("media_file", "")).To(BeNil())
})
It("generates LIKE filter for single word", func() {
expr := legacySearchExpr("media_file", "beatles")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("media_file.full_text LIKE"))
Expect(args).To(ContainElement("% beatles%"))
})
It("generates AND of LIKE filters for multiple words", func() {
expr := legacySearchExpr("media_file", "abbey road")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("AND"))
Expect(args).To(HaveLen(2))
})
})
var _ = Describe("likeSearchExpr", func() {
It("returns nil for empty query", func() {
Expect(likeSearchExpr("media_file", "")).To(BeNil())
})
It("returns nil for whitespace-only query", func() {
Expect(likeSearchExpr("media_file", " ")).To(BeNil())
})
It("generates LIKE filters against core columns for single CJK word", func() {
expr := likeSearchExpr("media_file", "周杰伦")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
// Should have OR between columns for the single word
Expect(sql).To(ContainSubstring("OR"))
Expect(sql).To(ContainSubstring("media_file.title LIKE"))
Expect(sql).To(ContainSubstring("media_file.album LIKE"))
Expect(sql).To(ContainSubstring("media_file.artist LIKE"))
Expect(sql).To(ContainSubstring("media_file.album_artist LIKE"))
Expect(args).To(HaveLen(4))
for _, arg := range args {
Expect(arg).To(Equal("%周杰伦%"))
}
})
It("generates AND of OR groups for multi-word query", func() {
expr := likeSearchExpr("media_file", "周杰伦 greatest")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
// Two groups AND'd together, each with 4 columns OR'd
Expect(sql).To(ContainSubstring("AND"))
Expect(args).To(HaveLen(8))
})
It("uses correct columns for album table", func() {
expr := likeSearchExpr("album", "周杰伦")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("album.name LIKE"))
Expect(sql).To(ContainSubstring("album.album_artist LIKE"))
Expect(args).To(HaveLen(2))
})
It("uses correct columns for artist table", func() {
expr := likeSearchExpr("artist", "周杰伦")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("artist.name LIKE"))
Expect(args).To(HaveLen(1))
})
It("returns nil for unknown table", func() {
Expect(likeSearchExpr("unknown_table", "周杰伦")).To(BeNil())
})
})
var _ = Describe("Legacy Integration Search", func() {
var mr model.MediaFileRepository
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "legacy"
ctx := log.NewContext(context.TODO())
ctx = request.WithUser(ctx, adminUser)
conn := GetDBXBuilder()
mr = NewMediaFileRepository(ctx, conn)
})
It("returns results using legacy LIKE-based search", func() {
results, err := mr.Search("Radioactivity", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Title).To(Equal("Radioactivity"))
})
It("returns empty results for single-char query (doSearch min-length guard)", func() {
results, err := mr.Search("a", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty(), "doSearch should reject single-char queries")
})
It("returns results with Max=0 (regression: must not produce LIMIT 0)", func() {
results, err := mr.Search("Beatles", model.QueryOptions{Max: 0})
Expect(err).ToNot(HaveOccurred())
Expect(results).ToNot(BeEmpty(), "Max=0 should mean no limit, not LIMIT 0")
})
})

View File

@@ -1,6 +1,8 @@
package persistence
import (
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
@@ -11,4 +13,100 @@ var _ = Describe("sqlRepository", func() {
Expect(formatFullText("legiao urbana")).To(Equal(" legiao urbana"))
})
})
Describe("getSearchStrategy", func() {
It("returns FTS strategy by default", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "fts"
conf.Server.Search.FullString = false
strategy := getSearchStrategy("media_file", "test")
Expect(strategy).ToNot(BeNil())
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("MATCH"))
})
It("returns legacy LIKE strategy when SearchBackend is legacy", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "legacy"
conf.Server.Search.FullString = false
strategy := getSearchStrategy("media_file", "test")
Expect(strategy).ToNot(BeNil())
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("LIKE"))
})
It("falls back to legacy LIKE strategy when SearchFullString is enabled", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "fts"
conf.Server.Search.FullString = true
strategy := getSearchStrategy("media_file", "test")
Expect(strategy).ToNot(BeNil())
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("LIKE"))
})
It("routes CJK queries to LIKE strategy instead of FTS", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "fts"
conf.Server.Search.FullString = false
strategy := getSearchStrategy("media_file", "周杰伦")
Expect(strategy).ToNot(BeNil())
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
// CJK should use LIKE, not MATCH
Expect(sql).To(ContainSubstring("LIKE"))
Expect(sql).NotTo(ContainSubstring("MATCH"))
})
It("routes non-CJK queries to FTS strategy", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "fts"
conf.Server.Search.FullString = false
strategy := getSearchStrategy("media_file", "beatles")
Expect(strategy).ToNot(BeNil())
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("MATCH"))
})
It("returns non-nil for single-character query (no min-length in strategy)", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "fts"
conf.Server.Search.FullString = false
strategy := getSearchStrategy("media_file", "a")
Expect(strategy).ToNot(BeNil(), "single-char queries must be accepted by strategies (min-length is enforced in doSearch)")
})
It("returns non-nil for single-character query with legacy backend", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "legacy"
conf.Server.Search.FullString = false
strategy := getSearchStrategy("media_file", "a")
Expect(strategy).ToNot(BeNil(), "single-char queries must be accepted by legacy strategy (min-length is enforced in doSearch)")
})
It("uses legacy for CJK when SearchBackend is legacy", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "legacy"
conf.Server.Search.FullString = false
strategy := getSearchStrategy("media_file", "周杰伦")
Expect(strategy).ToNot(BeNil())
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
// Legacy should still use full_text column LIKE
Expect(sql).To(ContainSubstring("LIKE"))
Expect(sql).To(ContainSubstring("full_text"))
})
})
})

View File

@@ -0,0 +1,27 @@
package capabilities
// TaskWorker provides task execution handling.
// This capability allows plugins to receive callbacks when their queued tasks
// are ready to execute. Plugins that use the taskqueue host service must
// implement this capability.
//
//nd:capability name=taskworker
type TaskWorker interface {
// OnTaskExecute is called when a queued task is ready to run.
// The returned string is a status/result message stored in the tasks table.
// Return an error to trigger retry (if retries are configured).
//nd:export name=nd_task_execute
OnTaskExecute(TaskExecuteRequest) (string, error)
}
// TaskExecuteRequest is the request provided when a task is ready to execute.
type TaskExecuteRequest struct {
// QueueName is the name of the queue this task belongs to.
QueueName string `json:"queueName"`
// TaskID is the unique identifier for this task.
TaskID string `json:"taskId"`
// Payload is the opaque data provided when the task was enqueued.
Payload []byte `json:"payload"`
// Attempt is the current attempt number (1-based: first attempt = 1).
Attempt int32 `json:"attempt"`
}

View File

@@ -0,0 +1,38 @@
version: v1-draft
exports:
nd_task_execute:
description: |-
OnTaskExecute is called when a queued task is ready to run.
The returned string is a status/result message stored in the tasks table.
Return an error to trigger retry (if retries are configured).
input:
$ref: '#/components/schemas/TaskExecuteRequest'
contentType: application/json
output:
type: string
contentType: application/json
components:
schemas:
TaskExecuteRequest:
description: TaskExecuteRequest is the request provided when a task is ready to execute.
properties:
queueName:
type: string
description: QueueName is the name of the queue this task belongs to.
taskId:
type: string
description: TaskID is the unique identifier for this task.
payload:
type: array
description: Payload is the opaque data provided when the task was enqueued.
items:
type: object
attempt:
type: integer
format: int32
description: 'Attempt is the current attempt number (1-based: first attempt = 1).'
required:
- queueName
- taskId
- payload
- attempt

View File

@@ -282,9 +282,6 @@ type ServiceB interface {
Entry("option pattern (value, exists bool)",
"config_service.go.txt", "config_client_expected.go.txt", "config_client_expected.py", "config_client_expected.rs"),
Entry("raw=true binary response",
"raw_service.go.txt", "raw_client_expected.go.txt", "raw_client_expected.py", "raw_client_expected.rs"),
)
It("generates compilable client code for comprehensive service", func() {

View File

@@ -256,6 +256,15 @@ func GenerateClientRust(svc Service) ([]byte, error) {
return nil, fmt.Errorf("parsing template: %w", err)
}
partialContent, err := templatesFS.ReadFile("templates/base64_bytes.rs.tmpl")
if err != nil {
return nil, fmt.Errorf("reading base64_bytes partial: %w", err)
}
tmpl, err = tmpl.Parse(string(partialContent))
if err != nil {
return nil, fmt.Errorf("parsing base64_bytes partial: %w", err)
}
data := templateData{
Service: svc,
}
@@ -622,6 +631,15 @@ func GenerateCapabilityRust(cap Capability) ([]byte, error) {
return nil, fmt.Errorf("parsing template: %w", err)
}
partialContent, err := templatesFS.ReadFile("templates/base64_bytes.rs.tmpl")
if err != nil {
return nil, fmt.Errorf("reading base64_bytes partial: %w", err)
}
tmpl, err = tmpl.Parse(string(partialContent))
if err != nil {
return nil, fmt.Errorf("parsing base64_bytes partial: %w", err)
}
data := capabilityTemplateData{
Package: cap.Name,
Capability: cap,

View File

@@ -264,96 +264,6 @@ var _ = Describe("Generator", func() {
Expect(codeStr).To(ContainSubstring(`extism "github.com/extism/go-sdk"`))
})
It("should generate binary framing for raw=true methods", func() {
svc := Service{
Name: "Stream",
Permission: "stream",
Interface: "StreamService",
Methods: []Method{
{
Name: "GetStream",
HasError: true,
Raw: true,
Params: []Param{NewParam("uri", "string")},
Returns: []Param{
NewParam("contentType", "string"),
NewParam("data", "[]byte"),
},
},
},
}
code, err := GenerateHost(svc, "host")
Expect(err).NotTo(HaveOccurred())
_, err = format.Source(code)
Expect(err).NotTo(HaveOccurred())
codeStr := string(code)
// Should include encoding/binary import for raw methods
Expect(codeStr).To(ContainSubstring(`"encoding/binary"`))
// Should NOT generate a response type for raw methods
Expect(codeStr).NotTo(ContainSubstring("type StreamGetStreamResponse struct"))
// Should generate request type (request is still JSON)
Expect(codeStr).To(ContainSubstring("type StreamGetStreamRequest struct"))
// Should build binary frame [0x00][4-byte CT len][CT][data]
Expect(codeStr).To(ContainSubstring("frame[0] = 0x00"))
Expect(codeStr).To(ContainSubstring("binary.BigEndian.PutUint32"))
// Should have writeRawError helper
Expect(codeStr).To(ContainSubstring("streamWriteRawError"))
// Should use writeRawError instead of writeError for raw methods
Expect(codeStr).To(ContainSubstring("streamWriteRawError(p, stack"))
})
It("should generate both writeError and writeRawError for mixed services", func() {
svc := Service{
Name: "API",
Permission: "api",
Interface: "APIService",
Methods: []Method{
{
Name: "Call",
HasError: true,
Params: []Param{NewParam("uri", "string")},
Returns: []Param{NewParam("response", "string")},
},
{
Name: "CallRaw",
HasError: true,
Raw: true,
Params: []Param{NewParam("uri", "string")},
Returns: []Param{
NewParam("contentType", "string"),
NewParam("data", "[]byte"),
},
},
},
}
code, err := GenerateHost(svc, "host")
Expect(err).NotTo(HaveOccurred())
_, err = format.Source(code)
Expect(err).NotTo(HaveOccurred())
codeStr := string(code)
// Should have both helpers
Expect(codeStr).To(ContainSubstring("apiWriteResponse"))
Expect(codeStr).To(ContainSubstring("apiWriteError"))
Expect(codeStr).To(ContainSubstring("apiWriteRawError"))
// Should generate response type for non-raw method only
Expect(codeStr).To(ContainSubstring("type APICallResponse struct"))
Expect(codeStr).NotTo(ContainSubstring("type APICallRawResponse struct"))
})
It("should always include json import for JSON protocol", func() {
// All services use JSON protocol, so json import is always needed
svc := Service{
@@ -717,49 +627,7 @@ var _ = Describe("Generator", func() {
Expect(codeStr).To(ContainSubstring(`response.get("boolVal", False)`))
})
It("should generate binary frame parsing for raw methods", func() {
svc := Service{
Name: "Stream",
Permission: "stream",
Interface: "StreamService",
Methods: []Method{
{
Name: "GetStream",
HasError: true,
Raw: true,
Params: []Param{NewParam("uri", "string")},
Returns: []Param{
NewParam("contentType", "string"),
NewParam("data", "[]byte"),
},
Doc: "GetStream returns raw binary stream data.",
},
},
}
code, err := GenerateClientPython(svc)
Expect(err).NotTo(HaveOccurred())
codeStr := string(code)
// Should import Tuple and struct for raw methods
Expect(codeStr).To(ContainSubstring("from typing import Any, Tuple"))
Expect(codeStr).To(ContainSubstring("import struct"))
// Should return Tuple[str, bytes]
Expect(codeStr).To(ContainSubstring("-> Tuple[str, bytes]:"))
// Should parse binary frame instead of JSON
Expect(codeStr).To(ContainSubstring("response_bytes = response_mem.bytes()"))
Expect(codeStr).To(ContainSubstring("response_bytes[0] == 0x01"))
Expect(codeStr).To(ContainSubstring("struct.unpack"))
Expect(codeStr).To(ContainSubstring("return content_type, data"))
// Should NOT use json.loads for response
Expect(codeStr).NotTo(ContainSubstring("json.loads(extism.memory.string(response_mem))"))
})
It("should not import Tuple or struct for non-raw services", func() {
It("should not import base64 for non-byte services", func() {
svc := Service{
Name: "Test",
Permission: "test",
@@ -779,8 +647,37 @@ var _ = Describe("Generator", func() {
codeStr := string(code)
Expect(codeStr).NotTo(ContainSubstring("Tuple"))
Expect(codeStr).NotTo(ContainSubstring("import struct"))
Expect(codeStr).NotTo(ContainSubstring("import base64"))
})
It("should generate base64 encoding/decoding for byte fields", func() {
svc := Service{
Name: "Codec",
Permission: "codec",
Interface: "CodecService",
Methods: []Method{
{
Name: "Encode",
HasError: true,
Params: []Param{NewParam("data", "[]byte")},
Returns: []Param{NewParam("result", "[]byte")},
},
},
}
code, err := GenerateClientPython(svc)
Expect(err).NotTo(HaveOccurred())
codeStr := string(code)
// Should import base64
Expect(codeStr).To(ContainSubstring("import base64"))
// Should base64-encode byte params in request
Expect(codeStr).To(ContainSubstring(`base64.b64encode(data).decode("ascii")`))
// Should base64-decode byte returns in response
Expect(codeStr).To(ContainSubstring(`base64.b64decode(response.get("result", ""))`))
})
})
@@ -939,46 +836,6 @@ var _ = Describe("Generator", func() {
Expect(codeStr).To(ContainSubstring("github.com/navidrome/navidrome/plugins/pdk/go/pdk"))
})
It("should include encoding/binary import for raw methods", func() {
svc := Service{
Name: "Stream",
Permission: "stream",
Interface: "StreamService",
Methods: []Method{
{
Name: "GetStream",
HasError: true,
Raw: true,
Params: []Param{NewParam("uri", "string")},
Returns: []Param{
NewParam("contentType", "string"),
NewParam("data", "[]byte"),
},
},
},
}
code, err := GenerateClientGo(svc, "host")
Expect(err).NotTo(HaveOccurred())
codeStr := string(code)
// Should include encoding/binary for raw binary frame parsing
Expect(codeStr).To(ContainSubstring(`"encoding/binary"`))
// Should NOT generate response type struct for raw methods
Expect(codeStr).NotTo(ContainSubstring("streamGetStreamResponse struct"))
// Should still generate request type
Expect(codeStr).To(ContainSubstring("streamGetStreamRequest struct"))
// Should parse binary frame
Expect(codeStr).To(ContainSubstring("responseBytes[0] == 0x01"))
Expect(codeStr).To(ContainSubstring("binary.BigEndian.Uint32"))
// Should return (string, []byte, error)
Expect(codeStr).To(ContainSubstring("func StreamGetStream(uri string) (string, []byte, error)"))
})
})
Describe("GenerateClientGoStub", func() {
@@ -1748,22 +1605,17 @@ var _ = Describe("Rust Generation", func() {
Expect(codeStr).NotTo(ContainSubstring("Option<bool>"))
})
It("should generate raw extern C import and binary frame parsing for raw methods", func() {
It("should generate base64 serde for Vec<u8> fields", func() {
svc := Service{
Name: "Stream",
Permission: "stream",
Interface: "StreamService",
Name: "Codec",
Permission: "codec",
Interface: "CodecService",
Methods: []Method{
{
Name: "GetStream",
Name: "Encode",
HasError: true,
Raw: true,
Params: []Param{NewParam("uri", "string")},
Returns: []Param{
NewParam("contentType", "string"),
NewParam("data", "[]byte"),
},
Doc: "GetStream returns raw binary stream data.",
Params: []Param{NewParam("data", "[]byte")},
Returns: []Param{NewParam("result", "[]byte")},
},
},
}
@@ -1773,24 +1625,36 @@ var _ = Describe("Rust Generation", func() {
codeStr := string(code)
// Should use extern "C" with wasm_import_module for raw methods, not #[host_fn] extern "ExtismHost"
Expect(codeStr).To(ContainSubstring(`#[link(wasm_import_module = "extism:host/user")]`))
Expect(codeStr).To(ContainSubstring(`extern "C"`))
Expect(codeStr).To(ContainSubstring("fn stream_getstream(offset: u64) -> u64"))
// Should generate base64_bytes serde module
Expect(codeStr).To(ContainSubstring("mod base64_bytes"))
Expect(codeStr).To(ContainSubstring("use base64::Engine as _"))
// Should NOT generate response type for raw methods
Expect(codeStr).NotTo(ContainSubstring("StreamGetStreamResponse"))
// Should add serde(with = "base64_bytes") on Vec<u8> fields
Expect(codeStr).To(ContainSubstring(`#[serde(with = "base64_bytes")]`))
})
// Should generate request type (request is still JSON)
Expect(codeStr).To(ContainSubstring("struct StreamGetStreamRequest"))
It("should not generate base64 module when no byte fields", func() {
svc := Service{
Name: "Test",
Permission: "test",
Interface: "TestService",
Methods: []Method{
{
Name: "Call",
HasError: true,
Params: []Param{NewParam("uri", "string")},
Returns: []Param{NewParam("response", "string")},
},
},
}
// Should return Result<(String, Vec<u8>), Error>
Expect(codeStr).To(ContainSubstring("Result<(String, Vec<u8>), Error>"))
code, err := GenerateClientRust(svc)
Expect(err).NotTo(HaveOccurred())
// Should parse binary frame
Expect(codeStr).To(ContainSubstring("response_bytes[0] == 0x01"))
Expect(codeStr).To(ContainSubstring("u32::from_be_bytes"))
Expect(codeStr).To(ContainSubstring("String::from_utf8_lossy"))
codeStr := string(code)
Expect(codeStr).NotTo(ContainSubstring("mod base64_bytes"))
Expect(codeStr).NotTo(ContainSubstring("use base64"))
})
})
})

View File

@@ -761,7 +761,6 @@ func parseMethod(name string, funcType *ast.FuncType, annotation map[string]stri
m := Method{
Name: name,
ExportName: annotation["name"],
Raw: annotation["raw"] == "true",
Doc: doc,
}
@@ -800,13 +799,6 @@ func parseMethod(name string, funcType *ast.FuncType, annotation map[string]stri
}
}
// Validate raw=true methods: must return exactly (string, []byte, error)
if m.Raw {
if !m.HasError || len(m.Returns) != 2 || m.Returns[0].Type != "string" || m.Returns[1].Type != "[]byte" {
return m, fmt.Errorf("raw=true method %s must return (string, []byte, error) — content-type, data, error", name)
}
}
return m, nil
}

View File

@@ -122,119 +122,6 @@ type TestService interface {
Expect(services[0].Methods[0].Name).To(Equal("Exported"))
})
It("should parse raw=true annotation", func() {
src := `package host
import "context"
//nd:hostservice name=Stream permission=stream
type StreamService interface {
//nd:hostfunc raw=true
GetStream(ctx context.Context, uri string) (contentType string, data []byte, err error)
}
`
err := os.WriteFile(filepath.Join(tmpDir, "stream.go"), []byte(src), 0600)
Expect(err).NotTo(HaveOccurred())
services, err := ParseDirectory(tmpDir)
Expect(err).NotTo(HaveOccurred())
Expect(services).To(HaveLen(1))
m := services[0].Methods[0]
Expect(m.Name).To(Equal("GetStream"))
Expect(m.Raw).To(BeTrue())
Expect(m.HasError).To(BeTrue())
Expect(m.Returns).To(HaveLen(2))
Expect(m.Returns[0].Name).To(Equal("contentType"))
Expect(m.Returns[0].Type).To(Equal("string"))
Expect(m.Returns[1].Name).To(Equal("data"))
Expect(m.Returns[1].Type).To(Equal("[]byte"))
})
It("should set Raw=false when raw annotation is absent", func() {
src := `package host
import "context"
//nd:hostservice name=Test permission=test
type TestService interface {
//nd:hostfunc
Call(ctx context.Context, uri string) (response string, err error)
}
`
err := os.WriteFile(filepath.Join(tmpDir, "test.go"), []byte(src), 0600)
Expect(err).NotTo(HaveOccurred())
services, err := ParseDirectory(tmpDir)
Expect(err).NotTo(HaveOccurred())
Expect(services[0].Methods[0].Raw).To(BeFalse())
})
It("should reject raw=true with invalid return signature", func() {
src := `package host
import "context"
//nd:hostservice name=Test permission=test
type TestService interface {
//nd:hostfunc raw=true
BadRaw(ctx context.Context, uri string) (result string, err error)
}
`
err := os.WriteFile(filepath.Join(tmpDir, "test.go"), []byte(src), 0600)
Expect(err).NotTo(HaveOccurred())
_, err = ParseDirectory(tmpDir)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("raw=true"))
Expect(err.Error()).To(ContainSubstring("must return (string, []byte, error)"))
})
It("should reject raw=true without error return", func() {
src := `package host
import "context"
//nd:hostservice name=Test permission=test
type TestService interface {
//nd:hostfunc raw=true
BadRaw(ctx context.Context, uri string) (contentType string, data []byte)
}
`
err := os.WriteFile(filepath.Join(tmpDir, "test.go"), []byte(src), 0600)
Expect(err).NotTo(HaveOccurred())
_, err = ParseDirectory(tmpDir)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("raw=true"))
})
It("should parse mixed raw and non-raw methods", func() {
src := `package host
import "context"
//nd:hostservice name=API permission=api
type APIService interface {
//nd:hostfunc
Call(ctx context.Context, uri string) (responseJSON string, err error)
//nd:hostfunc raw=true
CallRaw(ctx context.Context, uri string) (contentType string, data []byte, err error)
}
`
err := os.WriteFile(filepath.Join(tmpDir, "api.go"), []byte(src), 0600)
Expect(err).NotTo(HaveOccurred())
services, err := ParseDirectory(tmpDir)
Expect(err).NotTo(HaveOccurred())
Expect(services).To(HaveLen(1))
Expect(services[0].Methods).To(HaveLen(2))
Expect(services[0].Methods[0].Raw).To(BeFalse())
Expect(services[0].Methods[1].Raw).To(BeTrue())
Expect(services[0].HasRawMethods()).To(BeTrue())
})
It("should handle custom export name", func() {
src := `package host

View File

@@ -0,0 +1,25 @@
{{define "base64_bytes_module"}}
use base64::Engine as _;
use base64::engine::general_purpose::STANDARD as BASE64;
mod base64_bytes {
use serde::{self, Deserialize, Deserializer, Serializer};
use base64::Engine as _;
use base64::engine::general_purpose::STANDARD as BASE64;
pub fn serialize<S>(bytes: &Vec<u8>, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
serializer.serialize_str(&BASE64.encode(bytes))
}
pub fn deserialize<'de, D>(deserializer: D) -> Result<Vec<u8>, D::Error>
where
D: Deserializer<'de>,
{
let s = String::deserialize(deserializer)?;
BASE64.decode(&s).map_err(serde::de::Error::custom)
}
}
{{- end}}

View File

@@ -7,6 +7,7 @@ use serde::{Deserialize, Serialize};
{{- if hasHashMap .Capability}}
use std::collections::HashMap;
{{- end}}
{{- if .Capability.HasByteFields}}{{template "base64_bytes_module" .}}{{- end}}
// Helper functions for skip_serializing_if with numeric types
#[allow(dead_code)]
@@ -70,6 +71,9 @@ pub struct {{.Name}} {
#[serde(default, skip_serializing_if = "{{skipSerializingFunc .Type}}")]
{{- else}}
#[serde(default)]
{{- end}}
{{- if .IsByteSlice}}
#[serde(with = "base64_bytes")]
{{- end}}
pub {{rustFieldName .Name}}: {{fieldRustType .}},
{{- end}}

View File

@@ -8,9 +8,6 @@
package {{.Package}}
import (
{{- if .Service.HasRawMethods}}
"encoding/binary"
{{- end}}
"encoding/json"
{{- if .Service.HasErrors}}
"errors"
@@ -52,7 +49,7 @@ type {{requestType .}} struct {
{{- end}}
}
{{- end}}
{{- if and (not .IsErrorOnly) (not .Raw)}}
{{- if not .IsErrorOnly}}
type {{responseType .}} struct {
{{- range .Returns}}
@@ -98,27 +95,7 @@ func {{$.Service.Name}}{{.Name}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{
// Read the response from memory
responseMem := pdk.FindMemory(responsePtr)
responseBytes := responseMem.ReadBytes()
{{- if .Raw}}
// Parse binary-framed response
if len(responseBytes) == 0 {
return "", nil, errors.New("empty response from host")
}
if responseBytes[0] == 0x01 { // error
return "", nil, errors.New(string(responseBytes[1:]))
}
if responseBytes[0] != 0x00 {
return "", nil, errors.New("unknown response status")
}
if len(responseBytes) < 5 {
return "", nil, errors.New("malformed raw response: incomplete header")
}
ctLen := binary.BigEndian.Uint32(responseBytes[1:5])
if uint32(len(responseBytes)) < 5+ctLen {
return "", nil, errors.New("malformed raw response: content-type overflow")
}
return string(responseBytes[5 : 5+ctLen]), responseBytes[5+ctLen:], nil
{{- else if .IsErrorOnly}}
{{- if .IsErrorOnly}}
// Parse error-only response
var response struct {

View File

@@ -8,12 +8,12 @@
# main __init__.py file. Copy the needed functions from this file into your plugin.
from dataclasses import dataclass
from typing import Any{{- if .Service.HasRawMethods}}, Tuple{{end}}
from typing import Any
import extism
import json
{{- if .Service.HasRawMethods}}
import struct
{{- if .Service.HasByteFields}}
import base64
{{- end}}
@@ -32,7 +32,7 @@ def _{{exportName .}}(offset: int) -> int:
{{- end}}
{{- /* Generate dataclasses for multi-value returns */ -}}
{{range .Service.Methods}}
{{- if and .NeedsResultClass (not .Raw)}}
{{- if .NeedsResultClass}}
@dataclass
@@ -47,7 +47,7 @@ class {{pythonResultType .}}:
{{range .Service.Methods}}
def {{pythonFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.PythonName}}: {{$p.PythonType}}{{end}}){{if .Raw}} -> Tuple[str, bytes]{{else if .NeedsResultClass}} -> {{pythonResultType .}}{{else if .HasReturns}} -> {{(index .Returns 0).PythonType}}{{else}} -> None{{end}}:
def {{pythonFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.PythonName}}: {{$p.PythonType}}{{end}}){{if .NeedsResultClass}} -> {{pythonResultType .}}{{else if .HasReturns}} -> {{(index .Returns 0).PythonType}}{{else}} -> None{{end}}:
"""{{if .Doc}}{{.Doc}}{{else}}Call the {{exportName .}} host function.{{end}}
{{- if .HasParams}}
@@ -56,11 +56,7 @@ def {{pythonFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.PythonNam
{{.PythonName}}: {{.PythonType}} parameter.
{{- end}}
{{- end}}
{{- if .Raw}}
Returns:
Tuple of (content_type, data) with the raw binary response.
{{- else if .HasReturns}}
{{- if .HasReturns}}
Returns:
{{- if .NeedsResultClass}}
@@ -76,7 +72,11 @@ def {{pythonFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.PythonNam
{{- if .HasParams}}
request = {
{{- range .Params}}
{{- if .IsByteSlice}}
"{{.JSONName}}": base64.b64encode({{.PythonName}}).decode("ascii"),
{{- else}}
"{{.JSONName}}": {{.PythonName}},
{{- end}}
{{- end}}
}
request_bytes = json.dumps(request).encode("utf-8")
@@ -86,24 +86,6 @@ def {{pythonFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.PythonNam
request_mem = extism.memory.alloc(request_bytes)
response_offset = _{{exportName .}}(request_mem.offset)
response_mem = extism.memory.find(response_offset)
{{- if .Raw}}
response_bytes = response_mem.bytes()
if len(response_bytes) == 0:
raise HostFunctionError("empty response from host")
if response_bytes[0] == 0x01:
raise HostFunctionError(response_bytes[1:].decode("utf-8"))
if response_bytes[0] != 0x00:
raise HostFunctionError("unknown response status")
if len(response_bytes) < 5:
raise HostFunctionError("malformed raw response: incomplete header")
ct_len = struct.unpack(">I", response_bytes[1:5])[0]
if len(response_bytes) < 5 + ct_len:
raise HostFunctionError("malformed raw response: content-type overflow")
content_type = response_bytes[5:5 + ct_len].decode("utf-8")
data = response_bytes[5 + ct_len:]
return content_type, data
{{- else}}
response = json.loads(extism.memory.string(response_mem))
{{if .HasError}}
if response.get("error"):
@@ -112,10 +94,17 @@ def {{pythonFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.PythonNam
{{- if .NeedsResultClass}}
return {{pythonResultType .}}(
{{- range .Returns}}
{{- if .IsByteSlice}}
{{.PythonName}}=base64.b64decode(response.get("{{.JSONName}}", "")),
{{- else}}
{{.PythonName}}=response.get("{{.JSONName}}"{{pythonDefault .}}),
{{- end}}
{{- end}}
)
{{- else if .HasReturns}}
{{- if (index .Returns 0).IsByteSlice}}
return base64.b64decode(response.get("{{(index .Returns 0).JSONName}}", ""))
{{- else}}
return response.get("{{(index .Returns 0).JSONName}}"{{pythonDefault (index .Returns 0)}})
{{- end}}
{{- end}}

View File

@@ -5,6 +5,7 @@
use extism_pdk::*;
use serde::{Deserialize, Serialize};
{{- if .Service.HasByteFields}}{{template "base64_bytes_module" .}}{{- end}}
{{- /* Generate struct definitions */ -}}
{{- range .Service.Structs}}
{{if .Doc}}
@@ -16,6 +17,9 @@ pub struct {{.Name}} {
{{- range .Fields}}
{{- if .NeedsDefault}}
#[serde(default)]
{{- end}}
{{- if .IsByteSlice}}
#[serde(with = "base64_bytes")]
{{- end}}
pub {{.RustName}}: {{fieldRustType .}},
{{- end}}
@@ -29,17 +33,22 @@ pub struct {{.Name}} {
#[serde(rename_all = "camelCase")]
struct {{requestType .}} {
{{- range .Params}}
{{- if .IsByteSlice}}
#[serde(with = "base64_bytes")]
{{- end}}
{{.RustName}}: {{rustType .}},
{{- end}}
}
{{- end}}
{{- if not .Raw}}
#[derive(Debug, Clone, Deserialize)]
#[serde(rename_all = "camelCase")]
struct {{responseType .}} {
{{- range .Returns}}
#[serde(default)]
{{- if .IsByteSlice}}
#[serde(with = "base64_bytes")]
{{- end}}
{{.RustName}}: {{rustType .}},
{{- end}}
{{- if .HasError}}
@@ -48,92 +57,16 @@ struct {{responseType .}} {
{{- end}}
}
{{- end}}
{{- end}}
#[host_fn]
extern "ExtismHost" {
{{- range .Service.Methods}}
{{- if not .Raw}}
fn {{exportName .}}(input: Json<{{if .HasParams}}{{requestType .}}{{else}}serde_json::Value{{end}}>) -> Json<{{responseType .}}>;
{{- end}}
{{- end}}
}
{{- /* Declare raw extern "C" imports for raw methods */ -}}
{{- range .Service.Methods}}
{{- if .Raw}}
#[link(wasm_import_module = "extism:host/user")]
extern "C" {
fn {{exportName .}}(offset: u64) -> u64;
}
{{- end}}
{{- end}}
{{- /* Generate wrapper functions */ -}}
{{range .Service.Methods}}
{{- if .Raw}}
{{if .Doc}}{{rustDocComment .Doc}}{{else}}/// Calls the {{exportName .}} host function.{{end}}
{{- if .HasParams}}
///
/// # Arguments
{{- range .Params}}
/// * `{{.RustName}}` - {{rustType .}} parameter.
{{- end}}
{{- end}}
///
/// # Returns
/// A tuple of (content_type, data) with the raw binary response.
///
/// # Errors
/// Returns an error if the host function call fails.
pub fn {{rustFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.RustName}}: {{rustParamType $p}}{{end}}) -> Result<(String, Vec<u8>), Error> {
{{- if .HasParams}}
let req = {{requestType .}} {
{{- range .Params}}
{{.RustName}}: {{.RustName}}{{if .NeedsToOwned}}.to_owned(){{end}},
{{- end}}
};
let input_bytes = serde_json::to_vec(&req).map_err(|e| Error::msg(e.to_string()))?;
{{- else}}
let input_bytes = b"{}".to_vec();
{{- end}}
let input_mem = Memory::from_bytes(&input_bytes).map_err(|e| Error::msg(e.to_string()))?;
let response_offset = unsafe { {{exportName .}}(input_mem.offset()) };
let response_mem = Memory::find(response_offset)
.ok_or_else(|| Error::msg("empty response from host"))?;
let response_bytes = response_mem.to_vec();
if response_bytes.is_empty() {
return Err(Error::msg("empty response from host"));
}
if response_bytes[0] == 0x01 {
let msg = String::from_utf8_lossy(&response_bytes[1..]).to_string();
return Err(Error::msg(msg));
}
if response_bytes[0] != 0x00 {
return Err(Error::msg("unknown response status"));
}
if response_bytes.len() < 5 {
return Err(Error::msg("malformed raw response: incomplete header"));
}
let ct_len = u32::from_be_bytes([
response_bytes[1],
response_bytes[2],
response_bytes[3],
response_bytes[4],
]) as usize;
if ct_len > response_bytes.len() - 5 {
return Err(Error::msg("malformed raw response: content-type overflow"));
}
let ct_end = 5 + ct_len;
let content_type = String::from_utf8_lossy(&response_bytes[5..ct_end]).to_string();
let data = response_bytes[ct_end..].to_vec();
Ok((content_type, data))
}
{{- else}}
{{if .Doc}}{{rustDocComment .Doc}}{{else}}/// Calls the {{exportName .}} host function.{{end}}
{{- if .HasParams}}
@@ -209,4 +142,3 @@ pub fn {{rustFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.RustName
}
{{- end}}
{{- end}}
{{- end}}

View File

@@ -4,9 +4,6 @@ package {{.Package}}
import (
"context"
{{- if .Service.HasRawMethods}}
"encoding/binary"
{{- end}}
"encoding/json"
extism "github.com/extism/go-sdk"
@@ -23,7 +20,6 @@ type {{requestType .}} struct {
{{- end}}
}
{{- end}}
{{- if not .Raw}}
// {{responseType .}} is the response type for {{$.Service.Name}}.{{.Name}}.
type {{responseType .}} struct {
@@ -34,7 +30,6 @@ type {{responseType .}} struct {
Error string `json:"error,omitempty"`
{{- end}}
}
{{- end}}
{{end}}
// Register{{.Service.Name}}HostFunctions registers {{.Service.Name}} service host functions.
@@ -56,48 +51,18 @@ func new{{$.Service.Name}}{{.Name}}HostFunction(service {{$.Service.Interface}})
// Read JSON request from plugin memory
reqBytes, err := p.ReadBytes(stack[0])
if err != nil {
{{- if .Raw}}
{{$.Service.Name | lower}}WriteRawError(p, stack, err)
{{- else}}
{{$.Service.Name | lower}}WriteError(p, stack, err)
{{- end}}
return
}
var req {{requestType .}}
if err := json.Unmarshal(reqBytes, &req); err != nil {
{{- if .Raw}}
{{$.Service.Name | lower}}WriteRawError(p, stack, err)
{{- else}}
{{$.Service.Name | lower}}WriteError(p, stack, err)
{{- end}}
return
}
{{- end}}
// Call the service method
{{- if .Raw}}
{{range $i, $r := .Returns}}{{if $i}}, {{end}}{{lower $r.Name}}{{end}}, svcErr := service.{{.Name}}(ctx{{range .Params}}, req.{{title .Name}}{{end}})
if svcErr != nil {
{{$.Service.Name | lower}}WriteRawError(p, stack, svcErr)
return
}
// Write binary-framed response to plugin memory:
// [0x00][4-byte content-type length (big-endian)][content-type string][raw data]
ctBytes := []byte({{lower (index .Returns 0).Name}})
frame := make([]byte, 1+4+len(ctBytes)+len({{lower (index .Returns 1).Name}}))
frame[0] = 0x00 // success
binary.BigEndian.PutUint32(frame[1:5], uint32(len(ctBytes)))
copy(frame[5:5+len(ctBytes)], ctBytes)
copy(frame[5+len(ctBytes):], {{lower (index .Returns 1).Name}})
respPtr, err := p.WriteBytes(frame)
if err != nil {
stack[0] = 0
return
}
stack[0] = respPtr
{{- else if .HasReturns}}
{{- if .HasReturns}}
{{- if .HasError}}
{{range $i, $r := .Returns}}{{if $i}}, {{end}}{{lower $r.Name}}{{end}}, svcErr := service.{{.Name}}(ctx{{range .Params}}, req.{{title .Name}}{{end}})
if svcErr != nil {
@@ -162,16 +127,3 @@ func {{.Service.Name | lower}}WriteError(p *extism.CurrentPlugin, stack []uint64
respPtr, _ := p.WriteBytes(respBytes)
stack[0] = respPtr
}
{{- if .Service.HasRawMethods}}
// {{.Service.Name | lower}}WriteRawError writes a binary-framed error response to plugin memory.
// Format: [0x01][UTF-8 error message]
func {{.Service.Name | lower}}WriteRawError(p *extism.CurrentPlugin, stack []uint64, err error) {
errMsg := []byte(err.Error())
frame := make([]byte, 1+len(errMsg))
frame[0] = 0x01 // error
copy(frame[1:], errMsg)
respPtr, _ := p.WriteBytes(frame)
stack[0] = respPtr
}
{{- end}}

View File

@@ -173,16 +173,6 @@ func (s Service) HasErrors() bool {
return false
}
// HasRawMethods returns true if any method in the service uses raw binary framing.
func (s Service) HasRawMethods() bool {
for _, m := range s.Methods {
if m.Raw {
return true
}
}
return false
}
// Method represents a host function method within a service.
type Method struct {
Name string // Go method name (e.g., "Call")
@@ -191,7 +181,6 @@ type Method struct {
Returns []Param // Return values (excluding error)
HasError bool // Whether the method returns an error
Doc string // Documentation comment for the method
Raw bool // If true, response uses binary framing instead of JSON
}
// FunctionName returns the Extism host function export name.
@@ -343,6 +332,52 @@ type Param struct {
JSONName string // JSON field name (camelCase)
}
// IsByteSlice returns true if the parameter type is []byte.
func (p Param) IsByteSlice() bool {
return p.Type == "[]byte"
}
// IsByteSlice returns true if the field type is []byte.
func (f FieldDef) IsByteSlice() bool {
return f.Type == "[]byte"
}
// HasByteFields returns true if any method params, returns, or struct fields use []byte.
func (s Service) HasByteFields() bool {
for _, m := range s.Methods {
for _, p := range m.Params {
if p.IsByteSlice() {
return true
}
}
for _, r := range m.Returns {
if r.IsByteSlice() {
return true
}
}
}
for _, st := range s.Structs {
for _, f := range st.Fields {
if f.IsByteSlice() {
return true
}
}
}
return false
}
// HasByteFields returns true if any capability struct fields use []byte.
func (c Capability) HasByteFields() bool {
for _, st := range c.Structs {
for _, f := range st.Fields {
if f.IsByteSlice() {
return true
}
}
}
return false
}
// NewParam creates a Param with auto-generated JSON name.
func NewParam(name, typ string) Param {
return Param{

View File

@@ -12,6 +12,7 @@ from typing import Any
import extism
import json
import base64
class HostFunctionError(Exception):
@@ -38,7 +39,7 @@ def codec_encode(data: bytes) -> bytes:
HostFunctionError: If the host function returns an error.
"""
request = {
"data": data,
"data": base64.b64encode(data).decode("ascii"),
}
request_bytes = json.dumps(request).encode("utf-8")
request_mem = extism.memory.alloc(request_bytes)
@@ -49,4 +50,4 @@ def codec_encode(data: bytes) -> bytes:
if response.get("error"):
raise HostFunctionError(response["error"])
return response.get("result", b"")
return base64.b64decode(response.get("result", ""))

View File

@@ -5,10 +5,34 @@
use extism_pdk::*;
use serde::{Deserialize, Serialize};
use base64::Engine as _;
use base64::engine::general_purpose::STANDARD as BASE64;
mod base64_bytes {
use serde::{self, Deserialize, Deserializer, Serializer};
use base64::Engine as _;
use base64::engine::general_purpose::STANDARD as BASE64;
pub fn serialize<S>(bytes: &Vec<u8>, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
serializer.serialize_str(&BASE64.encode(bytes))
}
pub fn deserialize<'de, D>(deserializer: D) -> Result<Vec<u8>, D::Error>
where
D: Deserializer<'de>,
{
let s = String::deserialize(deserializer)?;
BASE64.decode(&s).map_err(serde::de::Error::custom)
}
}
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
struct CodecEncodeRequest {
#[serde(with = "base64_bytes")]
data: Vec<u8>,
}
@@ -16,6 +40,7 @@ struct CodecEncodeRequest {
#[serde(rename_all = "camelCase")]
struct CodecEncodeResponse {
#[serde(default)]
#[serde(with = "base64_bytes")]
result: Vec<u8>,
#[serde(default)]
error: Option<String>,

View File

@@ -12,6 +12,7 @@ from typing import Any
import extism
import json
import base64
class HostFunctionError(Exception):
@@ -327,7 +328,7 @@ def comprehensive_byte_slice(data: bytes) -> bytes:
HostFunctionError: If the host function returns an error.
"""
request = {
"data": data,
"data": base64.b64encode(data).decode("ascii"),
}
request_bytes = json.dumps(request).encode("utf-8")
request_mem = extism.memory.alloc(request_bytes)
@@ -338,4 +339,4 @@ def comprehensive_byte_slice(data: bytes) -> bytes:
if response.get("error"):
raise HostFunctionError(response["error"])
return response.get("result", b"")
return base64.b64decode(response.get("result", ""))

View File

@@ -5,6 +5,29 @@
use extism_pdk::*;
use serde::{Deserialize, Serialize};
use base64::Engine as _;
use base64::engine::general_purpose::STANDARD as BASE64;
mod base64_bytes {
use serde::{self, Deserialize, Deserializer, Serializer};
use base64::Engine as _;
use base64::engine::general_purpose::STANDARD as BASE64;
pub fn serialize<S>(bytes: &Vec<u8>, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
serializer.serialize_str(&BASE64.encode(bytes))
}
pub fn deserialize<'de, D>(deserializer: D) -> Result<Vec<u8>, D::Error>
where
D: Deserializer<'de>,
{
let s = String::deserialize(deserializer)?;
BASE64.decode(&s).map_err(serde::de::Error::custom)
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
@@ -144,6 +167,7 @@ struct ComprehensiveMultipleReturnsResponse {
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
struct ComprehensiveByteSliceRequest {
#[serde(with = "base64_bytes")]
data: Vec<u8>,
}
@@ -151,6 +175,7 @@ struct ComprehensiveByteSliceRequest {
#[serde(rename_all = "camelCase")]
struct ComprehensiveByteSliceResponse {
#[serde(default)]
#[serde(with = "base64_bytes")]
result: Vec<u8>,
#[serde(default)]
error: Option<String>,

View File

@@ -1,66 +0,0 @@
// Code generated by ndpgen. DO NOT EDIT.
//
// This file contains client wrappers for the Stream host service.
// It is intended for use in Navidrome plugins built with TinyGo.
//
//go:build wasip1
package ndpdk
import (
"encoding/binary"
"encoding/json"
"errors"
"github.com/navidrome/navidrome/plugins/pdk/go/pdk"
)
// stream_getstream is the host function provided by Navidrome.
//
//go:wasmimport extism:host/user stream_getstream
func stream_getstream(uint64) uint64
type streamGetStreamRequest struct {
Uri string `json:"uri"`
}
// StreamGetStream calls the stream_getstream host function.
// GetStream returns raw binary stream data with content type.
func StreamGetStream(uri string) (string, []byte, error) {
// Marshal request to JSON
req := streamGetStreamRequest{
Uri: uri,
}
reqBytes, err := json.Marshal(req)
if err != nil {
return "", nil, err
}
reqMem := pdk.AllocateBytes(reqBytes)
defer reqMem.Free()
// Call the host function
responsePtr := stream_getstream(reqMem.Offset())
// Read the response from memory
responseMem := pdk.FindMemory(responsePtr)
responseBytes := responseMem.ReadBytes()
// Parse binary-framed response
if len(responseBytes) == 0 {
return "", nil, errors.New("empty response from host")
}
if responseBytes[0] == 0x01 { // error
return "", nil, errors.New(string(responseBytes[1:]))
}
if responseBytes[0] != 0x00 {
return "", nil, errors.New("unknown response status")
}
if len(responseBytes) < 5 {
return "", nil, errors.New("malformed raw response: incomplete header")
}
ctLen := binary.BigEndian.Uint32(responseBytes[1:5])
if uint32(len(responseBytes)) < 5+ctLen {
return "", nil, errors.New("malformed raw response: content-type overflow")
}
return string(responseBytes[5 : 5+ctLen]), responseBytes[5+ctLen:], nil
}

View File

@@ -1,63 +0,0 @@
# Code generated by ndpgen. DO NOT EDIT.
#
# This file contains client wrappers for the Stream host service.
# It is intended for use in Navidrome plugins built with extism-py.
#
# IMPORTANT: Due to a limitation in extism-py, you cannot import this file directly.
# The @extism.import_fn decorators are only detected when defined in the plugin's
# main __init__.py file. Copy the needed functions from this file into your plugin.
from dataclasses import dataclass
from typing import Any, Tuple
import extism
import json
import struct
class HostFunctionError(Exception):
"""Raised when a host function returns an error."""
pass
@extism.import_fn("extism:host/user", "stream_getstream")
def _stream_getstream(offset: int) -> int:
"""Raw host function - do not call directly."""
...
def stream_get_stream(uri: str) -> Tuple[str, bytes]:
"""GetStream returns raw binary stream data with content type.
Args:
uri: str parameter.
Returns:
Tuple of (content_type, data) with the raw binary response.
Raises:
HostFunctionError: If the host function returns an error.
"""
request = {
"uri": uri,
}
request_bytes = json.dumps(request).encode("utf-8")
request_mem = extism.memory.alloc(request_bytes)
response_offset = _stream_getstream(request_mem.offset)
response_mem = extism.memory.find(response_offset)
response_bytes = response_mem.bytes()
if len(response_bytes) == 0:
raise HostFunctionError("empty response from host")
if response_bytes[0] == 0x01:
raise HostFunctionError(response_bytes[1:].decode("utf-8"))
if response_bytes[0] != 0x00:
raise HostFunctionError("unknown response status")
if len(response_bytes) < 5:
raise HostFunctionError("malformed raw response: incomplete header")
ct_len = struct.unpack(">I", response_bytes[1:5])[0]
if len(response_bytes) < 5 + ct_len:
raise HostFunctionError("malformed raw response: content-type overflow")
content_type = response_bytes[5:5 + ct_len].decode("utf-8")
data = response_bytes[5 + ct_len:]
return content_type, data

View File

@@ -1,73 +0,0 @@
// Code generated by ndpgen. DO NOT EDIT.
//
// This file contains client wrappers for the Stream host service.
// It is intended for use in Navidrome plugins built with extism-pdk.
use extism_pdk::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
struct StreamGetStreamRequest {
uri: String,
}
#[host_fn]
extern "ExtismHost" {
}
#[link(wasm_import_module = "extism:host/user")]
extern "C" {
fn stream_getstream(offset: u64) -> u64;
}
/// GetStream returns raw binary stream data with content type.
///
/// # Arguments
/// * `uri` - String parameter.
///
/// # Returns
/// A tuple of (content_type, data) with the raw binary response.
///
/// # Errors
/// Returns an error if the host function call fails.
pub fn get_stream(uri: &str) -> Result<(String, Vec<u8>), Error> {
let req = StreamGetStreamRequest {
uri: uri.to_owned(),
};
let input_bytes = serde_json::to_vec(&req).map_err(|e| Error::msg(e.to_string()))?;
let input_mem = Memory::from_bytes(&input_bytes).map_err(|e| Error::msg(e.to_string()))?;
let response_offset = unsafe { stream_getstream(input_mem.offset()) };
let response_mem = Memory::find(response_offset)
.ok_or_else(|| Error::msg("empty response from host"))?;
let response_bytes = response_mem.to_vec();
if response_bytes.is_empty() {
return Err(Error::msg("empty response from host"));
}
if response_bytes[0] == 0x01 {
let msg = String::from_utf8_lossy(&response_bytes[1..]).to_string();
return Err(Error::msg(msg));
}
if response_bytes[0] != 0x00 {
return Err(Error::msg("unknown response status"));
}
if response_bytes.len() < 5 {
return Err(Error::msg("malformed raw response: incomplete header"));
}
let ct_len = u32::from_be_bytes([
response_bytes[1],
response_bytes[2],
response_bytes[3],
response_bytes[4],
]) as usize;
if ct_len > response_bytes.len() - 5 {
return Err(Error::msg("malformed raw response: content-type overflow"));
}
let ct_end = 5 + ct_len;
let content_type = String::from_utf8_lossy(&response_bytes[5..ct_end]).to_string();
let data = response_bytes[ct_end..].to_vec();
Ok((content_type, data))
}

View File

@@ -1,10 +0,0 @@
package testpkg
import "context"
//nd:hostservice name=Stream permission=stream
type StreamService interface {
// GetStream returns raw binary stream data with content type.
//nd:hostfunc raw=true
GetStream(ctx context.Context, uri string) (contentType string, data []byte, err error)
}

View File

@@ -14,6 +14,7 @@ import (
"net/url"
"strings"
"github.com/navidrome/navidrome/plugins/pdk/go/host"
"github.com/navidrome/navidrome/plugins/pdk/go/metadata"
"github.com/navidrome/navidrome/plugins/pdk/go/pdk"
)
@@ -77,21 +78,28 @@ func sparqlQuery(endpoint, query string) (*SPARQLResult, error) {
form := url.Values{}
form.Set("query", query)
req := pdk.NewHTTPRequest(pdk.MethodPost, endpoint)
req.SetHeader("Accept", "application/sparql-results+json")
req.SetHeader("Content-Type", "application/x-www-form-urlencoded")
req.SetHeader("User-Agent", "NavidromeWikimediaPlugin/1.0")
req.SetBody([]byte(form.Encode()))
pdk.Log(pdk.LogDebug, fmt.Sprintf("SPARQL query to %s: %s", endpoint, query))
resp := req.Send()
if resp.Status() != 200 {
return nil, fmt.Errorf("SPARQL HTTP error: status %d", resp.Status())
resp, err := host.HTTPSend(host.HTTPRequest{
Method: "POST",
URL: endpoint,
Headers: map[string]string{
"Accept": "application/sparql-results+json",
"Content-Type": "application/x-www-form-urlencoded",
"User-Agent": "NavidromeWikimediaPlugin/1.0",
},
Body: []byte(form.Encode()),
TimeoutMs: 10000,
})
if err != nil {
return nil, fmt.Errorf("SPARQL HTTP error: %w", err)
}
if resp.StatusCode != 200 {
return nil, fmt.Errorf("SPARQL HTTP error: status %d", resp.StatusCode)
}
var result SPARQLResult
if err := json.Unmarshal(resp.Body(), &result); err != nil {
if err := json.Unmarshal(resp.Body, &result); err != nil {
return nil, fmt.Errorf("failed to parse SPARQL response: %w", err)
}
if len(result.Results.Bindings) == 0 {
@@ -104,15 +112,22 @@ func sparqlQuery(endpoint, query string) (*SPARQLResult, error) {
func mediawikiQuery(params url.Values) ([]byte, error) {
apiURL := fmt.Sprintf("%s?%s", mediawikiAPIEndpoint, params.Encode())
req := pdk.NewHTTPRequest(pdk.MethodGet, apiURL)
req.SetHeader("Accept", "application/json")
req.SetHeader("User-Agent", "NavidromeWikimediaPlugin/1.0")
resp := req.Send()
if resp.Status() != 200 {
return nil, fmt.Errorf("MediaWiki HTTP error: status %d", resp.Status())
resp, err := host.HTTPSend(host.HTTPRequest{
Method: "GET",
URL: apiURL,
Headers: map[string]string{
"Accept": "application/json",
"User-Agent": "NavidromeWikimediaPlugin/1.0",
},
TimeoutMs: 10000,
})
if err != nil {
return nil, fmt.Errorf("MediaWiki HTTP error: %w", err)
}
return resp.Body(), nil
if resp.StatusCode != 200 {
return nil, fmt.Errorf("MediaWiki HTTP error: status %d", resp.StatusCode)
}
return resp.Body, nil
}
// getWikidataWikipediaURL fetches the Wikipedia URL from Wikidata using MBID or name

40
plugins/host/http.go Normal file
View File

@@ -0,0 +1,40 @@
package host
import "context"
// HTTPRequest represents an outbound HTTP request from a plugin.
type HTTPRequest struct {
Method string `json:"method"`
URL string `json:"url"`
Headers map[string]string `json:"headers,omitempty"`
Body []byte `json:"body,omitempty"`
TimeoutMs int32 `json:"timeoutMs,omitempty"`
}
// HTTPResponse represents the response from an outbound HTTP request.
type HTTPResponse struct {
StatusCode int32 `json:"statusCode"`
Headers map[string]string `json:"headers,omitempty"`
Body []byte `json:"body,omitempty"`
}
// HTTPService provides outbound HTTP request capabilities for plugins.
//
// This service allows plugins to make HTTP requests to external services.
// Requests are validated against the plugin's declared requiredHosts patterns
// from the http permission in the manifest. Redirects are followed but each
// redirect destination is also validated against the allowed hosts.
//
//nd:hostservice name=HTTP permission=http
type HTTPService interface {
// Send executes an HTTP request and returns the response.
//
// Parameters:
// - request: The HTTP request to execute, including method, URL, headers, body, and timeout
//
// Returns the HTTP response with status code, headers, and body.
// Network errors, timeouts, and permission failures are returned as Go errors.
// Successful HTTP calls (including 4xx/5xx status codes) return a non-nil response with nil error.
//nd:hostfunc
Send(ctx context.Context, request HTTPRequest) (*HTTPResponse, error)
}

88
plugins/host/http_gen.go Normal file
View File

@@ -0,0 +1,88 @@
// Code generated by ndpgen. DO NOT EDIT.
package host
import (
"context"
"encoding/json"
extism "github.com/extism/go-sdk"
)
// HTTPSendRequest is the request type for HTTP.Send.
type HTTPSendRequest struct {
Request HTTPRequest `json:"request"`
}
// HTTPSendResponse is the response type for HTTP.Send.
type HTTPSendResponse struct {
Result *HTTPResponse `json:"result,omitempty"`
Error string `json:"error,omitempty"`
}
// RegisterHTTPHostFunctions registers HTTP service host functions.
// The returned host functions should be added to the plugin's configuration.
func RegisterHTTPHostFunctions(service HTTPService) []extism.HostFunction {
return []extism.HostFunction{
newHTTPSendHostFunction(service),
}
}
func newHTTPSendHostFunction(service HTTPService) extism.HostFunction {
return extism.NewHostFunctionWithStack(
"http_send",
func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) {
// Read JSON request from plugin memory
reqBytes, err := p.ReadBytes(stack[0])
if err != nil {
httpWriteError(p, stack, err)
return
}
var req HTTPSendRequest
if err := json.Unmarshal(reqBytes, &req); err != nil {
httpWriteError(p, stack, err)
return
}
// Call the service method
result, svcErr := service.Send(ctx, req.Request)
if svcErr != nil {
httpWriteError(p, stack, svcErr)
return
}
// Write JSON response to plugin memory
resp := HTTPSendResponse{
Result: result,
}
httpWriteResponse(p, stack, resp)
},
[]extism.ValueType{extism.ValueTypePTR},
[]extism.ValueType{extism.ValueTypePTR},
)
}
// httpWriteResponse writes a JSON response to plugin memory.
func httpWriteResponse(p *extism.CurrentPlugin, stack []uint64, resp any) {
respBytes, err := json.Marshal(resp)
if err != nil {
httpWriteError(p, stack, err)
return
}
respPtr, err := p.WriteBytes(respBytes)
if err != nil {
stack[0] = 0
return
}
stack[0] = respPtr
}
// httpWriteError writes an error response to plugin memory.
func httpWriteError(p *extism.CurrentPlugin, stack []uint64, err error) {
errResp := struct {
Error string `json:"error"`
}{Error: err.Error()}
respBytes, _ := json.Marshal(errResp)
respPtr, _ := p.WriteBytes(respBytes)
stack[0] = respPtr
}

View File

@@ -17,8 +17,8 @@ type SubsonicAPIService interface {
Call(ctx context.Context, uri string) (responseJSON string, err error)
// CallRaw executes a Subsonic API request and returns the raw binary response.
// Optimized for binary endpoints like getCoverArt and stream that return
// non-JSON data. The response is returned as raw bytes without JSON encoding overhead.
//nd:hostfunc raw=true
// Designed for binary endpoints like getCoverArt and stream that return
// non-JSON data. The data is base64-encoded over JSON on the wire.
//nd:hostfunc
CallRaw(ctx context.Context, uri string) (contentType string, data []byte, err error)
}

View File

@@ -4,7 +4,6 @@ package host
import (
"context"
"encoding/binary"
"encoding/json"
extism "github.com/extism/go-sdk"
@@ -26,6 +25,13 @@ type SubsonicAPICallRawRequest struct {
Uri string `json:"uri"`
}
// SubsonicAPICallRawResponse is the response type for SubsonicAPI.CallRaw.
type SubsonicAPICallRawResponse struct {
ContentType string `json:"contentType,omitempty"`
Data []byte `json:"data,omitempty"`
Error string `json:"error,omitempty"`
}
// RegisterSubsonicAPIHostFunctions registers SubsonicAPI service host functions.
// The returned host functions should be added to the plugin's configuration.
func RegisterSubsonicAPIHostFunctions(service SubsonicAPIService) []extism.HostFunction {
@@ -76,37 +82,28 @@ func newSubsonicAPICallRawHostFunction(service SubsonicAPIService) extism.HostFu
// Read JSON request from plugin memory
reqBytes, err := p.ReadBytes(stack[0])
if err != nil {
subsonicapiWriteRawError(p, stack, err)
subsonicapiWriteError(p, stack, err)
return
}
var req SubsonicAPICallRawRequest
if err := json.Unmarshal(reqBytes, &req); err != nil {
subsonicapiWriteRawError(p, stack, err)
subsonicapiWriteError(p, stack, err)
return
}
// Call the service method
contenttype, data, svcErr := service.CallRaw(ctx, req.Uri)
if svcErr != nil {
subsonicapiWriteRawError(p, stack, svcErr)
subsonicapiWriteError(p, stack, svcErr)
return
}
// Write binary-framed response to plugin memory:
// [0x00][4-byte content-type length (big-endian)][content-type string][raw data]
ctBytes := []byte(contenttype)
frame := make([]byte, 1+4+len(ctBytes)+len(data))
frame[0] = 0x00 // success
binary.BigEndian.PutUint32(frame[1:5], uint32(len(ctBytes)))
copy(frame[5:5+len(ctBytes)], ctBytes)
copy(frame[5+len(ctBytes):], data)
respPtr, err := p.WriteBytes(frame)
if err != nil {
stack[0] = 0
return
// Write JSON response to plugin memory
resp := SubsonicAPICallRawResponse{
ContentType: contenttype,
Data: data,
}
stack[0] = respPtr
subsonicapiWriteResponse(p, stack, resp)
},
[]extism.ValueType{extism.ValueTypePTR},
[]extism.ValueType{extism.ValueTypePTR},
@@ -137,14 +134,3 @@ func subsonicapiWriteError(p *extism.CurrentPlugin, stack []uint64, err error) {
respPtr, _ := p.WriteBytes(respBytes)
stack[0] = respPtr
}
// subsonicapiWriteRawError writes a binary-framed error response to plugin memory.
// Format: [0x01][UTF-8 error message]
func subsonicapiWriteRawError(p *extism.CurrentPlugin, stack []uint64, err error) {
errMsg := []byte(err.Error())
frame := make([]byte, 1+len(errMsg))
frame[0] = 0x01 // error
copy(frame[1:], errMsg)
respPtr, _ := p.WriteBytes(frame)
stack[0] = respPtr
}

68
plugins/host/task.go Normal file
View File

@@ -0,0 +1,68 @@
package host
import "context"
// TaskInfo holds the current state of a task.
type TaskInfo struct {
// Status is the current task status: "pending", "running",
// "completed", "failed", or "cancelled".
Status string `json:"status"`
// Message is the status/result message returned by the plugin callback.
Message string `json:"message"`
// Attempt is the current or last attempt number (1-based).
Attempt int32 `json:"attempt"`
}
// QueueConfig holds configuration for a task queue.
type QueueConfig struct {
// Concurrency is the max number of parallel workers. Default: 1.
// Capped by the plugin's manifest maxConcurrency.
Concurrency int32 `json:"concurrency"`
// MaxRetries is the number of times to retry a failed task. Default: 0.
MaxRetries int32 `json:"maxRetries"`
// BackoffMs is the initial backoff between retries in milliseconds.
// Doubles each retry (exponential: backoffMs * 2^(attempt-1)). Default: 1000.
BackoffMs int64 `json:"backoffMs"`
// DelayMs is the minimum delay between starting consecutive tasks
// in milliseconds. Useful for rate limiting. Default: 0.
DelayMs int64 `json:"delayMs"`
// RetentionMs is how long completed/failed/cancelled tasks are kept
// in milliseconds. Default: 3600000 (1h). Min: 60000 (1m). Max: 604800000 (1w).
RetentionMs int64 `json:"retentionMs"`
}
// TaskService provides persistent task queues for plugins.
//
// This service allows plugins to create named queues with configurable concurrency,
// retry policies, and rate limiting. Tasks are persisted to SQLite and survive
// server restarts. When a task is ready to execute, the host calls the plugin's
// nd_task_execute callback function.
//
//nd:hostservice name=Task permission=taskqueue
type TaskService interface {
// CreateQueue creates a named task queue with the given configuration.
// Zero-value fields in config use sensible defaults.
// If a queue with the same name already exists, returns an error.
// On startup, this also recovers any stale "running" tasks from a previous crash.
//nd:hostfunc
CreateQueue(ctx context.Context, name string, config QueueConfig) error
// Enqueue adds a task to the named queue. Returns the task ID.
// payload is opaque bytes passed back to the plugin on execution.
//nd:hostfunc
Enqueue(ctx context.Context, queueName string, payload []byte) (string, error)
// Get returns the current state of a task including its status,
// message, and attempt count.
//nd:hostfunc
Get(ctx context.Context, taskID string) (*TaskInfo, error)
// Cancel cancels a pending task. Returns error if already
// running, completed, or failed.
//nd:hostfunc
Cancel(ctx context.Context, taskID string) error
}

220
plugins/host/task_gen.go Normal file
View File

@@ -0,0 +1,220 @@
// Code generated by ndpgen. DO NOT EDIT.
package host
import (
"context"
"encoding/json"
extism "github.com/extism/go-sdk"
)
// TaskCreateQueueRequest is the request type for Task.CreateQueue.
type TaskCreateQueueRequest struct {
Name string `json:"name"`
Config QueueConfig `json:"config"`
}
// TaskCreateQueueResponse is the response type for Task.CreateQueue.
type TaskCreateQueueResponse struct {
Error string `json:"error,omitempty"`
}
// TaskEnqueueRequest is the request type for Task.Enqueue.
type TaskEnqueueRequest struct {
QueueName string `json:"queueName"`
Payload []byte `json:"payload"`
}
// TaskEnqueueResponse is the response type for Task.Enqueue.
type TaskEnqueueResponse struct {
Result string `json:"result,omitempty"`
Error string `json:"error,omitempty"`
}
// TaskGetRequest is the request type for Task.Get.
type TaskGetRequest struct {
TaskID string `json:"taskId"`
}
// TaskGetResponse is the response type for Task.Get.
type TaskGetResponse struct {
Result *TaskInfo `json:"result,omitempty"`
Error string `json:"error,omitempty"`
}
// TaskCancelRequest is the request type for Task.Cancel.
type TaskCancelRequest struct {
TaskID string `json:"taskId"`
}
// TaskCancelResponse is the response type for Task.Cancel.
type TaskCancelResponse struct {
Error string `json:"error,omitempty"`
}
// RegisterTaskHostFunctions registers Task service host functions.
// The returned host functions should be added to the plugin's configuration.
func RegisterTaskHostFunctions(service TaskService) []extism.HostFunction {
return []extism.HostFunction{
newTaskCreateQueueHostFunction(service),
newTaskEnqueueHostFunction(service),
newTaskGetHostFunction(service),
newTaskCancelHostFunction(service),
}
}
func newTaskCreateQueueHostFunction(service TaskService) extism.HostFunction {
return extism.NewHostFunctionWithStack(
"task_createqueue",
func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) {
// Read JSON request from plugin memory
reqBytes, err := p.ReadBytes(stack[0])
if err != nil {
taskWriteError(p, stack, err)
return
}
var req TaskCreateQueueRequest
if err := json.Unmarshal(reqBytes, &req); err != nil {
taskWriteError(p, stack, err)
return
}
// Call the service method
if svcErr := service.CreateQueue(ctx, req.Name, req.Config); svcErr != nil {
taskWriteError(p, stack, svcErr)
return
}
// Write JSON response to plugin memory
resp := TaskCreateQueueResponse{}
taskWriteResponse(p, stack, resp)
},
[]extism.ValueType{extism.ValueTypePTR},
[]extism.ValueType{extism.ValueTypePTR},
)
}
func newTaskEnqueueHostFunction(service TaskService) extism.HostFunction {
return extism.NewHostFunctionWithStack(
"task_enqueue",
func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) {
// Read JSON request from plugin memory
reqBytes, err := p.ReadBytes(stack[0])
if err != nil {
taskWriteError(p, stack, err)
return
}
var req TaskEnqueueRequest
if err := json.Unmarshal(reqBytes, &req); err != nil {
taskWriteError(p, stack, err)
return
}
// Call the service method
result, svcErr := service.Enqueue(ctx, req.QueueName, req.Payload)
if svcErr != nil {
taskWriteError(p, stack, svcErr)
return
}
// Write JSON response to plugin memory
resp := TaskEnqueueResponse{
Result: result,
}
taskWriteResponse(p, stack, resp)
},
[]extism.ValueType{extism.ValueTypePTR},
[]extism.ValueType{extism.ValueTypePTR},
)
}
func newTaskGetHostFunction(service TaskService) extism.HostFunction {
return extism.NewHostFunctionWithStack(
"task_get",
func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) {
// Read JSON request from plugin memory
reqBytes, err := p.ReadBytes(stack[0])
if err != nil {
taskWriteError(p, stack, err)
return
}
var req TaskGetRequest
if err := json.Unmarshal(reqBytes, &req); err != nil {
taskWriteError(p, stack, err)
return
}
// Call the service method
result, svcErr := service.Get(ctx, req.TaskID)
if svcErr != nil {
taskWriteError(p, stack, svcErr)
return
}
// Write JSON response to plugin memory
resp := TaskGetResponse{
Result: result,
}
taskWriteResponse(p, stack, resp)
},
[]extism.ValueType{extism.ValueTypePTR},
[]extism.ValueType{extism.ValueTypePTR},
)
}
func newTaskCancelHostFunction(service TaskService) extism.HostFunction {
return extism.NewHostFunctionWithStack(
"task_cancel",
func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) {
// Read JSON request from plugin memory
reqBytes, err := p.ReadBytes(stack[0])
if err != nil {
taskWriteError(p, stack, err)
return
}
var req TaskCancelRequest
if err := json.Unmarshal(reqBytes, &req); err != nil {
taskWriteError(p, stack, err)
return
}
// Call the service method
if svcErr := service.Cancel(ctx, req.TaskID); svcErr != nil {
taskWriteError(p, stack, svcErr)
return
}
// Write JSON response to plugin memory
resp := TaskCancelResponse{}
taskWriteResponse(p, stack, resp)
},
[]extism.ValueType{extism.ValueTypePTR},
[]extism.ValueType{extism.ValueTypePTR},
)
}
// taskWriteResponse writes a JSON response to plugin memory.
func taskWriteResponse(p *extism.CurrentPlugin, stack []uint64, resp any) {
respBytes, err := json.Marshal(resp)
if err != nil {
taskWriteError(p, stack, err)
return
}
respPtr, err := p.WriteBytes(respBytes)
if err != nil {
stack[0] = 0
return
}
stack[0] = respPtr
}
// taskWriteError writes an error response to plugin memory.
func taskWriteError(p *extism.CurrentPlugin, stack []uint64, err error) {
errResp := struct {
Error string `json:"error"`
}{Error: err.Error()}
respBytes, _ := json.Marshal(errResp)
respPtr, _ := p.WriteBytes(respBytes)
stack[0] = respPtr
}

190
plugins/host_httpclient.go Normal file
View File

@@ -0,0 +1,190 @@
package plugins
import (
"bytes"
"cmp"
"context"
"fmt"
"io"
"net"
"net/http"
"net/url"
"strings"
"time"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/plugins/host"
)
const (
httpClientDefaultTimeout = 10 * time.Second
httpClientMaxRedirects = 5
httpClientMaxResponseBodyLen = 10 * 1024 * 1024 // 10 MB
)
// httpServiceImpl implements host.HTTPService.
type httpServiceImpl struct {
pluginName string
requiredHosts []string
client *http.Client
}
// newHTTPService creates a new HTTPService for a plugin.
func newHTTPService(pluginName string, permission *HTTPPermission) *httpServiceImpl {
var requiredHosts []string
if permission != nil {
requiredHosts = permission.RequiredHosts
}
svc := &httpServiceImpl{
pluginName: pluginName,
requiredHosts: requiredHosts,
}
svc.client = &http.Client{
Transport: http.DefaultTransport,
// Timeout is set per-request via context deadline, not here.
// CheckRedirect validates hosts and enforces redirect limits.
CheckRedirect: func(req *http.Request, via []*http.Request) error {
if len(via) >= httpClientMaxRedirects {
log.Warn(req.Context(), "HTTP redirect limit exceeded", "plugin", svc.pluginName, "url", req.URL.String(), "redirectCount", len(via))
return http.ErrUseLastResponse
}
if err := svc.validateHost(req.Context(), req.URL.Host); err != nil {
log.Warn(req.Context(), "HTTP redirect blocked", "plugin", svc.pluginName, "url", req.URL.String(), "err", err)
return err
}
return nil
},
}
return svc
}
func (s *httpServiceImpl) Send(ctx context.Context, request host.HTTPRequest) (*host.HTTPResponse, error) {
// Parse and validate URL
parsedURL, err := url.Parse(request.URL)
if err != nil {
return nil, fmt.Errorf("invalid URL: %w", err)
}
// Validate URL scheme
if parsedURL.Scheme != "http" && parsedURL.Scheme != "https" {
return nil, fmt.Errorf("invalid URL scheme %q: must be http or https", parsedURL.Scheme)
}
// Validate host against allowed hosts and private IP restrictions
if err := s.validateHost(ctx, parsedURL.Host); err != nil {
return nil, err
}
// Apply per-request timeout via context deadline
timeout := cmp.Or(time.Duration(request.TimeoutMs)*time.Millisecond, httpClientDefaultTimeout)
ctx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
// Build request body
method := strings.ToUpper(request.Method)
var body io.Reader
if len(request.Body) > 0 {
body = bytes.NewReader(request.Body)
}
// Create HTTP request
httpReq, err := http.NewRequestWithContext(ctx, method, request.URL, body)
if err != nil {
return nil, fmt.Errorf("creating request: %w", err)
}
for k, v := range request.Headers {
httpReq.Header.Set(k, v)
}
// Execute request
resp, err := s.client.Do(httpReq) //nolint:gosec // URL is validated against requiredHosts
if err != nil {
return nil, err
}
defer resp.Body.Close()
log.Trace(ctx, "HTTP request", "plugin", s.pluginName, "method", method, "url", request.URL, "status", resp.StatusCode)
// Read response body (with size limit to prevent memory exhaustion)
respBody, err := io.ReadAll(io.LimitReader(resp.Body, httpClientMaxResponseBodyLen))
if err != nil {
return nil, fmt.Errorf("reading response body: %w", err)
}
// Flatten response headers (first value only)
headers := make(map[string]string, len(resp.Header))
for k, v := range resp.Header {
if len(v) > 0 {
headers[k] = v[0]
}
}
return &host.HTTPResponse{
StatusCode: int32(resp.StatusCode),
Headers: headers,
Body: respBody,
}, nil
}
// validateHost checks whether a request to the given host is permitted.
// When requiredHosts is set, it checks against the allowlist.
// When requiredHosts is empty, it blocks private/loopback IPs to prevent SSRF.
func (s *httpServiceImpl) validateHost(ctx context.Context, hostStr string) error {
hostname := extractHostname(hostStr)
if len(s.requiredHosts) > 0 {
if !s.isHostAllowed(hostname) {
return fmt.Errorf("host %q is not allowed", hostStr)
}
return nil
}
// No explicit allowlist: block private/loopback IPs
if isPrivateOrLoopback(hostname) {
log.Warn(ctx, "HTTP request to private/loopback address blocked", "plugin", s.pluginName, "host", hostStr)
return fmt.Errorf("host %q is not allowed: private/loopback addresses require explicit requiredHosts in manifest", hostStr)
}
return nil
}
func (s *httpServiceImpl) isHostAllowed(hostname string) bool {
for _, pattern := range s.requiredHosts {
if matchHostPattern(pattern, hostname) {
return true
}
}
return false
}
// extractHostname returns the hostname portion of a host string, stripping
// any port number and IPv6 brackets. It handles IPv6 addresses correctly
// (e.g. "[::1]:8080" → "::1", "[::1]" → "::1").
func extractHostname(hostStr string) string {
if h, _, err := net.SplitHostPort(hostStr); err == nil {
return h
}
// Strip IPv6 brackets when no port is present (e.g. "[::1]" → "::1")
if strings.HasPrefix(hostStr, "[") && strings.HasSuffix(hostStr, "]") {
return hostStr[1 : len(hostStr)-1]
}
return hostStr
}
// isPrivateOrLoopback returns true if the given hostname resolves to or is
// a private, loopback, or link-local IP address. This includes:
// IPv4: 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16
// IPv6: ::1, fc00::/7, fe80::/10
// It also blocks "localhost" by name.
func isPrivateOrLoopback(hostname string) bool {
if strings.EqualFold(hostname, "localhost") {
return true
}
ip := net.ParseIP(hostname)
if ip == nil {
return false
}
return ip.IsLoopback() || ip.IsPrivate() || ip.IsLinkLocalUnicast() || ip.IsLinkLocalMulticast()
}
// Verify interface implementation
var _ host.HTTPService = (*httpServiceImpl)(nil)

Some files were not shown because too many files have changed in this diff Show More