Compare commits

..
Author SHA1 Message Date
Deluan a998a329e6 test(artwork): use renamed ArtworkWorkerConcurrency in e2e tests 2026-07-23 14:09:39 -04:00
Deluan 42fd263cdf fix(artwork): clamp negative sizes to full-size; convert imghttp test to Ginkgo
- A negative size (Subsonic size / Jellyfin maxwidth accept signed ints) reached
  resizeStaticImage, where the square path builds image.NewNRGBA(Rect(0,0,size,size))
  — a giant rectangle that panics/OOMs. Clamp size<0 to 0 (full-size) at the Service
  entry. Positive sizes were already clamped to the original.
- imghttp used a plain func Test with a table; convert to a Ginkgo DescribeTable with
  the suite entry point in imghttp_suite_test.go (AGENTS.md test-framework requirement).
2026-07-23 14:08:08 -04:00
Deluan f92507b152 fix(artwork): preserve the drive when normalizing Windows file:// library paths
url.Parse puts the volume of file://C:/Music in Host, not Path, so localOSRoot dropped
it and returned /Music — os.Open/os.Stat then failed and folder/embedded art on Windows
looped as dangling. Rejoin the host volume, matching core/storage/local's newLocalStorage.
2026-07-23 14:08:08 -04:00
Deluan aba7ed925c fix(artwork): honor disabled per-track art at serve time; use nanosecond mtime provenance
Two serving-correctness fixes from review:
- serveMediaFile served a persisted mf embedded image even after EnableMediaFileCoverArt
  was turned off (the setting isn't in the config fingerprint, so found rows aren't
  reprocessed). Direct mf- URLs now honor the setting at serve time and fall back to
  disc/album art.
- The file-backed staleness check compared whole-second mtimes, so a same-second content
  replacement (two writes in one second, or timestamp-preserving tools) could serve
  different bytes under the old hash + immutable policy. RefMtime is now unix-nanoseconds
  (no schema change; int64 column), detecting sub-second changes where the filesystem
  records them.
2026-07-23 14:08:08 -04:00
Deluan aa6c0b1f17 fix(artwork): enqueue new empty playlists by id, and refresh on absent outcomes
Two worker/enqueue fixes from review:
- playlistRepository.Put assigned the generated id to the caller's Playlist but passed
  the stale copy (empty id) to refreshCounters, enqueueing a pl|"" row the worker
  failed until the daily dangling purge while the real playlist went unresolved. Set
  the id on the copy before enqueueing.
- The drain refresh batch only included found/foundStale, so a cover removed by a scan
  (found -> absent) never notified clients, leaving the old immutable image displayed.
  Broadcast absent outcomes too; precache still only warms found/foundStale.
2026-07-23 14:08:08 -04:00
Deluan e9c15d7fcb fix(artwork): don't stamp the album hash onto multi-disc tracks
The hydration fallback assigned a found album hash to every fallback track, but a
multi-disc track's CoverArtID emits a dc- id served from disc-specific art whose hash
is unknown at hydration time. Advertising dc-..._<albumHash> gave clients a content-
version that never changes when the disc image does, breaking id-based refresh. Only
stamp the album hash for single-disc tracks (DiscNumber == 0); multi-disc tracks stay
unhashed and rely on the correct ETag returned by the served response.
2026-07-23 14:08:08 -04:00
Deluan 3eaa21229d fix(artwork): version the artwork ETag with the served representation
The ETag was the pixel hash of the original image, so a CoverArtQuality or
EnableWebPEncoding change altered the resized bytes without changing the ETag —
revalidating clients got a spurious 304 and kept the old encoding. Resized responses
now carry a representation ETag (hash + size + square + encode settings) used for the
ETag header and If-None-Match, while the immutable decision stays on the pixel hash
(URLs remain pixel-identity per the spec, so hash-suffixed clients keep zero-request
caching). Full-size originals fall back to the pixel hash as before.
2026-07-23 14:08:08 -04:00
Deluan 9dd306eb10 fix(artwork): enforce entity visibility on the Subsonic getCoverArt path
serveEntity reads persisted item_artwork by id, bypassing the library and private-
playlist filters that the legacy entity-load applied. On the authenticated Subsonic
path a user could fetch artwork for an inaccessible album or someone else's private
playlist by guessing an id. getCoverArt now resolves the underlying entity through
the request-scoped (filtered) repositories and serves the placeholder when it is not
visible, so existence isn't leaked and the always-an-image invariant holds. The
public share (JWT-authorized) and Jellyfin (admin) paths are intentionally untouched.
2026-07-23 14:08:08 -04:00
Deluan 8d715ba2fa fix(artwork): restore synthetic-artist guard and unicode normalization in agent lookups
Moving agent calls into the worker bypassed two behaviors of the aggregate provider:
Agents.GetArtistImages' guard for Unknown/Various Artists (a direct retriever call
could assign an unrelated image to a synthetic artist), and auxAlbum/auxArtist.Name's
DevPreserveUnicodeInExternalCalls normalization (records with typographic quotes/dashes
missed exact-name searches). Re-apply both before enumerating retrievers.
2026-07-23 14:08:08 -04:00
Deluan 50ada9ad29 fix(artwork): invalidate artwork when an uploaded image is deleted
Deleting an artist/radio/playlist upload cleared the filename but left the found
item_artwork row and its hash, so lists kept advertising the deleted cover's
hash-suffixed immutable URL and clients could display it indefinitely. Call
EnqueueArtwork after the delete-side Put, symmetric with upload, so the state is
cleared and re-resolved to the next source (or absent).
2026-07-23 14:08:08 -04:00
Deluan ed4178a6a9 fix(artwork): only use disc resolution for multi-disc albums
DiscCoverArtID returns a dc- id for any track with DiscNumber>0, so serveDisc ran
the full DiscArtPriority chain even for single-disc albums, where a stray disc*/
embedded image could shadow higher-priority album art. Gate disc resolution on the
album having more than one disc, matching the legacy reader; single-disc tracks
serve album art directly.
2026-07-23 14:08:08 -04:00
Deluan ce06599288 fix(artwork): open library-backed artwork through its on-disk root
A library configured with a file:// path stored absRoot as the raw URI, so Abs
produced strings like file:/music/cover.jpg that os.Open/os.Stat reject — folder,
upload and embedded art were treated as dangling on every request, looping forever.
Normalize a file:// path to its parsed OS path (the same root os.DirFS uses);
non-local schemes are left unchanged (out of scope, per the artwork-musicfs TODO).
2026-07-23 14:08:08 -04:00
Deluan f016192eec fix(artwork): requeue playlist cover when its track set changes
A generated-grid cover went stale after track mutations: nothing re-resolved the
playlist's artwork, and the request path deliberately never rebuilds the grid, so
serveEntity kept returning the old grid hash indefinitely. Enqueue pl artwork from
refreshCounters (the choke point for every track-set change); no clear, so the old
cover keeps serving until the worker rebuilds.
2026-07-23 14:08:08 -04:00
Deluan cfca4a2433 fix(artwork): serve a local playlist ExternalImageURL as a file-backed reference
A local ExternalImageURL was resolved through the external step and labelled
external, so placeBytes copied it into the content-addressed store and dropped its
path/mtime — replacing the file never tripped the staleness check. Classify local
references as file-backed (resolved in place, even on the request path) and keep
store-backed behaviour only for http(s) URLs.
2026-07-23 14:08:08 -04:00
Deluan 49039fab47 fix(artwork): keep multi-disc tracks requestable when the album is absent
Round-1's hydration fix still copied the album's known-absent onto a non-eligible
(or own-absent) track, but MediaFile.CoverArtID routes a multi-disc track to disc
art, which resolves provisionally and is never known-absent. Marking it absent made
Subsonic omit coverArt so clients never requested a valid disc image. Only mark a
single-disc track absent, and only when its own art won't resolve.
2026-07-23 14:08:08 -04:00
Deluan 66d3d23149 fix(artwork): enqueue uploaded artwork only after the filename is persisted
SetImage cleared state and enqueued the bump before the caller stored the new
filename, so a worker drain in that window could resolve against the old (already
deleted) file and settle absent, leaving the upload unused until a later scan. Move
the invalidate+enqueue into EnqueueArtwork, which each caller now invokes after the
entity Put.
2026-07-23 14:08:08 -04:00
Deluan f4e14e9c1a fix(artwork): fall back to disc art, not the album, for multi-disc tracks
serveMediaFile delegated an absent/ineligible track straight to AlbumCoverArtID,
skipping the disc-specific lookup that MediaFile.CoverArtID (and the deleted legacy
reader) use. On multi-disc albums with per-disc images that served the album cover
instead of the configured disc artwork. Delegate through DiscCoverArtID.
2026-07-23 14:08:08 -04:00
Deluan d15d85ad0c fix(artwork): validate each agent image URL before picking the largest
bestImageURL selected the largest by size and only then parsed it, so a malformed
largest URL (e.g. a bad percent-escape) returned nil and shadowed a valid smaller
candidate, contradicting the documented skip-unparseable behavior. Parse per
candidate and compare sizes only among URLs that parse.
2026-07-23 14:08:08 -04:00
Deluan 0781c4a9b2 fix(artwork): keep an eligible track's cover requestable when its album is absent
An embedded-eligible track with no resolved item_artwork row inherited the album's
ImageAbsent, so when the album resolved absent (e.g. CoverArtPriority without
'embedded') the track's coverArt was omitted permanently — the client never
requested it, so the lazy mediafile path never resolved it — even though the
serving path would extract and serve the track's own embedded art. Hydration now
never copies the album's absence onto an eligible-but-unresolved track.
2026-07-23 14:08:08 -04:00
Deluan ca4220b029 fix(artwork): request read-through must not reset the failure backoff
The provisional read-through and dangling re-enqueue used Enqueue, whose upsert
resets retry_at, so any browse of an unresolved entity that was backing off after
an external failure made it immediately eligible again — defeating the exponential
backoff during a provider outage. Add EnqueueBump, which raises priority but leaves
an existing row's retry_at intact, and route the serving path through it. Scan and
manual re-resolve keep Enqueue's reset (a detected change wants immediate retry).
2026-07-23 14:08:08 -04:00
Deluan c2d7ae773c chore(artwork): generic 500 bodies on refresh endpoint, trim stale test comments 2026-07-23 14:08:08 -04:00
Deluan dfd4bec270 test(artwork): end-to-end coverage for the serving cutover 2026-07-23 14:08:08 -04:00
Deluan 0d1df1648e feat(artwork): precache on acquisition, bump on upload/radio changes, manual re-resolve API 2026-07-23 14:08:08 -04:00
Deluan 937e58e5fb refactor(artwork): delete the legacy reader chain, cache warmer, and provider image methods 2026-07-23 14:08:08 -04:00
Deluan 8fea7efa50 feat(subsonic): content-hash coverArt ids, omit artwork on known-absent 2026-07-23 14:08:08 -04:00
Deluan 25b32f9706 feat(server): serve artwork from persisted state with content-hash caching 2026-07-23 14:08:08 -04:00
Deluan 313998fd65 feat(artwork): state-backed serving path with provisional read-through 2026-07-23 14:08:08 -04:00
Deluan 3a9dadfe34 fix(artwork): broadcast refresh for stale-found artwork too 2026-07-23 14:08:08 -04:00
Deluan 05ba549843 feat(artwork): broadcast refresh events when artwork lands 2026-07-23 14:08:08 -04:00
Deluan 5179691811 feat(artwork): resolve media_file embedded art in the worker, invalidate on rescan 2026-07-23 14:07:52 -04:00
Deluan 2ab1323b28 feat(persistence): hydrate artwork hash and absence onto entity pages 2026-07-23 14:07:52 -04:00
Deluan 3b7cbf41dd feat(model): content-hash artwork id suffix and hydratable per-entity image state 2026-07-23 14:07:52 -04:00
Deluan 39e939686b fix(artwork): treat agent not-found as breaker success 2026-07-23 14:07:52 -04:00
Deluan 190c291e61 feat(artwork): worker fetches agent images directly with per-agent rate limits and breakers 2026-07-23 14:07:52 -04:00
Deluan b7f94f6727 feat(agents): enumerate enabled image-retriever agents per capability 2026-07-23 14:06:30 -04:00
Deluan f34a386137 fix(deezer): never return empty-image-id placeholder pictures 2026-07-23 14:06:30 -04:00
Deluan fc55e8bf16 feat(artwork): promote worker concurrency and external rate to real configs
The artwork worker's drain speed was governed by two hidden Dev flags,
DevArtworkWorkerConcurrency and DevArtworkExternalRPS, both defaulting to 2.
On a large library's one-time backfill the external rate limiter is the real
ceiling: every art-less item waits on it before the (rate-limited) external
lookup, so the drain crawls at ~RPS items/sec while local-art items are
unaffected.

Promote both to documented, supported options: ArtworkWorkerConcurrency
(default 4) sets local-resolution parallelism, ArtworkExternalMaxRPS
(default 2, 0 = unlimited) caps external-agent lookups to stay polite to
Last.fm/Deezer/etc. Operators can now trade first-backfill speed against
external-API rate limits. The old Dev names still map for backward compat.
2026-07-23 14:05:10 -04:00
Deluan 9ce51cf575 perf(artwork): fetch only IDs for backfill enumeration
Backfill enumerated every album, artist, playlist and radio via GetAll
and mapped out just the ID. GetAll materializes full entities (library
joins, participant/stats/tags JSON, annotation, artwork hydration), so on
a large library it loaded tens of thousands of heavy structs only to read
one field each — spiking transient RSS to ~1GB during the one-time
upgrade backfill, a memory risk on small NAS/Pi hardware.

Add GetAllIDs to the album, artist, playlist and radio repositories: it
reuses each repo's base row-set filter (library visibility, artist
content join, playlist userFilter) but projects only id, skipping the
heavy columns and post-processing. A per-repo parity test asserts
GetAllIDs returns exactly the same id set as GetAll.

Verified on a 727MB / 29k-artist production DB copy: peak RSS during
backfill dropped from ~1012MB to ~89MB, file descriptors flat, same
36,138 items enqueued.
2026-07-23 13:20:22 -04:00
Deluan b172ce4296 refactor(artwork): reuse auth.WithAdminUser and dedupe image cap guards 2026-07-22 18:02:54 -04:00
Deluan bba0eab3a5 fix(artwork): apply image limits to playlist tile decoding
decodeTile ran image.Decode on every sampled album's resolved bytes
before processItem's maxImageBytes/maxImagePixels guards applied,
letting an oversized or decompression-bomb tile fully decode
unbounded. Enforce both caps inside decodeTile itself.
2026-07-22 17:09:22 -04:00
Deluan f614850ff0 fix(artwork): store backing-file provenance per item, not per hash 2026-07-22 15:54:08 -04:00
Deluan ba2290af6d test(artwork): convert non-synctest timing tests to Ginkgo specs
TestArtworkBackoffSchedule and TestArtworkWorkerRunNoLeak needed no real
*testing.T (no synctest), so move them into worker_test.go as Ginkgo
specs. TestArtworkBreakerHalfOpen stays plain since testing/synctest
requires a real *testing.T, matching core/scrobbler's precedent.
2026-07-22 15:53:32 -04:00
Deluan 55608b2d20 fix(artwork): resolve private playlists with an admin context 2026-07-22 15:53:32 -04:00
Deluan 7713a6d6b2 fix(artwork): include M3U external art flag in the config fingerprint 2026-07-22 15:53:32 -04:00
Deluan bc30ce67c6 fix(artwork): keep fresh re-enqueues ahead of stale failure backoff 2026-07-22 15:53:32 -04:00
Deluan d6434b9929 fix(artwork): reject decompression-bomb dimensions before decoding 2026-07-22 15:53:32 -04:00
Deluan b3526c0fba test(artwork): make leak and permission tests pass on linux
goleak now ignores notify's nonrecursive-tree goroutines (linux uses
inotify, which spawns dispatch+internal instead of darwin's recursive
dispatch), and the read-only-dir prune spec skips under root, where
permission bits cannot make Remove fail.
2026-07-22 15:53:32 -04:00
Deluan 3d32157403 test(artwork): move soak test into the Ginkgo suite 2026-07-22 15:53:32 -04:00
Deluan 5482784bfc fix(artwork): treat playlist cover URL 404 as definitive miss
The playlist ExternalImageURL step used sources.go's fromURL, which maps any
non-200 to a generic error, so a stale URL returning 404/410 was classified
transient: infinite backoff plus it counted toward the circuit breaker,
blocking valid external work. Add a local fetch in resolve.go that maps
404/410 to model.ErrNotFound (definitive) while keeping other non-200s
transient. sources.go is left untouched.
2026-07-22 15:53:32 -04:00
Deluan 6afcb93a9b fix(artwork): retry higher-priority external art after fallback hit
With CoverArtPriority="external,cover.jpg", a transient external failure
followed by a folder hit dropped the external error: the worker recorded
found and deleted the queue row, so the configured higher-priority external
art was never retried. Carry extError onto the fallback resolution and add
an outcomeFoundStale that persists+serves the art but reschedules via
MarkFailed, giving the external source another chance. When external later
answers definitively-not-found, the hit is not stale and the row is deleted.
2026-07-22 15:53:32 -04:00
Deluan 67f6d8aee8 fix(artwork): cap resolved image reads
A user-editable ExternalImageURL can point at an arbitrarily large endpoint;
a fast server could make the worker buffer hundreds of MB inside the 5s HTTP
timeout. Bound the read to a fixed 20MB cap (no config knob) via io.LimitReader
and fail the item if it is exceeded.
2026-07-22 15:53:32 -04:00
Deluan 87095fab08 refactor(artwork): deduplicate purge loop, backfill table, and extGate alias 2026-07-22 15:53:32 -04:00
Deluan c57496d50d fix(artwork): treat missing local playlist cover as definitive, not transient
A playlist ExternalImageURL pointing at a local file that fails to open was
routed through extError, causing failed/48h-retry loops that burn a rate
limiter token forever instead of falling through to the generated grid.
2026-07-22 15:53:32 -04:00
Deluan 0fbbd01357 style(artwork): fix comment accuracy and budget; fingerprint ArtistImageFolder
Correct the inverted workerDeps.extGate comment, trim over-budget doc comments, and add conf.Server.ArtistImageFolder to the resolution fingerprint so an image-folder change re-resolves artist artwork.
2026-07-22 15:53:32 -04:00
Deluan bab9b5cd3a fix(artwork): purge dangling queue rows and guard concurrent re-enqueues
Queue rows for deleted entities failed forever (Get -> ErrNotFound -> failed -> capped retries, unbounded). Add ArtworkQueueRepository.PurgeDangling, called from Prune next to the item_artwork purge. Separately, the found/absent path unconditionally deleted the dequeued row, erasing a concurrent scan re-enqueue; switch to DeleteIfUnchanged, which deletes only while retry_at still matches the dequeued value (verified retry_at is the column an Enqueue upsert resets).
2026-07-22 15:53:32 -04:00
Deluan 454fd24833 fix(artwork): resolve full playlist source chain
resolvePlaylist only built the generated grid, dropping the uploaded-image, sidecar and ExternalImageURL sources the old reader_playlist.go chain serves. Port the full chain before the grid fallback: uploaded (upload), sidecar (folder), and ExternalImageURL routed through extGate with the same extError semantics as the other external steps. Also rewires the artist external step onto ArtistImageResult.
2026-07-22 15:53:32 -04:00
Deluan c01e9b3184 fix(artwork): propagate transient artist image errors to the worker
callGetImage swallowed all agent errors, so an agent outage surfaced as ErrNotFound and the worker settled artist artwork as a definitive absent (and reset the breaker). Add an additive ArtistImageResult path that returns the underlying agent error on transient failure while keeping ArtistImage byte-identical for existing callers; the worker's artist external step uses it via fromArtistExternalResult.
2026-07-22 15:53:32 -04:00
Deluan 1ed8ebf9b0 test(artwork): leak/soak coverage and deferred assertions 2026-07-22 15:53:32 -04:00
Deluan ad38cd1d58 feat(artwork): artwork backfill, fingerprint re-resolution and scheduled jobs 2026-07-22 15:53:32 -04:00
Deluan 25a05fd017 feat(artwork): enqueue artwork resolution from scan and CRUD paths 2026-07-22 15:53:32 -04:00
Deluan 57c64e386a feat(artwork): add acquisition worker service 2026-07-22 15:53:32 -04:00
Deluan d6fc829f84 style(artwork): tighten processor comments to budget 2026-07-22 15:53:32 -04:00
Deluan e0655dc882 feat(artwork): add acquisition processor
Resolves one queue item end to end: hash/dedup, decode + 128px thumbnail
blurhash, place bytes (store vs source file), and persist found/absent/
failed state for the worker (Task 4) to act on.
2026-07-22 15:53:32 -04:00
Deluan 608db503a7 fix(artwork): propagate playlist tile failures and dedupe external step 2026-07-22 15:53:32 -04:00
Deluan 967de74bf7 feat(artwork): add worker-side artwork resolvers 2026-07-22 15:53:32 -04:00
Deluan 2efa697e52 feat(artwork): import blurhash encoder from #5797 2026-07-22 15:53:32 -04:00
Deluan 1f818e7633 fix(artwork): store backing-file provenance per item, not per hash 2026-07-22 15:52:54 -04:00
Deluan c04c8ee02a fix(artwork): mock PutImage refreshes created_at like the SQL repository
Prune specs now age fixtures directly instead of seeding stale timestamps through the upsert.
2026-07-22 00:58:22 -04:00
Deluan ebbe533c6a fix(artwork): reject malformed hashes in ImageStore operations
Known-absent states carry an empty hash and malformed persisted hashes could panic path sharding or inject separators; Write/Open/Remove now return an error for anything but 16 lowercase hex chars.
2026-07-22 00:51:13 -04:00
Deluan 0147cc59b1 fix(artwork): honor the orphan cutoff in the repository mock
The mock's DeleteOrphans now applies createdBefore like the SQL implementation, and a new spec covers a freshly reacquired row surviving prune.
2026-07-22 00:42:42 -04:00
Deluan 034cd17498 fix(artwork): index artwork_queue in dequeue order
The previous leading retry_at range column forced a temp B-tree sort of the whole eligible set on every DequeueBatch; ordering the index by (priority DESC, enqueued_at) lets scans stop after the batch size.
2026-07-22 00:36:22 -04:00
Deluan 8147f7c40b fix(artwork): rewrite vanished duplicates and sweep stale mime variants
Write falls through to a real write when the liveness touch fails, and sweep retention now matches the recorded mime's extension so obsolete variants are reclaimed.
2026-07-22 00:28:51 -04:00
Deluan bf614e66ad fix(artwork): guard orphan file removal with the prune grace window
Duplicate ImageStore writes refresh the file mtime and Remove skips files newer than the cutoff, so overlapping acquisitions cannot lose their store files to a concurrent prune.
2026-07-22 00:19:45 -04:00
Deluan 623b7d6a6c fix(artwork): atomic orphan deletion and timestamp semantics from review
DeleteOrphans re-checks age+references at delete time, PutItemArtwork defaults attempted_at, queue mock timestamps mirror SQL.
2026-07-22 00:11:15 -04:00
Deluan 6f7f9c6463 fix(artwork): address review findings on prune/sweep races and mock fidelity
Sweep now honors an mtime grace window (in-flight acquisitions and temp files), reacquired orphans reset the prune grace window, and the queue mock implements real stale-absent semantics.
2026-07-21 23:57:58 -04:00
Deluan 8fd7ef19f3 refactor(artwork): apply simplify-pass cleanups
Internal item_artwork sqlRepository helper, toSQLArgs upserts, batched queue enqueue, EnqueueStaleAbsent moved to queue repo, snapshot-based prune sweep, mock/real semantics aligned.
2026-07-21 23:37:05 -04:00
Deluan 1041e45ca7 fix(artwork): chunk unbounded IN clauses and restore interface docs 2026-07-21 23:24:42 -04:00
Deluan 4f835437a9 refactor(artwork): merge item artwork state into ArtworkRepository 2026-07-21 23:14:01 -04:00
Deluan b16ef725c9 refactor(artwork): fold originals package into core/artwork as ImageStore 2026-07-21 23:09:31 -04:00
Deluan 3e7685adc2 fix(artwork): never sweep files on transient DB errors during prune 2026-07-21 23:02:47 -04:00
Deluan db16b3de9a feat(artwork): add artwork prune (orphan cleanup) 2026-07-21 22:58:20 -04:00
Deluan b72597821a feat(artwork): add content-addressed originals store 2026-07-21 22:53:00 -04:00
Deluan fcff9c63e7 feat(artwork): implement artwork_queue repository 2026-07-21 22:45:08 -04:00
Deluan 14dd57052e feat(artwork): implement item_artwork repository with batched hydration 2026-07-21 22:45:03 -04:00
Deluan f926539c04 feat(artwork): implement artwork repository 2026-07-21 22:44:24 -04:00
Deluan 6cce65f759 feat(artwork): add artwork models, repository interfaces and mocks 2026-07-21 22:35:34 -04:00
Deluan 7aacb01f4f feat(artwork): add artwork, item_artwork and artwork_queue tables 2026-07-21 22:31:03 -04:00
627 changed files with 7372 additions and 34761 deletions

No files matched your search

+1 -1
View File
@@ -4,7 +4,7 @@
"dockerfile": "Dockerfile",
"args": {
// Update the VARIANT arg to pick a version of Go: 1, 1.15, 1.14
"VARIANT": "1.27",
"VARIANT": "1.26",
// Options
"INSTALL_NODE": "true",
"NODE_VERSION": "v24"
+3 -3
View File
@@ -1,10 +1,10 @@
# These are supported funding model platforms
ko_fi: deluan
github: deluan
open_collective: navidrome
liberapay: deluan
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: deluan
liberapay: deluan
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
issuehunt: # Replace with a single IssueHunt username
@@ -68,11 +68,6 @@ runs:
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v4
with:
# Runner IPs are shared, so anonymous base image pulls get rate-limited.
buildkitd-config-inline: |
[registry."docker.io"]
mirrors = ["mirror.gcr.io"]
- name: Extract metadata for Docker image
id: meta
-60
View File
@@ -1,60 +0,0 @@
name: Report coverage on PR
on:
workflow_run:
workflows: ['Pipeline: Test, Lint, Build']
types: [completed]
jobs:
comment:
name: Comment coverage report
if: github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.conclusion == 'success'
runs-on: ubuntu-latest
permissions:
contents: read
actions: read
pull-requests: write
env:
COVERAGE_COMMENT: 'true'
steps:
# Only the config, from the base branch: this job holds a write token, so
# it must never check out the fork.
- name: Check out the octocov config
uses: actions/checkout@v7
with:
sparse-checkout: .octocov.yml
sparse-checkout-cone-mode: false
persist-credentials: false
# Into a subdirectory. A pull_request run executes the fork's own copy of
# pipeline.yml, so every file in here is attacker-controlled.
- uses: actions/download-artifact@v8
with:
name: octocov-pr
path: untrusted
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ github.token }}
- name: Verify the artifact and take the coverage profile
id: pr
env:
GH_TOKEN: ${{ github.token }}
HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
run: |
number=$(head -c 20 untrusted/pr_number | tr -d '[:space:]')
case "$number" in ''|*[!0-9]*)
echo "::error::artifact pr_number is not a number"; exit 1;;
esac
sha=$(gh api "repos/$GITHUB_REPOSITORY/pulls/$number" --jq .head.sha)
if [ "$sha" != "$HEAD_SHA" ]; then
echo "::error::artifact claims PR #$number, but its head $sha is not $HEAD_SHA"; exit 1
fi
cp untrusted/coverage.out coverage.out
echo "number=$number" >> "$GITHUB_OUTPUT"
- uses: k1LoW/octocov-action@v1
env:
# A workflow_run job looks like a push to the default branch. Point
# octocov back at the pull request and at the run that produced it.
GITHUB_PULL_REQUEST_NUMBER: ${{ steps.pr.outputs.number }}
OCTOCOV_GITHUB_REF: refs/pull/${{ steps.pr.outputs.number }}/merge
OCTOCOV_GITHUB_SHA: ${{ github.event.workflow_run.head_sha }}
OCTOCOV_GITHUB_RUN_ID: ${{ github.event.workflow_run.id }}
+4 -7
View File
@@ -34,19 +34,16 @@ jobs:
}
const {data: {artifacts}} = await github.rest.actions.listWorkflowRunArtifacts({owner, repo, run_id});
const downloadable = artifacts.filter((art) => !art.name.startsWith('octocov-'));
if (!downloadable.length) {
if (!artifacts.length) {
return core.error(`No artifacts found`);
}
const header = `Download the artifacts for this pull request:`;
let body = `${header}\n`;
for (const art of downloadable) {
let body = `Download the artifacts for this pull request:\n`;
for (const art of artifacts) {
body += `\n* [${art.name}.zip](https://nightly.link/${owner}/${repo}/actions/artifacts/${art.id}.zip)`;
}
const {data: comments} = await github.rest.issues.listComments({repo, owner, issue_number});
// Match on the body too: octocov also comments as github-actions[bot].
const existing_comment = comments.find((c) => c.user.login === 'github-actions[bot]' && c.body.startsWith(header));
const existing_comment = comments.find((c) => c.user.login === 'github-actions[bot]');
if (existing_comment) {
core.info(`Updating comment ${existing_comment.id}`);
await github.rest.issues.updateComment({repo, owner, comment_id: existing_comment.id, body});
+7 -93
View File
@@ -68,16 +68,10 @@ jobs:
with:
go-version-file: go.mod
# Keep CI on the same version `make lint` installs, so a clean local run
# cannot turn red in CI just because a new golangci-lint was released.
- name: Resolve golangci-lint version
id: golangci-version
run: echo "version=$(grep '^GOLANGCI_LINT_VERSION' Makefile | cut -d ' ' -f 3)" >> "$GITHUB_OUTPUT"
- name: golangci-lint
uses: golangci/golangci-lint-action@v9
with:
version: ${{ steps.golangci-version.outputs.version }}
version: latest
problem-matchers: true
args: --timeout 2m
@@ -137,10 +131,8 @@ jobs:
- name: Download dependencies
run: go mod download
# Name must stay unique across the workflow: octocov matches step names
# by name across every job, and waits for each match to finish.
- name: Test with coverage
run: go test -shuffle=on -tags netgo,sqlite_fts5 -race -v -covermode=atomic -coverprofile=coverage.out $(go list ./... | grep -v '/plugins$')
- name: Test
run: go test -shuffle=on -tags netgo,sqlite_fts5 -race ./... -v
- name: Test ndpgen
run: |
@@ -149,84 +141,6 @@ jobs:
go build -o ndpgen .
./ndpgen --help
- name: Upload coverage profile
uses: actions/upload-artifact@v7
with:
name: octocov-go
path: coverage.out
if-no-files-found: error
go-plugins:
name: Test Go plugins
runs-on: ubuntu-latest
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v7
- uses: actions/setup-go@v6
id: setup-go
with:
go-version-file: go.mod
# Without this, the suite recompiles every test plugin WASM module,
# which dominates its runtime under -race.
- name: Cache the WASM compilation cache
uses: actions/cache@v6
with:
path: plugins/testdata/.wazero-cache
key: wazero-${{ runner.os }}-go${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('plugins/testdata/*/*.go', 'plugins/testdata/*/go.*', 'plugins/pdk/go/**/*.go', 'plugins/pdk/go/go.*') }}
restore-keys: wazero-${{ runner.os }}-
- name: Test plugins
run: go tool ginkgo -p -race -tags netgo,sqlite_fts5 --cover --covermode=atomic --coverprofile=coverage.out --output-dir=. ./plugins/
- name: Upload coverage profile
uses: actions/upload-artifact@v7
with:
name: octocov-plugins
path: coverage.out
if-no-files-found: error
coverage:
name: Report coverage
runs-on: ubuntu-latest
needs: [go, go-plugins]
permissions:
contents: read
actions: write
env:
COVERAGE_COMMENT: 'false'
steps:
- uses: actions/checkout@v7
- uses: actions/download-artifact@v8
with:
pattern: octocov-*
# Merge here rather than letting octocov do it: octocov reports statement
# coverage for a single profile, but switches to line counting for several.
- name: Merge coverage profiles
run: |
echo "mode: atomic" > coverage.out
awk 'FNR==1 && /^mode:/ {next} {k=$1" "$2; c[k]+=$3} END {for (k in c) print k, c[k]}' \
octocov-*/coverage.out | sort >> coverage.out
- uses: k1LoW/octocov-action@v1
- name: Save the PR number for the comment workflow
if: github.event_name == 'pull_request'
run: echo "${{ github.event.pull_request.number }}" > pr_number
- name: Upload the merged profile for the comment workflow
if: github.event_name == 'pull_request'
uses: actions/upload-artifact@v7
with:
name: octocov-pr
path: |
coverage.out
pr_number
if-no-files-found: error
go-windows:
name: Test Go code (Windows)
runs-on: windows-2022
@@ -293,12 +207,12 @@ jobs:
run: go test -shuffle=on -tags netgo,sqlite_fts5 ./... -v
- name: Test ndpgen
shell: bash
shell: pwsh
run: |
cd plugins/cmd/ndpgen
cd plugins\cmd\ndpgen
go test -shuffle=on -v
go build -o ndpgen.exe .
./ndpgen.exe --help
.\ndpgen.exe --help
js:
name: Test JS code
@@ -364,7 +278,7 @@ jobs:
build:
name: Build
needs: [js, go, go-plugins, go-windows, go-lint, i18n-lint, git-version, check-push-enabled, validate-migrations]
needs: [js, go, go-windows, go-lint, i18n-lint, git-version, check-push-enabled, validate-migrations]
strategy:
matrix:
platform: [ linux/amd64, linux/arm64, linux/arm/v5, linux/arm/v6, linux/arm/v7, linux/386, linux/riscv64, darwin/amd64, darwin/arm64, windows/amd64, windows/386 ]
-8
View File
@@ -40,11 +40,3 @@ openspec/
.agents
go.work*
.worktrees/
.playwright-mcp/
# Temp benchmark files
zz_*_test.go
# wazero compilation cache for the plugins test suite
/plugins/testdata/.wazero-cache/
/plugins/testdata/*.stage/
-4
View File
@@ -27,9 +27,6 @@ linters:
disable:
- staticcheck
settings:
errcheck:
exclude-functions:
- (*github.com/zeebo/xxh3.Hasher).Write
gocritic:
disable-all: true
enabled-checks:
@@ -72,7 +69,6 @@ linters:
- examples$
- node_modules
- _gen\.go$
- .worktrees
formatters:
exclusions:
generated: lax
-44
View File
@@ -1,44 +0,0 @@
# Code coverage reporting for pull requests. See https://github.com/k1LoW/octocov
# The 30s default is not enough: scanning this repo's artifacts for the baseline
# eats most of it, leaving none for the report upload.
timeout: 5m
coverage:
# A single pre-merged profile: octocov reports statements for one path, but
# switches to line counting when it merges several itself.
paths:
- coverage.out
# Not code under test: tests/ holds the mocks and helpers, *_gen.go is generated.
# Both patterns need the '**/' prefix: the comment workflow has no source tree,
# so octocov cannot shorten the profile's import paths to repo-relative ones.
exclude:
- '**/tests/**'
- '**/*_gen.go'
codeToTestRatio:
# Needs the pull request's own source, which the comment workflow must not
# check out: it holds a write token.
if: env.COVERAGE_COMMENT != 'true'
code:
- '**/*.go'
- '!**/*_test.go'
- '!**/*_gen.go'
test:
- '**/*_test.go'
testExecutionTime:
if: true
steps:
- Test with coverage
- Test plugins
diff:
datastores:
- artifact://${GITHUB_REPOSITORY}
comment:
# Only the 'Report coverage on PR' workflow sets this: a pull_request run from
# a fork gets a read-only token, so commenting from here 403s.
if: env.COVERAGE_COMMENT == 'true'
updatePrevious: true
summary:
if: true
report:
if: is_default_branch
datastores:
- artifact://${GITHUB_REPOSITORY}
+6 -39
View File
@@ -2,7 +2,7 @@ FROM --platform=$BUILDPLATFORM ghcr.io/crazy-max/osxcross:14.5-debian AS osxcros
########################################################################################################################
### Build xx (original image: tonistiigi/xx)
FROM --platform=$BUILDPLATFORM alpine:3.22 AS xx-build
FROM --platform=$BUILDPLATFORM public.ecr.aws/docker/library/alpine:3.20 AS xx-build
# v1.9.0
ENV XX_VERSION=a5592eab7a57895e8d385394ff12241bc65ecd50
@@ -26,7 +26,7 @@ COPY --from=xx-build /out/ /usr/bin/
########################################################################################################################
### Build Navidrome UI
FROM --platform=$BUILDPLATFORM node:lts-alpine AS ui
FROM --platform=$BUILDPLATFORM public.ecr.aws/docker/library/node:lts-alpine AS ui
WORKDIR /app
# Install node dependencies
@@ -43,7 +43,7 @@ COPY --from=ui /build /build
########################################################################################################################
### Build Navidrome binary for Docker image (dynamic musl, enables native libwebp via dlopen)
FROM --platform=$BUILDPLATFORM golang:1.27-alpine AS build-alpine
FROM --platform=$BUILDPLATFORM public.ecr.aws/docker/library/golang:1.26-alpine AS build-alpine
COPY --from=xx / /
ARG TARGETPLATFORM
@@ -85,7 +85,7 @@ EOT
########################################################################################################################
### Build Navidrome binary for standalone distribution (static glibc, cross-compiled)
FROM --platform=$BUILDPLATFORM golang:1.27-trixie AS base
FROM --platform=$BUILDPLATFORM public.ecr.aws/docker/library/golang:1.26-trixie AS base
RUN apt-get update && apt-get install -y clang lld
COPY --from=xx / /
WORKDIR /workspace
@@ -152,52 +152,19 @@ RUN xx-verify --static /out/navidrome*
FROM scratch AS binary
COPY --from=build /out /
########################################################################################################################
### Build no-op stubs for mpv's video-output libraries
# mpv links libEGL/libgbm for video output only; Navidrome drives it headless, for audio.
# Real mesa pulls in LLVM + gallium (+218MB uncompressed), so ship stubs it never calls.
FROM --platform=$BUILDPLATFORM alpine:3.22 AS mpv-stubs
COPY --from=xx / /
RUN apk add --no-cache clang lld binutils mesa-egl mesa-gbm
ARG TARGETPLATFORM
RUN xx-apk add --no-cache musl-dev
RUN <<EOT
set -e
mkdir -p /out
for so in libEGL.so.1 libgbm.so.1; do
readelf -sW /usr/lib/$so \
| awk '$5 == "GLOBAL" && $7 != "UND" { print $8 }' \
| sed 's/@.*//' \
| grep -vE '^(_init|_fini|_edata|_end|__bss_start|_GLOBAL_OFFSET_TABLE_)$' \
| sort -u \
| awk '{ print "void " $1 "(void) {}" }' > /tmp/stub.c
test -s /tmp/stub.c
xx-clang -shared -nostdlib -fPIC -Wl,-soname,$so -o /out/$so /tmp/stub.c
xx-verify /out/$so
done
EOT
########################################################################################################################
### Build Final Image
FROM alpine:3.22 AS final
FROM public.ecr.aws/docker/library/alpine:3.20 AS final
LABEL maintainer="deluan@navidrome.org"
LABEL org.opencontainers.image.source="https://github.com/navidrome/navidrome"
# Install runtime dependencies
# - libwebp + symlinks: enables native WebP encoding via purego/dlopen
# The mesa/LLVM stack mpv pulls in for video output is dropped in this same layer,
# otherwise the deleted bytes still ship in the image.
RUN apk add -U --no-cache ffmpeg mpv sqlite libwebp libwebpdemux libwebpmux && \
for lib in libwebp libwebpdemux libwebpmux; do \
target=$(ls /usr/lib/$lib.so.* 2>/dev/null | head -1) && \
[ -n "$target" ] && ln -sf "$target" /usr/lib/$lib.so; \
done && \
rm -rf /usr/lib/gallium-pipe /usr/lib/dri \
/usr/lib/libEGL.so* /usr/lib/libgbm.so* /usr/lib/libgallium*.so /usr/lib/libLLVM.so* \
/usr/lib/libGL.so* /usr/lib/libGLESv2.so* /usr/lib/libglapi.so*
COPY --from=mpv-stubs /out/ /usr/lib/
RUN mpv --no-video --ao=null --version > /dev/null
done
# Copy navidrome binary (musl build for Docker, enables native libwebp)
COPY --from=build-alpine /out/navidrome /app/
+1 -1
View File
@@ -20,7 +20,7 @@ IMAGE_PLATFORMS ?= $(shell echo $(SUPPORTED_PLATFORMS) | tr ',' '\n' | grep "lin
PLATFORMS ?= $(SUPPORTED_PLATFORMS)
DOCKER_TAG ?= deluan/navidrome:develop
GOLANGCI_LINT_VERSION ?= v2.13.2
GOLANGCI_LINT_VERSION ?= v2.12.0
UI_SRC_FILES := $(shell find ui -type f -not -path "ui/build/*" -not -path "ui/node_modules/*")
+11 -33
View File
@@ -13,26 +13,15 @@ import (
"strings"
"github.com/microcosm-cc/bluemonday"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/log"
)
const apiBaseURL = "https://api.deezer.com"
const authBaseURL = "https://auth.deezer.com"
// errCodeQuota is Deezer's "Quota limit exceeded"; it arrives in the body, with HTTP 200
// and no rate-limit headers, so the body code is the only signal.
const errCodeQuota = 4
type deezerError struct {
Type string `json:"type"`
Message string `json:"message"`
Code int `json:"code"`
}
func (e *deezerError) Error() string {
return fmt.Sprintf("deezer error(%d): %s", e.Code, e.Message)
}
var (
ErrNotFound = errors.New("deezer: not found")
)
type httpDoer interface {
Do(req *http.Request) (*http.Response, error)
@@ -67,7 +56,7 @@ func (c *client) searchArtists(ctx context.Context, name string, limit int) ([]A
}
if len(results.Data) == 0 {
return nil, agents.ErrNotFound
return nil, ErrNotFound
}
return results.Data, nil
}
@@ -85,31 +74,20 @@ func (c *client) makeRequest(req *http.Request, response any) error {
return err
}
// Checked before the status: a throttled request still answers 200, and decoding its body
// into a result type yields an empty one, which reads as "nothing found".
if err := parseBodyError(data); err != nil {
return err
}
if resp.StatusCode != 200 {
return fmt.Errorf("deezer http status: (%d)", resp.StatusCode)
return c.parseError(data)
}
return json.Unmarshal(data, response)
}
// parseBodyError returns the error Deezer reported in the body, or nil when it reported none.
func parseBodyError(data []byte) error {
var body errorResponse
// Discarded: a payload that is not an error object leaves Error nil, which is the "none" answer.
_ = json.Unmarshal(data, &body)
switch {
case body.Error == nil:
return nil
case body.Error.Code == errCodeQuota:
return errors.Join(body.Error, agents.ErrRetryLater)
default:
return body.Error
func (c *client) parseError(data []byte) error {
var deezerError Error
err := json.Unmarshal(data, &deezerError)
if err != nil {
return err
}
return fmt.Errorf("deezer error(%d): %s", deezerError.Error.Code, deezerError.Error.Message)
}
func (c *client) getRelatedArtists(ctx context.Context, artistID int) ([]Artist, error) {
+1 -33
View File
@@ -2,14 +2,12 @@ package deezer
import (
"bytes"
"errors"
"fmt"
"io"
"net/http"
"os"
"time"
"github.com/navidrome/navidrome/core/agents"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
@@ -43,37 +41,7 @@ var _ = Describe("client", func() {
})
_, err := client.searchArtists(GinkgoT().Context(), "Michael Jackson", 20)
Expect(err).To(MatchError(agents.ErrNotFound))
})
// Deezer answers 200 with no rate-limit headers when throttling, so this body is the only signal.
It("reports an exhausted quota as a retryable error, not as a missing artist", func() {
httpClient.mock("https://api.deezer.com/search/artist", http.Response{
StatusCode: 200,
Body: io.NopCloser(bytes.NewBufferString(
`{"error":{"type":"Exception","message":"Quota limit exceeded","code":4}}`)),
})
_, err := client.searchArtists(GinkgoT().Context(), "Michael Jackson", 20)
Expect(err).To(HaveOccurred())
Expect(err).ToNot(MatchError(agents.ErrNotFound),
"a throttled lookup would otherwise settle the artist as having no image")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
Expect(err.Error()).To(ContainSubstring("Quota limit exceeded"))
})
It("reports a non-quota body error as a plain error", func() {
httpClient.mock("https://api.deezer.com/search/artist", http.Response{
StatusCode: 200,
Body: io.NopCloser(bytes.NewBufferString(
`{"error":{"type":"Exception","message":"Invalid query","code":100}}`)),
})
_, err := client.searchArtists(GinkgoT().Context(), "Michael Jackson", 20)
Expect(err).To(HaveOccurred())
Expect(err).ToNot(MatchError(agents.ErrNotFound))
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeFalse(),
"only a throttle asks the caller to come back later")
Expect(err).To(MatchError(ErrNotFound))
})
})
+7 -2
View File
@@ -5,6 +5,7 @@ import (
"context"
"errors"
"fmt"
"net/http"
"slices"
"strings"
@@ -14,7 +15,6 @@ import (
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/cache"
"github.com/navidrome/navidrome/utils/httpclient"
"github.com/navidrome/navidrome/utils/slice"
)
@@ -36,7 +36,9 @@ func deezerConstructor(dataStore model.DataStore) agents.Interface {
dataStore: dataStore,
languages: conf.Server.Deezer.Languages,
}
httpClient := httpclient.New(consts.DefaultHttpClientTimeOut)
httpClient := &http.Client{
Timeout: consts.DefaultHttpClientTimeOut,
}
cachedHttpClient := cache.NewHTTPClient(httpClient, consts.DefaultHttpClientTimeOut)
agent.client = newClient(cachedHttpClient)
return agent
@@ -91,6 +93,9 @@ func isPlaceholderPicture(url string) bool {
func (s *deezerAgent) searchArtist(ctx context.Context, name string) (*Artist, error) {
artists, err := s.client.searchArtists(ctx, name, deezerArtistSearchLimit)
if errors.Is(err, ErrNotFound) || len(artists) == 0 {
return nil, agents.ErrNotFound
}
if err != nil {
return nil, err
}
-17
View File
@@ -3,7 +3,6 @@ package deezer
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"net/http"
@@ -81,22 +80,6 @@ var _ = Describe("deezerAgent", func() {
Expect(artist.ID).To(Equal(2))
})
// The artwork worker settles an artist as "no image" on agents.ErrNotFound, so a throttled
// lookup reaching that here would record a permanent absence.
It("surfaces an exhausted quota instead of reporting the artist as not found", func() {
httpClient.mock("https://api.deezer.com/search/artist", http.Response{
StatusCode: 200,
Body: io.NopCloser(bytes.NewBufferString(
`{"error":{"type":"Exception","message":"Quota limit exceeded","code":4}}`)),
})
_, err := agent.searchArtist(ctx, "Queen")
Expect(err).To(HaveOccurred())
Expect(err).ToNot(MatchError(agents.ErrNotFound))
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
})
It("returns ErrNotFound when no result matches the name exactly", func() {
httpClient.mock("https://api.deezer.com/search/artist", http.Response{
StatusCode: 200,
+6 -2
View File
@@ -22,8 +22,12 @@ type Artist struct {
Type string `json:"type"`
}
type errorResponse struct {
Error *deezerError `json:"error"`
type Error struct {
Error struct {
Type string `json:"type"`
Message string `json:"message"`
Code int `json:"code"`
} `json:"error"`
}
type RelatedArtists struct {
+1 -1
View File
@@ -26,7 +26,7 @@ var _ = Describe("Responses", func() {
Describe("Error", func() {
It("parses the error response correctly", func() {
var errorResp errorResponse
var errorResp Error
body := []byte(`{"error":{"type":"MissingParameterException","message":"Missing parameters: q","code":501}}`)
err := json.Unmarshal(body, &errorResp)
Expect(err).To(BeNil())
+20 -22
View File
@@ -18,7 +18,6 @@ import (
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/cache"
"github.com/navidrome/navidrome/utils/httpclient"
"golang.org/x/net/html"
)
@@ -60,7 +59,9 @@ func lastFMConstructor(ds model.DataStore) *lastfmAgent {
secret: conf.Server.LastFM.Secret,
sessionKeys: &agents.SessionKeys{DataStore: ds, KeyName: sessionKeyProperty},
}
hc := httpclient.New(consts.DefaultHttpClientTimeOut)
hc := &http.Client{
Timeout: consts.DefaultHttpClientTimeOut,
}
chc := cache.NewHTTPClient(hc, consts.DefaultHttpClientTimeOut)
l.httpClient = chc
l.client = newClient(l.apiKey, l.secret, chc)
@@ -92,7 +93,7 @@ func (l *lastfmAgent) GetAlbumInfo(ctx context.Context, name, artist, mbid strin
var resp agents.AlbumInfo
for _, lang := range l.languages {
var err error
a, err = l.callAlbumGetInfo(ctx, name, artist, lang)
a, err = l.callAlbumGetInfo(ctx, name, artist, mbid, lang)
if err != nil {
return nil, err
}
@@ -113,7 +114,7 @@ func (l *lastfmAgent) GetAlbumInfo(ctx context.Context, name, artist, mbid strin
}
func (l *lastfmAgent) GetAlbumImages(ctx context.Context, name, artist, mbid string) ([]agents.ExternalImage, error) {
a, err := l.callAlbumGetInfo(ctx, name, artist, l.languages[0])
a, err := l.callAlbumGetInfo(ctx, name, artist, mbid, l.languages[0])
if err != nil {
return nil, err
}
@@ -285,18 +286,22 @@ func (l *lastfmAgent) GetArtistImages(ctx context.Context, _, name, mbid string)
return res, nil
}
// callAlbumGetInfo matches on name+artist only. Last.fm's album.getInfo by MBID is unreliable —
// a correct MBID can return a different album (or none) — so the MBID is deliberately not passed.
func (l *lastfmAgent) callAlbumGetInfo(ctx context.Context, name, artist, lang string) (*Album, error) {
a, err := l.client.albumGetInfo(ctx, name, artist, "", lang)
func (l *lastfmAgent) callAlbumGetInfo(ctx context.Context, name, artist, mbid string, lang string) (*Album, error) {
a, err := l.client.albumGetInfo(ctx, name, artist, mbid, lang)
var lfErr *lastFMError
isLastFMError := errors.As(err, &lfErr)
if mbid != "" && (isLastFMError && lfErr.Code == 6) {
log.Debug(ctx, "LastFM/album.getInfo could not find album by mbid, trying again", "album", name, "mbid", mbid)
return l.callAlbumGetInfo(ctx, name, artist, "", lang)
}
if err != nil {
if lfErr, ok := errors.AsType[*lastFMError](err); ok && lfErr.Code == 6 {
// A not-found is a definitive absence, not a fault: return the shared sentinel so the
// artwork worker's breaker/transient checks don't retry it, and log it at Debug.
log.Debug(ctx, "Album not found in Last.fm", "album", name, "artist", artist)
return nil, agents.ErrNotFound
if isLastFMError && lfErr.Code == 6 {
log.Debug(ctx, "Album not found", "album", name, "mbid", mbid, err)
} else {
log.Error(ctx, "Error calling LastFM/album.getInfo", "album", name, "mbid", mbid, err)
}
log.Error(ctx, "Error calling LastFM/album.getInfo", "album", name, "artist", artist, err)
return nil, err
}
return a, nil
@@ -308,12 +313,6 @@ func (l *lastfmAgent) callArtistGetInfo(ctx context.Context, name string, lang s
a, err := l.client.artistGetInfo(ctx, name, lang)
if err != nil {
if lfErr, ok := errors.AsType[*lastFMError](err); ok && lfErr.Code == 6 {
// A not-found is a definitive absence, not a fault: return the shared sentinel so it
// doesn't trip the artwork worker's breaker, and log at Debug instead of Error.
log.Debug(ctx, "Artist not found in Last.fm", "artist", name)
return nil, agents.ErrNotFound
}
log.Error(ctx, "Error calling LastFM/artist.getInfo", "artist", name, err)
return nil, err
}
@@ -405,8 +404,7 @@ func (l *lastfmAgent) Scrobble(ctx context.Context, userId string, s scrobbler.S
log.Warn(ctx, "Last.fm client.scrobble returned error", "track", s.Title, err)
return errors.Join(err, scrobbler.ErrRetryLater)
}
// 11: service offline; 16: temporarily unavailable. Rate limiting is mapped by the client.
if lfErr.Code == 11 || lfErr.Code == 16 || errors.Is(err, scrobbler.ErrRetryLater) {
if lfErr.Code == 11 || lfErr.Code == 16 {
return errors.Join(err, scrobbler.ErrRetryLater)
}
return errors.Join(err, scrobbler.ErrUnrecoverable)
+14 -37
View File
@@ -100,15 +100,6 @@ var _ = Describe("lastfmAgent", func() {
Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("U2"))
})
It("returns ErrRetryLater on error 29 (rate limit exceeded)", func() {
httpClient.Res = http.Response{
Body: io.NopCloser(bytes.NewBufferString(`{"error":29,"message":"Rate limit exceeded"}`)),
StatusCode: 200,
}
_, err := agent.GetArtistBiography(ctx, "123", "U2", "")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
})
})
Describe("Language Fallback", func() {
@@ -506,16 +497,6 @@ var _ = Describe("lastfmAgent", func() {
Expect(err).To(MatchError(scrobbler.ErrRetryLater))
})
It("returns ErrRetryLater on error 29 (rate limit exceeded)", func() {
httpClient.Res = http.Response{
Body: io.NopCloser(bytes.NewBufferString(`{"error":29,"message":"Rate limit exceeded"}`)),
StatusCode: 200,
}
err := agent.Scrobble(ctx, "user-1", scrobbler.Scrobble{MediaFile: *track, TimeStamp: time.Now()})
Expect(errors.Is(err, scrobbler.ErrRetryLater)).To(BeTrue())
})
It("returns ErrRetryLater on http errors", func() {
httpClient.Res = http.Response{
Body: io.NopCloser(bytes.NewBufferString(`internal server error`)),
@@ -558,10 +539,7 @@ var _ = Describe("lastfmAgent", func() {
URL: "https://www.last.fm/music/Cher/Believe",
}))
Expect(httpClient.RequestCount).To(Equal(1))
// MBID is deliberately not sent — album.getInfo matches on name+artist only.
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty())
Expect(httpClient.SavedRequest.URL.Query().Get("album")).To(Equal("Believe"))
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("Cher"))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("03c91c40-49a6-44a7-90e7-a700edf97a62"))
})
It("returns empty images if no images are available", func() {
@@ -580,7 +558,7 @@ var _ = Describe("lastfmAgent", func() {
_, err := agent.GetAlbumInfo(ctx, "123", "U2", "mbid-1234")
Expect(err).To(HaveOccurred())
Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty())
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("mbid-1234"))
})
It("returns an error if Last.fm call returns an error", func() {
@@ -588,17 +566,23 @@ var _ = Describe("lastfmAgent", func() {
_, err := agent.GetAlbumInfo(ctx, "123", "U2", "mbid-1234")
Expect(err).To(HaveOccurred())
Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty())
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("mbid-1234"))
})
It("returns an error when Last.fm returns an error 6 (album not found)", func() {
It("returns an error if Last.fm call returns an error 6 and mbid is empty", func() {
httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError6)), StatusCode: 200}
_, err := agent.GetAlbumInfo(ctx, "123", "U2", "mbid-1234")
_, err := agent.GetAlbumInfo(ctx, "123", "U2", "")
Expect(err).To(HaveOccurred())
// A definitive not-found must satisfy the sentinel, or the artwork worker retries it.
Expect(errors.Is(err, agents.ErrNotFound)).To(BeTrue())
Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty())
})
Context("MBID non existent in Last.fm", func() {
It("calls again when last.fm returns an error 6", func() {
httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError6)), StatusCode: 200}
_, _ = agent.GetAlbumInfo(ctx, "123", "U2", "mbid-1234")
Expect(httpClient.RequestCount).To(Equal(2))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty())
})
})
})
@@ -629,13 +613,6 @@ var _ = Describe("lastfmAgent", func() {
Expect(images[0].URL).To(Equal("https://lastfm.freetls.fastly.net/i/u/ar0/818148bf682d429dc21b59a73ef6f68e.png"))
})
It("maps a Last.fm error 6 (artist not found) to the shared not-found sentinel", func() {
apiClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError6)), StatusCode: 200}
_, err := agent.GetArtistImages(ctx, "123", "Nonexistent Artist", "")
// Not a fault: runs of missing artists must not trip the worker's circuit breaker.
Expect(errors.Is(err, agents.ErrNotFound)).To(BeTrue())
})
It("returns empty list if image is the ignored default image", func() {
fApi, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json")
apiClient.Res = http.Response{Body: fApi, StatusCode: 200}
+3 -2
View File
@@ -18,7 +18,6 @@ import (
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/server"
"github.com/navidrome/navidrome/utils/httpclient"
"github.com/navidrome/navidrome/utils/req"
)
@@ -42,7 +41,9 @@ func NewRouter(ds model.DataStore) *Router {
sessionKeys: &agents.SessionKeys{DataStore: ds, KeyName: sessionKeyProperty},
}
r.Handler = r.routes()
hc := httpclient.New(consts.DefaultHttpClientTimeOut)
hc := &http.Client{
Timeout: consts.DefaultHttpClientTimeOut,
}
r.client = newClient(r.apiKey, r.secret, hc)
return r
}
-9
View File
@@ -214,14 +214,5 @@ var _ = Describe("auth_router", func() {
_, err = verifyLinkToken(nonExpiringToken)
Expect(err).To(MatchError("link token missing expiration"))
})
It("rejects a Jellyfin access token", func() {
usr := &model.User{ID: "u1", UserName: "johndoe"}
tokenStr, err := auth.CreateAPIToken(usr, auth.AudienceJellyfin)
Expect(err).ToNot(HaveOccurred())
_, err = verifyLinkToken(tokenStr)
Expect(err).To(HaveOccurred())
})
})
})
+1 -10
View File
@@ -5,7 +5,6 @@ import (
"crypto/md5"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/url"
@@ -15,15 +14,11 @@ import (
"strings"
"time"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/log"
)
const (
apiBaseUrl = "https://ws.audioscrobbler.com/2.0/"
// errCodeRateLimit is Last.fm's "rate limit exceeded"; it arrives in the body, with HTTP 200
// and no rate-limit headers, so the body code is the only signal.
errCodeRateLimit = 29
)
type lastFMError struct {
@@ -230,11 +225,7 @@ func (c *client) makeRequest(ctx context.Context, method string, params url.Valu
return nil, jsonErr
}
if response.Error != 0 {
var err error = &lastFMError{Code: response.Error, Message: response.Message}
if response.Error == errCodeRateLimit {
err = errors.Join(err, &agents.RetryLaterError{})
}
return &response, err
return &response, &lastFMError{Code: response.Error, Message: response.Message}
}
return &response, nil
+4 -2
View File
@@ -3,6 +3,7 @@ package listenbrainz
import (
"context"
"errors"
"net/http"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts"
@@ -11,7 +12,6 @@ import (
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/cache"
"github.com/navidrome/navidrome/utils/httpclient"
"github.com/navidrome/navidrome/utils/slice"
)
@@ -33,7 +33,9 @@ func listenBrainzConstructor(ds model.DataStore) *listenBrainzAgent {
sessionKeys: &agents.SessionKeys{DataStore: ds, KeyName: sessionKeyProperty},
baseURL: conf.Server.ListenBrainz.BaseURL,
}
hc := httpclient.New(consts.DefaultHttpClientTimeOut)
hc := &http.Client{
Timeout: consts.DefaultHttpClientTimeOut,
}
chc := cache.NewHTTPClient(hc, consts.DefaultHttpClientTimeOut)
l.client = newClient(l.baseURL, chc)
return l
-13
View File
@@ -164,19 +164,6 @@ var _ = Describe("listenBrainzAgent", func() {
err := agent.Scrobble(ctx, "user-1", sc)
Expect(err).To(MatchError(scrobbler.ErrUnrecoverable))
})
It("keeps a 429 scrobble for retry and carries the delay", func() {
httpClient.Res = http.Response{
StatusCode: 429,
Header: http.Header{"X-Ratelimit-Reset-In": []string{"7"}},
Body: io.NopCloser(bytes.NewBufferString(`{"code":429,"error":"rate limited"}`)),
}
err := agent.Scrobble(ctx, "user-1", scrobbler.Scrobble{MediaFile: *track, TimeStamp: time.Now()})
Expect(errors.Is(err, scrobbler.ErrRetryLater)).To(BeTrue())
retry, ok := errors.AsType[*agents.RetryLaterError](err)
Expect(ok).To(BeTrue())
Expect(retry.RetryIn).To(Equal(7 * time.Second))
})
})
Describe("GetArtistUrl", func() {
+3 -2
View File
@@ -16,7 +16,6 @@ import (
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/server"
"github.com/navidrome/navidrome/utils/httpclient"
)
type sessionKeysRepo interface {
@@ -38,7 +37,9 @@ func NewRouter(ds model.DataStore) *Router {
sessionKeys: &agents.SessionKeys{DataStore: ds, KeyName: sessionKeyProperty},
}
r.Handler = r.routes()
hc := httpclient.New(consts.DefaultHttpClientTimeOut)
hc := &http.Client{
Timeout: consts.DefaultHttpClientTimeOut,
}
r.client = newClient(conf.Server.ListenBrainz.BaseURL, hc)
return r
}
-17
View File
@@ -13,7 +13,6 @@ import (
"slices"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/log"
)
@@ -22,12 +21,6 @@ const (
labsBase = "https://labs.api.listenbrainz.org/"
)
// retryLaterErr reads the wait ListenBrainz asked for. It sends X-RateLimit-Reset-In
// (delta-seconds) on every response, including the 429, and never Retry-After.
func retryLaterErr(h http.Header) *agents.RetryLaterError {
return &agents.RetryLaterError{RetryIn: agents.ParseRetryIn(h.Get("X-RateLimit-Reset-In"))}
}
var (
ErrorNotFound = errors.New("listenbrainz: not found")
)
@@ -181,9 +174,6 @@ func (c *client) makeAuthenticatedRequest(ctx context.Context, method string, en
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusTooManyRequests {
return nil, retryLaterErr(resp.Header)
}
decoder := json.NewDecoder(resp.Body)
var response listenBrainzResponse
@@ -195,10 +185,6 @@ func (c *client) makeAuthenticatedRequest(ctx context.Context, method string, en
return nil, jsonErr
}
if response.Code != 0 && response.Code != 200 {
// LB also reports rate limiting as a body code, not only as an HTTP status.
if response.Code == http.StatusTooManyRequests {
return &response, retryLaterErr(resp.Header)
}
return &response, &listenBrainzError{Code: response.Code, Message: response.Error}
}
@@ -225,9 +211,6 @@ func (c *client) makeGenericRequest(ctx context.Context, method string, endpoint
// On a 200 code, there is no code. Decode using using error message if it exists
if resp.StatusCode != 200 {
defer resp.Body.Close()
if resp.StatusCode == http.StatusTooManyRequests {
return nil, retryLaterErr(resp.Header)
}
decoder := json.NewDecoder(resp.Body)
var lbzError lbzHttpError
-73
View File
@@ -4,17 +4,13 @@ import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strings"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
@@ -465,73 +461,4 @@ var _ = Describe("client", func() {
}))
})
})
Describe("rate limiting", func() {
It("returns RetryLaterError with the header delay on 429", func() {
httpClient.Res = http.Response{
StatusCode: 429,
Header: http.Header{"X-Ratelimit-Reset-In": []string{"3"}},
Body: io.NopCloser(strings.NewReader(`{"code":429,"error":"You have exceeded your rate limit."}`)),
}
_, err := client.validateToken(context.Background(), "token")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
retry, ok := errors.AsType[*agents.RetryLaterError](err)
Expect(ok).To(BeTrue())
Expect(retry.RetryIn).To(Equal(3 * time.Second))
})
It("returns RetryLaterError with zero delay when no header is present", func() {
httpClient.Res = http.Response{
StatusCode: 429,
Body: io.NopCloser(strings.NewReader(`{"code":429,"error":"rate limited"}`)),
}
_, err := client.validateToken(context.Background(), "token")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
retry, _ := errors.AsType[*agents.RetryLaterError](err)
Expect(retry.RetryIn).To(BeZero())
})
DescribeTable("caps absurd header values at one hour",
func(header string) {
httpClient.Res = http.Response{
StatusCode: 429,
Header: http.Header{"X-Ratelimit-Reset-In": []string{header}},
Body: io.NopCloser(strings.NewReader(`{"code":429,"error":"rate limited"}`)),
}
_, err := client.validateToken(context.Background(), "token")
retry, _ := errors.AsType[*agents.RetryLaterError](err)
Expect(retry.RetryIn).To(Equal(time.Hour))
},
Entry("a large value", "999999"),
Entry("a huge value", "99999999999"),
// Scaling this to nanoseconds before capping wraps past 2^64, landing on ~0.29s.
Entry("a value that overflows int64 nanoseconds", "18446744074"),
)
It("maps a body-level 429 sent with a non-429 status", func() {
httpClient.Res = http.Response{
StatusCode: 200,
Header: http.Header{"X-Ratelimit-Reset-In": []string{"7"}},
Body: io.NopCloser(strings.NewReader(`{"code":429,"error":"You have exceeded your rate limit."}`)),
}
_, err := client.validateToken(context.Background(), "token")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
retry, ok := errors.AsType[*agents.RetryLaterError](err)
Expect(ok).To(BeTrue())
Expect(retry.RetryIn).To(Equal(7 * time.Second))
})
It("returns RetryLaterError on a 429 from makeGenericRequest", func() {
httpClient.Res = http.Response{
StatusCode: 429,
Header: http.Header{"X-Ratelimit-Reset-In": []string{"5"}},
Body: io.NopCloser(strings.NewReader(`{"code":429,"error":"rate limited"}`)),
}
_, err := client.getArtistUrl(context.Background(), "1")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
retry, ok := errors.AsType[*agents.RetryLaterError](err)
Expect(ok).To(BeTrue())
Expect(retry.RetryIn).To(Equal(5 * time.Second))
})
})
})
-1022
View File
File diff suppressed because it is too large. Load diff
-1206
View File
File diff suppressed because it is too large. Load diff
+9 -12
View File
@@ -6,14 +6,13 @@ import (
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"strconv"
"strings"
"github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/core/artwork"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
@@ -142,16 +141,14 @@ func findPlaylist(ctx context.Context, ds model.DataStore, nameOrID string) *mod
func runExporter(ctx context.Context) {
ds, ctx := getAdminContext(ctx)
playlist := findPlaylist(ctx, ds, playlistID)
writePlaylist(playlist.ToM3U8(), os.Stdout, outputFile)
}
func writePlaylist(m3u string, out io.Writer, file string) {
if file == "" || file == "-" {
fmt.Fprint(out, m3u)
pls := playlist.ToM3U8()
if outputFile == "-" || outputFile == "" {
println(pls)
return
}
if err := os.WriteFile(file, []byte(m3u), 0600); err != nil {
log.Fatal("Error writing to the output file", "file", file, err)
err := os.WriteFile(outputFile, []byte(pls), 0600)
if err != nil {
log.Fatal("Error writing to the output file", "file", outputFile, err)
}
}
@@ -160,7 +157,7 @@ func runExport(ctx context.Context) {
if playlistID != "" && outputFile == "" {
playlist := findPlaylist(ctx, ds, playlistID)
writePlaylist(playlist.ToM3U8(), os.Stdout, outputFile)
println(playlist.ToM3U8())
return
}
@@ -263,7 +260,7 @@ func runImport(ctx context.Context, files []string) {
ctx = request.WithUser(ctx, *user)
}
pls := playlists.NewPlaylists(ds, artwork.NewUploader(ds))
pls := playlists.NewPlaylists(ds, core.NewImageUploadService(ds))
for _, file := range files {
absPath, err := filepath.Abs(file)
-35
View File
@@ -1,35 +0,0 @@
package cmd
import (
"fmt"
"os"
"path/filepath"
"strings"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("writePlaylist", func() {
const m3u = "#EXTM3U\n#PLAYLIST:DJ Wave\n#EXTINF:364,Bel Canto - Dreaming Girl\n"
plsFile := filepath.Join(os.TempDir(), fmt.Sprintf("navidrome-pls-%d.m3u8", os.Getpid()))
BeforeEach(func() {
DeferCleanup(func() { _ = os.Remove(plsFile) })
})
DescribeTable("writes the playlist to exactly one destination",
func(file, wantStream, wantFile string) {
var out strings.Builder
writePlaylist(m3u, &out, file)
written, _ := os.ReadFile(plsFile)
Expect(out.String()).To(Equal(wantStream))
Expect(string(written)).To(Equal(wantFile))
},
Entry("no file name writes to the stream", "", m3u, ""),
Entry("a dash writes to the stream", "-", m3u, ""),
Entry("a path writes to the file", plsFile, "", m3u),
)
})
+2 -1
View File
@@ -9,6 +9,7 @@ import (
"os"
"strconv"
"strings"
"text/tabwriter"
"time"
"github.com/navidrome/navidrome/conf"
@@ -313,7 +314,7 @@ func formatPluginList(list model.Plugins, format string) (string, error) {
return sb.String(), w.Error()
case "table":
var sb strings.Builder
w := newTabWriter(&sb)
w := tabwriter.NewWriter(&sb, 0, 4, 2, ' ', 0)
fmt.Fprintln(w, "ID\tNAME\tVERSION\tENABLED\tLAST ERROR")
for _, p := range list {
name, version := manifestSummary(p)
+25 -24
View File
@@ -2,7 +2,6 @@ package cmd
import (
"context"
"net/http"
"os"
"os/signal"
"strings"
@@ -139,7 +138,7 @@ func startServer(ctx context.Context) func() error {
a.MountRouter("Prometheus metrics", conf.Server.Prometheus.MetricsPath, p.GetHandler())
}
if conf.Server.DevEnableProfiler {
a.MountRouter("Profiling", "/debug", profilerHandler())
a.MountRouter("Profiling", "/debug", middleware.Profiler())
}
if strings.HasPrefix(conf.Server.UILoginBackgroundURL, "/") {
a.MountRouter("Background images", conf.Server.UILoginBackgroundURL, backgrounds.NewHandler())
@@ -148,14 +147,6 @@ func startServer(ctx context.Context) func() error {
}
}
// profilerHandler returns the pprof handler. net/http/pprof resolves the profile
// name from the raw request path, so the BasePath has to come off first.
func profilerHandler() http.Handler {
// A trailing or root slash would make StripPrefix drop the leading slash chi needs.
basePath := strings.TrimRight(conf.Server.BasePath, "/")
return http.StripPrefix(basePath, middleware.Profiler())
}
// schedulePeriodicScan schedules a periodic scan of the music library, if configured.
func schedulePeriodicScan(ctx context.Context) func() error {
return func() error {
@@ -366,18 +357,19 @@ func startArtworkWorker(ctx context.Context, worker *artwork.Worker) func() erro
}
}
// scheduleArtworkHousekeeping registers the recurring missing-state and prune jobs, and
// reports an artwork config change without acting on it.
// scheduleArtworkHousekeeping runs the startup fingerprint backfill and registers the
// recurring stale-absent recheck and prune jobs. Scan-triggered prune lands in a later phase.
func scheduleArtworkHousekeeping(ctx context.Context, worker *artwork.Worker) func() error {
return func() error {
ds := CreateDataStore()
schedulerInstance := scheduler.GetInstance()
if _, err := schedulerInstance.Add(consts.ArtworkEnqueueMissingSchedule, func() {
if err := worker.EnqueueMissingAll(ctx); err != nil {
log.Error(ctx, "Error enqueueing missing artwork rechecks", err)
if _, err := schedulerInstance.Add(consts.ArtworkStaleAbsentRecheckSchedule, func() {
if err := artwork.EnqueueStaleAbsentAll(ctx, ds); err != nil {
log.Error(ctx, "Error enqueueing stale artwork rechecks", err)
}
}); err != nil {
log.Error(ctx, "Error scheduling artwork missing-state recheck", err)
log.Error(ctx, "Error scheduling artwork stale-absent recheck", err)
}
if _, err := schedulerInstance.Add(consts.ArtworkPruneSchedule, func() {
@@ -388,14 +380,23 @@ func scheduleArtworkHousekeeping(ctx context.Context, worker *artwork.Worker) fu
log.Error(ctx, "Error scheduling artwork prune", err)
}
// Also run the missing-row recheck once at startup so a never-scanned entity is picked up
// immediately, not only on the next hourly tick (e.g. after enabling the feature).
if err := worker.EnqueueMissingAll(ctx); err != nil {
log.Error(ctx, "Error enqueueing missing artwork rechecks", err)
backfilled, err := artwork.Backfill(ctx, ds)
if err != nil {
log.Error(ctx, "Error running artwork backfill", err)
return nil
}
if err := worker.ReconcileConfig(ctx); err != nil {
log.Error(ctx, "Error checking the artwork config fingerprint", err)
if !backfilled {
return nil
}
log.Info(ctx, "Artwork backfill enqueued, scheduling a follow-up prune")
timer := time.NewTimer(consts.ArtworkPostBackfillPruneDelay)
defer timer.Stop()
select {
case <-timer.C:
if err := worker.RunPrune(ctx); err != nil {
log.Error(ctx, "Error running post-backfill artwork prune", err)
}
case <-ctx.Done():
}
return nil
}
@@ -451,7 +452,7 @@ func init() {
rootCmd.Flags().String("albumplaycountmode", viper.GetString("albumplaycountmode"), "how to compute playcount for albums. absolute (default) or normalized")
rootCmd.Flags().Bool("autoimportplaylists", viper.GetBool("autoimportplaylists"), "enable/disable .m3u playlist auto-import`")
rootCmd.Flags().Bool("prometheus.enabled", viper.GetBool("prometheus.enabled"), "enable/disable prometheus metrics endpoint")
rootCmd.Flags().Bool("prometheus.enabled", viper.GetBool("prometheus.enabled"), "enable/disable prometheus metrics endpoint`")
rootCmd.Flags().String("prometheus.metricspath", viper.GetString("prometheus.metricspath"), "http endpoint for prometheus metrics")
_ = viper.BindPFlag("address", rootCmd.Flags().Lookup("address"))
-46
View File
@@ -1,46 +0,0 @@
package cmd
import (
"net/http"
"net/http/httptest"
"path"
"runtime/pprof"
"github.com/go-chi/chi/v5"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = pprof.NewProfile("nd-profiler-test")
var _ = Describe("profilerHandler", func() {
// Mirrors how server.MountRouter mounts the handler.
mount := func() http.Handler {
router := chi.NewRouter()
router.Mount(path.Join(conf.Server.BasePath, "/debug"), profilerHandler())
return router
}
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
})
DescribeTable("serves a named profile",
func(basePath string) {
conf.Server.BasePath = basePath
w := httptest.NewRecorder()
target := path.Join(basePath, "/debug/pprof/nd-profiler-test") + "?debug=1"
mount().ServeHTTP(w, httptest.NewRequest(http.MethodGet, target, nil))
Expect(w.Code).To(Equal(http.StatusOK))
Expect(w.Body.String()).To(HavePrefix("nd-profiler-test profile: total 0"))
},
Entry("without a BasePath", ""),
Entry("with a BasePath", "/music"),
Entry("with a root BasePath", "/"),
Entry("with a trailing-slash BasePath", "/music/"),
)
})
+2 -2
View File
@@ -9,7 +9,7 @@ import (
"os"
"strings"
"github.com/navidrome/navidrome/core/artwork"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/db"
"github.com/navidrome/navidrome/log"
@@ -82,7 +82,7 @@ func runScanner(ctx context.Context) {
sqlDB := db.Db()
defer db.Db().Close()
ds := persistence.New(sqlDB)
pls := playlists.NewPlaylists(ds, artwork.NewUploader(ds))
pls := playlists.NewPlaylists(ds, core.NewImageUploadService(ds))
// Parse targets from command line or file
var scanTargets []model.ScanTarget
-7
View File
@@ -4,8 +4,6 @@ import (
"context"
"errors"
"fmt"
"io"
"text/tabwriter"
"github.com/navidrome/navidrome/core/auth"
"github.com/navidrome/navidrome/db"
@@ -15,11 +13,6 @@ import (
"github.com/navidrome/navidrome/persistence"
)
// newTabWriter keeps every CLI table on the same column settings.
func newTabWriter(out io.Writer) *tabwriter.Writer {
return tabwriter.NewWriter(out, 0, 4, 2, ' ', 0)
}
func getAdminContext(ctx context.Context) (model.DataStore, context.Context) {
sqlDB := db.Db()
ds := persistence.New(sqlDB)
+24 -39
View File
@@ -65,8 +65,8 @@ func CreateNativeAPIRouter(ctx context.Context) *nativeapi.Router {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
share := core.NewShare(dataStore)
uploader := artwork.NewUploader(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, uploader)
imageUploadService := core.NewImageUploadService(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, imageUploadService)
insights := metrics.GetInstance(dataStore)
broker := events.GetBroker()
metricsMetrics := metrics.GetPrometheusInstance(dataStore)
@@ -76,10 +76,7 @@ func CreateNativeAPIRouter(ctx context.Context) *nativeapi.Router {
library := core.NewLibrary(dataStore, modelScanner, watcher, broker, manager)
user := core.NewUser(dataStore, manager)
maintenance := core.NewMaintenance(dataStore)
agentsAgents := agents.GetAgents(dataStore, manager)
matcherMatcher := matcher.New(dataStore)
provider := external.NewProvider(dataStore, agentsAgents, matcherMatcher, broker)
router := nativeapi.New(dataStore, share, playlistsPlaylists, insights, library, user, maintenance, manager, uploader, provider)
router := nativeapi.New(dataStore, share, playlistsPlaylists, insights, library, user, maintenance, manager, imageUploadService)
return router
}
@@ -87,9 +84,9 @@ func CreateSubsonicAPIRouter(ctx context.Context) *subsonic.Router {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
fileCache := artwork.GetImageCache()
imageStore := artwork.GetImageStore()
imageStore := artwork.ProvideImageStore()
fFmpeg := ffmpeg.New()
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, imageStore, fFmpeg)
service := artwork.NewService(dataStore, fileCache, imageStore, fFmpeg)
transcodingCache := stream.GetTranscodingCache()
mediaStreamer := stream.NewMediaStreamer(dataStore, fFmpeg, transcodingCache)
share := core.NewShare(dataStore)
@@ -100,16 +97,16 @@ func CreateSubsonicAPIRouter(ctx context.Context) *subsonic.Router {
manager := plugins.GetManager(dataStore, broker, metricsMetrics)
agentsAgents := agents.GetAgents(dataStore, manager)
matcherMatcher := matcher.New(dataStore)
provider := external.NewProvider(dataStore, agentsAgents, matcherMatcher, broker)
uploader := artwork.NewUploader(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, uploader)
provider := external.NewProvider(dataStore, agentsAgents, matcherMatcher)
imageUploadService := core.NewImageUploadService(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, imageUploadService)
modelScanner := scanner.New(ctx, dataStore, broker, playlistsPlaylists, metricsMetrics)
playTracker := scrobbler.GetPlayTracker(dataStore, broker, manager)
playbackServer := playback.GetInstance(dataStore)
lyricsLyrics := lyrics.NewLyrics(dataStore, manager)
transcodeDecider := stream.NewTranscodeDecider(dataStore, fFmpeg)
sonicSonic := sonic.New(dataStore, manager, matcherMatcher)
router := subsonic.New(dataStore, artworkArtwork, mediaStreamer, archiver, players, provider, modelScanner, broker, playlistsPlaylists, playTracker, share, playbackServer, metricsMetrics, lyricsLyrics, transcodeDecider, sonicSonic)
router := subsonic.New(dataStore, service, mediaStreamer, archiver, players, provider, modelScanner, broker, playlistsPlaylists, playTracker, share, playbackServer, metricsMetrics, lyricsLyrics, transcodeDecider, sonicSonic)
return router
}
@@ -117,9 +114,9 @@ func CreateJellyfinAPIRouter(ctx context.Context) *jellyfin.Router {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
fileCache := artwork.GetImageCache()
imageStore := artwork.GetImageStore()
imageStore := artwork.ProvideImageStore()
fFmpeg := ffmpeg.New()
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, imageStore, fFmpeg)
service := artwork.NewService(dataStore, fileCache, imageStore, fFmpeg)
transcodingCache := stream.GetTranscodingCache()
mediaStreamer := stream.NewMediaStreamer(dataStore, fFmpeg, transcodingCache)
transcodeDecider := stream.NewTranscodeDecider(dataStore, fFmpeg)
@@ -128,14 +125,14 @@ func CreateJellyfinAPIRouter(ctx context.Context) *jellyfin.Router {
metricsMetrics := metrics.GetPrometheusInstance(dataStore)
manager := plugins.GetManager(dataStore, broker, metricsMetrics)
playTracker := scrobbler.GetPlayTracker(dataStore, broker, manager)
uploader := artwork.NewUploader(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, uploader)
imageUploadService := core.NewImageUploadService(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, imageUploadService)
agentsAgents := agents.GetAgents(dataStore, manager)
matcherMatcher := matcher.New(dataStore)
provider := external.NewProvider(dataStore, agentsAgents, matcherMatcher, broker)
provider := external.NewProvider(dataStore, agentsAgents, matcherMatcher)
sonicSonic := sonic.New(dataStore, manager, matcherMatcher)
lyricsLyrics := lyrics.NewLyrics(dataStore, manager)
router := jellyfin.New(dataStore, artworkArtwork, mediaStreamer, transcodeDecider, players, playTracker, playlistsPlaylists, provider, sonicSonic, lyricsLyrics, broker)
router := jellyfin.New(dataStore, service, mediaStreamer, transcodeDecider, players, playTracker, playlistsPlaylists, provider, sonicSonic, lyricsLyrics, broker)
return router
}
@@ -143,14 +140,14 @@ func CreatePublicRouter() *public.Router {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
fileCache := artwork.GetImageCache()
imageStore := artwork.GetImageStore()
imageStore := artwork.ProvideImageStore()
fFmpeg := ffmpeg.New()
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, imageStore, fFmpeg)
service := artwork.NewService(dataStore, fileCache, imageStore, fFmpeg)
transcodingCache := stream.GetTranscodingCache()
mediaStreamer := stream.NewMediaStreamer(dataStore, fFmpeg, transcodingCache)
share := core.NewShare(dataStore)
archiver := core.NewArchiver(mediaStreamer, dataStore, share)
router := public.New(dataStore, artworkArtwork, mediaStreamer, share, archiver)
router := public.New(dataStore, service, mediaStreamer, share, archiver)
return router
}
@@ -186,8 +183,8 @@ func CreateScanner(ctx context.Context) model.Scanner {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
broker := events.GetBroker()
uploader := artwork.NewUploader(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, uploader)
imageUploadService := core.NewImageUploadService(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, imageUploadService)
metricsMetrics := metrics.GetPrometheusInstance(dataStore)
modelScanner := scanner.New(ctx, dataStore, broker, playlistsPlaylists, metricsMetrics)
return modelScanner
@@ -197,8 +194,8 @@ func CreateScanWatcher(ctx context.Context) scanner.Watcher {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
broker := events.GetBroker()
uploader := artwork.NewUploader(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, uploader)
imageUploadService := core.NewImageUploadService(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, imageUploadService)
metricsMetrics := metrics.GetPrometheusInstance(dataStore)
modelScanner := scanner.New(ctx, dataStore, broker, playlistsPlaylists, metricsMetrics)
watcher := scanner.GetWatcher(dataStore, modelScanner)
@@ -215,7 +212,7 @@ func GetPlaybackServer() playback.PlaybackServer {
func CreateArtworkWorker() *artwork.Worker {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
imageStore := artwork.GetImageStore()
imageStore := artwork.ProvideImageStore()
broker := events.GetBroker()
metricsMetrics := metrics.GetPrometheusInstance(dataStore)
manager := plugins.GetManager(dataStore, broker, metricsMetrics)
@@ -226,18 +223,6 @@ func CreateArtworkWorker() *artwork.Worker {
return worker
}
func CreateArtworkResolver(trace *artwork.ChainTrace, live bool) *artwork.TracingResolver {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
broker := events.GetBroker()
metricsMetrics := metrics.GetPrometheusInstance(dataStore)
manager := plugins.GetManager(dataStore, broker, metricsMetrics)
agentsAgents := agents.GetAgents(dataStore, manager)
fFmpeg := ffmpeg.New()
tracingResolver := artwork.NewTracingResolver(dataStore, agentsAgents, fFmpeg, trace, live)
return tracingResolver
}
func getPluginManager() *plugins.Manager {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
@@ -249,7 +234,7 @@ func getPluginManager() *plugins.Manager {
// wire_injectors.go:
var allProviders = wire.NewSet(core.Set, artwork.Set, server.New, subsonic.New, jellyfin.New, nativeapi.New, public.New, persistence.New, lastfm.NewRouter, listenbrainz.NewRouter, events.GetBroker, scanner.New, scanner.GetWatcher, metrics.GetPrometheusInstance, db.Db, plugins.GetManager, sonic.New, wire.Bind(new(agents.PluginLoader), new(*plugins.Manager)), wire.Bind(new(scrobbler.PluginLoader), new(*plugins.Manager)), wire.Bind(new(lyrics.PluginLoader), new(*plugins.Manager)), wire.Bind(new(sonic.PluginLoader), new(*plugins.Manager)), wire.Bind(new(sonic.Engine), new(*sonic.Sonic)), wire.Bind(new(nativeapi.PluginManager), new(*plugins.Manager)), wire.Bind(new(core.PluginUnloader), new(*plugins.Manager)), wire.Bind(new(plugins.PluginMetricsRecorder), new(metrics.Metrics)), wire.Bind(new(core.Watcher), new(scanner.Watcher)), wire.Bind(new(playlists.ImageUploadService), new(artwork.Uploader)))
var allProviders = wire.NewSet(core.Set, artwork.Set, server.New, subsonic.New, jellyfin.New, nativeapi.New, public.New, persistence.New, lastfm.NewRouter, listenbrainz.NewRouter, events.GetBroker, scanner.New, scanner.GetWatcher, metrics.GetPrometheusInstance, db.Db, plugins.GetManager, sonic.New, wire.Bind(new(agents.PluginLoader), new(*plugins.Manager)), wire.Bind(new(scrobbler.PluginLoader), new(*plugins.Manager)), wire.Bind(new(lyrics.PluginLoader), new(*plugins.Manager)), wire.Bind(new(sonic.PluginLoader), new(*plugins.Manager)), wire.Bind(new(sonic.Engine), new(*sonic.Sonic)), wire.Bind(new(nativeapi.PluginManager), new(*plugins.Manager)), wire.Bind(new(core.PluginUnloader), new(*plugins.Manager)), wire.Bind(new(plugins.PluginMetricsRecorder), new(metrics.Metrics)), wire.Bind(new(core.Watcher), new(scanner.Watcher)))
func GetPluginManager(ctx context.Context) *plugins.Manager {
manager := getPluginManager()
-9
View File
@@ -14,7 +14,6 @@ import (
"github.com/navidrome/navidrome/core/lyrics"
"github.com/navidrome/navidrome/core/metrics"
"github.com/navidrome/navidrome/core/playback"
"github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/core/scrobbler"
"github.com/navidrome/navidrome/core/sonic"
"github.com/navidrome/navidrome/db"
@@ -57,7 +56,6 @@ var allProviders = wire.NewSet(
wire.Bind(new(core.PluginUnloader), new(*plugins.Manager)),
wire.Bind(new(plugins.PluginMetricsRecorder), new(metrics.Metrics)),
wire.Bind(new(core.Watcher), new(scanner.Watcher)),
wire.Bind(new(playlists.ImageUploadService), new(artwork.Uploader)),
)
func CreateDataStore() model.DataStore {
@@ -144,13 +142,6 @@ func CreateArtworkWorker() *artwork.Worker {
))
}
func CreateArtworkResolver(trace *artwork.ChainTrace, live bool) *artwork.TracingResolver {
panic(wire.Build(
allProviders,
artwork.NewTracingResolver,
))
}
func getPluginManager() *plugins.Manager {
panic(wire.Build(
allProviders,
+49 -272
View File
@@ -2,19 +2,14 @@ package conf
import (
"cmp"
"encoding"
"encoding/json"
"fmt"
"math"
"net/url"
"os"
"path/filepath"
"reflect"
"regexp"
"runtime"
"slices"
"strings"
"sync"
"time"
"github.com/bmatcuk/doublestar/v4"
@@ -26,12 +21,11 @@ import (
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/scheduler"
"github.com/navidrome/navidrome/utils/run"
"github.com/navidrome/navidrome/utils/slice"
"github.com/spf13/viper"
)
type configOptions struct {
ConfigFile string `conf:"-"`
ConfigFile string
Address string
Port int
UnixSocketPerm string
@@ -63,6 +57,8 @@ type configOptions struct {
ImageCacheSize string
AlbumPlayCountMode string
EnableArtworkPrecache bool
ArtworkWorkerConcurrency int
ArtworkExternalMaxRPS int
AutoImportPlaylists bool
DefaultPlaylistPublicVisibility bool
PlaylistsPath string
@@ -73,7 +69,6 @@ type configOptions struct {
Matcher matcherOptions `json:",omitzero"`
RecentlyAddedByModTime bool
PreferSortTags bool
EnableNaturalSorting bool
IgnoredArticles string
IndexGroups string
FFmpegPath string
@@ -92,7 +87,6 @@ type configOptions struct {
EnableUserEditing bool
EnableArtworkUpload bool
MaxImageUploadSize string
MaxImageSize string
EnableSharing bool
ShareURL string
DefaultShareExpiration time.Duration
@@ -147,8 +141,6 @@ type configOptions struct {
DevArtworkThrottleBacklogLimit int
DevArtworkThrottleBacklogTimeout time.Duration
DevArtworkThrottleBuffered bool
DevArtworkWorkerConcurrency int
DevArtworkExternalMaxRPS int
DevArtistInfoTimeToLive time.Duration
DevAlbumInfoTimeToLive time.Duration
DevExternalScanner bool
@@ -211,7 +203,7 @@ type lastfmOptions struct {
ScrobbleFirstArtistOnly bool
// Computed values
Languages []string `conf:"-"` // Computed from Language, split by comma
Languages []string // Computed from Language, split by comma
}
type deezerOptions struct {
@@ -219,7 +211,7 @@ type deezerOptions struct {
Language string
// Computed values
Languages []string `conf:"-"` // Computed from Language, split by comma
Languages []string // Computed from Language, split by comma
}
type listenBrainzOptions struct {
@@ -315,12 +307,6 @@ var currentGOOS = func() string {
return runtime.GOOS
}
// TLSEnabled reports whether the server serves HTTPS. Both halves are required,
// so callers cannot infer it from the certificate alone.
func (c *configOptions) TLSEnabled() bool {
return c.TLSCert != "" && c.TLSKey != ""
}
var (
Server = &configOptions{}
hooks []func()
@@ -351,23 +337,19 @@ func LoadFromFile(confFile string) {
Load(true)
}
func durationNonNegativeOrDefault(val *time.Duration, original time.Duration) {
if val.Nanoseconds() < 0 {
log.Warn("Duration is a negative value. Using default value", "value", *val, "default", original)
*val = original
}
}
func Load(noConfigDump bool) {
parseIniFileConfiguration()
remapEnvVarKeysFromConfig()
// Map deprecated options to their new names for backwards compatibility
for _, o := range deprecatedOptions {
if o.replacement != "" {
mapDeprecatedOption(o.name, o.replacement)
}
}
mapDeprecatedOption("ReverseProxyWhitelist", "ExtAuth.TrustedSources")
mapDeprecatedOption("ReverseProxyUserHeader", "ExtAuth.UserHeader")
mapDeprecatedOption("HTTPSecurityHeaders.CustomFrameOptionsValue", "HTTPHeaders.FrameOptions")
mapDeprecatedOption("CoverJpegQuality", "CoverArtQuality")
mapDeprecatedOption("SimilarSongsMatchThreshold", "Matcher.FuzzyThreshold")
mapDeprecatedOption("EnableTranscodingCancellation", "Transcoding.EnableCancellation")
mapDeprecatedOption("DevArtworkWorkerConcurrency", "ArtworkWorkerConcurrency")
mapDeprecatedOption("DevArtworkExternalRPS", "ArtworkExternalMaxRPS")
err := viper.Unmarshal(&Server, viper.DecodeHook(
mapstructure.ComposeDecodeHookFunc(
@@ -425,34 +407,12 @@ func Load(noConfigDump bool) {
log.SetLogSourceLine(Server.DevLogSourceLine)
log.SetRedacting(Server.EnableLogRedacting)
durationNonNegativeOrDefault(&Server.SessionTimeout, consts.DefaultSessionTimeout)
durationNonNegativeOrDefault(&Server.SmartPlaylistRefreshDelay, consts.DefaultSmartRefresh)
durationNonNegativeOrDefault(&Server.DefaultShareExpiration, consts.DefaultShareExpiration)
durationNonNegativeOrDefault(&Server.UIPlaybackReportInterval, consts.DefaultUIPlaybackReportInterval)
durationNonNegativeOrDefault(&Server.AuthWindowLength, consts.DefaultAuthWindowLength)
durationNonNegativeOrDefault(&Server.Scanner.WatcherWait, consts.DefaultWatcherWait)
durationNonNegativeOrDefault(&Server.DevActivityPanelUpdateRate, consts.DefaultActivityPanelUpdateRate)
durationNonNegativeOrDefault(&Server.DevArtworkThrottleBacklogTimeout, consts.RequestThrottleBacklogTimeout)
durationNonNegativeOrDefault(&Server.DevArtistInfoTimeToLive, consts.ArtistInfoTimeToLive)
durationNonNegativeOrDefault(&Server.DevAlbumInfoTimeToLive, consts.AlbumInfoTimeToLive)
durationNonNegativeOrDefault(&Server.DevInsightsInitialDelay, consts.InsightsInitialDelay)
durationNonNegativeOrDefault(&Server.DevPluginCompilationTimeout, consts.DefaultPluginCompilationTimeout)
// Log deprecated, removed and unknown options
for _, o := range deprecatedOptions {
logDeprecatedOptions(o.name, o.replacement)
}
logRemovedOptions(removedOptions...)
logUnknownOptions()
err = run.Sequentially(
validateScanSchedule,
validateBackupSchedule,
validatePlaylistsPath,
validatePurgeMissingOption,
validateByteSize("MaxImageUploadSize", Server.MaxImageUploadSize),
validateByteSize("MaxImageSize", Server.MaxImageSize),
validateMaxImageUploadSize,
validateURL("ExtAuth.LogoutURL", Server.ExtAuth.LogoutURL),
)
if err != nil {
@@ -505,6 +465,21 @@ func Load(noConfigDump bool) {
// Parse Deezer.Language into Languages slice (comma-separated, with fallback to DefaultInfoLanguage)
Server.Deezer.Languages = parseLanguages(Server.Deezer.Language)
// Deprecated options
logDeprecatedOptions("Scanner.GenreSeparators", "")
logDeprecatedOptions("Scanner.GroupAlbumReleases", "")
logDeprecatedOptions("DevEnableBufferedScrobble", "") // Deprecated: Buffered scrobbling is now always enabled and this option is ignored
logDeprecatedOptions("SearchFullString", "Search.FullString")
logDeprecatedOptions("ReverseProxyWhitelist", "ExtAuth.TrustedSources")
logDeprecatedOptions("ReverseProxyUserHeader", "ExtAuth.UserHeader")
logDeprecatedOptions("HTTPSecurityHeaders.CustomFrameOptionsValue", "HTTPHeaders.FrameOptions")
logDeprecatedOptions("CoverJpegQuality", "CoverArtQuality")
logDeprecatedOptions("SimilarSongsMatchThreshold", "Matcher.FuzzyThreshold")
logDeprecatedOptions("EnableTranscodingCancellation", "Transcoding.EnableCancellation")
// Removed options
logRemovedOptions("Spotify.ID", "Spotify.Secret")
// Validate other options
if Server.UICoverArtSize < 200 || Server.UICoverArtSize > 1200 {
newValue := max(200, min(1200, Server.UICoverArtSize))
@@ -512,40 +487,15 @@ func Load(noConfigDump bool) {
Server.UICoverArtSize = newValue
}
// Floor MaxImageSize at MaxImageUploadSize so accepted uploads can always be read back.
imgSize, _ := humanize.ParseBytes(Server.MaxImageSize)
uploadSize, _ := humanize.ParseBytes(Server.MaxImageUploadSize)
if imgSize < uploadSize {
log.Warn("MaxImageSize must be at least MaxImageUploadSize, raising", "value", Server.MaxImageSize, "newValue", Server.MaxImageUploadSize)
Server.MaxImageSize = Server.MaxImageUploadSize
}
// Call init hooks
for _, hook := range hooks {
hook()
}
}
// deprecatedOptions still work, but will be removed in a future release. An empty
// replacement means the option is now ignored.
var deprecatedOptions = []struct{ name, replacement string }{
{"Scanner.GenreSeparators", ""},
{"Scanner.GroupAlbumReleases", ""},
{"DevEnableBufferedScrobble", ""},
{"SearchFullString", "Search.FullString"},
{"ReverseProxyWhitelist", "ExtAuth.TrustedSources"},
{"ReverseProxyUserHeader", "ExtAuth.UserHeader"},
{"HTTPSecurityHeaders.CustomFrameOptionsValue", "HTTPHeaders.FrameOptions"},
{"CoverJpegQuality", "CoverArtQuality"},
{"SimilarSongsMatchThreshold", "Matcher.FuzzyThreshold"},
{"EnableTranscodingCancellation", "Transcoding.EnableCancellation"},
}
var removedOptions = []string{"Spotify.ID", "Spotify.Secret"}
func logDeprecatedOptions(oldName, newName string) {
envVar := envVarName(oldName)
newEnvVar := envVarName(newName)
envVar := "ND_" + strings.ToUpper(strings.ReplaceAll(oldName, ".", "_"))
newEnvVar := "ND_" + strings.ToUpper(strings.ReplaceAll(newName, ".", "_"))
logWarning := func(oldName, newName string) {
if newName != "" {
log.Warn(fmt.Sprintf("Option '%s' is deprecated and will be ignored in a future release. Please use the new '%s'", oldName, newName))
@@ -565,7 +515,7 @@ func logDeprecatedOptions(oldName, newName string) {
// not available anymore
func logRemovedOptions(options ...string) {
for _, option := range options {
envVar := envVarName(option)
envVar := "ND_" + strings.ToUpper(strings.ReplaceAll(option, ".", "_"))
logWarning := func(option string) {
log.Warn(fmt.Sprintf("Option '%s' is not available anymore and will be ignored. Please remove it from your config", option))
}
@@ -586,193 +536,35 @@ func remapEnvVarKeysFromConfig() {
continue
}
stripped := strings.TrimPrefix(key, "nd_")
canonicalKey := ndKeyToCanonical(key)
canonicalKey := strings.ReplaceAll(stripped, "_", ".")
displayNDKey := "ND_" + strings.ToUpper(stripped)
canonicalName := canonicalOptionName(canonicalKey)
displayCanonical := toPascalCase(canonicalKey)
if viper.InConfig(canonicalKey) {
logFatal(fmt.Sprintf(
"Config file contains both '%s' and '%s'. Remove the ND_-prefixed version. "+
"The 'ND_' prefix is only needed for environment variables, not config file keys.",
displayNDKey, cmp.Or(canonicalName, toPascalCase(canonicalKey)),
displayNDKey, displayCanonical,
))
return
}
viper.Set(canonicalKey, viper.Get(key))
// Unknown keys get no advice here, logUnknownOptions reports them instead
if canonicalName != "" {
_, _ = fmt.Fprintf(os.Stderr, "WARNING: Config key '%s' uses environment variable naming. Use '%s' instead. "+
"The 'ND_' prefix is only needed for environment variables.\n",
displayNDKey, canonicalName,
)
}
_, _ = fmt.Fprintf(os.Stderr, "WARNING: Config key '%s' uses environment variable naming. Use '%s' instead. "+
"The 'ND_' prefix is only needed for environment variables.\n",
displayNDKey, displayCanonical,
)
}
}
// mapDeprecatedOption is used to provide backwards compatibility for deprecated options. It should be called after
// the config has been read by viper, but before unmarshalling it into the Config struct.
func mapDeprecatedOption(legacyName, newName string) {
// viper.Set outranks the config file, so an explicit replacement must win over the legacy value
if viper.IsSet(legacyName) && !explicitlySet(newName) {
if viper.IsSet(legacyName) {
viper.Set(newName, viper.Get(legacyName))
}
}
// explicitlySet reports whether the user provided the option, ignoring defaults,
// which viper.IsSet counts as set. The ND_ spelling is also accepted in the config
// file, and remapEnvVarKeysFromConfig has already moved it out of InConfig's reach.
func explicitlySet(name string) bool {
envVar := envVarName(name)
return viper.InConfig(name) || os.Getenv(envVar) != "" || viper.InConfig(strings.ToLower(envVar))
}
func envVarName(option string) string {
if option == "" {
return ""
}
return "ND_" + strings.ToUpper(strings.ReplaceAll(option, ".", "_"))
}
func logUnknownOptions() {
for _, key := range unknownConfigKeys() {
msg := fmt.Sprintf("Option '%s' is not recognized and will be ignored", key)
if matches := suggestOptions(key); len(matches) > 0 {
msg += fmt.Sprintf(". Did you mean '%s'?", strings.Join(matches, "' or '"))
}
log.Warn(msg)
}
}
// suggestOptions returns the known options sharing the last segment with key,
// catching options written outside their section.
func suggestOptions(key string) []string {
key = strings.ToLower(key)
leaf := leafKey(key)
canonical, _ := configKeys()
var matches []string
for known, name := range canonical {
// Removed options are known only so they get their own warning, never suggest them
if known != key && leafKey(known) == leaf && !slices.Contains(removedOptions, name) {
matches = append(matches, name)
}
}
slices.Sort(matches)
return matches
}
func leafKey(key string) string {
return key[strings.LastIndex(key, ".")+1:]
}
// unknownConfigKeys returns config file keys that don't match any known option, so
// typos and options written outside their section don't fail silently.
func unknownConfigKeys() []string {
// INI files keep the original [default] section alongside the merged one
skipDefault := strings.EqualFold(filepath.Ext(viper.ConfigFileUsed()), ".ini")
var unknown []string
for _, key := range viper.AllKeys() {
if !viper.InConfig(key) || canonicalOptionName(key) != "" {
continue
}
if skipDefault && strings.HasPrefix(key, "default.") {
continue
}
// Only ND_-prefixed keys that remapEnvVarKeysFromConfig could resolve are valid
if strings.HasPrefix(key, "nd_") && canonicalOptionName(ndKeyToCanonical(key)) != "" {
continue
}
unknown = append(unknown, key)
}
slices.Sort(unknown)
return asWrittenInConfigFile(unknown)
}
func ndKeyToCanonical(key string) string {
return strings.ReplaceAll(strings.TrimPrefix(key, "nd_"), "_", ".")
}
// canonicalOptionName returns the documented spelling of a known option key, or ""
// if it matches no option. Subkeys of free-form maps have no fixed spelling.
func canonicalOptionName(key string) string {
keys, prefixes := configKeys()
if name, ok := keys[key]; ok {
return name
}
if slices.ContainsFunc(prefixes, func(p string) bool { return strings.HasPrefix(key, p) }) {
return toPascalCase(key)
}
return ""
}
// asWrittenInConfigFile restores the casing the keys have in the config file, as
// viper lowercases every key it loads.
func asWrittenInConfigFile(keys []string) []string {
if len(keys) == 0 {
return nil
}
data, err := os.ReadFile(viper.ConfigFileUsed())
if err != nil {
return keys
}
casing := map[string]string{}
for _, match := range configFileKeyRx.FindAllStringSubmatch(string(data), -1) {
for segment := range strings.SplitSeq(match[1], ".") {
lower := strings.ToLower(segment)
casing[lower] = cmp.Or(casing[lower], segment)
}
}
return slice.Map(keys, func(key string) string {
segments := strings.Split(key, ".")
for i, s := range segments {
segments[i] = cmp.Or(casing[s], s)
}
return strings.Join(segments, ".")
})
}
// Matches keys and section headers in all supported config formats.
var configFileKeyRx = regexp.MustCompile(`(?m)^\s*\[?\s*"?([\w.]+)"?\s*[]=:]`)
// configKeys maps every accepted option name, lowercased, to its canonical spelling,
// plus the prefixes of free-form map options (Tags, DevLogLevels).
var configKeys = sync.OnceValues(func() (map[string]string, []string) {
keys := map[string]string{}
var prefixes []string
var collect func(t reflect.Type, prefix string)
collect = func(t reflect.Type, prefix string) {
for field := range t.Fields() {
// `conf:"-"` marks values computed during Load, not settable in the config
if !field.IsExported() || field.Tag.Get("conf") == "-" {
continue
}
name := prefix + field.Name
if field.Type.Kind() == reflect.Struct && !reflect.PointerTo(field.Type).Implements(textUnmarshalerType) {
collect(field.Type, name+".")
continue
}
lower := strings.ToLower(name)
keys[lower] = name
if field.Type.Kind() == reflect.Map {
prefixes = append(prefixes, lower+".")
}
}
}
collect(reflect.TypeFor[configOptions](), "")
for _, o := range deprecatedOptions {
keys[strings.ToLower(o.name)] = o.name
}
for _, o := range removedOptions {
keys[strings.ToLower(o)] = o
}
return keys, prefixes
})
var textUnmarshalerType = reflect.TypeFor[encoding.TextUnmarshaler]()
// parseIniFileConfiguration is used to parse the config file when it is in INI format. For INI files, it
// would require a nested structure, so instead we unmarshal it to a map and then merge the nested [default]
// section into the root level.
@@ -845,20 +637,11 @@ func validatePurgeMissingOption() error {
return nil
}
func validateByteSize(name, value string) func() error {
return func() error {
size, err := humanize.ParseBytes(value)
if err != nil {
return fmt.Errorf("invalid %s %q: use values like '10MB', '1GB', or raw bytes like '10485760': %w", name, value, err)
}
if size == 0 {
return fmt.Errorf("invalid %s %q: must be greater than zero", name, value)
}
if size > math.MaxInt64 {
return fmt.Errorf("invalid %s %q: value is too large", name, value)
}
return nil
func validateMaxImageUploadSize() error {
if _, err := humanize.ParseBytes(Server.MaxImageUploadSize); err != nil {
return fmt.Errorf("invalid MaxImageUploadSize %q: use values like '10MB', '1GB', or raw bytes like '10485760': %w", Server.MaxImageUploadSize, err)
}
return nil
}
func validateEnforceNonRootUser() error {
@@ -988,7 +771,7 @@ func setViperDefaults() {
viper.SetDefault("autoimportplaylists", true)
viper.SetDefault("defaultplaylistpublicvisibility", false)
viper.SetDefault("playlistspath", "")
viper.SetDefault("smartPlaylistRefreshDelay", consts.DefaultSmartRefresh)
viper.SetDefault("smartPlaylistRefreshDelay", 5*time.Second)
viper.SetDefault("enabledownloads", true)
viper.SetDefault("enableexternalservices", true)
viper.SetDefault("enablem3uexternalalbumart", false)
@@ -1001,7 +784,6 @@ func setViperDefaults() {
viper.SetDefault("matcher.fuzzythreshold", 85)
viper.SetDefault("recentlyaddedbymodtime", false)
viper.SetDefault("prefersorttags", false)
viper.SetDefault("enablenaturalsorting", false)
viper.SetDefault("ignoredarticles", "The El La Los Las Le Les Os As O A")
viper.SetDefault("indexgroups", "A B C D E F G H I J K L M N O P Q R S T U V W X-Z(XYZ) [Unknown]([)")
viper.SetDefault("ffmpegpath", "")
@@ -1029,17 +811,16 @@ func setViperDefaults() {
viper.SetDefault("uiplaybackreportinterval", consts.DefaultUIPlaybackReportInterval)
viper.SetDefault("enableartworkupload", true)
viper.SetDefault("maximageuploadsize", consts.DefaultMaxImageUploadSize)
viper.SetDefault("maximagesize", consts.DefaultMaxImageSize)
viper.SetDefault("enablesharing", true)
viper.SetDefault("shareurl", "")
viper.SetDefault("defaultshareexpiration", consts.DefaultShareExpiration)
viper.SetDefault("defaultshareexpiration", 8760*time.Hour)
viper.SetDefault("defaultdownloadableshare", false)
viper.SetDefault("gatrackingid", "")
viper.SetDefault("enableinsightscollector", true)
viper.SetDefault("enablescheduleddbanalyze", true)
viper.SetDefault("enablelogredacting", true)
viper.SetDefault("authrequestlimit", 5)
viper.SetDefault("authwindowlength", consts.DefaultAuthWindowLength)
viper.SetDefault("authwindowlength", 20*time.Second)
viper.SetDefault("passwordencryptionkey", "")
viper.SetDefault("extauth.userheader", "Remote-User")
viper.SetDefault("extauth.trustedsources", "")
@@ -1123,12 +904,8 @@ func setViperDefaults() {
viper.SetDefault("devartworkthrottlebackloglimit", consts.RequestThrottleBacklogLimit)
viper.SetDefault("devartworkthrottlebacklogtimeout", consts.RequestThrottleBacklogTimeout)
viper.SetDefault("devartworkthrottlebuffered", true)
// Half the CPU count (min 2), so local resolution scales with the host but stays under the
// SQLite pool (MaxOpenConns) — leaving connections for the scanner, scrobbles and the UI.
viper.SetDefault("devartworkworkerconcurrency", max(2, runtime.NumCPU()/2))
// External RPS gates outbound calls to third-party services (per service); it is bounded by
// their tolerance, not the host, so it stays a small constant regardless of CPU count.
viper.SetDefault("devartworkexternalmaxrps", 2)
viper.SetDefault("artworkworkerconcurrency", 4)
viper.SetDefault("artworkexternalmaxrps", 2)
viper.SetDefault("devartistinfotimetolive", consts.ArtistInfoTimeToLive)
viper.SetDefault("devalbuminfotimetolive", consts.AlbumInfoTimeToLive)
viper.SetDefault("devexternalscanner", true)
+13 -220
View File
@@ -1,17 +1,12 @@
package conf_test
import (
"bytes"
"fmt"
"os"
"path/filepath"
"testing"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/log"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/spf13/viper"
@@ -183,123 +178,6 @@ var _ = Describe("Configuration", func() {
})
})
Describe("unknownConfigKeys", func() {
BeforeEach(func() {
viper.Reset()
conf.SetViperDefaults()
viper.SetDefault("datafolder", GinkgoT().TempDir())
viper.SetDefault("loglevel", "error")
conf.ResetConf()
})
It("reports misplaced and misspelled options, as spelled in the config file", func() {
conf.InitConfig(filepath.Join("testdata", "cfg_unknown_keys.toml"), false)
conf.Load(true)
Expect(conf.UnknownConfigKeys()).To(ConsistOf(
"ArtistSplitExceptions", "EnableDownlods", "Whatever.Foo",
))
})
DescribeTable("recovers the original casing in all supported formats",
func(file string) {
conf.InitConfig(filepath.Join("testdata", file), false)
conf.Load(true)
Expect(conf.UnknownConfigKeys()).To(ConsistOf("NotAnOption"))
},
Entry("TOML", "cfg_unknown_casing.toml"),
Entry("YAML", "cfg_unknown_casing.yaml"),
Entry("JSON", "cfg_unknown_casing.json"),
Entry("INI", "cfg_unknown_casing.ini"),
)
It("does not report valid, deprecated or free-form keys", func() {
conf.InitConfig(filepath.Join("testdata", "cfg.toml"), false)
conf.Load(true)
Expect(conf.UnknownConfigKeys()).To(BeEmpty())
})
It("does not report the [default] section of INI files", func() {
conf.InitConfig(filepath.Join("testdata", "cfg.ini"), false)
conf.Load(true)
Expect(conf.UnknownConfigKeys()).To(BeEmpty())
})
DescribeTable("SuggestOptions",
func(key string, expected []string) {
Expect(conf.SuggestOptions(key)).To(Equal(expected))
},
Entry("suggests the section of a misplaced option", "artistsplitexceptions",
[]string{"Scanner.ArtistSplitExceptions"}),
Entry("suggests the section of a misplaced nested option", "backup.fuzzythreshold",
[]string{"Matcher.FuzzyThreshold"}),
Entry("suggests every section defining the option", "schedule",
[]string{"Backup.Schedule", "Scanner.Schedule"}),
Entry("suggests nothing for a typo", "enabledownlods", nil),
)
It("does not report ND_-prefixed keys, as they are remapped", func() {
conf.InitConfig(filepath.Join("testdata", "cfg_nd_keys.toml"), false)
conf.Load(true)
Expect(conf.UnknownConfigKeys()).To(BeEmpty())
})
It("reports ND_-prefixed keys that remap to no known option", func() {
conf.InitConfig(filepath.Join("testdata", "cfg_nd_bogus.toml"), false)
conf.Load(true)
Expect(conf.UnknownConfigKeys()).To(ConsistOf("ND_TOTALLY_BOGUS_OPTION"))
Expect(conf.Server.Scanner.Schedule).To(Equal("@every 1h"))
})
It("migrates every deprecated option that has a replacement", func() {
conf.InitConfig(filepath.Join("testdata", "cfg_deprecated_search.toml"), false)
conf.Load(true)
Expect(conf.Server.Search.FullString).To(BeTrue())
Expect(conf.UnknownConfigKeys()).To(BeEmpty())
})
It("warns about each unrecognized option at startup", func() {
var logBuf bytes.Buffer
log.SetOutput(&logBuf)
DeferCleanup(func() { log.SetOutput(GinkgoWriter) })
conf.InitConfig(filepath.Join("testdata", "cfg_warning_output.toml"), false)
conf.Load(true)
Expect(logBuf.String()).To(ContainSubstring(
"Option 'ArtistSplitExceptions' is not recognized and will be ignored. " +
"Did you mean 'Scanner.ArtistSplitExceptions'?"))
Expect(logBuf.String()).To(ContainSubstring(
"Option 'EnableDownlods' is not recognized and will be ignored"))
Expect(logBuf.String()).ToNot(ContainSubstring("ArtistJoiner"))
})
Context("with runtime-computed and removed options in the config", func() {
BeforeEach(func() {
conf.InitConfig(filepath.Join("testdata", "cfg_runtime_fields.toml"), false)
conf.Load(true)
})
It("reports values computed during Load, which the config cannot set", func() {
Expect(conf.UnknownConfigKeys()).To(ContainElements("ConfigFile", "LastFM.Languages"))
})
It("never suggests a removed option", func() {
Expect(conf.SuggestOptions("id")).To(BeEmpty())
})
It("keeps an explicit replacement over the deprecated value", func() {
Expect(conf.Server.Search.FullString).To(BeFalse())
})
})
})
Describe("logFatal", func() {
var invalidPath string
BeforeEach(func() {
@@ -339,10 +217,19 @@ var _ = Describe("Configuration", func() {
})
Describe("ValidateByteSize", func() {
Describe("ValidateMaxImageUploadSize", func() {
BeforeEach(func() {
viper.Reset()
conf.SetViperDefaults()
viper.SetDefault("datafolder", GinkgoT().TempDir())
viper.SetDefault("loglevel", "error")
conf.ResetConf()
})
DescribeTable("accepts valid size values",
func(input string) {
Expect(conf.ValidateByteSize("MaxImageSize", input)()).To(Succeed())
conf.Server.MaxImageUploadSize = input
Expect(conf.ValidateMaxImageUploadSize()).To(Succeed())
},
Entry("megabytes", "10MB"),
Entry("gigabytes", "1GB"),
@@ -353,39 +240,14 @@ var _ = Describe("Configuration", func() {
DescribeTable("rejects invalid size values",
func(input string) {
Expect(conf.ValidateByteSize("MaxImageSize", input)()).To(MatchError(ContainSubstring("invalid MaxImageSize")))
conf.Server.MaxImageUploadSize = input
Expect(conf.ValidateMaxImageUploadSize()).To(MatchError(ContainSubstring("invalid MaxImageUploadSize")))
},
Entry("garbage string", "not-a-size"),
Entry("negative-looking", "-10MB"),
Entry("zero", "0"),
Entry("zero with unit", "0MB"),
Entry("overflows int64", "9223372036854775808"),
)
})
Describe("MaxImageSize floor", func() {
BeforeEach(func() {
viper.Reset()
conf.SetViperDefaults()
viper.SetDefault("datafolder", GinkgoT().TempDir())
viper.SetDefault("loglevel", "error")
conf.ResetConf()
})
It("is raised to MaxImageUploadSize when configured lower", func() {
viper.SetDefault("maximagesize", "5MB")
viper.SetDefault("maximageuploadsize", "50MB")
conf.Load(true)
Expect(conf.Server.MaxImageSize).To(Equal("50MB"))
})
It("keeps a larger MaxImageSize unchanged", func() {
viper.SetDefault("maximagesize", "30MB")
conf.Load(true)
Expect(conf.Server.MaxImageSize).To(Equal("30MB"))
})
})
Describe("EnforceNonRootUser", func() {
It("defaults to false", func() {
conf.Load(true)
@@ -455,73 +317,4 @@ var _ = Describe("Configuration", func() {
Entry("INI format", "ini"),
Entry("JSON format", "json"),
)
It("should use default values for negative duration fields", func() {
filename := filepath.Join("testdata", "invalid_duration.toml")
conf.InitConfig(filename, false)
conf.Load(true)
server := conf.Server
Expect(server.SessionTimeout).To(Equal(consts.DefaultSessionTimeout))
Expect(server.SmartPlaylistRefreshDelay).To(Equal(consts.DefaultSmartRefresh))
Expect(server.DefaultShareExpiration).To(Equal(consts.DefaultShareExpiration))
Expect(server.UIPlaybackReportInterval).To(Equal(consts.DefaultUIPlaybackReportInterval))
Expect(server.AuthWindowLength).To(Equal(consts.DefaultAuthWindowLength))
Expect(server.Scanner.WatcherWait).To(Equal(consts.DefaultWatcherWait))
Expect(server.DevActivityPanelUpdateRate).To(Equal(consts.DefaultActivityPanelUpdateRate))
Expect(server.DevArtworkThrottleBacklogTimeout).To(Equal(consts.RequestThrottleBacklogTimeout))
Expect(server.DevArtistInfoTimeToLive).To(Equal(consts.ArtistInfoTimeToLive))
Expect(server.DevAlbumInfoTimeToLive).To(Equal(consts.AlbumInfoTimeToLive))
Expect(server.DevInsightsInitialDelay).To(Equal(consts.InsightsInitialDelay))
Expect(server.DevPluginCompilationTimeout).To(Equal(consts.DefaultPluginCompilationTimeout))
})
It("should use parsed values for duration fields", func() {
conf.InitConfig(filepath.Join("testdata", "valid_duration.toml"), false)
conf.Load(true)
configured := 1 * time.Second
server := conf.Server
Expect(server.SessionTimeout).To(Equal(configured))
Expect(server.SmartPlaylistRefreshDelay).To(Equal(configured))
Expect(server.DefaultShareExpiration).To(Equal(configured))
Expect(server.UIPlaybackReportInterval).To(Equal(configured))
Expect(server.AuthWindowLength).To(Equal(configured))
Expect(server.Scanner.WatcherWait).To(Equal(configured))
Expect(server.DevActivityPanelUpdateRate).To(Equal(configured))
Expect(server.DevArtworkThrottleBacklogTimeout).To(Equal(configured))
Expect(server.DevArtistInfoTimeToLive).To(Equal(configured))
Expect(server.DevAlbumInfoTimeToLive).To(Equal(configured))
Expect(server.DevInsightsInitialDelay).To(Equal(configured))
Expect(server.DevPluginCompilationTimeout).To(Equal(configured))
})
})
var _ = Describe("TLSEnabled", func() {
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
})
It("is false when neither the certificate nor the key is set", func() {
Expect(conf.Server.TLSEnabled()).To(BeFalse())
})
It("is true when both the certificate and the key are set", func() {
conf.Server.TLSCert = "cert.pem"
conf.Server.TLSKey = "key.pem"
Expect(conf.Server.TLSEnabled()).To(BeTrue())
})
It("is false when only the certificate is set", func() {
conf.Server.TLSCert = "cert.pem"
Expect(conf.Server.TLSEnabled()).To(BeFalse())
})
It("is false when only the key is set", func() {
conf.Server.TLSKey = "key.pem"
Expect(conf.Server.TLSEnabled()).To(BeFalse())
})
})
+1 -5
View File
@@ -14,7 +14,7 @@ var NormalizeSearchBackend = normalizeSearchBackend
var ToPascalCase = toPascalCase
var ValidateByteSize = validateByteSize
var ValidateMaxImageUploadSize = validateMaxImageUploadSize
func SetRuntimeInfoForTest(goos string, euid int) func() {
oldGOOS := currentGOOS
@@ -32,7 +32,3 @@ func SetLogFatal(f func(...any)) func() {
logFatal = f
return func() { logFatal = old }
}
var UnknownConfigKeys = unknownConfigKeys
var SuggestOptions = suggestOptions
-2
View File
@@ -1,2 +0,0 @@
MusicFolder = "/toml/music"
SearchFullString = true
-3
View File
@@ -1,3 +0,0 @@
MusicFolder = "/toml/music"
ND_TOTALLY_BOGUS_OPTION = true
ND_SCANNER_SCHEDULE = "@every 1h"
-10
View File
@@ -1,10 +0,0 @@
MusicFolder = "/toml/music"
SearchFullString = true
ConfigFile = "/somewhere/else"
ID = "oops"
[Search]
FullString = false
[LastFM]
Languages = ["pt"]
-3
View File
@@ -1,3 +0,0 @@
[default]
MusicFolder = /ini/music
NotAnOption = true
-4
View File
@@ -1,4 +0,0 @@
{
"MusicFolder": "/json/music",
"NotAnOption": true
}
-2
View File
@@ -1,2 +0,0 @@
MusicFolder = "/toml/music"
NotAnOption = true
-2
View File
@@ -1,2 +0,0 @@
MusicFolder: /yaml/music
NotAnOption: true
-18
View File
@@ -1,18 +0,0 @@
MusicFolder = "/toml/music"
# Valid option, but written at the root level instead of under Scanner
ArtistSplitExceptions = ["AC/DC", "Tyler, the creator"]
# Misspelled option
EnableDownlods = true
# Unknown section
[Whatever]
Foo = "bar"
# Valid options, must not be reported
[Scanner]
ArtistJoiner = " • "
[Tags.custom]
aliases = ["toml", "test"]
-7
View File
@@ -1,7 +0,0 @@
MusicFolder = "/toml/music"
LogLevel = "warn"
ArtistSplitExceptions = ["AC/DC"]
EnableDownlods = true
[Scanner]
ArtistJoiner = " • "
-12
View File
@@ -1,12 +0,0 @@
SessionTimeout = "-10s"
SmartPlaylistRefreshDelay = "-10s"
UIPlaybackReportInterval = "-10s"
AuthWindowLength = "-10s"
DefaultShareExpiration = "-10s"
Scanner.WatcherWait = "-10s"
DevActivityPanelUpdateRate = "-10s"
DevArtworkThrottleBacklogTimeout = "-10s"
DevArtistInfoTimeToLive = "-10s"
DevAlbumInfoTimeToLive = "-10s"
DevInsightsInitialDelay = "-10s"
DevPluginCompilationTimeout = "-10s"
-12
View File
@@ -1,12 +0,0 @@
SessionTimeout = "1s"
SmartPlaylistRefreshDelay = "1s"
UIPlaybackReportInterval = "1s"
AuthWindowLength = "1s"
DefaultShareExpiration = "1s"
Scanner.WatcherWait = "1s"
DevActivityPanelUpdateRate = "1s"
DevArtworkThrottleBacklogTimeout = "1s"
DevArtistInfoTimeToLive = "1s"
DevAlbumInfoTimeToLive = "1s"
DevInsightsInitialDelay = "1s"
DevPluginCompilationTimeout = "1s"
+4 -17
View File
@@ -24,25 +24,20 @@ const (
LastDBAnalyzeAttemptAtKey = "LastDBAnalyzeAttemptAt"
DBAnalyzePendingKey = "DBAnalyzePending"
DBAnalyzeFailureCountKey = "DBAnalyzeFailureCount"
// ArtConfFingerprintPropertyKey is the model.PropertyRepository key the artwork config check
// compares against to detect artwork-affecting config changes across restarts.
ArtConfFingerprintPropertyKey = "ArtConfFingerprint"
UIAuthorizationHeader = "X-ND-Authorization"
UIClientUniqueIDHeader = "X-ND-Client-Unique-Id"
JWTSecretKey = "JWTSecret"
JWTPublicSecretKey = "JWTPublicSecret"
JWTIssuer = "ND"
DefaultSessionTimeout = 48 * time.Hour
DefaultSmartRefresh = 5 * time.Second
DefaultShareExpiration = 8760 * time.Hour
CookieExpiry = 365 * 24 * 3600 // One year
DBAnalyzeCheckSchedule = "@every 30m"
DBAnalyzeMaxAge = 24 * time.Hour
ArtworkEnqueueMissingSchedule = "@every 1h"
ArtworkPruneSchedule = "@daily"
ArtworkStaleAbsentRecheckSchedule = "@every 1h"
ArtworkPruneSchedule = "@daily"
ArtworkPostBackfillPruneDelay = 10 * time.Minute
// DefaultEncryptionKey This is the encryption key used if none is specified in the `PasswordEncryptionKey` option
// Never ever change this! Or it will break all Navidrome installations that don't set the config option
@@ -73,7 +68,6 @@ const (
DefaultUILoginBackgroundURLOffline = "data:image/png;base64," + DefaultUILoginBackgroundOffline
DefaultMaxSidebarPlaylists = 100
DefaultAuthWindowLength = 20 * time.Second
RequestThrottleBacklogLimit = 100
RequestThrottleBacklogTimeout = time.Minute
@@ -89,9 +83,6 @@ const (
I18nFolder = "i18n"
ScanIgnoreFile = ".ndignore"
ArtworkFolder = "artwork"
// HashedArtworkFolder is a subtree of ArtworkFolder, kept apart from the name-addressed
// upload folders beside it so Prune's sweep never reaches them.
HashedArtworkFolder = "hashed"
PlaceholderArtistArt = "artist-placeholder.webp"
PlaceholderAlbumArt = "album-placeholder.webp"
@@ -109,15 +100,11 @@ const (
DefaultScannerExtractor = "taglib"
DefaultWatcherWait = 5 * time.Second
Zwsp = string('\u200b')
DefaultActivityPanelUpdateRate = 300 * time.Millisecond
DefaultPluginCompilationTimeout = time.Minute
)
const (
DefaultUICoverArtSize = 300
DefaultMaxImageUploadSize = "10MB"
DefaultMaxImageSize = "20MB"
)
// Prometheus options
@@ -206,7 +193,7 @@ var (
}
)
var HTTPUserAgent = "Navidrome/" + Version + " - https://github.com/navidrome"
var HTTPUserAgent = "Navidrome" + "/" + Version
var (
VariousArtists = "Various Artists"
+1 -1
View File
@@ -2,7 +2,7 @@
name=$RC_SVCNAME
command="/opt/navidrome/${RC_SVCNAME}"
command_args="--datafolder /opt/navidrome"
command_args="-datafolder /opt/navidrome"
command_user="${RC_SVCNAME}"
pidfile="/var/run/${RC_SVCNAME}.pid"
output_log="/opt/navidrome/${RC_SVCNAME}.log"
+46 -120
View File
@@ -1,13 +1,9 @@
package agents
import (
"cmp"
"context"
"errors"
"maps"
"slices"
"strings"
"sync"
"time"
"github.com/navidrome/navidrome/conf"
@@ -26,43 +22,11 @@ type PluginLoader interface {
LoadMediaAgent(name string) (Interface, bool)
}
// agentCooldown is the default cooldown duration for an agent that returns a RetryLaterError without a specific
// RetryIn duration.
const agentCooldown = time.Minute
// errUnsupported marks an agent that does not implement the requested method: it never ran,
// so it neither answered nor throttled.
var errUnsupported = errors.New("agent does not support this method")
// Agents is a meta-agent that aggregates multiple built-in and plugin agents. It tries each enabled agent in order
// until one returns valid data.
type Agents struct {
ds model.DataStore
pluginLoader PluginLoader
cooldowns cooldowns
}
// cooldowns remembers, across dispatches, which agents asked to be left alone and until when.
type cooldowns struct {
mu sync.RWMutex
until map[string]time.Time
}
func (c *cooldowns) active(name string) bool {
c.mu.RLock()
defer c.mu.RUnlock()
return time.Now().Before(c.until[name])
}
// park keeps whichever deadline is later, so a call still in flight when a longer cooldown
// starts cannot cut it short when it finally answers.
func (c *cooldowns) park(name string, d time.Duration) {
until := time.Now().Add(d)
c.mu.Lock()
defer c.mu.Unlock()
if until.After(c.until[name]) {
c.until[name] = until
}
}
// GetAgents returns the singleton instance of Agents
@@ -77,7 +41,6 @@ func createAgents(ds model.DataStore, pluginLoader PluginLoader) *Agents {
return &Agents{
ds: ds,
pluginLoader: pluginLoader,
cooldowns: cooldowns{until: map[string]time.Time{}},
}
}
@@ -127,19 +90,12 @@ func (a *Agents) getEnabledAgentNames() []enabledAgent {
} else if isPlugin {
validAgents = append(validAgents, enabledAgent{name: name, isPlugin: true})
} else {
log.Debug("Unknown agent ignored", "name", name, "available", availableAgentNames(availablePlugins))
log.Debug("Unknown agent ignored", "name", name)
}
}
return validAgents
}
// availableAgentNames returns every name accepted by the Agents config option.
func availableAgentNames(plugins []string) []string {
names := append(slices.Collect(maps.Keys(Map)), plugins...)
slices.Sort(names)
return names
}
func (a *Agents) getAgent(ea enabledAgent) Interface {
if ea.isPlugin {
// Try to load WASM plugin agent (if plugin loader is available)
@@ -185,7 +141,11 @@ type AlbumImageAgent struct {
func (a *Agents) ArtistImageAgents() []ArtistImageAgent {
var result []ArtistImageAgent
for _, ea := range a.getEnabledAgentNames() {
if retriever, ok := a.getAgent(ea).(ArtistImageRetriever); ok {
ag := a.getAgent(ea)
if ag == nil {
continue
}
if retriever, ok := ag.(ArtistImageRetriever); ok {
result = append(result, ArtistImageAgent{Name: ea.name, Retriever: retriever})
}
}
@@ -197,7 +157,11 @@ func (a *Agents) ArtistImageAgents() []ArtistImageAgent {
func (a *Agents) AlbumImageAgents() []AlbumImageAgent {
var result []AlbumImageAgent
for _, ea := range a.getEnabledAgentNames() {
if retriever, ok := a.getAgent(ea).(AlbumImageRetriever); ok {
ag := a.getAgent(ea)
if ag == nil {
continue
}
if retriever, ok := ag.(AlbumImageRetriever); ok {
result = append(result, AlbumImageAgent{Name: ea.name, Retriever: retriever})
}
}
@@ -215,7 +179,7 @@ func (a *Agents) GetArtistMBID(ctx context.Context, id string, name string) (str
return callAgentMethod(ctx, a, "GetArtistMBID", func(ag Interface) (string, error) {
retriever, ok := ag.(ArtistMBIDRetriever)
if !ok {
return "", errUnsupported
return "", ErrNotFound
}
return retriever.GetArtistMBID(ctx, id, name)
})
@@ -232,7 +196,7 @@ func (a *Agents) GetArtistURL(ctx context.Context, id, name, mbid string) (strin
return callAgentMethod(ctx, a, "GetArtistURL", func(ag Interface) (string, error) {
retriever, ok := ag.(ArtistURLRetriever)
if !ok {
return "", errUnsupported
return "", ErrNotFound
}
return retriever.GetArtistURL(ctx, id, name, mbid)
})
@@ -249,7 +213,7 @@ func (a *Agents) GetArtistBiography(ctx context.Context, id, name, mbid string)
return callAgentMethod(ctx, a, "GetArtistBiography", func(ag Interface) (string, error) {
retriever, ok := ag.(ArtistBiographyRetriever)
if !ok {
return "", errUnsupported
return "", ErrNotFound
}
return retriever.GetArtistBiography(ctx, id, name, mbid)
})
@@ -268,11 +232,7 @@ func (a *Agents) GetSimilarArtists(ctx context.Context, id, name, mbid string, l
overLimit := int(float64(limit) * conf.Server.DevExternalArtistFetchMultiplier)
start := time.Now()
attempts := newAttempts(&a.cooldowns)
for _, enabledAgent := range a.getEnabledAgentNames() {
if attempts.skip(enabledAgent.name) {
continue
}
ag := a.getAgent(enabledAgent)
if ag == nil {
continue
@@ -285,7 +245,6 @@ func (a *Agents) GetSimilarArtists(ctx context.Context, id, name, mbid string, l
continue
}
similar, err := retriever.GetSimilarArtists(ctx, id, name, mbid, overLimit)
attempts.record(enabledAgent.name, err)
if len(similar) > 0 && err == nil {
if log.IsGreaterOrEqualTo(log.LevelTrace) {
log.Debug(ctx, "Got Similar Artists", "agent", ag.AgentName(), "artist", name, "similar", similar, "elapsed", time.Since(start))
@@ -295,7 +254,7 @@ func (a *Agents) GetSimilarArtists(ctx context.Context, id, name, mbid string, l
return similar, err
}
}
return nil, attempts.noResultErr()
return nil, ErrNotFound
}
func (a *Agents) GetArtistImages(ctx context.Context, id, name, mbid string) ([]ExternalImage, error) {
@@ -309,7 +268,7 @@ func (a *Agents) GetArtistImages(ctx context.Context, id, name, mbid string) ([]
return callAgentSliceMethod(ctx, a, "GetArtistImages", func(ag Interface) ([]ExternalImage, error) {
retriever, ok := ag.(ArtistImageRetriever)
if !ok {
return nil, errUnsupported
return nil, ErrNotFound
}
return retriever.GetArtistImages(ctx, id, name, mbid)
})
@@ -330,7 +289,7 @@ func (a *Agents) GetArtistTopSongs(ctx context.Context, id, artistName, mbid str
return callAgentSliceMethod(ctx, a, "GetArtistTopSongs", func(ag Interface) ([]Song, error) {
retriever, ok := ag.(ArtistTopSongsRetriever)
if !ok {
return nil, errUnsupported
return nil, ErrNotFound
}
return retriever.GetArtistTopSongs(ctx, id, artistName, mbid, overLimit)
})
@@ -344,7 +303,7 @@ func (a *Agents) GetAlbumInfo(ctx context.Context, name, artist, mbid string) (*
return callAgentMethod(ctx, a, "GetAlbumInfo", func(ag Interface) (*AlbumInfo, error) {
retriever, ok := ag.(AlbumInfoRetriever)
if !ok {
return nil, errUnsupported
return nil, ErrNotFound
}
return retriever.GetAlbumInfo(ctx, name, artist, mbid)
})
@@ -358,7 +317,7 @@ func (a *Agents) GetAlbumImages(ctx context.Context, name, artist, mbid string)
return callAgentSliceMethod(ctx, a, "GetAlbumImages", func(ag Interface) ([]ExternalImage, error) {
retriever, ok := ag.(AlbumImageRetriever)
if !ok {
return nil, errUnsupported
return nil, ErrNotFound
}
return retriever.GetAlbumImages(ctx, name, artist, mbid)
})
@@ -369,7 +328,7 @@ func (a *Agents) GetSimilarSongsByTrack(ctx context.Context, id, name, artist, m
return callAgentSliceMethod(ctx, a, "GetSimilarSongsByTrack", func(ag Interface) ([]Song, error) {
retriever, ok := ag.(SimilarSongsByTrackRetriever)
if !ok {
return nil, errUnsupported
return nil, ErrNotFound
}
return retriever.GetSimilarSongsByTrack(ctx, id, name, artist, mbid, count)
})
@@ -380,7 +339,7 @@ func (a *Agents) GetSimilarSongsByAlbum(ctx context.Context, id, name, artist, m
return callAgentSliceMethod(ctx, a, "GetSimilarSongsByAlbum", func(ag Interface) ([]Song, error) {
retriever, ok := ag.(SimilarSongsByAlbumRetriever)
if !ok {
return nil, errUnsupported
return nil, ErrNotFound
}
return retriever.GetSimilarSongsByAlbum(ctx, id, name, artist, mbid, count)
})
@@ -398,61 +357,16 @@ func (a *Agents) GetSimilarSongsByArtist(ctx context.Context, id, name, mbid str
return callAgentSliceMethod(ctx, a, "GetSimilarSongsByArtist", func(ag Interface) ([]Song, error) {
retriever, ok := ag.(SimilarSongsByArtistRetriever)
if !ok {
return nil, errUnsupported
return nil, ErrNotFound
}
return retriever.GetSimilarSongsByArtist(ctx, id, name, mbid, count)
})
}
// agentAttempts tallies what the enabled agents did in one dispatch.
type agentAttempts struct {
cooldowns *cooldowns
throttled bool
answered bool
}
func newAttempts(c *cooldowns) agentAttempts {
return agentAttempts{cooldowns: c}
}
// skip reports whether name is still cooling down, counting it as throttled for this dispatch.
func (t *agentAttempts) skip(name string) bool {
if !t.cooldowns.active(name) {
return false
}
t.throttled = true
return true
}
// record files one agent's outcome, parking it when it asked to be retried later.
func (t *agentAttempts) record(name string, err error) {
switch retry, isRetryLater := errors.AsType[*RetryLaterError](err); {
case errors.Is(err, errUnsupported):
case isRetryLater:
t.cooldowns.park(name, cmp.Or(retry.RetryIn, agentCooldown))
t.throttled = true
default:
t.answered = true
}
}
// noResultErr tells a retryable empty dispatch (nobody answered) from a definitive miss.
func (t *agentAttempts) noResultErr() error {
if t.throttled && !t.answered {
return ErrRetryLater
}
return ErrNotFound
}
// callAgent tries each enabled agent in order until found reports a usable result.
func callAgent[T any](ctx context.Context, agents *Agents, methodName string, fn func(Interface) (T, error), found func(T) bool) (T, error) {
func callAgentMethod[T comparable](ctx context.Context, agents *Agents, methodName string, fn func(Interface) (T, error)) (T, error) {
var zero T
start := time.Now()
attempts := newAttempts(&agents.cooldowns)
for _, enabledAgent := range agents.getEnabledAgentNames() {
if attempts.skip(enabledAgent.name) {
continue
}
ag := agents.getAgent(enabledAgent)
if ag == nil {
continue
@@ -461,29 +375,41 @@ func callAgent[T any](ctx context.Context, agents *Agents, methodName string, fn
break
}
result, err := fn(ag)
attempts.record(enabledAgent.name, err)
if err != nil {
log.Trace(ctx, "Agent method call error", "method", methodName, "agent", ag.AgentName(), "error", err)
continue
}
if found(result) {
if result != zero {
log.Debug(ctx, "Got result", "method", methodName, "agent", ag.AgentName(), "elapsed", time.Since(start))
return result, nil
}
}
return zero, attempts.noResultErr()
}
func callAgentMethod[T comparable](ctx context.Context, agents *Agents, methodName string, fn func(Interface) (T, error)) (T, error) {
return callAgent(ctx, agents, methodName, fn, func(result T) bool {
var zero T
return result != zero
})
return zero, ErrNotFound
}
func callAgentSliceMethod[T any](ctx context.Context, agents *Agents, methodName string, fn func(Interface) ([]T, error)) ([]T, error) {
return callAgent(ctx, agents, methodName, fn, func(results []T) bool { return len(results) > 0 })
start := time.Now()
for _, enabledAgent := range agents.getEnabledAgentNames() {
ag := agents.getAgent(enabledAgent)
if ag == nil {
continue
}
if utils.IsCtxDone(ctx) {
break
}
results, err := fn(ag)
if err != nil {
log.Trace(ctx, "Agent method call error", "method", methodName, "agent", ag.AgentName(), "error", err)
continue
}
if len(results) > 0 {
log.Debug(ctx, "Got results", "method", methodName, "agent", ag.AgentName(), "count", len(results), "elapsed", time.Since(start))
return results, nil
}
}
return nil, ErrNotFound
}
var _ Interface = (*Agents)(nil)
+4 -143
View File
@@ -3,8 +3,6 @@ package agents
import (
"context"
"errors"
"slices"
"time"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/consts"
@@ -16,29 +14,6 @@ import (
. "github.com/onsi/gomega"
)
var _ = Describe("cooldowns", func() {
// Calls to one agent overlap, so a short cooldown can land after a long one started.
It("keeps the longer deadline when a shorter park lands after it", func() {
c := cooldowns{until: map[string]time.Time{}}
c.park("fake", time.Hour)
c.park("fake", time.Millisecond)
time.Sleep(10 * time.Millisecond)
Expect(c.active("fake")).To(BeTrue())
})
It("extends the deadline when the later park is longer", func() {
c := cooldowns{until: map[string]time.Time{}}
c.park("fake", time.Millisecond)
c.park("fake", time.Hour)
time.Sleep(10 * time.Millisecond)
Expect(c.active("fake")).To(BeTrue())
})
})
var _ = Describe("Agents", func() {
var ctx context.Context
var cancel context.CancelFunc
@@ -59,10 +34,10 @@ var _ = Describe("Agents", func() {
})
It("calls the placeholder GetArtistImages", func() {
mfRepo.SetData(model.MediaFiles{{ID: "1", Title: "One"}, {ID: "2", Title: "Two"}})
mfRepo.SetData(model.MediaFiles{{ID: "1", Title: "One", MbzReleaseTrackID: "111"}, {ID: "2", Title: "Two", MbzReleaseTrackID: "222"}})
songs, err := ag.GetArtistTopSongs(ctx, "123", "John Doe", "mb123", 2)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(ConsistOf([]Song{{ID: "1", Name: "One"}, {ID: "2", Name: "Two"}}))
Expect(songs).To(ConsistOf([]Song{{Name: "One", MBID: "111"}, {Name: "Two", MBID: "222"}}))
})
})
@@ -92,22 +67,6 @@ var _ = Describe("Agents", func() {
Expect(ags).ToNot(ContainElement("disabled"))
})
Describe("availableAgentNames", func() {
It("combines built-in agents with the given plugins", func() {
names := availableAgentNames([]string{"apple-music"})
Expect(names).To(ContainElements("apple-music", LocalAgentName, "fake", "empty"))
})
It("returns the names sorted", func() {
names := availableAgentNames([]string{"zz-plugin", "aa-plugin"})
Expect(slices.IsSorted(names)).To(BeTrue())
})
It("works when there are no plugins", func() {
Expect(availableAgentNames(nil)).To(ContainElement(LocalAgentName))
})
})
Describe("GetArtistMBID", func() {
It("returns on first match", func() {
Expect(ag.GetArtistMBID(ctx, "123", "test")).To(Equal("mbid"))
@@ -201,102 +160,6 @@ var _ = Describe("Agents", func() {
})
})
Describe("cooldown", func() {
It("skips an agent that returned RetryLaterError until the deadline", func() {
mock.Err = &RetryLaterError{RetryIn: time.Hour}
_, err := ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(errors.Is(err, ErrRetryLater)).To(BeTrue())
// Immediately after: agent is skipped, not called
mock.Err = nil
calls := mock.Calls
_, err = ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(mock.Calls).To(Equal(calls))
Expect(errors.Is(err, ErrRetryLater)).To(BeTrue())
})
// Providers that throttle without saying for how long (Last.fm sends no delay at all)
// must still be parked, or the aggregate keeps calling them on every request.
It("parks an agent that asked to be retried without a delay", func() {
mock.Err = ErrRetryLater
_, err := ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(errors.Is(err, ErrRetryLater)).To(BeTrue())
mock.Err = nil
calls := mock.Calls
_, err = ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(mock.Calls).To(Equal(calls), "the default cooldown must outlast the request")
Expect(errors.Is(err, ErrRetryLater)).To(BeTrue())
})
It("calls the agent again once the cooldown expires", func() {
mock.Err = &RetryLaterError{RetryIn: 10 * time.Millisecond}
_, err := ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(errors.Is(err, ErrRetryLater)).To(BeTrue())
mock.Err = nil
Eventually(func() (string, error) {
return ag.GetArtistBiography(ctx, "id", "name", "mbid")
}, 5*time.Second, 10*time.Millisecond).Should(Equal("bio"))
})
It("returns ErrNotFound, not ErrRetryLater, when agents failed for other reasons", func() {
mock.Err = errors.New("boom")
_, err := ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(errors.Is(err, ErrNotFound)).To(BeTrue())
Expect(errors.Is(err, ErrRetryLater)).To(BeFalse())
})
// ErrRetryLater tells the caller "nobody answered, do not cache this". A definitive
// answer from any other agent is an answer, throttled peer or not.
It("returns ErrNotFound when another agent answered with a definitive miss", func() {
other := &mockAgent{Err: ErrNotFound}
Register("fake2", func(model.DataStore) Interface { return other })
conf.Server.Agents = "fake,fake2"
ag = createAgents(ds, nil)
mock.Err = &RetryLaterError{RetryIn: time.Hour}
_, err := ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(errors.Is(err, ErrNotFound)).To(BeTrue())
Expect(errors.Is(err, ErrRetryLater)).To(BeFalse())
// The cooldown was still recorded for the throttled agent
calls := mock.Calls
_, _ = ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(mock.Calls).To(Equal(calls))
})
It("returns ErrNotFound when another agent answered with an empty slice", func() {
empty := &testImageAgent{Name: "emptyImages"}
Register("emptyImages", func(model.DataStore) Interface { return empty })
conf.Server.Agents = "fake,emptyImages"
ag = createAgents(ds, nil)
mock.Err = &RetryLaterError{RetryIn: time.Hour}
_, err := ag.GetArtistImages(ctx, "123", "test", "mb123")
Expect(errors.Is(err, ErrNotFound)).To(BeTrue())
Expect(errors.Is(err, ErrRetryLater)).To(BeFalse())
})
It("returns ErrRetryLater from GetSimilarArtists when only cooling agents remain", func() {
mock.Err = &RetryLaterError{RetryIn: time.Hour}
_, err := ag.GetSimilarArtists(ctx, "123", "test", "mb123", 2)
Expect(errors.Is(err, ErrRetryLater)).To(BeTrue())
})
It("returns ErrNotFound from GetSimilarArtists when another agent answered", func() {
other := &mockAgent{Err: ErrNotFound}
Register("fake2", func(model.DataStore) Interface { return other })
conf.Server.Agents = "fake,fake2"
ag = createAgents(ds, nil)
mock.Err = &RetryLaterError{RetryIn: time.Hour}
_, err := ag.GetSimilarArtists(ctx, "123", "test", "mb123", 2)
Expect(errors.Is(err, ErrNotFound)).To(BeTrue())
Expect(errors.Is(err, ErrRetryLater)).To(BeFalse())
})
})
Describe("GetArtistImages", func() {
It("returns on first match", func() {
Expect(ag.GetArtistImages(ctx, "123", "test", "mb123")).To(Equal([]ExternalImage{{
@@ -560,9 +423,8 @@ var _ = Describe("Agents", func() {
})
type mockAgent struct {
Args []any
Err error
Calls int
Args []any
Err error
}
func (a *mockAgent) AgentName() string {
@@ -587,7 +449,6 @@ func (a *mockAgent) GetArtistURL(_ context.Context, id, name, mbid string) (stri
func (a *mockAgent) GetArtistBiography(_ context.Context, id, name, mbid string) (string, error) {
a.Args = []any{id, name, mbid}
a.Calls++
if a.Err != nil {
return "", a.Err
}
+3 -46
View File
@@ -3,9 +3,6 @@ package agents
import (
"context"
"errors"
"fmt"
"strconv"
"time"
"github.com/gohugoio/hashstructure"
"github.com/navidrome/navidrome/model"
@@ -55,49 +52,9 @@ func (s Song) Equals(other Song) bool {
return h1 == h2
}
// ErrNotFound means the provider answered and had nothing. Return the underlying error
// for a fault instead, or callers that back off on faults will treat it as definitive.
var ErrNotFound = errors.New("not found")
// ErrRetryLater is the zero-delay RetryLaterError: the provider is temporarily unavailable
// or throttling us, but did not say for how long. Both errors.Is(err, ErrRetryLater) and
// errors.AsType[*RetryLaterError] match it and every delay-carrying variant.
// Treat it as immutable; build a new RetryLaterError to name a delay.
var ErrRetryLater = &RetryLaterError{}
// RetryLaterError asks callers to back off, optionally for the delay the provider requested.
type RetryLaterError struct {
RetryIn time.Duration
}
func (e *RetryLaterError) Error() string {
if e.RetryIn > 0 {
return fmt.Sprintf("retry later (in %s)", e.RetryIn)
}
return "retry later"
}
func (e *RetryLaterError) Is(target error) bool {
_, ok := target.(*RetryLaterError)
return ok
}
// MaxRetryIn caps a delay parsed from a provider, so a bogus value cannot park it indefinitely.
const MaxRetryIn = time.Hour
const maxRetryInSeconds = int(MaxRetryIn / time.Second)
// ParseRetryIn reads a provider's delay given in seconds, from a header or a plugin token.
// Anything unparseable or non-positive means unspecified.
func ParseRetryIn(seconds string) time.Duration {
// Clamp in seconds: scaling first would wrap a huge value past int64 nanoseconds,
// turning "wait an age" into a fraction of a second. Parse at a fixed width so the
// cap holds on the 32-bit targets we ship, where a plain Atoi would overflow first.
secs, err := strconv.ParseInt(seconds, 10, 64)
if err != nil || secs <= 0 {
return 0
}
return time.Duration(min(secs, int64(maxRetryInSeconds))) * time.Second
}
var (
ErrNotFound = errors.New("not found")
)
// AlbumInfoRetriever provides album info (no images)
type AlbumInfoRetriever interface {
+16 -31
View File
@@ -1,42 +1,27 @@
package agents_test
package agents
import (
"errors"
"fmt"
"time"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/core/scrobbler"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("RetryLaterError", func() {
It("matches the ErrRetryLater sentinel via errors.Is", func() {
err := &agents.RetryLaterError{RetryIn: 30 * time.Second}
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
var _ = Describe("Song.Equals", func() {
base := Song{ID: "1", Name: "S", Artists: []Artist{{ID: "x", Name: "A"}}}
It("true for identical songs incl Artists", func() {
Expect(base.Equals(base)).To(BeTrue())
})
It("matches through errors.Join and wrapping", func() {
err := fmt.Errorf("calling LB: %w", errors.Join(errors.New("http 429"), &agents.RetryLaterError{}))
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
It("false when Artists differ", func() {
other := base
other.Artists = []Artist{{ID: "y", Name: "B"}}
Expect(base.Equals(other)).To(BeFalse())
})
It("exposes the delay through the wrapped error", func() {
err := errors.Join(errors.New("http 429"), &agents.RetryLaterError{RetryIn: 42 * time.Second})
retry, ok := errors.AsType[*agents.RetryLaterError](err)
Expect(ok).To(BeTrue())
Expect(retry.RetryIn).To(Equal(42 * time.Second))
It("false when a scalar differs", func() {
other := base
other.Name = "T"
Expect(base.Equals(other)).To(BeFalse())
})
It("matches the sentinel too, reporting no delay", func() {
retry, ok := errors.AsType[*agents.RetryLaterError](agents.ErrRetryLater)
Expect(ok).To(BeTrue())
Expect(retry.RetryIn).To(BeZero())
})
It("is the same sentinel as scrobbler.ErrRetryLater", func() {
Expect(errors.Is(scrobbler.ErrRetryLater, agents.ErrRetryLater)).To(BeTrue())
Expect(errors.Is(&agents.RetryLaterError{}, scrobbler.ErrRetryLater)).To(BeTrue())
It("true when both have empty Artists and equal scalars", func() {
a := Song{ID: "1", Name: "S"}
Expect(a.Equals(a)).To(BeTrue())
})
})
+7 -46
View File
@@ -5,8 +5,6 @@ import (
"github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/persistence"
"github.com/navidrome/navidrome/utils/slice"
)
const LocalAgentName = "local"
@@ -39,51 +37,14 @@ func (p *localAgent) GetArtistTopSongs(ctx context.Context, id, artistName, mbid
if err != nil {
return nil, err
}
return songsFrom(top), nil
}
func (p *localAgent) GetSimilarSongsByTrack(ctx context.Context, id, name, artist, mbid string, count int) ([]Song, error) {
seed, err := p.ds.MediaFile(ctx).Get(id)
if err != nil {
return nil, err
var result []Song
for _, s := range top {
result = append(result, Song{
Name: s.Title,
MBID: s.MbzReleaseTrackID,
})
}
// Tag ids derive from (name, value), so the seed's genre ids need no extra query.
genreIDs := slice.Map(seed.Tags.Flatten(model.TagGenre), func(t model.Tag) string { return t.ID })
if len(genreIDs) == 0 {
return nil, nil
}
// Ask for extra so we can drop the seed itself and still fill the count.
candidates, err := p.ds.MediaFile(ctx).GetRandom(model.QueryOptions{
Filters: squirrel.And{
persistence.SongGenres.ByID(genreIDs),
squirrel.Eq{"missing": false},
},
Max: count + 1,
})
if err != nil {
return nil, err
}
filtered := make(model.MediaFiles, 0, len(candidates))
for _, s := range candidates {
if s.ID == id {
continue
}
filtered = append(filtered, s)
if len(filtered) >= count {
break
}
}
return songsFrom(filtered), nil
}
func songsFrom(mfs model.MediaFiles) []Song {
if len(mfs) == 0 {
return nil
}
return slice.Map(mfs, func(mf model.MediaFile) Song {
return Song{ID: mf.ID, Name: mf.Title}
})
return result, nil
}
func init() {
-96
View File
@@ -1,96 +0,0 @@
package agents
import (
"context"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
"github.com/navidrome/navidrome/utils/slice"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("localAgent GetSimilarSongsByTrack", func() {
var ds *tests.MockDataStore
var mfRepo *tests.MockMediaFileRepo
var agent *localAgent
var ctx context.Context
BeforeEach(func() {
ctx = context.Background()
mfRepo = &tests.MockMediaFileRepo{}
ds = &tests.MockDataStore{MockedMediaFile: mfRepo}
agent = &localAgent{ds: ds}
})
It("excludes the seed track from its own similars", func() {
seed := model.MediaFile{ID: "seed-1", Title: "Seed", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
related := model.MediaFile{ID: "rel-1", Title: "Related", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
// SetData keys by ID; a duplicate "seed-1" entry would clobber the real seed.
mfRepo.SetData(model.MediaFiles{seed, related})
songs, err := agent.GetSimilarSongsByTrack(ctx, "seed-1", "Seed", "", "", 10)
Expect(err).ToNot(HaveOccurred())
names := slice.Map(songs, func(s Song) string { return s.Name })
Expect(names).ToNot(ContainElement("Seed"))
})
// The mock ignores QueryOptions.Filters, so assert the predicate itself: otherwise this spec
// would pass just as well with no genre filter at all.
It("queries the indexed genre join for the seed's own genres, skipping missing files", func() {
rock := model.NewTag(model.TagGenre, "Rock")
seed := model.MediaFile{ID: "seed-4", Title: "Seed", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
mfRepo.SetData(model.MediaFiles{seed})
_, err := agent.GetSimilarSongsByTrack(ctx, "seed-4", "Seed", "", "", 10)
Expect(err).ToNot(HaveOccurred())
sql, args, sqlErr := mfRepo.Options.Filters.ToSql()
Expect(sqlErr).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("media_file_tags"), "must use the indexed join, not a json_tree scan")
Expect(sql).To(ContainSubstring("missing"))
Expect(args).To(ContainElement(false), "must exclude missing files, not select them")
Expect(args).To(ContainElement(rock.ID), "must filter on the seed's own genre tag id")
Expect(args).ToNot(ContainElement(model.NewTag(model.TagGenre, "Jazz").ID))
})
It("returns the library id so the matcher can resolve the song", func() {
seed := model.MediaFile{ID: "seed-3", Title: "Seed", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
// Without the id the matcher falls through to its MBID/title phases and resolves nothing,
// so the local fallback silently returns an empty mix.
related := model.MediaFile{ID: "rel-3", Title: "Related", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
mfRepo.SetData(model.MediaFiles{seed, related})
songs, err := agent.GetSimilarSongsByTrack(ctx, "seed-3", "Seed", "", "", 10)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(ContainElement(Song{ID: "rel-3", Name: "Related"}))
})
It("asks for one extra candidate so dropping the seed still fills the count", func() {
// The mock returns rows sorted by id, so the seed comes first and would consume the only
// slot if the query did not over-fetch.
seed := model.MediaFile{ID: "a-seed", Title: "Seed", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
related := model.MediaFile{ID: "b-rel", Title: "Related", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
mfRepo.SetData(model.MediaFiles{seed, related})
songs, err := agent.GetSimilarSongsByTrack(ctx, "a-seed", "Seed", "", "", 1)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(HaveLen(1))
Expect(songs[0].Name).To(Equal("Related"))
})
It("returns nil when the seed track has no genres", func() {
seed := model.MediaFile{ID: "seed-2", Title: "NoGenre"}
mfRepo.SetData(model.MediaFiles{seed})
songs, err := agent.GetSimilarSongsByTrack(ctx, "seed-2", "NoGenre", "", "", 10)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(BeEmpty())
// Without the early return an empty tag filter would scan the whole library.
Expect(mfRepo.Options).To(Equal(model.QueryOptions{}), "must not query at all")
})
})
-27
View File
@@ -1,27 +0,0 @@
package agents
import (
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("Song.Equals", func() {
base := Song{ID: "1", Name: "S", Artists: []Artist{{ID: "x", Name: "A"}}}
It("true for identical songs incl Artists", func() {
Expect(base.Equals(base)).To(BeTrue())
})
It("false when Artists differ", func() {
other := base
other.Artists = []Artist{{ID: "y", Name: "B"}}
Expect(base.Equals(other)).To(BeFalse())
})
It("false when a scalar differs", func() {
other := base
other.Name = "T"
Expect(base.Equals(other)).To(BeFalse())
})
It("true when both have empty Artists and equal scalars", func() {
a := Song{ID: "1", Name: "S"}
Expect(a.Equals(a)).To(BeTrue())
})
})
+8 -13
View File
@@ -14,7 +14,6 @@ import (
"github.com/navidrome/navidrome/core/stream"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/persistence"
"github.com/navidrome/navidrome/utils/slice"
"github.com/navidrome/navidrome/utils/str"
)
@@ -22,7 +21,7 @@ import (
type Archiver interface {
ZipAlbum(ctx context.Context, id string, format string, bitrate int, w io.Writer) error
ZipArtist(ctx context.Context, id string, format string, bitrate int, w io.Writer) error
ZipShare(ctx context.Context, s *model.Share, w io.Writer) error
ZipShare(ctx context.Context, id string, w io.Writer) error
ZipPlaylist(ctx context.Context, id string, format string, bitrate int, w io.Writer) error
}
@@ -41,13 +40,7 @@ func (a *archiver) ZipAlbum(ctx context.Context, id string, format string, bitra
}
func (a *archiver) ZipArtist(ctx context.Context, id string, format string, bitrate int, out io.Writer) error {
// Match by album-artist participation, not the deprecated album_artist_id
// column (first album artist only), so co-album-artists are included too.
filter := squirrel.And{
persistence.ParticipantIDFilter("media_file", id, model.RoleAlbumArtist),
squirrel.Eq{"missing": false},
}
return a.zipAlbums(ctx, id, format, bitrate, out, filter)
return a.zipAlbums(ctx, id, format, bitrate, out, squirrel.Eq{"album_artist_id": id})
}
func (a *archiver) zipAlbums(ctx context.Context, id string, format string, bitrate int, out io.Writer, filters squirrel.Sqlizer) error {
@@ -107,14 +100,16 @@ func (a *archiver) albumFilename(mf model.MediaFile, format string, isMultiDisc
return fmt.Sprintf("%s/%s", str.SanitizeFilename(mf.Album), file)
}
// ZipShare takes an already-loaded share: Share.Load records a visit, so
// loading it again here would count every download twice.
func (a *archiver) ZipShare(ctx context.Context, s *model.Share, out io.Writer) error {
func (a *archiver) ZipShare(ctx context.Context, id string, out io.Writer) error {
s, err := a.shares.Load(ctx, id)
if err != nil {
return err
}
if !s.Downloadable {
return model.ErrNotAuthorized
}
log.Debug(ctx, "Zipping share", "name", s.ID, "format", s.Format, "bitrate", s.MaxBitRate, "numTracks", len(s.Tracks))
return a.zipMediaFiles(ctx, s.ID, s.ID, s.Format, s.MaxBitRate, out, s.Tracks, false)
return a.zipMediaFiles(ctx, id, s.ID, s.Format, s.MaxBitRate, out, s.Tracks, false)
}
func (a *archiver) ZipPlaylist(ctx context.Context, id string, format string, bitrate int, out io.Writer) error {
+4 -11
View File
@@ -11,7 +11,6 @@ import (
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/stream"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/persistence"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/stretchr/testify/mock"
@@ -70,11 +69,8 @@ var _ = Describe("Archiver", func() {
mfRepo := &mockMediaFileRepository{}
mfRepo.On("GetAll", []model.QueryOptions{{
Filters: squirrel.And{
persistence.ParticipantIDFilter("media_file", "1", model.RoleAlbumArtist),
squirrel.Eq{"missing": false},
},
Sort: "album",
Filters: squirrel.Eq{"album_artist_id": "1"},
Sort: "album",
}}).Return(mfs, nil)
ds.On("MediaFile", mock.Anything).Return(mfRepo)
@@ -134,16 +130,13 @@ var _ = Describe("Archiver", func() {
Tracks: mfs,
}
sh.On("Load", mock.Anything, "1").Return(share, nil)
ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2)
out := new(bytes.Buffer)
err := arch.ZipShare(context.Background(), share, out)
err := arch.ZipShare(context.Background(), "1", out)
Expect(err).To(BeNil())
// Share.Load records a visit; re-loading here would double-count
// every download.
sh.AssertNotCalled(GinkgoT(), "Load", mock.Anything, mock.Anything)
zr, err := zip.NewReader(bytes.NewReader(out.Bytes()), int64(out.Len()))
Expect(err).To(BeNil())
+43 -51
View File
@@ -9,12 +9,13 @@ import (
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/str"
)
// externalName mirrors the normalization the aggregate provider applies, so agent searches match.
// externalName applies the DevPreserveUnicodeInExternalCalls normalization the aggregate
// provider used, so agent searches match the same way (typographic quotes/dashes cleared
// unless preserved).
func externalName(name string) string {
if conf.Server.DevPreserveUnicodeInExternalCalls {
return name
@@ -22,8 +23,23 @@ func externalName(name string) string {
return str.Clear(name)
}
// bestImageURL returns the largest fetchable image URL. Only one is returned and its failure ends
// the agent's turn, so an unfetchable candidate must never win: url.Parse alone accepts anything.
// gateFunc gates one named external fetch (rate limit + circuit breaker per name).
// resolveItem defaults to passthroughGate; the worker injects the per-agent gate.
type gateFunc = func(name string, f func() (io.ReadCloser, string, error)) (io.ReadCloser, string, error)
func passthroughGate(_ string, f func() (io.ReadCloser, string, error)) (io.ReadCloser, string, error) {
return f()
}
// denyGate refuses every external fetch with a definitive not-found, so local-only
// resolution never runs a network step even if an external branch is reached.
func denyGate(_ string, _ func() (io.ReadCloser, string, error)) (io.ReadCloser, string, error) {
return nil, "", model.ErrNotFound
}
// bestImageURL returns the largest-Size image URL, skipping empty or unparseable
// URLs; nil when none qualifies. Parsing happens per candidate so a malformed largest
// URL never shadows a valid smaller one.
func bestImageURL(imgs []agents.ExternalImage) *url.URL {
var best *url.URL
var bestSize int
@@ -32,7 +48,7 @@ func bestImageURL(imgs []agents.ExternalImage) *url.URL {
continue
}
u, err := url.Parse(imgs[i].URL)
if err != nil || !u.IsAbs() || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" {
if err != nil {
continue
}
if best == nil || imgs[i].Size > bestSize {
@@ -42,38 +58,19 @@ func bestImageURL(imgs []agents.ExternalImage) *url.URL {
return best
}
// longerRetry keeps whichever external failure asks for the longer wait, so one provider's
// short delay cannot shorten another's.
func longerRetry(a, b error) error {
if a == nil {
return b
}
var ra, rb *agents.RetryLaterError
if errors.As(b, &rb) && (!errors.As(a, &ra) || rb.RetryIn > ra.RetryIn) {
return b
}
return a
}
// fetchArtistImage tries each enabled artist-image agent in order. The error is non-nil only when no
// agent succeeded and at least one failed transiently.
func fetchArtistImage(ctx context.Context, ag *agents.Agents, gate gateFunc, ar model.Artist) (io.ReadCloser, string, error) {
// Synthetic artists would otherwise get an unrelated agent result assigned to them.
// fetchArtistImage tries each enabled artist-image agent in order, each under its own gate.
// Returns the winning reader + agent name; extErr is true only when NO agent succeeded and
// at least one failed transiently (a later success beats an earlier agent error).
func fetchArtistImage(ctx context.Context, ag *agents.Agents, gate gateFunc, ar model.Artist) (r io.ReadCloser, agentName string, extErr bool) {
// Synthetic artists have no real external image; mirror Agents.GetArtistImages' guard so a
// direct retriever call can't assign an unrelated result to Unknown/Various Artists.
switch ar.ID {
case consts.UnknownArtistID, consts.VariousArtistsID:
traceFrom(ctx).add(TraceStep{Candidate: externalCandidate, Outcome: OutcomeSkipped, Detail: "synthetic artist"})
return nil, "", nil
return nil, "", false
}
name := externalName(ar.Name)
imageAgents := ag.ArtistImageAgents()
if len(imageAgents) == 0 {
traceFrom(ctx).add(TraceStep{Candidate: externalCandidate, Outcome: OutcomeSkipped,
Detail: "no enabled agent provides artist images"})
return nil, "", nil
}
var extErr error
for _, a := range imageAgents {
reader, path, err := gate(a.Name, func() (io.ReadCloser, string, error) {
for _, a := range ag.ArtistImageAgents() {
reader, _, err := gate(a.Name, func() (io.ReadCloser, string, error) {
imgs, err := a.Retriever.GetArtistImages(ctx, ar.ID, name, ar.MbzArtistID)
if err != nil {
return nil, "", err
@@ -84,30 +81,21 @@ func fetchArtistImage(ctx context.Context, ag *agents.Agents, gate gateFunc, ar
}
return fromURL(ctx, u)
})
recordAgent(ctx, a.Name, reader, path, err)
if reader != nil {
return reader, a.Name, nil
return reader, a.Name, false
}
if isTransientExternal(err) {
extErr = longerRetry(extErr, err)
log.Debug(ctx, "Artwork: External artist-image lookup failed", "agent", a.Name, "artist", ar.Name, err)
extErr = true // includes errBreakerOpen and download failures: retry via the next agent
}
}
return nil, "", extErr
}
// fetchAlbumImage is the album counterpart of fetchArtistImage.
func fetchAlbumImage(ctx context.Context, ag *agents.Agents, gate gateFunc, al model.Album) (io.ReadCloser, string, error) {
func fetchAlbumImage(ctx context.Context, ag *agents.Agents, gate gateFunc, al model.Album) (r io.ReadCloser, agentName string, extErr bool) {
name, artist := externalName(al.Name), externalName(al.AlbumArtist)
imageAgents := ag.AlbumImageAgents()
if len(imageAgents) == 0 {
traceFrom(ctx).add(TraceStep{Candidate: externalCandidate, Outcome: OutcomeSkipped,
Detail: "no enabled agent provides album images"})
return nil, "", nil
}
var extErr error
for _, a := range imageAgents {
reader, path, err := gate(a.Name, func() (io.ReadCloser, string, error) {
for _, a := range ag.AlbumImageAgents() {
reader, _, err := gate(a.Name, func() (io.ReadCloser, string, error) {
imgs, err := a.Retriever.GetAlbumImages(ctx, name, artist, al.MbzAlbumID)
if err != nil {
return nil, "", err
@@ -118,14 +106,18 @@ func fetchAlbumImage(ctx context.Context, ag *agents.Agents, gate gateFunc, al m
}
return fromURL(ctx, u)
})
recordAgent(ctx, a.Name, reader, path, err)
if reader != nil {
return reader, a.Name, nil
return reader, a.Name, false
}
if isTransientExternal(err) {
extErr = longerRetry(extErr, err)
log.Debug(ctx, "Artwork: External album-image lookup failed", "agent", a.Name, "album", al.Name, err)
extErr = true
}
}
return nil, "", extErr
}
// isTransientExternal reports whether an external step failed in a way worth retrying;
// a not-found (from either package) is a definitive answer, not a fault.
func isTransientExternal(err error) bool {
return err != nil && !errors.Is(err, agents.ErrNotFound) && !errors.Is(err, model.ErrNotFound)
}
+19 -125
View File
@@ -2,13 +2,10 @@ package artwork
import (
"context"
"errors"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
@@ -31,22 +28,13 @@ type fakeImageAgent struct {
albumCalls int
gotArtistName string
gotAlbumName string
// block, when set, holds every lookup until closed, standing in for a slow/rate-limited agent.
block chan struct{}
// mu guards the call counters: the worker resolves several items concurrently.
mu sync.Mutex
}
func (f *fakeImageAgent) AgentName() string { return f.name }
func (f *fakeImageAgent) GetArtistImages(_ context.Context, _, name, _ string) ([]agents.ExternalImage, error) {
if f.block != nil {
<-f.block
}
f.mu.Lock()
f.artistCalls++
f.gotArtistName = name
f.mu.Unlock()
return f.imgs, f.err
}
@@ -56,8 +44,9 @@ func (f *fakeImageAgent) GetAlbumImages(_ context.Context, name, _, _ string) ([
return f.imgs, f.err
}
// imageAgents registers the fakes as built-in agents and enables them in order. The fakes
// ignore the DataStore, so reusing the process-wide GetAgents singleton across tests is safe.
// imageAgents registers the fakes as built-in agents (ignoring the DataStore) and
// enables them in order, returning the process-wide Agents. Because the fakes ignore
// ds, reusing the GetAgents singleton across tests is safe.
func imageAgents(fakes ...*fakeImageAgent) *agents.Agents {
names := make([]string, 0, len(fakes))
for _, f := range fakes {
@@ -112,42 +101,6 @@ var _ = Describe("agent images", func() {
Expect(bestImageURL(nil)).To(BeNil())
Expect(bestImageURL([]agents.ExternalImage{{URL: "", Size: 5}})).To(BeNil())
})
// Plugins hand these over as free-form strings, and url.Parse accepts them all. An
// unfetchable candidate that wins here ends the agent's turn before its valid images run.
DescribeTable("skips a candidate that cannot be fetched",
func(badURL string) {
u := bestImageURL([]agents.ExternalImage{
{URL: badURL, Size: 100}, // largest, and first
{URL: "https://cdn.example.com/ok.jpg", Size: 10},
})
Expect(u).ToNot(BeNil())
Expect(u.String()).To(Equal("https://cdn.example.com/ok.jpg"))
},
Entry("a relative path", "images/big.jpg"),
Entry("a root-relative path", "/images/big.jpg"),
Entry("a scheme we cannot fetch", "ftp://host/big.jpg"),
Entry("a scheme-relative URL", "//host/big.jpg"),
Entry("a URL with no host", "http:///big.jpg"),
)
// Size is often 0 for every candidate, and only a strictly larger one replaces the first,
// so an unfetchable entry in first position would otherwise stick.
It("skips an unfetchable first candidate when every Size is zero", func() {
u := bestImageURL([]agents.ExternalImage{
{URL: "images/rel.jpg"},
{URL: "https://cdn.example.com/ok.jpg"},
})
Expect(u).ToNot(BeNil())
Expect(u.String()).To(Equal("https://cdn.example.com/ok.jpg"))
})
It("returns nil when no candidate is fetchable", func() {
Expect(bestImageURL([]agents.ExternalImage{
{URL: "images/a.jpg", Size: 10},
{URL: "ftp://host/b.jpg", Size: 20},
})).To(BeNil())
})
})
Describe("fetchArtistImage", func() {
@@ -155,11 +108,11 @@ var _ = Describe("agent images", func() {
a := &fakeImageAgent{name: "agentA", imgs: []agents.ExternalImage{img("/a", 100)}}
ag := imageAgents(a)
r, name, err := fetchArtistImage(ctx, ag, passthroughGate, model.Artist{ID: "ar1", Name: "Artist"})
r, name, extErr := fetchArtistImage(ctx, ag, passthroughGate, model.Artist{ID: "ar1", Name: "Artist"})
Expect(r).ToNot(BeNil())
defer r.Close()
Expect(name).To(Equal("agentA"))
Expect(err).ToNot(HaveOccurred())
Expect(extErr).To(BeFalse())
})
It("skips the external lookup for synthetic artists", func() {
@@ -167,38 +120,14 @@ var _ = Describe("agent images", func() {
ag := imageAgents(a)
for _, id := range []string{consts.UnknownArtistID, consts.VariousArtistsID} {
r, name, err := fetchArtistImage(ctx, ag, passthroughGate, model.Artist{ID: id, Name: "Various Artists"})
r, name, extErr := fetchArtistImage(ctx, ag, passthroughGate, model.Artist{ID: id, Name: "Various Artists"})
Expect(r).To(BeNil())
Expect(name).To(BeEmpty())
Expect(err).ToNot(HaveOccurred())
Expect(extErr).To(BeFalse())
}
Expect(a.artistCalls).To(Equal(0), "synthetic artists never reach the agents")
})
It("records a skipped external candidate when no agent provides artist images", func() {
ag := imageAgents()
t := &ChainTrace{}
r, _, err := fetchArtistImage(withTrace(ctx, t), ag, passthroughGate, model.Artist{ID: "ar1"})
Expect(r).To(BeNil())
Expect(err).ToNot(HaveOccurred())
Expect(t.Steps()).To(Equal([]TraceStep{{Candidate: "external", Outcome: OutcomeSkipped,
Detail: "no enabled agent provides artist images"}}),
"a configured external token must never be silently absent from the chain")
})
It("records a skipped external candidate for synthetic artists", func() {
a := &fakeImageAgent{name: "agentA", imgs: []agents.ExternalImage{img("/a", 100)}}
ag := imageAgents(a)
t := &ChainTrace{}
_, _, _ = fetchArtistImage(withTrace(ctx, t), ag, passthroughGate,
model.Artist{ID: consts.VariousArtistsID, Name: "Various Artists"})
Expect(t.Steps()).To(HaveLen(1))
Expect(t.Steps()[0].Outcome).To(Equal(OutcomeSkipped))
Expect(t.Steps()[0].Detail).To(ContainSubstring("synthetic"))
})
It("clears typographic characters from the query name unless preserving unicode", func() {
conf.Server.DevPreserveUnicodeInExternalCalls = false
a := &fakeImageAgent{name: "agentA"}
@@ -213,11 +142,11 @@ var _ = Describe("agent images", func() {
b := &fakeImageAgent{name: "agentB", imgs: []agents.ExternalImage{img("/b", 50)}}
ag := imageAgents(a, b)
r, name, err := fetchArtistImage(ctx, ag, passthroughGate, model.Artist{ID: "ar1"})
r, name, extErr := fetchArtistImage(ctx, ag, passthroughGate, model.Artist{ID: "ar1"})
Expect(r).ToNot(BeNil())
defer r.Close()
Expect(name).To(Equal("agentB"))
Expect(err).ToNot(HaveOccurred(), "a later hit clears an earlier agent's error")
Expect(extErr).To(BeFalse(), "a later hit clears an earlier agent's error")
Expect(a.artistCalls).To(Equal(1))
Expect(b.artistCalls).To(Equal(1))
})
@@ -227,43 +156,20 @@ var _ = Describe("agent images", func() {
b := &fakeImageAgent{name: "agentB", err: agents.ErrNotFound}
ag := imageAgents(a, b)
r, name, err := fetchArtistImage(ctx, ag, passthroughGate, model.Artist{ID: "ar1"})
r, name, extErr := fetchArtistImage(ctx, ag, passthroughGate, model.Artist{ID: "ar1"})
Expect(r).To(BeNil())
Expect(name).To(BeEmpty())
Expect(err).ToNot(HaveOccurred(), "not-found is definitive, never a transient failure")
Expect(extErr).To(BeFalse(), "not-found is definitive, never a transient failure")
})
It("reports an error when one agent fails transiently and the rest find nothing", func() {
It("reports extErr when one agent fails transiently and the rest find nothing", func() {
a := &fakeImageAgent{name: "agentA", err: agents.ErrNotFound}
b := &fakeImageAgent{name: "agentB", err: context.DeadlineExceeded}
ag := imageAgents(a, b)
r, _, err := fetchArtistImage(ctx, ag, passthroughGate, model.Artist{ID: "ar1"})
r, _, extErr := fetchArtistImage(ctx, ag, passthroughGate, model.Artist{ID: "ar1"})
Expect(r).To(BeNil())
Expect(err).To(HaveOccurred())
})
// The worker reschedules on this delay, so it is only honored if the agent loop
// returns it. Two throttled agents: the longest wait is the one that must survive.
It("returns the longest retry delay the providers asked for", func() {
a := &fakeImageAgent{name: "agentA", err: &agents.RetryLaterError{RetryIn: 10 * time.Second}}
b := &fakeImageAgent{name: "agentB", err: &agents.RetryLaterError{RetryIn: 5 * time.Second}}
ag := imageAgents(a, b)
r, _, err := fetchArtistImage(ctx, ag, passthroughGate, model.Artist{ID: "ar1"})
Expect(r).To(BeNil())
retry, ok := errors.AsType[*agents.RetryLaterError](err)
Expect(ok).To(BeTrue())
Expect(retry.RetryIn).To(Equal(10 * time.Second))
})
It("returns no delay when the provider did not ask for one", func() {
ag := imageAgents(&fakeImageAgent{name: "agentA", err: errors.New("boom")})
_, _, err := fetchArtistImage(ctx, ag, passthroughGate, model.Artist{ID: "ar1"})
Expect(err).To(HaveOccurred())
_, ok := errors.AsType[*agents.RetryLaterError](err)
Expect(ok).To(BeFalse(), "a plain failure must not look like a throttle")
Expect(extErr).To(BeTrue())
})
})
@@ -272,33 +178,21 @@ var _ = Describe("agent images", func() {
a := &fakeImageAgent{name: "agentA", imgs: []agents.ExternalImage{img("/a", 100)}}
ag := imageAgents(a)
r, name, err := fetchAlbumImage(ctx, ag, passthroughGate, model.Album{Name: "Album", AlbumArtist: "Artist"})
r, name, extErr := fetchAlbumImage(ctx, ag, passthroughGate, model.Album{Name: "Album", AlbumArtist: "Artist"})
Expect(r).ToNot(BeNil())
defer r.Close()
Expect(name).To(Equal("agentA"))
Expect(err).ToNot(HaveOccurred())
Expect(extErr).To(BeFalse())
Expect(a.albumCalls).To(Equal(1))
})
It("records a skipped external candidate when no agent provides album images", func() {
ag := imageAgents()
t := &ChainTrace{}
r, _, err := fetchAlbumImage(withTrace(ctx, t), ag, passthroughGate, model.Album{Name: "Album"})
Expect(r).To(BeNil())
Expect(err).ToNot(HaveOccurred())
Expect(t.Steps()).To(Equal([]TraceStep{{Candidate: "external", Outcome: OutcomeSkipped,
Detail: "no enabled agent provides album images"}}),
"a configured external token must never be silently absent from the chain")
})
It("reports an error when the only agent fails transiently", func() {
It("reports extErr when the only agent fails transiently", func() {
a := &fakeImageAgent{name: "agentA", err: context.DeadlineExceeded}
ag := imageAgents(a)
r, _, err := fetchAlbumImage(ctx, ag, passthroughGate, model.Album{Name: "Album"})
r, _, extErr := fetchAlbumImage(ctx, ag, passthroughGate, model.Album{Name: "Album"})
Expect(r).To(BeNil())
Expect(err).To(HaveOccurred())
Expect(extErr).To(BeTrue())
})
})
-434
View File
@@ -1,434 +0,0 @@
package artwork
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"os"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/core/ffmpeg"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/resources"
"github.com/navidrome/navidrome/utils/cache"
)
var ErrUnavailable = errors.New("artwork unavailable")
// errStaleSource means the backing file's mtime no longer matches RefMtime, so the stored hash may be stale.
var errStaleSource = errors.New("artwork: source file changed since resolution")
// Image is one servable artwork response.
type Image struct {
io.ReadCloser
Hash string // pixel identity; "" for placeholders
ETag string // representation validator; "" means Hash applies (full-size original)
LastUpdated time.Time
Placeholder bool
}
// representationTag varies with dimensions and encode settings, so a config change invalidates
// a revalidating client's cache even though the pixel hash is unchanged.
func representationTag(hash string, size int, square bool) string {
return fmt.Sprintf("%s.%d.%v.%s", hash, size, square, formatQualityTag())
}
type Artwork interface {
// Get returns ErrUnavailable when there is nothing to serve and model.ErrNotFound when
// the id resolves to nothing, so the caller can pick placeholder vs 404.
Get(ctx context.Context, artID model.ArtworkID, size int, square bool) (*Image, error)
// GetOrPlaceholder accepts an artwork token or a raw entity id, falling back to the
// kind's placeholder image (never resized, Placeholder=true).
GetOrPlaceholder(ctx context.Context, id string, size int, square bool) (*Image, error)
}
func NewArtwork(ds model.DataStore, cache cache.FileCache, store *ImageStore, ffm ffmpeg.FFmpeg) Artwork {
return &service{ds: ds, cache: cache, store: store, ffmpeg: ffm}
}
// entityExists reports whether the entity an artwork id points at is still there: state rows
// outlive a deleted entity until the next prune, so a servable row is not evidence of its owner.
func entityExists(ctx context.Context, ds model.DataStore, artID model.ArtworkID) bool {
var found bool
var err error
switch artID.Kind {
case model.KindArtistArtwork:
found, err = ds.Artist(ctx).Exists(artID.ID)
case model.KindAlbumArtwork:
found, err = ds.Album(ctx).Exists(artID.ID)
case model.KindMediaFileArtwork:
found, err = ds.MediaFile(ctx).Exists(artID.ID)
case model.KindPlaylistArtwork:
found, err = ds.Playlist(ctx).Exists(artID.ID)
case model.KindRadioArtwork:
found, err = ds.Radio(ctx).Exists(artID.ID)
case model.KindDiscArtwork:
albumID, _, perr := model.ParseDiscArtworkID(artID.ID)
if perr != nil {
return false
}
found, err = ds.Album(ctx).Exists(albumID)
default:
return false
}
return err == nil && found
}
type service struct {
ds model.DataStore
cache cache.FileCache
store *ImageStore
ffmpeg ffmpeg.FFmpeg
}
func (s *service) GetOrPlaceholder(ctx context.Context, id string, size int, square bool) (*Image, error) {
artID, err := s.parseArtworkID(ctx, id)
var img *Image
if err == nil {
img, err = s.Get(ctx, artID, size, square)
}
// Only a resolvable entity with no art gets a placeholder; an unknown id must stay
// ErrNotFound so callers can still answer 404 / Subsonic error 70.
if errors.Is(err, ErrUnavailable) {
return placeholderImage(artID.Kind), nil
}
return img, err
}
func (s *service) Get(ctx context.Context, artID model.ArtworkID, size int, square bool) (*Image, error) {
if artID.ID == "" {
return nil, ErrUnavailable
}
if size < 0 {
size = 0 // a negative size means full-size, not a giant (OOM) resize rectangle
}
switch artID.Kind {
case model.KindDiscArtwork:
return s.serveDisc(ctx, artID, size, square)
case model.KindMediaFileArtwork:
return s.serveMediaFile(ctx, artID, size, square)
default:
return s.serveEntity(ctx, artID, size, square)
}
}
func (s *service) serveEntity(ctx context.Context, artID model.ArtworkID, size int, square bool) (*Image, error) {
ia, err := s.ds.Artwork(ctx).GetItemArtwork(artID.Kind, artID.ID, model.ImageTypePrimary)
switch {
case errors.Is(err, model.ErrNotFound):
return s.provisional(ctx, artID, size, square)
case err != nil:
return nil, err
case ia.Hash == "":
// Settled absent: only an explicit reprocess or refresh retries it.
return nil, ErrUnavailable
default:
return s.serveHash(ctx, artID, ia, size, square)
}
}
// serveSource is the one place bytes become an Image. hash is the pixel identity ("" for disc art)
// and doubles as the full-size validator, so an ETag is only needed when resized or hash is "".
func (s *service) serveSource(ctx context.Context, key, hash string, lastUpdate time.Time,
size int, square bool, open func() (io.ReadCloser, error),
) (*Image, error) {
if size == 0 && !square {
rc, err := open()
if err != nil {
return nil, err
}
if rc == nil {
return nil, ErrUnavailable
}
img := &Image{ReadCloser: rc, Hash: hash, LastUpdated: lastUpdate}
if hash == "" {
img.ETag = representationTag(key, size, square)
}
return img, nil
}
stream, err := s.cache.Get(ctx, &resizedItem{
hash: key, size: size, square: square, ffmpeg: s.ffmpeg, open: open,
})
if err != nil {
return nil, err
}
return &Image{ReadCloser: stream, Hash: hash, ETag: representationTag(key, size, square), LastUpdated: lastUpdate}, nil
}
// serveHash serves the bytes of a found state row. A mismatch/open error is dangling, but a
// cancelled request is not: it must not enqueue a re-resolution.
func (s *service) serveHash(ctx context.Context, artID model.ArtworkID, ia *model.ItemArtwork, size int, square bool) (*Image, error) {
// Only this path can hand back a deleted entity's bytes; the others load their entity anyway.
if !entityExists(ctx, s.ds, artID) {
return nil, ErrUnavailable
}
art, err := s.ds.Artwork(ctx).GetImage(ia.Hash)
if err != nil {
if errors.Is(err, model.ErrNotFound) {
return s.dangling(ctx, artID)
}
return nil, err
}
img, err := s.serveSource(ctx, ia.Hash, ia.Hash, ia.UpdatedAt, size, square,
func() (io.ReadCloser, error) { return openOriginal(ia, art.Mime, s.store) })
if err != nil {
if errors.Is(err, context.Canceled) {
return nil, err
}
log.Warn(ctx, "Artwork: Could not serve image", "artID", artID, "size", size, err)
return s.dangling(ctx, artID)
}
return img, nil
}
// openOriginal enforces the mtime invariant: bytes are never served under a hash they no longer match.
func openOriginal(ia *model.ItemArtwork, mime string, store *ImageStore) (io.ReadCloser, error) {
if isFileBacked(ia.Source) {
f, err := os.Open(ia.SourcePath)
if err != nil {
return nil, err
}
info, err := f.Stat()
if err != nil {
f.Close()
return nil, err
}
if ia.RefMtime != 0 && info.ModTime().UnixNano() != ia.RefMtime {
f.Close()
log.Debug("Artwork: Backing file changed since resolution", "path", ia.SourcePath,
"hash", ia.Hash, "resolvedMtime", ia.RefMtime, "currentMtime", info.ModTime().UnixNano())
return nil, errStaleSource
}
return f, nil
}
// Store-backed bytes still carry the source's mtime, to detect edits to embedded art.
if ia.SourcePath != "" && ia.RefMtime != 0 {
info, err := os.Stat(ia.SourcePath)
if err != nil {
return nil, err
}
if info.ModTime().UnixNano() != ia.RefMtime {
log.Debug("Artwork: Source file changed since resolution", "path", ia.SourcePath,
"hash", ia.Hash, "resolvedMtime", ia.RefMtime, "currentMtime", info.ModTime().UnixNano())
return nil, errStaleSource
}
}
return store.Open(ia.Hash, mime)
}
// provisional serves local bytes for an entity with no state row, enqueuing the worker but
// never writing a state row itself.
func (s *service) provisional(ctx context.Context, artID model.ArtworkID, size int, square bool) (*Image, error) {
item := model.ArtworkQueueItem{ItemKind: artID.Kind.Prefix(), ItemID: artID.ID, ImageType: model.ImageTypePrimary}
res, err := newLocalResolver(s.ds, s.ffmpeg).resolve(ctx, item)
if err != nil {
return nil, err
}
s.enqueue(ctx, artID, model.ArtworkPriorityBump)
log.Debug(ctx, "Artwork: Provisional read-through, no state row yet", "artID", artID,
"source", res.source, "hit", res.reader != nil)
return s.serveResolution(ctx, res, size, square)
}
// serveResolution turns a local resolution's bytes into a servable Image (byte-hash only, no decode).
func (s *service) serveResolution(ctx context.Context, res resolution, size int, square bool) (*Image, error) {
if res.reader == nil {
return nil, ErrUnavailable
}
defer res.reader.Close()
data, err := readCapped(res.reader)
if err != nil {
return nil, ErrUnavailable
}
hash, err := hashImage(bytes.NewReader(data))
if err != nil {
return nil, ErrUnavailable
}
// Keyed by the byte-hash, so the entry lines up with the worker's eventual store entry.
return s.serveSource(ctx, hash, hash, unixMtime(res.refMtime), size, square,
func() (io.ReadCloser, error) { return io.NopCloser(bytes.NewReader(data)), nil })
}
func (s *service) serveMediaFile(ctx context.Context, artID model.ArtworkID, size int, square bool) (*Image, error) {
// The setting is not in the config fingerprint, so honor it at serve time: a direct mf- URL
// must fall back to disc/album instead of serving stale persisted embedded art.
if !conf.Server.EnableMediaFileCoverArt {
mf, err := s.ds.MediaFile(ctx).Get(artID.ID)
if err != nil {
return nil, err
}
return s.Get(ctx, mf.DiscCoverArtID(), size, square)
}
ia, err := s.ds.Artwork(ctx).GetItemArtwork(model.KindMediaFileArtwork, artID.ID, model.ImageTypePrimary)
switch {
case err == nil && ia.Hash != "":
return s.serveHash(ctx, artID, ia, size, square)
case err == nil:
// absent row: fall through
case errors.Is(err, model.ErrNotFound):
// no row: fall through
default:
return nil, err
}
noRow := errors.Is(err, model.ErrNotFound)
mf, err := s.ds.MediaFile(ctx).Get(artID.ID)
if err != nil {
return nil, err
}
if noRow && conf.Server.EnableMediaFileCoverArt && mf.HasCoverArt {
return s.provisionalEmbedded(ctx, artID, *mf, size, square)
}
// Mirror MediaFile.CoverArtID: a track defers to its disc art, which falls back to the album.
return s.Get(ctx, mf.DiscCoverArtID(), size, square)
}
// provisionalEmbedded serves a track's embedded art immediately, leaving the state row to the worker.
func (s *service) provisionalEmbedded(ctx context.Context, artID model.ArtworkID, mf model.MediaFile, size int, square bool) (*Image, error) {
lib, err := loadLibraryView(ctx, s.ds, mf.LibraryID)
if err != nil {
return nil, err
}
res, ok := resolveEmbedded(ctx, lib, s.ffmpeg, mf.Path)
s.enqueue(ctx, artID, model.ArtworkPriorityBump)
if !ok {
// Eligible but unextractable: fall back the way CoverArtID does, not to a placeholder.
return s.Get(ctx, mf.DiscCoverArtID(), size, square)
}
return s.serveResolution(ctx, res, size, square)
}
// serveDisc reads disc art through with no state row and no enqueue, falling back to the album cover.
func (s *service) serveDisc(ctx context.Context, artID model.ArtworkID, size int, square bool) (*Image, error) {
dr, err := newDiscArtworkReader(ctx, s.ds, artID)
if err != nil {
return nil, err
}
// Single-disc albums run the chain too: a disc can carry art distinct from the album cover.
selectImage := func() (io.ReadCloser, error) {
res, err := dr.selectImage(ctx, s.ffmpeg, conf.Server.DiscArtPriority, &chainState{})
return res.reader, err
}
albumArtID := model.ArtworkID{Kind: model.KindAlbumArtwork, ID: dr.album.ID}
// Disc art has no state row, hence no content hash: keying on id, album mtime and
// DiscArtPriority lets a warm cache answer without running the chain or touching the disk.
key := fmt.Sprintf("%s|%d|%s", artID.ID, dr.cacheTime().UnixNano(), conf.Server.DiscArtPriority)
img, err := s.serveSource(ctx, key, "", dr.cacheTime(), size, square, selectImage)
if err != nil {
if errors.Is(err, context.Canceled) {
return nil, err
}
return s.Get(ctx, albumArtID, size, square)
}
return img, nil
}
// dangling enqueues a re-resolution and reports unavailable, leaving the state row untouched.
func (s *service) dangling(ctx context.Context, artID model.ArtworkID) (*Image, error) {
log.Debug(ctx, "Artwork: State row points at bytes we cannot serve, re-resolving", "artID", artID)
s.enqueue(ctx, artID, model.ArtworkPriorityScan)
return nil, ErrUnavailable
}
func (s *service) enqueue(ctx context.Context, artID model.ArtworkID, priority int) {
err := s.ds.ArtworkQueue(ctx).EnqueuePreservingBackoff(model.ArtworkQueueItem{
ItemKind: artID.Kind.Prefix(),
ItemID: artID.ID,
ImageType: model.ImageTypePrimary,
Priority: priority,
})
if err != nil {
log.Warn(ctx, "Artwork: Could not enqueue re-resolution", "artID", artID, err)
}
}
func placeholderImage(kind model.Kind) *Image {
path := consts.PlaceholderAlbumArt
if kind == model.KindArtistArtwork {
path = consts.PlaceholderArtistArt
}
r, _ := resources.FS().Open(path)
return &Image{ReadCloser: r, Placeholder: true}
}
type coverArtIDGetter interface {
CoverArtID() model.ArtworkID
}
// parseArtworkID accepts an artwork token or a raw entity id, resolving the latter to its CoverArtID.
func (s *service) parseArtworkID(ctx context.Context, id string) (model.ArtworkID, error) {
if id == "" {
return model.ArtworkID{}, ErrUnavailable
}
if artID, err := model.ParseArtworkID(id); err == nil {
return artID, nil
}
entity, err := model.GetEntityByID(ctx, s.ds, id)
if err != nil {
return model.ArtworkID{}, err
}
if e, ok := entity.(coverArtIDGetter); ok {
return e.CoverArtID(), nil
}
return model.ArtworkID{}, model.ErrNotFound
}
// TracingResolver is the CLI's read-only view of resolution: it walks the priority chain, records
// the walk and reports the winning source, without ever writing artwork state.
type TracingResolver struct {
inner *resolver
trace *ChainTrace
}
// NewTracingResolver builds a TracingResolver that records its priority-chain walk. Without live
// it gets no agents at all, so neither a chain nor any fallback added later can reach a provider;
// with it, one item is at most one call per agent, so the rate limiter and breaker are bypassed.
func NewTracingResolver(ds model.DataStore, ag *agents.Agents, ffm ffmpeg.FFmpeg, t *ChainTrace, live bool) *TracingResolver {
inner := newLocalResolver(ds, ffm)
if live {
inner = newResolver(ds, ag, ffm, passthroughGate)
}
return &TracingResolver{inner: inner, trace: t}
}
// Resolve walks kind's sources for id, recording the walk, and reports the winning source
// ("" when none produced an image).
func (r *TracingResolver) Resolve(ctx context.Context, kind model.Kind, id string) (string, error) {
switch kind {
case model.KindArtistArtwork:
return r.explain(ctx, r.inner.resolveArtist, id)
case model.KindAlbumArtwork:
return r.explain(ctx, r.inner.resolveAlbum, id)
case model.KindDiscArtwork:
return r.explain(ctx, r.inner.resolveDisc, id)
case model.KindMediaFileArtwork:
return r.explain(ctx, r.inner.resolveMediaFile, id)
}
return "", fmt.Errorf("artwork: %s artwork has no chain to explain", kind)
}
// explain discards the bytes: nothing downstream persists this resolution, so nothing else
// would close the reader either.
func (r *TracingResolver) explain(ctx context.Context, resolve func(context.Context, string) (resolution, error), id string) (string, error) {
res, err := resolve(withTrace(ctx, r.trace), id)
if err != nil {
return "", err
}
if res.reader != nil {
_ = res.reader.Close()
}
return res.source, nil
}
func unixMtime(mtime int64) time.Time {
if mtime <= 0 {
return time.Time{}
}
return time.Unix(0, mtime) // RefMtime is unix-nanoseconds
}
+1 -35
View File
@@ -11,7 +11,6 @@ import (
"github.com/navidrome/navidrome/core/storage"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/metadata"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
@@ -38,6 +37,7 @@ func TestArtwork(t *testing.T) {
}
// osDirFS wraps os.DirFS as a storage.MusicFS for integration tests.
// ReadTags is not exercised by these tests, so it is left as a stub.
type osDirFS struct{ fs.FS }
func (o osDirFS) ReadTags(...string) (map[string]metadata.Info, error) { return nil, nil }
@@ -81,37 +81,3 @@ func (s *osDirStorage) FS() (storage.MusicFS, error) {
}
return osDirFS{os.DirFS(s.root)}, nil
}
// fakeFolderRepo covers the three FolderRepository methods the resolvers reach for. The zero value
// answers as an unremarkable library does; the fields drive the album-root lookup and its failures.
type fakeFolderRepo struct {
model.FolderRepository
result []model.Folder
err error
parentResult *model.Folder
getErr error
getCallCount int
// hasOtherAudio is returned by HasAudioOutsideFolders (the album-root check).
// False means the parent qualifies as an album root.
hasOtherAudio bool
otherAudioErr error
}
func (f *fakeFolderRepo) GetAll(...model.QueryOptions) ([]model.Folder, error) {
return f.result, f.err
}
func (f *fakeFolderRepo) HasAudioOutsideFolders(model.Folder, []string) (bool, error) {
return f.hasOtherAudio, f.otherAudioErr
}
func (f *fakeFolderRepo) Get(string) (*model.Folder, error) {
f.getCallCount++
if f.getErr != nil {
return nil, f.getErr
}
if f.parentResult != nil {
return f.parentResult, nil
}
return nil, model.ErrNotFound
}
-9
View File
@@ -4,7 +4,6 @@ import (
"bytes"
"image"
"image/color"
"image/draw"
"image/jpeg"
"image/png"
"testing"
@@ -46,11 +45,3 @@ func generateGradientImage(width, height int) *image.RGBA {
}
return img
}
// gradientNRGBA mirrors generateGradientImage in the type makeThumbnail hands the encoders.
func gradientNRGBA(size int) *image.NRGBA {
src := generateGradientImage(size, size)
dst := image.NewNRGBA(src.Bounds())
draw.Draw(dst, dst.Bounds(), src, src.Bounds().Min, draw.Src)
return dst
}
+55 -86
View File
@@ -1,5 +1,5 @@
// Package blurhash implements the blurhash encoding (https://github.com/woltapp/blurhash),
// parameterized to match Jellyfin so clients see equivalent hashes.
// Package blurhash implements the blurhash encoding algorithm (https://github.com/woltapp/blurhash),
// matching Jellyfin's parameters so clients tuned against Jellyfin see equivalent hashes.
package blurhash
import (
@@ -15,25 +15,30 @@ import (
const alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz#$%*+,-.:;=?@[]^_{|}~"
// maxInputSize: larger inputs are slower with no visible difference in the result.
// maxInputSize matches Jellyfin: larger inputs are slower with no visually discernible difference.
const maxInputSize = 128
// components picks x/y component counts targeting ~16 near-square tiles.
func components(width, height int) (int, int) {
// Components picks x/y component counts for an image, targeting ~16 near-square tiles (Jellyfin's formula).
func Components(width, height int) (int, int) {
if width <= 0 || height <= 0 {
return 0, 0
}
xf := math.Sqrt(16.0 * float64(width) / float64(height))
yf := xf * float64(height) / float64(width)
return min(int(xf)+1, 9), min(int(yf)+1, 9)
}
// Encode returns the blurhash of img, deriving the component counts from its aspect ratio.
func Encode(img image.Image) (string, error) {
if img.Bounds().Dx() == 0 || img.Bounds().Dy() == 0 {
// Encode returns the blurhash of img using xComp x yComp components.
func Encode(img image.Image, xComp, yComp int) (string, error) {
if xComp < 1 || xComp > 9 || yComp < 1 || yComp > 9 {
return "", errors.New("blurhash: components must be between 1 and 9")
}
rgba := toRGBA(downscale(img))
bounds := rgba.Bounds()
w, h := bounds.Dx(), bounds.Dy()
if w == 0 || h == 0 {
return "", errors.New("blurhash: empty image")
}
// Pre-downscale: its rounding can flip a component count, and the hash is a client cache key.
xComp, yComp := components(img.Bounds().Dx(), img.Bounds().Dy())
src := pixelsOf(downscale(img))
w, h := src.w, src.h
cosX := make([][]float64, xComp)
for i := range cosX {
@@ -52,40 +57,19 @@ func Encode(img image.Image) (string, error) {
lin := srgbToLinearTable()
factors := make([][3]float64, xComp*yComp)
linR := make([]float64, w)
linG := make([]float64, w)
linB := make([]float64, w)
rowR := make([]float64, xComp)
rowG := make([]float64, xComp)
rowB := make([]float64, xComp)
for y := range h {
row := src.pix[y*src.stride:]
for x := range w {
for y := 0; y < h; y++ {
row := rgba.Pix[y*rgba.Stride:]
for x := 0; x < w; x++ {
p := x * 4
r, g, b := row[p], row[p+1], row[p+2]
if src.straight {
r, g, b = premultiply(r, g, b, row[p+3])
}
linR[x], linG[x], linB[x] = lin[r], lin[g], lin[b]
}
// The basis is separable, so a row costs xComp dot products plus one fold over yComp,
// rather than xComp*yComp multiply-accumulates per pixel.
for i := range xComp {
var sr, sg, sb float64
for x, c := range cosX[i] {
sr += c * linR[x]
sg += c * linG[x]
sb += c * linB[x]
}
rowR[i], rowG[i], rowB[i] = sr, sg, sb
}
for j := range yComp {
cy := cosY[j][y]
for i := range xComp {
f := &factors[j*xComp+i]
f[0] += cy * rowR[i]
f[1] += cy * rowG[i]
f[2] += cy * rowB[i]
lr, lg, lb := lin[row[p]], lin[row[p+1]], lin[row[p+2]]
for j := 0; j < yComp; j++ {
for i := 0; i < xComp; i++ {
basis := cosX[i][x] * cosY[j][y]
f := &factors[j*xComp+i]
f[0] += basis * lr
f[1] += basis * lg
f[2] += basis * lb
}
}
}
}
@@ -101,56 +85,40 @@ func Encode(img image.Image) (string, error) {
}
var sb strings.Builder
sb.WriteString(encode83((xComp-1)+(yComp-1)*9, 1))
sb.WriteString(Encode83((xComp-1)+(yComp-1)*9, 1))
// Derived counts are at least 1x9, so there is always at least one AC factor.
ac := factors[1:]
actualMax := 0.0
for _, f := range ac {
actualMax = max(actualMax, math.Abs(f[0]), math.Abs(f[1]), math.Abs(f[2]))
maxVal := 1.0
if len(ac) > 0 {
actualMax := 0.0
for _, f := range ac {
actualMax = max(actualMax, math.Abs(f[0]), math.Abs(f[1]), math.Abs(f[2]))
}
quantMax := int(math.Max(0, math.Min(82, math.Floor(actualMax*166-0.5))))
maxVal = float64(quantMax+1) / 166
sb.WriteString(Encode83(quantMax, 1))
} else {
sb.WriteString(Encode83(0, 1))
}
quantMax := int(max(0, min(82, math.Floor(actualMax*166-0.5))))
maxVal := float64(quantMax+1) / 166
sb.WriteString(encode83(quantMax, 1))
dc := factors[0]
sb.WriteString(encode83(linearToSRGB(dc[0])<<16|linearToSRGB(dc[1])<<8|linearToSRGB(dc[2]), 4))
sb.WriteString(Encode83(linearToSRGB(dc[0])<<16|linearToSRGB(dc[1])<<8|linearToSRGB(dc[2]), 4))
for _, f := range ac {
sb.WriteString(encode83(quantAC(f[0], maxVal)*19*19+quantAC(f[1], maxVal)*19+quantAC(f[2], maxVal), 2))
sb.WriteString(Encode83(quantAC(f[0], maxVal)*19*19+quantAC(f[1], maxVal)*19+quantAC(f[2], maxVal), 2))
}
return sb.String(), nil
}
// pixels is direct Pix access for the pixel loop, avoiding a per-pixel allocation via image.At.
type pixels struct {
pix []uint8
stride int
w, h int
// straight marks non-premultiplied alpha, which the loop premultiplies to keep the hash
// identical to the one an equivalent *image.RGBA produces.
straight bool
}
// pixelsOf accepts the two types the artwork pipeline produces without copying, and converts
// anything else.
func pixelsOf(img image.Image) pixels {
b := img.Bounds()
switch src := img.(type) {
case *image.RGBA:
return pixels{pix: src.Pix, stride: src.Stride, w: b.Dx(), h: b.Dy()}
case *image.NRGBA:
return pixels{pix: src.Pix, stride: src.Stride, w: b.Dx(), h: b.Dy(), straight: true}
// toRGBA gives the pixel loop direct Pix access, avoiding a per-pixel allocation through the
// image.At interface (~16k allocs per encode).
func toRGBA(img image.Image) *image.RGBA {
if rgba, ok := img.(*image.RGBA); ok {
return rgba
}
b := img.Bounds()
dst := image.NewRGBA(image.Rect(0, 0, b.Dx(), b.Dy()))
draw.Draw(dst, dst.Bounds(), img, b.Min, draw.Src)
return pixels{pix: dst.Pix, stride: dst.Stride, w: b.Dx(), h: b.Dy()}
}
func premultiply(r, g, b, a uint8) (uint8, uint8, uint8) {
if a == 255 {
return r, g, b
}
return uint8(uint32(r) * uint32(a) / 255), uint8(uint32(g) * uint32(a) / 255), uint8(uint32(b) * uint32(a) / 255)
return dst
}
var srgbToLinearTable = sync.OnceValue(func() *[256]float64 {
@@ -174,7 +142,7 @@ func downscale(img image.Image) image.Image {
}
func quantAC(v, maxVal float64) int {
return int(max(0, min(18, math.Floor(signPow(v/maxVal, 0.5)*9+9.5))))
return int(math.Max(0, math.Min(18, math.Floor(signPow(v/maxVal, 0.5)*9+9.5))))
}
func signPow(v, exp float64) float64 {
@@ -190,15 +158,16 @@ func srgbToLinear(v int) float64 {
}
func linearToSRGB(v float64) int {
v = min(max(0, v), 1)
v = math.Min(math.Max(0, v), 1)
if v <= 0.0031308 {
return int(v*12.92*255 + 0.5)
}
return int((1.055*math.Pow(v, 1/2.4)-0.055)*255 + 0.5)
}
// encode83 encodes value as a fixed-width, big-endian base83 string of the given length.
func encode83(value, length int) string {
// Encode83 encodes value as a fixed-width, big-endian base83 string of the given length, using the
// blurhash spec's alphabet.
func Encode83(value, length int) string {
b := make([]byte, length)
for i := length - 1; i >= 0; i-- {
b[i] = alphabet[value%83]
@@ -0,0 +1,41 @@
package blurhash_test
import (
"fmt"
"image"
"image/color"
"testing"
"github.com/navidrome/navidrome/core/artwork/blurhash"
)
// benchImage builds a deterministic gradient so runs are comparable across revisions.
func benchImage(size int) image.Image {
img := image.NewNRGBA(image.Rect(0, 0, size, size))
for y := 0; y < size; y++ {
for x := 0; x < size; x++ {
img.SetNRGBA(x, y, color.NRGBA{
R: uint8(255 * x / size),
G: uint8(255 * y / size),
B: uint8((x + y) * 255 / (2 * size)),
A: 255,
})
}
}
return img
}
func BenchmarkEncode(b *testing.B) {
for _, size := range []int{100, 300, 600, 900, 1200, 1500} {
img := benchImage(size)
x, y := blurhash.Components(size, size)
b.Run(fmt.Sprintf("%dx%d", size, size), func(b *testing.B) {
b.ReportAllocs()
for range b.N {
if _, err := blurhash.Encode(img, x, y); err != nil {
b.Fatal(err)
}
}
})
}
}
+38 -62
View File
@@ -22,8 +22,8 @@ func decode83(s string) int {
func solidImage(w, h int, c color.NRGBA) image.Image {
img := image.NewNRGBA(image.Rect(0, 0, w, h))
for y := range h {
for x := range w {
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
img.SetNRGBA(x, y, c)
}
}
@@ -32,79 +32,54 @@ func solidImage(w, h int, c color.NRGBA) image.Image {
func gradientImage(w, h int) image.Image {
img := image.NewNRGBA(image.Rect(0, 0, w, h))
for y := range h {
for x := range w {
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
img.SetNRGBA(x, y, color.NRGBA{R: uint8(255 * x / w), G: uint8(255 * y / h), B: 128, A: 255})
}
}
return img
}
var _ = Describe("Encode input types", func() {
// The pipeline hands Encode an *image.NRGBA; reading it must stay equivalent to the
// premultiplied *image.RGBA it used to receive, or every hash silently shifts.
buildPair := func(alpha uint8) (*image.NRGBA, *image.RGBA) {
const size = 40
nrgba := image.NewNRGBA(image.Rect(0, 0, size, size))
rgba := image.NewRGBA(image.Rect(0, 0, size, size))
for y := range size {
for x := range size {
c := color.NRGBA{
R: uint8(255 * x / size), G: uint8(255 * y / size),
B: uint8((x + y) * 255 / (2 * size)), A: alpha,
}
nrgba.SetNRGBA(x, y, c)
rgba.Set(x, y, c) // image.RGBA.Set premultiplies
}
}
return nrgba, rgba
}
DescribeTable("gives an NRGBA the same hash as the premultiplied RGBA it replaces",
func(alpha uint8) {
nrgba, rgba := buildPair(alpha)
fromNRGBA, err := blurhash.Encode(nrgba)
Expect(err).ToNot(HaveOccurred())
fromRGBA, err := blurhash.Encode(rgba)
Expect(err).ToNot(HaveOccurred())
Expect(fromNRGBA).To(Equal(fromRGBA))
var _ = Describe("Components", func() {
DescribeTable("derives component counts from aspect ratio (Jellyfin formula)",
func(w, h, expectedX, expectedY int) {
x, y := blurhash.Components(w, h)
Expect(x).To(Equal(expectedX))
Expect(y).To(Equal(expectedY))
},
Entry("opaque", uint8(255)),
Entry("partly transparent", uint8(128)),
Entry("fully transparent, which premultiplication crushes to black", uint8(0)),
Entry("square album art", 600, 600, 5, 5),
Entry("small square", 1, 1, 5, 5),
Entry("landscape 16:9", 1920, 1080, 6, 4),
Entry("portrait 9:16", 1080, 1920, 4, 6),
Entry("extreme landscape capped at 9", 10000, 100, 9, 1),
Entry("zero width", 0, 600, 0, 0),
Entry("zero height", 600, 0, 0, 0),
)
})
var _ = Describe("Encode", func() {
// The size flag encodes (xComp-1) + (yComp-1)*9.
DescribeTable("derives component counts from aspect ratio (Jellyfin formula)",
func(w, h, expectedX, expectedY int) {
hash, err := blurhash.Encode(gradientImage(w, h))
Expect(err).ToNot(HaveOccurred())
Expect(decode83(hash[:1])).To(Equal((expectedX - 1) + (expectedY-1)*9))
},
Entry("square album art", 60, 60, 5, 5),
Entry("smallest square", 1, 1, 5, 5),
Entry("landscape 16:9", 192, 108, 6, 4),
Entry("portrait 9:16", 108, 192, 4, 6),
Entry("extreme landscape capped at 9", 1000, 10, 9, 1),
Entry("extreme portrait capped at 9", 10, 1000, 1, 9),
)
It("rejects an empty image", func() {
_, err := blurhash.Encode(image.NewNRGBA(image.Rect(0, 0, 0, 0)))
It("rejects out-of-range components", func() {
_, err := blurhash.Encode(solidImage(8, 8, color.NRGBA{A: 255}), 0, 5)
Expect(err).To(HaveOccurred())
_, err = blurhash.Encode(solidImage(8, 8, color.NRGBA{A: 255}), 5, 10)
Expect(err).To(HaveOccurred())
})
It("produces the spec-mandated length", func() {
// 1 (size flag) + 1 (max AC) + 4 (DC) + 2 per AC component; a square derives 5x5
h, err := blurhash.Encode(solidImage(8, 8, color.NRGBA{R: 10, G: 20, B: 30, A: 255}))
// 1 (size flag) + 1 (max AC) + 4 (DC) + 2 per AC component
h, err := blurhash.Encode(solidImage(8, 8, color.NRGBA{R: 10, G: 20, B: 30, A: 255}), 4, 3)
Expect(err).ToNot(HaveOccurred())
Expect(h).To(HaveLen(4 + 2 + 2*(5*5-1)))
Expect(h).To(HaveLen(4 + 2 + 2*(4*3-1)))
})
It("encodes the size flag as the first character", func() {
h, err := blurhash.Encode(solidImage(8, 8, color.NRGBA{A: 255}), 4, 3)
Expect(err).ToNot(HaveOccurred())
Expect(decode83(h[:1])).To(Equal((4 - 1) + (3-1)*9))
})
It("stores the average color in the DC component", func() {
h, err := blurhash.Encode(solidImage(16, 16, color.NRGBA{R: 200, G: 100, B: 50, A: 255}))
h, err := blurhash.Encode(solidImage(16, 16, color.NRGBA{R: 200, G: 100, B: 50, A: 255}), 4, 3)
Expect(err).ToNot(HaveOccurred())
dc := decode83(h[2:6])
Expect(dc >> 16).To(BeNumerically("~", 200, 1))
@@ -114,23 +89,24 @@ var _ = Describe("Encode", func() {
It("is deterministic", func() {
img := gradientImage(64, 64)
h1, err1 := blurhash.Encode(img)
h2, err2 := blurhash.Encode(img)
h1, err1 := blurhash.Encode(img, 5, 5)
h2, err2 := blurhash.Encode(img, 5, 5)
Expect(err1).ToNot(HaveOccurred())
Expect(err2).ToNot(HaveOccurred())
Expect(h1).To(Equal(h2))
})
It("produces different hashes for different images", func() {
h1, _ := blurhash.Encode(solidImage(16, 16, color.NRGBA{R: 255, A: 255}))
h2, _ := blurhash.Encode(gradientImage(16, 16))
h1, _ := blurhash.Encode(solidImage(16, 16, color.NRGBA{R: 255, A: 255}), 4, 4)
h2, _ := blurhash.Encode(gradientImage(16, 16), 4, 4)
Expect(h1).ToNot(Equal(h2))
})
It("downscales large images internally without changing the result materially", func() {
big, err := blurhash.Encode(solidImage(1000, 1000, color.NRGBA{R: 60, G: 120, B: 180, A: 255}))
// A 1000px solid image must encode fine and carry the same DC as its small version.
big, err := blurhash.Encode(solidImage(1000, 1000, color.NRGBA{R: 60, G: 120, B: 180, A: 255}), 5, 5)
Expect(err).ToNot(HaveOccurred())
small, err := blurhash.Encode(solidImage(16, 16, color.NRGBA{R: 60, G: 120, B: 180, A: 255}))
small, err := blurhash.Encode(solidImage(16, 16, color.NRGBA{R: 60, G: 120, B: 180, A: 255}), 5, 5)
Expect(err).ToNot(HaveOccurred())
Expect(big[2:6]).To(Equal(small[2:6]))
})
+49 -99
View File
@@ -8,14 +8,11 @@ import (
"path/filepath"
"strconv"
"strings"
"time"
"github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/core/ffmpeg"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils"
"github.com/navidrome/navidrome/utils/slice"
)
// discArtworkReader resolves disc-level artwork from a library's folder images
@@ -28,15 +25,6 @@ type discArtworkReader struct {
isMultiFolder bool
firstTrackRel string // library-relative; for fromTag / ffmpeg via lib.Abs
lib libraryView
// Newest ImagesUpdatedAt across the album's and this disc's folders: an image can be
// replaced without the album row changing, so this is what makes a cache key notice it.
imagesUpdatedAt time.Time
}
// cacheTime is the disc image's validity stamp: any of these moving means the selection may
// have changed.
func (d *discArtworkReader) cacheTime() time.Time {
return utils.TimeNewest(d.album.UpdatedAt, d.album.ImportedAt, d.imagesUpdatedAt)
}
func newDiscArtworkReader(ctx context.Context, ds model.DataStore, artID model.ArtworkID) (*discArtworkReader, error) {
@@ -50,16 +38,11 @@ func newDiscArtworkReader(ctx context.Context, ds model.DataStore, artID model.A
return nil, err
}
_, imgFiles, albumImagesAt, err := loadAlbumFoldersPaths(ctx, ds, *al)
_, imgFiles, _, err := loadAlbumFoldersPaths(ctx, ds, *al)
if err != nil {
return nil, err
}
var imagesUpdatedAt time.Time
if albumImagesAt != nil {
imagesUpdatedAt = *albumImagesAt
}
// Query mediafiles for this album + disc to find folder associations and first track
mfs, err := ds.MediaFile(ctx).GetAll(model.QueryOptions{
Sort: "track_number",
@@ -77,17 +60,21 @@ func newDiscArtworkReader(ctx context.Context, ds model.DataStore, artID model.A
// Build disc folder set and find first track. mf.Path is already library-relative.
var firstTrackRel string
allFolderIDs := make(map[string]bool)
for _, mf := range mfs {
if mf.Path != "" {
allFolderIDs[mf.FolderID] = true
if firstTrackRel == "" {
firstTrackRel = filepath.ToSlash(mf.Path)
break
}
}
folderIDs := slice.Unique(slice.Map(mfs, func(mf model.MediaFile) string { return mf.FolderID }))
// Resolve folder IDs to library-relative paths
discFoldersRel := make(map[string]bool)
if len(folderIDs) > 0 {
if len(allFolderIDs) > 0 {
folderIDs := make([]string, 0, len(allFolderIDs))
for id := range allFolderIDs {
folderIDs = append(folderIDs, id)
}
folders, err := ds.Folder(ctx).GetAll(model.QueryOptions{
Filters: squirrel.Eq{"folder.id": folderIDs},
})
@@ -97,87 +84,41 @@ func newDiscArtworkReader(ctx context.Context, ds model.DataStore, artID model.A
for _, f := range folders {
rel := strings.TrimPrefix(path.Join(f.Path, f.Name), "/")
discFoldersRel[rel] = true
imagesUpdatedAt = utils.TimeNewest(imagesUpdatedAt, f.ImagesUpdatedAt)
}
}
return &discArtworkReader{
album: *al,
discNumber: discNumber,
imgFiles: imgFiles,
discFoldersRel: discFoldersRel,
isMultiFolder: len(al.FolderIDs) > 1,
firstTrackRel: firstTrackRel,
lib: lib,
imagesUpdatedAt: imagesUpdatedAt,
album: *al,
discNumber: discNumber,
imgFiles: imgFiles,
discFoldersRel: discFoldersRel,
isMultiFolder: len(al.FolderIDs) > 1,
firstTrackRel: firstTrackRel,
lib: lib,
}, nil
}
// discCandidate is one DiscArtPriority entry. skip is set when the entry maps to no source at
// all, so a chain walk can say why instead of leaving a configured entry unaccounted for.
type discCandidate struct {
pattern string
resolve func() (resolution, bool)
skip string
}
func (d *discArtworkReader) discCandidates(ctx context.Context, ffmpeg ffmpeg.FFmpeg, priority string) []discCandidate {
folder := func(sf sourceFunc) func() (resolution, bool) {
return func() (resolution, bool) { return resolveFolderSource(d.lib, sf) }
}
var cc []discCandidate
func (d *discArtworkReader) fromDiscArtPriority(ctx context.Context, ffmpeg ffmpeg.FFmpeg, priority string) []sourceFunc {
var ff []sourceFunc
for pattern := range strings.SplitSeq(strings.ToLower(priority), ",") {
pattern = strings.TrimSpace(pattern)
if pattern == "" {
continue
}
c := discCandidate{pattern: pattern}
switch {
case pattern == "embedded":
c.resolve = func() (resolution, bool) {
return resolveEmbedded(ctx, d.lib, ffmpeg, d.firstTrackRel)
}
case pattern == externalCandidate:
c.skip = "external sources are not supported for disc artwork"
ff = append(ff,
fromTag(ctx, d.lib.FS, d.firstTrackRel),
fromFFmpegTag(ctx, ffmpeg, d.lib.Abs(d.firstTrackRel)),
)
case pattern == "external":
// Not supported for disc art, silently ignore
case pattern == "discsubtitle":
subtitle := strings.TrimSpace(d.album.Discs[d.discNumber])
if subtitle == "" {
c.skip = "disc has no subtitle"
} else {
c.resolve = folder(d.fromDiscSubtitle(ctx, subtitle))
if subtitle := strings.TrimSpace(d.album.Discs[d.discNumber]); subtitle != "" {
ff = append(ff, d.fromDiscSubtitle(ctx, subtitle))
}
case len(d.imgFiles) == 0:
c.skip = "no images in album folder"
default:
c.resolve = folder(d.fromExternalFile(ctx, pattern))
}
cc = append(cc, c)
}
return cc
}
// selectImage walks the DiscArtPriority entries and returns the first that yields an image.
// chain records the walk; the serving path passes an untraced one and pays nothing for it.
func (d *discArtworkReader) selectImage(ctx context.Context, ffmpeg ffmpeg.FFmpeg, priority string,
chain *chainState) (resolution, error) {
for _, c := range d.discCandidates(ctx, ffmpeg, priority) {
if err := ctx.Err(); err != nil {
return resolution{}, err
}
if c.skip != "" {
chain.record(c.pattern, OutcomeSkipped, c.skip)
continue
}
start := time.Now()
res, ok := c.resolve()
log.Trace(ctx, "Artwork: Tried a disc artwork candidate", "albumID", d.album.ID,
"disc", d.discNumber, "pattern", c.pattern, "hit", ok, "path", res.sourcePath,
"elapsed", time.Since(start))
if res, ok = chain.try(c.pattern, res, ok); ok {
return res, nil
case len(d.imgFiles) > 0:
ff = append(ff, d.fromExternalFile(ctx, pattern))
}
}
return chain.exhausted(), nil
return ff
}
// fromDiscSubtitle returns a sourceFunc that matches image files whose stem
@@ -185,13 +126,14 @@ func (d *discArtworkReader) selectImage(ctx context.Context, ffmpeg ffmpeg.FFmpe
func (d *discArtworkReader) fromDiscSubtitle(ctx context.Context, subtitle string) sourceFunc {
return func() (io.ReadCloser, string, error) {
for _, file := range d.imgFiles {
stem := utils.BaseName(file)
name := path.Base(file)
stem := strings.TrimSuffix(name, path.Ext(name))
if !strings.EqualFold(stem, subtitle) {
continue
}
f, err := d.lib.FS.Open(file)
if err != nil {
log.Warn(ctx, "Artwork: Could not open disc art file", "file", file, err)
log.Warn(ctx, "Could not open disc art file", "file", file, err)
continue
}
return f, file, nil
@@ -200,12 +142,19 @@ func (d *discArtworkReader) fromDiscSubtitle(ctx context.Context, subtitle strin
}
}
// filepath.Match's '\' escape is excluded on purpose: treating it as a metachar
// would misalign the literal-prefix extraction in extractDiscNumber.
// globMetaChars holds the substitution metacharacters understood by
// filepath.Match. The '\' escape character is intentionally excluded:
// disc art patterns come from user config and never include escaped
// metachars in practice, and treating '\' as a metachar would misalign
// the literal-prefix extraction in extractDiscNumber.
const globMetaChars = "*?["
// extractDiscNumber parses the disc number from a filename matched by a filepath.Match-style
// glob. Caller must lowercase both args and have already verified the match.
// extractDiscNumber parses the disc number from a filename matched by a
// filepath.Match-style glob pattern.
//
// Both pattern and filename must already be lowercased by the caller, which
// is also expected to have verified that filepath.Match(pattern, filename)
// is true before calling this function.
func extractDiscNumber(pattern, filename string) (int, bool) {
metaIdx := strings.IndexAny(pattern, globMetaChars)
if metaIdx < 0 {
@@ -231,8 +180,9 @@ func extractDiscNumber(pattern, filename string) (int, bool) {
return num, true
}
// fromExternalFile matches image files against a (lowercase) glob pattern. A numbered
// filename whose number equals the target disc wins over any unnumbered candidate.
// fromExternalFile returns a sourceFunc that matches image files against a glob
// pattern. A numbered filename whose number equals the target disc wins over
// any unnumbered candidate; callers must pass a lowercase pattern.
func (d *discArtworkReader) fromExternalFile(ctx context.Context, pattern string) sourceFunc {
isLiteral := !strings.ContainsAny(pattern, globMetaChars)
return func() (io.ReadCloser, string, error) {
@@ -241,7 +191,7 @@ func (d *discArtworkReader) fromExternalFile(ctx context.Context, pattern string
name := strings.ToLower(path.Base(file))
match, err := filepath.Match(pattern, name)
if err != nil {
log.Warn(ctx, "Artwork: Error matching disc art file to pattern", "pattern", pattern, "file", file)
log.Warn(ctx, "Error matching disc art file to pattern", "pattern", pattern, "file", file)
continue
}
if !match {
@@ -255,7 +205,7 @@ func (d *discArtworkReader) fromExternalFile(ctx context.Context, pattern string
}
f, err := d.lib.FS.Open(file)
if err != nil {
log.Warn(ctx, "Artwork: Could not open disc art file", "file", file, err)
log.Warn(ctx, "Could not open disc art file", "file", file, err)
continue
}
return f, file, nil
@@ -271,7 +221,7 @@ func (d *discArtworkReader) fromExternalFile(ctx context.Context, pattern string
for _, file := range fallbacks {
f, err := d.lib.FS.Open(file)
if err != nil {
log.Warn(ctx, "Artwork: Could not open disc art file", "file", file, err)
log.Warn(ctx, "Could not open disc art file", "file", file, err)
continue
}
return f, file, nil
+66 -113
View File
@@ -6,8 +6,6 @@ import (
"path/filepath"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
"github.com/navidrome/navidrome/utils/slice"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
@@ -181,19 +179,19 @@ var _ = Describe("Disc Artwork Reader", func() {
lib: libraryView{FS: osDirFS{os.DirFS(tmpDir)}, absRoot: tmpDir},
}
cc := reader.discCandidates(ctx, nil, "disc*.*, cover.*")
Expect(cc).To(HaveLen(2))
res, ok := cc[0].resolve()
Expect(ok).To(BeTrue())
Expect(res.sourcePath).To(Equal(reader.lib.Abs(f2)))
res.reader.Close()
ff := reader.fromDiscArtPriority(ctx, nil, "disc*.*, cover.*")
Expect(ff).To(HaveLen(2))
r, path, err := ff[0]()
Expect(err).ToNot(HaveOccurred())
Expect(path).To(Equal(f2))
r.Close()
cc = reader.discCandidates(ctx, nil, "cover.*, disc*.*")
Expect(cc).To(HaveLen(2))
res, ok = cc[0].resolve()
Expect(ok).To(BeTrue())
Expect(res.sourcePath).To(Equal(reader.lib.Abs(f1)))
res.reader.Close()
ff = reader.fromDiscArtPriority(ctx, nil, "cover.*, disc*.*")
Expect(ff).To(HaveLen(2))
r, path, err = ff[0]()
Expect(err).ToNot(HaveOccurred())
Expect(path).To(Equal(f1))
r.Close()
})
DescribeTable("numbered match wins over shared fallback within a pattern",
@@ -430,109 +428,64 @@ var _ = Describe("Disc Artwork Reader", func() {
})
Describe("discArtworkReader", func() {
var (
reader *discArtworkReader
tmpDir string
)
BeforeEach(func() {
tmpDir = GinkgoT().TempDir()
reader = &discArtworkReader{
discNumber: 2,
isMultiFolder: true,
discFoldersRel: map[string]bool{"music/album/cd2": true},
imgFiles: []string{
"music/album/cd1/disc.jpg",
"music/album/cd2/disc.jpg",
"music/album/cd2/disc2.jpg",
},
firstTrackRel: "music/album/cd2/track1.flac",
lib: libraryView{FS: osDirFS{os.DirFS(tmpDir)}, absRoot: tmpDir},
}
})
Describe("selectImage", func() {
It("abandons the walk when the context is cancelled", func() {
ctx, cancel := context.WithCancel(context.Background())
cancel()
res, err := reader.selectImage(ctx, nil, "disc*.*, cover.*", &chainState{})
Expect(err).To(MatchError(context.Canceled))
Expect(res.reader).To(BeNil())
})
// "the track has no embedded art" and "the track is there but unreadable" are the two
// answers a wrong-artwork report needs told apart; only the second is worth retrying.
It("reports a track it cannot parse as unreadable, not as a miss", func() {
trace := &ChainTrace{}
track := filepath.Join(tmpDir, filepath.FromSlash(reader.firstTrackRel))
Expect(os.MkdirAll(filepath.Dir(track), 0755)).To(Succeed())
Expect(os.WriteFile(track, []byte("not audio"), 0600)).To(Succeed())
res, err := reader.selectImage(context.Background(), tests.NewMockFFmpeg(""), "embedded",
&chainState{trace: trace})
Expect(err).ToNot(HaveOccurred())
Expect(res.localError).To(BeTrue())
Expect(trace.Steps()).To(Equal([]TraceStep{{Candidate: "embedded", Outcome: OutcomeUnreadable}}))
})
It("reports a disc with no tracks to read as a miss", func() {
trace := &ChainTrace{}
reader.firstTrackRel = ""
res, err := reader.selectImage(context.Background(), tests.NewMockFFmpeg(""), "embedded",
&chainState{trace: trace})
Expect(err).ToNot(HaveOccurred())
Expect(res.localError).To(BeFalse(), "there was nothing to read, so nothing failed to read")
Expect(trace.Steps()).To(Equal([]TraceStep{{Candidate: "embedded", Outcome: OutcomeMiss}}))
})
})
Describe("discCandidates", func() {
It("returns a resolvable candidate for glob patterns", func() {
cc := reader.discCandidates(context.Background(), nil, "disc*.*")
Expect(cc).To(HaveLen(1))
Expect(cc[0].resolve).ToNot(BeNil())
})
It("returns one candidate per entry, in order", func() {
cc := reader.discCandidates(context.Background(), nil, "disc*.*, cd*.*, embedded")
Expect(slice.Map(cc, func(c discCandidate) string { return c.pattern })).
To(Equal([]string{"disc*.*", "cd*.*", "embedded"}))
})
It("skips an empty entry rather than building a glob that matches nothing", func() {
cc := reader.discCandidates(context.Background(), nil, "disc*.*,")
Expect(cc).To(HaveLen(1))
})
// The skip reasons below are what `artwork explain` prints, so an entry that maps to no
// source must say why instead of vanishing from the walk.
DescribeTable("keeps an entry that maps to no source, with its reason",
func(setup func(), priority, reason string) {
setup()
cc := reader.discCandidates(context.Background(), nil, priority)
Expect(cc).To(HaveLen(1))
Expect(cc[0].resolve).To(BeNil())
Expect(cc[0].skip).To(Equal(reason))
},
Entry("external is unsupported", func() {}, "external",
"external sources are not supported for disc artwork"),
Entry("no images in the album folder", func() { reader.imgFiles = nil }, "disc*.*",
"no images in album folder"),
Entry("the disc has no subtitle",
func() { reader.album = model.Album{Discs: model.Discs{2: ""}} }, "discsubtitle",
"disc has no subtitle"),
Describe("fromDiscArtPriority", func() {
var (
reader *discArtworkReader
tmpDir string
)
BeforeEach(func() {
tmpDir = GinkgoT().TempDir()
reader = &discArtworkReader{
discNumber: 2,
isMultiFolder: true,
discFoldersRel: map[string]bool{"music/album/cd2": true},
imgFiles: []string{
"music/album/cd1/disc.jpg",
"music/album/cd2/disc.jpg",
"music/album/cd2/disc2.jpg",
},
firstTrackRel: "music/album/cd2/track1.flac",
lib: libraryView{FS: osDirFS{os.DirFS(tmpDir)}, absRoot: tmpDir},
}
})
It("returns source funcs for glob patterns", func() {
ff := reader.fromDiscArtPriority(context.Background(), nil, "disc*.*")
Expect(ff).To(HaveLen(1))
})
It("returns source funcs for embedded pattern", func() {
ff := reader.fromDiscArtPriority(context.Background(), nil, "embedded")
Expect(ff).To(HaveLen(2)) // fromTag + fromFFmpegTag
})
It("handles multiple comma-separated patterns", func() {
ff := reader.fromDiscArtPriority(context.Background(), nil, "disc*.*, cd*.*, embedded")
Expect(ff).To(HaveLen(4)) // disc*.* + cd*.* + fromTag + fromFFmpegTag
})
It("ignores 'external' pattern silently", func() {
ff := reader.fromDiscArtPriority(context.Background(), nil, "external")
Expect(ff).To(HaveLen(0))
})
It("returns no source funcs when imgFiles is empty and pattern is not embedded", func() {
reader.imgFiles = nil
ff := reader.fromDiscArtPriority(context.Background(), nil, "disc*.*")
Expect(ff).To(HaveLen(0))
})
It("returns source func for discsubtitle pattern", func() {
reader.album = model.Album{Discs: model.Discs{2: "Bonus Tracks"}}
cc := reader.discCandidates(context.Background(), nil, "discsubtitle")
Expect(cc).To(HaveLen(1))
Expect(cc[0].resolve).ToNot(BeNil())
ff := reader.fromDiscArtPriority(context.Background(), nil, "discsubtitle")
Expect(ff).To(HaveLen(1))
})
It("returns no source func for discsubtitle when disc has no subtitle", func() {
reader.album = model.Album{Discs: model.Discs{2: ""}}
ff := reader.fromDiscArtPriority(context.Background(), nil, "discsubtitle")
Expect(ff).To(HaveLen(0))
})
})
})
-127
View File
@@ -1,127 +0,0 @@
// Package dominant extracts an image's dominant colour, for use as a flat placeholder while the
// real artwork loads.
package dominant
import (
"fmt"
"image"
"math"
"sort"
)
const (
// 4 bits per channel: coarse enough that near-identical pixels land together, fine enough that
// distinct colours stay apart.
bits = 4
nBins = 1 << (3 * bits)
// Only the heaviest bins can win, and merging is O(n^2) over whatever survives.
maxBins = 64
// Oklab distance below which two bins are the same colour to the eye. Merging matters because a
// gradient splits across adjacent bins and would otherwise lose to a smaller flat region.
mergeDist = 0.10
)
type bin struct {
r, g, b float64
n float64
}
// Color returns the dominant colour as "#rrggbb", or "" when the image has no pixels. It reports
// presence, not salience: a mostly white sleeve returns white.
func Color(img image.Image) string {
var bins [nBins]bin
total := 0
eachPixel(img, func(r, g, b uint8) {
i := int(r>>(8-bits))<<(2*bits) | int(g>>(8-bits))<<bits | int(b>>(8-bits))
bins[i].r += float64(r)
bins[i].g += float64(g)
bins[i].b += float64(b)
bins[i].n++
total++
})
if total == 0 {
return ""
}
used := make([]bin, 0, 32)
for i := range bins {
if bins[i].n > 0 {
used = append(used, bins[i])
}
}
sort.Slice(used, func(i, j int) bool { return used[i].n > used[j].n })
if len(used) > maxBins {
used = used[:maxBins]
}
merged := make([]bin, 0, len(used))
for _, b := range used {
if i := nearest(merged, b); i >= 0 {
merged[i].r += b.r
merged[i].g += b.g
merged[i].b += b.b
merged[i].n += b.n
continue
}
merged = append(merged, b)
}
best := merged[0]
for _, m := range merged[1:] {
if m.n > best.n {
best = m
}
}
return fmt.Sprintf("#%02x%02x%02x",
uint8(best.r/best.n+0.5), uint8(best.g/best.n+0.5), uint8(best.b/best.n+0.5))
}
func nearest(merged []bin, b bin) int {
bl, ba, bb := oklab(b.r/b.n, b.g/b.n, b.b/b.n)
for i, m := range merged {
ml, ma, mb := oklab(m.r/m.n, m.g/m.n, m.b/m.n)
if math.Sqrt((bl-ml)*(bl-ml)+(ba-ma)*(ba-ma)+(bb-mb)*(bb-mb)) < mergeDist {
return i
}
}
return -1
}
// eachPixel walks the image, taking the NRGBA fast path the artwork pipeline always hits: both hash
// encoders already read the shared thumbnail in that form.
func eachPixel(img image.Image, fn func(r, g, b uint8)) {
if p, ok := img.(*image.NRGBA); ok {
for y := range p.Rect.Dy() {
row := p.Pix[y*p.Stride : y*p.Stride+p.Rect.Dx()*4]
for x := 0; x < len(row); x += 4 {
fn(row[x], row[x+1], row[x+2])
}
}
return
}
b := img.Bounds()
for y := b.Min.Y; y < b.Max.Y; y++ {
for x := b.Min.X; x < b.Max.X; x++ {
r, g, bl, _ := img.At(x, y).RGBA()
fn(uint8(r>>8), uint8(g>>8), uint8(bl>>8))
}
}
}
func srgbToLinear(v float64) float64 {
v /= 255
if v <= 0.04045 {
return v / 12.92
}
return math.Pow((v+0.055)/1.055, 2.4)
}
func oklab(r, g, b float64) (float64, float64, float64) {
lr, lg, lb := srgbToLinear(r), srgbToLinear(g), srgbToLinear(b)
l := math.Cbrt(0.4122214708*lr + 0.5363325363*lg + 0.0514459929*lb)
m := math.Cbrt(0.2119034982*lr + 0.6806995451*lg + 0.1073969566*lb)
s := math.Cbrt(0.0883024619*lr + 0.2817188376*lg + 0.6299787005*lb)
return 0.2104542553*l + 0.7936177850*m - 0.0040720468*s,
1.9779984951*l - 2.4285922050*m + 0.4505937099*s,
0.0259040371*l + 0.7827717662*m - 0.8086757660*s
}
@@ -1,17 +0,0 @@
package dominant_test
import (
"testing"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
func TestDominant(t *testing.T) {
tests.Init(t, false)
log.SetLevel(log.LevelFatal)
RegisterFailHandler(Fail)
RunSpecs(t, "Dominant Suite")
}
-91
View File
@@ -1,91 +0,0 @@
package dominant_test
import (
"image"
"image/color"
"github.com/navidrome/navidrome/core/artwork/dominant"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// fill paints rect with c onto img.
func fill(img *image.NRGBA, r image.Rectangle, c color.NRGBA) {
for y := r.Min.Y; y < r.Max.Y; y++ {
for x := r.Min.X; x < r.Max.X; x++ {
img.SetNRGBA(x, y, c)
}
}
}
func newImg(w, h int, c color.NRGBA) *image.NRGBA {
img := image.NewNRGBA(image.Rect(0, 0, w, h))
fill(img, img.Bounds(), c)
return img
}
var _ = Describe("Color", func() {
It("returns a solid image's own colour", func() {
Expect(dominant.Color(newImg(20, 20, color.NRGBA{0x33, 0x66, 0x99, 255}))).To(Equal("#336699"))
})
It("returns empty for an image with no pixels", func() {
Expect(dominant.Color(image.NewNRGBA(image.Rect(0, 0, 0, 0)))).To(Equal(""))
})
// Presence, not salience: this is a placeholder, so the large field wins even though the small
// patch is the more interesting colour.
It("picks the largest area, not the most vivid one", func() {
img := newImg(20, 20, color.NRGBA{0xfa, 0xfa, 0xfa, 255})
fill(img, image.Rect(0, 0, 4, 4), color.NRGBA{0xff, 0x00, 0x00, 255})
Expect(dominant.Color(img)).To(Equal("#fafafa"))
})
It("reports a near-black cover as near-black", func() {
img := newImg(20, 20, color.NRGBA{0x05, 0x05, 0x05, 255})
fill(img, image.Rect(0, 0, 5, 5), color.NRGBA{0x00, 0xff, 0x00, 255})
Expect(dominant.Color(img)).To(Equal("#050505"))
})
// A gradient splits across many quantisation bins. Without merging, each slice is smaller than
// the flat block and the block would win despite covering far less of the image.
It("merges a gradient's bins so it beats a smaller flat block", func() {
img := image.NewNRGBA(image.Rect(0, 0, 40, 40))
for y := range 40 {
for x := range 40 {
// 30 columns of blue gradient == 75% of the image
if x < 30 {
img.SetNRGBA(x, y, color.NRGBA{0x10, 0x20, uint8(0xa0 + x), 255})
} else {
img.SetNRGBA(x, y, color.NRGBA{0xff, 0xcc, 0x00, 255})
}
}
}
got := dominant.Color(img)
Expect(got).To(HavePrefix("#1020"), "expected the blue gradient, got "+got)
})
It("is deterministic", func() {
img := image.NewNRGBA(image.Rect(0, 0, 30, 30))
for y := range 30 {
for x := range 30 {
img.SetNRGBA(x, y, color.NRGBA{uint8(x * 7), uint8(y * 5), uint8(x + y), 255})
}
}
first := dominant.Color(img)
for range 5 {
Expect(dominant.Color(img)).To(Equal(first))
}
})
It("handles images that are not NRGBA", func() {
src := newImg(10, 10, color.NRGBA{0x20, 0x40, 0x60, 255})
rgba := image.NewRGBA(src.Bounds())
for y := range 10 {
for x := range 10 {
rgba.Set(x, y, src.At(x, y))
}
}
Expect(dominant.Color(rgba)).To(Equal("#204060"))
})
})
+36 -159
View File
@@ -2,12 +2,9 @@ package e2e
import (
"context"
"encoding/base64"
"errors"
"io"
"os"
"path/filepath"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
@@ -22,7 +19,9 @@ import (
. "github.com/onsi/gomega"
)
// Covers the enqueue → drain → serve chain; per-source resolution rules live in the unit suites.
// These specs wire the real Worker and Service over the same ImageStore and mock repositories,
// then drive the full enqueue → drain → serve loop. They assert the integration of the chain, not
// the per-source resolution rules (which the unit suites in package artwork already cover).
var _ = Describe("Acquisition → serve loop", func() {
var (
ctx context.Context
@@ -37,31 +36,24 @@ var _ = Describe("Acquisition → serve loop", func() {
folderRepo *fakeFolderRepo
libRepo *tests.MockLibraryRepo
store *artwork.ImageStore
svc artwork.Artwork
svc artwork.Service
worker *artwork.Worker
coverBytes []byte
)
itemFound := func(kind model.Kind, id string) func() bool {
// itemFound reports whether the worker has persisted a resolved (hash-bearing) state row.
itemFound := func(kind, id string) func() bool {
return func() bool {
ia, err := artRepo.GetItemArtwork(kind, id, model.ImageTypePrimary)
return err == nil && ia.Hash != ""
}
}
itemAbsent := func(kind model.Kind, id string) func() bool {
itemAbsent := func(kind, id string) func() bool {
return func() bool {
ia, err := artRepo.GetItemArtwork(kind, id, model.ImageTypePrimary)
return err == nil && ia.Hash == ""
}
}
// Enqueues the way the serving paths do, so the drain is driven by a plain queue row.
bump := func(kind, id string) {
GinkgoHelper()
Expect(ds.ArtworkQueue(ctx).EnqueuePreservingBackoff(model.ArtworkQueueItem{
ItemKind: kind, ItemID: id, ImageType: model.ImageTypePrimary,
Priority: model.ArtworkPriorityBump,
})).To(Succeed())
}
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
@@ -73,9 +65,9 @@ var _ = Describe("Acquisition → serve loop", func() {
conf.Server.CacheFolder = conf.NewDir(GinkgoT().TempDir())
conf.Server.DataFolder = conf.NewDir(GinkgoT().TempDir())
conf.Server.CoverArtPriority = "cover.jpg"
conf.Server.ArtistArtPriority = "artist.png" // keeps artist resolution offline
conf.Server.ArtistArtPriority = "artist.png" // upload wins first; kept offline as a safety net
conf.Server.EnableMediaFileCoverArt = true
conf.Server.DevArtworkWorkerConcurrency = 1
conf.Server.ArtworkWorkerConcurrency = 1
folderRepo = &fakeFolderRepo{}
libRepo = &tests.MockLibraryRepo{}
@@ -101,47 +93,31 @@ var _ = Describe("Acquisition → serve loop", func() {
}
ffm := tests.NewMockFFmpeg("")
store = artwork.NewImageStore(GinkgoT().TempDir())
// size=0 requests stream originals, so this reader is never called (serving_test covers resizing).
// size=0 requests stream originals and never touch the resize cache, so this reader is a
// compile-time stand-in only; the resize path is covered by the package's serving_test.
imgCache := cache.NewFileCache("ArtworkPipelineE2E", "100MB", "images", 0,
func(context.Context, cache.Item) (io.Reader, error) {
return nil, errors.New("resize not exercised in e2e")
})
Eventually(func() bool { return imgCache.Available(ctx) }, 10*time.Second).Should(BeTrue())
Eventually(func() bool { return imgCache.Available(ctx) }).Should(BeTrue())
svc = artwork.NewArtwork(ds, imgCache, store, ffm)
svc = artwork.NewService(ds, imgCache, store, ffm)
worker = artwork.NewWorker(ds, store, agents.GetAgents(ds, nil), ffm, events.NoopBroker(), imgCache)
})
// seedFolderAlbum wires an album backed by the real fixture folder cover, shared by the album
// and playlist-grid scenarios.
seedFolderAlbum := func(albumID string) {
folderRepo.result = []model.Folder{{Path: albumFolderPath, ImageFiles: []string{"cover.jpg"}}}
albumRepo.SetData(model.Albums{{ID: albumID, Name: "Album", FolderIDs: []string{"f1"}, LibraryID: 0}})
}
It("acquires and serves a cover whose format has no registered decoder (#5950)", func() {
libDir := GinkgoT().TempDir()
Expect(os.MkdirAll(filepath.Join(libDir, "an-album"), 0755)).To(Succeed())
Expect(os.WriteFile(filepath.Join(libDir, "an-album", "cover.jxl"), jxlFixture, 0600)).To(Succeed())
conf.Server.CoverArtPriority = "cover.*"
libRepo.SetData(model.Libraries{{ID: 0, Path: libDir}})
folderRepo.result = []model.Folder{{Path: "an-album", ImageFiles: []string{"cover.jxl"}}}
albumRepo.SetData(model.Albums{{ID: "al1", Name: "Album", FolderIDs: []string{"f1"}, LibraryID: 0}})
bump("al", "al1")
runWorkerUntil(ctx, worker, itemFound(model.KindAlbumArtwork, "al1"))
img, err := svc.Get(ctx, model.MustParseArtworkID("al-al1"), 0, false)
Expect(err).ToNot(HaveOccurred())
Expect(img.Placeholder).To(BeFalse())
Expect(readAll(img)).To(Equal(jxlFixture))
})
It("acquires album folder art and serves the exact bytes under its hash", func() {
seedFolderAlbum("al1")
bump("al", "al1")
runWorkerUntil(ctx, worker, itemFound(model.KindAlbumArtwork, "al1"))
worker.Bump("al", "al1")
runWorkerUntil(ctx, worker, itemFound("al", "al1"))
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork("al", "al1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("folder"))
@@ -155,10 +131,10 @@ var _ = Describe("Acquisition → serve loop", func() {
It("acquires an artist's uploaded image and serves it", func() {
name := writeUpload(consts.EntityArtist, "artist-e2e.png", artistPngFixture)
artistRepo.SetData(model.Artists{{ID: "ar1", Name: "Artist", UploadedImage: name}})
bump("ar", "ar1")
runWorkerUntil(ctx, worker, itemFound(model.KindArtistArtwork, "ar1"))
worker.Bump("ar", "ar1")
runWorkerUntil(ctx, worker, itemFound("ar", "ar1"))
ia, err := artRepo.GetItemArtwork(model.KindArtistArtwork, "ar1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork("ar", "ar1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("upload"))
@@ -172,16 +148,17 @@ var _ = Describe("Acquisition → serve loop", func() {
seedFolderAlbum("al1")
plRepo.SetData(model.Playlists{{ID: "pl1", Name: "Playlist"}})
plRepo.TracksRepo = &tests.MockPlaylistTrackRepo{AlbumIDs: []string{"al1"}}
bump("pl", "pl1")
runWorkerUntil(ctx, worker, itemFound(model.KindPlaylistArtwork, "pl1"))
worker.Bump("pl", "pl1")
runWorkerUntil(ctx, worker, itemFound("pl", "pl1"))
ia, err := artRepo.GetItemArtwork(model.KindPlaylistArtwork, "pl1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork("pl", "pl1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("generated"))
img, err := svc.Get(ctx, model.MustParseArtworkID("pl-pl1"), 0, false)
Expect(err).ToNot(HaveOccurred())
Expect(img.Hash).To(Equal(ia.Hash))
// The generated grid is a fresh PNG placed in the content-addressed store.
art, err := artRepo.GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred())
Expect(art.Mime).To(Equal("image/png"))
@@ -191,10 +168,10 @@ var _ = Describe("Acquisition → serve loop", func() {
It("acquires a radio station's uploaded image and serves it", func() {
name := writeUpload(consts.EntityRadio, "radio-e2e.jpg", coverFixture)
radioRepo.Data["ra1"] = &model.Radio{ID: "ra1", Name: "Station", UploadedImage: name}
bump("ra", "ra1")
runWorkerUntil(ctx, worker, itemFound(model.KindRadioArtwork, "ra1"))
worker.Bump("ra", "ra1")
runWorkerUntil(ctx, worker, itemFound("ra", "ra1"))
ia, err := artRepo.GetItemArtwork(model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork("ra", "ra1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("upload"))
@@ -209,6 +186,7 @@ var _ = Describe("Acquisition → serve loop", func() {
ID: "mf1", AlbumID: "al1", HasCoverArt: true, LibraryID: 0, Path: mp3Fixture,
}})
// First read: no state row yet → extract embedded art provisionally and enqueue the track.
provisional, err := svc.Get(ctx, model.MustParseArtworkID("mf-mf1"), 0, false)
Expect(err).ToNot(HaveOccurred())
Expect(provisional.Placeholder).To(BeFalse())
@@ -216,110 +194,27 @@ var _ = Describe("Acquisition → serve loop", func() {
provisionalBytes := readAll(provisional)
Expect(len(provisionalBytes)).To(BeNumerically(">", 0))
_, err = artRepo.GetItemArtwork(model.KindMediaFileArtwork, "mf1", model.ImageTypePrimary)
_, err = artRepo.GetItemArtwork("mf", "mf1", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound), "provisional serving must not write a state row")
// The provisional read enqueued a Bump; drain it.
runWorkerUntil(ctx, worker, itemFound(model.KindMediaFileArtwork, "mf1"))
ia, err := artRepo.GetItemArtwork(model.KindMediaFileArtwork, "mf1", model.ImageTypePrimary)
// The provisional read enqueued a Bump; drain it and confirm the persisted hash matches.
runWorkerUntil(ctx, worker, itemFound("mf", "mf1"))
ia, err := artRepo.GetItemArtwork("mf", "mf1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("embedded"))
Expect(ia.Hash).To(Equal(provisional.Hash))
// Second read: now served from the persisted state row / store, same bytes.
resolved, err := svc.Get(ctx, model.MustParseArtworkID("mf-mf1"), 0, false)
Expect(err).ToNot(HaveOccurred())
Expect(resolved.Hash).To(Equal(ia.Hash))
Expect(readAll(resolved)).To(Equal(provisionalBytes))
})
It("stores dimensions, mime and a real blurhash alongside the acquired bytes", func() {
seedFolderAlbum("al1")
bump("al", "al1")
runWorkerUntil(ctx, worker, itemFound(model.KindAlbumArtwork, "al1"))
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
art, err := artRepo.GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred())
Expect(art.Mime).To(Equal("image/jpeg"))
Expect(art.Width).To(BeNumerically(">", 0))
Expect(art.Height).To(BeNumerically(">", 0))
Expect(art.SizeBytes).To(BeNumerically("==", len(coverBytes)))
// Never a synthesized value: both hashes are encoded from the real pixels.
Expect(art.BlurHash).ToNot(BeEmpty())
Expect(art.ThumbHash).ToNot(BeEmpty())
raw, err := base64.StdEncoding.DecodeString(art.ThumbHash)
Expect(err).ToNot(HaveOccurred())
Expect(len(raw)).To(BeNumerically(">=", 5))
})
It("acquires GIF artwork, whose decoder only core/artwork's blank import registers", func() {
writeUploadedImage(consts.EntityRadio, "station.gif", gifFixture)
radioRepo.Data["ra1"] = &model.Radio{ID: "ra1", Name: "Station", UploadedImage: "station.gif"}
bump("ra", "ra1")
runWorkerUntil(ctx, worker, itemFound(model.KindRadioArtwork, "ra1"))
ia, err := artRepo.GetItemArtwork(model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
art, err := artRepo.GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred())
Expect(art.Mime).To(Equal("image/gif"))
Expect(art.Width).To(BeNumerically("==", 4))
})
It("deduplicates byte-identical art across entities onto one image row", func() {
folderRepo.result = []model.Folder{{Path: albumFolderPath, ImageFiles: []string{"cover.jpg"}}}
albumRepo.SetData(model.Albums{
{ID: "al1", Name: "Album", FolderIDs: []string{"f1"}, LibraryID: 0},
{ID: "al2", Name: "Same Cover", FolderIDs: []string{"f1"}, LibraryID: 0},
})
bump("al", "al1")
bump("al", "al2")
runWorkerUntil(ctx, worker, func() bool {
return itemFound(model.KindAlbumArtwork, "al1")() && itemFound(model.KindAlbumArtwork, "al2")()
})
ia1, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
ia2, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al2", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia1.Hash).To(Equal(ia2.Hash), "identical bytes must share one content hash")
Expect(readAll(mustGet(svc.Get(ctx, model.MustParseArtworkID("al-al2"), 0, false)))).To(Equal(coverBytes))
})
It("stops serving a file-backed image once its source file changes underneath", func() {
name := writeUpload(consts.EntityRadio, "radio-stale.jpg", coverFixture)
radioRepo.Data["ra1"] = &model.Radio{ID: "ra1", Name: "Station", UploadedImage: name}
bump("ra", "ra1")
runWorkerUntil(ctx, worker, itemFound(model.KindRadioArtwork, "ra1"))
ia, err := artRepo.GetItemArtwork(model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
staleHash := ia.Hash
path := model.UploadedImagePath(consts.EntityRadio, name)
Expect(os.WriteFile(path, readFixture(artistPngFixture), 0o600)).To(Succeed())
newer := time.Now().Add(2 * time.Second)
Expect(os.Chtimes(path, newer, newer)).To(Succeed())
// The mtime no longer matches the state row, so the stale bytes are not served.
_, err = svc.Get(ctx, model.MustParseArtworkID("ra-ra1"), 0, false)
Expect(err).To(MatchError(artwork.ErrUnavailable))
// That failed read enqueued a re-resolution.
runWorkerUntil(ctx, worker, func() bool {
cur, gerr := artRepo.GetItemArtwork(model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
return gerr == nil && cur.Hash != "" && cur.Hash != staleHash
})
img, err := svc.Get(ctx, model.MustParseArtworkID("ra-ra1"), 0, false)
Expect(err).ToNot(HaveOccurred())
Expect(readAll(img)).To(Equal(readFixture(artistPngFixture)))
})
It("records an absent state for an entity with no art and reports it unavailable", func() {
albumRepo.SetData(model.Albums{{ID: "alx", Name: "Artless", LibraryID: 0}})
bump("al", "alx")
runWorkerUntil(ctx, worker, itemAbsent(model.KindAlbumArtwork, "alx"))
worker.Bump("al", "alx")
runWorkerUntil(ctx, worker, itemAbsent("al", "alx"))
_, err := svc.Get(ctx, model.MustParseArtworkID("al-alx"), 0, false)
Expect(err).To(MatchError(artwork.ErrUnavailable))
@@ -329,21 +224,3 @@ var _ = Describe("Acquisition → serve loop", func() {
Expect(img.Placeholder).To(BeTrue())
})
})
func mustGet(img *artwork.Image, err error) *artwork.Image {
GinkgoHelper()
Expect(err).ToNot(HaveOccurred())
return img
}
// Raw bytes on purpose: encoding a GIF here would register image/gif in the test binary, masking
// jxlFixture is a JPEG XL bare codestream header: a real image format, with no stdlib decoder.
var jxlFixture = []byte{0xff, 0x0a, 0x00, 0x10, 0x00}
// the production import the spec above guards.
var gifFixture = []byte{
0x47, 0x49, 0x46, 0x38, 0x39, 0x61, 0x04, 0x00, 0x04, 0x00, 0x80, 0x00,
0x00, 0x2e, 0x86, 0xc1, 0xf4, 0xd0, 0x3f, 0x2c, 0x00, 0x00, 0x00, 0x00,
0x04, 0x00, 0x04, 0x00, 0x00, 0x02, 0x05, 0x44, 0x7c, 0x67, 0xb8, 0x05,
0x00, 0x3b,
}
-470
View File
@@ -1,470 +0,0 @@
package e2e
import (
"testing/fstest"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/model"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// The in-memory library FS cannot satisfy the os.Open(SourcePath) used to serve folder art, so
// folder scenarios assert on the worker's state row (Source + SourcePath) instead of the bytes.
var _ = Describe("Album artwork resolution", func() {
BeforeEach(func() {
setupResolutionHarness()
})
When("an album has a single folder with cover.jpg at the album root", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// └── cover.jpg ← matched by cover.*
It("returns the album-root cover", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/cover.jpg": smallPNG("album-root"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Artist/Album/cover.jpg")
})
})
// https://github.com/navidrome/navidrome/issues/5376
// cover.* basenames tie across album-root and per-disc folders;
// compareImageFiles must prefer shallower paths.
When("a multi-disc album has a cover.jpg at the album root and per-disc covers", func() {
// Artist/
// └── Album/
// ├── CD1/
// │ ├── 01 - Track.mp3
// │ └── cover.jpg ← should not win
// ├── CD2/
// │ ├── 01 - Track.mp3
// │ └── cover.jpg
// └── cover.jpg ← should win (album-root fallback)
It("prefers the album-root cover over per-disc covers", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/Album/CD1/01 - Track.mp3": trackFile(1, "Track CD1"),
"Artist/Album/CD2/01 - Track.mp3": trackFile(1, "Track CD2"),
"Artist/Album/cover.jpg": smallPNG("album-root"),
"Artist/Album/CD1/cover.jpg": smallPNG("disc1"),
"Artist/Album/CD2/cover.jpg": smallPNG("disc2"),
})
scan()
al := firstAlbum()
Expect(al.FolderIDs).To(HaveLen(2),
"sanity check: the two disc subfolders should form one multi-disc album")
expectAlbumFolderCover(al, "Artist/Album/cover.jpg")
})
})
// https://github.com/navidrome/navidrome/issues/5376
// folder.jpg basenames tie across album-root and per-disc folders;
// compareImageFiles must prefer shallower paths.
When("a multi-disc album has folder.jpg at the album root AND in each disc subfolder", func() {
// Artist/
// └── Album/
// ├── CD1/
// │ ├── 01 - Track.mp3
// │ └── folder.jpg ← should not win
// ├── CD2/
// │ ├── 01 - Track.mp3
// │ └── folder.jpg
// └── folder.jpg ← should win (album-root fallback)
It("prefers the album-root folder.jpg over per-disc folder.jpg", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/Album/CD1/01 - Track.mp3": trackFile(1, "Track CD1"),
"Artist/Album/CD2/01 - Track.mp3": trackFile(1, "Track CD2"),
"Artist/Album/folder.jpg": smallPNG("album-root"),
"Artist/Album/CD1/folder.jpg": smallPNG("disc1"),
"Artist/Album/CD2/folder.jpg": smallPNG("disc2"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Artist/Album/folder.jpg")
})
})
// https://github.com/navidrome/navidrome/issues/5376
// Single-subfolder albums must still consider the parent folder's images.
When("an album lives entirely under a single disc subfolder with cover.jpg at the parent", func() {
// Artist/
// └── Album/
// ├── disc1/
// │ └── 01 - Track.mp3
// └── cover.jpg ← should win (parent-folder fallback)
It("uses the parent-folder cover for single-disc-subfolder albums", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/Album/disc1/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/cover.jpg": smallPNG("album-root"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Artist/Album/cover.jpg")
})
})
// https://github.com/navidrome/navidrome/issues/5456
When("a top-level multi-disc album has cover.jpg at the album root and per-disc folder.jpg", func() {
// Album/ (top-level folder, Path=".")
// ├── CD1/
// │ ├── 01 - Track.mp3
// │ └── folder.jpg
// ├── CD2/
// │ ├── 01 - Track.mp3
// │ └── folder.jpg
// └── cover.jpg ← should win (album-root)
It("prefers the album-root cover.jpg", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Album/CD1/01 - Track.mp3": trackFile(1, "Track CD1"),
"Album/CD2/01 - Track.mp3": trackFile(1, "Track CD2"),
"Album/cover.jpg": smallPNG("album-root"),
"Album/CD1/folder.jpg": smallPNG("disc1"),
"Album/CD2/folder.jpg": smallPNG("disc2"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Album/cover.jpg")
})
})
When("CoverArtPriority puts embedded first and the album has both embedded and external art", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3 ← has embedded picture (wins via "embedded")
// └── cover.jpg
It("returns the embedded image", func() {
conf.Server.CoverArtPriority = "embedded, cover.*, folder.*, front.*, external"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track", map[string]any{"has_picture": "true"}),
"Artist/Album/cover.jpg": smallPNG("external"),
})
scan()
replaceWithRealMP3("Artist/Album/01 - Track.mp3")
ia := acquire(model.KindAlbumArtwork, firstAlbum().ID)
Expect(ia.Source).To(Equal("embedded"))
Expect(storedBytes(ia)).To(Equal(embeddedArtBytes))
})
})
When("CoverArtPriority lists external first but no external file is present", func() {
// Artist/
// └── Album/
// └── 01 - Track.mp3 ← has embedded picture (falls through to "embedded")
It("falls through to embedded artwork", func() {
conf.Server.CoverArtPriority = "external, embedded"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track", map[string]any{"has_picture": "true"}),
})
scan()
replaceWithRealMP3("Artist/Album/01 - Track.mp3")
ia := acquire(model.KindAlbumArtwork, firstAlbum().ID)
Expect(ia.Source).To(Equal("embedded"))
Expect(storedBytes(ia)).To(Equal(embeddedArtBytes))
})
})
When("the only cover file uses uppercase extension and a different case in its name", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// └── Cover.JPG ← matched case-insensitively by cover.*
It("matches case-insensitively against cover.*", func() {
conf.Server.CoverArtPriority = "cover.*, folder.*"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/Cover.JPG": smallPNG("case-insensitive"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Artist/Album/Cover.JPG")
})
})
When("two cover files have basenames that tie under the natural-sort tiebreaker", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// ├── cover.jpg ← wins (no numeric suffix)
// └── cover.1.jpg
It("prefers the file without a numeric suffix", func() {
conf.Server.CoverArtPriority = "cover.*"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/cover.jpg": smallPNG("primary"),
"Artist/Album/cover.1.jpg": smallPNG("secondary"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Artist/Album/cover.jpg")
})
})
When("the album has no cover and CoverArtPriority lists only file patterns", func() {
// Artist/
// └── Album/
// └── 01 - Track.mp3 (no image files — settles absent)
It("settles absent", func() {
conf.Server.CoverArtPriority = "cover.*, folder.*"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
})
scan()
expectAlbumAbsent(firstAlbum())
})
})
// Doc scenarios from:
// https://www.navidrome.org/docs/usage/library/artwork/#albums
// Default CoverArtPriority is "cover.*, folder.*, front.*, embedded, external".
When("only folder.jpg is present (cover.* and front.* missing)", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// └── folder.jpg ← matched by folder.*
It("falls through to folder.jpg", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/folder.jpg": smallPNG("folder"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Artist/Album/folder.jpg")
})
})
When("only front.jpg is present (cover.* and folder.* missing)", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// └── front.jpg ← matched by front.*
It("falls through to front.jpg", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/front.jpg": smallPNG("front"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Artist/Album/front.jpg")
})
})
When("cover.*, folder.*, and front.* all exist in the same folder", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// ├── cover.jpg ← wins (cover.* is first in priority)
// ├── folder.jpg
// └── front.jpg
It("prefers cover.* (first in CoverArtPriority)", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/cover.jpg": smallPNG("cover"),
"Artist/Album/folder.jpg": smallPNG("folder"),
"Artist/Album/front.jpg": smallPNG("front"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Artist/Album/cover.jpg")
})
})
When("only folder.* and front.* exist (priority order check)", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// ├── folder.jpg ← wins (folder.* comes before front.*)
// └── front.jpg
It("prefers folder.* over front.*", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/folder.jpg": smallPNG("folder"),
"Artist/Album/front.jpg": smallPNG("front"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Artist/Album/folder.jpg")
})
})
When("three cover files tie by basename and differ only by numeric suffix", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// ├── cover.jpg ← wins (no numeric suffix)
// ├── cover.1.jpg
// └── cover.2.jpg
It("selects the unsuffixed file first regardless of numeric-suffix order", func() {
conf.Server.CoverArtPriority = "cover.*"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/cover.2.jpg": smallPNG("second"),
"Artist/Album/cover.jpg": smallPNG("primary"),
"Artist/Album/cover.1.jpg": smallPNG("first"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Artist/Album/cover.jpg")
})
})
When("CoverArtPriority contains an unknown pattern before a matching one", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// └── cover.jpg ← wins (unknown "bogus.*" is skipped)
It("skips the unknown pattern and falls through to the matching one", func() {
conf.Server.CoverArtPriority = "bogus.*, cover.*"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/cover.jpg": smallPNG("cover"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Artist/Album/cover.jpg")
})
})
// Regression introduced in v0.62.0 (#5451 + #5457): the parent-folder
// fallback can pick up images from the ARTIST folder, serving the artist
// thumbnail as album art for any album without its own image files.
When("an album has no images and the artist folder has folder.jpg", func() {
// Artist/
// ├── folder.jpg ← artist thumbnail, must NOT become album art
// ├── Album A/
// │ └── 01 - Track.mp3 (no images)
// └── Album B/
// ├── 01 - Track.mp3
// └── cover.jpg
It("does not use the artist image as album art", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/folder.jpg": smallPNG("artist-thumbnail"),
"Artist/Album A/01 - Track.mp3": trackFile(1, "Track A", map[string]any{"album": "Album A", "albumartist": "Artist"}),
"Artist/Album B/01 - Track.mp3": trackFile(1, "Track B", map[string]any{"album": "Album B", "albumartist": "Artist"}),
"Artist/Album B/cover.jpg": smallPNG("album-b"),
})
scan()
// Album B first: the acquire in expectAlbumAbsent would settle Album B too.
expectAlbumFolderCover(albumByName("Album B"), "Artist/Album B/cover.jpg")
expectAlbumAbsent(albumByName("Album A"))
})
})
When("a single-disc album is spread across sibling folders under the artist folder", func() {
// Artist/
// ├── folder.jpg ← artist thumbnail, must NOT become album art
// ├── Album A/
// │ └── 01 - Track.mp3 (album: "Album A")
// ├── Album A bonus/
// │ └── 02 - Track.mp3 (album: "Album A" — same album, second folder)
// └── Album B/
// ├── 01 - Track.mp3
// └── cover.jpg
It("does not use the artist image as album art for the spread album", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/folder.jpg": smallPNG("artist-thumbnail"),
"Artist/Album A/01 - Track.mp3": trackFile(1, "Track A1", map[string]any{"album": "Album A", "albumartist": "Artist"}),
"Artist/Album A bonus/02 - Track.mp3": trackFile(2, "Track A2", map[string]any{"album": "Album A", "albumartist": "Artist"}),
"Artist/Album B/01 - Track.mp3": trackFile(1, "Track B", map[string]any{"album": "Album B", "albumartist": "Artist"}),
"Artist/Album B/cover.jpg": smallPNG("album-b"),
})
scan()
alA := albumByName("Album A")
Expect(alA.FolderIDs).To(HaveLen(2),
"sanity check: the two sibling folders should form one spread album")
expectAlbumAbsent(alA)
})
})
// albumRootParent refuses the library root as an album root (parent.ParentID == "").
When("a multi-disc album sits directly at the library root with a cover.jpg beside it", func() {
// (library root)
// ├── cover.jpg ← must NOT be adopted
// ├── CD1/
// │ └── 01 - Track.mp3
// └── CD2/
// └── 01 - Track.mp3
It("does not adopt the library-root image as album art", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"cover.jpg": smallPNG("library-root"),
"CD1/01 - Track.mp3": trackFile(1, "T1", map[string]any{"album": "Rootless", "disc": "1"}),
"CD2/01 - Track.mp3": trackFile(1, "T2", map[string]any{"album": "Rootless", "disc": "2"}),
})
scan()
expectAlbumAbsent(firstAlbum())
})
})
// The shallower artist-folder cover.jpg would win the basename tie, but albumRootParent skips
// the parent folder for a single-folder album that has images of its own.
When("a single-folder album has its own cover.jpg and the artist folder has one too", func() {
// Artist/
// ├── cover.jpg ← shallower, but must NOT win
// └── Album/
// ├── 01 - Track.mp3
// └── cover.jpg ← should win
It("prefers the album's own cover over the shallower artist-folder cover", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/cover.jpg": smallPNG("artist-image"),
"Artist/Album/01 - Track.mp3": trackFile(1, "Track", map[string]any{"albumartist": "Artist"}),
"Artist/Album/cover.jpg": smallPNG("album-own"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Artist/Album/cover.jpg")
})
})
When("a spread album has its own front.jpg but the artist folder has cover.jpg", func() {
// Artist/
// ├── cover.jpg ← artist image; matches cover.* (first pattern),
// │ must NOT shadow the album's own front.jpg
// ├── Album A/
// │ ├── 01 - Track.mp3 (album: "Album A")
// │ └── front.jpg ← should win
// ├── Album A bonus/
// │ └── 02 - Track.mp3 (album: "Album A")
// └── Album B/
// └── 01 - Track.mp3 (other-album audio: rejects the artist folder as a root)
It("prefers the album's own art over the artist image", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/cover.jpg": smallPNG("artist-image"),
"Artist/Album A/01 - Track.mp3": trackFile(1, "Track A1", map[string]any{"album": "Album A", "albumartist": "Artist"}),
"Artist/Album A/front.jpg": smallPNG("album-a-front"),
"Artist/Album A bonus/02 - Track.mp3": trackFile(2, "Track A2", map[string]any{"album": "Album A", "albumartist": "Artist"}),
"Artist/Album B/01 - Track.mp3": trackFile(1, "Track B", map[string]any{"album": "Album B", "albumartist": "Artist"}),
})
scan()
alA := albumByName("Album A")
Expect(alA.FolderIDs).To(HaveLen(2),
"sanity check: the two sibling folders should form one spread album")
expectAlbumFolderCover(alA, "Artist/Album A/front.jpg")
})
})
When("embedded is first in CoverArtPriority but the track has no embedded art", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3 (no embedded picture)
// └── cover.jpg ← wins (embedded skipped, falls through)
It("falls through to the next priority entry", func() {
conf.Server.CoverArtPriority = "embedded, cover.*"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/cover.jpg": smallPNG("cover"),
})
scan()
expectAlbumFolderCover(firstAlbum(), "Artist/Album/cover.jpg")
})
})
})
-269
View File
@@ -1,269 +0,0 @@
package e2e
import (
"os"
"path/filepath"
"testing/fstest"
"github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/core/artwork"
"github.com/navidrome/navidrome/model"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// Doc reference:
// https://www.navidrome.org/docs/usage/library/artwork/#artists
// Default ArtistArtPriority is "artist.*, album/artist.*, external".
// Library-folder images are file-backed (asserted on the worker state row); uploaded and
// image-folder images are real files on disk (asserted byte-for-byte).
var _ = Describe("Artist artwork resolution", func() {
BeforeEach(func() {
setupResolutionHarness()
})
When("the artist folder contains an artist.jpg", func() {
// Artist/
// ├── artist.jpg ← matched by artist.*
// └── Album/
// └── 01 - Track.mp3
It("returns the artist.* image from the artist folder", func() {
conf.Server.ArtistArtPriority = "artist.*, album/artist.*, external"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track", map[string]any{"albumartist": "Artist"}),
"Artist/artist.jpg": smallPNG("artist-folder"),
})
scan()
expectArtistFolder(soleArtist(), "Artist/artist.jpg")
})
})
When("artist.* only exists inside an album folder", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// └── artist.jpg ← matched by album/artist.*
It("falls through to album/artist.* and returns that image", func() {
conf.Server.ArtistArtPriority = "artist.*, album/artist.*, external"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track", map[string]any{"albumartist": "Artist"}),
"Artist/Album/artist.jpg": smallPNG("album-artist"),
})
scan()
expectArtistFolder(soleArtist(), "Artist/Album/artist.jpg")
})
})
When("both the artist folder and an album folder have an artist.* image", func() {
// Artist/
// ├── artist.jpg ← wins (artist.* before album/artist.*)
// └── Album/
// ├── 01 - Track.mp3
// └── artist.jpg
It("prefers the artist-folder image (artist.* comes before album/artist.*)", func() {
conf.Server.ArtistArtPriority = "artist.*, album/artist.*, external"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track", map[string]any{"albumartist": "Artist"}),
"Artist/artist.jpg": smallPNG("artist-folder"),
"Artist/Album/artist.jpg": smallPNG("album-artist"),
})
scan()
expectArtistFolder(soleArtist(), "Artist/artist.jpg")
})
})
When("ArtistArtPriority has no album/ fallback", func() {
// Artist/
// ├── artist.jpg ← must resolve via the artist folder itself
// └── Album/
// └── 01 - Track.mp3
It("still resolves the artist folder and returns artist.*", func() {
conf.Server.ArtistArtPriority = "artist.*"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track", map[string]any{"albumartist": "Artist"}),
"Artist/artist.jpg": smallPNG("artist-folder"),
})
scan()
expectArtistFolder(soleArtist(), "Artist/artist.jpg")
})
})
When("the artist's only album has its tracks in disc subfolders", func() {
// Artist/
// ├── artist.jpg ← wins (artist.* before album/artist.*)
// └── Album/
// ├── artist.jpg
// ├── CD1/01 - Track.mp3
// └── CD2/02 - Track.mp3
It("prefers the artist-folder image over the album-folder one", func() {
conf.Server.ArtistArtPriority = "artist.*, album/artist.*, external"
setLayout(fstest.MapFS{
"Artist/Album/CD1/01 - Track.mp3": trackFile(1, "Track 1", map[string]any{"albumartist": "Artist", "album": "Album"}),
"Artist/Album/CD2/02 - Track.mp3": trackFile(2, "Track 2", map[string]any{"albumartist": "Artist", "album": "Album"}),
"Artist/artist.jpg": smallPNG("artist-folder"),
"Artist/Album/artist.jpg": smallPNG("album-artist"),
})
scan()
expectArtistFolder(soleArtist(), "Artist/artist.jpg")
})
})
When("one album has disc subfolders and another sits at artist level", func() {
// Artist/
// ├── artist.jpg ← wins
// ├── Album1/
// │ ├── artist.jpg
// │ ├── CD1/01 - Track.mp3
// │ └── CD2/02 - Track.mp3
// └── Album2/03 - Track.mp3
It("prefers the artist-folder image over the album-folder one", func() {
conf.Server.ArtistArtPriority = "artist.*, album/artist.*, external"
setLayout(fstest.MapFS{
"Artist/Album1/CD1/01 - Track.mp3": trackFile(1, "Track 1", map[string]any{"albumartist": "Artist", "album": "Album1"}),
"Artist/Album1/CD2/02 - Track.mp3": trackFile(2, "Track 2", map[string]any{"albumartist": "Artist", "album": "Album1"}),
"Artist/Album2/03 - Track.mp3": trackFile(3, "Track 3", map[string]any{"albumartist": "Artist", "album": "Album2"}),
"Artist/artist.jpg": smallPNG("artist-folder"),
"Artist/Album1/artist.jpg": smallPNG("album-artist"),
})
scan()
expectArtistFolder(soleArtist(), "Artist/artist.jpg")
})
})
When("every album of the artist has its tracks in disc subfolders", func() {
// Artist/
// ├── artist.jpg ← wins
// ├── Album1/
// │ ├── artist.jpg
// │ ├── CD1/01 - Track.mp3
// │ └── CD2/02 - Track.mp3
// └── Album2/
// ├── CD1/03 - Track.mp3
// └── CD2/04 - Track.mp3
It("prefers the artist-folder image over the album-folder one", func() {
conf.Server.ArtistArtPriority = "artist.*, album/artist.*, external"
setLayout(fstest.MapFS{
"Artist/Album1/CD1/01 - Track.mp3": trackFile(1, "Track 1", map[string]any{"albumartist": "Artist", "album": "Album1"}),
"Artist/Album1/CD2/02 - Track.mp3": trackFile(2, "Track 2", map[string]any{"albumartist": "Artist", "album": "Album1"}),
"Artist/Album2/CD1/03 - Track.mp3": trackFile(3, "Track 3", map[string]any{"albumartist": "Artist", "album": "Album2"}),
"Artist/Album2/CD2/04 - Track.mp3": trackFile(4, "Track 4", map[string]any{"albumartist": "Artist", "album": "Album2"}),
"Artist/artist.jpg": smallPNG("artist-folder"),
"Artist/Album1/artist.jpg": smallPNG("album-artist"),
})
scan()
expectArtistFolder(soleArtist(), "Artist/artist.jpg")
})
})
When("an artist has an uploaded image and a matching artist.* file", func() {
// <DataFolder>/
// └── artwork/
// └── artist/
// └── <id>_upload.jpg ← wins (uploaded image beats the priority chain)
// Library:
// Artist/
// ├── artist.jpg (ignored — uploaded image comes first)
// └── Album/
// └── 01 - Track.mp3
It("prefers the uploaded image over any priority-chain match", func() {
conf.Server.ArtistArtPriority = "artist.*, album/artist.*, external"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track", map[string]any{"albumartist": "Artist"}),
"Artist/artist.jpg": smallPNG("artist-folder"),
})
scan()
ar := soleArtist()
uploaded := ar.ID + "_upload.jpg"
writeUploadedImage(consts.EntityArtist, uploaded, pngBytes("artist-uploaded"))
ar.UploadedImage = uploaded
Expect(rds.Artist(rctx).Put(&ar)).To(Succeed())
ia := acquire(model.KindArtistArtwork, ar.ID)
Expect(ia.Source).To(Equal("upload"))
Expect(serveBytes(model.NewArtworkID(model.KindArtistArtwork, ar.ID, nil))).To(Equal(pngBytes("artist-uploaded")))
})
})
When("ArtistArtPriority uses album/<arbitrary pattern> (not just album/artist.*)", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// └── artist.jpg ← matched by album/artist.*
It("resolves the pattern against the artist's album image files", func() {
conf.Server.ArtistArtPriority = "album/artist.*, external"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track", map[string]any{"albumartist": "Artist"}),
"Artist/Album/artist.jpg": smallPNG("album-artist"),
})
scan()
expectArtistFolder(soleArtist(), "Artist/Album/artist.jpg")
})
})
// resolveArtist only samples albums where this artist is the SOLE album artist, so a
// collaboration or compilation never donates its images as the artist's own.
When("the artist's only album is credited to two album artists", func() {
// Artist/
// └── Collab Album/ (album artists: "Artist" + a collaborator)
// ├── 01 - Track.mp3
// └── artist.jpg ← must NOT become the artist image
It("ignores the album's images and settles absent", func() {
conf.Server.ArtistArtPriority = "album/artist.*"
// " / " is a default artists split separator, so this single tag yields two album artists.
setLayout(fstest.MapFS{
"Artist/Collab Album/01 - Track.mp3": trackFile(1, "Track", map[string]any{"albumartist": "Artist / Collaborator"}),
"Artist/Collab Album/artist.jpg": smallPNG("collab-artist"),
})
scan()
Expect(firstAlbum().Participants[model.RoleAlbumArtist]).To(HaveLen(2),
"sanity check: the album must be credited to two album artists")
ar := soleArtist()
ia := acquire(model.KindArtistArtwork, ar.ID)
Expect(ia.Hash).To(BeEmpty())
Expect(serveErr(model.NewArtworkID(model.KindArtistArtwork, ar.ID, nil))).
To(MatchError(artwork.ErrUnavailable))
})
})
When("ArtistArtPriority starts with image-folder and ArtistImageFolder has a name-matching image", func() {
// <ArtistImageFolder>/
// └── Artist.jpg ← matched by artist name (image-folder source)
// Library:
// Artist/
// └── Album/
// └── 01 - Track.mp3 (no artist.* present in library)
It("returns the image from the configured artist image folder", func() {
imgFolder := GinkgoT().TempDir()
Expect(os.WriteFile(filepath.Join(imgFolder, "Artist.jpg"), pngBytes("image-folder"), 0o600)).To(Succeed())
conf.Server.ArtistImageFolder = imgFolder
conf.Server.ArtistArtPriority = "image-folder, artist.*, album/artist.*"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track", map[string]any{"albumartist": "Artist"}),
})
scan()
ar := soleArtist()
ia := acquire(model.KindArtistArtwork, ar.ID)
Expect(ia.Source).To(Equal("folder"))
Expect(serveBytes(model.NewArtworkID(model.KindArtistArtwork, ar.ID, nil))).To(Equal(pngBytes("image-folder")))
})
})
})
func soleArtist() model.Artist {
GinkgoHelper()
artists, err := rds.Artist(rctx).GetAll(model.QueryOptions{
Filters: squirrel.Eq{"artist.name": "Artist"},
})
Expect(err).ToNot(HaveOccurred())
if len(artists) == 0 {
Fail("sole artist not found")
return model.Artist{}
}
return artists[0]
}
-333
View File
@@ -1,333 +0,0 @@
package e2e
import (
"fmt"
"testing/fstest"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/core/artwork"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// Disc art is a serve-time read through the library FS (no worker state row), so per-disc images
// are asserted byte-for-byte, while album-root covers are asserted on the state row.
var _ = Describe("Disc artwork resolution", func() {
BeforeEach(func() {
setupResolutionHarness()
})
When("the album is single-disc with a disc1.jpg in the only folder", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// └── disc1.jpg ← matched by disc*.*
It("returns the disc1.jpg image (matched as disc*.*)", func() {
conf.Server.DiscArtPriority = "disc*.*, cd*.*, cover.*, folder.*, front.*, embedded"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/disc1.jpg": smallPNG("disc1-image"),
})
scan()
expectDiscImage(firstAlbum(), 1, "disc1-image")
})
})
When("the album has no per-disc image and no album cover", func() {
// Artist/
// └── Album/
// └── 01 - Track.mp3 (no disc or album art — nothing to serve)
It("reports the disc lookup as unavailable", func() {
conf.Server.DiscArtPriority = "disc*.*, cd*.*"
conf.Server.CoverArtPriority = "cover.*, folder.*"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
})
scan()
Expect(serveErr(discArtID(firstAlbum(), 1))).To(MatchError(artwork.ErrUnavailable))
})
})
When("the album has no per-disc image but has an album cover", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// └── cover.jpg ← album-level fallback (no disc art present)
It("falls back to the album cover", func() {
conf.Server.DiscArtPriority = "disc*.*, cd*.*"
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/cover.jpg": smallPNG("album-cover"),
})
scan()
expectDiscImage(firstAlbum(), 1, "album-cover")
})
})
When("multiple disc images exist in the same folder (disc1 vs disc10)", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3
// ├── disc1.jpg ← matches request for disc 1
// └── disc10.jpg
It("matches the requested disc number, not a higher-numbered one", func() {
conf.Server.DiscArtPriority = "disc*.*"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/disc1.jpg": smallPNG("disc-one"),
"Artist/Album/disc10.jpg": smallPNG("disc-ten"),
})
scan()
expectDiscImage(firstAlbum(), 1, "disc-one")
})
})
When("a multi-disc album has per-disc covers", func() {
// Artist/
// └── Album/
// ├── CD1/
// │ ├── 01 - Track.mp3
// │ └── disc1.jpg ← matches request for disc 1
// └── CD2/
// ├── 01 - Track.mp3
// └── disc2.jpg ← matches request for disc 2
It("returns the requested disc's image", func() {
conf.Server.DiscArtPriority = "disc*.*"
setLayout(fstest.MapFS{
"Artist/Album/CD1/01 - Track.mp3": trackFile(1, "T1", map[string]any{"disc": "1"}),
"Artist/Album/CD2/01 - Track.mp3": trackFile(1, "T2", map[string]any{"disc": "2"}),
"Artist/Album/CD1/disc1.jpg": smallPNG("disc-1"),
"Artist/Album/CD2/disc2.jpg": smallPNG("disc-2"),
})
scan()
expectDiscImage(firstAlbum(), 2, "disc-2")
})
})
// Doc scenarios from:
// https://www.navidrome.org/docs/usage/library/artwork/#disc-cover-art
// Default DiscArtPriority is "disc*.*, cd*.*, cover.*, folder.*, front.*, discsubtitle, embedded".
When("a disc subfolder has a cd2.png image", func() {
// Artist/
// └── Album/
// ├── CD1/
// │ ├── 01 - Track.mp3
// │ └── disc1.jpg
// └── CD2/
// ├── 01 - Track.mp3
// └── cd2.png ← matched by cd*.* for disc 2
It("matches via the cd*.* pattern", func() {
conf.Server.DiscArtPriority = defaultDiscPriority
setLayout(fstest.MapFS{
"Artist/Album/CD1/01 - Track.mp3": trackFile(1, "T1", map[string]any{"disc": "1"}),
"Artist/Album/CD2/01 - Track.mp3": trackFile(1, "T2", map[string]any{"disc": "2"}),
"Artist/Album/CD1/disc1.jpg": smallPNG("disc-1"),
"Artist/Album/CD2/cd2.png": smallPNG("cd-2"),
})
scan()
expectDiscImage(firstAlbum(), 2, "cd-2")
})
})
When("a disc subfolder has cover.jpg but no disc*.*/cd*.* image", func() {
// Artist/
// └── Album/
// ├── CD1/
// │ ├── 01 - Track.mp3
// │ └── cover.jpg ← matched by cover.* inside disc folder
// └── CD2/
// ├── 01 - Track.mp3
// └── cover.jpg
It("falls through to cover.* inside the disc folder", func() {
conf.Server.DiscArtPriority = defaultDiscPriority
setLayout(fstest.MapFS{
"Artist/Album/CD1/01 - Track.mp3": trackFile(1, "T1", map[string]any{"disc": "1"}),
"Artist/Album/CD2/01 - Track.mp3": trackFile(1, "T2", map[string]any{"disc": "2"}),
"Artist/Album/CD1/cover.jpg": smallPNG("disc1-cover"),
"Artist/Album/CD2/cover.jpg": smallPNG("disc2-cover"),
})
scan()
expectDiscImage(firstAlbum(), 1, "disc1-cover")
})
})
When("DiscArtPriority is the empty string", func() {
// Artist/
// └── Album/
// ├── CD1/
// │ ├── 01 - Track.mp3
// │ └── disc1.jpg (ignored — DiscArtPriority is empty)
// ├── CD2/
// │ ├── 01 - Track.mp3
// │ └── cd2.png (ignored — DiscArtPriority is empty)
// └── cover.jpg ← used for every disc (album-level fallback)
It("skips every disc-level source and returns the album cover", func() {
conf.Server.DiscArtPriority = ""
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/Album/CD1/01 - Track.mp3": trackFile(1, "T1", map[string]any{"disc": "1"}),
"Artist/Album/CD2/01 - Track.mp3": trackFile(1, "T2", map[string]any{"disc": "2"}),
"Artist/Album/CD1/disc1.jpg": smallPNG("disc-1"),
"Artist/Album/CD2/cd2.png": smallPNG("cd-2"),
"Artist/Album/cover.jpg": smallPNG("album-cover"),
})
scan()
al := firstAlbum()
for _, n := range []int{1, 2} {
expectDiscImage(al, n, "album-cover")
}
})
})
When("the documented multi-disc layout is used (disc1.jpg + cd2.png + album-root cover.jpg)", func() {
// Artist/
// └── Album/
// ├── disc1/
// │ ├── disc1.jpg ← matched by disc*.* for disc 1
// │ ├── 01 - Track.mp3
// │ └── 02 - Track.mp3
// ├── disc2/
// │ ├── cd2.png ← matched by cd*.* for disc 2
// │ ├── 01 - Track.mp3
// │ └── 02 - Track.mp3
// └── cover.jpg (album-level fallback, unused here)
It("matches the per-disc image for each disc", func() {
conf.Server.DiscArtPriority = defaultDiscPriority
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/Album/disc1/01 - Track.mp3": trackFile(1, "T1", map[string]any{"disc": "1"}),
"Artist/Album/disc1/02 - Track.mp3": trackFile(2, "T2", map[string]any{"disc": "1"}),
"Artist/Album/disc2/01 - Track.mp3": trackFile(1, "T3", map[string]any{"disc": "2"}),
"Artist/Album/disc2/02 - Track.mp3": trackFile(2, "T4", map[string]any{"disc": "2"}),
"Artist/Album/disc1/disc1.jpg": smallPNG("disc-1"),
"Artist/Album/disc2/cd2.png": smallPNG("cd-2"),
"Artist/Album/cover.jpg": smallPNG("album-root"),
})
scan()
al := firstAlbum()
expectDiscImage(al, 1, "disc-1")
expectDiscImage(al, 2, "cd-2")
})
})
When("discsubtitle keyword matches an image whose stem equals the disc's subtitle", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3 (discsubtitle="Bonus Tracks")
// └── Bonus Tracks.jpg ← matched by "discsubtitle" keyword
It("selects the subtitle-named image", func() {
conf.Server.DiscArtPriority = "discsubtitle"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "T1", map[string]any{"disc": "1", "discsubtitle": "Bonus Tracks"}),
"Artist/Album/Bonus Tracks.jpg": smallPNG("bonus-tracks"),
})
scan()
expectDiscImage(firstAlbum(), 1, "bonus-tracks")
})
})
// Reproduces https://github.com/navidrome/navidrome/issues/5456
// Deeply nested layout matching the reporter's actual structure.
When("a deeply nested multi-disc album has cover.jpg and per-disc folder.jpg", func() {
// Genre/Artist/Album/ ← album root with cover.jpg
// ├── cover.jpg ← album-level cover
// ├── Disc 01 (Subtitle)/
// │ ├── 01 - Track.mp3
// │ └── folder.jpg ← disc 1 art
// ├── Disc 02 (Subtitle)/
// │ ├── 01 - Track.mp3
// │ └── folder.jpg
// └── ... (12 discs)
It("uses album-root cover.jpg for album art and per-disc folder.jpg for each disc", func() {
conf.Server.DiscArtPriority = defaultDiscPriority
conf.Server.CoverArtPriority = defaultCoverPriority
discNames := []string{
"Disc 01 (Birth of the Dead - The Studio Sides)",
"Disc 02 (Birth of the Dead - The Live Sides)",
"Disc 03 (The Grateful Dead)",
"Disc 04 (Anthem of the Sun)",
"Disc 05 (Aoxomoxoa)",
"Disc 06 (Live; Dead)",
"Disc 07 (Workingman's Dead)",
"Disc 08 (American Beauty)",
"Disc 09 (Grateful Dead)",
"Disc 10 (Europe '72)",
"Disc 11 (Europe '72)",
"Disc 12 (History of the Grateful Dead, Volume One (Bear's Choice))",
}
layout := fstest.MapFS{
"Pop; Rock/Grateful Dead/(2001) The Golden Road/cover.jpg": smallPNG("album-root-cover"),
}
for i, name := range discNames {
discNum := i + 1
prefix := fmt.Sprintf("Pop; Rock/Grateful Dead/(2001) The Golden Road/%s/", name)
layout[prefix+"01 - Track.mp3"] = trackFile(1, fmt.Sprintf("T%d", discNum), map[string]any{"disc": fmt.Sprintf("%d", discNum)})
layout[prefix+"folder.jpg"] = smallPNG(fmt.Sprintf("disc-%02d-folder", discNum))
}
setLayout(layout)
scan()
al := firstAlbum()
expectAlbumFolderCover(al, "(2001) The Golden Road/cover.jpg")
for i := range discNames {
discNum := i + 1
expectDiscImage(al, discNum, fmt.Sprintf("disc-%02d-folder", discNum))
}
})
})
// https://github.com/navidrome/navidrome/issues/5456
// Top-level album variant — album folder at library root (Path=".").
When("a top-level multi-disc album has cover.jpg and per-disc folder.jpg", func() {
// Album/ (top-level, Path=".")
// ├── cover.jpg ← album-level cover
// ├── Disc 01/
// │ ├── 01 - Track.mp3
// │ └── folder.jpg ← disc 1 art
// ├── Disc 02/
// │ ├── 01 - Track.mp3
// │ └── folder.jpg
// └── Disc 03/
// ├── 01 - Track.mp3
// └── folder.jpg
It("uses album-root cover.jpg for album art and per-disc folder.jpg for each disc", func() {
conf.Server.DiscArtPriority = defaultDiscPriority
conf.Server.CoverArtPriority = defaultCoverPriority
layout := fstest.MapFS{
"Album/cover.jpg": smallPNG("album-root-cover"),
}
for i := 1; i <= 3; i++ {
prefix := fmt.Sprintf("Album/Disc %02d/", i)
layout[prefix+"01 - Track.mp3"] = trackFile(1, fmt.Sprintf("T%d", i), map[string]any{"disc": fmt.Sprintf("%d", i)})
layout[prefix+"folder.jpg"] = smallPNG(fmt.Sprintf("disc-%02d-folder", i))
}
setLayout(layout)
scan()
al := firstAlbum()
expectAlbumFolderCover(al, "Album/cover.jpg")
for i := 1; i <= 3; i++ {
expectDiscImage(al, i, fmt.Sprintf("disc-%02d-folder", i))
}
})
})
When("discsubtitle is set but no image filename matches the subtitle", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3 (discsubtitle="Bonus Tracks")
// └── cover.jpg ← wins (discsubtitle has no match, falls through)
It("falls through to the next priority entry", func() {
conf.Server.DiscArtPriority = "discsubtitle, cover.*"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "T1", map[string]any{"disc": "1", "discsubtitle": "Bonus Tracks"}),
"Artist/Album/cover.jpg": smallPNG("cover"),
})
scan()
expectDiscImage(firstAlbum(), 1, "cover")
})
})
})
+11 -2
View File
@@ -1,5 +1,6 @@
// Package e2e exercises the artwork pipeline end to end: the real Worker drains the queue and the
// real Service serves the result, over a real ImageStore and real library files.
// Package e2e exercises the artwork pipeline end to end: it enqueues real entities, drives the
// real Worker to drain the queue, and serves the result through the real Service, over a real
// ImageStore and real library files.
package e2e
import (
@@ -35,6 +36,7 @@ const (
albumFolderPath = "tests/fixtures/artist/an-album"
)
// readFixture returns the raw bytes of a project-relative fixture file.
func readFixture(rel string) []byte {
GinkgoHelper()
data, err := os.ReadFile(rel)
@@ -42,6 +44,7 @@ func readFixture(rel string) []byte {
return data
}
// readAll drains an artwork image to bytes and closes it.
func readAll(img *artwork.Image) []byte {
GinkgoHelper()
Expect(img).ToNot(BeNil())
@@ -51,6 +54,8 @@ func readAll(img *artwork.Image) []byte {
return data
}
// runWorkerUntil starts the real worker loop, waits for a condition, then cancels and joins it,
// mirroring how cmd drives Worker.Run in production.
func runWorkerUntil(ctx context.Context, worker *artwork.Worker, until func() bool) {
GinkgoHelper()
runCtx, cancel := context.WithCancel(ctx)
@@ -61,6 +66,8 @@ func runWorkerUntil(ctx context.Context, worker *artwork.Worker, until func() bo
Eventually(done, 2*time.Second).Should(Receive(BeNil()))
}
// fakeFolderRepo is the minimal FolderRepository the album/playlist resolution chains touch:
// GetAll yields the seeded folders and the album-root parent lookup finds nothing.
type fakeFolderRepo struct {
model.FolderRepository
result []model.Folder
@@ -74,6 +81,8 @@ func (f *fakeFolderRepo) HasAudioOutsideFolders(model.Folder, []string) (bool, e
func (f *fakeFolderRepo) Get(string) (*model.Folder, error) { return nil, model.ErrNotFound }
// writeUpload copies a fixture into the per-entity upload folder under the data dir and returns
// the bare filename UploadedImagePath expects.
func writeUpload(entityType, name, srcFixture string) string {
GinkgoHelper()
dst := model.UploadedImagePath(entityType, name)
-149
View File
@@ -1,149 +0,0 @@
package e2e
import (
"testing/fstest"
"github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/model"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// Doc reference:
// https://www.navidrome.org/docs/usage/library/artwork/#mediafiles
// Navidrome resolves mediafile artwork in this order:
// 1. Embedded image from the mediafile itself
// 2. For multi-disc albums, disc-level artwork
// 3. Album cover art
//
// Embedded art lands in the content-addressed store (asserted byte-for-byte); disc-level art is a
// serve-time read through the library FS.
var _ = Describe("MediaFile artwork resolution", func() {
BeforeEach(func() {
setupResolutionHarness()
})
When("a multi-disc album track has no embedded art", func() {
// Artist/
// └── Album/
// ├── CD1/
// │ ├── 01 - Track.mp3
// │ └── disc1.jpg
// ├── CD2/
// │ ├── 01 - Track.mp3 ← track requested
// │ └── disc2.jpg ← wins (disc-level before album-level)
// └── cover.jpg
It("falls back to the disc-level artwork (not the album cover)", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
conf.Server.DiscArtPriority = defaultDiscPriority
setLayout(fstest.MapFS{
"Artist/Album/CD1/01 - Track.mp3": trackFile(1, "T1", map[string]any{"disc": "1"}),
"Artist/Album/CD2/01 - Track.mp3": trackFile(1, "T2", map[string]any{"disc": "2"}),
"Artist/Album/CD1/disc1.jpg": smallPNG("disc-1"),
"Artist/Album/CD2/disc2.jpg": smallPNG("disc-2"),
"Artist/Album/cover.jpg": smallPNG("album-root"),
})
scan()
mf := mediafileOn("Artist/Album/CD2/01 - Track.mp3")
Expect(serveBytes(mf.CoverArtID())).To(Equal(pngBytes("disc-2")))
})
})
When("a single-disc album track has no embedded art", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3 ← track requested
// └── cover.jpg ← wins (album-level fallback, no disc subfolder)
It("falls back to the album cover", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
conf.Server.DiscArtPriority = defaultDiscPriority
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track"),
"Artist/Album/cover.jpg": smallPNG("album-cover"),
})
scan()
mf := mediafileOn("Artist/Album/01 - Track.mp3")
Expect(serveBytes(mf.CoverArtID())).To(Equal(pngBytes("album-cover")))
})
})
When("a multi-disc album track has no embedded art and the disc has no disc-level image", func() {
// Artist/
// └── Album/
// ├── CD1/
// │ └── 01 - Track.mp3
// ├── CD2/
// │ └── 01 - Track.mp3 ← track requested
// └── cover.jpg ← wins (no disc image → album-level fallback)
It("falls through from disc to album cover", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
conf.Server.DiscArtPriority = defaultDiscPriority
setLayout(fstest.MapFS{
"Artist/Album/CD1/01 - Track.mp3": trackFile(1, "T1", map[string]any{"disc": "1"}),
"Artist/Album/CD2/01 - Track.mp3": trackFile(1, "T2", map[string]any{"disc": "2"}),
"Artist/Album/cover.jpg": smallPNG("album-root"),
})
scan()
mf := mediafileOn("Artist/Album/CD2/01 - Track.mp3")
Expect(serveBytes(mf.CoverArtID())).To(Equal(pngBytes("album-root")))
})
})
When("a track has its own embedded art", func() {
// Artist/
// └── Album/
// └── 01 - Track.mp3 ← has embedded picture (wins over every fallback)
It("resolves the track's embedded image into the store", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track", map[string]any{"has_picture": "true"}),
})
scan()
replaceWithRealMP3("Artist/Album/01 - Track.mp3")
mf := mediafileOn("Artist/Album/01 - Track.mp3")
ia := acquire(model.KindMediaFileArtwork, mf.ID)
Expect(ia.Source).To(Equal("embedded"))
Expect(storedBytes(ia)).To(Equal(embeddedArtBytes))
})
})
When("EnableMediaFileCoverArt is turned off after the track was scanned", func() {
// Artist/
// └── Album/
// ├── 01 - Track.mp3 ← has embedded picture (must NOT be served)
// └── cover.jpg ← wins (per-track art disabled at serve time)
It("serves the album cover instead of the track's embedded art", func() {
conf.Server.CoverArtPriority = defaultCoverPriority
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track", map[string]any{"has_picture": "true"}),
"Artist/Album/cover.jpg": smallPNG("album-cover"),
})
scan()
replaceWithRealMP3("Artist/Album/01 - Track.mp3")
// The setting is not part of the artwork fingerprint, so it must be honored at serve time.
conf.Server.EnableMediaFileCoverArt = false
mf := mediafileOn("Artist/Album/01 - Track.mp3")
trackArtID := model.NewArtworkID(model.KindMediaFileArtwork, mf.ID, nil)
Expect(serveBytes(trackArtID)).To(Equal(pngBytes("album-cover")))
})
})
})
func mediafileOn(relPath string) model.MediaFile {
GinkgoHelper()
mfs, err := rds.MediaFile(rctx).GetAll(model.QueryOptions{
Filters: squirrel.Like{"media_file.path": relPath},
})
Expect(err).ToNot(HaveOccurred())
if len(mfs) == 0 {
Fail("mediafile not found: " + relPath)
return model.MediaFile{}
}
return mfs[0]
}
-213
View File
@@ -1,213 +0,0 @@
package e2e
import (
"image/color"
"os"
"path/filepath"
"testing/fstest"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/core/artwork"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/slice"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// Playlist artwork resolves in this priority order:
// 1. Uploaded image (<DataFolder>/artwork/playlist/<file>)
// 2. Sidecar image next to the .m3u file (same basename, any image ext)
// 3. ExternalImageURL (http/https requires EnableM3UExternalAlbumArt; local path always allowed)
// 4. Generated 2x2 tiled cover from the playlist's albums
// 5. Absent
//
// The library is an in-memory FS, but uploaded/sidecar/local-external images are real files on
// disk — the resolver reads them via os.Open, so those tests place them in a real tempdir.
var _ = Describe("Playlist artwork resolution", func() {
BeforeEach(func() {
setupResolutionHarness()
})
When("a playlist has an uploaded image", func() {
// <DataFolder>/
// └── artwork/
// └── playlist/
// └── pl-1_upload.jpg ← matched by UploadedImagePath() (highest priority)
It("returns the uploaded image bytes", func() {
writeUploadedImage(consts.EntityPlaylist, "pl-1_upload.jpg", pngBytes("playlist-upload"))
pl := putPlaylist(model.Playlist{ID: "pl-1", Name: "Test", UploadedImage: "pl-1_upload.jpg"})
ia := acquire(model.KindPlaylistArtwork, pl.ID)
Expect(ia.Source).To(Equal("upload"))
Expect(serveBytes(pl.CoverArtID())).To(Equal(pngBytes("playlist-upload")))
})
})
When("a playlist has no uploaded image but a sidecar image beside its .m3u file", func() {
// <tempdir>/
// ├── MyList.m3u
// └── MyList.jpg ← matched by sidecar (same basename, case-insensitive)
It("returns the sidecar image", func() {
dir := GinkgoT().TempDir()
m3uPath := filepath.Join(dir, "MyList.m3u")
Expect(os.WriteFile(m3uPath, []byte("#EXTM3U\n"), 0o600)).To(Succeed())
Expect(os.WriteFile(filepath.Join(dir, "MyList.jpg"), pngBytes("sidecar"), 0o600)).To(Succeed())
pl := putPlaylist(model.Playlist{ID: "pl-2", Name: "MyList", Path: m3uPath})
ia := acquire(model.KindPlaylistArtwork, pl.ID)
Expect(ia.Source).To(Equal("folder"))
Expect(serveBytes(pl.CoverArtID())).To(Equal(pngBytes("sidecar")))
})
})
When("a playlist's sidecar uses a different extension case", func() {
// <tempdir>/
// ├── MyList.m3u
// └── MyList.PNG ← matched case-insensitively
It("matches case-insensitively", func() {
dir := GinkgoT().TempDir()
m3uPath := filepath.Join(dir, "MyList.m3u")
Expect(os.WriteFile(m3uPath, []byte("#EXTM3U\n"), 0o600)).To(Succeed())
Expect(os.WriteFile(filepath.Join(dir, "MyList.PNG"), pngBytes("sidecar-png"), 0o600)).To(Succeed())
pl := putPlaylist(model.Playlist{ID: "pl-3", Name: "MyList", Path: m3uPath})
Expect(serveBytes(pl.CoverArtID())).To(Equal(pngBytes("sidecar-png")))
})
})
When("a playlist has an ExternalImageURL pointing to a local file", func() {
// <tempdir>/
// └── cover.jpg ← absolute path stored in ExternalImageURL
It("returns the local file regardless of EnableM3UExternalAlbumArt", func() {
conf.Server.EnableM3UExternalAlbumArt = false // local paths bypass the toggle
dir := GinkgoT().TempDir()
imgPath := filepath.Join(dir, "cover.jpg")
Expect(os.WriteFile(imgPath, pngBytes("external-local"), 0o600)).To(Succeed())
pl := putPlaylist(model.Playlist{ID: "pl-4", Name: "WithExt", ExternalImageURL: imgPath})
Expect(serveBytes(pl.CoverArtID())).To(Equal(pngBytes("external-local")))
})
})
When("a playlist has an http(s) ExternalImageURL and EnableM3UExternalAlbumArt is false", func() {
// (no local files — the http source is gated off, so resolution settles absent)
It("skips the URL and settles absent", func() {
conf.Server.EnableM3UExternalAlbumArt = false
pl := putPlaylist(model.Playlist{ID: "pl-5", Name: "HttpGated", ExternalImageURL: "https://example.com/cover.jpg"})
ia := acquire(model.KindPlaylistArtwork, pl.ID)
Expect(ia.Hash).To(BeEmpty())
Expect(serveErr(pl.CoverArtID())).To(MatchError(artwork.ErrUnavailable))
img, err := rsvc.GetOrPlaceholder(rctx, pl.CoverArtID().String(), 0, false)
Expect(err).ToNot(HaveOccurred())
defer img.Close()
Expect(img.Placeholder).To(BeTrue())
})
})
When("a playlist has no images and no tracks", func() {
// (no uploaded/sidecar/external image and no album art to sample)
It("settles absent", func() {
pl := putPlaylist(model.Playlist{ID: "pl-6", Name: "Empty"})
ia := acquire(model.KindPlaylistArtwork, pl.ID)
Expect(ia.Hash).To(BeEmpty())
Expect(serveErr(pl.CoverArtID())).To(MatchError(artwork.ErrUnavailable))
})
})
When("a playlist has no uploaded/sidecar/external image but has tracks with album covers", func() {
// Library:
// Artist/
// ├── AlbumA/
// │ ├── 01 - Track.mp3
// │ └── cover.png ← tile 1 source
// └── AlbumB/
// ├── 01 - Track.mp3
// └── cover.png ← tile 2 source
// Playlist "pl-7" references tracks from both albums, so the worker generates a tiled
// cover from 2 distinct album art tiles (mirrored to fill the 2x2 grid).
It("generates a tiled cover from album art", func() {
conf.Server.CoverArtPriority = "cover.*"
setLayout(fstest.MapFS{
"Artist/AlbumA/01 - Track.mp3": trackFile(1, "TA", map[string]any{"album": "AlbumA"}),
"Artist/AlbumA/cover.png": smallPNG("albumA"),
"Artist/AlbumB/01 - Track.mp3": trackFile(1, "TB", map[string]any{"album": "AlbumB"}),
"Artist/AlbumB/cover.png": smallPNG("albumB"),
})
scan()
mfs, err := rds.MediaFile(rctx).GetAll(model.QueryOptions{})
Expect(err).ToNot(HaveOccurred())
Expect(mfs).To(HaveLen(2))
pl := model.Playlist{ID: "pl-7", Name: "Mix", OwnerID: "admin-1"}
pl.AddMediaFilesByID([]string{mfs[0].ID, mfs[1].ID})
Expect(rds.Playlist(rctx).Put(&pl)).To(Succeed())
ia := acquire(model.KindPlaylistArtwork, pl.ID)
Expect(ia.Source).To(Equal("generated"))
data := storedBytes(ia)
// The tiled cover is a PNG-encoded image; exact bytes vary (random album order).
Expect(data[:8]).To(Equal([]byte{0x89, 'P', 'N', 'G', 0x0d, 0x0a, 0x1a, 0x0a}))
// Two tiles are mirrored into the 2x2 grid as [A B B A], so opposite corners match.
q := gridQuadrants(data)
Expect(q[0]).To(Equal(q[3]))
Expect(q[1]).To(Equal(q[2]))
Expect(q[0]).ToNot(Equal(q[1]))
})
})
When("a playlist has tracks from four albums, each with its own cover", func() {
// Library:
// Artist/
// ├── AlbumA/{01 - Track.mp3, cover.png} ← tile 1
// ├── AlbumB/{01 - Track.mp3, cover.png} ← tile 2
// ├── AlbumC/{01 - Track.mp3, cover.png} ← tile 3
// └── AlbumD/{01 - Track.mp3, cover.png} ← tile 4
It("fills all four grid quadrants with distinct album art", func() {
conf.Server.CoverArtPriority = "cover.*"
layout := fstest.MapFS{}
for _, name := range []string{"AlbumA", "AlbumB", "AlbumC", "AlbumD"} {
layout["Artist/"+name+"/01 - Track.mp3"] = trackFile(1, "T"+name, map[string]any{"album": name})
layout["Artist/"+name+"/cover.png"] = smallPNG(name)
}
setLayout(layout)
scan()
mfs, err := rds.MediaFile(rctx).GetAll(model.QueryOptions{})
Expect(err).ToNot(HaveOccurred())
Expect(mfs).To(HaveLen(4))
ids := slice.Map(mfs, func(mf model.MediaFile) string { return mf.ID })
pl := model.Playlist{ID: "pl-8", Name: "Four", OwnerID: "admin-1"}
pl.AddMediaFilesByID(ids)
Expect(rds.Playlist(rctx).Put(&pl)).To(Succeed())
ia := acquire(model.KindPlaylistArtwork, pl.ID)
Expect(ia.Source).To(Equal("generated"))
q := gridQuadrants(storedBytes(ia))
Expect([]color.RGBA{q[0], q[1], q[2], q[3]}).To(HaveLen(4))
Expect(q[0]).ToNot(Equal(q[1]))
Expect(q[0]).ToNot(Equal(q[2]))
Expect(q[0]).ToNot(Equal(q[3]))
Expect(q[1]).ToNot(Equal(q[2]))
Expect(q[1]).ToNot(Equal(q[3]))
Expect(q[2]).ToNot(Equal(q[3]))
})
})
})
func putPlaylist(pl model.Playlist) model.Playlist {
GinkgoHelper()
if pl.OwnerID == "" {
pl.OwnerID = "admin-1"
}
Expect(rds.Playlist(rctx).Put(&pl)).To(Succeed())
return pl
}
-45
View File
@@ -1,45 +0,0 @@
package e2e
import (
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/core/artwork"
"github.com/navidrome/navidrome/model"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// Radio art is uploaded-image-only, with no fallback. Uploads are real files on disk, so they
// serve back byte-for-byte; a radio with no upload settles absent.
var _ = Describe("Radio artwork resolution", func() {
BeforeEach(func() {
setupResolutionHarness()
})
When("a radio has an uploaded image", func() {
// <DataFolder>/
// └── artwork/
// └── radio/
// └── rd-1_logo.jpg ← matched by UploadedImagePath()
It("returns the uploaded image bytes", func() {
writeUploadedImage(consts.EntityRadio, "rd-1_logo.jpg", pngBytes("radio-logo"))
rd := model.Radio{ID: "rd-1", Name: "Test Radio", StreamUrl: "https://example.com/stream", UploadedImage: "rd-1_logo.jpg"}
Expect(rds.Radio(rctx).Put(&rd)).To(Succeed())
ia := acquire(model.KindRadioArtwork, rd.ID)
Expect(ia.Source).To(Equal("upload"))
Expect(serveBytes(model.NewArtworkID(model.KindRadioArtwork, rd.ID, nil))).To(Equal(pngBytes("radio-logo")))
})
})
When("a radio has no uploaded image", func() {
// (no files on disk — the resolver has no sources to fall back to)
It("settles absent", func() {
rd := model.Radio{ID: "rd-2", Name: "Bare Radio", StreamUrl: "https://example.com/stream"}
Expect(rds.Radio(rctx).Put(&rd)).To(Succeed())
ia := acquire(model.KindRadioArtwork, rd.ID)
Expect(ia.Hash).To(BeEmpty())
Expect(serveErr(model.NewArtworkID(model.KindRadioArtwork, rd.ID, nil))).To(MatchError(artwork.ErrUnavailable))
})
})
})
-365
View File
@@ -1,365 +0,0 @@
package e2e
import (
"bytes"
"context"
"fmt"
"hash/fnv"
"image"
"image/color"
"image/png"
"io"
"maps"
"os"
"path/filepath"
"strings"
"sync"
"testing/fstest"
"time"
_ "github.com/navidrome/navidrome/adapters/gotaglib"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/core/artwork"
"github.com/navidrome/navidrome/core/metrics"
"github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/core/storage/storagetest"
"github.com/navidrome/navidrome/db"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/persistence"
"github.com/navidrome/navidrome/scanner"
"github.com/navidrome/navidrome/server/events"
"github.com/navidrome/navidrome/tests"
"github.com/navidrome/navidrome/tests/harness"
"github.com/navidrome/navidrome/utils/cache"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"go.senan.xyz/taglib"
)
const fakeLibScheme = "artworkfake"
const fakeLibPath = fakeLibScheme + ":///music"
const (
defaultCoverPriority = "cover.*, folder.*, front.*, embedded, external"
defaultDiscPriority = "disc*.*, cd*.*, cover.*, folder.*, front.*, discsubtitle, embedded"
)
var (
rctx context.Context
rds *tests.MockDataStore
rstore *artwork.ImageStore
rsvc artwork.Artwork
rworker *artwork.Worker
fakeFS *storagetest.FakeFS
)
// The go-sqlite3 singleton holds the file open for the whole suite, and Windows cannot unlink a
// file with a live handle, so the DB cannot live in Ginkgo's per-spec TempDir.
var suiteDBTempDir string
// Migrating the schema costs ~400ms, so it runs once per suite and specs reset by truncating.
var userTables []string
var _ = BeforeSuite(func() {
suiteDBTempDir = GinkgoT().TempDir()
DeferCleanup(configtest.SetupConfig())
conf.Server.DbPath = filepath.Join(suiteDBTempDir, "artwork-resolution-e2e.db") + "?_journal_mode=WAL"
conf.Server.DataFolder = conf.NewDir(GinkgoT().TempDir())
db.Db().SetMaxOpenConns(1)
db.Init(request.WithUser(context.Background(), model.User{ID: "admin-1", IsAdmin: true}))
userTables = harness.ResettableTables()
})
var _ = AfterSuite(func() {
db.Close(context.Background())
})
func setupResolutionHarness() {
DeferCleanup(configtest.SetupConfig())
tempDir := GinkgoT().TempDir()
conf.Server.DbPath = filepath.Join(suiteDBTempDir, "artwork-resolution-e2e.db") + "?_journal_mode=WAL"
conf.Server.DataFolder = conf.NewDir(tempDir)
conf.Server.MusicFolder = fakeLibPath
conf.Server.DevExternalScanner = false
conf.Server.ImageCacheSize = "0"
conf.Server.EnableExternalServices = false
conf.Server.EnableMediaFileCoverArt = true
conf.Server.DevArtworkWorkerConcurrency = 1
rctx = request.WithUser(GinkgoT().Context(), model.User{ID: "admin-1", UserName: "admin", IsAdmin: true})
harness.TruncateDB(userTables)
rds = &tests.MockDataStore{RealDS: persistence.New(db.Db())}
adminUser := model.User{ID: "admin-1", UserName: "admin", Name: "Admin", IsAdmin: true, NewPassword: "password"}
Expect(rds.User(rctx).Put(&adminUser)).To(Succeed())
lib := model.Library{ID: 1, Name: "Music", Path: fakeLibPath}
Expect(rds.Library(rctx).Put(&lib)).To(Succeed())
Expect(rds.User(rctx).SetUserLibraries(adminUser.ID, []int{lib.ID})).To(Succeed())
loadEmbeddedFixture()
fakeFS = &storagetest.FakeFS{}
storagetest.Register(fakeLibScheme, fakeFS)
ffm := tests.NewMockFFmpeg("")
rstore = artwork.NewImageStore(filepath.Join(tempDir, consts.HashedArtworkFolder))
// size=0 requests stream originals, so this reader is never called (serving_test covers resizing).
imgCache := cache.NewFileCache("ArtworkResolutionE2E", "100MB", "images", 0,
func(context.Context, cache.Item) (io.Reader, error) {
return nil, fmt.Errorf("resize not exercised in e2e")
})
Eventually(func() bool { return imgCache.Available(rctx) }, 10*time.Second).Should(BeTrue())
rsvc = artwork.NewArtwork(rds, imgCache, rstore, ffm)
rworker = artwork.NewWorker(rds, rstore, agents.GetAgents(rds, nil), ffm, events.NoopBroker(), imgCache)
}
// setLayout paths must be relative and forward-slash.
func setLayout(files fstest.MapFS) {
GinkgoHelper()
fakeFS.SetFiles(files)
}
func scan() {
GinkgoHelper()
s := scanner.New(rctx, rds, events.NoopBroker(),
playlists.NewPlaylists(rds, artwork.NewUploader(rds)), metrics.NewNoopInstance())
_, err := s.ScanAll(rctx, true)
Expect(err).ToNot(HaveOccurred())
}
func acquire(kind model.Kind, id string) model.ItemArtwork {
GinkgoHelper()
// Enqueues the way the serving paths do, so the drain is driven by a plain queue row.
Expect(rds.ArtworkQueue(rctx).EnqueuePreservingBackoff(model.ArtworkQueueItem{
ItemKind: kind.Prefix(), ItemID: id, ImageType: model.ImageTypePrimary,
Priority: model.ArtworkPriorityBump,
})).To(Succeed())
var ia *model.ItemArtwork
runResolutionWorkerUntil(func() bool {
got, err := rds.Artwork(rctx).GetItemArtwork(kind, id, model.ImageTypePrimary)
if err != nil {
return false
}
ia = got
return true
})
return *ia
}
func runResolutionWorkerUntil(until func() bool) {
GinkgoHelper()
runCtx, cancel := context.WithCancel(rctx)
done := make(chan error, 1)
go func() { done <- rworker.Run(runCtx) }()
Eventually(until, 5*time.Second, 10*time.Millisecond).Should(BeTrue())
cancel()
Eventually(done, 2*time.Second).Should(Receive(BeNil()))
}
func serveBytes(artID model.ArtworkID) []byte {
GinkgoHelper()
img, err := rsvc.Get(rctx, artID, 0, false)
Expect(err).ToNot(HaveOccurred())
defer img.Close()
data, err := io.ReadAll(img)
Expect(err).ToNot(HaveOccurred())
return data
}
func serveErr(artID model.ArtworkID) error {
img, err := rsvc.Get(rctx, artID, 0, false)
if img != nil {
img.Close()
}
return err
}
func libFileBytes(suffix string) []byte {
GinkgoHelper()
var match string
for name := range fakeFS.MapFS {
if strings.HasSuffix(name, suffix) {
Expect(match).To(BeEmpty(), "suffix %q is ambiguous: %q and %q", suffix, match, name)
match = name
}
}
Expect(match).ToNot(BeEmpty(), "no library file ends with %q", suffix)
return fakeFS.MapFS[match].Data
}
// Serving before acquiring is deliberate: with no state row the request resolves through the
// library FS, while a settled folder row is read with os.Open, which the in-memory FS cannot serve.
func expectAlbumFolderCover(al model.Album, suffix string) {
GinkgoHelper()
requireNoStateRow(model.KindAlbumArtwork, al.ID)
Expect(serveBytes(al.CoverArtID())).To(Equal(libFileBytes(suffix)))
ia := acquire(model.KindAlbumArtwork, al.ID)
Expect(ia.Source).To(Equal("folder"))
Expect(filepath.ToSlash(ia.SourcePath)).To(HaveSuffix(suffix))
}
// A drain settles every ready item, so byte-level folder assertions must precede any acquire.
func requireNoStateRow(kind model.Kind, id string) {
GinkgoHelper()
_, err := rds.Artwork(rctx).GetItemArtwork(kind, id, model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound),
"assert %s %q before acquiring any other entity in this spec", kind, id)
}
func expectAlbumAbsent(al model.Album) {
GinkgoHelper()
ia := acquire(model.KindAlbumArtwork, al.ID)
Expect(ia.Hash).To(BeEmpty())
Expect(serveErr(al.CoverArtID())).To(MatchError(artwork.ErrUnavailable))
}
func expectArtistFolder(ar model.Artist, suffix string) {
GinkgoHelper()
requireNoStateRow(model.KindArtistArtwork, ar.ID)
Expect(serveBytes(model.NewArtworkID(model.KindArtistArtwork, ar.ID, nil))).To(Equal(libFileBytes(suffix)))
ia := acquire(model.KindArtistArtwork, ar.ID)
Expect(ia.Source).To(Equal("folder"))
Expect(filepath.ToSlash(ia.SourcePath)).To(HaveSuffix(suffix))
}
func writeUploadedImage(entity, filename string, data []byte) {
GinkgoHelper()
dst := model.UploadedImagePath(entity, filename)
Expect(os.MkdirAll(filepath.Dir(dst), 0o755)).To(Succeed())
Expect(os.WriteFile(dst, data, 0o600)).To(Succeed())
}
func discArtID(al model.Album, disc int) model.ArtworkID {
return model.NewArtworkID(model.KindDiscArtwork, model.DiscArtworkID(al.ID, disc), &al.UpdatedAt)
}
// Disc art is a pure serve-time read through the library FS: no worker, no state row.
func expectDiscImage(al model.Album, disc int, label string) {
GinkgoHelper()
Expect(serveBytes(discArtID(al, disc))).To(Equal(pngBytes(label)))
}
// Samples in rect() order: top-left, top-right, bottom-left, bottom-right.
func gridQuadrants(data []byte) [4]color.RGBA {
GinkgoHelper()
img, _, err := image.Decode(bytes.NewReader(data))
Expect(err).ToNot(HaveOccurred())
b := img.Bounds()
qw, qh := b.Dx()/4, b.Dy()/4
at := func(x, y int) color.RGBA {
c := color.RGBAModel.Convert(img.At(b.Min.X+x, b.Min.Y+y))
return c.(color.RGBA)
}
return [4]color.RGBA{at(qw, qh), at(3*qw, qh), at(qw, 3*qh), at(3*qw, 3*qh)}
}
// Store-backed sources only (embedded/generated); file-backed ones assert on ia.SourcePath.
func storedBytes(ia model.ItemArtwork) []byte {
GinkgoHelper()
art, err := rds.Artwork(rctx).GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred())
r, err := rstore.Open(ia.Hash, art.Mime)
Expect(err).ToNot(HaveOccurred())
defer r.Close()
data, err := io.ReadAll(r)
Expect(err).ToNot(HaveOccurred())
return data
}
// The pixel color derives from label, so each label yields distinct, still-decodable bytes.
func smallPNG(label string) *fstest.MapFile {
h := fnv.New32a()
_, _ = h.Write([]byte(label))
sum := h.Sum32()
c := color.RGBA{R: byte(sum), G: byte(sum >> 8), B: byte(sum >> 16), A: 255}
img := image.NewRGBA(image.Rect(0, 0, 2, 2))
for y := range 2 {
for x := range 2 {
img.Set(x, y, c)
}
}
var buf bytes.Buffer
Expect(png.Encode(&buf, img)).To(Succeed())
return &fstest.MapFile{Data: buf.Bytes()}
}
func pngBytes(label string) []byte {
GinkgoHelper()
return smallPNG(label).Data
}
func trackFile(num int, title string, extra ...map[string]any) *fstest.MapFile {
tags := storagetest.Track(num, title)
for _, e := range extra {
maps.Copy(tags, e)
}
return storagetest.MP3(tags)
}
// FakeFS's JSON-encoded tags aren't taglib-readable, so embedded-art specs swap in these real MP3
// bytes after scanning. Loaded lazily: tests.Init must chdir to the project root first.
var (
embeddedFixtureOnce sync.Once
embeddedArtFixture []byte
embeddedArtBytes []byte
)
func loadEmbeddedFixture() {
embeddedFixtureOnce.Do(func() {
embeddedArtFixture = readFixture(mp3Fixture)
embeddedArtBytes = extractEmbeddedArt(embeddedArtFixture)
})
}
func extractEmbeddedArt(mp3 []byte) []byte {
tf, err := taglib.OpenStream(bytes.NewReader(mp3))
if err != nil {
panic("embedded-art fixture: taglib.OpenStream failed: " + err.Error())
}
defer tf.Close()
images := tf.Properties().Images
if len(images) == 0 {
panic("embedded-art fixture has no embedded images")
}
data, err := tf.Image(0)
if err != nil || len(data) == 0 {
panic("embedded-art fixture: could not read image 0")
}
return data
}
func replaceWithRealMP3(relPath string) {
GinkgoHelper()
fakeFS.MapFS[relPath] = &fstest.MapFile{Data: embeddedArtFixture}
}
func firstAlbum() model.Album {
GinkgoHelper()
albums, err := rds.Album(rctx).GetAll(model.QueryOptions{})
Expect(err).ToNot(HaveOccurred())
Expect(albums).To(HaveLen(1), "expected exactly one album, got %d", len(albums))
return albums[0]
}
func albumByName(name string) model.Album {
GinkgoHelper()
albums, err := rds.Album(rctx).GetAll(model.QueryOptions{})
Expect(err).ToNot(HaveOccurred())
for _, al := range albums {
if al.Name == name {
return al
}
}
Fail(fmt.Sprintf("album %q not found among %d albums", name, len(albums)))
return model.Album{}
}
+16 -18
View File
@@ -13,16 +13,19 @@ import (
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/natural"
"github.com/navidrome/navidrome/utils/slice"
)
func loadAlbumFoldersPaths(ctx context.Context, ds model.DataStore, album model.Album) ([]string, []string, *time.Time, error) {
folders, err := loadFolders(ctx, ds, album.FolderIDs)
func loadAlbumFoldersPaths(ctx context.Context, ds model.DataStore, albums ...model.Album) ([]string, []string, *time.Time, error) {
var folderIDs []string
for _, album := range albums {
folderIDs = append(folderIDs, album.FolderIDs...)
}
folders, err := ds.Folder(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"folder.id": folderIDs, "missing": false}})
if err != nil {
return nil, nil, nil, err
}
parent, err := albumRootParent(ctx, ds, folders, album.FolderIDs)
parent, err := albumRootParent(ctx, ds, folders, folderIDs)
if err != nil {
return nil, nil, nil, err
}
@@ -30,21 +33,11 @@ func loadAlbumFoldersPaths(ctx context.Context, ds model.DataStore, album model.
folders = append(folders, *parent)
}
paths := slice.Map(folders, func(f model.Folder) string { return f.AbsolutePath() })
imgFiles, updatedAt := folderImages(folders)
return paths, imgFiles, &updatedAt, nil
}
func loadFolders(ctx context.Context, ds model.DataStore, folderIDs []string) ([]model.Folder, error) {
return ds.Folder(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"folder.id": folderIDs, "missing": false}})
}
// folderImages collects the folders' image files, sorted so files without
// numeric suffixes win (e.g. cover.jpg over cover.1.jpg).
func folderImages(folders []model.Folder) ([]string, time.Time) {
var paths []string
var imgFiles []string
var updatedAt time.Time
for _, f := range folders {
paths = append(paths, f.AbsolutePath())
if f.ImagesUpdatedAt.After(updatedAt) {
updatedAt = f.ImagesUpdatedAt
}
@@ -53,8 +46,13 @@ func folderImages(folders []model.Folder) ([]string, time.Time) {
imgFiles = append(imgFiles, path.Join(rel, img))
}
}
// Sort image files to ensure consistent selection of cover art
// This prioritizes files without numeric suffixes (e.g., cover.jpg over cover.1.jpg)
// by comparing base filenames without extensions
slices.SortFunc(imgFiles, compareImageFiles)
return imgFiles, updatedAt
return paths, imgFiles, &updatedAt, nil
}
// albumRootParent returns the common parent of the album's folders when it
@@ -81,7 +79,7 @@ func albumRootParent(ctx context.Context, ds model.DataStore, folders []model.Fo
}
parent, err := ds.Folder(ctx).Get(commonParentID)
if errors.Is(err, model.ErrNotFound) {
log.Warn(ctx, "Artwork: Parent folder not found for album cover art lookup", "parentID", commonParentID)
log.Warn(ctx, "Parent folder not found for album cover art lookup", "parentID", commonParentID)
return nil, nil
}
if err != nil {
-214
View File
@@ -1,214 +0,0 @@
package artwork
import (
"context"
"errors"
"time"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// The e2e specs cover which image wins for a layout; these pin what a layout cannot reach: that
// the album-root parent is only fetched when it could qualify, and what happens when that fails.
var _ = Describe("loadAlbumFoldersPaths", func() {
var (
ctx context.Context
ds *tests.MockDataStore
repo *fakeFolderRepo
album model.Album
now time.Time
)
BeforeEach(func() {
ctx = context.Background()
now = time.Now().Truncate(time.Second)
repo = &fakeFolderRepo{}
ds = &tests.MockDataStore{MockedFolder: repo}
album = model.Album{
ID: "album1",
Name: "Album",
FolderIDs: []string{"folder1", "folder2", "folder3"},
}
})
It("does not query the parent when it is already one of the album's folders", func() {
repo.result = []model.Folder{
{ID: "folder1", Path: "Artist", Name: "Album", ParentID: "folder2",
ImagesUpdatedAt: now, ImageFiles: []string{"cover.jpg"}},
{ID: "folder2", Path: "", Name: "Artist", ImagesUpdatedAt: now},
}
_, imgFiles, _, err := loadAlbumFoldersPaths(ctx, ds, album)
Expect(err).ToNot(HaveOccurred())
Expect(imgFiles).To(ConsistOf("Artist/Album/cover.jpg"))
Expect(repo.getCallCount).To(BeZero())
})
It("does not query the parent when the album's folders have different parents", func() {
repo.result = []model.Folder{
{ID: "folder1", Path: "Artist1/Album", Name: "part1", ParentID: "parentA",
ImagesUpdatedAt: now, ImageFiles: []string{"cover.jpg"}},
{ID: "folder2", Path: "Artist2/Album", Name: "part2", ParentID: "parentB",
ImagesUpdatedAt: now},
}
_, imgFiles, _, err := loadAlbumFoldersPaths(ctx, ds, album)
Expect(err).ToNot(HaveOccurred())
Expect(imgFiles).To(ConsistOf("Artist1/Album/part1/cover.jpg"))
Expect(repo.getCallCount).To(BeZero())
})
It("does not query the parent for a single-folder album that has images of its own", func() {
repo.result = []model.Folder{
{ID: "folder1", Path: "Artist", Name: "Album", ParentID: "artistFolder",
ImagesUpdatedAt: now, ImageFiles: []string{"cover.jpg"}},
}
_, imgFiles, _, err := loadAlbumFoldersPaths(ctx, ds, album)
Expect(err).ToNot(HaveOccurred())
Expect(imgFiles).To(ConsistOf("Artist/Album/cover.jpg"))
Expect(repo.getCallCount).To(BeZero())
})
It("does not promote the library root, so its images never become album art", func() {
repo.result = []model.Folder{
{ID: "folder1", Path: ".", Name: "AlbumPart1", ParentID: "rootFolder",
ImagesUpdatedAt: now, ImageFiles: []string{"cover.jpg"}},
{ID: "folder2", Path: ".", Name: "AlbumPart2", ParentID: "rootFolder",
ImagesUpdatedAt: now},
}
repo.parentResult = &model.Folder{ID: "rootFolder", Name: ".", ImageFiles: []string{"unrelated.jpg"}}
_, imgFiles, _, err := loadAlbumFoldersPaths(ctx, ds, album)
Expect(err).ToNot(HaveOccurred())
Expect(imgFiles).To(ConsistOf("AlbumPart1/cover.jpg"))
Expect(repo.getCallCount).To(Equal(1))
})
It("does not promote a parent that holds another album's audio", func() {
repo.result = []model.Folder{
{ID: "folder1", Path: "Artist/Album", Name: "CD1", ParentID: "albumFolder",
ImagesUpdatedAt: now, ImageFiles: []string{"cover.jpg"}},
{ID: "folder2", Path: "Artist/Album", Name: "CD2", ParentID: "albumFolder",
ImagesUpdatedAt: now},
}
repo.parentResult = &model.Folder{ID: "albumFolder", Path: "Artist", Name: "Album",
ParentID: "artistFolder", ImageFiles: []string{"artist.jpg"}}
repo.hasOtherAudio = true
_, imgFiles, _, err := loadAlbumFoldersPaths(ctx, ds, album)
Expect(err).ToNot(HaveOccurred())
Expect(imgFiles).To(ConsistOf("Artist/Album/CD1/cover.jpg"))
})
It("promotes the album root parent into the returned paths", func() {
repo.result = []model.Folder{
{ID: "folder1", Path: "Artist/Album", Name: "CD1", ParentID: "albumFolder",
ImagesUpdatedAt: now},
{ID: "folder2", Path: "Artist/Album", Name: "CD2", ParentID: "albumFolder",
ImagesUpdatedAt: now},
}
repo.parentResult = &model.Folder{ID: "albumFolder", Path: "Artist", Name: "Album",
ParentID: "artistFolder", ImageFiles: []string{"cover.jpg"}}
paths, imgFiles, _, err := loadAlbumFoldersPaths(ctx, ds, album)
Expect(err).ToNot(HaveOccurred())
Expect(imgFiles).To(ConsistOf("Artist/Album/cover.jpg"))
Expect(paths).To(HaveLen(3))
})
It("propagates errors from the album-root check", func() {
repo.result = []model.Folder{
{ID: "folder1", Path: "Artist/Album", Name: "disc1", ParentID: "albumFolder",
ImagesUpdatedAt: now},
}
repo.parentResult = &model.Folder{ID: "albumFolder", Path: "Artist", Name: "Album",
ParentID: "artistFolder", ImageFiles: []string{"cover.jpg"}}
repo.otherAudioErr = errors.New("db connection failed")
_, _, _, err := loadAlbumFoldersPaths(ctx, ds, album)
Expect(err).To(MatchError("db connection failed"))
})
It("propagates non-ErrNotFound errors from the parent folder lookup", func() {
repo.result = []model.Folder{
{ID: "folder1", Path: "Artist/Album", Name: "CD1", ParentID: "parentFolder",
ImagesUpdatedAt: now, ImageFiles: []string{"cover.jpg"}},
{ID: "folder2", Path: "Artist/Album", Name: "CD2", ParentID: "parentFolder",
ImagesUpdatedAt: now},
}
repo.getErr = errors.New("db connection failed")
_, _, _, err := loadAlbumFoldersPaths(ctx, ds, album)
Expect(err).To(MatchError("db connection failed"))
Expect(repo.getCallCount).To(Equal(1))
})
It("continues when the parent folder has been deleted", func() {
repo.result = []model.Folder{
{ID: "folder1", Path: "Artist/Album", Name: "CD1", ParentID: "missingParent",
ImagesUpdatedAt: now, ImageFiles: []string{"cover.jpg"}},
{ID: "folder2", Path: "Artist/Album", Name: "CD2", ParentID: "missingParent",
ImagesUpdatedAt: now},
}
_, imgFiles, _, err := loadAlbumFoldersPaths(ctx, ds, album)
Expect(err).ToNot(HaveOccurred())
Expect(imgFiles).To(ConsistOf("Artist/Album/CD1/cover.jpg"))
Expect(repo.getCallCount).To(Equal(1))
})
})
// folderImages is the sort that decides which of several same-named images wins, so it is pinned
// directly rather than through a layout that can only show the winner.
var _ = Describe("folderImages", func() {
It("prefers base filenames over numeric-suffixed ones", func() {
imgFiles, _ := folderImages([]model.Folder{
{Path: "Artist", Name: "Album", ImageFiles: []string{"cover.1.jpg", "cover.jpg", "cover.2.jpg"}},
})
Expect(imgFiles).To(HaveExactElements(
"Artist/Album/cover.jpg", "Artist/Album/cover.1.jpg", "Artist/Album/cover.2.jpg"))
})
It("prefers shallower paths when the base filenames tie", func() {
imgFiles, _ := folderImages([]model.Folder{
{Path: "Artist/Album", Name: "CD1", ImageFiles: []string{"cover.jpg"}},
{Path: "Artist", Name: "Album", ImageFiles: []string{"cover.jpg"}},
})
Expect(imgFiles).To(HaveExactElements("Artist/Album/cover.jpg", "Artist/Album/CD1/cover.jpg"))
})
It("sorts case-insensitively", func() {
imgFiles, _ := folderImages([]model.Folder{
{Path: "Artist", Name: "Album", ImageFiles: []string{"Cover.jpg", "back.JPG"}},
})
Expect(imgFiles).To(HaveExactElements("Artist/Album/back.JPG", "Artist/Album/Cover.jpg"))
})
It("reports the newest ImagesUpdatedAt across the folders", func() {
now := time.Now().Truncate(time.Second)
newest := now.Add(5 * time.Minute)
_, updatedAt := folderImages([]model.Folder{
{Path: "Artist", Name: "Album", ImagesUpdatedAt: now},
{Path: "Artist/Album", Name: "CD1", ImagesUpdatedAt: newest},
})
Expect(updatedAt).To(Equal(newest))
})
})
+35 -81
View File
@@ -2,7 +2,6 @@ package artwork
import (
"context"
"errors"
"fmt"
"io"
"io/fs"
@@ -17,12 +16,12 @@ import (
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils"
"github.com/navidrome/navidrome/utils/slice"
"github.com/navidrome/navidrome/utils/str"
)
const (
// maxArtistFolderTraversalDepth defines how many directory levels to search
// when looking for artist images (artist folder + parent directories)
maxArtistFolderTraversalDepth = 3
)
@@ -35,64 +34,63 @@ func fromArtistFolder(ctx context.Context, libFS fs.FS, libPath, artistFolder, p
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return nil, "", fmt.Errorf(`artist folder '%s' is outside library '%s'`, artistFolder, libPath)
}
// fs.Glob needs forward slashes; filepath.Rel returns backslashes on Windows.
// fs.Glob / path.Join below expect forward-slash paths; filepath.Rel may
// return backslash separators on Windows.
rel = filepath.ToSlash(rel)
current := artistFolder
var unreadable error
for range maxArtistFolderTraversalDepth {
reader, hit, err := findImageInFolder(ctx, libFS, rel, current, pattern)
if err == nil {
return reader, hit, nil
}
if errors.Is(err, errSourceUnreadable) {
unreadable = err
}
if rel == "." {
break // reached library root
break // reached library root; don't traverse above it
}
rel = path.Dir(rel)
current = filepath.Dir(current)
}
if unreadable != nil {
return nil, "", unreadable
}
return nil, "", fmt.Errorf(`no matches for '%s' in '%s' or its parent directories (within library)`, pattern, artistFolder)
}
}
// findImageInFolder returns the first image matching pattern; absFolder is only used for
// the returned display path and log messages.
// findImageInFolder globs libFS at relFolder for pattern and returns the first
// matching image. absFolder is used only for the returned display path and log
// messages so callers see absolute-looking paths consistent with the rest of
// the artwork pipeline.
func findImageInFolder(ctx context.Context, libFS fs.FS, relFolder, absFolder, pattern string) (io.ReadCloser, string, error) {
log.Trace(ctx, "Artwork: Looking for artist image", "pattern", pattern, "folder", absFolder)
log.Trace(ctx, "looking for artist image", "pattern", pattern, "folder", absFolder)
globPattern := pattern
if relFolder != "." {
globPattern = path.Join(escapeGlobLiteral(relFolder), pattern)
}
matches, err := fs.Glob(libFS, globPattern)
if err != nil {
log.Warn(ctx, "Artwork: Error matching artist image pattern", "pattern", pattern, "folder", absFolder, err)
log.Warn(ctx, "Error matching artist image pattern", "pattern", pattern, "folder", absFolder, err)
return nil, "", err
}
imagePaths := slice.Filter(matches, model.IsImageFile)
// Filter to valid image files
var imagePaths []string
for _, m := range matches {
if !model.IsImageFile(m) {
continue
}
imagePaths = append(imagePaths, m)
}
// Prefer base filenames over numeric-suffixed ones (artist.jpg before artist.1.jpg)
// Sort image files by prioritizing base filenames without numeric
// suffixes (e.g., artist.jpg before artist.1.jpg)
slices.SortFunc(imagePaths, compareImageFiles)
var openErr error
for _, p := range imagePaths {
f, err := libFS.Open(p)
if err != nil {
log.Warn(ctx, "Artwork: Could not open cover art file", "file", p, err)
openErr = fmt.Errorf("%w: %s: %w", errSourceUnreadable, p, err)
log.Warn(ctx, "Could not open cover art file", "file", p, err)
continue
}
_, name := path.Split(p)
return f, filepath.Join(absFolder, name), nil
}
if openErr != nil {
return nil, "", openErr
}
return nil, "", fmt.Errorf(`no matches for '%s' in '%s'`, pattern, absFolder)
}
@@ -110,82 +108,38 @@ func escapeGlobLiteral(s string) string {
return b.String()
}
// loadArtistAlbumRoots returns one path per album — the deepest folder holding
// all of that album's tracks — so an album split into disc subfolders can't
// pull the artist folder's common prefix below the artist level.
func loadArtistAlbumRoots(ctx context.Context, ds model.DataStore, albums model.Albums) ([]string, []string, *time.Time, error) {
var folderIDs []string
for _, album := range albums {
folderIDs = append(folderIDs, album.FolderIDs...)
}
folders, err := loadFolders(ctx, ds, folderIDs)
if err != nil {
return nil, nil, nil, err
}
pathByID := slice.ToMap(folders, func(f model.Folder) (string, string) {
return f.ID, f.AbsolutePath()
})
var roots []string
for _, album := range albums {
var albumPaths []string
for _, fid := range album.FolderIDs {
if p, ok := pathByID[fid]; ok {
albumPaths = append(albumPaths, p)
}
}
if len(albumPaths) > 0 {
roots = append(roots, commonDir(albumPaths))
}
}
imgFiles, updatedAt := folderImages(folders)
return roots, imgFiles, &updatedAt, nil
}
// commonDir returns the deepest directory containing all paths. Trailing
// separators keep the comparison on segment boundaries, so a shared name
// fragment (".../Album" and ".../Album2") is never read as a shared directory.
func commonDir(paths []string) string {
sep := string(filepath.Separator)
common := str.LongestCommonPrefix(slice.Map(paths, func(p string) string { return p + sep }))
if !strings.HasSuffix(common, sep) {
common, _ = filepath.Split(common)
}
return filepath.Clean(common)
}
func loadArtistFolder(ctx context.Context, ds model.DataStore, albums model.Albums, paths []string) (string, time.Time, error) {
if len(albums) == 0 {
return "", time.Time{}, nil
}
libID := albums[0].LibraryID // TODO: Support albums spanning multiple libraries
libID := albums[0].LibraryID // Just need one of the albums, as they should all be in the same Library - for now! TODO: Support multiple libraries
// paths holds one root per album: two or more distinct roots already meet at
// the artist folder, while a single root is an album folder needing a climb.
roots := slices.Compact(slices.Sorted(slices.Values(paths)))
folderPath := commonDir(roots)
if len(roots) < 2 {
folderPath = filepath.Dir(folderPath)
folderPath := str.LongestCommonPrefix(paths)
if !strings.HasSuffix(folderPath, string(filepath.Separator)) {
folderPath, _ = filepath.Split(folderPath)
}
folderPath = filepath.Dir(folderPath)
// TODO: Hacky, but the easiest way to get the folder ID ATM
// Manipulate the path to get the folder ID
// TODO: This is a bit hacky, but it's the easiest way to get the folder ID, ATM
libPath := core.AbsolutePath(ctx, ds, libID, "")
folderID := model.FolderID(model.Library{ID: libID, Path: libPath}, folderPath)
log.Trace(ctx, "Artwork: Calculating artist folder details", "folderPath", folderPath, "folderID", folderID,
log.Trace(ctx, "Calculating artist folder details", "folderPath", folderPath, "folderID", folderID,
"libPath", libPath, "libID", libID, "albumPaths", paths)
// Get the last update time for the folder
folders, err := ds.Folder(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"folder.id": folderID, "missing": false}})
if err != nil || len(folders) == 0 {
log.Warn(ctx, "Artwork: Could not find folder for artist", "folderPath", folderPath, "id", folderID,
log.Warn(ctx, "Could not find folder for artist", "folderPath", folderPath, "id", folderID,
"libPath", libPath, "libID", libID, err)
return "", time.Time{}, err
}
return folderPath, folders[0].ImagesUpdatedAt, nil
}
// findImageInArtistFolder matches an image by MBID or artist name (case-insensitive), "" if none.
// findImageInArtistFolder scans a folder for an image file matching the artist's MBID or name
// (case-insensitive). Returns the full path, or empty string if not found.
func findImageInArtistFolder(folder, mbzArtistID, artistName string) string {
entries, err := os.ReadDir(folder)
if err != nil {
@@ -200,7 +154,7 @@ func findImageInArtistFolder(folder, mbzArtistID, artistName string) string {
continue
}
name := entry.Name()
base := utils.BaseName(name)
base := strings.TrimSuffix(name, filepath.Ext(name))
if strings.EqualFold(base, candidate) && model.IsImageFile(name) {
return filepath.Join(folder, name)
}
-117
View File
@@ -1,117 +0,0 @@
package artwork
import (
"context"
"errors"
"path/filepath"
"time"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// commonDir and loadArtistFolder decide how far above the albums the artist folder sits. A layout
// can only show the image that won, so the arithmetic is pinned here.
var _ = Describe("commonDir", func() {
It("returns the folder itself for a single path", func() {
Expect(commonDir([]string{filepath.FromSlash("/music/artist/album")})).
To(Equal(filepath.FromSlash("/music/artist/album")))
})
It("returns the deepest shared folder", func() {
Expect(commonDir([]string{
filepath.FromSlash("/music/artist/album/cd1"),
filepath.FromSlash("/music/artist/album/cd2"),
})).To(Equal(filepath.FromSlash("/music/artist/album")))
})
It("does not read a shared name fragment as a shared folder", func() {
Expect(commonDir([]string{
filepath.FromSlash("/music/artist/Album"),
filepath.FromSlash("/music/artist/Album2"),
})).To(Equal(filepath.FromSlash("/music/artist")))
})
})
var _ = Describe("loadArtistFolder", func() {
var (
ctx context.Context
ds *tests.MockDataStore
repo *fakeFolderRepo
albums model.Albums
updatedAt time.Time
)
BeforeEach(func() {
ctx = context.Background()
DeferCleanup(stubCoreAbsolutePath())
updatedAt = time.Now().Truncate(time.Second).Add(5 * time.Minute)
repo = &fakeFolderRepo{result: []model.Folder{{ImagesUpdatedAt: updatedAt}}}
ds = &tests.MockDataStore{MockedFolder: repo}
albums = model.Albums{{LibraryID: 1, ID: "album1", Name: "Album 1"}}
})
It("returns empty when the artist has no albums", func() {
folder, upd, err := loadArtistFolder(ctx, ds, model.Albums{}, []string{"/dummy/path"})
Expect(err).ToNot(HaveOccurred())
Expect(folder).To(BeEmpty())
Expect(upd).To(BeZero())
})
It("climbs above the album folder when the artist has a single album root", func() {
folder, upd, err := loadArtistFolder(ctx, ds, albums,
[]string{filepath.FromSlash("/music/artist/album1")})
Expect(err).ToNot(HaveOccurred())
Expect(folder).To(Equal(filepath.FromSlash("/music/artist")))
Expect(upd).To(Equal(updatedAt))
})
It("climbs above the shared folder when two albums live in the same one", func() {
folder, upd, err := loadArtistFolder(ctx, ds, albums, []string{
filepath.FromSlash("/music/artist/split"),
filepath.FromSlash("/music/artist/split"),
})
Expect(err).ToNot(HaveOccurred())
Expect(folder).To(Equal(filepath.FromSlash("/music/artist")))
Expect(upd).To(Equal(updatedAt))
})
It("stops at the folder where distinct album roots already meet", func() {
folder, upd, err := loadArtistFolder(ctx, ds, albums, []string{
filepath.FromSlash("/music/artist/album1"),
filepath.FromSlash("/music/artist/album2"),
})
Expect(err).ToNot(HaveOccurred())
Expect(folder).To(Equal(filepath.FromSlash("/music/artist")))
Expect(upd).To(Equal(updatedAt))
})
It("returns the error when the folder lookup fails", func() {
repo.err = errors.New("fake error")
folder, upd, err := loadArtistFolder(ctx, ds, albums, []string{
filepath.FromSlash("/music/artist/album1"),
filepath.FromSlash("/music/artist/album2"),
})
Expect(err).To(MatchError(ContainSubstring("fake error")))
Expect(folder).To(BeEmpty())
Expect(upd).To(BeZero())
})
})
func stubCoreAbsolutePath() func() {
original := core.AbsolutePath
core.AbsolutePath = func(context.Context, model.DataStore, int, string) string {
return filepath.FromSlash("/music")
}
return func() { core.AbsolutePath = original }
}
-47
View File
@@ -1,47 +0,0 @@
package artwork
import (
"context"
"errors"
"io/fs"
"testing/fstest"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// unreadableFS globs like its embedded MapFS but refuses to open anything. Injecting the error
// keeps this independent of the filesystem: os.Chmod does not restrict read access on Windows.
type unreadableFS struct{ fstest.MapFS }
func (u unreadableFS) Open(string) (fs.File, error) { return nil, fs.ErrPermission }
var _ = Describe("findImageInFolder", func() {
var ctx context.Context
var files fstest.MapFS
BeforeEach(func() {
ctx = context.Background()
files = fstest.MapFS{"artist.jpg": &fstest.MapFile{Data: []byte("img")}}
})
It("returns the first matching image", func() {
r, hit, err := findImageInFolder(ctx, files, ".", "/lib", "artist.*")
Expect(err).ToNot(HaveOccurred())
defer r.Close()
Expect(hit).To(HaveSuffix("artist.jpg"))
})
// The glob matched, so the image exists; failing to open it says nothing about whether the
// artist has one, and must not let the resolver settle on absent.
It("reports a matched but unreadable image as unreadable, not as a miss", func() {
_, _, err := findImageInFolder(ctx, unreadableFS{files}, ".", "/lib", "artist.*")
Expect(err).To(MatchError(errSourceUnreadable))
})
It("reports a plain miss when nothing matches", func() {
_, _, err := findImageInFolder(ctx, files, ".", "/lib", "nothing.*")
Expect(err).To(HaveOccurred())
Expect(errors.Is(err, errSourceUnreadable)).To(BeFalse(), "no match is definitive, not transient")
})
})
-171
View File
@@ -1,171 +0,0 @@
package artwork
import (
"cmp"
"context"
"errors"
"io"
"sync"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"golang.org/x/time/rate"
)
const (
breakerThreshold = 5
breakerProbeAfter = time.Minute
// breakerRecoveries is how many consecutive answers an open breaker needs before it trusts the
// provider again. One is not enough: a provider that is rate-limiting or blocking us still
// answers the occasional request, and closing on the first of those puts the agent straight
// back to full rate, which is what earns the next block.
breakerRecoveries = 3
)
var errBreakerOpen = errors.New("artwork: external circuit breaker open")
// gateFunc gates one named external fetch (rate limit + circuit breaker per name).
type gateFunc = func(name string, f func() (io.ReadCloser, string, error)) (io.ReadCloser, string, error)
func passthroughGate(_ string, f func() (io.ReadCloser, string, error)) (io.ReadCloser, string, error) {
return f()
}
// isTransientExternal reports whether an external failure is worth retrying; a not-found
// (from either package) is a definitive answer, not a fault.
func isTransientExternal(err error) bool {
return err != nil && !errors.Is(err, agents.ErrNotFound) && !errors.Is(err, model.ErrNotFound)
}
// extGate is one agent's rate limiter and circuit breaker, so a failing provider backs off
// in isolation from the others.
type extGate struct {
limiter *rate.Limiter
breaker *breaker
}
// gate runs a named external step through that agent's rate limiter and circuit breaker.
func (w *Worker) gate(name string, f func() (io.ReadCloser, string, error)) (io.ReadCloser, string, error) {
g := w.gateFor(name)
allowed, gen := g.breaker.allow()
if !allowed {
log.Debug(w.runCtx, "Artwork: Skipping agent, circuit breaker open", "agent", name)
return nil, "", errBreakerOpen
}
// Timed separately so a throttled agent isn't mistaken for a slow provider.
waitStart := time.Now()
if err := g.limiter.Wait(w.runCtx); err != nil {
return nil, "", err
}
callStart := time.Now()
r, path, err := f()
g.breaker.record(name, gen, err)
log.Trace(w.runCtx, "Artwork: External agent call", "agent", name, "hit", r != nil,
"limiterWait", callStart.Sub(waitStart), "elapsed", time.Since(callStart), err)
return r, path, err
}
// gateFor lazily creates the per-name gate on first use.
func (w *Worker) gateFor(name string) *extGate {
w.gatesMu.Lock()
defer w.gatesMu.Unlock()
if g, ok := w.gates[name]; ok {
return g
}
rps := conf.Server.DevArtworkExternalMaxRPS
limit := rate.Inf
if rps > 0 {
limit = rate.Limit(rps)
}
g := &extGate{limiter: rate.NewLimiter(limit, max(1, rps)), breaker: newBreaker()}
w.gates[name] = g
return g
}
// breaker opens after breakerThreshold consecutive errors and admits a single probe once
// breakerProbeAfter has elapsed; it closes after breakerRecoveries consecutive answers.
type breaker struct {
mu sync.Mutex
failures int
openedAt time.Time
// recoveries counts consecutive good answers while open; a single failure discards them.
recoveries int
// generation identifies the current open episode, so an answer from a call admitted before
// the breaker opened cannot be mistaken for evidence that it has recovered.
generation int
// probeAfter overrides the probe delay for the current episode when a provider named its own
// back-off; zero falls back to breakerProbeAfter.
probeAfter time.Duration
}
func newBreaker() *breaker { return &breaker{} }
// allow reports whether a call may proceed, and the open episode it was admitted under: zero
// when the breaker was closed, the current generation when admitted as a half-open probe.
func (b *breaker) allow() (bool, int) {
b.mu.Lock()
defer b.mu.Unlock()
if b.failures < breakerThreshold {
return true, 0
}
if time.Since(b.openedAt) >= cmp.Or(b.probeAfter, breakerProbeAfter) {
b.openedAt = time.Now() // start a fresh probe window so only one caller passes
return true, b.generation
}
return false, 0
}
func (b *breaker) record(name string, gen int, err error) {
// A cancelled run says nothing about the provider, so it neither counts nor clears.
if errors.Is(err, context.Canceled) {
return
}
b.mu.Lock()
defer b.mu.Unlock()
// An explicit back-off is a definitive "stop for this long", so it opens the breaker at once
// with the provider's own delay instead of waiting for the failure threshold.
if retry, ok := errors.AsType[*agents.RetryLaterError](err); ok && retry.RetryIn > 0 {
b.recoveries = 0
b.failures = breakerThreshold
b.openedAt = time.Now()
b.probeAfter = retry.RetryIn
b.generation++
log.Warn("Artwork: Circuit breaker opened for agent, provider asked to back off", "agent", name,
"probeAfter", retry.RetryIn)
return
}
if isTransientExternal(err) {
b.recoveries = 0
b.failures++
if b.failures == breakerThreshold {
b.openedAt = time.Now()
b.probeAfter = 0
b.generation++
log.Warn("Artwork: Circuit breaker opened for agent", "agent", name,
"consecutiveFailures", b.failures, "probeAfter", breakerProbeAfter, err)
}
return
}
if b.failures < breakerThreshold {
b.failures = 0
return
}
// Only a probe from this open episode is evidence of recovery. The worker drains concurrently,
// so answers keep arriving from calls admitted before the breaker opened; counting those would
// close it with no probe interval elapsed, which is the burst this exists to prevent.
if gen == 0 || gen != b.generation {
return
}
// A not-found counts because the provider did answer, but on its own it is thin evidence that
// a provider which just blocked us is well.
b.recoveries++
if b.recoveries < breakerRecoveries {
return
}
log.Info("Artwork: Circuit breaker closed for agent", "agent", name,
"consecutiveAnswers", b.recoveries)
b.failures, b.recoveries = 0, 0
}
-56
View File
@@ -1,56 +0,0 @@
package artwork
import (
"errors"
"time"
"github.com/navidrome/navidrome/core/agents"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// allowed drops the generation token when a caller only cares about admission.
func allowed(b *breaker) bool { ok, _ := b.allow(); return ok }
var _ = Describe("breaker", func() {
// The worker drains concurrently, so when the breaker opens there are already calls past
// allow(), queued in the rate limiter or waiting on a response. Their answers arrive
// afterwards. Counting those as recovery closes the breaker with no probe interval elapsed,
// which is the burst the ramp exists to prevent. No clock is involved: the race is an
// ordering, so it is reproduced by making the calls in the order concurrency produces.
It("ignores answers from calls admitted before it opened", func() {
b := newBreaker()
// A batch clears allow() while the breaker is still closed.
for range breakerThreshold + breakerRecoveries {
ok, gen := b.allow()
Expect(ok).To(BeTrue())
Expect(gen).To(BeZero(), "admitted with the breaker closed, so not a probe")
}
// The fast failures in that batch open it.
for range breakerThreshold {
b.record("agentA", 0, errors.New("blocked"))
}
Expect(allowed(b)).To(BeFalse(), "breaker is open")
// The slower answers from the same batch land now.
for range breakerRecoveries {
b.record("agentA", 0, nil)
}
Expect(allowed(b)).To(BeFalse(),
"answers from calls admitted before the breaker opened must not close it")
})
It("opens at once when a provider asks to retry later, honoring its delay", func() {
b := newBreaker()
Expect(allowed(b)).To(BeTrue(), "starts closed")
// A single explicit back-off opens the breaker without reaching the failure threshold.
b.record("agentA", 0, &agents.RetryLaterError{RetryIn: 5 * time.Second})
Expect(allowed(b)).To(BeFalse(), "an explicit back-off opens the breaker immediately")
Expect(b.probeAfter).To(Equal(5*time.Second), "the provider's delay drives the probe interval")
})
})
-51
View File
@@ -1,51 +0,0 @@
package artwork
import (
"fmt"
"image"
"testing"
"github.com/navidrome/navidrome/core/artwork/blurhash"
"github.com/navidrome/navidrome/core/artwork/thumbhash"
)
// hashEncoders are the two placeholder hashes decodeArtwork computes from one shared thumbnail.
var hashEncoders = []struct {
name string
encode func(image.Image) error
}{
{"blurhash", func(img image.Image) error { _, err := blurhash.Encode(img); return err }},
{"thumbhash", func(img image.Image) error { _, err := thumbhash.Encode(img); return err }},
}
func benchEncoder(b *testing.B, encode func(image.Image) error, img image.Image) {
b.Helper()
b.ReportAllocs()
for b.Loop() {
if err := encode(img); err != nil {
b.Fatal(err)
}
}
}
// BenchmarkHashEncodersAtInputSize is the bar: both encoders are handed the identical image
// makeThumbnail produces, so neither is measured with a conversion the other avoids.
func BenchmarkHashEncodersAtInputSize(b *testing.B) {
img := gradientNRGBA(thumbnailSize)
for _, e := range hashEncoders {
b.Run(e.name, func(b *testing.B) { benchEncoder(b, e.encode, img) })
}
}
// BenchmarkHashEncoders sweeps past the pipeline's input size, where each package's own defensive
// downscale starts to dominate. thumbnailSize itself is covered by the benchmark above.
func BenchmarkHashEncoders(b *testing.B) {
for _, size := range []int{300, 600, 900, 1200, 1500} {
img := gradientNRGBA(size)
for _, e := range hashEncoders {
b.Run(fmt.Sprintf("%s/%dx%d", e.name, size, size), func(b *testing.B) {
benchEncoder(b, e.encode, img)
})
}
}
}
+75 -146
View File
@@ -2,172 +2,101 @@ package artwork
import (
"context"
"crypto/md5"
"encoding/hex"
"fmt"
"slices"
"strconv"
"strings"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/core/auth"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/slice"
"github.com/zeebo/xxh3"
)
// ReprocessKinds omits media files: they resolve embedded-only, at scan or on view. Artists lead
// so bulk enqueues give the most external-dependent kind a queue headstart.
var ReprocessKinds = []model.Kind{
model.KindArtistArtwork, model.KindAlbumArtwork, model.KindPlaylistArtwork, model.KindRadioArtwork,
// FingerprintPropertyKey is the model.PropertyRepository key Backfill compares against
// to detect artwork-affecting config changes across restarts.
const FingerprintPropertyKey = "artwork.fingerprint"
// staleAbsentAge is how old an absent resolution must be before the recheck job retries it.
const staleAbsentAge = 24 * time.Hour
// staleAbsentKinds are the item kinds eligible for the periodic stale-absent recheck.
var staleAbsentKinds = []string{"ar", "al", "pl", "ra"}
// Fingerprint summarizes the config knobs that affect artwork resolution outcomes; a
// change means previously resolved (or absent) state may no longer be correct.
func Fingerprint() string {
raw := fmt.Sprintf("%s|%s|%s|%s|%t|%t|%s",
conf.Server.CoverArtPriority, conf.Server.ArtistArtPriority, conf.Server.ArtistImageFolder,
conf.Server.Agents, conf.Server.EnableExternalServices, conf.Server.EnableM3UExternalAlbumArt, consts.Version)
sum := md5.Sum([]byte(raw)) //nolint:gosec // fingerprint, not security-sensitive
return hex.EncodeToString(sum[:])
}
// KeepsState reports whether a kind is recorded in item_artwork and the artwork queue. Disc
// artwork is read through on every request and cached by content key, so it has neither.
func KeepsState(kind model.Kind) bool { return kind != model.KindDiscArtwork }
// RefreshableKinds is every kind Refresh can clear and re-queue, so it holds exactly the kinds
// KeepsState admits. Media files are absent from ReprocessKinds but belong here: the worker
// resolves them, it just never enumerates them in bulk.
var RefreshableKinds = append(slices.Clone(ReprocessKinds), model.KindMediaFileArtwork)
// settlesAbsentOnGiveUp reports whether an exhausted retry budget records an absent state. Media
// files are excluded because retrying one costs nothing: they resolve embedded-only, from a local
// read, and only a view ever enqueues them.
func settlesAbsentOnGiveUp(prefix string) bool {
kind, ok := model.ParseKind(prefix)
return ok && KeepsState(kind) && kind != model.KindMediaFileArtwork
}
// artworkEpoch invalidates all resolution state when bumped; bump it whenever resolution semantics change.
const artworkEpoch = 1
// FingerprintInput is one config value the fingerprint covers, named after the setting it came from.
type FingerprintInput struct {
Name string
Value string
}
// FingerprintInputs is the single listing of what ConfigFingerprint hashes.
func FingerprintInputs() []FingerprintInput {
return []FingerprintInput{
{"CoverArtPriority", conf.Server.CoverArtPriority},
{"ArtistArtPriority", conf.Server.ArtistArtPriority},
{"ArtistImageFolder", conf.Server.ArtistImageFolder},
{"Agents", conf.Server.Agents},
{"EnableExternalServices", strconv.FormatBool(conf.Server.EnableExternalServices)},
{"EnableM3UExternalAlbumArt", strconv.FormatBool(conf.Server.EnableM3UExternalAlbumArt)},
}
}
// ConfigFingerprint covers the inputs that affect resolution outcomes; a change invalidates stored state.
func ConfigFingerprint() string {
values := slice.Map(FingerprintInputs(), func(i FingerprintInput) string { return i.Value })
raw := fmt.Sprintf("%s|%d", strings.Join(values, "|"), artworkEpoch)
return fmt.Sprintf("%016x", xxh3.Hash([]byte(raw)))
}
// ReconcileConfigFingerprint warns when the artwork config changed since the library was last
// resolved under it. Nothing re-resolves on its own; applying a change is an explicit reprocess.
func ReconcileConfigFingerprint(ctx context.Context, ds model.DataStore) error {
current := ConfigFingerprint()
stored, err := ds.Property(ctx).DefaultGet(consts.ArtConfFingerprintPropertyKey, "")
// Backfill enqueues artwork resolution for every entity when the config fingerprint changed
// (or was never stored), artists first so those pages resolve before the larger backlog.
func Backfill(ctx context.Context, ds model.DataStore) (bool, error) {
ctx = auth.WithAdminUser(ctx, ds)
current := Fingerprint()
props := ds.Property(ctx)
stored, err := props.DefaultGet(FingerprintPropertyKey, "")
if err != nil {
return err
return false, err
}
switch stored {
case current:
case "":
// An unset fingerprint counts as current; the alternative warns every upgrading install once.
return MarkConfigApplied(ctx, ds)
default:
log.Warn(ctx, "Artwork: Config changed since the last full reprocess. Stored artwork keeps "+
"the old resolution; run 'navidrome artwork reprocess --all' to apply the change",
"stored", stored, "current", current, "inputs", FingerprintInputs())
if stored == current {
return false, nil
}
return nil
// Artists first: few entities, most external-dependent, so they get queue headstart.
kinds := []struct {
kind string
fetch func() ([]string, error)
}{
{"ar", func() ([]string, error) { return ds.Artist(ctx).GetAllIDs() }},
{"al", func() ([]string, error) { return ds.Album(ctx).GetAllIDs() }},
{"pl", func() ([]string, error) { return ds.Playlist(ctx).GetAllIDs() }},
{"ra", func() ([]string, error) { return ds.Radio(ctx).GetAllIDs() }},
}
for _, k := range kinds {
ids, err := k.fetch()
if err != nil {
return false, err
}
if err := enqueueBackfillKind(ctx, ds, k.kind, ids); err != nil {
return false, err
}
}
if err := props.Put(FingerprintPropertyKey, current); err != nil {
return false, err
}
log.Info(ctx, "Artwork: config fingerprint changed, backfill enqueued")
return true, nil
}
// MarkConfigApplied records the current fingerprint as the one the library is resolved under.
func MarkConfigApplied(ctx context.Context, ds model.DataStore) error {
return ds.Property(ctx).Put(consts.ArtConfFingerprintPropertyKey, ConfigFingerprint())
func enqueueBackfillKind(ctx context.Context, ds model.DataStore, kind string, ids []string) error {
if len(ids) == 0 {
return nil
}
items := make([]model.ArtworkQueueItem, len(ids))
for i, id := range ids {
items[i] = model.ArtworkQueueItem{
ItemKind: kind, ItemID: id, ImageType: model.ImageTypePrimary, Priority: model.ArtworkPriorityBackfill,
}
}
return ds.ArtworkQueue(ctx).Enqueue(items...)
}
// enqueueMissingAll is the safety net for entities a scan never enqueued (added between scans, or scanner off).
func enqueueMissingAll(ctx context.Context, ds model.DataStore) error {
// EnqueueStaleAbsentAll requeues absent-state entries older than staleAbsentAge, across
// every artwork-bearing kind, for the periodic recheck job.
func EnqueueStaleAbsentAll(ctx context.Context, ds model.DataStore) error {
cutoff := time.Now().Add(-staleAbsentAge)
queue := ds.ArtworkQueue(ctx)
for _, kind := range ReprocessKinds {
if _, err := queue.EnqueueAllMissing(kind, model.ArtworkPriorityRecheck); err != nil {
for _, kind := range staleAbsentKinds {
if _, err := queue.EnqueueStaleAbsent(kind, cutoff); err != nil {
return err
}
}
return nil
}
// ItemName resolves a kind+id to the entity's display name, and errors when the item
// does not exist. Callers use it to reject ids that would otherwise orphan a queue row.
func ItemName(ctx context.Context, ds model.DataStore, kind model.Kind, id string) (string, error) {
switch kind {
case model.KindArtistArtwork:
ar, err := ds.Artist(ctx).Get(id)
if err != nil {
return "", err
}
return ar.Name, nil
case model.KindAlbumArtwork:
al, err := ds.Album(ctx).Get(id)
if err != nil {
return "", err
}
return al.Name, nil
case model.KindPlaylistArtwork:
pls, err := ds.Playlist(ctx).Get(id)
if err != nil {
return "", err
}
return pls.Name, nil
case model.KindRadioArtwork:
rd, err := ds.Radio(ctx).Get(id)
if err != nil {
return "", err
}
return rd.Name, nil
case model.KindMediaFileArtwork:
mf, err := ds.MediaFile(ctx).Get(id)
if err != nil {
return "", err
}
return mf.Title, nil
case model.KindDiscArtwork:
return discArtworkName(ctx, ds, id)
}
return "", fmt.Errorf("unsupported kind %q", kind.Prefix())
}
func discArtworkName(ctx context.Context, ds model.DataStore, id string) (string, error) {
albumID, discNumber, err := model.ParseDiscArtworkID(id)
if err != nil {
return "", err
}
al, err := ds.Album(ctx).Get(albumID)
if err != nil {
return "", err
}
name := fmt.Sprintf("%s (disc %d)", al.Name, discNumber)
// The subtitle is itself a DiscArtPriority candidate, so name it where the chain can be read against it.
if subtitle := strings.TrimSpace(al.Discs[discNumber]); subtitle != "" {
name += ": " + subtitle
}
return name, nil
}
// Refresh drops an item's resolved artwork state and re-queues it at Bump priority.
func Refresh(ctx context.Context, ds model.DataStore, kind model.Kind, id string) error {
if err := ds.Artwork(ctx).DeleteForItems(kind, []string{id}); err != nil {
return fmt.Errorf("clearing artwork state: %w", err)
}
item := model.ArtworkQueueItem{ItemKind: kind.Prefix(), ItemID: id, ImageType: model.ImageTypePrimary, Priority: model.ArtworkPriorityBump}
if err := ds.ArtworkQueue(ctx).Enqueue(item); err != nil {
return fmt.Errorf("enqueuing artwork refresh: %w", err)
}
return nil
}
+162 -131
View File
@@ -2,35 +2,59 @@ package artwork
import (
"context"
"slices"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("RefreshableKinds", func() {
// The two are meant to describe the same fact. Nothing but this test stops them from drifting,
// and a drift would have `artwork explain` report state for a kind that keeps none.
It("holds exactly the kinds that keep state", func() {
for _, k := range []model.Kind{
model.KindArtistArtwork, model.KindAlbumArtwork, model.KindPlaylistArtwork,
model.KindRadioArtwork, model.KindMediaFileArtwork, model.KindDiscArtwork,
} {
Expect(slices.Contains(RefreshableKinds, k)).To(Equal(KeepsState(k)), k.String())
}
})
})
// visibilityPlaylistDS models playlist_repository's userFilter: a private playlist is only
// visible when the ctx carries an admin, so headless work must wrap ctx with one first.
type visibilityPlaylistDS struct {
*tests.MockDataStore
private model.Playlist
tracks model.PlaylistTrackRepository
}
func (v *visibilityPlaylistDS) Playlist(ctx context.Context) model.PlaylistRepository {
repo := tests.CreateMockPlaylistRepo()
repo.TracksRepo = v.tracks
if u, ok := request.UserFrom(ctx); ok && u.IsAdmin {
repo.SetData(model.Playlists{v.private})
}
return repo
}
func adminUserRepo() *tests.MockedUserRepo {
repo := tests.CreateMockUserRepo()
Expect(repo.Put(&model.User{ID: "admin", UserName: "admin", IsAdmin: true})).To(Succeed())
return repo
}
// orderTrackingQueueRepo records the item kind of each Enqueue call, so tests can
// assert phase ordering (artists-first) that same-priority timestamps can't guarantee.
type orderTrackingQueueRepo struct {
*tests.MockArtworkQueueRepo
callKinds []string
}
func (o *orderTrackingQueueRepo) Enqueue(items ...model.ArtworkQueueItem) error {
if len(items) > 0 {
o.callKinds = append(o.callKinds, items[0].ItemKind)
}
return o.MockArtworkQueueRepo.Enqueue(items...)
}
var _ = Describe("Housekeeping", func() {
var (
ctx context.Context
ds *tests.MockDataStore
queueRepo *tests.MockArtworkQueueRepo
queueRepo *orderTrackingQueueRepo
propRepo *tests.MockedPropertyRepo
)
@@ -42,154 +66,161 @@ var _ = Describe("Housekeeping", func() {
conf.Server.Agents = "spotify"
conf.Server.EnableExternalServices = true
queueRepo = tests.CreateMockArtworkQueueRepo()
queueRepo = &orderTrackingQueueRepo{MockArtworkQueueRepo: tests.CreateMockArtworkQueueRepo()}
propRepo = &tests.MockedPropertyRepo{}
ds = &tests.MockDataStore{MockedArtworkQueue: queueRepo, MockedProperty: propRepo}
})
seedEntities := func() {
artistRepo := tests.CreateMockArtistRepo()
artistRepo.SetData(model.Artists{{ID: "ar1"}, {ID: "ar2"}})
ds.MockedArtist = artistRepo
albumRepo := tests.CreateMockAlbumRepo()
albumRepo.SetData(model.Albums{{ID: "al1"}})
ds.MockedAlbum = albumRepo
playlistRepo := tests.CreateMockPlaylistRepo()
playlistRepo.SetData(model.Playlists{{ID: "pl1"}})
ds.MockedPlaylist = playlistRepo
radioRepo := tests.CreateMockedRadioRepo()
radioRepo.All = model.Radios{{ID: "ra1"}}
ds.MockedRadio = radioRepo
}
Describe("Fingerprint", func() {
DescribeTable("changes when a fingerprint-affecting config value changes",
func(change func()) {
before := ConfigFingerprint()
change()
Expect(ConfigFingerprint()).NotTo(Equal(before))
},
Entry("CoverArtPriority", func() { conf.Server.CoverArtPriority = "folder, embedded" }),
Entry("ArtistImageFolder", func() { conf.Server.ArtistImageFolder = "/after" }),
Entry("EnableM3UExternalAlbumArt", func() { conf.Server.EnableM3UExternalAlbumArt = true }),
)
It("changes when a fingerprint-affecting config value changes", func() {
f1 := Fingerprint()
conf.Server.CoverArtPriority = "folder, embedded"
f2 := Fingerprint()
Expect(f1).NotTo(Equal(f2))
})
// Pinned: a changed formula tells every existing install its artwork config went stale.
It("hashes a given config to a stable value", func() {
conf.Server.CoverArtPriority = "cover.*, embedded"
conf.Server.ArtistArtPriority = "artist.*, external"
conf.Server.ArtistImageFolder = ""
conf.Server.Agents = "lastfm,spotify"
conf.Server.EnableExternalServices = true
It("changes when ArtistImageFolder changes", func() {
conf.Server.ArtistImageFolder = "/before"
f1 := Fingerprint()
conf.Server.ArtistImageFolder = "/after"
Expect(Fingerprint()).NotTo(Equal(f1))
})
It("changes when EnableM3UExternalAlbumArt is toggled", func() {
conf.Server.EnableM3UExternalAlbumArt = false
Expect(ConfigFingerprint()).To(Equal("7b538a83a870c16d"))
})
It("reports the config inputs it hashes, so a change can be traced to a setting", func() {
conf.Server.Agents = "lastfm,spotify"
conf.Server.CoverArtPriority = "cover.*, embedded"
Expect(FingerprintInputs()).To(ContainElements(
FingerprintInput{Name: "Agents", Value: "lastfm,spotify"},
FingerprintInput{Name: "CoverArtPriority", Value: "cover.*, embedded"},
))
})
It("does not change when the server version changes", func() {
original := consts.Version
DeferCleanup(func() { consts.Version = original })
f1 := ConfigFingerprint()
consts.Version = original + "-next"
Expect(ConfigFingerprint()).To(Equal(f1),
"the version must not invalidate artwork state: every build would report a stale config")
f1 := Fingerprint()
conf.Server.EnableM3UExternalAlbumArt = true
Expect(Fingerprint()).NotTo(Equal(f1))
})
})
Describe("ReconcileConfigFingerprint", func() {
It("records the current fingerprint when none was ever stored", func() {
Expect(ReconcileConfigFingerprint(ctx, ds)).To(Succeed())
Describe("Backfill", func() {
It("enqueues nothing and returns false when the stored fingerprint matches", func() {
seedEntities()
Expect(propRepo.Put(FingerprintPropertyKey, Fingerprint())).To(Succeed())
Expect(propRepo.Get(consts.ArtConfFingerprintPropertyKey)).To(Equal(ConfigFingerprint()))
did, err := Backfill(ctx, ds)
Expect(err).ToNot(HaveOccurred())
Expect(did).To(BeFalse())
count, err := queueRepo.Count()
Expect(err).ToNot(HaveOccurred())
Expect(count).To(BeZero())
})
It("leaves a stale fingerprint stored, so the warning survives a restart", func() {
Expect(propRepo.Put(consts.ArtConfFingerprintPropertyKey, "stale-fingerprint")).To(Succeed())
It("runs the backfill when no fingerprint was ever stored", func() {
seedEntities()
Expect(ReconcileConfigFingerprint(ctx, ds)).To(Succeed())
did, err := Backfill(ctx, ds)
Expect(err).ToNot(HaveOccurred())
Expect(did).To(BeTrue())
Expect(propRepo.Get(consts.ArtConfFingerprintPropertyKey)).To(Equal("stale-fingerprint"))
count, err := queueRepo.Count()
Expect(err).ToNot(HaveOccurred())
Expect(count).To(Equal(int64(5))) // 2 artists + 1 album + 1 playlist + 1 radio
stored, err := propRepo.Get(FingerprintPropertyKey)
Expect(err).ToNot(HaveOccurred())
Expect(stored).To(Equal(Fingerprint()))
})
It("enqueues a private playlist by resolving it under an admin context", func() {
ds.MockedUser = adminUserRepo()
vds := &visibilityPlaylistDS{
MockDataStore: ds,
private: model.Playlist{ID: "plPrivate", OwnerID: "admin"},
tracks: &tests.MockPlaylistTrackRepo{},
}
did, err := Backfill(ctx, vds)
Expect(err).ToNot(HaveOccurred())
Expect(did).To(BeTrue())
Expect(findQueued(queueRepo.MockArtworkQueueRepo, "pl", "plPrivate")).ToNot(BeNil())
})
It("enqueues artists before albums/playlists/radios, all at Backfill priority", func() {
seedEntities()
Expect(propRepo.Put(FingerprintPropertyKey, "stale-fingerprint")).To(Succeed())
did, err := Backfill(ctx, ds)
Expect(err).ToNot(HaveOccurred())
Expect(did).To(BeTrue())
Expect(queueRepo.callKinds).ToNot(BeEmpty())
artistCallIdx := -1
for i, k := range queueRepo.callKinds {
if k == "ar" {
artistCallIdx = i
break
}
}
Expect(artistCallIdx).To(Equal(0), "artists must be the first Enqueue call")
for i, k := range queueRepo.callKinds {
if k != "ar" {
Expect(i).To(BeNumerically(">", artistCallIdx))
}
}
for _, it := range queueRepo.Data {
Expect(it.Priority).To(Equal(model.ArtworkPriorityBackfill))
Expect(it.ItemKind).To(BeElementOf("ar", "al", "pl", "ra"))
}
})
})
Describe("EnqueueMissingAll", func() {
Describe("EnqueueStaleAbsentAll", func() {
var artRepo *tests.MockArtworkRepo
BeforeEach(func() {
artRepo = tests.CreateMockArtworkRepo()
ds.MockedArtwork = artRepo
queueRepo.ItemArtworkSource = artRepo
queueRepo.ExistingIDs = map[string]map[string]bool{
"al": {"al1": true, "al2": true},
"ar": {"ar1": true},
"pl": {"pl1": true},
"ra": {"ra1": true},
}
})
It("enqueues only entities that have no item_artwork row, across all kinds", func() {
artRepo.ItemData["al-resolved"] = model.ItemArtwork{ItemKind: "al", ItemID: "al1", ImageType: model.ImageTypePrimary, Hash: "somehash"}
artRepo.ItemData["ar-absent"] = model.ItemArtwork{ItemKind: "ar", ItemID: "ar1", ImageType: model.ImageTypePrimary, Hash: ""}
It("enqueues only absent entries older than the recheck window, across all kinds", func() {
old := time.Now().Add(-48 * time.Hour)
recent := time.Now().Add(-time.Hour)
err := enqueueMissingAll(ctx, ds)
artRepo.ItemData["ar-stale"] = model.ItemArtwork{ItemKind: "ar", ItemID: "ar1", ImageType: model.ImageTypePrimary, Hash: "", AttemptedAt: old}
artRepo.ItemData["al-stale"] = model.ItemArtwork{ItemKind: "al", ItemID: "al1", ImageType: model.ImageTypePrimary, Hash: "", AttemptedAt: old}
artRepo.ItemData["pl-stale"] = model.ItemArtwork{ItemKind: "pl", ItemID: "pl1", ImageType: model.ImageTypePrimary, Hash: "", AttemptedAt: old}
artRepo.ItemData["ra-stale"] = model.ItemArtwork{ItemKind: "ra", ItemID: "ra1", ImageType: model.ImageTypePrimary, Hash: "", AttemptedAt: old}
// Not stale: too recent.
artRepo.ItemData["ar-recent"] = model.ItemArtwork{ItemKind: "ar", ItemID: "ar2", ImageType: model.ImageTypePrimary, Hash: "", AttemptedAt: recent}
// Not absent: has a resolved hash.
artRepo.ItemData["al-resolved"] = model.ItemArtwork{ItemKind: "al", ItemID: "al2", ImageType: model.ImageTypePrimary, Hash: "somehash", AttemptedAt: old}
err := EnqueueStaleAbsentAll(ctx, ds)
Expect(err).ToNot(HaveOccurred())
Expect(queueRepo.Data).To(HaveLen(4))
for _, it := range queueRepo.Data {
Expect(it.Priority).To(Equal(model.ArtworkPriorityRecheck))
}
Expect(findQueued(queueRepo, "al", "al2")).ToNot(BeNil())
Expect(findQueued(queueRepo, "pl", "pl1")).ToNot(BeNil())
Expect(findQueued(queueRepo, "ra", "ra1")).ToNot(BeNil())
Expect(findQueued(queueRepo, "al", "al1")).To(BeNil())
Expect(findQueued(queueRepo, "ar", "ar1")).To(BeNil())
})
})
})
var _ = Describe("ItemName", func() {
var ds *tests.MockDataStore
var ctx context.Context
BeforeEach(func() {
ctx = context.Background()
albumRepo := tests.CreateMockAlbumRepo()
albumRepo.SetData(model.Albums{
{ID: "al-1", Name: "Kid A"},
{ID: "al-2", Name: "Sandinista!", Discs: model.Discs{2: "Side Three"}},
})
ds = &tests.MockDataStore{MockedAlbum: albumRepo}
Expect(ds.Artist(ctx).(*tests.MockArtistRepo).Put(&model.Artist{ID: "ar-1", Name: "Radiohead"})).To(Succeed())
})
It("returns the album name", func() {
Expect(ItemName(ctx, ds, model.KindAlbumArtwork, "al-1")).To(Equal("Kid A"))
})
It("returns the artist name", func() {
Expect(ItemName(ctx, ds, model.KindArtistArtwork, "ar-1")).To(Equal("Radiohead"))
})
It("errors for an unknown album", func() {
_, err := ItemName(ctx, ds, model.KindAlbumArtwork, "nope")
Expect(err).To(MatchError(model.ErrNotFound))
})
It("errors for an unsupported kind", func() {
// model.Kind is a struct with unexported fields, so the zero value is the only
// unsupported Kind constructible from outside package model.
_, err := ItemName(ctx, ds, model.Kind{}, "al-1")
Expect(err).To(HaveOccurred())
})
Context("disc artwork", func() {
It("names the album, the disc and its subtitle", func() {
Expect(ItemName(ctx, ds, model.KindDiscArtwork, "al-2:2")).
To(Equal("Sandinista! (disc 2): Side Three"))
})
It("omits the subtitle when the disc has none", func() {
Expect(ItemName(ctx, ds, model.KindDiscArtwork, "al-2:1")).
To(Equal("Sandinista! (disc 1)"))
})
It("rejects an id that is not <albumID>:<disc>", func() {
_, err := ItemName(ctx, ds, model.KindDiscArtwork, "al-2")
Expect(err).To(HaveOccurred())
Expect(findQueued(queueRepo.MockArtworkQueueRepo, "ar", "ar1")).ToNot(BeNil())
Expect(findQueued(queueRepo.MockArtworkQueueRepo, "al", "al1")).ToNot(BeNil())
Expect(findQueued(queueRepo.MockArtworkQueueRepo, "pl", "pl1")).ToNot(BeNil())
Expect(findQueued(queueRepo.MockArtworkQueueRepo, "ra", "ra1")).ToNot(BeNil())
Expect(findQueued(queueRepo.MockArtworkQueueRepo, "ar", "ar2")).To(BeNil())
Expect(findQueued(queueRepo.MockArtworkQueueRepo, "al", "al2")).To(BeNil())
})
})
})
+5 -45
View File
@@ -1,13 +1,12 @@
package artwork
import (
"bytes"
"context"
"io"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/core/ffmpeg"
"github.com/navidrome/navidrome/utils/cache"
"github.com/navidrome/navidrome/utils/singleton"
)
@@ -16,7 +15,8 @@ import (
// produces the (possibly resized) bytes to store under Key.
type artworkReader interface {
cache.Item
Reader(ctx context.Context) (io.ReadCloser, error)
LastUpdated() time.Time
Reader(ctx context.Context) (io.ReadCloser, string, error)
}
type imageCache struct {
@@ -28,49 +28,9 @@ func GetImageCache() cache.FileCache {
return &imageCache{
FileCache: cache.NewFileCache("Image", conf.Server.ImageCacheSize, consts.ImageCacheDir, consts.DefaultImageCacheMaxItems,
func(ctx context.Context, arg cache.Item) (io.Reader, error) {
return arg.(artworkReader).Reader(ctx)
r, _, err := arg.(artworkReader).Reader(ctx)
return r, err
}),
}
})
}
// resizedItem is an artworkReader that resizes bytes opened by open() and caches the
// result under a hash-derived key.
type resizedItem struct {
hash string
size int
square bool
ffmpeg ffmpeg.FFmpeg
open func() (io.ReadCloser, error)
}
// Key is the ETag namespaced for the cache, so the validator a client holds and the entry it
// validates can never drift apart.
func (r *resizedItem) Key() string {
return "h-" + representationTag(r.hash, r.size, r.square)
}
func (r *resizedItem) Reader(ctx context.Context) (io.ReadCloser, error) {
orig, err := r.open()
if err != nil {
return nil, err
}
// An open() that reports "no image" as a nil reader would otherwise panic on the Close below.
if orig == nil {
return nil, ErrUnavailable
}
defer orig.Close()
data, err := readCapped(orig)
if err != nil {
return nil, err
}
resized, _, err := resizeImageData(ctx, r.ffmpeg, data, r.size, r.square)
if err != nil || resized == nil {
// Resize failed or image already within bounds: serve the original bytes.
return io.NopCloser(bytes.NewReader(data)), nil
}
if rc, ok := resized.(io.ReadCloser); ok {
return rc, nil
}
return io.NopCloser(resized), nil
}
-41
View File
@@ -1,41 +0,0 @@
package artwork
import (
"context"
"errors"
"io"
"strings"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("resizedItem", func() {
Describe("Reader", func() {
newItem := func(open func() (io.ReadCloser, error)) *resizedItem {
return &resizedItem{hash: "abc123", size: 300, open: open}
}
It("reports a nil reader as unavailable instead of panicking on it", func() {
// Every caller is expected to report "no image" as an error, but a nil reader reaches
// the deferred Close as a nil interface, which takes the whole request down.
_, err := newItem(func() (io.ReadCloser, error) { return nil, nil }).Reader(context.Background())
Expect(err).To(MatchError(ErrUnavailable))
})
It("propagates the open error", func() {
boom := errors.New("boom")
_, err := newItem(func() (io.ReadCloser, error) { return nil, boom }).Reader(context.Background())
Expect(err).To(MatchError(boom))
})
It("serves the original bytes when they cannot be resized", func() {
rc, err := newItem(func() (io.ReadCloser, error) {
return io.NopCloser(strings.NewReader("not an image")), nil
}).Reader(context.Background())
Expect(err).ToNot(HaveOccurred())
defer rc.Close()
Expect(io.ReadAll(rc)).To(Equal([]byte("not an image")))
})
})
})
+48 -31
View File
@@ -1,7 +1,6 @@
package artwork
import (
"context"
"errors"
"fmt"
"io"
@@ -13,11 +12,19 @@ import (
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/log"
"github.com/zeebo/xxh3"
)
// ImageStore is the content-addressed store for artwork images with no library file backing them.
func HashImage(r io.Reader) (string, error) {
d := xxh3.New()
if _, err := io.Copy(d, r); err != nil {
return "", err
}
return fmt.Sprintf("%016x", d.Sum64()), nil
}
// ImageStore is the content-addressed store for artwork images that have no
// library file backing them (external downloads, embedded extractions, generated).
type ImageStore struct {
root string
}
@@ -26,11 +33,14 @@ func NewImageStore(rootDir string) *ImageStore {
return &ImageStore{root: rootDir}
}
func GetImageStore() *ImageStore {
return NewImageStore(filepath.Join(conf.Server.DataFolder.String(), consts.ArtworkFolder, consts.HashedArtworkFolder))
// ProvideImageStore roots the store in its own subtree under the data folder, so
// Prune's recursive sweep never reaches the per-entity upload folders next to it.
func ProvideImageStore() *ImageStore {
return NewImageStore(filepath.Join(conf.Server.DataFolder.String(), consts.ArtworkFolder, "store"))
}
// extForMime must stay stable across OSes: extensions are baked into stored paths and re-derived on Open.
// extForMime is deliberately NOT mime.ExtensionsByType: extensions are baked into
// content-addressed paths and re-derived on Open, so they must be stable across OSes.
func extForMime(m string) string {
switch m {
case "image/jpeg":
@@ -45,15 +55,8 @@ func extForMime(m string) string {
return ".img"
}
func hashImage(r io.Reader) (string, error) {
d := xxh3.New()
if _, err := io.Copy(d, r); err != nil {
return "", err
}
return fmt.Sprintf("%016x", d.Sum64()), nil
}
// validHash guards path sharding: a malformed hash would slice-panic or inject path separators.
// validHash rejects anything but 16 lowercase hex chars: known-absent states carry "",
// and malformed persisted hashes must never reach path sharding (slice panics, separators).
func validHash(hash string) bool {
if len(hash) != 16 {
return false
@@ -81,7 +84,7 @@ func (s *ImageStore) Write(hash, mimeType string, r io.Reader) error {
if err := os.Chtimes(dst, now, now); err == nil {
return nil
}
// touch failed (likely pruned concurrently) — fall through and rewrite it
// touch failed (file likely pruned concurrently) — fall through and write it
}
if err := os.MkdirAll(filepath.Dir(dst), 0755); err != nil {
return err
@@ -108,18 +111,38 @@ func (s *ImageStore) Open(hash, mimeType string) (io.ReadCloser, error) {
return os.Open(s.path(hash, mimeType))
}
// Sweep removes store files not accepted by keep. Files modified after cutoff are always
// kept: their acquisition row may not be committed yet.
func (s *ImageStore) Sweep(ctx context.Context, cutoff time.Time, keep func(hash, ext string) bool) (int, error) {
removed, failed := 0, 0
var lastErr error
// Remove deletes the store file unless it is newer than olderThan, in which case
// an overlapping acquisition may have just touched it and be about to commit its row.
func (s *ImageStore) Remove(hash, mimeType string, olderThan time.Time) error {
if !validHash(hash) {
return fmt.Errorf("imagestore: invalid hash %q", hash)
}
path := s.path(hash, mimeType)
info, err := os.Stat(path)
if errors.Is(err, fs.ErrNotExist) {
return nil
}
if err != nil {
return err
}
if info.ModTime().After(olderThan) {
return nil
}
err = os.Remove(path)
if errors.Is(err, fs.ErrNotExist) {
return nil
}
return err
}
// Sweep removes store files not accepted by keep. Files modified after cutoff
// (including temp files) are always kept: their acquisition row may not be committed yet.
func (s *ImageStore) Sweep(cutoff time.Time, keep func(hash, ext string) bool) (int, error) {
removed := 0
err := filepath.WalkDir(s.root, func(path string, d fs.DirEntry, err error) error {
if err != nil || d.IsDir() {
return err
}
if err := ctx.Err(); err != nil {
return err
}
info, err := d.Info()
if err != nil {
return err
@@ -136,18 +159,12 @@ func (s *ImageStore) Sweep(ctx context.Context, cutoff time.Time, keep func(hash
if remove {
// #nosec G122 -- path comes from WalkDir over our own store root, no attacker-controlled symlinks
if err := os.Remove(path); err != nil {
// One unremovable file must not strand the rest of the store until the next prune.
failed, lastErr = failed+1, err
return nil //nolint:nilerr // counted and reported in aggregate below
return err
}
removed++
}
return nil
})
// Aggregated: a store that has gone read-only would otherwise warn once per file, every prune.
if failed > 0 {
log.Warn(ctx, "Artwork: Could not remove store files", "count", failed, "swept", removed, lastErr)
}
if errors.Is(err, fs.ErrNotExist) {
return removed, nil
}
+42 -66
View File
@@ -2,13 +2,11 @@ package artwork
import (
"bytes"
"context"
"io"
"os"
"path/filepath"
"time"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
@@ -16,27 +14,25 @@ import (
var _ = Describe("ImageStore", func() {
var store *ImageStore
var root string
var ctx context.Context
BeforeEach(func() {
root = GinkgoT().TempDir()
store = NewImageStore(root)
ctx = context.Background()
})
It("hashes deterministically", func() {
h1, err := hashImage(bytes.NewReader([]byte("some image bytes")))
h1, err := HashImage(bytes.NewReader([]byte("some image bytes")))
Expect(err).ToNot(HaveOccurred())
h2, _ := hashImage(bytes.NewReader([]byte("some image bytes")))
h2, _ := HashImage(bytes.NewReader([]byte("some image bytes")))
Expect(h1).To(Equal(h2))
Expect(h1).To(HaveLen(16))
h3, _ := hashImage(bytes.NewReader([]byte("other bytes")))
h3, _ := HashImage(bytes.NewReader([]byte("other bytes")))
Expect(h3).ToNot(Equal(h1))
})
It("writes sharded and reads back", func() {
data := []byte("jpeg-bytes")
h, _ := hashImage(bytes.NewReader(data))
h, _ := HashImage(bytes.NewReader(data))
Expect(store.Write(h, "image/jpeg", bytes.NewReader(data))).To(Succeed())
Expect(filepath.Join(root, h[0:2], h[2:4], h+".jpg")).To(BeAnExistingFile())
@@ -50,7 +46,7 @@ var _ = Describe("ImageStore", func() {
It("is idempotent on duplicate writes and preserves the original content", func() {
data := []byte("dup")
h, _ := hashImage(bytes.NewReader(data))
h, _ := HashImage(bytes.NewReader(data))
Expect(store.Write(h, "image/png", bytes.NewReader(data))).To(Succeed())
// A duplicate write only touches mtime; passing different bytes under the same
// hash proves the second reader is never consumed to overwrite the file.
@@ -66,7 +62,7 @@ var _ = Describe("ImageStore", func() {
It("refreshes the mtime on a duplicate write", func() {
data := []byte("touch-me")
h, _ := hashImage(bytes.NewReader(data))
h, _ := HashImage(bytes.NewReader(data))
Expect(store.Write(h, "image/png", bytes.NewReader(data))).To(Succeed())
old := time.Now().Add(-2 * time.Hour)
Expect(os.Chtimes(store.path(h, "image/png"), old, old)).To(Succeed())
@@ -80,7 +76,7 @@ var _ = Describe("ImageStore", func() {
It("rewrites the bytes when the existing file vanished before the liveness touch", func() {
data := []byte("vanishing")
h, _ := hashImage(bytes.NewReader(data))
h, _ := HashImage(bytes.NewReader(data))
for range 10 {
Expect(store.Write(h, "image/png", bytes.NewReader(data))).To(Succeed())
Expect(os.Remove(store.path(h, "image/png"))).To(Succeed())
@@ -99,26 +95,53 @@ var _ = Describe("ImageStore", func() {
Expect(os.IsNotExist(err)).To(BeTrue())
})
It("removes without error when already gone", func() {
Expect(store.Remove("beefbeefbeefbeef", "image/jpeg", time.Now())).To(Succeed())
})
It("rejects invalid hashes instead of panicking", func() {
for _, h := range []string{"", "ab", "BEEFBEEFBEEFBEEF", "../../../../etcpw", "beefbeefbeefbee/"} {
Expect(store.Write(h, "image/jpeg", bytes.NewReader([]byte("x")))).To(MatchError(ContainSubstring("invalid hash")))
_, err := store.Open(h, "image/jpeg")
Expect(err).To(MatchError(ContainSubstring("invalid hash")))
Expect(store.Remove(h, "image/jpeg", time.Now())).To(MatchError(ContainSubstring("invalid hash")))
}
})
It("spares a file newer than the cutoff, removes an aged one", func() {
fresh := []byte("fresh")
hf, _ := HashImage(bytes.NewReader(fresh))
Expect(store.Write(hf, "image/jpeg", bytes.NewReader(fresh))).To(Succeed())
aged := []byte("aged")
ha, _ := HashImage(bytes.NewReader(aged))
Expect(store.Write(ha, "image/jpeg", bytes.NewReader(aged))).To(Succeed())
old := time.Now().Add(-2 * time.Hour)
Expect(os.Chtimes(store.path(ha, "image/jpeg"), old, old)).To(Succeed())
cutoff := time.Now().Add(-time.Hour)
Expect(store.Remove(hf, "image/jpeg", cutoff)).To(Succeed())
Expect(store.Remove(ha, "image/jpeg", cutoff)).To(Succeed())
rc, err := store.Open(hf, "image/jpeg")
Expect(err).ToNot(HaveOccurred())
rc.Close()
_, err = store.Open(ha, "image/jpeg")
Expect(os.IsNotExist(err)).To(BeTrue())
})
It("sweeps unknown files, keeps known ones", func() {
d1 := []byte("keep-me")
h1, _ := hashImage(bytes.NewReader(d1))
h1, _ := HashImage(bytes.NewReader(d1))
Expect(store.Write(h1, "image/jpeg", bytes.NewReader(d1))).To(Succeed())
d2 := []byte("orphan")
h2, _ := hashImage(bytes.NewReader(d2))
h2, _ := HashImage(bytes.NewReader(d2))
Expect(store.Write(h2, "image/jpeg", bytes.NewReader(d2))).To(Succeed())
old := time.Now().Add(-2 * time.Hour)
Expect(os.Chtimes(store.path(h2, "image/jpeg"), old, old)).To(Succeed())
removed, err := store.Sweep(ctx, time.Now().Add(-time.Hour), func(h, _ string) bool { return h == h1 })
removed, err := store.Sweep(time.Now().Add(-time.Hour), func(h, _ string) bool { return h == h1 })
Expect(err).ToNot(HaveOccurred())
Expect(removed).To(Equal(1))
_, err = store.Open(h2, "image/jpeg")
@@ -130,7 +153,7 @@ var _ = Describe("ImageStore", func() {
It("sweeps a stale mime variant of a known hash, keeps the current one", func() {
data := []byte("same-bytes")
h, _ := hashImage(bytes.NewReader(data))
h, _ := HashImage(bytes.NewReader(data))
Expect(store.Write(h, "image/png", bytes.NewReader(data))).To(Succeed())
Expect(store.Write(h, "image/jpeg", bytes.NewReader(data))).To(Succeed())
old := time.Now().Add(-2 * time.Hour)
@@ -138,7 +161,7 @@ var _ = Describe("ImageStore", func() {
Expect(os.Chtimes(store.path(h, "image/jpeg"), old, old)).To(Succeed())
// The recorded mime is image/jpeg, so the .png variant is obsolete.
removed, err := store.Sweep(ctx, time.Now().Add(-time.Hour), func(hash, ext string) bool {
removed, err := store.Sweep(time.Now().Add(-time.Hour), func(hash, ext string) bool {
return hash == h && ext == ".jpg"
})
Expect(err).ToNot(HaveOccurred())
@@ -152,10 +175,10 @@ var _ = Describe("ImageStore", func() {
It("keeps young unknown files inside the grace window", func() {
d := []byte("fresh-orphan")
h, _ := hashImage(bytes.NewReader(d))
h, _ := HashImage(bytes.NewReader(d))
Expect(store.Write(h, "image/jpeg", bytes.NewReader(d))).To(Succeed())
removed, err := store.Sweep(ctx, time.Now().Add(-time.Hour), func(string, string) bool { return false })
removed, err := store.Sweep(time.Now().Add(-time.Hour), func(string, string) bool { return false })
Expect(err).ToNot(HaveOccurred())
Expect(removed).To(Equal(0))
rc, err := store.Open(h, "image/jpeg")
@@ -172,57 +195,10 @@ var _ = Describe("ImageStore", func() {
freshTmp := filepath.Join(root, ".fresh.tmp")
Expect(os.WriteFile(freshTmp, []byte("y"), 0600)).To(Succeed())
removed, err := store.Sweep(ctx, time.Now().Add(-time.Hour), func(string, string) bool { return true })
removed, err := store.Sweep(time.Now().Add(-time.Hour), func(string, string) bool { return true })
Expect(err).ToNot(HaveOccurred())
Expect(removed).To(Equal(1))
Expect(oldTmp).ToNot(BeAnExistingFile())
Expect(freshTmp).To(BeAnExistingFile())
})
It("keeps sweeping past a file it cannot remove", func() {
tests.SkipOnWindows("uses Unix file permission bits")
if os.Geteuid() == 0 {
Skip("read-only dir cannot block root (e.g. tests in a container)")
}
old := time.Now().Add(-2 * time.Hour)
// "blocked" sorts before "ok", so the walk hits the unremovable file first.
blockedDir := filepath.Join(root, "blocked")
Expect(os.MkdirAll(blockedDir, 0755)).To(Succeed())
blocked := filepath.Join(blockedDir, "a.jpg")
Expect(os.WriteFile(blocked, []byte("x"), 0600)).To(Succeed())
Expect(os.Chtimes(blocked, old, old)).To(Succeed())
okDir := filepath.Join(root, "ok")
Expect(os.MkdirAll(okDir, 0755)).To(Succeed())
reachable := filepath.Join(okDir, "b.jpg")
Expect(os.WriteFile(reachable, []byte("y"), 0600)).To(Succeed())
Expect(os.Chtimes(reachable, old, old)).To(Succeed())
Expect(os.Chmod(blockedDir, 0500)).To(Succeed())
DeferCleanup(func() { _ = os.Chmod(blockedDir, 0755) })
removed, err := store.Sweep(ctx, time.Now().Add(-time.Hour), func(string, string) bool { return false })
Expect(err).ToNot(HaveOccurred())
Expect(removed).To(Equal(1))
Expect(blocked).To(BeAnExistingFile())
Expect(reachable).ToNot(BeAnExistingFile())
})
// Prune holds the worker's write lock for the whole sweep, and shutdown waits on the
// worker, so an uncancellable walk over a large store stalls it until SIGKILL.
It("abandons the walk when the context is cancelled", func() {
old := time.Now().Add(-2 * time.Hour)
for _, name := range []string{"a", "b", "c", "d"} {
p := filepath.Join(root, name+".jpg")
Expect(os.WriteFile(p, []byte("x"), 0600)).To(Succeed())
Expect(os.Chtimes(p, old, old)).To(Succeed())
}
cancelCtx, cancel := context.WithCancel(ctx)
cancel()
_, err := store.Sweep(cancelCtx, time.Now().Add(-time.Hour), func(string, string) bool { return false })
Expect(err).To(MatchError(context.Canceled))
matches, _ := filepath.Glob(filepath.Join(root, "*.jpg"))
Expect(matches).To(HaveLen(4), "a cancelled sweep must not keep deleting")
})
})
+10 -40
View File
@@ -1,25 +1,23 @@
package artwork
import (
"bytes"
"context"
"image"
"image/draw"
"image/png"
"io"
"os"
"path/filepath"
"strings"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils"
xdraw "golang.org/x/image/draw"
)
const tileSize = 600
// findPlaylistSidecarPath finds an image beside plsPath with the same base name (case-insensitive).
// findPlaylistSidecarPath scans the directory of the playlist file for a sidecar
// image file with the same base name (case-insensitive). Returns empty string if
// no matching image is found or if plsPath is empty.
func findPlaylistSidecarPath(ctx context.Context, plsPath string) string {
if plsPath == "" {
return ""
@@ -29,12 +27,12 @@ func findPlaylistSidecarPath(ctx context.Context, plsPath string) string {
entries, err := os.ReadDir(dir)
if err != nil {
log.Warn(ctx, "Artwork: Could not read directory for playlist sidecar", "dir", dir, err)
log.Warn(ctx, "Could not read directory for playlist sidecar", "dir", dir, err)
return ""
}
for _, entry := range entries {
name := entry.Name()
nameBase := utils.BaseName(name)
nameBase := strings.TrimSuffix(name, filepath.Ext(name))
if !entry.IsDir() && strings.EqualFold(nameBase, base) && model.IsImageFile(name) {
return filepath.Join(dir, name)
}
@@ -58,21 +56,25 @@ func rect(pos int) image.Rectangle {
return r
}
// fillCenter center-crops src and scales it to fill dstW x dstH exactly.
// fillCenter crops the source image from the center and scales it to fill dstW x dstH exactly,
// equivalent to imaging.Fill with Center anchor.
func fillCenter(src image.Image, dstW, dstH int) image.Image {
srcBounds := src.Bounds()
srcW := srcBounds.Dx()
srcH := srcBounds.Dy()
// Calculate crop rectangle (center crop to match destination aspect ratio)
srcAspect := float64(srcW) / float64(srcH)
dstAspect := float64(dstW) / float64(dstH)
var cropRect image.Rectangle
if srcAspect > dstAspect {
// Source is wider — crop horizontally
cropW := int(float64(srcH) * dstAspect)
cropX := (srcW - cropW) / 2
cropRect = image.Rect(srcBounds.Min.X+cropX, srcBounds.Min.Y, srcBounds.Min.X+cropX+cropW, srcBounds.Max.Y)
} else {
// Source is taller — crop vertically
cropH := int(float64(srcW) / dstAspect)
cropY := (srcH - cropH) / 2
cropRect = image.Rect(srcBounds.Min.X, srcBounds.Min.Y+cropY, srcBounds.Max.X, srcBounds.Min.Y+cropY+cropH)
@@ -82,35 +84,3 @@ func fillCenter(src image.Image, dstW, dstH int) image.Image {
xdraw.CatmullRom.Scale(dst, dst.Bounds(), src, cropRect, draw.Src, nil)
return dst
}
// decodeTile runs before the processor's size guards apply, so it enforces the caps itself.
func decodeTile(r io.ReadCloser) (image.Image, error) {
data, err := readCapped(r)
if err != nil {
return nil, err
}
img, _, err := decodeCapped(data)
if err != nil {
return nil, err
}
return fillCenter(img, tileSize/2, tileSize/2), nil
}
func assembleTiles(tiles []image.Image) (io.ReadCloser, error) {
buf := new(bytes.Buffer)
var err error
if len(tiles) == 4 {
rgba := image.NewRGBA(image.Rectangle{Max: image.Point{X: tileSize - 1, Y: tileSize - 1}})
draw.Draw(rgba, rect(0), tiles[0], image.Point{}, draw.Src)
draw.Draw(rgba, rect(1), tiles[1], image.Point{}, draw.Src)
draw.Draw(rgba, rect(2), tiles[2], image.Point{}, draw.Src)
draw.Draw(rgba, rect(3), tiles[3], image.Point{}, draw.Src)
err = png.Encode(buf, rgba)
} else {
err = png.Encode(buf, tiles[0])
}
if err != nil {
return nil, err
}
return io.NopCloser(buf), nil
}
Loaded 100 of 627 files, more files were not shown because too many files have changed in this diff. Show more