Compare commits

..

138 Commits

Author SHA1 Message Date
Deluan
c7d7ec306e fix(mcp-agent): stream native mcp-server stderr to logs
The previous implementation buffered stderr from the native mcp-server process and only logged the full buffer content when the process exited. This prevented real-time viewing of logs from the server.

This change modifies the native process startup logic (`startProcess_locked`) to use `cmd.StderrPipe()` instead of assigning `cmd.Stderr` to a buffer. A separate goroutine is now launched within the process monitoring goroutine. This new goroutine uses a `bufio.Scanner` to continuously read lines from the stderr pipe and logs them using the Navidrome logger (`log.Info`) with an `[MCP-SERVER]` prefix.

This ensures logs from the native mcp-server appear in Navidrome's logs immediately as they are written.

(Note: Also includes update to McpServerPath constant to point to the native binary.)

Signed-off-by: Deluan <deluan@navidrome.org>
2025-04-19 20:53:10 -04:00
Deluan
bcc3643c81 chore: add debug logging to mcp-server
Added debug logging throughout the mcp-server components using the standard Go `log` package. All log messages are prefixed with `[MCP]` for easy identification.

This includes logging in: main function (startup, CLI execution, registration, serve loop), Tool handlers, Native and WASM fetcher implementations, Wikipedia, Wikidata, and DBpedia data fetching functions

Replaced previous `println` statements with `log.Println` or `log.Printf`.
2025-04-19 19:37:32 -04:00
Deluan
97b101685e perf: pre-compile WASM module for MCP agent
Modified the MCP agent constructor to pre-compile the WASM module when detected. This shifts the costly compilation step out of the first API request path.

The `MCPWasm` implementation now stores the `wazero.CompiledModule` provided by the constructor and uses it directly for instantiation via `runtime.InstantiateModule()` when the agent is first used or restarted. This significantly speeds up the initialization during the first request.

Updated tests and cleanup logic to accommodate the shared nature of the pre-compiled module.
2025-04-19 19:23:23 -04:00
Deluan
8660cb4fff refactor: centralize MCP agent method logic and cleanup comments
Centralized the argument preparation for GetArtistBiography and GetArtistURL within the main MCPAgent struct. Added callMCPTool to the mcpImplementation interface and removed the redundant GetArtist* methods from the native and WASM implementations.

Removed the embedded agents.Artist*Retriever interfaces from mcpImplementation as MCPAgent now directly fulfills these.

Also removed various redundant comments and leftover commented-out code from the agent, implementation, and test files.
2025-04-19 19:11:34 -04:00
Deluan
ae93e555c9 feat: refactor MCP agent to support native and WASM implementations
Refactored the MCPAgent to delegate core functionality to separate implementations for native process and WASM execution.

Introduced an `mcpImplementation` interface (`MCPNative`, `MCPWasm`) to abstract the underlying execution method. The main `MCPAgent` now holds an instance of this interface, selected by the `mcpConstructor` based on the `McpServerPath` (native executable or `.wasm` file).

Shared Wazero resources (runtime, cache, WASI, host functions) are now initialized once in the constructor and passed to the `MCPWasm` implementation, improving resource management and potentially startup performance for WASM modules.

Updated tests (`mcp_agent_test.go`) to use a new testing constructor (`NewAgentForTesting`) which injects a mock client into the appropriate implementation. Assertions were adjusted to reflect the refactored error handling and structure. Also updated the `McpServerPath` to use a relative path.
2025-04-19 19:05:15 -04:00
Deluan
2f71516dde fix: update MCP server path for agent initialization
Change the MCP server path in MCPAgent to point to the correct relative directory for the WASM file. This adjustment ensures proper initialization and access to the server resources, aligning with recent enhancements in the MCPAgent's handling of server types.
2025-04-19 18:43:50 -04:00
Deluan
73da7550d6 refactor: separate native and WASM process handling in MCPAgent
- Moved the native process handling logic from mcp_agent.go to a new file mcp_process_native.go for better organization.
- Introduced a new file mcp_host_functions.go to define and register host functions for WASM modules.
- Updated MCPAgent to ensure proper initialization and cleanup of both native and WASM processes, enhancing code clarity and maintainability.
- Added comments to clarify the purpose of changes and ensure future developers understand the structure.
2025-04-19 15:22:15 -04:00
Deluan
674129a34b fix: update MCP server path for agent initialization
Change the MCP server path in MCPAgent from a WASM file to a directory. This adjustment aligns with recent enhancements to the MCPAgent's handling of server types, ensuring proper initialization and access to the server resources.
2025-04-19 14:48:18 -04:00
Deluan
fb0714562d feat: grant filesystem access for WASM modules in MCPAgent
Enhance the MCPAgent's WASM module initialization by granting access to the host filesystem. This is necessary for DNS lookups and other operations that may depend on filesystem access. Added comments to highlight the security implications of this change and the need for potential restrictions in the future.
2025-04-19 14:46:54 -04:00
Deluan
6b89f7ab63 feat: integrate Wazero for WASM support in MCPAgent
Enhance MCPAgent to support both native executables and WASM modules using Wazero. This includes:
- Adding Wazero dependencies in go.mod and go.sum.
- Modifying MCPAgent to initialize a shared Wazero runtime and compile/load WASM modules.
- Implementing cleanup logic for WASM resources.
- Updating the process initialization to handle both native and WASM paths.

This change improves the flexibility of the MCPAgent in handling different server types.
2025-04-19 14:28:55 -04:00
Deluan
c548168503 fix: handle error messages returned in MCP tool response content
Modify callMCPTool helper to check if the text content returned by\nthe MCP server starts with 'handler returned an error:'.\n\nIf it does, log a warning and return agents.ErrNotFound, treating\nthe error signaled by the external tool as if the data was not found.\nAdded test cases to verify this behavior.
2025-04-19 14:01:51 -04:00
Deluan
8ebefe4065 refactor: DRY up MCPAgent implementation
Refactor the MCPAgent to reduce code duplication.\n\n- Consolidate GetArtistBiographyArgs and GetArtistURLArgs into a single\n  ArtistArgs struct.\n- Extract common logic (initialization check, locking, tool calling,\n  error handling, response validation) into a private callMCPTool helper method.\n- Simplify GetArtistBiography and GetArtistURL to delegate to callMCPTool.\n- Update tests to use the consolidated ArtistArgs struct.\n- Correct mutex locking in ensureClientInitialized to prevent race conditions.
2025-04-19 13:04:41 -04:00
Deluan
6e59060a01 fix: prevent double initialization of agents
Modify the createAgents function to call each agent constructor only once.\n\nPreviously, the constructor (init(ds)) was called first to check if the\nagent could be initialized, and then called again when appending the agent\nto the result slice. This caused unnecessary work and duplicate log messages.\n\nThe fix stores the result of the first call in the 'agent' variable and\nreuses this instance when appending, ensuring each constructor runs only once.
2025-04-19 13:01:41 -04:00
Deluan
be9e10db37 test: add mock-based tests for MCPAgent
Implement unit tests for MCPAgent using a mocked MCP client.\n\n- Define an mcpClient interface and a mock implementation in the test file.\n- Modify MCPAgent to use the interface and add an exported ClientOverride field\n  to allow injecting the mock during tests.\n- Export necessary constants and argument structs from the agent package.\n- Add test cases covering success, tool errors, empty responses, and pipe errors\n  for both GetArtistBiography and GetArtistURL.\n- Fix agent logic to handle empty TextContent in responses correctly.\n- Remove previous placeholder tests and unreliable initialization test.
2025-04-19 12:56:22 -04:00
Deluan
9c20520d59 feat: implement GetArtistURL in MCP agent
Add support for retrieving artist URLs via the MCP agent.\n\n- Implement the agents.ArtistURLRetriever interface.\n- Add the GetArtistURL method, which calls the 'get_artist_url'\n  tool on the external MCP server.\n- Define the necessary constants and argument struct.\n\nThis mirrors the existing implementation for GetArtistBiography, reusing\nthe persistent process and client logic.
2025-04-19 12:48:27 -04:00
Deluan
8b754a7c73 feat: implement MCP agent process auto-restart
Modify the MCPAgent to automatically attempt restarting the external\nserver process if it detects the process has exited.\n\n- Replaced sync.Once with mutex-protected checks (a.client == nil) to allow\n  re-initialization.\n- The monitoring goroutine now cleans up agent state (nils client, cmd, stdin)\n  upon process exit, signaling the need for restart.\n- ensureClientInitialized now attempts to start/initialize if the client is nil.\n- GetArtistBiography checks client validity again after locking to handle race\n  conditions where the process might die just after initialization check.
2025-04-19 12:45:41 -04:00
Deluan
8326a20eda refactor: keep MCP agent server process running
Refactor MCPAgent to maintain a persistent external server process.\n\nInstead of starting and stopping the MCP server for each request, the agent\nnow uses sync.Once for lazy initialization on the first call.\nIt stores the running exec.Cmd, stdio pipes, and the mcp.Client\nin the agent struct.\n\nA sync.Mutex protects concurrent access to the client/pipes.\nA goroutine monitors the process using cmd.Wait() and logs if it exits\nunexpectedly.\n\nThis avoids the overhead of process creation/destruction on every\nmetadata retrieval request.
2025-04-19 12:04:39 -04:00
Deluan
51567a0bdf feat: add proof-of-concept MCP agent
Add a new agent implementation MCPAgent in core/agents/mcp.\n\nThis agent interacts with an external Model Context Protocol (MCP) server\nusing the github.com/metoro-io/mcp-golang library via stdio.\n\nIt currently implements the ArtistBiographyRetriever interface to fetch\nbiographies by calling the 'get_artist_biography' tool on the MCP server.\nThe path to the MCP server executable is hardcoded for this PoC.\n\nIncludes basic Ginkgo test setup for the new agent.
2025-04-19 11:59:34 -04:00
Deluan
4944f8035a test: add tests for userName and AbsolutePath in core/common.go
Added Ginkgo/Gomega tests for userName and AbsolutePath functions in core/common.go. Tests cover normal and error cases, using existing mocks and helpers. This improves coverage and ensures correct behavior for user context extraction and library path resolution.
2025-04-18 11:53:47 -04:00
Ivan Pešić
0d5097d888 fix(ui): update Serbian translation (#3941) 2025-04-17 19:27:12 -04:00
Deluan Quintão
ed7ee3d9f8 fix(ui): always order album tracks by disc and track number (fixes #3720) (#3975)
* fix(ui): ensure album tracks are always ordered by disc and track number (fixes #3720)

* refactor(ui): remove obsolete release date grouping logic from SongDatagrid and AlbumSongs

* fix(ui): ensure correct album track ordering in context menu and play button

* fix: Update album sort to use album_id instead of release_date

* refactor: Adjust filters in PlayButton and AlbumContextMenu

* fix: correct typo in comment regarding participants in GetMissingAndMatching function

* fix: prevent visual separation of tracks on same disc

Removes the leftover `releaseDate` check from the `firstTracksOfDiscs` calculation in `SongDatagridBody`. This check caused unnecessary `DiscSubtitleRow` insertions when tracks on the same disc had different release dates, leading to an incorrect visual grouping that resembled a multi-disc layout.

This change ensures disc subtitles are only shown when the actual `discNumber` changes, correcting the UI presentation issue reported in issue #3720 after PR #3975.

* fix: remove remaining releaseDate references in SongDatagrid

Cleaned up leftover `releaseDate` references in `SongDatagrid.jsx`:

- Removed `releaseDate` parameter and usage from `handlePlaySubset` in `DiscSubtitleRow`.

- Removed `releaseDate` prop passed to `AlbumContextMenu` in `DiscSubtitleRow`.

- Removed `releaseDate` from the drag item data in `SongDatagridRow`.

- Removed `releaseDate` parameter and the corresponding `else` block from the `playSubset` function in `SongDatagridBody`.

This ensures the component consistently uses `discNumber` for grouping and playback actions initiated from the disc subtitle, fully resolving the inconsistencies related to issue #3720.
2025-04-17 19:23:53 -04:00
Deluan Quintão
74803bb43e fix(ui): update Russian, Turkish translations from POEditor (#3971)
Co-authored-by: navidrome-bot <navidrome-bot@navidrome.org>
2025-04-16 21:09:50 -04:00
marcbres
0159cf73e2 fix(ui): updated Catalan translations (#3973)
Co-authored-by: Marc Bres Gil <marc@helm>
2025-04-16 21:05:59 -04:00
Dongeun
ac1d51f9d0 fix(ui): update Chinese (Simplified) translations (#3938) 2025-04-16 21:05:26 -04:00
Thomas Johansen
91eb661db5 fix(ui): update Norwegian translation #3943 2025-04-16 21:04:10 -04:00
Guilherme Souza
524d508916 feat(ui): show sampleRate in song info dialog (#3960)
* feat(ui): show sampleRate in song info dialog

* npm run prettier --write
2025-04-12 20:52:47 -04:00
Deluan
a6f1f7b7e3 fix(server): albumtype in Smart Playlists
Signed-off-by: Deluan <deluan@navidrome.org>
2025-04-11 23:53:16 -04:00
Deluan Quintão
49b8cfc261 fix(artwork): always select the coverArt of the first disc in multi-disc albums (#3950)
* feat(artwork): sort image files for consistent cover art selection

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(artwork): improve album cover art selection by considering disc numbers

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-04-11 23:39:57 -04:00
Deluan
bcea8b832a chore(deps): update Go version to 1.24.2 in go.mod 2025-04-11 23:18:00 -04:00
Deluan Quintão
58367afaea refactor: external_metadata -> external.Provider (#3903)
* tests for TopSongs

Signed-off-by: Deluan <deluan@navidrome.org>

* convert to Ginkgo

Signed-off-by: Deluan <deluan@navidrome.org>

* consolidate tests

Signed-off-by: Deluan <deluan@navidrome.org>

* rename external metadata -wip

Signed-off-by: Deluan <deluan@navidrome.org>

* rename external metadata to extdata.Provider

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor tests - wip

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor test helpers

Signed-off-by: Deluan <deluan@navidrome.org>

* remove reflection

Signed-off-by: Deluan <deluan@navidrome.org>

* use mock.Mock

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor

Signed-off-by: Deluan <deluan@navidrome.org>

* fix

Signed-off-by: Deluan <deluan@navidrome.org>

* receive Agents interface in Provider constructor

Signed-off-by: Deluan <deluan@navidrome.org>

* use mock for Agents

Signed-off-by: Deluan <deluan@navidrome.org>

* tests for SimilarSongs

Signed-off-by: Deluan <deluan@navidrome.org>

* remove duplication

Signed-off-by: Deluan <deluan@navidrome.org>

* ArtistImage tests

Signed-off-by: Deluan <deluan@navidrome.org>

* AlbumImage tests

Signed-off-by: Deluan <deluan@navidrome.org>

* fix provider error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* UpdateAlbumInfo tests - wip

Signed-off-by: Deluan <deluan@navidrome.org>

* UpdateAlbumInfo tests - wip

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor

Signed-off-by: Deluan <deluan@navidrome.org>

* UpdateArtistInfo tests - wip

Signed-off-by: Deluan <deluan@navidrome.org>

* clean up

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor

Signed-off-by: Deluan <deluan@navidrome.org>

* fix test descriptions

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename extdata package to external

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-04-08 21:11:09 -04:00
Deluan
6b59f5f73a feat(ui): add genre and mood fields to AlbumSongs component
Signed-off-by: Deluan <deluan@navidrome.org>
2025-04-08 18:13:37 -04:00
Deluan Quintão
5f0c1e7387 chore(deps) upgrade Go dependencies, including golangci-lint (#3937)
* chore(deps): update Go dependencies

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(deps): upgrade golangci-lint

Signed-off-by: Deluan <deluan@navidrome.org>

* build: upgrade golangci-lint-action to v7

Signed-off-by: Deluan <deluan@navidrome.org>

* go mod tidy

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-04-07 19:42:00 -03:00
Deluan Quintão
a057a680f1 fix(ui): update Greek, Esperanto, Polish, Russian, Turkish translations from POEditor (#3894)
Co-authored-by: navidrome-bot <navidrome-bot@navidrome.org>
2025-04-05 08:54:29 -03:00
Deluan Quintão
f9081bbe6b fix(server): first user created should be admin, when using reverse proxy (#3920)
Fix #3902

Signed-off-by: Deluan <deluan@navidrome.org>
2025-04-05 08:24:14 -03:00
Deluan Quintão
73eb0e254b feat(ui): add mood column to Album and Song list views (#3925)
Signed-off-by: Deluan <deluan@navidrome.org>
2025-04-05 08:23:52 -03:00
Deluan Quintão
2b84c574ba fix: restore old date display/sort behaviour (#3862)
* fix(server): bring back legacy date mappings

Signed-off-by: Deluan <deluan@navidrome.org>

* reuse the mapDates logic in the legacyReleaseDate function

Signed-off-by: Deluan <deluan@navidrome.org>

* fix mappings

Signed-off-by: Deluan <deluan@navidrome.org>

* show original and release dates in album grid

Signed-off-by: Deluan <deluan@navidrome.org>

* fix tests based on new year mapping

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic): prefer returning original_year over (recording) year
when sorting albums

Signed-off-by: Deluan <deluan@navidrome.org>

* fix case when we don't have originalYear

Signed-off-by: Deluan <deluan@navidrome.org>

* show all dates in album's info, and remove the recording date from the album page

Signed-off-by: Deluan <deluan@navidrome.org>

* better?

Signed-off-by: Deluan <deluan@navidrome.org>

* add snapshot tests for Album Details

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic): sort order for getAlbumList?type=byYear

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-30 17:06:58 -04:00
Deluan
88f87e6c4f chore: replace album placeholder
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-30 13:41:32 -04:00
Deluan
cf100c4eb4 chore(subsonic): update snapshot tests to use version 1.16.1 2025-03-27 22:50:22 -04:00
Deluan Quintão
5ab345c83e chore(server): add more info to scrobble errors logs (#3889)
* chore(server): add more info to scrobble errors

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(server): add more info to scrobble errors

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(server): add more info to scrobble errors

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-27 18:57:06 -04:00
Deluan
46a2ec0ba1 feat(ui): hide absolute paths from regular users
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-25 20:05:24 -04:00
Deluan
3394580413 feat(ui): add Norwegian translation
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-25 17:43:25 -04:00
Michachatz
112ea281d9 feat(ui): add Greek translation (#3892)
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-25 17:33:58 -04:00
Deluan Quintão
c837838d58 fix(ui): update French, Polish, Turkish translations from POEditor (#3834)
Co-authored-by: navidrome-bot <navidrome-bot@navidrome.org>
2025-03-24 17:52:03 -04:00
matteo00gm
9e9465567d fix(ui): update Italian translations (#3885) 2025-03-24 17:49:23 -04:00
Deluan
651ce163c7 fix(ui): sort playlist by album_artist, bpm and channels
fix #3878

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-24 16:41:54 -04:00
Deluan Quintão
55ce28b2c6 fix(bfr): force upgrade to read all folders. (#3871)
* chore(scanner): add trace logs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(bfr): force upgrade to read all folders. It was skipping folders for certain timezones

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-24 15:22:59 -04:00
Deluan
d331ee904b fix(ui): sort playlist by year
fix #3878

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-24 15:08:17 -04:00
Deluan
3a0ce6aafa fix(scanner): elapsed time for folder processing is wrong in the logs
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-23 12:36:38 -04:00
Deluan
1806552ef6 chore: remove more outdated TODOs
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-23 11:53:43 -04:00
Deluan
223e88d481 chore: remove some BFR-related TODOs that are not valid anymore
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-23 11:37:20 -04:00
Deluan Quintão
57e0f6d3ea feat(server): custom ArtistJoiner config (#3873)
* feat(server): custom ArtistJoiner config

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(ui): organize ArtistLinkField, add tests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): use display artist

* feat(ui): use display artist

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-23 10:53:21 -04:00
Deluan
1c691ac0e6 feat(docker): automatically loads a navidrome.toml file from /data, if available
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-22 17:33:56 -04:00
Deluan
264d73d73e fix(server): don't break if the ND_CONFIGFILE does not exist
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-22 17:08:03 -04:00
Deluan
296259d781 feat(ui): show bitDepth in song info dialog
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-22 15:48:29 -04:00
Deluan
3f9d173495 fix(scanner): support ID3v2 embedded images in WAV files
Fix #3867

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-22 15:48:07 -04:00
Deluan
b386981b7f fix(scanner): better log message when AutoImportPlaylists is disabled
Fix #3861

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-22 15:08:26 -04:00
Deluan Quintão
be7cb59dc5 fix(scanner): allow disabling splitting with the Tags config option (#3869)
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-22 12:34:35 -04:00
Nicolas Derive
63dc0e2062 fix(ui): update Français, reorder translation according to en.json template (#3839)
Update french translation and reorder the file the same way as the en.json template, making comparison easier.
2025-03-22 12:31:32 -04:00
Xabi
1e1dce92b6 fix(ui): update Basque translation (#3864)
* Update Basque localisation

added missing strings

* Update eu.json
2025-03-22 12:29:43 -04:00
Deluan
d78c6f6a04 fix(subsonic): ArtistID3 should contain list of AlbumID3
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-20 22:10:46 -04:00
Deluan Quintão
59ece40393 fix(server): better embedded artwork extraction with ffmpeg (#3860)
- `-map 0:v` selects all video streams from the input
- `-map -0:V` excludes all "main" video streams (capital V)

This combination effectively selects only the attached pictures

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-20 19:26:40 -04:00
Deluan
491210ac12 fix(scanner): ignore NaN ReplayGain values
Fix: https://github.com/navidrome/navidrome/issues/3858
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-20 12:42:09 -04:00
Deluan
cd552a55ef fix(scanner): pass datafolder and cachefolder to scanner subprocess
Fix #3831

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-19 22:15:20 -04:00
Deluan
ee2c2b19e9 fix(dockerfile): remove the healthcheck, it gives more headaches than benefits.
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-19 20:18:56 -04:00
Deluan
0147bb5f12 chore(deps): upgrade viper to 1.20.0, add tests for the supported config formats
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-18 19:16:47 -04:00
Rob Emery
1ed8930107 fix(msi): don't override custom ini config (#3836)
Previously addLine would add-or-update, resulting in the custom settings being overriden on upgrade. createLine will only add to the ini if the key doesn't already exist.
2025-03-18 18:23:04 -04:00
Deluan
e457f21306 chore(server): show square flag in resize artwork logs
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-18 12:43:52 -04:00
Deluan Quintão
b04647309f chore(deps): upgrade to Go 1.24.1 (#3851)
* chore(deps): upgrade to Go 1.24.1

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(deps): add reflex as go.mod tool

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(deps): add wire as go.mod tool

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(deps): add goimports as go.mod tool

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(deps): add ginkgo as go.mod tool

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-17 21:08:10 -04:00
Deluan Quintão
2adb098f32 fix(scanner): fix displayArtist logic (#3835)
* fix displayArtist logic

Signed-off-by: Deluan <deluan@navidrome.org>

* remove unneeded value

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor

Signed-off-by: Deluan <deluan@navidrome.org>

* Use first albumartist if it cannot figure out the display name

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-17 19:21:33 -04:00
Kendall Garner
212887214c fix(ui): minor icon inconsistencies and "no missing files" translation (#3837)
* chore(ui): Fix minor inconsistencies

1. The icons in the user menu are a mix of MUI and react-icons. Move them all to react-icons, and use a standard size (24px)
2. On missing files page, provide a custom Empty component that just removes 'yet'

* use RA's builtin support for custom empty message

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>
2025-03-16 19:39:19 -04:00
Deluan Quintão
beb768cd9c feat(server): add Role filters to albums (#3829)
* navidrome artist filtering

* address discord feedback

* perPage min 36

* various artist artist_id -> albumartist_id

* artist_id, role_id separate

* remove all ui changes I guess

* Add tests, check for possible SQL injection

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Kendall Garner <17521368+kgarner7@users.noreply.github.com>
2025-03-14 21:43:52 -04:00
Kendall Garner
ed1109ddb2 fix(subsonic): fix albumCount in artists (#3827)
* only do subsonic instead

* make sure to actually populate response first

* navidrome artist filtering

* address discord feedback

* perPage min 36

* various artist artist_id -> albumartist_id

* artist_id, role_id separate

* remove all ui changes I guess

* Revert role filters

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>
2025-03-14 21:21:03 -04:00
Deluan
98808e4b6d docs(scanner): clarifies the purpose of the mappings.yaml file for regular users
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-14 19:32:26 -04:00
Deluan
422ba2284e chore(scanner): add logs to .ndignore processing
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-14 17:44:11 -04:00
Kendall Garner
938c3d44cc fix(scanner): restore setsubtitle as discsubtitle for non-WMA (#3821)
With old metadata, Disc Subtitle was one of `tsst`, `discsubtitle`, or `setsubtitle`.
With the updated, `setsubtitle` is only available for flac.
Update `mappings.yaml` to maintain prior behavior.
2025-03-14 07:01:07 -04:00
Deluan
2838ac36df feat(scanner): allow disabling tags with Tags.<tag>.Ignore=true
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-13 19:55:30 -04:00
Deluan
b952672877 fix(scanner): add back the Scanner.GenreSeparators as a deprecated option
This allows easy upgrade of containers in PikaPods

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-13 19:25:07 -04:00
Deluan Quintão
5c0b6fb9b7 fix(server): skip non-UTF encoding during the database migration. (#3803)
Fix #3787

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-13 07:10:45 -04:00
Deluan
5fb1db6031 fix(scanner): watcher not working with relative MusicFolder
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-12 18:13:22 -04:00
Deluan
226be78bf5 fix(scanner): full_text not being updated on scan
Fixes #3813

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-12 17:51:36 -04:00
Deluan
7c13878075 fix(subsonic): getRandomSongs with genre filter
fix https://github.com/dweymouth/supersonic/issues/577

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-12 17:35:06 -04:00
Rodrigo Iglesias
0bb4b881e9 fix(ui): update Español translation (#3805)
Corrected "aletorio" and added some more translations
2025-03-11 20:42:09 -04:00
Deluan Quintão
70f536e04d fix(ui): skip missing files in bulk operations (#3807)
* fix(ui): skip missing files when adding to playqueue

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): skip missing files when adding to playlists

* fix(ui): skip missing files when shuffling songs

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-11 20:19:46 -04:00
Deluan Quintão
2a15a217de fix(server): db migration does not work for MusicFolders ending with a trailing slash. (#3797)
* fix(server): db migration was not working for MusicFolders ending with a trailing slash.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(server): db migration for relative paths

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-11 10:09:09 -04:00
Kendall Garner
a28462a7ab fix(ui): fix make dev (#3795)
1. For some bizarre reason, importing inflection by itself is undefined. But you can import specific functions
2. Per https://github.com/vite-pwa/vite-plugin-pwa/issues/419, `type: 'module',` is only for non-chromium browsers
2025-03-10 14:50:16 -04:00
Deluan
5c67297dce fix(server): panic when logging tag type. Fix #3790
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-10 07:14:17 -04:00
Deluan Quintão
365df5220b fix(server): db migration not working when MusicFolder is a relative path (#3766)
* fix(server): db migration not working when MusicFolder is a relative path

Signed-off-by: Deluan <deluan@navidrome.org>

* remove todo

Signed-off-by: Deluan <deluan@navidrome.org>

* fix migration of paths in Windows

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-09 19:14:29 -04:00
Deluan Quintão
b2b5c00331 fix(ui): update Finnish, Hungarian, Russian, Ukrainian translations from POEditor (#3780)
Co-authored-by: navidrome-bot <navidrome-bot@navidrome.org>
2025-03-09 18:22:20 -04:00
Deluan
ee18489b85 fix(subsonic): don't return empty disctitles for a single disc album
See https://support.symfonium.app/t/hide-disc-header-for-albums-with-only-1-disc/6877/1

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-09 17:22:41 -04:00
Deluan
57d3be8604 feat(subsonic): rename AppendSubtitle conf to Subsonic.AppendSubtitle, for consistency
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-08 19:02:29 -05:00
Deluan
0d42b9a4a5 chore(deps): bump more JS dependencies
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-07 20:07:15 -05:00
Deluan
a1a6047c37 chore(deps): bump Vite version
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-07 19:59:35 -05:00
Deluan
2171c44503 chore(deps): bump JS dependencies
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-07 19:47:08 -05:00
Deluan
fac01ccecb chore(deps): bump Go dependencies
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-07 19:36:46 -05:00
Deluan
98a6819390 fix(ui): disable bulk action buttons if transcoding edit is disabled
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-07 18:01:49 -05:00
Deluan
4156602158 build(ci): show English names for changed languages in POEditor PRs
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-07 12:12:44 -05:00
Deluan
21a5528f5e feat(server): deprecate Scanner.GroupAlbumReleases config option
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-06 23:57:47 -05:00
Deluan
31e003e6f3 feat(ui): use webp for login backgrounds
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-06 23:32:52 -05:00
ChekeredList71
e467e32c06 fix(ui): updated Hungarian translation for BFR (#3773)
* Hungarian translation for v0.54.1 done

* Hungarian translation for v0.54.1 done

* Updated Hugarian translation

* Updated Hugarian translation

---------

Co-authored-by: ChekeredList71 <null@example.com>
Co-authored-by: ChekeredList71 <ads@asd.com>
2025-03-06 22:41:45 -05:00
Kendall Garner
36ed880e61 fix(scanner): always refresh folder image time when adding first image (#3764)
* fix(scanner): Always refresh folder image time when adding first image

Currently, the `images_updated_at` field is only set to the image modification time.
However, in cases where a new image is added _and_ said image is older than the folder mod time, the field is not updated properly.

In this the case where `images_updated_at` is null (no images were ever added) and a new images is found, use the folder modification time instead of image modification time.

**Note**, this doesn't handle cases such as replacing a newer image with an older one.

* simplify image update at

* we don't want to set imagesUpdatedAt when there's no images in the folder

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>
2025-03-06 22:16:37 -05:00
Deluan
1c192d8a6d fix(ui): replace bulk "delete" label with "remove" in playlists
Fix #3525

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-06 07:54:59 -05:00
Kendall Garner
5869f7caaf feat(subsonic): set sortName for OS AlbumList (#3776)
* feat(subsonic): Set SortName for OS AlbumList, test to JSON/XML

* albumlist2, star2 updated properly

* fix(subsonic): add sort or order name based on config

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>
2025-03-05 22:52:15 -05:00
Deluan
8732fc7226 fix(server): change log level for some unimportant messages
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-05 20:54:06 -05:00
Deluan
0372339e1b fix(server): only build core.Agents once
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-05 14:18:27 -08:00
Deluan
a04167672c fix(server): remove misleading "Agent not available" warning.
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-05 14:11:44 -08:00
Deluan
dc4e091622 feat(server): make appending subtitle to song title configurable
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-05 12:36:09 -08:00
Deluan
8ab2a11d22 feat(server): group Subsonic config options together
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-05 12:29:30 -08:00
Deluan
637c909e93 feat(server): removed GenreSeparator, replaced with Tag.Genre.Split
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-28 15:36:21 -10:00
Deluan
453873fa26 feat(insights): send scanner options
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-28 15:36:21 -10:00
Deluan
de37e0f720 feat(server): rename ScanSchedule conf to Scanner.Schedule, for consistency
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-28 15:36:21 -10:00
Deluan
f3cb85cb0d feat(server): warn users of ffmpeg extractor that it is not available anymore
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-28 12:39:30 -08:00
Deluan Quintão
0c4c223127 fix(server): import absolute paths in m3u (#3756)
* fix(server): import playlists with absolute paths

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(server): optimize playlist import

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(server): add test with multiple libraries

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(server): refactor

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-26 22:26:38 -05:00
Deluan Quintão
3892f70c35 fix(ui): update Deutsch, Español, Euskara, Galego, Bahasa Indonesia, 日本語, Português, Pусский, Türkçe translations from POEditor (#3681)
Co-authored-by: navidrome-bot <navidrome-bot@navidrome.org>
2025-02-26 22:20:48 -05:00
Deluan Quintão
1468a56808 fix(server): reduce SQLite "database busy" errors (#3760)
* fix(scanner): remove transactions where they are not strictly needed

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(server): force setStar transaction to start as IMMEDIATE

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(server): encapsulated way to upgrade tx to write mode

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(server): use tx immediate for some playlist endpoints

Signed-off-by: Deluan <deluan@navidrome.org>

* make more transactions immediate (#3759)

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Kendall Garner <17521368+kgarner7@users.noreply.github.com>
2025-02-26 22:01:49 -05:00
Deluan
d6ec52b9d4 fix(subsonic): check errors before setting headers for getCoverArt
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-25 08:22:38 -05:00
Deluan
5fa19f9cfa chore(server): add logs to begin/end transaction
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-24 19:13:42 -05:00
Deluan
15a3d2ca66 fix(server): disallow search engine crawlers in robots.txt
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-23 22:01:01 -05:00
Kendall Garner
efab198d4a test(server): validate play tracker participants, scrobble buffer (#3752)
* test(server): validate play tracker participants, scrobble buffer

* tests(server): nit: remove duplicated tests and small cleanups

Signed-off-by: Deluan <deluan@navidrome.org>

* tests(server): nit: replace panics with assertions

Signed-off-by: Deluan <deluan@navidrome.org>

* just use random ids, and store it instead

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>
2025-02-23 21:52:51 -05:00
Deluan
5ad9f546b2 fix(server): role filters in Smart Playlists.
See https://github.com/navidrome/navidrome/discussions/3676#discussioncomment-12286960

Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-23 14:08:53 -05:00
Deluan
20297c2aea fix(server): send artist mbids when scrobbling to ListenBrainz
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-23 13:30:39 -05:00
Kendall Garner
f6eee65955 feat(ui): Show performer subrole(s) where possible (#3747)
* feat(ui): Show performer subrole(s) where possible

* nit: simplify subrole formatting

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>
2025-02-22 12:05:19 -05:00
Kendall Garner
aee19e747c feat(ui): Improve Artist Album pagination (#3748)
* feat(ui): Improve Artist Album pagination

- use maximum of albumartist/artist credits for determining pagination
- reduce default maxPerPage considerably. This gives values of 36/72/108 at largest size

* enable pagination when over 90

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>
2025-02-22 09:31:20 -05:00
Deluan
f34f15ba1c feat(ui): make need for refresh more visible when upgrading server
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-21 18:15:25 -05:00
Deluan
74348a340f feat(server): new option to set the default for ReportRealPath on new players
Implements #3653

Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-20 22:24:09 -05:00
Deluan
09ae41a2da sec(subsonic): authentication bypass in Subsonic API with non-existent username
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-20 20:14:19 -05:00
Deluan
70487a09f4 fix(ui): paginate albums in artist page when needed
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-20 19:21:01 -05:00
Deluan
d4147c2330 fix(scanner): improve refresh artists stats query
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-20 14:55:45 -05:00
Deluan
dd4802c0c6 fix(ui): remove unused term
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-19 22:38:09 -05:00
Deluan
efed7f1b40 chore(deps): bump go dependencies
Signed-off-by: Deluan <deluan@navidrome.org>
2025-02-19 21:15:35 -05:00
Xabi
6cc95d53a9 fix(ui): update Basque translation (#3666) 2025-02-19 21:01:27 -05:00
Deluan Quintão
c795bcfcf7 feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709)
* fix(server): more race conditions when updating artist/album from external sources

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scanner): add .gitignore syntax to .ndignore. Resolves #1394

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): null

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): pass configfile option to child process

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): resume interrupted fullScans

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): remove old scanner code

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): rename old metadata package

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): move old metadata package

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: tests

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(deps): update Go to 1.23.4

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: logs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(test):

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: log level

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: remove log message

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add config for scanner watcher

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: children playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: replace `interface{}` with `any`

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: smart playlists with genres

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: allow any tags in smart playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: artist names in playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: smart playlist's sort by tags

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add moods to child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add moods to AlbumID3

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(subsonic): use generic JSONArray for OS arrays

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(subsonic): use https in test

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add releaseTypes to AlbumID3

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add recordLabels to AlbumID3

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(subsonic): rename JSONArray to Array

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add artists to AlbumID3

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add artists to Child

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): do not pre-populate smart playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): implement a simplified version of ArtistID3.

See https://github.com/opensubsonic/open-subsonic-api/discussions/120

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add artists to album child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add contributors to mediafile Child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add albumArtists to mediafile Child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add displayArtist and displayAlbumArtist

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add displayComposer to Child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add roles to ArtistID3

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic): use " • " separator for displayComposer

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor:

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic):

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic): respect `PreferSortTags` config option

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(subsonic):

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: optimize purging non-unused tags

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: don't run 'refresh artist stats' concurrently with other transactions

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor:

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: log message

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add Scanner.ScanOnStartup config option, default true

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: better json parsing error msg when importing NSPs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't update album's imported_time when updating external_metadata

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: handle interrupted scans and full scans after migrations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: run `analyze` when migration requires a full rescan

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: run `PRAGMA optimize` at the end of the scan

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't update artist's updated_at when updating external_metadata

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: handle multiple artists and roles in smart playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): dim missing tracks

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: album missing logic

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: error encoding in gob

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: separate warnings from errors

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: mark albums as missing if they were contained in a deleted folder

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: add participant names to media_file and album tables

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: use participations in criteria, instead of m2m relationship

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename participations to participants

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add moods to album child

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: albumartist role case

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scanner): run scanner as an external process by default

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): show albumArtist names

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): dim out missing albums

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: flaky test

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(server): scrobble buffer mapping. fix #3583

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: more participations renaming

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: listenbrainz scrobbling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: send release_group_mbid to listenbrainz

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): implement OpenSubsonic explicitStatus field (#3597)

* feat: implement OpenSubsonic explicitStatus field

* fix(subsonic): fix failing snapshot tests

* refactor: create helper for setting explicitStatus

* fix: store smaller values for explicit-status on database

* test: ToAlbum explicitStatus

* refactor: rename explicitStatus helper function

---------

Co-authored-by: Deluan Quintão <deluan@navidrome.org>

* fix: handle album and track tags in the DB based on the mappings.yaml file

Signed-off-by: Deluan <deluan@navidrome.org>

* save similar artists as JSONB

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: getAlbumList byGenre

Signed-off-by: Deluan <deluan@navidrome.org>

* detect changes in PID configuration

Signed-off-by: Deluan <deluan@navidrome.org>

* set default album PID to legacy_pid

Signed-off-by: Deluan <deluan@navidrome.org>

* fix tests

Signed-off-by: Deluan <deluan@navidrome.org>

* fix SIGSEGV

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't lose album stars/ratings when migrating

Signed-off-by: Deluan <deluan@navidrome.org>

* store full PID conf in properties

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: keep album annotations when changing PID.Album config

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: reassign album annotations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: use (display) albumArtist and add links to each artist

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: not showing albums by albumartist

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: error msgs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: hide PID from Native API

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: album cover art resolution

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: trim participant names

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: reduce watcher log spam

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: panic when initializing the watcher

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: various artists

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't store empty lyrics in the DB

Signed-off-by: Deluan <deluan@navidrome.org>

* remove unused methods

Signed-off-by: Deluan <deluan@navidrome.org>

* drop full_text indexes, as they are not being used by SQLite

Signed-off-by: Deluan <deluan@navidrome.org>

* keep album created_at when upgrading

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): null pointer

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: album artwork cache

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't expose missing files in Subsonic API

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: searchable interface

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: filter out missing items from subsonic search

* fix: filter out missing items from playlists

* fix: filter out missing items from shares

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): add filter by artist role

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): only return albumartists in getIndexes and getArtists endpoints

Signed-off-by: Deluan <deluan@navidrome.org>

* sort roles alphabetically

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: artist playcounts

Signed-off-by: Deluan <deluan@navidrome.org>

* change default Album PID conf

Signed-off-by: Deluan <deluan@navidrome.org>

* fix albumartist link when it does not match any albumartists values

Signed-off-by: Deluan <deluan@navidrome.org>

* fix `Ignoring filter not whitelisted` (role) message

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: trim any names/titles being imported

Signed-off-by: Deluan <deluan@navidrome.org>

* remove unused genre code

Signed-off-by: Deluan <deluan@navidrome.org>

* serialize calls to Last.fm's getArtist

Signed-off-by: Deluan <deluan@navidrome.org>

xxx

Signed-off-by: Deluan <deluan@navidrome.org>

* add counters to genres

Signed-off-by: Deluan <deluan@navidrome.org>

* nit: fix migration `notice` message

Signed-off-by: Deluan <deluan@navidrome.org>

* optimize similar artists query

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: last.fm.getInfo when mbid does not exist

Signed-off-by: Deluan <deluan@navidrome.org>

* ui only show missing items for admins

Signed-off-by: Deluan <deluan@navidrome.org>

* don't allow interaction with missing items

Signed-off-by: Deluan <deluan@navidrome.org>

* Add Missing Files view (WIP)

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: merged tag_counts into tag table

Signed-off-by: Deluan <deluan@navidrome.org>

* add option to completely disable automatic scanner

Signed-off-by: Deluan <deluan@navidrome.org>

* add delete missing files functionality

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: playlists not showing for regular users

Signed-off-by: Deluan <deluan@navidrome.org>

* reduce updateLastAccess frequency to once every minute

Signed-off-by: Deluan <deluan@navidrome.org>

* reduce update player frequency to once every minute

Signed-off-by: Deluan <deluan@navidrome.org>

* add timeout when updating player

Signed-off-by: Deluan <deluan@navidrome.org>

* remove dead code

Signed-off-by: Deluan <deluan@navidrome.org>

* fix duplicated roles in stats

Signed-off-by: Deluan <deluan@navidrome.org>

* add `; ` to artist splitters

Signed-off-by: Deluan <deluan@navidrome.org>

* fix stats query

Signed-off-by: Deluan <deluan@navidrome.org>

* more logs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* add record label filter

Signed-off-by: Deluan <deluan@navidrome.org>

* add release type filter

Signed-off-by: Deluan <deluan@navidrome.org>

* fix purgeUnused tags

Signed-off-by: Deluan <deluan@navidrome.org>

* add grouping filter to albums

Signed-off-by: Deluan <deluan@navidrome.org>

* allow any album tags to be used in as filters in the API

Signed-off-by: Deluan <deluan@navidrome.org>

* remove empty tags from album info

Signed-off-by: Deluan <deluan@navidrome.org>

* comments in the migration

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: Cannot read properties of undefined

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: listenbrainz scrobbling (#3640)

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: remove duplicated tag values

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't ignore the taglib folder!

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: show track subtitle tag

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: show artists stats based on selected role

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: inspect

Signed-off-by: Deluan <deluan@navidrome.org>

* add media type to album info/filters

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: change format of subtitle in the UI

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: subtitle in Subsonic API and search

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: subtitle in UI's player

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: split strings should be case-insensitive

Signed-off-by: Deluan <deluan@navidrome.org>

* disable ScanSchedule

Signed-off-by: Deluan <deluan@navidrome.org>

* increase default sessiontimeout

Signed-off-by: Deluan <deluan@navidrome.org>

* add sqlite command line tool to docker image

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: resources override

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: album PID conf

Signed-off-by: Deluan <deluan@navidrome.org>

* change migration to mark current artists as albumArtists

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): Allow filtering on multiple genres (#3679)

* feat(ui): Allow filtering on multiple genres

Signed-off-by: Henrik Nordvik <henrikno@gmail.com>
Signed-off-by: Deluan <deluan@navidrome.org>

* add multi-genre filter in Album list

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Henrik Nordvik <henrikno@gmail.com>
Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Henrik Nordvik <henrikno@gmail.com>

* add more multi-valued tag filters to Album and Song views

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): unselect missing files after removing

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): song filter

Signed-off-by: Deluan <deluan@navidrome.org>

* fix sharing tracks. fix #3687

Signed-off-by: Deluan <deluan@navidrome.org>

* use rowids when using search for sync (ex: Symfonium)

Signed-off-by: Deluan <deluan@navidrome.org>

* fix "Report Real Paths" option for subsonic clients

Signed-off-by: Deluan <deluan@navidrome.org>

* fix "Report Real Paths" option for subsonic clients for search

Signed-off-by: Deluan <deluan@navidrome.org>

* add libraryPath to Native API /songs endpoint

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add album version

Signed-off-by: Deluan <deluan@navidrome.org>

* made all tags lowercase as they are case-insensitive anyways.

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): Show full paths, extended properties for album/song (#3691)

* feat(ui): Show full paths, extended properties for album/song

- uses library path + os separator + path
- show participants (album/song) and tags (song)
- make album/participant clickable in show info

* add source to path

* fix pathSeparator in UI

Signed-off-by: Deluan <deluan@navidrome.org>

* fix local artist artwork (#3695)

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: parse vorbis performers

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: clean function into smaller functions

Signed-off-by: Deluan <deluan@navidrome.org>

* fix translations for en and pt

Signed-off-by: Deluan <deluan@navidrome.org>

* add trace log to show annotations reassignment

Signed-off-by: Deluan <deluan@navidrome.org>

* add trace log to show annotations reassignment

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: allow performers without instrument/subrole

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: metadata clean function again

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: optimize split function

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: split function is now a method of TagConf

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: humanize Artist total size

Signed-off-by: Deluan <deluan@navidrome.org>

* add album version to album details

Signed-off-by: Deluan <deluan@navidrome.org>

* don't display album-level tags in SongInfo

Signed-off-by: Deluan <deluan@navidrome.org>

* fix genre clicking in Album Page

Signed-off-by: Deluan <deluan@navidrome.org>

* don't use mbids in Last.fm api calls.

From https://discord.com/channels/671335427726114836/704303730660737113/1337574018143879248:

With MBID:
```
GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&mbid=a41ac10f-0a56-4672-9161-b83f9b223559&method=artist.getInfo

{
artist: {
name: "Bee Gees",
mbid: "bf0f7e29-dfe1-416c-b5c6-f9ebc19ea810",
url: "https://www.last.fm/music/Bee+Gees",
}
```

Without MBID:
```
GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&method=artist.getInfo

{
artist: {
name: "Van Morrison",
mbid: "a41ac10f-0a56-4672-9161-b83f9b223559",
url: "https://www.last.fm/music/Van+Morrison",
}
```

Signed-off-by: Deluan <deluan@navidrome.org>

* better logging for when the artist folder is not found

Signed-off-by: Deluan <deluan@navidrome.org>

* fix various issues with artist image resolution

Signed-off-by: Deluan <deluan@navidrome.org>

* hide "Additional Tags" header if there are none.

Signed-off-by: Deluan <deluan@navidrome.org>

* simplify tag rendering

Signed-off-by: Deluan <deluan@navidrome.org>

* enhance logging for artist folder detection

Signed-off-by: Deluan <deluan@navidrome.org>

* make folderID consistent for relative and absolute folderPaths

Signed-off-by: Deluan <deluan@navidrome.org>

* handle more folder paths scenarios

Signed-off-by: Deluan <deluan@navidrome.org>

* filter out other roles when SubsonicArtistParticipations = true

Signed-off-by: Deluan <deluan@navidrome.org>

* fix "Cannot read properties of undefined"

Signed-off-by: Deluan <deluan@navidrome.org>

* fix lyrics and comments being truncated (#3701)

* fix lyrics and comments being truncated

* specifically test for lyrics and comment length

* reorder assertions

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>

* fix(server): Expose library_path for playlist (#3705)

Allows showing absolute path for UI, and makes "report real path" work for playlists (Subsonic)

* fix BFR on Windows (#3704)

* fix potential reflected cross-site scripting vulnerability

Signed-off-by: Deluan <deluan@navidrome.org>

* hack to make it work on Windows

* ignore windows executables

* try fixing the pipeline

Signed-off-by: Deluan <deluan@navidrome.org>

* allow MusicFolder in other drives

* move windows local drive logic to local storage implementation

---------

Signed-off-by: Deluan <deluan@navidrome.org>

* increase pagination sizes for missing files

Signed-off-by: Deluan <deluan@navidrome.org>

* reduce level of "already scanning" watcher log message

Signed-off-by: Deluan <deluan@navidrome.org>

* only count folders with audio files in it

See https://github.com/navidrome/navidrome/discussions/3676#discussioncomment-11990930

Signed-off-by: Deluan <deluan@navidrome.org>

* add album version and catalog number to search

Signed-off-by: Deluan <deluan@navidrome.org>

* add `organization` alias for `recordlabel`

Signed-off-by: Deluan <deluan@navidrome.org>

* remove mbid from Last.fm agent

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: support inspect in ui (#3726)

* inspect in ui

* address round 1

* add catalogNum to AlbumInfo

Signed-off-by: Deluan <deluan@navidrome.org>

* remove dependency on metadata_old (deprecated) package

Signed-off-by: Deluan <deluan@navidrome.org>

* add `RawTags` to model

Signed-off-by: Deluan <deluan@navidrome.org>

* support parsing MBIDs for roles (from the https://github.com/kgarner7/picard-all-mbids plugin) (#3698)


* parse standard roles, vorbis/m4a work for now

* fix djmixer

* working roles, use DJ-mix

* add performers to file

* map mbids

* add a few more tests

* add test

Signed-off-by: Deluan <deluan@navidrome.org>

* try to simplify the performers logic

Signed-off-by: Deluan <deluan@navidrome.org>

* stylistic changes

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>

* remove param mutation

Signed-off-by: Deluan <deluan@navidrome.org>

* run automated SQLite optimizations

Signed-off-by: Deluan <deluan@navidrome.org>

* fix playlists import/export on Windows

* fix import playlists

* fix export playlists

* better handling of Windows volumes

Signed-off-by: Deluan <deluan@navidrome.org>

* handle more album ID reassignments

Signed-off-by: Deluan <deluan@navidrome.org>

* allow adding/overriding tags in the config file

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): Fix playlist track id, handle missing tracks better (#3734)

- Use `mediaFileId` instead of `id` for playlist tracks
- Only fetch if the file is not missing
- If extractor fails to get the file, also error (rather than panic)

* optimize DB after each scan.

Signed-off-by: Deluan <deluan@navidrome.org>

* remove sortable from AlbumSongs columns

Signed-off-by: Deluan <deluan@navidrome.org>

* simplify query to get missing tracks

Signed-off-by: Deluan <deluan@navidrome.org>

* mark Scanner.Extractor as deprecated

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Signed-off-by: Henrik Nordvik <henrikno@gmail.com>
Co-authored-by: Caio Cotts <caio@cotts.com.br>
Co-authored-by: Henrik Nordvik <henrikno@gmail.com>
Co-authored-by: Kendall Garner <17521368+kgarner7@users.noreply.github.com>
2025-02-19 20:35:17 -05:00
RTapeLoadingError
46a963a02a fix(ui): update Spanish translation (#3682)
Disambiguation for:
"recentlyAdded": "Añadidos recientemente",
"recentlyPlayed": "Reproducidos recientemente"
They share the same label: "Recientes".
2025-02-01 13:07:41 -05:00
Matvei Stefarov
195ae56001 fix(ui) Update Russian translation (#3678)
* fix(ui): Update Russian translations

- Adds missing strings added in the past couple releases
- Fixes a few confusing translations in the "share" section

* Add missing comma
2025-01-30 20:17:16 -05:00
Deluan Quintão
f9db449e7e fix(ui): update ไทย translations from POEditor (#3662)
Co-authored-by: navidrome-bot <navidrome-bot@navidrome.org>
2025-01-24 18:11:54 -05:00
Deluan
657fe11f53 fix: remove Access-Control-Allow-Origin. closes #3660
Signed-off-by: Deluan <deluan@navidrome.org>
2025-01-22 18:24:11 -05:00
Deluan Quintão
47e3fdb1b8 fix(server): do not try to validate credentials if the request is canceled (#3650)
Signed-off-by: Deluan <deluan@navidrome.org>
2025-01-16 20:32:11 -05:00
Deluan Quintão
c37583fa9f feat(server): create M3Us from shares (#3652) 2025-01-16 20:26:16 -05:00
Deluan
9d86f63f15 fix(server): add logs to public image endpoint
Signed-off-by: Deluan <deluan@navidrome.org>
2025-01-15 08:47:47 -05:00
522 changed files with 30575 additions and 10755 deletions

View File

@@ -4,7 +4,7 @@
"dockerfile": "Dockerfile",
"args": {
// Update the VARIANT arg to pick a version of Go: 1, 1.15, 1.14
"VARIANT": "1.23",
"VARIANT": "1.24",
// Options
"INSTALL_NODE": "true",
"NODE_VERSION": "v20"

View File

@@ -71,7 +71,7 @@ jobs:
version: ${{ env.CROSS_TAGLIB_VERSION }}
- name: golangci-lint
uses: golangci/golangci-lint-action@v6
uses: golangci/golangci-lint-action@v7
with:
version: latest
problem-matchers: true

View File

@@ -9,6 +9,7 @@ process_json() {
jq 'walk(if type == "object" then with_entries(select(.value != null and .value != "" and .value != [] and .value != {})) | to_entries | sort_by(.key) | from_entries else . end)' "$1"
}
# Function to check differences between local and remote translations
check_lang_diff() {
filename=${I18N_DIR}/"$1".json
url=$(curl -s -X POST https://poeditor.com/api/ \
@@ -35,19 +36,58 @@ check_lang_diff() {
rm -f poeditor.json poeditor.tmp "$filename".tmp
}
# Function to get the list of languages
get_language_list() {
response=$(curl -s -X POST https://api.poeditor.com/v2/languages/list \
-d api_token="${POEDITOR_APIKEY}" \
-d id="${POEDITOR_PROJECTID}")
echo $response
}
# Function to get the language name from the language code
get_language_name() {
lang_code="$1"
lang_list="$2"
lang_name=$(echo "$lang_list" | jq -r ".result.languages[] | select(.code == \"$lang_code\") | .name")
if [ -z "$lang_name" ]; then
echo "Error: Language code '$lang_code' not found" >&2
return 1
fi
echo "$lang_name"
}
# Function to get the language code from the file path
get_lang_code() {
filepath="$1"
# Extract just the filename
filename=$(basename "$filepath")
# Remove the extension
lang_code="${filename%.*}"
echo "$lang_code"
}
lang_list=$(get_language_list)
# Check differences for each language
for file in ${I18N_DIR}/*.json; do
name=$(basename "$file")
code=$(echo "$name" | cut -f1 -d.)
code=$(get_lang_code "$file")
lang=$(jq -r .languageName < "$file")
echo "Downloading $lang ($code)"
lang_name=$(get_language_name "$code" "$lang_list")
echo "Downloading $lang_name - $lang ($code)"
check_lang_diff "$code"
done
# List changed languages to stderr
languages=""
for file in $(git diff --name-only --exit-code | grep json); do
lang=$(jq -r .languageName < "$file")
languages="${languages}$(echo $lang | tr -d '\n'), "
lang_code=$(get_lang_code "$file")
lang_name=$(get_language_name "$lang_code" "$lang_list")
languages="${languages}$(echo "$lang_name" | tr -d '\n'), "
done
echo "${languages%??}" 1>&2

4
.gitignore vendored
View File

@@ -23,5 +23,5 @@ music
docker-compose.yml
!contrib/docker-compose.yml
binaries
taglib
navidrome-master
navidrome-master
*.exe

View File

@@ -1,7 +1,7 @@
version: "2"
run:
build-tags:
- netgo
linters:
enable:
- asasalint
@@ -11,31 +11,48 @@ linters:
- copyloopvar
- dogsled
- durationcheck
- errcheck
- errorlint
- gocritic
- gocyclo
- goprintffuncname
- gosec
- gosimple
- govet
- ineffassign
- misspell
- nakedret
- nilerr
- rowserrcheck
- staticcheck
- typecheck
- unconvert
- unused
- whitespace
linters-settings:
govet:
enable:
- nilness
gosec:
excludes:
- G501
- G401
- G505
- G115 # Can't check context, where the warning is clearly a false positive. See discussion in https://github.com/securego/gosec/pull/1149
disable:
- staticcheck
settings:
gocritic:
disable-all: true
enabled-checks:
- deprecatedComment
gosec:
excludes:
- G501
- G401
- G505
- G115
govet:
enable:
- nilness
exclusions:
generated: lax
presets:
- comments
- common-false-positives
- legacy
- std-error-handling
paths:
- third_party$
- builtin$
- examples$
formatters:
exclusions:
generated: lax
paths:
- third_party$
- builtin$
- examples$

View File

@@ -61,7 +61,7 @@ COPY --from=ui /build /build
########################################################################################################################
### Build Navidrome binary
FROM --platform=$BUILDPLATFORM public.ecr.aws/docker/library/golang:1.23-bookworm AS base
FROM --platform=$BUILDPLATFORM public.ecr.aws/docker/library/golang:1.24-bookworm AS base
RUN apt-get update && apt-get install -y clang lld
COPY --from=xx / /
WORKDIR /workspace
@@ -70,8 +70,6 @@ FROM --platform=$BUILDPLATFORM base AS build
# Install build dependencies for the target platform
ARG TARGETPLATFORM
ARG GIT_SHA
ARG GIT_TAG
RUN xx-apt install -y binutils gcc g++ libc6-dev zlib1g-dev
RUN xx-verify --setup
@@ -81,6 +79,9 @@ RUN --mount=type=bind,source=. \
--mount=type=cache,target=/go/pkg/mod \
go mod download
ARG GIT_SHA
ARG GIT_TAG
RUN --mount=type=bind,source=. \
--mount=from=ui,source=/build,target=./ui/build,ro \
--mount=from=osxcross,src=/osxcross/SDK,target=/xx-sdk,ro \
@@ -124,7 +125,7 @@ LABEL maintainer="deluan@navidrome.org"
LABEL org.opencontainers.image.source="https://github.com/navidrome/navidrome"
# Install ffmpeg and mpv
RUN apk add -U --no-cache ffmpeg mpv
RUN apk add -U --no-cache ffmpeg mpv sqlite
# Copy navidrome binary
COPY --from=build /out/navidrome /app/
@@ -132,12 +133,12 @@ COPY --from=build /out/navidrome /app/
VOLUME ["/data", "/music"]
ENV ND_MUSICFOLDER=/music
ENV ND_DATAFOLDER=/data
ENV ND_CONFIGFILE=/data/navidrome.toml
ENV ND_PORT=4533
ENV GODEBUG="asyncpreemptoff=1"
RUN touch /.nddockerenv
EXPOSE ${ND_PORT}
HEALTHCHECK CMD wget -O- http://localhost:${ND_PORT}/ping || exit 1
WORKDIR /app
ENTRYPOINT ["/app/navidrome"]

View File

@@ -29,23 +29,27 @@ dev: check_env ##@Development Start Navidrome in development mode, with hot-re
.PHONY: dev
server: check_go_env buildjs ##@Development Start the backend in development mode
@ND_ENABLEINSIGHTSCOLLECTOR="false" go run github.com/cespare/reflex@latest -d none -c reflex.conf
@ND_ENABLEINSIGHTSCOLLECTOR="false" go tool reflex -d none -c reflex.conf
.PHONY: server
watch: ##@Development Start Go tests in watch mode (re-run when code changes)
go run github.com/onsi/ginkgo/v2/ginkgo@latest watch -tags netgo -notify ./...
go tool ginkgo watch -tags=netgo -notify ./...
.PHONY: watch
test: ##@Development Run Go tests
go test -tags netgo ./...
.PHONY: test
testrace: ##@Development Run Go tests with race detector
go test -tags netgo -race -shuffle=on ./...
.PHONY: test
testall: test ##@Development Run Go and JS tests
testall: testrace ##@Development Run Go and JS tests
@(cd ./ui && npm run test:ci)
.PHONY: testall
lint: ##@Development Lint Go code
go run github.com/golangci/golangci-lint/cmd/golangci-lint@latest run -v --timeout 5m
go run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@latest run -v --timeout 5m
.PHONY: lint
lintall: lint ##@Development Lint Go and JS code
@@ -55,16 +59,16 @@ lintall: lint ##@Development Lint Go and JS code
format: ##@Development Format code
@(cd ./ui && npm run prettier)
@go run golang.org/x/tools/cmd/goimports@latest -w `find . -name '*.go' | grep -v _gen.go$$`
@go tool goimports -w `find . -name '*.go' | grep -v _gen.go$$`
@go mod tidy
.PHONY: format
wire: check_go_env ##@Development Update Dependency Injection
go run github.com/google/wire/cmd/wire@latest gen -tags=netgo ./...
go tool wire gen -tags=netgo ./...
.PHONY: wire
snapshots: ##@Development Update (GoLang) Snapshot tests
UPDATE_SNAPSHOTS=true go run github.com/onsi/ginkgo/v2/ginkgo@latest ./server/subsonic/...
UPDATE_SNAPSHOTS=true go tool ginkgo ./server/subsonic/responses/...
.PHONY: snapshots
migration-sql: ##@Development Create an empty SQL migration file

View File

@@ -1,2 +1,2 @@
JS: sh -c "cd ./ui && npm start"
GO: go run github.com/cespare/reflex@latest -d none -c reflex.conf
GO: go tool reflex -d none -c reflex.conf

View File

@@ -0,0 +1,154 @@
package taglib
import (
"io/fs"
"os"
"time"
"github.com/djherbis/times"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/metadata"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
type testFileInfo struct {
fs.FileInfo
}
func (t testFileInfo) BirthTime() time.Time {
if ts := times.Get(t.FileInfo); ts.HasBirthTime() {
return ts.BirthTime()
}
return t.FileInfo.ModTime()
}
var _ = Describe("Extractor", func() {
toP := func(name, sortName, mbid string) model.Participant {
return model.Participant{
Artist: model.Artist{Name: name, SortArtistName: sortName, MbzArtistID: mbid},
}
}
roles := []struct {
model.Role
model.ParticipantList
}{
{model.RoleComposer, model.ParticipantList{
toP("coma a", "a, coma", "bf13b584-f27c-43db-8f42-32898d33d4e2"),
toP("comb", "comb", "924039a2-09c6-4d29-9b4f-50cc54447d36"),
}},
{model.RoleLyricist, model.ParticipantList{
toP("la a", "a, la", "c84f648f-68a6-40a2-a0cb-d135b25da3c2"),
toP("lb", "lb", "0a7c582d-143a-4540-b4e9-77200835af65"),
}},
{model.RoleArranger, model.ParticipantList{
toP("aa", "", "4605a1d4-8d15-42a3-bd00-9c20e42f71e6"),
toP("ab", "", "002f0ff8-77bf-42cc-8216-61a9c43dc145"),
}},
{model.RoleConductor, model.ParticipantList{
toP("cona", "", "af86879b-2141-42af-bad2-389a4dc91489"),
toP("conb", "", "3dfa3c70-d7d3-4b97-b953-c298dd305e12"),
}},
{model.RoleDirector, model.ParticipantList{
toP("dia", "", "f943187f-73de-4794-be47-88c66f0fd0f4"),
toP("dib", "", "bceb75da-1853-4b3d-b399-b27f0cafc389"),
}},
{model.RoleEngineer, model.ParticipantList{
toP("ea", "", "f634bf6d-d66a-425d-888a-28ad39392759"),
toP("eb", "", "243d64ae-d514-44e1-901a-b918d692baee"),
}},
{model.RoleProducer, model.ParticipantList{
toP("pra", "", "d971c8d7-999c-4a5f-ac31-719721ab35d6"),
toP("prb", "", "f0a09070-9324-434f-a599-6d25ded87b69"),
}},
{model.RoleRemixer, model.ParticipantList{
toP("ra", "", "c7dc6095-9534-4c72-87cc-aea0103462cf"),
toP("rb", "", "8ebeef51-c08c-4736-992f-c37870becedd"),
}},
{model.RoleDJMixer, model.ParticipantList{
toP("dja", "", "d063f13b-7589-4efc-ab7f-c60e6db17247"),
toP("djb", "", "3636670c-385f-4212-89c8-0ff51d6bc456"),
}},
{model.RoleMixer, model.ParticipantList{
toP("ma", "", "53fb5a2d-7016-427e-a563-d91819a5f35a"),
toP("mb", "", "64c13e65-f0da-4ab9-a300-71ee53b0376a"),
}},
}
var e *extractor
BeforeEach(func() {
e = &extractor{}
})
Describe("Participants", func() {
DescribeTable("test tags consistent across formats", func(format string) {
path := "tests/fixtures/test." + format
mds, err := e.Parse(path)
Expect(err).ToNot(HaveOccurred())
info := mds[path]
fileInfo, _ := os.Stat(path)
info.FileInfo = testFileInfo{FileInfo: fileInfo}
metadata := metadata.New(path, info)
mf := metadata.ToMediaFile(1, "folderID")
for _, data := range roles {
role := data.Role
artists := data.ParticipantList
actual := mf.Participants[role]
Expect(actual).To(HaveLen(len(artists)))
for i := range artists {
actualArtist := actual[i]
expectedArtist := artists[i]
Expect(actualArtist.Name).To(Equal(expectedArtist.Name))
Expect(actualArtist.SortArtistName).To(Equal(expectedArtist.SortArtistName))
Expect(actualArtist.MbzArtistID).To(Equal(expectedArtist.MbzArtistID))
}
}
if format != "m4a" {
performers := mf.Participants[model.RolePerformer]
Expect(performers).To(HaveLen(8))
rules := map[string][]string{
"pgaa": {"2fd0b311-9fa8-4ff9-be5d-f6f3d16b835e", "Guitar"},
"pgbb": {"223d030b-bf97-4c2a-ad26-b7f7bbe25c93", "Guitar", ""},
"pvaa": {"cb195f72-448f-41c8-b962-3f3c13d09d38", "Vocals"},
"pvbb": {"60a1f832-8ca2-49f6-8660-84d57f07b520", "Vocals", "Flute"},
"pfaa": {"51fb40c-0305-4bf9-a11b-2ee615277725", "", "Flute"},
}
for name, rule := range rules {
mbid := rule[0]
for i := 1; i < len(rule); i++ {
found := false
for _, mapped := range performers {
if mapped.Name == name && mapped.MbzArtistID == mbid && mapped.SubRole == rule[i] {
found = true
break
}
}
Expect(found).To(BeTrue(), "Could not find matching artist")
}
}
}
},
Entry("FLAC format", "flac"),
Entry("M4a format", "m4a"),
Entry("OGG format", "ogg"),
Entry("WMA format", "wv"),
Entry("MP3 format", "mp3"),
Entry("WAV format", "wav"),
Entry("AIFF format", "aiff"),
)
})
})

151
adapters/taglib/taglib.go Normal file
View File

@@ -0,0 +1,151 @@
package taglib
import (
"io/fs"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/navidrome/navidrome/core/storage/local"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model/metadata"
)
type extractor struct {
baseDir string
}
func (e extractor) Parse(files ...string) (map[string]metadata.Info, error) {
results := make(map[string]metadata.Info)
for _, path := range files {
props, err := e.extractMetadata(path)
if err != nil {
continue
}
results[path] = *props
}
return results, nil
}
func (e extractor) Version() string {
return Version()
}
func (e extractor) extractMetadata(filePath string) (*metadata.Info, error) {
fullPath := filepath.Join(e.baseDir, filePath)
tags, err := Read(fullPath)
if err != nil {
log.Warn("extractor: Error reading metadata from file. Skipping", "filePath", fullPath, err)
return nil, err
}
// Parse audio properties
ap := metadata.AudioProperties{}
if length, ok := tags["_lengthinmilliseconds"]; ok && len(length) > 0 {
millis, _ := strconv.Atoi(length[0])
if millis > 0 {
ap.Duration = (time.Millisecond * time.Duration(millis)).Round(time.Millisecond * 10)
}
delete(tags, "_lengthinmilliseconds")
}
parseProp := func(prop string, target *int) {
if value, ok := tags[prop]; ok && len(value) > 0 {
*target, _ = strconv.Atoi(value[0])
delete(tags, prop)
}
}
parseProp("_bitrate", &ap.BitRate)
parseProp("_channels", &ap.Channels)
parseProp("_samplerate", &ap.SampleRate)
parseProp("_bitspersample", &ap.BitDepth)
// Parse track/disc totals
parseTuple := func(prop string) {
tagName := prop + "number"
tagTotal := prop + "total"
if value, ok := tags[tagName]; ok && len(value) > 0 {
parts := strings.Split(value[0], "/")
tags[tagName] = []string{parts[0]}
if len(parts) == 2 {
tags[tagTotal] = []string{parts[1]}
}
}
}
parseTuple("track")
parseTuple("disc")
// Adjust some ID3 tags
parseLyrics(tags)
parseTIPL(tags)
delete(tags, "tmcl") // TMCL is already parsed by TagLib
return &metadata.Info{
Tags: tags,
AudioProperties: ap,
HasPicture: tags["has_picture"] != nil && len(tags["has_picture"]) > 0 && tags["has_picture"][0] == "true",
}, nil
}
// parseLyrics make sure lyrics tags have language
func parseLyrics(tags map[string][]string) {
lyrics := tags["lyrics"]
if len(lyrics) > 0 {
tags["lyrics:xxx"] = lyrics
delete(tags, "lyrics")
}
}
// These are the only roles we support, based on Picard's tag map:
// https://picard-docs.musicbrainz.org/downloads/MusicBrainz_Picard_Tag_Map.html
var tiplMapping = map[string]string{
"arranger": "arranger",
"engineer": "engineer",
"producer": "producer",
"mix": "mixer",
"DJ-mix": "djmixer",
}
// parseTIPL parses the ID3v2.4 TIPL frame string, which is received from TagLib in the format:
//
// "arranger Andrew Powell engineer Chris Blair engineer Pat Stapley producer Eric Woolfson".
//
// and breaks it down into a map of roles and names, e.g.:
//
// {"arranger": ["Andrew Powell"], "engineer": ["Chris Blair", "Pat Stapley"], "producer": ["Eric Woolfson"]}.
func parseTIPL(tags map[string][]string) {
tipl := tags["tipl"]
if len(tipl) == 0 {
return
}
addRole := func(currentRole string, currentValue []string) {
if currentRole != "" && len(currentValue) > 0 {
role := tiplMapping[currentRole]
tags[role] = append(tags[role], strings.Join(currentValue, " "))
}
}
var currentRole string
var currentValue []string
for _, part := range strings.Split(tipl[0], " ") {
if _, ok := tiplMapping[part]; ok {
addRole(currentRole, currentValue)
currentRole = part
currentValue = nil
continue
}
currentValue = append(currentValue, part)
}
addRole(currentRole, currentValue)
delete(tags, "tipl")
}
var _ local.Extractor = (*extractor)(nil)
func init() {
local.RegisterExtractor("taglib", func(_ fs.FS, baseDir string) local.Extractor {
// ignores fs, as taglib extractor only works with local files
return &extractor{baseDir}
})
}

View File

@@ -0,0 +1,296 @@
package taglib
import (
"io/fs"
"os"
"strings"
"github.com/navidrome/navidrome/utils"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("Extractor", func() {
var e *extractor
BeforeEach(func() {
e = &extractor{}
})
Describe("Parse", func() {
It("correctly parses metadata from all files in folder", func() {
mds, err := e.Parse(
"tests/fixtures/test.mp3",
"tests/fixtures/test.ogg",
)
Expect(err).NotTo(HaveOccurred())
Expect(mds).To(HaveLen(2))
// Test MP3
m := mds["tests/fixtures/test.mp3"]
Expect(m.Tags).To(HaveKeyWithValue("title", []string{"Song"}))
Expect(m.Tags).To(HaveKeyWithValue("album", []string{"Album"}))
Expect(m.Tags).To(HaveKeyWithValue("artist", []string{"Artist"}))
Expect(m.Tags).To(HaveKeyWithValue("albumartist", []string{"Album Artist"}))
Expect(m.HasPicture).To(BeTrue())
Expect(m.AudioProperties.Duration.String()).To(Equal("1.02s"))
Expect(m.AudioProperties.BitRate).To(Equal(192))
Expect(m.AudioProperties.Channels).To(Equal(2))
Expect(m.AudioProperties.SampleRate).To(Equal(44100))
Expect(m.Tags).To(Or(
HaveKeyWithValue("compilation", []string{"1"}),
HaveKeyWithValue("tcmp", []string{"1"})),
)
Expect(m.Tags).To(HaveKeyWithValue("genre", []string{"Rock"}))
Expect(m.Tags).To(HaveKeyWithValue("date", []string{"2014-05-21"}))
Expect(m.Tags).To(HaveKeyWithValue("originaldate", []string{"1996-11-21"}))
Expect(m.Tags).To(HaveKeyWithValue("releasedate", []string{"2020-12-31"}))
Expect(m.Tags).To(HaveKeyWithValue("discnumber", []string{"1"}))
Expect(m.Tags).To(HaveKeyWithValue("disctotal", []string{"2"}))
Expect(m.Tags).To(HaveKeyWithValue("comment", []string{"Comment1\nComment2"}))
Expect(m.Tags).To(HaveKeyWithValue("bpm", []string{"123"}))
Expect(m.Tags).To(HaveKeyWithValue("replaygain_album_gain", []string{"+3.21518 dB"}))
Expect(m.Tags).To(HaveKeyWithValue("replaygain_album_peak", []string{"0.9125"}))
Expect(m.Tags).To(HaveKeyWithValue("replaygain_track_gain", []string{"-1.48 dB"}))
Expect(m.Tags).To(HaveKeyWithValue("replaygain_track_peak", []string{"0.4512"}))
Expect(m.Tags).To(HaveKeyWithValue("tracknumber", []string{"2"}))
Expect(m.Tags).To(HaveKeyWithValue("tracktotal", []string{"10"}))
Expect(m.Tags).ToNot(HaveKey("lyrics"))
Expect(m.Tags).To(Or(HaveKeyWithValue("lyrics:eng", []string{
"[00:00.00]This is\n[00:02.50]English SYLT\n",
"[00:00.00]This is\n[00:02.50]English",
}), HaveKeyWithValue("lyrics:eng", []string{
"[00:00.00]This is\n[00:02.50]English",
"[00:00.00]This is\n[00:02.50]English SYLT\n",
})))
Expect(m.Tags).To(Or(HaveKeyWithValue("lyrics:xxx", []string{
"[00:00.00]This is\n[00:02.50]unspecified SYLT\n",
"[00:00.00]This is\n[00:02.50]unspecified",
}), HaveKeyWithValue("lyrics:xxx", []string{
"[00:00.00]This is\n[00:02.50]unspecified",
"[00:00.00]This is\n[00:02.50]unspecified SYLT\n",
})))
// Test OGG
m = mds["tests/fixtures/test.ogg"]
Expect(err).To(BeNil())
Expect(m.Tags).To(HaveKeyWithValue("fbpm", []string{"141.7"}))
// TabLib 1.12 returns 18, previous versions return 39.
// See https://github.com/taglib/taglib/commit/2f238921824741b2cfe6fbfbfc9701d9827ab06b
Expect(m.AudioProperties.BitRate).To(BeElementOf(18, 39, 40, 43, 49))
Expect(m.AudioProperties.Channels).To(BeElementOf(2))
Expect(m.AudioProperties.SampleRate).To(BeElementOf(8000))
Expect(m.AudioProperties.SampleRate).To(BeElementOf(8000))
Expect(m.HasPicture).To(BeFalse())
})
DescribeTable("Format-Specific tests",
func(file, duration string, channels, samplerate, bitdepth int, albumGain, albumPeak, trackGain, trackPeak string, id3Lyrics bool) {
file = "tests/fixtures/" + file
mds, err := e.Parse(file)
Expect(err).NotTo(HaveOccurred())
Expect(mds).To(HaveLen(1))
m := mds[file]
Expect(m.HasPicture).To(BeFalse())
Expect(m.AudioProperties.Duration.String()).To(Equal(duration))
Expect(m.AudioProperties.Channels).To(Equal(channels))
Expect(m.AudioProperties.SampleRate).To(Equal(samplerate))
Expect(m.AudioProperties.BitDepth).To(Equal(bitdepth))
Expect(m.Tags).To(Or(
HaveKeyWithValue("replaygain_album_gain", []string{albumGain}),
HaveKeyWithValue("----:com.apple.itunes:replaygain_track_gain", []string{albumGain}),
))
Expect(m.Tags).To(Or(
HaveKeyWithValue("replaygain_album_peak", []string{albumPeak}),
HaveKeyWithValue("----:com.apple.itunes:replaygain_album_peak", []string{albumPeak}),
))
Expect(m.Tags).To(Or(
HaveKeyWithValue("replaygain_track_gain", []string{trackGain}),
HaveKeyWithValue("----:com.apple.itunes:replaygain_track_gain", []string{trackGain}),
))
Expect(m.Tags).To(Or(
HaveKeyWithValue("replaygain_track_peak", []string{trackPeak}),
HaveKeyWithValue("----:com.apple.itunes:replaygain_track_peak", []string{trackPeak}),
))
Expect(m.Tags).To(HaveKeyWithValue("title", []string{"Title"}))
Expect(m.Tags).To(HaveKeyWithValue("album", []string{"Album"}))
Expect(m.Tags).To(HaveKeyWithValue("artist", []string{"Artist"}))
Expect(m.Tags).To(HaveKeyWithValue("albumartist", []string{"Album Artist"}))
Expect(m.Tags).To(HaveKeyWithValue("genre", []string{"Rock"}))
Expect(m.Tags).To(HaveKeyWithValue("date", []string{"2014"}))
Expect(m.Tags).To(HaveKeyWithValue("bpm", []string{"123"}))
Expect(m.Tags).To(Or(
HaveKeyWithValue("tracknumber", []string{"3"}),
HaveKeyWithValue("tracknumber", []string{"3/10"}),
))
if !strings.HasSuffix(file, "test.wma") {
// TODO Not sure why this is not working for WMA
Expect(m.Tags).To(HaveKeyWithValue("tracktotal", []string{"10"}))
}
Expect(m.Tags).To(Or(
HaveKeyWithValue("discnumber", []string{"1"}),
HaveKeyWithValue("discnumber", []string{"1/2"}),
))
Expect(m.Tags).To(HaveKeyWithValue("disctotal", []string{"2"}))
// WMA does not have a "compilation" tag, but "wm/iscompilation"
Expect(m.Tags).To(Or(
HaveKeyWithValue("compilation", []string{"1"}),
HaveKeyWithValue("wm/iscompilation", []string{"1"})),
)
if id3Lyrics {
Expect(m.Tags).To(HaveKeyWithValue("lyrics:eng", []string{
"[00:00.00]This is\n[00:02.50]English",
}))
Expect(m.Tags).To(HaveKeyWithValue("lyrics:xxx", []string{
"[00:00.00]This is\n[00:02.50]unspecified",
}))
} else {
Expect(m.Tags).To(HaveKeyWithValue("lyrics:xxx", []string{
"[00:00.00]This is\n[00:02.50]unspecified",
"[00:00.00]This is\n[00:02.50]English",
}))
}
Expect(m.Tags).To(HaveKeyWithValue("comment", []string{"Comment1\nComment2"}))
},
// ffmpeg -f lavfi -i "sine=frequency=1200:duration=1" test.flac
Entry("correctly parses flac tags", "test.flac", "1s", 1, 44100, 16, "+4.06 dB", "0.12496948", "+4.06 dB", "0.12496948", false),
Entry("correctly parses m4a (aac) gain tags", "01 Invisible (RED) Edit Version.m4a", "1.04s", 2, 44100, 16, "0.37", "0.48", "0.37", "0.48", false),
Entry("correctly parses m4a (aac) gain tags (uppercase)", "test.m4a", "1.04s", 2, 44100, 16, "0.37", "0.48", "0.37", "0.48", false),
Entry("correctly parses ogg (vorbis) tags", "test.ogg", "1.04s", 2, 8000, 0, "+7.64 dB", "0.11772506", "+7.64 dB", "0.11772506", false),
// ffmpeg -f lavfi -i "sine=frequency=900:duration=1" test.wma
// Weird note: for the tag parsing to work, the lyrics are actually stored in the reverse order
Entry("correctly parses wma/asf tags", "test.wma", "1.02s", 1, 44100, 16, "3.27 dB", "0.132914", "3.27 dB", "0.132914", false),
// ffmpeg -f lavfi -i "sine=frequency=800:duration=1" test.wv
Entry("correctly parses wv (wavpak) tags", "test.wv", "1s", 1, 44100, 16, "3.43 dB", "0.125061", "3.43 dB", "0.125061", false),
// ffmpeg -f lavfi -i "sine=frequency=1000:duration=1" test.wav
Entry("correctly parses wav tags", "test.wav", "1s", 1, 44100, 16, "3.06 dB", "0.125056", "3.06 dB", "0.125056", true),
// ffmpeg -f lavfi -i "sine=frequency=1400:duration=1" test.aiff
Entry("correctly parses aiff tags", "test.aiff", "1s", 1, 44100, 16, "2.00 dB", "0.124972", "2.00 dB", "0.124972", true),
)
// Skip these tests when running as root
Context("Access Forbidden", func() {
var accessForbiddenFile string
var RegularUserContext = XContext
var isRegularUser = os.Getuid() != 0
if isRegularUser {
RegularUserContext = Context
}
// Only run permission tests if we are not root
RegularUserContext("when run without root privileges", func() {
BeforeEach(func() {
accessForbiddenFile = utils.TempFileName("access_forbidden-", ".mp3")
f, err := os.OpenFile(accessForbiddenFile, os.O_WRONLY|os.O_CREATE, 0222)
Expect(err).ToNot(HaveOccurred())
DeferCleanup(func() {
Expect(f.Close()).To(Succeed())
Expect(os.Remove(accessForbiddenFile)).To(Succeed())
})
})
It("correctly handle unreadable file due to insufficient read permission", func() {
_, err := e.extractMetadata(accessForbiddenFile)
Expect(err).To(MatchError(os.ErrPermission))
})
It("skips the file if it cannot be read", func() {
files := []string{
"tests/fixtures/test.mp3",
"tests/fixtures/test.ogg",
accessForbiddenFile,
}
mds, err := e.Parse(files...)
Expect(err).NotTo(HaveOccurred())
Expect(mds).To(HaveLen(2))
Expect(mds).ToNot(HaveKey(accessForbiddenFile))
})
})
})
})
Describe("Error Checking", func() {
It("returns a generic ErrPath if file does not exist", func() {
testFilePath := "tests/fixtures/NON_EXISTENT.ogg"
_, err := e.extractMetadata(testFilePath)
Expect(err).To(MatchError(fs.ErrNotExist))
})
It("does not throw a SIGSEGV error when reading a file with an invalid frame", func() {
// File has an empty TDAT frame
md, err := e.extractMetadata("tests/fixtures/invalid-files/test-invalid-frame.mp3")
Expect(err).ToNot(HaveOccurred())
Expect(md.Tags).To(HaveKeyWithValue("albumartist", []string{"Elvis Presley"}))
})
})
Describe("parseTIPL", func() {
var tags map[string][]string
BeforeEach(func() {
tags = make(map[string][]string)
})
Context("when the TIPL string is populated", func() {
It("correctly parses roles and names", func() {
tags["tipl"] = []string{"arranger Andrew Powell DJ-mix François Kevorkian DJ-mix Jane Doe engineer Chris Blair"}
parseTIPL(tags)
Expect(tags["arranger"]).To(ConsistOf("Andrew Powell"))
Expect(tags["engineer"]).To(ConsistOf("Chris Blair"))
Expect(tags["djmixer"]).To(ConsistOf("François Kevorkian", "Jane Doe"))
})
It("handles multiple names for a single role", func() {
tags["tipl"] = []string{"engineer Pat Stapley producer Eric Woolfson engineer Chris Blair"}
parseTIPL(tags)
Expect(tags["producer"]).To(ConsistOf("Eric Woolfson"))
Expect(tags["engineer"]).To(ConsistOf("Pat Stapley", "Chris Blair"))
})
It("discards roles without names", func() {
tags["tipl"] = []string{"engineer Pat Stapley producer engineer Chris Blair"}
parseTIPL(tags)
Expect(tags).ToNot(HaveKey("producer"))
Expect(tags["engineer"]).To(ConsistOf("Pat Stapley", "Chris Blair"))
})
})
Context("when the TIPL string is empty", func() {
It("does nothing", func() {
tags["tipl"] = []string{""}
parseTIPL(tags)
Expect(tags).To(BeEmpty())
})
})
Context("when the TIPL is not present", func() {
It("does nothing", func() {
parseTIPL(tags)
Expect(tags).To(BeEmpty())
})
})
})
})

View File

@@ -3,8 +3,11 @@
#include <typeinfo>
#define TAGLIB_STATIC
#include <apeproperties.h>
#include <apetag.h>
#include <aifffile.h>
#include <asffile.h>
#include <dsffile.h>
#include <fileref.h>
#include <flacfile.h>
#include <id3v2tag.h>
@@ -16,6 +19,8 @@
#include <tpropertymap.h>
#include <vorbisfile.h>
#include <wavfile.h>
#include <wavfile.h>
#include <wavpackfile.h>
#include "taglib_wrapper.h"
@@ -41,35 +46,31 @@ int taglib_read(const FILENAME_CHAR_T *filename, unsigned long id) {
// Add audio properties to the tags
const TagLib::AudioProperties *props(f.audioProperties());
go_map_put_int(id, (char *)"duration", props->lengthInSeconds());
go_map_put_int(id, (char *)"lengthinmilliseconds", props->lengthInMilliseconds());
go_map_put_int(id, (char *)"bitrate", props->bitrate());
go_map_put_int(id, (char *)"channels", props->channels());
go_map_put_int(id, (char *)"samplerate", props->sampleRate());
goPutInt(id, (char *)"_lengthinmilliseconds", props->lengthInMilliseconds());
goPutInt(id, (char *)"_bitrate", props->bitrate());
goPutInt(id, (char *)"_channels", props->channels());
goPutInt(id, (char *)"_samplerate", props->sampleRate());
// Create a map to collect all the tags
if (const auto* apeProperties{ dynamic_cast<const TagLib::APE::Properties*>(props) })
goPutInt(id, (char *)"_bitspersample", apeProperties->bitsPerSample());
if (const auto* asfProperties{ dynamic_cast<const TagLib::ASF::Properties*>(props) })
goPutInt(id, (char *)"_bitspersample", asfProperties->bitsPerSample());
else if (const auto* flacProperties{ dynamic_cast<const TagLib::FLAC::Properties*>(props) })
goPutInt(id, (char *)"_bitspersample", flacProperties->bitsPerSample());
else if (const auto* mp4Properties{ dynamic_cast<const TagLib::MP4::Properties*>(props) })
goPutInt(id, (char *)"_bitspersample", mp4Properties->bitsPerSample());
else if (const auto* wavePackProperties{ dynamic_cast<const TagLib::WavPack::Properties*>(props) })
goPutInt(id, (char *)"_bitspersample", wavePackProperties->bitsPerSample());
else if (const auto* aiffProperties{ dynamic_cast<const TagLib::RIFF::AIFF::Properties*>(props) })
goPutInt(id, (char *)"_bitspersample", aiffProperties->bitsPerSample());
else if (const auto* wavProperties{ dynamic_cast<const TagLib::RIFF::WAV::Properties*>(props) })
goPutInt(id, (char *)"_bitspersample", wavProperties->bitsPerSample());
else if (const auto* dsfProperties{ dynamic_cast<const TagLib::DSF::Properties*>(props) })
goPutInt(id, (char *)"_bitspersample", dsfProperties->bitsPerSample());
// Send all properties to the Go map
TagLib::PropertyMap tags = f.file()->properties();
// Make sure at least the basic properties are extracted
TagLib::Tag *basic = f.file()->tag();
if (!basic->isEmpty()) {
if (!basic->title().isEmpty()) {
tags.insert("title", basic->title());
}
if (!basic->artist().isEmpty()) {
tags.insert("artist", basic->artist());
}
if (!basic->album().isEmpty()) {
tags.insert("album", basic->album());
}
if (basic->year() > 0) {
tags.insert("date", TagLib::String::number(basic->year()));
}
if (basic->track() > 0) {
tags.insert("_track", TagLib::String::number(basic->track()));
}
}
TagLib::ID3v2::Tag *id3Tags = NULL;
// Get some extended/non-standard ID3-only tags (ex: iTunes extended frames)
@@ -114,7 +115,7 @@ int taglib_read(const FILENAME_CHAR_T *filename, unsigned long id) {
char *val = (char *)frame->text().toCString(true);
go_map_put_lyrics(id, language, val);
goPutLyrics(id, language, val);
}
} else if (kv.first == "SYLT") {
for (const auto &tag: kv.second) {
@@ -132,7 +133,7 @@ int taglib_read(const FILENAME_CHAR_T *filename, unsigned long id) {
for (const auto &line: frame->synchedText()) {
char *text = (char *)line.text.toCString(true);
go_map_put_lyric_line(id, language, text, line.time);
goPutLyricLine(id, language, text, line.time);
}
} else if (format == TagLib::ID3v2::SynchronizedLyricsFrame::AbsoluteMpegFrames) {
const int sampleRate = props->sampleRate();
@@ -141,12 +142,12 @@ int taglib_read(const FILENAME_CHAR_T *filename, unsigned long id) {
for (const auto &line: frame->synchedText()) {
const int timeInMs = (line.time * 1000) / sampleRate;
char *text = (char *)line.text.toCString(true);
go_map_put_lyric_line(id, language, text, timeInMs);
goPutLyricLine(id, language, text, timeInMs);
}
}
}
}
} else {
} else if (kv.first == "TIPL"){
if (!kv.second.isEmpty()) {
tags.insert(kv.first, kv.second.front()->toString());
}
@@ -154,7 +155,7 @@ int taglib_read(const FILENAME_CHAR_T *filename, unsigned long id) {
}
}
// M4A may have some iTunes specific tags
// M4A may have some iTunes specific tags not captured by the PropertyMap interface
TagLib::MP4::File *m4afile(dynamic_cast<TagLib::MP4::File *>(f.file()));
if (m4afile != NULL) {
const auto itemListMap = m4afile->tag()->itemMap();
@@ -162,12 +163,12 @@ int taglib_read(const FILENAME_CHAR_T *filename, unsigned long id) {
char *key = (char *)item.first.toCString(true);
for (const auto value: item.second.toStringList()) {
char *val = (char *)value.toCString(true);
go_map_put_m4a_str(id, key, val);
goPutM4AStr(id, key, val);
}
}
}
// WMA/ASF files may have additional tags not captured by the general iterator
// WMA/ASF files may have additional tags not captured by the PropertyMap interface
TagLib::ASF::File *asfFile(dynamic_cast<TagLib::ASF::File *>(f.file()));
if (asfFile != NULL) {
const TagLib::ASF::Tag *asfTags{asfFile->tag()};
@@ -184,13 +185,13 @@ int taglib_read(const FILENAME_CHAR_T *filename, unsigned long id) {
for (TagLib::StringList::ConstIterator j = i->second.begin();
j != i->second.end(); ++j) {
char *val = (char *)(*j).toCString(true);
go_map_put_str(id, key, val);
goPutStr(id, key, val);
}
}
// Cover art has to be handled separately
if (has_cover(f)) {
go_map_put_str(id, (char *)"has_picture", (char *)"true");
goPutStr(id, (char *)"has_picture", (char *)"true");
}
return 0;
@@ -200,41 +201,42 @@ int taglib_read(const FILENAME_CHAR_T *filename, unsigned long id) {
char has_cover(const TagLib::FileRef f) {
char hasCover = 0;
// ----- MP3
if (TagLib::MPEG::File *
mp3File{dynamic_cast<TagLib::MPEG::File *>(f.file())}) {
if (TagLib::MPEG::File * mp3File{dynamic_cast<TagLib::MPEG::File *>(f.file())}) {
if (mp3File->ID3v2Tag()) {
const auto &frameListMap{mp3File->ID3v2Tag()->frameListMap()};
hasCover = !frameListMap["APIC"].isEmpty();
}
}
// ----- FLAC
else if (TagLib::FLAC::File *
flacFile{dynamic_cast<TagLib::FLAC::File *>(f.file())}) {
else if (TagLib::FLAC::File * flacFile{dynamic_cast<TagLib::FLAC::File *>(f.file())}) {
hasCover = !flacFile->pictureList().isEmpty();
}
// ----- MP4
else if (TagLib::MP4::File *
mp4File{dynamic_cast<TagLib::MP4::File *>(f.file())}) {
else if (TagLib::MP4::File * mp4File{dynamic_cast<TagLib::MP4::File *>(f.file())}) {
auto &coverItem{mp4File->tag()->itemMap()["covr"]};
TagLib::MP4::CoverArtList coverArtList{coverItem.toCoverArtList()};
hasCover = !coverArtList.isEmpty();
}
// ----- Ogg
else if (TagLib::Ogg::Vorbis::File *
vorbisFile{dynamic_cast<TagLib::Ogg::Vorbis::File *>(f.file())}) {
else if (TagLib::Ogg::Vorbis::File * vorbisFile{dynamic_cast<TagLib::Ogg::Vorbis::File *>(f.file())}) {
hasCover = !vorbisFile->tag()->pictureList().isEmpty();
}
// ----- Opus
else if (TagLib::Ogg::Opus::File *
opusFile{dynamic_cast<TagLib::Ogg::Opus::File *>(f.file())}) {
else if (TagLib::Ogg::Opus::File * opusFile{dynamic_cast<TagLib::Ogg::Opus::File *>(f.file())}) {
hasCover = !opusFile->tag()->pictureList().isEmpty();
}
// ----- WMA
if (TagLib::ASF::File *
asfFile{dynamic_cast<TagLib::ASF::File *>(f.file())}) {
else if (TagLib::ASF::File * asfFile{dynamic_cast<TagLib::ASF::File *>(f.file())}) {
const TagLib::ASF::Tag *tag{asfFile->tag()};
hasCover = tag && tag->attributeListMap().contains("WM/Picture");
}
// ----- WAV
else if (TagLib::RIFF::WAV::File * wavFile{ dynamic_cast<TagLib::RIFF::WAV::File*>(f.file()) }) {
if (wavFile->hasID3v2Tag()) {
const auto& frameListMap{ wavFile->ID3v2Tag()->frameListMap() };
hasCover = !frameListMap["APIC"].isEmpty();
}
}
return hasCover;
}

View File

@@ -0,0 +1,157 @@
package taglib
/*
#cgo !windows pkg-config: --define-prefix taglib
#cgo windows pkg-config: taglib
#cgo illumos LDFLAGS: -lstdc++ -lsendfile
#cgo linux darwin CXXFLAGS: -std=c++11
#cgo darwin LDFLAGS: -L/opt/homebrew/opt/taglib/lib
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "taglib_wrapper.h"
*/
import "C"
import (
"encoding/json"
"fmt"
"os"
"runtime/debug"
"strconv"
"strings"
"sync"
"sync/atomic"
"unsafe"
"github.com/navidrome/navidrome/log"
)
const iTunesKeyPrefix = "----:com.apple.itunes:"
func Version() string {
return C.GoString(C.taglib_version())
}
func Read(filename string) (tags map[string][]string, err error) {
// Do not crash on failures in the C code/library
debug.SetPanicOnFault(true)
defer func() {
if r := recover(); r != nil {
log.Error("extractor: recovered from panic when reading tags", "file", filename, "error", r)
err = fmt.Errorf("extractor: recovered from panic: %s", r)
}
}()
fp := getFilename(filename)
defer C.free(unsafe.Pointer(fp))
id, m, release := newMap()
defer release()
log.Trace("extractor: reading tags", "filename", filename, "map_id", id)
res := C.taglib_read(fp, C.ulong(id))
switch res {
case C.TAGLIB_ERR_PARSE:
// Check additional case whether the file is unreadable due to permission
file, fileErr := os.OpenFile(filename, os.O_RDONLY, 0600)
defer file.Close()
if os.IsPermission(fileErr) {
return nil, fmt.Errorf("navidrome does not have permission: %w", fileErr)
} else if fileErr != nil {
return nil, fmt.Errorf("cannot parse file media file: %w", fileErr)
} else {
return nil, fmt.Errorf("cannot parse file media file")
}
case C.TAGLIB_ERR_AUDIO_PROPS:
return nil, fmt.Errorf("can't get audio properties from file")
}
if log.IsGreaterOrEqualTo(log.LevelDebug) {
j, _ := json.Marshal(m)
log.Trace("extractor: read tags", "tags", string(j), "filename", filename, "id", id)
} else {
log.Trace("extractor: read tags", "tags", m, "filename", filename, "id", id)
}
return m, nil
}
type tagMap map[string][]string
var allMaps sync.Map
var mapsNextID atomic.Uint32
func newMap() (uint32, tagMap, func()) {
id := mapsNextID.Add(1)
m := tagMap{}
allMaps.Store(id, m)
return id, m, func() {
allMaps.Delete(id)
}
}
func doPutTag(id C.ulong, key string, val *C.char) {
if key == "" {
return
}
r, _ := allMaps.Load(uint32(id))
m := r.(tagMap)
k := strings.ToLower(key)
v := strings.TrimSpace(C.GoString(val))
m[k] = append(m[k], v)
}
//export goPutM4AStr
func goPutM4AStr(id C.ulong, key *C.char, val *C.char) {
k := C.GoString(key)
// Special for M4A, do not catch keys that have no actual name
k = strings.TrimPrefix(k, iTunesKeyPrefix)
doPutTag(id, k, val)
}
//export goPutStr
func goPutStr(id C.ulong, key *C.char, val *C.char) {
doPutTag(id, C.GoString(key), val)
}
//export goPutInt
func goPutInt(id C.ulong, key *C.char, val C.int) {
valStr := strconv.Itoa(int(val))
vp := C.CString(valStr)
defer C.free(unsafe.Pointer(vp))
goPutStr(id, key, vp)
}
//export goPutLyrics
func goPutLyrics(id C.ulong, lang *C.char, val *C.char) {
doPutTag(id, "lyrics:"+C.GoString(lang), val)
}
//export goPutLyricLine
func goPutLyricLine(id C.ulong, lang *C.char, text *C.char, time C.int) {
language := C.GoString(lang)
line := C.GoString(text)
timeGo := int64(time)
ms := timeGo % 1000
timeGo /= 1000
sec := timeGo % 60
timeGo /= 60
minimum := timeGo % 60
formattedLine := fmt.Sprintf("[%02d:%02d.%02d]%s\n", minimum, sec, ms/10, line)
key := "lyrics:" + language
r, _ := allMaps.Load(uint32(id))
m := r.(tagMap)
k := strings.ToLower(key)
existing, ok := m[k]
if ok {
existing[0] += formattedLine
} else {
m[k] = []string{formattedLine}
}
}

View File

@@ -0,0 +1,24 @@
#define TAGLIB_ERR_PARSE -1
#define TAGLIB_ERR_AUDIO_PROPS -2
#ifdef __cplusplus
extern "C" {
#endif
#ifdef WIN32
#define FILENAME_CHAR_T wchar_t
#else
#define FILENAME_CHAR_T char
#endif
extern void goPutM4AStr(unsigned long id, char *key, char *val);
extern void goPutStr(unsigned long id, char *key, char *val);
extern void goPutInt(unsigned long id, char *key, int val);
extern void goPutLyrics(unsigned long id, char *lang, char *val);
extern void goPutLyricLine(unsigned long id, char *lang, char *text, int time);
int taglib_read(const FILENAME_CHAR_T *filename, unsigned long id);
char* taglib_version();
#ifdef __cplusplus
}
#endif

View File

@@ -5,25 +5,20 @@ import (
"fmt"
"strings"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/scanner"
"github.com/navidrome/navidrome/scanner/metadata"
"github.com/navidrome/navidrome/tests"
"github.com/pelletier/go-toml/v2"
"github.com/spf13/cobra"
"gopkg.in/yaml.v3"
)
var (
extractor string
format string
format string
)
func init() {
inspectCmd.Flags().StringVarP(&extractor, "extractor", "x", "", "extractor to use (ffmpeg or taglib, default: auto)")
inspectCmd.Flags().StringVarP(&format, "format", "f", "pretty", "output format (pretty, toml, yaml, json, jsonindent)")
inspectCmd.Flags().StringVarP(&format, "format", "f", "jsonindent", "output format (pretty, toml, yaml, json, jsonindent)")
rootCmd.AddCommand(inspectCmd)
}
@@ -48,7 +43,7 @@ var marshalers = map[string]func(interface{}) ([]byte, error){
}
func prettyMarshal(v interface{}) ([]byte, error) {
out := v.([]inspectorOutput)
out := v.([]core.InspectOutput)
var res strings.Builder
for i := range out {
res.WriteString(fmt.Sprintf("====================\nFile: %s\n\n", out[i].File))
@@ -60,39 +55,24 @@ func prettyMarshal(v interface{}) ([]byte, error) {
return []byte(res.String()), nil
}
type inspectorOutput struct {
File string
RawTags metadata.ParsedTags
MappedTags model.MediaFile
}
func runInspector(args []string) {
if extractor != "" {
conf.Server.Scanner.Extractor = extractor
}
log.Info("Using extractor", "extractor", conf.Server.Scanner.Extractor)
md, err := metadata.Extract(args...)
if err != nil {
log.Fatal("Error extracting tags", err)
}
mapper := scanner.NewMediaFileMapper(conf.Server.MusicFolder, &tests.MockedGenreRepo{})
marshal := marshalers[format]
if marshal == nil {
log.Fatal("Invalid format", "format", format)
}
var out []inspectorOutput
for k, v := range md {
if !model.IsAudioFile(k) {
var out []core.InspectOutput
for _, filePath := range args {
if !model.IsAudioFile(filePath) {
log.Warn("Not an audio file", "file", filePath)
continue
}
if len(v.Tags) == 0 {
output, err := core.Inspect(filePath, 1, "")
if err != nil {
log.Warn("Unable to process file", "file", filePath, "error", err)
continue
}
out = append(out, inspectorOutput{
File: k,
RawTags: v.Tags,
MappedTags: mapper.ToMediaFile(v),
})
out = append(out, *output)
}
data, _ := marshal(out)
fmt.Println(string(data))

View File

@@ -69,7 +69,7 @@ func runExporter() {
sqlDB := db.Db()
ds := persistence.New(sqlDB)
ctx := auth.WithAdminUser(context.Background(), ds)
playlist, err := ds.Playlist(ctx).GetWithTracks(playlistID, true)
playlist, err := ds.Playlist(ctx).GetWithTracks(playlistID, true, false)
if err != nil && !errors.Is(err, model.ErrNotFound) {
log.Fatal("Error retrieving playlist", "name", playlistID, err)
}
@@ -79,7 +79,7 @@ func runExporter() {
log.Fatal("Error retrieving playlist", "name", playlistID, err)
}
if len(playlists) > 0 {
playlist, err = ds.Playlist(ctx).GetWithTracks(playlists[0].ID, true)
playlist, err = ds.Playlist(ctx).GetWithTracks(playlists[0].ID, true, false)
if err != nil {
log.Fatal("Error retrieving playlist", "name", playlistID, err)
}

View File

@@ -9,11 +9,14 @@ import (
"time"
"github.com/go-chi/chi/v5/middleware"
_ "github.com/navidrome/navidrome/adapters/taglib"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/db"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/resources"
"github.com/navidrome/navidrome/scanner"
"github.com/navidrome/navidrome/scheduler"
"github.com/navidrome/navidrome/server/backgrounds"
"github.com/spf13/cobra"
@@ -45,8 +48,11 @@ Complete documentation is available at https://www.navidrome.org/docs`,
// Execute runs the root cobra command, which will start the Navidrome server by calling the runNavidrome function.
func Execute() {
ctx, cancel := mainContext(context.Background())
defer cancel()
rootCmd.SetVersionTemplate(`{{println .Version}}`)
if err := rootCmd.Execute(); err != nil {
if err := rootCmd.ExecuteContext(ctx); err != nil {
log.Fatal(err)
}
}
@@ -55,7 +61,7 @@ func preRun() {
if !noBanner {
println(resources.Banner())
}
conf.Load()
conf.Load(noBanner)
}
func postRun() {
@@ -66,19 +72,23 @@ func postRun() {
// If any of the services returns an error, it will log it and exit. If the process receives a signal to exit,
// it will cancel the context and exit gracefully.
func runNavidrome(ctx context.Context) {
defer db.Init()()
ctx, cancel := mainContext(ctx)
defer cancel()
defer db.Init(ctx)()
g, ctx := errgroup.WithContext(ctx)
g.Go(startServer(ctx))
g.Go(startSignaller(ctx))
g.Go(startScheduler(ctx))
g.Go(startPlaybackServer(ctx))
g.Go(schedulePeriodicScan(ctx))
g.Go(schedulePeriodicBackup(ctx))
g.Go(startInsightsCollector(ctx))
g.Go(scheduleDBOptimizer(ctx))
if conf.Server.Scanner.Enabled {
g.Go(runInitialScan(ctx))
g.Go(startScanWatcher(ctx))
g.Go(schedulePeriodicScan(ctx))
} else {
log.Warn(ctx, "Automatic Scanning is DISABLED")
}
if err := g.Wait(); err != nil {
log.Error("Fatal error in Navidrome. Aborting", err)
@@ -98,9 +108,9 @@ func mainContext(ctx context.Context) (context.Context, context.CancelFunc) {
// startServer starts the Navidrome web server, adding all the necessary routers.
func startServer(ctx context.Context) func() error {
return func() error {
a := CreateServer(conf.Server.MusicFolder)
a := CreateServer()
a.MountRouter("Native API", consts.URLPathNativeAPI, CreateNativeAPIRouter())
a.MountRouter("Subsonic API", consts.URLPathSubsonicAPI, CreateSubsonicAPIRouter())
a.MountRouter("Subsonic API", consts.URLPathSubsonicAPI, CreateSubsonicAPIRouter(ctx))
a.MountRouter("Public Endpoints", consts.URLPathPublic, CreatePublicRouter())
if conf.Server.LastFM.Enabled {
a.MountRouter("LastFM Auth", consts.URLPathNativeAPI+"/lastfm", CreateLastFMRouter())
@@ -127,29 +137,97 @@ func startServer(ctx context.Context) func() error {
// schedulePeriodicScan schedules a periodic scan of the music library, if configured.
func schedulePeriodicScan(ctx context.Context) func() error {
return func() error {
schedule := conf.Server.ScanSchedule
schedule := conf.Server.Scanner.Schedule
if schedule == "" {
log.Warn("Periodic scan is DISABLED")
log.Info(ctx, "Periodic scan is DISABLED")
return nil
}
scanner := GetScanner()
s := CreateScanner(ctx)
schedulerInstance := scheduler.GetInstance()
log.Info("Scheduling periodic scan", "schedule", schedule)
err := schedulerInstance.Add(schedule, func() {
_ = scanner.RescanAll(ctx, false)
_, err := s.ScanAll(ctx, false)
if err != nil {
log.Error(ctx, "Error executing periodic scan", err)
}
})
if err != nil {
log.Error("Error scheduling periodic scan", err)
log.Error(ctx, "Error scheduling periodic scan", err)
}
return nil
}
}
time.Sleep(2 * time.Second) // Wait 2 seconds before the initial scan
log.Debug("Executing initial scan")
if err := scanner.RescanAll(ctx, false); err != nil {
log.Error("Error executing initial scan", err)
func pidHashChanged(ds model.DataStore) (bool, error) {
pidAlbum, err := ds.Property(context.Background()).DefaultGet(consts.PIDAlbumKey, "")
if err != nil {
return false, err
}
pidTrack, err := ds.Property(context.Background()).DefaultGet(consts.PIDTrackKey, "")
if err != nil {
return false, err
}
return !strings.EqualFold(pidAlbum, conf.Server.PID.Album) || !strings.EqualFold(pidTrack, conf.Server.PID.Track), nil
}
func runInitialScan(ctx context.Context) func() error {
return func() error {
ds := CreateDataStore()
fullScanRequired, err := ds.Property(ctx).DefaultGet(consts.FullScanAfterMigrationFlagKey, "0")
if err != nil {
return err
}
inProgress, err := ds.Library(ctx).ScanInProgress()
if err != nil {
return err
}
pidHasChanged, err := pidHashChanged(ds)
if err != nil {
return err
}
scanNeeded := conf.Server.Scanner.ScanOnStartup || inProgress || fullScanRequired == "1" || pidHasChanged
time.Sleep(2 * time.Second) // Wait 2 seconds before the initial scan
if scanNeeded {
scanner := CreateScanner(ctx)
switch {
case fullScanRequired == "1":
log.Warn(ctx, "Full scan required after migration")
_ = ds.Property(ctx).Delete(consts.FullScanAfterMigrationFlagKey)
case pidHasChanged:
log.Warn(ctx, "PID config changed, performing full scan")
fullScanRequired = "1"
case inProgress:
log.Warn(ctx, "Resuming interrupted scan")
default:
log.Info("Executing initial scan")
}
_, err = scanner.ScanAll(ctx, fullScanRequired == "1")
if err != nil {
log.Error(ctx, "Scan failed", err)
} else {
log.Info(ctx, "Scan completed")
}
} else {
log.Debug(ctx, "Initial scan not needed")
}
return nil
}
}
func startScanWatcher(ctx context.Context) func() error {
return func() error {
if conf.Server.Scanner.WatcherWait == 0 {
log.Debug("Folder watcher is DISABLED")
return nil
}
w := CreateScanWatcher(ctx)
err := w.Run(ctx)
if err != nil {
log.Error("Error starting watcher", err)
}
log.Debug("Finished initial scan")
return nil
}
}
@@ -158,7 +236,7 @@ func schedulePeriodicBackup(ctx context.Context) func() error {
return func() error {
schedule := conf.Server.Backup.Schedule
if schedule == "" {
log.Warn("Periodic backup is DISABLED")
log.Info(ctx, "Periodic backup is DISABLED")
return nil
}
@@ -189,6 +267,21 @@ func schedulePeriodicBackup(ctx context.Context) func() error {
}
}
func scheduleDBOptimizer(ctx context.Context) func() error {
return func() error {
log.Info(ctx, "Scheduling DB optimizer", "schedule", consts.OptimizeDBSchedule)
schedulerInstance := scheduler.GetInstance()
err := schedulerInstance.Add(consts.OptimizeDBSchedule, func() {
if scanner.IsScanning() {
log.Debug(ctx, "Skipping DB optimization because a scan is in progress")
return
}
db.Optimize(ctx)
})
return err
}
}
// startScheduler starts the Navidrome scheduler, which is used to run periodic tasks.
func startScheduler(ctx context.Context) func() error {
return func() error {

View File

@@ -2,15 +2,28 @@ package cmd
import (
"context"
"encoding/gob"
"os"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/artwork"
"github.com/navidrome/navidrome/core/metrics"
"github.com/navidrome/navidrome/db"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/persistence"
"github.com/navidrome/navidrome/scanner"
"github.com/navidrome/navidrome/utils/pl"
"github.com/spf13/cobra"
)
var fullRescan bool
var (
fullScan bool
subprocess bool
)
func init() {
scanCmd.Flags().BoolVarP(&fullRescan, "full", "f", false, "check all subfolders, ignoring timestamps")
scanCmd.Flags().BoolVarP(&fullScan, "full", "f", false, "check all subfolders, ignoring timestamps")
scanCmd.Flags().BoolVarP(&subprocess, "subprocess", "", false, "run as subprocess (internal use)")
rootCmd.AddCommand(scanCmd)
}
@@ -19,16 +32,53 @@ var scanCmd = &cobra.Command{
Short: "Scan music folder",
Long: "Scan music folder for updates",
Run: func(cmd *cobra.Command, args []string) {
runScanner()
runScanner(cmd.Context())
},
}
func runScanner() {
scanner := GetScanner()
_ = scanner.RescanAll(context.Background(), fullRescan)
if fullRescan {
func trackScanInteractively(ctx context.Context, progress <-chan *scanner.ProgressInfo) {
for status := range pl.ReadOrDone(ctx, progress) {
if status.Warning != "" {
log.Warn(ctx, "Scan warning", "error", status.Warning)
}
if status.Error != "" {
log.Error(ctx, "Scan error", "error", status.Error)
}
// Discard the progress status, we only care about errors
}
if fullScan {
log.Info("Finished full rescan")
} else {
log.Info("Finished rescan")
}
}
func trackScanAsSubprocess(ctx context.Context, progress <-chan *scanner.ProgressInfo) {
encoder := gob.NewEncoder(os.Stdout)
for status := range pl.ReadOrDone(ctx, progress) {
err := encoder.Encode(status)
if err != nil {
log.Error(ctx, "Failed to encode status", err)
}
}
}
func runScanner(ctx context.Context) {
sqlDB := db.Db()
defer db.Db().Close()
ds := persistence.New(sqlDB)
pls := core.NewPlaylists(ds)
progress, err := scanner.CallScan(ctx, ds, artwork.NoopCacheWarmer(), pls, metrics.NewNoopInstance(), fullScan)
if err != nil {
log.Fatal(ctx, "Failed to scan", err)
}
// Wait for the scanner to finish
if subprocess {
trackScanAsSubprocess(ctx, progress)
} else {
trackScanInteractively(ctx, progress)
}
}

View File

@@ -16,7 +16,7 @@ const triggerScanSignal = syscall.SIGUSR1
func startSignaller(ctx context.Context) func() error {
log.Info(ctx, "Starting signaler")
scanner := GetScanner()
scanner := CreateScanner(ctx)
return func() error {
var sigChan = make(chan os.Signal, 1)
@@ -27,11 +27,11 @@ func startSignaller(ctx context.Context) func() error {
case sig := <-sigChan:
log.Info(ctx, "Received signal, triggering a new scan", "signal", sig)
start := time.Now()
err := scanner.RescanAll(ctx, false)
_, err := scanner.ScanAll(ctx, false)
if err != nil {
log.Error(ctx, "Error scanning", err)
}
log.Info(ctx, "Triggered scan complete", "elapsed", time.Since(start).Round(100*time.Millisecond))
log.Info(ctx, "Triggered scan complete", "elapsed", time.Since(start))
case <-ctx.Done():
return nil
}

View File

@@ -7,17 +7,20 @@
package cmd
import (
"context"
"github.com/google/wire"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/core/agents/lastfm"
"github.com/navidrome/navidrome/core/agents/listenbrainz"
"github.com/navidrome/navidrome/core/artwork"
"github.com/navidrome/navidrome/core/external"
"github.com/navidrome/navidrome/core/ffmpeg"
"github.com/navidrome/navidrome/core/metrics"
"github.com/navidrome/navidrome/core/playback"
"github.com/navidrome/navidrome/core/scrobbler"
"github.com/navidrome/navidrome/db"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/persistence"
"github.com/navidrome/navidrome/scanner"
"github.com/navidrome/navidrome/server"
@@ -27,9 +30,19 @@ import (
"github.com/navidrome/navidrome/server/subsonic"
)
import (
_ "github.com/navidrome/navidrome/adapters/taglib"
)
// Injectors from wire_injectors.go:
func CreateServer(musicFolder string) *server.Server {
func CreateDataStore() model.DataStore {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
return dataStore
}
func CreateServer() *server.Server {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
broker := events.GetBroker()
@@ -48,27 +61,27 @@ func CreateNativeAPIRouter() *nativeapi.Router {
return router
}
func CreateSubsonicAPIRouter() *subsonic.Router {
func CreateSubsonicAPIRouter(ctx context.Context) *subsonic.Router {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
fileCache := artwork.GetImageCache()
fFmpeg := ffmpeg.New()
agentsAgents := agents.New(dataStore)
externalMetadata := core.NewExternalMetadata(dataStore, agentsAgents)
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, externalMetadata)
agentsAgents := agents.GetAgents(dataStore)
provider := external.NewProvider(dataStore, agentsAgents)
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, provider)
transcodingCache := core.GetTranscodingCache()
mediaStreamer := core.NewMediaStreamer(dataStore, fFmpeg, transcodingCache)
share := core.NewShare(dataStore)
archiver := core.NewArchiver(mediaStreamer, dataStore, share)
players := core.NewPlayers(dataStore)
playlists := core.NewPlaylists(dataStore)
cacheWarmer := artwork.NewCacheWarmer(artworkArtwork, fileCache)
broker := events.GetBroker()
playlists := core.NewPlaylists(dataStore)
metricsMetrics := metrics.NewPrometheusInstance(dataStore)
scannerScanner := scanner.GetInstance(dataStore, playlists, cacheWarmer, broker, metricsMetrics)
scannerScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlists, metricsMetrics)
playTracker := scrobbler.GetPlayTracker(dataStore, broker)
playbackServer := playback.GetInstance(dataStore)
router := subsonic.New(dataStore, artworkArtwork, mediaStreamer, archiver, players, externalMetadata, scannerScanner, broker, playlists, playTracker, share, playbackServer)
router := subsonic.New(dataStore, artworkArtwork, mediaStreamer, archiver, players, provider, scannerScanner, broker, playlists, playTracker, share, playbackServer)
return router
}
@@ -77,9 +90,9 @@ func CreatePublicRouter() *public.Router {
dataStore := persistence.New(sqlDB)
fileCache := artwork.GetImageCache()
fFmpeg := ffmpeg.New()
agentsAgents := agents.New(dataStore)
externalMetadata := core.NewExternalMetadata(dataStore, agentsAgents)
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, externalMetadata)
agentsAgents := agents.GetAgents(dataStore)
provider := external.NewProvider(dataStore, agentsAgents)
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, provider)
transcodingCache := core.GetTranscodingCache()
mediaStreamer := core.NewMediaStreamer(dataStore, fFmpeg, transcodingCache)
share := core.NewShare(dataStore)
@@ -116,22 +129,39 @@ func CreatePrometheus() metrics.Metrics {
return metricsMetrics
}
func GetScanner() scanner.Scanner {
func CreateScanner(ctx context.Context) scanner.Scanner {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
playlists := core.NewPlaylists(dataStore)
fileCache := artwork.GetImageCache()
fFmpeg := ffmpeg.New()
agentsAgents := agents.New(dataStore)
externalMetadata := core.NewExternalMetadata(dataStore, agentsAgents)
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, externalMetadata)
agentsAgents := agents.GetAgents(dataStore)
provider := external.NewProvider(dataStore, agentsAgents)
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, provider)
cacheWarmer := artwork.NewCacheWarmer(artworkArtwork, fileCache)
broker := events.GetBroker()
playlists := core.NewPlaylists(dataStore)
metricsMetrics := metrics.NewPrometheusInstance(dataStore)
scannerScanner := scanner.GetInstance(dataStore, playlists, cacheWarmer, broker, metricsMetrics)
scannerScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlists, metricsMetrics)
return scannerScanner
}
func CreateScanWatcher(ctx context.Context) scanner.Watcher {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
fileCache := artwork.GetImageCache()
fFmpeg := ffmpeg.New()
agentsAgents := agents.GetAgents(dataStore)
provider := external.NewProvider(dataStore, agentsAgents)
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, provider)
cacheWarmer := artwork.NewCacheWarmer(artworkArtwork, fileCache)
broker := events.GetBroker()
playlists := core.NewPlaylists(dataStore)
metricsMetrics := metrics.NewPrometheusInstance(dataStore)
scannerScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlists, metricsMetrics)
watcher := scanner.NewWatcher(dataStore, scannerScanner)
return watcher
}
func GetPlaybackServer() playback.PlaybackServer {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
@@ -141,4 +171,4 @@ func GetPlaybackServer() playback.PlaybackServer {
// wire_injectors.go:
var allProviders = wire.NewSet(core.Set, artwork.Set, server.New, subsonic.New, nativeapi.New, public.New, persistence.New, lastfm.NewRouter, listenbrainz.NewRouter, events.GetBroker, scanner.GetInstance, db.Db, metrics.NewPrometheusInstance)
var allProviders = wire.NewSet(core.Set, artwork.Set, server.New, subsonic.New, nativeapi.New, public.New, persistence.New, lastfm.NewRouter, listenbrainz.NewRouter, events.GetBroker, scanner.New, scanner.NewWatcher, metrics.NewPrometheusInstance, db.Db)

View File

@@ -3,6 +3,8 @@
package cmd
import (
"context"
"github.com/google/wire"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/agents/lastfm"
@@ -11,6 +13,7 @@ import (
"github.com/navidrome/navidrome/core/metrics"
"github.com/navidrome/navidrome/core/playback"
"github.com/navidrome/navidrome/db"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/persistence"
"github.com/navidrome/navidrome/scanner"
"github.com/navidrome/navidrome/server"
@@ -31,12 +34,19 @@ var allProviders = wire.NewSet(
lastfm.NewRouter,
listenbrainz.NewRouter,
events.GetBroker,
scanner.GetInstance,
db.Db,
scanner.New,
scanner.NewWatcher,
metrics.NewPrometheusInstance,
db.Db,
)
func CreateServer(musicFolder string) *server.Server {
func CreateDataStore() model.DataStore {
panic(wire.Build(
allProviders,
))
}
func CreateServer() *server.Server {
panic(wire.Build(
allProviders,
))
@@ -48,7 +58,7 @@ func CreateNativeAPIRouter() *nativeapi.Router {
))
}
func CreateSubsonicAPIRouter() *subsonic.Router {
func CreateSubsonicAPIRouter(ctx context.Context) *subsonic.Router {
panic(wire.Build(
allProviders,
))
@@ -84,7 +94,13 @@ func CreatePrometheus() metrics.Metrics {
))
}
func GetScanner() scanner.Scanner {
func CreateScanner(ctx context.Context) scanner.Scanner {
panic(wire.Build(
allProviders,
))
}
func CreateScanWatcher(ctx context.Context) scanner.Watcher {
panic(wire.Build(
allProviders,
))

View File

@@ -9,9 +9,12 @@ import (
"strings"
"time"
"github.com/bmatcuk/doublestar/v4"
"github.com/go-viper/encoding/ini"
"github.com/kr/pretty"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/utils/chain"
"github.com/robfig/cron/v3"
"github.com/spf13/viper"
)
@@ -27,8 +30,6 @@ type configOptions struct {
DbPath string
LogLevel string
LogFile string
ScanInterval time.Duration
ScanSchedule string
SessionTimeout time.Duration
BaseURL string
BasePath string
@@ -59,7 +60,6 @@ type configOptions struct {
PreferSortTags bool
IgnoredArticles string
IndexGroups string
SubsonicArtistParticipations bool
FFmpegPath string
MPVPath string
MPVCmdTemplate string
@@ -90,11 +90,15 @@ type configOptions struct {
Scanner scannerOptions
Jukebox jukeboxOptions
Backup backupOptions
PID pidOptions
Inspect inspectOptions
Subsonic subsonicOptions
Agents string
LastFM lastfmOptions
Spotify spotifyOptions
ListenBrainz listenBrainzOptions
Tags map[string]TagConf
// DevFlags. These are used to enable/disable debugging and incomplete features
DevLogSourceLine bool
@@ -113,14 +117,37 @@ type configOptions struct {
DevArtworkThrottleBacklogTimeout time.Duration
DevArtistInfoTimeToLive time.Duration
DevAlbumInfoTimeToLive time.Duration
DevExternalScanner bool
DevScannerThreads uint
DevInsightsInitialDelay time.Duration
DevEnablePlayerInsights bool
}
type scannerOptions struct {
Enabled bool
Schedule string
WatcherWait time.Duration
ScanOnStartup bool
Extractor string
GenreSeparators string
GroupAlbumReleases bool
ArtistJoiner string
GenreSeparators string // Deprecated: Use Tags.genre.Split instead
GroupAlbumReleases bool // Deprecated: Use PID.Album instead
}
type subsonicOptions struct {
AppendSubtitle bool
ArtistParticipations bool
DefaultReportRealPath bool
LegacyClients string
}
type TagConf struct {
Ignore bool `yaml:"ignore"`
Aliases []string `yaml:"aliases"`
Type string `yaml:"type"`
MaxLength int `yaml:"maxLength"`
Split []string `yaml:"split"`
Album bool `yaml:"album"`
}
type lastfmOptions struct {
@@ -165,6 +192,18 @@ type backupOptions struct {
Schedule string
}
type pidOptions struct {
Track string
Album string
}
type inspectOptions struct {
Enabled bool
MaxRequests int
BacklogLimit int
BacklogTimeout int
}
var (
Server = &configOptions{}
hooks []func()
@@ -177,10 +216,10 @@ func LoadFromFile(confFile string) {
_, _ = fmt.Fprintln(os.Stderr, "FATAL: Error reading config file:", err)
os.Exit(1)
}
Load()
Load(true)
}
func Load() {
func Load(noConfigDump bool) {
parseIniFileConfiguration()
err := viper.Unmarshal(&Server)
@@ -232,11 +271,12 @@ func Load() {
log.SetLogSourceLine(Server.DevLogSourceLine)
log.SetRedacting(Server.EnableLogRedacting)
if err := validateScanSchedule(); err != nil {
os.Exit(1)
}
if err := validateBackupSchedule(); err != nil {
err = chain.RunSequentially(
validateScanSchedule,
validateBackupSchedule,
validatePlaylistsPath,
)
if err != nil {
os.Exit(1)
}
@@ -254,7 +294,7 @@ func Load() {
}
// Print current configuration if log level is Debug
if log.IsGreaterOrEqualTo(log.LevelDebug) {
if log.IsGreaterOrEqualTo(log.LevelDebug) && !noConfigDump {
prettyConf := pretty.Sprintf("Loaded configuration from '%s': %# v", Server.ConfigFile, Server)
if Server.EnableLogRedacting {
prettyConf = log.Redact(prettyConf)
@@ -266,12 +306,31 @@ func Load() {
disableExternalServices()
}
if Server.Scanner.Extractor != consts.DefaultScannerExtractor {
log.Warn(fmt.Sprintf("Extractor '%s' is not implemented, using 'taglib'", Server.Scanner.Extractor))
Server.Scanner.Extractor = consts.DefaultScannerExtractor
}
logDeprecatedOptions("Scanner.GenreSeparators")
logDeprecatedOptions("Scanner.GroupAlbumReleases")
// Call init hooks
for _, hook := range hooks {
hook()
}
}
func logDeprecatedOptions(options ...string) {
for _, option := range options {
envVar := "ND_" + strings.ToUpper(strings.ReplaceAll(option, ".", "_"))
if os.Getenv(envVar) != "" {
log.Warn(fmt.Sprintf("Option '%s' is deprecated and will be ignored in a future release", envVar))
}
if viper.InConfig(option) {
log.Warn(fmt.Sprintf("Option '%s' is deprecated and will be ignored in a future release", option))
}
}
}
// parseIniFileConfiguration is used to parse the config file when it is in INI format. For INI files, it
// would require a nested structure, so instead we unmarshal it to a map and then merge the nested [default]
// section into the root level.
@@ -309,26 +368,24 @@ func disableExternalServices() {
}
}
func validateScanSchedule() error {
if Server.ScanInterval != -1 {
log.Warn("ScanInterval is DEPRECATED. Please use ScanSchedule. See docs at https://navidrome.org/docs/usage/configuration-options/")
if Server.ScanSchedule != "@every 1m" {
log.Error("You cannot specify both ScanInterval and ScanSchedule, ignoring ScanInterval")
} else {
if Server.ScanInterval == 0 {
Server.ScanSchedule = ""
} else {
Server.ScanSchedule = fmt.Sprintf("@every %s", Server.ScanInterval)
}
log.Warn("Setting ScanSchedule", "schedule", Server.ScanSchedule)
func validatePlaylistsPath() error {
for _, path := range strings.Split(Server.PlaylistsPath, string(filepath.ListSeparator)) {
_, err := doublestar.Match(path, "")
if err != nil {
log.Error("Invalid PlaylistsPath", "path", path, err)
return err
}
}
if Server.ScanSchedule == "0" || Server.ScanSchedule == "" {
Server.ScanSchedule = ""
return nil
}
func validateScanSchedule() error {
if Server.Scanner.Schedule == "0" || Server.Scanner.Schedule == "" {
Server.Scanner.Schedule = ""
return nil
}
var err error
Server.ScanSchedule, err = validateSchedule(Server.ScanSchedule, "ScanSchedule")
Server.Scanner.Schedule, err = validateSchedule(Server.Scanner.Schedule, "Scanner.Schedule")
return err
}
@@ -337,10 +394,8 @@ func validateBackupSchedule() error {
Server.Backup.Schedule = ""
return nil
}
var err error
Server.Backup.Schedule, err = validateSchedule(Server.Backup.Schedule, "BackupSchedule")
Server.Backup.Schedule, err = validateSchedule(Server.Backup.Schedule, "Backup.Schedule")
return err
}
@@ -351,7 +406,7 @@ func validateSchedule(schedule, field string) (string, error) {
c := cron.New()
id, err := c.AddFunc(schedule, func() {})
if err != nil {
log.Error(fmt.Sprintf("Invalid %s. Please read format spec at https://pkg.go.dev/github.com/robfig/cron#hdr-CRON_Expression_Format", field), "schedule", field, err)
log.Error(fmt.Sprintf("Invalid %s. Please read format spec at https://pkg.go.dev/github.com/robfig/cron#hdr-CRON_Expression_Format", field), "schedule", schedule, err)
} else {
c.Remove(id)
}
@@ -373,8 +428,6 @@ func init() {
viper.SetDefault("port", 4533)
viper.SetDefault("unixsocketperm", "0660")
viper.SetDefault("sessiontimeout", consts.DefaultSessionTimeout)
viper.SetDefault("scaninterval", -1)
viper.SetDefault("scanschedule", "@every 1m")
viper.SetDefault("baseurl", "")
viper.SetDefault("tlscert", "")
viper.SetDefault("tlskey", "")
@@ -388,7 +441,7 @@ func init() {
viper.SetDefault("enableartworkprecache", true)
viper.SetDefault("autoimportplaylists", true)
viper.SetDefault("defaultplaylistpublicvisibility", false)
viper.SetDefault("playlistspath", consts.DefaultPlaylistsPath)
viper.SetDefault("playlistspath", "")
viper.SetDefault("smartPlaylistRefreshDelay", 5*time.Second)
viper.SetDefault("enabledownloads", true)
viper.SetDefault("enableexternalservices", true)
@@ -400,7 +453,6 @@ func init() {
viper.SetDefault("prefersorttags", false)
viper.SetDefault("ignoredarticles", "The El La Los Las Le Les Os As O A")
viper.SetDefault("indexgroups", "A B C D E F G H I J K L M N O P Q R S T U V W X-Z(XYZ) [Unknown]([)")
viper.SetDefault("subsonicartistparticipations", false)
viper.SetDefault("ffmpegpath", "")
viper.SetDefault("mpvcmdtemplate", "mpv --audio-device=%d --no-audio-display --pause %f --input-ipc-server=%s")
@@ -416,6 +468,9 @@ func init() {
viper.SetDefault("defaultuivolume", consts.DefaultUIVolume)
viper.SetDefault("enablereplaygain", true)
viper.SetDefault("enablecoveranimation", true)
viper.SetDefault("enablesharing", false)
viper.SetDefault("shareurl", "")
viper.SetDefault("defaultdownloadableshare", false)
viper.SetDefault("gatrackingid", "")
viper.SetDefault("enableinsightscollector", true)
viper.SetDefault("enablelogredacting", true)
@@ -435,10 +490,20 @@ func init() {
viper.SetDefault("jukebox.default", "")
viper.SetDefault("jukebox.adminonly", true)
viper.SetDefault("scanner.enabled", true)
viper.SetDefault("scanner.schedule", "0")
viper.SetDefault("scanner.extractor", consts.DefaultScannerExtractor)
viper.SetDefault("scanner.genreseparators", ";/,")
viper.SetDefault("scanner.watcherwait", consts.DefaultWatcherWait)
viper.SetDefault("scanner.scanonstartup", true)
viper.SetDefault("scanner.artistjoiner", consts.ArtistJoiner)
viper.SetDefault("scanner.genreseparators", "")
viper.SetDefault("scanner.groupalbumreleases", false)
viper.SetDefault("subsonic.appendsubtitle", true)
viper.SetDefault("subsonic.artistparticipations", false)
viper.SetDefault("subsonic.defaultreportrealpath", false)
viper.SetDefault("subsonic.legacyclients", "DSub")
viper.SetDefault("agents", "lastfm,spotify")
viper.SetDefault("lastfm.enabled", true)
viper.SetDefault("lastfm.language", "en")
@@ -455,6 +520,14 @@ func init() {
viper.SetDefault("backup.schedule", "")
viper.SetDefault("backup.count", 0)
viper.SetDefault("pid.track", consts.DefaultTrackPID)
viper.SetDefault("pid.album", consts.DefaultAlbumPID)
viper.SetDefault("inspect.enabled", true)
viper.SetDefault("inspect.maxrequests", 1)
viper.SetDefault("inspect.backloglimit", consts.RequestThrottleBacklogLimit)
viper.SetDefault("inspect.backlogtimeout", consts.RequestThrottleBacklogTimeout)
// DevFlags. These are used to enable/disable debugging and incomplete features
viper.SetDefault("devlogsourceline", false)
viper.SetDefault("devenableprofiler", false)
@@ -462,9 +535,6 @@ func init() {
viper.SetDefault("devautologinusername", "")
viper.SetDefault("devactivitypanel", true)
viper.SetDefault("devactivitypanelupdaterate", 300*time.Millisecond)
viper.SetDefault("enablesharing", false)
viper.SetDefault("shareurl", "")
viper.SetDefault("defaultdownloadableshare", false)
viper.SetDefault("devenablebufferedscrobble", true)
viper.SetDefault("devsidebarplaylists", true)
viper.SetDefault("devshowartistpage", true)
@@ -474,11 +544,17 @@ func init() {
viper.SetDefault("devartworkthrottlebacklogtimeout", consts.RequestThrottleBacklogTimeout)
viper.SetDefault("devartistinfotimetolive", consts.ArtistInfoTimeToLive)
viper.SetDefault("devalbuminfotimetolive", consts.AlbumInfoTimeToLive)
viper.SetDefault("devexternalscanner", true)
viper.SetDefault("devscannerthreads", 5)
viper.SetDefault("devinsightsinitialdelay", consts.InsightsInitialDelay)
viper.SetDefault("devenableplayerinsights", true)
}
func InitConfig(cfgFile string) {
codecRegistry := viper.NewCodecRegistry()
_ = codecRegistry.RegisterCodec("ini", ini.Codec{})
viper.SetOptions(viper.WithCodecRegistry(codecRegistry))
cfgFile = getConfigFile(cfgFile)
if cfgFile != "" {
// Use config file from the flag.
@@ -502,9 +578,17 @@ func InitConfig(cfgFile string) {
}
}
// getConfigFile returns the path to the config file, either from the flag or from the environment variable.
// If it is defined in the environment variable, it will check if the file exists.
func getConfigFile(cfgFile string) string {
if cfgFile != "" {
return cfgFile
}
return os.Getenv("ND_CONFIGFILE")
cfgFile = os.Getenv("ND_CONFIGFILE")
if cfgFile != "" {
if _, err := os.Stat(cfgFile); err == nil {
return cfgFile
}
}
return ""
}

View File

@@ -0,0 +1,50 @@
package conf_test
import (
"fmt"
"path/filepath"
"testing"
. "github.com/navidrome/navidrome/conf"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/spf13/viper"
)
func TestConfiguration(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "Configuration Suite")
}
var _ = Describe("Configuration", func() {
BeforeEach(func() {
// Reset viper configuration
viper.Reset()
viper.SetDefault("datafolder", GinkgoT().TempDir())
viper.SetDefault("loglevel", "error")
ResetConf()
})
DescribeTable("should load configuration from",
func(format string) {
filename := filepath.Join("testdata", "cfg."+format)
// Initialize config with the test file
InitConfig(filename)
// Load the configuration (with noConfigDump=true)
Load(true)
// Execute the format-specific assertions
Expect(Server.MusicFolder).To(Equal(fmt.Sprintf("/%s/music", format)))
Expect(Server.UIWelcomeMessage).To(Equal("Welcome " + format))
Expect(Server.Tags["custom"].Aliases).To(Equal([]string{format, "test"}))
// The config file used should be the one we created
Expect(Server.ConfigFile).To(Equal(filename))
},
Entry("TOML format", "toml"),
Entry("YAML format", "yaml"),
Entry("INI format", "ini"),
Entry("JSON format", "json"),
)
})

5
conf/export_test.go Normal file
View File

@@ -0,0 +1,5 @@
package conf
func ResetConf() {
Server = &configOptions{}
}

6
conf/testdata/cfg.ini vendored Normal file
View File

@@ -0,0 +1,6 @@
[default]
MusicFolder = /ini/music
UIWelcomeMessage = Welcome ini
[Tags]
Custom.Aliases = ini,test

12
conf/testdata/cfg.json vendored Normal file
View File

@@ -0,0 +1,12 @@
{
"musicFolder": "/json/music",
"uiWelcomeMessage": "Welcome json",
"Tags": {
"custom": {
"aliases": [
"json",
"test"
]
}
}
}

5
conf/testdata/cfg.toml vendored Normal file
View File

@@ -0,0 +1,5 @@
musicFolder = "/toml/music"
uiWelcomeMessage = "Welcome toml"
[Tags.custom]
aliases = ["toml", "test"]

7
conf/testdata/cfg.yaml vendored Normal file
View File

@@ -0,0 +1,7 @@
musicFolder: "/yaml/music"
uiWelcomeMessage: "Welcome yaml"
Tags:
custom:
aliases:
- yaml
- test

View File

@@ -1,27 +1,29 @@
package consts
import (
"crypto/md5"
"fmt"
"os"
"path/filepath"
"strings"
"time"
"github.com/navidrome/navidrome/model/id"
)
const (
AppName = "navidrome"
DefaultDbPath = "navidrome.db?cache=shared&_busy_timeout=15000&_journal_mode=WAL&_foreign_keys=on"
InitialSetupFlagKey = "InitialSetup"
DefaultDbPath = "navidrome.db?cache=shared&_busy_timeout=15000&_journal_mode=WAL&_foreign_keys=on&synchronous=normal"
InitialSetupFlagKey = "InitialSetup"
FullScanAfterMigrationFlagKey = "FullScanAfterMigration"
UIAuthorizationHeader = "X-ND-Authorization"
UIClientUniqueIDHeader = "X-ND-Client-Unique-Id"
JWTSecretKey = "JWTSecret"
JWTIssuer = "ND"
DefaultSessionTimeout = 24 * time.Hour
DefaultSessionTimeout = 48 * time.Hour
CookieExpiry = 365 * 24 * 3600 // One year
OptimizeDBSchedule = "@every 24h"
// DefaultEncryptionKey This is the encryption key used if none is specified in the `PasswordEncryptionKey` option
// Never ever change this! Or it will break all Navidrome installations that don't set the config option
DefaultEncryptionKey = "just for obfuscation"
@@ -51,11 +53,13 @@ const (
ServerReadHeaderTimeout = 3 * time.Second
ArtistInfoTimeToLive = 24 * time.Hour
AlbumInfoTimeToLive = 7 * 24 * time.Hour
ArtistInfoTimeToLive = 24 * time.Hour
AlbumInfoTimeToLive = 7 * 24 * time.Hour
UpdateLastAccessFrequency = time.Minute
UpdatePlayerFrequency = time.Minute
I18nFolder = "i18n"
SkipScanFile = ".ndignore"
I18nFolder = "i18n"
ScanIgnoreFile = ".ndignore"
PlaceholderArtistArt = "artist-placeholder.webp"
PlaceholderAlbumArt = "album-placeholder.webp"
@@ -66,8 +70,8 @@ const (
DefaultHttpClientTimeOut = 10 * time.Second
DefaultScannerExtractor = "taglib"
Zwsp = string('\u200b')
DefaultWatcherWait = 5 * time.Second
Zwsp = string('\u200b')
)
// Prometheus options
@@ -93,6 +97,14 @@ const (
AlbumPlayCountModeNormalized = "normalized"
)
const (
//DefaultAlbumPID = "album_legacy"
DefaultAlbumPID = "musicbrainz_albumid|albumartistid,album,albumversion,releasedate"
DefaultTrackPID = "musicbrainz_trackid|albumid,discnumber,tracknumber,title"
PIDAlbumKey = "PIDAlbum"
PIDTrackKey = "PIDTrack"
)
const (
InsightsIDKey = "InsightsID"
InsightsEndpoint = "https://insights.navidrome.org/collect"
@@ -127,25 +139,29 @@ var (
Command: "ffmpeg -i %s -ss %t -map 0:a:0 -b:a %bk -v 0 -c:a aac -f adts -",
},
}
DefaultPlaylistsPath = strings.Join([]string{".", "**/**"}, string(filepath.ListSeparator))
)
var (
VariousArtists = "Various Artists"
VariousArtistsID = fmt.Sprintf("%x", md5.Sum([]byte(strings.ToLower(VariousArtists))))
UnknownAlbum = "[Unknown Album]"
UnknownArtist = "[Unknown Artist]"
UnknownArtistID = fmt.Sprintf("%x", md5.Sum([]byte(strings.ToLower(UnknownArtist))))
VariousArtists = "Various Artists"
// TODO This will be dynamic when using disambiguation
VariousArtistsID = "63sqASlAfjbGMuLP4JhnZU"
UnknownAlbum = "[Unknown Album]"
UnknownArtist = "[Unknown Artist]"
// TODO This will be dynamic when using disambiguation
UnknownArtistID = id.NewHash(strings.ToLower(UnknownArtist))
VariousArtistsMbzId = "89ad4ac3-39f7-470e-963a-56509c546377"
ServerStart = time.Now()
ArtistJoiner = " • "
)
var InContainer = func() bool {
// Check if the /.nddockerenv file exists
if _, err := os.Stat("/.nddockerenv"); err == nil {
return true
}
return false
}()
var (
ServerStart = time.Now()
InContainer = func() bool {
// Check if the /.nddockerenv file exists
if _, err := os.Stat("/.nddockerenv"); err == nil {
return true
}
return false
}()
)

View File

@@ -10,6 +10,7 @@ import (
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils"
"github.com/navidrome/navidrome/utils/singleton"
)
type Agents struct {
@@ -17,22 +18,36 @@ type Agents struct {
agents []Interface
}
func New(ds model.DataStore) *Agents {
func GetAgents(ds model.DataStore) *Agents {
return singleton.GetInstance(func() *Agents {
return createAgents(ds)
})
}
func createAgents(ds model.DataStore) *Agents {
var order []string
if conf.Server.Agents != "" {
order = strings.Split(conf.Server.Agents, ",")
}
order = append(order, LocalAgentName)
var res []Interface
var enabled []string
for _, name := range order {
init, ok := Map[name]
if !ok {
log.Error("Agent not available. Check configuration", "name", name)
log.Error("Invalid agent. Check `Agents` configuration", "name", name, "conf", conf.Server.Agents)
continue
}
res = append(res, init(ds))
agent := init(ds)
if agent == nil {
log.Debug("Agent not available. Missing configuration?", "name", name)
continue
}
enabled = append(enabled, name)
res = append(res, agent)
}
log.Debug("List of agents enabled", "names", enabled)
return &Agents{ds: ds, agents: res}
}

View File

@@ -7,6 +7,7 @@ import (
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
"github.com/navidrome/navidrome/utils/slice"
"github.com/navidrome/navidrome/conf"
. "github.com/onsi/ginkgo/v2"
@@ -28,7 +29,7 @@ var _ = Describe("Agents", func() {
var ag *Agents
BeforeEach(func() {
conf.Server.Agents = ""
ag = New(ds)
ag = createAgents(ds)
})
It("calls the placeholder GetArtistImages", func() {
@@ -44,19 +45,21 @@ var _ = Describe("Agents", func() {
var mock *mockAgent
BeforeEach(func() {
mock = &mockAgent{}
Register("fake", func(ds model.DataStore) Interface {
return mock
})
Register("empty", func(ds model.DataStore) Interface {
return struct {
Interface
}{}
})
conf.Server.Agents = "empty,fake"
ag = New(ds)
Register("fake", func(model.DataStore) Interface { return mock })
Register("disabled", func(model.DataStore) Interface { return nil })
Register("empty", func(model.DataStore) Interface { return &emptyAgent{} })
conf.Server.Agents = "empty,fake,disabled"
ag = createAgents(ds)
Expect(ag.AgentName()).To(Equal("agents"))
})
It("does not register disabled agents", func() {
ags := slice.Map(ag.agents, func(a Interface) string { return a.AgentName() })
// local agent is always appended to the end of the agents list
Expect(ags).To(HaveExactElements("empty", "fake", "local"))
Expect(ags).ToNot(ContainElement("disabled"))
})
Describe("GetArtistMBID", func() {
It("returns on first match", func() {
Expect(ag.GetArtistMBID(ctx, "123", "test")).To(Equal("mbid"))
@@ -344,3 +347,11 @@ func (a *mockAgent) GetAlbumInfo(ctx context.Context, name, artist, mbid string)
},
}, nil
}
type emptyAgent struct {
Interface
}
func (e *emptyAgent) AgentName() string {
return "empty"
}

View File

@@ -8,6 +8,7 @@ import (
"regexp"
"strconv"
"strings"
"sync"
"github.com/andybalholm/cascadia"
"github.com/navidrome/navidrome/conf"
@@ -31,15 +32,19 @@ var ignoredBiographies = []string{
}
type lastfmAgent struct {
ds model.DataStore
sessionKeys *agents.SessionKeys
apiKey string
secret string
lang string
client *client
ds model.DataStore
sessionKeys *agents.SessionKeys
apiKey string
secret string
lang string
client *client
getInfoMutex sync.Mutex
}
func lastFMConstructor(ds model.DataStore) *lastfmAgent {
if !conf.Server.LastFM.Enabled || conf.Server.LastFM.ApiKey == "" || conf.Server.LastFM.Secret == "" {
return nil
}
l := &lastfmAgent{
ds: ds,
lang: conf.Server.LastFM.Language,
@@ -107,7 +112,7 @@ func (l *lastfmAgent) GetAlbumInfo(ctx context.Context, name, artist, mbid strin
}
func (l *lastfmAgent) GetArtistMBID(ctx context.Context, id string, name string) (string, error) {
a, err := l.callArtistGetInfo(ctx, name, "")
a, err := l.callArtistGetInfo(ctx, name)
if err != nil {
return "", err
}
@@ -118,7 +123,7 @@ func (l *lastfmAgent) GetArtistMBID(ctx context.Context, id string, name string)
}
func (l *lastfmAgent) GetArtistURL(ctx context.Context, id, name, mbid string) (string, error) {
a, err := l.callArtistGetInfo(ctx, name, mbid)
a, err := l.callArtistGetInfo(ctx, name)
if err != nil {
return "", err
}
@@ -129,7 +134,7 @@ func (l *lastfmAgent) GetArtistURL(ctx context.Context, id, name, mbid string) (
}
func (l *lastfmAgent) GetArtistBiography(ctx context.Context, id, name, mbid string) (string, error) {
a, err := l.callArtistGetInfo(ctx, name, mbid)
a, err := l.callArtistGetInfo(ctx, name)
if err != nil {
return "", err
}
@@ -146,7 +151,7 @@ func (l *lastfmAgent) GetArtistBiography(ctx context.Context, id, name, mbid str
}
func (l *lastfmAgent) GetSimilarArtists(ctx context.Context, id, name, mbid string, limit int) ([]agents.Artist, error) {
resp, err := l.callArtistGetSimilar(ctx, name, mbid, limit)
resp, err := l.callArtistGetSimilar(ctx, name, limit)
if err != nil {
return nil, err
}
@@ -164,7 +169,7 @@ func (l *lastfmAgent) GetSimilarArtists(ctx context.Context, id, name, mbid stri
}
func (l *lastfmAgent) GetArtistTopSongs(ctx context.Context, id, artistName, mbid string, count int) ([]agents.Song, error) {
resp, err := l.callArtistGetTopTracks(ctx, artistName, mbid, count)
resp, err := l.callArtistGetTopTracks(ctx, artistName, count)
if err != nil {
return nil, err
}
@@ -184,15 +189,19 @@ func (l *lastfmAgent) GetArtistTopSongs(ctx context.Context, id, artistName, mbi
var artistOpenGraphQuery = cascadia.MustCompile(`html > head > meta[property="og:image"]`)
func (l *lastfmAgent) GetArtistImages(ctx context.Context, _, name, mbid string) ([]agents.ExternalImage, error) {
a, err := l.callArtistGetInfo(ctx, name, mbid)
log.Debug(ctx, "Getting artist images from Last.fm", "name", name)
hc := http.Client{
Timeout: consts.DefaultHttpClientTimeOut,
}
a, err := l.callArtistGetInfo(ctx, name)
if err != nil {
return nil, fmt.Errorf("get artist info: %w", err)
}
req, err := http.NewRequest(http.MethodGet, a.URL, nil)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, a.URL, nil)
if err != nil {
return nil, fmt.Errorf("create artist image request: %w", err)
}
resp, err := l.client.hc.Do(req)
resp, err := hc.Do(req)
if err != nil {
return nil, fmt.Errorf("get artist url: %w", err)
}
@@ -240,48 +249,31 @@ func (l *lastfmAgent) callAlbumGetInfo(ctx context.Context, name, artist, mbid s
return a, nil
}
func (l *lastfmAgent) callArtistGetInfo(ctx context.Context, name string, mbid string) (*Artist, error) {
a, err := l.client.artistGetInfo(ctx, name, mbid)
var lfErr *lastFMError
isLastFMError := errors.As(err, &lfErr)
if mbid != "" && ((err == nil && a.Name == "[unknown]") || (isLastFMError && lfErr.Code == 6)) {
log.Debug(ctx, "LastFM/artist.getInfo could not find artist by mbid, trying again", "artist", name, "mbid", mbid)
return l.callArtistGetInfo(ctx, name, "")
}
func (l *lastfmAgent) callArtistGetInfo(ctx context.Context, name string) (*Artist, error) {
l.getInfoMutex.Lock()
defer l.getInfoMutex.Unlock()
a, err := l.client.artistGetInfo(ctx, name)
if err != nil {
log.Error(ctx, "Error calling LastFM/artist.getInfo", "artist", name, "mbid", mbid, err)
log.Error(ctx, "Error calling LastFM/artist.getInfo", "artist", name, err)
return nil, err
}
return a, nil
}
func (l *lastfmAgent) callArtistGetSimilar(ctx context.Context, name string, mbid string, limit int) ([]Artist, error) {
s, err := l.client.artistGetSimilar(ctx, name, mbid, limit)
var lfErr *lastFMError
isLastFMError := errors.As(err, &lfErr)
if mbid != "" && ((err == nil && s.Attr.Artist == "[unknown]") || (isLastFMError && lfErr.Code == 6)) {
log.Debug(ctx, "LastFM/artist.getSimilar could not find artist by mbid, trying again", "artist", name, "mbid", mbid)
return l.callArtistGetSimilar(ctx, name, "", limit)
}
func (l *lastfmAgent) callArtistGetSimilar(ctx context.Context, name string, limit int) ([]Artist, error) {
s, err := l.client.artistGetSimilar(ctx, name, limit)
if err != nil {
log.Error(ctx, "Error calling LastFM/artist.getSimilar", "artist", name, "mbid", mbid, err)
log.Error(ctx, "Error calling LastFM/artist.getSimilar", "artist", name, err)
return nil, err
}
return s.Artists, nil
}
func (l *lastfmAgent) callArtistGetTopTracks(ctx context.Context, artistName, mbid string, count int) ([]Track, error) {
t, err := l.client.artistGetTopTracks(ctx, artistName, mbid, count)
var lfErr *lastFMError
isLastFMError := errors.As(err, &lfErr)
if mbid != "" && ((err == nil && t.Attr.Artist == "[unknown]") || (isLastFMError && lfErr.Code == 6)) {
log.Debug(ctx, "LastFM/artist.getTopTracks could not find artist by mbid, trying again", "artist", artistName, "mbid", mbid)
return l.callArtistGetTopTracks(ctx, artistName, "", count)
}
func (l *lastfmAgent) callArtistGetTopTracks(ctx context.Context, artistName string, count int) ([]Track, error) {
t, err := l.client.artistGetTopTracks(ctx, artistName, count)
if err != nil {
log.Error(ctx, "Error calling LastFM/artist.getTopTracks", "artist", artistName, "mbid", mbid, err)
log.Error(ctx, "Error calling LastFM/artist.getTopTracks", "artist", artistName, err)
return nil, err
}
return t.Track, nil
@@ -304,7 +296,7 @@ func (l *lastfmAgent) NowPlaying(ctx context.Context, userId string, track *mode
})
if err != nil {
log.Warn(ctx, "Last.fm client.updateNowPlaying returned error", "track", track.Title, err)
return scrobbler.ErrUnrecoverable
return errors.Join(err, scrobbler.ErrUnrecoverable)
}
return nil
}
@@ -312,7 +304,7 @@ func (l *lastfmAgent) NowPlaying(ctx context.Context, userId string, track *mode
func (l *lastfmAgent) Scrobble(ctx context.Context, userId string, s scrobbler.Scrobble) error {
sk, err := l.sessionKeys.Get(ctx, userId)
if err != nil || sk == "" {
return scrobbler.ErrNotAuthorized
return errors.Join(err, scrobbler.ErrNotAuthorized)
}
if s.Duration <= 30 {
@@ -336,12 +328,12 @@ func (l *lastfmAgent) Scrobble(ctx context.Context, userId string, s scrobbler.S
isLastFMError := errors.As(err, &lfErr)
if !isLastFMError {
log.Warn(ctx, "Last.fm client.scrobble returned error", "track", s.Title, err)
return scrobbler.ErrRetryLater
return errors.Join(err, scrobbler.ErrRetryLater)
}
if lfErr.Code == 11 || lfErr.Code == 16 {
return scrobbler.ErrRetryLater
return errors.Join(err, scrobbler.ErrRetryLater)
}
return scrobbler.ErrUnrecoverable
return errors.Join(err, scrobbler.ErrUnrecoverable)
}
func (l *lastfmAgent) IsAuthorized(ctx context.Context, userId string) bool {
@@ -351,15 +343,19 @@ func (l *lastfmAgent) IsAuthorized(ctx context.Context, userId string) bool {
func init() {
conf.AddHook(func() {
if conf.Server.LastFM.Enabled {
if conf.Server.LastFM.ApiKey != "" && conf.Server.LastFM.Secret != "" {
agents.Register(lastFMAgentName, func(ds model.DataStore) agents.Interface {
return lastFMConstructor(ds)
})
scrobbler.Register(lastFMAgentName, func(ds model.DataStore) scrobbler.Scrobbler {
return lastFMConstructor(ds)
})
agents.Register(lastFMAgentName, func(ds model.DataStore) agents.Interface {
a := lastFMConstructor(ds)
if a != nil {
return a
}
}
return nil
})
scrobbler.Register(lastFMAgentName, func(ds model.DataStore) scrobbler.Scrobbler {
a := lastFMConstructor(ds)
if a != nil {
return a
}
return nil
})
})
}

View File

@@ -11,6 +11,7 @@ import (
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/core/scrobbler"
"github.com/navidrome/navidrome/model"
@@ -30,16 +31,38 @@ var _ = Describe("lastfmAgent", func() {
BeforeEach(func() {
ds = &tests.MockDataStore{}
ctx = context.Background()
DeferCleanup(configtest.SetupConfig())
conf.Server.LastFM.Enabled = true
conf.Server.LastFM.ApiKey = "123"
conf.Server.LastFM.Secret = "secret"
})
Describe("lastFMConstructor", func() {
It("uses configured api key and language", func() {
conf.Server.LastFM.ApiKey = "123"
conf.Server.LastFM.Secret = "secret"
conf.Server.LastFM.Language = "pt"
agent := lastFMConstructor(ds)
Expect(agent.apiKey).To(Equal("123"))
Expect(agent.secret).To(Equal("secret"))
Expect(agent.lang).To(Equal("pt"))
When("Agent is properly configured", func() {
It("uses configured api key and language", func() {
conf.Server.LastFM.Language = "pt"
agent := lastFMConstructor(ds)
Expect(agent.apiKey).To(Equal("123"))
Expect(agent.secret).To(Equal("secret"))
Expect(agent.lang).To(Equal("pt"))
})
})
When("Agent is disabled", func() {
It("returns nil", func() {
conf.Server.LastFM.Enabled = false
Expect(lastFMConstructor(ds)).To(BeNil())
})
})
When("ApiKey is empty", func() {
It("returns nil", func() {
conf.Server.LastFM.ApiKey = ""
Expect(lastFMConstructor(ds)).To(BeNil())
})
})
When("Secret is empty", func() {
It("returns nil", func() {
conf.Server.LastFM.Secret = ""
Expect(lastFMConstructor(ds)).To(BeNil())
})
})
})
@@ -56,48 +79,25 @@ var _ = Describe("lastfmAgent", func() {
It("returns the biography", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
Expect(agent.GetArtistBiography(ctx, "123", "U2", "mbid-1234")).To(Equal("U2 é uma das mais importantes bandas de rock de todos os tempos. Formada em 1976 em Dublin, composta por Bono (vocalista e guitarrista), The Edge (guitarrista, pianista e backing vocal), Adam Clayton (baixista), Larry Mullen, Jr. (baterista e percussionista).\n\nDesde a década de 80, U2 é uma das bandas mais populares no mundo. Seus shows são únicos e um verdadeiro festival de efeitos especiais, além de serem um dos que mais arrecadam anualmente. <a href=\"https://www.last.fm/music/U2\">Read more on Last.fm</a>"))
Expect(agent.GetArtistBiography(ctx, "123", "U2", "")).To(Equal("U2 é uma das mais importantes bandas de rock de todos os tempos. Formada em 1976 em Dublin, composta por Bono (vocalista e guitarrista), The Edge (guitarrista, pianista e backing vocal), Adam Clayton (baixista), Larry Mullen, Jr. (baterista e percussionista).\n\nDesde a década de 80, U2 é uma das bandas mais populares no mundo. Seus shows são únicos e um verdadeiro festival de efeitos especiais, além de serem um dos que mais arrecadam anualmente. <a href=\"https://www.last.fm/music/U2\">Read more on Last.fm</a>"))
Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("mbid-1234"))
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("U2"))
})
It("returns an error if Last.fm call fails", func() {
httpClient.Err = errors.New("error")
_, err := agent.GetArtistBiography(ctx, "123", "U2", "mbid-1234")
_, err := agent.GetArtistBiography(ctx, "123", "U2", "")
Expect(err).To(HaveOccurred())
Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("mbid-1234"))
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("U2"))
})
It("returns an error if Last.fm call returns an error", func() {
httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError3)), StatusCode: 200}
_, err := agent.GetArtistBiography(ctx, "123", "U2", "mbid-1234")
Expect(err).To(HaveOccurred())
Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("mbid-1234"))
})
It("returns an error if Last.fm call returns an error 6 and mbid is empty", func() {
httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError6)), StatusCode: 200}
_, err := agent.GetArtistBiography(ctx, "123", "U2", "")
Expect(err).To(HaveOccurred())
Expect(httpClient.RequestCount).To(Equal(1))
})
Context("MBID non existent in Last.fm", func() {
It("calls again when the response is artist == [unknown]", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.unknown.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
_, _ = agent.GetArtistBiography(ctx, "123", "U2", "mbid-1234")
Expect(httpClient.RequestCount).To(Equal(2))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty())
})
It("calls again when last.fm returns an error 6", func() {
httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError6)), StatusCode: 200}
_, _ = agent.GetArtistBiography(ctx, "123", "U2", "mbid-1234")
Expect(httpClient.RequestCount).To(Equal(2))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty())
})
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("U2"))
})
})
@@ -114,51 +114,28 @@ var _ = Describe("lastfmAgent", func() {
It("returns similar artists", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.getsimilar.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
Expect(agent.GetSimilarArtists(ctx, "123", "U2", "mbid-1234", 2)).To(Equal([]agents.Artist{
Expect(agent.GetSimilarArtists(ctx, "123", "U2", "", 2)).To(Equal([]agents.Artist{
{Name: "Passengers", MBID: "e110c11f-1c94-4471-a350-c38f46b29389"},
{Name: "INXS", MBID: "481bf5f9-2e7c-4c44-b08a-05b32bc7c00d"},
}))
Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("mbid-1234"))
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("U2"))
})
It("returns an error if Last.fm call fails", func() {
httpClient.Err = errors.New("error")
_, err := agent.GetSimilarArtists(ctx, "123", "U2", "mbid-1234", 2)
_, err := agent.GetSimilarArtists(ctx, "123", "U2", "", 2)
Expect(err).To(HaveOccurred())
Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("mbid-1234"))
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("U2"))
})
It("returns an error if Last.fm call returns an error", func() {
httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError3)), StatusCode: 200}
_, err := agent.GetSimilarArtists(ctx, "123", "U2", "mbid-1234", 2)
Expect(err).To(HaveOccurred())
Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("mbid-1234"))
})
It("returns an error if Last.fm call returns an error 6 and mbid is empty", func() {
httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError6)), StatusCode: 200}
_, err := agent.GetSimilarArtists(ctx, "123", "U2", "", 2)
Expect(err).To(HaveOccurred())
Expect(httpClient.RequestCount).To(Equal(1))
})
Context("MBID non existent in Last.fm", func() {
It("calls again when the response is artist == [unknown]", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.getsimilar.unknown.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
_, _ = agent.GetSimilarArtists(ctx, "123", "U2", "mbid-1234", 2)
Expect(httpClient.RequestCount).To(Equal(2))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty())
})
It("calls again when last.fm returns an error 6", func() {
httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError6)), StatusCode: 200}
_, _ = agent.GetSimilarArtists(ctx, "123", "U2", "mbid-1234", 2)
Expect(httpClient.RequestCount).To(Equal(2))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty())
})
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("U2"))
})
})
@@ -175,51 +152,28 @@ var _ = Describe("lastfmAgent", func() {
It("returns top songs", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.gettoptracks.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
Expect(agent.GetArtistTopSongs(ctx, "123", "U2", "mbid-1234", 2)).To(Equal([]agents.Song{
Expect(agent.GetArtistTopSongs(ctx, "123", "U2", "", 2)).To(Equal([]agents.Song{
{Name: "Beautiful Day", MBID: "f7f264d0-a89b-4682-9cd7-a4e7c37637af"},
{Name: "With or Without You", MBID: "6b9a509f-6907-4a6e-9345-2f12da09ba4b"},
}))
Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("mbid-1234"))
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("U2"))
})
It("returns an error if Last.fm call fails", func() {
httpClient.Err = errors.New("error")
_, err := agent.GetArtistTopSongs(ctx, "123", "U2", "mbid-1234", 2)
_, err := agent.GetArtistTopSongs(ctx, "123", "U2", "", 2)
Expect(err).To(HaveOccurred())
Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("mbid-1234"))
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("U2"))
})
It("returns an error if Last.fm call returns an error", func() {
httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError3)), StatusCode: 200}
_, err := agent.GetArtistTopSongs(ctx, "123", "U2", "mbid-1234", 2)
Expect(err).To(HaveOccurred())
Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("mbid-1234"))
})
It("returns an error if Last.fm call returns an error 6 and mbid is empty", func() {
httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError6)), StatusCode: 200}
_, err := agent.GetArtistTopSongs(ctx, "123", "U2", "", 2)
Expect(err).To(HaveOccurred())
Expect(httpClient.RequestCount).To(Equal(1))
})
Context("MBID non existent in Last.fm", func() {
It("calls again when the response is artist == [unknown]", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.gettoptracks.unknown.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
_, _ = agent.GetArtistTopSongs(ctx, "123", "U2", "mbid-1234", 2)
Expect(httpClient.RequestCount).To(Equal(2))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty())
})
It("calls again when last.fm returns an error 6", func() {
httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError6)), StatusCode: 200}
_, _ = agent.GetArtistTopSongs(ctx, "123", "U2", "mbid-1234", 2)
Expect(httpClient.RequestCount).To(Equal(2))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty())
})
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("U2"))
})
})

View File

@@ -59,11 +59,10 @@ func (c *client) albumGetInfo(ctx context.Context, name string, artist string, m
return &response.Album, nil
}
func (c *client) artistGetInfo(ctx context.Context, name string, mbid string) (*Artist, error) {
func (c *client) artistGetInfo(ctx context.Context, name string) (*Artist, error) {
params := url.Values{}
params.Add("method", "artist.getInfo")
params.Add("artist", name)
params.Add("mbid", mbid)
params.Add("lang", c.lang)
response, err := c.makeRequest(ctx, http.MethodGet, params, false)
if err != nil {
@@ -72,11 +71,10 @@ func (c *client) artistGetInfo(ctx context.Context, name string, mbid string) (*
return &response.Artist, nil
}
func (c *client) artistGetSimilar(ctx context.Context, name string, mbid string, limit int) (*SimilarArtists, error) {
func (c *client) artistGetSimilar(ctx context.Context, name string, limit int) (*SimilarArtists, error) {
params := url.Values{}
params.Add("method", "artist.getSimilar")
params.Add("artist", name)
params.Add("mbid", mbid)
params.Add("limit", strconv.Itoa(limit))
response, err := c.makeRequest(ctx, http.MethodGet, params, false)
if err != nil {
@@ -85,11 +83,10 @@ func (c *client) artistGetSimilar(ctx context.Context, name string, mbid string,
return &response.SimilarArtists, nil
}
func (c *client) artistGetTopTracks(ctx context.Context, name string, mbid string, limit int) (*TopTracks, error) {
func (c *client) artistGetTopTracks(ctx context.Context, name string, limit int) (*TopTracks, error) {
params := url.Values{}
params.Add("method", "artist.getTopTracks")
params.Add("artist", name)
params.Add("mbid", mbid)
params.Add("limit", strconv.Itoa(limit))
response, err := c.makeRequest(ctx, http.MethodGet, params, false)
if err != nil {

View File

@@ -42,10 +42,10 @@ var _ = Describe("client", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
artist, err := client.artistGetInfo(context.Background(), "U2", "123")
artist, err := client.artistGetInfo(context.Background(), "U2")
Expect(err).To(BeNil())
Expect(artist.Name).To(Equal("U2"))
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&lang=pt&mbid=123&method=artist.getInfo"))
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&lang=pt&method=artist.getInfo"))
})
It("fails if Last.fm returns an http status != 200", func() {
@@ -54,7 +54,7 @@ var _ = Describe("client", func() {
StatusCode: 500,
}
_, err := client.artistGetInfo(context.Background(), "U2", "123")
_, err := client.artistGetInfo(context.Background(), "U2")
Expect(err).To(MatchError("last.fm http status: (500)"))
})
@@ -64,7 +64,7 @@ var _ = Describe("client", func() {
StatusCode: 400,
}
_, err := client.artistGetInfo(context.Background(), "U2", "123")
_, err := client.artistGetInfo(context.Background(), "U2")
Expect(err).To(MatchError(&lastFMError{Code: 3, Message: "Invalid Method - No method with that name in this package"}))
})
@@ -74,14 +74,14 @@ var _ = Describe("client", func() {
StatusCode: 200,
}
_, err := client.artistGetInfo(context.Background(), "U2", "123")
_, err := client.artistGetInfo(context.Background(), "U2")
Expect(err).To(MatchError(&lastFMError{Code: 6, Message: "The artist you supplied could not be found"}))
})
It("fails if HttpClient.Do() returns error", func() {
httpClient.Err = errors.New("generic error")
_, err := client.artistGetInfo(context.Background(), "U2", "123")
_, err := client.artistGetInfo(context.Background(), "U2")
Expect(err).To(MatchError("generic error"))
})
@@ -91,7 +91,7 @@ var _ = Describe("client", func() {
StatusCode: 200,
}
_, err := client.artistGetInfo(context.Background(), "U2", "123")
_, err := client.artistGetInfo(context.Background(), "U2")
Expect(err).To(MatchError("invalid character '<' looking for beginning of value"))
})
@@ -102,10 +102,10 @@ var _ = Describe("client", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.getsimilar.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
similar, err := client.artistGetSimilar(context.Background(), "U2", "123", 2)
similar, err := client.artistGetSimilar(context.Background(), "U2", 2)
Expect(err).To(BeNil())
Expect(len(similar.Artists)).To(Equal(2))
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&limit=2&mbid=123&method=artist.getSimilar"))
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&limit=2&method=artist.getSimilar"))
})
})
@@ -114,10 +114,10 @@ var _ = Describe("client", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.gettoptracks.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
top, err := client.artistGetTopTracks(context.Background(), "U2", "123", 2)
top, err := client.artistGetTopTracks(context.Background(), "U2", 2)
Expect(err).To(BeNil())
Expect(len(top.Track)).To(Equal(2))
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&limit=2&mbid=123&method=artist.getTopTracks"))
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&limit=2&method=artist.getTopTracks"))
})
})

View File

@@ -12,6 +12,7 @@ import (
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/cache"
"github.com/navidrome/navidrome/utils/slice"
)
const (
@@ -45,6 +46,12 @@ func (l *listenBrainzAgent) AgentName() string {
}
func (l *listenBrainzAgent) formatListen(track *model.MediaFile) listenInfo {
artistMBIDs := slice.Map(track.Participants[model.RoleArtist], func(p model.Participant) string {
return p.MbzArtistID
})
artistNames := slice.Map(track.Participants[model.RoleArtist], func(p model.Participant) string {
return p.Name
})
li := listenInfo{
TrackMetadata: trackMetadata{
ArtistName: track.Artist,
@@ -54,9 +61,11 @@ func (l *listenBrainzAgent) formatListen(track *model.MediaFile) listenInfo {
SubmissionClient: consts.AppName,
SubmissionClientVersion: consts.Version,
TrackNumber: track.TrackNumber,
ArtistMbzIDs: []string{track.MbzArtistID},
RecordingMbzID: track.MbzRecordingID,
ReleaseMbID: track.MbzAlbumID,
ArtistNames: artistNames,
ArtistMBIDs: artistMBIDs,
RecordingMBID: track.MbzRecordingID,
ReleaseMBID: track.MbzAlbumID,
ReleaseGroupMBID: track.MbzReleaseGroupID,
DurationMs: int(track.Duration * 1000),
},
},
@@ -67,14 +76,14 @@ func (l *listenBrainzAgent) formatListen(track *model.MediaFile) listenInfo {
func (l *listenBrainzAgent) NowPlaying(ctx context.Context, userId string, track *model.MediaFile) error {
sk, err := l.sessionKeys.Get(ctx, userId)
if err != nil || sk == "" {
return scrobbler.ErrNotAuthorized
return errors.Join(err, scrobbler.ErrNotAuthorized)
}
li := l.formatListen(track)
err = l.client.updateNowPlaying(ctx, sk, li)
if err != nil {
log.Warn(ctx, "ListenBrainz updateNowPlaying returned error", "track", track.Title, err)
return scrobbler.ErrUnrecoverable
return errors.Join(err, scrobbler.ErrUnrecoverable)
}
return nil
}
@@ -82,7 +91,7 @@ func (l *listenBrainzAgent) NowPlaying(ctx context.Context, userId string, track
func (l *listenBrainzAgent) Scrobble(ctx context.Context, userId string, s scrobbler.Scrobble) error {
sk, err := l.sessionKeys.Get(ctx, userId)
if err != nil || sk == "" {
return scrobbler.ErrNotAuthorized
return errors.Join(err, scrobbler.ErrNotAuthorized)
}
li := l.formatListen(&s.MediaFile)
@@ -96,12 +105,12 @@ func (l *listenBrainzAgent) Scrobble(ctx context.Context, userId string, s scrob
isListenBrainzError := errors.As(err, &lbErr)
if !isListenBrainzError {
log.Warn(ctx, "ListenBrainz Scrobble returned HTTP error", "track", s.Title, err)
return scrobbler.ErrRetryLater
return errors.Join(err, scrobbler.ErrRetryLater)
}
if lbErr.Code == 500 || lbErr.Code == 503 {
return scrobbler.ErrRetryLater
return errors.Join(err, scrobbler.ErrRetryLater)
}
return scrobbler.ErrUnrecoverable
return errors.Join(err, scrobbler.ErrUnrecoverable)
}
func (l *listenBrainzAgent) IsAuthorized(ctx context.Context, userId string) bool {

View File

@@ -32,24 +32,26 @@ var _ = Describe("listenBrainzAgent", func() {
agent = listenBrainzConstructor(ds)
agent.client = newClient("http://localhost:8080", httpClient)
track = &model.MediaFile{
ID: "123",
Title: "Track Title",
Album: "Track Album",
Artist: "Track Artist",
TrackNumber: 1,
MbzRecordingID: "mbz-123",
MbzAlbumID: "mbz-456",
MbzArtistID: "mbz-789",
Duration: 142.2,
ID: "123",
Title: "Track Title",
Album: "Track Album",
Artist: "Track Artist",
TrackNumber: 1,
MbzRecordingID: "mbz-123",
MbzAlbumID: "mbz-456",
MbzReleaseGroupID: "mbz-789",
Duration: 142.2,
Participants: map[model.Role]model.ParticipantList{
model.RoleArtist: []model.Participant{
{Artist: model.Artist{ID: "ar-1", Name: "Artist 1", MbzArtistID: "mbz-111"}},
{Artist: model.Artist{ID: "ar-2", Name: "Artist 2", MbzArtistID: "mbz-222"}},
},
},
}
})
Describe("formatListen", func() {
It("constructs the listenInfo properly", func() {
var idArtistId = func(element interface{}) string {
return element.(string)
}
lr := agent.formatListen(track)
Expect(lr).To(MatchAllFields(Fields{
"ListenedAt": Equal(0),
@@ -61,12 +63,12 @@ var _ = Describe("listenBrainzAgent", func() {
"SubmissionClient": Equal(consts.AppName),
"SubmissionClientVersion": Equal(consts.Version),
"TrackNumber": Equal(track.TrackNumber),
"RecordingMbzID": Equal(track.MbzRecordingID),
"ReleaseMbID": Equal(track.MbzAlbumID),
"ArtistMbzIDs": MatchAllElements(idArtistId, Elements{
"mbz-789": Equal(track.MbzArtistID),
}),
"DurationMs": Equal(142200),
"RecordingMBID": Equal(track.MbzRecordingID),
"ReleaseMBID": Equal(track.MbzAlbumID),
"ReleaseGroupMBID": Equal(track.MbzReleaseGroupID),
"ArtistNames": ConsistOf("Artist 1", "Artist 2"),
"ArtistMBIDs": ConsistOf("mbz-111", "mbz-222"),
"DurationMs": Equal(142200),
}),
}),
}))

View File

@@ -76,9 +76,11 @@ type additionalInfo struct {
SubmissionClient string `json:"submission_client,omitempty"`
SubmissionClientVersion string `json:"submission_client_version,omitempty"`
TrackNumber int `json:"tracknumber,omitempty"`
RecordingMbzID string `json:"recording_mbid,omitempty"`
ArtistMbzIDs []string `json:"artist_mbids,omitempty"`
ReleaseMbID string `json:"release_mbid,omitempty"`
ArtistNames []string `json:"artist_names,omitempty"`
ArtistMBIDs []string `json:"artist_mbids,omitempty"`
RecordingMBID string `json:"recording_mbid,omitempty"`
ReleaseMBID string `json:"release_mbid,omitempty"`
ReleaseGroupMBID string `json:"release_group_mbid,omitempty"`
DurationMs int `json:"duration_ms,omitempty"`
}

View File

@@ -74,11 +74,12 @@ var _ = Describe("client", func() {
TrackName: "Track Title",
ReleaseName: "Track Album",
AdditionalInfo: additionalInfo{
TrackNumber: 1,
RecordingMbzID: "mbz-123",
ArtistMbzIDs: []string{"mbz-789"},
ReleaseMbID: "mbz-456",
DurationMs: 142200,
TrackNumber: 1,
ArtistNames: []string{"Artist 1", "Artist 2"},
ArtistMBIDs: []string{"mbz-789", "mbz-012"},
RecordingMBID: "mbz-123",
ReleaseMBID: "mbz-456",
DurationMs: 142200,
},
},
}

2
core/agents/mcp/mcp-server/.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
mcp-server
*.wasm

View File

@@ -0,0 +1,17 @@
# MCP Server (Proof of Concept)
This directory contains the source code for the `mcp-server`, a simple server implementation used as a proof-of-concept (PoC) for the Navidrome Plugin/MCP agent system.
This server is designed to be compiled into a WebAssembly (WASM) module (`.wasm`) using the `wasip1` target.
## Compilation
To compile the server into a WASM module (`mcp-server.wasm`), navigate to this directory in your terminal and run the following command:
```bash
CGO_ENABLED=0 GOOS=wasip1 GOARCH=wasm go build -o mcp-server.wasm .
```
**Note:** This command compiles the WASM module _without_ the `netgo` tag. Networking operations (like HTTP requests) are expected to be handled by host functions provided by the embedding application (Navidrome's `MCPAgent`) rather than directly within the WASM module itself.
Place the resulting `mcp-server.wasm` file where the Navidrome `MCPAgent` expects it (currently configured via the `McpServerPath` constant in `core/agents/mcp/mcp_agent.go`).

View File

@@ -0,0 +1,172 @@
package main
import (
"context"
"encoding/json" // Reusing ErrNotFound from wikidata.go (implicitly via main)
"fmt"
"log"
"net/http"
"net/url"
"strings"
"time"
)
const dbpediaEndpoint = "https://dbpedia.org/sparql"
// Default timeout for DBpedia requests
const defaultDbpediaTimeout = 20 * time.Second
// Can potentially reuse SparqlResult, SparqlBindings, SparqlValue from wikidata.go
// if the structure is identical. Assuming it is for now.
// GetArtistBioFromDBpedia queries DBpedia for an artist's abstract using their name.
func GetArtistBioFromDBpedia(fetcher Fetcher, ctx context.Context, name string) (string, error) {
log.Printf("[MCP] Debug: GetArtistBioFromDBpedia called for name: %s", name)
if name == "" {
log.Printf("[MCP] Error: GetArtistBioFromDBpedia requires a name.")
return "", fmt.Errorf("name is required to query DBpedia by name")
}
// Escape name for SPARQL query literal
escapedName := strings.ReplaceAll(name, "\"", "\\\"")
// SPARQL query using DBpedia ontology (dbo)
// Prefixes are recommended but can be omitted if endpoint resolves them.
// Searching case-insensitively on the label.
// Filtering for dbo:MusicalArtist or dbo:Band.
// Selecting the English abstract.
sparqlQuery := fmt.Sprintf(`
PREFIX dbo: <http://dbpedia.org/ontology/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
SELECT DISTINCT ?abstract WHERE {
?artist rdfs:label ?nameLabel .
FILTER(LCASE(STR(?nameLabel)) = LCASE("%s") && LANG(?nameLabel) = "en")
# Ensure it's a musical artist or band
{ ?artist rdf:type dbo:MusicalArtist } UNION { ?artist rdf:type dbo:Band }
?artist dbo:abstract ?abstract .
FILTER(LANG(?abstract) = "en")
} LIMIT 1`, escapedName)
// Prepare and execute HTTP request
queryValues := url.Values{}
queryValues.Set("query", sparqlQuery)
queryValues.Set("format", "application/sparql-results+json") // DBpedia standard format
reqURL := fmt.Sprintf("%s?%s", dbpediaEndpoint, queryValues.Encode())
log.Printf("[MCP] Debug: DBpedia Bio Request URL: %s", reqURL)
timeout := defaultDbpediaTimeout
if deadline, ok := ctx.Deadline(); ok {
timeout = time.Until(deadline)
}
log.Printf("[MCP] Debug: Fetching from DBpedia with timeout: %v", timeout)
statusCode, bodyBytes, err := fetcher.Fetch(ctx, "GET", reqURL, nil, timeout)
if err != nil {
log.Printf("[MCP] Error: Fetcher failed for DBpedia bio request (name: '%s'): %v", name, err)
return "", fmt.Errorf("failed to execute DBpedia request: %w", err)
}
if statusCode != http.StatusOK {
log.Printf("[MCP] Error: DBpedia bio query failed for name '%s' with status %d: %s", name, statusCode, string(bodyBytes))
return "", fmt.Errorf("DBpedia query failed with status %d: %s", statusCode, string(bodyBytes))
}
log.Printf("[MCP] Debug: DBpedia bio query successful (status %d), %d bytes received.", statusCode, len(bodyBytes))
var result SparqlResult
if err := json.Unmarshal(bodyBytes, &result); err != nil {
// Try reading the raw body for debugging if JSON parsing fails
// (Seek back to the beginning might be needed if already read for error)
// For simplicity, just return the parsing error now.
log.Printf("[MCP] Error: Failed to decode DBpedia bio response for name '%s': %v", name, err)
return "", fmt.Errorf("failed to decode DBpedia response: %w", err)
}
// Extract the abstract
if len(result.Results.Bindings) > 0 {
if abstractVal, ok := result.Results.Bindings[0]["abstract"]; ok {
log.Printf("[MCP] Debug: Found DBpedia abstract for '%s'.", name)
return abstractVal.Value, nil
}
}
// Use the shared ErrNotFound
log.Printf("[MCP] Warn: No abstract found on DBpedia for name '%s'.", name)
return "", ErrNotFound
}
// GetArtistWikipediaURLFromDBpedia queries DBpedia for an artist's Wikipedia URL using their name.
func GetArtistWikipediaURLFromDBpedia(fetcher Fetcher, ctx context.Context, name string) (string, error) {
log.Printf("[MCP] Debug: GetArtistWikipediaURLFromDBpedia called for name: %s", name)
if name == "" {
log.Printf("[MCP] Error: GetArtistWikipediaURLFromDBpedia requires a name.")
return "", fmt.Errorf("name is required to query DBpedia by name for URL")
}
escapedName := strings.ReplaceAll(name, "\"", "\\\"")
// SPARQL query using foaf:isPrimaryTopicOf
sparqlQuery := fmt.Sprintf(`
PREFIX dbo: <http://dbpedia.org/ontology/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT DISTINCT ?wikiPage WHERE {
?artist rdfs:label ?nameLabel .
FILTER(LCASE(STR(?nameLabel)) = LCASE("%s") && LANG(?nameLabel) = "en")
{ ?artist rdf:type dbo:MusicalArtist } UNION { ?artist rdf:type dbo:Band }
?artist foaf:isPrimaryTopicOf ?wikiPage .
# Ensure it links to the English Wikipedia
FILTER(STRSTARTS(STR(?wikiPage), "https://en.wikipedia.org/"))
} LIMIT 1`, escapedName)
// Prepare and execute HTTP request (similar structure to bio query)
queryValues := url.Values{}
queryValues.Set("query", sparqlQuery)
queryValues.Set("format", "application/sparql-results+json")
reqURL := fmt.Sprintf("%s?%s", dbpediaEndpoint, queryValues.Encode())
log.Printf("[MCP] Debug: DBpedia URL Request URL: %s", reqURL)
timeout := defaultDbpediaTimeout
if deadline, ok := ctx.Deadline(); ok {
timeout = time.Until(deadline)
}
log.Printf("[MCP] Debug: Fetching DBpedia URL with timeout: %v", timeout)
statusCode, bodyBytes, err := fetcher.Fetch(ctx, "GET", reqURL, nil, timeout)
if err != nil {
log.Printf("[MCP] Error: Fetcher failed for DBpedia URL request (name: '%s'): %v", name, err)
return "", fmt.Errorf("failed to execute DBpedia URL request: %w", err)
}
if statusCode != http.StatusOK {
log.Printf("[MCP] Error: DBpedia URL query failed for name '%s' with status %d: %s", name, statusCode, string(bodyBytes))
return "", fmt.Errorf("DBpedia URL query failed with status %d: %s", statusCode, string(bodyBytes))
}
log.Printf("[MCP] Debug: DBpedia URL query successful (status %d), %d bytes received.", statusCode, len(bodyBytes))
var result SparqlResult
if err := json.Unmarshal(bodyBytes, &result); err != nil {
log.Printf("[MCP] Error: Failed to decode DBpedia URL response for name '%s': %v", name, err)
return "", fmt.Errorf("failed to decode DBpedia URL response: %w", err)
}
// Extract the URL
if len(result.Results.Bindings) > 0 {
if pageVal, ok := result.Results.Bindings[0]["wikiPage"]; ok {
log.Printf("[MCP] Debug: Found DBpedia Wikipedia URL for '%s': %s", name, pageVal.Value)
return pageVal.Value, nil
}
}
log.Printf("[MCP] Warn: No Wikipedia URL found on DBpedia for name '%s'.", name)
return "", ErrNotFound
}

View File

@@ -0,0 +1,16 @@
package main
import (
"context"
"time"
)
// Fetcher defines an interface for making HTTP requests, abstracting
// over native net/http and WASM host functions.
type Fetcher interface {
// Fetch performs an HTTP request.
// Returns the status code, response body, and any error encountered.
// Note: Implementations should aim to return the body even on non-2xx status codes
// if the body was successfully read, allowing callers to potentially inspect it.
Fetch(ctx context.Context, method, url string, requestBody []byte, timeout time.Duration) (statusCode int, responseBody []byte, err error)
}

View File

@@ -0,0 +1,83 @@
//go:build !wasm
package main
import (
"bytes"
"context"
"fmt"
"io"
"log"
"net/http"
"time"
)
type nativeFetcher struct {
// We could hold a shared client, but creating one per request
// with the specific timeout is simpler for this adapter.
}
// Ensure nativeFetcher implements Fetcher
var _ Fetcher = (*nativeFetcher)(nil)
// NewFetcher creates the default native HTTP fetcher.
func NewFetcher() Fetcher {
log.Println("[MCP] Debug: Using Native HTTP fetcher")
return &nativeFetcher{}
}
func (nf *nativeFetcher) Fetch(ctx context.Context, method, urlStr string, requestBody []byte, timeout time.Duration) (statusCode int, responseBody []byte, err error) {
log.Printf("[MCP] Debug: Native Fetch: Method=%s, URL=%s, Timeout=%v", method, urlStr, timeout)
// Create a client with the specific timeout for this request
client := &http.Client{Timeout: timeout}
var bodyReader io.Reader
if requestBody != nil {
bodyReader = bytes.NewReader(requestBody)
}
req, err := http.NewRequestWithContext(ctx, method, urlStr, bodyReader)
if err != nil {
log.Printf("[MCP] Error: Native Fetch failed to create request: %v", err)
return 0, nil, fmt.Errorf("failed to create native request: %w", err)
}
// Set headers consistent with previous direct client usage
req.Header.Set("Accept", "application/sparql-results+json, application/json")
// Note: Specific User-Agent was set per call site previously, might need adjustment
// if different user agents are desired per service.
req.Header.Set("User-Agent", "MCPGoServerExample/0.1 (Native Client)")
log.Printf("[MCP] Debug: Native Fetch executing request...")
resp, err := client.Do(req)
if err != nil {
// Let context cancellation errors pass through
if ctx.Err() != nil {
log.Printf("[MCP] Debug: Native Fetch context cancelled: %v", ctx.Err())
return 0, nil, ctx.Err()
}
log.Printf("[MCP] Error: Native Fetch HTTP request failed: %v", err)
return 0, nil, fmt.Errorf("native HTTP request failed: %w", err)
}
defer resp.Body.Close()
statusCode = resp.StatusCode
log.Printf("[MCP] Debug: Native Fetch received status code: %d", statusCode)
responseBodyBytes, readErr := io.ReadAll(resp.Body)
if readErr != nil {
// Still return status code if body read fails
log.Printf("[MCP] Error: Native Fetch failed to read response body: %v", readErr)
return statusCode, nil, fmt.Errorf("failed to read native response body: %w", readErr)
}
responseBody = responseBodyBytes
log.Printf("[MCP] Debug: Native Fetch read %d bytes from response body", len(responseBodyBytes))
// Mimic behavior of returning body even on error status
if statusCode < 200 || statusCode >= 300 {
log.Printf("[MCP] Warn: Native Fetch request failed with status %d. Body: %s", statusCode, string(responseBody))
return statusCode, responseBody, fmt.Errorf("native request failed with status %d", statusCode)
}
log.Printf("[MCP] Debug: Native Fetch completed successfully.")
return statusCode, responseBody, nil
}

View File

@@ -0,0 +1,171 @@
//go:build wasm
package main
import (
"context"
"errors"
"fmt"
"log"
"time"
"unsafe"
)
// --- WASM Host Function Import --- (Copied from user prompt)
//go:wasmimport env http_fetch
//go:noescape
func http_fetch(
// Request details
urlPtr, urlLen uint32,
methodPtr, methodLen uint32,
bodyPtr, bodyLen uint32,
timeoutMillis uint32,
// Result pointers
resultStatusPtr uint32,
resultBodyPtr uint32, resultBodyCapacity uint32, resultBodyLenPtr uint32,
resultErrorPtr uint32, resultErrorCapacity uint32, resultErrorLenPtr uint32,
) uint32 // 0 on success, 1 on host error
// --- Go Wrapper for Host Function --- (Copied from user prompt)
const (
defaultResponseBodyCapacity = 1024 * 10 // 10 KB for response body
defaultResponseErrorCapacity = 1024 // 1 KB for error messages
)
// callHostHTTPFetch provides a Go-friendly interface to the http_fetch host function.
func callHostHTTPFetch(ctx context.Context, method, url string, requestBody []byte, timeout time.Duration) (statusCode int, responseBody []byte, err error) {
log.Printf("[MCP] Debug: WASM Fetch (Host Call): Method=%s, URL=%s, Timeout=%v", method, url, timeout)
// --- Prepare Input Pointers ---
urlPtr, urlLen := stringToPtr(url)
methodPtr, methodLen := stringToPtr(method)
bodyPtr, bodyLen := bytesToPtr(requestBody)
timeoutMillis := uint32(timeout.Milliseconds())
if timeoutMillis <= 0 {
timeoutMillis = 30000 // Default 30 seconds if 0 or negative
}
if timeout == 0 {
// Handle case where context might already be cancelled
select {
case <-ctx.Done():
log.Printf("[MCP] Debug: WASM Fetch context cancelled before host call: %v", ctx.Err())
return 0, nil, ctx.Err()
default:
}
}
// --- Prepare Output Buffers and Pointers ---
resultBodyBuffer := make([]byte, defaultResponseBodyCapacity)
resultErrorBuffer := make([]byte, defaultResponseErrorCapacity)
resultStatus := uint32(0)
resultBodyLen := uint32(0)
resultErrorLen := uint32(0)
resultStatusPtr := &resultStatus
resultBodyPtr, resultBodyCapacity := bytesToPtr(resultBodyBuffer)
resultBodyLenPtr := &resultBodyLen
resultErrorPtr, resultErrorCapacity := bytesToPtr(resultErrorBuffer)
resultErrorLenPtr := &resultErrorLen
// --- Call the Host Function ---
log.Printf("[MCP] Debug: WASM Fetch calling host function http_fetch...")
hostReturnCode := http_fetch(
urlPtr, urlLen,
methodPtr, methodLen,
bodyPtr, bodyLen,
timeoutMillis,
uint32(uintptr(unsafe.Pointer(resultStatusPtr))),
resultBodyPtr, resultBodyCapacity, uint32(uintptr(unsafe.Pointer(resultBodyLenPtr))),
resultErrorPtr, resultErrorCapacity, uint32(uintptr(unsafe.Pointer(resultErrorLenPtr))),
)
log.Printf("[MCP] Debug: WASM Fetch host function returned code: %d", hostReturnCode)
// --- Process Results ---
if hostReturnCode != 0 {
err = errors.New("host function http_fetch failed internally")
log.Printf("[MCP] Error: WASM Fetch host function failed: %v", err)
return 0, nil, err
}
statusCode = int(resultStatus)
log.Printf("[MCP] Debug: WASM Fetch received status code from host: %d", statusCode)
if resultErrorLen > 0 {
actualErrorLen := min(resultErrorLen, resultErrorCapacity)
errMsg := string(resultErrorBuffer[:actualErrorLen])
err = errors.New(errMsg)
log.Printf("[MCP] Error: WASM Fetch received error from host: %s", errMsg)
return statusCode, nil, err
}
if resultBodyLen > 0 {
actualBodyLen := min(resultBodyLen, resultBodyCapacity)
responseBody = make([]byte, actualBodyLen)
copy(responseBody, resultBodyBuffer[:actualBodyLen])
log.Printf("[MCP] Debug: WASM Fetch received %d bytes from host body (reported size: %d)", actualBodyLen, resultBodyLen)
if resultBodyLen > resultBodyCapacity {
err = fmt.Errorf("response body truncated: received %d bytes, but actual size was %d", actualBodyLen, resultBodyLen)
log.Printf("[MCP] Warn: WASM Fetch %v", err)
return statusCode, responseBody, err // Return truncated body with error
}
log.Printf("[MCP] Debug: WASM Fetch completed successfully.")
return statusCode, responseBody, nil
}
log.Printf("[MCP] Debug: WASM Fetch completed successfully (no body, no error).")
return statusCode, nil, nil
}
// --- Pointer Helper Functions --- (Copied from user prompt)
func stringToPtr(s string) (ptr uint32, length uint32) {
if len(s) == 0 {
return 0, 0
}
// Use unsafe.StringData for potentially safer pointer access in modern Go
// Needs Go 1.20+
// return uint32(uintptr(unsafe.Pointer(unsafe.StringData(s)))), uint32(len(s))
// Fallback to slice conversion for broader compatibility / if StringData isn't available
buf := []byte(s)
return bytesToPtr(buf)
}
func bytesToPtr(b []byte) (ptr uint32, length uint32) {
if len(b) == 0 {
return 0, 0
}
// Use unsafe.SliceData for potentially safer pointer access in modern Go
// Needs Go 1.20+
// return uint32(uintptr(unsafe.Pointer(unsafe.SliceData(b)))), uint32(len(b))
// Fallback for broader compatibility
return uint32(uintptr(unsafe.Pointer(&b[0]))), uint32(len(b))
}
func min(a, b uint32) uint32 {
if a < b {
return a
}
return b
}
// --- WASM Fetcher Implementation ---
type wasmFetcher struct{}
// Ensure wasmFetcher implements Fetcher
var _ Fetcher = (*wasmFetcher)(nil)
// NewFetcher creates the WASM host function fetcher.
func NewFetcher() Fetcher {
log.Println("[MCP] Debug: Using WASM host fetcher")
return &wasmFetcher{}
}
func (wf *wasmFetcher) Fetch(ctx context.Context, method, url string, requestBody []byte, timeout time.Duration) (statusCode int, responseBody []byte, err error) {
// Directly call the wrapper which now contains logging
return callHostHTTPFetch(ctx, method, url, requestBody, timeout)
}

View File

@@ -0,0 +1,19 @@
module mcp-server
go 1.24.2
require github.com/metoro-io/mcp-golang v0.11.0
require (
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/buger/jsonparser v1.1.1 // indirect
github.com/invopop/jsonschema v0.12.0 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

View File

@@ -0,0 +1,34 @@
github.com/bahlo/generic-list-go v0.2.0 h1:5sz/EEAK+ls5wF+NeqDpk5+iNdMDXrh3z3nPnH1Wvgk=
github.com/bahlo/generic-list-go v0.2.0/go.mod h1:2KvAjgMlE5NNynlg/5iLrrCCZ2+5xWbdbCW3pNTGyYg=
github.com/buger/jsonparser v1.1.1 h1:2PnMjfWD7wBILjqQbt530v576A/cAbQvEW9gGIpYMUs=
github.com/buger/jsonparser v1.1.1/go.mod h1:6RYKKt7H4d4+iWqouImQ9R2FZql3VbhNgx27UK13J/0=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/invopop/jsonschema v0.12.0 h1:6ovsNSuvn9wEQVOyc72aycBMVQFKz7cPdMJn10CvzRI=
github.com/invopop/jsonschema v0.12.0/go.mod h1:ffZ5Km5SWWRAIN6wbDXItl95euhFz2uON45H2qjYt+0=
github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/metoro-io/mcp-golang v0.11.0 h1:1k+VSE9QaeMTLn0gJ3FgE/DcjsCBsLFnz5eSFbgXUiI=
github.com/metoro-io/mcp-golang v0.11.0/go.mod h1:ifLP9ZzKpN1UqFWNTpAHOqSvNkMK6b7d1FSZ5Lu0lN0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
github.com/wk8/go-ordered-map/v2 v2.1.8 h1:5h/BUHu93oj4gIdvHHHGsScSTMijfx5PeYkE/fJgbpc=
github.com/wk8/go-ordered-map/v2 v2.1.8/go.mod h1:5nJHM5DyteebpVlHnWMV0rPz6Zp7+xBAnxjb1X5vnTw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -0,0 +1,289 @@
package main
import (
"context"
"errors"
"flag"
"fmt"
"log"
"net/url"
"os"
mcp_golang "github.com/metoro-io/mcp-golang"
"github.com/metoro-io/mcp-golang/transport/stdio"
)
type Content struct {
Title string `json:"title" jsonschema:"required,description=The title to submit"`
Description *string `json:"description" jsonschema:"description=The description to submit"`
}
type MyFunctionsArguments struct {
Submitter string `json:"submitter" jsonschema:"required,description=The name of the thing calling this tool (openai, google, claude, etc)"`
Content Content `json:"content" jsonschema:"required,description=The content of the message"`
}
type ArtistBiography struct {
ID string `json:"id" jsonschema:"required,description=The id of the artist"`
Name string `json:"name" jsonschema:"required,description=The name of the artist"`
MBID string `json:"mbid" jsonschema:"description=The mbid of the artist"`
}
type ArtistURLArgs struct {
ID string `json:"id" jsonschema:"required,description=The id of the artist"`
Name string `json:"name" jsonschema:"required,description=The name of the artist"`
MBID string `json:"mbid" jsonschema:"description=The mbid of the artist"`
}
func main() {
log.Println("[MCP] Starting mcp-server...")
done := make(chan struct{})
// Create the appropriate fetcher (native or WASM based on build tags)
log.Printf("[MCP] Debug: Creating fetcher...")
fetcher := NewFetcher()
log.Printf("[MCP] Debug: Fetcher created successfully.")
// --- Command Line Flag Handling ---
nameFlag := flag.String("name", "", "Artist name to query directly")
mbidFlag := flag.String("mbid", "", "Artist MBID to query directly")
flag.Parse()
if *nameFlag != "" || *mbidFlag != "" {
log.Printf("[MCP] Debug: Running tools directly via CLI flags (Name: '%s', MBID: '%s')", *nameFlag, *mbidFlag)
fmt.Println("--- Running Tools Directly ---")
// Call getArtistBiography
fmt.Printf("Calling get_artist_biography (Name: '%s', MBID: '%s')...\n", *nameFlag, *mbidFlag)
if *mbidFlag == "" && *nameFlag == "" {
fmt.Println(" Error: --mbid or --name is required for get_artist_biography")
} else {
// Use context.Background for CLI calls
log.Printf("[MCP] Debug: CLI calling getArtistBiography...")
bio, bioErr := getArtistBiography(fetcher, context.Background(), "cli", *nameFlag, *mbidFlag)
if bioErr != nil {
fmt.Printf(" Error: %v\n", bioErr)
log.Printf("[MCP] Error: CLI getArtistBiography failed: %v", bioErr)
} else {
fmt.Printf(" Result: %s\n", bio)
log.Printf("[MCP] Debug: CLI getArtistBiography succeeded.")
}
}
// Call getArtistURL
fmt.Printf("Calling get_artist_url (Name: '%s', MBID: '%s')...\n", *nameFlag, *mbidFlag)
if *mbidFlag == "" && *nameFlag == "" {
fmt.Println(" Error: --mbid or --name is required for get_artist_url")
} else {
log.Printf("[MCP] Debug: CLI calling getArtistURL...")
urlResult, urlErr := getArtistURL(fetcher, context.Background(), "cli", *nameFlag, *mbidFlag)
if urlErr != nil {
fmt.Printf(" Error: %v\n", urlErr)
log.Printf("[MCP] Error: CLI getArtistURL failed: %v", urlErr)
} else {
fmt.Printf(" Result: %s\n", urlResult)
log.Printf("[MCP] Debug: CLI getArtistURL succeeded.")
}
}
fmt.Println("-----------------------------")
log.Printf("[MCP] Debug: CLI execution finished.")
return // Exit after direct execution
}
// --- End Command Line Flag Handling ---
log.Printf("[MCP] Debug: Initializing MCP server...")
server := mcp_golang.NewServer(stdio.NewStdioServerTransport())
log.Printf("[MCP] Debug: Registering tool 'hello'...")
err := server.RegisterTool("hello", "Say hello to a person", func(arguments MyFunctionsArguments) (*mcp_golang.ToolResponse, error) {
log.Printf("[MCP] Debug: Tool 'hello' called with args: %+v", arguments)
return mcp_golang.NewToolResponse(mcp_golang.NewTextContent(fmt.Sprintf("Hello, %server!", arguments.Submitter))), nil
})
if err != nil {
log.Fatalf("[MCP] Fatal: Failed to register tool 'hello': %v", err)
}
log.Printf("[MCP] Debug: Registering tool 'get_artist_biography'...")
err = server.RegisterTool("get_artist_biography", "Get the biography of an artist", func(arguments ArtistBiography) (*mcp_golang.ToolResponse, error) {
log.Printf("[MCP] Debug: Tool 'get_artist_biography' called with args: %+v", arguments)
// Using background context in handlers as request context isn't passed through MCP library currently
bio, err := getArtistBiography(fetcher, context.Background(), arguments.ID, arguments.Name, arguments.MBID)
if err != nil {
log.Printf("[MCP] Error: getArtistBiography handler failed: %v", err)
return nil, fmt.Errorf("handler returned an error: %w", err) // Return structured error
}
log.Printf("[MCP] Debug: Tool 'get_artist_biography' succeeded.")
return mcp_golang.NewToolResponse(mcp_golang.NewTextContent(bio)), nil
})
if err != nil {
log.Fatalf("[MCP] Fatal: Failed to register tool 'get_artist_biography': %v", err)
}
log.Printf("[MCP] Debug: Registering tool 'get_artist_url'...")
err = server.RegisterTool("get_artist_url", "Get the artist's specific Wikipedia URL via MBID, or a search URL using name as fallback", func(arguments ArtistURLArgs) (*mcp_golang.ToolResponse, error) {
log.Printf("[MCP] Debug: Tool 'get_artist_url' called with args: %+v", arguments)
urlResult, err := getArtistURL(fetcher, context.Background(), arguments.ID, arguments.Name, arguments.MBID)
if err != nil {
log.Printf("[MCP] Error: getArtistURL handler failed: %v", err)
return nil, fmt.Errorf("handler returned an error: %w", err)
}
log.Printf("[MCP] Debug: Tool 'get_artist_url' succeeded.")
return mcp_golang.NewToolResponse(mcp_golang.NewTextContent(urlResult)), nil
})
if err != nil {
log.Fatalf("[MCP] Fatal: Failed to register tool 'get_artist_url': %v", err)
}
log.Printf("[MCP] Debug: Registering prompt 'prompt_test'...")
err = server.RegisterPrompt("prompt_test", "This is a test prompt", func(arguments Content) (*mcp_golang.PromptResponse, error) {
log.Printf("[MCP] Debug: Prompt 'prompt_test' called with args: %+v", arguments)
return mcp_golang.NewPromptResponse("description", mcp_golang.NewPromptMessage(mcp_golang.NewTextContent(fmt.Sprintf("Hello, %server!", arguments.Title)), mcp_golang.RoleUser)), nil
})
if err != nil {
log.Fatalf("[MCP] Fatal: Failed to register prompt 'prompt_test': %v", err)
}
log.Printf("[MCP] Debug: Registering resource 'test://resource'...")
err = server.RegisterResource("test://resource", "resource_test", "This is a test resource", "application/json", func() (*mcp_golang.ResourceResponse, error) {
log.Printf("[MCP] Debug: Resource 'test://resource' called")
return mcp_golang.NewResourceResponse(mcp_golang.NewTextEmbeddedResource("test://resource", "This is a test resource", "application/json")), nil
})
if err != nil {
log.Fatalf("[MCP] Fatal: Failed to register resource 'test://resource': %v", err)
}
log.Printf("[MCP] Debug: Registering resource 'file://app_logs'...")
err = server.RegisterResource("file://app_logs", "app_logs", "The app logs", "text/plain", func() (*mcp_golang.ResourceResponse, error) {
log.Printf("[MCP] Debug: Resource 'file://app_logs' called")
return mcp_golang.NewResourceResponse(mcp_golang.NewTextEmbeddedResource("file://app_logs", "This is a test resource", "text/plain")), nil
})
if err != nil {
log.Fatalf("[MCP] Fatal: Failed to register resource 'file://app_logs': %v", err)
}
log.Println("[MCP] MCP server initialized and starting to serve...")
err = server.Serve()
if err != nil {
log.Fatalf("[MCP] Fatal: Server exited with error: %v", err)
}
log.Println("[MCP] Server exited cleanly.")
<-done // Keep running until interrupted (though server.Serve() is blocking)
}
func getArtistBiography(fetcher Fetcher, ctx context.Context, id, name, mbid string) (string, error) {
log.Printf("[MCP] Debug: getArtistBiography called (id: %s, name: %s, mbid: %s)", id, name, mbid)
if mbid == "" {
fmt.Fprintf(os.Stderr, "MBID not provided, attempting DBpedia lookup by name: %s\n", name)
log.Printf("[MCP] Debug: MBID not provided, attempting DBpedia lookup by name: %s", name)
} else {
// 1. Attempt Wikidata MBID lookup first
log.Printf("[MCP] Debug: Attempting Wikidata URL lookup for MBID: %s", mbid)
wikiURL, err := GetArtistWikipediaURL(fetcher, ctx, mbid)
if err == nil {
// 1a. Found Wikidata URL, now fetch from Wikipedia API
log.Printf("[MCP] Debug: Found Wikidata URL '%s', fetching bio from Wikipedia API...", wikiURL)
bio, errBio := GetBioFromWikipediaAPI(fetcher, ctx, wikiURL)
if errBio == nil {
log.Printf("[MCP] Debug: Successfully fetched bio from Wikipedia API for '%s'.", name)
return bio, nil // Success via Wikidata/Wikipedia!
} else {
// Failed to get bio even though URL was found
log.Printf("[MCP] Error: Found Wikipedia URL (%s) via MBID %s, but failed to fetch bio: %v", wikiURL, mbid, errBio)
fmt.Fprintf(os.Stderr, "Found Wikipedia URL (%s) via MBID %s, but failed to fetch bio: %v\n", wikiURL, mbid, errBio)
// Fall through to try DBpedia by name as a last resort?
// Let's fall through for now.
}
} else if !errors.Is(err, ErrNotFound) {
// Wikidata lookup failed for a reason other than not found (e.g., network)
log.Printf("[MCP] Error: Wikidata URL lookup failed for MBID %s (non-NotFound error): %v", mbid, err)
fmt.Fprintf(os.Stderr, "Wikidata URL lookup failed for MBID %s (non-NotFound error): %v\n", mbid, err)
// Don't proceed to DBpedia name lookup if Wikidata had a technical failure
return "", fmt.Errorf("Wikidata lookup failed: %w", err)
} else {
// Wikidata lookup returned ErrNotFound for MBID
log.Printf("[MCP] Debug: MBID %s not found on Wikidata, attempting DBpedia lookup by name: %s", mbid, name)
fmt.Fprintf(os.Stderr, "MBID %s not found on Wikidata, attempting DBpedia lookup by name: %s\n", mbid, name)
}
}
// 2. Attempt DBpedia lookup by name (if MBID was missing or failed with ErrNotFound)
if name == "" {
log.Printf("[MCP] Error: Cannot find artist bio: MBID lookup failed/missing, and no name provided.")
return "", fmt.Errorf("cannot find artist: MBID lookup failed or MBID not provided, and no name provided for DBpedia fallback")
}
log.Printf("[MCP] Debug: Attempting DBpedia bio lookup by name: %s", name)
dbpediaBio, errDb := GetArtistBioFromDBpedia(fetcher, ctx, name)
if errDb == nil {
log.Printf("[MCP] Debug: Successfully fetched bio from DBpedia for '%s'.", name)
return dbpediaBio, nil // Success via DBpedia!
}
// 3. If both Wikidata (MBID) and DBpedia (Name) failed
if errors.Is(errDb, ErrNotFound) {
log.Printf("[MCP] Error: Artist '%s' (MBID: %s) not found via Wikidata or DBpedia name lookup.", name, mbid)
return "", fmt.Errorf("artist '%s' (MBID: %s) not found via Wikidata MBID or DBpedia Name lookup", name, mbid)
}
// Return DBpedia's error if it wasn't ErrNotFound
log.Printf("[MCP] Error: DBpedia lookup failed for name '%s': %v", name, errDb)
return "", fmt.Errorf("DBpedia lookup failed for name '%s': %w", name, errDb)
}
// getArtistURL attempts to find the specific Wikipedia URL using MBID (via Wikidata),
// then by Name (via DBpedia), falling back to a search URL using name.
func getArtistURL(fetcher Fetcher, ctx context.Context, id, name, mbid string) (string, error) {
log.Printf("[MCP] Debug: getArtistURL called (id: %s, name: %s, mbid: %s)", id, name, mbid)
if mbid == "" {
fmt.Fprintf(os.Stderr, "getArtistURL: MBID not provided, attempting DBpedia lookup by name: %s\n", name)
log.Printf("[MCP] Debug: getArtistURL: MBID not provided, attempting DBpedia lookup by name: %s", name)
} else {
// Try to get the specific URL from Wikidata using MBID
log.Printf("[MCP] Debug: getArtistURL: Attempting Wikidata URL lookup for MBID: %s", mbid)
wikiURL, err := GetArtistWikipediaURL(fetcher, ctx, mbid)
if err == nil && wikiURL != "" {
log.Printf("[MCP] Debug: getArtistURL: Found specific URL '%s' via Wikidata MBID.", wikiURL)
return wikiURL, nil // Found specific URL via MBID
}
// Log error if Wikidata lookup failed for reasons other than not found
if err != nil && !errors.Is(err, ErrNotFound) {
log.Printf("[MCP] Error: getArtistURL: Wikidata URL lookup failed for MBID %s (non-NotFound error): %v", mbid, err)
fmt.Fprintf(os.Stderr, "getArtistURL: Wikidata URL lookup failed for MBID %s (non-NotFound error): %v\n", mbid, err)
// Fall through to try DBpedia if name is available
} else if errors.Is(err, ErrNotFound) {
log.Printf("[MCP] Debug: getArtistURL: MBID %s not found on Wikidata, attempting DBpedia lookup by name: %s", mbid, name)
fmt.Fprintf(os.Stderr, "getArtistURL: MBID %s not found on Wikidata, attempting DBpedia lookup by name: %s\n", mbid, name)
}
}
// Fallback 1: Try DBpedia lookup by name
if name != "" {
log.Printf("[MCP] Debug: getArtistURL: Attempting DBpedia URL lookup by name: %s", name)
dbpediaWikiURL, errDb := GetArtistWikipediaURLFromDBpedia(fetcher, ctx, name)
if errDb == nil && dbpediaWikiURL != "" {
log.Printf("[MCP] Debug: getArtistURL: Found specific URL '%s' via DBpedia Name lookup.", dbpediaWikiURL)
return dbpediaWikiURL, nil // Found specific URL via DBpedia Name lookup
}
// Log error if DBpedia lookup failed for reasons other than not found
if errDb != nil && !errors.Is(errDb, ErrNotFound) {
log.Printf("[MCP] Error: getArtistURL: DBpedia URL lookup failed for name '%s' (non-NotFound error): %v", name, errDb)
fmt.Fprintf(os.Stderr, "getArtistURL: DBpedia URL lookup failed for name '%s' (non-NotFound error): %v\n", name, errDb)
// Fall through to search URL fallback
} else if errors.Is(errDb, ErrNotFound) {
log.Printf("[MCP] Debug: getArtistURL: Name '%s' not found on DBpedia, attempting search fallback", name)
fmt.Fprintf(os.Stderr, "getArtistURL: Name '%s' not found on DBpedia, attempting search fallback\n", name)
}
}
// Fallback 2: Generate a search URL if name is provided
if name != "" {
searchURL := fmt.Sprintf("https://en.wikipedia.org/w/index.php?search=%s", url.QueryEscape(name))
log.Printf("[MCP] Debug: getArtistURL: Falling back to search URL: %s", searchURL)
fmt.Fprintf(os.Stderr, "getArtistURL: Falling back to search URL: %s\n", searchURL)
return searchURL, nil
}
// Final error: MBID lookup failed (or no MBID given) AND no name provided for fallback
log.Printf("[MCP] Error: getArtistURL: Cannot generate Wikipedia URL: Lookups failed and no name provided.")
return "", fmt.Errorf("cannot generate Wikipedia URL: Wikidata/DBpedia lookups failed and no artist name provided for search fallback")
}

View File

@@ -0,0 +1,205 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"log"
"net/http"
"net/url"
"os"
"time"
)
const wikidataEndpoint = "https://query.wikidata.org/sparql"
// ErrNotFound indicates a specific item (like an artist or URL) was not found on Wikidata.
var ErrNotFound = errors.New("item not found on Wikidata")
// Wikidata SPARQL query result structures
type SparqlResult struct {
Results SparqlBindings `json:"results"`
}
type SparqlBindings struct {
Bindings []map[string]SparqlValue `json:"bindings"`
}
type SparqlValue struct {
Type string `json:"type"`
Value string `json:"value"`
Lang string `json:"xml:lang,omitempty"` // Handle language tags like "en"
}
// GetArtistBioFromWikidata queries Wikidata for an artist's description using their MBID.
// NOTE: This function is currently UNUSED as the main logic prefers Wikipedia/DBpedia.
func GetArtistBioFromWikidata(client *http.Client, mbid string) (string, error) {
log.Printf("[MCP] Debug: GetArtistBioFromWikidata called for MBID: %s", mbid)
if mbid == "" {
log.Printf("[MCP] Error: GetArtistBioFromWikidata requires an MBID.")
return "", fmt.Errorf("MBID is required to query Wikidata")
}
// SPARQL query to find the English description for an entity with a specific MusicBrainz ID
sparqlQuery := fmt.Sprintf(`
SELECT ?artistDescription WHERE {
?artist wdt:P434 "%s" . # P434 is the property for MusicBrainz artist ID
OPTIONAL {
?artist schema:description ?artistDescription .
FILTER(LANG(?artistDescription) = "en")
}
SERVICE wikibase:label { bd:serviceParam wikibase:language "en". }
}
LIMIT 1`, mbid)
// Prepare the HTTP request
queryValues := url.Values{}
queryValues.Set("query", sparqlQuery)
queryValues.Set("format", "json")
reqURL := fmt.Sprintf("%s?%s", wikidataEndpoint, queryValues.Encode())
log.Printf("[MCP] Debug: Wikidata Bio Request URL: %s", reqURL)
req, err := http.NewRequest("GET", reqURL, nil)
if err != nil {
log.Printf("[MCP] Error: Failed to create Wikidata bio request: %v", err)
return "", fmt.Errorf("failed to create Wikidata request: %w", err)
}
req.Header.Set("Accept", "application/sparql-results+json")
req.Header.Set("User-Agent", "MCPGoServerExample/0.1 (https://example.com/contact)") // Good practice to identify your client
// Execute the request
log.Printf("[MCP] Debug: Executing Wikidata bio request...")
resp, err := client.Do(req)
if err != nil {
log.Printf("[MCP] Error: Failed to execute Wikidata bio request: %v", err)
return "", fmt.Errorf("failed to execute Wikidata request: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
// Attempt to read body for more error info, but don't fail if it doesn't work
bodyBytes, readErr := io.ReadAll(resp.Body)
errorMsg := "Could not read error body"
if readErr == nil {
errorMsg = string(bodyBytes)
}
log.Printf("[MCP] Error: Wikidata bio query failed with status %d: %s", resp.StatusCode, errorMsg)
return "", fmt.Errorf("Wikidata query failed with status %d: %s", resp.StatusCode, errorMsg)
}
log.Printf("[MCP] Debug: Wikidata bio query successful (status %d).", resp.StatusCode)
// Parse the response
var result SparqlResult
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
log.Printf("[MCP] Error: Failed to decode Wikidata bio response: %v", err)
return "", fmt.Errorf("failed to decode Wikidata response: %w", err)
}
// Extract the description
if len(result.Results.Bindings) > 0 {
if descriptionVal, ok := result.Results.Bindings[0]["artistDescription"]; ok {
log.Printf("[MCP] Debug: Found description for MBID %s", mbid)
return descriptionVal.Value, nil
}
}
log.Printf("[MCP] Warn: No English description found on Wikidata for MBID %s", mbid)
return "", fmt.Errorf("no English description found on Wikidata for MBID %s", mbid)
}
// GetArtistWikipediaURL queries Wikidata for an artist's English Wikipedia page URL using MBID.
// It tries searching by MBID first, then falls back to searching by name.
func GetArtistWikipediaURL(fetcher Fetcher, ctx context.Context, mbid string) (string, error) {
log.Printf("[MCP] Debug: GetArtistWikipediaURL called for MBID: %s", mbid)
// 1. Try finding by MBID
if mbid == "" {
log.Printf("[MCP] Error: GetArtistWikipediaURL requires an MBID.")
return "", fmt.Errorf("MBID is required to find Wikipedia URL on Wikidata")
} else {
// SPARQL query to find the enwiki URL for an entity with a specific MusicBrainz ID
sparqlQuery := fmt.Sprintf(`
SELECT ?article WHERE {
?artist wdt:P434 "%s" . # P434 is MusicBrainz artist ID
?article schema:about ?artist ;
schema:isPartOf <https://en.wikipedia.org/> .
}
LIMIT 1`, mbid)
log.Printf("[MCP] Debug: Executing Wikidata URL query for MBID: %s", mbid)
foundURL, err := executeWikidataURLQuery(fetcher, ctx, sparqlQuery)
if err == nil && foundURL != "" {
log.Printf("[MCP] Debug: Found Wikipedia URL '%s' via MBID %s", foundURL, mbid)
return foundURL, nil // Found via MBID
}
// Use the specific ErrNotFound
if errors.Is(err, ErrNotFound) {
log.Printf("[MCP] Debug: MBID %s not found on Wikidata for URL lookup.", mbid)
return "", ErrNotFound // Explicitly return ErrNotFound
}
// Log other errors
if err != nil {
log.Printf("[MCP] Error: Wikidata URL lookup via MBID %s failed: %v", mbid, err)
fmt.Fprintf(os.Stderr, "Wikidata URL lookup via MBID %s failed: %v\n", mbid, err)
return "", fmt.Errorf("Wikidata URL lookup via MBID failed: %w", err)
}
}
// Should ideally not be reached if MBID is required and lookup failed or was not found
log.Printf("[MCP] Warn: Reached end of GetArtistWikipediaURL unexpectedly for MBID %s", mbid)
return "", ErrNotFound // Return ErrNotFound if somehow reached
}
// executeWikidataURLQuery is a helper to run SPARQL and extract the first bound URL for '?article'.
func executeWikidataURLQuery(fetcher Fetcher, ctx context.Context, sparqlQuery string) (string, error) {
log.Printf("[MCP] Debug: executeWikidataURLQuery called.")
queryValues := url.Values{}
queryValues.Set("query", sparqlQuery)
queryValues.Set("format", "json")
reqURL := fmt.Sprintf("%s?%s", wikidataEndpoint, queryValues.Encode())
log.Printf("[MCP] Debug: Wikidata Sparql Request URL: %s", reqURL)
// Directly use the fetcher
// Note: Headers (Accept, User-Agent) are now handled by the Fetcher implementation
// The WASM fetcher currently doesn't support setting them via the host func interface.
// Timeout is handled via context for native, and passed to host func for WASM.
// Let's use a default timeout here if not provided via context (e.g., 15s)
// TODO: Consider making timeout configurable or passed down
timeout := 15 * time.Second
if deadline, ok := ctx.Deadline(); ok {
timeout = time.Until(deadline)
}
log.Printf("[MCP] Debug: Fetching from Wikidata with timeout: %v", timeout)
statusCode, bodyBytes, err := fetcher.Fetch(ctx, "GET", reqURL, nil, timeout)
if err != nil {
log.Printf("[MCP] Error: Fetcher failed for Wikidata SPARQL request: %v", err)
return "", fmt.Errorf("failed to execute Wikidata request: %w", err)
}
// Check status code. Fetcher interface implies body might be returned even on error.
if statusCode != http.StatusOK {
log.Printf("[MCP] Error: Wikidata SPARQL query failed with status %d: %s", statusCode, string(bodyBytes))
return "", fmt.Errorf("Wikidata query failed with status %d: %s", statusCode, string(bodyBytes))
}
log.Printf("[MCP] Debug: Wikidata SPARQL query successful (status %d), %d bytes received.", statusCode, len(bodyBytes))
var result SparqlResult
if err := json.Unmarshal(bodyBytes, &result); err != nil { // Use Unmarshal for byte slice
log.Printf("[MCP] Error: Failed to decode Wikidata SPARQL response: %v", err)
return "", fmt.Errorf("failed to decode Wikidata response: %w", err)
}
if len(result.Results.Bindings) > 0 {
if articleVal, ok := result.Results.Bindings[0]["article"]; ok {
log.Printf("[MCP] Debug: Found Wikidata article URL: %s", articleVal.Value)
return articleVal.Value, nil
}
}
log.Printf("[MCP] Debug: No Wikidata article URL found in SPARQL response.")
return "", ErrNotFound
}

View File

@@ -0,0 +1,121 @@
package main
import (
"context"
"encoding/json"
"fmt"
"log"
"net/http"
"net/url"
"strings"
"time"
)
const mediaWikiAPIEndpoint = "https://en.wikipedia.org/w/api.php"
// Structures for parsing MediaWiki API response (query extracts)
type MediaWikiQueryResult struct {
Query MediaWikiQuery `json:"query"`
}
type MediaWikiQuery struct {
Pages map[string]MediaWikiPage `json:"pages"`
}
type MediaWikiPage struct {
PageID int `json:"pageid"`
Ns int `json:"ns"`
Title string `json:"title"`
Extract string `json:"extract"`
}
// Default timeout for Wikipedia API requests
const defaultWikipediaTimeout = 15 * time.Second
// GetBioFromWikipediaAPI fetches the introductory text of a Wikipedia page.
func GetBioFromWikipediaAPI(fetcher Fetcher, ctx context.Context, wikipediaURL string) (string, error) {
log.Printf("[MCP] Debug: GetBioFromWikipediaAPI called for URL: %s", wikipediaURL)
pageTitle, err := extractPageTitleFromURL(wikipediaURL)
if err != nil {
log.Printf("[MCP] Error: Could not extract title from Wikipedia URL '%s': %v", wikipediaURL, err)
return "", fmt.Errorf("could not extract title from Wikipedia URL %s: %w", wikipediaURL, err)
}
log.Printf("[MCP] Debug: Extracted Wikipedia page title: %s", pageTitle)
// Prepare API request parameters
apiParams := url.Values{}
apiParams.Set("action", "query")
apiParams.Set("format", "json")
apiParams.Set("prop", "extracts") // Request page extracts
apiParams.Set("exintro", "true") // Get only the intro section
apiParams.Set("explaintext", "true") // Get plain text instead of HTML
apiParams.Set("titles", pageTitle) // Specify the page title
apiParams.Set("redirects", "1") // Follow redirects
reqURL := fmt.Sprintf("%s?%s", mediaWikiAPIEndpoint, apiParams.Encode())
log.Printf("[MCP] Debug: MediaWiki API Request URL: %s", reqURL)
timeout := defaultWikipediaTimeout
if deadline, ok := ctx.Deadline(); ok {
timeout = time.Until(deadline)
}
log.Printf("[MCP] Debug: Fetching from MediaWiki with timeout: %v", timeout)
statusCode, bodyBytes, err := fetcher.Fetch(ctx, "GET", reqURL, nil, timeout)
if err != nil {
log.Printf("[MCP] Error: Fetcher failed for MediaWiki request (title: '%s'): %v", pageTitle, err)
return "", fmt.Errorf("failed to execute MediaWiki request for title '%s': %w", pageTitle, err)
}
if statusCode != http.StatusOK {
log.Printf("[MCP] Error: MediaWiki query for '%s' failed with status %d: %s", pageTitle, statusCode, string(bodyBytes))
return "", fmt.Errorf("MediaWiki query for '%s' failed with status %d: %s", pageTitle, statusCode, string(bodyBytes))
}
log.Printf("[MCP] Debug: MediaWiki query successful (status %d), %d bytes received.", statusCode, len(bodyBytes))
// Parse the response
var result MediaWikiQueryResult
if err := json.Unmarshal(bodyBytes, &result); err != nil {
log.Printf("[MCP] Error: Failed to decode MediaWiki response for '%s': %v", pageTitle, err)
return "", fmt.Errorf("failed to decode MediaWiki response for '%s': %w", pageTitle, err)
}
// Extract the text - MediaWiki API returns pages keyed by page ID
for pageID, page := range result.Query.Pages {
log.Printf("[MCP] Debug: Processing MediaWiki page ID: %s, Title: %s", pageID, page.Title)
if page.Extract != "" {
// Often includes a newline at the end, trim it
log.Printf("[MCP] Debug: Found extract for '%s'. Length: %d", pageTitle, len(page.Extract))
return strings.TrimSpace(page.Extract), nil
}
}
log.Printf("[MCP] Warn: No extract found in MediaWiki response for title '%s'", pageTitle)
return "", fmt.Errorf("no extract found in MediaWiki response for title '%s' (page might not exist or be empty)", pageTitle)
}
// extractPageTitleFromURL attempts to get the page title from a standard Wikipedia URL.
// Example: https://en.wikipedia.org/wiki/The_Beatles -> The_Beatles
func extractPageTitleFromURL(wikiURL string) (string, error) {
parsedURL, err := url.Parse(wikiURL)
if err != nil {
return "", err
}
if parsedURL.Host != "en.wikipedia.org" {
return "", fmt.Errorf("URL host is not en.wikipedia.org: %s", parsedURL.Host)
}
pathParts := strings.Split(strings.TrimPrefix(parsedURL.Path, "/"), "/")
if len(pathParts) < 2 || pathParts[0] != "wiki" {
return "", fmt.Errorf("URL path does not match /wiki/<title> format: %s", parsedURL.Path)
}
title := pathParts[1]
if title == "" {
return "", fmt.Errorf("extracted title is empty")
}
// URL Decode the title (e.g., %27 -> ')
decodedTitle, err := url.PathUnescape(title)
if err != nil {
return "", fmt.Errorf("failed to decode title '%s': %w", title, err)
}
return decodedTitle, nil
}

View File

@@ -0,0 +1,207 @@
package mcp
import (
"context"
"errors"
"os"
"path/filepath"
"strings"
"time"
mcp "github.com/metoro-io/mcp-golang"
"github.com/tetratelabs/wazero"
"github.com/tetratelabs/wazero/imports/wasi_snapshot_preview1"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
)
// Constants used by the MCP agent
const (
McpAgentName = "mcp"
initializationTimeout = 5 * time.Second
// McpServerPath defines the location of the MCP server executable or WASM module.
// McpServerPath = "./core/agents/mcp/mcp-server/mcp-server"
McpServerPath = "./core/agents/mcp/mcp-server/mcp-server.wasm"
McpToolNameGetBio = "get_artist_biography"
McpToolNameGetURL = "get_artist_url"
)
// mcpClient interface matching the methods used from mcp.Client.
type mcpClient interface {
Initialize(ctx context.Context) (*mcp.InitializeResponse, error)
CallTool(ctx context.Context, toolName string, args any) (*mcp.ToolResponse, error)
}
// mcpImplementation defines the common interface for both native and WASM MCP agents.
// This allows the main MCPAgent to delegate calls without knowing the underlying type.
type mcpImplementation interface {
Close() error // For cleaning up resources associated with this specific implementation.
// callMCPTool is the core method implemented differently by native/wasm
callMCPTool(ctx context.Context, toolName string, args any) (string, error)
}
// MCPAgent is the public-facing agent registered with Navidrome.
// It acts as a wrapper around the actual implementation (native or WASM).
type MCPAgent struct {
// No mutex needed here if impl is set once at construction
// and the implementation handles its own internal state synchronization.
impl mcpImplementation
// Shared Wazero resources (runtime, cache) are managed externally
// and closed separately, likely during application shutdown.
}
// mcpConstructor creates the appropriate MCP implementation (native or WASM)
// and wraps it in the MCPAgent.
func mcpConstructor(ds model.DataStore) agents.Interface {
if _, err := os.Stat(McpServerPath); os.IsNotExist(err) {
log.Warn("MCP server executable/WASM not found, disabling agent", "path", McpServerPath, "error", err)
return nil
}
var agentImpl mcpImplementation
var err error
if strings.HasSuffix(McpServerPath, ".wasm") {
log.Info("Configuring MCP agent for WASM execution", "path", McpServerPath)
ctx := context.Background()
// Setup Shared Wazero Resources
var cache wazero.CompilationCache
cacheDir := filepath.Join(conf.Server.DataFolder, "cache", "wazero")
if errMkdir := os.MkdirAll(cacheDir, 0755); errMkdir != nil {
log.Error(ctx, "Failed to create Wazero cache directory, WASM caching disabled", "path", cacheDir, "error", errMkdir)
} else {
cache, err = wazero.NewCompilationCacheWithDir(cacheDir)
if err != nil {
log.Error(ctx, "Failed to create Wazero compilation cache, WASM caching disabled", "path", cacheDir, "error", err)
cache = nil
}
}
runtimeConfig := wazero.NewRuntimeConfig()
if cache != nil {
runtimeConfig = runtimeConfig.WithCompilationCache(cache)
}
runtime := wazero.NewRuntimeWithConfig(ctx, runtimeConfig)
if err = registerHostFunctions(ctx, runtime); err != nil {
_ = runtime.Close(ctx)
if cache != nil {
_ = cache.Close(ctx)
}
return nil // Fatal error: Host functions required
}
if _, err = wasi_snapshot_preview1.Instantiate(ctx, runtime); err != nil {
log.Error(ctx, "Failed to instantiate WASI on shared Wazero runtime, MCP WASM agent disabled", "error", err)
_ = runtime.Close(ctx)
if cache != nil {
_ = cache.Close(ctx)
}
return nil // Fatal error: WASI required
}
// Compile the module
log.Debug(ctx, "Pre-compiling WASM module...", "path", McpServerPath)
wasmBytes, err := os.ReadFile(McpServerPath)
if err != nil {
log.Error(ctx, "Failed to read WASM module file, disabling agent", "path", McpServerPath, "error", err)
_ = runtime.Close(ctx)
if cache != nil {
_ = cache.Close(ctx)
}
return nil
}
compiledModule, err := runtime.CompileModule(ctx, wasmBytes)
if err != nil {
log.Error(ctx, "Failed to pre-compile WASM module, disabling agent", "path", McpServerPath, "error", err)
_ = runtime.Close(ctx)
if cache != nil {
_ = cache.Close(ctx)
}
return nil
}
agentImpl = newMCPWasm(runtime, cache, compiledModule)
log.Info(ctx, "Shared Wazero runtime, WASI, cache, host functions initialized, and module pre-compiled for MCP agent")
} else {
log.Info("Configuring MCP agent for native execution", "path", McpServerPath)
agentImpl = newMCPNative()
}
log.Info("MCP Agent implementation created successfully")
return &MCPAgent{impl: agentImpl}
}
// NewAgentForTesting is a constructor specifically for tests.
// It creates the appropriate implementation based on McpServerPath
// and injects a mock mcpClient into its ClientOverride field.
func NewAgentForTesting(mockClient mcpClient) agents.Interface {
// We need to replicate the logic from mcpConstructor to determine
// the implementation type, but without actually starting processes.
var agentImpl mcpImplementation
if strings.HasSuffix(McpServerPath, ".wasm") {
// For WASM testing, we might not need the full runtime setup,
// just the struct. Pass nil for shared resources for now.
// We rely on the mockClient being used before any real WASM interaction.
wasmImpl := newMCPWasm(nil, nil, nil)
wasmImpl.ClientOverride = mockClient
agentImpl = wasmImpl
} else {
nativeImpl := newMCPNative()
nativeImpl.ClientOverride = mockClient
agentImpl = nativeImpl
}
return &MCPAgent{impl: agentImpl}
}
func (a *MCPAgent) AgentName() string {
return McpAgentName
}
func (a *MCPAgent) GetArtistBiography(ctx context.Context, id, name, mbid string) (string, error) {
if a.impl == nil {
return "", errors.New("MCP agent implementation is nil")
}
// Construct args and call the implementation's specific tool caller
args := ArtistArgs{ID: id, Name: name, Mbid: mbid}
return a.impl.callMCPTool(ctx, McpToolNameGetBio, args)
}
func (a *MCPAgent) GetArtistURL(ctx context.Context, id, name, mbid string) (string, error) {
if a.impl == nil {
return "", errors.New("MCP agent implementation is nil")
}
// Construct args and call the implementation's specific tool caller
args := ArtistArgs{ID: id, Name: name, Mbid: mbid}
return a.impl.callMCPTool(ctx, McpToolNameGetURL, args)
}
// Note: A Close method on MCPAgent itself isn't part of agents.Interface.
// Cleanup of the specific implementation happens via impl.Close().
// Cleanup of shared Wazero resources needs separate handling (e.g., on app shutdown).
// ArtistArgs defines the structure for MCP tool arguments requiring artist info.
type ArtistArgs struct {
ID string `json:"id"`
Name string `json:"name"`
Mbid string `json:"mbid,omitempty"`
}
var _ agents.ArtistBiographyRetriever = (*MCPAgent)(nil)
var _ agents.ArtistURLRetriever = (*MCPAgent)(nil)
func init() {
agents.Register(McpAgentName, mcpConstructor)
}

View File

@@ -0,0 +1,237 @@
package mcp_test
import (
"context"
"errors"
"fmt"
"io"
mcp_client "github.com/metoro-io/mcp-golang" // Renamed alias for clarity
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/core/agents/mcp"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// Define the mcpClient interface locally for mocking, matching the one
// used internally by MCPNative/MCPWasm.
type mcpClient interface {
Initialize(ctx context.Context) (*mcp_client.InitializeResponse, error)
CallTool(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error)
}
// mockMCPClient is a mock implementation of mcpClient for testing.
type mockMCPClient struct {
InitializeFunc func(ctx context.Context) (*mcp_client.InitializeResponse, error)
CallToolFunc func(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error)
callToolArgs []any // Store args for verification
callToolName string // Store tool name for verification
}
func (m *mockMCPClient) Initialize(ctx context.Context) (*mcp_client.InitializeResponse, error) {
if m.InitializeFunc != nil {
return m.InitializeFunc(ctx)
}
return &mcp_client.InitializeResponse{}, nil // Default success
}
func (m *mockMCPClient) CallTool(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error) {
m.callToolName = toolName
m.callToolArgs = append(m.callToolArgs, args)
if m.CallToolFunc != nil {
return m.CallToolFunc(ctx, toolName, args)
}
return &mcp_client.ToolResponse{}, nil
}
// Ensure mock implements the local interface (compile-time check)
var _ mcpClient = (*mockMCPClient)(nil)
var _ = Describe("MCPAgent", func() {
var (
ctx context.Context
// We test the public MCPAgent wrapper, which uses the implementations internally.
// The actual agent instance might be native or wasm depending on McpServerPath
agent agents.Interface // Use the public agents.Interface
mockClient *mockMCPClient
)
BeforeEach(func() {
ctx = context.Background()
mockClient = &mockMCPClient{
callToolArgs: make([]any, 0), // Reset args on each test
}
// Instantiate the real agent using a testing constructor
// This constructor needs to be added to the mcp package.
agent = mcp.NewAgentForTesting(mockClient)
Expect(agent).NotTo(BeNil(), "Agent should be created")
})
// Helper to get the concrete agent type for calling specific methods
getConcreteAgent := func() *mcp.MCPAgent {
concreteAgent, ok := agent.(*mcp.MCPAgent)
Expect(ok).To(BeTrue(), "Agent should be of type *mcp.MCPAgent")
return concreteAgent
}
Describe("GetArtistBiography", func() {
It("should call the correct tool and return the biography", func() {
expectedBio := "This is the artist bio."
mockClient.CallToolFunc = func(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error) {
Expect(toolName).To(Equal(mcp.McpToolNameGetBio))
Expect(args).To(BeAssignableToTypeOf(mcp.ArtistArgs{}))
typedArgs := args.(mcp.ArtistArgs)
Expect(typedArgs.ID).To(Equal("id1"))
Expect(typedArgs.Name).To(Equal("Artist Name"))
Expect(typedArgs.Mbid).To(Equal("mbid1"))
return mcp_client.NewToolResponse(mcp_client.NewTextContent(expectedBio)), nil
}
bio, err := getConcreteAgent().GetArtistBiography(ctx, "id1", "Artist Name", "mbid1")
Expect(err).NotTo(HaveOccurred())
Expect(bio).To(Equal(expectedBio))
})
It("should return error if CallTool fails", func() {
expectedErr := errors.New("mcp tool error")
mockClient.CallToolFunc = func(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error) {
return nil, expectedErr
}
bio, err := getConcreteAgent().GetArtistBiography(ctx, "id1", "Artist Name", "mbid1")
Expect(err).To(HaveOccurred())
// The error originates from the implementation now, check for specific part
Expect(err.Error()).To(SatisfyAny(
ContainSubstring("native MCP agent not ready"), // Error from native
ContainSubstring("WASM MCP agent not ready"), // Error from WASM
ContainSubstring("failed to call native MCP tool"),
ContainSubstring("failed to call WASM MCP tool"),
))
Expect(errors.Is(err, expectedErr)).To(BeTrue())
Expect(bio).To(BeEmpty())
})
It("should return ErrNotFound if CallTool response is empty", func() {
mockClient.CallToolFunc = func(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error) {
// Return a response created with no content parts
return mcp_client.NewToolResponse(), nil
}
bio, err := getConcreteAgent().GetArtistBiography(ctx, "id1", "Artist Name", "mbid1")
Expect(err).To(MatchError(agents.ErrNotFound))
Expect(bio).To(BeEmpty())
})
It("should return ErrNotFound if CallTool response has nil TextContent (simulated by empty string)", func() {
mockClient.CallToolFunc = func(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error) {
// Simulate nil/empty text content by creating response with empty string text
return mcp_client.NewToolResponse(mcp_client.NewTextContent("")), nil
}
bio, err := getConcreteAgent().GetArtistBiography(ctx, "id1", "Artist Name", "mbid1")
Expect(err).To(MatchError(agents.ErrNotFound))
Expect(bio).To(BeEmpty())
})
It("should return comm error if CallTool returns pipe error", func() {
mockClient.CallToolFunc = func(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error) {
return nil, io.ErrClosedPipe
}
bio, err := getConcreteAgent().GetArtistBiography(ctx, "id1", "Artist Name", "mbid1")
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(SatisfyAny(
ContainSubstring("native MCP agent process communication error"),
ContainSubstring("WASM MCP agent module communication error"),
))
Expect(errors.Is(err, io.ErrClosedPipe)).To(BeTrue())
Expect(bio).To(BeEmpty())
})
It("should return ErrNotFound if MCP tool returns an error string", func() {
mcpErrorString := "handler returned an error: something went wrong on the server"
mockClient.CallToolFunc = func(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error) {
return mcp_client.NewToolResponse(mcp_client.NewTextContent(mcpErrorString)), nil
}
bio, err := getConcreteAgent().GetArtistBiography(ctx, "id1", "Artist Name", "mbid1")
Expect(err).To(MatchError(agents.ErrNotFound))
Expect(bio).To(BeEmpty())
})
})
Describe("GetArtistURL", func() {
It("should call the correct tool and return the URL", func() {
expectedURL := "http://example.com/artist"
mockClient.CallToolFunc = func(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error) {
Expect(toolName).To(Equal(mcp.McpToolNameGetURL))
Expect(args).To(BeAssignableToTypeOf(mcp.ArtistArgs{}))
typedArgs := args.(mcp.ArtistArgs)
Expect(typedArgs.ID).To(Equal("id2"))
Expect(typedArgs.Name).To(Equal("Another Artist"))
Expect(typedArgs.Mbid).To(Equal("mbid2"))
return mcp_client.NewToolResponse(mcp_client.NewTextContent(expectedURL)), nil
}
url, err := getConcreteAgent().GetArtistURL(ctx, "id2", "Another Artist", "mbid2")
Expect(err).NotTo(HaveOccurred())
Expect(url).To(Equal(expectedURL))
})
It("should return error if CallTool fails", func() {
expectedErr := errors.New("mcp tool error url")
mockClient.CallToolFunc = func(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error) {
return nil, expectedErr
}
url, err := getConcreteAgent().GetArtistURL(ctx, "id2", "Another Artist", "mbid2")
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(SatisfyAny(
ContainSubstring("native MCP agent not ready"), // Error from native
ContainSubstring("WASM MCP agent not ready"), // Error from WASM
ContainSubstring("failed to call native MCP tool"),
ContainSubstring("failed to call WASM MCP tool"),
))
Expect(errors.Is(err, expectedErr)).To(BeTrue())
Expect(url).To(BeEmpty())
})
It("should return ErrNotFound if CallTool response is empty", func() {
mockClient.CallToolFunc = func(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error) {
// Return a response created with no content parts
return mcp_client.NewToolResponse(), nil
}
url, err := getConcreteAgent().GetArtistURL(ctx, "id2", "Another Artist", "mbid2")
Expect(err).To(MatchError(agents.ErrNotFound))
Expect(url).To(BeEmpty())
})
It("should return comm error if CallTool returns pipe error", func() {
mockClient.CallToolFunc = func(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error) {
return nil, fmt.Errorf("write: %w", io.ErrClosedPipe)
}
url, err := getConcreteAgent().GetArtistURL(ctx, "id2", "Another Artist", "mbid2")
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(SatisfyAny(
ContainSubstring("native MCP agent process communication error"),
ContainSubstring("WASM MCP agent module communication error"),
))
Expect(errors.Is(err, io.ErrClosedPipe)).To(BeTrue())
Expect(url).To(BeEmpty())
})
It("should return ErrNotFound if MCP tool returns an error string", func() {
mcpErrorString := "handler returned an error: could not find url"
mockClient.CallToolFunc = func(ctx context.Context, toolName string, args any) (*mcp_client.ToolResponse, error) {
return mcp_client.NewToolResponse(mcp_client.NewTextContent(mcpErrorString)), nil
}
url, err := getConcreteAgent().GetArtistURL(ctx, "id2", "Another Artist", "mbid2")
Expect(err).To(MatchError(agents.ErrNotFound))
Expect(url).To(BeEmpty())
})
})
})

View File

@@ -0,0 +1,189 @@
package mcp
import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"time"
"github.com/navidrome/navidrome/log"
"github.com/tetratelabs/wazero"
"github.com/tetratelabs/wazero/api"
)
// httpClient is a shared HTTP client for host function reuse.
var httpClient = &http.Client{
// Consider adding a default timeout
Timeout: 30 * time.Second,
}
// registerHostFunctions defines and registers the host functions (e.g., http_fetch)
// into the provided Wazero runtime.
func registerHostFunctions(ctx context.Context, runtime wazero.Runtime) error {
// Define and Instantiate Host Module "env"
_, err := runtime.NewHostModuleBuilder("env"). // "env" is the conventional module name
NewFunctionBuilder().
WithFunc(httpFetch). // Register our Go function
Export("http_fetch"). // Export it with the name WASM will use
Instantiate(ctx)
if err != nil {
log.Error(ctx, "Failed to instantiate 'env' host module with httpFetch", "error", err)
return fmt.Errorf("instantiate host module 'env': %w", err)
}
log.Info(ctx, "Instantiated 'env' host module with http_fetch function")
return nil
}
// httpFetch is the host function exposed to WASM.
// ... (full implementation as provided previously) ...
// Returns:
// - 0 on success (request completed, results written).
// - 1 on host-side failure (e.g., memory access error, invalid input).
func httpFetch(
ctx context.Context, mod api.Module, // Standard Wazero host function params
// Request details
urlPtr, urlLen uint32,
methodPtr, methodLen uint32,
bodyPtr, bodyLen uint32,
timeoutMillis uint32,
// Result pointers
resultStatusPtr uint32,
resultBodyPtr uint32, resultBodyCapacity uint32, resultBodyLenPtr uint32,
resultErrorPtr uint32, resultErrorCapacity uint32, resultErrorLenPtr uint32,
) uint32 { // Using uint32 for status code convention (0=success, 1=failure)
mem := mod.Memory()
// --- Read Inputs ---
urlBytes, ok := mem.Read(urlPtr, urlLen)
if !ok {
log.Error(ctx, "httpFetch host error: failed to read URL from WASM memory")
// Cannot write error back as we don't have the pointers validated yet
return 1
}
url := string(urlBytes)
methodBytes, ok := mem.Read(methodPtr, methodLen)
if !ok {
log.Error(ctx, "httpFetch host error: failed to read method from WASM memory", "url", url)
return 1 // Bail out
}
method := string(methodBytes)
if method == "" {
method = "GET" // Default to GET
}
var reqBody io.Reader
if bodyLen > 0 {
bodyBytes, ok := mem.Read(bodyPtr, bodyLen)
if !ok {
log.Error(ctx, "httpFetch host error: failed to read body from WASM memory", "url", url, "method", method)
return 1 // Bail out
}
reqBody = bytes.NewReader(bodyBytes)
}
timeout := time.Duration(timeoutMillis) * time.Millisecond
if timeout <= 0 {
timeout = 30 * time.Second // Default timeout matching httpClient
}
// --- Prepare and Execute Request ---
log.Debug(ctx, "httpFetch executing request", "method", method, "url", url, "timeout", timeout)
// Use a specific context for the request, derived from the host function's context
// but with the specific timeout for this call.
reqCtx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
req, err := http.NewRequestWithContext(reqCtx, method, url, reqBody)
if err != nil {
errMsg := fmt.Sprintf("failed to create request: %v", err)
log.Error(ctx, "httpFetch host error", "url", url, "method", method, "error", errMsg)
writeStringResult(mem, resultErrorPtr, resultErrorCapacity, resultErrorLenPtr, errMsg)
mem.WriteUint32Le(resultStatusPtr, 0) // Write 0 status on creation error
mem.WriteUint32Le(resultBodyLenPtr, 0) // No body
return 0 // Indicate results (including error) were written
}
// TODO: Consider adding a User-Agent?
// req.Header.Set("User-Agent", "Navidrome/MCP-Agent-Host")
resp, err := httpClient.Do(req)
if err != nil {
// Handle client-side errors (network, DNS, timeout)
errMsg := fmt.Sprintf("failed to execute request: %v", err)
log.Error(ctx, "httpFetch host error", "url", url, "method", method, "error", errMsg)
writeStringResult(mem, resultErrorPtr, resultErrorCapacity, resultErrorLenPtr, errMsg)
mem.WriteUint32Le(resultStatusPtr, 0) // Write 0 status on transport error
mem.WriteUint32Le(resultBodyLenPtr, 0)
return 0 // Indicate results written
}
defer resp.Body.Close()
// --- Process Response ---
statusCode := uint32(resp.StatusCode)
responseBodyBytes, readErr := io.ReadAll(resp.Body)
if readErr != nil {
errMsg := fmt.Sprintf("failed to read response body: %v", readErr)
log.Error(ctx, "httpFetch host error", "url", url, "method", method, "status", statusCode, "error", errMsg)
writeStringResult(mem, resultErrorPtr, resultErrorCapacity, resultErrorLenPtr, errMsg)
mem.WriteUint32Le(resultStatusPtr, statusCode) // Write actual status code
mem.WriteUint32Le(resultBodyLenPtr, 0)
return 0 // Indicate results written
}
// --- Write Results Back to WASM Memory ---
log.Debug(ctx, "httpFetch writing results", "url", url, "method", method, "status", statusCode, "bodyLen", len(responseBodyBytes))
// Write status code
if !mem.WriteUint32Le(resultStatusPtr, statusCode) {
log.Error(ctx, "httpFetch host error: failed to write status code to WASM memory")
return 1 // Host error
}
// Write response body (checking capacity)
if !writeBytesResult(mem, resultBodyPtr, resultBodyCapacity, resultBodyLenPtr, responseBodyBytes) {
// If body write fails (likely due to capacity), write an error message instead.
errMsg := fmt.Sprintf("response body size (%d) exceeds buffer capacity (%d)", len(responseBodyBytes), resultBodyCapacity)
log.Error(ctx, "httpFetch host error", "url", url, "method", method, "status", statusCode, "error", errMsg)
writeStringResult(mem, resultErrorPtr, resultErrorCapacity, resultErrorLenPtr, errMsg)
mem.WriteUint32Le(resultBodyLenPtr, 0) // Ensure body length is 0 if we wrote an error
} else {
// Write empty error string if body write was successful
mem.WriteUint32Le(resultErrorLenPtr, 0)
}
return 0 // Success
}
// Helper to write string results, respecting capacity. Returns true on success.
func writeStringResult(mem api.Memory, ptr, capacity, lenPtr uint32, result string) bool {
bytes := []byte(result)
return writeBytesResult(mem, ptr, capacity, lenPtr, bytes)
}
// Helper to write byte results, respecting capacity. Returns true on success.
func writeBytesResult(mem api.Memory, ptr, capacity, lenPtr uint32, result []byte) bool {
resultLen := uint32(len(result))
writeLen := resultLen
if writeLen > capacity {
log.Warn(context.Background(), "WASM host write truncated", "requested", resultLen, "capacity", capacity)
writeLen = capacity // Truncate if too large for buffer
}
if writeLen > 0 {
if !mem.Write(ptr, result[:writeLen]) {
log.Error(context.Background(), "WASM host memory write failed", "ptr", ptr, "len", writeLen)
return false // Memory write failed
}
}
// Write the *original* length of the data (even if truncated) so the WASM side knows.
if !mem.WriteUint32Le(lenPtr, resultLen) {
log.Error(context.Background(), "WASM host memory length write failed", "lenPtr", lenPtr, "len", resultLen)
return false // Memory write failed
}
return true
}

View File

@@ -0,0 +1,252 @@
package mcp
import (
"bufio"
"context"
"errors"
"fmt"
"io"
"os"
"os/exec"
"strings"
"sync"
mcp "github.com/metoro-io/mcp-golang"
"github.com/metoro-io/mcp-golang/transport/stdio"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/log"
)
// MCPNative implements the mcpImplementation interface for running the MCP server as a native process.
type MCPNative struct {
mu sync.Mutex
cmd *exec.Cmd // Stores the running command
stdin io.WriteCloser
client mcpClient
// ClientOverride allows injecting a mock client for testing this specific implementation.
ClientOverride mcpClient // TODO: Consider if this is the best way to test
}
// newMCPNative creates a new instance of the native MCP agent implementation.
func newMCPNative() *MCPNative {
return &MCPNative{}
}
// --- mcpImplementation interface methods ---
func (n *MCPNative) Close() error {
n.mu.Lock()
defer n.mu.Unlock()
n.cleanupResources_locked()
return nil // Currently, cleanup doesn't return errors
}
// --- Internal Helper Methods ---
// ensureClientInitialized starts the MCP server process and initializes the client if needed.
// MUST be called with the mutex HELD.
func (n *MCPNative) ensureClientInitialized_locked(ctx context.Context) error {
// Use override if provided (for testing)
if n.ClientOverride != nil {
if n.client == nil {
n.client = n.ClientOverride
log.Debug(ctx, "Using provided MCP client override for native testing")
}
return nil
}
// Check if already initialized
if n.client != nil {
return nil
}
log.Info(ctx, "Initializing Native MCP client and starting/restarting server process...", "serverPath", McpServerPath)
// Clean up any old resources *before* starting new ones
n.cleanupResources_locked()
hostStdinWriter, hostStdoutReader, nativeCmd, startErr := n.startProcess_locked(ctx)
if startErr != nil {
log.Error(ctx, "Failed to start Native MCP server process", "error", startErr)
// Ensure pipes are closed if start failed (startProcess might handle this, but be sure)
if hostStdinWriter != nil {
_ = hostStdinWriter.Close()
}
if hostStdoutReader != nil {
_ = hostStdoutReader.Close()
}
return fmt.Errorf("failed to start native MCP server: %w", startErr)
}
// --- Initialize MCP client ---
transport := stdio.NewStdioServerTransportWithIO(hostStdoutReader, hostStdinWriter)
clientImpl := mcp.NewClient(transport)
initCtx, cancel := context.WithTimeout(context.Background(), initializationTimeout)
defer cancel()
if _, initErr := clientImpl.Initialize(initCtx); initErr != nil {
err := fmt.Errorf("failed to initialize native MCP client: %w", initErr)
log.Error(ctx, "Native MCP client initialization failed", "error", err)
// Cleanup the newly started process and close pipes
n.cmd = nativeCmd // Temporarily set cmd so cleanup can kill it
n.cleanupResources_locked()
if hostStdinWriter != nil {
_ = hostStdinWriter.Close()
}
if hostStdoutReader != nil {
_ = hostStdoutReader.Close()
}
return err
}
// --- Initialization successful, update agent state ---
n.cmd = nativeCmd
n.stdin = hostStdinWriter // This is the pipe the agent writes to
n.client = clientImpl
log.Info(ctx, "Native MCP client initialized successfully", "pid", n.cmd.Process.Pid)
return nil // Success
}
// callMCPTool handles ensuring initialization and calling the MCP tool.
func (n *MCPNative) callMCPTool(ctx context.Context, toolName string, args any) (string, error) {
// Ensure the client is initialized and the server is running (attempts restart if needed)
n.mu.Lock()
err := n.ensureClientInitialized_locked(ctx)
if err != nil {
n.mu.Unlock()
log.Error(ctx, "Native MCP agent initialization/restart failed", "tool", toolName, "error", err)
return "", fmt.Errorf("native MCP agent not ready: %w", err)
}
// Keep a reference to the client while locked
currentClient := n.client
// Unlock mutex *before* making the potentially blocking MCP call
n.mu.Unlock()
// Call the tool using the client reference
log.Debug(ctx, "Calling Native MCP tool", "tool", toolName, "args", args)
response, callErr := currentClient.CallTool(ctx, toolName, args)
if callErr != nil {
// Handle potential pipe closures or other communication errors
log.Error(ctx, "Failed to call Native MCP tool", "tool", toolName, "error", callErr)
// Check if the error indicates a broken pipe, suggesting the server died
// The monitoring goroutine will handle cleanup, just return error here.
if errors.Is(callErr, io.ErrClosedPipe) || strings.Contains(callErr.Error(), "broken pipe") || strings.Contains(callErr.Error(), "EOF") {
log.Warn(ctx, "Native MCP tool call failed, possibly due to server process exit.", "tool", toolName)
// No need to explicitly call cleanup, monitoring goroutine handles it.
return "", fmt.Errorf("native MCP agent process communication error: %w", callErr)
}
return "", fmt.Errorf("failed to call native MCP tool '%s': %w", toolName, callErr)
}
// Process the response (same logic as before)
if response == nil || len(response.Content) == 0 || response.Content[0].TextContent == nil || response.Content[0].TextContent.Text == "" {
log.Warn(ctx, "Native MCP tool returned empty/invalid response", "tool", toolName)
// Treat as not found for agent interface consistency
return "", agents.ErrNotFound // Import agents package if needed, or define locally
}
resultText := response.Content[0].TextContent.Text
if strings.HasPrefix(resultText, "handler returned an error:") {
log.Warn(ctx, "Native MCP tool returned an error message", "tool", toolName, "mcpError", resultText)
return "", agents.ErrNotFound // Treat MCP tool errors as "not found"
}
log.Debug(ctx, "Received response from Native MCP agent", "tool", toolName, "length", len(resultText))
return resultText, nil
}
// cleanupResources closes existing resources (stdin, server process).
// MUST be called with the mutex HELD.
func (n *MCPNative) cleanupResources_locked() {
log.Debug(context.Background(), "Cleaning up Native MCP instance resources...")
if n.stdin != nil {
_ = n.stdin.Close()
n.stdin = nil
}
if n.cmd != nil && n.cmd.Process != nil {
pid := n.cmd.Process.Pid
log.Debug(context.Background(), "Killing native MCP process", "pid", pid)
// Kill the process. Ignore error if it's already done.
if err := n.cmd.Process.Kill(); err != nil && !errors.Is(err, os.ErrProcessDone) {
log.Error(context.Background(), "Failed to kill native process", "pid", pid, "error", err)
}
// Wait for the process to release resources. Ignore error.
_ = n.cmd.Wait()
n.cmd = nil
}
// Mark client as invalid
n.client = nil
}
// startProcess starts the MCP server as a native executable and sets up monitoring.
// MUST be called with the mutex HELD.
func (n *MCPNative) startProcess_locked(ctx context.Context) (stdin io.WriteCloser, stdout io.ReadCloser, cmd *exec.Cmd, err error) {
log.Debug(ctx, "Starting native MCP server process", "path", McpServerPath)
// Use Background context for the command itself, as it should outlive the request context (ctx)
cmd = exec.CommandContext(context.Background(), McpServerPath)
stdinPipe, err := cmd.StdinPipe()
if err != nil {
return nil, nil, nil, fmt.Errorf("native stdin pipe: %w", err)
}
stdoutPipe, err := cmd.StdoutPipe()
if err != nil {
_ = stdinPipe.Close()
return nil, nil, nil, fmt.Errorf("native stdout pipe: %w", err)
}
// Get stderr pipe to stream logs
stderrPipe, err := cmd.StderrPipe()
if err != nil {
_ = stdinPipe.Close()
_ = stdoutPipe.Close()
return nil, nil, nil, fmt.Errorf("native stderr pipe: %w", err)
}
if err = cmd.Start(); err != nil {
_ = stdinPipe.Close()
_ = stdoutPipe.Close()
// stderrPipe gets closed implicitly if cmd.Start() fails
return nil, nil, nil, fmt.Errorf("native start: %w", err)
}
currentPid := cmd.Process.Pid
currentCmd := cmd // Capture the current cmd pointer for the goroutine
log.Info(ctx, "Native MCP server process started", "pid", currentPid)
// Start monitoring goroutine for process exit
go func() {
// Start separate goroutine to stream stderr
go func() {
scanner := bufio.NewScanner(stderrPipe)
for scanner.Scan() {
log.Info("[MCP-SERVER] " + scanner.Text())
}
if err := scanner.Err(); err != nil {
log.Error("Error reading MCP server stderr", "pid", currentPid, "error", err)
}
log.Debug("MCP server stderr pipe closed", "pid", currentPid)
}()
waitErr := currentCmd.Wait() // Wait for the specific process this goroutine monitors
n.mu.Lock()
// Stderr is now streamed, so we don't capture it here anymore.
log.Warn("Native MCP server process exited", "pid", currentPid, "error", waitErr)
// Critical: Check if the agent's current command is STILL the one we were monitoring.
// If it's different, it means cleanup/restart already happened, so we shouldn't cleanup again.
if n.cmd == currentCmd {
n.cleanupResources_locked() // Use the locked version as we hold the lock
log.Info("MCP Native agent state cleaned up after process exit", "pid", currentPid)
} else {
log.Debug("Native MCP process exited, but state already updated/cmd mismatch", "exitedPid", currentPid)
}
n.mu.Unlock()
}()
// Return the pipes connected to the process and the Cmd object
return stdinPipe, stdoutPipe, cmd, nil
}

View File

@@ -0,0 +1,305 @@
package mcp
import (
"context"
"errors"
"fmt"
"io"
"os"
"strings"
"sync"
"time"
mcp "github.com/metoro-io/mcp-golang"
"github.com/metoro-io/mcp-golang/transport/stdio"
"github.com/navidrome/navidrome/core/agents" // Needed for ErrNotFound
"github.com/navidrome/navidrome/log"
"github.com/tetratelabs/wazero" // Needed for types
"github.com/tetratelabs/wazero/api" // Needed for types
)
// MCPWasm implements the mcpImplementation interface for running the MCP server as a WASM module.
type MCPWasm struct {
mu sync.Mutex
wasmModule api.Module // Stores the instantiated module
wasmCompiled api.Closer // Stores the compiled module reference for this instance
stdin io.WriteCloser
client mcpClient
// Shared resources (passed in, not owned by this struct)
wasmRuntime api.Closer // Shared Wazero Runtime
wasmCache wazero.CompilationCache // Shared Compilation Cache (can be nil)
preCompiledModule wazero.CompiledModule // Pre-compiled module from constructor
// ClientOverride allows injecting a mock client for testing this specific implementation.
ClientOverride mcpClient // TODO: Consider if this is the best way to test
}
// newMCPWasm creates a new instance of the WASM MCP agent implementation.
// It stores the shared runtime, cache, and the pre-compiled module.
func newMCPWasm(runtime api.Closer, cache wazero.CompilationCache, compiledModule wazero.CompiledModule) *MCPWasm {
return &MCPWasm{
wasmRuntime: runtime,
wasmCache: cache,
preCompiledModule: compiledModule,
}
}
// --- mcpImplementation interface methods ---
// Close cleans up instance-specific WASM resources.
// It does NOT close the shared runtime or cache.
func (w *MCPWasm) Close() error {
w.mu.Lock()
defer w.mu.Unlock()
w.cleanupResources_locked()
return nil
}
// --- Internal Helper Methods ---
// ensureClientInitialized starts the MCP WASM module and initializes the client if needed.
// MUST be called with the mutex HELD.
func (w *MCPWasm) ensureClientInitialized_locked(ctx context.Context) error {
// Use override if provided (for testing)
if w.ClientOverride != nil {
if w.client == nil {
w.client = w.ClientOverride
log.Debug(ctx, "Using provided MCP client override for WASM testing")
}
return nil
}
// Check if already initialized
if w.client != nil {
return nil
}
log.Info(ctx, "Initializing WASM MCP client and starting/restarting server module...", "serverPath", McpServerPath)
w.cleanupResources_locked()
// Check if shared runtime exists
if w.wasmRuntime == nil {
return errors.New("shared Wazero runtime not initialized for MCPWasm")
}
hostStdinWriter, hostStdoutReader, mod, err := w.startModule_locked(ctx)
if err != nil {
log.Error(ctx, "Failed to start WASM MCP server module", "error", err)
// Ensure pipes are closed if start failed
if hostStdinWriter != nil {
_ = hostStdinWriter.Close()
}
if hostStdoutReader != nil {
_ = hostStdoutReader.Close()
}
// startModule_locked handles cleanup of mod/compiled on error
return fmt.Errorf("failed to start WASM MCP server: %w", err)
}
transport := stdio.NewStdioServerTransportWithIO(hostStdoutReader, hostStdinWriter)
clientImpl := mcp.NewClient(transport)
initCtx, cancel := context.WithTimeout(context.Background(), initializationTimeout)
defer cancel()
if _, initErr := clientImpl.Initialize(initCtx); initErr != nil {
err := fmt.Errorf("failed to initialize WASM MCP client: %w", initErr)
log.Error(ctx, "WASM MCP client initialization failed", "error", err)
// Cleanup the newly started module and close pipes
w.wasmModule = mod // Temporarily set so cleanup can close it
w.wasmCompiled = nil // We don't store the compiled instance ref anymore, just the module instance
w.cleanupResources_locked()
if hostStdinWriter != nil {
_ = hostStdinWriter.Close()
}
if hostStdoutReader != nil {
_ = hostStdoutReader.Close()
}
return err
}
w.wasmModule = mod
w.wasmCompiled = nil // We don't store the compiled instance ref anymore, just the module instance
w.stdin = hostStdinWriter
w.client = clientImpl
log.Info(ctx, "WASM MCP client initialized successfully")
return nil
}
// callMCPTool handles ensuring initialization and calling the MCP tool.
func (w *MCPWasm) callMCPTool(ctx context.Context, toolName string, args any) (string, error) {
w.mu.Lock()
err := w.ensureClientInitialized_locked(ctx)
if err != nil {
w.mu.Unlock()
log.Error(ctx, "WASM MCP agent initialization/restart failed", "tool", toolName, "error", err)
return "", fmt.Errorf("WASM MCP agent not ready: %w", err)
}
// Keep a reference to the client while locked
currentClient := w.client
// Unlock mutex *before* making the potentially blocking MCP call
w.mu.Unlock()
// Call the tool using the client reference
log.Debug(ctx, "Calling WASM MCP tool", "tool", toolName, "args", args)
response, callErr := currentClient.CallTool(ctx, toolName, args)
if callErr != nil {
// Handle potential pipe closures or other communication errors
log.Error(ctx, "Failed to call WASM MCP tool", "tool", toolName, "error", callErr)
// Check if the error indicates a broken pipe, suggesting the server died
// The monitoring goroutine will handle cleanup, just return error here.
if errors.Is(callErr, io.ErrClosedPipe) || strings.Contains(callErr.Error(), "broken pipe") || strings.Contains(callErr.Error(), "EOF") {
log.Warn(ctx, "WASM MCP tool call failed, possibly due to server module exit.", "tool", toolName)
// No need to explicitly call cleanup, monitoring goroutine handles it.
return "", fmt.Errorf("WASM MCP agent module communication error: %w", callErr)
}
return "", fmt.Errorf("failed to call WASM MCP tool '%s': %w", toolName, callErr)
}
// Process the response (same logic as native)
if response == nil || len(response.Content) == 0 || response.Content[0].TextContent == nil || response.Content[0].TextContent.Text == "" {
log.Warn(ctx, "WASM MCP tool returned empty/invalid response", "tool", toolName)
return "", agents.ErrNotFound
}
resultText := response.Content[0].TextContent.Text
if strings.HasPrefix(resultText, "handler returned an error:") {
log.Warn(ctx, "WASM MCP tool returned an error message", "tool", toolName, "mcpError", resultText)
return "", agents.ErrNotFound // Treat MCP tool errors as "not found"
}
log.Debug(ctx, "Received response from WASM MCP agent", "tool", toolName, "length", len(resultText))
return resultText, nil
}
// cleanupResources closes instance-specific WASM resources (stdin, module, compiled ref).
// It specifically avoids closing the shared runtime or cache.
// MUST be called with the mutex HELD.
func (w *MCPWasm) cleanupResources_locked() {
log.Debug(context.Background(), "Cleaning up WASM MCP instance resources...")
if w.stdin != nil {
_ = w.stdin.Close()
w.stdin = nil
}
// Close the module instance
if w.wasmModule != nil {
log.Debug(context.Background(), "Closing WASM module instance")
ctxClose, cancel := context.WithTimeout(context.Background(), 2*time.Second)
if err := w.wasmModule.Close(ctxClose); err != nil && !errors.Is(err, context.DeadlineExceeded) {
log.Error(context.Background(), "Failed to close WASM module instance", "error", err)
}
cancel()
w.wasmModule = nil
}
// DO NOT close w.wasmCompiled (instance ref)
// DO NOT close w.preCompiledModule (shared pre-compiled code)
// DO NOT CLOSE w.wasmRuntime or w.wasmCache here!
w.client = nil
}
// startModule loads and starts the MCP server as a WASM module.
// It now uses the pre-compiled module.
// MUST be called with the mutex HELD.
func (w *MCPWasm) startModule_locked(ctx context.Context) (hostStdinWriter io.WriteCloser, hostStdoutReader io.ReadCloser, mod api.Module, err error) {
// Check for pre-compiled module
if w.preCompiledModule == nil {
return nil, nil, nil, errors.New("pre-compiled WASM module is nil")
}
// Create pipes for stdio redirection
wasmStdinReader, hostStdinWriter, err := os.Pipe()
if err != nil {
return nil, nil, nil, fmt.Errorf("wasm stdin pipe: %w", err)
}
// Defer close pipes on error exit
shouldClosePipesOnError := true
defer func() {
if shouldClosePipesOnError {
if wasmStdinReader != nil {
_ = wasmStdinReader.Close()
}
if hostStdinWriter != nil {
_ = hostStdinWriter.Close()
}
// hostStdoutReader and wasmStdoutWriter handled below
}
}()
hostStdoutReader, wasmStdoutWriter, err := os.Pipe()
if err != nil {
return nil, nil, nil, fmt.Errorf("wasm stdout pipe: %w", err)
}
// Defer close pipes on error exit
defer func() {
if shouldClosePipesOnError {
if hostStdoutReader != nil {
_ = hostStdoutReader.Close()
}
if wasmStdoutWriter != nil {
_ = wasmStdoutWriter.Close()
}
}
}()
// Use the SHARDED runtime from the agent struct
runtime, ok := w.wasmRuntime.(wazero.Runtime)
if !ok || runtime == nil {
return nil, nil, nil, errors.New("wasmRuntime is not initialized or not a wazero.Runtime")
}
// Prepare module configuration (host funcs/WASI already instantiated on runtime)
config := wazero.NewModuleConfig().
WithStdin(wasmStdinReader).
WithStdout(wasmStdoutWriter).
WithStderr(os.Stderr).
WithArgs(McpServerPath).
WithFS(os.DirFS("/")) // Keep FS access for now
log.Info(ctx, "Instantiating pre-compiled WASM module (will run _start)...")
var moduleInstance api.Module
instanceErrChan := make(chan error, 1)
go func() {
var instantiateErr error
// Use context.Background() for the module's main execution context
moduleInstance, instantiateErr = runtime.InstantiateModule(context.Background(), w.preCompiledModule, config)
instanceErrChan <- instantiateErr
}()
// Wait briefly for immediate instantiation errors
select {
case instantiateErr := <-instanceErrChan:
if instantiateErr != nil {
log.Error(ctx, "Failed to instantiate pre-compiled WASM module", "error", instantiateErr)
// Pipes closed by defer
return nil, nil, nil, fmt.Errorf("instantiate wasm module: %w", instantiateErr)
}
log.Warn(ctx, "WASM module instantiation returned (exited?) immediately without error.")
case <-time.After(1 * time.Second): // Shorter wait now, instantiation should be faster
log.Debug(ctx, "WASM module instantiation likely blocking (server running), proceeding...")
}
// Start a monitoring goroutine for WASM module exit/error
go func(instanceToMonitor api.Module, errChan chan error) {
// This blocks until the instance created by InstantiateModule exits or errors.
instantiateErr := <-errChan
w.mu.Lock() // Lock the specific MCPWasm instance
log.Warn("WASM module exited/errored", "error", instantiateErr)
// Critical: Check if the agent's current module is STILL the one we were monitoring.
if w.wasmModule == instanceToMonitor {
w.cleanupResources_locked() // Use the locked version
log.Info("MCP WASM agent state cleaned up after module exit/error")
} else {
log.Debug("WASM module exited, but state already updated/module mismatch. No cleanup needed by this goroutine.")
// No need to close anything here, the pre-compiled module is shared
}
w.mu.Unlock()
}(moduleInstance, instanceErrChan)
// Success: prevent deferred cleanup of pipes
shouldClosePipesOnError = false
return hostStdinWriter, hostStdoutReader, moduleInstance, nil
}

View File

@@ -0,0 +1,17 @@
package mcp_test
import (
"testing"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
func TestMCPAgent(t *testing.T) {
tests.Init(t, false)
log.SetLevel(log.LevelFatal)
RegisterFailHandler(Fail)
RunSpecs(t, "MCP Agent Test Suite")
}

View File

@@ -27,6 +27,9 @@ type spotifyAgent struct {
}
func spotifyConstructor(ds model.DataStore) agents.Interface {
if conf.Server.Spotify.ID == "" || conf.Server.Spotify.Secret == "" {
return nil
}
l := &spotifyAgent{
ds: ds,
id: conf.Server.Spotify.ID,
@@ -88,8 +91,6 @@ func (s *spotifyAgent) searchArtist(ctx context.Context, name string) (*Artist,
func init() {
conf.AddHook(func() {
if conf.Server.Spotify.ID != "" && conf.Server.Spotify.Secret != "" {
agents.Register(spotifyAgentName, spotifyConstructor)
}
agents.Register(spotifyAgentName, spotifyConstructor)
})
}

View File

@@ -53,11 +53,11 @@ func (a *archiver) zipAlbums(ctx context.Context, id string, format string, bitr
})
for _, album := range albums {
discs := slice.Group(album, func(mf model.MediaFile) int { return mf.DiscNumber })
isMultDisc := len(discs) > 1
isMultiDisc := len(discs) > 1
log.Debug(ctx, "Zipping album", "name", album[0].Album, "artist", album[0].AlbumArtist,
"format", format, "bitrate", bitrate, "isMultDisc", isMultDisc, "numTracks", len(album))
"format", format, "bitrate", bitrate, "isMultiDisc", isMultiDisc, "numTracks", len(album))
for _, mf := range album {
file := a.albumFilename(mf, format, isMultDisc)
file := a.albumFilename(mf, format, isMultiDisc)
_ = a.addFileToZip(ctx, z, mf, format, bitrate, file)
}
}
@@ -78,12 +78,12 @@ func createZipWriter(out io.Writer, format string, bitrate int) *zip.Writer {
return z
}
func (a *archiver) albumFilename(mf model.MediaFile, format string, isMultDisc bool) string {
func (a *archiver) albumFilename(mf model.MediaFile, format string, isMultiDisc bool) string {
_, file := filepath.Split(mf.Path)
if format != "raw" {
file = strings.TrimSuffix(file, mf.Suffix) + format
}
if isMultDisc {
if isMultiDisc {
file = fmt.Sprintf("Disc %02d/%s", mf.DiscNumber, file)
}
return fmt.Sprintf("%s/%s", sanitizeName(mf.Album), file)
@@ -91,18 +91,18 @@ func (a *archiver) albumFilename(mf model.MediaFile, format string, isMultDisc b
func (a *archiver) ZipShare(ctx context.Context, id string, out io.Writer) error {
s, err := a.shares.Load(ctx, id)
if !s.Downloadable {
return model.ErrNotAuthorized
}
if err != nil {
return err
}
if !s.Downloadable {
return model.ErrNotAuthorized
}
log.Debug(ctx, "Zipping share", "name", s.ID, "format", s.Format, "bitrate", s.MaxBitRate, "numTracks", len(s.Tracks))
return a.zipMediaFiles(ctx, id, s.Format, s.MaxBitRate, out, s.Tracks)
}
func (a *archiver) ZipPlaylist(ctx context.Context, id string, format string, bitrate int, out io.Writer) error {
pls, err := a.ds.Playlist(ctx).GetWithTracks(id, true)
pls, err := a.ds.Playlist(ctx).GetWithTracks(id, true, false)
if err != nil {
log.Error(ctx, "Error loading mediafiles from playlist", "id", id, err)
return err
@@ -138,13 +138,14 @@ func sanitizeName(target string) string {
}
func (a *archiver) addFileToZip(ctx context.Context, z *zip.Writer, mf model.MediaFile, format string, bitrate int, filename string) error {
path := mf.AbsolutePath()
w, err := z.CreateHeader(&zip.FileHeader{
Name: filename,
Modified: mf.UpdatedAt,
Method: zip.Store,
})
if err != nil {
log.Error(ctx, "Error creating zip entry", "file", mf.Path, err)
log.Error(ctx, "Error creating zip entry", "file", path, err)
return err
}
@@ -152,22 +153,22 @@ func (a *archiver) addFileToZip(ctx context.Context, z *zip.Writer, mf model.Med
if format != "raw" && format != "" {
r, err = a.ms.DoStream(ctx, &mf, format, bitrate, 0)
} else {
r, err = os.Open(mf.Path)
r, err = os.Open(path)
}
if err != nil {
log.Error(ctx, "Error opening file for zipping", "file", mf.Path, "format", format, err)
log.Error(ctx, "Error opening file for zipping", "file", path, "format", format, err)
return err
}
defer func() {
if err := r.Close(); err != nil && log.IsGreaterOrEqualTo(log.LevelDebug) {
log.Error(ctx, "Error closing stream", "id", mf.ID, "file", mf.Path, err)
log.Error(ctx, "Error closing stream", "id", mf.ID, "file", path, err)
}
}()
_, err = io.Copy(w, r)
if err != nil {
log.Error(ctx, "Error zipping file", "file", mf.Path, err)
log.Error(ctx, "Error zipping file", "file", path, err)
return err
}

View File

@@ -25,8 +25,8 @@ var _ = Describe("Archiver", func() {
BeforeEach(func() {
ms = &mockMediaStreamer{}
ds = &mockDataStore{}
sh = &mockShare{}
ds = &mockDataStore{}
arch = core.NewArchiver(ms, ds, sh)
})
@@ -134,7 +134,7 @@ var _ = Describe("Archiver", func() {
}
plRepo := &mockPlaylistRepository{}
plRepo.On("GetWithTracks", "1", true).Return(pls, nil)
plRepo.On("GetWithTracks", "1", true, false).Return(pls, nil)
ds.On("Playlist", mock.Anything).Return(plRepo)
ms.On("DoStream", mock.Anything, mock.Anything, "mp3", 128, 0).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2)
@@ -167,6 +167,19 @@ func (m *mockDataStore) Playlist(ctx context.Context) model.PlaylistRepository {
return args.Get(0).(model.PlaylistRepository)
}
func (m *mockDataStore) Library(context.Context) model.LibraryRepository {
return &mockLibraryRepository{}
}
type mockLibraryRepository struct {
mock.Mock
model.LibraryRepository
}
func (m *mockLibraryRepository) GetPath(id int) (string, error) {
return "/music", nil
}
type mockMediaFileRepository struct {
mock.Mock
model.MediaFileRepository
@@ -182,8 +195,8 @@ type mockPlaylistRepository struct {
model.PlaylistRepository
}
func (m *mockPlaylistRepository) GetWithTracks(id string, includeTracks bool) (*model.Playlist, error) {
args := m.Called(id, includeTracks)
func (m *mockPlaylistRepository) GetWithTracks(id string, refreshSmartPlaylists, includeMissing bool) (*model.Playlist, error) {
args := m.Called(id, refreshSmartPlaylists, includeMissing)
return args.Get(0).(*model.Playlist), args.Error(1)
}

View File

@@ -8,7 +8,7 @@ import (
"time"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/external"
"github.com/navidrome/navidrome/core/ffmpeg"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
@@ -24,15 +24,15 @@ type Artwork interface {
GetOrPlaceholder(ctx context.Context, id string, size int, square bool) (io.ReadCloser, time.Time, error)
}
func NewArtwork(ds model.DataStore, cache cache.FileCache, ffmpeg ffmpeg.FFmpeg, em core.ExternalMetadata) Artwork {
return &artwork{ds: ds, cache: cache, ffmpeg: ffmpeg, em: em}
func NewArtwork(ds model.DataStore, cache cache.FileCache, ffmpeg ffmpeg.FFmpeg, provider external.Provider) Artwork {
return &artwork{ds: ds, cache: cache, ffmpeg: ffmpeg, provider: provider}
}
type artwork struct {
ds model.DataStore
cache cache.FileCache
ffmpeg ffmpeg.FFmpeg
em core.ExternalMetadata
ds model.DataStore
cache cache.FileCache
ffmpeg ffmpeg.FFmpeg
provider external.Provider
}
type artworkReader interface {
@@ -115,9 +115,9 @@ func (a *artwork) getArtworkReader(ctx context.Context, artID model.ArtworkID, s
} else {
switch artID.Kind {
case model.KindArtistArtwork:
artReader, err = newArtistReader(ctx, a, artID, a.em)
artReader, err = newArtistReader(ctx, a, artID, a.provider)
case model.KindAlbumArtwork:
artReader, err = newAlbumArtworkReader(ctx, a, artID, a.em)
artReader, err = newAlbumArtworkReader(ctx, a, artID, a.provider)
case model.KindMediaFileArtwork:
artReader, err = newMediafileArtworkReader(ctx, a, artID)
case model.KindPlaylistArtwork:

View File

@@ -4,15 +4,10 @@ import (
"context"
"errors"
"image"
"image/jpeg"
"image/png"
"io"
"os"
"path/filepath"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
@@ -20,7 +15,8 @@ import (
. "github.com/onsi/gomega"
)
var _ = Describe("Artwork", func() {
// TODO Fix tests
var _ = XDescribe("Artwork", func() {
var aw *artwork
var ds model.DataStore
var ffmpeg *tests.MockFFmpeg
@@ -37,17 +33,17 @@ var _ = Describe("Artwork", func() {
ds = &tests.MockDataStore{MockedTranscoding: &tests.MockTranscodingRepo{}}
alOnlyEmbed = model.Album{ID: "222", Name: "Only embed", EmbedArtPath: "tests/fixtures/artist/an-album/test.mp3"}
alEmbedNotFound = model.Album{ID: "333", Name: "Embed not found", EmbedArtPath: "tests/fixtures/NON_EXISTENT.mp3"}
alOnlyExternal = model.Album{ID: "444", Name: "Only external", ImageFiles: "tests/fixtures/artist/an-album/front.png"}
alExternalNotFound = model.Album{ID: "555", Name: "External not found", ImageFiles: "tests/fixtures/NON_EXISTENT.png"}
//alOnlyExternal = model.Album{ID: "444", Name: "Only external", ImageFiles: "tests/fixtures/artist/an-album/front.png"}
//alExternalNotFound = model.Album{ID: "555", Name: "External not found", ImageFiles: "tests/fixtures/NON_EXISTENT.png"}
arMultipleCovers = model.Artist{ID: "777", Name: "All options"}
alMultipleCovers = model.Album{
ID: "666",
Name: "All options",
EmbedArtPath: "tests/fixtures/artist/an-album/test.mp3",
Paths: "tests/fixtures/artist/an-album",
ImageFiles: "tests/fixtures/artist/an-album/cover.jpg" + consts.Zwsp +
"tests/fixtures/artist/an-album/front.png" + consts.Zwsp +
"tests/fixtures/artist/an-album/artist.png",
//Paths: []string{"tests/fixtures/artist/an-album"},
//ImageFiles: "tests/fixtures/artist/an-album/cover.jpg" + consts.Zwsp +
// "tests/fixtures/artist/an-album/front.png" + consts.Zwsp +
// "tests/fixtures/artist/an-album/artist.png",
AlbumArtistID: "777",
}
mfWithEmbed = model.MediaFile{ID: "22", Path: "tests/fixtures/test.mp3", HasCoverArt: true, AlbumID: "222"}
@@ -245,11 +241,11 @@ var _ = Describe("Artwork", func() {
DescribeTable("resize",
func(format string, landscape bool, size int) {
coverFileName := "cover." + format
dirName := createImage(format, landscape, size)
//dirName := createImage(format, landscape, size)
alCover = model.Album{
ID: "444",
Name: "Only external",
ImageFiles: filepath.Join(dirName, coverFileName),
ID: "444",
Name: "Only external",
//ImageFiles: filepath.Join(dirName, coverFileName),
}
ds.Album(ctx).(*tests.MockAlbumRepo).SetData(model.Albums{
alCover,
@@ -274,24 +270,24 @@ var _ = Describe("Artwork", func() {
})
})
func createImage(format string, landscape bool, size int) string {
var img image.Image
if landscape {
img = image.NewRGBA(image.Rect(0, 0, size, size/2))
} else {
img = image.NewRGBA(image.Rect(0, 0, size/2, size))
}
tmpDir := GinkgoT().TempDir()
f, _ := os.Create(filepath.Join(tmpDir, "cover."+format))
defer f.Close()
switch format {
case "png":
_ = png.Encode(f, img)
case "jpg":
_ = jpeg.Encode(f, img, &jpeg.Options{Quality: 75})
}
return tmpDir
}
//func createImage(format string, landscape bool, size int) string {
// var img image.Image
//
// if landscape {
// img = image.NewRGBA(image.Rect(0, 0, size, size/2))
// } else {
// img = image.NewRGBA(image.Rect(0, 0, size/2, size))
// }
//
// tmpDir := GinkgoT().TempDir()
// f, _ := os.Create(filepath.Join(tmpDir, "cover."+format))
// defer f.Close()
// switch format {
// case "png":
// _ = png.Encode(f, img)
// case "jpg":
// _ = jpeg.Encode(f, img, &jpeg.Options{Quality: 75})
// }
//
// return tmpDir
//}

View File

@@ -22,6 +22,9 @@ type CacheWarmer interface {
PreCache(artID model.ArtworkID)
}
// NewCacheWarmer creates a new CacheWarmer instance. The CacheWarmer will pre-cache Artwork images in the background
// to speed up the response time when the image is requested by the UI. The cache is pre-populated with the original
// image size, as well as the size defined in the UICoverArtSize constant.
func NewCacheWarmer(artwork Artwork, cache cache.FileCache) CacheWarmer {
// If image cache is disabled, return a NOOP implementation
if conf.Server.ImageCacheSize == "0" || !conf.Server.EnableArtworkPrecache {
@@ -49,15 +52,7 @@ type cacheWarmer struct {
wakeSignal chan struct{}
}
var ignoredIds = map[string]struct{}{
consts.VariousArtistsID: {},
consts.UnknownArtistID: {},
}
func (a *cacheWarmer) PreCache(artID model.ArtworkID) {
if _, shouldIgnore := ignoredIds[artID.ID]; shouldIgnore {
return
}
a.mutex.Lock()
defer a.mutex.Unlock()
a.buffer[artID] = struct{}{}
@@ -104,14 +99,8 @@ func (a *cacheWarmer) run(ctx context.Context) {
}
func (a *cacheWarmer) waitSignal(ctx context.Context, timeout time.Duration) {
var to <-chan time.Time
if !a.cache.Available(ctx) {
tmr := time.NewTimer(timeout)
defer tmr.Stop()
to = tmr.C
}
select {
case <-to:
case <-time.After(timeout):
case <-a.wakeSignal:
case <-ctx.Done():
}
@@ -142,6 +131,10 @@ func (a *cacheWarmer) doCacheImage(ctx context.Context, id model.ArtworkID) erro
return nil
}
func NoopCacheWarmer() CacheWarmer {
return &noopCacheWarmer{}
}
type noopCacheWarmer struct{}
func (a *noopCacheWarmer) PreCache(model.ArtworkID) {}

View File

@@ -5,34 +5,52 @@ import (
"crypto/md5"
"fmt"
"io"
"path/filepath"
"slices"
"strings"
"time"
"github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/external"
"github.com/navidrome/navidrome/core/ffmpeg"
"github.com/navidrome/navidrome/model"
)
type albumArtworkReader struct {
cacheKey
a *artwork
em core.ExternalMetadata
album model.Album
a *artwork
provider external.Provider
album model.Album
updatedAt *time.Time
imgFiles []string
rootFolder string
}
func newAlbumArtworkReader(ctx context.Context, artwork *artwork, artID model.ArtworkID, em core.ExternalMetadata) (*albumArtworkReader, error) {
func newAlbumArtworkReader(ctx context.Context, artwork *artwork, artID model.ArtworkID, provider external.Provider) (*albumArtworkReader, error) {
al, err := artwork.ds.Album(ctx).Get(artID.ID)
if err != nil {
return nil, err
}
_, imgFiles, imagesUpdateAt, err := loadAlbumFoldersPaths(ctx, artwork.ds, *al)
if err != nil {
return nil, err
}
a := &albumArtworkReader{
a: artwork,
em: em,
album: *al,
a: artwork,
provider: provider,
album: *al,
updatedAt: imagesUpdateAt,
imgFiles: imgFiles,
rootFolder: core.AbsolutePath(ctx, artwork.ds, al.LibraryID, ""),
}
a.cacheKey.artID = artID
a.cacheKey.lastUpdate = al.UpdatedAt
if a.updatedAt != nil && a.updatedAt.After(al.UpdatedAt) {
a.cacheKey.lastUpdate = *a.updatedAt
} else {
a.cacheKey.lastUpdate = al.UpdatedAt
}
return a, nil
}
@@ -63,12 +81,43 @@ func (a *albumArtworkReader) fromCoverArtPriority(ctx context.Context, ffmpeg ff
pattern = strings.TrimSpace(pattern)
switch {
case pattern == "embedded":
ff = append(ff, fromTag(ctx, a.album.EmbedArtPath), fromFFmpegTag(ctx, ffmpeg, a.album.EmbedArtPath))
embedArtPath := filepath.Join(a.rootFolder, a.album.EmbedArtPath)
ff = append(ff, fromTag(ctx, embedArtPath), fromFFmpegTag(ctx, ffmpeg, embedArtPath))
case pattern == "external":
ff = append(ff, fromAlbumExternalSource(ctx, a.album, a.em))
case a.album.ImageFiles != "":
ff = append(ff, fromExternalFile(ctx, a.album.ImageFiles, pattern))
ff = append(ff, fromAlbumExternalSource(ctx, a.album, a.provider))
case len(a.imgFiles) > 0:
ff = append(ff, fromExternalFile(ctx, a.imgFiles, pattern))
}
}
return ff
}
func loadAlbumFoldersPaths(ctx context.Context, ds model.DataStore, albums ...model.Album) ([]string, []string, *time.Time, error) {
var folderIDs []string
for _, album := range albums {
folderIDs = append(folderIDs, album.FolderIDs...)
}
folders, err := ds.Folder(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"folder.id": folderIDs, "missing": false}})
if err != nil {
return nil, nil, nil, err
}
var paths []string
var imgFiles []string
var updatedAt time.Time
for _, f := range folders {
path := f.AbsolutePath()
paths = append(paths, path)
if f.ImagesUpdatedAt.After(updatedAt) {
updatedAt = f.ImagesUpdatedAt
}
for _, img := range f.ImageFiles {
imgFiles = append(imgFiles, filepath.Join(path, img))
}
}
// Sort image files to ensure consistent selection of cover art
// This prioritizes files from lower-numbered disc folders by sorting the paths
slices.Sort(imgFiles)
return paths, imgFiles, &updatedAt, nil
}

View File

@@ -0,0 +1,76 @@
package artwork
import (
"context"
"path/filepath"
"time"
"github.com/navidrome/navidrome/model"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("Album Artwork Reader", func() {
Describe("loadAlbumFoldersPaths", func() {
var (
ctx context.Context
ds *fakeDataStore
repo *fakeFolderRepo
album model.Album
now time.Time
expectedAt time.Time
)
BeforeEach(func() {
ctx = context.Background()
now = time.Now().Truncate(time.Second)
expectedAt = now.Add(5 * time.Minute)
// Set up the test folders with image files
repo = &fakeFolderRepo{
result: []model.Folder{
{
Path: "Artist/Album/Disc1",
ImagesUpdatedAt: expectedAt,
ImageFiles: []string{"cover.jpg", "back.jpg"},
},
{
Path: "Artist/Album/Disc2",
ImagesUpdatedAt: now,
ImageFiles: []string{"cover.jpg"},
},
{
Path: "Artist/Album/Disc10",
ImagesUpdatedAt: now,
ImageFiles: []string{"cover.jpg"},
},
},
err: nil,
}
ds = &fakeDataStore{
folderRepo: repo,
}
album = model.Album{
ID: "album1",
Name: "Album",
FolderIDs: []string{"folder1", "folder2", "folder3"},
}
})
It("returns sorted image files", func() {
_, imgFiles, imagesUpdatedAt, err := loadAlbumFoldersPaths(ctx, ds, album)
Expect(err).ToNot(HaveOccurred())
Expect(*imagesUpdatedAt).To(Equal(expectedAt))
// Check that image files are sorted alphabetically
Expect(imgFiles).To(HaveLen(4))
// The files should be sorted by full path
Expect(imgFiles[0]).To(Equal(filepath.FromSlash("Artist/Album/Disc1/back.jpg")))
Expect(imgFiles[1]).To(Equal(filepath.FromSlash("Artist/Album/Disc1/cover.jpg")))
Expect(imgFiles[2]).To(Equal(filepath.FromSlash("Artist/Album/Disc10/cover.jpg")))
Expect(imgFiles[3]).To(Equal(filepath.FromSlash("Artist/Album/Disc2/cover.jpg")))
})
})
})

View File

@@ -13,8 +13,8 @@ import (
"github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/external"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/str"
@@ -23,42 +23,49 @@ import (
type artistReader struct {
cacheKey
a *artwork
em core.ExternalMetadata
provider external.Provider
artist model.Artist
artistFolder string
files string
imgFiles []string
}
func newArtistReader(ctx context.Context, artwork *artwork, artID model.ArtworkID, em core.ExternalMetadata) (*artistReader, error) {
func newArtistReader(ctx context.Context, artwork *artwork, artID model.ArtworkID, provider external.Provider) (*artistReader, error) {
ar, err := artwork.ds.Artist(ctx).Get(artID.ID)
if err != nil {
return nil, err
}
als, err := artwork.ds.Album(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"album_artist_id": artID.ID}})
// Only consider albums where the artist is the sole album artist.
als, err := artwork.ds.Album(ctx).GetAll(model.QueryOptions{
Filters: squirrel.And{
squirrel.Eq{"album_artist_id": artID.ID},
squirrel.Eq{"json_array_length(participants, '$.albumartist')": 1},
},
})
if err != nil {
return nil, err
}
albumPaths, imgFiles, imagesUpdatedAt, err := loadAlbumFoldersPaths(ctx, artwork.ds, als...)
if err != nil {
return nil, err
}
artistFolder, artistFolderLastUpdate, err := loadArtistFolder(ctx, artwork.ds, als, albumPaths)
if err != nil {
return nil, err
}
a := &artistReader{
a: artwork,
em: em,
artist: *ar,
a: artwork,
provider: provider,
artist: *ar,
artistFolder: artistFolder,
imgFiles: imgFiles,
}
// TODO Find a way to factor in the ExternalUpdateInfoAt in the cache key. Problem is that it can
// change _after_ retrieving from external sources, making the key invalid
//a.cacheKey.lastUpdate = ar.ExternalInfoUpdatedAt
var files []string
var paths []string
for _, al := range als {
files = append(files, al.ImageFiles)
paths = append(paths, splitList(al.Paths)...)
if a.cacheKey.lastUpdate.Before(al.UpdatedAt) {
a.cacheKey.lastUpdate = al.UpdatedAt
}
}
a.files = strings.Join(files, consts.Zwsp)
a.artistFolder = str.LongestCommonPrefix(paths)
if !strings.HasSuffix(a.artistFolder, string(filepath.Separator)) {
a.artistFolder, _ = filepath.Split(a.artistFolder)
a.cacheKey.lastUpdate = *imagesUpdatedAt
if artistFolderLastUpdate.After(a.cacheKey.lastUpdate) {
a.cacheKey.lastUpdate = artistFolderLastUpdate
}
a.cacheKey.artID = artID
return a, nil
@@ -89,9 +96,9 @@ func (a *artistReader) fromArtistArtPriority(ctx context.Context, priority strin
pattern = strings.TrimSpace(pattern)
switch {
case pattern == "external":
ff = append(ff, fromArtistExternalSource(ctx, a.artist, a.em))
ff = append(ff, fromArtistExternalSource(ctx, a.artist, a.provider))
case strings.HasPrefix(pattern, "album/"):
ff = append(ff, fromExternalFile(ctx, a.files, strings.TrimPrefix(pattern, "album/")))
ff = append(ff, fromExternalFile(ctx, a.imgFiles, strings.TrimPrefix(pattern, "album/")))
default:
ff = append(ff, fromArtistFolder(ctx, a.artistFolder, pattern))
}
@@ -125,3 +132,33 @@ func fromArtistFolder(ctx context.Context, artistFolder string, pattern string)
return nil, "", nil
}
}
func loadArtistFolder(ctx context.Context, ds model.DataStore, albums model.Albums, paths []string) (string, time.Time, error) {
if len(albums) == 0 {
return "", time.Time{}, nil
}
libID := albums[0].LibraryID // Just need one of the albums, as they should all be in the same Library
folderPath := str.LongestCommonPrefix(paths)
if !strings.HasSuffix(folderPath, string(filepath.Separator)) {
folderPath, _ = filepath.Split(folderPath)
}
folderPath = filepath.Dir(folderPath)
// Manipulate the path to get the folder ID
// TODO: This is a bit hacky, but it's the easiest way to get the folder ID, ATM
libPath := core.AbsolutePath(ctx, ds, libID, "")
folderID := model.FolderID(model.Library{ID: libID, Path: libPath}, folderPath)
log.Trace(ctx, "Calculating artist folder details", "folderPath", folderPath, "folderID", folderID,
"libPath", libPath, "libID", libID, "albumPaths", paths)
// Get the last update time for the folder
folders, err := ds.Folder(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"folder.id": folderID, "missing": false}})
if err != nil || len(folders) == 0 {
log.Warn(ctx, "Could not find folder for artist", "folderPath", folderPath, "id", folderID,
"libPath", libPath, "libID", libID, err)
return "", time.Time{}, err
}
return folderPath, folders[0].ImagesUpdatedAt, nil
}

View File

@@ -0,0 +1,141 @@
package artwork
import (
"context"
"errors"
"path/filepath"
"time"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/model"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("artistReader", func() {
var _ = Describe("loadArtistFolder", func() {
var (
ctx context.Context
fds *fakeDataStore
repo *fakeFolderRepo
albums model.Albums
paths []string
now time.Time
expectedUpdTime time.Time
)
BeforeEach(func() {
ctx = context.Background()
DeferCleanup(stubCoreAbsolutePath())
now = time.Now().Truncate(time.Second)
expectedUpdTime = now.Add(5 * time.Minute)
repo = &fakeFolderRepo{
result: []model.Folder{
{
ImagesUpdatedAt: expectedUpdTime,
},
},
err: nil,
}
fds = &fakeDataStore{
folderRepo: repo,
}
albums = model.Albums{
{LibraryID: 1, ID: "album1", Name: "Album 1"},
}
})
When("no albums provided", func() {
It("returns empty and zero time", func() {
folder, upd, err := loadArtistFolder(ctx, fds, model.Albums{}, []string{"/dummy/path"})
Expect(err).ToNot(HaveOccurred())
Expect(folder).To(BeEmpty())
Expect(upd).To(BeZero())
})
})
When("artist has only one album", func() {
It("returns the parent folder", func() {
paths = []string{
filepath.FromSlash("/music/artist/album1"),
}
folder, upd, err := loadArtistFolder(ctx, fds, albums, paths)
Expect(err).ToNot(HaveOccurred())
Expect(folder).To(Equal("/music/artist"))
Expect(upd).To(Equal(expectedUpdTime))
})
})
When("the artist have multiple albums", func() {
It("returns the common prefix for the albums paths", func() {
paths = []string{
filepath.FromSlash("/music/library/artist/one"),
filepath.FromSlash("/music/library/artist/two"),
}
folder, upd, err := loadArtistFolder(ctx, fds, albums, paths)
Expect(err).ToNot(HaveOccurred())
Expect(folder).To(Equal(filepath.FromSlash("/music/library/artist")))
Expect(upd).To(Equal(expectedUpdTime))
})
})
When("the album paths contain same prefix", func() {
It("returns the common prefix", func() {
paths = []string{
filepath.FromSlash("/music/artist/album1"),
filepath.FromSlash("/music/artist/album2"),
}
folder, upd, err := loadArtistFolder(ctx, fds, albums, paths)
Expect(err).ToNot(HaveOccurred())
Expect(folder).To(Equal("/music/artist"))
Expect(upd).To(Equal(expectedUpdTime))
})
})
When("ds.Folder().GetAll returns an error", func() {
It("returns an error", func() {
paths = []string{
filepath.FromSlash("/music/artist/album1"),
filepath.FromSlash("/music/artist/album2"),
}
repo.err = errors.New("fake error")
folder, upd, err := loadArtistFolder(ctx, fds, albums, paths)
Expect(err).To(MatchError(ContainSubstring("fake error")))
// Folder and time are empty on error.
Expect(folder).To(BeEmpty())
Expect(upd).To(BeZero())
})
})
})
})
type fakeFolderRepo struct {
model.FolderRepository
result []model.Folder
err error
}
func (f *fakeFolderRepo) GetAll(...model.QueryOptions) ([]model.Folder, error) {
return f.result, f.err
}
type fakeDataStore struct {
model.DataStore
folderRepo *fakeFolderRepo
}
func (fds *fakeDataStore) Folder(_ context.Context) model.FolderRepository {
return fds.folderRepo
}
func stubCoreAbsolutePath() func() {
// Override core.AbsolutePath to return a fixed string during tests.
original := core.AbsolutePath
core.AbsolutePath = func(_ context.Context, ds model.DataStore, libID int, p string) string {
return filepath.FromSlash("/music")
}
return func() {
core.AbsolutePath = original
}
}

View File

@@ -54,9 +54,10 @@ func (a *mediafileArtworkReader) LastUpdated() time.Time {
func (a *mediafileArtworkReader) Reader(ctx context.Context) (io.ReadCloser, string, error) {
var ff []sourceFunc
if a.mediafile.CoverArtID().Kind == model.KindMediaFileArtwork {
path := a.mediafile.AbsolutePath()
ff = []sourceFunc{
fromTag(ctx, a.mediafile.Path),
fromFFmpegTag(ctx, a.a.ffmpeg, a.mediafile.Path),
fromTag(ctx, path),
fromFFmpegTag(ctx, a.a.ffmpeg, path),
}
}
ff = append(ff, fromAlbum(ctx, a.a, a.mediafile.AlbumCoverArtID()))

View File

@@ -61,7 +61,7 @@ func (a *playlistArtworkReader) fromGeneratedTiledCover(ctx context.Context) sou
}
}
func toArtworkIDs(albumIDs []string) []model.ArtworkID {
func toAlbumArtworkIDs(albumIDs []string) []model.ArtworkID {
return slice.Map(albumIDs, func(id string) model.ArtworkID {
al := model.Album{ID: id}
return al.CoverArtID()
@@ -75,24 +75,21 @@ func (a *playlistArtworkReader) loadTiles(ctx context.Context) ([]image.Image, e
log.Error(ctx, "Error getting album IDs for playlist", "id", a.pl.ID, "name", a.pl.Name, err)
return nil, err
}
ids := toArtworkIDs(albumIds)
ids := toAlbumArtworkIDs(albumIds)
var tiles []image.Image
for len(tiles) < 4 {
if len(ids) == 0 {
for _, id := range ids {
r, _, err := fromAlbum(ctx, a.a, id)()
if err == nil {
tile, err := a.createTile(ctx, r)
if err == nil {
tiles = append(tiles, tile)
}
_ = r.Close()
}
if len(tiles) == 4 {
break
}
id := ids[len(ids)-1]
ids = ids[0 : len(ids)-1]
r, _, err := fromAlbum(ctx, a.a, id)()
if err != nil {
continue
}
tile, err := a.createTile(ctx, r)
if err == nil {
tiles = append(tiles, tile)
}
_ = r.Close()
}
switch len(tiles) {
case 0:

View File

@@ -63,12 +63,12 @@ func (a *resizedArtworkReader) Reader(ctx context.Context) (io.ReadCloser, strin
resized, origSize, err := resizeImage(orig, a.size, a.square)
if resized == nil {
log.Trace(ctx, "Image smaller than requested size", "artID", a.artID, "original", origSize, "resized", a.size)
log.Trace(ctx, "Image smaller than requested size", "artID", a.artID, "original", origSize, "resized", a.size, "square", a.square)
} else {
log.Trace(ctx, "Resizing artwork", "artID", a.artID, "original", origSize, "resized", a.size)
log.Trace(ctx, "Resizing artwork", "artID", a.artID, "original", origSize, "resized", a.size, "square", a.square)
}
if err != nil {
log.Warn(ctx, "Could not resize image. Will return image as is", "artID", a.artID, "size", a.size, err)
log.Warn(ctx, "Could not resize image. Will return image as is", "artID", a.artID, "size", a.size, "square", a.square, err)
}
if err != nil || resized == nil {
// if we couldn't resize the image, return the original

View File

@@ -17,7 +17,7 @@ import (
"github.com/dhowden/tag"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/external"
"github.com/navidrome/navidrome/core/ffmpeg"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
@@ -53,13 +53,9 @@ func (f sourceFunc) String() string {
return name
}
func splitList(s string) []string {
return strings.Split(s, consts.Zwsp)
}
func fromExternalFile(ctx context.Context, files string, pattern string) sourceFunc {
func fromExternalFile(ctx context.Context, files []string, pattern string) sourceFunc {
return func() (io.ReadCloser, string, error) {
for _, file := range splitList(files) {
for _, file := range files {
_, name := filepath.Split(file)
match, err := filepath.Match(pattern, strings.ToLower(name))
if err != nil {
@@ -161,9 +157,9 @@ func fromAlbumPlaceholder() sourceFunc {
return r, consts.PlaceholderAlbumArt, nil
}
}
func fromArtistExternalSource(ctx context.Context, ar model.Artist, em core.ExternalMetadata) sourceFunc {
func fromArtistExternalSource(ctx context.Context, ar model.Artist, provider external.Provider) sourceFunc {
return func() (io.ReadCloser, string, error) {
imageUrl, err := em.ArtistImage(ctx, ar.ID)
imageUrl, err := provider.ArtistImage(ctx, ar.ID)
if err != nil {
return nil, "", err
}
@@ -172,9 +168,9 @@ func fromArtistExternalSource(ctx context.Context, ar model.Artist, em core.Exte
}
}
func fromAlbumExternalSource(ctx context.Context, al model.Album, em core.ExternalMetadata) sourceFunc {
func fromAlbumExternalSource(ctx context.Context, al model.Album, provider external.Provider) sourceFunc {
return func() (io.ReadCloser, string, error) {
imageUrl, err := em.AlbumImage(ctx, al.ID)
imageUrl, err := provider.AlbumImage(ctx, al.ID)
if err != nil {
return nil, "", err
}

View File

@@ -8,12 +8,12 @@ import (
"time"
"github.com/go-chi/jwtauth/v5"
"github.com/google/uuid"
"github.com/lestrrat-go/jwx/v2/jwt"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/id"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/utils"
)
@@ -125,7 +125,7 @@ func WithAdminUser(ctx context.Context, ds model.DataStore) context.Context {
}
func createNewSecret(ctx context.Context, ds model.DataStore) string {
secret := uuid.NewString()
secret := id.NewRandom()
encSecret, err := utils.Encrypt(ctx, getEncKey(), secret)
if err != nil {
log.Error(ctx, "Could not encrypt JWT secret", err)

View File

@@ -2,7 +2,9 @@ package core
import (
"context"
"path/filepath"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
)
@@ -13,3 +15,13 @@ func userName(ctx context.Context) string {
return user.UserName
}
}
// BFR We should only access files through the `storage.Storage` interface. This will require changing how
// TagLib and ffmpeg access files
var AbsolutePath = func(ctx context.Context, ds model.DataStore, libId int, path string) string {
libPath, err := ds.Library(ctx).GetPath(libId)
if err != nil {
return path
}
return filepath.Join(libPath, path)
}

55
core/common_test.go Normal file
View File

@@ -0,0 +1,55 @@
package core
import (
"context"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/tests"
)
var _ = Describe("common.go", func() {
Describe("userName", func() {
It("returns the username from context", func() {
ctx := request.WithUser(context.Background(), model.User{UserName: "testuser"})
Expect(userName(ctx)).To(Equal("testuser"))
})
It("returns 'UNKNOWN' if no user in context", func() {
ctx := context.Background()
Expect(userName(ctx)).To(Equal("UNKNOWN"))
})
})
Describe("AbsolutePath", func() {
var (
ds *tests.MockDataStore
libId int
path string
)
BeforeEach(func() {
ds = &tests.MockDataStore{}
libId = 1
path = "music/file.mp3"
mockLib := &tests.MockLibraryRepo{}
mockLib.SetData(model.Libraries{{ID: libId, Path: "/library/root"}})
ds.MockedLibrary = mockLib
})
It("returns the absolute path when library exists", func() {
ctx := context.Background()
abs := AbsolutePath(ctx, ds, libId, path)
Expect(abs).To(Equal("/library/root/music/file.mp3"))
})
It("returns the original path if library not found", func() {
ctx := context.Background()
abs := AbsolutePath(ctx, ds, 999, path)
Expect(abs).To(Equal(path))
})
})
})

270
core/external/extdata_helper_test.go vendored Normal file
View File

@@ -0,0 +1,270 @@
package external_test
import (
"context"
"errors"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/model"
"github.com/stretchr/testify/mock"
)
// --- Shared Mock Implementations ---
// mockArtistRepo mocks model.ArtistRepository
type mockArtistRepo struct {
mock.Mock
model.ArtistRepository
}
func newMockArtistRepo() *mockArtistRepo {
return &mockArtistRepo{}
}
// SetData sets up basic Get expectations.
func (m *mockArtistRepo) SetData(artists model.Artists) {
for _, a := range artists {
artistCopy := a
m.On("Get", artistCopy.ID).Return(&artistCopy, nil)
}
}
// Get implements model.ArtistRepository.
func (m *mockArtistRepo) Get(id string) (*model.Artist, error) {
args := m.Called(id)
if args.Get(0) == nil {
return nil, args.Error(1)
}
return args.Get(0).(*model.Artist), args.Error(1)
}
// GetAll implements model.ArtistRepository.
func (m *mockArtistRepo) GetAll(options ...model.QueryOptions) (model.Artists, error) {
argsSlice := make([]interface{}, len(options))
for i, v := range options {
argsSlice[i] = v
}
args := m.Called(argsSlice...)
if args.Get(0) == nil {
return nil, args.Error(1)
}
return args.Get(0).(model.Artists), args.Error(1)
}
// SetError is a helper to set up a generic error for GetAll.
func (m *mockArtistRepo) SetError(hasError bool) {
if hasError {
m.On("GetAll", mock.Anything).Return(nil, errors.New("mock repo error"))
}
}
// FindByName is a helper to set up a GetAll expectation for finding by name.
func (m *mockArtistRepo) FindByName(name string, artist model.Artist) {
m.On("GetAll", mock.MatchedBy(func(opt model.QueryOptions) bool {
return opt.Filters != nil
})).Return(model.Artists{artist}, nil).Once()
}
// mockMediaFileRepo mocks model.MediaFileRepository
type mockMediaFileRepo struct {
mock.Mock
model.MediaFileRepository
}
func newMockMediaFileRepo() *mockMediaFileRepo {
return &mockMediaFileRepo{}
}
// SetData sets up basic Get expectations.
func (m *mockMediaFileRepo) SetData(mediaFiles model.MediaFiles) {
for _, mf := range mediaFiles {
mfCopy := mf
m.On("Get", mfCopy.ID).Return(&mfCopy, nil)
}
}
// Get implements model.MediaFileRepository.
func (m *mockMediaFileRepo) Get(id string) (*model.MediaFile, error) {
args := m.Called(id)
if args.Get(0) == nil {
return nil, args.Error(1)
}
return args.Get(0).(*model.MediaFile), args.Error(1)
}
// GetAll implements model.MediaFileRepository.
func (m *mockMediaFileRepo) GetAll(options ...model.QueryOptions) (model.MediaFiles, error) {
argsSlice := make([]interface{}, len(options))
for i, v := range options {
argsSlice[i] = v
}
args := m.Called(argsSlice...)
if args.Get(0) == nil {
return nil, args.Error(1)
}
return args.Get(0).(model.MediaFiles), args.Error(1)
}
// SetError is a helper to set up a generic error for GetAll.
func (m *mockMediaFileRepo) SetError(hasError bool) {
if hasError {
m.On("GetAll", mock.Anything).Return(nil, errors.New("mock repo error"))
}
}
// FindByMBID is a helper to set up a GetAll expectation for finding by MBID.
func (m *mockMediaFileRepo) FindByMBID(mbid string, mediaFile model.MediaFile) {
m.On("GetAll", mock.MatchedBy(func(opt model.QueryOptions) bool {
return opt.Filters != nil
})).Return(model.MediaFiles{mediaFile}, nil).Once()
}
// FindByArtistAndTitle is a helper to set up a GetAll expectation for finding by artist/title.
func (m *mockMediaFileRepo) FindByArtistAndTitle(artistID string, title string, mediaFile model.MediaFile) {
m.On("GetAll", mock.MatchedBy(func(opt model.QueryOptions) bool {
return opt.Filters != nil
})).Return(model.MediaFiles{mediaFile}, nil).Once()
}
// mockAlbumRepo mocks model.AlbumRepository
type mockAlbumRepo struct {
mock.Mock
model.AlbumRepository
}
func newMockAlbumRepo() *mockAlbumRepo {
return &mockAlbumRepo{}
}
// Get implements model.AlbumRepository.
func (m *mockAlbumRepo) Get(id string) (*model.Album, error) {
args := m.Called(id)
if args.Get(0) == nil {
return nil, args.Error(1)
}
return args.Get(0).(*model.Album), args.Error(1)
}
// GetAll implements model.AlbumRepository.
func (m *mockAlbumRepo) GetAll(options ...model.QueryOptions) (model.Albums, error) {
argsSlice := make([]interface{}, len(options))
for i, v := range options {
argsSlice[i] = v
}
args := m.Called(argsSlice...)
if args.Get(0) == nil {
return nil, args.Error(1)
}
return args.Get(0).(model.Albums), args.Error(1)
}
// mockSimilarArtistAgent mocks agents implementing ArtistTopSongsRetriever and ArtistSimilarRetriever
type mockSimilarArtistAgent struct {
mock.Mock
agents.Interface // Embed to satisfy methods not explicitly mocked
}
func (m *mockSimilarArtistAgent) AgentName() string {
return "mockSimilar"
}
func (m *mockSimilarArtistAgent) GetArtistTopSongs(ctx context.Context, id, artistName, mbid string, count int) ([]agents.Song, error) {
args := m.Called(ctx, id, artistName, mbid, count)
if args.Get(0) != nil {
return args.Get(0).([]agents.Song), args.Error(1)
}
return nil, args.Error(1)
}
func (m *mockSimilarArtistAgent) GetSimilarArtists(ctx context.Context, id, name, mbid string, limit int) ([]agents.Artist, error) {
args := m.Called(ctx, id, name, mbid, limit)
if args.Get(0) != nil {
return args.Get(0).([]agents.Artist), args.Error(1)
}
return nil, args.Error(1)
}
// mockAgents mocks the main Agents interface used by Provider
type mockAgents struct {
mock.Mock // Embed testify mock
topSongsAgent agents.ArtistTopSongsRetriever
similarAgent agents.ArtistSimilarRetriever
imageAgent agents.ArtistImageRetriever
albumInfoAgent agents.AlbumInfoRetriever
bioAgent agents.ArtistBiographyRetriever
mbidAgent agents.ArtistMBIDRetriever
urlAgent agents.ArtistURLRetriever
agents.Interface
}
func (m *mockAgents) AgentName() string {
return "mockCombined"
}
func (m *mockAgents) GetSimilarArtists(ctx context.Context, id, name, mbid string, limit int) ([]agents.Artist, error) {
if m.similarAgent != nil {
return m.similarAgent.GetSimilarArtists(ctx, id, name, mbid, limit)
}
args := m.Called(ctx, id, name, mbid, limit)
if args.Get(0) != nil {
return args.Get(0).([]agents.Artist), args.Error(1)
}
return nil, args.Error(1)
}
func (m *mockAgents) GetArtistTopSongs(ctx context.Context, id, artistName, mbid string, count int) ([]agents.Song, error) {
if m.topSongsAgent != nil {
return m.topSongsAgent.GetArtistTopSongs(ctx, id, artistName, mbid, count)
}
args := m.Called(ctx, id, artistName, mbid, count)
if args.Get(0) != nil {
return args.Get(0).([]agents.Song), args.Error(1)
}
return nil, args.Error(1)
}
func (m *mockAgents) GetAlbumInfo(ctx context.Context, name, artist, mbid string) (*agents.AlbumInfo, error) {
if m.albumInfoAgent != nil {
return m.albumInfoAgent.GetAlbumInfo(ctx, name, artist, mbid)
}
args := m.Called(ctx, name, artist, mbid)
if args.Get(0) != nil {
return args.Get(0).(*agents.AlbumInfo), args.Error(1)
}
return nil, args.Error(1)
}
func (m *mockAgents) GetArtistMBID(ctx context.Context, id string, name string) (string, error) {
if m.mbidAgent != nil {
return m.mbidAgent.GetArtistMBID(ctx, id, name)
}
args := m.Called(ctx, id, name)
return args.String(0), args.Error(1)
}
func (m *mockAgents) GetArtistURL(ctx context.Context, id, name, mbid string) (string, error) {
if m.urlAgent != nil {
return m.urlAgent.GetArtistURL(ctx, id, name, mbid)
}
args := m.Called(ctx, id, name, mbid)
return args.String(0), args.Error(1)
}
func (m *mockAgents) GetArtistBiography(ctx context.Context, id, name, mbid string) (string, error) {
if m.bioAgent != nil {
return m.bioAgent.GetArtistBiography(ctx, id, name, mbid)
}
args := m.Called(ctx, id, name, mbid)
return args.String(0), args.Error(1)
}
func (m *mockAgents) GetArtistImages(ctx context.Context, id, name, mbid string) ([]agents.ExternalImage, error) {
if m.imageAgent != nil {
return m.imageAgent.GetArtistImages(ctx, id, name, mbid)
}
args := m.Called(ctx, id, name, mbid)
if args.Get(0) != nil {
return args.Get(0).([]agents.ExternalImage), args.Error(1)
}
return nil, args.Error(1)
}

17
core/external/extdata_suite_test.go vendored Normal file
View File

@@ -0,0 +1,17 @@
package external
import (
"testing"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
func TestExternal(t *testing.T) {
tests.Init(t, false)
log.SetLevel(log.LevelFatal)
RegisterFailHandler(Fail)
RunSpecs(t, "External Suite")
}

View File

@@ -1,4 +1,4 @@
package core
package external
import (
"context"
@@ -13,25 +13,26 @@ import (
"github.com/navidrome/navidrome/core/agents"
_ "github.com/navidrome/navidrome/core/agents/lastfm"
_ "github.com/navidrome/navidrome/core/agents/listenbrainz"
_ "github.com/navidrome/navidrome/core/agents/mcp"
_ "github.com/navidrome/navidrome/core/agents/spotify"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils"
. "github.com/navidrome/navidrome/utils/gg"
"github.com/navidrome/navidrome/utils/random"
"github.com/navidrome/navidrome/utils/slice"
"github.com/navidrome/navidrome/utils/str"
"golang.org/x/sync/errgroup"
)
const (
unavailableArtistID = "-1"
maxSimilarArtists = 100
refreshDelay = 5 * time.Second
refreshTimeout = 15 * time.Second
refreshQueueLength = 2000
maxSimilarArtists = 100
refreshDelay = 5 * time.Second
refreshTimeout = 15 * time.Second
refreshQueueLength = 2000
)
type ExternalMetadata interface {
type Provider interface {
UpdateAlbumInfo(ctx context.Context, id string) (*model.Album, error)
UpdateArtistInfo(ctx context.Context, id string, count int, includeNotPresent bool) (*model.Artist, error)
SimilarSongs(ctx context.Context, id string, count int) (model.MediaFiles, error)
@@ -40,9 +41,9 @@ type ExternalMetadata interface {
AlbumImage(ctx context.Context, id string) (*url.URL, error)
}
type externalMetadata struct {
type provider struct {
ds model.DataStore
ag *agents.Agents
ag Agents
artistQueue refreshQueue[auxArtist]
albumQueue refreshQueue[auxAlbum]
}
@@ -57,14 +58,24 @@ type auxArtist struct {
Name string
}
func NewExternalMetadata(ds model.DataStore, agents *agents.Agents) ExternalMetadata {
e := &externalMetadata{ds: ds, ag: agents}
type Agents interface {
agents.AlbumInfoRetriever
agents.ArtistBiographyRetriever
agents.ArtistMBIDRetriever
agents.ArtistImageRetriever
agents.ArtistSimilarRetriever
agents.ArtistTopSongsRetriever
agents.ArtistURLRetriever
}
func NewProvider(ds model.DataStore, agents Agents) Provider {
e := &provider{ds: ds, ag: agents}
e.artistQueue = newRefreshQueue(context.TODO(), e.populateArtistInfo)
e.albumQueue = newRefreshQueue(context.TODO(), e.populateAlbumInfo)
return e
}
func (e *externalMetadata) getAlbum(ctx context.Context, id string) (auxAlbum, error) {
func (e *provider) getAlbum(ctx context.Context, id string) (auxAlbum, error) {
var entity interface{}
entity, err := model.GetEntityByID(ctx, e.ds, id)
if err != nil {
@@ -81,10 +92,11 @@ func (e *externalMetadata) getAlbum(ctx context.Context, id string) (auxAlbum, e
default:
return auxAlbum{}, model.ErrNotFound
}
return album, nil
}
func (e *externalMetadata) UpdateAlbumInfo(ctx context.Context, id string) (*model.Album, error) {
func (e *provider) UpdateAlbumInfo(ctx context.Context, id string) (*model.Album, error) {
album, err := e.getAlbum(ctx, id)
if err != nil {
log.Info(ctx, "Not found", "id", id)
@@ -109,7 +121,7 @@ func (e *externalMetadata) UpdateAlbumInfo(ctx context.Context, id string) (*mod
return &album.Album, nil
}
func (e *externalMetadata) populateAlbumInfo(ctx context.Context, album auxAlbum) (auxAlbum, error) {
func (e *provider) populateAlbumInfo(ctx context.Context, album auxAlbum) (auxAlbum, error) {
start := time.Now()
info, err := e.ag.GetAlbumInfo(ctx, album.Name, album.AlbumArtist, album.MbzAlbumID)
if errors.Is(err, agents.ErrNotFound) {
@@ -144,7 +156,7 @@ func (e *externalMetadata) populateAlbumInfo(ctx context.Context, album auxAlbum
}
}
err = e.ds.Album(ctx).Put(&album.Album)
err = e.ds.Album(ctx).UpdateExternalInfo(&album.Album)
if err != nil {
log.Error(ctx, "Error trying to update album external information", "id", album.ID, "name", album.Name,
"elapsed", time.Since(start), err)
@@ -155,7 +167,7 @@ func (e *externalMetadata) populateAlbumInfo(ctx context.Context, album auxAlbum
return album, nil
}
func (e *externalMetadata) getArtist(ctx context.Context, id string) (auxArtist, error) {
func (e *provider) getArtist(ctx context.Context, id string) (auxArtist, error) {
var entity interface{}
entity, err := model.GetEntityByID(ctx, e.ds, id)
if err != nil {
@@ -177,7 +189,7 @@ func (e *externalMetadata) getArtist(ctx context.Context, id string) (auxArtist,
return artist, nil
}
func (e *externalMetadata) UpdateArtistInfo(ctx context.Context, id string, similarCount int, includeNotPresent bool) (*model.Artist, error) {
func (e *provider) UpdateArtistInfo(ctx context.Context, id string, similarCount int, includeNotPresent bool) (*model.Artist, error) {
artist, err := e.refreshArtistInfo(ctx, id)
if err != nil {
return nil, err
@@ -187,7 +199,7 @@ func (e *externalMetadata) UpdateArtistInfo(ctx context.Context, id string, simi
return &artist.Artist, err
}
func (e *externalMetadata) refreshArtistInfo(ctx context.Context, id string) (auxArtist, error) {
func (e *provider) refreshArtistInfo(ctx context.Context, id string) (auxArtist, error) {
artist, err := e.getArtist(ctx, id)
if err != nil {
return auxArtist{}, err
@@ -211,7 +223,7 @@ func (e *externalMetadata) refreshArtistInfo(ctx context.Context, id string) (au
return artist, nil
}
func (e *externalMetadata) populateArtistInfo(ctx context.Context, artist auxArtist) (auxArtist, error) {
func (e *provider) populateArtistInfo(ctx context.Context, artist auxArtist) (auxArtist, error) {
start := time.Now()
// Get MBID first, if it is not yet available
if artist.MbzArtistID == "" {
@@ -236,7 +248,7 @@ func (e *externalMetadata) populateArtistInfo(ctx context.Context, artist auxArt
}
artist.ExternalInfoUpdatedAt = P(time.Now())
err := e.ds.Artist(ctx).Put(&artist.Artist)
err := e.ds.Artist(ctx).UpdateExternalInfo(&artist.Artist)
if err != nil {
log.Error(ctx, "Error trying to update artist external information", "id", artist.ID, "name", artist.Name,
"elapsed", time.Since(start), err)
@@ -246,7 +258,7 @@ func (e *externalMetadata) populateArtistInfo(ctx context.Context, artist auxArt
return artist, nil
}
func (e *externalMetadata) SimilarSongs(ctx context.Context, id string, count int) (model.MediaFiles, error) {
func (e *provider) SimilarSongs(ctx context.Context, id string, count int) (model.MediaFiles, error) {
artist, err := e.getArtist(ctx, id)
if err != nil {
return nil, err
@@ -304,7 +316,7 @@ func (e *externalMetadata) SimilarSongs(ctx context.Context, id string, count in
return similarSongs, nil
}
func (e *externalMetadata) ArtistImage(ctx context.Context, id string) (*url.URL, error) {
func (e *provider) ArtistImage(ctx context.Context, id string) (*url.URL, error) {
artist, err := e.getArtist(ctx, id)
if err != nil {
return nil, err
@@ -318,24 +330,35 @@ func (e *externalMetadata) ArtistImage(ctx context.Context, id string) (*url.URL
imageUrl := artist.ArtistImageUrl()
if imageUrl == "" {
return nil, agents.ErrNotFound
return nil, model.ErrNotFound
}
return url.Parse(imageUrl)
}
func (e *externalMetadata) AlbumImage(ctx context.Context, id string) (*url.URL, error) {
func (e *provider) AlbumImage(ctx context.Context, id string) (*url.URL, error) {
album, err := e.getAlbum(ctx, id)
if err != nil {
return nil, err
}
info, err := e.ag.GetAlbumInfo(ctx, album.Name, album.AlbumArtist, album.MbzAlbumID)
if errors.Is(err, agents.ErrNotFound) {
if err != nil {
switch {
case errors.Is(err, agents.ErrNotFound):
log.Trace(ctx, "Album not found in agent", "albumID", id, "name", album.Name, "artist", album.AlbumArtist)
return nil, model.ErrNotFound
case errors.Is(err, context.Canceled):
log.Debug(ctx, "GetAlbumInfo call canceled", err)
default:
log.Warn(ctx, "Error getting album info from agent", "albumID", id, "name", album.Name, "artist", album.AlbumArtist, err)
}
return nil, err
}
if utils.IsCtxDone(ctx) {
log.Warn(ctx, "AlbumImage call canceled", ctx.Err())
return nil, ctx.Err()
if info == nil {
log.Warn(ctx, "Agent returned nil info without error", "albumID", id, "name", album.Name, "artist", album.AlbumArtist)
return nil, model.ErrNotFound
}
// Return the biggest image
@@ -346,26 +369,37 @@ func (e *externalMetadata) AlbumImage(ctx context.Context, id string) (*url.URL,
}
}
if img.URL == "" {
return nil, agents.ErrNotFound
return nil, model.ErrNotFound
}
return url.Parse(img.URL)
}
func (e *externalMetadata) TopSongs(ctx context.Context, artistName string, count int) (model.MediaFiles, error) {
func (e *provider) TopSongs(ctx context.Context, artistName string, count int) (model.MediaFiles, error) {
artist, err := e.findArtistByName(ctx, artistName)
if err != nil {
log.Error(ctx, "Artist not found", "name", artistName, err)
return nil, nil
}
return e.getMatchingTopSongs(ctx, e.ag, artist, count)
songs, err := e.getMatchingTopSongs(ctx, e.ag, artist, count)
if err != nil {
switch {
case errors.Is(err, agents.ErrNotFound):
log.Trace(ctx, "TopSongs not found", "name", artistName)
return nil, model.ErrNotFound
case errors.Is(err, context.Canceled):
log.Debug(ctx, "TopSongs call canceled", err)
default:
log.Warn(ctx, "Error getting top songs from agent", "artist", artistName, err)
}
return nil, err
}
return songs, nil
}
func (e *externalMetadata) getMatchingTopSongs(ctx context.Context, agent agents.ArtistTopSongsRetriever, artist *auxArtist, count int) (model.MediaFiles, error) {
func (e *provider) getMatchingTopSongs(ctx context.Context, agent agents.ArtistTopSongsRetriever, artist *auxArtist, count int) (model.MediaFiles, error) {
songs, err := agent.GetArtistTopSongs(ctx, artist.ID, artist.Name, artist.MbzArtistID, count)
if errors.Is(err, agents.ErrNotFound) {
return nil, nil
}
if err != nil {
return nil, err
}
@@ -386,13 +420,17 @@ func (e *externalMetadata) getMatchingTopSongs(ctx context.Context, agent agents
} else {
log.Debug(ctx, "Found matching top songs", "name", artist.Name, "numSongs", len(mfs))
}
return mfs, nil
}
func (e *externalMetadata) findMatchingTrack(ctx context.Context, mbid string, artistID, title string) (*model.MediaFile, error) {
func (e *provider) findMatchingTrack(ctx context.Context, mbid string, artistID, title string) (*model.MediaFile, error) {
if mbid != "" {
mfs, err := e.ds.MediaFile(ctx).GetAll(model.QueryOptions{
Filters: squirrel.Eq{"mbz_recording_id": mbid},
Filters: squirrel.And{
squirrel.Eq{"mbz_recording_id": mbid},
squirrel.Eq{"missing": false},
},
})
if err == nil && len(mfs) > 0 {
return &mfs[0], nil
@@ -406,6 +444,7 @@ func (e *externalMetadata) findMatchingTrack(ctx context.Context, mbid string, a
squirrel.Eq{"album_artist_id": artistID},
},
squirrel.Like{"order_title": str.SanitizeFieldForSorting(title)},
squirrel.Eq{"missing": false},
},
Sort: "starred desc, rating desc, year asc, compilation asc ",
Max: 1,
@@ -416,7 +455,7 @@ func (e *externalMetadata) findMatchingTrack(ctx context.Context, mbid string, a
return &mfs[0], nil
}
func (e *externalMetadata) callGetURL(ctx context.Context, agent agents.ArtistURLRetriever, artist *auxArtist) {
func (e *provider) callGetURL(ctx context.Context, agent agents.ArtistURLRetriever, artist *auxArtist) {
artisURL, err := agent.GetArtistURL(ctx, artist.ID, artist.Name, artist.MbzArtistID)
if err != nil {
return
@@ -424,7 +463,7 @@ func (e *externalMetadata) callGetURL(ctx context.Context, agent agents.ArtistUR
artist.ExternalUrl = artisURL
}
func (e *externalMetadata) callGetBiography(ctx context.Context, agent agents.ArtistBiographyRetriever, artist *auxArtist) {
func (e *provider) callGetBiography(ctx context.Context, agent agents.ArtistBiographyRetriever, artist *auxArtist) {
bio, err := agent.GetArtistBiography(ctx, artist.ID, str.Clear(artist.Name), artist.MbzArtistID)
if err != nil {
return
@@ -434,7 +473,7 @@ func (e *externalMetadata) callGetBiography(ctx context.Context, agent agents.Ar
artist.Biography = strings.ReplaceAll(bio, "<a ", "<a target='_blank' ")
}
func (e *externalMetadata) callGetImage(ctx context.Context, agent agents.ArtistImageRetriever, artist *auxArtist) {
func (e *provider) callGetImage(ctx context.Context, agent agents.ArtistImageRetriever, artist *auxArtist) {
images, err := agent.GetArtistImages(ctx, artist.ID, artist.Name, artist.MbzArtistID)
if err != nil {
return
@@ -452,7 +491,7 @@ func (e *externalMetadata) callGetImage(ctx context.Context, agent agents.Artist
}
}
func (e *externalMetadata) callGetSimilar(ctx context.Context, agent agents.ArtistSimilarRetriever, artist *auxArtist,
func (e *provider) callGetSimilar(ctx context.Context, agent agents.ArtistSimilarRetriever, artist *auxArtist,
limit int, includeNotPresent bool) {
similar, err := agent.GetSimilarArtists(ctx, artist.ID, artist.Name, artist.MbzArtistID, limit)
if len(similar) == 0 || err != nil {
@@ -467,24 +506,43 @@ func (e *externalMetadata) callGetSimilar(ctx context.Context, agent agents.Arti
artist.SimilarArtists = sa
}
func (e *externalMetadata) mapSimilarArtists(ctx context.Context, similar []agents.Artist, includeNotPresent bool) (model.Artists, error) {
func (e *provider) mapSimilarArtists(ctx context.Context, similar []agents.Artist, includeNotPresent bool) (model.Artists, error) {
var result model.Artists
var notPresent []string
// First select artists that are present.
artistNames := slice.Map(similar, func(artist agents.Artist) string { return artist.Name })
// Query all artists at once
clauses := slice.Map(artistNames, func(name string) squirrel.Sqlizer {
return squirrel.Like{"artist.name": name}
})
artists, err := e.ds.Artist(ctx).GetAll(model.QueryOptions{
Filters: squirrel.Or(clauses),
})
if err != nil {
return nil, err
}
// Create a map for quick lookup
artistMap := make(map[string]model.Artist)
for _, artist := range artists {
artistMap[artist.Name] = artist
}
// Process the similar artists
for _, s := range similar {
sa, err := e.findArtistByName(ctx, s.Name)
if err != nil {
if artist, found := artistMap[s.Name]; found {
result = append(result, artist)
} else {
notPresent = append(notPresent, s.Name)
continue
}
result = append(result, sa.Artist)
}
// Then fill up with non-present artists
if includeNotPresent {
for _, s := range notPresent {
sa := model.Artist{ID: unavailableArtistID, Name: s}
// Let the ID empty to indicate that the artist is not present in the DB
sa := model.Artist{Name: s}
result = append(result, sa)
}
}
@@ -492,7 +550,7 @@ func (e *externalMetadata) mapSimilarArtists(ctx context.Context, similar []agen
return result, nil
}
func (e *externalMetadata) findArtistByName(ctx context.Context, artistName string) (*auxArtist, error) {
func (e *provider) findArtistByName(ctx context.Context, artistName string) (*auxArtist, error) {
artists, err := e.ds.Artist(ctx).GetAll(model.QueryOptions{
Filters: squirrel.Like{"artist.name": artistName},
Max: 1,
@@ -510,10 +568,10 @@ func (e *externalMetadata) findArtistByName(ctx context.Context, artistName stri
return artist, nil
}
func (e *externalMetadata) loadSimilar(ctx context.Context, artist *auxArtist, count int, includeNotPresent bool) error {
func (e *provider) loadSimilar(ctx context.Context, artist *auxArtist, count int, includeNotPresent bool) error {
var ids []string
for _, sa := range artist.SimilarArtists {
if sa.ID == unavailableArtistID {
if sa.ID == "" {
continue
}
ids = append(ids, sa.ID)
@@ -544,7 +602,7 @@ func (e *externalMetadata) loadSimilar(ctx context.Context, artist *auxArtist, c
continue
}
la = sa
la.ID = unavailableArtistID
la.ID = ""
}
loaded = append(loaded, la)
}

View File

@@ -0,0 +1,303 @@
package external_test
import (
"context"
"errors"
"net/url"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/core/agents"
. "github.com/navidrome/navidrome/core/external"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/stretchr/testify/mock"
)
var _ = Describe("Provider - AlbumImage", func() {
var ds *tests.MockDataStore
var provider Provider
var mockArtistRepo *mockArtistRepo
var mockAlbumRepo *mockAlbumRepo
var mockMediaFileRepo *mockMediaFileRepo
var mockAlbumAgent *mockAlbumInfoAgent
var agentsCombined *mockAgents
var ctx context.Context
BeforeEach(func() {
ctx = GinkgoT().Context()
DeferCleanup(configtest.SetupConfig())
conf.Server.Agents = "mockAlbum" // Configure mock agent
mockArtistRepo = newMockArtistRepo()
mockAlbumRepo = newMockAlbumRepo()
mockMediaFileRepo = newMockMediaFileRepo()
ds = &tests.MockDataStore{
MockedArtist: mockArtistRepo,
MockedAlbum: mockAlbumRepo,
MockedMediaFile: mockMediaFileRepo,
}
mockAlbumAgent = newMockAlbumInfoAgent()
agentsCombined = &mockAgents{
albumInfoAgent: mockAlbumAgent,
}
provider = NewProvider(ds, agentsCombined)
// Default mocks
// Mocks for GetEntityByID sequence (initial failed lookups)
mockArtistRepo.On("Get", "album-1").Return(nil, model.ErrNotFound).Once()
mockArtistRepo.On("Get", "mf-1").Return(nil, model.ErrNotFound).Once()
mockAlbumRepo.On("Get", "mf-1").Return(nil, model.ErrNotFound).Once()
// Default mock for non-existent entities - Use Maybe() for flexibility
mockArtistRepo.On("Get", "not-found").Return(nil, model.ErrNotFound).Maybe()
mockAlbumRepo.On("Get", "not-found").Return(nil, model.ErrNotFound).Maybe()
mockMediaFileRepo.On("Get", "not-found").Return(nil, model.ErrNotFound).Maybe()
})
It("returns the largest image URL when successful", func() {
// Arrange
mockArtistRepo.On("Get", "album-1").Return(nil, model.ErrNotFound).Once() // Expect GetEntityByID sequence
mockAlbumRepo.On("Get", "album-1").Return(&model.Album{ID: "album-1", Name: "Album One", AlbumArtistID: "artist-1"}, nil).Once()
// Explicitly mock agent call for this test
mockAlbumAgent.On("GetAlbumInfo", ctx, "Album One", "", "").
Return(&agents.AlbumInfo{
Images: []agents.ExternalImage{
{URL: "http://example.com/large.jpg", Size: 1000},
{URL: "http://example.com/medium.jpg", Size: 500},
{URL: "http://example.com/small.jpg", Size: 200},
},
}, nil).Once()
expectedURL, _ := url.Parse("http://example.com/large.jpg")
imgURL, err := provider.AlbumImage(ctx, "album-1")
Expect(err).ToNot(HaveOccurred())
Expect(imgURL).To(Equal(expectedURL))
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "album-1") // From GetEntityByID
mockAlbumRepo.AssertCalled(GinkgoT(), "Get", "album-1")
mockArtistRepo.AssertNotCalled(GinkgoT(), "Get", "artist-1") // Artist lookup no longer happens in getAlbum
mockAlbumAgent.AssertCalled(GinkgoT(), "GetAlbumInfo", ctx, "Album One", "", "") // Expect empty artist name
})
It("returns ErrNotFound if the album is not found in the DB", func() {
// Arrange: Explicitly expect the full GetEntityByID sequence for "not-found"
mockArtistRepo.On("Get", "not-found").Return(nil, model.ErrNotFound).Once()
mockAlbumRepo.On("Get", "not-found").Return(nil, model.ErrNotFound).Once()
mockMediaFileRepo.On("Get", "not-found").Return(nil, model.ErrNotFound).Once()
imgURL, err := provider.AlbumImage(ctx, "not-found")
Expect(err).To(MatchError("data not found"))
Expect(imgURL).To(BeNil())
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "not-found")
mockAlbumRepo.AssertCalled(GinkgoT(), "Get", "not-found")
mockMediaFileRepo.AssertCalled(GinkgoT(), "Get", "not-found")
mockAlbumAgent.AssertNotCalled(GinkgoT(), "GetAlbumInfo", mock.Anything, mock.Anything, mock.Anything, mock.Anything)
})
It("returns the agent error if the agent fails", func() {
// Arrange
mockArtistRepo.On("Get", "album-1").Return(nil, model.ErrNotFound).Once() // Expect GetEntityByID sequence
mockAlbumRepo.On("Get", "album-1").Return(&model.Album{ID: "album-1", Name: "Album One", AlbumArtistID: "artist-1"}, nil).Once()
agentErr := errors.New("agent failure")
// Explicitly mock agent call for this test
mockAlbumAgent.On("GetAlbumInfo", ctx, "Album One", "", "").Return(nil, agentErr).Once() // Expect empty artist
imgURL, err := provider.AlbumImage(ctx, "album-1")
Expect(err).To(MatchError("agent failure"))
Expect(imgURL).To(BeNil())
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "album-1")
mockAlbumRepo.AssertCalled(GinkgoT(), "Get", "album-1")
mockArtistRepo.AssertNotCalled(GinkgoT(), "Get", "artist-1")
mockAlbumAgent.AssertCalled(GinkgoT(), "GetAlbumInfo", ctx, "Album One", "", "") // Expect empty artist
})
It("returns ErrNotFound if the agent returns ErrNotFound", func() {
// Arrange
mockArtistRepo.On("Get", "album-1").Return(nil, model.ErrNotFound).Once() // Expect GetEntityByID sequence
mockAlbumRepo.On("Get", "album-1").Return(&model.Album{ID: "album-1", Name: "Album One", AlbumArtistID: "artist-1"}, nil).Once()
// Explicitly mock agent call for this test
mockAlbumAgent.On("GetAlbumInfo", ctx, "Album One", "", "").Return(nil, agents.ErrNotFound).Once() // Expect empty artist
imgURL, err := provider.AlbumImage(ctx, "album-1")
Expect(err).To(MatchError("data not found"))
Expect(imgURL).To(BeNil())
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "album-1")
mockAlbumRepo.AssertCalled(GinkgoT(), "Get", "album-1")
mockAlbumAgent.AssertCalled(GinkgoT(), "GetAlbumInfo", ctx, "Album One", "", "") // Expect empty artist
})
It("returns ErrNotFound if the agent returns no images", func() {
// Arrange
mockArtistRepo.On("Get", "album-1").Return(nil, model.ErrNotFound).Once() // Expect GetEntityByID sequence
mockAlbumRepo.On("Get", "album-1").Return(&model.Album{ID: "album-1", Name: "Album One", AlbumArtistID: "artist-1"}, nil).Once()
// Explicitly mock agent call for this test
mockAlbumAgent.On("GetAlbumInfo", ctx, "Album One", "", "").
Return(&agents.AlbumInfo{Images: []agents.ExternalImage{}}, nil).Once() // Expect empty artist
imgURL, err := provider.AlbumImage(ctx, "album-1")
Expect(err).To(MatchError("data not found"))
Expect(imgURL).To(BeNil())
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "album-1")
mockAlbumRepo.AssertCalled(GinkgoT(), "Get", "album-1")
mockAlbumAgent.AssertCalled(GinkgoT(), "GetAlbumInfo", ctx, "Album One", "", "") // Expect empty artist
})
It("returns context error if context is canceled", func() {
// Arrange
cctx, cancelCtx := context.WithCancel(ctx)
// Mock the necessary DB calls *before* canceling the context
mockArtistRepo.On("Get", "album-1").Return(nil, model.ErrNotFound).Once()
mockAlbumRepo.On("Get", "album-1").Return(&model.Album{ID: "album-1", Name: "Album One", AlbumArtistID: "artist-1"}, nil).Once()
// Expect the agent call even if context is cancelled, returning the context error
mockAlbumAgent.On("GetAlbumInfo", cctx, "Album One", "", "").Return(nil, context.Canceled).Once()
// Cancel the context *before* calling the function under test
cancelCtx()
imgURL, err := provider.AlbumImage(cctx, "album-1")
Expect(err).To(MatchError("context canceled"))
Expect(imgURL).To(BeNil())
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "album-1")
mockAlbumRepo.AssertCalled(GinkgoT(), "Get", "album-1")
// Agent should now be called, verify this expectation
mockAlbumAgent.AssertCalled(GinkgoT(), "GetAlbumInfo", cctx, "Album One", "", "")
})
It("derives album ID from MediaFile ID", func() {
// Arrange: Mock full GetEntityByID for "mf-1" and recursive "album-1"
mockArtistRepo.On("Get", "mf-1").Return(nil, model.ErrNotFound).Once()
mockAlbumRepo.On("Get", "mf-1").Return(nil, model.ErrNotFound).Once()
mockMediaFileRepo.On("Get", "mf-1").Return(&model.MediaFile{ID: "mf-1", Title: "Track One", ArtistID: "artist-1", AlbumID: "album-1"}, nil).Once()
mockArtistRepo.On("Get", "album-1").Return(nil, model.ErrNotFound).Once()
mockAlbumRepo.On("Get", "album-1").Return(&model.Album{ID: "album-1", Name: "Album One", AlbumArtistID: "artist-1"}, nil).Once()
// Explicitly mock agent call for this test
mockAlbumAgent.On("GetAlbumInfo", ctx, "Album One", "", "").
Return(&agents.AlbumInfo{
Images: []agents.ExternalImage{
{URL: "http://example.com/large.jpg", Size: 1000},
{URL: "http://example.com/medium.jpg", Size: 500},
{URL: "http://example.com/small.jpg", Size: 200},
},
}, nil).Once()
expectedURL, _ := url.Parse("http://example.com/large.jpg")
imgURL, err := provider.AlbumImage(ctx, "mf-1")
Expect(err).ToNot(HaveOccurred())
Expect(imgURL).To(Equal(expectedURL))
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "mf-1")
mockAlbumRepo.AssertCalled(GinkgoT(), "Get", "mf-1")
mockMediaFileRepo.AssertCalled(GinkgoT(), "Get", "mf-1")
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "album-1")
mockAlbumRepo.AssertCalled(GinkgoT(), "Get", "album-1")
mockArtistRepo.AssertNotCalled(GinkgoT(), "Get", "artist-1")
mockAlbumAgent.AssertCalled(GinkgoT(), "GetAlbumInfo", ctx, "Album One", "", "")
})
It("handles different image orders from agent", func() {
// Arrange
mockArtistRepo.On("Get", "album-1").Return(nil, model.ErrNotFound).Once() // Expect GetEntityByID sequence
mockAlbumRepo.On("Get", "album-1").Return(&model.Album{ID: "album-1", Name: "Album One", AlbumArtistID: "artist-1"}, nil).Once()
// Explicitly mock agent call for this test
mockAlbumAgent.On("GetAlbumInfo", ctx, "Album One", "", "").
Return(&agents.AlbumInfo{
Images: []agents.ExternalImage{
{URL: "http://example.com/small.jpg", Size: 200},
{URL: "http://example.com/large.jpg", Size: 1000},
{URL: "http://example.com/medium.jpg", Size: 500},
},
}, nil).Once()
expectedURL, _ := url.Parse("http://example.com/large.jpg")
imgURL, err := provider.AlbumImage(ctx, "album-1")
Expect(err).ToNot(HaveOccurred())
Expect(imgURL).To(Equal(expectedURL)) // Should still pick the largest
mockAlbumAgent.AssertCalled(GinkgoT(), "GetAlbumInfo", ctx, "Album One", "", "")
})
It("handles agent returning only one image", func() {
// Arrange
mockArtistRepo.On("Get", "album-1").Return(nil, model.ErrNotFound).Once() // Expect GetEntityByID sequence
mockAlbumRepo.On("Get", "album-1").Return(&model.Album{ID: "album-1", Name: "Album One", AlbumArtistID: "artist-1"}, nil).Once()
// Explicitly mock agent call for this test
mockAlbumAgent.On("GetAlbumInfo", ctx, "Album One", "", "").
Return(&agents.AlbumInfo{
Images: []agents.ExternalImage{
{URL: "http://example.com/single.jpg", Size: 700},
},
}, nil).Once()
expectedURL, _ := url.Parse("http://example.com/single.jpg")
imgURL, err := provider.AlbumImage(ctx, "album-1")
Expect(err).ToNot(HaveOccurred())
Expect(imgURL).To(Equal(expectedURL))
mockAlbumAgent.AssertCalled(GinkgoT(), "GetAlbumInfo", ctx, "Album One", "", "")
})
It("returns ErrNotFound if deriving album ID fails", func() {
// Arrange: Mock full GetEntityByID for "mf-no-album" and recursive "not-found"
mockArtistRepo.On("Get", "mf-no-album").Return(nil, model.ErrNotFound).Once()
mockAlbumRepo.On("Get", "mf-no-album").Return(nil, model.ErrNotFound).Once()
mockMediaFileRepo.On("Get", "mf-no-album").Return(&model.MediaFile{ID: "mf-no-album", Title: "Track No Album", ArtistID: "artist-1", AlbumID: "not-found"}, nil).Once()
mockArtistRepo.On("Get", "not-found").Return(nil, model.ErrNotFound).Once()
mockAlbumRepo.On("Get", "not-found").Return(nil, model.ErrNotFound).Once()
mockMediaFileRepo.On("Get", "not-found").Return(nil, model.ErrNotFound).Once()
imgURL, err := provider.AlbumImage(ctx, "mf-no-album")
Expect(err).To(MatchError("data not found"))
Expect(imgURL).To(BeNil())
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "mf-no-album")
mockAlbumRepo.AssertCalled(GinkgoT(), "Get", "mf-no-album")
mockMediaFileRepo.AssertCalled(GinkgoT(), "Get", "mf-no-album")
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "not-found")
mockAlbumRepo.AssertCalled(GinkgoT(), "Get", "not-found")
mockMediaFileRepo.AssertCalled(GinkgoT(), "Get", "not-found")
mockAlbumAgent.AssertNotCalled(GinkgoT(), "GetAlbumInfo", mock.Anything, mock.Anything, mock.Anything, mock.Anything)
})
})
// mockAlbumInfoAgent implementation
type mockAlbumInfoAgent struct {
mock.Mock
agents.AlbumInfoRetriever // Embed interface
}
func newMockAlbumInfoAgent() *mockAlbumInfoAgent {
m := new(mockAlbumInfoAgent)
m.On("AgentName").Return("mockAlbum").Maybe()
return m
}
func (m *mockAlbumInfoAgent) AgentName() string {
args := m.Called()
return args.String(0)
}
func (m *mockAlbumInfoAgent) GetAlbumInfo(ctx context.Context, name, artist, mbid string) (*agents.AlbumInfo, error) {
args := m.Called(ctx, name, artist, mbid)
if args.Get(0) == nil {
return nil, args.Error(1)
}
return args.Get(0).(*agents.AlbumInfo), args.Error(1)
}
// Ensure mockAgent implements the interface
var _ agents.AlbumInfoRetriever = (*mockAlbumInfoAgent)(nil)

View File

@@ -0,0 +1,301 @@
package external_test
import (
"context"
"errors"
"net/url"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/core/agents"
. "github.com/navidrome/navidrome/core/external"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/stretchr/testify/mock"
)
var _ = Describe("Provider - ArtistImage", func() {
var ds *tests.MockDataStore
var provider Provider
var mockArtistRepo *mockArtistRepo
var mockAlbumRepo *mockAlbumRepo
var mockMediaFileRepo *mockMediaFileRepo
var mockImageAgent *mockArtistImageAgent
var agentsCombined *mockAgents
var ctx context.Context
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Agents = "mockImage" // Configure only the mock agent
ctx = GinkgoT().Context()
mockArtistRepo = newMockArtistRepo()
mockAlbumRepo = newMockAlbumRepo()
mockMediaFileRepo = newMockMediaFileRepo()
ds = &tests.MockDataStore{
MockedArtist: mockArtistRepo,
MockedAlbum: mockAlbumRepo,
MockedMediaFile: mockMediaFileRepo,
}
mockImageAgent = newMockArtistImageAgent()
// Use the mockAgents from helper, setting the specific agent
agentsCombined = &mockAgents{
imageAgent: mockImageAgent,
}
provider = NewProvider(ds, agentsCombined)
// Default mocks for successful Get calls
mockArtistRepo.On("Get", "artist-1").Return(&model.Artist{ID: "artist-1", Name: "Artist One"}, nil).Maybe()
mockAlbumRepo.On("Get", "album-1").Return(&model.Album{ID: "album-1", Name: "Album One", AlbumArtistID: "artist-1"}, nil).Maybe()
mockMediaFileRepo.On("Get", "mf-1").Return(&model.MediaFile{ID: "mf-1", Title: "Track One", ArtistID: "artist-1"}, nil).Maybe()
// Default mock for non-existent entities
mockArtistRepo.On("Get", "not-found").Return(nil, model.ErrNotFound).Maybe()
mockAlbumRepo.On("Get", "not-found").Return(nil, model.ErrNotFound).Maybe()
mockMediaFileRepo.On("Get", "not-found").Return(nil, model.ErrNotFound).Maybe()
// Default successful image agent response
mockImageAgent.On("GetArtistImages", mock.Anything, "artist-1", "Artist One", "").
Return([]agents.ExternalImage{
{URL: "http://example.com/large.jpg", Size: 1000},
{URL: "http://example.com/medium.jpg", Size: 500},
{URL: "http://example.com/small.jpg", Size: 200},
}, nil).Maybe()
})
AfterEach(func() {
mockArtistRepo.AssertExpectations(GinkgoT())
mockAlbumRepo.AssertExpectations(GinkgoT())
mockMediaFileRepo.AssertExpectations(GinkgoT())
mockImageAgent.AssertExpectations(GinkgoT())
})
It("returns the largest image URL when successful", func() {
// Arrange
expectedURL, _ := url.Parse("http://example.com/large.jpg")
// Act
imgURL, err := provider.ArtistImage(ctx, "artist-1")
// Assert
Expect(err).ToNot(HaveOccurred())
Expect(imgURL).To(Equal(expectedURL))
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "artist-1")
mockImageAgent.AssertCalled(GinkgoT(), "GetArtistImages", ctx, "artist-1", "Artist One", "")
})
It("returns ErrNotFound if the artist is not found in the DB", func() {
// Arrange
// Act
imgURL, err := provider.ArtistImage(ctx, "not-found")
// Assert
Expect(err).To(MatchError(model.ErrNotFound))
Expect(imgURL).To(BeNil())
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "not-found")
mockImageAgent.AssertNotCalled(GinkgoT(), "GetArtistImages", mock.Anything, mock.Anything, mock.Anything, mock.Anything)
})
It("returns the agent error if the agent fails", func() {
// Arrange
agentErr := errors.New("agent failure")
mockImageAgent.Mock = mock.Mock{} // Reset default expectation
mockImageAgent.On("GetArtistImages", ctx, "artist-1", "Artist One", "").Return(nil, agentErr).Once()
// Act
imgURL, err := provider.ArtistImage(ctx, "artist-1")
// Assert
Expect(err).To(MatchError(model.ErrNotFound)) // Corrected Expectation: The provider maps agent errors (other than canceled) to ErrNotFound if no image was found/populated
Expect(imgURL).To(BeNil())
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "artist-1")
mockImageAgent.AssertCalled(GinkgoT(), "GetArtistImages", ctx, "artist-1", "Artist One", "")
})
It("returns ErrNotFound if the agent returns ErrNotFound", func() {
// Arrange
mockImageAgent.Mock = mock.Mock{} // Reset default expectation
mockImageAgent.On("GetArtistImages", ctx, "artist-1", "Artist One", "").Return(nil, agents.ErrNotFound).Once()
// Act
imgURL, err := provider.ArtistImage(ctx, "artist-1")
// Assert
Expect(err).To(MatchError(model.ErrNotFound))
Expect(imgURL).To(BeNil())
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "artist-1")
mockImageAgent.AssertCalled(GinkgoT(), "GetArtistImages", ctx, "artist-1", "Artist One", "")
})
It("returns ErrNotFound if the agent returns no images", func() {
// Arrange
mockImageAgent.Mock = mock.Mock{} // Reset default expectation
mockImageAgent.On("GetArtistImages", ctx, "artist-1", "Artist One", "").Return([]agents.ExternalImage{}, nil).Once()
// Act
imgURL, err := provider.ArtistImage(ctx, "artist-1")
// Assert
Expect(err).To(MatchError(model.ErrNotFound)) // Implementation maps empty result to ErrNotFound
Expect(imgURL).To(BeNil())
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "artist-1")
mockImageAgent.AssertCalled(GinkgoT(), "GetArtistImages", ctx, "artist-1", "Artist One", "")
})
It("returns context error if context is canceled before agent call", func() {
// Arrange
cctx, cancelCtx := context.WithCancel(context.Background())
mockArtistRepo.Mock = mock.Mock{} // Reset default expectation for artist repo as well
mockArtistRepo.On("Get", "artist-1").Return(&model.Artist{ID: "artist-1", Name: "Artist One"}, nil).Run(func(args mock.Arguments) {
cancelCtx() // Cancel context *during* the DB call simulation
}).Once()
// Act
imgURL, err := provider.ArtistImage(cctx, "artist-1")
// Assert
Expect(err).To(MatchError(context.Canceled))
Expect(imgURL).To(BeNil())
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "artist-1")
})
It("derives artist ID from MediaFile ID", func() {
// Arrange: Add mocks for the initial GetEntityByID lookups
mockArtistRepo.On("Get", "mf-1").Return(nil, model.ErrNotFound).Once()
mockAlbumRepo.On("Get", "mf-1").Return(nil, model.ErrNotFound).Once()
// Default mocks for MediaFileRepo.Get("mf-1") and ArtistRepo.Get("artist-1") handle the rest
expectedURL, _ := url.Parse("http://example.com/large.jpg")
// Act
imgURL, err := provider.ArtistImage(ctx, "mf-1")
// Assert
Expect(err).ToNot(HaveOccurred())
Expect(imgURL).To(Equal(expectedURL))
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "mf-1") // GetEntityByID sequence
mockAlbumRepo.AssertCalled(GinkgoT(), "Get", "mf-1") // GetEntityByID sequence
mockMediaFileRepo.AssertCalled(GinkgoT(), "Get", "mf-1")
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "artist-1") // Should be called after getting MF
mockImageAgent.AssertCalled(GinkgoT(), "GetArtistImages", ctx, "artist-1", "Artist One", "")
})
It("derives artist ID from Album ID", func() {
// Arrange: Add mock for the initial GetEntityByID lookup
mockArtistRepo.On("Get", "album-1").Return(nil, model.ErrNotFound).Once()
// Default mocks for AlbumRepo.Get("album-1") and ArtistRepo.Get("artist-1") handle the rest
expectedURL, _ := url.Parse("http://example.com/large.jpg")
// Act
imgURL, err := provider.ArtistImage(ctx, "album-1")
// Assert
Expect(err).ToNot(HaveOccurred())
Expect(imgURL).To(Equal(expectedURL))
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "album-1") // GetEntityByID sequence
mockAlbumRepo.AssertCalled(GinkgoT(), "Get", "album-1")
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "artist-1") // Should be called after getting Album
mockImageAgent.AssertCalled(GinkgoT(), "GetArtistImages", ctx, "artist-1", "Artist One", "")
})
It("returns ErrNotFound if derived artist is not found", func() {
// Arrange
// Add mocks for the initial GetEntityByID lookups
mockArtistRepo.On("Get", "mf-bad-artist").Return(nil, model.ErrNotFound).Once()
mockAlbumRepo.On("Get", "mf-bad-artist").Return(nil, model.ErrNotFound).Once()
mockMediaFileRepo.On("Get", "mf-bad-artist").Return(&model.MediaFile{ID: "mf-bad-artist", ArtistID: "not-found"}, nil).Once()
// Add expectation for the recursive GetEntityByID call for the MediaFileRepo
mockMediaFileRepo.On("Get", "not-found").Return(nil, model.ErrNotFound).Maybe()
// The default mocks for ArtistRepo/AlbumRepo handle the final "not-found" lookups
// Act
imgURL, err := provider.ArtistImage(ctx, "mf-bad-artist")
// Assert
Expect(err).To(MatchError(model.ErrNotFound))
Expect(imgURL).To(BeNil())
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "mf-bad-artist") // GetEntityByID sequence
mockAlbumRepo.AssertCalled(GinkgoT(), "Get", "mf-bad-artist") // GetEntityByID sequence
mockMediaFileRepo.AssertCalled(GinkgoT(), "Get", "mf-bad-artist")
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "not-found")
mockImageAgent.AssertNotCalled(GinkgoT(), "GetArtistImages", mock.Anything, mock.Anything, mock.Anything, mock.Anything)
})
It("handles different image orders from agent", func() {
// Arrange
mockImageAgent.Mock = mock.Mock{} // Reset default expectation
mockImageAgent.On("GetArtistImages", ctx, "artist-1", "Artist One", "").
Return([]agents.ExternalImage{
{URL: "http://example.com/small.jpg", Size: 200},
{URL: "http://example.com/large.jpg", Size: 1000},
{URL: "http://example.com/medium.jpg", Size: 500},
}, nil).Once()
expectedURL, _ := url.Parse("http://example.com/large.jpg")
// Act
imgURL, err := provider.ArtistImage(ctx, "artist-1")
// Assert
Expect(err).ToNot(HaveOccurred())
Expect(imgURL).To(Equal(expectedURL)) // Still picks the largest
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "artist-1")
mockImageAgent.AssertCalled(GinkgoT(), "GetArtistImages", ctx, "artist-1", "Artist One", "")
})
It("handles agent returning only one image", func() {
// Arrange
mockImageAgent.Mock = mock.Mock{} // Reset default expectation
mockImageAgent.On("GetArtistImages", ctx, "artist-1", "Artist One", "").
Return([]agents.ExternalImage{
{URL: "http://example.com/medium.jpg", Size: 500},
}, nil).Once()
expectedURL, _ := url.Parse("http://example.com/medium.jpg")
// Act
imgURL, err := provider.ArtistImage(ctx, "artist-1")
// Assert
Expect(err).ToNot(HaveOccurred())
Expect(imgURL).To(Equal(expectedURL))
mockArtistRepo.AssertCalled(GinkgoT(), "Get", "artist-1")
mockImageAgent.AssertCalled(GinkgoT(), "GetArtistImages", ctx, "artist-1", "Artist One", "")
})
})
// mockArtistImageAgent implementation using testify/mock
// This remains local as it's specific to testing the ArtistImage functionality
type mockArtistImageAgent struct {
mock.Mock
agents.ArtistImageRetriever // Embed interface
}
// Constructor for the mock agent
func newMockArtistImageAgent() *mockArtistImageAgent {
mock := new(mockArtistImageAgent)
// Set default AgentName if needed, although usually called via mockAgents
mock.On("AgentName").Return("mockImage").Maybe()
return mock
}
func (m *mockArtistImageAgent) AgentName() string {
args := m.Called()
return args.String(0)
}
func (m *mockArtistImageAgent) GetArtistImages(ctx context.Context, id, artistName, mbid string) ([]agents.ExternalImage, error) {
args := m.Called(ctx, id, artistName, mbid)
// Need careful type assertion for potentially nil slice
var res []agents.ExternalImage
if args.Get(0) != nil {
res = args.Get(0).([]agents.ExternalImage)
}
return res, args.Error(1)
}
// Ensure mockAgent implements the interface
var _ agents.ArtistImageRetriever = (*mockArtistImageAgent)(nil)

View File

@@ -0,0 +1,198 @@
package external_test
import (
"context"
"errors"
"github.com/navidrome/navidrome/core/agents"
. "github.com/navidrome/navidrome/core/external"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/stretchr/testify/mock"
)
var _ = Describe("Provider - SimilarSongs", func() {
var ds model.DataStore
var provider Provider
var mockAgent *mockSimilarArtistAgent
var mockTopAgent agents.ArtistTopSongsRetriever
var mockSimilarAgent agents.ArtistSimilarRetriever
var agentsCombined Agents
var artistRepo *mockArtistRepo
var mediaFileRepo *mockMediaFileRepo
var ctx context.Context
BeforeEach(func() {
ctx = GinkgoT().Context()
artistRepo = newMockArtistRepo()
mediaFileRepo = newMockMediaFileRepo()
ds = &tests.MockDataStore{
MockedArtist: artistRepo,
MockedMediaFile: mediaFileRepo,
}
mockAgent = &mockSimilarArtistAgent{}
mockTopAgent = mockAgent
mockSimilarAgent = mockAgent
agentsCombined = &mockAgents{
topSongsAgent: mockTopAgent,
similarAgent: mockSimilarAgent,
}
provider = NewProvider(ds, agentsCombined)
})
It("returns similar songs from main artist and similar artists", func() {
artist1 := model.Artist{ID: "artist-1", Name: "Artist One"}
similarArtist := model.Artist{ID: "artist-3", Name: "Similar Artist"}
song1 := model.MediaFile{ID: "song-1", Title: "Song One", ArtistID: "artist-1"}
song2 := model.MediaFile{ID: "song-2", Title: "Song Two", ArtistID: "artist-1"}
song3 := model.MediaFile{ID: "song-3", Title: "Song Three", ArtistID: "artist-3"}
artistRepo.On("Get", "artist-1").Return(&artist1, nil).Maybe()
artistRepo.On("Get", "artist-3").Return(&similarArtist, nil).Maybe()
artistRepo.On("GetAll", mock.MatchedBy(func(opt model.QueryOptions) bool {
return opt.Max == 1 && opt.Filters != nil
})).Return(model.Artists{artist1}, nil).Once()
similarAgentsResp := []agents.Artist{
{Name: "Similar Artist", MBID: "similar-mbid"},
}
mockAgent.On("GetSimilarArtists", mock.Anything, "artist-1", "Artist One", "", 15).
Return(similarAgentsResp, nil).Once()
artistRepo.On("GetAll", mock.MatchedBy(func(opt model.QueryOptions) bool {
return opt.Max == 0 && opt.Filters != nil
})).Return(model.Artists{similarArtist}, nil).Once()
mockAgent.On("GetArtistTopSongs", mock.Anything, "artist-1", "Artist One", "", mock.Anything).
Return([]agents.Song{
{Name: "Song One", MBID: "mbid-1"},
{Name: "Song Two", MBID: "mbid-2"},
}, nil).Once()
mockAgent.On("GetArtistTopSongs", mock.Anything, "artist-3", "Similar Artist", "", mock.Anything).
Return([]agents.Song{
{Name: "Song Three", MBID: "mbid-3"},
}, nil).Once()
mediaFileRepo.FindByMBID("mbid-1", song1)
mediaFileRepo.FindByMBID("mbid-2", song2)
mediaFileRepo.FindByMBID("mbid-3", song3)
songs, err := provider.SimilarSongs(ctx, "artist-1", 3)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(HaveLen(3))
for _, song := range songs {
Expect(song.ID).To(BeElementOf("song-1", "song-2", "song-3"))
}
})
It("returns ErrNotFound when artist is not found", func() {
artistRepo.On("Get", "artist-unknown-artist").Return(nil, model.ErrNotFound)
mediaFileRepo.On("Get", "artist-unknown-artist").Return(nil, model.ErrNotFound)
artistRepo.On("GetAll", mock.MatchedBy(func(opt model.QueryOptions) bool {
return opt.Max == 1 && opt.Filters != nil
})).Return(model.Artists{}, nil).Maybe()
songs, err := provider.SimilarSongs(ctx, "artist-unknown-artist", 5)
Expect(err).To(Equal(model.ErrNotFound))
Expect(songs).To(BeNil())
})
It("returns songs from main artist when GetSimilarArtists returns error", func() {
artist1 := model.Artist{ID: "artist-1", Name: "Artist One"}
song1 := model.MediaFile{ID: "song-1", Title: "Song One", ArtistID: "artist-1"}
artistRepo.On("Get", "artist-1").Return(&artist1, nil).Maybe()
artistRepo.On("GetAll", mock.MatchedBy(func(opt model.QueryOptions) bool {
return opt.Max == 1 && opt.Filters != nil
})).Return(model.Artists{artist1}, nil).Maybe()
mockAgent.On("GetSimilarArtists", mock.Anything, "artist-1", "Artist One", "", 15).
Return(nil, errors.New("error getting similar artists")).Once()
artistRepo.On("GetAll", mock.MatchedBy(func(opt model.QueryOptions) bool {
return opt.Max == 0 && opt.Filters != nil
})).Return(model.Artists{}, nil).Once()
mockAgent.On("GetArtistTopSongs", mock.Anything, "artist-1", "Artist One", "", mock.Anything).
Return([]agents.Song{
{Name: "Song One", MBID: "mbid-1"},
}, nil).Once()
mediaFileRepo.FindByMBID("mbid-1", song1)
songs, err := provider.SimilarSongs(ctx, "artist-1", 5)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(HaveLen(1))
Expect(songs[0].ID).To(Equal("song-1"))
})
It("returns empty list when GetArtistTopSongs returns error", func() {
artist1 := model.Artist{ID: "artist-1", Name: "Artist One"}
artistRepo.On("Get", "artist-1").Return(&artist1, nil).Maybe()
artistRepo.On("GetAll", mock.MatchedBy(func(opt model.QueryOptions) bool {
return opt.Max == 1 && opt.Filters != nil
})).Return(model.Artists{artist1}, nil).Maybe()
mockAgent.On("GetSimilarArtists", mock.Anything, "artist-1", "Artist One", "", 15).
Return([]agents.Artist{}, nil).Once()
artistRepo.On("GetAll", mock.MatchedBy(func(opt model.QueryOptions) bool {
return opt.Max == 0 && opt.Filters != nil
})).Return(model.Artists{}, nil).Once()
mockAgent.On("GetArtistTopSongs", mock.Anything, "artist-1", "Artist One", "", mock.Anything).
Return(nil, errors.New("error getting top songs")).Once()
songs, err := provider.SimilarSongs(ctx, "artist-1", 5)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(BeEmpty())
})
It("respects count parameter", func() {
artist1 := model.Artist{ID: "artist-1", Name: "Artist One"}
song1 := model.MediaFile{ID: "song-1", Title: "Song One", ArtistID: "artist-1"}
song2 := model.MediaFile{ID: "song-2", Title: "Song Two", ArtistID: "artist-1"}
artistRepo.On("Get", "artist-1").Return(&artist1, nil).Maybe()
artistRepo.On("GetAll", mock.MatchedBy(func(opt model.QueryOptions) bool {
return opt.Max == 1 && opt.Filters != nil
})).Return(model.Artists{artist1}, nil).Maybe()
mockAgent.On("GetSimilarArtists", mock.Anything, "artist-1", "Artist One", "", 15).
Return([]agents.Artist{}, nil).Once()
artistRepo.On("GetAll", mock.MatchedBy(func(opt model.QueryOptions) bool {
return opt.Max == 0 && opt.Filters != nil
})).Return(model.Artists{}, nil).Once()
mockAgent.On("GetArtistTopSongs", mock.Anything, "artist-1", "Artist One", "", mock.Anything).
Return([]agents.Song{
{Name: "Song One", MBID: "mbid-1"},
{Name: "Song Two", MBID: "mbid-2"},
}, nil).Once()
mediaFileRepo.FindByMBID("mbid-1", song1)
mediaFileRepo.FindByMBID("mbid-2", song2)
songs, err := provider.SimilarSongs(ctx, "artist-1", 1)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(HaveLen(1))
Expect(songs[0].ID).To(BeElementOf("song-1", "song-2"))
})
})

193
core/external/provider_topsongs_test.go vendored Normal file
View File

@@ -0,0 +1,193 @@
package external_test
import (
"context"
"errors"
"github.com/navidrome/navidrome/core/agents"
_ "github.com/navidrome/navidrome/core/agents/lastfm"
_ "github.com/navidrome/navidrome/core/agents/listenbrainz"
_ "github.com/navidrome/navidrome/core/agents/spotify"
. "github.com/navidrome/navidrome/core/external"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/stretchr/testify/mock"
)
var _ = Describe("Provider - TopSongs", func() {
var (
p Provider
artistRepo *mockArtistRepo // From provider_helper_test.go
mediaFileRepo *mockMediaFileRepo // From provider_helper_test.go
ag *mockAgents // Consolidated mock from export_test.go
ctx context.Context
)
BeforeEach(func() {
ctx = GinkgoT().Context()
artistRepo = newMockArtistRepo() // Use helper mock
mediaFileRepo = newMockMediaFileRepo() // Use helper mock
// Configure tests.MockDataStore to use the testify/mock-based repos
ds := &tests.MockDataStore{
MockedArtist: artistRepo,
MockedMediaFile: mediaFileRepo,
}
ag = new(mockAgents)
p = NewProvider(ds, ag)
})
BeforeEach(func() {
// Setup expectations in individual tests
})
It("returns top songs for a known artist", func() {
// Mock finding the artist
artist1 := model.Artist{ID: "artist-1", Name: "Artist One", MbzArtistID: "mbid-artist-1"}
artistRepo.On("GetAll", mock.AnythingOfType("model.QueryOptions")).Return(model.Artists{artist1}, nil).Once()
// Mock agent response
agentSongs := []agents.Song{
{Name: "Song One", MBID: "mbid-song-1"},
{Name: "Song Two", MBID: "mbid-song-2"},
}
ag.On("GetArtistTopSongs", ctx, "artist-1", "Artist One", "mbid-artist-1", 2).Return(agentSongs, nil).Once()
// Mock finding matching tracks
song1 := model.MediaFile{ID: "song-1", Title: "Song One", ArtistID: "artist-1", MbzRecordingID: "mbid-song-1"}
song2 := model.MediaFile{ID: "song-2", Title: "Song Two", ArtistID: "artist-1", MbzRecordingID: "mbid-song-2"}
mediaFileRepo.On("GetAll", mock.AnythingOfType("model.QueryOptions")).Return(model.MediaFiles{song1}, nil).Once()
mediaFileRepo.On("GetAll", mock.AnythingOfType("model.QueryOptions")).Return(model.MediaFiles{song2}, nil).Once()
songs, err := p.TopSongs(ctx, "Artist One", 2)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(HaveLen(2))
Expect(songs[0].ID).To(Equal("song-1"))
Expect(songs[1].ID).To(Equal("song-2"))
artistRepo.AssertExpectations(GinkgoT())
ag.AssertExpectations(GinkgoT())
mediaFileRepo.AssertExpectations(GinkgoT())
})
It("returns nil for an unknown artist", func() {
// Mock artist not found
artistRepo.On("GetAll", mock.AnythingOfType("model.QueryOptions")).Return(model.Artists{}, nil).Once()
songs, err := p.TopSongs(ctx, "Unknown Artist", 5)
Expect(err).ToNot(HaveOccurred()) // TopSongs returns nil error if artist not found
Expect(songs).To(BeNil())
artistRepo.AssertExpectations(GinkgoT())
ag.AssertNotCalled(GinkgoT(), "GetArtistTopSongs", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything)
})
It("returns error when the agent returns an error", func() {
// Mock finding the artist
artist1 := model.Artist{ID: "artist-1", Name: "Artist One", MbzArtistID: "mbid-artist-1"}
artistRepo.On("GetAll", mock.AnythingOfType("model.QueryOptions")).Return(model.Artists{artist1}, nil).Once()
// Mock agent error
agentErr := errors.New("agent error")
ag.On("GetArtistTopSongs", ctx, "artist-1", "Artist One", "mbid-artist-1", 5).Return(nil, agentErr).Once()
songs, err := p.TopSongs(ctx, "Artist One", 5)
Expect(err).To(MatchError(agentErr))
Expect(songs).To(BeNil())
artistRepo.AssertExpectations(GinkgoT())
ag.AssertExpectations(GinkgoT())
})
It("returns ErrNotFound when the agent returns ErrNotFound", func() {
// Mock finding the artist
artist1 := model.Artist{ID: "artist-1", Name: "Artist One", MbzArtistID: "mbid-artist-1"}
artistRepo.On("GetAll", mock.AnythingOfType("model.QueryOptions")).Return(model.Artists{artist1}, nil).Once()
// Mock agent ErrNotFound
ag.On("GetArtistTopSongs", ctx, "artist-1", "Artist One", "mbid-artist-1", 5).Return(nil, agents.ErrNotFound).Once()
songs, err := p.TopSongs(ctx, "Artist One", 5)
Expect(err).To(MatchError(model.ErrNotFound))
Expect(songs).To(BeNil())
artistRepo.AssertExpectations(GinkgoT())
ag.AssertExpectations(GinkgoT())
})
It("returns fewer songs if count is less than available top songs", func() {
// Mock finding the artist
artist1 := model.Artist{ID: "artist-1", Name: "Artist One", MbzArtistID: "mbid-artist-1"}
artistRepo.On("GetAll", mock.AnythingOfType("model.QueryOptions")).Return(model.Artists{artist1}, nil).Once()
// Mock agent response (only need 1 for the test)
agentSongs := []agents.Song{{Name: "Song One", MBID: "mbid-song-1"}}
ag.On("GetArtistTopSongs", ctx, "artist-1", "Artist One", "mbid-artist-1", 1).Return(agentSongs, nil).Once()
// Mock finding matching track
song1 := model.MediaFile{ID: "song-1", Title: "Song One", ArtistID: "artist-1", MbzRecordingID: "mbid-song-1"}
mediaFileRepo.On("GetAll", mock.AnythingOfType("model.QueryOptions")).Return(model.MediaFiles{song1}, nil).Once()
songs, err := p.TopSongs(ctx, "Artist One", 1)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(HaveLen(1))
Expect(songs[0].ID).To(Equal("song-1"))
artistRepo.AssertExpectations(GinkgoT())
ag.AssertExpectations(GinkgoT())
mediaFileRepo.AssertExpectations(GinkgoT())
})
It("returns fewer songs if fewer matching tracks are found", func() {
// Mock finding the artist
artist1 := model.Artist{ID: "artist-1", Name: "Artist One", MbzArtistID: "mbid-artist-1"}
artistRepo.On("GetAll", mock.AnythingOfType("model.QueryOptions")).Return(model.Artists{artist1}, nil).Once()
// Mock agent response
agentSongs := []agents.Song{
{Name: "Song One", MBID: "mbid-song-1"},
{Name: "Song Two", MBID: "mbid-song-2"},
}
ag.On("GetArtistTopSongs", ctx, "artist-1", "Artist One", "mbid-artist-1", 2).Return(agentSongs, nil).Once()
// Mock finding matching tracks (only find song 1)
song1 := model.MediaFile{ID: "song-1", Title: "Song One", ArtistID: "artist-1", MbzRecordingID: "mbid-song-1"}
mediaFileRepo.On("GetAll", mock.AnythingOfType("model.QueryOptions")).Return(model.MediaFiles{song1}, nil).Once()
mediaFileRepo.On("GetAll", mock.AnythingOfType("model.QueryOptions")).Return(model.MediaFiles{}, nil).Once() // For mbid-song-2 (fails)
mediaFileRepo.On("GetAll", mock.AnythingOfType("model.QueryOptions")).Return(model.MediaFiles{}, nil).Once() // For title fallback (fails)
songs, err := p.TopSongs(ctx, "Artist One", 2)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(HaveLen(1))
Expect(songs[0].ID).To(Equal("song-1"))
artistRepo.AssertExpectations(GinkgoT())
ag.AssertExpectations(GinkgoT())
mediaFileRepo.AssertExpectations(GinkgoT())
})
It("returns error when context is canceled during agent call", func() {
// Mock finding the artist
artist1 := model.Artist{ID: "artist-1", Name: "Artist One", MbzArtistID: "mbid-artist-1"}
artistRepo.On("GetAll", mock.AnythingOfType("model.QueryOptions")).Return(model.Artists{artist1}, nil).Once()
// Setup context that will be canceled
canceledCtx, cancel := context.WithCancel(ctx)
// Mock agent call to return context canceled error
ag.On("GetArtistTopSongs", canceledCtx, "artist-1", "Artist One", "mbid-artist-1", 5).Return(nil, context.Canceled).Once()
cancel() // Cancel the context before calling
songs, err := p.TopSongs(canceledCtx, "Artist One", 5)
Expect(err).To(MatchError(context.Canceled))
Expect(songs).To(BeNil())
artistRepo.AssertExpectations(GinkgoT())
ag.AssertExpectations(GinkgoT())
})
})

View File

@@ -0,0 +1,170 @@
package external_test
import (
"context"
"errors"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/core/external"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
"github.com/navidrome/navidrome/utils/gg"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/stretchr/testify/mock"
)
func init() {
log.SetLevel(log.LevelDebug)
}
var _ = Describe("Provider - UpdateAlbumInfo", func() {
var (
ctx context.Context
p external.Provider
ds *tests.MockDataStore
ag *mockAgents
mockAlbumRepo *tests.MockAlbumRepo
)
BeforeEach(func() {
ctx = GinkgoT().Context()
ds = new(tests.MockDataStore)
ag = new(mockAgents)
p = external.NewProvider(ds, ag)
mockAlbumRepo = ds.Album(ctx).(*tests.MockAlbumRepo)
conf.Server.DevAlbumInfoTimeToLive = 1 * time.Hour
})
It("returns error when album is not found", func() {
album, err := p.UpdateAlbumInfo(ctx, "al-not-found")
Expect(err).To(MatchError(model.ErrNotFound))
Expect(album).To(BeNil())
ag.AssertNotCalled(GinkgoT(), "GetAlbumInfo", mock.Anything, mock.Anything, mock.Anything, mock.Anything)
})
It("populates info when album exists but has no external info", func() {
originalAlbum := &model.Album{
ID: "al-existing",
Name: "Test Album",
AlbumArtist: "Test Artist",
MbzAlbumID: "mbid-album",
}
mockAlbumRepo.SetData(model.Albums{*originalAlbum})
expectedInfo := &agents.AlbumInfo{
URL: "http://example.com/album",
Description: "Album Description",
Images: []agents.ExternalImage{
{URL: "http://example.com/large.jpg", Size: 300},
{URL: "http://example.com/medium.jpg", Size: 200},
{URL: "http://example.com/small.jpg", Size: 100},
},
}
ag.On("GetAlbumInfo", ctx, "Test Album", "Test Artist", "mbid-album").Return(expectedInfo, nil)
updatedAlbum, err := p.UpdateAlbumInfo(ctx, "al-existing")
Expect(err).NotTo(HaveOccurred())
Expect(updatedAlbum).NotTo(BeNil())
Expect(updatedAlbum.ID).To(Equal("al-existing"))
Expect(updatedAlbum.ExternalUrl).To(Equal("http://example.com/album"))
Expect(updatedAlbum.Description).To(Equal("Album Description"))
Expect(updatedAlbum.LargeImageUrl).To(Equal("http://example.com/large.jpg"))
Expect(updatedAlbum.MediumImageUrl).To(Equal("http://example.com/medium.jpg"))
Expect(updatedAlbum.SmallImageUrl).To(Equal("http://example.com/small.jpg"))
Expect(updatedAlbum.ExternalInfoUpdatedAt).NotTo(BeNil())
Expect(*updatedAlbum.ExternalInfoUpdatedAt).To(BeTemporally("~", time.Now(), time.Second))
ag.AssertExpectations(GinkgoT())
})
It("returns cached info when album exists and info is not expired", func() {
now := time.Now()
originalAlbum := &model.Album{
ID: "al-cached",
Name: "Cached Album",
AlbumArtist: "Cached Artist",
ExternalUrl: "http://cached.com/album",
Description: "Cached Desc",
LargeImageUrl: "http://cached.com/large.jpg",
ExternalInfoUpdatedAt: gg.P(now.Add(-conf.Server.DevAlbumInfoTimeToLive / 2)),
}
mockAlbumRepo.SetData(model.Albums{*originalAlbum})
updatedAlbum, err := p.UpdateAlbumInfo(ctx, "al-cached")
Expect(err).NotTo(HaveOccurred())
Expect(updatedAlbum).NotTo(BeNil())
Expect(*updatedAlbum).To(Equal(*originalAlbum))
ag.AssertNotCalled(GinkgoT(), "GetAlbumInfo", mock.Anything, mock.Anything, mock.Anything, mock.Anything)
})
It("returns cached info and triggers background refresh when info is expired", func() {
now := time.Now()
expiredTime := now.Add(-conf.Server.DevAlbumInfoTimeToLive * 2)
originalAlbum := &model.Album{
ID: "al-expired",
Name: "Expired Album",
AlbumArtist: "Expired Artist",
ExternalUrl: "http://expired.com/album",
Description: "Expired Desc",
LargeImageUrl: "http://expired.com/large.jpg",
ExternalInfoUpdatedAt: gg.P(expiredTime),
}
mockAlbumRepo.SetData(model.Albums{*originalAlbum})
updatedAlbum, err := p.UpdateAlbumInfo(ctx, "al-expired")
Expect(err).NotTo(HaveOccurred())
Expect(updatedAlbum).NotTo(BeNil())
Expect(*updatedAlbum).To(Equal(*originalAlbum))
ag.AssertNotCalled(GinkgoT(), "GetAlbumInfo", mock.Anything, mock.Anything, mock.Anything, mock.Anything)
})
It("returns error when agent fails to get album info", func() {
originalAlbum := &model.Album{
ID: "al-agent-error",
Name: "Agent Error Album",
AlbumArtist: "Agent Error Artist",
MbzAlbumID: "mbid-agent-error",
}
mockAlbumRepo.SetData(model.Albums{*originalAlbum})
expectedErr := errors.New("agent communication failed")
ag.On("GetAlbumInfo", ctx, "Agent Error Album", "Agent Error Artist", "mbid-agent-error").Return(nil, expectedErr)
updatedAlbum, err := p.UpdateAlbumInfo(ctx, "al-agent-error")
Expect(err).To(MatchError(expectedErr))
Expect(updatedAlbum).To(BeNil())
ag.AssertExpectations(GinkgoT())
})
It("returns original album when agent returns ErrNotFound", func() {
originalAlbum := &model.Album{
ID: "al-agent-notfound",
Name: "Agent NotFound Album",
AlbumArtist: "Agent NotFound Artist",
MbzAlbumID: "mbid-agent-notfound",
}
mockAlbumRepo.SetData(model.Albums{*originalAlbum})
ag.On("GetAlbumInfo", ctx, "Agent NotFound Album", "Agent NotFound Artist", "mbid-agent-notfound").Return(nil, agents.ErrNotFound)
updatedAlbum, err := p.UpdateAlbumInfo(ctx, "al-agent-notfound")
Expect(err).NotTo(HaveOccurred())
Expect(updatedAlbum).NotTo(BeNil())
Expect(*updatedAlbum).To(Equal(*originalAlbum))
Expect(updatedAlbum.ExternalInfoUpdatedAt).To(BeNil())
ag.AssertExpectations(GinkgoT())
})
})

View File

@@ -0,0 +1,229 @@
package external_test
import (
"context"
"errors"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/core/external"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
"github.com/navidrome/navidrome/utils/gg"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/stretchr/testify/mock"
)
func init() {
log.SetLevel(log.LevelDebug)
}
var _ = Describe("Provider - UpdateArtistInfo", func() {
var (
ctx context.Context
p external.Provider
ds *tests.MockDataStore
ag *mockAgents
mockArtistRepo *tests.MockArtistRepo
)
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.DevArtistInfoTimeToLive = 1 * time.Hour
ctx = GinkgoT().Context()
ds = new(tests.MockDataStore)
ag = new(mockAgents)
p = external.NewProvider(ds, ag)
mockArtistRepo = ds.Artist(ctx).(*tests.MockArtistRepo)
})
It("returns error when artist is not found", func() {
artist, err := p.UpdateArtistInfo(ctx, "ar-not-found", 10, false)
Expect(err).To(MatchError(model.ErrNotFound))
Expect(artist).To(BeNil())
ag.AssertNotCalled(GinkgoT(), "GetArtistMBID")
ag.AssertNotCalled(GinkgoT(), "GetArtistImages")
ag.AssertNotCalled(GinkgoT(), "GetArtistBiography")
ag.AssertNotCalled(GinkgoT(), "GetArtistURL")
ag.AssertNotCalled(GinkgoT(), "GetSimilarArtists")
})
It("populates info when artist exists but has no external info", func() {
originalArtist := &model.Artist{
ID: "ar-existing",
Name: "Test Artist",
}
mockArtistRepo.SetData(model.Artists{*originalArtist})
expectedMBID := "mbid-artist-123"
expectedBio := "Artist Bio"
expectedURL := "http://artist.url"
expectedImages := []agents.ExternalImage{
{URL: "http://large.jpg", Size: 300},
{URL: "http://medium.jpg", Size: 200},
{URL: "http://small.jpg", Size: 100},
}
rawSimilar := []agents.Artist{
{Name: "Similar Artist 1", MBID: "mbid-similar-1"},
{Name: "Similar Artist 2", MBID: "mbid-similar-2"},
{Name: "Similar Artist 3", MBID: "mbid-similar-3"},
}
similarInDS := model.Artist{ID: "ar-similar-2", Name: "Similar Artist 2"}
ag.On("GetArtistMBID", ctx, "ar-existing", "Test Artist").Return(expectedMBID, nil).Once()
ag.On("GetArtistImages", ctx, "ar-existing", "Test Artist", expectedMBID).Return(expectedImages, nil).Once()
ag.On("GetArtistBiography", ctx, "ar-existing", "Test Artist", expectedMBID).Return(expectedBio, nil).Once()
ag.On("GetArtistURL", ctx, "ar-existing", "Test Artist", expectedMBID).Return(expectedURL, nil).Once()
ag.On("GetSimilarArtists", ctx, "ar-existing", "Test Artist", expectedMBID, 100).Return(rawSimilar, nil).Once()
mockArtistRepo.SetData(model.Artists{*originalArtist, similarInDS})
updatedArtist, err := p.UpdateArtistInfo(ctx, "ar-existing", 10, false)
Expect(err).NotTo(HaveOccurred())
Expect(updatedArtist).NotTo(BeNil())
Expect(updatedArtist.ID).To(Equal("ar-existing"))
Expect(updatedArtist.MbzArtistID).To(Equal(expectedMBID))
Expect(updatedArtist.Biography).To(Equal("Artist Bio"))
Expect(updatedArtist.ExternalUrl).To(Equal(expectedURL))
Expect(updatedArtist.LargeImageUrl).To(Equal("http://large.jpg"))
Expect(updatedArtist.MediumImageUrl).To(Equal("http://medium.jpg"))
Expect(updatedArtist.SmallImageUrl).To(Equal("http://small.jpg"))
Expect(updatedArtist.ExternalInfoUpdatedAt).NotTo(BeNil())
Expect(*updatedArtist.ExternalInfoUpdatedAt).To(BeTemporally("~", time.Now(), time.Second))
Expect(updatedArtist.SimilarArtists).To(HaveLen(1))
Expect(updatedArtist.SimilarArtists[0].ID).To(Equal("ar-similar-2"))
Expect(updatedArtist.SimilarArtists[0].Name).To(Equal("Similar Artist 2"))
ag.AssertExpectations(GinkgoT())
})
It("returns cached info when artist exists and info is not expired", func() {
now := time.Now()
originalArtist := &model.Artist{
ID: "ar-cached",
Name: "Cached Artist",
MbzArtistID: "mbid-cached",
ExternalUrl: "http://cached.url",
Biography: "Cached Bio",
LargeImageUrl: "http://cached_large.jpg",
ExternalInfoUpdatedAt: gg.P(now.Add(-conf.Server.DevArtistInfoTimeToLive / 2)),
SimilarArtists: model.Artists{
{ID: "ar-similar-present", Name: "Similar Present"},
{ID: "ar-similar-absent", Name: "Similar Absent"},
},
}
similarInDS := model.Artist{ID: "ar-similar-present", Name: "Similar Present Updated"}
mockArtistRepo.SetData(model.Artists{*originalArtist, similarInDS})
updatedArtist, err := p.UpdateArtistInfo(ctx, "ar-cached", 5, false)
Expect(err).NotTo(HaveOccurred())
Expect(updatedArtist).NotTo(BeNil())
Expect(updatedArtist.ID).To(Equal(originalArtist.ID))
Expect(updatedArtist.Name).To(Equal(originalArtist.Name))
Expect(updatedArtist.MbzArtistID).To(Equal(originalArtist.MbzArtistID))
Expect(updatedArtist.ExternalUrl).To(Equal(originalArtist.ExternalUrl))
Expect(updatedArtist.Biography).To(Equal(originalArtist.Biography))
Expect(updatedArtist.LargeImageUrl).To(Equal(originalArtist.LargeImageUrl))
Expect(updatedArtist.ExternalInfoUpdatedAt).To(Equal(originalArtist.ExternalInfoUpdatedAt))
Expect(updatedArtist.SimilarArtists).To(HaveLen(1))
Expect(updatedArtist.SimilarArtists[0].ID).To(Equal(similarInDS.ID))
Expect(updatedArtist.SimilarArtists[0].Name).To(Equal(similarInDS.Name))
ag.AssertNotCalled(GinkgoT(), "GetArtistMBID")
ag.AssertNotCalled(GinkgoT(), "GetArtistImages")
ag.AssertNotCalled(GinkgoT(), "GetArtistBiography")
ag.AssertNotCalled(GinkgoT(), "GetArtistURL")
})
It("returns cached info and triggers background refresh when info is expired", func() {
now := time.Now()
expiredTime := now.Add(-conf.Server.DevArtistInfoTimeToLive * 2)
originalArtist := &model.Artist{
ID: "ar-expired",
Name: "Expired Artist",
ExternalInfoUpdatedAt: gg.P(expiredTime),
SimilarArtists: model.Artists{
{ID: "ar-exp-similar", Name: "Expired Similar"},
},
}
similarInDS := model.Artist{ID: "ar-exp-similar", Name: "Expired Similar Updated"}
mockArtistRepo.SetData(model.Artists{*originalArtist, similarInDS})
updatedArtist, err := p.UpdateArtistInfo(ctx, "ar-expired", 5, false)
Expect(err).NotTo(HaveOccurred())
Expect(updatedArtist).NotTo(BeNil())
Expect(updatedArtist.ID).To(Equal(originalArtist.ID))
Expect(updatedArtist.Name).To(Equal(originalArtist.Name))
Expect(updatedArtist.ExternalInfoUpdatedAt).To(Equal(originalArtist.ExternalInfoUpdatedAt))
Expect(updatedArtist.SimilarArtists).To(HaveLen(1))
Expect(updatedArtist.SimilarArtists[0].ID).To(Equal(similarInDS.ID))
Expect(updatedArtist.SimilarArtists[0].Name).To(Equal(similarInDS.Name))
ag.AssertNotCalled(GinkgoT(), "GetArtistMBID")
ag.AssertNotCalled(GinkgoT(), "GetArtistImages")
ag.AssertNotCalled(GinkgoT(), "GetArtistBiography")
ag.AssertNotCalled(GinkgoT(), "GetArtistURL")
})
It("includes non-present similar artists when includeNotPresent is true", func() {
now := time.Now()
originalArtist := &model.Artist{
ID: "ar-similar-test",
Name: "Similar Test Artist",
ExternalInfoUpdatedAt: gg.P(now.Add(-conf.Server.DevArtistInfoTimeToLive / 2)),
SimilarArtists: model.Artists{
{ID: "ar-sim-present", Name: "Similar Present"},
{ID: "", Name: "Similar Absent Raw"},
{ID: "ar-sim-absent-lookup", Name: "Similar Absent Lookup"},
},
}
similarInDS := model.Artist{ID: "ar-sim-present", Name: "Similar Present Updated"}
mockArtistRepo.SetData(model.Artists{*originalArtist, similarInDS})
updatedArtist, err := p.UpdateArtistInfo(ctx, "ar-similar-test", 5, true)
Expect(err).NotTo(HaveOccurred())
Expect(updatedArtist).NotTo(BeNil())
Expect(updatedArtist.SimilarArtists).To(HaveLen(3))
Expect(updatedArtist.SimilarArtists[0].ID).To(Equal(similarInDS.ID))
Expect(updatedArtist.SimilarArtists[0].Name).To(Equal(similarInDS.Name))
Expect(updatedArtist.SimilarArtists[1].ID).To(BeEmpty())
Expect(updatedArtist.SimilarArtists[1].Name).To(Equal("Similar Absent Raw"))
Expect(updatedArtist.SimilarArtists[2].ID).To(BeEmpty())
Expect(updatedArtist.SimilarArtists[2].Name).To(Equal("Similar Absent Lookup"))
})
It("updates ArtistInfo even if an optional agent call fails", func() {
originalArtist := &model.Artist{
ID: "ar-agent-fail",
Name: "Agent Fail Artist",
}
mockArtistRepo.SetData(model.Artists{*originalArtist})
expectedErr := errors.New("agent MBID failed")
ag.On("GetArtistMBID", ctx, "ar-agent-fail", "Agent Fail Artist").Return("", expectedErr).Once()
ag.On("GetArtistImages", ctx, "ar-agent-fail", "Agent Fail Artist", mock.Anything).Return(nil, nil).Maybe()
ag.On("GetArtistBiography", ctx, "ar-agent-fail", "Agent Fail Artist", mock.Anything).Return("", nil).Maybe()
ag.On("GetArtistURL", ctx, "ar-agent-fail", "Agent Fail Artist", mock.Anything).Return("", nil).Maybe()
ag.On("GetSimilarArtists", ctx, "ar-agent-fail", "Agent Fail Artist", mock.Anything, 100).Return(nil, nil).Maybe()
updatedArtist, err := p.UpdateArtistInfo(ctx, "ar-agent-fail", 10, false)
Expect(err).NotTo(HaveOccurred())
Expect(updatedArtist).NotTo(BeNil())
Expect(updatedArtist.ID).To(Equal("ar-agent-fail"))
ag.AssertExpectations(GinkgoT())
})
})

View File

@@ -29,7 +29,7 @@ func New() FFmpeg {
}
const (
extractImageCmd = "ffmpeg -i %s -an -vcodec copy -f image2pipe -"
extractImageCmd = "ffmpeg -i %s -map 0:v -map -0:V -vcodec copy -f image2pipe -"
probeCmd = "ffmpeg %s -f ffmetadata"
)
@@ -39,6 +39,10 @@ func (e *ffmpeg) Transcode(ctx context.Context, command, path string, maxBitRate
if _, err := ffmpegCmd(); err != nil {
return nil, err
}
// First make sure the file exists
if err := fileExists(path); err != nil {
return nil, err
}
args := createFFmpegCommand(command, path, maxBitRate, offset)
return e.start(ctx, args)
}
@@ -47,10 +51,25 @@ func (e *ffmpeg) ExtractImage(ctx context.Context, path string) (io.ReadCloser,
if _, err := ffmpegCmd(); err != nil {
return nil, err
}
// First make sure the file exists
if err := fileExists(path); err != nil {
return nil, err
}
args := createFFmpegCommand(extractImageCmd, path, 0, 0)
return e.start(ctx, args)
}
func fileExists(path string) error {
s, err := os.Stat(path)
if err != nil {
return err
}
if s.IsDir() {
return fmt.Errorf("'%s' is a directory", path)
}
return nil
}
func (e *ffmpeg) Probe(ctx context.Context, files []string) (string, error) {
if _, err := ffmpegCmd(); err != nil {
return "", err

51
core/inspect.go Normal file
View File

@@ -0,0 +1,51 @@
package core
import (
"path/filepath"
"github.com/navidrome/navidrome/core/storage"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/metadata"
. "github.com/navidrome/navidrome/utils/gg"
)
type InspectOutput struct {
File string `json:"file"`
RawTags model.RawTags `json:"rawTags"`
MappedTags *model.MediaFile `json:"mappedTags,omitempty"`
}
func Inspect(filePath string, libraryId int, folderId string) (*InspectOutput, error) {
path, file := filepath.Split(filePath)
s, err := storage.For(path)
if err != nil {
return nil, err
}
fs, err := s.FS()
if err != nil {
return nil, err
}
tags, err := fs.ReadTags(file)
if err != nil {
return nil, err
}
tag, ok := tags[file]
if !ok {
log.Error("Could not get tags for path", "path", filePath)
return nil, model.ErrNotFound
}
md := metadata.New(path, tag)
result := &InspectOutput{
File: filePath,
RawTags: tags[file].Tags,
MappedTags: P(md.ToMediaFile(libraryId, folderId)),
}
return result, nil
}

View File

@@ -36,11 +36,12 @@ type mediaStreamer struct {
}
type streamJob struct {
ms *mediaStreamer
mf *model.MediaFile
format string
bitRate int
offset int
ms *mediaStreamer
mf *model.MediaFile
filePath string
format string
bitRate int
offset int
}
func (j *streamJob) Key() string {
@@ -68,13 +69,14 @@ func (ms *mediaStreamer) DoStream(ctx context.Context, mf *model.MediaFile, reqF
format, bitRate = selectTranscodingOptions(ctx, ms.ds, mf, reqFormat, reqBitRate)
s := &Stream{ctx: ctx, mf: mf, format: format, bitRate: bitRate}
filePath := mf.AbsolutePath()
if format == "raw" {
log.Debug(ctx, "Streaming RAW file", "id", mf.ID, "path", mf.Path,
log.Debug(ctx, "Streaming RAW file", "id", mf.ID, "path", filePath,
"requestBitrate", reqBitRate, "requestFormat", reqFormat, "requestOffset", reqOffset,
"originalBitrate", mf.BitRate, "originalFormat", mf.Suffix,
"selectedBitrate", bitRate, "selectedFormat", format)
f, err := os.Open(mf.Path)
f, err := os.Open(filePath)
if err != nil {
return nil, err
}
@@ -85,11 +87,12 @@ func (ms *mediaStreamer) DoStream(ctx context.Context, mf *model.MediaFile, reqF
}
job := &streamJob{
ms: ms,
mf: mf,
format: format,
bitRate: bitRate,
offset: reqOffset,
ms: ms,
mf: mf,
filePath: filePath,
format: format,
bitRate: bitRate,
offset: reqOffset,
}
r, err := ms.cache.Get(ctx, job)
if err != nil {
@@ -101,7 +104,7 @@ func (ms *mediaStreamer) DoStream(ctx context.Context, mf *model.MediaFile, reqF
s.ReadCloser = r
s.Seeker = r.Seeker
log.Debug(ctx, "Streaming TRANSCODED file", "id", mf.ID, "path", mf.Path,
log.Debug(ctx, "Streaming TRANSCODED file", "id", mf.ID, "path", filePath,
"requestBitrate", reqBitRate, "requestFormat", reqFormat, "requestOffset", reqOffset,
"originalBitrate", mf.BitRate, "originalFormat", mf.Suffix,
"selectedBitrate", bitRate, "selectedFormat", format, "cached", cached, "seekable", s.Seekable())
@@ -201,7 +204,7 @@ func NewTranscodingCache() TranscodingCache {
log.Error(ctx, "Error loading transcoding command", "format", job.format, err)
return nil, os.ErrInvalid
}
out, err := job.ms.transcoder.Transcode(ctx, t.Command, job.mf.Path, job.bitRate, job.offset)
out, err := job.ms.transcoder.Transcode(ctx, t.Command, job.filePath, job.bitRate, job.offset)
if err != nil {
log.Error(ctx, "Error starting transcoder", "id", job.mf.ID, err)
return nil, os.ErrInvalid

View File

@@ -187,7 +187,6 @@ var staticData = sync.OnceValue(func() insights.Data {
data.Config.EnablePrometheus = conf.Server.Prometheus.Enabled
data.Config.TranscodingCacheSize = conf.Server.TranscodingCacheSize
data.Config.ImageCacheSize = conf.Server.ImageCacheSize
data.Config.ScanSchedule = conf.Server.ScanSchedule
data.Config.SessionTimeout = uint64(math.Trunc(conf.Server.SessionTimeout.Seconds()))
data.Config.SearchFullString = conf.Server.SearchFullString
data.Config.RecentlyAddedByModTime = conf.Server.RecentlyAddedByModTime
@@ -195,6 +194,10 @@ var staticData = sync.OnceValue(func() insights.Data {
data.Config.BackupSchedule = conf.Server.Backup.Schedule
data.Config.BackupCount = conf.Server.Backup.Count
data.Config.DevActivityPanel = conf.Server.DevActivityPanel
data.Config.ScannerEnabled = conf.Server.Scanner.Enabled
data.Config.ScanSchedule = conf.Server.Scanner.Schedule
data.Config.ScanWatcherWait = uint64(math.Trunc(conf.Server.Scanner.WatcherWait.Seconds()))
data.Config.ScanOnStartup = conf.Server.Scanner.ScanOnStartup
return data
})

View File

@@ -43,7 +43,10 @@ type Data struct {
LogLevel string `json:"logLevel,omitempty"`
LogFileConfigured bool `json:"logFileConfigured,omitempty"`
TLSConfigured bool `json:"tlsConfigured,omitempty"`
ScannerEnabled bool `json:"scannerEnabled,omitempty"`
ScanSchedule string `json:"scanSchedule,omitempty"`
ScanWatcherWait uint64 `json:"scanWatcherWait,omitempty"`
ScanOnStartup bool `json:"scanOnStartup,omitempty"`
TranscodingCacheSize string `json:"transcodingCacheSize,omitempty"`
ImageCacheSize string `json:"imageCacheSize,omitempty"`
EnableArtworkPrecache bool `json:"enableArtworkPrecache,omitempty"`

View File

@@ -28,7 +28,14 @@ type metrics struct {
}
func NewPrometheusInstance(ds model.DataStore) Metrics {
return &metrics{ds: ds}
if conf.Server.Prometheus.Enabled {
return &metrics{ds: ds}
}
return noopMetrics{}
}
func NewNoopInstance() Metrics {
return noopMetrics{}
}
func (m *metrics) WriteInitialMetrics(ctx context.Context) {
@@ -144,3 +151,12 @@ func processSqlAggregateMetrics(ctx context.Context, ds model.DataStore, targetG
}
targetGauge.With(prometheus.Labels{"model": "user"}).Set(float64(usersCount))
}
type noopMetrics struct {
}
func (n noopMetrics) WriteInitialMetrics(context.Context) {}
func (n noopMetrics) WriteAfterScanMetrics(context.Context, bool) {}
func (n noopMetrics) GetHandler() http.Handler { return nil }

View File

@@ -5,13 +5,13 @@ package mpv
import (
"path/filepath"
"github.com/google/uuid"
"github.com/navidrome/navidrome/model/id"
)
func socketName(prefix, suffix string) string {
// Windows needs to use a named pipe for the socket
// see https://mpv.io/manual/master#using-mpv-from-other-programs-or-scripts
return filepath.Join(`\\.\pipe\mpvsocket`, prefix+uuid.NewString()+suffix)
return filepath.Join(`\\.\pipe\mpvsocket`, prefix+id.NewRandom()+suffix)
}
func removeSocket(string) {

View File

@@ -5,10 +5,13 @@ import (
"fmt"
"time"
"github.com/google/uuid"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/id"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/utils"
)
type Players interface {
@@ -17,46 +20,57 @@ type Players interface {
}
func NewPlayers(ds model.DataStore) Players {
return &players{ds}
return &players{
ds: ds,
limiter: utils.Limiter{Interval: consts.UpdatePlayerFrequency},
}
}
type players struct {
ds model.DataStore
ds model.DataStore
limiter utils.Limiter
}
func (p *players) Register(ctx context.Context, id, client, userAgent, ip string) (*model.Player, *model.Transcoding, error) {
func (p *players) Register(ctx context.Context, playerID, client, userAgent, ip string) (*model.Player, *model.Transcoding, error) {
var plr *model.Player
var trc *model.Transcoding
var err error
user, _ := request.UserFrom(ctx)
if id != "" {
plr, err = p.ds.Player(ctx).Get(id)
if playerID != "" {
plr, err = p.ds.Player(ctx).Get(playerID)
if err == nil && plr.Client != client {
id = ""
playerID = ""
}
}
if err != nil || id == "" {
username := userName(ctx)
if err != nil || playerID == "" {
plr, err = p.ds.Player(ctx).FindMatch(user.ID, client, userAgent)
if err == nil {
log.Debug(ctx, "Found matching player", "id", plr.ID, "client", client, "username", userName(ctx), "type", userAgent)
log.Debug(ctx, "Found matching player", "id", plr.ID, "client", client, "username", username, "type", userAgent)
} else {
plr = &model.Player{
ID: uuid.NewString(),
ID: id.NewRandom(),
UserId: user.ID,
Client: client,
ScrobbleEnabled: true,
ReportRealPath: conf.Server.Subsonic.DefaultReportRealPath,
}
log.Info(ctx, "Registering new player", "id", plr.ID, "client", client, "username", userName(ctx), "type", userAgent)
log.Info(ctx, "Registering new player", "id", plr.ID, "client", client, "username", username, "type", userAgent)
}
}
plr.Name = fmt.Sprintf("%s [%s]", client, userAgent)
plr.UserAgent = userAgent
plr.IP = ip
plr.LastSeen = time.Now()
err = p.ds.Player(ctx).Put(plr)
if err != nil {
return nil, nil, err
}
p.limiter.Do(plr.ID, func() {
ctx, cancel := context.WithTimeout(ctx, time.Second)
defer cancel()
err = p.ds.Player(ctx).Put(plr)
if err != nil {
log.Warn(ctx, "Could not save player", "id", plr.ID, "client", client, "username", username, "type", userAgent, err)
}
})
if plr.TranscodingId != "" {
trc, err = p.ds.Transcoding(ctx).Get(plr.TranscodingId)
}

View File

@@ -9,10 +9,12 @@ import (
"net/url"
"os"
"path/filepath"
"regexp"
"strings"
"time"
"github.com/RaveNoX/go-jsoncommentstrip"
"github.com/bmatcuk/doublestar/v4"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
@@ -22,7 +24,7 @@ import (
)
type Playlists interface {
ImportFile(ctx context.Context, dir string, fname string) (*model.Playlist, error)
ImportFile(ctx context.Context, folder *model.Folder, filename string) (*model.Playlist, error)
Update(ctx context.Context, playlistID string, name *string, comment *string, public *bool, idsToAdd []string, idxToRemove []int) error
ImportM3U(ctx context.Context, reader io.Reader) (*model.Playlist, error)
}
@@ -35,16 +37,29 @@ func NewPlaylists(ds model.DataStore) Playlists {
return &playlists{ds: ds}
}
func (s *playlists) ImportFile(ctx context.Context, dir string, fname string) (*model.Playlist, error) {
pls, err := s.parsePlaylist(ctx, fname, dir)
func InPlaylistsPath(folder model.Folder) bool {
if conf.Server.PlaylistsPath == "" {
return true
}
rel, _ := filepath.Rel(folder.LibraryPath, folder.AbsolutePath())
for _, path := range strings.Split(conf.Server.PlaylistsPath, string(filepath.ListSeparator)) {
if match, _ := doublestar.Match(path, rel); match {
return true
}
}
return false
}
func (s *playlists) ImportFile(ctx context.Context, folder *model.Folder, filename string) (*model.Playlist, error) {
pls, err := s.parsePlaylist(ctx, filename, folder)
if err != nil {
log.Error(ctx, "Error parsing playlist", "path", filepath.Join(dir, fname), err)
log.Error(ctx, "Error parsing playlist", "path", filepath.Join(folder.AbsolutePath(), filename), err)
return nil, err
}
log.Debug("Found playlist", "name", pls.Name, "lastUpdated", pls.UpdatedAt, "path", pls.Path, "numTracks", len(pls.Tracks))
err = s.updatePlaylist(ctx, pls)
if err != nil {
log.Error(ctx, "Error updating playlist", "path", filepath.Join(dir, fname), err)
log.Error(ctx, "Error updating playlist", "path", filepath.Join(folder.AbsolutePath(), filename), err)
}
return pls, err
}
@@ -56,7 +71,7 @@ func (s *playlists) ImportM3U(ctx context.Context, reader io.Reader) (*model.Pla
Public: false,
Sync: false,
}
err := s.parseM3U(ctx, pls, "", reader)
err := s.parseM3U(ctx, pls, nil, reader)
if err != nil {
log.Error(ctx, "Error parsing playlist", err)
return nil, err
@@ -69,8 +84,8 @@ func (s *playlists) ImportM3U(ctx context.Context, reader io.Reader) (*model.Pla
return pls, nil
}
func (s *playlists) parsePlaylist(ctx context.Context, playlistFile string, baseDir string) (*model.Playlist, error) {
pls, err := s.newSyncedPlaylist(baseDir, playlistFile)
func (s *playlists) parsePlaylist(ctx context.Context, playlistFile string, folder *model.Folder) (*model.Playlist, error) {
pls, err := s.newSyncedPlaylist(folder.AbsolutePath(), playlistFile)
if err != nil {
return nil, err
}
@@ -86,7 +101,7 @@ func (s *playlists) parsePlaylist(ctx context.Context, playlistFile string, base
case ".nsp":
err = s.parseNSP(ctx, pls, file)
default:
err = s.parseM3U(ctx, pls, baseDir, file)
err = s.parseM3U(ctx, pls, folder, file)
}
return pls, err
}
@@ -112,14 +127,35 @@ func (s *playlists) newSyncedPlaylist(baseDir string, playlistFile string) (*mod
return pls, nil
}
func (s *playlists) parseNSP(ctx context.Context, pls *model.Playlist, file io.Reader) error {
func getPositionFromOffset(data []byte, offset int64) (line, column int) {
line = 1
for _, b := range data[:offset] {
if b == '\n' {
line++
column = 1
} else {
column++
}
}
return
}
func (s *playlists) parseNSP(_ context.Context, pls *model.Playlist, reader io.Reader) error {
nsp := &nspFile{}
reader := jsoncommentstrip.NewReader(file)
dec := json.NewDecoder(reader)
err := dec.Decode(nsp)
reader = io.LimitReader(reader, 100*1024) // Limit to 100KB
reader = jsoncommentstrip.NewReader(reader)
input, err := io.ReadAll(reader)
if err != nil {
log.Error(ctx, "Error parsing SmartPlaylist", "playlist", pls.Name, err)
return err
return fmt.Errorf("reading SmartPlaylist: %w", err)
}
err = json.Unmarshal(input, nsp)
if err != nil {
var syntaxErr *json.SyntaxError
if errors.As(err, &syntaxErr) {
line, col := getPositionFromOffset(input, syntaxErr.Offset)
return fmt.Errorf("JSON syntax error in SmartPlaylist at line %d, column %d: %w", line, col, err)
}
return fmt.Errorf("JSON parsing error in SmartPlaylist: %w", err)
}
pls.Rules = &nsp.Criteria
if nsp.Name != "" {
@@ -131,7 +167,7 @@ func (s *playlists) parseNSP(ctx context.Context, pls *model.Playlist, file io.R
return nil
}
func (s *playlists) parseM3U(ctx context.Context, pls *model.Playlist, baseDir string, reader io.Reader) error {
func (s *playlists) parseM3U(ctx context.Context, pls *model.Playlist, folder *model.Folder, reader io.Reader) error {
mediaFileRepository := s.ds.MediaFile(ctx)
var mfs model.MediaFiles
for lines := range slice.CollectChunks(slice.LinesFrom(reader), 400) {
@@ -150,12 +186,17 @@ func (s *playlists) parseM3U(ctx context.Context, pls *model.Playlist, baseDir s
line = strings.TrimPrefix(line, "file://")
line, _ = url.QueryUnescape(line)
}
if baseDir != "" && !filepath.IsAbs(line) {
line = filepath.Join(baseDir, line)
if !model.IsAudioFile(line) {
continue
}
filteredLines = append(filteredLines, line)
}
found, err := mediaFileRepository.FindByPaths(filteredLines)
paths, err := s.normalizePaths(ctx, pls, folder, filteredLines)
if err != nil {
log.Warn(ctx, "Error normalizing paths in playlist", "playlist", pls.Name, err)
continue
}
found, err := mediaFileRepository.FindByPaths(paths)
if err != nil {
log.Warn(ctx, "Error reading files from DB", "playlist", pls.Name, err)
continue
@@ -164,7 +205,7 @@ func (s *playlists) parseM3U(ctx context.Context, pls *model.Playlist, baseDir s
for idx := range found {
existing[strings.ToLower(found[idx].Path)] = idx
}
for _, path := range filteredLines {
for _, path := range paths {
idx, ok := existing[strings.ToLower(path)]
if ok {
mfs = append(mfs, found[idx])
@@ -182,6 +223,64 @@ func (s *playlists) parseM3U(ctx context.Context, pls *model.Playlist, baseDir s
return nil
}
// TODO This won't work for multiple libraries
func (s *playlists) normalizePaths(ctx context.Context, pls *model.Playlist, folder *model.Folder, lines []string) ([]string, error) {
libRegex, err := s.compileLibraryPaths(ctx)
if err != nil {
return nil, err
}
res := make([]string, 0, len(lines))
for idx, line := range lines {
var libPath string
var filePath string
if folder != nil && !filepath.IsAbs(line) {
libPath = folder.LibraryPath
filePath = filepath.Join(folder.AbsolutePath(), line)
} else {
cleanLine := filepath.Clean(line)
if libPath = libRegex.FindString(cleanLine); libPath != "" {
filePath = cleanLine
}
}
if libPath != "" {
if rel, err := filepath.Rel(libPath, filePath); err == nil {
res = append(res, rel)
} else {
log.Debug(ctx, "Error getting relative path", "playlist", pls.Name, "path", line, "libPath", libPath,
"filePath", filePath, err)
}
} else {
log.Warn(ctx, "Path in playlist not found in any library", "path", line, "line", idx)
}
}
return slice.Map(res, filepath.ToSlash), nil
}
func (s *playlists) compileLibraryPaths(ctx context.Context) (*regexp.Regexp, error) {
libs, err := s.ds.Library(ctx).GetAll()
if err != nil {
return nil, err
}
// Create regex patterns for each library path
patterns := make([]string, len(libs))
for i, lib := range libs {
cleanPath := filepath.Clean(lib.Path)
escapedPath := regexp.QuoteMeta(cleanPath)
patterns[i] = fmt.Sprintf("^%s(?:/|$)", escapedPath)
}
// Combine all patterns into a single regex
combinedPattern := strings.Join(patterns, "|")
re, err := regexp.Compile(combinedPattern)
if err != nil {
return nil, fmt.Errorf("compiling library paths `%s`: %w", combinedPattern, err)
}
return re, nil
}
func (s *playlists) updatePlaylist(ctx context.Context, newPls *model.Playlist) error {
owner, _ := request.UserFrom(ctx)
@@ -216,7 +315,7 @@ func (s *playlists) Update(ctx context.Context, playlistID string,
needsInfoUpdate := name != nil || comment != nil || public != nil
needsTrackRefresh := len(idxToRemove) > 0
return s.ds.WithTx(func(tx model.DataStore) error {
return s.ds.WithTxImmediate(func(tx model.DataStore) error {
var pls *model.Playlist
var err error
repo := tx.Playlist(ctx)
@@ -225,7 +324,7 @@ func (s *playlists) Update(ctx context.Context, playlistID string,
return fmt.Errorf("%w: playlist '%s'", model.ErrNotFound, playlistID)
}
if needsTrackRefresh {
pls, err = repo.GetWithTracks(playlistID, true)
pls, err = repo.GetWithTracks(playlistID, true, false)
pls.RemoveTracks(idxToRemove)
pls.AddTracks(idsToAdd)
} else {

View File

@@ -7,6 +7,8 @@ import (
"strings"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/criteria"
"github.com/navidrome/navidrome/model/request"
@@ -18,43 +20,56 @@ import (
var _ = Describe("Playlists", func() {
var ds *tests.MockDataStore
var ps Playlists
var mp mockedPlaylist
var mockPlsRepo mockedPlaylistRepo
var mockLibRepo *tests.MockLibraryRepo
ctx := context.Background()
BeforeEach(func() {
mp = mockedPlaylist{}
mockPlsRepo = mockedPlaylistRepo{}
mockLibRepo = &tests.MockLibraryRepo{}
ds = &tests.MockDataStore{
MockedPlaylist: &mp,
MockedPlaylist: &mockPlsRepo,
MockedLibrary: mockLibRepo,
}
ctx = request.WithUser(ctx, model.User{ID: "123"})
// Path should be libPath, but we want to match the root folder referenced in the m3u, which is `/`
mockLibRepo.SetData([]model.Library{{ID: 1, Path: "/"}})
})
Describe("ImportFile", func() {
var folder *model.Folder
BeforeEach(func() {
ps = NewPlaylists(ds)
ds.MockedMediaFile = &mockedMediaFileRepo{}
libPath, _ := os.Getwd()
folder = &model.Folder{
ID: "1",
LibraryID: 1,
LibraryPath: libPath,
Path: "tests/fixtures",
Name: "playlists",
}
})
Describe("M3U", func() {
It("parses well-formed playlists", func() {
pls, err := ps.ImportFile(ctx, "tests/fixtures", "playlists/pls1.m3u")
pls, err := ps.ImportFile(ctx, folder, "pls1.m3u")
Expect(err).ToNot(HaveOccurred())
Expect(pls.OwnerID).To(Equal("123"))
Expect(pls.Tracks).To(HaveLen(3))
Expect(pls.Tracks[0].Path).To(Equal("tests/fixtures/test.mp3"))
Expect(pls.Tracks[1].Path).To(Equal("tests/fixtures/test.ogg"))
Expect(pls.Tracks[2].Path).To(Equal("/tests/fixtures/01 Invisible (RED) Edit Version.mp3"))
Expect(mp.last).To(Equal(pls))
Expect(pls.Tracks).To(HaveLen(2))
Expect(pls.Tracks[0].Path).To(Equal("tests/fixtures/playlists/test.mp3"))
Expect(pls.Tracks[1].Path).To(Equal("tests/fixtures/playlists/test.ogg"))
Expect(mockPlsRepo.last).To(Equal(pls))
})
It("parses playlists using LF ending", func() {
pls, err := ps.ImportFile(ctx, "tests/fixtures/playlists", "lf-ended.m3u")
pls, err := ps.ImportFile(ctx, folder, "lf-ended.m3u")
Expect(err).ToNot(HaveOccurred())
Expect(pls.Tracks).To(HaveLen(2))
})
It("parses playlists using CR ending (old Mac format)", func() {
pls, err := ps.ImportFile(ctx, "tests/fixtures/playlists", "cr-ended.m3u")
pls, err := ps.ImportFile(ctx, folder, "cr-ended.m3u")
Expect(err).ToNot(HaveOccurred())
Expect(pls.Tracks).To(HaveLen(2))
})
@@ -62,9 +77,9 @@ var _ = Describe("Playlists", func() {
Describe("NSP", func() {
It("parses well-formed playlists", func() {
pls, err := ps.ImportFile(ctx, "tests/fixtures", "playlists/recently_played.nsp")
pls, err := ps.ImportFile(ctx, folder, "recently_played.nsp")
Expect(err).ToNot(HaveOccurred())
Expect(mp.last).To(Equal(pls))
Expect(mockPlsRepo.last).To(Equal(pls))
Expect(pls.OwnerID).To(Equal("123"))
Expect(pls.Name).To(Equal("Recently Played"))
Expect(pls.Comment).To(Equal("Recently played tracks"))
@@ -73,6 +88,10 @@ var _ = Describe("Playlists", func() {
Expect(pls.Rules.Limit).To(Equal(100))
Expect(pls.Rules.Expression).To(BeAssignableToTypeOf(criteria.All{}))
})
It("returns an error if the playlist is not well-formed", func() {
_, err := ps.ImportFile(ctx, folder, "invalid_json.nsp")
Expect(err.Error()).To(ContainSubstring("line 19, column 1: invalid character '\\n'"))
})
})
})
@@ -82,79 +101,136 @@ var _ = Describe("Playlists", func() {
repo = &mockedMediaFileFromListRepo{}
ds.MockedMediaFile = repo
ps = NewPlaylists(ds)
mockLibRepo.SetData([]model.Library{{ID: 1, Path: "/music"}, {ID: 2, Path: "/new"}})
ctx = request.WithUser(ctx, model.User{ID: "123"})
})
It("parses well-formed playlists", func() {
repo.data = []string{
"tests/fixtures/test.mp3",
"tests/fixtures/test.ogg",
"/tests/fixtures/01 Invisible (RED) Edit Version.mp3",
"tests/test.mp3",
"tests/test.ogg",
"tests/01 Invisible (RED) Edit Version.mp3",
"downloads/newfile.flac",
}
f, _ := os.Open("tests/fixtures/playlists/pls-with-name.m3u")
defer f.Close()
m3u := strings.Join([]string{
"#PLAYLIST:playlist 1",
"/music/tests/test.mp3",
"/music/tests/test.ogg",
"/new/downloads/newfile.flac",
"file:///music/tests/01%20Invisible%20(RED)%20Edit%20Version.mp3",
}, "\n")
f := strings.NewReader(m3u)
pls, err := ps.ImportM3U(ctx, f)
Expect(err).ToNot(HaveOccurred())
Expect(pls.OwnerID).To(Equal("123"))
Expect(pls.Name).To(Equal("playlist 1"))
Expect(pls.Sync).To(BeFalse())
Expect(pls.Tracks).To(HaveLen(3))
Expect(pls.Tracks[0].Path).To(Equal("tests/fixtures/test.mp3"))
Expect(pls.Tracks[1].Path).To(Equal("tests/fixtures/test.ogg"))
Expect(pls.Tracks[2].Path).To(Equal("/tests/fixtures/01 Invisible (RED) Edit Version.mp3"))
Expect(mp.last).To(Equal(pls))
f.Close()
Expect(pls.Tracks).To(HaveLen(4))
Expect(pls.Tracks[0].Path).To(Equal("tests/test.mp3"))
Expect(pls.Tracks[1].Path).To(Equal("tests/test.ogg"))
Expect(pls.Tracks[2].Path).To(Equal("downloads/newfile.flac"))
Expect(pls.Tracks[3].Path).To(Equal("tests/01 Invisible (RED) Edit Version.mp3"))
Expect(mockPlsRepo.last).To(Equal(pls))
})
It("sets the playlist name as a timestamp if the #PLAYLIST directive is not present", func() {
repo.data = []string{
"tests/fixtures/test.mp3",
"tests/fixtures/test.ogg",
"/tests/fixtures/01 Invisible (RED) Edit Version.mp3",
"tests/test.mp3",
"tests/test.ogg",
"/tests/01 Invisible (RED) Edit Version.mp3",
}
f, _ := os.Open("tests/fixtures/playlists/pls-without-name.m3u")
defer f.Close()
m3u := strings.Join([]string{
"/music/tests/test.mp3",
"/music/tests/test.ogg",
}, "\n")
f := strings.NewReader(m3u)
pls, err := ps.ImportM3U(ctx, f)
Expect(err).ToNot(HaveOccurred())
_, err = time.Parse(time.RFC3339, pls.Name)
Expect(err).ToNot(HaveOccurred())
Expect(pls.Tracks).To(HaveLen(3))
Expect(pls.Tracks).To(HaveLen(2))
})
It("returns only tracks that exist in the database and in the same other as the m3u", func() {
repo.data = []string{
"test1.mp3",
"test2.mp3",
"test3.mp3",
"album1/test1.mp3",
"album2/test2.mp3",
"album3/test3.mp3",
}
m3u := strings.Join([]string{
"test3.mp3",
"test1.mp3",
"test4.mp3",
"test2.mp3",
"/music/album3/test3.mp3",
"/music/album1/test1.mp3",
"/music/album4/test4.mp3",
"/music/album2/test2.mp3",
}, "\n")
f := strings.NewReader(m3u)
pls, err := ps.ImportM3U(ctx, f)
Expect(err).ToNot(HaveOccurred())
Expect(pls.Tracks).To(HaveLen(3))
Expect(pls.Tracks[0].Path).To(Equal("test3.mp3"))
Expect(pls.Tracks[1].Path).To(Equal("test1.mp3"))
Expect(pls.Tracks[2].Path).To(Equal("test2.mp3"))
Expect(pls.Tracks[0].Path).To(Equal("album3/test3.mp3"))
Expect(pls.Tracks[1].Path).To(Equal("album1/test1.mp3"))
Expect(pls.Tracks[2].Path).To(Equal("album2/test2.mp3"))
})
It("is case-insensitive when comparing paths", func() {
repo.data = []string{
"tEsT1.Mp3",
"abc/tEsT1.Mp3",
}
m3u := strings.Join([]string{
"TeSt1.mP3",
"/music/ABC/TeSt1.mP3",
}, "\n")
f := strings.NewReader(m3u)
pls, err := ps.ImportM3U(ctx, f)
Expect(err).ToNot(HaveOccurred())
Expect(pls.Tracks).To(HaveLen(1))
Expect(pls.Tracks[0].Path).To(Equal("tEsT1.Mp3"))
Expect(pls.Tracks[0].Path).To(Equal("abc/tEsT1.Mp3"))
})
})
Describe("InPlaylistsPath", func() {
var folder model.Folder
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
folder = model.Folder{
LibraryPath: "/music",
Path: "playlists/abc",
Name: "folder1",
}
})
It("returns true if PlaylistsPath is empty", func() {
conf.Server.PlaylistsPath = ""
Expect(InPlaylistsPath(folder)).To(BeTrue())
})
It("returns true if PlaylistsPath is any (**/**)", func() {
conf.Server.PlaylistsPath = "**/**"
Expect(InPlaylistsPath(folder)).To(BeTrue())
})
It("returns true if folder is in PlaylistsPath", func() {
conf.Server.PlaylistsPath = "other/**:playlists/**"
Expect(InPlaylistsPath(folder)).To(BeTrue())
})
It("returns false if folder is not in PlaylistsPath", func() {
conf.Server.PlaylistsPath = "other"
Expect(InPlaylistsPath(folder)).To(BeFalse())
})
It("returns true if for a playlist in root of MusicFolder if PlaylistsPath is '.'", func() {
conf.Server.PlaylistsPath = "."
Expect(InPlaylistsPath(folder)).To(BeFalse())
folder2 := model.Folder{
LibraryPath: "/music",
Path: "",
Name: ".",
}
Expect(InPlaylistsPath(folder2)).To(BeTrue())
})
})
})
@@ -192,16 +268,16 @@ func (r *mockedMediaFileFromListRepo) FindByPaths([]string) (model.MediaFiles, e
return mfs, nil
}
type mockedPlaylist struct {
type mockedPlaylistRepo struct {
last *model.Playlist
model.PlaylistRepository
}
func (r *mockedPlaylist) FindByPath(string) (*model.Playlist, error) {
func (r *mockedPlaylistRepo) FindByPath(string) (*model.Playlist, error) {
return nil, model.ErrNotFound
}
func (r *mockedPlaylist) Put(pls *model.Playlist) error {
func (r *mockedPlaylistRepo) Put(pls *model.Playlist) error {
r.last = pls
return nil
}

View File

@@ -53,18 +53,25 @@ func newPlayTracker(ds model.DataStore, broker events.Broker) *playTracker {
m := cache.NewSimpleCache[string, NowPlayingInfo]()
p := &playTracker{ds: ds, playMap: m, broker: broker}
p.scrobblers = make(map[string]Scrobbler)
var enabled []string
for name, constructor := range constructors {
s := constructor(ds)
if s == nil {
log.Debug("Scrobbler not available. Missing configuration?", "name", name)
continue
}
enabled = append(enabled, name)
if conf.Server.DevEnableBufferedScrobble {
s = newBufferedScrobbler(ds, s, name)
}
p.scrobblers[name] = s
}
log.Debug("List of scrobblers enabled", "names", enabled)
return p
}
func (p *playTracker) NowPlaying(ctx context.Context, playerId string, playerName string, trackId string) error {
mf, err := p.ds.MediaFile(ctx).Get(trackId)
mf, err := p.ds.MediaFile(ctx).GetWithParticipants(trackId)
if err != nil {
log.Error(ctx, "Error retrieving mediaFile", "id", trackId, err)
return err
@@ -124,7 +131,7 @@ func (p *playTracker) Submit(ctx context.Context, submissions []Submission) erro
success := 0
for _, s := range submissions {
mf, err := p.ds.MediaFile(ctx).Get(s.TrackID)
mf, err := p.ds.MediaFile(ctx).GetWithParticipants(s.TrackID)
if err != nil {
log.Error(ctx, "Cannot find track for scrobbling", "id", s.TrackID, "user", username, err)
continue
@@ -158,7 +165,9 @@ func (p *playTracker) incPlay(ctx context.Context, track *model.MediaFile, times
if err != nil {
return err
}
err = tx.Artist(ctx).IncPlayCount(track.ArtistID, timestamp)
for _, artist := range track.Participants[model.RoleArtist] {
err = tx.Artist(ctx).IncPlayCount(artist.ID, timestamp)
}
return err
})
}

View File

@@ -22,7 +22,8 @@ var _ = Describe("PlayTracker", func() {
var tracker PlayTracker
var track model.MediaFile
var album model.Album
var artist model.Artist
var artist1 model.Artist
var artist2 model.Artist
var fake fakeScrobbler
BeforeEach(func() {
@@ -34,9 +35,12 @@ var _ = Describe("PlayTracker", func() {
ctx = request.WithPlayer(ctx, model.Player{ScrobbleEnabled: true})
ds = &tests.MockDataStore{}
fake = fakeScrobbler{Authorized: true}
Register("fake", func(ds model.DataStore) Scrobbler {
Register("fake", func(model.DataStore) Scrobbler {
return &fake
})
Register("disabled", func(model.DataStore) Scrobbler {
return nil
})
tracker = newPlayTracker(ds, events.GetBroker())
track = model.MediaFile{
@@ -44,20 +48,27 @@ var _ = Describe("PlayTracker", func() {
Title: "Track Title",
Album: "Track Album",
AlbumID: "al-1",
Artist: "Track Artist",
ArtistID: "ar-1",
AlbumArtist: "Track AlbumArtist",
TrackNumber: 1,
Duration: 180,
MbzRecordingID: "mbz-123",
Participants: map[model.Role]model.ParticipantList{
model.RoleArtist: []model.Participant{_p("ar-1", "Artist 1"), _p("ar-2", "Artist 2")},
},
}
_ = ds.MediaFile(ctx).Put(&track)
artist = model.Artist{ID: "ar-1"}
_ = ds.Artist(ctx).Put(&artist)
artist1 = model.Artist{ID: "ar-1"}
_ = ds.Artist(ctx).Put(&artist1)
artist2 = model.Artist{ID: "ar-2"}
_ = ds.Artist(ctx).Put(&artist2)
album = model.Album{ID: "al-1"}
_ = ds.Album(ctx).(*tests.MockAlbumRepo).Put(&album)
})
It("does not register disabled scrobblers", func() {
Expect(tracker.(*playTracker).scrobblers).To(HaveKey("fake"))
Expect(tracker.(*playTracker).scrobblers).ToNot(HaveKey("disabled"))
})
Describe("NowPlaying", func() {
It("sends track to agent", func() {
err := tracker.NowPlaying(ctx, "player-1", "player-one", "123")
@@ -65,6 +76,7 @@ var _ = Describe("PlayTracker", func() {
Expect(fake.NowPlayingCalled).To(BeTrue())
Expect(fake.UserID).To(Equal("u-1"))
Expect(fake.Track.ID).To(Equal("123"))
Expect(fake.Track.Participants).To(Equal(track.Participants))
})
It("does not send track to agent if user has not authorized", func() {
fake.Authorized = false
@@ -129,6 +141,7 @@ var _ = Describe("PlayTracker", func() {
Expect(fake.ScrobbleCalled).To(BeTrue())
Expect(fake.UserID).To(Equal("u-1"))
Expect(fake.LastScrobble.ID).To(Equal("123"))
Expect(fake.LastScrobble.Participants).To(Equal(track.Participants))
})
It("increments play counts in the DB", func() {
@@ -140,7 +153,10 @@ var _ = Describe("PlayTracker", func() {
Expect(err).ToNot(HaveOccurred())
Expect(track.PlayCount).To(Equal(int64(1)))
Expect(album.PlayCount).To(Equal(int64(1)))
Expect(artist.PlayCount).To(Equal(int64(1)))
// It should increment play counts for all artists
Expect(artist1.PlayCount).To(Equal(int64(1)))
Expect(artist2.PlayCount).To(Equal(int64(1)))
})
It("does not send track to agent if user has not authorized", func() {
@@ -180,9 +196,11 @@ var _ = Describe("PlayTracker", func() {
Expect(track.PlayCount).To(Equal(int64(1)))
Expect(album.PlayCount).To(Equal(int64(1)))
Expect(artist.PlayCount).To(Equal(int64(1)))
})
// It should increment play counts for all artists
Expect(artist1.PlayCount).To(Equal(int64(1)))
Expect(artist2.PlayCount).To(Equal(int64(1)))
})
})
})
@@ -220,3 +238,11 @@ func (f *fakeScrobbler) Scrobble(ctx context.Context, userId string, s Scrobble)
f.LastScrobble = s
return nil
}
func _p(id, name string, sortName ...string) model.Participant {
p := model.Participant{Artist: model.Artist{ID: id, Name: name}}
if len(sortName) > 0 {
p.Artist.SortArtistName = sortName[0]
}
return p
}

View File

@@ -167,7 +167,10 @@ func (r *shareRepositoryWrapper) contentsLabelFromPlaylist(shareID string, id st
func (r *shareRepositoryWrapper) contentsLabelFromMediaFiles(shareID string, ids string) string {
idList := strings.Split(ids, ",")
mfs, err := r.ds.MediaFile(r.ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"id": idList}})
mfs, err := r.ds.MediaFile(r.ctx).GetAll(model.QueryOptions{Filters: squirrel.And{
squirrel.Eq{"media_file.id": idList},
squirrel.Eq{"missing": false},
}})
if err != nil {
log.Error(r.ctx, "Error retrieving media files for share", "share", shareID, err)
return ""

25
core/storage/interface.go Normal file
View File

@@ -0,0 +1,25 @@
package storage
import (
"context"
"io/fs"
"github.com/navidrome/navidrome/model/metadata"
)
type Storage interface {
FS() (MusicFS, error)
}
// MusicFS is an interface that extends the fs.FS interface with the ability to read tags from files
type MusicFS interface {
fs.FS
ReadTags(path ...string) (map[string]metadata.Info, error)
}
// Watcher is a storage with the ability watch the FS and notify changes
type Watcher interface {
// Start starts a watcher on the whole FS and returns a channel to send detected changes.
// The watcher must be stopped when the context is done.
Start(context.Context) (<-chan string, error)
}

View File

@@ -0,0 +1,29 @@
package local
import (
"io/fs"
"sync"
"github.com/navidrome/navidrome/model/metadata"
)
// Extractor is an interface that defines the methods that a tag/metadata extractor must implement
type Extractor interface {
Parse(files ...string) (map[string]metadata.Info, error)
Version() string
}
type extractorConstructor func(fs.FS, string) Extractor
var (
extractors = map[string]extractorConstructor{}
lock sync.RWMutex
)
// RegisterExtractor registers a new extractor, so it can be used by the local storage. The one to be used is
// defined with the configuration option Scanner.Extractor.
func RegisterExtractor(id string, f extractorConstructor) {
lock.Lock()
defer lock.Unlock()
extractors[id] = f
}

Some files were not shown because too many files have changed in this diff Show More