Compare commits

...

2 Commits

Author SHA1 Message Date
Deluan
fffbfc5f08 feat(db): statically compile spellfix1 SQLite extension
Vendor the spellfix1 extension source and compile it directly into the
binary via cgo, avoiding runtime shared library dependencies across
platforms. The extension is registered via sqlite3_auto_extension so
every new SQLite connection has spellfix1 available.

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-23 15:51:10 -05:00
Deluan Quintão
b59eb32961 feat(subsonic): sort search3 results by relevance (#5086)
* fix(subsonic): optimize search3 for high-cardinality FTS queries

Use a two-phase query strategy for FTS5 searches to avoid the
performance penalty of expensive LEFT JOINs (annotation, bookmark,
library) on high-cardinality results like "the".

Phase 1 runs a lightweight query (main table + FTS index only) to get
sorted, paginated rowids. Phase 2 hydrates only those few rowids with
the full JOINs, making them nearly free.

For queries with complex ORDER BY expressions that reference joined
tables (e.g. artist search sorted by play count), the optimization is
skipped and the original single-query approach is used.

* fix(search): update order by clauses to include 'rank' for FTS queries

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): reintroduce 'rank' in Phase 2 ORDER BY for FTS queries

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): remove 'rank' from ORDER BY in non-FTS queries and adjust two-phase query handling

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): update FTS ranking to use bm25 weights and simplify ORDER BY qualification

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): refine FTS query handling and improve comments for clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): refactor full-text search handling to streamline query strategy selection and improve LIKE fallback logic.

Increase e2e coverage for search3

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance FTS column definitions and relevance weights

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): refactor Search method signatures to remove offset and size parameters, streamline query handling

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): allow single-character queries in search strategies and update related tests

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): make FTS Phase 1 treat Max=0 as no limit, reorganize tests

FTS Phase 1 unconditionally called Limit(uint64(options.Max)), which
produced LIMIT 0 when Max was zero. This diverged from applyOptions
where Max=0 means no limit. Now Phase 1 mirrors applyOptions: only add
LIMIT/OFFSET when the value is positive. Also moved legacy backend
integration tests from sql_search_fts_test.go to sql_search_like_test.go
and added regression tests for the Max=0 behavior on both backends.

* refactor: simplify callSearch function by removing variadic options and directly using QueryOptions

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(search): implement ftsQueryDegraded function to detect significant content loss in FTS queries

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-23 08:51:54 -05:00
37 changed files with 4971 additions and 419 deletions

View File

@@ -117,7 +117,7 @@ jobs:
- name: Test
run: |
pkg-config --define-prefix --cflags --libs taglib # for debugging
go test -shuffle=on -tags netgo,sqlite_fts5 -race ./... -v
go test -shuffle=on -tags netgo,sqlite_fts5,sqlite_spellfix -race ./... -v
- name: Test ndpgen
run: |

View File

@@ -3,6 +3,7 @@ run:
build-tags:
- netgo
- sqlite_fts5
- sqlite_spellfix
linters:
enable:
- asasalint

View File

@@ -109,7 +109,7 @@ RUN --mount=type=bind,source=. \
export EXT=".exe"
fi
go build -tags=netgo,sqlite_fts5 -ldflags="${LD_EXTRA} -w -s \
go build -tags=netgo,sqlite_fts5,sqlite_spellfix -ldflags="${LD_EXTRA} -w -s \
-X github.com/navidrome/navidrome/consts.gitSha=${GIT_SHA} \
-X github.com/navidrome/navidrome/consts.gitTag=${GIT_TAG}" \
-o /out/navidrome${EXT} .

View File

@@ -1,6 +1,6 @@
GO_VERSION=$(shell grep "^go " go.mod | cut -f 2 -d ' ')
NODE_VERSION=$(shell cat .nvmrc)
GO_BUILD_TAGS=netgo,sqlite_fts5
GO_BUILD_TAGS=netgo,sqlite_fts5,sqlite_spellfix
# Set global environment variables, required for most targets
export CGO_CFLAGS_ALLOW=--define-prefix

View File

@@ -0,0 +1,8 @@
//go:build sqlite_spellfix
package buildtags
// SPELLFIX is required for the spellfix1 virtual table, used for fuzzy/approximate
// string matching. Without this tag, the SQLite driver won't include spellfix1 support.
var SQLITE_SPELLFIX = true

View File

@@ -10,6 +10,7 @@ import (
"github.com/mattn/go-sqlite3"
"github.com/navidrome/navidrome/conf"
_ "github.com/navidrome/navidrome/db/migrations"
_ "github.com/navidrome/navidrome/db/spellfix"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/utils/hasher"
"github.com/navidrome/navidrome/utils/singleton"

24
db/spellfix/README.md Normal file
View File

@@ -0,0 +1,24 @@
# spellfix1 SQLite Extension
This package statically compiles the [spellfix1](https://sqlite.org/spellfix1.html) SQLite extension
into the Navidrome binary. It is registered via `sqlite3_auto_extension` so that every new SQLite
connection has `spellfix1` available without loading a shared library.
## Vendored Files
The C source files are vendored because cgo cannot reference headers from other Go modules:
- **`spellfix.c`** — from the SQLite source tree: [`ext/misc/spellfix.c`](https://github.com/sqlite/sqlite/blob/master/ext/misc/spellfix.c)
- **`sqlite3ext.h`** — from the SQLite source tree: [`src/sqlite3ext.h`](https://github.com/sqlite/sqlite/blob/master/src/sqlite3ext.h)
## Updating
When upgrading `github.com/mattn/go-sqlite3`, run the update script to download
the matching spellfix1 source files for the bundled SQLite version:
```bash
./db/spellfix/update.sh
```
The script reads the SQLite version from go-sqlite3's `sqlite3-binding.h` and
downloads the corresponding files from the [SQLite GitHub mirror](https://github.com/sqlite/sqlite).

3076
db/spellfix/spellfix.c Normal file
View File

File diff suppressed because it is too large Load Diff

33
db/spellfix/spellfix.go Normal file
View File

@@ -0,0 +1,33 @@
//go:build sqlite_spellfix
package spellfix
/*
#cgo CFLAGS: -I${SRCDIR} -Wno-deprecated-declarations
// Avoid duplicate symbol conflicts with go-sqlite3.
// Rename the api pointer and entry point to unique names for this compilation unit.
#define sqlite3_api sqlite3_api_spellfix
#define sqlite3_spellfix_init sqlite3_spellfix_init_local
// Compile the extension into this binary.
// spellfix.c includes sqlite3ext.h and declares SQLITE_EXTENSION_INIT1.
#include "spellfix.c"
// sqlite3ext.h redefines sqlite3_auto_extension as a macro through the api
// struct. Undo that so we can call the real C function directly.
#undef sqlite3_auto_extension
// Provided by the SQLite library linked via go-sqlite3.
extern int sqlite3_auto_extension(void(*)(void));
// Register spellfix so it is available on every new sqlite3_open() connection.
static void register_spellfix(void) {
sqlite3_auto_extension((void(*)(void))sqlite3_spellfix_init_local);
}
*/
import "C"
func init() {
C.register_spellfix()
}

View File

@@ -0,0 +1,57 @@
//go:build sqlite_spellfix
package spellfix_test
import (
"database/sql"
"testing"
_ "github.com/mattn/go-sqlite3"
_ "github.com/navidrome/navidrome/db/spellfix"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
func TestSpellfix(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "Spellfix Suite")
}
var _ = Describe("spellfix1", func() {
var db *sql.DB
BeforeEach(func() {
var err error
db, err = sql.Open("sqlite3", ":memory:")
Expect(err).ToNot(HaveOccurred())
})
AfterEach(func() {
_ = db.Close()
})
It("creates a spellfix1 virtual table", func() {
_, err := db.Exec("CREATE VIRTUAL TABLE demo USING spellfix1")
Expect(err).ToNot(HaveOccurred())
})
It("returns fuzzy matches", func() {
_, err := db.Exec("CREATE VIRTUAL TABLE demo USING spellfix1")
Expect(err).ToNot(HaveOccurred())
_, err = db.Exec("INSERT INTO demo(word) VALUES ('hello'), ('world'), ('help')")
Expect(err).ToNot(HaveOccurred())
rows, err := db.Query("SELECT word FROM demo WHERE word MATCH 'helo' AND top=3")
Expect(err).ToNot(HaveOccurred())
defer rows.Close()
var words []string
for rows.Next() {
var word string
Expect(rows.Scan(&word)).To(Succeed())
words = append(words, word)
}
Expect(words).To(ContainElement("hello"))
})
})

730
db/spellfix/sqlite3ext.h Normal file
View File

@@ -0,0 +1,730 @@
/*
** 2006 June 7
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
*************************************************************************
** This header file defines the SQLite interface for use by
** shared libraries that want to be imported as extensions into
** an SQLite instance. Shared libraries that intend to be loaded
** as extensions by SQLite should #include this file instead of
** sqlite3.h.
*/
#ifndef SQLITE3EXT_H
#define SQLITE3EXT_H
#include "sqlite3.h"
/*
** The following structure holds pointers to all of the SQLite API
** routines.
**
** WARNING: In order to maintain backwards compatibility, add new
** interfaces to the end of this structure only. If you insert new
** interfaces in the middle of this structure, then older different
** versions of SQLite will not be able to load each other's shared
** libraries!
*/
struct sqlite3_api_routines {
void * (*aggregate_context)(sqlite3_context*,int nBytes);
int (*aggregate_count)(sqlite3_context*);
int (*bind_blob)(sqlite3_stmt*,int,const void*,int n,void(*)(void*));
int (*bind_double)(sqlite3_stmt*,int,double);
int (*bind_int)(sqlite3_stmt*,int,int);
int (*bind_int64)(sqlite3_stmt*,int,sqlite_int64);
int (*bind_null)(sqlite3_stmt*,int);
int (*bind_parameter_count)(sqlite3_stmt*);
int (*bind_parameter_index)(sqlite3_stmt*,const char*zName);
const char * (*bind_parameter_name)(sqlite3_stmt*,int);
int (*bind_text)(sqlite3_stmt*,int,const char*,int n,void(*)(void*));
int (*bind_text16)(sqlite3_stmt*,int,const void*,int,void(*)(void*));
int (*bind_value)(sqlite3_stmt*,int,const sqlite3_value*);
int (*busy_handler)(sqlite3*,int(*)(void*,int),void*);
int (*busy_timeout)(sqlite3*,int ms);
int (*changes)(sqlite3*);
int (*close)(sqlite3*);
int (*collation_needed)(sqlite3*,void*,void(*)(void*,sqlite3*,
int eTextRep,const char*));
int (*collation_needed16)(sqlite3*,void*,void(*)(void*,sqlite3*,
int eTextRep,const void*));
const void * (*column_blob)(sqlite3_stmt*,int iCol);
int (*column_bytes)(sqlite3_stmt*,int iCol);
int (*column_bytes16)(sqlite3_stmt*,int iCol);
int (*column_count)(sqlite3_stmt*pStmt);
const char * (*column_database_name)(sqlite3_stmt*,int);
const void * (*column_database_name16)(sqlite3_stmt*,int);
const char * (*column_decltype)(sqlite3_stmt*,int i);
const void * (*column_decltype16)(sqlite3_stmt*,int);
double (*column_double)(sqlite3_stmt*,int iCol);
int (*column_int)(sqlite3_stmt*,int iCol);
sqlite_int64 (*column_int64)(sqlite3_stmt*,int iCol);
const char * (*column_name)(sqlite3_stmt*,int);
const void * (*column_name16)(sqlite3_stmt*,int);
const char * (*column_origin_name)(sqlite3_stmt*,int);
const void * (*column_origin_name16)(sqlite3_stmt*,int);
const char * (*column_table_name)(sqlite3_stmt*,int);
const void * (*column_table_name16)(sqlite3_stmt*,int);
const unsigned char * (*column_text)(sqlite3_stmt*,int iCol);
const void * (*column_text16)(sqlite3_stmt*,int iCol);
int (*column_type)(sqlite3_stmt*,int iCol);
sqlite3_value* (*column_value)(sqlite3_stmt*,int iCol);
void * (*commit_hook)(sqlite3*,int(*)(void*),void*);
int (*complete)(const char*sql);
int (*complete16)(const void*sql);
int (*create_collation)(sqlite3*,const char*,int,void*,
int(*)(void*,int,const void*,int,const void*));
int (*create_collation16)(sqlite3*,const void*,int,void*,
int(*)(void*,int,const void*,int,const void*));
int (*create_function)(sqlite3*,const char*,int,int,void*,
void (*xFunc)(sqlite3_context*,int,sqlite3_value**),
void (*xStep)(sqlite3_context*,int,sqlite3_value**),
void (*xFinal)(sqlite3_context*));
int (*create_function16)(sqlite3*,const void*,int,int,void*,
void (*xFunc)(sqlite3_context*,int,sqlite3_value**),
void (*xStep)(sqlite3_context*,int,sqlite3_value**),
void (*xFinal)(sqlite3_context*));
int (*create_module)(sqlite3*,const char*,const sqlite3_module*,void*);
int (*data_count)(sqlite3_stmt*pStmt);
sqlite3 * (*db_handle)(sqlite3_stmt*);
int (*declare_vtab)(sqlite3*,const char*);
int (*enable_shared_cache)(int);
int (*errcode)(sqlite3*db);
const char * (*errmsg)(sqlite3*);
const void * (*errmsg16)(sqlite3*);
int (*exec)(sqlite3*,const char*,sqlite3_callback,void*,char**);
int (*expired)(sqlite3_stmt*);
int (*finalize)(sqlite3_stmt*pStmt);
void (*free)(void*);
void (*free_table)(char**result);
int (*get_autocommit)(sqlite3*);
void * (*get_auxdata)(sqlite3_context*,int);
int (*get_table)(sqlite3*,const char*,char***,int*,int*,char**);
int (*global_recover)(void);
void (*interruptx)(sqlite3*);
sqlite_int64 (*last_insert_rowid)(sqlite3*);
const char * (*libversion)(void);
int (*libversion_number)(void);
void *(*malloc)(int);
char * (*mprintf)(const char*,...);
int (*open)(const char*,sqlite3**);
int (*open16)(const void*,sqlite3**);
int (*prepare)(sqlite3*,const char*,int,sqlite3_stmt**,const char**);
int (*prepare16)(sqlite3*,const void*,int,sqlite3_stmt**,const void**);
void * (*profile)(sqlite3*,void(*)(void*,const char*,sqlite_uint64),void*);
void (*progress_handler)(sqlite3*,int,int(*)(void*),void*);
void *(*realloc)(void*,int);
int (*reset)(sqlite3_stmt*pStmt);
void (*result_blob)(sqlite3_context*,const void*,int,void(*)(void*));
void (*result_double)(sqlite3_context*,double);
void (*result_error)(sqlite3_context*,const char*,int);
void (*result_error16)(sqlite3_context*,const void*,int);
void (*result_int)(sqlite3_context*,int);
void (*result_int64)(sqlite3_context*,sqlite_int64);
void (*result_null)(sqlite3_context*);
void (*result_text)(sqlite3_context*,const char*,int,void(*)(void*));
void (*result_text16)(sqlite3_context*,const void*,int,void(*)(void*));
void (*result_text16be)(sqlite3_context*,const void*,int,void(*)(void*));
void (*result_text16le)(sqlite3_context*,const void*,int,void(*)(void*));
void (*result_value)(sqlite3_context*,sqlite3_value*);
void * (*rollback_hook)(sqlite3*,void(*)(void*),void*);
int (*set_authorizer)(sqlite3*,int(*)(void*,int,const char*,const char*,
const char*,const char*),void*);
void (*set_auxdata)(sqlite3_context*,int,void*,void (*)(void*));
char * (*xsnprintf)(int,char*,const char*,...);
int (*step)(sqlite3_stmt*);
int (*table_column_metadata)(sqlite3*,const char*,const char*,const char*,
char const**,char const**,int*,int*,int*);
void (*thread_cleanup)(void);
int (*total_changes)(sqlite3*);
void * (*trace)(sqlite3*,void(*xTrace)(void*,const char*),void*);
int (*transfer_bindings)(sqlite3_stmt*,sqlite3_stmt*);
void * (*update_hook)(sqlite3*,void(*)(void*,int ,char const*,char const*,
sqlite_int64),void*);
void * (*user_data)(sqlite3_context*);
const void * (*value_blob)(sqlite3_value*);
int (*value_bytes)(sqlite3_value*);
int (*value_bytes16)(sqlite3_value*);
double (*value_double)(sqlite3_value*);
int (*value_int)(sqlite3_value*);
sqlite_int64 (*value_int64)(sqlite3_value*);
int (*value_numeric_type)(sqlite3_value*);
const unsigned char * (*value_text)(sqlite3_value*);
const void * (*value_text16)(sqlite3_value*);
const void * (*value_text16be)(sqlite3_value*);
const void * (*value_text16le)(sqlite3_value*);
int (*value_type)(sqlite3_value*);
char *(*vmprintf)(const char*,va_list);
/* Added ??? */
int (*overload_function)(sqlite3*, const char *zFuncName, int nArg);
/* Added by 3.3.13 */
int (*prepare_v2)(sqlite3*,const char*,int,sqlite3_stmt**,const char**);
int (*prepare16_v2)(sqlite3*,const void*,int,sqlite3_stmt**,const void**);
int (*clear_bindings)(sqlite3_stmt*);
/* Added by 3.4.1 */
int (*create_module_v2)(sqlite3*,const char*,const sqlite3_module*,void*,
void (*xDestroy)(void *));
/* Added by 3.5.0 */
int (*bind_zeroblob)(sqlite3_stmt*,int,int);
int (*blob_bytes)(sqlite3_blob*);
int (*blob_close)(sqlite3_blob*);
int (*blob_open)(sqlite3*,const char*,const char*,const char*,sqlite3_int64,
int,sqlite3_blob**);
int (*blob_read)(sqlite3_blob*,void*,int,int);
int (*blob_write)(sqlite3_blob*,const void*,int,int);
int (*create_collation_v2)(sqlite3*,const char*,int,void*,
int(*)(void*,int,const void*,int,const void*),
void(*)(void*));
int (*file_control)(sqlite3*,const char*,int,void*);
sqlite3_int64 (*memory_highwater)(int);
sqlite3_int64 (*memory_used)(void);
sqlite3_mutex *(*mutex_alloc)(int);
void (*mutex_enter)(sqlite3_mutex*);
void (*mutex_free)(sqlite3_mutex*);
void (*mutex_leave)(sqlite3_mutex*);
int (*mutex_try)(sqlite3_mutex*);
int (*open_v2)(const char*,sqlite3**,int,const char*);
int (*release_memory)(int);
void (*result_error_nomem)(sqlite3_context*);
void (*result_error_toobig)(sqlite3_context*);
int (*sleep)(int);
void (*soft_heap_limit)(int);
sqlite3_vfs *(*vfs_find)(const char*);
int (*vfs_register)(sqlite3_vfs*,int);
int (*vfs_unregister)(sqlite3_vfs*);
int (*xthreadsafe)(void);
void (*result_zeroblob)(sqlite3_context*,int);
void (*result_error_code)(sqlite3_context*,int);
int (*test_control)(int, ...);
void (*randomness)(int,void*);
sqlite3 *(*context_db_handle)(sqlite3_context*);
int (*extended_result_codes)(sqlite3*,int);
int (*limit)(sqlite3*,int,int);
sqlite3_stmt *(*next_stmt)(sqlite3*,sqlite3_stmt*);
const char *(*sql)(sqlite3_stmt*);
int (*status)(int,int*,int*,int);
int (*backup_finish)(sqlite3_backup*);
sqlite3_backup *(*backup_init)(sqlite3*,const char*,sqlite3*,const char*);
int (*backup_pagecount)(sqlite3_backup*);
int (*backup_remaining)(sqlite3_backup*);
int (*backup_step)(sqlite3_backup*,int);
const char *(*compileoption_get)(int);
int (*compileoption_used)(const char*);
int (*create_function_v2)(sqlite3*,const char*,int,int,void*,
void (*xFunc)(sqlite3_context*,int,sqlite3_value**),
void (*xStep)(sqlite3_context*,int,sqlite3_value**),
void (*xFinal)(sqlite3_context*),
void(*xDestroy)(void*));
int (*db_config)(sqlite3*,int,...);
sqlite3_mutex *(*db_mutex)(sqlite3*);
int (*db_status)(sqlite3*,int,int*,int*,int);
int (*extended_errcode)(sqlite3*);
void (*log)(int,const char*,...);
sqlite3_int64 (*soft_heap_limit64)(sqlite3_int64);
const char *(*sourceid)(void);
int (*stmt_status)(sqlite3_stmt*,int,int);
int (*strnicmp)(const char*,const char*,int);
int (*unlock_notify)(sqlite3*,void(*)(void**,int),void*);
int (*wal_autocheckpoint)(sqlite3*,int);
int (*wal_checkpoint)(sqlite3*,const char*);
void *(*wal_hook)(sqlite3*,int(*)(void*,sqlite3*,const char*,int),void*);
int (*blob_reopen)(sqlite3_blob*,sqlite3_int64);
int (*vtab_config)(sqlite3*,int op,...);
int (*vtab_on_conflict)(sqlite3*);
/* Version 3.7.16 and later */
int (*close_v2)(sqlite3*);
const char *(*db_filename)(sqlite3*,const char*);
int (*db_readonly)(sqlite3*,const char*);
int (*db_release_memory)(sqlite3*);
const char *(*errstr)(int);
int (*stmt_busy)(sqlite3_stmt*);
int (*stmt_readonly)(sqlite3_stmt*);
int (*stricmp)(const char*,const char*);
int (*uri_boolean)(const char*,const char*,int);
sqlite3_int64 (*uri_int64)(const char*,const char*,sqlite3_int64);
const char *(*uri_parameter)(const char*,const char*);
char *(*xvsnprintf)(int,char*,const char*,va_list);
int (*wal_checkpoint_v2)(sqlite3*,const char*,int,int*,int*);
/* Version 3.8.7 and later */
int (*auto_extension)(void(*)(void));
int (*bind_blob64)(sqlite3_stmt*,int,const void*,sqlite3_uint64,
void(*)(void*));
int (*bind_text64)(sqlite3_stmt*,int,const char*,sqlite3_uint64,
void(*)(void*),unsigned char);
int (*cancel_auto_extension)(void(*)(void));
int (*load_extension)(sqlite3*,const char*,const char*,char**);
void *(*malloc64)(sqlite3_uint64);
sqlite3_uint64 (*msize)(void*);
void *(*realloc64)(void*,sqlite3_uint64);
void (*reset_auto_extension)(void);
void (*result_blob64)(sqlite3_context*,const void*,sqlite3_uint64,
void(*)(void*));
void (*result_text64)(sqlite3_context*,const char*,sqlite3_uint64,
void(*)(void*), unsigned char);
int (*strglob)(const char*,const char*);
/* Version 3.8.11 and later */
sqlite3_value *(*value_dup)(const sqlite3_value*);
void (*value_free)(sqlite3_value*);
int (*result_zeroblob64)(sqlite3_context*,sqlite3_uint64);
int (*bind_zeroblob64)(sqlite3_stmt*, int, sqlite3_uint64);
/* Version 3.9.0 and later */
unsigned int (*value_subtype)(sqlite3_value*);
void (*result_subtype)(sqlite3_context*,unsigned int);
/* Version 3.10.0 and later */
int (*status64)(int,sqlite3_int64*,sqlite3_int64*,int);
int (*strlike)(const char*,const char*,unsigned int);
int (*db_cacheflush)(sqlite3*);
/* Version 3.12.0 and later */
int (*system_errno)(sqlite3*);
/* Version 3.14.0 and later */
int (*trace_v2)(sqlite3*,unsigned,int(*)(unsigned,void*,void*,void*),void*);
char *(*expanded_sql)(sqlite3_stmt*);
/* Version 3.18.0 and later */
void (*set_last_insert_rowid)(sqlite3*,sqlite3_int64);
/* Version 3.20.0 and later */
int (*prepare_v3)(sqlite3*,const char*,int,unsigned int,
sqlite3_stmt**,const char**);
int (*prepare16_v3)(sqlite3*,const void*,int,unsigned int,
sqlite3_stmt**,const void**);
int (*bind_pointer)(sqlite3_stmt*,int,void*,const char*,void(*)(void*));
void (*result_pointer)(sqlite3_context*,void*,const char*,void(*)(void*));
void *(*value_pointer)(sqlite3_value*,const char*);
int (*vtab_nochange)(sqlite3_context*);
int (*value_nochange)(sqlite3_value*);
const char *(*vtab_collation)(sqlite3_index_info*,int);
/* Version 3.24.0 and later */
int (*keyword_count)(void);
int (*keyword_name)(int,const char**,int*);
int (*keyword_check)(const char*,int);
sqlite3_str *(*str_new)(sqlite3*);
char *(*str_finish)(sqlite3_str*);
void (*str_appendf)(sqlite3_str*, const char *zFormat, ...);
void (*str_vappendf)(sqlite3_str*, const char *zFormat, va_list);
void (*str_append)(sqlite3_str*, const char *zIn, int N);
void (*str_appendall)(sqlite3_str*, const char *zIn);
void (*str_appendchar)(sqlite3_str*, int N, char C);
void (*str_reset)(sqlite3_str*);
int (*str_errcode)(sqlite3_str*);
int (*str_length)(sqlite3_str*);
char *(*str_value)(sqlite3_str*);
/* Version 3.25.0 and later */
int (*create_window_function)(sqlite3*,const char*,int,int,void*,
void (*xStep)(sqlite3_context*,int,sqlite3_value**),
void (*xFinal)(sqlite3_context*),
void (*xValue)(sqlite3_context*),
void (*xInv)(sqlite3_context*,int,sqlite3_value**),
void(*xDestroy)(void*));
/* Version 3.26.0 and later */
const char *(*normalized_sql)(sqlite3_stmt*);
/* Version 3.28.0 and later */
int (*stmt_isexplain)(sqlite3_stmt*);
int (*value_frombind)(sqlite3_value*);
/* Version 3.30.0 and later */
int (*drop_modules)(sqlite3*,const char**);
/* Version 3.31.0 and later */
sqlite3_int64 (*hard_heap_limit64)(sqlite3_int64);
const char *(*uri_key)(const char*,int);
const char *(*filename_database)(const char*);
const char *(*filename_journal)(const char*);
const char *(*filename_wal)(const char*);
/* Version 3.32.0 and later */
const char *(*create_filename)(const char*,const char*,const char*,
int,const char**);
void (*free_filename)(const char*);
sqlite3_file *(*database_file_object)(const char*);
/* Version 3.34.0 and later */
int (*txn_state)(sqlite3*,const char*);
/* Version 3.36.1 and later */
sqlite3_int64 (*changes64)(sqlite3*);
sqlite3_int64 (*total_changes64)(sqlite3*);
/* Version 3.37.0 and later */
int (*autovacuum_pages)(sqlite3*,
unsigned int(*)(void*,const char*,unsigned int,unsigned int,unsigned int),
void*, void(*)(void*));
/* Version 3.38.0 and later */
int (*error_offset)(sqlite3*);
int (*vtab_rhs_value)(sqlite3_index_info*,int,sqlite3_value**);
int (*vtab_distinct)(sqlite3_index_info*);
int (*vtab_in)(sqlite3_index_info*,int,int);
int (*vtab_in_first)(sqlite3_value*,sqlite3_value**);
int (*vtab_in_next)(sqlite3_value*,sqlite3_value**);
/* Version 3.39.0 and later */
int (*deserialize)(sqlite3*,const char*,unsigned char*,
sqlite3_int64,sqlite3_int64,unsigned);
unsigned char *(*serialize)(sqlite3*,const char *,sqlite3_int64*,
unsigned int);
const char *(*db_name)(sqlite3*,int);
/* Version 3.40.0 and later */
int (*value_encoding)(sqlite3_value*);
/* Version 3.41.0 and later */
int (*is_interrupted)(sqlite3*);
/* Version 3.43.0 and later */
int (*stmt_explain)(sqlite3_stmt*,int);
/* Version 3.44.0 and later */
void *(*get_clientdata)(sqlite3*,const char*);
int (*set_clientdata)(sqlite3*, const char*, void*, void(*)(void*));
/* Version 3.50.0 and later */
int (*setlk_timeout)(sqlite3*,int,int);
/* Version 3.51.0 and later */
int (*set_errmsg)(sqlite3*,int,const char*);
int (*db_status64)(sqlite3*,int,sqlite3_int64*,sqlite3_int64*,int);
};
/*
** This is the function signature used for all extension entry points. It
** is also defined in the file "loadext.c".
*/
typedef int (*sqlite3_loadext_entry)(
sqlite3 *db, /* Handle to the database. */
char **pzErrMsg, /* Used to set error string on failure. */
const sqlite3_api_routines *pThunk /* Extension API function pointers. */
);
/*
** The following macros redefine the API routines so that they are
** redirected through the global sqlite3_api structure.
**
** This header file is also used by the loadext.c source file
** (part of the main SQLite library - not an extension) so that
** it can get access to the sqlite3_api_routines structure
** definition. But the main library does not want to redefine
** the API. So the redefinition macros are only valid if the
** SQLITE_CORE macros is undefined.
*/
#if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION)
#define sqlite3_aggregate_context sqlite3_api->aggregate_context
#ifndef SQLITE_OMIT_DEPRECATED
#define sqlite3_aggregate_count sqlite3_api->aggregate_count
#endif
#define sqlite3_bind_blob sqlite3_api->bind_blob
#define sqlite3_bind_double sqlite3_api->bind_double
#define sqlite3_bind_int sqlite3_api->bind_int
#define sqlite3_bind_int64 sqlite3_api->bind_int64
#define sqlite3_bind_null sqlite3_api->bind_null
#define sqlite3_bind_parameter_count sqlite3_api->bind_parameter_count
#define sqlite3_bind_parameter_index sqlite3_api->bind_parameter_index
#define sqlite3_bind_parameter_name sqlite3_api->bind_parameter_name
#define sqlite3_bind_text sqlite3_api->bind_text
#define sqlite3_bind_text16 sqlite3_api->bind_text16
#define sqlite3_bind_value sqlite3_api->bind_value
#define sqlite3_busy_handler sqlite3_api->busy_handler
#define sqlite3_busy_timeout sqlite3_api->busy_timeout
#define sqlite3_changes sqlite3_api->changes
#define sqlite3_close sqlite3_api->close
#define sqlite3_collation_needed sqlite3_api->collation_needed
#define sqlite3_collation_needed16 sqlite3_api->collation_needed16
#define sqlite3_column_blob sqlite3_api->column_blob
#define sqlite3_column_bytes sqlite3_api->column_bytes
#define sqlite3_column_bytes16 sqlite3_api->column_bytes16
#define sqlite3_column_count sqlite3_api->column_count
#define sqlite3_column_database_name sqlite3_api->column_database_name
#define sqlite3_column_database_name16 sqlite3_api->column_database_name16
#define sqlite3_column_decltype sqlite3_api->column_decltype
#define sqlite3_column_decltype16 sqlite3_api->column_decltype16
#define sqlite3_column_double sqlite3_api->column_double
#define sqlite3_column_int sqlite3_api->column_int
#define sqlite3_column_int64 sqlite3_api->column_int64
#define sqlite3_column_name sqlite3_api->column_name
#define sqlite3_column_name16 sqlite3_api->column_name16
#define sqlite3_column_origin_name sqlite3_api->column_origin_name
#define sqlite3_column_origin_name16 sqlite3_api->column_origin_name16
#define sqlite3_column_table_name sqlite3_api->column_table_name
#define sqlite3_column_table_name16 sqlite3_api->column_table_name16
#define sqlite3_column_text sqlite3_api->column_text
#define sqlite3_column_text16 sqlite3_api->column_text16
#define sqlite3_column_type sqlite3_api->column_type
#define sqlite3_column_value sqlite3_api->column_value
#define sqlite3_commit_hook sqlite3_api->commit_hook
#define sqlite3_complete sqlite3_api->complete
#define sqlite3_complete16 sqlite3_api->complete16
#define sqlite3_create_collation sqlite3_api->create_collation
#define sqlite3_create_collation16 sqlite3_api->create_collation16
#define sqlite3_create_function sqlite3_api->create_function
#define sqlite3_create_function16 sqlite3_api->create_function16
#define sqlite3_create_module sqlite3_api->create_module
#define sqlite3_create_module_v2 sqlite3_api->create_module_v2
#define sqlite3_data_count sqlite3_api->data_count
#define sqlite3_db_handle sqlite3_api->db_handle
#define sqlite3_declare_vtab sqlite3_api->declare_vtab
#define sqlite3_enable_shared_cache sqlite3_api->enable_shared_cache
#define sqlite3_errcode sqlite3_api->errcode
#define sqlite3_errmsg sqlite3_api->errmsg
#define sqlite3_errmsg16 sqlite3_api->errmsg16
#define sqlite3_exec sqlite3_api->exec
#ifndef SQLITE_OMIT_DEPRECATED
#define sqlite3_expired sqlite3_api->expired
#endif
#define sqlite3_finalize sqlite3_api->finalize
#define sqlite3_free sqlite3_api->free
#define sqlite3_free_table sqlite3_api->free_table
#define sqlite3_get_autocommit sqlite3_api->get_autocommit
#define sqlite3_get_auxdata sqlite3_api->get_auxdata
#define sqlite3_get_table sqlite3_api->get_table
#ifndef SQLITE_OMIT_DEPRECATED
#define sqlite3_global_recover sqlite3_api->global_recover
#endif
#define sqlite3_interrupt sqlite3_api->interruptx
#define sqlite3_last_insert_rowid sqlite3_api->last_insert_rowid
#define sqlite3_libversion sqlite3_api->libversion
#define sqlite3_libversion_number sqlite3_api->libversion_number
#define sqlite3_malloc sqlite3_api->malloc
#define sqlite3_mprintf sqlite3_api->mprintf
#define sqlite3_open sqlite3_api->open
#define sqlite3_open16 sqlite3_api->open16
#define sqlite3_prepare sqlite3_api->prepare
#define sqlite3_prepare16 sqlite3_api->prepare16
#define sqlite3_prepare_v2 sqlite3_api->prepare_v2
#define sqlite3_prepare16_v2 sqlite3_api->prepare16_v2
#define sqlite3_profile sqlite3_api->profile
#define sqlite3_progress_handler sqlite3_api->progress_handler
#define sqlite3_realloc sqlite3_api->realloc
#define sqlite3_reset sqlite3_api->reset
#define sqlite3_result_blob sqlite3_api->result_blob
#define sqlite3_result_double sqlite3_api->result_double
#define sqlite3_result_error sqlite3_api->result_error
#define sqlite3_result_error16 sqlite3_api->result_error16
#define sqlite3_result_int sqlite3_api->result_int
#define sqlite3_result_int64 sqlite3_api->result_int64
#define sqlite3_result_null sqlite3_api->result_null
#define sqlite3_result_text sqlite3_api->result_text
#define sqlite3_result_text16 sqlite3_api->result_text16
#define sqlite3_result_text16be sqlite3_api->result_text16be
#define sqlite3_result_text16le sqlite3_api->result_text16le
#define sqlite3_result_value sqlite3_api->result_value
#define sqlite3_rollback_hook sqlite3_api->rollback_hook
#define sqlite3_set_authorizer sqlite3_api->set_authorizer
#define sqlite3_set_auxdata sqlite3_api->set_auxdata
#define sqlite3_snprintf sqlite3_api->xsnprintf
#define sqlite3_step sqlite3_api->step
#define sqlite3_table_column_metadata sqlite3_api->table_column_metadata
#define sqlite3_thread_cleanup sqlite3_api->thread_cleanup
#define sqlite3_total_changes sqlite3_api->total_changes
#define sqlite3_trace sqlite3_api->trace
#ifndef SQLITE_OMIT_DEPRECATED
#define sqlite3_transfer_bindings sqlite3_api->transfer_bindings
#endif
#define sqlite3_update_hook sqlite3_api->update_hook
#define sqlite3_user_data sqlite3_api->user_data
#define sqlite3_value_blob sqlite3_api->value_blob
#define sqlite3_value_bytes sqlite3_api->value_bytes
#define sqlite3_value_bytes16 sqlite3_api->value_bytes16
#define sqlite3_value_double sqlite3_api->value_double
#define sqlite3_value_int sqlite3_api->value_int
#define sqlite3_value_int64 sqlite3_api->value_int64
#define sqlite3_value_numeric_type sqlite3_api->value_numeric_type
#define sqlite3_value_text sqlite3_api->value_text
#define sqlite3_value_text16 sqlite3_api->value_text16
#define sqlite3_value_text16be sqlite3_api->value_text16be
#define sqlite3_value_text16le sqlite3_api->value_text16le
#define sqlite3_value_type sqlite3_api->value_type
#define sqlite3_vmprintf sqlite3_api->vmprintf
#define sqlite3_vsnprintf sqlite3_api->xvsnprintf
#define sqlite3_overload_function sqlite3_api->overload_function
#define sqlite3_prepare_v2 sqlite3_api->prepare_v2
#define sqlite3_prepare16_v2 sqlite3_api->prepare16_v2
#define sqlite3_clear_bindings sqlite3_api->clear_bindings
#define sqlite3_bind_zeroblob sqlite3_api->bind_zeroblob
#define sqlite3_blob_bytes sqlite3_api->blob_bytes
#define sqlite3_blob_close sqlite3_api->blob_close
#define sqlite3_blob_open sqlite3_api->blob_open
#define sqlite3_blob_read sqlite3_api->blob_read
#define sqlite3_blob_write sqlite3_api->blob_write
#define sqlite3_create_collation_v2 sqlite3_api->create_collation_v2
#define sqlite3_file_control sqlite3_api->file_control
#define sqlite3_memory_highwater sqlite3_api->memory_highwater
#define sqlite3_memory_used sqlite3_api->memory_used
#define sqlite3_mutex_alloc sqlite3_api->mutex_alloc
#define sqlite3_mutex_enter sqlite3_api->mutex_enter
#define sqlite3_mutex_free sqlite3_api->mutex_free
#define sqlite3_mutex_leave sqlite3_api->mutex_leave
#define sqlite3_mutex_try sqlite3_api->mutex_try
#define sqlite3_open_v2 sqlite3_api->open_v2
#define sqlite3_release_memory sqlite3_api->release_memory
#define sqlite3_result_error_nomem sqlite3_api->result_error_nomem
#define sqlite3_result_error_toobig sqlite3_api->result_error_toobig
#define sqlite3_sleep sqlite3_api->sleep
#define sqlite3_soft_heap_limit sqlite3_api->soft_heap_limit
#define sqlite3_vfs_find sqlite3_api->vfs_find
#define sqlite3_vfs_register sqlite3_api->vfs_register
#define sqlite3_vfs_unregister sqlite3_api->vfs_unregister
#define sqlite3_threadsafe sqlite3_api->xthreadsafe
#define sqlite3_result_zeroblob sqlite3_api->result_zeroblob
#define sqlite3_result_error_code sqlite3_api->result_error_code
#define sqlite3_test_control sqlite3_api->test_control
#define sqlite3_randomness sqlite3_api->randomness
#define sqlite3_context_db_handle sqlite3_api->context_db_handle
#define sqlite3_extended_result_codes sqlite3_api->extended_result_codes
#define sqlite3_limit sqlite3_api->limit
#define sqlite3_next_stmt sqlite3_api->next_stmt
#define sqlite3_sql sqlite3_api->sql
#define sqlite3_status sqlite3_api->status
#define sqlite3_backup_finish sqlite3_api->backup_finish
#define sqlite3_backup_init sqlite3_api->backup_init
#define sqlite3_backup_pagecount sqlite3_api->backup_pagecount
#define sqlite3_backup_remaining sqlite3_api->backup_remaining
#define sqlite3_backup_step sqlite3_api->backup_step
#define sqlite3_compileoption_get sqlite3_api->compileoption_get
#define sqlite3_compileoption_used sqlite3_api->compileoption_used
#define sqlite3_create_function_v2 sqlite3_api->create_function_v2
#define sqlite3_db_config sqlite3_api->db_config
#define sqlite3_db_mutex sqlite3_api->db_mutex
#define sqlite3_db_status sqlite3_api->db_status
#define sqlite3_extended_errcode sqlite3_api->extended_errcode
#define sqlite3_log sqlite3_api->log
#define sqlite3_soft_heap_limit64 sqlite3_api->soft_heap_limit64
#define sqlite3_sourceid sqlite3_api->sourceid
#define sqlite3_stmt_status sqlite3_api->stmt_status
#define sqlite3_strnicmp sqlite3_api->strnicmp
#define sqlite3_unlock_notify sqlite3_api->unlock_notify
#define sqlite3_wal_autocheckpoint sqlite3_api->wal_autocheckpoint
#define sqlite3_wal_checkpoint sqlite3_api->wal_checkpoint
#define sqlite3_wal_hook sqlite3_api->wal_hook
#define sqlite3_blob_reopen sqlite3_api->blob_reopen
#define sqlite3_vtab_config sqlite3_api->vtab_config
#define sqlite3_vtab_on_conflict sqlite3_api->vtab_on_conflict
/* Version 3.7.16 and later */
#define sqlite3_close_v2 sqlite3_api->close_v2
#define sqlite3_db_filename sqlite3_api->db_filename
#define sqlite3_db_readonly sqlite3_api->db_readonly
#define sqlite3_db_release_memory sqlite3_api->db_release_memory
#define sqlite3_errstr sqlite3_api->errstr
#define sqlite3_stmt_busy sqlite3_api->stmt_busy
#define sqlite3_stmt_readonly sqlite3_api->stmt_readonly
#define sqlite3_stricmp sqlite3_api->stricmp
#define sqlite3_uri_boolean sqlite3_api->uri_boolean
#define sqlite3_uri_int64 sqlite3_api->uri_int64
#define sqlite3_uri_parameter sqlite3_api->uri_parameter
#define sqlite3_uri_vsnprintf sqlite3_api->xvsnprintf
#define sqlite3_wal_checkpoint_v2 sqlite3_api->wal_checkpoint_v2
/* Version 3.8.7 and later */
#define sqlite3_auto_extension sqlite3_api->auto_extension
#define sqlite3_bind_blob64 sqlite3_api->bind_blob64
#define sqlite3_bind_text64 sqlite3_api->bind_text64
#define sqlite3_cancel_auto_extension sqlite3_api->cancel_auto_extension
#define sqlite3_load_extension sqlite3_api->load_extension
#define sqlite3_malloc64 sqlite3_api->malloc64
#define sqlite3_msize sqlite3_api->msize
#define sqlite3_realloc64 sqlite3_api->realloc64
#define sqlite3_reset_auto_extension sqlite3_api->reset_auto_extension
#define sqlite3_result_blob64 sqlite3_api->result_blob64
#define sqlite3_result_text64 sqlite3_api->result_text64
#define sqlite3_strglob sqlite3_api->strglob
/* Version 3.8.11 and later */
#define sqlite3_value_dup sqlite3_api->value_dup
#define sqlite3_value_free sqlite3_api->value_free
#define sqlite3_result_zeroblob64 sqlite3_api->result_zeroblob64
#define sqlite3_bind_zeroblob64 sqlite3_api->bind_zeroblob64
/* Version 3.9.0 and later */
#define sqlite3_value_subtype sqlite3_api->value_subtype
#define sqlite3_result_subtype sqlite3_api->result_subtype
/* Version 3.10.0 and later */
#define sqlite3_status64 sqlite3_api->status64
#define sqlite3_strlike sqlite3_api->strlike
#define sqlite3_db_cacheflush sqlite3_api->db_cacheflush
/* Version 3.12.0 and later */
#define sqlite3_system_errno sqlite3_api->system_errno
/* Version 3.14.0 and later */
#define sqlite3_trace_v2 sqlite3_api->trace_v2
#define sqlite3_expanded_sql sqlite3_api->expanded_sql
/* Version 3.18.0 and later */
#define sqlite3_set_last_insert_rowid sqlite3_api->set_last_insert_rowid
/* Version 3.20.0 and later */
#define sqlite3_prepare_v3 sqlite3_api->prepare_v3
#define sqlite3_prepare16_v3 sqlite3_api->prepare16_v3
#define sqlite3_bind_pointer sqlite3_api->bind_pointer
#define sqlite3_result_pointer sqlite3_api->result_pointer
#define sqlite3_value_pointer sqlite3_api->value_pointer
/* Version 3.22.0 and later */
#define sqlite3_vtab_nochange sqlite3_api->vtab_nochange
#define sqlite3_value_nochange sqlite3_api->value_nochange
#define sqlite3_vtab_collation sqlite3_api->vtab_collation
/* Version 3.24.0 and later */
#define sqlite3_keyword_count sqlite3_api->keyword_count
#define sqlite3_keyword_name sqlite3_api->keyword_name
#define sqlite3_keyword_check sqlite3_api->keyword_check
#define sqlite3_str_new sqlite3_api->str_new
#define sqlite3_str_finish sqlite3_api->str_finish
#define sqlite3_str_appendf sqlite3_api->str_appendf
#define sqlite3_str_vappendf sqlite3_api->str_vappendf
#define sqlite3_str_append sqlite3_api->str_append
#define sqlite3_str_appendall sqlite3_api->str_appendall
#define sqlite3_str_appendchar sqlite3_api->str_appendchar
#define sqlite3_str_reset sqlite3_api->str_reset
#define sqlite3_str_errcode sqlite3_api->str_errcode
#define sqlite3_str_length sqlite3_api->str_length
#define sqlite3_str_value sqlite3_api->str_value
/* Version 3.25.0 and later */
#define sqlite3_create_window_function sqlite3_api->create_window_function
/* Version 3.26.0 and later */
#define sqlite3_normalized_sql sqlite3_api->normalized_sql
/* Version 3.28.0 and later */
#define sqlite3_stmt_isexplain sqlite3_api->stmt_isexplain
#define sqlite3_value_frombind sqlite3_api->value_frombind
/* Version 3.30.0 and later */
#define sqlite3_drop_modules sqlite3_api->drop_modules
/* Version 3.31.0 and later */
#define sqlite3_hard_heap_limit64 sqlite3_api->hard_heap_limit64
#define sqlite3_uri_key sqlite3_api->uri_key
#define sqlite3_filename_database sqlite3_api->filename_database
#define sqlite3_filename_journal sqlite3_api->filename_journal
#define sqlite3_filename_wal sqlite3_api->filename_wal
/* Version 3.32.0 and later */
#define sqlite3_create_filename sqlite3_api->create_filename
#define sqlite3_free_filename sqlite3_api->free_filename
#define sqlite3_database_file_object sqlite3_api->database_file_object
/* Version 3.34.0 and later */
#define sqlite3_txn_state sqlite3_api->txn_state
/* Version 3.36.1 and later */
#define sqlite3_changes64 sqlite3_api->changes64
#define sqlite3_total_changes64 sqlite3_api->total_changes64
/* Version 3.37.0 and later */
#define sqlite3_autovacuum_pages sqlite3_api->autovacuum_pages
/* Version 3.38.0 and later */
#define sqlite3_error_offset sqlite3_api->error_offset
#define sqlite3_vtab_rhs_value sqlite3_api->vtab_rhs_value
#define sqlite3_vtab_distinct sqlite3_api->vtab_distinct
#define sqlite3_vtab_in sqlite3_api->vtab_in
#define sqlite3_vtab_in_first sqlite3_api->vtab_in_first
#define sqlite3_vtab_in_next sqlite3_api->vtab_in_next
/* Version 3.39.0 and later */
#ifndef SQLITE_OMIT_DESERIALIZE
#define sqlite3_deserialize sqlite3_api->deserialize
#define sqlite3_serialize sqlite3_api->serialize
#endif
#define sqlite3_db_name sqlite3_api->db_name
/* Version 3.40.0 and later */
#define sqlite3_value_encoding sqlite3_api->value_encoding
/* Version 3.41.0 and later */
#define sqlite3_is_interrupted sqlite3_api->is_interrupted
/* Version 3.43.0 and later */
#define sqlite3_stmt_explain sqlite3_api->stmt_explain
/* Version 3.44.0 and later */
#define sqlite3_get_clientdata sqlite3_api->get_clientdata
#define sqlite3_set_clientdata sqlite3_api->set_clientdata
/* Version 3.50.0 and later */
#define sqlite3_setlk_timeout sqlite3_api->setlk_timeout
/* Version 3.51.0 and later */
#define sqlite3_set_errmsg sqlite3_api->set_errmsg
#define sqlite3_db_status64 sqlite3_api->db_status64
#endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */
#if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION)
/* This case when the file really is being compiled as a loadable
** extension */
# define SQLITE_EXTENSION_INIT1 const sqlite3_api_routines *sqlite3_api=0;
# define SQLITE_EXTENSION_INIT2(v) sqlite3_api=v;
# define SQLITE_EXTENSION_INIT3 \
extern const sqlite3_api_routines *sqlite3_api;
#else
/* This case when the file is being statically linked into the
** application */
# define SQLITE_EXTENSION_INIT1 /*no-op*/
# define SQLITE_EXTENSION_INIT2(v) (void)v; /* unused parameter */
# define SQLITE_EXTENSION_INIT3 /*no-op*/
#endif
#endif /* SQLITE3EXT_H */

31
db/spellfix/update.sh Executable file
View File

@@ -0,0 +1,31 @@
#!/usr/bin/env bash
#
# Updates the vendored spellfix1 source files to match the SQLite version
# bundled with the current go-sqlite3 dependency.
#
set -euo pipefail
cd "$(dirname "$0")"
SQLITE_VERSION=$(grep '#define SQLITE_VERSION ' \
"$(go env GOMODCACHE)/$(go list -m -f '{{.Path}}@{{.Version}}' github.com/mattn/go-sqlite3)/sqlite3-binding.h" \
| awk '{gsub(/"/, "", $3); print $3}')
if [ -z "$SQLITE_VERSION" ]; then
echo "ERROR: Could not determine SQLite version from go-sqlite3" >&2
exit 1
fi
TAG="version-${SQLITE_VERSION}"
BASE_URL="https://raw.githubusercontent.com/sqlite/sqlite/${TAG}"
echo "SQLite version from go-sqlite3: ${SQLITE_VERSION}"
echo "Downloading from tag: ${TAG}"
curl -sfL "${BASE_URL}/ext/misc/spellfix.c" -o spellfix.c
echo " Updated spellfix.c"
curl -sfL "${BASE_URL}/src/sqlite3ext.h" -o sqlite3ext.h
echo " Updated sqlite3ext.h"
echo "Done."

View File

@@ -15,6 +15,7 @@ func main() {
// To avoid these kind of errors, you should use `make build` to compile the project.
_ = buildtags.NETGO
_ = buildtags.SQLITE_FTS5
_ = buildtags.SQLITE_SPELLFIX
cmd.Execute()
}

View File

@@ -1,5 +1,5 @@
package model
type SearchableRepository[T any] interface {
Search(q string, offset, size int, options ...QueryOptions) (T, error)
Search(q string, options ...QueryOptions) (T, error)
}

View File

@@ -12,7 +12,6 @@ import (
. "github.com/Masterminds/squirrel"
"github.com/deluan/rest"
"github.com/google/uuid"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
@@ -353,18 +352,21 @@ func (r *albumRepository) purgeEmpty(libraryIDs ...int) error {
return nil
}
func (r *albumRepository) Search(q string, offset int, size int, options ...model.QueryOptions) (model.Albums, error) {
var albumSearchConfig = searchConfig{
NaturalOrder: "album.rowid",
OrderBy: []string{"name"},
MBIDFields: []string{"mbz_album_id", "mbz_release_group_id"},
}
func (r *albumRepository) Search(q string, options ...model.QueryOptions) (model.Albums, error) {
var opts model.QueryOptions
if len(options) > 0 {
opts = options[0]
}
var res dbAlbums
if uuid.Validate(q) == nil {
err := r.searchByMBID(r.selectAlbum(options...), q, []string{"mbz_album_id", "mbz_release_group_id"}, &res)
if err != nil {
return nil, fmt.Errorf("searching album by MBID %q: %w", q, err)
}
} else {
err := r.doSearch(r.selectAlbum(options...), q, offset, size, &res, "album.rowid", "name")
if err != nil {
return nil, fmt.Errorf("searching album by query %q: %w", q, err)
}
err := r.doSearch(r.selectAlbum(options...), q, &res, albumSearchConfig, opts)
if err != nil {
return nil, fmt.Errorf("searching album %q: %w", q, err)
}
return res.toModels(), nil
}

View File

@@ -11,7 +11,6 @@ import (
. "github.com/Masterminds/squirrel"
"github.com/deluan/rest"
"github.com/google/uuid"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
@@ -513,20 +512,25 @@ func (r *artistRepository) RefreshStats(allArtists bool) (int64, error) {
return totalRowsAffected, nil
}
func (r *artistRepository) Search(q string, offset int, size int, options ...model.QueryOptions) (model.Artists, error) {
var res dbArtists
if uuid.Validate(q) == nil {
err := r.searchByMBID(r.selectArtist(options...), q, []string{"mbz_artist_id"}, &res)
if err != nil {
return nil, fmt.Errorf("searching artist by MBID %q: %w", q, err)
}
} else {
func (r *artistRepository) searchCfg() searchConfig {
return searchConfig{
// Natural order for artists is more performant by ID, due to GROUP BY clause in selectArtist
err := r.doSearch(r.selectArtist(options...), q, offset, size, &res, "artist.id",
"sum(json_extract(stats, '$.total.m')) desc", "name")
if err != nil {
return nil, fmt.Errorf("searching artist by query %q: %w", q, err)
}
NaturalOrder: "artist.id",
OrderBy: []string{"sum(json_extract(stats, '$.total.m')) desc", "name"},
MBIDFields: []string{"mbz_artist_id"},
LibraryFilter: r.applyLibraryFilterToArtistQuery,
}
}
func (r *artistRepository) Search(q string, options ...model.QueryOptions) (model.Artists, error) {
var opts model.QueryOptions
if len(options) > 0 {
opts = options[0]
}
var res dbArtists
err := r.doSearch(r.selectArtist(options...), q, &res, r.searchCfg(), opts)
if err != nil {
return nil, fmt.Errorf("searching artist %q: %w", q, err)
}
return res.toModels(), nil
}

View File

@@ -512,7 +512,7 @@ var _ = Describe("ArtistRepository", func() {
Expect(err).ToNot(HaveOccurred())
// Test the search
results, err := (*testRepo).Search("550e8400-e29b-41d4-a716-446655440010", 0, 10)
results, err := (*testRepo).Search("550e8400-e29b-41d4-a716-446655440010", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
if shouldFind {
@@ -543,12 +543,12 @@ var _ = Describe("ArtistRepository", func() {
Expect(err).ToNot(HaveOccurred())
// Restricted user should not find this artist
results, err := restrictedRepo.Search("a74b1b7f-71a5-4011-9441-d0b5e4122711", 0, 10)
results, err := restrictedRepo.Search("a74b1b7f-71a5-4011-9441-d0b5e4122711", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())
// But admin should find it
results, err = repo.Search("a74b1b7f-71a5-4011-9441-d0b5e4122711", 0, 10)
results, err = repo.Search("a74b1b7f-71a5-4011-9441-d0b5e4122711", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
@@ -560,7 +560,7 @@ var _ = Describe("ArtistRepository", func() {
Context("Text Search", func() {
It("allows admin to find artists by name regardless of library", func() {
results, err := repo.Search("Beatles", 0, 10)
results, err := repo.Search("Beatles", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Name).To(Equal("The Beatles"))
@@ -580,7 +580,7 @@ var _ = Describe("ArtistRepository", func() {
Expect(err).ToNot(HaveOccurred())
// Restricted user should not find this artist
results, err := restrictedRepo.Search("Unique Search Name", 0, 10)
results, err := restrictedRepo.Search("Unique Search Name", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty(), "Text search should respect library filtering")
@@ -688,7 +688,7 @@ var _ = Describe("ArtistRepository", func() {
Expect(artists).To(HaveLen(5)) // Including the missing artist
// Search never returns missing artists (hardcoded behavior)
results, err := repo.Search("Missing Artist", 0, 10)
results, err := repo.Search("Missing Artist", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())
})
@@ -742,11 +742,11 @@ var _ = Describe("ArtistRepository", func() {
})
It("Search returns empty results for users without library access", func() {
results, err := restrictedRepo.Search("Beatles", 0, 10)
results, err := restrictedRepo.Search("Beatles", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())
results, err = restrictedRepo.Search("Kraftwerk", 0, 10)
results, err = restrictedRepo.Search("Kraftwerk", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())
})

View File

@@ -11,7 +11,6 @@ import (
. "github.com/Masterminds/squirrel"
"github.com/deluan/rest"
"github.com/google/uuid"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
@@ -428,18 +427,21 @@ func (r *mediaFileRepository) FindRecentFilesByProperties(missing model.MediaFil
return res.toModels(), nil
}
func (r *mediaFileRepository) Search(q string, offset int, size int, options ...model.QueryOptions) (model.MediaFiles, error) {
var mediaFileSearchConfig = searchConfig{
NaturalOrder: "media_file.rowid",
OrderBy: []string{"title"},
MBIDFields: []string{"mbz_recording_id", "mbz_release_track_id"},
}
func (r *mediaFileRepository) Search(q string, options ...model.QueryOptions) (model.MediaFiles, error) {
var opts model.QueryOptions
if len(options) > 0 {
opts = options[0]
}
var res dbMediaFiles
if uuid.Validate(q) == nil {
err := r.searchByMBID(r.selectMediaFile(options...), q, []string{"mbz_recording_id", "mbz_release_track_id"}, &res)
if err != nil {
return nil, fmt.Errorf("searching media_file by MBID %q: %w", q, err)
}
} else {
err := r.doSearch(r.selectMediaFile(options...), q, offset, size, &res, "media_file.rowid", "title")
if err != nil {
return nil, fmt.Errorf("searching media_file by query %q: %w", q, err)
}
err := r.doSearch(r.selectMediaFile(options...), q, &res, mediaFileSearchConfig, opts)
if err != nil {
return nil, fmt.Errorf("searching media_file %q: %w", q, err)
}
return res.toModels(), nil
}

View File

@@ -527,7 +527,7 @@ var _ = Describe("MediaRepository", func() {
Describe("Search", func() {
Context("text search", func() {
It("finds media files by title", func() {
results, err := mr.Search("Antenna", 0, 10)
results, err := mr.Search("Antenna", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(3)) // songAntenna, songAntennaWithLyrics, songAntenna2
for _, result := range results {
@@ -536,7 +536,7 @@ var _ = Describe("MediaRepository", func() {
})
It("finds media files case insensitively", func() {
results, err := mr.Search("antenna", 0, 10)
results, err := mr.Search("antenna", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(3))
for _, result := range results {
@@ -545,7 +545,7 @@ var _ = Describe("MediaRepository", func() {
})
It("returns empty result when no matches found", func() {
results, err := mr.Search("nonexistent", 0, 10)
results, err := mr.Search("nonexistent", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())
})
@@ -578,7 +578,7 @@ var _ = Describe("MediaRepository", func() {
})
It("finds media file by mbz_recording_id", func() {
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440020", 0, 10)
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440020", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].ID).To(Equal("test-mbid-mediafile"))
@@ -586,7 +586,7 @@ var _ = Describe("MediaRepository", func() {
})
It("finds media file by mbz_release_track_id", func() {
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440021", 0, 10)
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440021", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].ID).To(Equal("test-mbid-mediafile"))
@@ -594,7 +594,7 @@ var _ = Describe("MediaRepository", func() {
})
It("returns empty result when MBID is not found", func() {
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440099", 0, 10)
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440099", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())
})
@@ -614,7 +614,7 @@ var _ = Describe("MediaRepository", func() {
Expect(err).ToNot(HaveOccurred())
// Search never returns missing media files (hardcoded behavior)
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440022", 0, 10)
results, err := mr.Search("550e8400-e29b-41d4-a716-446655440022", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty())

View File

@@ -109,12 +109,10 @@ func booleanFilter(field string, value any) Sqlizer {
func fullTextFilter(tableName string, mbidFields ...string) func(string, any) Sqlizer {
return func(field string, value any) Sqlizer {
v := strings.ToLower(value.(string))
searchExpr := getSearchExpr()
cond := cmp.Or(
return cmp.Or[Sqlizer](
mbidExpr(tableName, v, mbidFields...),
searchExpr(tableName, v),
getSearchStrategy(tableName, v),
)
return cond
}
}

View File

@@ -102,11 +102,11 @@ var _ = Describe("sqlRestful", func() {
uuid := "550e8400-e29b-41d4-a716-446655440000"
result := noMbidFilter("search", uuid)
// mbidExpr with no fields returns nil, so cmp.Or falls back to fullTextExpr
expected := squirrel.And{
squirrel.Like{"test_table.full_text": "% 550e8400-e29b-41d4-a716-446655440000%"},
}
Expect(result).To(Equal(expected))
// mbidExpr with no fields returns nil, so cmp.Or falls back to search strategy
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(ContainElement("% 550e8400-e29b-41d4-a716-446655440000%"))
})
})
@@ -114,26 +114,25 @@ var _ = Describe("sqlRestful", func() {
It("returns full text search condition only", func() {
result := filter("search", "beatles")
// mbidExpr returns nil for non-UUIDs, so fullTextExpr result is returned directly
expected := squirrel.And{
squirrel.Like{"test_table.full_text": "% beatles%"},
}
Expect(result).To(Equal(expected))
// mbidExpr returns nil for non-UUIDs, so search strategy result is returned directly
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(ContainElement("% beatles%"))
})
It("handles multi-word search terms", func() {
result := filter("search", "the beatles abbey road")
// Should return And condition directly
andCondition, ok := result.(squirrel.And)
Expect(ok).To(BeTrue())
Expect(andCondition).To(HaveLen(4))
// Check that all words are present (order may vary)
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% the%"}))
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% beatles%"}))
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% abbey%"}))
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% road%"}))
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
// All words should be present as LIKE conditions
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(HaveLen(4))
Expect(args).To(ContainElement("% the%"))
Expect(args).To(ContainElement("% beatles%"))
Expect(args).To(ContainElement("% abbey%"))
Expect(args).To(ContainElement("% road%"))
})
})
@@ -142,26 +141,48 @@ var _ = Describe("sqlRestful", func() {
conf.Server.Search.FullString = false
result := filter("search", "test query")
andCondition, ok := result.(squirrel.And)
Expect(ok).To(BeTrue())
Expect(andCondition).To(HaveLen(2))
// Check that all words are present with leading space (order may vary)
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% test%"}))
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% query%"}))
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(HaveLen(2))
Expect(args).To(ContainElement("% test%"))
Expect(args).To(ContainElement("% query%"))
})
It("uses no separator with SearchFullString=true", func() {
conf.Server.Search.FullString = true
result := filter("search", "test query")
andCondition, ok := result.(squirrel.And)
Expect(ok).To(BeTrue())
Expect(andCondition).To(HaveLen(2))
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(HaveLen(2))
Expect(args).To(ContainElement("%test%"))
Expect(args).To(ContainElement("%query%"))
})
})
// Check that all words are present without leading space (order may vary)
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "%test%"}))
Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "%query%"}))
Context("single-character queries (regression: must not be rejected)", func() {
It("returns valid filter for single-char query with legacy backend", func() {
conf.Server.Search.Backend = "legacy"
result := filter("search", "a")
Expect(result).ToNot(BeNil(), "single-char REST filter must not be dropped")
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("LIKE"))
Expect(args).ToNot(BeEmpty())
})
It("returns valid filter for single-char query with FTS backend", func() {
conf.Server.Search.Backend = "fts"
conf.Server.Search.FullString = false
ftsFilter := fullTextFilter(tableName, mbidFields...)
result := ftsFilter("search", "a")
Expect(result).ToNot(BeNil(), "single-char REST filter must not be dropped")
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("MATCH"))
Expect(args).ToNot(BeEmpty())
})
})
@@ -179,10 +200,10 @@ var _ = Describe("sqlRestful", func() {
It("handles special characters that are sanitized", func() {
result := filter("search", "don't")
expected := squirrel.And{
squirrel.Like{"test_table.full_text": "% dont%"}, // str.SanitizeStrings removes quotes
}
Expect(result).To(Equal(expected))
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(ContainElement("% dont%"))
})
It("returns nil for single quote (SQL injection protection)", func() {
@@ -206,31 +227,30 @@ var _ = Describe("sqlRestful", func() {
result := filter("search", "550e8400-invalid-uuid")
// Should return full text filter since UUID is invalid
expected := squirrel.And{
squirrel.Like{"test_table.full_text": "% 550e8400-invalid-uuid%"},
}
Expect(result).To(Equal(expected))
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(ContainElement("% 550e8400-invalid-uuid%"))
})
It("handles empty mbid fields array", func() {
emptyMbidFilter := fullTextFilter(tableName, []string{}...)
result := emptyMbidFilter("search", "test")
// mbidExpr with empty fields returns nil, so cmp.Or falls back to fullTextExpr
expected := squirrel.And{
squirrel.Like{"test_table.full_text": "% test%"},
}
Expect(result).To(Equal(expected))
// mbidExpr with empty fields returns nil, so search strategy result is returned directly
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(ContainElement("% test%"))
})
It("converts value to lowercase before processing", func() {
result := filter("search", "TEST")
// The function converts to lowercase internally
expected := squirrel.And{
squirrel.Like{"test_table.full_text": "% test%"},
}
Expect(result).To(Equal(expected))
sql, args, err := result.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("test_table.full_text LIKE"))
Expect(args).To(ContainElement("% test%"))
})
})
})

View File

@@ -6,7 +6,6 @@ import (
. "github.com/Masterminds/squirrel"
"github.com/google/uuid"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/str"
)
@@ -16,57 +15,71 @@ func formatFullText(text ...string) string {
return " " + fullText
}
// searchExprFunc is the function signature for search expression builders.
type searchExprFunc func(tableName string, query string) Sqlizer
// getSearchExpr returns the active search expression function based on config.
// It falls back to legacySearchExpr when Search.FullString is enabled, because
// FTS5 is token-based and cannot match substrings within words.
// CJK queries are routed to likeSearchExpr, since FTS5's unicode61 tokenizer
// cannot segment CJK text.
func getSearchExpr() searchExprFunc {
if conf.Server.Search.Backend == "legacy" || conf.Server.Search.FullString {
return legacySearchExpr
}
return func(tableName, query string) Sqlizer {
if containsCJK(query) {
return likeSearchExpr(tableName, query)
}
return ftsSearchExpr(tableName, query)
}
// searchConfig holds per-repository constants for doSearch.
type searchConfig struct {
NaturalOrder string // ORDER BY for empty-query results (e.g. "album.rowid")
OrderBy []string // ORDER BY for text search results (e.g. ["name"])
MBIDFields []string // columns to match when query is a UUID
// LibraryFilter overrides the default applyLibraryFilter for FTS Phase 1.
// Needed when library access requires a junction table (e.g. artist → library_artist).
LibraryFilter func(sq SelectBuilder) SelectBuilder
}
// doSearch performs a full-text search with the specified parameters.
// The naturalOrder is used to sort results when no full-text filter is applied. It is useful for cases like
// OpenSubsonic, where an empty search query should return all results in a natural order. Normally the parameter
// should be `tableName + ".rowid"`, but some repositories (ex: artist) may use a different natural order.
func (r sqlRepository) doSearch(sq SelectBuilder, q string, offset, size int, results any, naturalOrder string, orderBys ...string) error {
// searchStrategy defines how to execute a text search against a repository table.
// options carries filters and pagination that must reach all query phases,
// including FTS Phase 1 which builds its own query outside sq.
type searchStrategy interface {
Sqlizer
execute(r sqlRepository, sq SelectBuilder, dest any, cfg searchConfig, options model.QueryOptions) error
}
// getSearchStrategy returns the appropriate search strategy based on config and query content.
// Returns nil when the query produces no searchable tokens.
func getSearchStrategy(tableName, query string) searchStrategy {
if conf.Server.Search.Backend == "legacy" || conf.Server.Search.FullString {
return newLegacySearch(tableName, query)
}
if containsCJK(query) {
return newLikeSearch(tableName, query)
}
return newFTSSearch(tableName, query)
}
// doSearch dispatches a search query: empty → natural order, UUID → MBID match,
// otherwise delegates to getSearchStrategy. sq must already have LIMIT/OFFSET set
// via newSelect(options...). options is forwarded so FTS Phase 1 can apply the same
// filters and pagination independently.
func (r sqlRepository) doSearch(sq SelectBuilder, q string, results any, cfg searchConfig, options model.QueryOptions) error {
q = strings.TrimSpace(q)
q = strings.TrimSuffix(q, "*")
sq = sq.Where(Eq{r.tableName + ".missing": false})
// Empty query (OpenSubsonic `search3?query=""`) — return all in natural order.
if q == "" || q == `""` {
sq = sq.OrderBy(cfg.NaturalOrder)
return r.queryAll(sq, results, options)
}
// MBID search: if query is a valid UUID, search by MBID fields instead
if uuid.Validate(q) == nil && len(cfg.MBIDFields) > 0 {
sq = sq.Where(mbidExpr(r.tableName, q, cfg.MBIDFields...))
return r.queryAll(sq, results)
}
// Min-length guard: single-character queries are too broad for search3.
// This check lives here (not in the strategies) so that fullTextFilter
// (REST filter path) can still use single-character queries.
if len(q) < 2 {
return nil
}
searchExpr := getSearchExpr()
filter := searchExpr(r.tableName, q)
if filter != nil {
sq = sq.Where(filter)
sq = sq.OrderBy(orderBys...)
} else {
// This is to speed up the results of `search3?query=""`, for OpenSubsonic
// If the filter is empty, we sort by the specified natural order.
sq = sq.OrderBy(naturalOrder)
strategy := getSearchStrategy(r.tableName, q)
if strategy == nil {
return nil
}
sq = sq.Where(Eq{r.tableName + ".missing": false})
sq = sq.Limit(uint64(size)).Offset(uint64(offset))
return r.queryAll(sq, results, model.QueryOptions{Offset: offset})
}
func (r sqlRepository) searchByMBID(sq SelectBuilder, mbid string, mbidFields []string, results any) error {
sq = sq.Where(mbidExpr(r.tableName, mbid, mbidFields...))
sq = sq.Where(Eq{r.tableName + ".missing": false})
return r.queryAll(sq, results)
return strategy.execute(r, sq, results, cfg, options)
}
func mbidExpr(tableName, mbid string, mbidFields ...string) Sqlizer {
@@ -80,24 +93,3 @@ func mbidExpr(tableName, mbid string, mbidFields ...string) Sqlizer {
}
return Or(cond)
}
// legacySearchExpr generates LIKE-based search filters against the full_text column.
// This is the original search implementation, used when Search.Backend="legacy".
func legacySearchExpr(tableName string, s string) Sqlizer {
q := str.SanitizeStrings(s)
if q == "" {
log.Trace("Search using legacy backend, query is empty", "table", tableName)
return nil
}
var sep string
if !conf.Server.Search.FullString {
sep = " "
}
parts := strings.Split(q, " ")
filters := And{}
for _, part := range parts {
filters = append(filters, Like{tableName + ".full_text": "%" + sep + part + "%"})
}
log.Trace("Search using legacy backend", "query", filters, "table", tableName)
return filters
}

View File

@@ -9,6 +9,7 @@ import (
. "github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
)
// containsCJK returns true if the string contains any CJK (Chinese/Japanese/Korean) characters.
@@ -187,75 +188,235 @@ func buildFTS5Query(userInput string) string {
return result
}
// likeSearchColumns defines the core columns to search with LIKE queries.
// These are the primary user-visible fields for each entity type.
// Used as a fallback when FTS5 cannot handle the query (e.g., CJK text, punctuation-only input).
var likeSearchColumns = map[string][]string{
"media_file": {"title", "album", "artist", "album_artist"},
"album": {"name", "album_artist"},
"artist": {"name"},
// ftsColumn pairs an FTS5 column name with its BM25 relevance weight.
type ftsColumn struct {
Name string
Weight float64
}
// likeSearchExpr generates LIKE-based search filters against core columns.
// Each word in the query must match at least one column (AND between words),
// and each word can match any column (OR within a word).
// Used as a fallback when FTS5 cannot handle the query (e.g., CJK text, punctuation-only input).
func likeSearchExpr(tableName string, s string) Sqlizer {
s = strings.TrimSpace(s)
if s == "" {
log.Trace("Search using LIKE backend, query is empty", "table", tableName)
return nil
}
columns, ok := likeSearchColumns[tableName]
if !ok {
log.Trace("Search using LIKE backend, couldn't find columns for this table", "table", tableName)
return nil
}
words := strings.Fields(s)
wordFilters := And{}
for _, word := range words {
colFilters := Or{}
for _, col := range columns {
colFilters = append(colFilters, Like{tableName + "." + col: "%" + word + "%"})
// ftsColumnDefs defines FTS5 columns and their BM25 relevance weights.
// The order MUST match the column order in the FTS5 table definition (see migrations).
// All columns are both searched and ranked. When adding indexed-but-not-searched
// columns in the future, use Weight: 0 to exclude from the search column filter.
var ftsColumnDefs = map[string][]ftsColumn{
"media_file": {
{"title", 10.0},
{"album", 5.0},
{"artist", 3.0},
{"album_artist", 3.0},
{"sort_title", 1.0},
{"sort_album_name", 1.0},
{"sort_artist_name", 1.0},
{"sort_album_artist_name", 1.0},
{"disc_subtitle", 1.0},
{"search_participants", 2.0},
{"search_normalized", 1.0},
},
"album": {
{"name", 10.0},
{"sort_album_name", 1.0},
{"album_artist", 3.0},
{"search_participants", 2.0},
{"discs", 1.0},
{"catalog_num", 1.0},
{"album_version", 1.0},
{"search_normalized", 1.0},
},
"artist": {
{"name", 10.0},
{"sort_artist_name", 1.0},
{"search_normalized", 1.0},
},
}
// ftsColumnFilters and ftsBM25Weights are precomputed from ftsColumnDefs at init time
// to avoid per-query allocations.
var (
ftsColumnFilters = map[string]string{}
ftsBM25Weights = map[string]string{}
)
func init() {
for table, cols := range ftsColumnDefs {
var names []string
weights := make([]string, len(cols))
for i, c := range cols {
if c.Weight > 0 {
names = append(names, c.Name)
}
weights[i] = fmt.Sprintf("%.1f", c.Weight)
}
wordFilters = append(wordFilters, colFilters)
ftsColumnFilters[table] = "{" + strings.Join(names, " ") + "}"
ftsBM25Weights[table] = strings.Join(weights, ", ")
}
log.Trace("Search using LIKE backend", "query", wordFilters, "table", tableName)
return wordFilters
}
// ftsSearchColumns defines which FTS5 columns are included in general search.
// Columns not listed here are indexed but not searched by default,
// enabling future additions (comments, lyrics, bios) without affecting general search.
var ftsSearchColumns = map[string]string{
"media_file": "{title album artist album_artist sort_title sort_album_name sort_artist_name sort_album_artist_name disc_subtitle search_participants search_normalized}",
"album": "{name sort_album_name album_artist search_participants discs catalog_num album_version search_normalized}",
"artist": "{name sort_artist_name search_normalized}",
// ftsSearch implements searchStrategy using FTS5 full-text search with BM25 ranking.
type ftsSearch struct {
tableName string
ftsTable string
matchExpr string
rankExpr string
}
// ftsSearchExpr generates an FTS5 MATCH-based search filter.
// If the query produces no FTS tokens (e.g., punctuation-only like "!!!!!!!"),
// it falls back to LIKE-based search.
func ftsSearchExpr(tableName string, s string) Sqlizer {
q := buildFTS5Query(s)
if q == "" {
s = strings.TrimSpace(strings.ReplaceAll(s, `"`, ""))
if s != "" {
log.Trace("Search using LIKE fallback for non-tokenizable query", "table", tableName, "query", s)
return likeSearchExpr(tableName, s)
// ToSql returns a single-query fallback for the REST filter path (no two-phase split).
func (s *ftsSearch) ToSql() (string, []interface{}, error) {
sql := s.tableName + ".rowid IN (SELECT rowid FROM " + s.ftsTable + " WHERE " + s.ftsTable + " MATCH ?)"
return sql, []interface{}{s.matchExpr}, nil
}
// execute runs a two-phase FTS5 search:
// - Phase 1: lightweight rowid query (main table + FTS + library filter) for ranking and pagination.
// - Phase 2: full SELECT with all JOINs, scoped to Phase 1's rowid set.
//
// Complex ORDER BY (function calls, aggregations) are dropped from Phase 1.
func (s *ftsSearch) execute(r sqlRepository, sq SelectBuilder, dest any, cfg searchConfig, options model.QueryOptions) error {
qualifiedOrderBys := []string{s.rankExpr}
for _, ob := range cfg.OrderBy {
if qualified := qualifyOrderBy(s.tableName, ob); qualified != "" {
qualifiedOrderBys = append(qualifiedOrderBys, qualified)
}
}
// Phase 1: fresh query — must set LIMIT/OFFSET from options explicitly.
// Mirror applyOptions behavior: Max=0 means no limit, not LIMIT 0.
rowidQuery := Select(s.tableName+".rowid").
From(s.tableName).
Join(s.ftsTable+" ON "+s.ftsTable+".rowid = "+s.tableName+".rowid AND "+s.ftsTable+" MATCH ?", s.matchExpr).
Where(Eq{s.tableName + ".missing": false}).
OrderBy(qualifiedOrderBys...)
if options.Max > 0 {
rowidQuery = rowidQuery.Limit(uint64(options.Max))
}
if options.Offset > 0 {
rowidQuery = rowidQuery.Offset(uint64(options.Offset))
}
// Library filter + musicFolderId must be applied here, before pagination.
if cfg.LibraryFilter != nil {
rowidQuery = cfg.LibraryFilter(rowidQuery)
} else {
rowidQuery = r.applyLibraryFilter(rowidQuery)
}
if options.Filters != nil {
rowidQuery = rowidQuery.Where(options.Filters)
}
rowidSQL, rowidArgs, err := rowidQuery.ToSql()
if err != nil {
return fmt.Errorf("building FTS rowid query: %w", err)
}
// Phase 2: strip LIMIT/OFFSET from sq (Phase 1 handled pagination),
// join on the ranked rowid set to hydrate with full columns.
sq = sq.RemoveLimit().RemoveOffset()
rankedSubquery := fmt.Sprintf(
"(SELECT rowid as _rid, row_number() OVER () AS _rn FROM (%s)) AS _ranked",
rowidSQL,
)
sq = sq.Join(rankedSubquery+" ON "+s.tableName+".rowid = _ranked._rid", rowidArgs...)
sq = sq.OrderBy("_ranked._rn")
return r.queryAll(sq, dest)
}
// qualifyOrderBy prepends tableName to a simple column name. Returns empty string for
// complex expressions (function calls, aggregations) that can't be used in Phase 1.
func qualifyOrderBy(tableName, orderBy string) string {
orderBy = strings.TrimSpace(orderBy)
if orderBy == "" || strings.ContainsAny(orderBy, "(,") {
return ""
}
parts := strings.Fields(orderBy)
if !strings.Contains(parts[0], ".") {
parts[0] = tableName + "." + parts[0]
}
return strings.Join(parts, " ")
}
// ftsQueryDegraded returns true when the FTS query lost significant discriminating
// content compared to the original input. This happens when special characters that
// are part of the entity name (e.g., "1+", "C++", "!!!", "C#") get stripped by FTS
// tokenization, leaving only very short/broad tokens. Also detects quoted phrases
// that would be degraded by FTS5's unicode61 tokenizer (e.g., "1+" → token "1").
func ftsQueryDegraded(original, ftsQuery string) bool {
original = strings.TrimSpace(original)
if original == "" || ftsQuery == "" {
return false
}
// Strip quotes from original for comparison — we want the raw content
stripped := strings.ReplaceAll(original, `"`, "")
// Extract the alphanumeric content from the original query
alphaNum := fts5PunctStrip.ReplaceAllString(stripped, "")
// If the original is entirely alphanumeric, nothing was stripped — not degraded
if len(alphaNum) == len(stripped) {
return false
}
// Check if all effective FTS tokens are very short (≤2 chars).
// Short tokens with prefix matching are too broad when special chars were stripped.
// For quoted phrases, extract the content and check the tokens inside.
tokens := strings.Fields(ftsQuery)
for _, t := range tokens {
t = strings.TrimSuffix(t, "*")
// Skip internal phrase placeholders
if strings.HasPrefix(t, "\x00") {
return false
}
// For OR groups from processPunctuatedWords (e.g., ("a ha" OR aha*)),
// the punctuated word was already handled meaningfully — not degraded.
if strings.HasPrefix(t, "(") {
return false
}
// For quoted phrases, check the tokens inside as FTS5 will tokenize them
if strings.HasPrefix(t, `"`) {
// Extract content between quotes
inner := strings.Trim(t, `"`)
innerAlpha := fts5PunctStrip.ReplaceAllString(inner, " ")
for _, it := range strings.Fields(innerAlpha) {
if len(it) > 2 {
return false
}
}
continue
}
if len(t) > 2 {
return false
}
}
return true
}
// newFTSSearch creates an FTS5 search strategy. Falls back to LIKE search if the
// query produces no FTS tokens (e.g., punctuation-only like "!!!!!!!") or if FTS
// tokenization stripped significant content from the query (e.g., "1+" → "1*").
// Returns nil when the query produces no searchable tokens at all.
func newFTSSearch(tableName, query string) searchStrategy {
q := buildFTS5Query(query)
if q == "" || ftsQueryDegraded(query, q) {
// Fallback: try LIKE search with the raw query
cleaned := strings.TrimSpace(strings.ReplaceAll(query, `"`, ""))
if cleaned != "" {
log.Trace("Search using LIKE fallback for non-tokenizable query", "table", tableName, "query", cleaned)
return newLikeSearch(tableName, cleaned)
}
return nil
}
ftsTable := tableName + "_fts"
matchExpr := q
if cols, ok := ftsSearchColumns[tableName]; ok {
if cols, ok := ftsColumnFilters[tableName]; ok {
matchExpr = cols + " : (" + q + ")"
}
filter := Expr(
tableName+".rowid IN (SELECT rowid FROM "+ftsTable+" WHERE "+ftsTable+" MATCH ?)",
matchExpr,
)
log.Trace("Search using FTS5 backend", "table", tableName, "query", q, "filter", filter)
return filter
rankExpr := ftsTable + ".rank"
if weights, ok := ftsBM25Weights[tableName]; ok {
rankExpr = "bm25(" + ftsTable + ", " + weights + ")"
}
s := &ftsSearch{
tableName: tableName,
ftsTable: ftsTable,
matchExpr: matchExpr,
rankExpr: rankExpr,
}
log.Trace("Search using FTS5 backend", "table", tableName, "query", q, "filter", s)
return s
}

View File

@@ -3,8 +3,6 @@ package persistence
import (
"context"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
@@ -52,6 +50,23 @@ var _ = DescribeTable("buildFTS5Query",
Entry("returns empty string for empty quoted phrase", `""`, ""),
)
var _ = DescribeTable("ftsQueryDegraded",
func(original, ftsQuery string, expected bool) {
Expect(ftsQueryDegraded(original, ftsQuery)).To(Equal(expected))
},
Entry("not degraded for empty original", "", "1*", false),
Entry("not degraded for empty ftsQuery", "1+", "", false),
Entry("not degraded for purely alphanumeric query", "beatles", "beatles*", false),
Entry("not degraded when long tokens remain", "test^val", "test* val*", false),
Entry("not degraded for quoted phrase with long tokens", `"the beatles"`, `"the beatles"`, false),
Entry("degraded for quoted phrase with only short tokens after tokenizer strips special chars", `"1+"`, `"1+"`, true),
Entry("not degraded for quoted phrase with meaningful content", `"C++ programming"`, `"C++ programming"`, false),
Entry("degraded when special chars stripped leaving short token", "1+", "1*", true),
Entry("degraded when special chars stripped leaving two short tokens", "C# 1", "C* 1*", true),
Entry("not degraded when at least one long token remains", "1+ beatles", "1* beatles*", false),
Entry("not degraded for OR groups from processPunctuatedWords", "AC/DC", `("AC DC" OR ACDC*)`, false),
)
var _ = DescribeTable("normalizeForFTS",
func(expected string, values ...string) {
Expect(normalizeForFTS(values...)).To(Equal(expected))
@@ -81,133 +96,211 @@ var _ = DescribeTable("containsCJK",
Entry("detects single CJK character", "a曲b", true),
)
var _ = Describe("likeSearchExpr", func() {
It("returns nil for empty query", func() {
Expect(likeSearchExpr("media_file", "")).To(BeNil())
var _ = DescribeTable("qualifyOrderBy",
func(tableName, orderBy, expected string) {
Expect(qualifyOrderBy(tableName, orderBy)).To(Equal(expected))
},
Entry("returns empty string for empty input", "artist", "", ""),
Entry("qualifies simple column with table name", "artist", "name", "artist.name"),
Entry("qualifies column with direction", "artist", "name desc", "artist.name desc"),
Entry("preserves already-qualified column", "artist", "artist.name", "artist.name"),
Entry("preserves already-qualified column with direction", "artist", "artist.name desc", "artist.name desc"),
Entry("returns empty for function call expression", "artist", "sum(json_extract(stats, '$.total.m')) desc", ""),
Entry("returns empty for expression with comma", "artist", "a, b", ""),
Entry("qualifies media_file column", "media_file", "title", "media_file.title"),
)
var _ = Describe("ftsColumnDefs helpers", func() {
Describe("ftsColumnFilters", func() {
It("returns column filter for media_file", func() {
Expect(ftsColumnFilters).To(HaveKeyWithValue("media_file",
"{title album artist album_artist sort_title sort_album_name sort_artist_name sort_album_artist_name disc_subtitle search_participants search_normalized}",
))
})
It("returns column filter for album", func() {
Expect(ftsColumnFilters).To(HaveKeyWithValue("album",
"{name sort_album_name album_artist search_participants discs catalog_num album_version search_normalized}",
))
})
It("returns column filter for artist", func() {
Expect(ftsColumnFilters).To(HaveKeyWithValue("artist",
"{name sort_artist_name search_normalized}",
))
})
It("has no entry for unknown table", func() {
Expect(ftsColumnFilters).ToNot(HaveKey("unknown"))
})
})
It("returns nil for whitespace-only query", func() {
Expect(likeSearchExpr("media_file", " ")).To(BeNil())
Describe("ftsBM25Weights", func() {
It("returns weight CSV for media_file", func() {
Expect(ftsBM25Weights).To(HaveKeyWithValue("media_file",
"10.0, 5.0, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 1.0",
))
})
It("returns weight CSV for album", func() {
Expect(ftsBM25Weights).To(HaveKeyWithValue("album",
"10.0, 1.0, 3.0, 2.0, 1.0, 1.0, 1.0, 1.0",
))
})
It("returns weight CSV for artist", func() {
Expect(ftsBM25Weights).To(HaveKeyWithValue("artist",
"10.0, 1.0, 1.0",
))
})
It("has no entry for unknown table", func() {
Expect(ftsBM25Weights).ToNot(HaveKey("unknown"))
})
})
It("generates LIKE filters against core columns for single CJK word", func() {
expr := likeSearchExpr("media_file", "周杰伦")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
// Should have OR between columns for the single word
Expect(sql).To(ContainSubstring("OR"))
Expect(sql).To(ContainSubstring("media_file.title LIKE"))
Expect(sql).To(ContainSubstring("media_file.album LIKE"))
Expect(sql).To(ContainSubstring("media_file.artist LIKE"))
Expect(sql).To(ContainSubstring("media_file.album_artist LIKE"))
Expect(args).To(HaveLen(4))
for _, arg := range args {
Expect(arg).To(Equal("%周杰伦%"))
It("has definitions for all known tables", func() {
for _, table := range []string{"media_file", "album", "artist"} {
Expect(ftsColumnDefs).To(HaveKey(table))
Expect(ftsColumnDefs[table]).ToNot(BeEmpty())
}
})
It("generates AND of OR groups for multi-word query", func() {
expr := likeSearchExpr("media_file", "周杰伦 greatest")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
// Two groups AND'd together, each with 4 columns OR'd
Expect(sql).To(ContainSubstring("AND"))
Expect(args).To(HaveLen(8))
})
It("uses correct columns for album table", func() {
expr := likeSearchExpr("album", "周杰伦")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("album.name LIKE"))
Expect(sql).To(ContainSubstring("album.album_artist LIKE"))
Expect(args).To(HaveLen(2))
})
It("uses correct columns for artist table", func() {
expr := likeSearchExpr("artist", "周杰伦")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("artist.name LIKE"))
Expect(args).To(HaveLen(1))
})
It("returns nil for unknown table", func() {
Expect(likeSearchExpr("unknown_table", "周杰伦")).To(BeNil())
It("has matching column count between filter and weights", func() {
for table, cols := range ftsColumnDefs {
// Column filter only includes Weight > 0 columns
filterCount := 0
for _, c := range cols {
if c.Weight > 0 {
filterCount++
}
}
// For now, all columns have Weight > 0, so filter count == total count
Expect(filterCount).To(Equal(len(cols)), "table %s: all columns should have positive weights", table)
}
})
})
var _ = Describe("ftsSearchExpr", func() {
var _ = Describe("newFTSSearch", func() {
It("returns nil for empty query", func() {
Expect(ftsSearchExpr("media_file", "")).To(BeNil())
Expect(newFTSSearch("media_file", "")).To(BeNil())
})
It("generates rowid IN subquery with MATCH and column filter", func() {
expr := ftsSearchExpr("media_file", "beatles")
sql, args, err := expr.ToSql()
It("returns non-nil for single-character query", func() {
strategy := newFTSSearch("media_file", "a")
Expect(strategy).ToNot(BeNil(), "single-char queries must not be rejected; min-length is enforced in doSearch, not here")
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("MATCH"))
})
It("returns ftsSearch with correct table names and MATCH expression", func() {
strategy := newFTSSearch("media_file", "beatles")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.tableName).To(Equal("media_file"))
Expect(fts.ftsTable).To(Equal("media_file_fts"))
Expect(fts.matchExpr).To(HavePrefix("{title album artist album_artist"))
Expect(fts.matchExpr).To(ContainSubstring("beatles*"))
})
It("ToSql generates rowid IN subquery with MATCH (fallback path)", func() {
strategy := newFTSSearch("media_file", "beatles")
sql, args, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("media_file.rowid IN"))
Expect(sql).To(ContainSubstring("media_file_fts"))
Expect(sql).To(ContainSubstring("MATCH"))
Expect(args).To(HaveLen(1))
Expect(args[0]).To(HavePrefix("{title album artist album_artist"))
Expect(args[0]).To(ContainSubstring("beatles*"))
})
It("generates correct FTS table name per entity", func() {
for _, table := range []string{"media_file", "album", "artist"} {
expr := ftsSearchExpr(table, "test")
sql, _, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring(table + ".rowid IN"))
Expect(sql).To(ContainSubstring(table + "_fts"))
strategy := newFTSSearch(table, "test")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.tableName).To(Equal(table))
Expect(fts.ftsTable).To(Equal(table + "_fts"))
}
})
It("builds bm25() rank expression with column weights", func() {
strategy := newFTSSearch("media_file", "beatles")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.rankExpr).To(HavePrefix("bm25(media_file_fts,"))
Expect(fts.rankExpr).To(ContainSubstring("10.0"))
strategy = newFTSSearch("artist", "beatles")
fts, ok = strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.rankExpr).To(HavePrefix("bm25(artist_fts,"))
})
It("falls back to ftsTable.rank for unknown tables", func() {
strategy := newFTSSearch("unknown_table", "test")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.rankExpr).To(Equal("unknown_table_fts.rank"))
})
It("wraps query with column filter for known tables", func() {
expr := ftsSearchExpr("artist", "Beatles")
_, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(args[0]).To(Equal("{name sort_artist_name search_normalized} : (Beatles*)"))
strategy := newFTSSearch("artist", "Beatles")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.matchExpr).To(Equal("{name sort_artist_name search_normalized} : (Beatles*)"))
})
It("passes query without column filter for unknown tables", func() {
expr := ftsSearchExpr("unknown_table", "test")
_, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(args[0]).To(Equal("test*"))
strategy := newFTSSearch("unknown_table", "test")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.matchExpr).To(Equal("test*"))
})
It("preserves phrase queries inside column filter", func() {
expr := ftsSearchExpr("media_file", `"the beatles"`)
_, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(args[0]).To(ContainSubstring(`"the beatles"`))
strategy := newFTSSearch("media_file", `"the beatles"`)
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.matchExpr).To(ContainSubstring(`"the beatles"`))
})
It("preserves prefix queries inside column filter", func() {
expr := ftsSearchExpr("media_file", "beat*")
_, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(args[0]).To(ContainSubstring("beat*"))
strategy := newFTSSearch("media_file", "beat*")
fts, ok := strategy.(*ftsSearch)
Expect(ok).To(BeTrue())
Expect(fts.matchExpr).To(ContainSubstring("beat*"))
})
It("falls back to LIKE search for punctuation-only query", func() {
expr := ftsSearchExpr("media_file", "!!!!!!!")
Expect(expr).ToNot(BeNil())
sql, args, err := expr.ToSql()
strategy := newFTSSearch("media_file", "!!!!!!!")
Expect(strategy).ToNot(BeNil())
_, ok := strategy.(*ftsSearch)
Expect(ok).To(BeFalse(), "punctuation-only should fall back to LIKE, not FTS")
sql, args, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("LIKE"))
Expect(args).To(ContainElement("%!!!!!!!%"))
})
It("falls back to LIKE search for degraded query (special chars stripped leaving short tokens)", func() {
strategy := newFTSSearch("album", "1+")
Expect(strategy).ToNot(BeNil())
_, ok := strategy.(*ftsSearch)
Expect(ok).To(BeFalse(), "degraded query should fall back to LIKE, not FTS")
sql, args, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("LIKE"))
Expect(args).To(ContainElement("%1+%"))
})
It("returns nil for empty string even with LIKE fallback", func() {
Expect(ftsSearchExpr("media_file", "")).To(BeNil())
Expect(ftsSearchExpr("media_file", " ")).To(BeNil())
Expect(newFTSSearch("media_file", "")).To(BeNil())
Expect(newFTSSearch("media_file", " ")).To(BeNil())
})
It("returns nil for empty quoted phrase", func() {
Expect(ftsSearchExpr("media_file", `""`)).To(BeNil())
Expect(newFTSSearch("media_file", `""`)).To(BeNil())
})
})
@@ -229,7 +322,7 @@ var _ = Describe("FTS5 Integration Search", func() {
Describe("MediaFile search", func() {
It("finds media files by title", func() {
results, err := mr.Search("Radioactivity", 0, 10)
results, err := mr.Search("Radioactivity", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Title).To(Equal("Radioactivity"))
@@ -237,7 +330,7 @@ var _ = Describe("FTS5 Integration Search", func() {
})
It("finds media files by artist name", func() {
results, err := mr.Search("Beatles", 0, 10)
results, err := mr.Search("Beatles", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(3))
for _, r := range results {
@@ -248,7 +341,7 @@ var _ = Describe("FTS5 Integration Search", func() {
Describe("Album search", func() {
It("finds albums by name", func() {
results, err := alr.Search("Sgt Peppers", 0, 10)
results, err := alr.Search("Sgt Peppers", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Name).To(Equal("Sgt Peppers"))
@@ -256,7 +349,7 @@ var _ = Describe("FTS5 Integration Search", func() {
})
It("finds albums with multi-word search", func() {
results, err := alr.Search("Abbey Road", 0, 10)
results, err := alr.Search("Abbey Road", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(2))
})
@@ -264,7 +357,7 @@ var _ = Describe("FTS5 Integration Search", func() {
Describe("Artist search", func() {
It("finds artists by name", func() {
results, err := arr.Search("Kraftwerk", 0, 10)
results, err := arr.Search("Kraftwerk", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Name).To(Equal("Kraftwerk"))
@@ -274,7 +367,7 @@ var _ = Describe("FTS5 Integration Search", func() {
Describe("CJK search", func() {
It("finds media files by CJK title", func() {
results, err := mr.Search("プラチナ", 0, 10)
results, err := mr.Search("プラチナ", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Title).To(Equal("プラチナ・ジェット"))
@@ -282,14 +375,14 @@ var _ = Describe("FTS5 Integration Search", func() {
})
It("finds media files by CJK artist name", func() {
results, err := mr.Search("シートベルツ", 0, 10)
results, err := mr.Search("シートベルツ", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Artist).To(Equal("シートベルツ"))
})
It("finds albums by CJK artist name", func() {
results, err := alr.Search("シートベルツ", 0, 10)
results, err := alr.Search("シートベルツ", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Name).To(Equal("COWBOY BEBOP"))
@@ -297,7 +390,7 @@ var _ = Describe("FTS5 Integration Search", func() {
})
It("finds artists by CJK name", func() {
results, err := arr.Search("シートベルツ", 0, 10)
results, err := arr.Search("シートベルツ", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Name).To(Equal("シートベルツ"))
@@ -307,7 +400,7 @@ var _ = Describe("FTS5 Integration Search", func() {
Describe("Album version search", func() {
It("finds albums by version tag via FTS", func() {
results, err := alr.Search("Deluxe", 0, 10)
results, err := alr.Search("Deluxe", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].ID).To(Equal(albumWithVersion.ID))
@@ -316,7 +409,7 @@ var _ = Describe("FTS5 Integration Search", func() {
Describe("Punctuation-only search", func() {
It("finds media files with punctuation-only title", func() {
results, err := mr.Search("!!!!!!!", 0, 10)
results, err := mr.Search("!!!!!!!", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Title).To(Equal("!!!!!!!"))
@@ -324,15 +417,19 @@ var _ = Describe("FTS5 Integration Search", func() {
})
})
Describe("Legacy backend fallback", func() {
It("returns results using legacy LIKE-based search when configured", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "legacy"
results, err := mr.Search("Radioactivity", 0, 10)
Describe("Single-character search (doSearch min-length guard)", func() {
It("returns empty results for single-char query via Search", func() {
results, err := mr.Search("a", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Title).To(Equal("Radioactivity"))
Expect(results).To(BeEmpty(), "doSearch should reject single-char queries")
})
})
Describe("Max=0 means no limit (regression: must not produce LIMIT 0)", func() {
It("returns results with Max=0", func() {
results, err := mr.Search("Beatles", model.QueryOptions{Max: 0})
Expect(err).ToNot(HaveOccurred())
Expect(results).ToNot(BeEmpty(), "Max=0 should mean no limit, not LIMIT 0")
})
})
})

View File

@@ -0,0 +1,106 @@
package persistence
import (
"strings"
. "github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/str"
)
// likeSearch implements searchStrategy using LIKE-based SQL filters.
// Used for legacy full_text searches, CJK fallback, and punctuation-only fallback.
type likeSearch struct {
filter Sqlizer
}
func (s *likeSearch) ToSql() (string, []interface{}, error) {
return s.filter.ToSql()
}
func (s *likeSearch) execute(r sqlRepository, sq SelectBuilder, dest any, cfg searchConfig, options model.QueryOptions) error {
sq = sq.Where(s.filter)
sq = sq.OrderBy(cfg.OrderBy...)
return r.queryAll(sq, dest, options)
}
// newLegacySearch creates a LIKE search against the full_text column.
// Returns nil when the query produces no searchable tokens.
func newLegacySearch(tableName, query string) searchStrategy {
filter := legacySearchExpr(tableName, query)
if filter == nil {
return nil
}
return &likeSearch{filter: filter}
}
// newLikeSearch creates a LIKE search against core entity columns (CJK, punctuation fallback).
// No minimum length is enforced, since single CJK characters are meaningful words.
// Returns nil when the query produces no searchable tokens.
func newLikeSearch(tableName, query string) searchStrategy {
filter := likeSearchExpr(tableName, query)
if filter == nil {
return nil
}
return &likeSearch{filter: filter}
}
// legacySearchExpr generates LIKE-based search filters against the full_text column.
// This is the original search implementation, used when Search.Backend="legacy".
func legacySearchExpr(tableName string, s string) Sqlizer {
q := str.SanitizeStrings(s)
if q == "" {
log.Trace("Search using legacy backend, query is empty", "table", tableName)
return nil
}
var sep string
if !conf.Server.Search.FullString {
sep = " "
}
parts := strings.Split(q, " ")
filters := And{}
for _, part := range parts {
filters = append(filters, Like{tableName + ".full_text": "%" + sep + part + "%"})
}
log.Trace("Search using legacy backend", "query", filters, "table", tableName)
return filters
}
// likeSearchColumns defines the core columns to search with LIKE queries.
// These are the primary user-visible fields for each entity type.
// Used as a fallback when FTS5 cannot handle the query (e.g., CJK text, punctuation-only input).
var likeSearchColumns = map[string][]string{
"media_file": {"title", "album", "artist", "album_artist"},
"album": {"name", "album_artist"},
"artist": {"name"},
}
// likeSearchExpr generates LIKE-based search filters against core columns.
// Each word in the query must match at least one column (AND between words),
// and each word can match any column (OR within a word).
// Used as a fallback when FTS5 cannot handle the query (e.g., CJK text, punctuation-only input).
func likeSearchExpr(tableName string, s string) Sqlizer {
s = strings.TrimSpace(s)
if s == "" {
log.Trace("Search using LIKE backend, query is empty", "table", tableName)
return nil
}
columns, ok := likeSearchColumns[tableName]
if !ok {
log.Trace("Search using LIKE backend, couldn't find columns for this table", "table", tableName)
return nil
}
words := strings.Fields(s)
wordFilters := And{}
for _, word := range words {
colFilters := Or{}
for _, col := range columns {
colFilters = append(colFilters, Like{tableName + "." + col: "%" + word + "%"})
}
wordFilters = append(wordFilters, colFilters)
}
log.Trace("Search using LIKE backend", "query", wordFilters, "table", tableName)
return wordFilters
}

View File

@@ -0,0 +1,134 @@
package persistence
import (
"context"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("newLegacySearch", func() {
It("returns non-nil for single-character query", func() {
strategy := newLegacySearch("media_file", "a")
Expect(strategy).ToNot(BeNil(), "single-char queries must not be rejected; min-length is enforced in doSearch, not here")
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("LIKE"))
})
})
var _ = Describe("legacySearchExpr", func() {
It("returns nil for empty query", func() {
Expect(legacySearchExpr("media_file", "")).To(BeNil())
})
It("generates LIKE filter for single word", func() {
expr := legacySearchExpr("media_file", "beatles")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("media_file.full_text LIKE"))
Expect(args).To(ContainElement("% beatles%"))
})
It("generates AND of LIKE filters for multiple words", func() {
expr := legacySearchExpr("media_file", "abbey road")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("AND"))
Expect(args).To(HaveLen(2))
})
})
var _ = Describe("likeSearchExpr", func() {
It("returns nil for empty query", func() {
Expect(likeSearchExpr("media_file", "")).To(BeNil())
})
It("returns nil for whitespace-only query", func() {
Expect(likeSearchExpr("media_file", " ")).To(BeNil())
})
It("generates LIKE filters against core columns for single CJK word", func() {
expr := likeSearchExpr("media_file", "周杰伦")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
// Should have OR between columns for the single word
Expect(sql).To(ContainSubstring("OR"))
Expect(sql).To(ContainSubstring("media_file.title LIKE"))
Expect(sql).To(ContainSubstring("media_file.album LIKE"))
Expect(sql).To(ContainSubstring("media_file.artist LIKE"))
Expect(sql).To(ContainSubstring("media_file.album_artist LIKE"))
Expect(args).To(HaveLen(4))
for _, arg := range args {
Expect(arg).To(Equal("%周杰伦%"))
}
})
It("generates AND of OR groups for multi-word query", func() {
expr := likeSearchExpr("media_file", "周杰伦 greatest")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
// Two groups AND'd together, each with 4 columns OR'd
Expect(sql).To(ContainSubstring("AND"))
Expect(args).To(HaveLen(8))
})
It("uses correct columns for album table", func() {
expr := likeSearchExpr("album", "周杰伦")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("album.name LIKE"))
Expect(sql).To(ContainSubstring("album.album_artist LIKE"))
Expect(args).To(HaveLen(2))
})
It("uses correct columns for artist table", func() {
expr := likeSearchExpr("artist", "周杰伦")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("artist.name LIKE"))
Expect(args).To(HaveLen(1))
})
It("returns nil for unknown table", func() {
Expect(likeSearchExpr("unknown_table", "周杰伦")).To(BeNil())
})
})
var _ = Describe("Legacy Integration Search", func() {
var mr model.MediaFileRepository
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "legacy"
ctx := log.NewContext(context.TODO())
ctx = request.WithUser(ctx, adminUser)
conn := GetDBXBuilder()
mr = NewMediaFileRepository(ctx, conn)
})
It("returns results using legacy LIKE-based search", func() {
results, err := mr.Search("Radioactivity", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(HaveLen(1))
Expect(results[0].Title).To(Equal("Radioactivity"))
})
It("returns empty results for single-char query (doSearch min-length guard)", func() {
results, err := mr.Search("a", model.QueryOptions{Max: 10})
Expect(err).ToNot(HaveOccurred())
Expect(results).To(BeEmpty(), "doSearch should reject single-char queries")
})
It("returns results with Max=0 (regression: must not produce LIMIT 0)", func() {
results, err := mr.Search("Beatles", model.QueryOptions{Max: 0})
Expect(err).ToNot(HaveOccurred())
Expect(results).ToNot(BeEmpty(), "Max=0 should mean no limit, not LIMIT 0")
})
})

View File

@@ -14,98 +14,99 @@ var _ = Describe("sqlRepository", func() {
})
})
Describe("legacySearchExpr", func() {
It("returns nil for empty query", func() {
Expect(legacySearchExpr("media_file", "")).To(BeNil())
})
It("generates LIKE filter for single word", func() {
expr := legacySearchExpr("media_file", "beatles")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("media_file.full_text LIKE"))
Expect(args).To(ContainElement("% beatles%"))
})
It("generates AND of LIKE filters for multiple words", func() {
expr := legacySearchExpr("media_file", "abbey road")
sql, args, err := expr.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("AND"))
Expect(args).To(HaveLen(2))
})
})
Describe("getSearchExpr", func() {
It("returns ftsSearchExpr by default", func() {
Describe("getSearchStrategy", func() {
It("returns FTS strategy by default", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "fts"
conf.Server.Search.FullString = false
expr := getSearchExpr()("media_file", "test")
sql, _, err := expr.ToSql()
strategy := getSearchStrategy("media_file", "test")
Expect(strategy).ToNot(BeNil())
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("MATCH"))
})
It("returns legacySearchExpr when SearchBackend is legacy", func() {
It("returns legacy LIKE strategy when SearchBackend is legacy", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "legacy"
conf.Server.Search.FullString = false
expr := getSearchExpr()("media_file", "test")
sql, _, err := expr.ToSql()
strategy := getSearchStrategy("media_file", "test")
Expect(strategy).ToNot(BeNil())
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("LIKE"))
})
It("falls back to legacySearchExpr when SearchFullString is enabled", func() {
It("falls back to legacy LIKE strategy when SearchFullString is enabled", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "fts"
conf.Server.Search.FullString = true
expr := getSearchExpr()("media_file", "test")
sql, _, err := expr.ToSql()
strategy := getSearchStrategy("media_file", "test")
Expect(strategy).ToNot(BeNil())
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("LIKE"))
})
It("routes CJK queries to likeSearchExpr instead of ftsSearchExpr", func() {
It("routes CJK queries to LIKE strategy instead of FTS", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "fts"
conf.Server.Search.FullString = false
expr := getSearchExpr()("media_file", "周杰伦")
sql, _, err := expr.ToSql()
strategy := getSearchStrategy("media_file", "周杰伦")
Expect(strategy).ToNot(BeNil())
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
// CJK should use LIKE, not MATCH
Expect(sql).To(ContainSubstring("LIKE"))
Expect(sql).NotTo(ContainSubstring("MATCH"))
})
It("routes non-CJK queries to ftsSearchExpr", func() {
It("routes non-CJK queries to FTS strategy", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "fts"
conf.Server.Search.FullString = false
expr := getSearchExpr()("media_file", "beatles")
sql, _, err := expr.ToSql()
strategy := getSearchStrategy("media_file", "beatles")
Expect(strategy).ToNot(BeNil())
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("MATCH"))
})
It("returns non-nil for single-character query (no min-length in strategy)", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "fts"
conf.Server.Search.FullString = false
strategy := getSearchStrategy("media_file", "a")
Expect(strategy).ToNot(BeNil(), "single-char queries must be accepted by strategies (min-length is enforced in doSearch)")
})
It("returns non-nil for single-character query with legacy backend", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "legacy"
conf.Server.Search.FullString = false
strategy := getSearchStrategy("media_file", "a")
Expect(strategy).ToNot(BeNil(), "single-char queries must be accepted by legacy strategy (min-length is enforced in doSearch)")
})
It("uses legacy for CJK when SearchBackend is legacy", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "legacy"
conf.Server.Search.FullString = false
expr := getSearchExpr()("media_file", "周杰伦")
sql, _, err := expr.ToSql()
strategy := getSearchStrategy("media_file", "周杰伦")
Expect(strategy).ToNot(BeNil())
sql, _, err := strategy.ToSql()
Expect(err).ToNot(HaveOccurred())
// Legacy should still use full_text column LIKE
Expect(sql).To(ContainSubstring("LIKE"))
Expect(sql).To(ContainSubstring("full_text"))
})
})
})

View File

@@ -1 +1 @@
-s -r "(\.go$$|\.cpp$$|\.h$$|navidrome.toml|resources|token_received.html)" -R "(^ui|^data|^db/migrations)" -R "_test\.go$$" -- go run -race -tags netgo,sqlite_fts5 .
-s -r "(\.go$$|\.cpp$$|\.h$$|navidrome.toml|resources|token_received.html)" -R "(^ui|^data|^db/migrations)" -R "_test\.go$$" -- go run -race -tags netgo,sqlite_fts5,sqlite_spellfix .

View File

@@ -115,6 +115,7 @@ func buildTestFS() storagetest.FakeFS {
ledZepIV := template(_t{"albumartist": "Led Zeppelin", "artist": "Led Zeppelin", "album": "IV", "year": 1971, "genre": "Rock"})
kindOfBlue := template(_t{"albumartist": "Miles Davis", "artist": "Miles Davis", "album": "Kind of Blue", "year": 1959, "genre": "Jazz"})
popTrack := template(_t{"albumartist": "Various", "artist": "Various", "album": "Pop", "year": 2020, "genre": "Pop"})
cowboyBebop := template(_t{"albumartist": "シートベルツ", "artist": "シートベルツ", "album": "COWBOY BEBOP", "year": 1998, "genre": "Jazz"})
return createFS(fstest.MapFS{
// Rock / The Beatles / Abbey Road (with MBIDs)
@@ -132,6 +133,8 @@ func buildTestFS() storagetest.FakeFS {
"Jazz/Miles Davis/Kind of Blue/01 - So What.mp3": kindOfBlue(track(1, "So What")),
// Pop (standalone track, no MBIDs)
"Pop/01 - Standalone Track.mp3": popTrack(track(1, "Standalone Track")),
// CJK / シートベルツ / COWBOY BEBOP (Japanese artist, for CJK search tests)
"CJK/シートベルツ/COWBOY BEBOP/01 - プラチナ・ジェット.mp3": cowboyBebop(track(1, "プラチナ・ジェット")),
// _empty folder (directory with no audio)
"_empty/.keep": &fstest.MapFile{Data: []byte{}, ModTime: time.Now()},
})

View File

@@ -19,7 +19,7 @@ var _ = Describe("Album List Endpoints", func() {
Expect(resp.Status).To(Equal(responses.StatusOK))
Expect(resp.AlbumList).ToNot(BeNil())
Expect(resp.AlbumList.Album).To(HaveLen(5))
Expect(resp.AlbumList.Album).To(HaveLen(6))
})
It("type=alphabeticalByName sorts albums by name", func() {
@@ -27,13 +27,14 @@ var _ = Describe("Album List Endpoints", func() {
Expect(resp.AlbumList).ToNot(BeNil())
albums := resp.AlbumList.Album
Expect(albums).To(HaveLen(5))
// Verify alphabetical order: Abbey Road, Help!, IV, Kind of Blue, Pop
Expect(albums).To(HaveLen(6))
// Verify alphabetical order: Abbey Road, COWBOY BEBOP, Help!, IV, Kind of Blue, Pop
Expect(albums[0].Title).To(Equal("Abbey Road"))
Expect(albums[1].Title).To(Equal("Help!"))
Expect(albums[2].Title).To(Equal("IV"))
Expect(albums[3].Title).To(Equal("Kind of Blue"))
Expect(albums[4].Title).To(Equal("Pop"))
Expect(albums[1].Title).To(Equal("COWBOY BEBOP"))
Expect(albums[2].Title).To(Equal("Help!"))
Expect(albums[3].Title).To(Equal("IV"))
Expect(albums[4].Title).To(Equal("Kind of Blue"))
Expect(albums[5].Title).To(Equal("Pop"))
})
It("type=alphabeticalByArtist sorts albums by artist name", func() {
@@ -41,29 +42,32 @@ var _ = Describe("Album List Endpoints", func() {
Expect(resp.AlbumList).ToNot(BeNil())
albums := resp.AlbumList.Album
Expect(albums).To(HaveLen(5))
Expect(albums).To(HaveLen(6))
// Articles like "The" are stripped for sorting, so "The Beatles" sorts as "Beatles"
// Non-compilations first: Beatles (x2), Led Zeppelin, Miles Davis, then compilations: Various
// Non-compilations first: Beatles (x2), Led Zeppelin, Miles Davis, then compilations: Various, then CJK: シートベルツ
Expect(albums[0].Artist).To(Equal("The Beatles"))
Expect(albums[1].Artist).To(Equal("The Beatles"))
Expect(albums[2].Artist).To(Equal("Led Zeppelin"))
Expect(albums[3].Artist).To(Equal("Miles Davis"))
Expect(albums[4].Artist).To(Equal("Various"))
Expect(albums[5].Artist).To(Equal("シートベルツ"))
})
It("type=random returns albums", func() {
resp := doReq("getAlbumList", "type", "random")
Expect(resp.AlbumList).ToNot(BeNil())
Expect(resp.AlbumList.Album).To(HaveLen(5))
Expect(resp.AlbumList.Album).To(HaveLen(6))
})
It("type=byGenre filters by genre parameter", func() {
resp := doReq("getAlbumList", "type", "byGenre", "genre", "Jazz")
Expect(resp.AlbumList).ToNot(BeNil())
Expect(resp.AlbumList.Album).To(HaveLen(1))
Expect(resp.AlbumList.Album[0].Title).To(Equal("Kind of Blue"))
Expect(resp.AlbumList.Album).To(HaveLen(2))
for _, a := range resp.AlbumList.Album {
Expect(a.Genre).To(Equal("Jazz"))
}
})
It("type=byYear filters by fromYear/toYear range", func() {
@@ -184,7 +188,7 @@ var _ = Describe("Album List Endpoints", func() {
Expect(resp.Status).To(Equal(responses.StatusOK))
Expect(resp.AlbumList2).ToNot(BeNil())
albums := resp.AlbumList2.Album
Expect(albums).To(HaveLen(5))
Expect(albums).To(HaveLen(6))
// Verify AlbumID3 format fields
Expect(albums[0].Name).To(Equal("Abbey Road"))
Expect(albums[0].Id).ToNot(BeEmpty())
@@ -195,7 +199,7 @@ var _ = Describe("Album List Endpoints", func() {
resp := doReq("getAlbumList2", "type", "newest")
Expect(resp.AlbumList2).ToNot(BeNil())
Expect(resp.AlbumList2.Album).To(HaveLen(5))
Expect(resp.AlbumList2.Album).To(HaveLen(6))
})
})
@@ -240,7 +244,7 @@ var _ = Describe("Album List Endpoints", func() {
Expect(resp.Status).To(Equal(responses.StatusOK))
Expect(resp.RandomSongs).ToNot(BeNil())
Expect(resp.RandomSongs.Songs).ToNot(BeEmpty())
Expect(len(resp.RandomSongs.Songs)).To(BeNumerically("<=", 6))
Expect(resp.RandomSongs.Songs).To(HaveLen(7))
})
It("respects size parameter", func() {
@@ -254,8 +258,10 @@ var _ = Describe("Album List Endpoints", func() {
resp := doReq("getRandomSongs", "size", "500", "genre", "Jazz")
Expect(resp.RandomSongs).ToNot(BeNil())
Expect(resp.RandomSongs.Songs).To(HaveLen(1))
Expect(resp.RandomSongs.Songs[0].Genre).To(Equal("Jazz"))
Expect(resp.RandomSongs.Songs).To(HaveLen(2))
for _, s := range resp.RandomSongs.Songs {
Expect(s.Genre).To(Equal("Jazz"))
}
})
})

View File

@@ -327,8 +327,8 @@ var _ = Describe("Browsing Endpoints", func() {
}
}
Expect(jazzGenre).ToNot(BeNil())
Expect(jazzGenre.SongCount).To(Equal(int32(1)))
Expect(jazzGenre.AlbumCount).To(Equal(int32(1)))
Expect(jazzGenre.SongCount).To(Equal(int32(2)))
Expect(jazzGenre.AlbumCount).To(Equal(int32(2)))
})
It("reports correct song and album counts for Pop", func() {

View File

@@ -141,7 +141,7 @@ var _ = Describe("Multi-Library Support", Ordered, func() {
resp := doReqWithUser(adminWithLibs, "getAlbumList", "type", "alphabeticalByName", "musicFolderId", fmt.Sprintf("%d", lib.ID))
Expect(resp.AlbumList).ToNot(BeNil())
Expect(resp.AlbumList.Album).To(HaveLen(5))
Expect(resp.AlbumList.Album).To(HaveLen(6))
for _, a := range resp.AlbumList.Album {
Expect(a.Title).ToNot(Equal("Symphony No. 9"))
}
@@ -275,5 +275,21 @@ var _ = Describe("Multi-Library Support", Ordered, func() {
Expect(artistNames).To(ContainElements("The Beatles", "Led Zeppelin", "Miles Davis"))
Expect(artistNames).ToNot(ContainElement("Ludwig van Beethoven"))
})
It("non-admin user search returns only their library's content", func() {
resp := doReqWithUser(userLib1Only, "search3", "query", "Beethoven")
Expect(resp.SearchResult3).ToNot(BeNil())
Expect(resp.SearchResult3.Artist).To(BeEmpty(), "userLib1Only should not see Beethoven (lib2)")
Expect(resp.SearchResult3.Album).To(BeEmpty())
Expect(resp.SearchResult3.Song).To(BeEmpty())
})
It("non-admin user search finds content from their library", func() {
resp := doReqWithUser(userLib1Only, "search3", "query", "Beatles")
Expect(resp.SearchResult3).ToNot(BeNil())
Expect(resp.SearchResult3.Artist).ToNot(BeEmpty(), "userLib1Only should find Beatles (lib1)")
})
})
})

View File

@@ -2,6 +2,8 @@ package e2e
import (
"github.com/google/uuid"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/server/subsonic/responses"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
@@ -113,9 +115,9 @@ var _ = Describe("Search Endpoints", func() {
Expect(resp.Status).To(Equal(responses.StatusOK))
Expect(resp.SearchResult3).ToNot(BeNil())
Expect(resp.SearchResult3.Artist).To(HaveLen(4))
Expect(resp.SearchResult3.Album).To(HaveLen(5))
Expect(resp.SearchResult3.Song).To(HaveLen(6))
Expect(resp.SearchResult3.Artist).To(HaveLen(5))
Expect(resp.SearchResult3.Album).To(HaveLen(6))
Expect(resp.SearchResult3.Song).To(HaveLen(7))
})
It("finds across all entity types simultaneously", func() {
@@ -217,5 +219,56 @@ var _ = Describe("Search Endpoints", func() {
Expect(resp.SearchResult3.Song).To(BeEmpty())
})
})
Describe("CJK search", func() {
It("finds songs by CJK title", func() {
resp := doReq("search3", "query", "プラチナ")
Expect(resp.Status).To(Equal(responses.StatusOK))
Expect(resp.SearchResult3).ToNot(BeNil())
Expect(resp.SearchResult3.Song).To(HaveLen(1))
Expect(resp.SearchResult3.Song[0].Title).To(Equal("プラチナ・ジェット"))
})
It("finds artists by CJK name", func() {
resp := doReq("search3", "query", "シートベルツ")
Expect(resp.Status).To(Equal(responses.StatusOK))
Expect(resp.SearchResult3).ToNot(BeNil())
Expect(resp.SearchResult3.Artist).To(HaveLen(1))
Expect(resp.SearchResult3.Artist[0].Name).To(Equal("シートベルツ"))
})
It("finds albums by CJK artist name", func() {
resp := doReq("search3", "query", "シートベルツ")
Expect(resp.Status).To(Equal(responses.StatusOK))
Expect(resp.SearchResult3).ToNot(BeNil())
Expect(resp.SearchResult3.Album).To(HaveLen(1))
Expect(resp.SearchResult3.Album[0].Name).To(Equal("COWBOY BEBOP"))
})
})
Describe("Legacy backend", func() {
It("returns results using legacy LIKE-based search when configured", func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.Search.Backend = "legacy"
resp := doReq("search3", "query", "Beatles")
Expect(resp.Status).To(Equal(responses.StatusOK))
Expect(resp.SearchResult3).ToNot(BeNil())
Expect(resp.SearchResult3.Artist).ToNot(BeEmpty())
found := false
for _, a := range resp.SearchResult3.Artist {
if a.Name == "The Beatles" {
found = true
break
}
}
Expect(found).To(BeTrue(), "expected to find artist 'The Beatles' with legacy backend")
})
})
})
})

View File

@@ -42,17 +42,17 @@ func (api *Router) getSearchParams(r *http.Request) (*searchParams, error) {
return sp, nil
}
type searchFunc[T any] func(q string, offset int, size int, options ...model.QueryOptions) (T, error)
type searchFunc[T any] func(q string, options ...model.QueryOptions) (T, error)
func callSearch[T any](ctx context.Context, s searchFunc[T], q string, offset, size int, result *T, options ...model.QueryOptions) func() error {
func callSearch[T any](ctx context.Context, s searchFunc[T], q string, options model.QueryOptions, result *T) func() error {
return func() error {
if size == 0 {
if options.Max == 0 {
return nil
}
typ := strings.TrimPrefix(reflect.TypeOf(*result).String(), "model.")
var err error
start := time.Now()
*result, err = s(q, offset, size, options...)
*result, err = s(q, options)
if err != nil {
log.Error(ctx, "Error searching "+typ, "query", q, "elapsed", time.Since(start), err)
} else {
@@ -66,27 +66,22 @@ func (api *Router) searchAll(ctx context.Context, sp *searchParams, musicFolderI
start := time.Now()
q := sanitize.Accents(strings.ToLower(strings.TrimSuffix(sp.query, "*")))
// Create query options for library filtering
var options []model.QueryOptions
var artistOptions []model.QueryOptions
// Build options with offset/size/filters packed in
songOpts := model.QueryOptions{Max: sp.songCount, Offset: sp.songOffset}
albumOpts := model.QueryOptions{Max: sp.albumCount, Offset: sp.albumOffset}
artistOpts := model.QueryOptions{Max: sp.artistCount, Offset: sp.artistOffset}
if len(musicFolderIds) > 0 {
// For MediaFiles and Albums, use direct library_id filter
options = append(options, model.QueryOptions{
Filters: Eq{"library_id": musicFolderIds},
})
// For Artists, use the repository's built-in library filtering mechanism
// which properly handles the library_artist table joins
// TODO Revisit library filtering in sql_base_repository.go
artistOptions = append(artistOptions, model.QueryOptions{
Filters: Eq{"library_artist.library_id": musicFolderIds},
})
songOpts.Filters = Eq{"library_id": musicFolderIds}
albumOpts.Filters = Eq{"library_id": musicFolderIds}
artistOpts.Filters = Eq{"library_artist.library_id": musicFolderIds}
}
// Run searches in parallel
g, ctx := errgroup.WithContext(ctx)
g.Go(callSearch(ctx, api.ds.MediaFile(ctx).Search, q, sp.songOffset, sp.songCount, &mediaFiles, options...))
g.Go(callSearch(ctx, api.ds.Album(ctx).Search, q, sp.albumOffset, sp.albumCount, &albums, options...))
g.Go(callSearch(ctx, api.ds.Artist(ctx).Search, q, sp.artistOffset, sp.artistCount, &artists, artistOptions...))
g.Go(callSearch(ctx, api.ds.MediaFile(ctx).Search, q, songOpts, &mediaFiles))
g.Go(callSearch(ctx, api.ds.Album(ctx).Search, q, albumOpts, &albums))
g.Go(callSearch(ctx, api.ds.Artist(ctx).Search, q, artistOpts, &artists))
err := g.Wait()
if err == nil {
log.Debug(ctx, fmt.Sprintf("Search resulted in %d songs, %d albums and %d artists",

View File

@@ -119,7 +119,7 @@ func (m *MockAlbumRepo) UpdateExternalInfo(album *model.Album) error {
return nil
}
func (m *MockAlbumRepo) Search(q string, offset int, size int, options ...model.QueryOptions) (model.Albums, error) {
func (m *MockAlbumRepo) Search(q string, options ...model.QueryOptions) (model.Albums, error) {
if len(options) > 0 {
m.Options = options[0]
}

View File

@@ -145,7 +145,7 @@ func (m *MockArtistRepo) GetIndex(includeMissing bool, libraryIds []int, roles .
return result, nil
}
func (m *MockArtistRepo) Search(q string, offset int, size int, options ...model.QueryOptions) (model.Artists, error) {
func (m *MockArtistRepo) Search(q string, options ...model.QueryOptions) (model.Artists, error) {
if len(options) > 0 {
m.Options = options[0]
}

View File

@@ -238,7 +238,7 @@ func (m *MockMediaFileRepo) NewInstance() any {
return &model.MediaFile{}
}
func (m *MockMediaFileRepo) Search(q string, offset int, size int, options ...model.QueryOptions) (model.MediaFiles, error) {
func (m *MockMediaFileRepo) Search(q string, options ...model.QueryOptions) (model.MediaFiles, error) {
if len(options) > 0 {
m.Options = options[0]
}