Lucene 3 has no numeric type, so a number field is indexed zero-padded to make
a range sort: 600 minutes is stored as "00000600.00". Every other field keeps
the number as written, so the novel "1984" is stored as "1984". Padding every
number in a query regardless of the field it was being compared against made a
numeric title unfindable - "title:1984" looked for "00001984.00" among the
titles - and a bare "1984" could only ever mean "some number field is 1984".
A number now takes the spelling its field is indexed in. With no field named,
the default field holds both spellings, so both are searched. A range and a
phrase cannot hold that disjunction: a range stays padded, a phrase does not.
Expanding a bare number to a group exposed the pure-negation workaround, which
judged a query by every clause in its tree, so "-(a OR b)" was read as mixed
and returned nothing. It is now decided per query by that query's own clauses.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Search field names are ordinary words - absent, podcast, plus, series - and
a book's own text is full of them. The sanitizer read any known field name as
a field name wherever it appeared, so a value in that position picked up the
implied ":True" of a bool field: "title:absent" became "title:absent:True",
which Lucene refuses to parse, and "title:\"absent friend\"" quietly matched
nothing. A colon straight after a field name now marks the next term as that
field's value.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A book the index does not hold cannot be found by any positive filter, and
every negated filter - which resolves to every document in the index - drops
it from the grid instead. That is why issue #1989 reads as half a working
filter: Absent found nothing while -Absent removed exactly the absent books.
The index is only ever written as a whole, and a rebuild that fails is
deliberately swallowed so a bad index cannot fail a good scan, so a short
index stayed short until something else changed the library. Count the
index against the library once per run and rebuild when it is short.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
<title short> stops at the first colon, so it shortens Audible titles that
contain one just as readily as it drops Audible's subtitle, and distinct books
then collapse onto the same name. A colon cannot be searched for: the analyzer
discards punctuation and Lucene reads a colon in a query as a field separator.
Two bool index fields find the affected books instead.
Document how the two title tags differ, since <audible title> already drops
Audible's subtitle without ever cutting a title, and how to audit for names
that actually collide in a spreadsheet export.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Both grids restored the last good filter by recursing into the filter handler, which
never terminated once the search index rather than the query was the problem: the
restore fails the same way, and the retry uses the same filter. The user got an
endless run of dialogs, each of them blaming a filter string that was fine. Only an
empty last-good filter broke the loop, because that short-circuits before reaching
the search engine.
The fallback is now a bounded sequence -- last good filter, then no filter -- and
the message distinguishes an index Libation cannot reach from a query it cannot
parse. Only the first failure is reported, so restoring is quiet. A malformed query
never surfaces as an IO-family exception, which QueryFailureShapeTests pins against
the real engine, so a typo is never mistaken for index trouble or made to trigger a
rebuild.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Windows CI caught real over-reach. When another holder has write.lock, Windows
raises the sharing violation before Lucene can turn it into a
LockObtainFailedException, so it arrives as a plain IOException. Repairing anything
that is not a recognised lock conflict then meant deleting the index the other
holder was using -- exactly the second-instance case the retry exists for.
An IOException naming Lucene's write lock now counts as a lock conflict. Matching
the file name rather than the message wording keeps it working on non-English
Windows. The end-to-end test asserts the property instead of the exception type,
since the type legitimately differs by platform.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A truncated or zero-length segments file is reported by Lucene 3 as a plain
IOException ("read past EOF") rather than a CorruptIndexException, so it was
misclassified as a write.lock conflict: CreateNewIndex burned its whole backoff
budget and rethrew, and the delete-and-rebuild recovery never ran. Passing
create/overwrite to IndexWriter does not repair it either, because
IndexFileDeleter reads every segments_* file in the directory and tolerates only
missing ones, so a single unreadable segments file -- even a stale one from an
older commit -- leaves the index permanently unopenable. The user's only cure
was deleting the SearchEngine folder by hand.
Retries are now reserved for genuine lock conflicts (LockObtainFailedException,
which derives from IOException, and UnauthorizedAccessException), and any other
open failure gets one delete-and-rebuild pass before giving up with a message
that says which folder to remove. The query path recovers too, since
IsRecoverableCorruptIndexException now recognizes the truncated-segments
signature.
Search index updates are also no longer allowed to fail the library change that
triggered them. Both events fire after the database is committed, so an escaping
exception reported a successful scan as "Error importing library" and, being the
first subscriber, stopped the handlers that refresh the grid and backup counts.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
- Use the main display grid control to display deleted books
- Added search functionality for deleted books. This required creating a temporary search index in the `InProgress` folder. The products grid control now uses an instance of `ISearchEngine` to filter its grid entries. The main grid uses a singleton instance of `MainSearchEngine`, which merely wraps `SearchEngineCommands.Search()`. The TrashBinDialogs use `TempSearchEngine`.
- Users can now batch select `Everyting` as well as `Audible Plus Books`
Avalonia:
- Refactor main grid context menus to no longer require reflection
- Add `IsAudiblePlus` to search engine
- Add `IsAudiblePlus` and `AbsentFromLastScan` properties to library export
- Refactor library export ToXlsx method
- Make nullable
- Improve readability and extensability
- Use same column header names as CSV
- Extend export methods to accept optional list of books (future use)
- Update all project runtime targets
- Update all dependencies
- NOTE: Using Npgsql.EntityFrameworkCore.PostgreSQL RTM build from MyGet
- Delete unused pubxml files (they were made redundant by recent workflow changes)
- Replace Libation.sln with Libation.slnx
- Move all settings file logic into new LibationFiles class
- Configuration.LibationFiles is a singleton instance of the LibationFiles class.
- A LibationFiles instance is bound to a single appsettings.json path. All updates to LibationFiles location are updated in that appsettings.json file
- Unify initial setup and settings validation process
- Add LibationSetup which handles all startup validation of settings files and prompts users for setup if needed
- Added a new LibationUiBase.Tests test project with tests for various
- Add Book.IsSpatial property and add it to search index
- Read audio format of actual output files and store it in UserDefinedItem. Now works with MP3s.
- Store last downloaded audio file version
- Add IsSpatial, file version, and Audio Format to library exports and to template tags. Updated docs.
- Add last downloaded audio file version and format info to the Last Downloaded tab
- Migrated the DB
- Update AAXClean with some bug fixes
- Fixed error converting xHE-AAC audio files to mp3 when splitting by chapter (or trimming the audible branding from the beginning of the file)
- Improve mp3 ID# tags support. Chapter titles are now preserved.
- Add support for reading EC-3 and AC-4 audio format metadata
Robert McRackan