Stephen King's 11/22/63 sits either side of the line the number handling draws.
The analyzer keeps a date like this as one token, so the title is text and must
not be zero-padded, while "title:11*" is a bare number in a text field and must
not be padded either - that one fails against the code from before the numeric
fix, going looking for "title:00000011.00*".
Audible lists it with slashes and a hyphenated spelling turns up elsewhere, so
both are covered, as are books 11, 22 and 63 minutes long so that nothing can
pass by matching a number field instead of the title.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
It was a generic primer whose examples used title, body and mod_date. Only one
of those exists here, so half the guide could not be typed into the search box.
Every example is now a real field and was run against an index before being
written down, including the exclusive range and the leading-wildcard error the
old text described but never showed.
Covers the two forms that reach the new number handling and had no test: a
wildcard on a numeric title, and a curly-brace range.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A bare number has to keep every book it already matched and add the one whose
title is that number. The library had a book 14 minutes long but none 14 hours
long, so the Hours field went untested - the same field that made an earlier
draft of these tests pass against the bug by accident.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Lucene 3 has no numeric type, so a number field is indexed zero-padded to make
a range sort: 600 minutes is stored as "00000600.00". Every other field keeps
the number as written, so the novel "1984" is stored as "1984". Padding every
number in a query regardless of the field it was being compared against made a
numeric title unfindable - "title:1984" looked for "00001984.00" among the
titles - and a bare "1984" could only ever mean "some number field is 1984".
A number now takes the spelling its field is indexed in. With no field named,
the default field holds both spellings, so both are searched. A range and a
phrase cannot hold that disjunction: a range stays padded, a phrase does not.
Expanding a bare number to a group exposed the pure-negation workaround, which
judged a query by every clause in its tree, so "-(a OR b)" was read as mixed
and returned nothing. It is now decided per query by that query's own clauses.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Search field names are ordinary words - absent, podcast, plus, series - and
a book's own text is full of them. The sanitizer read any known field name as
a field name wherever it appeared, so a value in that position picked up the
implied ":True" of a bool field: "title:absent" became "title:absent:True",
which Lucene refuses to parse, and "title:\"absent friend\"" quietly matched
nothing. A colon straight after a field name now marks the next term as that
field's value.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A book the index does not hold cannot be found by any positive filter, and
every negated filter - which resolves to every document in the index - drops
it from the grid instead. That is why issue #1989 reads as half a working
filter: Absent found nothing while -Absent removed exactly the absent books.
The index is only ever written as a whole, and a rebuild that fails is
deliberately swallowed so a bad index cannot fail a good scan, so a short
index stayed short until something else changed the library. Count the
index against the library once per run and rebuild when it is short.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Libation already runs its tests on Microsoft.Testing.Platform via the MSTest
metapackage; this only moves the version off 4.2.2 so all three repos name the
same MSTest release.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
<title short> stops at the first colon, so it shortens Audible titles that
contain one just as readily as it drops Audible's subtitle, and distinct books
then collapse onto the same name. A colon cannot be searched for: the analyzer
discards punctuation and Lucene reads a colon in a query as a field separator.
Two bool index fields find the affected books instead.
Document how the two title tags differ, since <audible title> already drops
Audible's subtitle without ever cutting a title, and how to audit for names
that actually collide in a spreadsheet export.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Both grids restored the last good filter by recursing into the filter handler, which
never terminated once the search index rather than the query was the problem: the
restore fails the same way, and the retry uses the same filter. The user got an
endless run of dialogs, each of them blaming a filter string that was fine. Only an
empty last-good filter broke the loop, because that short-circuits before reaching
the search engine.
The fallback is now a bounded sequence -- last good filter, then no filter -- and
the message distinguishes an index Libation cannot reach from a query it cannot
parse. Only the first failure is reported, so restoring is quiet. A malformed query
never surfaces as an IO-family exception, which QueryFailureShapeTests pins against
the real engine, so a typo is never mistaken for index trouble or made to trigger a
rebuild.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Windows CI caught real over-reach. When another holder has write.lock, Windows
raises the sharing violation before Lucene can turn it into a
LockObtainFailedException, so it arrives as a plain IOException. Repairing anything
that is not a recognised lock conflict then meant deleting the index the other
holder was using -- exactly the second-instance case the retry exists for.
An IOException naming Lucene's write lock now counts as a lock conflict. Matching
the file name rather than the message wording keeps it working on non-English
Windows. The end-to-end test asserts the property instead of the exception type,
since the type legitimately differs by platform.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Releasing the write.lock part way through the retry budget raced with Lucene 3's
own lock bookkeeping: on Windows a competing Obtain left a handle on the file, so
Release and the temp directory cleanup both failed with a sharing violation. Hold
the lock for the whole budget instead and assert what actually matters, that a lock
conflict is retried and leaves the index files alone.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Lucene 3's base-36 filename formatter overruns its buffer when segments.gen names
an absurd generation, so the rebuild path has to survive an IndexOutOfRangeException
as well as the IOException shapes. Damage does not always announce itself as IO.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A truncated or zero-length segments file is reported by Lucene 3 as a plain
IOException ("read past EOF") rather than a CorruptIndexException, so it was
misclassified as a write.lock conflict: CreateNewIndex burned its whole backoff
budget and rethrew, and the delete-and-rebuild recovery never ran. Passing
create/overwrite to IndexWriter does not repair it either, because
IndexFileDeleter reads every segments_* file in the directory and tolerates only
missing ones, so a single unreadable segments file -- even a stale one from an
older commit -- leaves the index permanently unopenable. The user's only cure
was deleting the SearchEngine folder by hand.
Retries are now reserved for genuine lock conflicts (LockObtainFailedException,
which derives from IOException, and UnauthorizedAccessException), and any other
open failure gets one delete-and-rebuild pass before giving up with a message
that says which folder to remove. The query path recovers too, since
IsRecoverableCorruptIndexException now recognizes the truncated-segments
signature.
Search index updates are also no longer allowed to fail the library change that
triggered them. Both events fire after the database is committed, so an escaping
exception reported a successful scan as "Error importing library" and, being the
first subscriber, stopped the handlers that refresh the grid and backup counts.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
- Update all project runtime targets
- Update all dependencies
- NOTE: Using Npgsql.EntityFrameworkCore.PostgreSQL RTM build from MyGet
- Delete unused pubxml files (they were made redundant by recent workflow changes)
- Replace Libation.sln with Libation.slnx
Robert McRackan