The dialog names the account in full because it is shown to whoever owns it, but
log files get attached to public issue reports, which is why the codebase has
MaskedLogEntry. Naming the account in the log the same way the dialog does would
have put real email addresses into every shared log.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
This reverts 6daaf33d. Master is 13.7.8 and the next release is the 0.0.1
increment from it, so the original 13.7.9 references were correct.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Both grids restored the last good filter by recursing into the filter handler, which
never terminated once the search index rather than the query was the problem: the
restore fails the same way, and the retry uses the same filter. The user got an
endless run of dialogs, each of them blaming a filter string that was fine. Only an
empty last-good filter broke the loop, because that short-circuits before reaching
the search engine.
The fallback is now a bounded sequence -- last good filter, then no filter -- and
the message distinguishes an index Libation cannot reach from a query it cannot
parse. Only the first failure is reported, so restoring is quiet. A malformed query
never surfaces as an IO-family exception, which QueryFailureShapeTests pins against
the real engine, so a typo is never mistaken for index trouble or made to trigger a
rebuild.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
The two path assertions added with the PDF fix compared a path the test built
itself against one that had been through LongPath, which on Windows prefixes a
drive-rooted path with \\?\ so paths past the 260 character limit work. Linux
adds no prefix, so this only showed up on the Windows job.
Normalising both sides is not just about the false failure. The inequality
assertion guarding 'the PDF was saved loose in the Books directory' compared a
raw temp path against a prefixed one, so on Windows it passed on the prefix
alone and would not have caught the bug it exists to catch.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Windows CI caught real over-reach. When another holder has write.lock, Windows
raises the sharing violation before Lucene can turn it into a
LockObtainFailedException, so it arrives as a plain IOException. Repairing anything
that is not a recognised lock conflict then meant deleting the index the other
holder was using -- exactly the second-instance case the retry exists for.
An IOException naming Lucene's write lock now counts as a lock conflict. Matching
the file name rather than the message wording keeps it working on non-English
Windows. The end-to-end test asserts the property instead of the exception type,
since the type legitimately differs by platform.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Releasing the write.lock part way through the retry budget raced with Lucene 3's
own lock bookkeeping: on Windows a competing Obtain left a handle on the file, so
Release and the temp directory cleanup both failed with a sharing violation. Hold
the lock for the whole budget instead and assert what actually matters, that a lock
conflict is retried and leaves the index files alone.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A PDF is fetched through the same license request as the audiobook, so following
a refusal with a PDF request reproduced, through the PDF, exactly the per-run
refusal the wait exists to stop. The follow-up pass now skips the titles the
first pass deliberately left alone as well as the ones it attempted.
Also stop a failed PDF download leaving an empty folder in the library: a
PDF-only download is the one case that has to create the book's folder before it
has anything to put in it, so it now removes a folder it created and did not
fill. GetProposedDownloadFilePath goes back to being a pure path computation.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A plain 'libationcli liberate' iterates the titles DownloadDecryptBook selects,
and that step selects on '!AudioExists'. A title needing nothing but its PDF was
therefore never reached by the verb documented as 'book and pdf backups' - only
'liberate --pdf' picked it up. For a library that was liberated before its PDFs
were, that is every title with a PDF.
Give the bulk run an optional second pass and have liberate use it for PDFs, the
way the app's Liberate All always has. Skipped when the first pass stopped early
so a run cut short by its download limit does not carry on doing other work, and
titles the first pass attempted are excluded by product id rather than by asking
Validate again, so a step that just failed is not immediately retried.
Left alone: the Audiobookshelf upload stays tied to a fresh liberation. Its
Validate passes for any liberated title, so including it here would walk the
whole library on the next run. 'abs upload' already exists for that.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
getProposedDownloadFilePath looked for the book's audio file and fell back to
the Books directory itself when it found none. That lookup matches on the
product id appearing in the path, so it finds nothing for a library whose
folder and file templates omit <id>, and nothing for a title marked downloaded
whose files are not on this machine. Those PDFs landed in the library root,
where they also shared one namespace and so could collide with each other.
Fall back to the folder template instead - the same folder the audiobook itself
would go in - and create it, since nothing else does on the PDF-only path.
Also give MockLibraryBook a three-field version: ToVersionString formats to at
least three fields, so the two-field default threw as soon as anything rendered
a naming template for a mock book.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Ported from #1949. The reporter's log paused auto-scan on a second account that
had never been logged in, while the dialog blamed an expired session and named no
account, so there was nothing to act on.
AccountCredentialStatus tells a never-registered account apart from one holding an
expired access token, by looking for a refresh token to renew from. AutoScanRunner
now hands the AuthenticationRequiredException to the notification so the prompt can
name the account, which means digging that exception back out of the wrappers the
scan adds on the way up. Same distinction in the log line and in the exception
message ApiExtended throws when interactive login is unavailable, which is what the
CLI and Docker users see.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Ported from #1949, which surfaces the manual recovery steps the maintainer had
been giving out by hand instead of leaving the user with a raw Lucene error.
Adapted to the failure now being contained: with the exception no longer escaping
into the library change, the scan-failure catch blocks #1949 hooked would never
see it, and hooking only those would still miss every other trigger -- removing
books is what crashed the GUI. So the guard moves from AppScaffolding into
SearchEngineCommands next to the update commands it protects, and raises
UpdateFailed from there. Both GUIs subscribe, so any trigger is covered, and the
event carries the exception rather than needing #1949's StackTrace string sniffing
to find it. The dialog is shown once per session: a damaged index fails on every
library change and these steps only need following once.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Fold the pdf-only exclusion into HonorsDeferredRetries instead of also checking
the processable type in the run loop, split the user-facing message building
into its own file next to the store, and leave GC.Collect on the success path
where it was.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Lucene 3's base-36 filename formatter overruns its buffer when segments.gen names
an absurd generation, so the rebuild path has to survive an IndexOutOfRangeException
as well as the IOException shapes. Damage does not always announce itself as IO.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Asserting on the generated JSON alone would pass just as happily with a
misspelled sink argument, which Serilog ignores in silence - and silently not
rolling is the bug. These build a real logger from Libation's own config and
write until it rolls, including a test that pins the old unbounded behaviour so
a future change to the defaults cannot quietly restore it.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
The classifier's inputs are the actual denials from the log attached to issue
#1947: owned titles on an inactive account, a Plus title no longer in the
catalog, and an unreleased preorder Audible has no audio for.
The backoff tests also caught a real overflow: first * 2^n exceeds a TimeSpan
long before the cap matters.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A license denial left no trace: BookStatus stayed NotLiberated, so every
liberate run asked again. Only the GUI's bad-book dialog could mark a title
Error, and license denials take their own path and never reach that dialog, so
a headless install had no way at all to stop the retries. A cron schedule then
re-requested the same refused licenses every run and printed the same warning
block for each one, which is both wasted API traffic and the log noise reported
in issue #1947.
Record the refusal instead, with a wait that doubles per consecutive failure:
one day for an eligibility denial (up to 30), six hours for a title Audible has
no audio for yet such as an unreleased preorder (up to 7 days), one hour when
the denial names GenericError, which the GUI already reads as an outage
(up to 12). Nothing is permanent - every kind is attempted again on its own.
Only failures attributable to Audible are recorded. A dropped connection, a
decrypt error or a full disk keeps being retried on the next run as before.
Naming a title, --force, and setting a download status all clear the record:
asking for a title explicitly overrides the wait.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A truncated or zero-length segments file is reported by Lucene 3 as a plain
IOException ("read past EOF") rather than a CorruptIndexException, so it was
misclassified as a write.lock conflict: CreateNewIndex burned its whole backoff
budget and rethrew, and the delete-and-rebuild recovery never ran. Passing
create/overwrite to IndexWriter does not repair it either, because
IndexFileDeleter reads every segments_* file in the directory and tolerates only
missing ones, so a single unreadable segments file -- even a stale one from an
older commit -- leaves the index permanently unopenable. The user's only cure
was deleting the SearchEngine folder by hand.
Retries are now reserved for genuine lock conflicts (LockObtainFailedException,
which derives from IOException, and UnauthorizedAccessException), and any other
open failure gets one delete-and-rebuild pass before giving up with a message
that says which folder to remove. The query path recovers too, since
IsRecoverableCorruptIndexException now recognizes the truncated-segments
signature.
Search index updates are also no longer allowed to fail the library change that
triggered them. Both events fire after the database is committed, so an escaping
exception reported a successful scan as "Error importing library" and, being the
first subscriber, stopped the handlers that refresh the grid and backup counts.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
The default Serilog config set rollingInterval only, so Serilog's own defaults
applied: no size-based roll and a 1 GB ceiling after which the sink silently
stops writing. A busy install (many accounts scanned several times an hour)
reaches tens of MB in a month, past the point where the log can be attached to
a bug report.
Add fileSizeLimitBytes, rollOnFileSizeLimit and retainedFileCountLimit to the
default File sink, and fill in whichever of the three an existing Settings.json
is missing so installs that already have a Serilog section benefit too. Only
absent keys are written, so a hand-tuned config is left alone.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Drop the series-parent reason, which no caller can reach: parents are expanded
into their children before anything is queued. Replace the enum and its three
switches with a record carrying the label and the advice, which is all the
switches were mapping to.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
"Absent from your last library scan (run Scan, ...): 2" buries the number behind a
parenthetical. Lead with the label and count, then the advice.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
The menu item is enabled from the grid's display status, so it can be clicked for
episodes the queue will reject - episodes absent from the last scan are the usual
case. Pre-filtering the children threw that reason away and left the queue with an
empty request it could only answer in general terms.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Names the local function ProcessOrStopAsync so it does not shadow
Processable.TryProcessAsync, which means something else entirely, and asserts
the whole sentence a stopped run prints rather than a fragment of it.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Two levels. The pure tests supply a run's history directly and check the
decision to stop: the count and size thresholds, the always-allow-one rule,
and that titles another process downloaded, or that this run attempted
without downloading, are not counted.
The loop tests drive the real run loop against a real library database with
a processable that records downloads the way DownloadDecryptBook does, which
is as close to a limited run as is possible without an Audible account. They
pin the behavior that matters at the boundary: five books under a limit of
two download two, and a run whose books end exactly at its limit reports
nothing, because nothing was cut short.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
The multi-book branch of QueueDownloadDecryptAsync returned false with no log
entry and no message whenever UnLiberated() came back empty, so a request
Libation understood and declined looked exactly like a dead button. Callers that
pre-filter with UnLiberated() land here with an empty list, which is the common
way to hit it.
Classify each title that cannot be queued and report the breakdown: already
downloaded, previously failed, absent from the last scan, or a series parent with
no audio of its own. Log it either way; show it only when a person is waiting,
so the automatic post-scan download stays silent.
Fixes#1940
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Adds three mutually exclusive options to `liberate`: --limit-books,
--limit-mb and --limit-gb. Each stops the run once it has downloaded that
much, leaving the rest of the library un-liberated for the next run.
Requested in #1920: a scripted or scheduled run has no way to take only a
slice of a large library, so users resort to feeding the CLI a handful of
ASINs at a time. The GUI needs no equivalent, where selecting rows already
says exactly which titles to download.
Counting reuses the daily limit's history rows rather than a private tally,
so a book and a byte mean the same thing to both limits, and failed,
cancelled and pdf-only work is never counted. Only titles this run attempted
are counted, so a Libation window or a second container downloading at the
same time does not consume this run's allowance.
The limit is checked before each title rather than at the top of the run, so
a run whose books happen to end exactly at the limit reports nothing: nothing
was cut short. The daily download limit keeps applying on top, unchanged.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Cancel All sets a flag the daily-limit wait loop watches. If books were queued
while the cancelled book was still settling, the queue was still running, so
those new books inherited the cancellation at the gate. Queueing work now
clears the flag.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Adds a feature page covering the rolling window, what counts, the MB/GB
estimate and the Docker/CLI keys, plus a one-line pointer from getting started
where a new user with a large Plus library clicks Begin Book Backups.
Scripts/seed-download-history.cs seeds fake completed downloads so the limit
can be exercised without downloading, including dating rows just under 24 hours
old to turn the multi-day pause-and-resume behavior into a one minute test.
Also shortens the queue status text: the process queue column clips rather than
wrapping, so the resume time was being cut off.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Avalonia and WinForms both get a Daily download limit group on the
Download/Decrypt tab: scope drop-down, and when a limit is chosen a quantity
spinner (minimum 1, no practical maximum) plus a Books/MB/GB unit, with the
MB/GB approximation note shown only for those units.
A license denial that looks like Audible throttling now suggests turning the
limit on, quoting the real number of downloads Libation recorded in the last 24
hours. Audible reports no distinct throttling reason, so the suggestion stays
silent unless that record makes throttling plausible.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Records every successful audiobook download in the library database (a new
DownloadHistory table) and, when the user opts in, stops downloading once the
rolling 24 hour window is full.
The history lives in the database rather than a file under LibationFiles
because in Docker only the database is on a volume; a file there is discarded
on every container restart.
The limit is checked immediately before each book downloads, never at queueing
time, so a full queue stays full and the user can raise or disable the limit
mid-run. When nothing in the queue can proceed the queue pauses and re-checks
every 15 seconds, recomputing settings, history and clock from scratch, so a
queue left running for days drip-feeds itself as downloads age out. The CLI
never waits: it skips covered titles and reports a count.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Empty-args usage now lists nested command groups alongside root verbs.
Nested `abs upload --help` brands the public command path instead of the
internal parser verb.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
The Liberate icon offers an action rather than reporting a state, so an expanded
series shows a minus to collapse it and a collapsed one shows a plus to expand
it. Naming the geometry after the state it belonged to had these swapped.
Name it after its shape instead, and pin the direction with a test: the plus is
the minus plus a bar, so it is strictly the inkier of the two.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Cover every icon in the finite set: each renders a valid PNG, no two render
alike, stoplights keep a common height, a PDF overlay only widens them, and
repeat requests are served from the cache.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
