TrackedQueue<T> is a pure data structure with no dependencies beyond
System, so the behaviour parallel downloads relies on can be asserted
directly rather than inferred from the UI.
Every case here needs more than one book active at once, which is what
made these defects unreachable before. The two that mattered: a book
finishing out of order now reports the reorder, so a list bound only to
CollectionChanged stays in step with the queue; and the queue can be
enumerated while it is being mutated, which used to throw.
The sequential path is covered too - one book at a time still raises no
Move, because the book that finishes is already first.
Same unlocked-enumeration problem as the others: the speed limit is
changed from the UI thread while book tasks are starting and finishing,
and Active is the live list.
Faulted book tasks were dropped unobserved. The reaping pass removes any
completed task from the active set before the closing WhenAll can
rethrow, so an exception out of ProcessOneAsync - which can happen via
GetFailureActionAsync in its finally - went nowhere. In the sequential
loop it reached the outer catch and was logged. It is logged again.
The bad-book dialog did not survive concurrency. The license and
Widevine messages are guarded to appear once per run, but the "skip this
book?" dialog is per book and its "apply to all remaining books" answer
lands in a shared BadBookSessionContext. Three books failing together
put three modals on screen racing to set the same override. The dialog
is now serialised on the session, and each book re-checks the override
after its turn comes: if the book ahead answered "apply to all", the
question is not asked again.
Machine capability no longer overwrites the stored concurrency setting.
MaxAllowedConcurrentDownloads clamped both the getter and the setter, so
a user who chose 8 and then opened the same config on a two-core machine
read back 2 - and had 2 written over their 8. The stored value is now
bounded only by the hard limit, which is machine-independent; processor
count bounds the spinner and is applied at the point of use, where the
loop decides how many books to run.
The opt-in daily download limit landed after this branch was written and
lives inside the sequential while (Queue.MoveNext()) loop this change
replaces. It is sequential by construction, so rebasing alone leaves it
subtly wrong rather than merely conflicted.
The gate now runs in the dispatch loop, between taking a book off the
queue and starting its task. That keeps the existing semantics - checked
as a book is about to start so the queue keeps its contents and the
limit can be changed mid-run - while books already in flight carry on.
Putting it inside the book task instead would have every blocked book
polling the history at once.
Three concrete defects that fell out of the collision:
RequeueLast deferred a book with Queue.ClearCurrent(), which drops
Active[0]. With one book at a time that is the book being deferred; with
three in flight it is somebody else's download, so deferring the second
active book silently evicted the first. It now removes the book it was
given.
A book cancelled at the gate was left on the active list. The sequential
loop retired it on the next MoveNext(); there is no next MoveNext(), so
it is marked completed explicitly.
CancelAllAsync existed twice after the rebase - the sequential version
that sets cancelAllRequested and cancels Queue.Current, and this
branch's version that cancels every active book. Unified into one that
does both. The flag matters: a queue paused on the limit is sitting in
WaitForDailyLimitAsync and that flag is how it learns to stop.
AnyOtherQueuedBookAllowed enumerates the queue with Queue.Any(...) while
book tasks mutate it; that is safe now that GetAllItems snapshots under
the lock, and it also now takes a copy of the active list to cancel.
Both of these are reachable only once more than one book is active at a
time, and both were found by compiling TrackedQueue<T> on its own.
MarkCompleted moved a book from Active to Completed and raised only
CompletedCountChanged. The display index of an active book is derived
from Completed.Count, so when the second of two active books finishes
first the two swap places - with no CollectionChanged, a bound list
keeps painting the old order and rows show the wrong book's progress.
Out-of-order completion is the normal case with parallel downloads. Now
raises a Move for the book whose position actually changed; with one
book at a time the indices are equal and nothing is raised, so the
sequential path behaves exactly as before.
GetAllItems built a lazy Concat inside the lock and returned it, so the
enumeration ran unlocked. Any foreach or LINQ over the queue while a
book task mutated it threw InvalidOperationException. It now returns a
snapshot taken while the lock is held. This is the source of the crash
that setLiberatedVisibleMenuItem was patched for at the symptom.
RemoveActive and ClearCurrent also read _completed.Count outside the
lock when computing the index to report; both now capture it inside.
The queue loop only woke when a book finished, so books queued a moment
after it started sat idle until an in-flight download happened to
complete. Observed as one book downloading alone for 15 seconds with a
limit of 5, then four starting within 37ms of the first finishing.
The loop now waits on whichever comes first: an active task completing,
or a new book being queued. The enqueue signal is captured before the
queue is inspected, so a book queued between TryDequeueNext and the wait
completes the token already held rather than a fresh one - otherwise the
wakeup is lost and the loop sleeps until something else finishes. The
'nothing left, exit' path re-checks the same signal for the same reason.
The 'At once' spinner sat at x136-181 on the second row, but the
'DL Limit:' label starts at x148. Moves the label and spinner up to the
first row beside the Auto-scroll checkbox, where there is clear space
between Cancel All and Clear Finished.
Chardonnay had parallel downloads with no way to configure them, since
the queue logic lives in shared UI code but each UI supplies its own
controls. Adds the Auto-scroll toggle and the 'At once' spinner to
Chardonnay's queue panel, bound to the same view model properties the
WinForms panel uses.
Also uses Environment.ProcessorCount as the spinner's ceiling rather
than its default: min(ProcessorCount, 10). Downloading is bound by
Audible's license throttling rather than local CPU, so core count says
nothing about how many concurrent downloads will succeed - it only
bounds how many decrypts can usefully run at once. The default stays 3.
Spinner bounds are bound rather than hardcoded, so the two UIs cannot
drift apart.
Replaces the MultiThreadEnabled checkbox and the ProcessorCount default
with a single MaxConcurrentDownloads value where 1 means serial - the
behaviour Libation had before parallel downloads existed.
The old pairing could not express 'off' at all: the bool was hardcoded
true in the view model constructor and the int setter clamped to a
minimum of 2, so there was no way back to one-at-a-time downloads.
Collapsing both into one value makes that state unreachable rather than
merely fixed, and removes the risk of the two settings disagreeing.
Bounds live in Configuration as named constants: minimum 1, default 3,
maximum 10. Audible throttles license requests, so the default is
deliberately conservative and the cap keeps users from choosing a number
that produces license denials instead of speed.
WinForms swaps the 'Parallel downloads' checkbox for an 'At once:'
spinner that reads its bounds from those constants.
ClearQueue() only stops new work from starting. With parallel downloads,
books already running kept going after an abort or a disk-full result,
so the queue reported itself stopped while downloads continued.
Adds ProcessQueueViewModel.CancelAllAsync(), which clears the queue and
cancels every active book, and uses it for both the abort and disk-full
paths. Both Cancel All buttons now route through it as well - the Avalonia
one previously cancelled only Queue.Current, so it missed every book but
the first.
Snapshot GetVisible() on the UI thread before handing off to Task.Run,
preventing InvalidOperationException when parallel downloads modify the
collection concurrently. Also fix installer script to use 64-bit dotnet.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The dialog names the account in full because it is shown to whoever owns it, but
log files get attached to public issue reports, which is why the codebase has
MaskedLogEntry. Naming the account in the log the same way the dialog does would
have put real email addresses into every shared log.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
This reverts 6daaf33d. Master is 13.7.8 and the next release is the 0.0.1
increment from it, so the original 13.7.9 references were correct.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Both grids restored the last good filter by recursing into the filter handler, which
never terminated once the search index rather than the query was the problem: the
restore fails the same way, and the retry uses the same filter. The user got an
endless run of dialogs, each of them blaming a filter string that was fine. Only an
empty last-good filter broke the loop, because that short-circuits before reaching
the search engine.
The fallback is now a bounded sequence -- last good filter, then no filter -- and
the message distinguishes an index Libation cannot reach from a query it cannot
parse. Only the first failure is reported, so restoring is quiet. A malformed query
never surfaces as an IO-family exception, which QueryFailureShapeTests pins against
the real engine, so a typo is never mistaken for index trouble or made to trigger a
rebuild.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
The two path assertions added with the PDF fix compared a path the test built
itself against one that had been through LongPath, which on Windows prefixes a
drive-rooted path with \\?\ so paths past the 260 character limit work. Linux
adds no prefix, so this only showed up on the Windows job.
Normalising both sides is not just about the false failure. The inequality
assertion guarding 'the PDF was saved loose in the Books directory' compared a
raw temp path against a prefixed one, so on Windows it passed on the prefix
alone and would not have caught the bug it exists to catch.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Windows CI caught real over-reach. When another holder has write.lock, Windows
raises the sharing violation before Lucene can turn it into a
LockObtainFailedException, so it arrives as a plain IOException. Repairing anything
that is not a recognised lock conflict then meant deleting the index the other
holder was using -- exactly the second-instance case the retry exists for.
An IOException naming Lucene's write lock now counts as a lock conflict. Matching
the file name rather than the message wording keeps it working on non-English
Windows. The end-to-end test asserts the property instead of the exception type,
since the type legitimately differs by platform.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Releasing the write.lock part way through the retry budget raced with Lucene 3's
own lock bookkeeping: on Windows a competing Obtain left a handle on the file, so
Release and the temp directory cleanup both failed with a sharing violation. Hold
the lock for the whole budget instead and assert what actually matters, that a lock
conflict is retried and leaves the index files alone.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A PDF is fetched through the same license request as the audiobook, so following
a refusal with a PDF request reproduced, through the PDF, exactly the per-run
refusal the wait exists to stop. The follow-up pass now skips the titles the
first pass deliberately left alone as well as the ones it attempted.
Also stop a failed PDF download leaving an empty folder in the library: a
PDF-only download is the one case that has to create the book's folder before it
has anything to put in it, so it now removes a folder it created and did not
fill. GetProposedDownloadFilePath goes back to being a pure path computation.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A plain 'libationcli liberate' iterates the titles DownloadDecryptBook selects,
and that step selects on '!AudioExists'. A title needing nothing but its PDF was
therefore never reached by the verb documented as 'book and pdf backups' - only
'liberate --pdf' picked it up. For a library that was liberated before its PDFs
were, that is every title with a PDF.
Give the bulk run an optional second pass and have liberate use it for PDFs, the
way the app's Liberate All always has. Skipped when the first pass stopped early
so a run cut short by its download limit does not carry on doing other work, and
titles the first pass attempted are excluded by product id rather than by asking
Validate again, so a step that just failed is not immediately retried.
Left alone: the Audiobookshelf upload stays tied to a fresh liberation. Its
Validate passes for any liberated title, so including it here would walk the
whole library on the next run. 'abs upload' already exists for that.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
getProposedDownloadFilePath looked for the book's audio file and fell back to
the Books directory itself when it found none. That lookup matches on the
product id appearing in the path, so it finds nothing for a library whose
folder and file templates omit <id>, and nothing for a title marked downloaded
whose files are not on this machine. Those PDFs landed in the library root,
where they also shared one namespace and so could collide with each other.
Fall back to the folder template instead - the same folder the audiobook itself
would go in - and create it, since nothing else does on the PDF-only path.
Also give MockLibraryBook a three-field version: ToVersionString formats to at
least three fields, so the two-field default threw as soon as anything rendered
a naming template for a mock book.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Ported from #1949. The reporter's log paused auto-scan on a second account that
had never been logged in, while the dialog blamed an expired session and named no
account, so there was nothing to act on.
AccountCredentialStatus tells a never-registered account apart from one holding an
expired access token, by looking for a refresh token to renew from. AutoScanRunner
now hands the AuthenticationRequiredException to the notification so the prompt can
name the account, which means digging that exception back out of the wrappers the
scan adds on the way up. Same distinction in the log line and in the exception
message ApiExtended throws when interactive login is unavailable, which is what the
CLI and Docker users see.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Ported from #1949, which surfaces the manual recovery steps the maintainer had
been giving out by hand instead of leaving the user with a raw Lucene error.
Adapted to the failure now being contained: with the exception no longer escaping
into the library change, the scan-failure catch blocks #1949 hooked would never
see it, and hooking only those would still miss every other trigger -- removing
books is what crashed the GUI. So the guard moves from AppScaffolding into
SearchEngineCommands next to the update commands it protects, and raises
UpdateFailed from there. Both GUIs subscribe, so any trigger is covered, and the
event carries the exception rather than needing #1949's StackTrace string sniffing
to find it. The dialog is shown once per session: a damaged index fails on every
library change and these steps only need following once.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Fold the pdf-only exclusion into HonorsDeferredRetries instead of also checking
the processable type in the run loop, split the user-facing message building
into its own file next to the store, and leave GC.Collect on the success path
where it was.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Lucene 3's base-36 filename formatter overruns its buffer when segments.gen names
an absurd generation, so the rebuild path has to survive an IndexOutOfRangeException
as well as the IOException shapes. Damage does not always announce itself as IO.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Asserting on the generated JSON alone would pass just as happily with a
misspelled sink argument, which Serilog ignores in silence - and silently not
rolling is the bug. These build a real logger from Libation's own config and
write until it rolls, including a test that pins the old unbounded behaviour so
a future change to the defaults cannot quietly restore it.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
The classifier's inputs are the actual denials from the log attached to issue
#1947: owned titles on an inactive account, a Plus title no longer in the
catalog, and an unreleased preorder Audible has no audio for.
The backoff tests also caught a real overflow: first * 2^n exceeds a TimeSpan
long before the cap matters.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A license denial left no trace: BookStatus stayed NotLiberated, so every
liberate run asked again. Only the GUI's bad-book dialog could mark a title
Error, and license denials take their own path and never reach that dialog, so
a headless install had no way at all to stop the retries. A cron schedule then
re-requested the same refused licenses every run and printed the same warning
block for each one, which is both wasted API traffic and the log noise reported
in issue #1947.
Record the refusal instead, with a wait that doubles per consecutive failure:
one day for an eligibility denial (up to 30), six hours for a title Audible has
no audio for yet such as an unreleased preorder (up to 7 days), one hour when
the denial names GenericError, which the GUI already reads as an outage
(up to 12). Nothing is permanent - every kind is attempted again on its own.
Only failures attributable to Audible are recorded. A dropped connection, a
decrypt error or a full disk keeps being retried on the next run as before.
Naming a title, --force, and setting a download status all clear the record:
asking for a title explicitly overrides the wait.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A truncated or zero-length segments file is reported by Lucene 3 as a plain
IOException ("read past EOF") rather than a CorruptIndexException, so it was
misclassified as a write.lock conflict: CreateNewIndex burned its whole backoff
budget and rethrew, and the delete-and-rebuild recovery never ran. Passing
create/overwrite to IndexWriter does not repair it either, because
IndexFileDeleter reads every segments_* file in the directory and tolerates only
missing ones, so a single unreadable segments file -- even a stale one from an
older commit -- leaves the index permanently unopenable. The user's only cure
was deleting the SearchEngine folder by hand.
Retries are now reserved for genuine lock conflicts (LockObtainFailedException,
which derives from IOException, and UnauthorizedAccessException), and any other
open failure gets one delete-and-rebuild pass before giving up with a message
that says which folder to remove. The query path recovers too, since
IsRecoverableCorruptIndexException now recognizes the truncated-segments
signature.
Search index updates are also no longer allowed to fail the library change that
triggered them. Both events fire after the database is committed, so an escaping
exception reported a successful scan as "Error importing library" and, being the
first subscriber, stopped the handlers that refresh the grid and backup counts.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
The default Serilog config set rollingInterval only, so Serilog's own defaults
applied: no size-based roll and a 1 GB ceiling after which the sink silently
stops writing. A busy install (many accounts scanned several times an hour)
reaches tens of MB in a month, past the point where the log can be attached to
a bug report.
Add fileSizeLimitBytes, rollOnFileSizeLimit and retainedFileCountLimit to the
default File sink, and fill in whichever of the three an existing Settings.json
is missing so installs that already have a Serilog section benefit too. Only
absent keys are written, so a hand-tuned config is left alone.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Drop the series-parent reason, which no caller can reach: parents are expanded
into their children before anything is queued. Replace the enum and its three
switches with a record carrying the label and the advice, which is all the
switches were mapping to.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
"Absent from your last library scan (run Scan, ...): 2" buries the number behind a
parenthetical. Lead with the label and count, then the advice.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
The menu item is enabled from the grid's display status, so it can be clicked for
episodes the queue will reject - episodes absent from the last scan are the usual
case. Pre-filtering the children threw that reason away and left the queue with an
empty request it could only answer in general terms.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Names the local function ProcessOrStopAsync so it does not shadow
Processable.TryProcessAsync, which means something else entirely, and asserts
the whole sentence a stopped run prints rather than a fragment of it.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Two levels. The pure tests supply a run's history directly and check the
decision to stop: the count and size thresholds, the always-allow-one rule,
and that titles another process downloaded, or that this run attempted
without downloading, are not counted.
The loop tests drive the real run loop against a real library database with
a processable that records downloads the way DownloadDecryptBook does, which
is as close to a limited run as is possible without an Audible account. They
pin the behavior that matters at the boundary: five books under a limit of
two download two, and a run whose books end exactly at its limit reports
nothing, because nothing was cut short.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
The multi-book branch of QueueDownloadDecryptAsync returned false with no log
entry and no message whenever UnLiberated() came back empty, so a request
Libation understood and declined looked exactly like a dead button. Callers that
pre-filter with UnLiberated() land here with an empty list, which is the common
way to hit it.
Classify each title that cannot be queued and report the breakdown: already
downloaded, previously failed, absent from the last scan, or a series parent with
no audio of its own. Log it either way; show it only when a person is waiting,
so the automatic post-scan download stays silent.
Fixes#1940
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Adds three mutually exclusive options to `liberate`: --limit-books,
--limit-mb and --limit-gb. Each stops the run once it has downloaded that
much, leaving the rest of the library un-liberated for the next run.
Requested in #1920: a scripted or scheduled run has no way to take only a
slice of a large library, so users resort to feeding the CLI a handful of
ASINs at a time. The GUI needs no equivalent, where selecting rows already
says exactly which titles to download.
Counting reuses the daily limit's history rows rather than a private tally,
so a book and a byte mean the same thing to both limits, and failed,
cancelled and pdf-only work is never counted. Only titles this run attempted
are counted, so a Libation window or a second container downloading at the
same time does not consume this run's allowance.
The limit is checked before each title rather than at the top of the run, so
a run whose books happen to end exactly at the limit reports nothing: nothing
was cut short. The daily download limit keeps applying on top, unchanged.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Cancel All sets a flag the daily-limit wait loop watches. If books were queued
while the cancelled book was still settling, the queue was still running, so
those new books inherited the cancellation at the gate. Queueing work now
clears the flag.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Adds a feature page covering the rolling window, what counts, the MB/GB
estimate and the Docker/CLI keys, plus a one-line pointer from getting started
where a new user with a large Plus library clicks Begin Book Backups.
Scripts/seed-download-history.cs seeds fake completed downloads so the limit
can be exercised without downloading, including dating rows just under 24 hours
old to turn the multi-day pause-and-resume behavior into a one minute test.
Also shortens the queue status text: the process queue column clips rather than
wrapping, so the resume time was being cut off.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Avalonia and WinForms both get a Daily download limit group on the
Download/Decrypt tab: scope drop-down, and when a limit is chosen a quantity
spinner (minimum 1, no practical maximum) plus a Books/MB/GB unit, with the
MB/GB approximation note shown only for those units.
A license denial that looks like Audible throttling now suggests turning the
limit on, quoting the real number of downloads Libation recorded in the last 24
hours. Audible reports no distinct throttling reason, so the suggestion stays
silent unless that record makes throttling plausible.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Records every successful audiobook download in the library database (a new
DownloadHistory table) and, when the user opts in, stops downloading once the
rolling 24 hour window is full.
The history lives in the database rather than a file under LibationFiles
because in Docker only the database is on a volume; a file there is discarded
on every container restart.
The limit is checked immediately before each book downloads, never at queueing
time, so a full queue stays full and the user can raise or disable the limit
mid-run. When nothing in the queue can proceed the queue pauses and re-checks
every 15 seconds, recomputing settings, history and clock from scratch, so a
queue left running for days drip-feeds itself as downloads age out. The CLI
never waits: it skips covered titles and reports a count.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Allamagoosa