Parsing HTML belongs before the data reaches the data layer, not in it. Move the
flattening to AudibleUtilities, where the rest of the Item translation lives, so
Book.Description holds plain text and DataLayer no longer references
HtmlAgilityPack.
Import now updates a description as well as creating one, which it never did
before: books imported while descriptions were stored as raw HTML are still
holding markup, and a re-scan is what clears it. A scan that reports no summary
leaves the stored one alone.
This also cleans up the CSV/Excel export, which was emitting raw HTML too.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
FillMissingTags is driven against a real MetadataItems built from a hand-rolled
ilst box: empty, as a Widevine download arrives, and pre-populated, as an .aaxc
download arrives. Reverting any of the three fixes fails a test with the exact
value from the issue report.
MockLibraryBook.CreateBook gains a copyright knob to match UpdateBookDetails.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
DownloadPdfLicenseTests drives the step with and without a license in hand. No
Audible account exists in those tests, so a test that completes at all is the
proof that the step did not go asking for a license of its own.
BulkRunSelectionTests drives the real run loop against a real library database:
which titles each pass reaches, that an absent title is left alone by both, that
--force still attempts it, and that a refused title is waited on by the follow-up
pass and reported once rather than twice.
SupplementSyncTests covers the scan rules, in a new DtoImporterService.Tests
project. The two replaced assertions - that a PDF-only title is never waited on,
and that a --pdf run is never held back - encoded the belief that a PDF is a
different request from the audiobook, which is what this branch disproves.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Only a newly imported book ever recorded a supplement, so a title that gained a
PDF after its first import never got one, and a title that lost its PDF went on
claiming one - which in issue #1973 is why three titles Audible has no PDF for
were still being asked for.
Sync from updateBook as well, and give Book set-semantics for the one supplement
Audible reports per title. The duplicate guard compared the incoming url to
itself, so it happened to mean 'this book already has a supplement' and a url
that had changed was silently ignored.
A supplement is dropped only when Audible says outright that no supplement url is
available. A missing url says nothing by itself: episodes come from the catalog,
which is never asked for pdf_url, so there it means 'not asked'. A PDF already
downloaded is left alone either way, since the file is on disk and the library
should go on saying so.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Libation already runs its tests on Microsoft.Testing.Platform via the MSTest
metapackage; this only moves the version off 4.2.2 so all three repos name the
same MSTest release.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Seven tests failed in a Windows CI leg, every one thrown from a
TestInitialize rather than an assertion, and on a path no test in the
failing class had created. The cause is [assembly: Parallelize] over
classes that reach for state which is process-wide by nature: the
Configuration singleton, the LIBATION_FILES_DIR environment variable, and
AudibleFileStorage's static file cache. Two classes running at once means
one swapping the config or deleting its temp directory while the other
enumerates it.
Opting out per class was the previous arrangement and it does not hold:
three classes carried [DoNotParallelize] while two touching the same
state did not, and nothing tells whoever adds the sixth. The attribute is
gone, with a comment saying why, and the existing [DoNotParallelize]
markers stay as a statement of intent if parallelism ever returns.
Costs nothing measurable: 68 tests in about 3.1 seconds serially, against
9.8 seconds for the same assembly in the CI leg that failed. Ran three
times over to check.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
That the file existed said only that the server sent a body, and an Audible error is
a 200 with a JSON body like any other response. Dinah's downloader renames by
Content-Disposition, so such a body landed in the book's folder under whatever
Audible called it and the title was recorded as having its PDF.
Check the payload: a file named .pdf must carry the PDF header, and nothing may begin
with the opening character of a JSON or markup document. A rejected download is
deleted rather than left in the library, which also lets the empty-folder cleanup
run, and the file is added to the path cache only once it has passed.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A storefront that no longer lists a title still answers a request for it: HTTP 200,
total_results 1, and a product carrying an asin and a few always-returned flags.
Nothing between the request and the file noticed, so a re-download replaced a
metadata file written while the title was still listed - the only copy of that data -
with the placeholder.
Fetch the product before touching the destination, and leave the file alone when the
product carries no title. Reported in issue #1947, where a Canada-only title produced
{"asin":...,"asset_details":[],"is_preview_enabled":false,"is_vvab":false,"rating":{...}}
against every other storefront.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
The two path assertions added with the PDF fix compared a path the test built
itself against one that had been through LongPath, which on Windows prefixes a
drive-rooted path with \\?\ so paths past the 260 character limit work. Linux
adds no prefix, so this only showed up on the Windows job.
Normalising both sides is not just about the false failure. The inequality
assertion guarding 'the PDF was saved loose in the Books directory' compared a
raw temp path against a prefixed one, so on Windows it passed on the prefix
alone and would not have caught the bug it exists to catch.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A PDF is fetched through the same license request as the audiobook, so following
a refusal with a PDF request reproduced, through the PDF, exactly the per-run
refusal the wait exists to stop. The follow-up pass now skips the titles the
first pass deliberately left alone as well as the ones it attempted.
Also stop a failed PDF download leaving an empty folder in the library: a
PDF-only download is the one case that has to create the book's folder before it
has anything to put in it, so it now removes a folder it created and did not
fill. GetProposedDownloadFilePath goes back to being a pure path computation.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
A plain 'libationcli liberate' iterates the titles DownloadDecryptBook selects,
and that step selects on '!AudioExists'. A title needing nothing but its PDF was
therefore never reached by the verb documented as 'book and pdf backups' - only
'liberate --pdf' picked it up. For a library that was liberated before its PDFs
were, that is every title with a PDF.
Give the bulk run an optional second pass and have liberate use it for PDFs, the
way the app's Liberate All always has. Skipped when the first pass stopped early
so a run cut short by its download limit does not carry on doing other work, and
titles the first pass attempted are excluded by product id rather than by asking
Validate again, so a step that just failed is not immediately retried.
Left alone: the Audiobookshelf upload stays tied to a fresh liberation. Its
Validate passes for any liberated title, so including it here would walk the
whole library on the next run. 'abs upload' already exists for that.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
getProposedDownloadFilePath looked for the book's audio file and fell back to
the Books directory itself when it found none. That lookup matches on the
product id appearing in the path, so it finds nothing for a library whose
folder and file templates omit <id>, and nothing for a title marked downloaded
whose files are not on this machine. Those PDFs landed in the library root,
where they also shared one namespace and so could collide with each other.
Fall back to the folder template instead - the same folder the audiobook itself
would go in - and create it, since nothing else does on the PDF-only path.
Also give MockLibraryBook a three-field version: ToVersionString formats to at
least three fields, so the two-field default threw as soon as anything rendered
a naming template for a mock book.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Asserting on the generated JSON alone would pass just as happily with a
misspelled sink argument, which Serilog ignores in silence - and silently not
rolling is the bug. These build a real logger from Libation's own config and
write until it rolls, including a test that pins the old unbounded behaviour so
a future change to the defaults cannot quietly restore it.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
The classifier's inputs are the actual denials from the log attached to issue
#1947: owned titles on an inactive account, a Plus title no longer in the
catalog, and an unreleased preorder Audible has no audio for.
The backoff tests also caught a real overflow: first * 2^n exceeds a TimeSpan
long before the cap matters.
Co-authored-by: rmcrackan <rmcrackan@gmail.com>
Auto-upload only fires when a book is liberated. Books liberated before
Audiobookshelf was configured, or while it was disabled, had no path to the
server short of re-downloading the whole library.
'libationcli upload' backfills them from the files already on disk. Bulk or
targeted by ASIN. Nothing is re-downloaded and no local file is deleted.
Also fixes a latent defect this exposes. Validate() reads a database status
(Book.AudioExists) while GetFilesToUpload() read only FilePathCache. A book
liberated long ago passes validation but has no cache entry, so the upload
found no files and returned success having sent nothing. File lookup now uses
AudibleFileStorage.Audio.GetPaths, which unions the cache with a live scan of
the Books directory.
Other changes:
- Validate() now requires LiberatedStatus.Liberated. It previously accepted
Error too, whose partial files should not be uploaded.
- New OutcomeDetermined event classifies each book as Uploaded, AlreadyExists,
NoFilesFound or Failed. Failures travel on this event rather than through
StatusHandler: the GUI process queue treats a non-success StatusHandler as a
bad book and raises the Abort/Retry/Ignore dialog, and an upload problem must
never fail a liberation.
- The verb prints an end-of-run summary and exits 0, matching other verbs.
No database migration. Duplicate detection already runs server-side inside
UploadBookAsync, so repeat runs are safe without local upload state.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Update all project runtime targets
- Update all dependencies
- NOTE: Using Npgsql.EntityFrameworkCore.PostgreSQL RTM build from MyGet
- Delete unused pubxml files (they were made redundant by recent workflow changes)
- Replace Libation.sln with Libation.slnx
