🐛 Raise default binfile import text total to 500MiB

Giant legitimate exports hold hundreds of MB of JSON across
hundreds of thousands of KB-sized entries without tripping any
per-entry cap, so the 200MiB cumulative default rejected them out
of the box. Raise it to 500MiB; the 20MiB per-entry cap still
bounds single-entry inflation. Pin the floor with a regression
test.

AI-assisted-by: muse-spark-1.3-contributor
This commit is contained in:
Andrey Antukh committed 2026-10-05 18:31:16 +02:00
1 parent 2c4da98131
commit 753e98bb9f
2 files changed
+14 -3

No files matched your search

+7 -3
View File
@@ -68,11 +68,15 @@
(* 1024 1024 20))
;; Maximum total decompressed size allowed for all JSON/text zip entries
;; combined within a single import job: 200MiB. Bounds the case where many
;; combined within a single import job: 500MiB. Bounds the case where many
;; entries, each individually under default-max-text-entry-size, still sum
;; to an unreasonable total.
;; to an unreasonable total. Set well above the 200MiB it replaces because
;; legitimate giant exports exist: hundreds of thousands of KB-sized
;; entries (pages, shapes) that never trip the per-entry cap but sum past
;; 400MiB. The per-entry cap still bounds single-entry DEFLATE
;; amplification, so this only moves the cumulative ceiling.
(def ^:const default-max-text-total-size
(* 1024 1024 200))
(* 1024 1024 500))
;; Maximum number of entries allowed in the import zip: 500,000.
(def ^:const default-max-zip-entries
@@ -2153,6 +2153,13 @@
(t/is (= :validation (:type out)))
(t/is (= :max-file-size-reached (:code out))))))
(t/deftest default-text-total-budget-fits-giant-legit-files
;; A real-world giant export held ~437 MiB of JSON across ~178k
;; KB-sized entries without tripping any per-entry cap. The default
;; cumulative budget must clear that scale, or such files cannot be
;; imported out of the box.
(t/is (>= bfc/default-max-text-total-size (* 1024 1024 450))))
(defn- text-entries-sizes
"Returns the decompressed sizes of every `.json` entry in the zip at
`zip-path`. Used to pick a cumulative budget that sits between the