this is a bit dangerous but necessary for more efficient delta updates.
we must be very careful to avoid a bug in systemd caused by putting
foo.erofs.caibx into the sha256sums. so instead we have foo.caibx that
gets installed with the correct name by sysupdate
we still have the foo.erofs.caibx on the server for backwards compat,
but continue to not put it into the sha256sums
this is a bit of a shot in the dark, but I believe we may have
unnecessary delta in our images caused by the rebuilding of software on
a daily basis. this would result in mtimes changing when the files
actually do not.
a tiny mtimer tool is meant to work around that by consuming an input
json file of mtimes+checksums and if mtimes change it will checksum the
affected file to verify it actually has changed in content as well.
assuming reproducible builds this should result in far less delta in the
erofs and by extension the delta download
this should make things more amazing. from my testing it looks like
aligning the sizes improves compression and improves caibx generation by
aligning chunk boundaries better.
in a test scenario of adding a single 128M random data file to /usr/lib
this brought the fragmentation from a couple thousand segments down to 8
(of which 5 are in the superblock and the new file appears as large
contiguous chunk delta). the actual download size is a 135M delta
in the hopes of getting upgrades listed again
the trouble appears to be that since we no longer create the tar or
erofs addon systemd-sysupdate considers the candidates not viable and
just ignores the new versions. by having mock artifacts I hope things
will work again
we'll need to invent some tech to eventually rotate the old artifacts
out of /system though
our generator now always runs to generate the relevant units for erofs
use
tar build has been removed, instead we spend some extra time on zstd
compression of the erofs
shouldn't have much practical impact other than smaller images
dedupe would also be nice to use but unfortunately is single threaded
right now and that would cause **substantial** slow down of builds.
hours for in fact. for only about 300mb reduction :(
unfrotunately we need a code copy for this because bootstrap must be
self contained. not too much to worry about since they are both in the
same repo so I expect people will grep for stuff (hopefully :D)
with ephemeral VMs the base image is old so we need to update the
mirrorlist during build.sh as well. also turn the copies around. now we
always pull copies out of /etc
