mirror of
https://github.com/KDE/kde-linux.git
synced 2025-12-23 16:08:29 -05:00
2be5c59c92
Alright, this is the main set of changes done in `build.sh`. After this, as far as `build.sh` is concerned, it's mostly style and documentation changes I will write MRs for. There was no way for me to split this into smaller independent changes without leaving the tree in an inconsistent state, this is really the most "atomic" I can personally do. The changes may look scary at first, but I promise you they're not that major, breaking or hard to review. The MR focuses on how the image is created, populated and shrunk from the rootfs generated by mkosi. Firstly, the variables that are set at the start of the program were simplified and refactored so they can be used in (in my opinion) a more elegant and useful way with the rest of the now-modified build process. They now focus more on simply the paths/filenames of the outputted artifacts, so they can be changed conveniently, with the internal naming now still staying a static `kde-linux`. As a result of this, the image ID (`kde-linux`) is now in `mkosi.conf`. `OUTPUT_IS_BTRFS_SUBVOLUME` was dropped. This is a remnant of when we could populate the images without a btrfs'd build environment, but those times are now gone and if this variable was determined to be false, the build would fail pretty quickly anyways, so really, it should always be true in a properly configured build environment. Because of it being used as a condition in if-else statements (if $OUTPUT_IS_BTRFS_SUBVOLUME is true... else...), those statements were dropped and only their main branch was kept with their "else" branch (which, again, should never even run on a working build environment) discarded. One script that was only ever called from one of those discarded else-branches was `btrfs-copy.sh`, so it was deleted as it is now fully unused. The way everything used to work after this was fairly convoluted, which is probably the result of piling new shrinking methods and the likes onto old existing code. I will drop some details for the sake of brevity. After mkosi finished, three btrfs subvolumes were created and the files from the mkosi-generated rootfs were copied to those (`/var/lib/flatpak` was copied to a flatpak subvolume, `/live` was copied to a live subvolume, and the rest to a kde-linux subvolume). These subvolumes were then all packed into three respective files. Using `systemd-repart` and the definitions inside `mkosi.repart`, a full disk image containing both a 260M ESP populated with files from `/efi-template` and a btrfs partition of a size large enough to house all of those subvolumes was created. Using `systemd-dissect`, only the btrfs partition from the full disk image was mounted, and the script `btrfs-send-receive.sh` was called inside of the mounted partition, which unpacked all of the previously packed subvolumes onto the filesystem. It then called `btrfs-shrink.py` which would try and shrink the filesystem to the minimum possible size and write that size into a file for future usage by `part-rebuild.py` after it's done. `part-rebuild.py` got called which extracted the btrfs partition and ESP from the full disk image separately using `dd` and shrunk the btrfs part of the image to the size that the actual inner filesystem was shrunk to by `btrfs-shrink.py`. Then `systemd-repart` was called AGAIN, this time with `--size=auto` instead of a fixed size and using definitions inside `mkosi.repart-rebuild`, which would merge the two part-rebuild.py-dd-separated partition images of the ESP and shrunk btrfs filesystem back into one full disk image. The way it now works is the ESP (of 260M) and btrfs filesystem (of a size large enough, 8G) are created inside of two files using native tools (`fallocate` and `mkfs.btrfs`/`fat`). The ESP file is mounted, files from `/efi-template` are copied to it, and then it's unmounted. Then, the btrfs file is mounted, files from the mkosi-generated rootfs are copied to it (what `btrfs-send-receive.sh` effectively did), the filesystem is shrunk to the minimum possible size (what `btrfs-shrink.py` effectively did), and then it's unmounted. Since the underlying btrfs filesystem inside of the btrfs file was shrunk but the actual file is still 8G, the file is, using `truncate`, shrunk to the size of the actual btrfs filesystem inside (what `part-rebuild.py` effectively did). The two files are now ready to be merged into one full bootable disk image, so `systemd-repart` is called a single time right away with `--size=auto` using the definitions in `mkosi.repart` to do so. Everything is done in `build.sh`. As a result, the files no longer necessary were deleted from the tree - `btrfs-copy.sh`, `btrfs-send-receive.sh`, `btrfs-shrink.py`, `part-rebuild.py`, and the definitions inside `mkosi.repart` were simplified to conform to the new direct disk image build method. I'm pretty sure it's faster (30min build time down to 20min) because it's less convoluted, and generates smaller images (6ish G down to 5ish G). I'm not sure what the reasoning is behind the latter, but it's also a welcome change :) !91 will also be updated to refer to this MR instead of the now-closed !90 and can then be merged to fully get rid of Python for now.