Alright, this is the main set of changes done in `build.sh`. After this, as far as `build.sh` is concerned, it's mostly style and documentation changes I will write MRs for. There was no way for me to split this into smaller independent changes without leaving the tree in an inconsistent state, this is really the most "atomic" I can personally do. The changes may look scary at first, but I promise you they're not that major, breaking or hard to review. The MR focuses on how the image is created, populated and shrunk from the rootfs generated by mkosi.
Firstly, the variables that are set at the start of the program were simplified and refactored so they can be used in (in my opinion) a more elegant and useful way with the rest of the now-modified build process. They now focus more on simply the paths/filenames of the outputted artifacts, so they can be changed conveniently, with the internal naming now still staying a static `kde-linux`. As a result of this, the image ID (`kde-linux`) is now in `mkosi.conf`.
`OUTPUT_IS_BTRFS_SUBVOLUME` was dropped. This is a remnant of when we could populate the images without a btrfs'd build environment, but those times are now gone and if this variable was determined to be false, the build would fail pretty quickly anyways, so really, it should always be true in a properly configured build environment. Because of it being used as a condition in if-else statements (if $OUTPUT_IS_BTRFS_SUBVOLUME is true... else...), those statements were dropped and only their main branch was kept with their "else" branch (which, again, should never even run on a working build environment) discarded. One script that was only ever called from one of those discarded else-branches was `btrfs-copy.sh`, so it was deleted as it is now fully unused.
The way everything used to work after this was fairly convoluted, which is probably the result of piling new shrinking methods and the likes onto old existing code. I will drop some details for the sake of brevity. After mkosi finished, three btrfs subvolumes were created and the files from the mkosi-generated rootfs were copied to those (`/var/lib/flatpak` was copied to a flatpak subvolume, `/live` was copied to a live subvolume, and the rest to a kde-linux subvolume). These subvolumes were then all packed into three respective files. Using `systemd-repart` and the definitions inside `mkosi.repart`, a full disk image containing both a 260M ESP populated with files from `/efi-template` and a btrfs partition of a size large enough to house all of those subvolumes was created. Using `systemd-dissect`, only the btrfs partition from the full disk image was mounted, and the script `btrfs-send-receive.sh` was called inside of the mounted partition, which unpacked all of the previously packed subvolumes onto the filesystem. It then called `btrfs-shrink.py` which would try and shrink the filesystem to the minimum possible size and write that size into a file for future usage by `part-rebuild.py` after it's done. `part-rebuild.py` got called which extracted the btrfs partition and ESP from the full disk image separately using `dd` and shrunk the btrfs part of the image to the size that the actual inner filesystem was shrunk to by `btrfs-shrink.py`. Then `systemd-repart` was called AGAIN, this time with `--size=auto` instead of a fixed size and using definitions inside `mkosi.repart-rebuild`, which would merge the two part-rebuild.py-dd-separated partition images of the ESP and shrunk btrfs filesystem back into one full disk image.
The way it now works is the ESP (of 260M) and btrfs filesystem (of a size large enough, 8G) are created inside of two files using native tools (`fallocate` and `mkfs.btrfs`/`fat`). The ESP file is mounted, files from `/efi-template` are copied to it, and then it's unmounted. Then, the btrfs file is mounted, files from the mkosi-generated rootfs are copied to it (what `btrfs-send-receive.sh` effectively did), the filesystem is shrunk to the minimum possible size (what `btrfs-shrink.py` effectively did), and then it's unmounted. Since the underlying btrfs filesystem inside of the btrfs file was shrunk but the actual file is still 8G, the file is, using `truncate`, shrunk to the size of the actual btrfs filesystem inside (what `part-rebuild.py` effectively did). The two files are now ready to be merged into one full bootable disk image, so `systemd-repart` is called a single time right away with `--size=auto` using the definitions in `mkosi.repart` to do so. Everything is done in `build.sh`.
As a result, the files no longer necessary were deleted from the tree - `btrfs-copy.sh`, `btrfs-send-receive.sh`, `btrfs-shrink.py`, `part-rebuild.py`, and the definitions inside `mkosi.repart` were simplified to conform to the new direct disk image build method.
I'm pretty sure it's faster (30min build time down to 20min) because it's less convoluted, and generates smaller images (6ish G down to 5ish G). I'm not sure what the reasoning is behind the latter, but it's also a welcome change :)
!91 will also be updated to refer to this MR instead of the now-closed !90 and can then be merged to fully get rid of Python for now.
Fixes: https://invent.kde.org/kde-linux/kde-linux/-/issues/103
Shouldn't affect non-nvidia hardware (I tested on an Intel-only system)
If you'd like to test on non-nvidia hardware, enable the nvidia services like so:
```
systemctl enable nvidia-suspend.service nvidia-resume.service nvidia-hibernate.service
```
Let me know if you see any issues!
Signed-off-by: Ryan Brue <ryanbrue.dev@gmail.com>
This fixes the recent issues of `cannot stat /efi/EFI/Linux` we get when using `update.sh`
systemd no longer auto-mounts the ESP at `/efi`. Reason is explained in a comment inline.
If/when !95 merges, this will be the only Python script in the build process left. Rewrite to (ba)sh like the rest for consistency, then we can port everything to some other language if desired one day.
This also makes the logo bigger by comparison. I don't know how that'll look in plymouth. If it's wanted, I could keep the effective size the same by having transparent space, just let me know.
CC: @ngraham @justinzobel
Signed-off-by: Ryan Brue <ryanbrue.dev@gmail.com>
The consensus on the matrix chat is that distrobox is better than toolbox as a container and development solution.
I've used them both on Fedora Silverblue (and its derivatives) and found that to be true.
build.sh unnecessarily generates some temporary configs for mkosi.
Instead, pass flags to mkosi to configure it without polluting.
The distro being arch is already defined in mkosi.conf.
We install all packages in the `kde-linux` repo:
```bash
# All the KDE we plan to include in the base image
$(pacman --sync --list --quiet kde-linux)
```
I recently added `calamares-git` (and some other AUR packages) to that
repo so we don't have to build them as part of every image build. This
is causing it to get included in the base image.
This change makes pacman restrict to just the `kde-linux` group instead
of the whole repo.
Resolves#42
archlinux-keyring is a part of base already. Good global mirrors are already defined in Arch Docker images, the explicit 23m mirror stuff is unnecessary.
This commit adds power-profiles-daemon to kde-linux. Reportedly this had been added in the past but it may have just been refactored out by accident.
Signed-off-by: Ryan Brue <ryanbrue.dev@gmail.com>
the tombstone is a bit broken in that it doesn't use the correct name.
sort the names correctly (first is preferred) and also bring back the
old names so they eventually get cleaned up
