Web Excursions 2021-11-24
The stability of the Linux desktop has dramatically improved in recent years.
Core library developers are finally seeing the benefits of maintaining compatibility.
Despite this, many developers are not interested in depending on a stable base of libraries for binary software.
Instead, they have decided to ignore and override almost all libraries pre-installed on the user’s system.
The current solutions involve packaging entire alternate runtimes in containerized environments.
Flatpak, Snap, AppImage, Docker, and Steam:
these all provide an app packaging mechanism that replaces most or all of the system’s runtime libraries,
and they now all use containerization to accomplish this.
some of the technical, security and usability problems with Flatpak and others
Size
KCalc AppImage (if you can even figure out how to download it) is 152 MB. For a calculator.
Other solutions like Flatpak or Steam download the runtime separately.
Your app metadata specifies what runtime it wants to use and a service downloads it and runs your app against it.
KCalc from Flathub ~ 900 MB
Note that the app package itself is only 4.4 MB. The rest is all redundant libraries that are already on my system.
Flatpak says this is so that apps can share runtimes.
But the whole point of their runtime system is to let apps use different runtimes.
Each branch of a runtime (representing, say, a different base version of Ubuntu) is a completely independent runtime image.
They say disk space is cheap.
This is not true, not for the root devices of modern computers.
Built-in storage has in fact been shrinking.
Why shouldn’t storage shrink anyway?
Software should be getting more efficient, not less.
Even if bandwidth were free and hard drives grew on trees, it would not excuse such drastic bloat.
Memory Usage, Startup Time
The penalties of huge alternate runtimes aren’t just in storage and bandwidth.
Each app with a new runtime adds another hundred megs or more of RAM usage.
A bigger problem is that these apps can actually take several seconds to start up.
They have to load all their own libraries from disk instead of using what’s already on the system, already in memory.
Snap is the slowest of all, largely because it stores all its data in squashfs images.
Snap mounts all registered snaps at startup instead of just extracting the metadata they need beforehand,
possibly in an effort to mitigate this slowness.
They’re just moving part of the slow startup time to the boot time of your computer.
All sorts of snap crap now shows up in mount and fdisk -l.
The more snaps you have installed, the slower your computer will start, even if you don’t use them.
Canonical started converting their basic desktop apps like the GNOME Calculator to Snap in Ubuntu 18.04.
The user experience was so terrible that they quietly converted them back to normal apps in Ubuntu 20.04
Drivers
A major problem with alternate runtimes is drivers.
New graphics hardware needs new graphics libraries which have a ton of dependencies.
Linux distributions, especially those with rolling releases or hardware enablement packages, do a great job of keeping these libraries up-to-date for new hardware.
Bundled runtimes do not.
Steam Runtime uses various heuristics to determine whether to override each library in the runtime with that of the host system,
so each app is running on a Frankenstein hodge-podge of libraries.
Flatpak recognizes that mixing libraries with the host system is a mess.
Instead, they want to bundle their own graphics drivers in the runtimes, keeping them regularly updated for new hardware.
But they can’t control the most important part, the kernel, so they can’t directly package drivers that depend on specific kernel versions or proprietary kernel modules.
So the runtime is broken into extensions, and extensions can go the Steam route of pulling in the native driver or even download specific NVidia drivers on-the-fly to match the kernel module.
AppImage mixes host libraries with the runtime as well, but not dynamically;
they just decide what to exclude at packaging time so it’s even less reliable than Steam.
Docker has an NVidia toolkit that must be installed on both the host system and the containerized runtime to get drivers working.
Snap is doing who knows what.
None of these are proper solutions.
Security
Flatpak allows apps to declare that they need full access to your filesystem or your home folder, yet graphical software stores still claim such apps are sandboxed.
In reality it’s actually worse than nothing because it leads people to place more trust than they should in random apps they find on the internet.
Flatpak is working on a fine-grained permission system to improve the security of its sandbox.
Permissions are things like whether the app is allowed to access the microphone or the printer.
Portals are things like a file open dialog that runs outside the sandbox, so the app in the sandbox gets only the file the user chose.
They want the apps running on Flatpak to be unaware of Flatpak.
They would rather modify the core libraries like GTK to integrate with Flatpak.
Flathub doesn’t enforce DNS ownership.
They let anyone publish anything; they merely prefer that apps are controlled by their authors.
A sane security strategy would be to require a DNS challenge to allow a publisher to use a specific app prefix.
Complexity
You would think that these packaging mechanisms would embrace simplicity if they want to attract software developers.
In fact they are doing the opposite.
The driver, sandboxing, and permission/portal issues we’ve discussed barely scratch the surface of the complexity involved in making this all work.
Services
All of these app packaging systems require that the user have some service installed on their PC before any packages can be installed.
free software and privacy people don’t want to allow proprietary apps this level of freedom on their PCs. But the vast majority of users and independent software vendors don’t care.
What users want is easy (and efficient!) installation.
What the biggest software vendors want is the freedom to ship software directly to customers,
without the intervention of a service or hub.
The backwards compatibility situation will continue to get better with time.
The more proprietary software is shipped for the native Linux desktop, the less likely major distributions will accept upstream breakage.
I strongly recommend that all app developers (open source or otherwise) delay upgrading to GTK 4 as long as possible.
The longer we can delay the switch, the more stable GTK 3 will be.
If the GTK team want us to use GTK 4, they will need to prove its stability over many years, not simply replace it with GTK 5.
If the Flatpak developers truly want an app distribution system that rivals Android and iOS, the sandbox, permissions and portal system should be the only focus.
Abandon everything related to runtimes, and instead mount the native /usr (or a restricted set of core libraries from /usr) read-only into each container;
Add metadata to Flatpaks to declare library dependencies that the Flatpak service will install with the host distribution’s package manager (perhaps with generated meta-packages that can be cleanly removed);
Build a fine-grained user-interactive runtime permission system that requires the app to make Flatpak-specific API calls to activate permission dialogs; and
Deprecate install-time permissions (especially filesystem access) and remove all apps from Flathub that use them.