Web Excursions 2021-11-24

Flatpak Is Not the Future

  • The stability of the Linux desktop has dramatically improved in recent years.

    • Core library developers are finally seeing the benefits of maintaining compatibility.

    • Despite this, many developers are not interested in depending on a stable base of libraries for binary software.

    • Instead, they have decided to ignore and override almost all libraries pre-installed on the user’s system.

  • The current solutions involve packaging entire alternate runtimes in containerized environments.

    • Flatpak, Snap, AppImage, Docker, and Steam:

    • these all provide an app packaging mechanism that replaces most or all of the system’s runtime libraries,

    • and they now all use containerization to accomplish this.

  • some of the technical, security and usability problems with Flatpak and others

  • Size

    • KCalc AppImage (if you can even figure out how to download it) is 152 MB. For a calculator.

    • Other solutions like Flatpak or Steam download the runtime separately.

      • Your app metadata specifies what runtime it wants to use and a service downloads it and runs your app against it.

      • KCalc from Flathub ~ 900 MB

      • Note that the app package itself is only 4.4 MB. The rest is all redundant libraries that are already on my system.

    • Flatpak says this is so that apps can share runtimes.

      • But the whole point of their runtime system is to let apps use different runtimes.

      • Each branch of a runtime (representing, say, a different base version of Ubuntu) is a completely independent runtime image.

    • They say disk space is cheap.

      • This is not true, not for the root devices of modern computers.

      • Built-in storage has in fact been shrinking.

    • Why shouldn’t storage shrink anyway?

      • Software should be getting more efficient, not less.

      • Even if bandwidth were free and hard drives grew on trees, it would not excuse such drastic bloat.

  • Memory Usage, Startup Time

    • The penalties of huge alternate runtimes aren’t just in storage and bandwidth.

    • Each app with a new runtime adds another hundred megs or more of RAM usage.

    • A bigger problem is that these apps can actually take several seconds to start up.

      • They have to load all their own libraries from disk instead of using what’s already on the system, already in memory.

      • Snap is the slowest of all, largely because it stores all its data in squashfs images.

        • Snap mounts all registered snaps at startup instead of just extracting the metadata they need beforehand,

          • possibly in an effort to mitigate this slowness.

        • They’re just moving part of the slow startup time to the boot time of your computer.

        • All sorts of snap crap now shows up in mount and fdisk -l.

        • The more snaps you have installed, the slower your computer will start, even if you don’t use them.

        • Canonical started converting their basic desktop apps like the GNOME Calculator to Snap in Ubuntu 18.04.

          • The user experience was so terrible that they quietly converted them back to normal apps in Ubuntu 20.04

  • Drivers

    • A major problem with alternate runtimes is drivers.

    • New graphics hardware needs new graphics libraries which have a ton of dependencies.

    • Linux distributions, especially those with rolling releases or hardware enablement packages, do a great job of keeping these libraries up-to-date for new hardware.

      • Bundled runtimes do not.

    • Steam Runtime uses various heuristics to determine whether to override each library in the runtime with that of the host system,

      • so each app is running on a Frankenstein hodge-podge of libraries.

    • Flatpak recognizes that mixing libraries with the host system is a mess.

      • Instead, they want to bundle their own graphics drivers in the runtimes, keeping them regularly updated for new hardware.

      • But they can’t control the most important part, the kernel, so they can’t directly package drivers that depend on specific kernel versions or proprietary kernel modules.

      • So the runtime is broken into extensions, and extensions can go the Steam route of pulling in the native driver or even download specific NVidia drivers on-the-fly to match the kernel module.

    • AppImage mixes host libraries with the runtime as well, but not dynamically;

      • they just decide what to exclude at packaging time so it’s even less reliable than Steam.

    • Docker has an NVidia toolkit that must be installed on both the host system and the containerized runtime to get drivers working.

    • Snap is doing who knows what.

    • None of these are proper solutions.

  • Security

    • Flatpak allows apps to declare that they need full access to your filesystem or your home folder, yet graphical software stores still claim such apps are sandboxed.

    • In reality it’s actually worse than nothing because it leads people to place more trust than they should in random apps they find on the internet.

    • Flatpak is working on a fine-grained permission system to improve the security of its sandbox.

      • Permissions are things like whether the app is allowed to access the microphone or the printer.

      • Portals are things like a file open dialog that runs outside the sandbox, so the app in the sandbox gets only the file the user chose.

      • They want the apps running on Flatpak to be unaware of Flatpak.

      • They would rather modify the core libraries like GTK to integrate with Flatpak.

    • Flathub doesn’t enforce DNS ownership.

      • They let anyone publish anything; they merely prefer that apps are controlled by their authors.

      • A sane security strategy would be to require a DNS challenge to allow a publisher to use a specific app prefix.

  • Complexity

    • You would think that these packaging mechanisms would embrace simplicity if they want to attract software developers.

    • In fact they are doing the opposite.

    • The driver, sandboxing, and permission/portal issues we’ve discussed barely scratch the surface of the complexity involved in making this all work.

  • Services

    • All of these app packaging systems require that the user have some service installed on their PC before any packages can be installed.

    • free software and privacy people don’t want to allow proprietary apps this level of freedom on their PCs. But the vast majority of users and independent software vendors don’t care.

    • What users want is easy (and efficient!) installation.

    • What the biggest software vendors want is the freedom to ship software directly to customers,

      • without the intervention of a service or hub.

  • The backwards compatibility situation will continue to get better with time.

    • The more proprietary software is shipped for the native Linux desktop, the less likely major distributions will accept upstream breakage.

  • I strongly recommend that all app developers (open source or otherwise) delay upgrading to GTK 4 as long as possible.

    • The longer we can delay the switch, the more stable GTK 3 will be.

    • If the GTK team want us to use GTK 4, they will need to prove its stability over many years, not simply replace it with GTK 5.

  • If the Flatpak developers truly want an app distribution system that rivals Android and iOS, the sandbox, permissions and portal system should be the only focus.

    • Abandon everything related to runtimes, and instead mount the native /usr (or a restricted set of core libraries from /usr) read-only into each container;

    • Add metadata to Flatpaks to declare library dependencies that the Flatpak service will install with the host distribution’s package manager (perhaps with generated meta-packages that can be cleanly removed);

    • Build a fine-grained user-interactive runtime permission system that requires the app to make Flatpak-specific API calls to activate permission dialogs; and

    • Deprecate install-time permissions (especially filesystem access) and remove all apps from Flathub that use them.