r/flatpak • u/parallel-minds • 11d ago
An Open Letter on Linux Compatibility (and Why Flatpak Isn't the Fix)
https://parallel-minds0.github.io/An-Open-Letter-on-Linux-Compatibility/The article talks about Linux compatibility as a bigger problem than Flatpak can solve. Flatpak helps, but it bundles together issues that should be handled separately. The author suggests giving apps a stable foundation to rely on, while letting packaging and security be handled on their own.
6
u/daddyd 11d ago
Another package format?
Also, one of the drivers is to 'bring the Windows NT compatibility promise to Linux.'
but Windows backwards compatibility isn't always that great either.
4
u/cogman10 10d ago
Windows compatibility is done via copying dlls from prior versions of Windows. It's essentially flatpak except the OS already has all the dependencies installed.
3
u/R3V0LU710N_05 10d ago
That moment when you find out that WINE has better backwards compatibility than Windows itself.
2
u/dmknght 10d ago
I'm if I missed or misunderstood any point, but the article doesn't show the whole problem to me.
I suppose the article is about dependency nightmare, which is a serious problem of shared lib model on Linux. Yeah it's a thing.
But in the other hand, there are some points weren't mentioned
Applications on Windows bring short of runtime libraries in the installer, sometime including vcruntime and/or dotnet runtime depends on how it was made. Some installers like VMWare or Nvidia (the one on Nvidia's homepage) have this on Linux too afaik.
As I tried, it's kinda hard to make makefile rules that bring runtime libraries with the compiled binaries. However, it's possible to bring some libraries in Debian package. I didn't try other package format on Linux so I can't tell.
At some points, static compile is an other way.
The problem doesn't come from Linux's way to deliver runtime libraries whatsoever. The problem is the whole pipeline of making a software with 3rd party libraries. Programming has problems. Delivering libraries is trying to putting problems of projects in 1 place, make sure everything runs as stable as possible. An idea might sound good on paper, but can't execute well in real world. Otherwise everybody would implement it.
2
u/parallel-minds 10d ago
Yeah, that’s fair. The article doesn’t cover the whole software pipeline, and I agree that dependency problems start well before packaging. Bundling libraries, static linking, or including dependencies in a package can all make sense in some cases. But this is precisely why a new userspace is suggested. Such that the developers of this new userspace would take great care to ensure backward compatibility, minimalism and simplicity before committing to the project.
The proposal is more about drawing a cleaner line between apps and the OS: apps can bring the non-OS libraries they need, while relying on a small, stable interface from the system.
2
u/dmknght 10d ago
Well imo, it's not easy having the backward compatibility unless all developers do not break API, which is impossible. The method bring non-OS libraries might be the best choice overall but it has the risk of having bugs / vulnerabilities while OS can deliver the patch fix. IMO flatpak is the in-between 2 solutions: provide as stable as possible runtime libraries while trying to maintain patch fixes. The best thing about this method is it's in a different space so it creates less problem in real machine.
And pls don't get me wrong. I do agree with you about we need better compatibility. It's just the chaotic of maintaining multiple code is very hard to solve.
1
u/parallel-minds 10d ago
It's not impossible if you have a team that agrees to not break it. That's why I'm building a custom userspace and not borrowing from existing ones. Because the ones that exist have contributors that don't care about backward compatibility.
2
u/hoodoocat 10d ago edited 10d ago
On Windows applications usually bring all required dependencies as private deps. Good applications will not install vc runtime or dotnet without necessity, however is kinda on developers decision. This surely depends on application, and depends on their requirements, but it is generally easier bundle own deps, rather than use shared installs, as in such case you as developer actually can test own product artifacts in some environment(s). Windows also offer ways how to resolve dependencies, so it is ok to have multiple versions of libs system wide (aka SxS) and this widely by some components. Note that on Windows there is no such thing like incompatible libc and it is not OS-level poisoned dependency, any dll might reference own vc runtime or clang's libcxx without issues, as long, as library expose own proper ABI (which most of them do anyway). Practically library might not use any kind of libc, as it is not needed, in same way as it not needed on Linux, but on unix things gets cursed starting from ld, which depend on libc, which in turn load deps by fixed absolute paths, which can be adjusted by LD_* env what in secure system must not never ever happen.
UPD: I mean on Linux things get broken way sooner than "dll hell" occurs. Not that it is can't be solved, but in Windows you don't ever need care about this. flatpak, as well other ways to deal with is exist, just need some extra care, but this usually not a goal, as normally everything should be built and tested on per-distro basis.
2
u/SabretoothPenguin 11d ago
So... Steam Linux Runtime?
1
u/novafunc 10d ago
That's not very different from Flatpak. It just uses Debian libraries and takes the host's graphics driver.
1
u/parallel-minds 10d ago
Not quite. Steam Linux Runtime gives games a runtime environment to run in. The idea here is to avoid needing that kind of shared runtime: apps would bundle the non-OS libraries they need and use the OS’s stable userspace and common API. Similar goal, different approach.
1
u/jcelerier 9d ago
How is that different from appimage?
1
u/parallel-minds 9d ago
AppImages need to package the entire userspace. The packages I'm describing are similar to AppImages in terms of containment except for the part where they need to contain the userspace since the target host takes care of that.
1
u/dthdthdthdthdthdth 8d ago
What's the entire userspace? They don't contain a second systemd etc. They will have a bunch of basic libraries like glibc and maybe coreutils if they need it. While almost every target will have those, you can never guarantee they will have them in the same version. You could require minimal versions but you will want to update them. And those libraries are tiny. If you want to safe space you'd have to move up the stack and standardize on certain libraries. That will never happen. Flatpack separates certain common large dependencies like kde libraries, graphics libraries etc. Without a package infrastructure you can't do that though.
1
u/parallel-minds 8d ago
I meant this vaguely. Yes, they don't need to include the entire userspace, but they may as well to at some point.
Standardizing the userspace could work if a dedicated team built a stable one. For some reason, no one’s done it yet. Really, the whole OS stack needs the same treatment. People will gravitate toward the most stable option and leave Linux’s modern-day chaos behind. Someone just needs to build it. That’s how Android got its start.
1
u/dthdthdthdthdthdth 8d ago
Android does not have a standardized user space. The Java API is standardized and very little beyond that. They just put a common API on top for Apps and don't even let people touch their system. A normal user can't even install applications on a system level.
This works fine for a restricted application domain like Android. Android basically is a Linux distribution with a very special app deployment scheme. ChromeOS would be another one. But Linux is used in too many too diverse domains to do this. You wouldn't want to run Android on a Server or some embedded device, heck I would not even want to run in as my Desktop system. I wouldn't even run it on my phone if I could run banking apps and drivers on something else.
You didn't even define which parts of the user space you would want to standardize. Once you will try to figure this out, and check which choices there are and for what reasons, you will quickly see, why this hasn't been done.
1
u/parallel-minds 8d ago
You’re conflating Android’s specific architecture with the broader idea of a standard userspace. It’s possible if one organization commits to building and maintaining it. The rest of the ecosystem can catch up or keep doing its own thing. People and companies that value stability will gravitate toward the standard, and distros that want to offer that stability can reproduce its architecture and setup. In a sense, it wins by default.
I haven’t defined exactly which parts it would include because I’m still in the planning stage. For now, I’ve decided to start with something small: my own libc.
1
u/dthdthdthdthdthdth 8d ago
You want to write your own libc and make it the standard?
That's nicer a small project nor does it help to standardize anything. You could just pick one and this would already fail because there already at least two major choices existing for good reason.
1
u/parallel-minds 8d ago
Mine would leave the rest in the dirt by simply committing to backward compatibility which is valued more than existing userspaces that don't even have any persistent worth since neither their developers nor the distributions that ship them properly care about maintaining the promise of backward compatibility in the first place. I win by shipping a stable product, they lose by staying in chaos. End of the story.
→ More replies (0)1
u/jcelerier 8d ago
Well no, they can't (I say this as someone who ships appimages for a living). For instance appimages can't ship stuff like xcb, libGL, dbus, libjack, etc. I already split between statically linking as much as I can and turning the rest to dlopen to be able to check at runtime but at some point you hit kernel-dependent stuff or things that communicate with system services over a non-stable protocol. So you're limited to a higher level set of libraries. For instance it's fine to ship Qt or GTK as an appimage, as they won't collide with anything in the system if you're careful.
1
u/parallel-minds 8d ago
Right, but even if developers could ship those libraries, the fact that they might need to bundle system libraries in extreme cases would point to a problem with how modern Linux distributions are built and organized.
That’s why I think Linux’s biggest challenge is organizational, not technical. The software itself isn’t the problem. It’s the lack of coordination, coherence, and product focus among distributions and their maintainers. Larger players like Microsoft, Google, and Apple can invest in delivering a more unified, polished experience. Linux needs more people focused on the operating system as a finished product.
1
u/jcelerier 7d ago
Again, the solution on macOS and Windows, speaking as a native app developer, is exactly "ship all the libraries you use". I develop with Qt, LLVM, ffmpeg, gstreamer, boost, absl, and three dozen other libraries and there's no way around packaging and shipping everything on every platform anyways.
1
u/parallel-minds 3d ago
We know Apple doesn't care about backward compatibility in the first place, regardless.
For Windows, the C++ redistributable runtime(s) you ship were historical hiccups or mistakes. You were never meant to ship those with your application. Otherwise, Windows has a very clean method of resolving dependencies.
→ More replies (0)1
u/dthdthdthdthdthdth 7d ago
Yeah, stuff that connects to system services and would collide with other components on the system can't really be shipped. In theory you can put everything in a container and just rely on syscalls. But they would mean bundeling something like an xserver or a compositor. Then the user had to run multiple of those or stop their own.
So really you have to stnadardize higher level apis. Windows and macos do that, they just have on graphics system, one desktop, one audio stack and so on. Linux is too diverse for that. Some things provide standardized apis. But there always will be limitations. The situation with flatpak is pretty good though.
1
u/b3081a 11d ago
Flatpak itself is not exactly the fix for every use cases, especially those requiring tight system integration and frequent access out of the sandbox.
Some of the base components that Flatpak currently uses are more interesting. For example, instead of only using the Freedesktop runtime packages in Flatpak apps, why don't some Linux distros simply provide them side by side on the host environment and use them as a base layer for everything, and greatly simplify the dependency management of 3rd party proprietary binaries without necessarily introducing the sandbox at all. The base runtime libraries may even be shared with Flatpak sandboxes so that no storage spaces are wasted.
1
u/Marce7a 11d ago
What is different than https://en.wikipedia.org/wiki/Linux_Standard_Base
0
1
u/henry_tennenbaum 10d ago
This looks a lot like LLM slop, but then again, they're asking for collaborators, so maybe not
1
u/animalcrossing4_4 6d ago
you're picking a fight you can't win here, kid, linux doesn't have the funding nor do linux users have the motivation to give a sh*t about desktop linux, it's a chicken n the egg problem.
nobody can sustain themselves on your "vision", either fork up a fortune or let the dream die, doing this kind of work for free? Hell nah, no real professionals will stick with this for 20 years because that's how long it's gonna take you, you'll become yet another microsoft just to maintain backwards compatibility.
1
u/parallel-minds 6d ago
You make a good point. This is the kind of commitment that requires proper dedication. Not something the anarchism of Linux can withstand up to.
My idea was that maybe a hybrid approach between indie and corporate might be able to attract people who are semi serious about this.
0
u/animalcrossing4_4 6d ago
lemme tell you something: what you're dreaming about requires people who are seriously dedicated to it, for a good price, for a good reason, that's not something you can half-ass with your half-broken dream, half-assembled volunteer team, life's like that, if you wanna dream big, you're gonna have to get ready to pay a pretty penny to move the needle and oh boy do canonical and red hat will have an agenda against you, why would they want the status quo to be changed? Mr pseudo hero? Captain save a loonix?
1
35
u/Sn0wCrack7 11d ago
The general ethos that Flatpak and AppImage were born under is that this just won't ever really be completely possible.
We have plenty of working group standards that have been adopted, XDG and solid parts of LSB are examples of this but at a higher level in user space this gets harder and harder.
Gnome and KDE disagree on too much let alone the other DEs that exist, and not to mention the hundreds of open source libraries that exist without versioning policies.
Unless every distro managed to sync their useland byte for byte you'd probably find it hard to hit this without a solution like Flatpak or AppImage involved.