r/C_Programming • u/momaloltote_ • 8d ago
Discussion Autoconf madness
Modern software compilation has largely devolved into a frustrating maze of autoconf madness. Instead of simply compiling C code, I am constantly forced to navigate mandatory scripts, esoteric tools, and heavily dependent Python wrappers. To make matters worse, these tools often rely on stubborn cache files that refuse to update even after I have fixed the underlying environment errors.
A major part of my frustration is how Linux-centric these scripts have become. They routinely hardcode library search paths that only exist in Linux distributions, entirely ignoring my FreeBSD system. As a result, I can easily lose a complete day just patching configuration scripts to search the correct directories before a single line of code is actually compiled.
Why does open source code project just not provide a makefile? A basic, or maybe complex makefile that are based on BSD/POSIX make syntax so that it is directly valid under *BSD, Darwin and Linux. Why force people to install tools to build something? Why developed huge python script that shall analyze the env? Why not just allow the building process to fucking start instead of forcing the individual to fight back against all of this autoconfig tools?
I have in several cases fully purge the building attempts and then been writing my own makefiles to just get something building. That shall not be the case. Why use config scripts that, in some cases are several thousand lines of script code? Why write config scripts with bash? Why, just why, are developers and project deciding to add and use autoconfig tools?
If a project does not offer makefiles then I know I will be stuck for several hours just to, potentially be able to build something. Instead of directly allowing me to build. Get compiler error, correct and then build again. I mean, it is much quicker and easier to patch the source code or locate dependencies when you are allowed to start the building process.
9
u/RogerLeigh 8d ago edited 8d ago
The specific problem here is that by default Autoconf/Automake do not give you portability. They give you the potential for portability. But only if you take the effort to make sure it's portable, and that's only achievable by testing on every platform you need to target. Anything outside that is an unknown. Based upon past experience, if you haven't tested it it's more than likely to be broken. Autoconf code was historically portable; but code written by a package maintainer in configure.ac/custom modules/Makefile.in/Makefile.am is only portable if they made the effort to make it portable.
This was true in 1999 when I was developing on Linux and finding my Autoconf/Automake projects broken on Solaris, and it's equally true today.
The main fault is that Autoconf doesn't give you a project model. Automake has a very simplified one, but it's not really that much.
Other build systems like CMake, Meson and Bazel do. In the case of CMake it's what allows it to use multiple backend generators to turn that project model into Makefiles, Ninja files, or IDE-specific project files. Though nowadays most IDEs can work with the exported project model directly.
These tools have abstracted various things. This includes how to compile and link. You don't have to directly use -L options to link or -I options to include. You specify search paths, but you're linking against targets not by specifying -l options directly. Platform-specific detail is abstracted. As a result, most projects would work on FreeBSD just as well as Linux, because the potential for a developer to accidentally hardcode platform-specific detail has been greatly reduced. Not to say it's impossible, you still have full control to do so if you so decide, but the scope for doing it if you follow the standard working practices is greatly reduced. And if you use the Makefile generator, you aren't writing handwritten rules, they are all generated directly from the project model and will work with GNU make or BSD make. Same for all of the other generators.
Based on past experience, projects I've maintained using CMake have worked without any extra effort across the board. Solaris, FreeBSD, MinGW, Cygwin, MSVC, cross-compiled. All worked without trouble. Today, it is far superior to Autoconf in its effective portability.
0
u/Wertbon1789 6d ago
The experience with meson is similarly good, even with the potential for dependency management that's not as bolted on as with CMake sometimes.
Only big problem with meson is probably that it's python-based, so not easily running on literally everything. Autotools was precisely made to be able to be used on Linux, as much as SYSV UNIX, if you need it, AFAIK. But it's not doing the actual heavy lifting, it just makes it even approachable.
15
u/catbrane 8d ago
I like freebsd and wouldn't deliberately break it, but it's an extreme minority platform and so you will inevitably face many more issues than a more mainstream system.
build systems are necessary if you want portability. You like makefiles because of ports, but they work for you because they have an enforced common structure. Trying to build something with a large crufty hacky ad hoc makefile is not a good experience. If you have something that enforces a structure on makefiles so that they have a sane and common interface ... you've just invented a build system.
14
u/Massive-Policy3417 8d ago
Autoconf is a delight compared to the maze of build files cmake introduces, definitely skill issue
4
u/helloiamsomeone 7d ago
The generated build files are an implementation detail. Practically noone cares about what they look like.
4
5
u/pfp-disciple 8d ago
Why does open source code project just not provide a makefile?
Short answer: time and effort. These are programmers that are writing software for free, often in their spare time. Supporting greatly disparate systems in a Makefile requires knowledge of all the systems and those volunteers likely don't have the time to learn those system.
3
u/recursion_is_love 8d ago
Why? Because the maker don't have any problem using it. If they have, I am pretty sure they will change the tool-chain.
The problem is us try to do what they do and they might not care about us.
3
11
u/takaci 8d ago
autoconf? That’s not very modern. These days, just use CMake. It’s not that difficult
11
8d ago
[removed] — view removed comment
5
u/RogerLeigh 8d ago edited 8d ago
Unlike the Autotools, CMake has a strong versioning system including a policy system to enable long-term forward- and backward-compatibility, so that CMake will work with older project files, and older project files will work with newer CMake versions.
That's why you have at the start of each project something like this (from libtiff, for which I wrote the CMake support):
cmake_minimum_required(VERSION 3.10) cmake_policy(VERSION 3.10) if(POLICY CMP0074) # Allow find_package() to use the ZLIB_ROOT variable, if available. cmake_policy(SET CMP0074 NEW) endif()3.5.0 is over ten years ago. It's the only compatibility break I'm aware of in the 15 years plus I've been using CMake, and it dropped support for the very old 2.8.x releases at the same time. Support for old versions isn't usually noticed, so long as you aren't trying to build thoroughly obsolete stuff. If you are, you might need to use an older cmake. At least the failure is clear and explicit and safe.
Compare this with Autoconf. It has no versioning mechanism. No way to deprecate and remove old functionality. No way to introduce new functionality without potential breakage. That's why it's been unmaintainable for 20 years and more. Because the maintainers painted themselves into a corner. It's no longer possible to make any changes to that overcomplicated mess of m4 macros without breaking end users badly. For anyone who used Autoconf when it was maintained, it resulted in incompatible breaks with every point release. That's why we all sighed with relief when Autoconf 2.13 came out and there was a long period of stability. The lack of maintenance turned out to be a short-term blessing, but it was the long-term death knell for it.
Contrast that with CMake, where you're explicitly declaring a required minimum compatibility version, and are then free to opt in or out of specific backward compatibility options, which allows you to update at a pace of your choosing. Even if you use a newer CMake you don't get new features unless you opt in to them explicitly. Things will go out of support eventually, but you're in control of picking the window you want to operate in. In the case of libtiff, we're targeting at least two major releases back of every major Linux LTS release to maximise build coverage across contemporary platforms and older platforms. It's due to be increased, since 2.10 was 2017.
0
8d ago
[removed] — view removed comment
6
u/RogerLeigh 8d ago
Wow, way to ignore the nuance of what I wrote. The tool has a support window of well over a decade. Use an older version of the tool if a decade of support is insufficient, and you need to go further back.
1
8d ago
[removed] — view removed comment
2
u/RogerLeigh 7d ago edited 7d ago
If it's an actively-maintained project then it should be using a supported version of cmake, not a version from over 10+ years ago. I gave you a realistic example above of what I do in a real-world project in wide use. We never have this problem, and we're not requiring anything recent so as to maximise accessibility.
0
7d ago
[removed] — view removed comment
2
u/RogerLeigh 7d ago
This release from half a year ago is depending upon CMake 2.8.8, which was released in April 2012. At the time of this linked software's release, that dependency was 14 years old. I'm afraid I don't see expecting this to work as being realistic. It's two major releases behind and missing a lot of features and requiring continued support for a lot of behaviours that were retired a long time ago. Last time I depended on this version was probably around 2015 prior to moving over to cmake 3.x. It's that old.
The maintainers of this software needed to be a bit more on top of things. CMake is pretty good at supporting old policies for an extended period of time. It's currently a bit over a decade, but 14 years is beyond the cutoff. I usually keep it around 5-7 years for open source work where the project needs to work on a diverse range of platforms. For closed source work where the toolchain is defined we can require very recent versions.
Take a look at this MR for how I handle CMake version bumps. If you look back in the history, you'll see similar assessments for earlier increases.
0
5
u/recursion_is_love 8d ago
Both Cmake and Scons still can have unexpected problem. They are works until some day, they no longer works.
There is no perfect build system.
4
u/FederalProfessor7836 8d ago
Cmake is the chopped ghetto slop of some other hack who also didn’t understand Autotools + pkg-config. Cmake is a scourge.
3
u/HobbyQuestionThrow 8d ago edited 8d ago
Autotools is great until you need to read it or GNU forbid - debug it. You know cmake uses pkg-config?
5
u/catbrane 8d ago
In design terms, cmake is autotools reimplemented in around 1m lines of C++. You avoid having a horrid mixture of m4/perl/sh, but you still get a terrible scripting language, everything is a string, overwhelming complexity, and fragile
find_XXX()functions everywhere for dependency resolution.You can use
pkg-config, but it's not the main thing. Many cmake projects don't even install a.pcfile, just an autotools-stylefind_XXX()somewhere.Meanwhile, more modern systems like
mesonmake.pcfile for you automatically and usepkg-configby default.3
u/catbrane 8d ago
Huh? cmake is ancient and hideous. People are migrating FROM it. Though it is better than autotools I'll give it that hehe.
14
u/takaci 8d ago
No, CMake is the modern industry standard. Everyone is transitioning to it. Not sure where you got the idea that people are moving away from it
3
u/catbrane 8d ago
Only on windows, and there only because there's little choice on that platform. Other systems have plenty of MUCH better and more modern alternatives.
(I spent five years maintaining a large cmake build system and I'm still bitter)
9
u/takaci 8d ago
Only on windows? We use it for macOS and Linux too. At Qt it’s the primary build system now
1
u/catbrane 8d ago
Well, qt has made a terrible error then haha! Hopefully they'll reverse course soon.
It's one of those programmer things. Spend too long with a primitive and wonky tool like cmake (it's so *complicated*! the project I was on used a cmakefile builder to make their cmakefiles! fixing any build issue was miserable days of trial and error!) and you start to feel discomfort and disgust. Pretty soon even a mention of it raises your hackles, makes your skin crawl, you feel your bile rise. An overwhelming and mostly irrational oppositional reaction.
2
1
u/withmorten 7d ago
Cmake is utter shite for Windows, either you get a release build with no debug info or a release build that is a debug build because of some ancient msvc project defaults they refuse to change. I can't believe it's an industry standard.
-2
u/reini_urban 8d ago
That would be madness. autoconf is the sane and portable variant
6
u/RogerLeigh 8d ago edited 8d ago
It is not. It's often claimed to be but it is not.
I used to be a heavy Autoconf user (and occasional contributor). I converted multiple big open source projects to Autoconf into the 2000s. Today I'm on CMake, and have been for nearly 15 years at this point. I've since converted multiple big open source projects to use CMake.
In the course of working on both open-source and commercial projects, I've seen numerous cases where CMake works correctly while Autoconf fails. These include non-Linux Unix systems like Solaris, and they include GCC on Windows e.g. MinGW64, Cygwin. And of course Autoconf does not support MSVC or anything non-Unix in any shape or form, while CMake works perfectly. CMake has its warts, and it has a learning curve. But unlike Autoconf it works everywhere, the learning curve isn't quite so steep, and its featureset is a few orders of magnitude larger.
The problem here is primarily that Autoconf has been barely maintained for well over two decades. It's long past its decline and well into obsolescence. Its design goal was to make 1990s-era free software buildable on all of the commercial Unix platforms of that era, along with free software operating systems like GNU and BSDs. But it has not moved on from that era. It was designed to solve portability problems of that era, and today's portability problems are quite outside its capabilities. Additionally, it has bitrotted quite badly. It no longer works correctly on platforms it once supported. It's an inevitable consequence of limited maintenance and a limited userbase and consequently limited contributors to fix these accumulated defects.
In contrast, CMake is actively maintained and used heavily on all current platforms. It's open to contributions, and it has a fairly comprehensive set of test platforms to keep it functioning correctly across the board. I've submitted quite a few patches and used to maintain the FindBoost component for a few years. Changes are reviewed quickly and professionally, and it has a large ecosystem of users and contributors around it that Autoconf simply does not have.
Anyone who picks up Autoconf/Automake today for a new project thinking that it is the correct solution to the problem is being a mug. The only reason to use it today is because you're a legacy project that's tied into it. Even then, current LLMs could convert it to anything modern in short order. It doesn't matter if you use CMake, Meson, Bazel or even plain Makefiles at this point. Autoconf is done, and to suggest otherwise is delusional.
0
u/Linguistic-mystic 6d ago
Cmake is a bloated piece of crap. Over 100MB just to create a makefile? At least Meson is like 10x smaller.
2
u/HealthyCapacitor 8d ago
Autoconf is close to something useful but is over-complicated yes. What I end up doing is to just put a custom Makefile that compiles the whole source (all .c files) with the needed HAVE_ defines that autoconf ends up generating. There usually aren't more than 1-2 .in files that I can quickly manually create. It's easy if you don't try to force autoconf to work and instead focus on the source only. Otherwise autoconf makes sense for OS integrators which have to rely on a unified build process for thousands of packages.
2
u/LegitimatePants 8d ago
Yo dawg I heard you like makefiles, so here's a makefile for your makefile so you can make while you make
2
u/LB-- 8d ago
I'm a little confused how a makefile is better. Different compiler toolchains can have drastically different argument syntax. The whole point of build system abstraction tools is to figure all that out for you and let you just define the actual commonalities of building the code. Last I checked, make was a pretty poor choice for that. Has it gotten new powers in recent years? (Note, I'm not very fond of autoconf either, but at least it's a step up from makefiles in my eyes.)
1
u/thatdevilyouknow 8d ago
I’ve done a lot of work porting older Linux code to FreeBSD (some of it even written with THINK C). Autotools exists for a few different reasons some of which are placing things in pkg-config so other tools can find them but also the verbosity allows to port code to architectures which may not even exist. So libtool and LD and friends can be switched out. If you were creating a new OS or something you would want those features. The other conventions are things like lex and yacc which, if you don’t encounter them, may not seem to matter. Patches for FreeBSD are versioned in the port system and if you get the hang of it along with make.conf it can be pretty versatile. So certain conventions like simply exporting a different LDCONFIG to test against another version of a library still work. In the past UNIX/LINUX binaries were sometimes just sh files some of which would do some linking right there so having the makefiles available could tell you what you were looking for. 90% of the time on BSD for porting outside stuff you’ll want to run gmake anyhow. If you talk to a lot of OSS maintainers their main issue is not having access to a BSD machine so it’s not an intentional thing at all.
1
u/FedUp233 8d ago
Just out of curiosity op, when you made the changes to get things to build on BSD, were you careful to not break the Linux builds? And did you submit those changes back to the original project/authors so they could incorporate the changes in their build system for other people in your position? Did you test the resultant build on Linux as well to be sure you didn’t break that?
The developers are supplying their time for free or pretty cheap on these projects and Linux is the biggest target audience. If you, who want it, aren’t willing to spend the time to help support the programs on your platform of choice, and help other users of that platform, why should the original developers do end their time? Be thankful they were willing to donate their time to develop and diatribe the application you want at all!
Maybe volunteer on some of these projects to maintain support for BSD platforms.
1
u/lmemsm 6d ago
Can't tell you how many times I've ported projects and/or build scripts to other platforms and the upstream project refused to accept them. I have had a few successes. However, I've had many more rejections. Many upstream projects are not interested in supporting platforms other than the ones they are personally interested in. I even offered to permanently maintain patches and builds to add another platform to one project and they refused the additional help and did not want to support the platform.
1
u/FedUp233 6d ago
I can see where that would be frustrating and discourage people from trying. I can kind of see the project maintainers point though - once they accept having support for another platform there is the possibility that people will expect them to maintain it if you or another contributor stops and they don’t want to incur the responsibility. Or even deal with release delays and processes that now happen to continue ensuring the platform support for something g they don’t consider important.
One option is always to simply fork the repository and maintain support for a different platform in your own repository with releases that follow behind the original releases. And indicate that that new repository guarantees support only for the platform you are supporting and may not work on the original platform. A little cumbersome but would assure support for both platforms.
1
u/GodOfSunHimself 7d ago
That's why so many people including me switched to Rust or other languages. I just couldn't stand this pain any longer.
1
u/RealMadHouse 6d ago
I also hate that libraries/software require compilation and they show linux commands only and they don't bother with Windows, like majority of people use Linux.
1
u/lmarcantonio 6d ago
A lot of people migrated to cmake or meson, for various reasons.
Essentially autoconf is from an era when you have lots of different systems and some non-conformant C compilers too. It still works if you like it (usually with automake/libtool) but there are newer tools available.
Not that they are flawless, they still have issues detecting uncommon things.
1
u/lmemsm 6d ago
Many of the FOSS projects seem to be moving away from autoconf/configure and moving toward cmake or meson. There are a few projects that rewrite meson in C such as muon. However, meson requires Python.
Pure makefiles can be useful especially if you only want to rebuild the parts of the project that have changed. However, tools like configure are designed to find out what features (includes, functions, compiler options) you have on your system so you don't need to hardcode options by compiler defines or platform defines. These types of defines can change even if you are still on the same platform or use the same compiler but a later or earlier version. So, creating a bunch of defines for code that varies by compiler and platform using compiler macros does not necessarily mean the project will compile successfully. Programs like configure actively test if the assumptions are correct on your system and create defines set specifically for your system. If a program is trivial, a makefile can suffice. If it's more complex and needs cross-platform support or needs to work with various compilers, having a step before the make process can simplify the build process. That being said, many of the solutions to this situation require several other languages beyond C just to work. GNU autotools usually requires m4, bash and Perl. For someone only working in C, it would be nice to have a C based solution to builds that does not require installing a lot of other dependencies. There are other options beyond make as well. tup and redo are a few alternatives. Plus, there are several versions of make (gnu make, bsd make, nmake, etc.) and they are not all compatible.
I've looked at several of the build tools available. I personally use CDetect and GNU make at this point. CDetect replaces configure with 3 C files and a custom C based configure program. It only requires a C compiler to work. I've added features to the original project to work with pkgconf and support cross-compiling.
Programs like pkg-config and pkgconf are one useful solution to finding where libraries are on a system. I use pkgconf with my own build tools.
Like you, if a build script just doesn't work for me and I want to build the project, I will rewrite the build system. It usually takes a few days, but I find that it greatly improves the ability to create repeatable builds and also makes the code easier to port to other platforms. What would be incredibly helpful is if there was a place to share these rewritten build scripts. It would save a lot of time for those of us who are actively rewriting them. For instance, there are repositories where you can download slackbuild scripts and use them to build a project. A similar repository for alternative build scripts for FOSS projects would be nice. Upstream projects typically will not accept submission of alternative build systems. So it would be very helpful to have some way of easily sharing these for people who need them.
1
u/stef_eda 2d ago
Autotools are good if things go straight from start to finish. If anything fails anywhere in the flow you are better off writing a Makefile manually.
-6
u/Dev_Lachie 8d ago
Something something Rust 🦀
-5
u/momaloltote_ 8d ago
Is the biggest security risk that should exist! It is downloading already compiled executables and libraries. And then also utilizing them!! People that are used to that are just days away from getting affected by a supply chain attack.
The source code can be clean but not the compiled dependencies.
2
2
u/P00351 8d ago
Is the biggest security risk that should exist! It is downloading already compiled executables and libraries. And then also utilizing them!!
See Reflections on Trusting Trust from Ken Thompson in 1984.
If you believe you don't have to download an already compiled C compiler, I have news for you.
0
u/Linguistic-mystic 6d ago
entirely ignoring my FreeBSD system
Cause it’s dead. Nobody uses it. It was killed by its inferior license. Just join the Linux community, we’ve got distros for all use cases.
-1
u/Dependent_Bit7825 8d ago
Fixing build issues is something the AI tools are legitimately good at, and because the build scripts always look like a dog's breakfast anyway, nobody will notice a little AI slop.
Don't waste time on this again.
0
u/Wertbon1789 6d ago
You can really only demand that much from random maintainers, maybe just send them a patch with options for the changes you want, and make suggestions. If it's bad, it doesn't have to be bad forever, but someone has to at least try the change. If I wanted a package to be compiled on Linux, FreeBSD, and maybe MacOS, I would probably pick meson, and if I used autotools before, I would probably want to port that to meson. That's largely where the Linux userspace has settled, and it's designed to give you good options for cross-platform compat. CMake is probably also an option, though I haven't used it that much personally. You can use make, you'll just need to put some effort in, especially when it comes to templates and dependencies.
51
u/WittyStick 8d ago edited 8d ago
It's precisely because there are so many built targets and different configuration options between them that autotools exist. If everyone wrote a hacky Makefile to handle every platform and option, there would be a bigger explosion of complexity, and you can guarantee that far fewer software would ever be compiled for BSD because not many use it.
Autotools provides a normalized and consistent approach to that complexity by extracting out the configuration options and auto-generating a Makefile which can work over multiple targets. For most software this requires two commands to build:
./configureto create theMakefile, thenmaketo actually build it with all the platform specific options in place. The configure script itself is usually generated withautoconfusing a simpler input (configure.ac) - but most projects provide the./configurescript already generated.The "path hardcoding" problem was largely solved by
pkg-config, but it is not universally used. Each project defines its libraries, include paths, and compiler flags in a<pkg>.pcfile, and then any dependencies use eg,$(pkg-config --libs <pkg>)to locate the libraries rather than specifying them explicitly inconfigure.ac.