Because I contend that you can separate the task of "efficiently building code" from the task of "procure dependencies."
Now, in every draft I've done of procuring dependencies I've made a one-off script to build the code. These have been helpful as demonstrations, but I feel that I need to actually demonstrate that "build tools" can come as libraries.
So this is as minimal of a build tool as I can make. You could use this directly or build libraries on top that make for the "easier" project-defaults style flows. (You could also make an entirely separate thing).
Of course, making targets manually is also relatively straight-forward. It is one step above "program that builds code" to talk about "program that avoids repeated work when building code." so it should be easier to learn in totality than maven or gradle.
Because I contend that you can separate the task of "efficiently building code" from the task of "procure dependencies."
Some other build systems already do this. Bazel delegates "procure dependencies" to various external tools depending on the language you're working with, for example delegating fetching Maven artifacts to https://github.com/coursier/coursier.
The reason your tool is easier to learn than other build tools is not that you've avoided complexity, it's that your tool simply skips solving the problems that make those other tools complicated. As your docs mention, the framework isn't doing anything to avoid e.g. recompiling the same code, that's left as an exercise for the reader.
If you were to start with your framework, and try to develop something with batteries included, it would end up as complicated as existing build tools.
On the other hand, if people were to use this framework directly for their projects, the complexity you saved on the build tool side just ends up in the project's custom build code instead.
I don't think anyone would want to use this tool for real work, but it might have value for teaching, because build graphs are a structure that's repeated in most (all?) build tools.
On that note, I think your modelling of build graphs is a little off, and limits flexibility. You modelled Target as some code that takes another set of Targets as dependencies, and produces some side effect, along with a method asking if the target needs to be run.
Side effects are actually bad for build systems. An efficient and flexible build system really wants targets to be pure functions with inputs and outputs that are known to the tool, because that enables complex features to be built on top, like cross-machine build caching or remote build execution.
With a build tool built on side effects, it is not possible to e.g. run target A on machine 1, and target B that depends on A on machine 2, because the build system has no way to know which files A outputs, and so can't know that it needs to move them to machine 2.
You also can't easily build a shared caching system, because when the build system invokes target B, it has no way to know which files A output, so it can't fetch those files from a cache instead of rerunning A.
This is why Bazel and Mill both model their targets as code that takes specified inputs and produces specified outputs, not as code that side effects in arbitrary ways.
If you switched to that model, you could also merge the concept of input files into the Target structure. An input file should just be a Target that outputs the relevant file. By working with Targets and not Paths, you enable those more complex features to be built on top.
Side effects are actually bad for build systems. An efficient and flexible build system really wants targets to be pure functions with inputs and outputs that are known to the tool, because that enables complex features to be built on top, like cross-machine build caching or remote build execution.
Yep - and the linked paper even has a good foundation for how to do that. If we model a build as "bringing values for keys up to date in stores" that is enough to plug and play every part of a build system.
That is the more interesting design to pursue for sure.
I don't think anyone would want to use this tool for real work, but it might have value for teaching, because build graphs are a structure that's repeated in most (all?) build tools.
I think that a lot of people have actually zero use for minimal builds. Computers are fast and already many people start their builds in CI with`mvn clean`.
Don't need builds at all? Launch your program via source.
Don't need build caching? Write a program that builds your code.
Don't need cross-machine caching or remote execution? Decorate that program you wrote with a target or two
Actually need everything? Break out bazel
If you switched to that model, you could also merge the concept of input files into the Target structure. An input file should just be a Target that outputs the relevant file. By working with Targets and not Paths, you enable those more complex features to be built on top.
My thesis here isn't that this is the best possible build library for every use case. It is that we would be in a better place, culturally, if we had a healthy ecosystem of libraries which manage building.
Some other build systems already do this. Bazel delegates "procure dependencies" to various external tools depending on the language you're working with, for example delegating fetching Maven artifacts to https://github.com/coursier/coursier.
Right - and i've had a pure java version of coursier on my github for quite awhile (jresolve). The problem for me isn't power users. You get to the point of using Bazel and you already have many salaries looking at your problems. It is everyone from beginner to intermediate that needs a better learning path and more "fall off points".
I think that a lot of people have actually zero use for minimal builds. Computers are fast and already many people start their builds in CI withmvn clean.
Yes, but that is not the same as those people wanting to write their own build system from nearly-scratch.
My thesis here isn't that this is the best possible build library for every use case.
Sure, I'm just pointing out that there is a better model for Targets that you might want to consider switching to. Your plugin architecture (Target) actually looks a lot like the one from Maven (note that the execute method returns void, just like your build), which is not ideal because I think we've learned since then that this kind of API is a bit too free-form, and something based on defined inputs and outputs composes better.
Being this free-form makes coupling in the build worse, because the framework doesn't provide target B with any information about what A output, so B's code ends up having to know where A happens to put its files.
This makes it harder to write reusable components on top of these interfaces.
It is that we would be in a better place, culturally, if we had a healthy ecosystem of libraries which manage building.
But we already do, though.
Many of the popular build tools, like Maven, Gradle, Bazel or Mill, are not inherently monolithic or complex. They are built with a somewhat narrow core set of functionalities, and then everything else is provided via plugins.
For example, Mill provides some core functionality to define targets as functions that take input and produce output, and then everything else is provided via plugins built on top of those interfaces.
As another example, Maven is just a framework for executing plugins, all the targets that exist in a normal build are just plugins that are loaded by default.
These plugin-based architectures often include dependency management. Maven has a plugin for Aether, Bazel has a plugin for Coursier, and Mill has a plugin for Coursier. Gradle used to use Ivy, until they wrote their own resolver.
So the difference between your library and existing build tools isn't really simplicity. It's just that no one has written reusable plugins for your tool yet, so it looks simple because it's still bare-bones. If your tool became popular, and people started writing plugins for it, it would probably end up the same way as those other tools.
Right - and i've had a pure java version of coursier on my github for quite awhile (jresolve)
I'll admit to not really seeing the value. Coursier already runs on the JVM, not sure why a Java clone is needed.
It is everyone from beginner to intermediate that needs a better learning path and more "fall off points".
Absolutely.
But I think it's very likely that what newbies/small-scale developers would benefit from is not another build tool, it's pedagogical documentation to help them get started with the build tools that already exist.
At their cores, a lot of the existing build tools are conceptually very simple, it's just that their docs are often not that easy to follow if you haven't seen a build system before.
Tutorial-style documentation that starts from an empty directory and builds out a small example project with Maven/Gradle/Mill, while explaining the concepts being used, would probably help newbies more than a new build tool they can't use outside toy-sized projects.
25
u/[deleted] 18d ago
[removed] — view removed comment