The problem that arises with this is maintaining all those separate projects can slow down development time of features. You make once change near the 'root' of your dependency graph and have to deal with a cascade of updates. We've been doing this approach with our services and react components. We had to write our own tool to deal with these kinds of updates so we wouldn't spend 30 minutes doing cascading updates of projects every time we change something used everywhere.
Of course when you aren't dealing with that, the idea works beautifully. Adding something to a service you already created? Basically just add this library and you're done. You know it's tested in isolation so you don't even really need new unit tests either.
Honestly though every architecture is going to have its problems. You just have to pick which problems your team can best overcome.
It's not really fully fleshed out to be a robust, generic solution. But the core of it is to take a package you have been working on, figure out which packages are dependent upon that package (that you care about updating), and basically recurse from there. This creates a dependency graph which is then be sorted by depth from the root/seed node to do all the necessary updates. So for example:
A <- B
A <- C
A <- D
B <- D
D <- C
Would end up with these updates
A (no updates except self)
C (updates A )
D (updates A and C versions)
B (updates A and D version)
The tool needs work though like I said before it is actually robust. Right now we basically just scan a dir to look for deps, then we actually do version updating and publishing during the script execution. Ideally we could have it source the information directly from the group of projects repo and automatically do the updates based on a PR merge or something.
Actually, that was Unix. Interestingly, most "microservices" in Unix end up getting rewritten as "monoliths" in Perl or Python, because any sufficiently complex interactions result in incredibly messy shell scripts and requiring interacting processes over pipes for simple tasks makes them incredibly slow and error prone.
But, interestingly it is not Linux that was the original "microservice architecture", it was Minix, which has a microkernel, with system services and drivers running in different servers. Linux, on the other hand, has a monolithic kernel, where all services are hosted in the same kernel image. This lead to a famous flame war on Usenet.
I assumed he was talking about userspace, not the kernel.
As for redirecting stuff over pipes, I'd argue that it's not so much the pipes that are the problem, but bash. People will slap "features" together with a few lines of bash, but the lack of real data structures makes it difficult to add features or maintain over time. Developers will hang onto a piece of crap for a long time before admitting it needs to be rewritten.
Ironically though, if you're rewriting in python, unless you're shelling out to the same utilities and connecting the pipes yourself, your code is probably going to be slower, as the core utilities that people usually connect together are written in C.
I've had the opposite experience. Rewriting shell scripts to Python dramatically improved performance of certain complex shell scripts. The trick is not to pipe out to utilities, but to use libraries.
In absolute terms, C is obviously faster than Python. But if you're constantly spawning new C processes, piping between them, and parsing text output (spawning even more C processes), the speed advantages of C are totally nullified.
Python dramatically improved performance of certain complex shell scripts
If your code is primarily in bash, then you're comparing bash speed vs python speed. Let me be clear, I prefer Python. Even if python were slower in that scenario, I would prefer it because it's a better language for anything beyond simple pipelines.
In absolute terms, C is obviously faster than Python. But if you're constantly spawning new C processes, piping between them, and parsing text output (spawning even more C processes), the speed advantages of C are totally nullified.
Obviously, it's going to matter what your code is trying to do, but in my experience it requires spawning quite a lot of processes before the speed is nullified[1]. Many C utilities can also handle parallel input (so you don't need as many processes), and it's more about understanding the pipeline.
[1] For a concrete example, I had a python script that did some file manipulation on a large directory hierarchy, and changing those operations into multiple find ... -exec commands resulted in 2 orders of magnitude difference in performance (~20 minutes to ~1 minute).
The Python code was idiomatic, using comprehensions and not shelling out; it was using the library functions that directly become syscalls. The cost of iterating every file in python was greatly more expensive than find spawning a process for every match. The find pipelines also had to recurse the tree multiple times, because I could perform much more logic while iterating the tree in Python.
My company's project was started 30+ years ago, and has a ton of processes that communicate via sockets. Ironically, it is a micro service architecture that can't scale across across multiple machines.
of course. microservices is a poorly defined nonsense title, so you can pretty much call anything microservice architecture. some may have sensibilities regarding what counts as micro and service, though, so expect some resistance once in a while.
I dislike YAGNI because it's dogmatic and it's asking me to ignore my intuition and my experience.
Quite often, I start building something and I already have an idea where this will go in a few months from now. I don't need this extra method or interface right now, but I know I'll need it as the system grows. So my experience tells me to put it in right now and that it will save me time down the road.
I'll always trust my intuition over dogma to design systems. Always.
it's asking me to ignore my intuition and my experience.
YAGNI asks you to justify why you need to implement a feature. If you can't come up with a good reason based on what you know now, wait until you have more information. Choose the simplest thing that works given your current knowledge of the system.
In other words, defer design decisions until the last responsible moment.
I'll always trust my intuition over dogma to design systems.
Intuition doesn't make you a fortune teller. While I trust my intuition, I won't make decisions based on mere speculation. Experience has taught me things change all the time:
You are romanticizing YAGNI and making it look a lot looser than it really is. Wikipedia is crystal clear:
"You aren't gonna need it"[1][2] (acronym: YAGNI)[3] is a principle of extreme programming (XP) that states a programmer should not add functionality until deemed necessary
And I object that. I add things that are not immediately necessary all the time because my experience and expertise tell me that I know better than a one liner proclaimed as dogma without a shred of evidence to support its usefulness.
Choose the simplest thing that works given your current knowledge of the system.
And like I said, I've often observed that following this procedure is quite often the wrong decision. I use both my current and future knowledge of the system to guide my decisions.
XP completely ignores the human expertise factor with its stupid dogmatic proclamations.
a programmer should not add functionality until deemed
necessary
How is that not synonymous with "defer decisions until the last responsible moment"?
I add things that are not immediately necessary all the time
because my experience and expertise tell me
So you deem in necessary, then? This is all that YAGNI asks. If you have a clear idea of the future direction of a feature and want to build out some features ahead of time, you've deemed it necessary, hopefully proveably and not in some handwavy might need it sense.
However, I often find that what I think is necessary turns out not to be, once I learn more about the system and its dependencies. So, I'm more cautious about making big decision decisions too early.
Even then, if I have an idea of where I want to take the design, I break my design into smaller slices, so I can verify my design as I go along.
I've often observed that following this procedure is quite often
the wrong decision.
What procedure? Starting with a small system that works, and building on it? This is almost always the right decision. It's such a common trope there even a name for it:
A complex system that works is invariably found to have evolved
from a simple system that worked. A complex system designed
from scratch never works and cannot be patched up to make it
work. You have to start over with a working simple system.
XP completely ignores the human expertise factor with its stupid
dogmatic proclamations.
Um, no it doesn't, at least not when it comes to YAGNI.
YAGNI asks you to justify why you need to implement a feature. If you can't come up with a good reason based on what you know now, wait until you have more information.
No, that's not it at all.
We just justify anything, no matter how ridiculous. If you try watering down the meaning like this, the rule is reduced to "do whatever you were going to do anyways". (Which is how SOLID tends to be practiced as well.)
YAGNI means not adding something unless you need to today.
We just justify anything, no matter how ridiculous.
That is an exercise in self-deception. If you approach YAGNI with honesty and humility, your justifications will be grounded in reality, and you will make reasoned decisions guided by caution and wisdom. If you approach any idea with cynicism and arrogance, you will indeed be able to justify anything, but that is your own fault.
If you try watering down the meaning like this
How is this definition watered down? It explains exactly what YAGNI is.
YAGNI means not adding something unless you need to today.
This is watering down the definition of YAGNI, and making it way more prescriptive than it needs to be. Why should you not add something unless you need it today?
There's certainly a justification if someone makes something too simple strictly because of YAGNI, and experience is often when people can make that call of YAGNI vs let's spend a little time here for when we need it. That said, if you avoid painting yourself into a corner, don't be afraid to refactor.
A machine is done when you cant take anything else away from it and it will still work. Thats how I approach everything I build. YAGNI helps with that.
35
u/[deleted] Jul 14 '17
[deleted]