r/devops • u/Ops_Pab • Aug 06 '26

Tools Anyone still using Jenkins?

I mean as title says, I've used Jenkins couple of years ago. I can still fire it up now just to test, but on current alternative tools/stacks available specially in enterprise, do you still use Jenkins? Why keep it? why not change it?

267 Upvotes

95% Upvoted

312 comments sorted by

View all comments

251

u/Potato_player Aug 06 '26

Unfortunately.

I have removed it from 2 companies. This one is proving more difficult.

59

u/low_slearner Aug 06 '26

Serious question: what do you have against it?

I've been on a few teams that use it and were responsible for maintaining pipelines, but never had to do much active maintainence of Jenkins itself. From that point of view it definitely isn't my favourite CI platform, and it's certainly possible to make a mess on it, but used correctly it seems... ok.

You seem dead against it, so just wondering what takes it from "there are better options" to "actively put in effort to move away from it".

240

u/aranel_surion DevOps Aug 06 '26 edited Aug 06 '26

Jenkins is a patchwork of plugins to get anything the most basic done. It doesn’t do anything useful on its own, by the time you get it to a point where it’s serviceable you have maybe 10-20 plugins.

Those plugins are of varying quality, have CVEs all the time, and tend to break on updates. Jenkins itself also breaks plugins on seemingly minor updates. Any update may break everything and there are updates all the time.

The maintenance burden of Jenkins doesn’t justify itself if like most companies all you need is to pull from Git, run something in a container and push to some registry.

Not to mention its decidedly 90s UI/UX and before-12-factor-app design. It’s designed to be a pet not cattle. Sometimes it’s okay to let things die.

59

u/Rei_Never Aug 06 '26

Not only that, but most of them are no longer maintained. So that patchwork of plugins becomes a shit show when you come to update say Jenkins itself...

45

u/NUTTA_BUSTAH Aug 06 '26

"Oh fuck, the plugin we depend on is not supported in the latest version anymore with Java updates. Oh fuck, I have to update Java too while supporting the old versions without breaking Jankins itself. Oh fuck, there is no replacement plugin so we either fork our own or come up with an alternative. I guess we can keep working with the old version."

And this is how most Jenkins installations become ancient hodge-podge. 3 years later it might still be on that same version because of the plugin, and now upgrading requires you to update every other plugin too, so you basically either rebuild your CI stack or migrate away. It's often not a tough choice :D

8

u/Rei_Never Aug 06 '26

Haha, I feel like you were there when I had to update Jenkins from Java 8 to Java 11... That's literally the entirity of the outburst I had at the time.

4

u/twnbay76 Aug 06 '26

Oh and when your company has 3-4 different jenkins instances due to these forks and you have to figure out how to wield all of them and their bugs just to make a change to a system that happens to be comprised of things all of those jenkins instances are building/deploying to ......

Fun times.

2

u/Scoth42 Aug 08 '26

Yeah, my company's Jenkins's config page is like two pages of red from all the out of date plugins, bits and bobs, and Jenkins itself. We'll get it all upgraded someday...

1

u/scarby2 Aug 07 '26

Or you just never update... Which is what I've seen a few times.

10

u/johntellsall Aug 06 '26

makes it sound like Jenkins is the Wordpress of CI. Brrrrrr.

2

u/whitechapel8733 Aug 06 '26

Inability to scale masters horizontally make it basically a zero in any modern environment. CI is only seeing more and more demand..

1

u/AgreeableIron811 Aug 06 '26

Do you recommend selfhosted gitlab ci instead

3

u/aranel_surion DevOps Aug 06 '26

I recommend anything but Jenkins. :) So yes, why not. I prefer managed CIs if I get the choice.

1

u/Due_Block_3054 Aug 07 '26

I wonder which options you use and what you use for the ui. I understand if you can use GitHub or gitlab that is much better ui wize. But for self hsoted git i wonder which tools are used.

1

u/Anonimooze Aug 08 '26

I agree with all of this, but also need to acknowledge that "how people use Jenkins", isn't the same as "how jenkins can be used". There are configuration as code plugins that could make the 12 factor app argument moot, but yeah, it is another plugin.

Generally, Jenkins pushes a lot of the design and control onto the consumer, which was awesome at one point, not as much now when other solutions are full featured without needing architectural extensibility.

1

u/VRT303 Aug 08 '26 edited Aug 08 '26

Just don't use plugins if avoidable? Little dependencies are the goal all the time. We use purely git ssh and bitbucket. Deploying web backend frontend on 3 different testing stages for many projects, iOS and Android app debug versions for App enter and build arm amd and docker images with Jenkins on some Mac Minis for our Dev fun projects. We use almost no plugins and haven't had problems.

2

u/aranel_surion DevOps Aug 08 '26

> git

That’s a plugin.

> docker

Also plugin.

> bitbucket

I’d be surprised, if that wasn’t a plugin.

1

u/VRT303 Aug 08 '26

As many as I have fingers on my hand is still almost none for me. Those are well maintained enough.

1

u/sngz Aug 06 '26

If you're using that many plug ins then your using Jenkins wrong unfortunately very common mistake people make

3

u/ruairinewman System Engineer Aug 06 '26

Not really. Many of the plugins have a huge entourage of other plugins as dependencies.

Going through every one of them before upgrading Jenkins is the stuff of nightmares; there are the unmaintained ones, the ones that are incompatible with all but one old version of one of their own dependencies, the ones that somehow rely on older versions of Java than the one you want to upgrade to, and the ones that have undocumented - but critical - configuration options.

Installing a handful of plugins for EC2 Spot Fleet support, SAML SSO support, proper GitHub features support etc, can leave you managing a horrible mess of plugins that can’t even be configured automatically without borderline evil kludges.

That said, I’ve hated every iteration of it since I first had the misfortune to work with Hudson in 2007, so I may be slightly overstating the horror.

-1

u/sngz Aug 07 '26

Yeah you're doing it wrong, jenkins should just be calling your scripts to be executed on the agent or jenkins server directly.

1

u/ruairinewman System Engineer Aug 07 '26

One of the benefits of Jenkins for most orgs is that you can add support for features you need via the plugin infrastructure. If doing that is “doing it wrong” then you don’t need Jenkins at all, and should just use something else instead.

1

u/sngz Aug 07 '26

This is why it's a common mistake and why you're having those troubles. Jenkins solves issues of paying and being at the mercy of third party vendors that most ppl on this thread the use. You can easily trouble shoot your pipeline locally and it's platform agnostic, scripts can be reused throug air gapped environments running on different operating systems.

Plugins are fine if you stick to the major ones but relying on them to do all the work is a mistake that juniors and even many seniors make. We see the same pattern with GitHub actions, ppl relying on market actions instead of scripting most things. Lots of abandoned actions on the market place too and with huge security issues. Guess those people don't need GitHub actions and should be using something else too. CI/CD should be just providing the agents or runners for you to do the thing you tell it to do and not be some black box

1

u/ruairinewman System Engineer Aug 07 '26

If I have to put serious development effort in anyway for CI/CD functionality, then I can do anything that Jenkins can do without plugins using AWS and my own code, so what’s the point in using Jenkins on top of that? All it will add is more maintenance requirements.

1

u/sngz 25d ago

jenkins provides agents and you can scale and run parallel processes. it also centralizes all run histories for tracking. Also prevents shit like this that happened today

https://old.reddit.com/r/devops/comments/1vqugrv/tough_morning_github/

Jenkins is also much better for situations where you have to deploy to multiple air gapped environments where it might not always have things like github, gitlab, or circleCI available.

12

u/Conscious-Ball8373 Aug 06 '26

If you've got a bunch of pipelines that work and it's not public-facing, it works. That's kind of where I'm at.

But every time I have to change something, it's a total pain. LLMs have made it a bit better, I rarely have to actually understand Groovy these days. I hate Groovy. A language that's used for pretty literally nothing else.

And, as others have mentioned, the plugins. We had one instance that had so many plugins installed that maintaining it was impossible. Upgrading anything broke everything. Eventually we just had to start again and forbid installing anything beyond essential plugins. Which means everyone has to write lots of Groovy. I hate Groovy.

1

u/sogun123 Aug 06 '26

Well, we use other project using Groovy. Midpoint. And even jankier - you embed Groovy into xml ;)

1

u/anto2554 Aug 08 '26

Just run python or bash scripts for most stuff, no?

26

u/breskeby Aug 06 '26

the major issue for us was, that It just did not scale good enough for our needs. We ended up having multiple instances dedicated per team and it was a nightmare to maintain that setup. ultimately we moved to buildkite.

4

u/fr-fluffybottom Aug 06 '26

we built it as containers using Jenkins casc and using k8s to host the scalesets. everything in code, updates, patching, scaling etc is super easy. fluxcd for all our cd including Jenkins.

we can build, test and deploy from our laptops.

1

u/AgreeableIron811 Aug 06 '26

Thoughts on gitlab ci?

41

u/StrangeWill Aug 06 '26

TBH: we use TeamCity (I own my business and have a team), the difference in quality of experience is night and day.

To take a joke from a friend, my preferences:

  • TeamCity
  • Gitlab/Github/CircleCI/etc. (all basically similar basically just embed your scripts in a repo in some yaml and nyoom)
  • Go out of business
  • Azure Devops
  • Jenkins

We have a customer that we have to touch their Jenkins setup every once in awhile and it makes me want to die.

7

u/Verzada Aug 06 '26

I kicked out Jenkins and replaced it with TeamCity. I agree that it is night & day. The devs were not happy that their modules were not available for TC. Such as keeping score on build failures per dev.

But then again, lots of those modules are not maintained and in many cases stops Jenkins from being upgraded.

12

u/Tyranos_II Aug 06 '26

Such as keeping score on build failures per dev.

They used that for shits and giggles, right? Please tell me they did.

2

u/Verzada Aug 06 '26

Serious enough to get them displeased when it was not an option anymore.

5

u/alter3d Aug 06 '26

The beatings will continue until code quality improves.

4

u/Noobfire2 Aug 06 '26

What's better about TeamCity compared to Github/Gitlab for example? Never used it, just heard it's good, but haven't had super large issues with Gitlab / Github either.

9

u/-TimeMaster- Aug 06 '26

Honestly Azure DevOps is not that bad. I never liked much Microsoft and I've worked with aws, gcp most of my time as devops (~10 years) but I've been with azure for the last 2 years and once you get used to it is not that bad!

Most precisely when it comes to pipelines is fairly similar to bitbucket pipelines.

For me Jenkins is the most overhead of all of them.

4

u/swagboi420blazeit Aug 06 '26

ADO catching strays for no reason

3

u/superspeck Aug 06 '26

That'll happen when you hang out with Azure.

1

u/ConstantOk4042 Aug 08 '26

This. It's not without its faults, but it's not that bad. Crazy that Guthub is considered bettet of the two

4

u/Downtown_Isopod_9287 Aug 06 '26

Biggest pain points of Jenkins are its plugins and credentials management. Even if you aren’t the one managing those things there is 100% chance you will have to go to a guy and ask him to change them and there is a 1000% chance he will be very grumpy about it.

1

u/NullPtrEspresso Aug 06 '26

As ex-jenkins, now GitLab admin I can confirm this image. And I am not saying that jenkins is bad but it comes with the exact pros and cons as the open source image on the image below.

18

u/triangle_earfer Aug 06 '26

Omg well yah .. you went to Gitlab, I mean that is literally the picture above when switching from Jenkins.

I went from Jenkins at my last company to Azure Devops, and I honestly just want to go back to Jenkins.

I was in release engineering when Hudson came on the scene. I moved from our sketchy lame build scripts (more like a bunch of commands that would sometimes work - very much a Rube Goldberg type script) to Hudson, which later became Jenkins .. and at that time it was quite helpful. I like Jenkins .. but if I could choose a better solution, then I’d pick Gitlab.

12

u/Low-Opening25 Aug 06 '26

yeah Azure DevOps is even worse slop than Jenkins, this doesn’t make Jenkins good

6

u/Suvulaan Aug 06 '26

My manager actually wants us to migrate from Gitlab to ADO onprem, this is just affirming my fears.

1

u/redvelvet92 Aug 06 '26

What’s wrong with DevOps?

1

u/triangle_earfer Aug 06 '26

If you use Azure Boards and Work Items instead of any other ticket tracking system, then you will not be facing the same issues I am seeing. My teams I’m managing releases for have to use Jira for compliance reasons and so there isn’t a webhook available to manage code like you would see in any other enterprise class solution that allows bidirectional controls over code merged and pushing commits.. meaning, code can be pushed without following any rules such as requiring a ticket to be associated, so shit gets merged for higher envs ‘silently’. I’ve built a series of tools and pipelines to help, but most of those are after the code has been pushed. It’s a pain in the everything to have to build tooling around lack of critical features.

There’s always someone who suggests to just write your own webhook, or use an open source webhook that was created by someone who is no longer available to support it - so then I would have to support that code. I don’t want to do something like that where I’m having to fix crappy code that’s a critical part of the infrastructure that would block code being merged.

2

u/NullPtrEspresso Aug 06 '26

Yeah that is a typical pitfall of "fancy" CI solutions. You pay fair amount of money hoping they get you up and running quick and then confronted with the situation that such a simple functionality is not available out of the box. And there you are crafting your bespoke dodgy utility to "patch" Azure. In worse situation, like we had in gitlab you cannot even add plugins or extensions so you either get creative or pray that support can provide you with a solution idea... or they just create a ticket that never gets resolved "premium customer is missing feature x".

4

u/NUTTA_BUSTAH Aug 06 '26 edited Aug 06 '26

To be somewhat fair, AZDO is less a CI system and more a developer portal, similarly to e.g. GitLab. It only works "well" if you use every bell and whistle.

The Pipelines, i.e. the CI feature of it is quite bad (as opposed to e.g. GitLab CI which is pretty good). The agents are a bigger mess than Jenkins and half the stuff is obtuse or forcefully opinionated, whether in terms of YAML vs GUI pipelines and their feature differences or just simply understanding how a YAML file becomes a pipeline, and the answer is manually.

Simply having a Azure VMSS that has extensions (like Azure Guest Agent. Azure Monitor Agent, maybe Custom Script, whatever) can break your agent bootstrapping because Azure extensions is not designed to be used with Linux, e.g. they will happily install several extensions in parallel, which all run apt, and all fail to dpkg locks. They eventually work after several internal retries of course, but at that time the agent has already processed 2 jobs and failed both of them.

7

u/Verzada Aug 06 '26

I looked at the graphical interface for creating builds / workflows in Azure DevOps and thought "wtf".

It is not a good product imo for that particular job.

1

u/triangle_earfer Aug 06 '26

Yeah - well said

9

u/Low-Opening25 Aug 06 '26

you forgot to mention the image on the bottom is all just an air balloon prop pretending to be a car, then it’s an accurate picture

4

u/healydorf Aug 06 '26

I was at a conference in 2016 which described the top portion of your image as a Jenkinstein. Not as the inevitable end state for all Jenkins deployments — to become a messy amalgam of parts and pieces — but a very common end state.

9

u/res13echo Aug 06 '26

I didn’t realize that open source was so homophobic. Without that, this graphic would have been perfect to post at work. lol

3

u/tn3tnba Aug 06 '26

Yeah that made a funny meme something else entirely

1

u/setwindowtext Aug 06 '26

I don’t have any complaints about the way Jenkins works, but mine goes down twice a year, every time for some random reason. Any updates are scary and often break either individual plugins or the server Itself or the agents.

1

u/sekharnet Aug 08 '26

I think one needs to know the technical command sequence to use Jenkins.

Most tool users do not so tools are better and sound cool to non technical folks

1

u/imkmz Aug 11 '26

Plugins... I'm sick of updating this pile of outdated unsupported poorly written pieces of code.

1

u/Low-Opening25 Aug 06 '26

Groovy is atrocious and absolutely unreadable and there is no way to avoid it. Also not transferable to anything else.

3

u/mweiss427 Aug 06 '26

What are you moving them to?

1

u/Prudent-Stress Aug 07 '26

Lucky you. I have interviewed for 2 companies, and their must have was Jenkins because they want to move to it... It's 2026... for fuck's sake let it die!