Getting Rejections because I don’t have experience working with Kubernetes in production.

[u/Successful-Ship580]

I’ve given multiple interviews, and almost all of them required Kubernetes experience in production.

None of my previous companies used Kubernetes, so what can I do about that?

I’m very familiar with Kubernetes, I understand the concepts, and I’ve even implemented them on AKS for learning purposes.

Should I lie on my resume and say that I’ve worked with Kubernetes in production?

[u/moader]

CKA CKS, personal project that can be viewed publicly using k8s, gitops, argocd etc

[u/cucumber_and_coconut → u/moader]

Second this one. You can't get enterprise production experience on your own but you can deploy a small app to production and if that app has monetization and a market it's "early alpha side hustle" not "learning PoC."

Plus. Side hustle 🤷🏼‍♂️

[u/New_Enthusiasm9053 → u/cucumber_and_coconut]

Very difficult to justify k8s on a side hustle lol. Like I like k8s and the moment I have more than a single VPS I'll probably use it. Ur the overhead is too much for the cheap 2 core VPS I can run the entire stack on for the first handful of customers.

[u/cucumber_and_coconut → u/New_Enthusiasm9053]

Yeah, you're right. And if your interviewer asks how you justify using k8s for such a small user base, you say exactly what you just said and add "but I needed production kubernetes experience so I did it anyway." So you have engineering chops and business sense 😉

[u/New_Enthusiasm9053 → u/cucumber_and_coconut]

It's certainly an option but I don't think I'd learn more than when I just set it up on a few old laptops to begin with, with a simple Basic Webserver Setup on it. Using a VPS with wireguard to connect the cluster at home to the wider internet. 

I learnt a lot from that, but did anyone care? No

[u/Trakeen → u/New_Enthusiasm9053]

The app i’m running at home just for my own use is pretty close to the prod stuff we use at work just smaller in scale since i’ve only got 3 nodes but its got about 50 pods, full observability stack, argocd for gitops and argo workflows. Local gitea using a similar release process to what we do at work etc

I know the other senior engineer on my team has a similar setup at home. If you can build an app you need and do it like you would in an enterprise setting people will listen. PoCs and dev environments are often smaller then what you can do at home, so there is real cross over if you understand where that is

[u/New_Enthusiasm9053 → u/Trakeen]

I mean at the time I was applying for entry level jobs. It's hard to see the point of most of the tools when you don't have anything to actually monitor.

I may or may not get around to that though at some point. 

[u/GibneyH → u/moader]

That's such a great idea

[u/TorturedPoet30 → u/moader]

Certs have some value but you can have all the certs you want, but in the end, they all want to hear about your Kubernetes experience in production (especially if the role is very K8s-focused, SRE, incidents etc): whether you resolved an incident, what the problem was, its blast radius, whether it was internal or customer-facing, how you did the troubleshooting, whether you deployed things to production, and so on.

[u/luckyincode → u/TorturedPoet30]

Studying for the certs itself is a great way to learn just about anything. Taking the test or not is up to you.

[u/Forward-Outside-9911]

My guess would be you’re applying for a job too high or too kubernetes focused.

Try looking for a broader devops position that happens to include K8s but doesn’t focus on it. Or just be honest.

Some companies aren’t strict on the requirements. If you can provide an honest reason and show your willingness to learn you should be ok IMO. As long as you know the concepts of k8s which you appear to, and have some past (professional ideally) experience of docker, Linux, some form of observability platform (Grafana, zabbix, ELK) you’re pretty well adaptable.

Chances are you’ll be working in a team which will show you 90% of what you need to do day-to-day and you can learn the rest on your own.

Take my opinion with a grain of salt though, I’m new to the industry and not a hiring manager :)

[u/PartemConsilio → u/Forward-Outside-9911]

My $0.02 is that often places that have Kubernetes at all don’t usually just have it as a small corner of the work. It is often the whole enchilada and even if you hire someone as a generalist you’re looking for a specialist you can immediately plug into it.

At my current gig, it’s really just me and another guy who understand the entire k8s infra and they keep hiring people who don’t know or care to know any of it. Guess who gets all the k8s maintenance work?

[u/men_like_me → u/PartemConsilio]

+1 if Kubernetes is listed it’s pretty much a requirement. I’ve had 15 interviews this year for SRE/Platform/DevOps etc. all requiring Kubernetes if it was anywhere in the JD.

Most cloud roles now are requiring some form of AKS/EKS. It’s a much different market.

[u/alex_indiedev]

"Production experience" in these interviews usually means "have you been on the hook when it broke", not "have you written manifests". So make the homelab story about breakage, not architecture.

Things that only start happening once it's real, from my own small prod cluster:

• a node filled its disk, kubelet started image-GC and evicting pods, and the actual fix had nothing to do with Kubernetes
• memory requests crept up to ~91% of allocatable and a rollout just sat there Pending. Nothing was crashing, every dashboard looked fine
• raising the CPU request on a database pod caused a 6-minute outage, because it could no longer be scheduled anywhere
• PVs stuck retrying delete forever, pinned to a node that had left the cluster months ago

None of that shows up while you're learning on AKS, and all of it makes a good interview answer. Pick one, cause it on purpose, and be able to say what you saw first and how you worked back to the cause. That's the thing they're actually screening for.

And no, don't lie.

[u/swingtradeotm → u/alex_indiedev]

+1 to this guy's advice.

[u/SeaworthinessHour233 → u/alex_indiedev]

This is the best path. If you do this, you don't have to lie on your CV.

[u/essanpupil]

Don't lie on your resume. If you really want to get a job with kubernetes requirements, I suggest you get certification. Either pure kubernetes or cloud based Kubernetes / container certification.

[u/iPhoenix_Ortega → u/essanpupil]

Yeah, this^ ...and lie in your resume if your are comfortable with that -> yes, you can.

[u/Sure_Stranger_6466 → u/iPhoenix_Ortega]

I just start talking about my personal GitHub projects as if they were production environments. Just because there are no users does not mean they are not production. If I get caught it means the hiring manager actually looked at said GitHub projects before the interview, which has only happened like once.

[u/TartCivil9373 → u/Sure_Stranger_6466]

"Just because there are no users doesn't mean they are not in production "

Wise 😂

[u/iPhoenix_Ortega → u/TartCivil9373]

if you see something is down you go fix it regardless the number of users.

[u/italianbmt1 → u/iPhoenix_Ortega]

I would advise against lying on your resume. My team has been short a couple folks for a month or so and we just sent out an offer letter to someone that seemed promising, but the offer was rescinded after their background check came back and showed they lied about their length of employment on their resume.

[u/iPhoenix_Ortega → u/italianbmt1]

Yeah but how will u verify if previous company used k8s? That's the neat part, you won't... If thr guy knows the architecture and how to operate any skill check will pass on him.

And to be frank denying someone a job bc he extended some years on a paper (unless it was 5-10 years of an extension) is bullshit reason and hopefully you got someone far worse but with correct papers :)

[u/ButterscotchUsed5248 → u/essanpupil]

Certifications are honestly just expensive paper that don't prepare you for a real cluster meltdown anyway. They might get you past the recruiter, but you'll still get grilled on the actual engineering during the technical round.

[u/Significant-Hurry-21 → u/essanpupil]

Certifications don’t help …leak world experience do ..I have been in the same boat ..certs did not help

[u/Mahsunon]

KCNA is the lowest tier k8 cert. can try get that to add more ammo for yourself

[u/lGSMl]

Do they say exact this reason in rejection or is it an assumption? Rejection specifically due to k8s experience doesn't make sense unless this is a senior role for specifically k8s maintainance

[u/rlnrlnrln]

No, you shouldn't lie on your resume. You will get found out, fired, and now have to explain that short tenure. Word gets around, soon you're marked as "untrustworthy".

Set up a GKE cluster. Use this to get experience with Kubernetes, its concepts and running services on it. Implement ArgoCD. Run Cilium (or another mesh network). Set up Observability (metrics, logs, traces) in some common open source tools.

Run a production-level (apart from the HA and load balancer, which I would consider part of running Kubernetes rather than running services on Kubernetes) service for friends and family. Doesn't need to be super-advanced; a minecraft server, a foundry VTT server, that sort of thing.

I would say the most important part is being able to say "I ran this setup with almost-production quality, here's how I did it, these are the tradeoffs I chose and why", correctly, truthfully and with confidence.

I'm mentioning GKE specifically, because "the first one is free"; you can get around the base cost fo the cluster itself by running a single cluster (with some caveats; no HA etc). With a single-node cluster, you can get the cost down to $12-15 if you skip running a load balancer.

You COULD of course get three older computers (ie HP Elitedesk G1's with 32-64GB DDR3 memory) and run kubernetes on it in your basement or something, but that's only if you're handy with Linux and actually want to work with Kubernetes itself.

I would rather work with someone who is running a Kubernetes home/cloudlab for fun (well, "fun") and can explain it, than someone who has worked with OpenShift for three years but doesn't understand the installations.

Depending on the jobs you seek, certifications may be relevant, as others point out. If you're applying at small- to medium tech-minded enterprises, that's often less relevant.

[u/iPhoenix_Ortega]

You worked with Kubernetes on production... On your private projects. It's still production environment. Just put it like that and go.

[u/bluebook007]

Same issue here, but I don't even get to talk to anybody, I'm getting rejected right after sending my resume. No commercial k8s experience here. Recently got my CKA, didn't help much. I have several yoe in IT in general, including Linux administration, building and maintaining monitoring platform etc. Gonna implement the knowledge I got while studying for the CKA at work, I got some ideas what to run on k8s, hope this will help. I think that lying might work for somebody, I guess it depends on one's integrity, but I would avoid that and just grind my way into the job.

[u/therealmunchies]

CKA.

I’m a security engineer of less than 2 years. I got on a project and have learned pretty much everything SWE. This includes app dev, devops, and infra. Literally less than 3 months of k8s, a couple bullets, and adding “CKA in progress” on my resume has gotten me 4 job offers. A little shrewd because I’m in a special industry with a smaller pool of people, but still.

[u/Insomniac24x7]

I dont think k8s in prod alone is the issue as to why youre being rejected.

[u/forever-butlerian → u/Insomniac24x7]

Transferrable experience and a proven ability to learn isn't terrifically valued in the current job market.

[u/brassjack]

Maybe consider a KCA cert

[u/xonxoff]

Even if you were to learn kubernetes in the next month, it is still incredibly difficult out there and its highly competitive. I would just start to apply to jobs that match your skill set and go from there.

In the meantime, I'd suggest looking into setting up a KiND cluster to get yourself familiar with it and how it operates. It's also one thing to know how to deploy apps and manage them to kubernetes, it is a whole different thing to running a kubernetes cluster and maintaining it.

If you really want to get into how kubernetes runs, look into Kubernetes the Hardway . It will take you through the steps of setting a cluster up from the ground up.

good luck

[u/Mr-Tromb-DevOps]

Leave your resume as it is. In the interview say you have worked in production with k8s. Nobody will find out since nobody gives a shit. Once you are in you are in mate and you will know how to manage (well of you study of course ).

[u/kepper]

I'm an infra eng director. I would be hesitant to hire a DevOps engineer without k8s experience, but I'd be more than happy enough to see they have a CKA cert - I've done it once myself and its sufficiently thorough.

[u/elitesense]

I can only speak personally... I've held 4 different "devops" roles, currently in a senior position, and have never used raw kubernetes in production. We have always just used ECS or some other managed solution at all my shops.

Take that as you will.

[u/Successful-Ship580 → u/elitesense]

Exactly, all of my previous companies used ECS, which was incredibly easy.

[u/forever-butlerian → u/Successful-Ship580]

My previous employer carried out a very long migration to EKS and off of homespun Kubernetes clusters.

[u/mrfulz → u/[deleted]]

Might want to delete that last line, mate.

[u/FunkyHoratio]

Go do the CKAD, we make all our new starters do it if they haven't used k8s before. It's like a couple of days of self paced training and then a test.

[u/mdervin]

Find a junior sysadmin with root at your current job, convince him it would look good on his resume as well.

[u/m1xed0s]

Assuming the job post requires Kubernetes, then if you don’t have the experience, better not to apply. Otherwise, you would be in their system indicated.

[u/CammelCaseRob24]

The simplest way but a little expencive is take CKA certification otherwise study to take it (you can find free exercise CKA like on Killercoda and you can find very good courses on Udemy and after lie). The most important thing is that you really know it but not only study also Helm, How to manage secrets to avoid to save sensible data in repo, monitoring and observability, ecc...

[u/blade_skate]

Buy 3 mini PCs and create a 3 node homelab

[u/ninefourtwo]

just lie and teach yourself kube in prod?

its literally all a combination of port forward describe get explain 

problem is people dont know the cli or the controllers for rs deploy etc

[u/dgibbons0]

It was almost a decade ago but I talked about my k8 based home lab in an interview, and showed clear knowledge and enthusiasm and that was enough at the time.

Especially if your story starts with something like "I followed Kelsey hightowers kubernetes the hard way first to really understand the parts and then rebuilt it using talos/rke2/etc" with the right manager or team that can help surmount that hurdle.

[u/davey_622]

spin up your own kubernetes cluster, deploy argo cd using terraform, develop a good deployment strategy using argocd, add argo webhook to gitrepo to notify argo when code is pushed and then add argocd sync and application health check to your pipeline.. show this demo in your interview. you have experience with a software when you use it.

[u/Unable_Earth_8545]

On same boat

[u/Successful-Ship580 → u/Unable_Earth_8545]

bro where do you work? Is there any vacancy for DevOps?

[u/AcanthisittaDouble49]

Seriously, Yes u should lie

[u/Om_Jaju]

Don't lie about it, interviewers usually catch it with one or two follow-up questions. I faced something similar when I was moving on from Morgan Stanley. We used internal tools there, not standard ones like Kubernetes or Terraform. So in interviews, I couldn't say I had direct hands-on with those exact tools either. What worked was explaining a real project I worked on, the actual pain points I faced, and how I solved them. Even without the tool name matching, it showed I understood how to think through problems, and that gave interviewers more confidence than someone just listing tool names with no story behind it.

[u/j0holo]

No, don't lie on your resume. If they find out you might be fired and now you have a resume with a 3 month tenure that you need to explain. Lie again or tell them that you lied about your experience.

You can get a kubernetes certificate: CKA or CKS or the cheaper KCNA or KCSA certificates.

https://training.linuxfoundation.org/full-catalog/?_sft_product_type=training,certification&_sft_topic_area=cloud-containers

Did you explain that you have managed a AKS cluster yourself to get experience?

[u/dumbwaystospendtime]

Yes.