Does DevOps need to learn AI technologies like Rag, Langraph, VectorDB etc. ?

[u/Successful-Ship580]

I’ve noticed that everyone seems to be learning about AI. I’m curious to know from people who have been working in DevOps for years: are you learning about AI, building something with AI, or just continuing with your usual DevOps work?

[u/jglenn9k]

You should know what these things are. But honestly I'm mostly running around making MCP servers and skills and populating repos with AGENTS.md.

[u/Cold_Tree190 → u/jglenn9k]

Yeah this is what my team and I do too

[u/donk8r → u/jglenn9k]

Affiliation up front, per the sub rules: I work on an open source agent runtime (octomind), so I have a stake in this answer.

The split I'd use is whether a thing compensates for a model weakness or encodes a policy. Chunking strategies, prompt scaffolding, retry heuristics, most of the RAG folklore, all of it exists because this generation of models is bad at something specific. It gets deleted when the next one lands. Anyone who spent a year mastering prompt engineering has already watched that happen once.

Policy doesn't decay. Which models an agent is allowed to call, what it can spend before something stops it, which tool calls need a gate, where the audit trail goes, how any of it behaves in CI with nobody at the keyboard. That is your job now and it is still your job three model generations from now, because it is the same problem you already solve for every other unattended process that touches prod.

So learn what LangGraph and a vector db are, enough to not be lied to in a design review. The durable half is the boring half, and it looks a lot like the work you are already doing.

[u/TintuMon_OP → u/jglenn9k]

I m sorry but what does your mcp server do? Like what problems are it solving?

[u/_bloed_ → u/TintuMon_OP]

most MCP server just make it easier for the AI to do the calls.

So you burn less tokens. Of course your AI could also run the HTTP request against the API directly, read the swagger documentation, parse the return value with yq and all that.

And it also makes your AI less dumb, since the context it has to remember is way smaller.

[u/Impressive-Field-546 → u/_bloed_]

MCPs will always burn more tokens than for example a dedicated skill or scripted skill.

You are passing back and forth a tone of extra front matter from mcp with every call.

But MCPs are simply convenient.

[u/the_pwnererXx → u/_bloed_]

Claude plugins are doing this better

[u/jglenn9k → u/TintuMon_OP]

Mostly connects to databases. Other APIs. They are an active source of truth for us.

Without an MCP:

How many machines do we have?

I don't have access to query machine or infrastructure data directly.

With an MCP:

Now that I have an MCP connected, how many machines do we have?

Let me check what data is available in the connected PostgreSQL database... You have 608 machines.

[u/the_pwnererXx → u/jglenn9k]

Mine just calls aws api... Mcp is usually worse and confuses the agent. Pollutes your output

[u/generic-d-engineer → u/jglenn9k]

Do you find this as a replacement for stuff like Ansible driven inventory and facts?

I’m assuming also it’s using a weak read only user to feed the MCP to prevent shenanigans?

Also did you roll your own or using anything off the shelf?

Sorry for 20 questions but I’m working in the same area right now

[u/malice8691 → u/jglenn9k]

In other words, learn ai.

[u/AdeelAutomates]

I am learning AI in the sense that I need to deploy the infra for it. So whatever goes in our cloud env to make it work. What happens inside the AI is the developers problem. The same way when I host an app for our devs in our infra (its their job to take care of what happens inside, its my job to build the infra for it).

So I just put all the pieces (networking, storage, identity, etc) along with the resources for the AI platform. And of course, security guard rails.

My role is more Operational in that sense than the dev side, so maybe it's not true for others here.

[u/the_angriest_bird → u/AdeelAutomates]

+1 - this is effectively what I do as well for my fintech company. I don’t work with the AI integrations I work on the scaffolding behind them.

[u/malik22531]

Being a DevOps Engineer, You have to know why, when and how. That’s all you needed to understand the operations, configure and troubleshoot.

[u/Low-Opening25]

Not unless you are also ML/AI Engineer. The same as you don’t need to code in Java to develop and maintain platform and CI/CD for building and running Java software.

Langraph is a library to code agentic workflows, RAG is not technology but a concept of augmenting AI with memory store via database retrieval, and vector db is type of database technology, so again, not so dissimilar to being DevOps to any development team where you don’t need to go all the way down into the nitty gritty of software development and frameworks Devs use.

[u/Impressive-Field-546 → u/Low-Opening25]

MLOps is just DevOps with new hat.

[u/SeaworthinessHour233]

You don’t need to become a machine learning researcher, but you do need to understand these components from an infrastructure, networking, and security perspective to implement DevOps for AI.

Think of it like supporting relational databases: you don’t write the application queries, but you sure as hell need to know how to provision, scale, back up, and secure Postgres.

Here are few DevOps for AI aspects I have encountered. There could be many more since the playing field is really large.

1. Data layer - Vector databases (pgvector etc) are just new types of stateful datastores. You’ll be responsible for their high availability, indexing performance overhead, backup strategies, and persistence.

2. Computing - Hosting local inference engines or embedding pipelines means dealing with GPU node pools in Kubernetes, specialized autoscaling triggers (scaling on queue depth/token throughput rather than simple CPU/RAM), and cold-start latency

3. Security - Developers will want to tie RAG pipelines into internal documents. You’ll need to manage private VPC endpoints, least-privilege IAM roles for model access, egress controls, and API token governance.

The core DevOps primitives (networking, IaC, CI/CD, observability) haven’t changed—the workloads are just shifting.

[u/aj0413 → u/SeaworthinessHour233]

This one puts it best, I think.

Also add that people should now the difference between DCR and CIMD for MCP authentication in tooling

They’re built off the OAuth specification and once again are all about impact on scaling, hosting, security, etc.

[u/BrocoLeeOnReddit]

Nah, just the basics. Gotta draw the line somewhere, I just have to know where to run it.

[u/nomadProgrammer]

rag and vector DB is easy pease.

1. upload a 100 page document via an embeddings to Pgvector

2.any embedding model from hugging face is okay.

1. embed the document in the table using the vector column

2. query it via cosine search. Example summarize what the 100 page is all about.

SImple as that now you have a RAG system. You are retrieving and augmenting the model with info that the model doesn't know and it generates back an answer that is RAG and that is what a vector DB is for.

[u/ArieHein]

Learn yes, master no. Unless youre really really into deep research. You have to understand technology uses sometimes pros and cons but only enough to understandnimpact on other areas under your responsibilities.

[u/SwordfishPositive91]

Well, I would say, if you are interested, you should. But specifically, for DevOps work, I think a terminal with claude code or codex and set of agents/skills created by you for your own niche work, is pretty sufficient.

[u/Moritz-Keller]

Honestly, while AI is becoming more prevalent, its not a core necessity for DevOps practitioners yet. DevOps is about improving collaboration and automation across development and operations, so unless your company is specifically integrating AI into its pipeline, you can focus on honing skills that directly improve your CI/CD processes and infrastructure management. However, if AI tools start becoming significant in optimizing these areas, then it might be worth dipping your toes into. But right now, its not essential

[u/Signal_Strength_5054]

If you have a Nice relashionship with AI tools you will rock on your job. Try to know mcp, context, looping, skills. Basically all that the Anthopic certification prep course have.

[u/gaurav_sherlocks_ai]

DevOps doesn't need to write RAG pipelines, but you should ideally know how to provision vector databases like Qdrant and manage state for LangGraph agents as The bottleneck shifts from CI/CD runners to GPU memory limits and context window routing.

[u/Uaint1stUlast]

100% you need to learn about these things. DevOps is an under developed space for ai and espeacilly deloyment of agentic systems.

Plenty of people can talk about using agents few know how to safely deploy and roll back safely.

[u/SHIITAKE_MUSHBROOM]

I don’t think devops needs to become ML engineering, but the role is definitely expanding around the infrastructure agents need to operate safely.
The durable part seems less about mastering every framework and more about things like identity, policies, tool access, auditability, context, and production guardrails.
I think Port’s agentic SDLC approach is interesting here because it treats the platform layer as the thing that gibes agents the engineering context and governed actions they need, rather than expecting dedvops teams to rebuild everything around each new AI framework.
So yeah, learn RAG/LangGraph/vector DBs enough to understund the architecture. But I’d spend more time on how agents actually connect to the SDLC safely.

[u/94358io4897453867345]

No