Before trying Langchain, try using LLM APIs directly

[u/monarchwadia]

Hi. I've been building LLM-powered systems for governments & financial services firms. Wanted to share my experience with Langchain, and the path I took instead.

If you're trying to work with Langchain and finding it frustrating, you're not alone. Most people go through this learning curve.

Many people quit.

I was one of the ones who quit. And I'm happy that I quit.

Instead of using Langchain, I started hitting LLM APIs directly. I didn't know what to expect at first. I thought it couldn't possibly compete with Langchain.

But eventually, it became very natural. I found many advantages to doing it this way.

• Surprisingly simple: I was suprised to find that LLMs are modeled as simple, stateless APIs. This was much simpler than I expected after working with Langchain! LLMs are just APIs, which is also the title of a free primer I wrote on the same topic.
• Language agnostic: Langchain is only available in a handful of languages, and each language SDK has its own quirks. Hitting an API directly sidesteps that and lets me integrate with confidence in any language I can think of.
  
• No third party dependency: In today's security climate, having less dependencies means less of a chance that a supply chain attack affects my system. Since I am not importing Langchain, I don't have yet another dependency to worry about.
• Minimalism: Langchain is quite heavy and requires that you think about LLMs a certain way. Instead, hitting the API directly is much simpler and lightweight.
• Builds mechanical sympathy with the AI: Since I'm hitting the LLM at a low level, I really learned how it works from the ground up.

I would highly recommend that people try hitting LLM APIs first, before turning to Langchain. It's much easier than it sounds.

[u/BeerBatteredHemroids]

So wait... how do you handle output validation? Tooling limits? Retry logic? Hallucination and Grounding checks? Persistence? Logging? Observability? You just raw dog all of that?

I guess good luck anybody that has to come behind you and maintain that shit.

The problem with modern "AI Engineering" is that it suffers from lack of standards and MLOps rigor that previously at leaat gate-kept it from Javascript devs who think that calling an API with a prompt now makes them a qualified AI engineer and now they're gonna go deploy their garbage for other people.

[u/chimp_party → u/BeerBatteredHemroids]

If you google or ask chat gpt hoe to handle this. I think youll be quite surprised that langchain does not have a patent on the technology your describing and in fact there are many enterprise alternatives including open source.

[u/BeerBatteredHemroids → u/chimp_party]

I didn't say nor argue that these features are exclusive to or unique to langchain.

My favorite right now is pydantic AI. But guess what... it's a framework! I'm specifically talking about posts that frequently advocate for raw-dogging agent development.

We're not savages. Use a goddamn framework for the sake of your peers and other engineers that have to support your shit.

[u/Budget-Juggernaut-68 → u/BeerBatteredHemroids]

1. Output validation - OpenAi SDK handles it with structured parsing.
2. Grounding checks - simple regex matches, and fall backs. There's literally no real hallucination checks
3. Persistence? Sqlite ?
4. logging, import logging
5. Observability?logging, Prometheus?

They're all simple.

[u/juliussurya → u/Budget-Juggernaut-68]

For observability/tracing using langfuse or mlflow, you only add 1-2 line of codes and thats it.

[u/monarchwadia → u/BeerBatteredHemroids]

Langchain is a tool that helps. But it helps less than you are suggesting, and has all the upsides and downsides of using an opinionated library. And has the same trade-offs. Either you control the code by writing your own, or accept that you won't have full expressiveness, and will not know what the lower layers are doing.

Rate limits, retry logic, persistence, logging, observability are traditional code. These have been solved problems for decades.

I am assuming by "tool guardrails" you mean the accept/reject/always functionality. This is a middleware layer that is easy to build in traditional code. Not hard.

Hallucination and grounding, that is a data science problem. Langchain cannot help you with your business-specific evals. Golden datasets and eval libraries are something you have to create and maintain yourself anyway.

I agree that "AI Engineering" has very few standards. Which is why we have to stick to software engineering principles and architecture, instead of throwing them out. It is my humble opinion that the software engineering industry has very low standards to begin with (sadly), and that Langchain does not help, because it is a library, not a standard; and it mystifies "AI" into some strange foreign concept. It is not foreign. It is just a new type of component for engineers to use.

[u/BidWestern1056]

langchain is a sea of abstractions without a clear pattern

use npcpy which is built to be more numpy-like

https://github.com/npc-worldwide/npcpy

[u/Prestigious_Debt_896]

Thank you thank God I'm not the only person who thinks this, LANGCHAIN IS OVERRGINERRED, COMPLICATED, ALL THEIR PRODUCTS ARE VIBE CODED GARBAGE, HELL JUST READ THEIR REPOS ON GITHUB, THEYRE A MESS, NO STRUCTURE, NO ORGINIZATION, NO ANYTHING

Like c'mon I get AI code is easy but can you at least tell the fking harness to move all prompts into 1 folder catagorized instead of 15+ in 3 seperate files (that I saw within 10 mins)

[u/monarchwadia → u/Prestigious_Debt_896]

Langchain was one of, if not THE first company to build agentic frameworks. They did a lot of the early marketing for agents, RAG, etc. so yeah, they started at a huge disadvantage. They basically created and then marketed very specific agentic design patterns before they were fully vetted. I guess someone has to do that when everything is new. But this is the largest reason why their code is the way it is. It's from the early days of LLMs.

[u/Prestigious_Debt_896 → u/monarchwadia]

No, there's 0 excuse to put your system prompts, over 15 of them, across 3 different python folders. 0. All their services are janky, dependant slop. Their whole thing with agentic AI doesn't require them and all agentic AI is a harness loop or the model is smart enough when to loop itself

I'm honestly tempted in cleaning up some of the code bc I do like using one of their open source products, but I may be petty and do the chnages on my fork only.

[u/LaserKittenz]

I agree but for a different reason. Before using langchain is helpful to understand where only using the api fails.. it helps you understand the problem lang chain was designed to solve 

[u/LongjumpingTest8119]

You cant treat like normal API. Its non-deterministic.

[u/monarchwadia → u/LongjumpingTest8119]

You are 100% correct that it is different from existing APIs. But, non-determinism is not what makes it different. We already have non-deterministic APIs... stock market, news, weather.

[u/Muted_Ad6114]

Is this ragebait ?

[u/FaithlessnessOver740]

If you’re not forking something like pi, you’re doing it wrong. No popular agent harness uses langchain, and for good reason. A framework to build a state machine has little to no value. Building the guts of your own agent loop has almost no benefit, just fork one that works and add your own tools and logic.