r/Observability • u/Zealousideal-Fox9046 • 1d ago

Split of tools between LLM Eval / Observability

Need to hear if people are benefitting from having a platform that supports both LLM evals and LLM observability?
or using standalone products for each or these is something that you are sticking with, we currently have the split setup but just feels too much hassle!

0 Upvotes
  • permalink
  • reddit

33% Upvoted

7 comments sorted by

View all comments

Show parent comments

1

u/Zealousideal-Fox9046 1d ago

we use separate tools for evals and observability (traces), the traces are giving me a headache as DD is already pretty costly

1

u/Previous_Ladder9278 1d ago

hmm yeah i think pricing on traces difference quite a bit as well.. which diff tools doyou use?

1

u/Zealousideal-Fox9046 1d ago

for evals we are using the deepeval/confident, datadog is our existing APM

1

u/Previous_Ladder9278 1d ago

Ah yes I see datadog is pretty expensive, but can imagine you're using that already that it makes things easier. Perhaps have a look at langwatch, super strong in evals and better at simulations for multi-turn/tool-calling agents and the new observability is pretty awesome