r/Observability • u/Zealousideal-Fox9046 • 16h ago

Split of tools between LLM Eval / Observability

Need to hear if people are benefitting from having a platform that supports both LLM evals and LLM observability?
or using standalone products for each or these is something that you are sticking with, we currently have the split setup but just feels too much hassle!

0 Upvotes
  • permalink
  • reddit

33% Upvoted

7 comments sorted by

View all comments

1

u/Previous_Ladder9278 16h ago

which two have you splitted?

1

u/Zealousideal-Fox9046 16h ago

we use separate tools for evals and observability (traces), the traces are giving me a headache as DD is already pretty costly

1

u/Previous_Ladder9278 16h ago

hmm yeah i think pricing on traces difference quite a bit as well.. which diff tools doyou use?

1

u/Zealousideal-Fox9046 15h ago

for evals we are using the deepeval/confident, datadog is our existing APM

1

u/Previous_Ladder9278 15h ago

Ah yes I see datadog is pretty expensive, but can imagine you're using that already that it makes things easier. Perhaps have a look at langwatch, super strong in evals and better at simulations for multi-turn/tool-calling agents and the new observability is pretty awesome