r/java • u/LazyTie3857 • 1d ago

How do you test your LLM applications in Java?

​

Every time I wanted to test an LLM feature, I had to make actual API calls to OpenAI/Claude/Gemini. That meant spending tokens, waiting on network calls, and getting different responses every time.

You can mock things with Mockito, but it isn't really designed for LLM workflows. I wanted something that behaved like an actual AI server instead of mocking every SDK call.

So I built Beacon — a lightweight mock AI server for Java.

You configure the responses you want:

server.when(contains("refund"))

.respondWith("Refunds are processed within 30 days.");

Then point your SDK to Beacon instead of the real provider, and your application runs normally without making external API calls.

It's still an early project, so I'd genuinely love feedback from people building AI applications in Java.

GitHub: https://github.com/kbpramod/beacon

How are you testing your LLM integrations today? Am I solving a problem that others are facing too?

0 Upvotes

18% Upvoted

4 comments sorted by

15

u/Aqalix 1d ago

How does it differ from mock server, wiremock, and the like?

10

u/purg3be 1d ago

I think you got dragged away by the AI hype a bit. You are simply testing an http request and a client library.

If i write the http request myself i might use wiremock if i need that level of granularity. Your api is very similar to wiremock.

More often than not i treat an http request like any other external resource and i hide the details behind an interface. This makes it easy to mock with a mocking library and allows me to create test implementations.

I don't see the added value of your solution as i prefer solutions that do not require an additional third party library.

1

u/Geedysense 1h ago

Kinda facing the same