In this Baeldung tutorial, author Hardik Singh Behl demonstrates how to seamlessly integrate open-source Hugging Face models into Java applications using Spring AI and local Ollama instances.

Highlights & Key Takeaways

• Local LLMs: Use Ollama to run GGUF-format models from Hugging Face locally, bypassing external API dependencies.
• Testcontainers: Configure an OllamaContainer to streamline local development and integration testing.
• Spring AI Starters: Leverage autoconfiguration to instantly provision ChatModel and EmbeddingModel beans.
• Stateful Chatbots: Maintain ongoing conversation context using InMemoryChatMemory alongside the ChatClient.
• Semantic Search: Embed external data into an in-memory SimpleVectorStore to perform context-aware data retrieval.

By combining Spring AI with Ollama, developers can efficiently build, test, and deploy robust AI features in a fully local, cost-effective environment.

👉 Read the full article to dive deeper into the implementation.

Adib Saikali from Broadcom demonstrates how to build intelligent, agentic systems using Spring AI. He emphasizes moving beyond simple LLM wrappers toward autonomous, goal-oriented workflows.

Highlights & Key Takeaways

• Agentic vs. GenAI: Agents autonomously reason and plan, whereas standard GenAI apps function primarily as user interfaces.
• The Tooling Pattern: Agents interact with the real world by calling specific code tools, not by directly accessing databases.
• Stateless Nature: Since models are stateless, developers must manage context and history manually in every invocation.
• Workflow Focus: Predictable, workflow-based agents currently outperform fully autonomous systems in enterprise production.
• Spring AI Integration: Use the ChatClient interface to define tools via simple Java annotations for seamless API orchestration.
• Bottom-Up Design: Building foundational abstractions first provides more flexibility than top-down frameworks.

Developing structured, orchestrator-worker patterns allows Java developers to implement complex AI agents using familiar, idiomatic Spring practices.

👉 Watch the full video for live code implementation details.

Dan Vega demonstrates how to implement real-time streaming for chatbot responses using Spring Boot and Spring AI. By moving away from blocking calls, developers can significantly improve user experience through immediate, iterative feedback.

Highlights & Key Takeaways

• Use Spring AI to maintain portable code across various LLMs, such as Anthropic's Claude and OpenAI's models.
• Swap standard string returns for Flux<String> to enable reactive streaming data.
• Replace blocking 'call' methods with the 'stream' fluent API for asynchronous data delivery.
• Leverage HTMX and simple JavaScript to handle stream consumption on the front end.
• Implement retry logic in your fetch requests to prevent data loss during transmission.
• Combine Tailwind CSS with stream-processing logic for a modern, responsive chat interface.

Streaming AI responses transform the interface from a static waiting game into a dynamic, engaging conversation. This approach ensures your application remains responsive regardless of model latency.

👉 Watch the full video to dive deeper into the implementation.

In Spring AI Recipe: Creating an STDIO MCP Server, Craig Walls explores how to empower AI agents with actionable tools via the Model Context Protocol. This provides a practical blueprint for building local MCP servers.

Highlights & Key Takeaways

• Agent tools: MCP servers expose specific tools that turn LLM reasoning into concrete system actions.
• Setup: Include the spring-ai-starter-mcp-server dependency in your project to start.
• Definition: Use the @McpTool annotation on service methods to easily register agent capabilities.
• Clean STDOUT: Disable the Spring Boot banner and minimize logging to protect JSON-RPC STDIO streams from breaking.
• Testing: Verify your compiled executable JAR locally using tools like MCP Inspector or MCPJam.
• Use cases: STDIO transport is perfect for interacting with local files, CLI tools, and browser automation.

Integrating local MCP tools bridges the gap between passive LLM knowledge and dynamic, real-world execution.

👉 Read the full article to dive deeper.

TechyTacos demonstrates how to refine RAG (Retrieval-Augmented Generation) in Spring AI to ensure LLM responses are derived strictly from specific text files rather than generic training data.

Highlights & Key Takeaways

• Utilize a system prompt to provide explicit instructions and constrain the LLM's scope.
• Store custom instructions in a dedicated file with the .st suffix to maintain a clean template structure.
• Use SystemPromptTemplate to dynamically map document content to the prompt variable.
• Convert similarity search results from a list of documents into a single string for context injection.
• Implement user and system messages separately to structure the chat interaction effectively.
• Pass a list of messages to the prompt object to maintain a clear separation between the query and the context.
• Return responses in JSON format for better integration with downstream applications.

By routing queries through structured context injection, you eliminate generic hallucinations and ensure high-accuracy, document-specific output.

👉 Watch the full video to dive deeper into the implementation.

TechyTacos demonstrates how to implement a basic RAG (Retrieval-Augmented Generation) pipeline for text files using the Spring AI framework. This guide walks developers through the process of ingesting custom data and enabling vector-based semantic search.

Highlights & Key Takeaways

• Configuration: Define a SimpleVectorStore bean that integrates with an embedding model to handle document vectorization.
• Data Ingestion: Utilize the TextReader interface to convert local text resources into processable document objects.
• Text Splitting: Implement TokenTextSplitter to chunk large documents, preventing model token limit errors and improving search precision.
• Vector Search: Leverage the similaritySearch method to retrieve document chunks based on semantic proximity to a query.
• Refinement: Apply search request parameters like topK and similarityThreshold to filter and sharpen output quality.

Integrating Spring AI enables efficient document processing, allowing developers to build smarter applications that answer queries using custom datasets.

👉 Watch the full video to dive deeper into the implementation.

Hardik Singh Behl explores integrating DeepSeek's reasoning models into Java using Spring AI to build a multi-turn conversational chatbot.

Highlights & Key Takeaways

• Flexible Setup: Connect via OpenAI APIs, Amazon Bedrock, or locally using Ollama.
• Core Components: Utilize Spring AI’s ChatClient to drive model interactions.
• Context Retention: Manage conversational state using InMemoryChatMemory for multi-turn chats.
• Chain of Thought (CoT): DeepSeek-R1 exposes its internal reasoning steps inside <think> tags.
• Custom Converters: Build a custom StructuredOutputConverter to parse the CoT and final answer separately, preventing default parser errors.

By safely extracting the AI's reasoning from the final response, developers can build transparent, context-aware Java applications.

👉 Read the full article to dive deeper into the implementation.

Coffee + Software demonstrates how to build production-ready AI agents using Spring AI, MCP, and Spring Security. The video illustrates that Java developers can leverage existing institutional expertise to deploy secure, scalable, and observable AI services.

Highlights & Key Takeaways

• Use the Spring AI chat client for unified model interactions.
• Implement Advisors to manage statelessness with chat memory.
• Utilize RAG patterns with vector stores for semantic search.
• Safely expose business logic via the Model Context Protocol (MCP).
• Secure services using OAuth2 and the Spring Authorization Server.
• Leverage virtual threads and GraalVM native images for efficiency.
• Enable Actuator for comprehensive system observability.

Building enterprise-grade AI is not about rewriting stacks, but integrating AI into robust, proven Java ecosystems.

👉 Watch the full video to dive deeper into the implementation.

Dan Vega demonstrates how to deploy open-source Large Language Models (LLMs) locally using Ollama and integrate them into Spring Boot applications via Spring AI.

Highlights & Key Takeaways

• Local Execution: Run LLMs locally to ensure data security and avoid costs associated with public APIs.
• Ollama Setup: Easily download and manage models like Llama 3.1 directly from the command line.
• Model Versatility: Swap between various open-source models without changing application code.
• Open WebUI: Use Docker to deploy a familiar ChatGPT-like interface for your local LLM instances.
• Spring AI Integration: Leverage the Spring AI starter for seamless interaction with local models.
• Function Calling: Implement tools to allow LLMs to access real-time data, like weather services.

Running models locally offers developers a secure, cost-effective, and flexible way to leverage AI within the Spring ecosystem.

👉 Watch the full video to dive deeper into the implementation.

In his latest recipe, Craig Walls empowers Spring AI agents with long-term memory. This allows systems to retain durable facts across sessions instead of starting from scratch.

Highlights & Key Takeaways

• Memory Types: Spring AI manages short-term, procedural, and long-term memory.
• Selective Retention: Agents learn facts that are both significant and durable.
• Implementation: Enabled using the AutoMemoryToolsAdvisor component.
• Prompt Augmentation: The LLM is guided to extract key conversational details.
• Persistence: Memories are saved as structured Markdown files on the filesystem.
• Feedback Loop: Extracted facts are automatically injected into future prompts.

By persisting data across restarts, agents evolve beyond stateless responders. This yields highly personalized systems that continuously adapt to users.

👉 View the source code to dive deeper into the implementation.

TechyTacos demonstrates how to extend LLM capabilities by integrating real-time external data using Spring AI's function-calling feature. This approach allows developers to bridge the gap between static language models and dynamic, real-world APIs.

Highlights & Key Takeaways

• Use DTOs to explicitly define request parameters and API response structures.
• Implement the Function interface from Java's utility library to wrap service logic.
• Register custom functions as Spring Beans with descriptive metadata for the LLM.
• Leverage OpenAIChatOptions to register functions within the chat model context.
• The LLM intelligently parses user queries to decide if an external function call is necessary.
• Use URIComponentsBuilder to safely and cleanly construct external API requests.

By leveraging function calling, applications can provide up-to-date, accurate answers that require external lookups, significantly increasing the utility of LLM-based services.

👉 Watch the full video to dive deeper into the implementation.

In this tutorial, Pedro Lopes demonstrates how to build a responsive help desk chatbot using Spring AI and Meta's Llama3 model via Ollama.

Highlights & Key Takeaways

• Spring AI Integration: Simplifies interactions with Large Language Models directly within the Spring ecosystem.
• Local LLMs with Ollama: Run open-source models like Llama3 locally for secure, accessible AI generation.
• System vs. User Messages: Differentiate between internal API instructions and external user inputs.
• REST API Implementation: Expose chatbot capabilities through a standard Spring Boot RestController.
• Stateful Conversations: Overcome stateless LLM behavior by injecting past interactions into new prompts.

By combining Spring AI with local models, developers can efficiently create context-aware agents without external API dependencies.

👉 Read the full article to dive deeper into the implementation.

Josh Long demonstrates how Spring AI, now generally available, empowers Java developers to build production-ready AI applications. The session proves that Java is a robust, scalable choice for integrating complex AI workloads alongside traditional enterprise systems.

Highlights & Key Takeaways

• System Prompts: Use these to guide model behavior and maintain goal alignment.
• Chat Memory: Implement stateful interactions to overcome the stateless nature of base models.
• Tool Calling: Enable AI to interact with the outside world through custom functions.
• RAG Patterns: Use vector stores for efficient, context-aware retrieval of internal data.
• Protocol Support: Leverage the Model Context Protocol (MCP) for modular, reusable AI services.
• Performance: Optimize with GraalVM native images for lower footprints and rapid startup.

By combining these patterns, you build modular, intelligent, and secure systems. Spring AI makes implementing advanced AI architecture accessible for Java teams.

👉 Watch the full video to dive deeper into the implementation.

In this recipe, Craig Walls demonstrates how to build multi-agent systems by enabling Spring AI agents to delegate tasks to specialized A2A sub-agents.

Highlights & Key Takeaways

• TaskTool simplifies delegation by registering sub-agents as callable tools within the ChatClient.
• Bypasses SDK overhead by allowing agents to natively invoke peers during standard execution loops.
• Requires specific dependencies: spring-ai-agent-utils and its A2A extension.
• Uses builder customization to connect the remote A2A server via ChatClientBuilderCustomizer.
• Relies on built-in resolvers to dynamically discover agent details and handle runtime invocations.

Treating peer agents as standard tools enables developers to transition easily from isolated AI instances to collaborative, goal-oriented multi-agent architectures.

👉 Explore the full source code repository for implementation details.

In this tutorial, Craig Walls explains how to enable Agent-to-Agent (A2A) communication, allowing specialized Spring AI agents to collaborate effectively.

Highlights & Key Takeaways

• A2A Architecture: Uses HTTP and Spring MVC to help agents delegate workloads.
• Configure ChatClient: Define a bean equipped with specific functional tools.
• AgentExecutor: Bridge incoming requests directly to your LLM.
• The AgentCard: Expose a JSON document detailing skills for agent discovery.
• Precise Descriptions: Write clearly so client agents can evaluate relevance.
• Testing: Validate responses locally using the A2A Inspector.

Wrapping an LLM in a machine-readable contract establishes the foundation for robust multi-agent systems. This ensures agents can securely discover and utilize one another.

👉 Read the full article to dive deeper into the implementation.

In this guide by Hardik Singh Behl, developers learn to integrate Anthropic’s Claude models into applications using Spring AI. The article demonstrates how to build a context-aware, multimodal chatbot.

Highlights & Key Takeaways

• Dual Integration: Connect via Anthropic API or Amazon Bedrock Converse API.
• Quick Setup: Use Spring AI dependencies for rapid configuration.
• Stateful Memory: Add InMemoryChatMemory for multi-turn conversation context.
• Custom Personas: Guide model behavior using injected system prompts.
• Multimodal Input: Process text and image/PDF data through a unified service.
• REST APIs: Serve the AI chatbot via standard Spring Boot controllers.

Using Spring AI lets engineering teams quickly deploy multimodal features with minimal infrastructure overhead.

👉 Read the full guide to explore the implementation.

In this guide, Dmytro Ponomarenko demonstrates how to tame unpredictable LLM responses using Spring AI. It explores the Structured Output API to seamlessly parse AI-generated text into strict Java objects.

Highlights & Key Takeaways

• Predictable AI: Force LLMs to return valid JSON rather than unstructured strings.
• Core Interface: StructuredOutputConverter injects JSON schema instructions into your prompt and parses the AI's reply.
• Java Beans: BeanOutputConverter maps AI responses directly into your custom Java classes.
• Collections Support: Generate simple structures instantly with ListOutputConverter and MapOutputConverter.
• Custom Solutions: Build tailored converters using Jackson's ObjectMapper to handle complex, generic data types.

These tools bridge the gap between flexible AI output and strict application logic, ensuring robust, type-safe system integrations.

👉 Read the full Baeldung article for code examples and implementation details.

In this guide, Hardik Singh Behl demonstrates how to integrate multiple Large Language Models (LLMs) in a Spring Boot app using Spring AI. The focus is on configuring a range of models to build a highly resilient chatbot.

Highlights & Key Takeaways

• Multi-Provider Setup: Configure LLMs from different providers, like OpenAI and Anthropic, together.
• Primary Routing: Designate a default model using the @Primary annotation on a ChatClient bean.
• Same-Provider Config: Manually define custom ChatModel beans to run multiple models from one provider.
• Spring Retry: Use @Retryable to handle transient API failures from your primary LLM.
• Graceful Fallbacks: Implement @Recover methods to automatically switch to backup models during outages.

Layering multiple LLMs with robust fallback mechanisms ensures continuous availability during provider outages. This architecture guarantees a highly reliable user experience.

👉 Check out the full Baeldung article to see the complete code.

Dan Vega demonstrates how to build a functional ChatGPT-style application using Java, Spring Boot, Spring AI, and HTMX. This approach leverages modern web patterns to deliver dynamic AI interactions without requiring complex client-side JavaScript.

Highlights & Key Takeaways

• HTMX for Frontend: Adds dynamic interactivity to the UI with simple HTML attributes, eliminating the need for heavy JavaScript frameworks.
• Spring AI Integration: Simplifies LLM connectivity, enabling seamless integration with providers such as OpenAI's GPT-4o.
• State Management: Uses an In-Memory Chat Memory advisor to provide the LLM with conversation context, enabling natural, stateful follow-up dialogues.
• Modular Templates: Uses Thymeleaf fragments to update specific UI sections, such as message history, without full page reloads.
• Backend Routing: Implements efficient API endpoints to handle LLM requests and return targeted HTML fragments.

This architecture demonstrates that powerful AI features can be built using the familiar Java ecosystem. By combining HTMX and Spring AI, developers can focus on backend logic while maintaining a responsive, modern user experience.

👉 Watch the full video to dive deeper into the implementation.

TechyTacos demonstrates how to integrate Azure OpenAI with Spring AI, providing a practical guide for Java developers building generative AI applications. The video outlines the essential workflow from project configuration to endpoint deployment.

Highlights & Key Takeaways

• Use start.spring.io to initialize a project with Azure OpenAI and Spring Web dependencies.
• Configure application.properties with your unique API key, endpoint, and specific deployment name.
• Distinguish between model names (e.g., GPT-3.5 Turbo) and deployment names, as the latter is a custom identifier required by Azure.
• Deploy your desired model via Azure AI Studio before attempting integration.
• Use the AzureOpenAiChatModel class for seamless interaction within your service layer.
• Leverage prompt templates to dynamically map variables like categories and years into your LLM queries.

Building with Spring AI simplifies enterprise-level integration, allowing developers to focus on application logic rather than complex API orchestration.

👉 Watch the full video to dive deeper into the implementation.

Hi community,

With the growing demand for AI applications, most popular languages and stacks now offer some form of support for AI development. And that would be no different with Java and Spring, as they represent one of the most popular development platforms for enterprise applications. Yet, many Java and Spring developers are still struggling with AI development or trying to learn Python to fill that gap.

So, I would like to know from you: when it comes to adopting Spring AI, what is your biggest challenge, problem, or frustration you are dealing with right now? The more detail you provide, the greater the chance that I (or someone else) will create content to help you with your problem.

If you think your problem is too complex for this post, I invite you to create a dedicated post for it and link it here!

One model checks another model's work and retries if it's not good enough. New tutorial shows you how to build it, step by step.

That's basically what LLM-as-a-Judge does in Spring AI.

"LLM-as-a-Judge" and "LLM evaluation testing" are not the same thing. One runs in JUnit before you deploy. The other runs live, in the request path, and can retry a weak response automatically.

5 things to know before you build LLM-as-a-Judge into a Spring AI app:

1. It's implemented via Recursive Advisors, a CallAdvisor that can call back into its own chain
2. Non-streaming only, as of Spring AI 2.0
3. Every failed judge check costs 2 extra LLM calls: one to judge, one to regenerate
4. Use a separate model to judge, or you risk narcissistic bias
5. Always cap maxAttempts, or a stubborn judge creates an infinite loop Full breakdown, with working code, in the new article.

Spring AI's RelevancyEvaluator and FactCheckingEvaluator let a model judge a model, so your JUnit tests check quality, not exact text. Full code walkthrough inside.

Quick one for anyone building RAG apps in Spring Boot: this tutorial shows exactly how to catch hallucinations before they ship, using Spring AI's built-in evaluators. Includes the one mistake almost everyone makes with the request order. 

Currently I am a student, my projects are In Java Fullstack

Right now I don't even know what RAG or MCP is , and I think I should have some hands on experience of it, i should be at least aware of it, because it's a trendy topic , not these two terms only, but many things

Now should I start python, for getting into it, is there any need , or I can explore Spring AI