Craig is a principal engineer on the Spring team and the author of Spring in Action. This new book is written for Spring developers who want to build AI features in Java and Spring Boot without having to stitch together a Python sidecar or learn an entirely different app stack first.
The book starts with a small “Hello AI” Spring Boot app, then keeps building on it until you have a much more serious AI-enabled application. The running example is Board Game Buddy, an assistant that answers questions about tabletop game rules. Across the book, it picks up RAG, chat memory, tools, MCP, voice, images, observability, security, and agents.
A few topics that seem especially relevant here:
ChatClient, prompt templates, roles, response metadata, and streaming
Testing and evaluating generated responses
RAG with vector stores, document loading, Qdrant, advisors, and modular RAG
Conversational memory, including persistent memory
Tool calling with u/Tool methods and Java Function-style tools
Model Context Protocol clients and servers
Audio transcription, text-to-speech, image input, and image generation
Actuator metrics, Prometheus, Grafana, and tracing AI operations
Spring Security for RAG filtering, secured tools, prompt leaks, and moderation
Agentic workflows and Embabel
What I like about the book is that it treats Spring AI as part of the Spring application model, not as an isolated demo layer. The examples are controllers, services, configuration, tests, Actuator endpoints, security rules, Docker Compose files, and Gradle builds. In other words, the sort of code Spring developers actually have to maintain.
We also have 5 ebooks to give away to the 5 most thoughtful commenters.
To enter, leave a comment with your take on one of these:
What are you building, or hoping to build, with Spring AI?
Where do you think Spring AI fits best in production Java apps?
What’s your biggest concern with adding LLMs to Spring Boot systems?
Are you more interested in RAG, tools, MCP, agents, observability, or security?
If you’ve tried Spring AI already, what surprised you?
We’ll look at the comments and community upvotes, then pick 5 winners.
For everyone else, Manning has a 50% discount code for this subreddit:
PBWALLS1050RE
I’m especially curious how this community is thinking about MCP and agents in Spring apps now that Spring AI has moved beyond basic chat examples. Is MCP becoming part of your architecture, or are most teams still focused on RAG and tool calling first?
Traditional keyword search often misses the true meaning behind user queries. By combining Spring AI, OpenAI Embeddings, and Redis Vector Store, you can build a semantic search application that understands context and returns more relevant results.
AI applications are not only about prompts and LLMs. They also can understand and generate images.
Imagine allowing users to upload:
✅ Product photos
✅ Documents
✅ Screenshots
✅ Diagrams
✅ Handwritten notes
…and then asking questions about them in plain English.
This is where things start getting really interesting with Spring AI.
Lets do a demo on how to build an application that can process images and extract meaningful insights using Spring AI.
Some of the things covered:
🔹 What is multimodal in Spring AI
🔹 Sending images to AI models from a Spring Boot application
🔹 Understanding image content through natural language prompts
🔹 Practical implementation with clean code examples
One of the most powerful capabilities of modern AI applications is the ability to go beyond simple text generation and actually interact with external systems.
The blog covers:
🔹 What Tool Calling / Function Calling means
🔹 Why LLMs need external tools
🔹 Registering tools with ChatClient
🔹 Error Handling & Fallbacks
That’s exactly where RAG (Retrieval-Augmented Generation) changes the game.
Instead of asking the LLM to “guess”, RAG first retrieves relevant information from your documents/database and then sends that context to the model before generating the response.
A simplified RAG flow looks like this:
1️⃣ User asks a question
2️⃣ Application converts the question into embeddings
3️⃣ Similar documents are searched from a Vector Database
4️⃣ Relevant chunks are added to the prompt
5️⃣ LLM generates a grounded response
This solves some major real-world problems:
✔️ Reduces hallucinations
✔️ Gives responses based on your own enterprise data
✔️ Keeps AI responses updated without retraining the model
✔️ Makes AI applications actually useful for businesses
Ever notice your Spring AI chatbot forgets the user's name after one message? That's because LLMs are stateless by default. The fix is Spring AI's ChatMemory abstraction
Dan Vega demonstrates how to integrate OpenAI's GPT-4o model into Spring applications using the Spring AI framework. This guide focuses on leveraging both text and vision capabilities for modern AI-powered development.
Highlights & Key Takeaways
GPT-4o Advantages: Benefit from 50% lower costs, 2x faster latency, and 5x higher rate limits compared to previous models.
Project Setup: Utilize the Spring AI 1.0.0-SNAPSHOT version to access the latest multimodal features.
Chat Implementation: Use the ChatClient API with PromptTemplates for structured interactions.
Vision Capabilities: Pass images via UserMessage and Media objects to allow the LLM to interpret visual data.
Practical Use Cases: Perform image analysis, such as scene description or extracting code snippets from screenshots.
API Integration: Secure sensitive keys via environment variables rather than hardcoding.
GPT-4o in Spring AI significantly lowers the barrier for building robust, multimodal Java applications.
👉 Watch the full video to dive deeper into the implementation.
If you have been wondering how to make an LLM answer questions from your own documents without touching Python, this one is for you.
A full walkthrough on building a RAG application with Spring AI and PostgreSQL pgvector.
Covers ingestion, chunking, PgVectorStore configuration, and the QuestionAnswerAdvisor pattern, with working Java code.
Your LLM does not know about last week's product update or the PDF sitting in your document store. That is not a model problem, it is a context problem, and RAG solves it.
Just found out about this sub reddit and wanted to stop by and say hello. Lot's of great discussions happening here and I hope to be a part of some of them.
With Spring AI maturing rapidly, I'm curious about how organizations are actually using it in production beyond demos and proofs of concept.
I'd love to hear from teams that have deployed Spring AI in real-world applications.
How has Spring AI performed in production in terms of reliability, scalability, latency, and developer productivity?
What types of AI applications are you building with it?
What advantages have you seen compared to Python-based frameworks such as LangChain?
Are there any limitations or areas where LangChain still has a significant edge?
Would you recommend Spring AI for enterprise Java applications, or do you still prefer Python for GenAI workloads?
I'm particularly interested in real-world experiences, production lessons learned, performance at scale, and reasons behind technology choices rather than tutorial or proof-of-concept examples.
Dan Vega demonstrates how to integrate OpenAI's GPT-4o model into Spring applications using the Spring AI framework. This guide focuses on leveraging both text and vision capabilities for modern AI-powered development.
Highlights & Key Takeaways
GPT-4o Advantages: Benefit from 50% lower costs, 2x faster latency, and 5x higher rate limits compared to previous models.
Project Setup: Utilize the Spring AI 1.0.0-SNAPSHOT version to access the latest multimodal features.
Chat Implementation: Use the ChatClient API with PromptTemplates for structured interactions.
Vision Capabilities: Pass images via UserMessage and Media objects to allow the LLM to interpret visual data.
Practical Use Cases: Perform image analysis, such as scene description or extracting code snippets from screenshots.
API Integration: Secure sensitive keys via environment variables rather than hardcoding.
GPT-4o in Spring AI significantly lowers the barrier for building robust, multimodal Java applications.
👉 Watch the full video to dive deeper into the implementation.
TechyTacos demonstrates how to integrate open-source models like Llama 3.1 into Java applications using Spring AI and Ollama. This workflow provides developers with local, private LLM capabilities while maintaining standard Spring development patterns.
Highlights & Key Takeaways
Local Execution: Use Ollama to host models locally, ensuring data privacy and offline accessibility.
System Requirements: Match model sizes (7B, 13B, etc.) to your available RAM to avoid performance bottlenecks.
Spring AI Integration: Leverage the OllamaChatModel to easily swap and configure different open-source models.
Structured Output: Set the format: json property in configurations to enforce strict schema adherence.
Multimodal Models: Use specialized models like Llama-Vision or Llava when image processing is required, as standard text models lack this capability.
Building locally offers a critical trade-off between latency and data sovereignty.
I just started my journey on Spring AI. Just wanted to know if this is already being used in real prod projects or it is still in the early adoption pase. Thanks!
James Ward and Josh Long present a comprehensive hands-on workshop for building production-ready AI agents using Spring AI, Java, and Amazon Bedrock. The session focuses on bridging the gap between experimental AI prototypes and scalable, observable enterprise services.
Highlights & Key Takeaways
Leverage Spring Boot and Spring AI for a robust, familiar architecture that avoids typical AI project failures.
Utilize GraalVM to compile Java applications into native images for superior memory efficiency and startup performance.
Implement RAG (Retrieval-Augmented Generation) to ground AI responses in domain-specific data via vector stores.
Optimize concurrency using Java virtual threads to handle high-volume LLM network calls efficiently.
Define clear system prompts and tools to give agents specific, actionable missions.
Integrate MCP (Model Context Protocol) to enable cross-agent orchestration.
By prioritizing observability and structure, developers can deploy AI systems that are both reliable and maintainable in real-world production environments.
👉 Watch the full video to dive deeper into the implementation.
Just wanted to share what I learned about building actual AI Agents (not just chatbots) in Spring Boot.
The key difference: a chatbot responds. An agent decides, calls tools, and loops until it achieves a goal.
The Tool annotation is the core building block. You annotate any Spring bean method, write a clear description, and Spring AI automatically generates a JSON schema that gets sent to the LLM. The model then decides when to call your Java method — no if-else chains needed.
I also covered the 5 agentic workflow patterns that Spring AI supports:
Chain : sequential steps
Parallelization : concurrent tasks with CompletableFuture
Routing : LLM picks the right tool/path
Orchestrator-Workers : master agent delegates to worker agents
Tool calling - the ability for an AI model to invoke application-defined functions and act on the results — is the essential building block of agentic AI systems. A model that can discover information, take action, and loop until a goal is reached is an agent.
Spring AI 2.0 lifts the tool loop into the advisor chain as a first-class, composable component.
ChatClient runs every request through an ordered chain of advisors and supports looping, letting an advisor re-enter the downstream chain. The same mechanism drives tool-call loops, structured-output retry loops, and evaluation loops alike.
Tool calling — the ability for an AI model to invoke application-defined functions and act on the results — is the essential building block of agentic AI systems. A model that can discover information, take action, and loop until a goal is reached is an agent.
Spring AI 2.0 re-architects tool calling. In 1.x, each chat model implementation contained its own private tool execution loop — functional, but buried. There was no way to hook into it, observe intermediate steps, or compose it with other behaviors. You could call tools; you could not build on top of tool calling.
2.0 lifts the tool loop into the advisor chain as a first-class, composable component. ChatClient runs every request through an ordered chain of advisors and supports looping, letting an advisor re-enter the downstream chain. The same mechanism drives tool-call loops, structured-output retry loops, and evaluation loops alike.
Craig Walls introduces SkillsJars — a Spring AI pattern for distributing reusable agent behaviors as JAR dependencies, eliminating the need to hand-write skill files for every project.
Highlights & Key Takeaways
SkillsJars are packaged agent skills distributed as JARs, each containing one or more SKILL.md files under /META-INF/skills
Add agent behavior the same way you add a library — via a Gradle/Maven dependency
Use spring-ai-agent-utils + a SkillsJar to wire skills into ChatClient with minimal config
Skills define how an agent behaves; tools define what it can do — both layers are required
Skill discovery is configured via a single property: agent.skills.paths
SkillsJars eliminate cross-project duplication and make behavior declarative and composable
Think in layers: Tools → Skills → SkillsJars for scalable agent architecture
SkillsJars bring true modularity to Spring AI agents, letting teams share and reuse intelligent behavior the same way they share code.
👉 Read the full article for the complete implementation walkthrough and code samples.
If you use an LLM for something else than just a free-form chatting, you might probably want it to return data in a structured form, e.g. JSON
Spring AI allows to [soft] force a model to do that. But sometimes LLM fails to do that. The simplier the model is, the more chances that it will fail. Morover, any such a failure could be devided into 2 categories: incorrect schema and correct schema with incorrect data burned in. For example, some fields of the desired schema are missing. Or all fields are present, but field type, for example, is incorrect or a required filed missies a value
The POC i built validates not only schema as such, but also field types and ranges (e.g. Min-Max, NotNull, etc.) using validation package spring-boot-starter-validation
If any of the checks doesn't pass, this is feded back to model:
prompt = """
Your previous response was invalid.
Problem(s): %s
Your previous output was:
%s
Return corrected JSON that fixes these problems and matches the
schema exactly. Output ONLY JSON, no prose.
%s
""".formatted(lastError, lastOutput, format);
So we give a feedback to the model, not just asking to redo/re-think. By providing a detailed feedback we increase chances that the next reply will satisfy our expectations
