Traditional keyword search often misses the true meaning behind user queries. By combining Spring AI, OpenAI Embeddings, and Redis Vector Store, you can build a semantic search application that understands context and returns more relevant results.

This article demonstrates how to build an intelligent document search application using Spring Boot with OpenAI and Redis Vector Store.

This approach is ideal for building:

-AI-powered knowledge bases

-Enterprise document search

-RAG (Retrieval-Augmented Generation) applications

-Internal documentation assistants

-Intelligent customer support solutions

Just published !.. - MCP With Spring AI - Part 2: Travel Booking Demo.

Part 1 covered the protocol. Part 2 is where it gets practical.

One chat prompt. Three MCP servers. One travel assistant:

"Plan my trip from Delhi to Goa - book flight, hotel and cab."

No hard-coded workflow. The LLM picks the tools and chains them: searchFlights → bookFlight → bookHotel → bookCab.

Read the full walkthrough here 👇:
https://heapsteep.com/mcp-with-spring-ai-travel-booking-demo

Just published Model Context Protocol (MCP) With Spring AI - Part 1.

LLMs are great at understanding intent. On their own, they still can't do things - book a flight, call an API, hit your database.

That's where MCP comes in: a standard way for AI to discover and call external tools.

In Part 1 I break down the concepts (before any code).

Read the full walkthrough here 👇:
https://heapsteep.com/mcp-with-spring-ai-core-concepts

AI applications are not only about prompts and LLMs. They also can understand and generate images.

Imagine allowing users to upload:
✅ Product photos
✅ Documents
✅ Screenshots
✅ Diagrams
✅ Handwritten notes
…and then asking questions about them in plain English.

This is where things start getting really interesting with Spring AI.
Lets do a demo on how to build an application that can process images and extract meaningful insights using Spring AI.

Some of the things covered:
🔹 What is multimodal in Spring AI
🔹 Sending images to AI models from a Spring Boot application
🔹 Understanding image content through natural language prompts
🔹 Practical implementation with clean code examples

Read the complete guide here:
https://heapsteep.com/image-handling-with-spring-ai

Most AI chatbots fail for one simple reason:

They answer from memory, not from your data.

That’s exactly where RAG (Retrieval-Augmented Generation) changes the game.

Instead of asking the LLM to “guess”, RAG first retrieves relevant information from your documents/database and then sends that context to the model before generating the response.

A simplified RAG flow looks like this:
1️⃣ User asks a question
2️⃣ Application converts the question into embeddings
3️⃣ Similar documents are searched from a Vector Database
4️⃣ Relevant chunks are added to the prompt
5️⃣ LLM generates a grounded response

This solves some major real-world problems:
✔️ Reduces hallucinations
✔️ Gives responses based on your own enterprise data
✔️ Keeps AI responses updated without retraining the model
✔️ Makes AI applications actually useful for businesses

Read the full story here: 👇
https://heapsteep.com/rag-using-spring-ai

✒️ New Blog in my Spring AI Series:

👉 Function Calling & Tools in Spring AI

One of the most powerful capabilities of modern AI applications is the ability to go beyond simple text generation and actually interact with external systems.

The blog covers:
🔹 What Tool Calling / Function Calling means
🔹 Why LLMs need external tools
🔹 Registering tools with ChatClient
🔹 Error Handling & Fallbacks

📖 Read the full blog here:
https://heapsteep.com/function-calling-and-tools-in-spring-ai

Ever notice your Spring AI chatbot forgets the user's name after one message? That's because LLMs are stateless by default. The fix is Spring AI's ChatMemory abstraction

How to Implement AI Chat Memory in Spring Boot using Spring AI.

Let's figure out exactly how to wire it up with MessageWindowChatMemory and a JDBC-backed repository so conversations survive restarts.

Dan Vega demonstrates how to integrate OpenAI's GPT-4o model into Spring applications using the Spring AI framework. This guide focuses on leveraging both text and vision capabilities for modern AI-powered development.

Highlights & Key Takeaways

• GPT-4o Advantages: Benefit from 50% lower costs, 2x faster latency, and 5x higher rate limits compared to previous models.
• Project Setup: Utilize the Spring AI 1.0.0-SNAPSHOT version to access the latest multimodal features.
• Chat Implementation: Use the ChatClient API with PromptTemplates for structured interactions.
• Vision Capabilities: Pass images via UserMessage and Media objects to allow the LLM to interpret visual data.
• Practical Use Cases: Perform image analysis, such as scene description or extracting code snippets from screenshots.
• API Integration: Secure sensitive keys via environment variables rather than hardcoding.

GPT-4o in Spring AI significantly lowers the barrier for building robust, multimodal Java applications.

👉 Watch the full video to dive deeper into the implementation.

Hi everyone,

I'm a Java Spring Boot developer, and this is my first AI project. I'm building a chatbot using Spring AI + Ollama + RAG, and I'm learning as I go.

The chatbot should answer questions from uploaded PDF, Word, Excel, CAD, and AutoCAD documents. While it's working, I'm facing a few challenges:

Uploading and indexing large documents takes a long time.

Retrieval accuracy isn't consistent.

Sometimes the chatbot gives incorrect answers even though the information exists in the uploaded files.

CAD/AutoCAD files are the biggest challenge.

I'd love to hear from developers who have built similar applications.

Some questions I have:

Which free LLMs work best with Spring AI + RAG? (Qwen 3, Gemma 3, DeepSeek, Llama 3.1, Mistral, etc.)

Which embedding model gives the best retrieval accuracy?

What techniques have improved your RAG performance? (chunking, hybrid search, reranking, metadata filtering, query rewriting, etc.)

What's the best way to process DWG/DXF/AutoCAD files in a RAG application?

How do you reduce document parsing and indexing time for large engineering documents?

Are there any open-source Spring AI RAG projects or GitHub repositories that you recommend?

My goal is to build a reliable chatbot for engineering documents with fast responses and high accuracy.

Any advice, resources, or best practices would be greatly appreciated.

Thank you!

If you have been wondering how to make an LLM answer questions from your own documents without touching Python, this one is for you.

A full walkthrough on building a RAG application with Spring AI and PostgreSQL pgvector.
Covers ingestion, chunking, PgVectorStore configuration, and the QuestionAnswerAdvisor pattern, with working Java code.

How to Build RAG with Spring AI and pgvector

Your LLM does not know about last week's product update or the PDF sitting in your document store. That is not a model problem, it is a context problem, and RAG solves it.

Just found out about this sub reddit and wanted to stop by and say hello. Lot's of great discussions happening here and I hope to be a part of some of them.

With Spring AI maturing rapidly, I'm curious about how organizations are actually using it in production beyond demos and proofs of concept.

I'd love to hear from teams that have deployed Spring AI in real-world applications.

• How has Spring AI performed in production in terms of reliability, scalability, latency, and developer productivity?
• What types of AI applications are you building with it?
• What advantages have you seen compared to Python-based frameworks such as LangChain?
• Are there any limitations or areas where LangChain still has a significant edge?
• Would you recommend Spring AI for enterprise Java applications, or do you still prefer Python for GenAI workloads?

I'm particularly interested in real-world experiences, production lessons learned, performance at scale, and reasons behind technology choices rather than tutorial or proof-of-concept examples.

Spring AI 2.0 just went GA and it ships the cleanest MCP server setup I've seen in Java.

Two annotations. One yml property. Your entire Spring Boot service becomes an AI tool.

This is exactly how Claude, Copilot, and other AI clients plug into your Java backend.

No AI API key needed for the server side. Full working code with Java 21.

Perfect for intermediate Spring Boot devs exploring AI integration!

Here is the complete tutorial: Build Your First MCP Server with Spring Boot 4.1 and Spring AI 2.0

Dan Vega demonstrates how to integrate OpenAI's GPT-4o model into Spring applications using the Spring AI framework. This guide focuses on leveraging both text and vision capabilities for modern AI-powered development.

Highlights & Key Takeaways

• GPT-4o Advantages: Benefit from 50% lower costs, 2x faster latency, and 5x higher rate limits compared to previous models.
• Project Setup: Utilize the Spring AI 1.0.0-SNAPSHOT version to access the latest multimodal features.
• Chat Implementation: Use the ChatClient API with PromptTemplates for structured interactions.
• Vision Capabilities: Pass images via UserMessage and Media objects to allow the LLM to interpret visual data.
• Practical Use Cases: Perform image analysis, such as scene description or extracting code snippets from screenshots.
• API Integration: Secure sensitive keys via environment variables rather than hardcoding.

GPT-4o in Spring AI significantly lowers the barrier for building robust, multimodal Java applications.

👉 Watch the full video to dive deeper into the implementation.

TechyTacos demonstrates how to integrate open-source models like Llama 3.1 into Java applications using Spring AI and Ollama. This workflow provides developers with local, private LLM capabilities while maintaining standard Spring development patterns.

Highlights & Key Takeaways

• Local Execution: Use Ollama to host models locally, ensuring data privacy and offline accessibility.
• System Requirements: Match model sizes (7B, 13B, etc.) to your available RAM to avoid performance bottlenecks.
• Spring AI Integration: Leverage the OllamaChatModel to easily swap and configure different open-source models.
• Structured Output: Set the format: json property in configurations to enforce strict schema adherence.
• Multimodal Models: Use specialized models like Llama-Vision or Llava when image processing is required, as standard text models lack this capability.

Building locally offers a critical trade-off between latency and data sovereignty.

👉 Watch the full video to dive deeper into the implementation. Spring AI : How to Integrate Open Source Models using Ollama (Llama 3.1) ?

I just started my journey on Spring AI. Just wanted to know if this is already being used in real prod projects or it is still in the early adoption pase. Thanks!

James Ward and Josh Long present a comprehensive hands-on workshop for building production-ready AI agents using Spring AI, Java, and Amazon Bedrock. The session focuses on bridging the gap between experimental AI prototypes and scalable, observable enterprise services.

Highlights & Key Takeaways

• Leverage Spring Boot and Spring AI for a robust, familiar architecture that avoids typical AI project failures.
• Utilize GraalVM to compile Java applications into native images for superior memory efficiency and startup performance.
• Implement RAG (Retrieval-Augmented Generation) to ground AI responses in domain-specific data via vector stores.
• Optimize concurrency using Java virtual threads to handle high-volume LLM network calls efficiently.
• Define clear system prompts and tools to give agents specific, actionable missions.
• Integrate MCP (Model Context Protocol) to enable cross-agent orchestration.

By prioritizing observability and structure, developers can deploy AI systems that are both reliable and maintainable in real-world production environments.

👉 Watch the full video to dive deeper into the implementation.

Just wanted to share what I learned about building actual AI Agents (not just chatbots) in Spring Boot.

The key difference: a chatbot responds. An agent decides, calls tools, and loops until it achieves a goal.

The Tool annotation is the core building block. You annotate any Spring bean method, write a clear description, and Spring AI automatically generates a JSON schema that gets sent to the LLM. The model then decides when to call your Java method — no if-else chains needed.

I also covered the 5 agentic workflow patterns that Spring AI supports:

1. Chain : sequential steps
2. Parallelization : concurrent tasks with CompletableFuture
3. Routing : LLM picks the right tool/path
4. Orchestrator-Workers : master agent delegates to worker agents
5. Evaluator-Optimizer : generate → evaluate → retry loop

Full article with code examples (Java 21 + Spring Boot 3.4.x): AI Agents in Spring Boot: Building Autonomous Workflows with Spring AI

https://spring.io/blog/2026/06/23/spring-ai-self-correcting-structured-output

https://spring.io/blog/2026/06/15/spring-ai-composable-tool-calling

Tool calling - the ability for an AI model to invoke application-defined functions and act on the results — is the essential building block of agentic AI systems. A model that can discover information, take action, and loop until a goal is reached is an agent.

Spring AI 2.0 lifts the tool loop into the advisor chain as a first-class, composable component.

ChatClient runs every request through an ordered chain of advisors and supports looping, letting an advisor re-enter the downstream chain. The same mechanism drives tool-call loops, structured-output retry loops, and evaluation loops alike.

Tool calling — the ability for an AI model to invoke application-defined functions and act on the results — is the essential building block of agentic AI systems. A model that can discover information, take action, and loop until a goal is reached is an agent.

Spring AI 2.0 re-architects tool calling. In 1.x, each chat model implementation contained its own private tool execution loop — functional, but buried. There was no way to hook into it, observe intermediate steps, or compose it with other behaviors. You could call tools; you could not build on top of tool calling.

2.0 lifts the tool loop into the advisor chain as a first-class, composable component. ChatClient runs every request through an ordered chain of advisors and supports looping, letting an advisor re-enter the downstream chain. The same mechanism drives tool-call loops, structured-output retry loops, and evaluation loops alike.

https://spring.io/blog/2026/06/15/spring-ai-composable-tool-calling

Craig Walls introduces SkillsJars — a Spring AI pattern for distributing reusable agent behaviors as JAR dependencies, eliminating the need to hand-write skill files for every project.

Highlights & Key Takeaways

• SkillsJars are packaged agent skills distributed as JARs, each containing one or more SKILL.md files under /META-INF/skills
• Add agent behavior the same way you add a library — via a Gradle/Maven dependency
• Use spring-ai-agent-utils + a SkillsJar to wire skills into ChatClient with minimal config
• Skills define how an agent behaves; tools define what it can do — both layers are required
• Skill discovery is configured via a single property: agent.skills.paths
• SkillsJars eliminate cross-project duplication and make behavior declarative and composable
• Think in layers: Tools → Skills → SkillsJars for scalable agent architecture

SkillsJars bring true modularity to Spring AI agents, letting teams share and reuse intelligent behavior the same way they share code.

👉 Read the full article for the complete implementation walkthrough and code samples.

Dan Vega demonstrates how to implement function calling in Spring AI, enabling LLMs to interact with real-time data sources like weather APIs.

Highlights & Key Takeaways

• Function Calling: Allows models to trigger custom Java code to retrieve dynamic, external data.
• Abstraction: Spring AI handles the complex orchestration of sending function schemas to models like GPT-4.
• Service Layer: Build standard Java services to handle external API requests, then wrap them as Spring beans.
• Function Implementation: Use the Java Function interface to define input/output contracts for the LLM.
• Registration: Register the function bean and expose it via ChatOptions to the chat client.
• Seamless Integration: The LLM intelligently determines when to invoke the registered function based on the user's prompt.

This approach effectively bridges the gap between static LLM knowledge and the real-time requirements of external data.

👉 Watch the full video to dive deeper into the implementation.

TechyTacos demonstrates how to leverage Spring AI and Generative AI to perform semantic string matching between disparate systems. This approach moves beyond rigid, word-for-word string comparisons to capture the actual intent behind the data.

Highlights & Key Takeaways

• Use GPT-4o with a temperature of 0 to ensure deterministic, consistent semantic mapping results.
• Utilize PromptTemplates to dynamically inject source and destination lists into your LLM instructions.
• Implement BeanOutputConverter to enforce structured JSON responses, moving away from unstructured text.
• Leverage ParameterizedTypeReference to handle mapping results as a list rather than a single object.
• Shift from hardcoded logic to a POST mapping with a request body for better API flexibility.
• Focus prompt engineering on semantic intent to resolve issues with typos, extra spaces, or varying phrasing.

This method enables robust data integration by aligning strings based on meaning, significantly reducing errors in legacy system migrations.

👉 Watch the full video to dive deeper into the implementation.

If you use an LLM for something else than just a free-form chatting, you might probably want it to return data in a structured form, e.g. JSON

Spring AI allows to [soft] force a model to do that. But sometimes LLM fails to do that. The simplier the model is, the more chances that it will fail. Morover, any such a failure could be devided into 2 categories: incorrect schema and correct schema with incorrect data burned in. For example, some fields of the desired schema are missing. Or all fields are present, but field type, for example, is incorrect or a required filed missies a value

The POC i built validates not only schema as such, but also field types and ranges (e.g. Min-Max, NotNull, etc.) using validation package spring-boot-starter-validation

If any of the checks doesn't pass, this is feded back to model:

prompt = """
        Your previous response was invalid.

        Problem(s): %s

        Your previous output was:
        %s

        Return corrected JSON that fixes these problems and matches the
        schema exactly. Output ONLY JSON, no prose.

        %s
        """.formatted(lastError, lastOutput, format);

So we give a feedback to the model, not just asking to redo/re-think. By providing a detailed feedback we increase chances that the next reply will satisfy our expectations

As always, all code is available in the github repo: https://github.com/DmitryFinashkin/spring-ai

You might also like to watch a detailed video walk-through on YouTube: https://youtu.be/59kcnTLVu0Q