Setting up Spring AI in a Spring Boot project — step by step

Dev had the mental model. Tokens, context windows, temperature, prompts — all clear. Time to open the IDE.

The first question: how do you actually connect a Spring Boot application to an LLM? The answer turns out to be remarkably familiar — a starter dependency, a few properties, and an injected bean. If you have set up Spring Data or Spring Security before, Spring AI follows the same pattern.

This post gets you from a blank Spring Boot project to a working LLM API call.

Table of contents

Project prerequisites

Before adding Spring AI, confirm your project has:

• Java 21 (Spring AI requires Java 17+, but this course uses Java 21 features)
• Spring Boot 3.3 or later
• Maven 3.9+

If you are starting fresh, generate a project at start.spring.io with Spring Boot 3.x, Java 21, and the Web dependency.

Step 1 — Add the Spring AI BOM

Spring AI uses a BOM (Bill of Materials) to manage consistent dependency versions. Add it to your pom.xml under <dependencyManagement>:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-bom</artifactId>
            <version>1.0.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

Step 2 — Add the model starter dependency

Choose the model provider you want to use. For this course we use OpenAI in staging/production and Ollama locally. Add the starter — no version needed because the BOM manages it:

OpenAI:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>

Ollama (for local development):

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-ollama</artifactId>
</dependency>

You can have both in the same project and switch between them using Spring profiles. We will set that up shortly.

Important: Never commit your API key to source control. Store it as an environment variable and reference it in properties with ${OPENAI_API_KEY}. If you accidentally expose an API key, rotate it immediately at platform.openai.com.

Step 3 — Configure the model

For OpenAI (src/main/resources/application.properties):

spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.openai.chat.options.model=gpt-4o-mini
spring.ai.openai.chat.options.temperature=0.2
spring.ai.openai.chat.options.max-tokens=500

For Ollama — first, install and run Ollama:

# macOS
brew install ollama
ollama serve

# Pull a model (one-time download, ~2-4 GB)
ollama pull llama3.2

Then in application-dev.properties (used when spring.profiles.active=dev):

spring.ai.ollama.base-url=http://localhost:11434
spring.ai.ollama.chat.options.model=llama3.2
spring.ai.ollama.chat.options.temperature=0.2

To use the dev profile locally:

SPRING_PROFILES_ACTIVE=dev ./mvnw spring-boot:run

Tip: Use Ollama during development and CI — it is free, works offline, and requires no API key. Switch to OpenAI only in staging and production where output quality matters. This alone can reduce your AI spend to near zero during development.

Step 4 — Verify auto-configuration

Spring AI auto-configures a ChatClient.Builder bean when it detects the right starter and properties. Check your application starts without errors:

./mvnw spring-boot:run

If you see an error like Could not resolve placeholder 'OPENAI_API_KEY', the environment variable is not set. Export it first:

export OPENAI_API_KEY=sk-your-key-here
./mvnw spring-boot:run

Step 5 — Make your first LLM call

Inject ChatClient.Builder, build a ChatClient, and make a call:

@RestController
class HelloAiController {

    private final ChatClient chatClient;

    HelloAiController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    @GetMapping("/ai/hello")
    String hello(@RequestParam String question) {
        return chatClient.prompt()
                .user(question)
                .call()
                .content();
    }
}

Start the app and test it:

curl "http://localhost:8080/ai/hello?question=What+is+Spring+Boot+in+one+sentence"

You should get a one-sentence response from the model. If you do, the integration is working.

Step 6 — Configure a shared ChatClient bean

For production code, configure a ChatClient bean with shared defaults rather than building a new one per controller:

@Configuration
class AiConfig {

    @Bean
    ChatClient chatClient(ChatClient.Builder builder) {
        return builder
                .defaultSystem("""
                        You are a helpful customer support assistant for TechGadgets, an online electronics store.
                        Answer only questions about products, orders, and store policies.
                        If you cannot answer from the provided context, say: "I don't have that information right now."
                        Keep responses concise — 2 to 4 sentences unless more detail is explicitly requested.
                        """)
                .build();
    }
}

Inject this bean wherever you need it:

@Service
class SupportService {

    private final ChatClient chatClient;

    SupportService(ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    String answer(String customerQuestion) {
        return chatClient.prompt()
                .user(customerQuestion)
                .call()
                .content();
    }
}

The system prompt is set once in the bean. All calls through this client use it automatically.

Caution: Do not put your system prompt text directly in the @Bean method as a hardcoded string in production code. Move it to a .st file in src/main/resources/prompts/. The next post covers this with PromptTemplate.

Setup checklist

1. Add spring-ai-bom to <dependencyManagement> with the target version.
2. Add the model starter (spring-ai-starter-model-openai or spring-ai-starter-model-ollama).
3. Set API key via environment variable — never hardcode.
4. Configure model name and temperature in application.properties.
5. Create a ChatClient @Bean with a default system prompt.
6. Verify: start the app, hit one endpoint, confirm a response comes back.
7. Set up Spring profiles: dev → Ollama, prod → OpenAI.

Note: The entire setup takes about 10 minutes for a developer who has done Spring Boot before. If something is not working, check the auto-configuration report: start the app with --debug and search for OpenAiAutoConfiguration or OllamaAutoConfiguration to see what was matched and what was skipped.

References

• Spring AI getting started guide
• Spring AI ChatClient reference
• Ollama model library