Spring Cloud AWS Bedrock Starter

Lets you use the AWS Bedrock Service with Spring Boot and Spring Cloud

Bedrock

This project provides a developer friendly way to access the services of AWS Bedrock, an AWS service that provides a managed Language Model (LLM) service with a catalog of supported LLMs. This is a Spring Cloud Starter for the Bedrock service that removes all the boilerplate code required to access the service. It uses the standard Spring Cloud AWS Starter to provide the credentials for the AWS SDK for Java, and the standards of Spring Boot and Spring Cloud to provide the configuration and the auto-wiring of the service objects.

This project leverages Spring Cloud AWS for credential management

Just provide the right properties as defined below and the appropriate AWSCredentialsProvider object will be activated for injection into your Spring objects. An example for the creation of BasicAwsCredentials is shown below. You can follow the link below to understand all the other supported credential models.

# for BasicAwsCredentials
spring.cloud.aws.credentials.access-key=YOUR_ACCESS_KEY
spring.cloud.aws.credentials.secret-key=YOUR_SECRET_KEY

# an optional method to do the same
spring.cloud.aws.credentials.profile.name=YOUR_PROFILE
spring.cloud.aws.credentials.profile.path=~/.aws/credentials

See the Credentials section of Spring Cloud AWS Starter documentation for alternate configuration options.

Properties that will enable ClaudeService

aws.bedrock.model.id=anthropic.claude-v2
aws.bedrock.model.claude.temperature=0.5
aws.bedrock.model.claude.maxTokensToSample=100

The Bedrock and BedrockRuntimeClient

Once the right AWSCredentialsProvider is injected, you can expect the activation of the appropriate versions of the BedrockClient and the BedrockRuntimeClient. These will be conditional on the bedrock.client.type property. The options there are sync and async, which drive the activation of the synchronous or the asynchronous versions of the clients.

E.g.,

bedrock.client.type=sync

One can use the activated clients to interact with the AWS Bedrock service based on the docs for the Bedrock Service.

Bedrock Service

The best part of the Starter is the activation of a service object which has the invoke method.

All you need to do is provide the properties like one of the sets below to activate the appropriate implementations of the BedrockService interface (Claude, Jurassic, Llama, StableDiffusion, or Titan).

Claude model parameters

aws.bedrock.model.claude.id=anthropic.claude-v2
aws.bedrock.model.claude.temperature=0.5
aws.bedrock.model.claude.maxTokensToSample=100

Jurassic model parameters

aws.bedrock.model.jurassic.id=ai21.j2-mid-v1
aws.bedrock.model.jurassic.maxTokens=200
aws.bedrock.model.jurassic.temperature=0.5

Llama2 model parameters

aws.bedrock.model.llama2.id=anthropic.llama2-v1
aws.bedrock.model.llama2.maxTokens=200
aws.bedrock.model.llama2.temperature=0.5

Titan model parameters

aws.bedrock.model.titan.id=amazon.titan-text-express-v1
aws.bedrock.model.titan.maxTokens=4096
aws.bedrock.model.titan.temperature=0.0

Stability model parameters

aws.bedrock.model.stability.id=stability.stable-diffusion-xl
aws.bedrock.model.stability.promptStrength=10.0
aws.bedrock.model.stability.generationStep=30
aws.bedrock.model.stability.seed=75

Usage

Once activated, a Service can be autowired and used as below.

@RestController
public class ClaudeController {

    private final ClaudeService claudeService;


    @Autowired
    public ClaudeController(ClaudeService claudeService) {
        this.claudeService = claudeService;
    }

    @GetMapping("/invoke")
    public String invoke(@RequestParam String prompt) {
        try {
            return claudeService.invoke(prompt);
        } catch (Exception e) {
            return "Error invoking Claude: " + e.getMessage();
        }
    }
}

If you activate more than one model, then you will need to use the @Qualifier annotation to eliminate ambiguity when choosing to inject by the BedrockService interface. See @Qualifier vs @Primary with Examples for background. You may not activate more than one model variant from a single provider.

You might also want to take a look at and/or run the Service tests. To do that, make sure you've set the appropriate AWS_* environment variables, then execute

mvn test -Dspring.profiles.active=authorized