From 4ef572b5882a097195a331302ad23904c42278a5 Mon Sep 17 00:00:00 2001 From: Chris Phillipson Date: Sun, 26 Nov 2023 18:59:13 -0800 Subject: [PATCH 1/4] Update README * Add hyperlinks for additional context * Fix grammar and adjust section titles --- Readme.md | 64 ++++++++++++++++++++++++++++++++----------------------- 1 file changed, 37 insertions(+), 27 deletions(-) diff --git a/Readme.md b/Readme.md index f1c9d78..07e627c 100644 --- a/Readme.md +++ b/Readme.md @@ -4,12 +4,12 @@ [![Alpha](https://img.shields.io/badge/Release-Alpha-darkred)](https://img.shields.io/badge/Release-Alpha-darkred) ![Github Action CI Workflow Status](https://github.com/clue2solve/aws-bedrock-springtboot-starter/actions/workflows/ci.yml/badge.svg) [![Known Vulnerabilities](https://snyk.io/test/github/clue2solve/aws-bedrock-springtboot-starter/badge.svg?style=plastic)](https://snyk.io/test/github/clue2solve/aws-bedrock-springtboot-starter) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) ### Bedrock -This project provides a developer friendly way to access the services of AWS Bedrock, an AWS service that provides a managed Language Model (LLM) service with a catalog of supported LLMs. -This is a Spring Cloud Starter for the Bedrock service that removes all the boilerplate code required to access the service. It uses the standard Spring Cloud AWS Starter to provide the credentials and the AWS SDK, and the standards of Spring boot and Spring Cloud to provide the configuration and the auto-wiring of the service objects. +This project provides a developer friendly way to access the services of [AWS Bedrock](https://aws.amazon.com/bedrock/), an AWS service that provides a managed Language Model (LLM) service with a catalog of supported LLMs. +This is a [Spring Cloud](https://spring.io/projects/spring-cloud) Starter for the Bedrock service that removes all the boilerplate code required to access the service. It uses the standard [Spring Cloud AWS](https://docs.awspring.io/spring-cloud-aws/docs/3.0.3/reference/html/index.html) Starter to provide the credentials for the [AWS SDK for Java](https://aws.amazon.com/sdk-for-java/), and the standards of Spring Boot and Spring Cloud to provide the configuration and the auto-wiring of the service objects. -#### This Project Levereges Spring Cloud AWS for credential management. +#### This project leverages Spring Cloud AWS for credential management -Just Provide the right properties as defined below and the apprpriate CredentialProvider object will be activated for injection into your Spring Objects. An Example for the creation of the BasicCredentials Provider is shown below. You can follow the link to understand all the other supported Credential models. +Just provide the right properties as defined below and the appropriate [AWSCredentialsProvider](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentialsProvider.html) object will be activated for injection into your Spring objects. An example for the creation of [BasicAwsCredentials](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/BasicAWSCredentials.html) is shown below. You can follow the link below to understand all the other supported credential models. ```properties # For BasicAwsCredentialsConfig @@ -18,10 +18,10 @@ spring.cloud.aws.credentials.secret-key=YOUR_SECRET_KEY spring.cloud.aws.credentials.profile.name=YOUR_PROFILE spring.cloud.aws.credentials.profile.path=~/.aws/credentials ``` -See the [Credentials](https://docs.awspring.io/spring-cloud-aws/docs/3.0.3/reference/html/index.html#credentials) section of Spring Cloud AWS Starter documentation for alternate configuration options. - +> See the [Credentials](https://docs.awspring.io/spring-cloud-aws/docs/3.0.3/reference/html/index.html#credentials) section of Spring Cloud AWS Starter documentation for alternate configuration options. ### Properties that will enable ClaudeService + ```properties aws.bedrock.model.id=anthropic.claude-v2 aws.bedrock.model.claude.prompt=defaultPrompt @@ -32,7 +32,11 @@ aws.bedrock.model.claude.maxTokensToSample=100 ``` ### The Bedrock and BedrockRuntimeClient -Once the right Credentials provider is injected, you can expect the activation of the appropriate versions of the BedrockClient and the BedrockRuntimeClient. These will be conditional on the `bedrock.client.type` property. the options there are `sync` and `async`, which drives the activation of the regular or the async versions of the client.s. Example: + +Once the right AWSCredentialsProvider is injected, you can expect the activation of the appropriate versions of the [BedrockClient](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrock/BedrockClient.html) and the [BedrockRuntimeClient](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrockruntime/BedrockRuntimeClient.html). These will be conditional on the `bedrock.client.type` property. The options there are `sync` and `async`, which drive the activation of the _regular_ or the _asynchronous_ versions of the clients. + +E.g., + ```property bedrock.client.type=sync ``` @@ -40,10 +44,13 @@ bedrock.client.type=sync One can use the activated clients to interact with the AWS Bedrock service based on the docs for the Bedrock Service. ### Bedrock Service -The best part of the Starter is the activation of the individual service objects which has the `invoke` methods. + +The best part of the Starter is the activation of a service object which has the `invoke` method. + All you need to do is provide the properties like one one of the sets below to activate the appropriate implementations of the BedrockService Interface ( Claude, Jurasic or Llama). -#### Claude Model Params +#### Claude model parameters + ```properties aws.bedrock.model.id=anthropic.claude-v2 @@ -53,7 +60,9 @@ aws.bedrock.model.claude.topP=0.5 aws.bedrock.model.claude.topK=50 aws.bedrock.model.claude.maxTokensToSample=100 ``` -#### Jurasic Model Params + +#### Jurassic model parameters + ```properties aws.bedrock.model.id=ai21.j2-mid-v1 @@ -62,33 +71,34 @@ aws.bedrock.model.jurassic.maxTokens=200 aws.bedrock.model.jurassic.temperature=0.5 ``` -#### Llama2 Model Params +#### Llama2 model parameters + ```properties aws.bedrock.model.id=anthropic.llama2-v1 aws.bedrock.model.llama2.prompt=Your prompt here aws.bedrock.model.llama2.maxTokens=200 aws.bedrock.model.llama2.temperature=0.5 - ``` ### Usage -Once activated the Service can be autowired and used as below. -```Java - private final ClaudeService claudeService; +Once activated, the Service can be autowired and used as below. - @Autowired - public ClaudeController(ClaudeService claudeService) { - this.claudeService = claudeService; - } +```java +private final ClaudeService claudeService; + +@Autowired +public ClaudeController(ClaudeService claudeService) { + this.claudeService = claudeService; +} - @GetMapping("/invoke") - public String invokeClaude(@RequestParam String prompt) { - try { - return claudeService.invokeClaude(prompt); - } catch (Exception e) { - return "Error invoking Claude: " + e.getMessage(); - } +@GetMapping("/invoke") +public String invokeClaude(@RequestParam String prompt) { + try { + return claudeService.invokeClaude(prompt); + } catch (Exception e) { + return "Error invoking Claude: " + e.getMessage(); } -``` \ No newline at end of file +} +``` From 17f933d459313611455ebf7e9b23d32cf3679840 Mon Sep 17 00:00:00 2001 From: Chris Phillipson Date: Sun, 26 Nov 2023 19:00:13 -0800 Subject: [PATCH 2/4] s/regular/synchronous --- Readme.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Readme.md b/Readme.md index 07e627c..7d276b7 100644 --- a/Readme.md +++ b/Readme.md @@ -33,7 +33,7 @@ aws.bedrock.model.claude.maxTokensToSample=100 ### The Bedrock and BedrockRuntimeClient -Once the right AWSCredentialsProvider is injected, you can expect the activation of the appropriate versions of the [BedrockClient](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrock/BedrockClient.html) and the [BedrockRuntimeClient](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrockruntime/BedrockRuntimeClient.html). These will be conditional on the `bedrock.client.type` property. The options there are `sync` and `async`, which drive the activation of the _regular_ or the _asynchronous_ versions of the clients. +Once the right AWSCredentialsProvider is injected, you can expect the activation of the appropriate versions of the [BedrockClient](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrock/BedrockClient.html) and the [BedrockRuntimeClient](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrockruntime/BedrockRuntimeClient.html). These will be conditional on the `bedrock.client.type` property. The options there are `sync` and `async`, which drive the activation of the _synchronous_ or the _asynchronous_ versions of the clients. E.g., From 02359b85189f59b7763563801fe65aaf046d4dac Mon Sep 17 00:00:00 2001 From: Chris Phillipson Date: Sun, 26 Nov 2023 19:01:17 -0800 Subject: [PATCH 3/4] Rename Conventions, conventions --- Readme.md => README.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename Readme.md => README.md (100%) diff --git a/Readme.md b/README.md similarity index 100% rename from Readme.md rename to README.md From cf844be8e6267e233ecfb94cbdb254643efeed74 Mon Sep 17 00:00:00 2001 From: Chris Phillipson Date: Sun, 26 Nov 2023 19:51:51 -0800 Subject: [PATCH 4/4] Replace typed local variables inside service methods with var --- .../starter/service/impl/ClaudeService.java | 17 ++++++++--------- .../starter/service/impl/JurassicService.java | 17 +++++++---------- .../starter/service/impl/LlamaService.java | 17 +++++++---------- 3 files changed, 22 insertions(+), 29 deletions(-) diff --git a/src/main/java/io/clue2solve/aws/bedrock/springboot/starter/service/impl/ClaudeService.java b/src/main/java/io/clue2solve/aws/bedrock/springboot/starter/service/impl/ClaudeService.java index c2a7652..91e0492 100644 --- a/src/main/java/io/clue2solve/aws/bedrock/springboot/starter/service/impl/ClaudeService.java +++ b/src/main/java/io/clue2solve/aws/bedrock/springboot/starter/service/impl/ClaudeService.java @@ -31,27 +31,26 @@ public ClaudeService(BedrockRuntimeClient client, ClaudeProperties properties) { @Override public String invoke(String prompt) throws JsonProcessingException { try { - String enclosedPrompt = "Human: " + prompt + "\n\nAssistant:"; - ObjectMapper mapper = new ObjectMapper(); - ObjectNode payload = mapper.createObjectNode(); + var enclosedPrompt = "Human: " + prompt + "\n\nAssistant:"; + var mapper = new ObjectMapper(); + var payload = mapper.createObjectNode(); payload.put("prompt", enclosedPrompt); payload.put("max_tokens_to_sample", properties.maxTokensToSample()); payload.put("temperature", properties.temperature()); // Create an ArrayNode and add elements to it - ArrayNode stopSequencesNode = mapper.createArrayNode(); + var stopSequencesNode = mapper.createArrayNode(); properties.stopSequences().forEach(stopSequencesNode::add); payload.set("stop_sequences", stopSequencesNode); - SdkBytes body = SdkBytes.fromUtf8String(payload.toString()); + var body = SdkBytes.fromUtf8String(payload.toString()); - InvokeModelRequest request = InvokeModelRequest.builder().modelId(properties.modelId()).body(body).build(); + var request = InvokeModelRequest.builder().modelId(properties.modelId()).body(body).build(); - InvokeModelResponse response = client.invokeModel(request); + var response = client.invokeModel(request); - ObjectNode responseBody = new ObjectMapper().readValue(response.body().asUtf8String(), ObjectNode.class); + var responseBody = new ObjectMapper().readValue(response.body().asUtf8String(), ObjectNode.class); return responseBody.get("completion").asText(); - } catch (AwsServiceException e) { System.err.println(e.awsErrorDetails().errorMessage()); diff --git a/src/main/java/io/clue2solve/aws/bedrock/springboot/starter/service/impl/JurassicService.java b/src/main/java/io/clue2solve/aws/bedrock/springboot/starter/service/impl/JurassicService.java index b9986d0..4261970 100644 --- a/src/main/java/io/clue2solve/aws/bedrock/springboot/starter/service/impl/JurassicService.java +++ b/src/main/java/io/clue2solve/aws/bedrock/springboot/starter/service/impl/JurassicService.java @@ -25,24 +25,21 @@ public JurassicService(BedrockRuntimeClient client, JurassicProperties propertie @Override public String invoke(String prompt) throws JsonProcessingException { try { - ObjectMapper mapper = new ObjectMapper(); - ObjectNode payload = mapper.createObjectNode(); + var mapper = new ObjectMapper(); + var payload = mapper.createObjectNode(); payload.put("prompt", properties.prompt() + prompt); payload.put("maxTokens", properties.maxTokens()); payload.put("temperature", properties.temperature()); - SdkBytes body = SdkBytes.fromUtf8String(payload.toString()); + var body = SdkBytes.fromUtf8String(payload.toString()); - InvokeModelRequest request = InvokeModelRequest.builder().modelId(properties.modelId()).body(body).build(); + var request = InvokeModelRequest.builder().modelId(properties.modelId()).body(body).build(); - InvokeModelResponse response = client.invokeModel(request); + var response = client.invokeModel(request); - ObjectNode responseBody = new ObjectMapper().readValue(response.body().asUtf8String(), ObjectNode.class); - - String completion = responseBody.get("completions").elements().next().get("data").get("text").asText(); - - return completion; + var responseBody = new ObjectMapper().readValue(response.body().asUtf8String(), ObjectNode.class); + return responseBody.get("completions").elements().next().get("data").get("text").asText(); } catch (AwsServiceException e) { System.err.println(e.awsErrorDetails().errorMessage()); diff --git a/src/main/java/io/clue2solve/aws/bedrock/springboot/starter/service/impl/LlamaService.java b/src/main/java/io/clue2solve/aws/bedrock/springboot/starter/service/impl/LlamaService.java index ffb38b3..879c042 100644 --- a/src/main/java/io/clue2solve/aws/bedrock/springboot/starter/service/impl/LlamaService.java +++ b/src/main/java/io/clue2solve/aws/bedrock/springboot/starter/service/impl/LlamaService.java @@ -25,24 +25,21 @@ public LlamaService(BedrockRuntimeClient client, LlamaProperties properties) { @Override public String invoke(String prompt) throws JsonProcessingException { try { - ObjectMapper mapper = new ObjectMapper(); - ObjectNode payload = mapper.createObjectNode(); + var mapper = new ObjectMapper(); + var payload = mapper.createObjectNode(); payload.put("prompt", properties.prompt() + prompt); payload.put("maxTokens", properties.maxTokens()); payload.put("temperature", properties.temperature()); - SdkBytes body = SdkBytes.fromUtf8String(payload.toString()); + var body = SdkBytes.fromUtf8String(payload.toString()); - InvokeModelRequest request = InvokeModelRequest.builder().modelId(properties.modelId()).body(body).build(); + var request = InvokeModelRequest.builder().modelId(properties.modelId()).body(body).build(); - InvokeModelResponse response = client.invokeModel(request); + var response = client.invokeModel(request); - ObjectNode responseBody = new ObjectMapper().readValue(response.body().asUtf8String(), ObjectNode.class); - - String completion = responseBody.get("completions").elements().next().get("data").get("text").asText(); - - return completion; + var responseBody = new ObjectMapper().readValue(response.body().asUtf8String(), ObjectNode.class); + return responseBody.get("completions").elements().next().get("data").get("text").asText(); } catch (AwsServiceException e) { System.err.println(e.awsErrorDetails().errorMessage());