With Bedrock you get access a range of different large language models, ( for instance, Claude, Mistral, Llama and Amazon Titan) with new versions becoming available all the time.
Having choice is great, but having to code your requests differently for each model is a pain.
Here’s why the Amazon Bedrock Converse API is going to save you a bunch of time and effort, when comparing the output of different foundation models!
Consistency is key!
The Converse API is a consistent interface that works with
1) We can do everything using the CloudShell in the AWS console.
3) Download the file named converse_demo.py from
converse_demo.py
#first we import boto3 and json
import boto3, json
#create a boto3 session - stores config state and allows you to create service clients
session = boto3.Session()
#create a Bedrock Runtime Client instance - used to send API calls to AI models in Bedrock
bedrock = session.client(service_name='bedrock-runtime')
#here's our prompt telling the model what we want it to do, we can change this later
system_prompts = [{"text": "You are an app that creates reading lists for book groups."}]
#define an empty message list - to be used to pass the messages to the model
message_list = []
#here’s the message that I want to send to the model, we can change this later if we want
initial_message = {
"role": "user",
"content": [{"text": "Create a list of five novels suitable for a book group who are interested in classic novels."}],
}
#the message above is appended to the message_list
message_list.append(initial_message)
#make an API call to the Bedrock Converse API, we define the model to use, the message, and inference parameters to use as well
response = bedrock.converse(
modelId="anthropic.claude-v2",
messages=message_list,
system=system_prompts,
inferenceConfig={
"maxTokens": 2048,
"temperature": 0,
"topP": 1
},
)
#invoke converse with all the parameters we provided above and after that, print the result
response_message = response['output']['message']
print(json.dumps(response_message, indent=4))
4) Run the Python code like this:
python converse_demo.py
It should give you an output similar to this:
6) We can test again with another version:
anthropic.claude-3-5-sonnet-20240620-v1:0
7) I also tried it with a different model provider, by changing the model id to:
mistral.mistral-small-2402-v1:0
So the Converse API gives you a simple, consistent API, that works with all Amazon Bedrock models that support messages. And this means that you can write your code once and use it with different models to compare the results!
So next time you’re working with Bedrock, do yourself a favour, try out the Converse API, and thank me later!
SOCIAL SHARE CARD GENERATOR