CODEWITHSUNDEEP
pythonlarge-language-modelamazon-bedrock
Chedva
Chedva

Reputation: 171

How to call AWS Bedrock asynchronously

Is there a way to call Bedrock claude3 model with Python SDK asynchronously?

More specifically, I want the results to be sent to S3.

Upvotes: 2

Views: 1862

Answers (2)

Willem
Willem

Reputation: 1099

from anthropic import AsyncAnthropicBedrock

model_id = "my_model_id"
user_message = "Hello Claude!"

client = AsyncAnthropicBedrock()

message = await client.messages.create(
    model=model_id,
    max_tokens=1024,
    messages=[
        {"role": "user", "content": user_message}
    ]
)

For the S3 part you'll need to use boto3 to upload the files manually. This solution is just doing the async part. But hopefully it is already helpful. (Uploading to S3 should be relatively easy.)

Upvotes: 0

lsc
lsc

Reputation: 323

Batch inference is asynchronous, but it's in preview.

Upvotes: 1

Related Questions