CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaaws-sdk-jsamazon-bedrock
AndrewB
AndrewB

Reputation: 836

Why does the InvokeModelCommand for AWS Bedrock Cohere model return numbers and not text?

I have a file called bedrock.mjs in AWS Lambda with the below code:

import {
    BedrockRuntimeClient,
    InvokeModelCommand,
} from "@aws-sdk/client-bedrock-runtime"

export const callBedrock = async (prompt, max_tokens, temperature) => {
    const client = new BedrockRuntimeClient()

    const input = {
        modelId: "cohere.command-text-v14",
        contentType: "application/json",
        accept: "application/json",
        body: JSON.stringify({
            prompt,
            max_tokens,
            temperature
        })
    }

    const command = new InvokeModelCommand(input)

    const response = await client.send(command)

    return response
}

When I call callBedrock("Respond with a nice greeting", 100, 0.2), the body from the response is:

{\"0\":123,\"1\":34,\"2\":103,\"3\":101,\"4\":110,\"5\":101,\"6\":114,\"7\":97,\"8\":116,\"9\":105,\"10\":111,\"11\":110,\"12\":115,\"13\":34,\"14\":58,\"15\":91,\"16\":123,\"17\":34,\"18\":105,\"19\":100,\"20\":34,\"21\":58,\"22\":34,\"23\":98,\"24\":98,\"25\":101,\"26\":54,\"27\":56,\"28\":49,\"29\":54,\"30\":101,\"31\":45,\"32\":48,\"33\":99,\"34\":49,\"35\":50,\"36\":45,\"37\":52,\"38\":50,\"39\":53,\"40\":99,\"41\":45,\"42\":56,\"43\":56,\"44\":101,\"45\":52,\"46\":45,\"47\":98,\"48\":102,\"49\":49,\"50\":51,\"51\":49,\"52\":101,\"53\":98,\"54\":56,\"55\":49,\"56\":49,\"57\":100,\"58\":50,\"59\":34,\"60\":44,\"61\":34,\"62\":116,\"63\":101,\"64\":120,\"65\":116,\"66\":34,\"67\":58,\"68\":34,\"69\":32,\"70\":72,\"71\":101,\"72\":108,\"73\":108,\"74\":111,\"75\":33,\"76\":32,\"77\":72,\"78\":111,\"79\":119,\"80\":32,\"81\":99,\"82\":97,\"83\":110,\"84\":32,\"85\":73,\"86\":32,\"87\":104,\"88\":101,\"89\":108,\"90\":112,\"91\":32,\"92\":121,\"93\":111,\"94\":117,\"95\":32,\"96\":116,\"97\":111,\"98\":100,\"99\":97,\"100\":121,\"101\":63,\"102\":34,\"103\":125,\"104\":93,\"105\":44,\"106\":34,\"107\":105,\"108\":100,\"109\":34,\"110\":58,\"111\":34,\"112\":99,\"113\":97,\"114\":100,\"115\":99,\"116\":57,\"117\":50,\"118\":102,\"119\":51,\"120\":45,\"121\":53,\"122\":50,\"123\":102,\"124\":97,\"125\":45,\"126\":52,\"127\":56,\"128\":101,\"129\":56,\"130\":45,\"131\":98,\"132\":100,\"133\":57,\"134\":54,\"135\":45,\"136\":48,\"137\":57,\"138\":57,\"139\":57,\"140\":102,\"141\":98,\"142\":55,\"143\":56,\"144\":53,\"145\":97,\"146\":101,\"147\":101,\"148\":34,\"149\":44,\"150\":34,\"151\":112,\"152\":114,\"153\":111,\"154\":109,\"155\":112,\"156\":116,\"157\":34,\"158\":58,\"159\":34,\"160\":82,\"161\":101,\"162\":115,\"163\":112,\"164\":111,\"165\":110,\"166\":100,\"167\":32,\"168\":119,\"169\":105,\"170\":116,\"171\":104,\"172\":32,\"173\":97,\"174\":32,\"175\":110,\"176\":105,\"177\":99,\"178\":101,\"179\":32,\"180\":103,\"181\":114,\"182\":101,\"183\":101,\"184\":116,\"185\":105,\"186\":110,\"187\":103,\"188\":34,\"189\":125}

How is this supposed to be interpreted or used?

Upvotes: 3

Views: 2149

Answers (2)

Extender
Extender

Reputation: 260

Just in case someone needs complete example:

    import { BedrockRuntime } from "@aws-sdk/client-bedrock-runtime";
    import dotenv from "dotenv";
    dotenv.config({ override: true });
    
    const textDecoder = new TextDecoder("utf-8");
    export const getTextClaude = async (prompt, temperature) => {
        const bedrock = new BedrockRuntime({
            credentials: {
                accessKeyId: process.env.AWS_ACCESS_KEY,
                secretAccessKey: process.env.AWS_SECRET_KEY,
            },
            region: "eu-central-1",
        });
    
        const params = {
            modelId: "anthropic.claude-v2:1",
            contentType: "application/json",
            accept: "application/json",
            body: JSON.stringify({
                prompt: `\n\nHuman:\n  ${prompt}\n\nAssistant:\n`,
                max_tokens_to_sample: 2048,
                temperature: temperature || 0.5,
                top_k: 250,
                top_p: 1,
                stop_sequences: ["\\n\\nHuman:"],
            }),
        };
    
        const data = await bedrock.invokeModel(params);
    
        if (!data) {
            throw new Error("AWS Bedrock Claude Error");
        } else {
            const response_body = JSON.parse(textDecoder.decode(data.body));
            return response_body.completion;
        }
    };

Upvotes: 2

Ermiya Eskandary
Ermiya Eskandary

Reputation: 23682

The body is a Uint8Array, which you can convert to text using the TextDecoder:

const response = await client.send(command)

let decoder = new TextDecoder();
let text = decoder.decode(response.body);

console.log(text);

For your input, this returns:

{
  "generations": [
    {
      "id": "f7652e8a-b50b-4251-82c4-cf346a997075",
      "text": " Hello! How can I help you today?"
    }
  ],
  "id": "6d267bf5-35fa-4d08-9083-70595f6fdbab",
  "prompt": "Respond with a nice greeting"
}

This should be based on the mime type (accept) but any other mime type other than application/json for this model throws the error:

ValidationException: The provided Accept Type is invalid or not supported for this model

Upvotes: 6

Related Questions