Aryan Orpe
Aryan Orpe

Reputation: 1

Getting error AttributeError: 'SagemakerEndpoint' object has no attribute 'bind_tools' when building LangChain/LangGraph Agent

I'm getting this error AttributeError: 'SagemakerEndpoint' object has no attribute 'bind_tools' when building a Gen AI agent with LangChain/LangGraph/Deployed LLM on AWS Sagemaker Endpoint.

I'm trying to build a Gen AI Agent using a model deployed on AWS Sagemaker Endpoint but I'm getting this error AttributeError: 'SagemakerEndpoint' object has no attribute 'bind_tools'.

Using the prebuilt create_react_agent function from LangGraph. I was expecting this to work since it was working with LangChain ChatBedrock but this might be since it is a 'chat' model and SageMakerEndpoint is a 'llm' model which is older.

Am I building the agent the right way and if not how should I do it using a deployed model on AWS Sagemaker Endpoint?

Minimal Reproducible Example:

import json
from typing import Dict

import boto3
from langchain_community.llms import SagemakerEndpoint
from langchain_community.llms.sagemaker_endpoint import LLMContentHandler
from langchain_core.messages import HumanMessage
from langgraph.prebuilt import create_react_agent
from langchain_core.tools import tool

roleARN = "arn:aws:iam::123456789:role/cross-account-role"
sts_client = boto3.client("sts")
response = sts_client.assume_role(
    RoleArn=roleARN, RoleSessionName="CrossAccountSession"
)

client = boto3.client(
    "sagemaker-runtime",
    region_name="us-west-2",
    aws_access_key_id=response["Credentials"]["AccessKeyId"],
    aws_secret_access_key=response["Credentials"]["SecretAccessKey"],
    aws_session_token=response["Credentials"]["SessionToken"],
)


class ContentHandler(LLMContentHandler):
    content_type = "application/json"
    accepts = "application/json"

    def transform_input(self, prompt: str, model_kwargs: Dict) -> bytes:
        input_str = json.dumps({"inputs": prompt, "parameters": model_kwargs})
        return input_str.encode("utf-8")

    def transform_output(self, output: bytes) -> str:
        response_json = json.loads(output.read().decode("utf-8"))
        return response_json[0]["generated_text"]


content_handler = ContentHandler()

model = SagemakerEndpoint(
    endpoint_name=<endpoint-name>,
    client=client,
    model_kwargs={"temperature": 1e-10},
    content_handler=content_handler,
)

tools = [<enter-tools>]

agent_executor = create_react_agent(model, tools)

agent_executor.invoke({"messages": [HumanMessage(content=<message>)]})

Upvotes: 0

Views: 108

Answers (0)

Related Questions