Why does Bedrock and CrewAI produce incomplete responses?

Question

Im writing a crewAI backed by AWS Bedrock. application that generates code.

The issue Im seeing is that it produces incomplete responses. The code produce some python functions but will

1. Produce an incomplete function that is syntacticly incorrect.
2. Produce not enough code to compelete the applciation. i.e. it misses some needed functions or code in the implementation.

Im therefore trying to understand how bedrock and crewai works to get it to produce more complete answers.

Code snippets:

MODEL_ID = "anthropic.claude-3-5-sonnet-20240620-v1:0"
bedrock = boto3.client(service_name="bedrock-runtime", region_name="us-east-1")

docs_tool = DirectoryReadTool(directory=DIRECTORY_PATH)
file_tool = FileReadTool()

model_kwargs = {
        "max_tokens": 10000,
        "temperature": 0.1,
        "top_k": 250,
        "top_p": 1,
        "stop_sequences": ["\n\nHuman"],
    }


agent_llm = ChatBedrock(client=bedrock,model_id=MODEL_ID,model_kwargs=model_kwargs)

def developer(tools):
    """ An agent representing a senior developer"""
    agent = Agent(
        role='Senior application developer and team lead',
        goal='Make sure the code produced by your team is at the required standard',
        backstory=("You are a senior application developer"),
        memory=True,
        verbose=True,
        allow_delegation=True,
        llm=agent_llm,
        tools=tools
    )
    return agent

def dev_task(tools, agent):
    """ Tasks for writing unit tests"""
    task = Task(
        description = (
            "Review this code and refactor it to make it more readable"
            "do not return comments, feedback, breakdown or notes, just code."
        ),
        expected_output = ("A refactored code base that is more readable"),
        tools=tools,
        agent= agent
    )

    return task

crew = Crew(
        agents=agent,
        tasks=tasks, # Optional: Sequential task execution is default
    )
    return crew, tasks

Ive tried to use other models like llama3 and Mistral but they both seem incompatible with crewai, throwing errors. It appears claude anthropic is the only compatiable one. Ive tried using langchain but the vecotrisation takes too long when applying Rag on my machine. Not sure if there is a programmatic way to get bedrock to do it.

Is the tokenization too short? Am I using crewAI properly? Am I using Bedrock Properly? Is the code base Im providing too big? Im asking the Directory tool to analyse a directory.

Thanks for your thoughts.

Ive tried to use other models like llama3 and Mistral but they both seem incompatible with crewai, throwing errors. It appears claude anthropic is the only compatiable one. Ive tried using langchain but the vecotrisation takes too long when applying Rag on my machine. Not sure if there is a programmatic way to get bedrock to do it.