CODEWITHSUNDEEP
machine-learninglarge-language-modelgoogle-cloud-vertex-aiquotagoogle-gemini
Ben
Ben

Reputation: 33

VertexAIException - list index out of range Error when calling Gemini-Pro API

I am calling Google Gemini-Pro API in a consecutive manner (like about 50 queries per minute). I believe I have properly set my VertexAI project and credentials. When the number of consecutive queries I used was below a constant bar, the queries would run through and the responses would be received just fine. However, once the number of queries increased over the aforementioned bar, the following error would show up:

IndexError - list index out of range

Note that the number of queries "bar" over which this error will occur depend on the length of each query and is consistent if the length of the queries stays the same across program executions. For example, after trying to increase my query length by rougly 20%, the bar dropped from roughly 330 queries to roughly 60 queries.

File "/Users/user/anaconda3/envs/chat1/lib/python3.11/site-packages/vertexai/generative_models/_generative_models.py", line 1315, in text return self.candidates[0].text ~~~~~~~~~~~~~~~^^^ IndexError: list index out of range

What is causing this? I have set the VertexAI server location to be: "us-central1", which should only have a quota of 300 queries/minute as far as I know. Since I am doing the API call consecutively but below the rate of 60 queries/minute, I think I am in the ok zone for usage. I am currently using the free VertexAI trial account (with 300 USD free credit).

The Gemini Pro API Call function that I have written is:

def gemini_response(message: str) -> str:
    # Initialize Vertex AI
    vertexai.init(project="project-id-0123", location="us-central1")

    # Load the model
    model = GenerativeModel("gemini-pro")

    # Query the model
    response = model.generate_content(message)
    return response.text

When debugging what's wrong with the candidates variable, the variable inspection results look like:

> self 
> prompt_feedback {block_reason: OTHER} 
> usage_metadata {prompt_token_count: 505   total_token_count: 505 } 

> self.candidates 
> []

> self._raw_response 
> prompt_feedback {block_reason: OTHER}
> usage_metadata {prompt_token_count: 505   total_token_count: 505 }

Upvotes: 3

Views: 1218

Answers (2)

Ark-kun
Ark-kun

Reputation: 6787

prompt_feedback {block_reason: OTHER}

This means that you input prompt was blocked (rather than output response or response candidates). The block reason is not specified here. It's expected that those prompts will always be blocked. Please report the concrete prompts that are being blocked to https://github.com/googleapis/python-aiplatform/issues or https://issuetracker.google.com/issues?q=status:open%20componentid:571646&s=created_time:desc

Upvotes: 1

Prisoner
Prisoner

Reputation: 50701

This suggests there are no candidates that were returned by the prompt.

If other runs of the same prompt work, then this is likely due to rate limiting, even if you think you are under the limit. You can check this in the Cloud Console API dashboard, but all these numbers are rough approximations.

If this prompt never works, then it is likely from the safety system flagging the prompt, possibly in error.

In either case, it can be useful to look at the JSON representation of the response object to see exactly what has been returned and check out the fields as appropriate. If this does indicate a rate error, then you should be able to try again in short order.

Upvotes: 0

Related Questions