CODEWITHSUNDEEP
pythongoogle-bigquerygoogle-cloud-vertex-aigoogle-gemini
Linda Lawton - DaImTo
Linda Lawton - DaImTo

Reputation: 117166

Get batch predictions for Gemini - Vertex AI returns unsupported

I have been chasing my tail all day trying to get Get batch predictions for Gemini to work.

I have been digging around in all the samples and documentation that i can find and have not been able to get this to work. From what i can tell the library supports this yet. I am faced with this error.

grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with: status = StatusCode.UNIMPLEMENTED details = "Received http2 header with status: 404" debug_error_string = "UNKNOWN:Error received from peer {grpc_message:"Received http2 header with status: 404", grpc_status:12, created_time:"2024-06-05T17:00:46.9053534+00:00"}"

Any help would be greatly apricated

async def sample_create_batch_prediction_job():
    input_config = aiplatform_v1.types.BatchPredictionJob.InputConfig(
        bigquery_source=aiplatform_v1.types.BigQuerySource(
            input_uri="bq://sample.text_input"
        )
    )
    output_config = aiplatform_v1.types.BatchPredictionJob.OutputConfig(
        bigquery_destination=aiplatform_v1.types.BigQueryDestination(
            output_uri="bq://sample.llm_dataset.embedding_out_BP_sample_publisher_BQ_20230712_134650"
        )
    )
    # https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/batch-prediction-api
    batch_prediction_job_1 = BatchPredictionJob(
        name=os.getenv("PROJECT_ID"),
        display_name='my-batch-prediction-job',
        model=f"projects/{os.getenv("PROJECT_ID")}/locations/us-central1/models/gemini-1.0-pro-001",
        input_config=input_config,
        output_config=output_config,
        )

    # Create a client
    client = aiplatform_v1.JobServiceAsyncClient()

    request = aiplatform_v1.CreateBatchPredictionJobRequest(
        parent=f"projects/{os.getenv("PROJECT_ID")}/locations/us-central1",
        batch_prediction_job=batch_prediction_job_1,
    )

    # Make the request
    response = await client.create_batch_prediction_job(request=request)

    # Handle the response
    print(response)

I thought it might be the big query URLs i tried adding the table ID for my own tables but there is no change. The documentation doesn't actually state how or what to pass for these big query URLs. Nor what the data model should look like.

Upvotes: 3

Views: 1292

Answers (1)

Linda Lawton - DaImTo
Linda Lawton - DaImTo

Reputation: 117166

With some help from #3871 I got this working however the docs say that 1.5 is working with this it does not it returns an error.

import os
import time
import vertexai
from vertexai.preview.batch_prediction import BatchPredictionJob
import asyncio
from dotenv import load_dotenv

load_dotenv()

vertexai.init(project=os.getenv("PROJECT_ID"), location=os.getenv("PROJECT_LOCATION"))

job = BatchPredictionJob.submit(
    source_model=os.getenv("MODEL_ID"),  # source_model
    input_dataset=f"bq://{os.getenv("BQ_TABLE_ID")}",  # input_dataset
    output_uri_prefix=f"bq://{os.getenv("BQ_OUTPUT_DATASET")}"  # Optional, output_uri_prefix
)

# Check job status
print(f"Job resouce name: {job.resource_name}")
print(f"Model resource name with the job: {job.model_name}")
print(f"Job state: {job.state.name}")

# Refresh the job until complete
while not job.has_ended:
    time.sleep(5)
    job.refresh()

# Check if the job succeeds
if job.has_succeeded:
    print("Job succeeded!")
else:
    print(f"Job failed: {job.error}")

# Check the location of the output
print(f"Job output location: {job.output_location}")

# List all the GenAI batch prediction jobs under the project
for bpj in BatchPredictionJob.list():
    print(f"Job ID: '{bpj.name}', Job state: {bpj.state.name}, Job model: {bpj.model_name}")

Upvotes: 3

Related Questions