CODEWITHSUNDEEP
google-cloud-platformcontinuous-integrationgcloudgoogle-cloud-vertex-aikubeflow-pipelines
bayes2021
bayes2021

Reputation: 202

How do I create a Vertex AI Pipeline with the gcloud CLI?

As part of a CI/CD pipeline, I'd like to automatically deploy or update Vertex AI pipelines. However, I couldn't figure out how to create a Vertex AI pipeline with the gcloud CLI-tool.

Using the Python SDK I have two files: one in which the pipeline is defined and compiled (vertex_pipeline.py), and another one (submit_to_vertex.py) in which the compiled pipeline (greet_pipeline.json) is used to create or submit the pipeline to Vertex AI.

These files look as follows:

# vertex_pipeline.py
from kfp import dsl
from kfp.compile import Compiler

@dsl.component(base_image=IMAGE)
def greet(name: str):
    print(f"Hi {name=}. How are you today?")

@dsl.pipeline(name="greet_pipeline"):
def greet_pipeline(people_to_greet: str = "Tom"):
    greet_task=greet(name=people_to_greet)


if __name__ == "__main__":
    Compiler().compile(pipeline_func=greet_pipeline, package_path="greet_pipeline.json")


# submit_to_vertex.py
import google.cloud.aiplatform as aiplatform

PROJECT_ID=...
LOCATION=...
...

aiplatform.init(project=PROJECT_ID,location=LOCATION)

pipeline_=aiplatform.pipeline_jobs.PipelineJob(...)
pipeline_.submit(service_account=SERVICE_ACCOUNT)

What is the equivalent of the second file with gcloud? I've read the documentation of gcloud but couldn't find anything related.

Upvotes: 0

Views: 494

Answers (1)

Khatuntcov Mikhail
Khatuntcov Mikhail

Reputation: 1

As of today (2024-09-19) gcloud doesn't have this functionality.

However you could use pure API to perform such things (see orginal example Using the Pipelines REST API): https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/bb2c8c502d281f49fb5b5fa2a40fada0c8b66144/notebooks/official/pipelines/pipelines_intro_kfp.ipynb

Upvotes: 0

Related Questions