CODEWITHSUNDEEP
python-3.xkubeflow-pipelinesgoogle-cloud-vertex-ai
Optimus
Optimus

Reputation: 1604

Optionally use component functions added in VertexAI python SDK

I am using vertex ai's python SDK and it's built on top of Kubeflow pipelines. In it, you supposedly can do this:

train_op = (sklearn_classification_train(
        train_data = data_op.outputs['train_out']
    ).
    set_cpu_limit(training_cpu_limit).
    set_memory_limit(training_memory_limit).
    add_node_selector_constraint(training_node_selector).
    set_gpu_limit(training_gpu_limit)
)

where you can add these functions (set_cpu_limit, set_memory_limit, add_node_selector, and set_gpu_limit) onto your component. I've haven't used this syntax before.

How I can optionally use each 'sub function' only if the variables are specified each function?

For example, if training_gpu_limit isn't set, I don't want to execute set_gpu_limit on the component.

Upvotes: 0

Views: 165

Answers (1)

Optimus
Optimus

Reputation: 1604

These functions are not appended to the function, but to the component.

In your code, if you do print(type(train_op)) it will be an object type component. So there is no way to add parameters to the function which will influence behavior of the component.

You can do it in pipeline function by changing code to:

train_op = sklearn_classification_train(train_data = data_op.outputs['train_out'])

if training_cpu_limit:
    train_op.set_cpu_limit(training_cpu_limit)

# etc

Upvotes: 1

Related Questions