CODEWITHSUNDEEP
kubernetesgoogle-kubernetes-enginekubeflowkubeflow-pipelinesseldon
kevin.w.johnson
kevin.w.johnson

Reputation: 1794

Kubeflow Pipeline in serving model

I'm beginning to dig into kubeflow pipelines for a project and have a beginner's question. It seems like kubeflow pipelines work well for training, but how about serving in production?

I have a fairly intensive pre processing pipeline for training and must apply that same pipeline for production predictions. Can I use something like Seldon Serving to create an endpoint to kickoff the pre processing pipeline, apply the model, then to return the prediction? Or is the better approach to just put everything in one docker container?

Upvotes: 1

Views: 667

Answers (2)

Ark-kun
Ark-kun

Reputation: 6787

KF Pipelines is designed for pipelines that run from start to finish. Serving process does not have an end, so, although possible, serving itself should be handled outside of a pipeline.

What the pipeline should do is to push a trained model to the long-lasting serving service in the end.

The serving can performed by CMLE serving, Kubeflow's TFServe, Seldon, etc.

Can I use something like Seldon Serving to create an endpoint to kickoff the pre processing pipeline, apply the model, then to return the prediction?

Due to container starting overhead, Kubeflow Pipelines usually handle batch jobs. Of course you can run a pipeline for a single prediction, but the latency might not be acceptable. For serving it might be better to have a dedicated long-lived container/service that accepts requests, transforms data and makes predictions.

Upvotes: 0

Gabriel Wen
Gabriel Wen

Reputation: 21

Yes, you can definitely use Seldon for serving. In fact, Kubeflow team offers an easy way to link between training and serving: fairing

Fairing provides a programmatic way of deploying your prediction endpoint. You could also take a look at this example on how to deploy your Seldon endpoint with your training result.

Upvotes: 2

Related Questions