Alternative to AWS lambda serverless concept in Kubernetes?

Question

I am new to devops world and have a doubt in Kubernetes. My usecase is as follows.
My organization is currently hosting all its microservices in Docker containers using Kubernetes container orchestration platform.

How I want my microservice container to be scaled and automated using Kubernetes:

I have 100s of AI models stored in Amazon S3 bucket. When a request comes from user1 to the container with a particular model name, corresponding model from the S3 should be fetched and loaded in the container memory then user1 should start the inference. Similarly if another request comes from user2 with another model name, new container should spin off and user2 should start the inference. For n model requests, n containers should spin off and serve each session. Containers should automatically scale up with different requests.

Since the above logic is similar to how AWS Lambda function works, I was recommending the same to our devops team. "They informed that they cannot use AWS Lambda for a single microservice since it is serverless and all other microservices are non-serverless which runs in Kubernetes"

My question is, can the same concept be implemented with Kubernetes?. If so please suggest a method or doc.

Thanks in advance.

Answer 1

MicroFunctions is open-source Kubernetes Native Serverless platform bthat lets you deploy small bits of code without having to worry about the underlying infrastructure plumbing. It leverages Kubernetes resources to provide auto-scaling, API routing, monitoring, troubleshooting and supports every programming language. (Nodejs,Go,python,..).

https://github.com/microfunctionsio/microfunctions

Answer 2

Check Knative project. It has serverless workload management (serving component) and also eventing component. https://knative.dev

It had been adopted by many other projects. A few of the commercial project are listed here https://knative.dev/docs/knative-offerings/

Knative is widely adopted and also has good open source community ( active contribution, etc)

Answer 3

Have you looked at Kubeless https://kubeless.io/?

I haven't but something I'd like to play with at some point. Here's a quote from their site:

Kubeless is a Kubernetes-native serverless framework that lets you deploy small bits of code (functions) without having to worry about the underlying infrastructure. It is designed to be deployed on top of a Kubernetes cluster and take advantage of all the great Kubernetes primitives. If you are looking for an open source serverless solution that clones what you can find on AWS Lambda, Azure Functions, and Google Cloud Functions, Kubeless is for you!

Answer 4

I would use KFServing for serverless inference on k8s. You can install it as a standalone component or you can run it with the full Kubeflow toolkit. You can inject your models from s3 into the serving containers which are optimized for whichever ML framework or language you might use. It uses Knative & Istio under the hood in order to achieve the "scale-to-zero" functionality.