Converting my Node app to Docker/Kubernetes?

Question

I have a Node app that's composed of several independent modules, talking to each other over AMQP. This app is started by an index.js file that instatiates every other index.js in the different folders of my project, which in turn instantiate the actual modules.

I've been reading about microservices and I'd like to convert my application to Docker containers. I've found examples of how to convert a simple Node app to Docker, but I want to separate my app so that every module is in an independent Node container (this is because each module is independent and doesn't depend on other modules to work. They receive work from the message queue, and put results on the message queue).

What I can't find is information on how I should organize and deploy my code. Should I have a different Node project (with separate packages.json) for each of my modules? Or should I have a single Node project for all my modules and deploy each of them individually?

After I have my project organized, is there a script tool that will generate (build* and deploy) each of my modules to its own container? All examples I've found so far are "hello world" level samples that just pack one app.

During development, will I have to deploy new containers for each change I test?

*: build because I use ES6 and I have to use Babel.

Answer 1

Should I have a separate packages.json for each Dockerfile? From what I've read, I understand the Dockerfile will actually run NPM and download the actual packages. If I have a single packages file I would have a lot of bloat in each of my images, with unused modules.

Sure you will have bloat, that's the beauty of Microservice, and That's the curse of Microservice.

Should I have a separate packages.json?

Okay, let me try to address this question specifically. Let's just say your moduleA uses lodash.x.x.x. And you would like to use lodash.x+1.x.x in moduleB. You are sure that lodash.x+1.x.x is not backward compatible with lodash.x.x.x. So, now you are forced to make ModuleA's code compatible with lodash.x+1.x.x. If the above said sounds true, we call those applications as Monoliths, not microservices. To answer your question, yes, you might need separate packages.json unless you can have a parent package.json with common dependencies and submodules with it's own package.json (I'm not from node-land, so not sure whether package.json have such capabilities)

What I can't find is information on how I should organize and deploy my code. Should I have a different Node project (with separate packages.json) for each of my modules? Or should I have a single Node project for all my modules and deploy each of them individually?

I have seen both the patterns (separate projects as well as submodules in same project). My personal opinion (for what it's worth) is to have separate project and separate code base, because your "modules" are talking over a language-agnostic protocol-AMQP already. Tomorrow, you might want to use golang/kotlin for "ModuleB" (microserviceB, depending on how you look at it :)), who knows.

After I have my project organized, is there a script tool that will generate (build* and deploy) each of my modules to its own container?

One thing to clarify here is, development and deployment of one microservice should not force you to change/deploy other microservices as long as the contract between them is kept intact, otherwise it is a distributed-monolith (whoa..did I just invent a new term? fancy!!). When there is a contract change you should bump the version and have both the version run for a while until all the concerned parties (your other microservices) get a chance to upgrade.

Answer 2

There are several layers that you need to pass.

First, you need to containerize each microservice into its own container image. Usually you will use Docker for that.

Each app will have a separate Dockerfile which you will use to create docker images which you will push to docker image registries which will be pulled by whoever wants to run your app.

As for the codebase organisation, you can have one repository with all your microservices, but you will need multiple Dockerfiles to create images for all of them.

Then you need to orchestrate your containers created from your images (run them). If you want your app to run on one host you may get away with Docker compose, which lets you define which containers are run in what order in single yaml manifest file. Docs are here: https://docs.docker.com/compose/

If you want to run your app in a kubernetes cluster, you will want to create a k8s Deployment (https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) which will run your containers as pods on your cluster nodes.