Monolith to microservice

Question

We are planning to migrate from monolith to micro-services based architecture. Now i own the responsibility of talking a module out of monolith.

Existing Monolith:

1) Code is very tightly coupled.

2) APIs are called recursively with different parameters.

3) Some of the calls with-in the module which i am planning to extract out contains calls to a system which takes approx 9 minutes to complete. Unfortunately that's a synchronous.

Points to note:

1) I am starting with a single api migration which is a very important one and is not performing well. 2) This api consists of parallel calls to another system for performing bunch of tasks. All the calls are blocking and time-consuming (consider avg response time to be 5-6 min)

Moving to microservice based architecture : There are 2 approaches that comes to my mind while moving the aforementioned api from monolith to a separate microservice, along with solving the problem of blocking threads due to time taking blocking calls.

a) moving in phases :

 - Create a separate module 
 - In this module provide an api to push events to kafka, another 
   module will in-turn process the request and push the response back 
   to kafka
 - monolith for now will call above mentioned api to push events to 
   kafka
 - New module will inturn call back the monolith when the task 
   complete (received response on a separate topic in kafka)
 - Monolith once get response for all the tasks will trigger some post 
   processing activity.

 Advantage:
 1) It will solve the problem of sync- blocking call.

 Disadvantage:
 1) Changes are required in the monolith, which could introduce some 
    bugs.
 2) No fallbacks are available for the case if bug gets introduced.

b) Move the API at once to the microservice :

• Initially which will share common data source with the monolith and solve the problem of blocking calls via introduction of kafka between new microservice and the module which takes time to process the request.

  Advantage:

  1) Fallback is available in monolith

  Disadvantage:

  1) Initially data source is shared between the systems.

What should be the best approach to do these kinds of complex tasks ?

Answer 1

I believe the best approach would be moving option 1: Moving in phases. However, it is possible to do it while having a fallback strategy. You can keep the a version of the untouched backend to serve as a backup if your new service encounters issues.

The approach is described in more details in the article: Low risk monolith to microservice evolution It provides more details in the implementation and the thought processes behind why a phased approach has lower risk. However, the need to change the backend would still be present, but hopefully mitigated through unit testing.

Answer 2

You have to take care of several things.

First

Going to microservice will be slower (90% of the time) than a monolith because you introduce latency. So never forget it when you go with it.

Second

You ask if it is a good way to go with kafka. I may answer yes in most of the case but you mentioned that today the process is synchronous. If it is for transactional reasons, you won't be able to solve it with a message broker I guess because you'll update your strong consistency system to an eventually one. https://en.wikipedia.org/wiki/Eventual_consistency

I am not saying that it is a bad solution only that it change your workflow and may impact some business rules.

As a solution I offer this:

1 - Break the seams in your monolith by introducing functional key and api composition inside the monolith (read Sam Newman's book to help).

2 - Introduce the eventual consistency inside the monolith to test if it fits the purpose. It will be easier to rollback if not.

No you have to possibility:

The second step went well so go ahead and put the code of the service into a microservice out of the monolith.

The second step did not fit then think about doing the risky thing in a specific service or use distributed transactions (be careful with this solution it could be hard to manage).