CODEWITHSUNDEEP
springspring-dataspring-integrationspring-batch
user1052610
user1052610

Reputation: 4719

Data Migration using Spring

We are beginning the process of re-architecting the systems within our company. One of the key components of the work is a new data model which better meets our requirements. A major part of the initial phase of the work is to design and build a data migration tool. This will take data from one or more existing systems and migrate it to the new model. Some requirements:

  1. Transformation of data to the new model
  2. Enrichment of data, with default values or according to business rules
  3. Integration with existing systems to pull data
  4. Integration with Salesforce CRM which is being introduced into the company.
  5. Logging and notification about failures

Within the Spring world, which is the best Spring project to use as the underlying framework for such a data migration tool?

My initial thoughts are to look at implementing the tool using Spring Integration. This would:

  1. Through the XML or DSL, allow for the high level data flow to be seen, understood, and edited (possibly using a visual tool such as a STS plugin). Being able to view the high level flow in such a way is a big advantage.
  2. Connectors to work with different data sources.
  3. Transformers components to be built to migrate data formats.
  4. Routers to route the data in the new model to endpoints which connect with systems.

However, are there other Spring projects, such as Spring Data or Spring Batch, which are a better match for the requirements?

Very much appreciate feedback and ideas.

Upvotes: 0

Views: 211

Answers (1)

Oleg Zhurakousky
Oleg Zhurakousky

Reputation: 6106

I would certainly start with spring-integration which exposes bare bones implementation for Enterprise Integration Patterns which are at the core of most/all of your requirements listed. It is also an exceptionally great problem modelling tool which helps you better understand the problem and then envision its implementation in one cohesive integration flow Later on, once you have a clear understanding of how things are working it would be extremely simple to take it to the next level by introducing the "other frameworks" you mentioned/tagged adding #spring-cloud-data-flow and #spring-cloud-stream.

Overall this question is rather broad, so consider following the above pointers and get started and raise more concrete questions.

Upvotes: 2

Related Questions