CODEWITHSUNDEEP
apiarchitecturebiztalk
Robert Persson
Robert Persson

Reputation: 309

How to architect scheduled API to API integration

My organization moves data for customers between systems, these integrations are in BizTalk and are done by file, sometimes to/from APIs. More and more customers are switching to APIs so we are facing more and more API to API integrations.

I'm mostly a backend developer but have been tasked with finding out how we can find a more generic pattern or system to make these integrations, we are talking close to a thousand of integrations. But not thousands of different APIs, many customers use the same sort of systems.

What I want is a solution that:

  1. Fetches data from the source api
  2. Transforms the data to the format for the target api
  3. Sends the data to the target api

Another requirement is that it should be possible to set a schedule when these jobs should run. This is easily done in BizTalk but as mentioned there will be thousands of integrations and if we need to change something in one of the steps it will be a lot of work.

My vision is something that holds interfaces to all APIs that we communicate with and also contains the scheduled jobs we want to be run between them. Preferrably with logging/tracking.

There must be something out there that does this? Suggestions?

NOTE: No cloud-based solutions since they are not allowed in our organization.

Upvotes: 1

Views: 270

Answers (1)

Maxim Fateev
Maxim Fateev

Reputation: 6890

You can easily implement this using temporal.io open source project. You can code your integrations using a general-purpose programming language. Temporal ensures that the integration runs to completion in the presence of all sorts of intermittent failures. Scheduling is also supported out of the box.

Disclaimer: I'm a founder of the Temporal project.

Upvotes: 3

Related Questions