Can Apache Ignite update when 3rd party SQL Server database changed something directly?

Question

Can I get some advice on whether it is possible to proceed like the steps below?

• SQL Server data is loaded in Ignite Cluster
• The data in SQL Server has been changed.

-> Is there any other way to reflect this changed data without reloading the data from SQL Server?

When used as a cache in front of the database, when changes are made directly to the DB without going through the Ignite Cluster, can the already loaded cache data be directly reflected in the Ignite cache? Is it possible to set only the value to change without loading the data again?

If possible, which part should I set? Please.

Answer 1

I suppose the real question is - how to propagate changes applied to SQL Server first to the Apache Ignite cluster. And the short answer is - you need to do it by yourself, i.e. you need to implement some synchronization logic between the two databases. This should not be a complex task if most of the data updates come through Ignite and SQL Server-first updates are rare.

As for the general approach, you can check for the Change Data Capture (CDC) pattern implementations. There are multiple articles on how you can achieve it using external tools, for sample, CDC Between MySQL and GridGain With Debezium or this video.

It's worth mentioning that Apache Ignite is currently working on its own native implementation of CDC.

Answer 2

Take a look at Ignite's external storage integration, and the read/write through features. See: https://ignite.apache.org/docs/latest/persistence/external-storage
and https://ignite.apache.org/docs/latest/persistence/custom-cache-store

examples here: https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples/datagrid/store