CODEWITHSUNDEEP
databricksdelta-live-tables
dwolfeu
dwolfeu

Reputation: 1251

Databricks Delta Live Tables: Difference between STREAMING and INCREMENTAL

Is there a difference between CREATE STREAMING LIVE TABLE and CREATE INCREMENTAL LIVE TABLE? The documentation is mixed: For instance, STREAMING is used here, while INCREMENTAL is used here. I have tested both and so far I have not noticed any difference.

Upvotes: 11

Views: 7419

Answers (1)

Alex Ott
Alex Ott

Reputation: 87119

There are two aspects here:

  1. Conceptual - incremental means that the minimal data changes are applied to a destination table, we don't recompute full data set when new data arrive. This is how is explained in the Getting Started book.

  2. Syntax - CREATE INCREMENTAL LIVE TABLE was the original syntax for pipelines that were processing streaming data. But it was deprecated in the favor of CREATE STREAMING LIVE TABLE, but the old syntax is still supported for compatibility reasons.

Upvotes: 11

Related Questions