CODEWITHSUNDEEP
azure-databricks
user13128577
user13128577

Reputation:

Do Databricks Tables Support Transaction Isolation Levels

I am working on databricks tables in Azure databricks service, however, it looks to me that databricks tables do not support transaction isolation levels? What happens when a table is being updated/deleted/inserted while another process accessing(reading/modifying) the same table?

Upvotes: 1

Views: 1496

Answers (1)

CHEEKATLAPRADEEP
CHEEKATLAPRADEEP

Reputation: 12768

Azure Databricks table schema is immutable.

Delta Lake on Azure Databricks supports two isolation levels: Serializable and WriteSerializable.

Delta Lake provides ACID transaction guarantees between reads and writes. This means that:

  • Multiple writers across multiple clusters can simultaneously modify a table partition and see a consistent snapshot view of the table and there will be a serial order for these writes.
  • Readers continue to see a consistent snapshot view of the table that the Azure Databricks job started with, even when a table is modified during a job.

The isolation level of a table defines the degree to which a transaction must be isolated from modifications made by concurrent transactions. Delta Lake on Azure Databricks supports two isolation levels: Serializable and WriteSerializable.

Serializable: The strongest isolation level. It ensures that committed write operations and all reads are Serializable. Operations are allowed as long as there exists a serial sequence of executing them one-at-a-time that generates the same outcome as that seen in the table. For the write operations, the serial sequence is exactly the same as that seen in the table’s history.

WriteSerializable (Default): A weaker isolation level than Serializable. It ensures only that the write operations (that is, not reads) are serializable. However, this is still stronger than Snapshot isolation. WriteSerializable is the default isolation level because it provides great balance of data consistency and availability for most common operations.

In this mode, the content of the Delta table may be different from that which is expected from the sequence of operations seen in the table history. This is because this mode allows certain pairs of concurrent writes (say, operations X and Y) to proceed such that the result would be as if Y was performed before X (that is, serializable between them) even though the history would show that Y was committed after X. To disallow this reordering, set the table isolation level to be Serializable to cause these transactions to fail.

For more information on which types of operations can conflict with each other in each isolation level and the possible errors, see Concurrency control.

For more details, refer "Azure Databricks - Isolation levels".

Upvotes: 1

Related Questions