CODEWITHSUNDEEP
apache-sparkdatabricksdelta-lake
Ged
Ged

Reputation: 18098

Spark Delta format on non-Databricks platforms

To improve query speed, Delta Lake on Databricks supports the ability to optimize the layout of data stored in cloud storage. Delta Lake on Databricks supports two layout algorithms: bin-packing and Z-Ordering.

If you run on-prem (not in the cloud) and use the delta format with Spark, thus not on Databricks, can the Z-Ordering be used? Or is it only available on Databricks run-time?

My assumption is yes, but just want to be really clear as I do not have RHEL cluster at hand.

Upvotes: 2

Views: 225

Answers (1)

Betta
Betta

Reputation: 416

The z-ordering is supported only in the Delta Lake on Databricks Runtime.

Update Delta Lake 2.0 has announced the support of Z-Ordering

Upvotes: 2

Related Questions