CODEWITHSUNDEEP
apache-sparkapache-spark-sqldatabricksdelta-lake
Sandie
Sandie

Reputation: 987

What are databricks spark delta tables? Does they also stores data for a specific session and how can I view these delta tables and their structure

What is the purpose of spark delta tables? Does they meant to store data permanently or only holds the processing data till the session lasts. How can I view them in spark cluster and what database they belongs to.

Upvotes: 2

Views: 1055

Answers (1)

user10249612
user10249612

Reputation: 31

What is the purpose of spark delta tables?

The primary goal is to enable single table transnational writes in multicluster setups. This is achieved by keeping a transaction log (idea very similar to append-only tables in typical database systems).

Does they meant to store data permanently or only holds the processing data till the session lasts.

There are persistent, and by definition scoped across the sessions.

. How can I view them in spark cluster and what database

Same as any other table in Spark. There are not specific to any database, and written using delta format.

Upvotes: 3

Related Questions