CODEWITHSUNDEEP
databricksazure-databricksdelta-lake
user3661384
user3661384

Reputation: 544

DataBricks DELTA VACUUM

I'm trying to delete historical data from DELTA using the VACUUM command but it doesn't do anything

I ran the DRYRUN command to show which files have to be deleted, but nothing comes back, but looking at the JSON file in the delta folder is already in time to erase the data.

I ran this command to delete data, but without success, and analyzing the JSON timestamp it would be in the delete time. Am I doing something wrong?

%sql

delta VACUUM.`/mnt/deltaTestVacuum/myTable/`

Json with timestamp remove inside in directory _delta_log enter image description here

Command DRY RUN enter image description here

Upvotes: 2

Views: 3792

Answers (1)

Donghwa Kim
Donghwa Kim

Reputation: 11

What is the retention internal you have for the table? See: https://docs.delta.io/latest/delta-utility.html#remove-files-no-longer-referenced-by-a-delta-table

You can remove files no longer referenced by a Delta table and are older than the retention threshold by running the vacuum command on the table.

Upvotes: 1

Related Questions