CODEWITHSUNDEEP
apache-sparkpysparkapache-spark-sqldatabricksdelta-lake
uvan
uvan

Reputation: 55

Auto increment column in delta table without rekey

I have a dim delta table so far i am calculating dim_id using row_number() + max(dim_id).

Dim_id | user_id

1001   |   1
1002   |   3
1003   |   5
1004   |   9

For example if i deleted 1004 id then insert a new user_id like 7 (row_number() + max(dim_id) = 1004) 1004 id repeated. Is there any way to prevent already used ids not created once it deleted from the delta table

Upvotes: 0

Views: 411

Answers (1)

Kevin Lee
Kevin Lee

Reputation: 96

  • The perfect way to solve this is Primary Key, yet not support until now.
  • You can combine monotonically_increasing_id() with row_number() for two columns. Here is the example: Generate unique increasing values

RD：

  1. monotonically_increasing_id()
  2. Constraints on Databricks

Upvotes: 2

Related Questions