CODEWITHSUNDEEP
apache-spark-sqlazure-synapsedelta-lake
Steve Platz
Steve Platz

Reputation: 2263

Add a column to a delta table in Azure Synapse

I have a delta table that I created in Azure Synapse using a mapping data flow. The data flow reads append-only changes from Dataverse, finds the latest value, and upserts them to the table.

Now, I'd like to add a column to the delta table. When you select Upsert in a mapping dataflow, the Merge Schema is disabled, so it doesn't appear I can use that.

I tried creating a notebook and executing the following SQL, but I get an error.

ALTER TABLE delta.`https://xxxx.dfs.core.windows.net/path/to/table` ADD COLUMNS (mytest STRING)

Error: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: null path

The path provided is not in the default Synapse container.

How can I alter the table and add the column?

Upvotes: 0

Views: 1200

Answers (1)

Steve Platz
Steve Platz

Reputation: 2263

The issue I was running into was that the default Synapse container wasn't present in the storage account. After creating the container, the command executed successfully.

  1. Open Azure Portal
  2. Navigate to Synapse Workspace
  3. Click Properties
  4. Value of Primary ADLS Gen2 file system

This helped me track down the issue. https://learn.microsoft.com/en-us/answers/questions/706694/unable-to-run-sql-queries-in-azure-synapse-error-o.html

Upvotes: 0

Related Questions