Liquibase with Databricks: "Unsupported data type DATETIME" error during liquibase update

Question

I’m trying to use Liquibase with Databricks SQL Warehouse to manage schema migrations. However, when I run the liquibase update command, I encounter the following error:

System and Version info:

1. Liquibase Version: 4.31.0
2. Java version :23.0.2
3. Database: Databricks SQL Warehouse
4. Databricks JDBC Driver: DatabricksJDBC42.jar

Error: I am getting the following error when running the liquibase update command:

[Failed SQL: (500051) CREATE TABLE DATABASECHANGELOG (ID VARCHAR(255) NOT NULL, AUTHOR VARCHAR(255) NOT NULL, FILENAME VARCHAR(255) NOT NULL, DATEEXECUTED datetime NOT NULL, ORDEREXECUTED INT NOT NULL, EXECTYPE VARCHAR(10) NOT NULL, MD5SUM VARCHAR(35), DESCRIPTION VARCHAR(255), COMMENTS VARCHAR(255), TAG VARCHAR(255), LIQUIBASE VARCHAR(20), CONTEXTS VARCHAR(255), LABELS VARCHAR(255), DEPLOYMENT_ID VARCHAR(10))]

Caused by: org.apache.spark.sql.catalyst.parser.ParseException:[UNSUPPORTED_DATATYPE] Unsupported data type "DATETIME". SQLSTATE: 0A000== SQL (line 1, position 134) ==...HAR(255) NOT NULL, DATEEXECUTED datetime NOT NULL, ORDEREXECUTED INT NOT...

Below is my liquibase.properties:

#Liquibase configuration for Databricks SQL Warehouse

changeLogFile=changelog.xml

#JDBC connection URL for Databricks

url=jdbc:databricks://adb-xxx.azuredatabricks.net:443;httpPath=/sql/1.0/warehouses/xxx;AuthMech=3;SSL=1;ConnCatalog=catalog_name;connSchema=schema_name

#Databricks authentication (using personal access token)

username=token

password=XXXXXX

#JDBC driver for Databricks

driver=com.databricks.client.jdbc.Driver

#Path to the Databricks JDBC driver JAR fileclasspath=jdbc/DatabricksJDBC42.jar

#Liquibase logging level

logLevel=debug

#Set the default datetime type for Liquibase

liquibase.datatype.datetime=TIMESTAMP

Below is my changelog.xml

<?xml version="1.0" encoding="UTF-8"?>
\<databaseChangeLog 
    xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
    xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-latest.xsd http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd"\>
\<changeSet id="1" author="demo"\>
\<createTable catalogName="catalog_name" schemaName="schema_name" tableName="employee"\>
\<column name="id" type="bigint"\>
\<constraints primaryKey="true" nullable="false"/\>
\</column\>
\<column name="name" type="string"\>
\<constraints nullable="false"/\>
\</column\>
\<column name="department" type="string"/\>
\</createTable\>
\</changeSet\>
\</databaseChangeLog\>

What I’ve Tried So Far

1. Manually Created the DATABASECHANGELOG Table: I manually created the DATABASECHANGELOG table in Databricks with the correct schema (using TIMESTAMP instead of DATETIME):

CREATE TABLE catalog_name.schema_name.DATABASECHANGELOG (
ID STRING NOT NULL,
AUTHOR STRING NOT NULL,
FILENAME STRING NOT NULL,
DATEEXECUTED TIMESTAMP NOT NULL,
ORDEREXECUTED INT NOT NULL,
EXECTYPE STRING NOT NULL,
MD5SUM STRING,
DESCRIPTION STRING,
COMMENTS STRING,
TAG STRING,
LIQUIBASE STRING,
CONTEXTS STRING,
LABELS STRING,
DEPLOYMENT_ID STRING
);

But whenever I run liquibase update command it again trying to create DATABASECHANGELOG and giving above error

2. Updated liquibase.properties: I configured the liquibase.properties file to override the DATETIME behavior:

liquibase.datatype.datetime=TIMESTAMP

But still getting same error

My Questions

1.Why is Liquibase still trying to create the DATABASECHANGELOG table with DATETIME even after I manually created it with TIMESTAMP?

2.Is there a way to force Liquibase to use TIMESTAMP instead of DATETIME for all date/time columns?

3 Are there any known compatibility issues between Liquibase and Databricks that could cause this behavior?