Unable to generate difference from liquibase gradle plugin

Question

I'm trying to implement liquibase in an existing SpringBoot project with MYSQL database. I want to be able to generate changesets which specify the differences when an entity is changed.

What I've done:

I've added liquibase dependencies and the gradle liquibase plugin in my build.gradle file. After making a domain change, I've run gradle generateChangeLog. The command executes successfully but nothing happens.

I read somewhere that this gradle plugin works only for the inmemory h2 database? Is that true? If yes then what alternative should I use to generate changelogs automatically.

I could not find a working SpringBoot gradle based example which uses MYSQL and has liquibase implemented WITH automatic change generation ability. It would be great if someone could provide that.

References:

https://github.com/stevesaliman/liquibase-workshop

https://github.com/liquibase/liquibase-gradle-plugin

Answer 1

NB. To apply the excellent answer from Hazim for Spring boot 2 referenceUrl argument should have "&hibernate.physical_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy&hibernate.implicit_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy"

Answer 2

liquibase {
activities {
    main {
        changeLogFile 'src/main/resources/liquibase-changeLog-db1.xml'
        classpath "$projectDir/src/main/resources"
        url props["database.db1.url"]
        referenceUrl props["database.db3.url"]
        username props["database.db1.user"]
        password props["database.db1.password"]
        referenceUsername props["database.db3.user"]
        referencePassword props["database.db3.password"]

    }
    secondary {
        changeLogFile 'src/main/resources/liquibase-changeLog-db2.xml'
        classpath "$projectDir/src/main/resources"
        url props["database.db2.url"]
        username props["database.db2.user"]
        password props["database.db2.password"]
    }
    tertiary {
        changeLogFile 'src/main/resources/liquibase-changeLog-db1.xml'
        classpath "$projectDir/src/main/resources"
        url props["database.db3.url"]
        username props["database.db3.user"]
        password props["database.db3.password"]
    }
    runList =  project.ext.runList
}

}

here when you run the command ./gradlew diff prunList=main it will take the main db and compare it with the reference db and will printout the diff in the console in the below format. you may need to add the db urls and passwords in applications.properties file.here in my application.properties file i have defined 3 dbs.in my 1st and 3rd they are almost the same except for a minor column addition. in the below diff it has identified the missing column.

Compared Schemas: liquibase_new -> liquibase2
Product Name: EQUAL
Product Version: EQUAL
Missing Catalog(s): NONE
Unexpected Catalog(s): NONE
Changed Catalog(s): NONE
Missing Column(s):
     liquibase_new.business_center.new
Unexpected Column(s):
     liquibase2.business_center.new_column
Changed Column(s): NONE
Missing Foreign Key(s): NONE
Unexpected Foreign Key(s): NONE
Changed Foreign Key(s): NONE
Missing Index(s): NONE
Unexpected Index(s): NONE
Changed Index(s): NONE
Missing Primary Key(s): NONE
Unexpected Primary Key(s): NONE
Changed Primary Key(s): NONE
Missing Sequence(s): NONE
Unexpected Sequence(s): NONE
Changed Sequence(s): NONE
Missing Table(s): NONE
Unexpected Table(s): NONE
Changed Table(s): NONE
Missing Unique Constraint(s): NONE
Unexpected Unique Constraint(s): NONE
Changed Unique Constraint(s): NONE
Missing View(s): NONE
Unexpected View(s): NONE
Changed View(s): NONE

Answer 3

With the following setup it can be used in conjunction with the liquibase-hibernate and liquibase-gradle extensions:

plugins {
    id 'org.liquibase.gradle' version '2.0.1'
}

dependencies {
    implementation 'org.liquibase:liquibase-core:3.8.0'

    liquibaseRuntime 'org.liquibase.ext:liquibase-hibernate5:3.8'
    liquibaseRuntime sourceSets.main.runtimeClasspath
    liquibaseRuntime sourceSets.main.output
}

def props = new Properties()
file("src/main/resources/liquibase.properties").withInputStream { props.load(it) }

diff.dependsOn assemble
diffChangeLog.dependsOn assemble

liquibase {
    activities {
        main {
            changeLogFile props.getProperty("liquibase.changelog.main")
            referenceUrl props.getProperty("liquibase.changelog.referenceUrl")
            url props.getProperty("spring.datasource.url")
            username props.getProperty("spring.datasource.username")
            password props.getProperty("spring.datasource.password")
            referenceDriver "liquibase.ext.hibernate.database.connection.HibernateDriver"
        }
    }
}

This will generate a changelog in the specified changelog file. You can first generate an initial changelog with gradle generateChangelog, run your application to apply these changes to the database and then after each change in your entity models run the gradle diffChangelog task to get these changes in the chanlog file. These should then be applied to the database before running the diffChangeLog task again to prevent duplicate operations in the changelog.

For this to work you will need the following properties in liquibase.properties:

liquibase.changelog.main=src/main/resources/db/changelog/db.changelog-master.xml
liquibase.changelog.classpath=classpath:db/changelog/db.changelog-master.xml
liquibase.changelog.referenceUrl=hibernate:spring:<MODEL_PACKAGE>?dialect=org.hibernate.dialect.MySQL5Dialect

IMPORTANT: Be sure to replace <MODEL_PACKAGE> with the package where your hibernate models are located.

Answer 4

plugins {
    id 'org.liquibase.gradle' version '2.0.1'
}

The Gradle liquibase plugin worked for me after I added built resources and classes to its runtime dependencies, as follows:

dependencies {
    liquibaseRuntime 'org.liquibase:liquibase-core:3.5.3'
    liquibaseRuntime 'org.liquibase:liquibase-groovy-dsl:2.0.1'
    liquibaseRuntime 'mysql:mysql-connector-java:5.1.34'
    liquibaseRuntime 'org.liquibase.ext:liquibase-hibernate4:3.6'
    liquibaseRuntime 'javax.persistence:javax.persistence-api:2.2'
    liquibaseRuntime files('build/classes/java/main')
    liquibaseRuntime files('build/resources/main')

    // Your other dependencies...
}

I defined its main activity as:

liquibase {
  activities {
    main {
      changeLogFile 'build/liquibase_change_log.xml'
      url 'jdbc:mysql://localhost/YOURDATABASE'
      username 'YOURUSER'
      password 'YOURPASSWORD'
      driver 'com.mysql.jdbc.Driver'
      referenceUrl 'hibernate:classic:/hibernate.cfg.xml'      
    }
  }
}

Note that I'm just using a classic Hibernate configuration to define the source schema.

The hibernate4 integration of liquibase looks for /hibernate.cfg.xml in the classpath of the JVM that is running liquibase. It will also need to find your schema classes.

I also added this:

diffChangeLog.dependsOn build

Answer 5

The solutions is to write a gradle task which invokes liquibase diffChangeLog

Create a liquibase.gradle file in the project root directory, add liquibase-hibernate extension and write a gradle task that invokes the liquibase diffChangeLog command.

configurations {
  liquibase
}

dependencies {
  liquibase group: 'org.liquibase.ext', name: 'liquibase-hibernate4', version: 3.5
}

//loading properties file.
Properties liquibaseProps = new Properties()
liquibaseProps.load(new FileInputStream("src/main/resources/liquibase-task.properties"))

Properties applicationProps = new Properties()
applicationProps.load(new FileInputStream("src/main/resources/application.properties"))

task liquibaseDiffChangelog(type: JavaExec) {
  group = "liquibase"


  classpath sourceSets.main.runtimeClasspath
  classpath configurations.liquibase
  main = "liquibase.integration.commandline.Main"

  args "--changeLogFile=" + liquibaseProps.getProperty('liquibase.changelog.path')+ buildTimestamp() +"_changelog.xml"
  args "--referenceUrl=hibernate:spring:" + liquibaseProps.getProperty('liquibase.domain.package') + "?dialect=" + applicationProps.getProperty('spring.jpa.properties.hibernate.dialect')
  args "--username=" + applicationProps.getProperty('spring.datasource.username')
  args "--password=" + applicationProps.getProperty('spring.datasource.password')
  args "--url=" + applicationProps.getProperty('spring.datasource.url')
  args "--driver=com.mysql.jdbc.Driver"
  args "diffChangeLog"
}

def buildTimestamp() {
  def date = new Date()
  def formattedDate = date.format('yyyyMMddHHmmss')
  return formattedDate
}

NOTE: I have used properties files to pass arguments to the liquibase command, you could add the values directly, but that would not be a good practice.

Next, you would need to apply the liquibase.gradle file from within the project's build.gradle file. and add the liquibase dependency

apply from: 'liquibase.gradle'
//code omitted
dependencies {
    compile (group: 'org.liquibase', name: 'liquibase-core', version: "3.4.2")
}

After this step liquibase would be setup completely.

You can now use gradle liquibaseDiffChangeLog to generate changelogs.