Room db migration fallbackToDestructiveMigration() not working

Question

I am using Room with a prepopulated database in the assets folder. For an app update, I would like to alter this database by adding a new column and prepopulating this column with new data.

The database was auto-migrated from version 1 to 2 (a table was added). From version 2 to 3, I would now like to apply abovementioned changes by providing a different 'database.db' file in the assets folder and allowing for destructive migration.

@Database(entities = [Object1::class, Object2::class], version = 3, autoMigrations = [
    AutoMigration (from = 1, to = 2)], exportSchema = true)
abstract class AppDatabase : RoomDatabase() {

    abstract fun dao(): Dao

    companion object {

        private const val DB_NAME = "database.db"

        @Volatile
        private var instance: AppDatabase? = null

        fun getInstance(context: Context): AppDatabase {
            return instance ?: synchronized(this) {
                instance ?: buildDatabase(context).also { instance = it }
            }
        }

        private fun buildDatabase(context: Context): AppDatabase {
            return Room.databaseBuilder(
                context,
                AppDatabase::class.java, "AppDB.db")
                .fallbackToDestructiveMigration()
                .createFromAsset(DB_NAME)
                .build()
        }
    }
}

The problem is that I still get the following exception:

   java.lang.IllegalStateException: A migration from 1 to 3 was required but not found. Please provide the necessary Migration path via RoomDatabase.Builder.addMigration(Migration ...) or allow for destructive migrations via one of the RoomDatabase.Builder.fallbackToDestructiveMigration* methods.

I am unsure why this would still happen. I thought it was either providing a migration script or allowing for destructive migration that makes the migration work.

Added Comment:-

I have tried an implemented migration, but the same exception as above happened again. When I try starting over with versionCode 1, I am getting "java.lang.IllegalStateException: Room cannot verify the data integrity. Looks like you've changed schema but forgot to update the version number. You can simply fix this by increasing the version number." I have also changed the database name and added android:allowBackup="false" in the manifest.

Any ideas?

Answer 1

I had problems using fallbackToDestructiveMigration and createFromAsset together. I would like to share my experience because it took me hours to find it. When you provide an asset db, you have to update the user version pragma of the default database file that you are providing with createFromAsset. If not, you always lose the data that you insert while the app is working.

Answer 2

I finally figured out what the problem was, it had nothing to do with the versioning or anything else related to room or the asset db file.

It was dependency injection.

I provided my database to Dagger in a DatabaseModule class as follows:

private const val DB_NAME = "database.db"

@InstallIn(SingletonComponent::class)
@Module
class DatabaseModule {

@Provides
fun provideDao(appDatabase: AppDatabase): Dao {
    return appDatabase.dao()
}

@Provides
@Singleton
fun provideAppDatabase(@ApplicationContext appContext: Context): AppDatabase {
    return Room.databaseBuilder(
        appContext,
        AppDatabase::class.java, "AppDB.db")
        .createFromAsset(DB_NAME)
        .build()
}

}

It was missing the fallBackToDestructiveMigration() call, so this messed up Room's internal onUpgrade call in RoomOpenHelper.java.

To fix it, I made my buildDatabase call in AppDatabase public and used it to provide the database to Dagger in the DatabaseModule class.

Answer 3

After extensive methodical testing, the only way that I can replicate your (1-3 required) failure is by excluding fallbackToDestructiveMigation. In which case the exception happens if the migration is from 1 to 3 or the migration is 3 to 1 (i.e. Asset Version at 3 but Room version at 1)

• as per the spreadsheet screenshot below

• 1-3 exception when AssetDB Version =3 Database Version = 1 Room Version = 3

• also 3-1 exception when AssetDB Version =3 Database Version = -1 Room Version = 1

  • -1 version means file does not exist (i.e. initial install)

I suspect that you have somehow inadvertently introduced one of the above two scanrios. What I haven't tested is alternative Room library versions. The above was tested with 2.4.0-alpha04 as per :-

implementation 'androidx.core:core-ktx:1.6.0'
implementation 'androidx.appcompat:appcompat:1.3.1'
implementation 'com.google.android.material:material:1.4.0'
implementation 'androidx.constraintlayout:constraintlayout:2.1.0'
implementation 'androidx.room:room-ktx:2.4.0-alpha04'
implementation 'androidx.room:room-runtime:2.4.0-alpha04'
testImplementation 'junit:junit:4.+'
androidTestImplementation 'androidx.test.ext:junit:1.1.3'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'
kapt 'androidx.room:room-compiler:2.4.0-alpha0

For the testing, I had two copies of the asset file, one at version 1 the other at version 2 (v1dbbase.db and v3dbbase.db), the data in a common column indicating the if the data was for version3. The actual asset file used was deleted before a test and the appropriate version copied and pasted to database.db

I had the two entities Object1 and Object2 and could comment in or out an extra column in either. e.g.:-

/*TESTING INCLUDE FOR V2+ >>>>>*///, @ColumnInfo(name = COL_EXTRAV2, defaultValue = "x") val object1_extra: String
- as above it is excluded
/*TESTING INCLUDE FOR V2+ >>>>>*/, @ColumnInfo(name = COL_EXTRAV2, defaultValue = "x") val object1_extra: String
- with the two //'s before the comma now included

• both the extra columns commented out = Version 1
• Object1's extra column included = Version 3
• Object1's and Object2's extra column included = Version 3
  • Object2's extra column included but not Object1's was not considered.

A few constants were added to cater for logging.

Additionally to cater for logging a callback function was added (.addCallback) and onOpen, onCreate and onDestructiveMigration were all overridden to log the Room Version and Database Version.

To further enhance the logging, two functions were added, to get the version from the sqlite database header. One for the asset file, the other for the database. The functions being called/invoked BEFORE the database build.

To run a test it meant:-

1. Ensuring that the device had the App at the appropriate level.
2. Deleting the database.db asset
3. Copying and pasting the appropriate asset file as database.db (from either v1dbbase.db or v3dbbase.db)
4. Amending the Object1 class to include/exclude the extra column (as explained above)
5. Amending the Object2 class to include/exclude the extra columns (as explained above)
6. Amended the Room Version to the appropriate level.

The code used for testing:-

Object1

@Entity(tableName = TABLE_NAME)
data class Object1(
    @PrimaryKey
    @ColumnInfo(name = COL_ID)
    val object1_id: Long,
    @ColumnInfo(name = COL_NAME)
    val object1_name: String
    /*TESTING INCLUDE FOR V2+ >>>>>*///, @ColumnInfo(name = COL_EXTRAV2, defaultValue = "x") val object1_extra: String
) {
    companion object {
        const val TABLE_NAME = "object1"
        const val COL_ID = TABLE_NAME + "_object1_id"
        const val COL_NAME = TABLE_NAME + "_object1_name"
        const val COL_EXTRAV2 = TABLE_NAME + "_object1_extrav2"
    }
}

Object2

@Entity(tableName = TABLE_NAME)
data class Object2(
    @PrimaryKey
    @ColumnInfo(name = COL_ID)
    val object2_id: Long,
    @ColumnInfo(name = COL_NAME)
    val object2_name: String
    /*TESTING INCLUDE FOR V3>>>>>*///, @ColumnInfo(name = COL_EXTRAV3, defaultValue = "x") val object3_extrav3: String
) {
    companion object {
        const val TABLE_NAME = "object2"
        const val COL_ID = TABLE_NAME + "_object2_id"
        const val COL_NAME = TABLE_NAME + "_object2_name"
        const val COL_EXTRAV3 = TABLE_NAME + "_object2_extrav3"
    }
}

Dao

@Dao
abstract class Dao {

    @Insert
    abstract fun insert(object1: Object1): Long
    @Insert
    abstract fun insert(object2: Object2): Long
    @Query("SELECT * FROM ${Object1.TABLE_NAME}")
    abstract fun getAllFromObject1(): List<Object1>
    @Query("SELECT * FROM ${Object2.TABLE_NAME}")
    abstract fun getAllFromObject2(): List<Object2>
}

AppDatabase

@Database(
    entities = [Object1::class, Object2::class],
    version = AppDatabase.DBVERSION,
    autoMigrations = [AutoMigration (from = 1, to = 2)],
    exportSchema = true
)
abstract class AppDatabase : RoomDatabase() {
    abstract fun dao(): Dao

    companion object {

        private const val DB_NAME = "database.db"
        private const val DB_FILENAME = "AppDB.db" //<<<<< ADDED for getting header
        const val TAG = "DBINFO" //<<<< ADDED for logging
        const val DBVERSION = 1 //<<<<<ADDED for logging

        @Volatile
        private var instance: AppDatabase? = null

        fun getInstance(context: Context): AppDatabase {
            return instance ?: synchronized(this) {
                //ADDED>>>>> to get database version from dbfile and assets before building the database
                Log.d(TAG,
                    "AssetDB Version =${getAssetDBVersion(context, DB_NAME)} " +
                            "Database Version = ${getDBVersion(context, DB_FILENAME)} " +
                            "Room Version = ${DBVERSION}")
                instance ?: buildDatabase(context).also { instance = it }
            }
        }

        private fun buildDatabase(context: Context): AppDatabase {
            return Room.databaseBuilder(
                context,
                AppDatabase::class.java, DB_FILENAME)
                .fallbackToDestructiveMigration()
                .createFromAsset(DB_NAME)
                .allowMainThreadQueries()
                .addCallback(rdc)
                .build()
        }

        /* Call Backs for discovery */
        object rdc: RoomDatabase.Callback(){
            override fun onCreate(db: SupportSQLiteDatabase) {
                super.onCreate(db)
                Log.d(TAG,"onCreate called. DB Version = ${db.version}, Room Version is ${DBVERSION}")
            }

            override fun onOpen(db: SupportSQLiteDatabase) {
                super.onOpen(db)
                Log.d(TAG,"onOpen called. DB Version = ${db.version}, Room Version is ${DBVERSION}")
            }

            override fun onDestructiveMigration(db: SupportSQLiteDatabase) {
                super.onDestructiveMigration(db)
                Log.d(TAG,"onDestructiveMigration called. DB Version = ${db.version}, Room Version is ${DBVERSION}")
            }
        }

        fun getAssetDBVersion(context: Context, assetFilePath: String): Int {
            var assetFileHeader = ByteArray(100)
            try {
                var assetFileStream = context.assets.open(assetFilePath)
                assetFileStream.read(assetFileHeader,0,100)
                assetFileStream.close()
            } catch (e: IOException) {
                return -2 // Indicates file not found (no asset)
            }
            return ByteBuffer.wrap(assetFileHeader,60,4).getInt()
        }

        fun getDBVersion(context: Context, dbFileName: String): Int {
            var SQLiteHeader = ByteArray(100)
            val dbFile = context.getDatabasePath(dbFileName)
            if(dbFile.exists()) {
                var inputStream = dbFile.inputStream()
                inputStream.read(SQLiteHeader, 0, 100)
                inputStream.close()
                return ByteBuffer.wrap(SQLiteHeader, 60, 4).getInt()
            } else {
                return -1 // Indicates no database file (e.g. new install)
            }
        }
    }
}

• you may wish to consider including the logging above, it could very easily detect issues with the version(s) being used.

MainActivity

class MainActivity : AppCompatActivity() {

    lateinit var db: AppDatabase
    lateinit var dao: Dao
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        db = AppDatabase.getInstance(this)
        dao = db.dao()
        for(o1: Object1 in dao.getAllFromObject1()) {
            logObject1(o1)
        }
        for(o2: Object2 in dao.getAllFromObject2()) {
            logObject2(o2)
        }

    }

    fun logObject1(object1: Object1) {
        Log.d(TAG,"ID is ${object1.object1_id}, Name is ${object1.object1_name}")
    }

    fun logObject2(object2: Object2) {
        Log.d(TAG,"ID is ${object2.object2_id}, Name is ${object2.object2_name}")
    }
    companion object {
        const val TAG = AppDatabase.TAG
    }
}

In addition to utilising the above code and ensuring that the 6 tasks were undertaken I also kept a spreadsheet of the versions and the results e.g. :-

Previous answer (not the case after testing)

I believe that your issue may be with the pre-populated database, in that it's version number (user_version) hasn't been changed to 3.

• you can change the version using the SQL (from an SQlite tool ) PRAGMA user_version = 3;

The documentation says :-

Here is what happens in this situation:

• Because the database defined in your app is on version 3 and the database instance already installed on the device is on version 2, a migration is necessary.
• Because there is no implemented migration plan from version 2 to version 3, the migration is a fallback migration.
• Because the fallbackToDestructiveMigration() builder method is called, the fallback migration is destructive. Room drops the database instance that's installed on the device.
• Because there is a prepackaged database file that is on version 3, Room recreates the database and populates it using the contents of the prepackaged database file.
  • If, on the other hand, you prepackaged database file were on version 2, then Room would note that it does not match the target version and would not use it as part of the fallback migration.

• By note perhaps by the way of an exception?

Answer 4

Digging through the room documentation doesn't turn much up, my hunch is that it has to do with the fact that you are using Automigrations instead of implemented migrations. Have you tried changing that Automigration from 1->2 to an implemented migration?

Also, since you are manually replacing it with a new database that has prepopulated data my solution would be to just get rid of the old migrations, change the name of the DB slightly and start over from version 1. There's no reason to maintain the old migrations if anyone going from older versions to the current version are having their DB deleted.