Room - Using external SQLites as well as internal DB

Question

At our company, we are currently evaluating switching to Room for managing our mobile devices databases. The main reasons we want to do that is because of inheritance support and increased testability.

Currently, we are using GreenDao to access an internal database (on which the session data, settings, user data and similar things are stored) and to access our external databases (sqlites that we retrieve from an FTP-Server and store in subfolders in /sdcard/Android/app-folder/; these provide the data we use in our business logic).

What we need: We need to be able to use attach the external databases to the internal Room-Database without changing the path they are stored in. After adding them, we want to be able to use the databases with the Room Entity, Dao and Database objects. Is there any possible way to achieve this?

Room-Version: 1.1.1 (Support-Libraries)

API-Level: 15

MikeT · Accepted Answer

What we need:

We need to be able to use attach the external databases to the internal Room-Database without changing the path they are stored in. After adding them, we want to be able to use the databases with the Room Entity, Dao and Database objects. Is there any possible way to achieve this?

It might be easier to not attach, the reason being that you could utilise a separate Room Database instance. Without you'd need to have separate DAO's to cater for the attached schema name (I believe). Saying that the example below (based upon something I was playing with for someone and hence the rather confusing column names).

e.g. assuming ATTACH DATABASE .... AS other (the attached schema being other) then instead of (for the main database)

@Query("SELECT * FROM AllUsers")
List getAllAllUsers();

You would need a complimentary :-

@SkipQueryVerification
@Query("SELECT * FROM other.AllUsers")
List getOtherAllAllUsers();

etc.

However, if you instead had something like (for the main) :-

    mLPDB = Room.databaseBuilder(this,LoanPaymentDatabase.class,"mydb").allowMainThreadQueries().build();
    mLPDB_DAO = mLPDB.mDao();

Along with (for the other) :-

    mOtherDB = Room.databaseBuilder(this,LoanPaymentDatabase.class,OtherDatabaseHelper.DBNAME).allowMainThreadQueries().build();
    mOtherDAO = mOtherDB.mDao();

Then you can access both using the same DAO.

Note of course the above assumes that the schema are complimentary (not necessarily exact).

Not necessarily exact?

Also covering the comment :-

You have to migrate data to room itself first.

Having a little play, you can get around the having to migrate by fooling room by setting the user_version to 0. In which case Room sets the version number (limited testing). However, I'm not sure what GreenDao or your Server would make of this (your homework).
My limited testing was for a common issue when migrating that of having a column with INTEGER PRIMARY KEY i.e. without AUTOINCREMENT. Room if migrating will complain that the schema's mismatch. So I intentionally did not code AUTOINCREMENT, set the user_version to 0 and no complaints accessing the database via Room. Also used a column type of rumplestilskin and no complaint.

As such I believe that you can get around migration issues with the dreaded expected/found by setting the user_version to 0 (and hence I believe circumvent migration). Obviously though column names have to match, if defined in an Entity and not ignored.

I've also tried adding a column not defined the the Entity and using the above results in no complaint (these tests should be obvious in the code below).

Example(s)

The following is an example of a 2 Entity Room Database and for the purposes of testing the other database one built outside of room that sufficiently matches the room database to be able to be used i.e. Entity column names are matched.

The other database

The other non-room database is created via an SQLiteOpenHelper subclass as per OtherDatabaseHelper.java :-

public class OtherDatabaseHelper extends SQLiteOpenHelper {

    public static final String DBNAME = "lpolddb";
    public static final int DBVERSION = 1;
    public static final String ALLUSERS_TBL = "AllUsers";
    public static final String PAIDUNPAID_TBL = "PaidUnpaid";

    /*
        @PrimaryKey(autoGenerate = true)
        private long auid;
        private String Name;
        private int Loan;
        private int TimeInMonths;
     */
    public static final String ALLUSERS_COL_AUID = "auid";
    public static final String ALLUSERS_COL_NAME = "Name";
    public static final String ALLUSERS_COL_LOAN = "Loan";
    public static final String ALLUSERS_COL_TIMEINMONTHS = "TimeInMonths";

    private static final String crt_allusers_table_sql =
            "CREATE TABLE IF NOT EXISTS " + ALLUSERS_TBL + "(" +
                    //ALLUSERS_COL_AUID + " INTEGER PRIMARY KEY AUTOINCREMENT," +
                    ALLUSERS_COL_AUID + " INTEGER PRIMARY KEY," +
                    ALLUSERS_COL_NAME + " TEXT, " +
                    ALLUSERS_COL_LOAN + " INTEGER, " +
                    "someothercolumnnotdefineinroom TEXT, " + //!!!!!!!!!! not a column in an entity
                    ALLUSERS_COL_TIMEINMONTHS + " INTEGER" +
                    ")";

    /*
        @PrimaryKey(autoGenerate = true)
        private long puid;
        private int TimeInMonths;
        private String PaidUnpaid;
        @ForeignKey(
            entity = AllUsers.class,
            parentColumns = {"auid"},
            childColumns = {"AllUsersReference"},
            onUpdate = ForeignKey.CASCADE, onDelete = ForeignKey.CASCADE)
        private long AllUsersReference;
     */

    public static final String PAIDUNPAID_COL_PUID = "puid";
    public static final String PAIDUNPAID_TIMEINMONTHS = ALLUSERS_COL_TIMEINMONTHS;
    public static final String PAIDUNPAID_COL_PAIDUNPAID = "PaidUnpaid";
    public static final String PAIDUNPAID_COL_ALLUSERSREFERENCE = "AllUsersReference";

    public static final String crt_paidunpaid_table_sql =
            "CREATE TABLE IF NOT EXISTS " + PAIDUNPAID_TBL + "(" +
                    PAIDUNPAID_COL_PUID + " INTEGER PRIMARY KEY AUTOINCREMENT," +
                    PAIDUNPAID_TIMEINMONTHS + " rumplestilskin, " + // !!!!!!!!!!!
                    PAIDUNPAID_COL_PAIDUNPAID + " TEXT," +
                    PAIDUNPAID_COL_ALLUSERSREFERENCE + " INTEGER " +
                    " REFERENCES " + ALLUSERS_TBL + "(" + ALLUSERS_COL_AUID + ") " +
                    "ON UPDATE CASCADE ON DELETE CASCADE" +
                    ")";


    SQLiteDatabase mDB;
    public OtherDatabaseHelper(Context context) {
        super(context, DBNAME, null, DBVERSION);
        mDB = this.getWritableDatabase();
    }

    @Override
    public void onCreate(SQLiteDatabase db) {
        db.execSQL(crt_allusers_table_sql);
        db.execSQL(crt_paidunpaid_table_sql);
    }

    @Override
    public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {

    }

    public long insertAllUsers(String name, int loanamount, int periodofloan) {
        ContentValues cv = new ContentValues();
        cv.put(ALLUSERS_COL_NAME,name);
        cv.put(ALLUSERS_COL_LOAN,loanamount);
        cv.put(ALLUSERS_COL_TIMEINMONTHS,periodofloan);
        return mDB.insert(ALLUSERS_TBL,null,cv);
    }

    public long insertPaidUnpaid(int formonth, String status, long allUserreferenced) {
        ContentValues cv = new ContentValues();
        cv.put(PAIDUNPAID_TIMEINMONTHS,formonth);
        cv.put(PAIDUNPAID_COL_PAIDUNPAID,status);
        cv.put(PAIDUNPAID_COL_ALLUSERSREFERENCE,allUserreferenced);
        return mDB.insert(PAIDUNPAID_TBL,null,cv);
    }
}

see comments for oddities/purposefully added discrepancies
this is popluated, accessed by alternate Room database and via attached DB in MainActivity.java below

The Room Database

The 2 Entities :-

AllUsers.java

@Entity
public class AllUsers {
    @PrimaryKey(autoGenerate = true)
    private long auid;
    private String Name;
    private int Loan;
    private int TimeInMonths;

    public AllUsers() {
    }

    @Ignore
    public AllUsers(String Name, int Loan, int TimeInMonths) {
        this.Name = Name;
        this.Loan = Loan;
        this.TimeInMonths = TimeInMonths;
    }

    public long getAuid() {
        return auid;
    }

    public void setAuid(long auid) {
        this.auid = auid;
    }

    public String getName() {
        return Name;
    }

    public void setName(String name) {
        Name = name;
    }

    public int getLoan() {
        return Loan;
    }

    public void setLoan(int loan) {
        Loan = loan;
    }

    public int getTimeInMonths() {
        return TimeInMonths;
    }

    public void setTimeInMonths(int timeInMonths) {
        TimeInMonths = timeInMonths;
    }
}

and PaidUnpaid.java :-

@Entity
public class PaidUnpaid {
    @PrimaryKey(autoGenerate = true)
    private long puid;
    private int TimeInMonths;
    private String PaidUnpaid;
    @ForeignKey(
            entity = AllUsers.class,
            parentColumns = {"auid"},
            childColumns = {"AllUsersReference"},
            onUpdate = ForeignKey.CASCADE, onDelete = ForeignKey.CASCADE)
    private long AllUsersReference;


    public PaidUnpaid() {
    }

    @Ignore
    public PaidUnpaid(int TimeInMonths, String PaidUnpaid, long AllUsersreference) {
        this.TimeInMonths = TimeInMonths;
        this.PaidUnpaid = PaidUnpaid;
        this.AllUsersReference = AllUsersreference;
    }

    public long getPuid() {
        return puid;
    }

    public void setPuid(long puid) {
        this.puid = puid;
    }

    public int getTimeInMonths() {
        return TimeInMonths;
    }

    public void setTimeInMonths(int timeInMonths) {
        TimeInMonths = timeInMonths;
    }

    public String getPaidUnpaid() {
        return PaidUnpaid;
    }

    public void setPaidUnpaid(String paidUnpaid) {
        PaidUnpaid = paidUnpaid;
    }

    public long getAllUsersReference() {
        return AllUsersReference;
    }

    public void setAllUsersReference(long allUsersReference) {
        AllUsersReference = allUsersReference;
    }
}

An additonal POJO class, AllUsersAndPaidUnpaidsList.java that was being played with so incorporated and used :-

public class AllUsersAndPaidUnpaidsList {

    @Embedded
    AllUsers allUsers;
    @Ignore
    @PrimaryKey
    long auid;

    @Ignore
    @Relation(entity = PaidUnpaid.class,parentColumn = "auid",entityColumn = "puid")
    List paidUnpaidList;

    @Ignore
    public AllUsersAndPaidUnpaidsList(AllUsers au, List pulist) {
        this.allUsers = au;
        this.paidUnpaidList = pulist;
    }

    public List getPaidUnpaidList() {
        return this.paidUnpaidList;
    }


    public void setPaidUnpaidList(List paidUnpaidList) {
        this.paidUnpaidList = paidUnpaidList;
    }

    public AllUsers getAllUsers() {
        return allUsers;
    }

    public void setAllUsers(AllUsers allUsers) {
        this.allUsers = allUsers;
    }

    public void outputToLog(String tag) {
        StringBuilder sb = new StringBuilder("AllUsersName = ")
                .append(this.allUsers.getName())
                .append(" TimeInMonths = ")
                .append(String.valueOf(this.allUsers.getTimeInMonths()))
                ;
        for (PaidUnpaid pu: this.getPaidUnpaidList()) {
            sb.append("
	 TimeInMonths = ")
                    .append(String.valueOf(pu.getTimeInMonths()))
                    .append(" Paid/Unpaid = ")
                    .append(pu.getPaidUnpaid());
        }
        Log.d(tag,sb.toString());
    }
}

A single interface Dao.java :-

@androidx.room.Dao
public interface Dao {

    @Insert(onConflict = OnConflictStrategy.IGNORE)
    long[] insertAllUsers(AllUsers... allUsers);

    @Insert(onConflict = OnConflictStrategy.IGNORE)
    long insertAllUsers(AllUsers allUsers);

    @Insert(onConflict = OnConflictStrategy.IGNORE)
    long[] insertPaidUnpaid(PaidUnpaid... paidUnpaids);

    @Insert(onConflict = OnConflictStrategy.IGNORE)
    long insertPaidUnpaid(PaidUnpaid paidUnpaid);

    @Update(onConflict = OnConflictStrategy.IGNORE)
    int updateAllUsers(AllUsers... allUsers);

    @Update(onConflict =  OnConflictStrategy.IGNORE)
    int updateAllUsers(AllUsers allUsers);

    @Update(onConflict = OnConflictStrategy.IGNORE)
    int updatePaidUnpaid(PaidUnpaid... paidUnpaids);

    @Update(onConflict = OnConflictStrategy.IGNORE)
    int updatePaidUnpaid(PaidUnpaid paidUnpaid);

    @Delete
    int deleteAllUsers(AllUsers... allUsers);

    @Delete
    int deleteAllUsers(AllUsers allUsers);

    @Delete
    int deletePaidUnpaid(PaidUnpaid... paidUnpaids);

    @Delete
    int deletePaidUnpaid(PaidUnpaid paidUnpaid);

    @Query("SELECT * FROM AllUsers")
    List getAllAllUsers();

    @Query("SELECT * FROM AllUsers WHERE auid = :id")
    List getOneAllUsersById(long id);

    @Query("SELECT * FROM PaidUnpaid")
    List getAllPaidUnpaids();

    @Query("SELECT * FROM PaidUnpaid WHERE puid = :id")
    List getOnePaidUnpaidById(long id);

    @Query("SELECT * FROM PaidUnpaid WHERE AllUsersReference = :allUsersid")
    List getPaidUnpaidsForAllUsersId(long allUsersid);

    /*************
     * Some Additional DAO's for attached not required for alternative helper
     * in practice you would likely need attached versions of all
     ************/

    @Query("SELECT * FROM other.PaidUnpaid WHERE AllUsersReference = :allUsersid")
    @SkipQueryVerification
    List getOtherPaidUnpaidForAllUsersId(long allUsersid);

    @SkipQueryVerification
    @Query("SELECT * FROM other.AllUsers")
    List getOtherAllAllUsers();
}

The Room Database class LoanPaymentDatabase.java

@Database(entities = {AllUsers.class,PaidUnpaid.class},exportSchema = false,version = 1)
public abstract class LoanPaymentDatabase extends RoomDatabase {
    public abstract Dao mDao();
}

Putting it all together

Finally an activity that :-

Creates and populates the other (non-room) database (if it doesn't exist) for testing. Setting the user_version (database version in Android talk) to 0.
Creates, if needed, the Room version of the database.
Adds a couple of rows to the Room version.
Outputs the data in the Room version to the log.
Creates an alternative RoomDatabase using the other database.
Outputs the data in the other database via Room.
Attaches the other database to the Room version.
Outputs the data from both via the original Roomdatabase accessing the attached other database via the additional DAO interfaces that include other.????.

MainActivity.java

public class MainActivity extends AppCompatActivity {

    LoanPaymentDatabase mLPDB;
    Dao mLPDB_DAO;

    LoanPaymentDatabase mOtherDB;
    Dao mOtherDAO;
    Random rnd = new Random();

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        manageOtherDatabase();

        mLPDB = Room.databaseBuilder(this,LoanPaymentDatabase.class,"mydb").allowMainThreadQueries().build();
        mLPDB_DAO = mLPDB.mDao();
        // Add some(2) AllUsers
        mLPDB_DAO.insertAllUsers(new AllUsers("Fred",5000,5));
        mLPDB_DAO.insertAllUsers(new AllUsers("Mary", 4000,6));

        // Add Some PaidUnpaid's for each AllUsers
        // Random amount with random paid or unpaid
        // This is just for demonstration and doesn't reflect what would typically be done
        List allusers =  mLPDB_DAO.getAllAllUsers();
        for (AllUsers au: allusers) {
            int lc = rnd.nextInt(4) + 1;
            int month = 1;
            for (int i=0; i < lc; i++) {
                String paid = "Paid";
                if((rnd.nextInt(2) % 2) > 0 ) {
                    paid = "Unpaid";
                }
                mLPDB_DAO.insertPaidUnpaid(new PaidUnpaid(month++, paid, au.getAuid()));
            }
        }

        //Extract all AllUsersAndPaidUnpaid (i.e  each AllUsers with the related PaidUnpaid for the AllUsers)
        ArrayList aupulist = new ArrayList<>();
        for (AllUsers au: allusers) {
            List pulist = mLPDB_DAO.getPaidUnpaidsForAllUsersId(au.getAuid());
            aupulist.add(new AllUsersAndPaidUnpaidsList(au,pulist));
        }

        // Output the results
        for (AllUsersAndPaidUnpaidsList aupu: aupulist) {
            aupu.outputToLog("INITALAUPU");
        }

        //Use separate openHelper rather than ATTACH
        mOtherDB = Room.databaseBuilder(this,LoanPaymentDatabase.class,OtherDatabaseHelper.DBNAME).allowMainThreadQueries().build();
        mOtherDAO = mOtherDB.mDao();
        ArrayList otheraupulist = new ArrayList<>();
        for (AllUsers oau: mOtherDAO.getAllAllUsers() ) {
            otheraupulist.add(new AllUsersAndPaidUnpaidsList(oau,mOtherDAO.getPaidUnpaidsForAllUsersId(oau.getAuid())));
        }
        for (AllUsersAndPaidUnpaidsList aupu: otheraupulist) {
            aupu.outputToLog("ALTDBAUPU");
        }

        // User Attach
        SupportSQLiteDatabase main_sdb = mLPDB.getOpenHelper().getWritableDatabase();
        SupportSQLiteDatabase other_sdb = mOtherDB.getOpenHelper().getWritableDatabase();
        main_sdb.execSQL("ATTACH DATABASE '" + other_sdb.getPath() + "' AS other");
        ArrayList attachaupulist = new ArrayList<>();
        for (AllUsers aau: mLPDB_DAO.getAllAllUsers()) {
            attachaupulist.add(new AllUsersAndPaidUnpaidsList(aau,mLPDB_DAO.getPaidUnpaidsForAllUsersId(aau.getAuid())));
        }
        for (AllUsers aauother: mLPDB_DAO.getOtherAllAllUsers()) {
            attachaupulist.add(new AllUsersAndPaidUnpaidsList(aauother,mLPDB_DAO.getOtherPaidUnpaidForAllUsersId(aauother.getAuid())));
        }
        for (AllUsersAndPaidUnpaidsList aupu: attachaupulist) {
            aupu.outputToLog("ATTACHEDAUPU");
        }

        mLPDB.close();
    }

    /*********
     *  For testing purposes - Populate the OTHER database to be used
     *********/
    private void manageOtherDatabase() {
        OtherDatabaseHelper mODBHlpr = new OtherDatabaseHelper(this);
        SQLiteDatabase db = mODBHlpr.getWritableDatabase();
        db.execSQL("PRAGMA user_version = 0");
        if (DatabaseUtils.queryNumEntries(db,OtherDatabaseHelper.ALLUSERS_TBL) > 0) {
            db.close();
            mODBHlpr.close();
            return;
        }
        db.beginTransaction();
        for (int i= 0; i < 5; i++) {
            long auid = mODBHlpr.insertAllUsers("AU" + String.valueOf(i),10000 + 1,5 + i);
            for(int ii = 0; ii < 5; ii++) {
                mODBHlpr.insertPaidUnpaid(ii,"Paid",auid);
            }
        }
        db.setTransactionSuccessful();
        db.endTransaction();
        db.close();
        mODBHlpr.close();
    }
}