What native search engine can be used for my use case?

Question

I am working on an android application where some secured data will be stored in android device and frequently be queried from web app side. Right now I am storing the data in SQLite DB. When the application is opening I am loading the full data from DB and preparing a filter map for efficient in memory queries for web app. However, for large amount of data in-memory query might not be a scalable solution. My search queries are basically full text search. Where can I start from?

Update 1

As suggested by Gaurav, I am trying to use virtual table,

private const val SQL_CREATE_ENTRIES =
    "CREATE VIRTUAL TABLE ${FeedReaderContract.FeedEntry.TABLE_NAME} USING FTS5 (" +
            "${FeedReaderContract.FeedEntry.COLUMN_NAME_TITLE}," +
            "${FeedReaderContract.FeedEntry.COLUMN_NAME_SUBTITLE})"

It gives me error while creating table,

[CREATE VIRTUAL TABLE entry USING FTS5 (title,subtitle);] no such module: FTS5

E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.achellies.kotlin, PID: 30458
java.lang.RuntimeException: Unable to start activity ComponentInfo{com.achellies.kotlin/com.achellies.kotlin.MainActivity}: android.database.sqlite.SQLiteException: no such module: FTS5 (code 1 SQLITE_ERROR[1])
    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3086)
    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3229)
    at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:78)
    at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:108)
    at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:68)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1926)
    at android.os.Handler.dispatchMessage(Handler.java:106)
    at android.os.Looper.loop(Looper.java:214)
    at android.app.ActivityThread.main(ActivityThread.java:6981)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1445)
 Caused by: android.database.sqlite.SQLiteException: no such module: FTS5 (code 1 SQLITE_ERROR[1])
    at android.database.sqlite.SQLiteConnection.nativeExecuteForChangedRowCount(Native Method)
    at android.database.sqlite.SQLiteConnection.executeForChangedRowCount(SQLiteConnection.java:946)
    at android.database.sqlite.SQLiteSession.executeForChangedRowCount(SQLiteSession.java:754)
    at android.database.sqlite.SQLiteStatement.executeUpdateDelete(SQLiteStatement.java:64)
    at android.database.sqlite.SQLiteDatabase.executeSql(SQLiteDatabase.java:2228)
    at android.database.sqlite.SQLiteDatabase.execSQL(SQLiteDatabase.java:2155)
    at com.achellies.kotlin.FeedReaderDbHelper.onCreate(FeedReaderDbHelper.kt:19)
    at android.database.sqlite.SQLiteOpenHelper.getDatabaseLocked(SQLiteOpenHelper.java:393)
    at android.database.sqlite.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:298)
    at com.achellies.kotlin.MainActivity.onCreate(MainActivity.kt:15)
    at android.app.Activity.performCreate(Activity.java:7326)
    at android.app.Activity.performCreate(Activity.java:7317)
    at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1271)
    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3066)
    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3229) 
    at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:78) 
    at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:108) 
    at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:68) 
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1926) 
    at android.os.Handler.dispatchMessage(Handler.java:106) 
    at android.os.Looper.loop(Looper.java:214) 
    at android.app.ActivityThread.main(ActivityThread.java:6981) 
    at java.lang.reflect.Method.invoke(Native Method) 
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493) 
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1445) 

Update 2

private const val SQL_CREATE_ENTRIES =
    "CREATE VIRTUAL TABLE ${FeedReaderContract.FeedEntry.TABLE_NAME} USING FTS4 (" +
            "${FeedReaderContract.FeedEntry.COLUMN_NAME_TITLE}," +
            "${FeedReaderContract.FeedEntry.COLUMN_NAME_SUBTITLE})"

This works fine for me. But I cannot run raw query with sanitized input.

val query = "SELECT * FROM entry WHERE title MATCH '?'"
        val args = arrayOf("Test")
        val cursor: Cursor? = db.rawQuery(query, args)

Runtime error,

java.lang.RuntimeException: Unable to start activity ComponentInfo{com.achellies.kotlin/com.achellies.kotlin.MainActivity}: java.lang.IllegalArgumentException: Cannot bind argument at index 1 because the index is out of range.  The statement has 0 parameters.

Answer 1

Sqlite provides a Full Text Search Option. I have used it successfully for my app that searches on 0.22 Million strings in real-time.

You need to create A virtual table. And insert all your data in this table.Then you need to update this table when new data comes along.

CREATE VIRTUAL TABLE table_name 
USING FTS4(column1,column2...);

ex -

public boolean createFTS4Table() {
        database = getWritableDatabase();
        boolean success = true;
        try {

            database.execSQL("CREATE VIRTUAL TABLE if not exists fts_dict USING fts4 (id ,word)");
            database.execSQL("INSERT INTO fts_dict SELECT rowid,word FROM entries");
        } catch (Exception ex) {
            ex.printStackTrace();
            success = false;
            Toast.makeText(ctx, "ERROR in FTS", Toast.LENGTH_SHORT).show();
        }

        return success;

    }

For searching you can use MATCH -

SELECT * 
FROM posts 
WHERE posts MATCH 'fts5';

Java -

getReadableDatabase().rawQuery("SELECT * FROM " + FTS_TABLE_NAME + " WHERE word match ?", new String[]{word + "*"}

For more in-depth documentation -

http://www.sqlitetutorial.net/sqlite-full-text-search/