CODEWITHSUNDEEP
c#sqliteencoding
adi ohaion
adi ohaion

Reputation: 374

Bad Encoding when inserting non-latin data to sqlite database using c#

I'm trying to insert non-latin data into sqlite database using bind variables using System.Data.SQLite. when I inserting the data without bind variables everything is ok:

cmd.CommandText = "INSERT INTO mytab VALUES (/" some non-latin data /")"; cmd.ExecuteNonQuery();

when I try to use bind variables the data is being insert but the encoding is bad and all I can see is stuff like ??? or some weird letters. I read the solutions here and tried to use PRAGMA ENCODING="UTF-8" in the connection string. also I've tried to encode it from "ISO-8859-1" to UTF-8, and also from the default encoding to UTF-8. nothing fix the problem.

here's my code, please tell me what im doing wrong:

public void addItem(List updateProperties) {

try
{
    _Conn.Open();
    SQLiteCommand oCmd = new SQLiteCommand ( "BEGIN", _dtflsConn );
    oCmd.ExecuteNonQuery (  );
    oCmd.Dispose (  );


    foreach (List<string> lst in updateProperties)
    {    
        string table = lst[0];
        lst.RemoveAt(0);

        //building query
        oCmd = new SQLiteCommand ( "INSERT INTO " + table + " VALUES (", _dtflsConn);
        foreach (string str in lst)
            oCmd.CommandText +=   "?,";

        oCmd.CommandText = (oCmd.CommandText.Substring(0, oCmd.CommandText.Length -1)) + ");";
        int i = 0;
        foreach ( string str in lst )
        {
            var val = oCmd.CreateParameter();
            val.ParameterName = i.ToString();

            //Encoding
            byte[] b = Encoding.Default.GetBytes(str);
            b = Encoding.Convert(Encoding.Default, Encoding.UTF8, b);
            val.Value = b;
            oCmd.Parameters.Add(val);
            i++;
        }

        //executing
        oCmd.ExecuteNonQuery();
        oCmd.Dispose();
    }

    oCmd = new SQLiteCommand ( "END", _dtflsConn);
    oCmd.ExecuteNonQuery (  );
    oCmd.Dispose (  );

    _Conn.Close();
}

catch (Exception e)
{
    _Conn.Close();
    logger.LogMessageToFile(e.Message);
}

}

Upvotes: 0

Views: 2019

Answers (2)

user1286858
user1286858

Reputation: 31

In a project we inserted non-UTF8 (sqlite default encoding) into a SQLite database. This was done by inserting the text as binary type. First convert the string (in your favorite encoding) to byte array, then add a parameter of type binary with your bytearray as value.

Upvotes: 2

Jon Skeet
Jon Skeet

Reputation: 1500155

Well this doesn't look like a good idea:

byte[] b = Encoding.Default.GetBytes(str);

Why are you even trying any encoding yourself? Just use the string:

val.Value = str;

That's what you should be doing, as this is text. If that doesn't work, look at the schema to find out how the field is declared, and so on - but I would strongly advise against performing the encoding yourself. That's fundamentally the job of the database/driver combination - it shouldn't be in application code.

Upvotes: 1

Related Questions