CODEWITHSUNDEEP
pythonnumpysqlite
Mike
Mike

Reputation: 33

Create numpy array with fromiter

I am trying to make numpy arrays from data stored in sqlite3 and have an issue with the strings. From what I understand, fromiter is the most efficient way to accomplish this task. After getting the data with cursor.fetchall(), the string is correct. However after going into the numpy array it gets lost.

import sqlite3
import numpy as np

connection = sqlite3.connect('fermControlData.db')

with connection:
    cursor = connection.cursor()
    sql = "SELECT Time, bTemp, aTemp, heaterStatus, gasMonitor, Location FROM dataLog WHERE beerName= 'Red Sled' ORDER BY Time"
    cursor.execute(sql)
    data = cursor.fetchall()
    print(data[0], type(data[0][5]))
    dataLog = np.fromiter(data, dtype=('f, f, f, i, i8, S'))
    print(dataLog[0])

And this is the output-

(1480890498.4052606, 65.53, 66.42, 0, 0.0, 'Warm1') <class 'str'>
(1480890496.0, 65.52999877929688, 66.41999816894531, 0, 0, b'')

As you can see the string 'Warm1' is getting converted to b' '.

Do you know why that happens and how to fix it so that the string is imported?

Many thanks!

Upvotes: 0

Views: 2080

Answers (1)

user7138814
user7138814

Reputation: 2041

Strings in numpy arrays are fixed lenght and in the case of fromiter you have to specify that length in advance. E.g. with dtype=('f, f, f, i, i8, S5').

Upvotes: 1

Related Questions