CODEWITHSUNDEEP
pythonwindowslmdb
Hossein
Hossein

Reputation: 26004

No data is read when using lmdb cursor in Python

I have a lmdb database and I'm trying to read its contents. The irony is nothing gets printed on screen. This is the code snippet that I have written for reading from lmdb:

import caffe
import lmdb
import numpy as np
from caffe.proto import caffe_pb2
import cv2
import sys

db_train = lmdb.open('mnist_train_lmdb')
db_train_txn = db_train.begin()
cursor = db_train_txn.cursor()

print db_train
print db_train_txn
print db_train_txn.cursor()

datum = caffe_pb2.Datum()

index = sys.argv[0]
size_train = 50000
size_test = 10000
data_train = np.zeros((size_train, 1, 28, 28))
label_train = np.zeros(size_train, dtype=int)

print 'Reading training data...'
i = -1
for key, value in cursor:
    i = i + 1
    if i % 1000 == 0:
        print i
    if i == size_train:
        break
    datum.ParseFromString(value)
    label = datum.label
    data = caffe.io.datum_to_array(datum)
    data_train[i] = data
    label_train[i] = label

This prints :

<Environment object at 0x0000000009CE3990>
<Transaction object at 0x0000000009CE1810>
<Cursor object at 0x0000000009863738>
Reading training data...
Reading test data...

It seems the for loop doesn't run at all. What am I missing here?

I checked and it seems this is the normal way of reading from lmdb, all source examples that I have seen have similar approach.

Upvotes: 0

Views: 1720

Answers (2)

Dale
Dale

Reputation: 1608

Correct myself:

Both way of using lmdb.Cursor()

for key, value in cursor:

and

while cursor.next():

are right and I was wrong in the original answer.

You didn't use cursor properly and a slight modification should be made in your code like:

... # original stuff
print 'Reading training data...'
i = -1
while cursor.next(): #  Move to the next element, and
    i = i + 1        #  note cursor starts in an unpositioned state
    if i % 1000 == 0:
        print i
    if i == size_train:
        break
    datum.ParseFromString(cursor.value())
    label = datum.label
    data = caffe.io.datum_to_array(datum)
    data_train[i] = data
    label_train[i] = label

For more usage about lmdb python binding, you can refer here.

Upvotes: 2

Hossein
Hossein

Reputation: 26004

OK, it seems the database was faulty! I used another database and it worked just fine. both my code snippet and what was suggested by @DaleSong.

Upvotes: 1

Related Questions