CODEWITHSUNDEEP
python-3.xgenerator
Asteroid098
Asteroid098

Reputation: 2825

Python - why can't I get at dict items in a generator?

I created a generator that reads each row from a large CSV file, and returns each row:

def csv_reader(filename):
    import csv
    with open(filename) as f:
        reader = csv.DictReader(f)
        for line in reader:
            yield  dict(line)


filename = "output.csv"
generator = csv_reader(filename)

I know how to iterate the generator and get the output:

# Iterate Generator
for g in generator:
    print(g)

This prints dictionaries.

However, how can I access the value of each key in each dictionary? The following did not return anything.

for g in generator:
    print(g['user'])

This would not work either:

>>> for g in generator:
...     alist.append(g.keys())
...     alist
... 
>>> alist
[]

Upvotes: 0

Views: 311

Answers (1)

Julien Palard
Julien Palard

Reputation: 11536

Looks like you're iterating a now-empty generator.

I mean, once you iterated your generator once, it's emptied, and iterating over it again will give nothing:

>>> def some_generator():
...     yield 1
...     yield 2
...     yield 3
... 
>>> generator = some_generator()
>>> for g in generator:
...     print(g)
... 
1
2
3
>>> for g in generator:
...     print(g)
...

There's two paths around this: either store the generator result in a list so you can iterate the list multiple times (consumes memory), either re-create the generator (consumes I/O as you're reading a file).

So either:

>>> for g in some_generator():
...     print(g)
... 
1
2
3
>>> for g in some_generator():
...     print(g)
... 
1
2
3

or:

>>> values = list(some_generator())
>>> for g in values:
...     print(g)
... 
1
2
3
>>> for g in values:
...     print(g)
... 
1
2
3

Upvotes: 1

Related Questions