CODEWITHSUNDEEP
pythonpython-3.xgenerator
Jens de Bruijn
Jens de Bruijn

Reputation: 969

Does collections.Counter eat a generator while counting?

I have been looking at collections.Counter.

I am using the following (simplified) code:

choices = ['foo', 'bar']

def generator(n=100000000):
  yield random.choice(choices)

counts = collections.Counter(generator())

Now my question, in my implementation Counter takes a generator as input. To first convert the generator to a list would take way too much space in memory.

So I wonder if collections.Counter first converts the data to a list and then 'counts' or it 'eats' the generator while counting.

If it first converts the data to a list. How would I best go about implementing it without converting it to a list.

Upvotes: 0

Views: 563

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1122152

No, the iterable you pass in (be it a generator or another iterable object) is not converted to a list.

There is no need to convert the iterable; counting is done as you iterate. The implementation is essentially the same as:

counts = {}
for element in generator():
    if element in counts:
        counts[element] += 1
    else:
        counts[element] = 1

but a Counter() object does this much faster (the counting code is implemented in C).

Upvotes: 3

Related Questions