Python 2.7 yield is creating strange behavior

Question

Using Python 2.7....

The print thread["Subject"] should return:

please ignore - test2 

and

Please ignore - test

It does successfully with this:

def thread_generator(threads):
    tz = pytz.timezone('America/Chicago')
    POSTS_SINCE_HOURS = 24
    now_date = datetime.datetime.now(tz)
    time_stamp = now_date - datetime.timedelta(hours=POSTS_SINCE_HOURS)
for thread in threads:
    last_post_time = convert_time(thread["LatestPostDate"])

    if last_post_time > time_stamp:
        print thread["Subject"]
        row = {
            "url": thread["Url"],
            "author_username": thread["Author"]["Username"],
            "author_name": thread["Author"]["DisplayName"],
            "thread_id": thread["Id"],
            "forum_id": thread["ForumId"],
            "subject": thread["Subject"],
            "created_date": thread["Content"]["CreatedDate"],
            "reply_count": thread["ReplyCount"],
            "latest_post_date": thread["LatestPostDate"],
            "latest_reply_author": thread["LatestForumReplyAuthorId"] }

But, when adding the yield row the print thread["Subject"] does not show Please ignore - test as it should.

def thread_generator(threads):
    tz = pytz.timezone('America/Chicago')
    POSTS_SINCE_HOURS = 24
    now_date = datetime.datetime.now(tz)
    time_stamp = now_date - datetime.timedelta(hours=POSTS_SINCE_HOURS)

    for thread in threads:
        last_post_time = convert_time(thread["LatestPostDate"])

        if last_post_time > time_stamp:
            print thread["Subject"]
            row = {
                "url": thread["Url"],
                "author_username": thread["Author"]["Username"],
                "author_name": thread["Author"]["DisplayName"],
                "thread_id": thread["Id"],
                "forum_id": thread["ForumId"],
                "subject": thread["Subject"],
                "created_date": thread["Content"]["CreatedDate"],
                "reply_count": thread["ReplyCount"],
                "latest_post_date": thread["LatestPostDate"],
                "latest_reply_author": thread["LatestForumReplyAuthorId"] }
            yield row

Why is this? Please ignore - test should still show with print thread["Subject"]. Makes no sense to me.

UPDATE: How the generators is called

def sanitize_threads(threads):
    for thread in thread_generator(threads):
        do stuff
        thread_batch.append(thread)
     return thread_batch

def get_unanswered_threads():
    slug = 'forums/threads/unanswered.json?PageSize=100'
    status_code, threads = do_request(slug)
    if status_code == 200:
        threads = threads["Threads"]
        thread_batch = sanitize_threads(threads)
        database_update(thread_batch)

Answer 1

Have you tried actually calling next() on the resultant generator? If you call the function with yield the same way you call the function without, in the yield case you'll get a generator object as a result. A generator doesn't evaluate what's inside it until you actually require a value of it, which can be done with next(generator).

For example:

>>> def nogen():
...   '''Create a list of values 0-3 and print each one as you build the list.'''
...   r = []
...   for i in range(3):
...     print(i)
...     r.append(i)
...   return r
... 
>>> def firstgen():
...   '''Create an iterator of values 0-3 and print each one as you yield the value.'''
...   for i in range(3):
...     print(i)
...     yield i
... 
>>> a = nogen() # This function is eager and does everything at once.
0
1
2
>>> a
[0, 1, 2]
>>> b = firstgen() # Note there's no output immediately after this line.
>>> list(b) # The output happens as the list coercion causes the generator to execute.
0
1
2
[0, 1, 2]
>>> c = firstgen() # You can also see this a step at a time using `next`
>>> next(c)
0
0
>>> next(c)
1
1
>>> next(c)
2
2
>>> next(c)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration