The yield + recursion in python annoyed me

Question

It is so complicated for me to understand when yield and recursion happen simultaneously。 I want to traverse file directory with my code:

import os
def doc_iter(fpath):
  if os.path.isdir(fpath):
    for child in os.listdir(fpath):
      child=os.path.join(fpath,child)
      print  "this is ",child
      for cn in doc_iter(child):
        print "i am here1"
        yield cn
        print "yiedl1",cn
  else:
    print "i am here2"
    yield fpath
    print "yield2",fpath

There is a diretory test ,three child directory test1,test2,test3 in it
In directory test1 ,there are two files test11,test12
In directory test2 ,there are two files test21,test22
In directory test3 ,there are two files test31,test32

    >>> a.next()
this is  /home/debian/test/test2
this is  /home/debian/test/test2/test22
i am here2
i am here1
i am here1
'/home/debian/test/test2/test22'
>>> a.next()
yiedl1 /home/debian/test/test2/test22
yiedl1 /home/debian/test/test2/test22
yield2 /home/debian/test/test2/test22
this is  /home/debian/test/test2/test21
i am here2
i am here1
i am here1
'/home/debian/test/test2/test21'
>>> a.next()
yiedl1 /home/debian/test/test2/test21
yiedl1 /home/debian/test/test2/test21
yield2 /home/debian/test/test2/test21
this is  /home/debian/test/test3
this is  /home/debian/test/test3/test32
i am here2
i am here1
i am here1
'/home/debian/test/test3/test32'
>>> a.next()
yiedl1 /home/debian/test/test3/test32
yiedl1 /home/debian/test/test3/test32
yield2 /home/debian/test/test3/test32
this is  /home/debian/test/test3/test31
i am here2
i am here1
i am here1
'/home/debian/test/test3/test31'
>>> a.next()
yiedl1 /home/debian/test/test3/test31
yiedl1 /home/debian/test/test3/test31
yield2 /home/debian/test/test3/test31
this is  /home/debian/test/test1
this is  /home/debian/test/test1/test11
i am here2
i am here1
i am here1
'/home/debian/test/test1/test11'
>>> a.next()
yiedl1 /home/debian/test/test1/test11
yiedl1 /home/debian/test/test1/test11
yield2 /home/debian/test/test1/test11
this is  /home/debian/test/test1/test12
i am here2
i am here1
i am here1
'/home/debian/test/test1/test12'
>>> a.next()
yiedl1 /home/debian/test/test1/test12
yiedl1 /home/debian/test/test1/test12
yield2 /home/debian/test/test1/test12
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

The output annoyed me,
1.The print in yield 1 is equal to in yield 2?
2.There are at least two yield statement to run,in textbook,it is saidwhen run into yield ,the program will be halt, the next next() make it continue?? 3.let's analyse the output of first next(),why there are two i am here1s after i am here2" 4.what is the function ofyield cn`?
5.how to draw the calculation tree in detail? 6.if you write a function to traverse the directory,

bottom=[]
import os 
def doc_iter(fpath):
    if os.path.isdir(fpath):
        for child in os.listdir(fpath):
            child=os.path.join(fpath,child)
            doc_iter(child)
    else:
        bottom.append(fpath)
    return bottom

the output is :

doc_iter("/home/debian/test")  

['/home/debian/test/test2/test22', '/home/debian/test/test2/test21', '/home/debian/test/test', '/home/debian/test/test3/test32', '/home/debian/test/test3/test31', '/home/debian/test/test~', '/home/debian/test/test1/test11', '/home/debian/test/test1/test12']

there are different between the function and iterator, in the function:

doc_iter(child)

in the iterator :

for cn in doc_iter(child):
    yield 

how complicated it is in this example!

Answer 1

So your example here is a bit contrived and leads to most if not all the difficulty you are having with understanding how yield works.

Each iteration of the walk_dir results in a new root path being passed to the entire function which will result in a new iteration of the function with its own yields

Now the reason you are seeing the same path printed is because each invocation of next moves the deeper into the directory structure.

So each next() returns the next directory or file. The stop iteration exception you are seeing is how the idiom

for x in something_that_yeilds() knows how to stop.

I would look at this pretty awesome write up on iterators, iterable, and generators.