Close file after open() doesn't work

Question

When I run this function, it hangs because I am using the open() function to read a csv and I need to close it. I put the close() function where I think it's supposed to go, however it doesn't seem to be working. I've put the close() function in line with the "while True" indent, "for i in byte" indent and it doesn't work on either. What am I doing wrong?

def parse(text):
    #states
    is_token = False
    previous_character_is_escape = False
    no_quote_value = True
    quote_value = False


    file_base = os.path.basename('"app/csv_upload_directory/%s' % text)
    new_base = os.path.splitext(file_base)[0]

    row_counter = 1
    token_counter = 0
    fo = open("csv_upload_directory/%s_results.csv" % new_base, "w+")

    fo.write("Row %i" % row_counter + '\n')
    row_counter += 1
    with io.open(text,'rb',newline=None) as f:
        while True:
            byte = f.read(1)
            for i in byte:
                #print "%s,%s" % (no_quote_value,previous_character_is_escape)
                if is_token == False:
                    if i == '"':
                        fo.write(i)
                        token_counter = 0
                        is_token = True
                        no_quote_value = False
                        quote_value = True
                    elif i == '\n':
                        fo.write(",")
                        fo.write("%i" % token_counter)
                        fo.write('\n')
                        fo.write("Row %i" % (row_counter))
                        fo.write("\n")
                        token_counter = 0
                        row_counter += 1
                    elif i == ',':
                        fo.write(",")
                        fo.write("%i" % token_counter)
                        fo.write('\n')
                        token_counter = 0
                    elif no_quote_value == True:
                        fo.write(i)
                        token_counter += 1
                        is_token = True
                        quote_value = False
                    else:
                        fo.write(i)
                        token_counter += 1


                elif is_token == True:
                    # start of an escape sequence
                    if i == '\\':
                        fo.write(i)
                        token_counter += 1
                        previous_character_is_escape = True
                    # for line delimiter, the quoted values are being processed outside token
                    elif no_quote_value == True and i == '\n':
                        fo.write(",")
                        fo.write("%i" % token_counter)
                        fo.write('\n')
                        fo.write("Row %i" % (row_counter))
                        fo.write("\n")
                        token_counter = 0
                        row_counter += 1
                        is_token = False
                    # if token is not a quoted value but ends with quotes, and there is no escape character
                    elif no_quote_value == True and previous_character_is_escape == False and i == '"':
                        fo.write(i)
                        fo.write("This is a not a valid token, this is not a quoted value but there is an ending quote")
                        return False
                    # builds off previous_character_is_escape and captures any escape sequence
                    elif previous_character_is_escape == True:
                        fo.write(i)
                        token_counter += 1
                        previous_character_is_escape = False
                    # this type of quote is the end of token, returns back to other if statement
                    elif previous_character_is_escape == False and i == '"':
                        fo.write(i)
                        no_quote_value = True
                        quote_value = False
                        is_token = False
                    # if token starts as a quote but ends without quotes
                    elif quote_value == True and previous_character_is_escape == False and i == ',':
                        fo.write(i)
                        fo.write("This is not a valid token, there should be a quote at the end of this token")
                        return False
                    # this comma marks the end of a non quoted token, this invokes a newline
                    elif no_quote_value == True and previous_character_is_escape == False and i == ',':
                        fo.write(",")
                        fo.write("%i" % token_counter)
                        fo.write('\n')
                        token_counter = 0
                        is_token = False
                    elif no_quote_value == False and i == ',':
                        fo.write(i)
                        fo.write("DONG")
                    else:
                        fo.write(i)
                        token_counter += 1
        fo.close()

parse('example.csv')

Answer 1

From your comment, it sounds like closing the files is not actually your issue (though it is something you should take care of). The real issue is that your function never ends.

This is because you're looping forever, attempting to read one character per iteration. When the file has all been read, you're not noticing it that you get empty bytestrings from f.read(1). You should add some logic to break out of the loop when that happens.

A further issue: you're currently using a for loop to iterate over the one-byte string you're getting from read(1). There's no need for that loop, and it makes it hard to break out of the while loop using a break statement.

Try:

with io.open(text,'rb',newline=None) as f, fo:   # add fo to the with statement
    while True:
        i = f.read(1)        # read directly to "i", no need for the extra loop on bytes
        if i == '':          # check if read gave us an empty string (happens at EOF)
            break
        if is_token == True:
            # ...

Answer 2

So, you are opening two files here. One with the reference of fo and the other with f.

For fo, you are using open() method to perform file operations and you need to close it appropriately with fo.close().

However, this is not the case with f. Since you are using with... open() method, you don't need to close it as it efficiently handles closing the file after its completes code block execution. Read about the related documentation here.