load text file then load different words into different lists

Question

I have searched on strings, lists, append etc. but can't seem to handle this.

I have created some files from android based on selections done in an app. The output looks like this and is in a text file:

House 1,bwfront3,colorfront2,bwtilt3,colortilt3
House 2,bwfront6,colorfront6,bwtilt6,colortilt6
House 3,bwfront5,colorfront5,bwtilt5,colortilt5
House 4,bwfront4,colorfront4,bwtilt4,colortilt4
House 5,bwfront2,colorfront2,bwtilt2,colortilt2

the reason for the naming:

• I have 5 houses, where the user firstly selects from 9 'bwfront..' pictures. and then between color images and so on.

• THe exercise is to map different pictures to the 'house'.

• I now wish to load the text file(s) and count how many of each of the different 'bwfront' have been selected etc. To clarify, the user selects four times per 'house'.

• This will continue with all houses + types of pictures,but if any of you can get me started, I should be able to apply the solution to all my 23 files.

Does it make sense?

Answer 1

Possible way to parse such a file to count different bwfronts:

import csv
from collections import Counter

def count_bwfronts():
    """Counts occurrences of different bwfronts in 
    yourfile.txt. Returns a Counter object which
    maps different bwfront values to a number of their
    occurances in the file."""

    reader = csv.reader(open('yourfile.txt', 'rb'), delimiter=",")
    counter = Counter()
    counter.update(row[1] for row in reader)
    return counter

if __name__ == "__main__":
    print count_bwfronts()

As you might have guessed, each row taken from reader is just a list of strings which used to be separated by comma in your input file. In order to do more complex calculations you might want to rewrite generator expression into a loop.

Answer 2

This will loop through the file and extract each part of a line and store it in a list of 5-tuples, from there you can do whatever you need with the house/color/etc. This is just an example because it is hard to determine exactly what you need out of the script, but this should help get you started:

houses = open("houses.txt", "r")

house_list = []

for line in houses:

    # This assumes each line will always have 5 items separated by commas.
    house, bwfront, colorfront, bwtilt, colortilt = line.split(",")

    # This strips off the initial word ("bwfront") and just gives you the number
    house_num = house[6:]
    bwfront_num = bwfront[7:]
    colorfront_num = colorfront[10:]
    bwtilt_num = bwtilt[6:]
    colortilt_num = colortilt[9:]

    house_list.append((house_num, bwfront_num, colorfront_num, bwtilt_num, colortilt_num))

print house_list

Results in:

[('1', '3', '2', '3', '3'), ('2', '6', '6', '6', '6'), ('3', '5', '5', '5', '5'), ('4', '4', '4', '4', '4'), ('5', '2', '2', '2', '2')]

From there, you can do something like [h[1] for h in house_list] to get all of the bwfront numbers for each house, etc:

['3', '6', '5', '4', '2']

Answer 3

# First, create a dictionary for each column, that maps each
# value (eg colorfront2) to a list of house names.
results = [{}, {}, {}, {}]
for filename in os.listdir('/wherever'):
    s = open(os.path.join('/wherever', filename), 'rb').read()
    for line in s.split('\n'):
        if not line.strip(): continue # skip blank lines
        house, values = line.split(',', 1)
        values = values.split(',')
        assert len(values) == len(results) # sanity check
        for value, result in zip(values, results):
            if value not in result:
                result[value] = []
            result[value].append(house)


# Then, do something with it -- e.g., show them in order
for i, result in enumerate(results):
    print 'COLUMN %d' % i
    def sortkey(pair): return len(pair[1]) # number of houses
    for (val, houses) in sorted(result.items(), key=sortkey, reverse=True):
        print '   %20s occurs for %d houses' % (val, len(houses))