CODEWITHSUNDEEP
pythondjangocsv
CMcorpse
CMcorpse

Reputation: 83

Taking *ANY csv column format into Django form

My code is currently hard coded to only accept csv files in the column format:

first_name,last_name,phone,email,address,company

However I would like users to be able to upload csv files that are in any* format order and naming scheme and correctly populate our forms. For example:

Email,LastName,FirstName,Company,Phone,Address

would be a valid column format. How would I go about doing that? Relevant code as follows:

        dr = csv.reader(open(tmp_file,'r')) 
        data_dict = {}

        headers = next(dr)
        print (headers)

        #skips over first line in csv
        iterlines = iter(dr) 
        next(iterlines)
        for row in iterlines:

            data_dict["first_name"] = row[0]
            #print(data_dict["first_name"])
            data_dict["last_name"] = row[1]
            #print(data_dict["last_name"])
            data_dict["phone"] = row[2]
            #print(data_dict["phone"])
            data_dict["email"] = row[3]
            #print(data_dict["email"])
            data_dict["address"] = row[4]
            #print(data_dict["address"])
            data_dict["company"] = row[5]
            #print(data_dict["company"])      
            #adds to form
            try:
                form = AddContactForm(data_dict)
                if form.is_valid():  
                     obj = form.save(commit=False)
                     obj.owner = request.user.username
                     first_name = form.cleaned_data.get(data_dict["first_name"])
                     last_name = form.cleaned_data.get(data_dict["last_name"])
                     phone = form.cleaned_data.get(data_dict["phone"])
                     email = form.cleaned_data.get(data_dict["email"])
                     address = form.cleaned_data.get(data_dict["address"])
                     company = form.cleaned_data.get(data_dict["company"])
                     obj.save() 
                else:
                 logging.getLogger("error_logger").error(form.errors.as_json())   

            except Exception as e:
                logging.getLogger("error_logger").error(repr(e))                    
                pass

Upvotes: 0

Views: 409

Answers (2)

snakecharmerb
snakecharmerb

Reputation: 55699

There are a number of approaches to handling inconsistent header names in the file. The best approach is to prevent it by rejecting such files at upload time, obliging the uploader to correct them. Assuming this isn't possible, you could try to transform the provided headers into what you want

import csv
import io
import re

with open(tmp_file, 'r') as f:
    reader = csv.reader  
    headers = next(reader)
# Make a new header list with placeholders
fixed_headers = [None * len(headers)]


for i, value in enumerate(headers) 
    fixed = re.sub(r'(\w+)(?<=[a-z])([A-Z]\w+)', r'\1_\2', v).lower()
    new_headers[i] = fixed

The regex finds capital letters in the middle of strings and inserts an underscore; then str.lower is called on the result (so values like 'Email' will be converted to 'email'.

Now rewrite the csv with the fixed headers:

with open(tmp_file, 'r') as f:
    reader = csv.reader(f)
    next(reader)
    new_file = io.StringIO()
    writer = csv.writer(new_file)
    writer.writerow(fixed_headers)
    for row in reader:
        writer.writerow(row)
# Rewind the file pointer
new_file.seek(0)

Use csv.DictReader to get rows as dictionaries of values mapped to headers.

dr = csv.DictReader(new_file) 

for data_dict in dr:

    #adds to form
    try:
        form = AddContactForm(data_dict)
        if form.is_valid():  
             obj = form.save(commit=False)
             obj.owner = request.user.username
             first_name = form.cleaned_data.get(data_dict["first_name"])
             last_name = form.cleaned_data.get(data_dict["last_name"])
             phone = form.cleaned_data.get(data_dict["phone"])
             email = form.cleaned_data.get(data_dict["email"])
             address = form.cleaned_data.get(data_dict["address"])
             company = form.cleaned_data.get(data_dict["company"])
             obj.save() 
        else:
         logging.getLogger("error_logger").error(form.errors.as_json())   

    except Exception as e:
        logging.getLogger("error_logger").error(repr(e))                    
        pass

Upvotes: 1

paulsbecks
paulsbecks

Reputation: 126

headers = "first_name,last_name,email"
headers_array = headers.split(',')
headers_map = {}

for i, column_name in enumerate(headers_array):
    headers_map[column_name] = i

#creates {'first_name': 0, 'last_name': 1, 'email': 2}

Now you can now use headers_map to get the row element

row[headers_map['first_name']]

Edit: For those loving one liners

headers_map = {column_name: i for i, column_name in enumerate(headers.split(','))}

Upvotes: 1

Related Questions