CODEWITHSUNDEEP
pythoncsvfor-loopsyntax
Toma
Toma

Reputation: 171

Using a loop to open and process txt files with csv

I have data (mixed text and numbers in txt files) and I'd like to write a for loop that creates a list of lists, such that I can process the data from all the files using fewer lines.

So far I have written this:

import csv

path = (some path...)
files = [path + 'file1.txt',path + 'file2.txt', path + 
'file3.txt', ...]

for i in files:
    with open(i, 'r') as j:
        Reader = csv.reader(j)
        List = [List for List in Reader]

I think I overwrite List instead of creating a nested list, since I get Reader with size of 1 and a list that's with dimensions for one of the files.

My questions:

  1. Given that the files may contain different line numbers, is it the right approach to save some lines of code? (What could be done better?)
  2. I think the problem is in [List for List in Reader], is there a way to change it so I don't overwrite List? Something like add to List?

Upvotes: 0

Views: 489

Answers (2)

martineau
martineau

Reputation: 123463

You can use the list append() method to add to an existing list. Since csv.reader instances are iterable objects, you can just pass one of them to the method as shown below:

import csv
from pathlib import Path

path = Path('./')
filenames = ['in_file1.txt', 'in_file2.txt']  # etc ...
List = []

for filename in filenames:
    with open(path / filename, 'r', newline='') as file:
        List.append(list(csv.reader(file)))

print(List)

Update

An even more succinct way to do it would be to use something called a "list comprehension":

import csv
from pathlib import Path

path = Path('./')
filenames = ['in_file1.txt', 'in_file2.txt']  # etc ...
List = [list(csv.reader(open(path / filename, 'r', newline='')))
            for filename in filenames]

print(List)

Upvotes: 1

wasif
wasif

Reputation: 15478

Yes, use .append():

import numpy as np
import matplotlib.pyplot as plt
import csv
path = (some path...)
files = [path+x for x in ['FILESLIST']]

for i in files:
    with open(i, 'r') as j:
        Reader = csv.reader(j)
        List.append([L for L in Reader])

Upvotes: 1

Related Questions