CODEWITHSUNDEEP
pythonstringlistcsv
J'e
J'e

Reputation: 3706

python: searching a text file for values in a list

im trying to search a large text file for a list of words. rather than running a command over and over for each word, I thought a list would be easier but I'm not sure how to go about it. the script below more or less work with the string value but I would like to replace the 'string' below with every value for the 'dict' list. 

import csv

count = 0
dic = open('dictionary','r') #changed from "dict" in original post
reader = csv.reader(dic)
allRows = [row for row in reader]
with open('bigfile.log','r') in inF:
   for line in inF:
      if 'string' in line: #<---replace the 'string' with dict values
         count += 1
count

Upvotes: 2

Views: 4981

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1121744

Convert your file to a set instead:

 with open('dictionary','r') as d:
     sites = set(l.strip() for l in d)

Now you can do efficient membership tests per line, provided you can split your lines:

with open('bigfile.log','r') as inF:
   for line in inF:
       elements = line.split()
       if sites.intersection(elements):
           count += 1

Upvotes: 4

Related Questions