Reputation: 105
I have a function which searches for relationships inside each paragraph of a text file. If found, it returns the relationship found with the paragraph and searches if there is any colour present in that paragraph. A sample of the metadata.csv file which searches for relationship in paragraphs is as follows:
Blister Base Web PVC/PVDC
Blister Foil Aluminium
Blister Base Web PVC/PVDC
Blister Foil Aluminium
Vial Glass Borosilicate Glass
Vial Stopper Bromobutyl Rubber
Vial Cap Aluminium
The sample text file is as follows:
The tablets are filled into cylindrically shaped bottles made of white coloured
polyethylene. The volumes of the bottles depend on the tablet strength and amount of
tablets, ranging from 20 to 175 ml. The screw type cap is made of white coloured
polypropylene and is equipped with a tamper proof ring.
PVC/PVDC blister pack
Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tablet
is filled into a separate blister and a lidding foil of aluminium is welded on. The blisters
are opened by pressing the tablets through the lidding foil. PVDC foil is in contact with
the tablets.
Aluminium blister pack
Blisters are made in a cold-forming process from an aluminium base web. Each tablet is
filled into a separate blister and a lidding foil of aluminium is welded on. The blisters
are opened by pressing the tablets through the lidding foil.
The function to extract this is as follows:
import csv
import re
import os
#import pdb
def extractor(filepath):
#pdb.set_trace()
#pdb.set_trace()
TEXT_WITHOUT_COLOUR = 'Stage {counter} : Package Description: {sen} Values: {values}'
TEXT_WITH_COLOUR = TEXT_WITHOUT_COLOUR + ',' ' Colour: {colour}'
colours = ['White', 'Yellow', 'Blue', 'Green', 'Black', 'Brown', 'Silver', 'Purple', 'Navy blue', 'Gray', 'Orange', 'Maroon', 'pink', 'colourless', 'blue']
counter = 1
result = []
unique_desc = [] #every unique description is stored
outputs = []
with open(filepath, encoding='utf-8') as f:
data=f.read()
paragraphs=data.split("\n\n")
inputfile = r"C:\Users\metadata.csv"
inputm = []
with open(inputfile, "r") as f:
reader = csv.reader(f, delimiter="\t")
for row in reader:
#types = row.split(',')
inputm.append(row)
final_ref = []
for lists in inputm:
final_ref.append(str(lists[0]).split(','))
def is_missing(words, sen):
for w in words:
if w.lower() not in sen.lower():
return True
return False
#pdb.set_trace()
for sen in paragraphs:
for words in final_ref:
# print(sen)
# print("HHHHHHHHHHHHHHHHHHHHXXXXXXXXXXXXXX")
if is_missing(words, sen):
continue
kwargs = {
'counter': counter,
'sen': sen,
'values': str(words)
}
if (words[0] == 'Bottle') or (words[0]=='Vial') or (words[0] =='Container') or (words[0] =='Ampoules') or (words[0] =='Occlusive Dressing'):
for wd in colours:
if wd.lower() in sen.lower():
kwargs['colour'] = wd
break
text_const = TEXT_WITH_COLOUR
else:
text_const = TEXT_WITHOUT_COLOUR
result.append(text_const.format(**kwargs).replace('\n', '').replace('\t', ''))
#
for desc in result:
compare = re.search(r'Package Description:(.*?)Values:',desc).group(1).replace(' ','') #clean spaces
if compare in unique_desc:
group = str(unique_desc.index(compare)+1) #index starts in 0 and group in 1
desc = re.sub('Stage \d','Group '+group, desc)
outputs.append(desc)
else:
unique_desc.append(compare)
group = str(len(unique_desc)) #new group
desc = re.sub('Stage \d','Group '+group, desc)
outputs.append(desc)
counter+=1
#continue
#break
#counter += 1
# return output
return (sorted(set(outputs)))
For few files, I am getting the error "
File "<ipython-input-56-f725b589f198>", line 63, in extractor
result.append(text_const.format(**kwargs).replace('\n', '').replace('\t', ''))
KeyError: 'colour'
Any idea on how to fix this.
Upvotes: 0
Views: 650
Reputation: 8045
You can't assign dictionary keys to values with .get()
:
kwargs.get['colour'] = wd
should be
kwargs['colour'] = wd
Edit regarding comments below:
if (words[0] == 'Bottle') or (words[0]=='Vial') or (words[0] =='Container') or (words[0] =='Ampoules') or (words[0] =='Occlusive Dressing'):
for wd in colours:
if wd.lower() in sen.lower():
kwargs['colour'] = wd
break
text_const = TEXT_WITH_COLOUR
else:
text_const = TEXT_WITHOUT_COLOUR
should be:
text_const = TEXT_WITHOUT_COLOUR
if (words[0] == 'Bottle') or (words[0]=='Vial') or (words[0] =='Container') or (words[0] =='Ampoules') or (words[0] =='Occlusive Dressing'):
for wd in colours:
if wd.lower() in sen.lower():
kwargs['colour'] = wd
text_const = TEXT_WITH_COLOUR
break
Upvotes: 2