CODEWITHSUNDEEP
pythonpandasscikit-learnnlp
pankaj
pankaj

Reputation: 460

how to find and write a frequency of occurrence of words inside text data into csv file using pandas

How Can I create csv file containing words and its frequency of occurrence in python .

I removed the stop words, tokenized and countvectorized the text data

My code

 data['Clean_addr'] = data['Adj_Addr'].apply(lambda x: ' '.join([item.lower() for item in x.split()]))
        data['Clean_addr']=data['Clean_addr'].apply(lambda x:"".join([item.lower() for item in x if  not  item.isdigit()]))
        data['Clean_addr']=data['Clean_addr'].apply(lambda x:"".join([item.lower() for item in x if item not in string.punctuation]))
        data['Clean_addr'] = data['Clean_addr'].apply(lambda x: ' '.join([item.lower() for item in x.split() if item not in (new_stop_words)]))
        cv = CountVectorizer( max_features = 200,analyzer='word')
        cv_addr = cv.fit_transform(data.pop('Clean_addr'))

Sample Dump of the File I am using

https://www.dropbox.com/s/allhfdxni0kfyn6/Test.csv?dl=0

**Expected output**
Word       Freq
Industry    40
Limited     23
House       45
flat        56

Upvotes: 1

Views: 141

Answers (1)

jezrael
jezrael

Reputation: 862591

You can create DataFrame first and then sum:

df1 = pd.DataFrame(cv_addr.todense(), columns=cv.get_feature_names())
df1 = df1.sum().rename_axis('Word').reset_index(name='Freq')

Upvotes: 1

Related Questions