misguided
misguided

Reputation: 3799

reading large string and split each word taking too much time in python

I am reading a dataframe column having comments. The data is taking forever to read using the code below. Is there a way to make this faster ?

for val in df.Description:
    val = str(val)
    tokens = val.split()  
    for i in range(len(tokens)):
        tokens[i] = tokens[i].lower()  
        for words in tokens:
            comment = comment + words + ''

df.Description is a column of comments (basically email text)

Upvotes: 0

Views: 193

Answers (1)

Scott
Scott

Reputation: 5848

Update: Assuming df.Description is your column, this might be helpful:

arr_string = df.Description.astype(str).values.tolist()
for val in arr_string:
    for words in val:
            comment = ''.join([comment, words])

Take a look at this.

Upvotes: 2

Related Questions