Reputation: 45
I want to remove url's from a column in a data-frame. The column I am interested in is called comment, and example entry in comment is:
|comment |
|:--------------------------------------:|
| """Drone Strikes Up 432 Percent Under. |
|Donald Trump"" by Joe Wolverton, II, |
|J.D. |
|<a |
|href=""https://www.thenewamerican.com/ |
|usne |
|ws/foreign-policy/item/25604-drone- |
|strikes-up-432-percent-under-donald- |
|trump"" |
|title=""https://www.thenewamerican.com/ |
|usn |
|ews/foreign-policy/item/25604-drone- |
|strikes-up-432-percent-under-donald- |
|trump"" |
|target=""_blank"">https://www.thenewamer|
|c |
|an.com/usnews/foreign-policy/item/25604-|
|drone-st...</a><br/>""Trump is weighing |
| major escalation in Yemen's devastating|
|war<br/>The war has already killed at |
|least 10,000, displaced 3 million, and. |
|left millions more at risk of famine."" |
|<br/>" |
This above entry shows the issue I am trying to solve. I want to completely remove:
<a href=""https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-strikes-up-432-percent-under-donald-trump"" title=""https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-strikes-up-432-percent-under-donald-trump"" target=""_blank"">https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-st...</a>
I've tried:
df['comment'] = df['comment'].replace(r'https\S+', ' ', regex=True).replace(r'www\S+', ' ', regex=True).replace(r'http\S+', ' ', regex=True)
However this likes with me in
href title targetblank com
Upvotes: 1
Views: 44
Reputation: 1153
You can try to perform a substitution using a regular expression, use re.sub.
For example:
import re
s = """Drone Strikes Up 432 Percent Under. Donald Trump"" by Joe Wolverton, II, J.D. <a href=""https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-strikes-up-432-percent-under-donald-trump"" title=""https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-strikes-up-432-percent-under-donald-trump"" target=""_blank"">https://www.thenewamerc an.com/usnews/foreign-policy/item/25604-drone-st...</a><br/>""Trump is weighing major escalation in Yemen's devastating war<br/>The war has already killed at least 10,000, displaced 3 million, and. left millions more at risk of famine."" <br/>"""
print(re.sub('<a\s[^>]*.*?<\/a>', '', s))
In your case, you can use .applay
to achieve your goal:
import re
import pandas as pd
df = pd.DataFrame({'comment': ["Drone Strikes Up 432 Percent Under. Donald Trump"" by Joe Wolverton, II, J.D. <a href=""https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-strikes-up-432-percent-under-donald-trump"" title=""https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-strikes-up-432-percent-under-donald-trump"" target=""_blank"">https://www.thenewamerc an.com/usnews/foreign-policy/item/25604-drone-st...</a><br/>""Trump is weighing major escalation in Yemen's devastating war<br/>The war has already killed at least 10,000, displaced 3 million, and. left millions more at risk of famine."" <br/>"""]})
df['comment'] = df['comment'].apply(lambda x: re.sub('<a\s[^>]*.*?<\/a>', '', x))
print(df)
Upvotes: 1
Reputation: 120559
Try:
df['comment'] = df['comment'].str.replace('<a\s[^>]*.*?<\/a>', '')
Output:
>>> df.loc[0, 'comment']
'Drone Strikes Up 432 Percent Under. Donald Trump"" by Joe Wolverton, II, J.D. <br/>""Trump is weighing major escalation in Yemen\'s devastating war<br/>The war has already killed at least 10,000, displaced 3 million, and. left millions more at risk of famine."" <br/>'
Upvotes: 2