CODEWITHSUNDEEP
javascriptpythonpandasdataframeurl
h354
h354

Reputation: 45

Removing urls from a data-frame column with targetblank tag

I want to remove url's from a column in a data-frame. The column I am interested in is called comment, and example entry in comment is:

|comment                                 |
|:--------------------------------------:|
| """Drone Strikes Up 432 Percent Under. |
|Donald Trump"" by Joe Wolverton, II,    |
|J.D.                                    |
|<a                                      |
|href=""https://www.thenewamerican.com/  |
|usne                                    |
|ws/foreign-policy/item/25604-drone-     |
|strikes-up-432-percent-under-donald-    |
|trump""                                 |
|title=""https://www.thenewamerican.com/ |
|usn                                     |
|ews/foreign-policy/item/25604-drone-    |
|strikes-up-432-percent-under-donald-    |
|trump""                                 |
|target=""_blank"">https://www.thenewamer|
|c                                       |
|an.com/usnews/foreign-policy/item/25604-|
|drone-st...</a><br/>""Trump is weighing |
| major escalation in Yemen's devastating| 
|war<br/>The war has already killed at   |
|least 10,000, displaced 3 million, and. | 
|left millions more at risk of famine."" |
|<br/>"                                  |

This above entry shows the issue I am trying to solve. I want to completely remove:

<a href=""https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-strikes-up-432-percent-under-donald-trump"" title=""https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-strikes-up-432-percent-under-donald-trump"" target=""_blank"">https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-st...</a>

I've tried:

df['comment'] = df['comment'].replace(r'https\S+', ' ', regex=True).replace(r'www\S+', ' ', regex=True).replace(r'http\S+', ' ', regex=True)

However this likes with me in

href title targetblank com

Upvotes: 1

Views: 44

Answers (2)

Danil Perestoronin
Danil Perestoronin

Reputation: 1153

You can try to perform a substitution using a regular expression, use re.sub.

For example:

import re

s = """Drone Strikes Up 432 Percent Under. Donald Trump"" by Joe Wolverton, II, J.D. <a href=""https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-strikes-up-432-percent-under-donald-trump"" title=""https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-strikes-up-432-percent-under-donald-trump"" target=""_blank"">https://www.thenewamerc an.com/usnews/foreign-policy/item/25604-drone-st...</a><br/>""Trump is weighing  major escalation in Yemen's devastating war<br/>The war has already killed at   least 10,000, displaced 3 million, and.  left millions more at risk of famine."" <br/>"""

print(re.sub('<a\s[^>]*.*?<\/a>', '', s))

In your case, you can use .applay to achieve your goal:

import re
import pandas as pd


df = pd.DataFrame({'comment': ["Drone Strikes Up 432 Percent Under. Donald Trump"" by Joe Wolverton, II, J.D. <a href=""https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-strikes-up-432-percent-under-donald-trump"" title=""https://www.thenewamerican.com/usnews/foreign-policy/item/25604-drone-strikes-up-432-percent-under-donald-trump"" target=""_blank"">https://www.thenewamerc an.com/usnews/foreign-policy/item/25604-drone-st...</a><br/>""Trump is weighing  major escalation in Yemen's devastating war<br/>The war has already killed at   least 10,000, displaced 3 million, and.  left millions more at risk of famine."" <br/>"""]})

df['comment'] = df['comment'].apply(lambda x: re.sub('<a\s[^>]*.*?<\/a>', '', x))

print(df)

Upvotes: 1

Corralien
Corralien

Reputation: 120559

Try:

df['comment'] = df['comment'].str.replace('<a\s[^>]*.*?<\/a>', '')

Output:

>>> df.loc[0, 'comment']

'Drone Strikes Up 432 Percent Under. Donald Trump"" by Joe Wolverton, II, J.D. <br/>""Trump is weighing  major escalation in Yemen\'s devastating war<br/>The war has already killed at   least 10,000, displaced 3 million, and.  left millions more at risk of famine."" <br/>'

Upvotes: 2

Related Questions