CODEWITHSUNDEEP
rubyregexstringencoding
btw
btw

Reputation: 7154

Problem With Regular Expression to Remove HTML Tags

In my Ruby app, I've used the following method and regular expression to remove all HTML tags from a string:

str.gsub(/<\/?[^>]*>/,"")

This regular expression did just about all I was expecting it to, except it caused all quotation marks to be transformed into &#8220; and all single quotes to be changed to &#8221; .

What's the obvious thing I'm missing to convert the messy codes back into their proper characters?

Edit: The problem occurs with or without the Regular Expression, so it's clear my problem has nothing to do with it. My question now is how to deal with this formatting error and correct it. Thanks!

Upvotes: 5

Views: 4905

Answers (5)

vladr
vladr

Reputation: 66661

Use CGI::unescapeHTML after you perform your regular expression substitution:

CGI::unescapeHTML(str.gsub(/<\/?[^>]*>/,""))

See http://www.ruby-doc.org/core/classes/CGI.html#M000547

In the above code snippet, gsub removes all HTML tags. Then, unescapeHTML() reverts all HTML entities (such as <, &#8220) to their actual characters (<, quotes, etc.)

With respect to another post on this page, note that you will never ever be passed HTML such as

<tag attribute="<value>">2 + 3 < 6</tag>

(which is invalid HTML); what you may receive is, instead:

<tag attribute="&lt;value&gt;">2 + 3 &lt; 6</tag>

The call to gsub will transform the above to:

2 + 3 &lt; 6

And unescapeHTML will finish the job:

2 + 3 < 6

Upvotes: 5

lazyfly
lazyfly

Reputation:

I've run into a similar problem with character changes, this happened when my code ran through another module that enforced UTF-8 encoding and then when it came back, I had a different file (slurped array of lines) on my hands.

Upvotes: 0

Georg Sch&#246;lly
Georg Sch&#246;lly

Reputation: 126105

This regular expression did just about all I was expecting it to, except it caused all quotation marks to be transformed into “ and all single quotes to be changed to ” .

This doesn't sound as if the RegExp would be doing this. Are you sure it's different before?

See this question here for information about the problem, it has got an excellent answer:
Get non UTF-8 form fields as UTF-8 in php.

Upvotes: 2

Sniggerfardimungus
Sniggerfardimungus

Reputation: 11831

You're going to run into more trouble when you see something like:

<doohickey name="<foobar>">

You'll want to apply something like:

gsub(/<[^<>]*>/, "")

...for as long as the pattern matches.

Upvotes: 2

Tim
Tim

Reputation: 1874

You could use a multi-pass system to get the results you are looking for.

After running your regular expression, run an expression to convert &8220; to quotes and another to convert &8221; to single quotes.

Upvotes: -3

Related Questions