CODEWITHSUNDEEP
htmlperlweb-scraping
user2027571
user2027571

Reputation: 49

how to bypass html escape signs and extract text only from html file in perl using web::scraper

I am trying to extract the text only from the html page and want to ignore or bypass the html escape signs "<" and ">". I am copying the part the html page that i used for extraction of text:

        <table class="reference">
    <tr>
    <th align="left" width="25%">Tag</th>
    <th align="left" width="75%">Description</th>
    </tr>
    <tr>
    <td><a href="tag_comment.asp">&lt;!--...--&gt;</a></td>
    <td>Defines a comment</td>
    </tr>
    <tr>
    <td><a href="tag_doctype.asp">&lt;!DOCTYPE&gt;</a>&nbsp;</td>
    <td>Defines the document type</td>
    </tr>
    <tr>
    <td><a href="tag_a.asp">&lt;a&gt;</a></td>
    <td>Defines a hyperlink</td>
    </tr>
    <tr>
    <td><a href="tag_abbr.asp">&lt;abbr&gt;</a></td>
    <td>Defines an abbreviation</td>
    </tr>
    <tr>
...

My perl code is:

my $urlToScrape = "http://www.w3schools.com/tags/";

# prepare data
my $teamsdata = scraper {
process "table.reference > tr > td > a ", 'tags[]' => 'TEXT';
process "table.reference > tr > td > a ", 'urls[]' => '@href';
};

# scrape the data
my $res = $teamsdata->scrape(URI->new($urlToScrape));

print "<HTML_tags>\n";
for my $i ( 0 .. $#{$res->{urls}}) {
 print FILE "   <tag_Name> $res->{tags}[$i] </tag_Name>\n ";
}
print "</HTML_tags>\n";

The output I get is the following:

<HTML_tags>
    <tag_Name> <!--...--> </tag_Name>
        <tag_Name> <!DOCTYPE> </tag_Name>
        <tag_Name> <a> </tag_Name>
        <tag_Name> <abbr> </tag_Name>
</HTML_tags>

whereas I want output as:

<HTML_tags>
    <tag_Name> !--...-- </tag_Name>
        <tag_Name> !DOCTYPE </tag_Name>
        <tag_Name> a </tag_Name>
        <tag_Name> abbr </tag_Name>
</HTML_tags>

Can anyone tell what do I have to change inorder to get the above output? Many Thanks.

Upvotes: 0

Views: 645

Answers (1)

Krishnachandra Sharma
Krishnachandra Sharma

Reputation: 1342

Brute Force:

$res->{tags}[$i] =~ s/[\<\>]//gs; ## Added line 
print FILE "   <tag_Name> $res->{tags}[$i] </tag_Name>\n ";

Upvotes: 2

Related Questions