CODEWITHSUNDEEP
c#regexparsinghtml-agility-packfizzler
J.Skeet
J.Skeet

Reputation: 1

Parsing HTML page using C# having problems

I've been trying to accomplish something using HtmlAgilityPack, Fizzler and Regular Expressions but had no luck.

The page I am trying to scrape and parse to elements is here http://www.sczg.unizg.hr/student-servis/vijest/2015-04-14-poslovi-u-administraciji/

Example of an item in item list:
<p> 
  <b>1628/ SomeBoldedTitle
  </b> 
    Some Description. 
    Some price 20,00kuna. 
  <strong>Contact somenumber
       098/1234-567 some mail
  </strong> 
</p>

I would like to parse this item to:

Here is some code I tried to get at least some output, I expected all p elements with b's but nothing has come out.

 using System;
    using HtmlAgilityPack;
    using Fizzler.Systems.HtmlAgilityPack;

namespace Sample
{
    class Program
    {

        static void Main(string[] args)
        {
            var web = new HtmlWeb();
            var document = web.Load("http://www.sczg.unizg.hr/student-servis/vijest/2015-04-14-poslovi-u-administraciji/");
            var page = document.DocumentNode;
                foreach (var item in page.QuerySelectorAll("p.item"))
            {
                Console.WriteLine(item.QuerySelector("p:has(b)").InnerHtml);
            }
        }
    }
}

Here is the link to the fizzler "documentation" I used to get this code https://fizzlerex.codeplex.com/

Upvotes: 0

Views: 95

Answers (1)

Ro Yo Mi
Ro Yo Mi

Reputation: 15000

Forward

I recommend using an HTML parsing module because HTML can lead to some crazy edge cases that will really skew your data. But if you're in control of your source text and still want/need to use a regex, I offer this possible solution.

Description

Given the following text

Example of an item in item list:
<p> 
  <b>1628/ SomeBoldedTitle
  </b> 
    Some Description. 
    Some price 20,00kuna. 
  <strong>Contact somenumber
       098/1234-567 some mail
  </strong> 
</p>

This Regex 

<p>(?:(?!<p>).)*<b>([0-9]+)/\s*((?:(?!</b>).)*?)\s*</b>\s*((?:(?!<strong>|<b>).)*?)\s*<(?:strong|b)>\s*((?:(?!</).)*?)\s*</

Will parse your text into the following capture groups:

  • Group 0 will be most of the string
  • Group 1 will be the multi digit code
  • Group 2 will be the title
  • Group 3 will be the description
  • Group 4 will be the phone number

Capture Groups

[0][0] = <p> 
  <b>1628/ SomeBoldedTitle
  </b> 
    Some Description. 
    Some price 20,00kuna. 
  <strong>Contact somenumber
       098/1234-567 some mail
  </
[0][1] = 1628
[0][2] = SomeBoldedTitle
[0][3] = Some Description. 
    Some price 20,00kuna.
[0][4] = Contact somenumber
       098/1234-567 some mail

Explained

Regular expression visualization

Note: right click the image and select view in new window.

NODE                     EXPLANATION
----------------------------------------------------------------------
  <p>                      '<p>'
----------------------------------------------------------------------
  (?:                      group, but do not capture (0 or more times
                           (matching the most amount possible)):
----------------------------------------------------------------------
    (?!                      look ahead to see if there is not:
----------------------------------------------------------------------
      <p>                      '<p>'
----------------------------------------------------------------------
    )                        end of look-ahead
----------------------------------------------------------------------
    .                        any character
----------------------------------------------------------------------
  )*                       end of grouping
----------------------------------------------------------------------
  <b>                      '<b>'
----------------------------------------------------------------------
  (                        group and capture to \1:
----------------------------------------------------------------------
    [0-9]+                   any character of: '0' to '9' (1 or more
                             times (matching the most amount
                             possible))
----------------------------------------------------------------------
  )                        end of \1
----------------------------------------------------------------------
  /                        '/'
----------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                           more times (matching the most amount
                           possible))
----------------------------------------------------------------------
  (                        group and capture to \2:
----------------------------------------------------------------------
    (?:                      group, but do not capture (0 or more
                             times (matching the least amount
                             possible)):
----------------------------------------------------------------------
      (?!                      look ahead to see if there is not:
----------------------------------------------------------------------
        </b>                     '</b>'
----------------------------------------------------------------------
      )                        end of look-ahead
----------------------------------------------------------------------
      .                        any character
----------------------------------------------------------------------
    )*?                      end of grouping
----------------------------------------------------------------------
  )                        end of \2
----------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                           more times (matching the most amount
                           possible))
----------------------------------------------------------------------
  </b>                     '</b>'
----------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                           more times (matching the most amount
                           possible))
----------------------------------------------------------------------
  (                        group and capture to \3:
----------------------------------------------------------------------
    (?:                      group, but do not capture (0 or more
                             times (matching the least amount
                             possible)):
----------------------------------------------------------------------
      (?!                      look ahead to see if there is not:
----------------------------------------------------------------------
        <strong>                 '<strong>'
----------------------------------------------------------------------
       |                        OR
----------------------------------------------------------------------
        <b>                      '<b>'
----------------------------------------------------------------------
      )                        end of look-ahead
----------------------------------------------------------------------
      .                        any character
----------------------------------------------------------------------
    )*?                      end of grouping
----------------------------------------------------------------------
  )                        end of \3
----------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                           more times (matching the most amount
                           possible))
----------------------------------------------------------------------
  <                        '<'
----------------------------------------------------------------------
  (?:                      group, but do not capture:
----------------------------------------------------------------------
    strong                   'strong'
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    b                        'b'
----------------------------------------------------------------------
  )                        end of grouping
----------------------------------------------------------------------
  >                        '>'
----------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                           more times (matching the most amount
                           possible))
----------------------------------------------------------------------
  (                        group and capture to \4:
----------------------------------------------------------------------
    (?:                      group, but do not capture (0 or more
                             times (matching the least amount
                             possible)):
----------------------------------------------------------------------
      (?!                      look ahead to see if there is not:
----------------------------------------------------------------------
        </                       '</'
----------------------------------------------------------------------
      )                        end of look-ahead
----------------------------------------------------------------------
      .                        any character
----------------------------------------------------------------------
    )*?                      end of grouping
----------------------------------------------------------------------
  )                        end of \4
----------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                           more times (matching the most amount
                           possible))
----------------------------------------------------------------------
  </                       '</'

Upvotes: 1

Related Questions