CODEWITHSUNDEEP
javaparsingjsoup
ZedBrannigan
ZedBrannigan

Reputation: 611

How do I correctly parse data using JSoup (java)

I want to parse the data out of this HTML (CompanyName, Location, jobDescription,...) using JSoup (java). I get stuck when trying to iterate the joblistings

The extract from the HTML is one of many "JOBLISTING" divs which I want to iterate and extract the Data out of it. I just can't handle how to iterate the specific div objects. Sorry for this noob question, but maybe someone can help me who already knows which function to use. Select?

<div class="between_listings"><!-- local.spacer --></div>

<div id="joblisting-2944914" class="joblisting listing-even listing-even company-98028 " itemscope itemtype="http://schema.org/JobPosting">


<div class="company_logo" itemprop="hiringOrganization" itemscope itemtype="http://schema.org/Organization">
     <a href="/stellenangebote-des-unternehmens--Delivery-Hero-Holding-GmbH--98028.html" title="Jobs Delivery Hero Holding GmbH" itemprop="url">
       <img src="/upload_de/logo/D/logoDelivery-Hero-Holding-GmbH-98028DE.gif" alt="Logo Delivery Hero Holding GmbH" itemprop="image" width="160" height="80" />
     </a>
</div>


<div class="job_info">


<div class="h3 job_title">
   <a id="jobtitle-2944914" href="/stellenangebote--Junior-Business-Intelligence-Analyst-CRM-m-f-Berlin-Delivery-Hero-Holding-GmbH--2944914-inline.html?ssaPOP=204&ssaPOR=203" title="Arbeiten bei Delivery Hero Holding GmbH" itemprop="url">
      <span itemprop="title">Junior Business Intelligence Analyst / CRM (m/f)</span>
   </a>
</div>

<div class="h3 company_name" itemprop="hiringOrganization" itemscope itemtype="http://schema.org/Organization">

    <span itemprop="name">Delivery Hero Holding GmbH</span>

</div>

</div>




<div class="job_location_date">

    <div class="job_location target-location">
         <div class="job_location_info" itemprop="jobLocation" itemscope itemtype="http://schema.org/Place">


            <div class="h3 locality" itemprop="address" itemscope itemtype="http://schema.org/PostalAddress">
                  <span itemprop="addressLocality"> Berlin</span>
            </div>


            <span class="location_actions">
                <a href="javaScript:PopUp('http://www.stepstone.de/5/standort.html?OfferId=2944914&ssaPOP=203&ssaPOR=203','resultList',800,520,1)" class="action_showlistingonmap showlabel" title="Google Maps" itemprop="maps">
                   <span class="location-icon"><!-- --></span>
                   <span class="location-label">Google Maps</span>
                </a>
            </span>

          </div>
       </div>

       <div class="job_date_added" itemprop="datePosted"><time datetime="2014-07-04">04.07.14</time></div>
</div>


<div class="job_actions">


</div>

</div>
<div class="between_listings"><!-- local.spacer --></div>

File input = new File("C:/Talend/workspace/WEBCRAWLER/output/keywords_SOA.txt"); // Load file into extraction1 Document ParseResult = Jsoup.parse(input, "UTF-8", "http://example.com/"); Elements jobListingElements = ParseResult.select(".joblisting"); for (Element jobListingElement: jobListingElements) { jobListingElement.select(".companyName span[itemprop=\"name\"]"); // other element properties System.out.println(jobListingElements);

Java code:

File input = new File("C:/Talend/workspace/WEBCRAWLER/output/keywords_SOA.txt");
// Load file into extraction1       
Document ParseResult = Jsoup.parse(input, "UTF-8", "http://example.com/");                          
Elements jobListingElements = ParseResult.select(".joblisting");        
for (Element jobListingElement: jobListingElements) {         
    jobListingElement.select(".companyName span[itemprop=\"name\"]");         
    // other element properties         
    System.out.println(jobListingElements);
}

Thank you!

Upvotes: 0

Views: 1719

Answers (1)

meistermeier
meistermeier

Reputation: 8262

So you got your Jsoup document right? Than it seems pretty easy if the css class joblisting does not appear anywhere else.

Document document = Jsoup.parse(new File("d:/bla.html"), "utf-8");
Elements elements = document.select(".joblisting");
for (Element element : elements) {
    Elements jobTitleElement = element.select(".job_title span");
    Elements companyNameElement = element.select(".company_name spanspan[itemprop=name]");
    String companyName = companyNameElement.text();
    String jobTitle = jobTitleElement.text();

    System.out.println(companyName);
    System.out.println(jobTitle);
}

I don't know why the attribute [itemprop*=\"name\"] selector does not find the span (Further reading: http://jsoup.org/cookbook/extracting-data/selector-syntax )

Got it: span[itemprop=name] without any quotes or escapes. Other attributes or values also should work to get a more specific selection.

Upvotes: 2

Related Questions