CODEWITHSUNDEEP
web-scrapingpython-requestsweb-crawler
송민찬
송민찬

Reputation: 35

Is there anyway to scrape specific information

How can I get that 'star_recom' information which is located in Base Url?

Star_recom's type is number(ex.49%) as you can see in the BaseUrl.

please, check the code and tell me if there is any problem. 

BaseUrl = 'https://www.jobplanet.co.kr/companies/90364/'
req = requests.get(BaseUrl)
soup = BeautifulSoup(req.text,'html.parser')
body = soup.find("div",{"class":"body_wrap"})
sbody= body.find("dl", {"class":"rate_bar_set"})


star_recom = body.find('div', class_='pie1').find('span', class_='txt_point').text.strip()

Upvotes: 0

Views: 35

Answers (1)

Ahmed Soliman
Ahmed Soliman

Reputation: 1710

your Code is correct but it won't return anything because the data you are trying to scrape is being written by a JavaScript function in the body.

<div class="review_stats-pagination"></div>
<script>
  ;(function($){
    // Fill animations


    // Dummy data
    var data = [
      {label:'직원의 기업 추천율',val : 0.85},
      {label:'직원이 전망하는 성장 가능성',val : 0.81},
      {label:'이 기업의 CEO 지지율',val : 0.93 }
    ];

You can try :

import re
BaseUrl = 'https://www.jobplanet.co.kr/companies/90364/'
req = requests.get(BaseUrl).text

# extract the values as it is in the dom
spans = re.findall( r',val\s*:\s*(.*?)}', req )
print(spans)

Output :

['0.85', '0.81', '0.93 ']

and If you want the exact same info :

# convert it to look like the data displayed on the html
text_as_website = ['{}%'.format(int(float(span) * 100)) for span in spans]
print(text_as_website)

Output :

['85%', '81%', '93%']

Upvotes: 1

Related Questions