CODEWITHSUNDEEP
htmlrxpathweb-scrapingrvest
Barsssk
Barsssk

Reputation: 93

Rvest Web Scraping Brings Empty Data Table With Column Names only

I have been trying to scrape share market usinf "rvest" package from this url: http://finans.mynet.com/borsa/canliborsa/#A which requires to sign-up. I created dummy account for you to try. Username and password below are real and work okay. What I have come up with is the following:

library("rvest")
library("data.table")

url<- "http://uyeler.mynet.com/login/login.asp?rurl=http%3A%2F%2Ffinans.mynet.com%2Fborsa%2Fcanliborsa%2F&formname=finans#A"
session<-html_session(url)
form<- html_form(session)[[1]]

login<- set_values(form, "username" ="muharrem_babaogul_1991","password"="q1w2e3")
submit_form(session,login)

jumped<-jump_to(session,url = 'http://finans.mynet.com/borsa/canliborsa/#A')

page<- read_html(jumped)
page<-html_nodes(page,xpath='//*[@id="canliLeftColumn"]/div[3]/table')
page<-  html_table(page)
head(page[[1]])

And the result:

[1] Hisse         Hisse         Hisse         Son           Alış         
[6] Satış         %Fark         En Düşük      En Yüksek     AOF          
[11] Hacim (Lot)   Hacim (TL)    Son İşlem     Ekle / Kaldır
<0 rows> (or 0-length row.names)

As you can see, I can reach the table with xpath, I get the column names but without any data inside. Table is completely empty. Is there anyone who can help ? Thanks in advance.

Upvotes: 1

Views: 822

Answers (1)

Barsssk
Barsssk

Reputation: 93

Problem solved. Thanks for help @Dave2e . I found out that jump_to function can scrape the data and contains it in $response. Code below gives the data as txt only spliting it remained.

url<- "http://uyeler.mynet.com/login/login.asp?rurl=http%3A%2F%2Ffinans.mynet.com%2Fborsa%2Fcanliborsa%2F&formname=finans#A"
session<-html_session(url)
form<- html_form(session)[[1]]

login<- set_values(form, "username" ="muharrem_babaogul_1991","password"="q1w2e3")
submit_form(session,login)


jumped<-jump_to(session,url = 'http://finans.mynet.com/borsa/canliborsadata/data.fdata')

file<-content(jumped$response,as="text")

Upvotes: 2

Related Questions