CODEWITHSUNDEEP
rweb-scrapingrcurlrvesthttr
thatsawinner
thatsawinner

Reputation: 151

getting next results using rvest submit_form

I have the following from a website which I am trying to scrape

    <td align="right">
                <span id="ctl00_ContentPlaceHolder1_lblCount1">825 records found, </span>
                Page
                <input name="ctl00$ContentPlaceHolder1$txtCurrent1" type="text" value="1" maxlength="4" id="ctl00_ContentPlaceHolder1_txtCurrent1" style="width:30px;" />
                of
                <span id="ctl00_ContentPlaceHolder1_lblTotalPage1">83</span>
                <input type="submit" name="ctl00$ContentPlaceHolder1$btnGo1" value="GO" id="ctl00_ContentPlaceHolder1_btnGo1" class="inputbtn" />
            </td>

I have tried the following code using the rvest package

pgsession <- html_session(url)
pgform <- html_form(pgsession)[[1]]
filled_form <- set_values(pgform,`ctl00$ContentPlaceHolder1$txtCurrent1` = 2)
result <- submit_form(pgsession,filled_form)

I am not getting the next table in the website returned to me. How I do use this package to submit a value and get back the resulting HTML? I've done some poking around and maybe I should be using the R packages httr and rcurl to do this.

Upvotes: 1

Views: 1306

Answers (1)

thatsawinner
thatsawinner

Reputation: 151

I figured it out. The correct code is:

pgsession <- html_session("url")
pgform <- html_form(read_html(pgsession))[[1]]
filled_form <- set_values(pgform, `ctl00$ContentPlaceHolder1$txtCurrent1` =2)
result <- submit_form(pgsession,filled_form, submit='ctl00$ContentPlaceHolder1$btnGo1')
case_home <- read_html(result)

Upvotes: 3

Related Questions