CODEWITHSUNDEEP
rweb-scrapingrvest
Martin
Martin

Reputation: 39

R: Using Rvest to loop through list

I try to scape the prices, area and addresses from all flats of this homepage (https://www.immobilienscout24.de/Suche/S-T/P-1/Wohnung-Miete/Sachsen/Dresden)

Getting the data for one list element with Rvest and xpath works fine (see code), but I don´t know how to get the ID of each list element to loop through all elements.

Here is a part of the html-code with the data-go-to-expose-id I need for the loop. How can I get all IDs?

<a href="/expose/103049161" data-go-to-expose-id="103049161" data-go-to-expose-referrer="RESULT_LIST_LISTING" class="slick-slide" data-slick-index="1" aria-hidden="true" style="width: 268px;"><span class="slick-bg-layer"></span><img alt="Immobilienbild" class="gallery__image block height-full" src="https://pictures.immobilienscout24.de/listings/541dfd45-c75a-4da7-a831-3339264d578b-1193970198.jpg/ORIG/legacy_thumbnail/532x399/format/jpg/quality/80"></a>a831-3339264d578b-1193970198.jpg/ORIG/legacy_thumbnail/532x399/format/jpg/quality/80"></a>

And here is my current R-code to fetch the data from one list element: 

library(rvest)

url <- "https://www.immobilienscout24.de/Suche/S-T/P-1/Wohnung-Miete/Sachsen/Dresden"

address <- url %>% read_html(encoding = "UTF-8") %>% html_node(xpath = '//*[@id="result-103049161"]/div[2]/div[2]/div[1]/div[2]/div[2]/a') %>% html_text()
price <- url %>% read_html(encoding = "UTF-8") %>% html_node(xpath = '//*[@id="result-103049161"]/div[2]/div[2]/div[1]/div[3]/div/div[1]/dl[1]/dd') %>% html_text()
area <- url %>% read_html(encoding = "UTF-8") %>% html_node(xpath = '//*[@id="result-103049161"]/div[2]/div[2]/div[1]/div[3]/div/div[1]/dl[2]/dd') %>% html_text()

Upvotes: 0

Views: 679

Answers (1)

davidski
davidski

Reputation: 581

Does this get what you are after

library("tidyverse")
library("httr")
library("rvest")

url <- "https://www.immobilienscout24.de/Suche/S-T/P-1/Wohnung-Miete/Sachsen/Dresden"

x <- read_html(url)

x %>% 
  html_nodes("#listings") %>% 
  html_nodes(".result-list__listing") %>% 
  html_attr("data-id")

Upvotes: 1

Related Questions