CODEWITHSUNDEEP
rcsvdataframetext-miningcran
Nicholas Chen
Nicholas Chen

Reputation: 3

How to do Text Mining from a HTML document, and convert it into a CSV file?

So I'm trying to do a bit of text mining from this website "https://www.bmkg.go.id/gempabumi/gempabumi-terkini.bmkg" - particularly from lines 452 until 1050 through the Developer's Sources. I haven't been able to do that successfully; and my goal is, after I succeed in doing so, I'll have to convert it into a dataframe with custom labels, then save it as a CSV file into my local drive.

Is my logic on achieving this goal correct, or am I getting it wrong to even begin with?

Here's what I have so far:

    library(httr)
    library(dplyr)

    bmkg_current <- GET("https://www.bmkg.go.id/gempabumi/gempabumi-terkini.bmkg")

    stringi::stri_enc_detect(content(bmkg_current, "raw"))      //just to check encoding type
    bmkg_text <- content(bmkg_current, type ="text", encoding = "ISO-8859-1")
    bmkg_df <- tibble(line = 452:1050, text = bmkg_text)
    bmkg_df          //tried to output, but not want I wanted

Output:

 # A tibble: 599 x 2
   line text                                                      
   <int> <chr>                                                     
   1   452 "<!DOCTYPE html>\r\n<!--[if IE 8]> <html lang=\"en\" clas~
   2   453 "<!DOCTYPE html>\r\n<!--[if IE 8]> <html lang=\"en\" clas~
   3   454 "<!DOCTYPE html>\r\n<!--[if IE 8]> <html lang=\"en\" clas~
   4   455 "<!DOCTYPE html>\r\n<!--[if IE 8]> <html lang=\"en\" clas~
   5   456 "<!DOCTYPE html>\r\n<!--[if IE 8]> <html lang=\"en\" clas~

These are what lines 452 - 1050 look like in the HTML, from Developer Source:

                            <tr>
                                <td>2</td>
                                <td>29-Mar-20 <br>06:10:35 WIB</td>
                                <td>-7.39</td>
                                <td>124.19</td>
                                <td>5.2</td>
                                <td>631 Km</td>
                                <td>108 km BaratLaut ALOR-NTT</td>
                            </tr>

Any help on this would be much appreciated! Thank you :)

Upvotes: 0

Views: 368

Answers (1)

Ronak Shah
Ronak Shah

Reputation: 389205

If you need the information from the table on the website using rvest you can do : 

library(rvest)
url <- 'https://www.bmkg.go.id/gempabumi/gempabumi-terkini.bmkg'
out_df <- url %>% read_html() %>% html_table() %>% .[[1]]

head(out_df)
#  #            Waktu Gempa Lintang  Bujur Magnitudo Kedalaman                                   Wilayah
#1 1 02-Apr-20 09:13:13 WIB   -7.93 125.62       5.5     10 Km                 125 km TimurLaut ALOR-NTT
#2 2 29-Mar-20 06:10:35 WIB   -7.39 124.19       5.2    631 Km                 108 km BaratLaut ALOR-NTT
#3 3 28-Mar-20 22:43:17 WIB   -1.72 120.14       5.8     10 Km               46 km Tenggara SIGI-SULTENG
#4 4 27-Mar-20 21:32:48 WIB    0.28 133.53       5.5     10 Km       139 km BaratLaut MANOKWARI-PAPUABRT
#5 5 27-Mar-20 04:36:40 WIB   -2.72 139.26       5.9     11 Km        72 km BaratLaut KAB-JAYAPURA-PAPUA
#6 6 26-Mar-20 22:38:03 WIB    5.58 125.16       6.3     10 Km 221 km BaratLaut TAHUNA-KEP.SANGIHE-SULUT

You could use write.csv to write this data into csv

write.csv(out_df, 'earthquake_data.csc', row.names = FALSE)

Upvotes: 2

Related Questions