Reputation: 4082
Problem removing whitespace from a string in r
I know this seems like a duplicate, but I have tried several solutions to this problem including this one, maybe I have a problem with my scraping that produces this problem
Example
This is what I am trying to do:
I scraped this dataset from Wikipedia:
library(tidyverse)
library(rvest)
Sueldos <- read_html("https://es.wikipedia.org/wiki/Anexo:Salario_m%C3%ADnimo_en_Chile") %>% html_nodes(".wikitable")
Sueldos <-rvest::html_table(Sueldos[[1]])
colnames(Sueldos) <- make.names(colnames(Sueldos))
Sueldos <- Sueldos %>% rename(Sueldo = Monto.bruto.enpesos.chilenos) %>% dplyr::select(Desde, Sueldo)
The dataset looks like this:
|Desde |Sueldo |
|:-----------------------|:------|
|1 de septiembre de 1987 |11 335 |
|1 de febrero de 1989 |15 488 |
|1 de junio de 1989 |18 000 |
|1 de junio de 1990 |26 000 |
|1 de junio de 1991 |33 000 |
|1 de junio de 1992 |38 600 |
|1 de junio de 1993 |46 000 |
|1 de junio de 1994 |52 150 |
|1 de junio de 1995 |58 900 |
|1 de junio de 1996 |65 500 |
I want to trim whitespace in Sueldos to turn them to numeric, usually, I have no problem trimming this whitespace, but several trials have given me the same result:
Test 1
I first thought I would try with str_remove_all from stringr
Sueldos <- Sueldos %>% str_remove_all(Sueldo, pattern = " ")
But it didn't change
Test 2
Then I thought of using str_replace_all
Sueldos <- Sueldos %>% mutate(Sueldo = str_replace_all(Sueldo, pattern = " ", replacement = ""))
And nothing
Test 3
I thought maybe as in the link above if I used fixed it would work
Sueldos <- Sueldos %>% mutate(Sueldo = str_replace_all(Sueldo, pattern = fixed(" "), replacement = ""))
Nothing
Test 4
I finally tried gsub, just in case
Sueldos$Sueldo <- gsub(x = Sueldos$Sueldo, pattern = " ", replacement = "")
Still nothing, I dont get what is going on
SessionInfo:
R version 3.6.1 (2019-07-05)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=es_CL.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=es_CL.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=es_CL.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=es_CL.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] forcats_0.4.0 stringr_1.4.0 dplyr_0.8.3 purrr_0.3.3 readr_1.3.1 tidyr_1.0.0 tibble_2.1.3
[8] ggplot2_3.2.1 tidyverse_1.2.1 rvest_0.3.4 xml2_1.2.2
loaded via a namespace (and not attached):
[1] tidyselect_0.2.5 xfun_0.10 haven_2.1.1 lattice_0.20-38 colorspace_1.4-1 vctrs_0.2.0 generics_0.0.2
[8] htmltools_0.4.0 yaml_2.2.0 rlang_0.4.1 pillar_1.4.2 glue_1.3.1 withr_2.1.2 selectr_0.4-1
[15] modelr_0.1.5 readxl_1.3.1 lifecycle_0.1.0 munsell_0.5.0 gtable_0.3.0 cellranger_1.1.0 htmlwidgets_1.5.1
[22] evaluate_0.14 knitr_1.25 curl_4.2 highr_0.8 htmlTable_1.13.2 broom_0.5.2 Rcpp_1.0.2
[29] scales_1.0.0 backports_1.1.5 checkmate_1.9.4 jsonlite_1.6 hms_0.5.1 digest_0.6.22 stringi_1.4.3
[36] grid_3.6.1 cli_1.1.0 tools_3.6.1 magrittr_1.5 lazyeval_0.2.2 crayon_1.3.4 pkgconfig_2.0.3
[43] zeallot_0.1.0 lubridate_1.7.4 assertthat_0.2.1 rmarkdown_1.16 httr_1.4.1 rstudioapi_0.10 R6_2.4.0
[50] nlme_3.1-141 compiler_3.6.1
Upvotes: 1
Views: 1103
Answers (4)
Reputation: 5788
Base R (regex) solution:
Sueldos$Sueldo <- gsub("\\s+", "", Sueldos$Sueldo)
Upvotes: 0
Reputation: 4082
The Answer of @StupidWolf worked for me, I wanted to add what my final code was just to keep the tidyverse format for whoever likes this better:
library(rvest)
library(tidyverse)
Sueldos <- read_html("https://es.wikipedia.org/wiki/Anexo:Salario_m%C3%ADnimo_en_Chile") %>% html_nodes(".wikitable")
Sueldos <-rvest::html_table(Sueldos[[1]])
colnames(Sueldos) <- make.names(colnames(Sueldos))
Sueldos <- Sueldos %>% rename(Sueldo = Monto.bruto.enpesos.chilenos) %>% dplyr::select(Desde, Sueldo) %>% mutate(Desde = lubridate::dmy(Desde), Sueldo = gsub(x = Sueldo, pattern = intToUtf8(160),replacement = ""))
This works great
if you prefer stringr:
replace the last line of code with:
Sueldos <- Sueldos %>% rename(Sueldo = Monto.bruto.enpesos.chilenos) %>% dplyr::select(Desde, Sueldo) %>% mutate(Desde = lubridate::dmy(Desde), Sueldo = str_remove_all(Sueldo, pattern = intToUtf8(160)))
Upvotes: 1
Reputation: 46898
Normally a whitespace has ascii 32, but when I look under that is the character in the column Sueldo, the character between the numbers has ascii 160. One solution i can offer is to replace the column using intToUtf8(160). As for what exactly is ascii score 160 in this data, I am not sure.
It might be called non breaking space, as @WiktorStribiżew pointed out below, I am not sure about the distinction between this and white space.
sapply(unlist(strsplit(Sueldos$Sueldo[2],"")),utf8ToInt)
1 1 3 3 5
49 49 160 51 51 53
utf8ToInt(" ")
[1] 32
Now if we gsub the correct character:
gsub(intToUtf8(160),"",Sueldos$Sueldo)
[1] "11335" "15488" "18000" "26000" "33000" "38600"
[7] "46000" "52150" "58900" "65500" "71400" "80500"
[13] "90500" "100000" "105500" "111200" "115648" "120000"
[19] "127500" "135000" "144000" "159000" "165000" "172000"
[25] "182000" "193000" "210000" "225000" "241000" "250000"
[31] "257500" "264000" "270000" "276000" "288000[1]" "301000[2]"
Upvotes: 1
Reputation: 1763
It works when you use str_replace_all on a variable (and not the tibble) :
suppressPackageStartupMessages(library(dplyr))
suppressPackageStartupMessages(library(rvest))
Sueldos <- read_html("https://es.wikipedia.org/wiki/Anexo:Salario_m%C3%ADnimo_en_Chile") %>% html_nodes(".wikitable")
Sueldos <-rvest::html_table(Sueldos[[1]])
colnames(Sueldos) <- make.names(colnames(Sueldos))
Sueldos <- Sueldos %>% rename(Sueldo = Monto.bruto.enpesos.chilenos) %>% dplyr::select(Desde, Sueldo)
# test <- Sueldos$Desde[1]
# test %>%
# stringr::str_remove_all(string = ., pattern = " ")
Sueldos %>%
mutate(Desde = Desde %>% stringr::str_remove_all(string = ., pattern = " "))
#> Desde Sueldo
#> 1 1deseptiembrede1987 11 335
#> 2 1defebrerode1989 15 488
#> 3 1dejuniode1989 18 000
#> 4 1dejuniode1990 26 000
#> 5 1dejuniode1991 33 000
#> 6 1dejuniode1992 38 600
#> 7 1dejuniode1993 46 000
#> 8 1dejuniode1994 52 150
#> 9 1dejuniode1995 58 900
#> 10 1dejuniode1996 65 500
#> 11 1dejuniode1997 71 400
#> 12 1dejuniode1998 80 500
#> 13 1dejuniode1999 90 500
#> 14 1dejuniode2000 100 000
#> 15 1dejuniode2001 105 500
#> 16 1dejuniode2002 111 200
#> 17 1dejuliode2003 115 648
#> 18 1dejuliode2004 120 000
#> 19 1dejuliode2005 127 500
#> 20 1dejuliode2006 135 000
#> 21 1dejuliode2007 144 000
#> 22 1dejuliode2008 159 000
#> 23 1dejuliode2009 165 000
#> 24 1dejuliode2010 172 000
#> 25 1dejuliode2011 182 000
#> 26 1dejuliode2012 193 000
#> 27 1deagostode2013 210 000
#> 28 1dejuliode2014 225 000
#> 29 1dejuliode2015 241 000
#> 30 1deenerode2016 250 000
#> 31 1dejuliode2016 257 500
#> 32 1deenerode2017 264 000
#> 33 1dejuliode2017 270 000
#> 34 1deenerode2018 276 000
#> 35 1deseptiembrede2018 288 000[1]<U+200B>
#> 36 1demarzode2019 301 000[2]<U+200B>
Upvotes: 0
Related Questions
- Removing Whitespace From a Whole Data Frame in R
- Using gsub to extract character string before white space in R
- Use gsub remove all string before first white space in R
- Using gsub I want to remove white spaces
- Cannot remove leading/trailing white space with gsub or trimws
- R: gsub keep the space between characters in a string without removing
- replace "." with space using gsub() in R?
- Remove extra white space from between letters in R using gsub()
- Remove trailing and leading spaces and extra internal whitespace with one gsub call
- How to replace the white spaces and character with specified character in R?