Why can't I get st_read to open a shapefile from a compressed file? (Map is read if saved on local Onedrive folder but not unzipped in R)

Question

I'm creating a report that includes a vector map of the state of Oklahoma in the US. I found shapefiles that I need, but these are in zipped files.

I found a similar question on Stackoverflow (link here) about downloading and extracting zipped data, and used the approach suggested there. However, I am getting an error about being unable to find the file in the zipped file--even though it is there. The original question is linked below:

https://stackoverflow.com/questions/3053833/using-r-to-download-zipped-data-file-extract-and-import-data#:~:text=8%20Answers.%201%20Create%20a%20temp.%20file%20name,simpler%29%20looks%20like%20temp%20%3C-%20tempfile%28%29%20download.%20

This is the R code that I used, which includes linked to the zipped vector data:

library(tidyverse)
library(sf)
library(tmap)

ok_zip_code_tempfile <- tempfile()
download.file("https://okmaps.org/OGI/Downloads/okzip/OGR-SHP-ZIP/okzip__state_EPSG_32124.zip", ok_zip_code_tempfile)
ok_zip_map <- st_read(unzip(ok_zip_code_tempfile, files="okzip.shp"))
unlink(ok_zip_code_tempfile)

ok_municipalities_tempfile <- tempfile()
download.file("https://www.odot.org/hqdiv/p-r-div/maps/shp-files/munibnd.zip", ok_municipalities_tempfile)
ok_municipalities <- st_read(unzip(ok_municipalities_tempfile, files='munibnd.shp'))
unlink(ok_municipalities_tempfile)

When running these, I get the following errors:

> Error: Cannot open
> "C:\Users\[myname]\OneDrive\Desktop\Documents\okzip.shp"; The source
> could be corrupt or not supported. See `st_drivers()` for a list of
> supported formats. In addition: Warning message: In CPL_read_ogr(dsn,
> layer, query, as.character(options), quiet,  :   GDAL Error 4: Unable
> to open C:\Users\[myname]\OneDrive\Desktop\Documents\okzip.shx or
> C:\Users\[myname]\OneDrive\Desktop\Documents\okzip.SHX. Set
> SHAPE_RESTORE_SHX config option to YES to restore or create it.
> 
> 
> Error: Cannot open
> "C:\Users\[myname]\OneDrive\Desktop\Documents\munibnd.shp"; The source
> could be corrupt or not supported. See `st_drivers()` for a list of
> supported formats. In addition: Warning message: In CPL_read_ogr(dsn,
> layer, query, as.character(options), quiet,  :   GDAL Error 4: Unable
> to open C:\Users\[myname]\OneDrive\Desktop\Documents\munibnd.shx or
> C:\Users\[myname]\OneDrive\Desktop\Documents\munibnd.SHX. Set
> SHAPE_RESTORE_SHX config option to YES to restore or create it.

Further confusing the mix, if I save a copy on my Desktop on OneDrive, then the files load correctly:

ok_zip_map <- st_read("C:\\Users\\[myname]\\OneDrive\\Desktop\\OK_Data\\okzip.shp")

> Reading layer `okzip' from data source
> `C:\Users\[myname]\OneDrive\Desktop\OK_Data\okzip.shp' using driver
> `ESRI Shapefile' Simple feature collection with 682 features and 4
> fields geometry type:  MULTIPOLYGON dimension:      XY bbox:          
> xmin: -103.0025 ymin: 33.61579 xmax: -94.43066 ymax: 37.00231
> geographic CRS: WGS 84
 
 
ok_municipalities <- st_read("C:\\Users\\[myname]\\OneDrive\\Desktop\\OK_Data\\munibnd.shp")

> Reading layer `munibnd' from data source
> `C:\Users\[myname]\OneDrive\Desktop\OK_Data\munibnd.shp' using driver
> `ESRI Shapefile' Simple feature collection with 1414 features and 8
> fields geometry type:  MULTIPOLYGON dimension:      XY bbox:          
> xmin: -102.5219 ymin: 33.76577 xmax: -94.43182 ymax: 36.99937 CRS:    
> NA

However, if I save the same files on my local computer (and not in a Onedrive folder), then I get an error.

ok_zip_map <- st_read("C:\\Users\\[myname]\\Documents\\okzip.shp")

> Error: Cannot open "C:\Users\[myname]\Documents\okzip.shp"; The source
> could be corrupt or not supported. See `st_drivers()` for a list of
> supported formats. In addition: Warning message: In CPL_read_ogr(dsn,
> layer, query, as.character(options), quiet,  :   GDAL Error 4: Unable
> to open C:\Users\[myname]\Documents\okzip.shx or
> C:\Users\[myname]\Documents\okzip.SHX. Set SHAPE_RESTORE_SHX config
> option to YES to restore or create it.
 
 
ok_municipalities <- st_read("C:\\Users\\[myname]\\Documents\\munibnd.shp") 

> Error: Cannot
> open "C:\Users\[myname]\Documents\munibnd.shp"; The source could be
> corrupt or not supported. See `st_drivers()` for a list of supported
> formats. In addition: Warning message: In CPL_read_ogr(dsn, layer,
> query, as.character(options), quiet,  :   GDAL Error 4: Unable to open
> C:\Users\[myname]\Documents\munibnd.shx or
> C:\Users\[myname]\Documents\munibnd.SHX. Set SHAPE_RESTORE_SHX config
> option to YES to restore or create it

I've also tried running the original code (downloading the link into a temporary file and unzipping it in R) on another computer (a Mac) and it failed with a similar error.

I would ideally like to produce a report that doesn't need dependency files (i.e., local copies of the maps), as it will be run by others on their own computers. Can someone help me understand what may be going on?

Thanks in advance for any guidance you might have!

Tony

Answer 1

As mentioned in this answer, the key here is to unzip the downloaded file, and read the unzipped folder using sf::read_sf or sf::st_read. Here's a wrapper function that I used before:

library(sf)

read_shape_URL <- function(URL){
  cur_tempfile <- tempfile()
  download.file(url = URL, destfile = cur_tempfile)
  out_directory <- tempfile()
  unzip(cur_tempfile, exdir = out_directory)
  
  st_read(dsn = out_directory) #read_sf also works here
}

Using this, we can load the zipped shapefile:

ok_zip_map <- read_shape_URL("https://okmaps.org/OGI/Downloads/okzip/OGR-SHP-ZIP/okzip__state_EPSG_32124.zip")