CODEWITHSUNDEEP
ftpwget
LonelySoul
LonelySoul

Reputation: 1232

Download data from FTP Website

There is something which I am missing or might be the whole case. So I am trying to download NCDC data from NCDC Datasets and unable to do it the unix box.

The command which I have used this far are 

wget ftp://ftp.ncdc.noaa.gov:21/pub/data/noaa/1901/029070-99999-1901.gz">029070-99999-1901.gz

This is for one file, but will be very happy if I can downlaod the entire parent directory.

Upvotes: 2

Views: 890

Answers (2)

Cristian
Cristian

Reputation: 578

for i in {1990..1993} 
do
    echo "$i"
    cd /home/chile/data
    # -nH Disable generation of host-prefixed directories
    # -nd all files will get saved to the current directory
    # -np Do not ever ascend to the parent directory when retrieving recursively. 
    # -R index.html*,227070*.gz* don't download files with this regex

wget -r -nH -nd -np -R *.html,999999-99999-$i.gz* http://www1.ncdc.noaa.gov/pub/data/noaa/$i/
    /data/noaa/$i/

done

Upvotes: 0

legrandviking
legrandviking

Reputation: 2424

You seem to have a lonely " just before the >

to download everything you can try this command to get the whole directory content

wget -r ftp://ftp.ncdc.noaa.gov:21/pub/data/noaa/1901/*

Upvotes: 2

Related Questions