CODEWITHSUNDEEP
recursionftpwgetlftp
Gabriel Fair
Gabriel Fair

Reputation: 4272

How can I wget to traverse all symbolic links it comes across?

Issue

wget is not traversing symbolic links to directories to download the linked folders recursively.

Backgroud

As per the man page of wget:

By default, when retrieving FTP directories recursively and a symbolic link is encountered, the symbolic link is traversed and the pointed-to files are retrieved. Currently, Wget does not traverse symbolic links to directories to download them recursively, though this feature may be added in the future.

Problem

I am trying to backup/mirror all of ftp://l5ftl01.larc.nasa.gov/ and there are thousands of symbolic/hardlinks to other folders and servers.

I tried using wget -N -m ftp://l5ftl01.larc.nasa.gov/* but that does not follow the links, it just downloads the shortcut as a file.

Question

What should I do instead?

Is there a way for wget to recursively traverse all symbolic links to get all the files?

Upvotes: 0

Views: 1542

Answers (1)

Earthdata Support
Earthdata Support

Reputation: 26

This method of acquiring data is possibly outdated. As of January 1, all DAACs including LARC and LARC_ASDC are moving to https. This should solve your recursion problem.

Please contact [email protected] they will help you get the data you require.

Upvotes: 1

Related Questions