Reputation: 682

Get list of files from FTP server

I'm trying to get a list of all the files we have on a server ( specifically every pdf file we have there ). I've tried using total commander and search for the files. It worked to some extent, as in, i got a list of every pdf we had there, but no way of exporting the results ( we have 100.000+ files there )

I've tried using a bash script to get the information, but i'm not very experienced with linux, and i don't really know what i'm doing.

My script looks like this :

#!/bin/bash
hostname="<host>"
ftp -i -nv $hostname << EOF
user <username> <password>
ls -R 
EOF

Running the above script i get

?Invalid command
501 Illegal PORT command
ftp: bind: Address already in use
221 Goodbye

Any help or pointing me on what to search would be greatly appreciated.

Upvotes: 23

Answers (8)

James

Reputation: 80

try .netrc file :

#!/bin/sh
 
HOST=11.22.33.44 
USER=myuser 
PWD="mypwd"
 
cat >> ~/.netrc <<END_SCRIPT 
machine $HOST login $USER password $PWD
macdef init 
ls 
cd upload 
lcd patch 
bin 
hash 
prompt 
mput *.patch 
put update.log 
ls 
quit
 
END_SCRIPT
chmod 600 ~/.netrc 
exit 0

run it once, and

$ chmod 600 ~/.netrc ; ls -la ~/.netrc

make sure the file perm : -rw------- ... ~/.netrc

$ ftp $HOST >> ~/ftp.log

if the .netrc file is already exist, you have only to keyin ftp $HOST, and it will finish all commands in .netrc file.

and, if you have to leave for a while, you can use nohup ftp $HOST , and logout.

The process log will be in the file: ’nohup.out’ ( or '$HOME/nohup.out' )

Store it in your own file: ' nohup ftp $HOST >> ~/MyLogFile.log'

or No Output file needed: ' nohup ftp $HOST > /dev/null '

more information about .netrc file, reference :

https://www.unix.com/man-page/ultrix/5/netrc/

-or-

$ man .netrc

in your own *NIX OS

.....hmmm... not support by Microsoft OS...

Upvotes: 1

alperenkilic

Reputation: 47

curl --list-only ftp://user:[email protected]/directory/JPG/

with curl request lists files

Upvotes: 1

Whome

Reputation: 10400

I have always run this FTP script in a linux system, after login ls command is submitted to a ftp connection(passive). List of files is written to a stdout stream. Script is inline in a sh script file.

#!/bin/sh

HOST=11.22.33.44
USER=myuser
PWD="mypwd"

ftp -p -n $HOST <<END_SCRIPT
quote USER $USER
quote PASS $PWD
ls
quit
END_SCRIPT
exit 0

Upvotes: 3

ioCron

Reputation: 963

As mentionend in one of my comments, do not use user:password as plain text in your commands ever, otherwise you are running at risk of beeing history hjacked! Instead use at least a protected/restricted file with the username+password and substitute it in your command, e.g.:

ftp://yourftpserver/dir/ --user "$(cat .userpw)"

Whereas .userpw is your protected/restricted file with the example content: myusername:mypassword

Upvotes: 4

Talespin_Kit

Reputation: 21877

curl ftp://user:password@<ip>/path/

The last / is a must if it is a directory. This worked in curl version 7.29.0

Upvotes: 9

Patrick B.

Reputation: 12333

Try to configure ftp to use the PASV (passive) mode for data transfers. This done with the -p switch.

I'm not sure if you will be able to do a recursive file listing with this ftp-client. ls -R in my case just gave the list of files and directories in the current working dir. Maybe Recursive FTP directory listing in shell/bash with a single session (using cURL or ftp) will help you.

Upvotes: 6