CODEWITHSUNDEEP
shellftpsftp
oskarlin
oskarlin

Reputation: 113

Good workflow for uploading thousands of files

Every day ~30000 files are created on one server and then they should all be moved to a web server. Until now I've been using ncftpput for transferring all the files but it's very slow and stops now and then.

My plan now is to gzip all the files and then use scp to transfer the archive to my web server and then unpack it there.

Is this a good solution or is there a better way?

Upvotes: 1

Views: 79

Answers (2)

Mark Setchell
Mark Setchell

Reputation: 207465

I would use "rsync" if available on your platform. It is restartable, only does the minimum necessary and very fast and easy to use.

Upvotes: 4

LosAngeles
LosAngeles

Reputation: 83

I think it is better than upload raw files. But I found this: Slow Upload Speeds via SCP - https://discussion.dreamhost.com/thread-134902.html

Upvotes: 0

Related Questions