CODEWITHSUNDEEP
linuxshellsshscpinotify
Chan-ho Lee
Chan-ho Lee

Reputation: 51

Linux file transfer between server automatically when a file is created

In my work, I use 2 Linux servers. The first one is used for web-crawling and create it as a text file. The other one is used for analyzing the text file from the web crawler.

So the issue is that when a text file is created on web-crawling server, it needs to be transferred automatically on the analysis server.

I've used shell programming guides referring some tips, and set up the crawling server to be able to execute the scp command without requiring the password (By using ssh-keygen command, Add ssh-key on authorized_keys file located in /root/.ssh directory) But I cannot figure out how to programmatically transfer the file when it is created.

My job position is just data analyze (Not programming) So, the lack of background programming knowledge is my big concern

If there is a way to trigger the scp to copy the file when it is created, please let me know.

Upvotes: 1

Views: 2158

Answers (1)

wottle
wottle

Reputation: 13619

You could use inotifywait to monitor the directory and run a command every time a file is created in the directory. In this case, you would fire off the scp command. IF you have it set up to not prompt for the password, you should be all set.

inotifywait -mrq -e CREATE --format %w%f /path/to/dir | while read FILE; do scp "$FILE"analysis_server:/path/on/anaylsis/server/; done

You can find out more about inotifywait at http://techarena51.com/index.php/inotify-tools-example/

Upvotes: 2

Related Questions