Bash: Script output to terminal session stops but script finishes normal

Question

I'm opening an ssh-session to a remote server and execute a larger (around 1000 lines) bash-script on the remote machine. It involves several very CPU-intensive calls which run for up to three minutes each. To track the scripts progress it echoes messages placed at several points in the script.

In general the script runs smoothly. From time to time the script runs trough (the resulting file on the remote machine is correct) but the output to the terminal stops. Ctrl-C doesn't help, no prompt, just a frozen session. top in a separate session shows normal execution of the script.

My question: How keep the session alive?

local machine:

$ sw_vers  
ProductName:    Mac OS X  
ProductVersion: 10.9  
BuildVersion:   13A603

remote machine:

$ lsb_release -d  
Description:    Ubuntu 12.04.3 LTS

Answer 1

Make it log to a file instead (perhaps via syslog), and tail that file from wherever is convenient for you. This also helps detach the script so you can run it headless, from a cron job, etc. Also, if the log file has read access for others, they too can monitor it.

Answer 2

Start a screen on the remote machine and run your command from it:

screen -S largeScript

And then

./yourLargeScript.sh

Whenever your ssh session gets frozen, you can kill it with ~.

If you ssh again, you can grab back your screen by:

screen -dr largeScript

Answer 3

Personally, I would recommend using screen or tmux on the remote terminal for exactly this reason.

Those apps will allow the remote process to continue even if your local SSH session times out.

• http://www.bangmoney.org/presentations/screen.html
• http://tmux.sourceforge.net/