Unix/Bash Compare Files from Remote Servers

Question

I've updated the code below and created a separate variable for the time comparison. However, it's still not comparing the second file even after setting the loop after the variables.

I'm trying to compare files from multiple servers remotely on when they were last updated

ssh -q -l user svr "ls -ltr /path/to/file/svr.log | tail -n1"

It returns with this output, which is what I want.

-rwxr-xr-x 1 user user 1658367654 Jul 21 01:40 /path/to/file/svr.log

So I do the same for the rest of the servers. I need to stat them to get the last modified date and check against the threshold of when it's supposed to be last updated

   #!/bin/bash
set -x
s11=$(ssh -q -l user svr1 "find /path/to/log/svr.log -printf '"%T+ %p"' | sort")
s12=$(ssh -q -l user svr2 "find /path/to/log/svr.log -printf '"%T+ %p"' | sort")
s11r=$(echo $s11 | awk '{print $2}')
s12r=$(echo $s12  | awk '{print $2}')

for myservers in $s11r $s12r;
do
OLDTIME=1200 #20 minutes file threshold
FILETIME=$(stat -c %Y "${myservers}")
CURTIME=$(perl -e 'print time')
TIMEDIFF=$(($CURTIME - $FILETIME))


if [[ $TIMEDIFF -gt $OLDTIME ]]; then
echo -e "$myservers: old time"
else
echo "$myservers: all good"
fi
done

OUTPUT below

    ++ ssh -q -l user svr1 'find /path/to/log/svr.log -printf '\''%T+' '%p'\'' | sort'
+ s11='2022-07-21+03:09:19.3550802140 /path/to/log/svr.log'
++ ssh -q -l user svr2 'find /path/to/log/svr.log -printf '\''%T+' '%p'\'' | sort'
+ s12='2022-07-21+03:25:53.3758441030 /path/to/log/svr.log'
++ echo 2022-07-21+03:09:19.3550802140 /path/to/log/svr.log
++ awk '{print $2}'
+ s11r=/path/to/log/svr.log
++ echo 2022-07-21+03:25:53.3758441030 /path/to/log/svr.log
++ awk '{print $2}'
+ s12r=/path/to/log/svr.log
+ for myservers in '$s11r' '$s12r'
+ OLDTIME=1200
++ stat -c %Y /path/to/log/svr.log
+ FILETIME=1658381370
++ perl -e 'print time'
+ CURTIME=1658381855
+ TIMEDIFF=485
+ [[ 485 -gt 1200 ]]
+ echo '/path/to/log/svr.log: all good'
/path/to/log/svr.log: all good
+ for myservers in '$s11r' '$s12r'
+ OLDTIME=1200
++ stat -c %Y /path/to/log/svr.log
+ FILETIME=1658381370
++ perl -e 'print time'
+ CURTIME=1658381855
+ TIMEDIFF=485
+ [[ 485 -gt 1200 ]]
+ echo '/path/to/log/svr.log: all good'
/path/to/log/svr.log: all good

As per output, it does the job but it only processes the log file for server 1 and ignores server 2. Even though all the servers have updated file, they do not have the same size.

Need help with

1. How to make it process all the servers
2. How to skip the server check comparison if 1 file is already updated

Thank you!

Answer 1

Here is a solution for this issue:

I'm trying to compare files from multiple servers remotely on when they were last updated

how

• check the stat of that file and grep the Modify: time
• convert the time to seconds
• compare the seconds (= numbers)

I am going to test the /var/log/auth.log file which by SSHing is getting updated.

script

# host names
HOST_1=docker
HOST_2=irj

# SHH to host and stat a file
declare -ir HOST_1_TIME=$(ssh $HOST_1 "stat -c %Y /var/log/auth.log");
declare -ir HOST_2_TIME=$(ssh $HOST_2 "stat -c %Y /var/log/auth.log");

# print
echo HOST_1_TIME: $HOST_1_TIME
echo HOST_2_TIME: $HOST_2_TIME
echo diff: $(( $HOST_1_TIME - $HOST_2_TIME ));

# check which was access first
if (( $HOST_1_TIME < $HOST_2_TIME )); then
    echo "HOST_1: $HOST_1 logged in first"
else
    echo "HOST_2: $HOST_2 logged in first"
fi

sample output for me

HOST_1_TIME: 1658414665
HOST_2_TIME: 1658414666
diff: -1
HOST_1: docker logged in first

if we swap the hosts

declare -ir HOST_2_TIME=$(ssh $HOST_2 "stat -c %Y /var/log/auth.log");
declare -ir HOST_1_TIME=$(ssh $HOST_1 "stat -c %Y /var/log/auth.log");

sample output for me

HOST_1_TIME: 1658414653
HOST_2_TIME: 1658414651
diff: 2
HOST_2: irj logged in first

---

NOTE

• find is not for query over a file: find /path/to/log/svr.log
• if your output is more than a single line, save it in an array -- you can use mapfile -t array_name < <(your-cmd)
• probably you would like to use find -printf this way: -printf '%T+ %p\n' -- a missing \n newline

---

a cleaner way

#!/bin/bash

declare -a servers=(docker irj);
declare -A stat_time;

for server in ${servers[*]}; do
    stat_time[$server]=$(ssh $server "stat -c %Y /var/log/auth.log");
done

for server in ${servers[*]}; do
    printf "%-20s %s\n" stat_time[$server]: ${stat_time[$server]};
done

sample output for me

stat_time[docker]:   1658415177
stat_time[irj]:      1658415179

---

if there are more than a single file (multiple files)

#!/bin/bash

declare -r server=docker

# save list of files and their time in an array
mapfile -t files < <(ssh $server "find -maxdepth 1 -type f -printf '%T+@%p\n'");

OLDIFS=$IFS
IFS='@'
for file in "${files[@]}"; do
    # extra time and filename
    read _time _filename <<< "$file";

    printf "time: %s and filename: %s\n" $_time "$_filename";
done

echo

for file in "${files[@]}"; do
    # extra time and filename
    read _time _filename <<< "$file";

    # extract valid format for date --date
    _time=$(sed 's/+/ /' <<< $_time);

    # cover time to seconds
    _time=$(date --date $_time +%s);
    printf "time(seconds): %s and filename: %s\n" $_time "$_filename";
done
IFS=$OLDIFS

sample output for me

time: 2022-07-21+05:35:28.1242915540 and filename: ./.viminfo
time: 2022-05-17+08:59:32.3547403750 and filename: ./.cloud-locale-test.skip
time: 2022-07-10+10:06:06.3498292130 and filename: ./.lesshst
time: 2022-07-21+15:09:39.1537378410 and filename: ./.bash_history
time: 2022-07-19+14:12:51.0595533570 and filename: ./.bashrc
time: 2022-07-21+08:54:02.6645262280 and filename: ./.gitconfig
time: 2022-07-19+05:45:24.5526313330 and filename: ./.vimrc
time: 2019-12-05+14:39:21.0000000000 and filename: ./.profile
time: 2022-07-19+13:33:53.2693524620 and filename: ./.wget-hsts

time(seconds): 1658365528 and filename: ./.viminfo
time(seconds): 1652761772 and filename: ./.cloud-locale-test.skip
time(seconds): 1657431366 and filename: ./.lesshst
time(seconds): 1658399979 and filename: ./.bash_history
time(seconds): 1658223771 and filename: ./.bashrc
time(seconds): 1658377442 and filename: ./.gitconfig
time(seconds): 1658193324 and filename: ./.vimrc
time(seconds): 1575544161 and filename: ./.profile
time(seconds): 1658221433 and filename: ./.wget-hsts