Reputation: 5660
I am attempting to untar multiple tar files over ssh:
ssh user@hostname "cat /dir/file*.tgz" | tar xvzf -
The above only works on the first file match on the remote server. The local (dest) server only receives one file. The wildcard has been verified to match multiple files though.
Is there another way to do this?
Upvotes: 5
Views: 7148
Reputation: 89
You could wrap this in a find call and then use the exec switch to iterate over each found file.
ssh user@hostname 'find /path/to/dir -name "*.tar.gz" -exec tar xvf "{}" ";"'
Single ssh call, no need to write bash script.
Upvotes: 1
Reputation: 74800
(Edit after first try didn't work:)
Another idea, avoiding multiple ssh
calls and also scp
(since this needs at least one roundtrip between each file):
ssh user@hostname 'tar cf - /dir/file*.tgz' | tar xf - --to-command='tar xzvf -'
We use one more tar
call on the server side to wrap all the files together, a second tar
call on the client side to unwrap them again, which will then be calling tar xzv
for each entry once. This works similar to the base64-answer from sehe, but will be more efficient since it does not blow up the files.
Upvotes: 4
Reputation: 107899
It's easier to work with files than with commands. So rather than use the ssh
command to access remote hosts, and the tar
command to access archives, use the SSHFS filesystem to access files on remote hosts
mkdir hostname-dir.tmp
sshfs user@hostname:/dir hostname-dir.tmp
for a in hostname-dir.tmp/*.tgz; do tar xvf "$a"; done
fusermount -u hostname-dir.tmp
You can even go one step further and the AVFS to access archives as directories¹. All brought to you by FUSE, a generic framework to access all kinds of things as files.
mountavfs
mkdir hostname-dir.tmp
sshfs user@hostname:/dir hostname-dir.tmp
for a in ~/.avfs$PWD/hostname-dir.tmp/*.tgz; do cp -Rp "$a#"/* .; done
fusermount -u hostname-dir.tmp
If your shell is zsh, that for
loop can be replaced by a glob qualifier.
cp -Rp ~/.avfs$PWD/hostname-dir.tmp/*.tgz(e\''reply=($REPLY\#/*)'\') .
¹ AVFS can also access remote files over SSH, but I don't think you can access remote archives that way: AVFS paths only have a single hop. So SSHFS is still necessary.
Upvotes: 0
Reputation: 393969
CarpeNoctem inspired me to go even further and provide a one-stop solution that require only a single ssh connection and does all the stuff streaming.
A tested proof concept (oneline formatted for readability):
(echo '#!/bin/sh';
for a in /dir/file*.tgz
do
echo 'base64 -d <<"TARIMAGE" | tar xvzf -'
base64 "$a"
echo "TARIMAGE"
done) | ssh -C remote 'cd /targetdir && sh -'
Of course you'd actually want to reverse the roles of local and remote, in which case I'd recommend getting the script generation into a shell script instead:
gen_script.sh, chmod +x
#!/bin/bash
echo '#!/bin/sh';
for a in /dir/file*.tgz
do
echo 'base64 -d <<"TARIMAGE" | tar xvzf -'
base64 "$a"
echo "TARIMAGE"
done
invocation
scp gen_script.sh user@hostname:
ssh -C user@hostname /home/user/gen_script.sh | (cd targetdir && sh -)
Upvotes: 0
Reputation: 393969
mkdir /tmp/tars
scp 'user@hostname:/dir/file*.tgz' /tmp/tars/
foreach tarname in /tmp/tars/*.tgz; do tar xzvf "$tarname"; done
If you absolutely cannot store temp copies:
ssh user@hostname 'ls /dir/file*.tgz' | while read tarname;
do
ssh user@hostname "cat '/dir/$tarname'" | tar xzvf -
done
Upvotes: 2