CODEWITHSUNDEEP
bashwgetunzip
Roger
Roger

Reputation: 8576

How to unzip a piped zip file (from "wget -qO-")?

Any ideas on how to unzip a piped zip file like this:

wget -qO- http://downloads.wordpress.org/plugin/akismet.2.5.3.zip

I wished to unzip the file to a directory, like we used to do with a normal file:

wget -qO- http://downloads.wordpress.org/plugin/akismet.2.5.3.zip | unzip -d ~/Desktop

Upvotes: 58

Views: 75038

Answers (8)

adamency
adamency

Reputation: 1500

Another solution if you already have unzip, you should also have funzip which comes from the same [unzip] package.

This utility is made for reading from pipes/stdin. However it seems very primitive and can apparently extract only the first file from the archive (as per the manpage).

Anyway for the sake of completeness on the context of the question, here is a way I found to do it with funzip for a single-file .zip archive:

curl -sL https://<url_to_my_archive>.zip | funzip - > <my_extracted_file>

replace <url_to_my_archive> and <my_extracted_file> with your values.

Upvotes: 0

Ian Robertson
Ian Robertson

Reputation: 1348

While the following will not work in bash, it will work in zsh. Since many zsh users may end up here, it may still be useful:

% unzip =( wget -qO- http://downloads.wordpress.org/plugin/akismet.2.5.3.zip )
Archive:  /tmp/zshLCod6x
   creating: akismet/
  inflating: akismet/admin.php       
  inflating: akismet/akismet.css     
  inflating: akismet/akismet.gif     
  inflating: akismet/akismet.js      
  inflating: akismet/akismet.php     
  inflating: akismet/legacy.php      
  inflating: akismet/readme.txt      
  inflating: akismet/widget.php      
%

As you can notice the temporary downloaded zip file has been deleted straight away :

% ls /tmp/zshLCod6x
ls: cannot access '/tmp/zshLCod6x': No such file or directory
%

Upvotes: 14

pts
pts

Reputation: 87221

Reposting my answer:

I wrote a Python (2.x) script to do streaming extraction of ZIP archives, you can get it from here: https://raw.githubusercontent.com/pts/unzip_scan/master/unzip_scan.py . Usage: cat file.zip | sh unzip_scan.py -.

Upvotes: 2

Leon
Leon

Reputation: 2896

wget -q -O tmp.zip http://downloads.wordpress.org/plugin/akismet.2.5.3.zip && unzip tmp.zip && rm tmp.zip

Upvotes: 11

Saftever
Saftever

Reputation: 785

BusyBox's unzip can take stdin and extract all the files.

wget -qO- http://downloads.wordpress.org/plugin/akismet.2.5.3.zip | busybox unzip -

The dash after unzip is to use stdin as input.

You can even,

cat file.zip | busybox unzip -

But that's just redundant of unzip file.zip.

If your distro uses BusyBox by default (e.g. Alpine), just run unzip -.

Upvotes: 28

lanzalibre
lanzalibre

Reputation: 370

just use zcat

wget -qO- http://downloads.wordpress.org/plugin/akismet.2.5.3.zip | zcat >> myfile.txt
  • This will only extract first file. You will see this error message "gzip: stdin has more than one entry--rest ignored" after the first file is extracted.

Upvotes: 18

ruario
ruario

Reputation: 1707

The ZIP file format includes a directory (index) at the end of the archive. This directory says where, within the archive each file is located and thus allows for quick, random access, without reading the entire archive.

This would appear to pose a problem when attempting to read a ZIP archive through a pipe, in that the index is not accessed until the very end and so individual members cannot be correctly extracted until after the file has been entirely read and is no longer available. As such it appears unsurprising that most ZIP decompressors simply fail when the archive is supplied through a pipe.

The directory at the end of the archive is not the only location where file meta information is stored in the archive. In addition, individual entries also include this information in a local file header, for redundancy purposes.

Although not every ZIP decompressor will use local file headers when the index is unavailable, the tar and cpio front ends to libarchive (a.k.a. bsdtar and bsdcpio) can and will do so when reading through a pipe, meaning that the following is possible:

wget -qO- http://downloads.wordpress.org/plugin/akismet.2.5.3.zip | bsdtar -xvf- -C ~/Desktop

Upvotes: 72

Sam Cantrell
Sam Cantrell

Reputation: 585

I'd take a look at funzip (http://www.info-zip.org/mans/funzip.html). The man page for it notes,

...filter for extracting from a ZIP archive in a pipe

Sorry I don't have an example, but it looks like it does come with the Linux unzip utility.

Upvotes: 5

Related Questions