CODEWITHSUNDEEP
rubyshell
7wonders
7wonders

Reputation: 1648

Rewriting 3 commands into one command or script that can be run on cron

Im currently using 3 different commands to achieve my goal of downloading a zip, extracting it, converting the txt file to utf8 and then converting the csv to json!

First I have:

wget https://www.example.com/example.zip -O temp.zip; unzip -o temp.zip; rm temp.zip

Which is good, but the problem to start with is how do I rename the file that is extracted so its the same every time for the next processes as it can be a different name within the zip every day? Next I run this script depending on the filename that converts the ISO to utf8:

sh dir_iconv.sh example1.txt ISO8859-1 UTF-8

Which is this script:

#!/bin/bash

ICONVBIN='/usr/bin/iconv' # path to iconv binary

if [ $# -lt 3 ]
then
echo "$0 dir from_charset to_charset"
exit
fi

for f in $1/*
do
if test -f $f
then
    echo -e "\nConverting $f"
    /bin/mv $f $f.old
    $ICONVBIN -f $2 -t $3 $f.old > $f
    rm -f $f.old
else
    echo -e "\nSkipping $f - not a regular file";
fi
done

And then finally I run a ruby script csv2json - https://github.com/darwin/csv2json - that is called as follows (pipe delimited) to give me a json output:

csv2json -s '|' example1.txt > example1.json

Is there a simple way to roll this into one command or script that can be called?

Upvotes: 1

Views: 151

Answers (1)

i-blis
i-blis

Reputation: 3179

Pipe all your commands one after another and, if necessary, throw them in a shell script file.

wget -qO- https://www.example.com/example.zip | unzip | iconv -f ISO8859-1 -t UTF-8 | csv2json > example.json

Upvotes: 3

Related Questions