CODEWITHSUNDEEP
bashsed
HarryH
HarryH

Reputation: 165

Command substitution in sed

I want to read the first six characters of a text file into a string and prepend every other non-blank line in that file with that string. An example of such file could be:

04/17 Walmart .toys $ 70 .cash $ -70

Caltex .gas 20 $ .cheque $ -20

McDonalds .burger 1 $ .cash $ -1

Each entry, i.e.: each non-empty line, needs a date, which for reasons of easy data entry has been entered on the first line only. Entries are separated by 1 or more empty lines. The output would look like this:

04/17 Walmart .toys $ 70 .cash $ -70

04/17 Caltex .gas 20 $ .cheque $ -20

04/17 McDonalds .burger 1 $ .cash $ -1

I can match the non empty strings with things like ^[^@]+[ ]*.[ ]([^;{}:]+)[ ]*$, but I don't know how to actually implement that for non blank lines.

This Bash script looks attractive to me, but I don't know how then to insert my string at the start.

I also can't find the straight answer to my question on Stack Overflow.

I tried a script which accepts a file name:

read -n 6 date < $1
sed 's/^/$(echo $date)/' | \
sed 's/^$(echo $date)\n//' | > $newName

I was able to come up with prepending the date with space (e.g. the string: '04/17 ') to every line, and then remove the same from every line in which nothing follows.

However, it seems that sed doesn't accept the command substitution:

sed: -e expression #1, char 10: unknown option to `s'

Upvotes: 4

Views: 619

Answers (4)

clt60
clt60

Reputation: 63912

Using Perl:

perl -plE 'if($.==1){$d=substr($_,0,6);next}elsif(/./){s/^/$d/}' file > new

Output

04/17 Walmart .toys $ 70 .cash $ -70

04/17 Caltex .gas 20 $ .cheque $ -20

04/17 McDonalds .burger 1 $ .cash $ -1

Or in same file with backup to file.bak

perl -i.bak -plE 'if($.==1){$d=substr($_,0,6);next}elsif(/./){s/^/$d/}' file

Or same file without backup

perl -i -plE 'if($.==1){$d=substr($_,0,6);next}elsif(/./){s/^/$d/}' file

Or, if you're unsure about the leading zeros in the date,

perl -plE 'if($.==1){($d)=m|^(\d+/\d+\s)|;next}elsif(/./){s/^/$d/}' file

will match any digit(s) / digit(s) space on the beginning of the 1st line.

As l'L'l mentioned in comment, the above adds the date to pseudo-empty lines too (where the line only looks-like empty) e.g. it contains at least one space. In such case, instead of /./:

  • Use /\w/ - so, prepend the date only to lines containing at least one word character;
  • Or use /\S/ - when containing at least one non-space character

Explanation:

perl -plE '                    # Run the commands on every input line and print them.
    if( $. == 1) {             # If it is the 1st line
        $d = substr($_, 0, 6); # take the first 6 characters and store it to $d
        next                   # And continue to the next line.
    }
    elsif( /\S/ ) {            # Else if the line contains any nonspace character
        s/^/$d/                # add to the beginning the content of $d
    }
    ' file > new

Upvotes: 3

user000001
user000001

Reputation: 33327

The slash is terminating the sed command, change the delimiter to something else:

"s#^#$(echo $date)#"

You could probably write like this too:

"s#^#$date#"

Note however that this approach is generally fragile (as you have discovered), because you can't treat the variable as a literal string.

Based on the samples on the updated question, I would suggest to use a single awk command to do your text processing. Something like this could give you the sample output:

$ cat file
04/17 Walmart .toys $ 70 .cash $ -70

Caltex .gas 20 $ .cheque $ -20

McDonalds .burger 1 $ .cash $ -1

$ awk 'NR==1{d=$1}NR>1&&NF>0{$0=d" "$0}1' file
04/17 Walmart .toys $ 70 .cash $ -70

04/17 Caltex .gas 20 $ .cheque $ -20

04/17 McDonalds .burger 1 $ .cash $ -1

Upvotes: 2

l&#39;L&#39;l
l&#39;L&#39;l

Reputation: 47169

You should be able to do this with one sed command:

read -rn 6 date < "$1"
sed -E 's#^([a-zA-Z]+)#'"$date"' \1#g' "$1" > newfile

The capture group ensures there's at least one character on the line before inserting the date.

EDIT: Based on the revision to your question:

newfile="output.txt"
lineone=$(head -1 "$1");

read -rn 6 date <<< "$lineone"
sed -E 's#^([a-zA-Z]+)#'"$date"' \1#g; 1s#^.*$#'"$lineone"'#' "$1" > "$newfile"

Since you aren't doing an in-place edit, you can do the $date insertions, and then go back and swap out the first line since it would end up with two dates. There might be "better" ways to do this such as using Perl, or losing the second sed command, although this should at least give you a basic idea though on how it works...

Result (newfile):

04/17 Walmart .toys $ 70 .cash $ -70

04/17 Caltex .gas 20 $ .cheque $ -20

04/17 McDonalds .burger 1 $ .cash $ -1

NOTE: In some versions of sed the option for extended regex can either be -r or -E.

Upvotes: 4

agc
agc

Reputation: 8406

Pure bash answer:

unset n
while read -r x ; do
    case "${#n}$x" in 6) ;; 6*) x="$n$x" ;; *) n="${x:0:6}" ;; esac
    echo "$x"
done < file > newfile

Output:

04/17 Walmart .toys $ 70 .cash $ -70

04/17 Caltex .gas 20 $ .cheque $ -20

04/17 McDonalds .burger 1 $ .cash $ -1

Upvotes: 3

Related Questions