How to join lines not starting with specific pattern to the previous line in UNIX?

Question

Please take a look at the sample file and the desired output below to understand what I am looking for.

It can be done with loops in a shell script but I am struggling to get an awk/sed one liner.

SampleFile.txt

These are leaves.
These are branches.
These are greenery which gives
oxygen, provides control over temperature
and maintains cleans the air.
These are tigers
These are bears
and deer and squirrels and other animals.
These are something you want to kill
Which will see you killed in the end.
These are things you must to think to save your tomorrow.

Desired output

These are leaves.
These are branches.
These are greenery which gives oxygen, provides control over temperature and maintains cleans the air.
These are tigers
These are bears and deer and squirrels and other animals.
These are something you want to kill Which will see you killed in the end.
These are things you must to think to save your tomorrow.

Answer 1

Here is a sed program which avoids branches. I tested it with the --posix option. The trick is to use an "anchor" (a string which does not occur in the file):

 sed --posix -n '/^These/!{;s/^/DOES_NOT_OCCUR/;};H;${;x;s/^\n//;s/\nDOES_NOT_OCCUR/ /g;p;}'

Explanation:

1. write DOES_NOT_OCCUR at the beginning of lines not starting with "These":

   /^These/!{;s/^/DOES_NOT_OCCUR/;};

2. append the pattern space to the hold space

   H;

3. If the last line is read, exchange pattern space and hold space

   ${;x;

4. Remove the newline at the beginning of the pattern space which is added by the H command when it added the first line to the hold space

   s/^\n//;

5. Replace all newlines followed by DOES_NOT_OCCUR with blanks and print the result

   s/\nDOES_NOT_OCCUR/ /g;p;}

Note that the whole file is read in sed's process memory, but with only 4GB this should not be a problem.

Answer 2

With sed:

sed ':a;N;/\nThese/!s/\n/ /;ta;P;D' infile

resulting in

These are leaves.
These are branches.
These are greenery which gives oxygen, provides control over temperature and maintains cleans the air.
These are tigers
These are bears and deer and squirrels and other animals.
These are something you want to kill Which will see you killed in the end.
These are things you must to think to save your tomorrow.

Here is how it works:

sed '
:a                   # Label to jump to
N                    # Append next line to pattern space
/\nThese/!s/\n/ /    # If the newline is NOT followed by "These", append
                     # the line by replacing the newline with a space
ta                   # If we changed something, jump to label
P                    # Print part until newline
D                    # Delete part until newline
' infile

The N;P;D is the idiomatic way of keeping multiple lines in the pattern space; the conditional branching part takes care of the situation where we append more than one line.

This works with GNU sed; for other seds like the one found in Mac OS, the oneliner has to be split up so branching and label are in separate commands, the newlines may have to be escaped, and we need an extra semicolon:

sed -e ':a' -e 'N;/'$'\n''These/!s/'$'\n''/ /;ta' -e 'P;D;' infile

_{This last command is untested; see this answer for differences between different seds and how to handle them.}

Another alternative is to enter the newlines literally:

sed -e ':a' -e 'N;/\
These/!s/\
/ /;ta' -e 'P;D;' infile

But then, by definition, it's no longer a one-liner.

Answer 3

Please try the following:

awk 'BEGIN {accum_line = "";} /^These/{if(length(accum_line)){print accum_line; accum_line = "";}} {accum_line = accum_line " " $0;} END {if(length(accum_line)){print accum_line; }}' < data.txt

The code consists of three parts:

1. The block marked by BEGIN is executed before anything else. It's useful for global initialization
2. The block marked by END is executed when the regular processing finished. It is good for wrapping the things. Like printing the last collected data if this line has no These at the beginning (this case)
3. The rest is the code performed for each line. First, the pattern is searched for and the relevant things are done. Second, data collection is done regardless of the string contents.

Answer 4

$ awk '{printf "%s%s", (NR>1 ? (/^These/?ORS:OFS) : ""), $0} END{print ""}' file
These are leaves.
These are branches.
These are greenery which gives oxygen, provides control over temperature and maintains cleans the air.
These are tigers
These are bears and deer and squirrels and other animals.
These are something you want to kill Which will see you killed in the end.
These are things you must to think to save your tomorrow.

Answer 5

another awk if you have support for multi-char RS (gawk has)

$ awk -v RS="These" 'NR>1{$1=$1; print RS, $0}' file

These are leaves.
These are branches.
These are greenery which gives oxygen, provides control over temperature and maintains cleans the air.
These are tigers
These are bears and deer and squirrels and other animals.
These are something you want to kill Which will see you killed in the end.
These are things you must to think to save your tomorrow.

Explanation Set the record delimiter as "These", skip the first (empty) record. Reassign field to force awk to restructure the record; print record separator and the rest of the record.

Answer 6

Not a one-liner (but see end of answer!), but an awk-script:

#!/usr/bin/awk -f

NR == 1     { line = $0 }
/^These/    { print line; line  = $0 }
! /^These/  { line = line " " $0 }
END         { print line }

Explanation:

I'm accumulating, building up, lines that start with "These" with lines not starting with "These", outputting the completed lines whenever I find the next line with "These" at the beginning.

1. Store the first line (the first "record").
2. If the line starts with "These", print the accumulated (previous, now complete) line and replace whatever we have found so far with the current line.
3. If it doesn't start with "These", accumulate the line (i.e concatenate it with the previously read incomplete lines, with a space in between).
4. When there's no more input, print the last accumulated (now complete) line.

Run like this:

$ ./script.awk data.in

As a one-liner:

$ awk 'NR==1{c=$0} /^These/{print c;c=$0} !/^These/{c=c" "$0} END{print c}' data.in

... but why you would want to run anything like that on the command line is beyond me.

EDIT Saw that it was the specific string "These" (/^These/) that was what should be looked for. Previously had my code look for uppercase letters at the start of the line (/^[A-Z]/).

Answer 7

awk '$1==These{print row;row=$0}$1!=These{row=row " " $0}'

you can take it from there. blank lines, separators,
other unspecified behaviors (untested)