CODEWITHSUNDEEP
linuxawksedappend
Rajeev
Rajeev

Reputation: 1413

how to continue to append starting from a pattern using sed or awk?

How to add consecutive lines using sed (or awk) untill a pattern is found?

I have this data:

max system cycles: 9087AED
max system scans: 900
Secure connection

date:....


1:    ID                 = XX_34_4_7_6e
      field1          = trc_1_1
      field2               = 24
      Blank field2          = 
      field3 XX       = OK (45)
      field4 XX               = assfsdf
      field5 XX YY      = sdffee (ssddT)
      field6 a b c d = no
      field7 ID            = zip
      field8               = city

2:    ID                 = YY_z3_4_t20
      field1          = xyz1_1_t
      field2               = 20
      Blank field2          = 
      field3 XX       = OK (5)

I intend to normalize this text by merging all rows for a record (starting with a numeral) into one line. I am not sure how to append more then one line?...

problem with below is: only one line gets appended. lines at top which are not exactly the "record" gets appended too....

$ sed  -e '/[0-9]/ N; s/\n/,/' /var/tmp/tmp1
max system cycles: 9087AED,max system scans: 900
Secure connection

date:....


1:    ID                 = XX_34_4_7_6e,      field1          = trc_1_1
      field2               = 24,      Blank field2          = 
      field3 XX       = OK (45),      field4 XX               = assfsdf
      field5 XX YY      = sdffee (ssddT),      field6 a b c d = no
      field7 ID            = zip,      field8               = city

2:    ID                 = YY_z3_4_t20,      field1          = xyz1_1_t
      field2               = 20,      Blank field2          = 
      field3 XX       = OK (5)

How can we continue to append till next record starts?

Thank you.

Edit: Adding the output I was able to get using some help from internet. The only issue is it operates on all lines. I have to figure how to restrict to 'records' only....

sed '/./{H;$!d} ; x ; s/.\n/,/g; s/ *//g' /var/tmp/tmp1

maxsystemcycles:9087AE,maxsystemscans:90,Secureconnection

date:....


1:ID=XX_34_4_7_6,field1=trc_1_,field2=2,Blankfield2=,field3XX=OK(45,field4XX=assfsd,field5XXYY=sdffee(ssddT,field6abcd=n,field7ID=zi,field8=city

2:ID=YY_z3_4_t2,field1=xyz1_1_,field2=2,Blankfield2=,field3XX=OK(5)

Upvotes: 3

Views: 68

Answers (2)

Ed Morton
Ed Morton

Reputation: 204381

This, using any awk, might be what you're trying to do but without seeing the expected output it's just a guess:

$ awk -v RS= -F'\n' -v OFS=, '/^[0-9]/{$1=$1} 1' file
max system cycles: 9087AED
max system scans: 900
Secure connection
date:....
1:    ID                 = XX_34_4_7_6e,      field1          = trc_1_1,      field2               = 24,      Blank field2          = ,      field3 XX       = OK (45),      field4 XX               = assfsdf,      field5 XX YY      = sdffee (ssddT),      field6 a b c d = no,      field7 ID            = zip,      field8               = city
2:    ID                 = YY_z3_4_t20,      field1          = xyz1_1_t,      field2               = 20,      Blank field2          = ,      field3 XX       = OK (5)

or maybe you'd prefer less white space on each output line:

$ awk -v RS= -F'\n' -v OFS=, '/^[0-9]/{$1=$1; gsub(/ +/," ")} 1' file
max system cycles: 9087AED
max system scans: 900
Secure connection
date:....
1: ID = XX_34_4_7_6e, field1 = trc_1_1, field2 = 24, Blank field2 = , field3 XX = OK (45), field4 XX = assfsdf, field5 XX YY = sdffee (ssddT), field6 a b c d = no, field7 ID = zip, field8 = city
2: ID = YY_z3_4_t20, field1 = xyz1_1_t, field2 = 20, Blank field2 = , field3 XX = OK (5)

or:

$ awk -v RS= -F'\n' -v OFS=, '/^[0-9]/{$1=$1; gsub(/ +/,"")} 1' file
max system cycles: 9087AED
max system scans: 900
Secure connection
date:....
1:ID=XX_34_4_7_6e,field1=trc_1_1,field2=24,Blankfield2=,field3XX=OK(45),field4XX=assfsdf,field5XXYY=sdffee(ssddT),field6abcd=no,field7ID=zip,field8=city
2:ID=YY_z3_4_t20,field1=xyz1_1_t,field2=20,Blankfield2=,field3XX=OK(5)

There are lots of possibilities...

Upvotes: 5

Daweo
Daweo

Reputation: 36700

If your file has not NUL byte, then you might exploit GNU sed following way, let file.txt content be

max system cycles: 9087AED
max system scans: 900
Secure connection

date:....


1:    ID                 = XX_34_4_7_6e
      field1          = trc_1_1
      field2               = 24
      Blank field2          = 
      field3 XX       = OK (45)
      field4 XX               = assfsdf
      field5 XX YY      = sdffee (ssddT)
      field6 a b c d = no
      field7 ID            = zip
      field8               = city

2:    ID                 = YY_z3_4_t20
      field1          = xyz1_1_t
      field2               = 20
      Blank field2          = 
      field3 XX       = OK (5)

then

sed -z 's/\n /, /g' file.txt

gives output

max system cycles: 9087AED
max system scans: 900
Secure connection

date:....


1:    ID                 = XX_34_4_7_6e,      field1          = trc_1_1,      field2               = 24,      Blank field2          = ,      field3 XX       = OK (45),      field4 XX               = assfsdf,      field5 XX YY      = sdffee (ssddT),      field6 a b c d = no,      field7 ID            = zip,      field8               = city

2:    ID                 = YY_z3_4_t20,      field1          = xyz1_1_t,      field2               = 20,      Blank field2          = ,      field3 XX       = OK (5)

Explanation: I use -z option to get whole content of file as one line, then I globally replace newline following by space using comma followed by space, thus merging line starting with space to previous line.

(tested in GNU sed 4.9)

Upvotes: 3

Related Questions