CODEWITHSUNDEEP
linuxawksednewlinetrim
kwanx
kwanx

Reputation: 11

Bash trim or sed newline

Have log file that has SOME new lines to remove (not all); have tried many ways of trim and sed with no luck (usually ends up one big line).

I think i need to have a specific search, when trimming or sedding, but not sure.

example file:

JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=X;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams)
   at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams)
   at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX ; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX:VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:09.313
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457

Need out:

JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=X;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams) at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams) at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX ; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX:VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:09.313
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457

Any help greatly apprecaited!

Upvotes: 0

Views: 440

Answers (3)

Graham
Graham

Reputation: 1759

If what you're doing is joining lines that start with whitespace to the previous line, you could do this in bash.

last=""
while IFS="" read line; do
  if [[ "$line" =~ ^[[:space:]] ]]; then
    echo -n "$last "
  else
    echo "$last"
  fi
  last="$line"
done < inp
echo "$line"

Or even in POSIX (non-bash) shell:

last=""
while IFS="" read line; do
  if expr "$line" : '[[:space:]]' >/dev/null; then
    echo -n "$last "
  else
    echo "$last"
  fi
  last="$line"
done < inp
echo "$line"

Upvotes: 1

jayant
jayant

Reputation: 2389

You can do this with awk:

awk 'BEGIN {ORS=""} /^ /{ print $0; next;} {print "\n"$0;}' input_file | sed 's/\r//g'

which gives

JOBV2|MISC: PURCHASE_DENIED|Puchasemode = sync ; iEoCode = 992 ; teminalId = XXXXXXXXXXXX; offeingId = XXXXXX; entalType = ps ; pice = 1649 ; entalWindowHs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TansactionID = IsUG=N;TId=X;VID=XXXX;Pice=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTansactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pid=0;RetPice=16.49;DisPice=0.00;NetPice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Puchase_Denied Eo sting: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PuchasePaams objPuchasePaams)   at STRING.JOBV3(Sting stInputPaams)|stInputPaams = stUnitAddess=XXXXXXXXXXXX,teminalid=XXXXXXXXXXXX,offeingid=XXXXXX,entalType=PS,pice=1649,entalWindowHs=0,IsAdult=false,IsRedeemTitle=false,pomotionType=,isUpgade=N,catalogpice=0,stpomotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Puchase_Denied Eo sting: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PuchasePaams objPuchasePaams)   at STRING.JOBV3(Sting stInputPaams)|stInputPaams = stUnitAddess=XXXXXXXXXXXX,teminalid=XXXXXXXXXXXX,offeingid=XXXXXX,entalType=PS,pice=1649,entalWindowHs=0,IsAdult=false,IsRedeemTitle=false,pomotionType=,isUpgade=N,catalogpice=0,stpomotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: PURCHASE_DENIED|Puchasemode = sync ; iEoCode = 992 ; teminalId = XXXXXXXXXXXX ; offeingId = XXXXXX; entalType = ps ; pice = 1649 ; entalWindowHs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TansactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pice=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTansactionID = IsUG=N;TId=XXXXXXXXXXXX:VID=XXXX;Pid=0;RetPice=16.49;DisPice=0.00;NetPice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:09.313
JOBV2|MISC: Non-Puchase_Denied Eo sting: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457

Alternative method:

awk 'BEGIN {ORS=""} /\)$/{print $0; next;} {print $0"\n";}' input_file | sed 's/\r//g'

Upvotes: 2

Jose Ricardo Bustos M.
Jose Ricardo Bustos M.

Reputation: 8164

Another solution, you can use sed

sed -r ':a;N;$!ba;s/\n\s+/ /g' file

or, without to use extended regular expressions option -r

sed ':a;N;$!ba;s/\n[ \t][ \t]*/ /g' file

you get

JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=X;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams) at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams) at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX ; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX:VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:09.313
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457

Upvotes: 0

Related Questions