CODEWITHSUNDEEP
bashfileshelltext-parsingfileparsing
user1584119
user1584119

Reputation: 11

I need to parse a log file into multiple files based on delimiters

I have a log file which i need to parse it into multiple files.

############################################################################################
6610
############################################################################################
GTI02152 I    gtirreqi 20130906 000034 TC SJ014825         GTT_E_REQ_INF テーブル挿入件数 16件

############################################################################################
Z5000
############################################################################################
GTP10000 I NIPS     gtgZ5000 20130906 000054 TC SJ014825         シェル開始
############################################################################################

I need to create files like 6610.txt which will have all values under 6610 like(GTI02152..) and for z5000(GTP10000) respectively. Any help will be greatly appreciated!

Upvotes: 0

Views: 471

Answers (4)

Digital Trauma
Digital Trauma

Reputation: 15996

This script makes the following assumptions:

  • Each record is separated by an empty line
  • #### lines are purely comment/space filler and can be ignored during parsing
  • The first line of each record (ignoring ####) contains the basename for the filename

The name of the logfile is passed as the first argument to this script.

#!/bin/bash

# write records to this temporary file, rename later
tempfile=$(mktemp)
while read line; do
    if [[ $line == "" ]] ; then
        # line is empty - separator - save existing record and start a new one
        mv $tempfile $filename
        filename=""
        tempfile=$(mktemp)
    else
        # output non-empty line to record file
        echo $line >> $tempfile
        if [[ $filename == "" ]] ; then
            # we haven't yet figured out the filename for this record
            if [[ $line =~ ^#+$ ]] ; then
                # ignore #### comment lines
                :
            else
                # 1st non-comment line in record is filename
                filename=${line}.txt
            fi
        fi
    fi
done < $1
# end of input file might not have explicit empty line separator -
# make sure last record file is moved correctly
if [[ -e $tempfile ]] ; then
    mv $tempfile $filename
fi

Upvotes: 0

konsolebox
konsolebox

Reputation: 75488

You can do it with Ruby as well:

ruby -e 'File.read(ARGV.shift).scan(/^[^#].*?(?=^[#])/m).each{|e| name = e.split[0]; File.write("#{name}.txt", e)}' file

Example output:

> for A in *.txt; do echo "---- $A ----"; cat "$A"; done
---- 6610.txt ----
6610
---- GTI02152.txt ----
GTI02152 I    gtirreqi 20130906 000034 TC SJ014825         GTT_E_REQ_INF テーブル挿入件数 16件

---- GTP10000.txt ----
GTP10000 I NIPS     gtgZ5000 20130906 000054 TC SJ014825         シェル開始
---- Z5000.txt ----
Z5000

Upvotes: 0

n0741337
n0741337

Reputation: 2514

Here's an awk styled answer:

I put the following into a file named awko and chmod +x it to use it:

#!/usr/bin/awk -f

BEGIN { p = 0 }    # look for filename flag - start at zero

/^\#/ { p = !p }   # turn it on to find the filename

    # either make a filename or write to the last filename based on the flag
$0 !~ /^\#/ {
    if( p == 1 ) filename = $1 ".txt"
    else print $0 > filename
    }

Running awko data.txt produced two files, 6610.txt and Z5000.txt from your example data. It's capable of sending more data lines to the output files as well.

Upvotes: 0

kumar_m_kiran
kumar_m_kiran

Reputation: 4022

Below script would help you to get the information. You can modify them to create the data you require. 

#!/bin/sh

cmd=`cat data.dat | paste -d, - - - - - | cut -d ',' -f 2,4 > file.out`
$cmd

while read p; do
    fileName=`echo $p | cut -d ',' -f 1`
    echo $fileName
    dataInfo=`echo $p | cut -d ',' -f 2`
    echo $dataInfo
done< file.out

Upvotes: 1

Related Questions