CODEWITHSUNDEEP
bashsedgrep
khalessi22
khalessi22

Reputation: 25

Using bash to parse data from log files?

My log file looks something like this: 

<lun compNum="3" partitionNum="1" failed="1" backup="1" free_mem="1024" max_mem="2048" />
<lun compNum="22" partitionNum="11" failed="1" backup="0" free_mem="1024" max_mem="2048" />

I want a output that goes through all of those lines and saves the pair of compNum and paritionNum into this:

PAIR=($compNumValue parititonNumValue)

A solution without using any external libraries and only using native bash would be ideal. Any idea how to get started for someone who is new to bash?

Upvotes: 1

Views: 98

Answers (1)

Charles Duffy
Charles Duffy

Reputation: 295291

The Wrong Way (What You Asked For)

lun_re='<lun compNum="([[:digit:]]+)" partitionNum="([[:digit:]]+)"[^/]* />'
while IFS= read -r line; do
  if [[ $line =~ $lun_re ]]; then
    compNum=${BASH_REMATCH[1]}
    partitionNum=${BASH_REMATCH[2]}
    echo "Read a line with compNum $compNum and partitionNum $partitionNum"
  fi
done

See:

The Right Way

Use an XML parser when reading XML. Really. Anything else won't know how to deal with comments, namespaces, CDATA sections, or even simple changes like a new version of the program generating this log putting all the elements on the same line (or putting newlines between the attributes, or even just changing the attributes to have partitionNum come before partNum).

In this context, that might look like:

while IFS='|' read -r -d '' compNum partitionNum; do
  echo "Read a line with compNum $compNum and partitionNum $partitionNum"
done < <(xmlstarlet sel -t -m '//lun[@compNum][@partitionNum]' \
           -v ./@compNum -o '|' \
           -v ./@partitionNum -n)

In addition to the links given above, see BashFAQ #24 to understand why the < <(...) syntax is in use.

If you don't have XMLStarlet installed on the systems where this is being run, you can generate an XSLT template and use xsltproc to evaluate it. xmlstarlet -C emits the following XSLT as equivalent to the above command:

<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:exslt="http://exslt.org/common" version="1.0" extension-element-prefixes="exslt">
  <xsl:output omit-xml-declaration="yes" indent="no"/>
  <xsl:template match="/">
    <xsl:for-each select="//lun[@compNum][@partitionNum]">
      <xsl:call-template name="value-of-template">
        <xsl:with-param name="select" select="./@compNum"/>
      </xsl:call-template>
      <xsl:text>|</xsl:text>
      <xsl:call-template name="value-of-template">
        <xsl:with-param name="select" select="./@partitionNum"/>
      </xsl:call-template>
      <xsl:value-of select="'&#10;'"/>
    </xsl:for-each>
  </xsl:template>
  <xsl:template name="value-of-template">
    <xsl:param name="select"/>
    <xsl:value-of select="$select"/>
    <xsl:for-each select="exslt:node-set($select)[position()&gt;1]">
      <xsl:value-of select="'&#10;'"/>
      <xsl:value-of select="."/>
    </xsl:for-each>
  </xsl:template>
</xsl:stylesheet>

...such that you can use xsltproc /path/to/template.xslt - as equivalent to the given xmlstarlet command.

Upvotes: 4

Related Questions