CODEWITHSUNDEEP
parsingcsvawkseddig
user3666682
user3666682

Reputation: 23

Parse dig output export csv

using a dig command in a shell script and want to output into csv format flags and authority section

dig @ns1.hosangit.com djzah.com +noall +authority +comments

output

; <<>> DiG 9.8.3-P1 <<>> @ns1.hosangit.com djzah.com +noall +authority +comments
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64505
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; AUTHORITY SECTION:
djzah.com.  3600    IN  NS  ns3.eventguyz.com.
djzah.com.  3600    IN  NS  ns1.eventguyz.com.
djzah.com.  3600    IN  NS  ns2.eventguyz.com.

Expected output for csv is ( domain, flags (not always these three), authority section (could be 5) ):

djzah.com,qr,aa,rd,ns3.eventguyz.com,ns1.eventguyz.com,ns2.eventguyz.com

I was trying to use awk and/or sed but am having difficulty searching for a pattern like for the flags section ;; flags: (then use a space delimiter until you reach ;)

Then the Authority section, I assume you would search for ;; AUTHORITY SECTION: Then create an array and only use the last.

I don't know what I'm doing.

Upvotes: 2

Views: 3258

Answers (1)

konsolebox
konsolebox

Reputation: 75608

#!/usr/bin/awk -f
BEGIN { OFS = "," }
/^;; flags:/ {
    sub(/;; flags: /, "")
    sub(/;.*$/, "")
    $1 = $1
    flags = "," $0
    next
}
/^;/ || NF < 5 { next }
!($1 in a) {
    keys[++k] = $1
}
{
    t = $5
    sub(/[.][ \t\r]*$/, "", t)
    a[$1] = a[$1] "," t
}
END {
    for (i = 1; i <= k; ++i) {
        key = keys[k]
        t = key
        sub(/[.][ \t\r]*$/, "", t)
        print t flags a[key]
    }
}

Usage:

dig @ns1.hosangit.com djzah.com +noall +authority +comments | awk -f script.awk

Test:

awk -f script.awk sample

Output:

djzah.com,qr,aa,rd,ns3.eventguyz.com,ns1.eventguyz.com,ns2.eventguyz
  • BEGIN { OFS = "," }: Every section in awk always run everytime a record is processed. BEGIN block only run once at start. This basically sets OFS to ,.
  • /^;; flags:/ matches ;; flags:. The section that is presented by it basically extracts the flags from the record (line). The sub commands basically remove unnecessary parts from the record. $1 = $1 just makes sure $0 is updated with OFS. flags = "," $0 assigns the now comma-separated flags into flags variable. next makes awk jump to the next record.
  • /^;/ || NF < 5 { next } basically makes awk skip unnecessary lines.
  • !($1 in a) { keys[++k] = $1 } if $1 e.g. djzah.com. is first encountered, add to keys array.
  • { t = $5; sub(/[.][ \t\r]*$/, "", t); a[$1] = a[$1] "," t } adds the value of the 5th column e.g. ns3.eventguyz.com to the collection with the leading . removed.
  • When processing is finished, END block executes. It iterates through the keys found and prints the data bound to it.

Upvotes: 1

Related Questions