expand ipv6 address in shell script

Question

I'd like to update a djbdns (dbndns) configuration file based on a given IPv6 address, e.g. 2a01:488:66:1000:523:f116:0:1 or ::1.

dbndns requires expanded IPv6 addresses, e.g. 2a010488006610000523f11600000001 for 2a01:488:66:1000:523:f116:0:1.

What's the most simple way to expand such an IPv6 address?

Answer 1

POSIX shell solution:

#!/bin/sh
expand_ipv6()
{
  __expand_ipv6_ip="${1%%/*}"
  __expand_ipv6_mask=""

  # extract and filter mask at end of address
  case "$1" in
    */*)
      __expand_ipv6_mask="${1#*/}"
      __expand_ipv6_mask="/${__expand_ipv6_mask%%[^0-9/]*}"
  esac

  case "$__expand_ipv6_ip" in
    :*) __expand_ipv6_ip="0$__expand_ipv6_ip"
  esac

  case "$__expand_ipv6_ip" in
    *::*)
      __expand_ipv6_colons="$(echo "$__expand_ipv6_ip" | tr -c -d ':')"
      __expand_ipv6_expanded="$(echo ":::::::::" | sed -e "s/$__expand_ipv6_colons//" -e 's/:/:0/g')"
      __expand_ipv6_ip="$(echo "$__expand_ipv6_ip" | sed "s/::/$__expand_ipv6_expanded/")"
    ;;
  esac

  __expand_ipv6_blocks="$(echo "$__expand_ipv6_ip" | grep -o '[0-9a-fA-F]\+' | while read -r __expand_ipv6_hex; do [ -n "$__expand_ipv6_hex" ] && printf " %d" "$((0x$__expand_ipv6_hex % 65536))"; done)"
  printf "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x" $__expand_ipv6_blocks
  printf "%s\n" "$__expand_ipv6_mask"
}

Testing:

> expand_ipv6 ab:12:345f::2/32
00ab:0012:345f:0000:0000:0000:0000:0002/32

This is a slightly improved version based on this answer by user48768, who copied it from this github source, which:

• Includes support for optional mask (if the input has no mask, it is not printed either)
• Is secure against code injection (due to using double quotes)
• Does not overwrite existing variables (due to __expand_ipv6_-prefix)

Note: The expand_ipv6 function expects a valid IPv6 address, or else the result is not necessarily valid either. To test if a string is a valid IPv6 address, use:

#!/bin/sh
is_ipv6()
{
  ip -6 route get "$1" >/dev/null 2>/dev/null || [ $? -ne 1 ]
}

Furthermore, to compress IPv6 addresses (the reverse of expand_ipv6), here is also an improved solution for that:

#!/bin/sh
compress_ipv6()
{
  __compress_ipv6_ip="$(echo "$1" | sed -e 's/::/:0:/g' | grep -o "[0-9a-fA-F]\+" | while read -r __compress_ipv6_hex; do [ -n "$__compress_ipv6_hex" ] && printf ":%x" "$((0x$__compress_ipv6_hex))"; done)"

  for __compress_ipv6_chain in :0:0:0:0:0:0:0:0 :0:0:0:0:0:0:0 :0:0:0:0:0:0 :0:0:0:0:0 :0:0:0:0 :0:0:0 :0:0
  do
    case "$__compress_ipv6_ip" in
      *$__compress_ipv6_chain*)
        __compress_ipv6_ip="$(echo "$__compress_ipv6_ip" | sed -e "s/$__compress_ipv6_chain/::/" -e 's/:::/::/')"
        break
    esac
  done

  case "$__compress_ipv6_ip" in
    ::*) ;;
    :*) __compress_ipv6_ip="${__compress_ipv6_ip#:}"
  esac
  echo "$__compress_ipv6_ip"
}

Answer 2

I recently wanted a no-dependency solution that would be portable across shells and work on platforms such as openwrt. I came up with the following snippet:

# helper to convert hex to dec (portable version)
hex2dec(){
    [ "$1" != "" ] && printf "%d" "$(( 0x$1 ))"
}

# expand an ipv6 address
expand_ipv6() {
    ip=$1

    # prepend 0 if we start with :
    echo $ip | grep -qs "^:" && ip="0${ip}"

    # expand ::
    if echo $ip | grep -qs "::"; then
        colons=$(echo $ip | sed 's/[^:]//g')
        missing=$(echo ":::::::::" | sed "s/$colons//")
        expanded=$(echo $missing | sed 's/:/:0/g')
        ip=$(echo $ip | sed "s/::/$expanded/")
    fi

    blocks=$(echo $ip | grep -o "[0-9a-f]\+")
    set $blocks

    printf "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x\n" \
        $(hex2dec $1) \
        $(hex2dec $2) \
        $(hex2dec $3) \
        $(hex2dec $4) \
        $(hex2dec $5) \
        $(hex2dec $6) \
        $(hex2dec $7) \
        $(hex2dec $8)
}

I also have this function to compress

# returns a compressed ipv6 address under the form recommended by RFC5952
compress_ipv6() {
    ip=$1

    blocks=$(echo $ip | grep -o "[0-9a-f]\+")
    set $blocks

    # compress leading zeros
    ip=$(printf "%x:%x:%x:%x:%x:%x:%x:%x\n" \
        $(hex2dec $1) \
        $(hex2dec $2) \
        $(hex2dec $3) \
        $(hex2dec $4) \
        $(hex2dec $5) \
        $(hex2dec $6) \
        $(hex2dec $7) \
        $(hex2dec $8)
    )

    # prepend : for easier matching
    ip=:$ip

    # :: must compress the longest chain
    for pattern in :0:0:0:0:0:0:0:0 \
            :0:0:0:0:0:0:0 \
            :0:0:0:0:0:0 \
            :0:0:0:0:0 \
            :0:0:0:0 \
            :0:0:0 \
            :0:0; do
        if echo $ip | grep -qs $pattern; then
            ip=$(echo $ip | sed "s/$pattern/::/")
            # if the substitution occured before the end, we have :::
            ip=$(echo $ip | sed 's/:::/::/')
            break # only one substitution
        fi
    done

    # remove prepending : if necessary
    echo $ip | grep -qs "^:[^:]" && ip=$(echo $ip | sed 's/://')

    echo $ip
}

You can combine them to test if a given input is an ipv6

# a valid ipv6 is either the expanded form or the compressed one
is_ipv6(){
    expanded="$(expand_ipv6 $1)"
    [ "$1" = "$expanded" ] && return 0
    compressed="$(compress_ipv6 $expanded)"
    [ "$1" = "$compressed" ] && return 0
    return 1
}

I hope this helps! Those snippets are taken from https://github.com/chmduquesne/wg-ip. If you spot any bug, please contribute!

Answer 3

Here's my simple solution that works in Bash and BusyBox sh.

The ip variable is the input, and the expanded_ip variable is the output.

# Add enough leading zeros to each field in the address so that each field is
# exactly 4 hexadecimal digits long.
expanded_ip="$(
    printf '%s\n' "$ip" \
    | sed -E 's/[0-9a-f]+/000&/gi; s/0+([0-9a-f]{4})/\1/gi'
)"

# If the address contains `::`, expand it into zeros.
if [[ "$expanded_ip" == *"::"* ]]; then
    # The part of the address before the `::`.
    ip_start="${expanded_ip/::*/}"
    # The part of the address after the `::`.
    ip_end="${expanded_ip/*::/}"

    # Start with an IPv6 address of all zeros.
    zeros="0000:0000:0000:0000:0000:0000:0000:0000"

    # Slice out just the zeros that should replace the `::`.
    zeros="${zeros:"${#ip_start}"}"
    if [[ "$ip_end" ]]; then
        zeros="${zeros:0:-"${#ip_end}"}"
    fi

    # Join together the part before the `::`, the zeros replacing the `::`,
    # and the part after the `::` to get the final expanded IPv6 address.
    expanded_ip="$ip_start$zeros$ip_end"
fi

As a bonus, since this was my use case, here's another command to convert the expanded IP into a PTR record address:

# Remove the colons, separate each hexadecimal digit with a `.`, reverse it, and
# append `.ip6.arpa` to get the final PTR record name.
record_name="$(
    printf '%s\n' "$expanded_ip" \
    | sed -E 's/://g; s/./.&/g; s/^\.//' \
    | rev
).ip6.arpa"

Answer 4

Awk verbose version:

{
    OFS=":"
    FS=":"
    j=NF;
    for(i=8;i>=1;i--) {
        # If the i field is empty and there are still missing fields (j<i)
        if (!$(i) && j<i) {
            # if j has a value, copy to i and clean j
            if($(j)) {
                $(i)=$(j);
                $(j)="";
                j--
            # if not, fill i it with zero
            } else {
                $(i)=0
            }
        }
        # Now just add the leading 0
        $(i)=gensub(" ","0","g",sprintf("%4s",$(i)))
    }
    print
}

And the oneliner compact one:

awk -v OFS=":" -F: '{j=NF;for(i=8;i>=1;i--) {if (!$(i) && j<i) {if($(j)) {$(i)=$(j);$(j)="";j--} else $(i)=0} $(i)=gensub(" ","0","g",sprintf("%4s",$(i))) } print}'

You can use it reading from input:

$ echo ::1 | awk -v OFS=":" -F: '{j=NF;for(i=8;i>=1;i--) {if (!$(i) && j<i) {if($(j)) {$(i)=$(j);$(j)="";j--} else $(i)=0} $(i)=gensub(" ","0","g",sprintf("%4s",$(i))) } print}'
0000:0000:0000:0000:0000:0000:0000:0001
$ echo 2001::1 | awk -v OFS=":" -F: '{j=NF;for(i=8;i>=1;i--) {if (!$(i) && j<i) {if($(j)) {$(i)=$(j);$(j)="";j--} else $(i)=0} $(i)=gensub(" ","0","g",sprintf("%4s",$(i))) } print}'
2001:0000:0000:0000:0000:0000:0000:0001
$ echo 2001:1:1:1:1:1:1:1 | awk -v OFS=":" -F: '{j=NF;for(i=8;i>=1;i--) {if (!$(i) && j<i) {if($(j)) {$(i)=$(j);$(j)="";j--} else $(i)=0} $(i)=gensub(" ","0","g",sprintf("%4s",$(i)))
 } print}'
2001:0001:0001:0001:0001:0001:0001:0001
$ echo 2001:1:1::1:1:1:1 | awk -v OFS=":" -F: '{j=NF;for(i=8;i>=1;i--) {if (!$(i) && j<i) {if($(j)) {$(i)=$(j);$(j)="";j--} else $(i)=0} $(i)=gensub(" ","0","g",sprintf("%4s",$(i))) 
} print}'
2001:0001:0001:0000:0001:0001:0001:0001
$ echo 2001:1:1:1:1:1:1:1 | awk -v OFS=":" -F: '{j=NF;for(i=8;i>=1;i--) {if (!$(i) && j<i) {if($(j)) {$(i)=$(j);$(j)="";j--} else $(i)=0} $(i)=gensub(" ","0","g",sprintf("%4s",$(i)))
 } print}'
2001:0001:0001:0001:0001:0001:0001:0001

Do not expect it to work when it is not a valid IPv6 address. There is no validation as IPv6 with more than 8 fields or the use of :: twice.

Answer 5

Using awk, you can do this:

$ echo 2001::1 | awk '{if(NF<8){inner = "0"; for(missing = (8 - NF);missing>0;--missing){inner = inner ":0"}; if($2 == ""){$2 = inner} else if($3 == ""){$3 = inner} else if($4 == ""){$4 = inner} else if($5 == ""){$5 = inner} else if($6 == ""){$6 = inner} else if($7 == ""){$7 = inner}}; print $0}' FS=":" OFS=":" | awk '{for(i=1;i<9;++i){len = length($(i)); if(len < 1){$(i) = "0000"} else if(len < 2){$(i) = "000" $(i)} else if(len < 3){$(i) = "00" $(i)} else if(len < 4){$(i) = "0" $(i)} }; print $0}' FS=":" OFS=":"
$ 2001:0000:0000:0000:0000:0000:0000:0001

The first call to awk add missing zeros and colons between a "::"; The second call to awk add missing 0 to each group.

To trim the colons, just replace the last OFS=":" by OFS="".

Answer 6

__rfc5952_expand () {
    read addr mask < <(IFS=/; echo $1)
    quads=$(grep -oE "[a-fA-F0-9]{1,4}" <<< ${addr/\/*} | wc -l)
    #[ "${addr:${#addr}-1}" == ":" ] && { addr="${addr}0000"; (( quads++ )); }
    grep -qs ":$" <<< $addr && { addr="${addr}0000"; (( quads++ )); }
    grep -qs "^:" <<< $addr && { addr="0000${addr}"; (( quads++ )); }
    [ $quads -lt 8 ] && addr=${addr/::/:$(for (( i=1; i<=$(( 8 - quads )) ; i++ )); do printf "0000:"; done)}
    #addr=$(
    #for quad in $(IFS=:; echo ${addr}); do
    #    [ "${#quad}" -lt 4 ] && for (( i=${#quad}; i<4 ; i++ )); do quad=0${quad}; done
    #    printf "${delim}${quad}"; delim=":";
    # Or so if you need result without colon, as asked in first post
    #   printf "${quad}";
    #done)
    addr=$(for quad in $(IFS=:; echo ${addr}); do printf "${delim}%04x" "0x${quad}"; delim=":"; done)
    #addr=$(for quad in $(IFS=:; echo ${addr}); do printf "%04x" "0x${quad}"; done)  
    [ ! -z $mask ] && echo $addr/$mask || echo $addr
}

for ip in 2a01:4f8:211:9e::/64 ::1/128; do __rfc5952_expand $ip; done

2a01:04f8:0211:009e:0000:0000:0000:0000/64
0000:0000:0000:0000:0000:0000:0000:0001/128

__rfc5952_compact () {
    read addr mask < <(IFS=/; echo $1)
    addr=$(for quad in $(IFS=:; echo ${addr}); do printf "${delim}%x" "0x${quad}"; delim=":"; done)
    for zeros in $(grep -oE "((^|:)0)+:?" <<< $addr | sort -r | head -1); do addr=${addr/$zeros/::}; done
    [ ! -z $mask ] && echo $addr/$mask || echo $addr
}

for ip in 2a01:04f8:0211:009e:00:0001:0000:0000/64 0000:0000:0000:0000:0000:0000:0000:0001/128; do __rfc5952_compact $ip; done

2a01:4f8:211:9e:0:1::/64
::1/128

Answer 7

Using sipcalc might do it. It gives more information than you need, but a bit of grep and cut can solve that :-)

$ EXPANDED=`sipcalc 2001::1 | fgrep Expanded | cut -d '-' -f 2`
$ echo $EXPAND
2001:0000:0000:0000:0000:0000:0000:0001

For reference, this is the full output of sipcalc:

$ sipcalc 2001::1
-[ipv6 : 2001::1] - 0

[IPV6 INFO]
Expanded Address        - 2001:0000:0000:0000:0000:0000:0000:0001
Compressed address      - 2001::1
Subnet prefix (masked)  - 2001:0:0:0:0:0:0:1/128
Address ID (masked)     - 0:0:0:0:0:0:0:0/128
Prefix address          - ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff
Prefix length           - 128
Address type            - Aggregatable Global Unicast Addresses
Network range           - 2001:0000:0000:0000:0000:0000:0000:0001 -
                          2001:0000:0000:0000:0000:0000:0000:0001

Answer 8

is this ok for you?

kent$  echo "2a01:488:66:1000:523:f116:0:1"|awk -F: '{for(i=1;i<=NF;i++)x=x""sprintf ("%4s", $i);gsub(/ /,"0",x);print x}'
2a010488006610000523f11600000001