Use of associative array with a variable name under bash 4.1

Question

I trying to parse multiples files like this under bash-4.1

$cat hostname_abc.txt
host_type type_foo
SoftA version123
SoftB version456

to obtain an output where you can see how many times a version of Soft[A,B] is used, grouped by host type :

$./list_versions.sh
[type_foo] 11 times
    SoftA:
        [version123] 1 times
        [version444] 5 times
        [version567] 5 times
    SoftB:
        [version456] 9 times
        [version777] 2 times
[type_bar] 6 times
    SoftA:
        [version444] 6 times
    SoftB:
        [version111] 4 times
        [version777] 2 times

I don't know in advance the list of host_type and the versions. So I tried to save in an associative array the count of each host_type and create dynamically the names of the associatives arrays which stored the count of each version of Soft[A,B] per host_type based base on a template host_type_Soft[A,B]

I tried many times with different variations of syntax and indirections so I remade below a more readable script that follow my aim :

#!/usr/bin/env bash

# ----- generated test conditions -----
echo -e "host_type typeA\nSoftA v2\nSoftB v1" > hostname_1.txt
echo -e "host_type typeB\nSoftA v1\nSoftB v1" > hostname_2.txt
echo -e "host_type typeB\nSoftA v1\nSoftB v0" > hostname_3.txt
echo -e "host_type typeA\nSoftA v0\nSoftB v0" > hostname_4.txt
echo -e "host_type typeA\nSoftA v3\nSoftB v2" > hostname_5.txt
echo -e "host_type typeB\nSoftA v3\nSoftB v1" > hostname_6.txt
echo -e "host_type typeB\nSoftA v2\nSoftB v2" > hostname_7.txt
echo -e "host_type typeA\nSoftA v1\nSoftB v2" > hostname_8.txt
echo -e "host_type typeC\nSoftA v0\nSoftB v4" > hostname_9.txt
list_hostname() {
    for i in {1..9}; do
        echo "hostname_${i}.txt"
    done
}

declare -A list_host_type

while read f; do

    #parse the hostname files
    while read l; do
        [[ $l = *"host_type"* ]] && host_type="$( echo $l | cut -d' ' -f2)"
        [[ $l = *"SoftA"* ]] && versionA="$( echo $l | cut -d' ' -f2)"
        [[ $l = *"SoftB"* ]] && versionB="$( echo $l | cut -d' ' -f2)"
    done < <( cat "$f" )

    #count the number of hosts by host_type
    [[ ${list_host_type[$host_type]} ]] && ((list_host_type[$host_type]++)) || list_host_type[$host_type]='1'

    #create associative arrays with a name only know at runtime
    declare -A "${host_type}_SoftA"
    declare -A "${host_type}_SoftB"

    #count the number of host for the couple host_type and Soft[A,B], stored on the dynamically named assiociative array
    [[ ${${host_type}_SoftA[$versionA]} ]] && ((${host_type}_SoftA[$versionA]++)) || ${host_type}_SoftA[$versionA]='1'
    [[ ${${host_type}_SoftB[$versionB]} ]] && ((${host_type}_SoftB[$versionB]++)) || ${host_type}_SoftB[$versionB]='1'
done < <( list_hostname )

#print a non pretty-formated output
echo '==== result ====='
for m in "${!list_host_type[@]}"; do
    echo "host type: $m  count: ${list_model[$m]}"
    for versionA in "${!${m}_softA[@]}"; do
        echo "    SoftA  version: $versionA  count: ${${m}_SoftA[$versionA]}"
    done
    for versionB in "${!${m}_softB[@]}"; do
        echo "    SoftB  version: $versionB  count: ${${m}_SoftB[$versionB]}"
    done
done

I know they are others methods to achieve my goal but I want to know if I can use associative this way with bash-4.1.

Answer 1

I don't think you can use dynamic variable names with arrays in Bash. (I tried a few things but couldn't figure out the syntax.) Even if possible, I think it would be extremely difficult to understand.

---

A possible workaround could be using a single associative array, with "composite keys". That is, for example use a comma separated value of host type, soft and version:

while read f; do
    line=0
    while read col1 col2; do
        if [[ $line = 0 ]]; then
            host_type=$col2
        else
            soft=$col1
            version=$col2
            index=$host_type,$soft,$version
            ((list_host_type[$index]++))
        fi
        ((line++))
    done < <( cat "$f" )
done < <( list_hostname )

for m in "${!list_host_type[@]}"; do
    echo $m = ${list_host_type[$m]}
done

For your sample data this would produce:

typeA,SoftA,v2 = 1
typeA,SoftA,v3 = 1
typeA,SoftA,v0 = 1
typeA,SoftA,v1 = 1
typeB,SoftA,v3 = 1
typeB,SoftA,v2 = 1
typeB,SoftA,v1 = 2
typeA,SoftB,v2 = 2
typeA,SoftB,v1 = 1
typeA,SoftB,v0 = 1
typeC,SoftB,v4 = 1
typeB,SoftB,v2 = 1
typeB,SoftB,v0 = 1
typeB,SoftB,v1 = 2
typeC,SoftA,v0 = 1

And then work with this associative array to compute the statistics you need. Here's a rough example implementation:

get_host_types() {
    local names=(${!list_host_type[@]})
    printf "%s\n" "${names[@]%%,*}" | sort -u
}

get_soft() {
    local host_type=$1
    local names=(${!list_host_type[@]})
    for name in "${names[@]}"; do
        [[ ${name%%,*} = $host_type ]] && echo $name
    done | cut -d, -f2 | sort -u
}

get_versions() {
    local prefix=$1
    local names=(${!list_host_type[@]})
    for name in "${names[@]}"; do
        [[ ${name%,*} = $prefix ]] && echo $name
    done | cut -d, -f3 | sort -u
}

indent="    "
for host_type in $(get_host_types); do
    echo "[$host_type]"
    for soft in $(get_soft $host_type); do
        echo "$indent$soft:"
        for version in $(get_versions $host_type,$soft); do
            index=$host_type,$soft,$version
            echo "$indent$indent[$version] ${list_host_type[$index]} times"
        done
    done
done

Producing as output:

[typeA]
    SoftA:
        [v0] 1 times
        [v1] 1 times
        [v2] 1 times
        [v3] 1 times
    SoftB:
        [v0] 1 times
        [v1] 1 times
        [v2] 2 times
[typeB]
    SoftA:
        [v1] 2 times
        [v2] 1 times
        [v3] 1 times
    SoftB:
        [v0] 1 times
        [v1] 2 times
        [v2] 1 times
[typeC]
    SoftA:
        [v0] 1 times
    SoftB:
        [v4] 1 times

All in all, it would be better to implement this using a proper programming language.