How to keep associative array order?

Question

I try to iterate over an associative array in Bash.

It seems to be simple, but the loop doesn't follow the initial order of the array.

Here is a simple script to try:

#!/bin/bash

echo -e "Workspace\n----------";
lsb_release -a

echo -e "\nBash version\n----------";
echo -e $BASH_VERSION."\n";

declare -A groups;
groups["group1"]="123";
groups["group2"]="456";
groups["group3"]="789";
groups["group4"]="abc";
groups["group5"]="def";

echo -e "Result\n----------";
for i in "${!groups[@]}"
do
    echo "$i => ${groups[$i]}";
done

The output:

Workspace
----------
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.2 LTS
Release:    14.04
Codename:   trusty

Bash version
----------
4.3.11(1)-release.

Result
----------
group3 => 789
group2 => 456
group1 => 123
group5 => def
group4 => abc

Why I don't have group1, group2, etc.?

I don't want to have an alphanum order, I just want that the loop follow the initial declaration's order of the array...

Is there a way?

Answer 1

Another way to sort entries in your associative array is to keep a list of the groups as you add them as an entry in the associative array. Call this entry key "group_list". As you add each new group, append it to the group_list field, adding a blank space to separate subsequent additions. Here's one I did for an associative array I called master_array:

master_array["group_list"]+="${new_group}";

To sequence through the groups in the order you added them, sequence through the group_list field in a for loop, then you can access the group fields in the associative array. Here's a code snippet for one I wrote for master_array:

for group in ${master_array["group_list"]}; do
    echo "${group}";
    echo "${master_array[${group},destination_directory]}";
done

and here's the output from that code:

"linux"
"${HOME}/Backup/home4"
"data"
"${HOME}/Backup/home4/data"
"pictures"
"${HOME}/Backup/home4/pictures"
"pictures-archive"
"${HOME}/Backup/home4/pictures-archive"
"music"
"${HOME}/Backup/home4/music"

This is similar to the suggestion by Jonathan Leffler, but keeps the data with the associative array rather than needing to keep two separate disjoint arrays. As you can see, it's not in random order, nor in alphabetical order, but the order in which I added them to the array.

Also, if you have subgroups, you can create subgroup lists for each group, and sequence through those as well. That's the reason I did it this way, to alleviate the need for multiple arrays to access the associative array, and also to allow for expansion to new subgroups without having to modify the code.

Answer 2

Here are some ready-to use functions for that purpose. Based on @joe-hillenbrand's version this code is less likely to trigger namespace-collisions (due to declare -n) and provides a native array-key loop without subshell invocation and potential quoting issues (among others). It also correctly handles adding the same key twice (preserving the original order). Usage:

# avoid shellcheck warnings by declaring first.
declare myhash
hash_init myhash

hash_set myhash key1 val1
hash_set myhash key2 val2
hash_append myhash key1 ' appending works'

declare val
for key in "${myhash[@]}"; do
    hash_get myhash "$key" val
    echo "key »$key« val »$val«"
done

hash_key_exists myhash "key_not_set"; echo "key_is set: $?"
# throws error
hash_get myhash "key_not_set" val
echo "key_not_set »$val«"

The code:

# $1 map
hash_init(){
    declare -ga "$1"
    declare -gA "_hash_${1}_map"
}

# $1 map, $2 key, $3 value
# Setting an existing value does not change its order.
hash_set(){
    _hash_set "$@" false
}

# $1 map, $2 key, $3 value
hash_append(){
    _hash_set "$@" true
}

# $1 map, $2 key, $3: named parameter to be filled
hash_get(){
    local -n _hash_get_map
    local -n _hash_get_key
    _hash_get_map="_hash_${1}_map"
    _hash_get_key="$3"
    _hash_get_key="${_hash_get_map["$2"]?$2 is unset}"

}

# $1 map, $2 key
hash_key_exists(){
    local -n _hash_key_exists_map
    _hash_key_exists_map="_hash_${1}_map"
    [[ -n ${_hash_key_exists_map["$2"]+x} ]]
}

# Private member $1 map, $2 key, $3 value, $4 append=true
_hash_set(){
    local -n _hash_set_arr
    local -n _hash_set_map
    _hash_set_arr="$1"
    _hash_set_map="_hash_${1}_map"
    [[ -n ${_hash_set_map["$2"]+x} ]] || _hash_set_arr+=("$2")
    if [[ "$4" == true ]]; then
        if [[ -z ${_hash_set_map["$2"]+x} ]]; then
            echo "${FUNCNAME[1]}: cannot append »$3« to non-existing key »$2«" >&2
            return 1
        fi
        # append it
        _hash_set_map["$2"]+="$3"
    else
        # set it
        _hash_set_map["$2"]="$3"
    fi
}

Answer 3

Here are some ergonomic abstractions:

add() {
  local var=$1
  local key=$2
  local val=$3
  declare -ga "${var}_ORDER"
  declare -gA "${var}_MAP"
  local -n map=${var}_MAP
  local -n order=${var}_ORDER
  order+=("$key")
  map["$key"]=$val
}

add groups group1 123
add groups group2 456
add groups group3 789
add groups group4 abc
add groups group5 def

get() {
  local var=$1
  local key=$2
  local -n tmp=${var}_MAP
  echo "${tmp[$key]}"
}

get groups group2 # 456
get groups group5 # def

keys() {
  local var=$1
  local -n tmp=${var}_ORDER
  echo "${tmp[@]}"
}

keys groups # group1 group2 group3 group4 group5

values() {
  local var=$1
  local -n map=${var}_MAP
  local -n order=${var}_ORDER
  for k in "${order[@]}"; do
    echo -n "${map[$k]} "
  done
  echo
}

values groups # 123 456 789 abc def

for i in $(keys groups); do
  echo "$i => $(get groups "$i")"
done

# group1 => 123
# group2 => 456
# group3 => 789
# group4 => abc
# group5 => def

Answer 4

My approach is to create a sorted array of keys first:

keys=( $( echo ${!dict[@]} | tr ' ' $'\n' | sort ) )
for k in ${keys[@]}; do
    echo "$k=${dict[$k]}"
done

Answer 5

As already pointed out, there is no mistake. Associative arrays are stored in a 'hash' order. If you want ordering, you don't use associative arrays. Or, you use a non-associative array as well as an associative array.

Keep a second (non-associative) array that identifies the keys in the order that they're created. Then step through the second array, using its contents to key the first (associative) array when printing the data. Like this:

declare -A groups;      declare -a orders;
groups["group1"]="123"; orders+=( "group1" )
groups["group2"]="456"; orders+=( "group2" )
groups["group3"]="789"; orders+=( "group3" )
groups["group4"]="abc"; orders+=( "group4" )
groups["group5"]="def"; orders+=( "group5" )

# Convoluted option 1
for i in "${!orders[@]}"
do
    echo "${orders[$i]}: ${groups[${orders[$i]}]}"
done
echo

# Convoluted option 1 - 'explained'
for i in "${!orders[@]}"
do
    echo "$i: ${orders[$i]}: ${groups[${orders[$i]}]}"
done
echo

# Simpler option 2 - thanks, PesaThe
for i in "${orders[@]}"
do
    echo "$i: ${groups[$i]}"
done

The 'simpler option 2' was suggested by PesaThe in a comment, and should be used in preference to the 'convoluted option'.

Sample output:

group1: 123
group2: 456
group3: 789
group4: abc
group5: def

0: group1: 123
1: group2: 456
2: group3: 789
3: group4: abc
4: group5: def

group1: 123
group2: 456
group3: 789
group4: abc
group5: def

You probably don't want to have two statements per line like that, but it emphasizes the parallelism between the handling of the two arrays.

The semicolons after the assignments in the question are not really necessary (though they do no active harm, beyond leaving the reader wondering 'why?').