Arrays of Hosts
Problem statement
Load a list of hosts from a file into a bash array, count and extend it, look up each host's role in an associative array, and filter it with a pattern. Scripts that act on many servers, services or regions keep them in arrays, and doing it right keeps items with spaces intact.
hosts.txt
web-1web-2db-1cache-1- Read the file into an array and print how many hosts there are, the first and the last.
- Add
lb-1to the array. - Print each host with its role from an associative array, sorted, with
unknownfor a host that has no role. - Print only the
web-*hosts. - Show that
"${a[@]}"keeps an item with a space as one item, and that${a[*]}joins all items into one string.
Expected output:
== loaded ==count: 4, first: web-1, last: cache-1after adding lb-1: 5 hosts== host and role ==cache-1 cachedb-1 databaselb-1 unknownweb-1 webweb-2 web== only web hosts ==web-1web-2== quoting arrays ==[my server][db]unquoted gives:[my][server][db]joined with commas: web-1,web-2,db-1,cache-1,lb-1Hints
mapfile -t hosts < hosts.txt reads each line into one array item. ${#hosts[@]} is the count, ${hosts[0]} the first item and ${hosts[-1]} the last.Approach
Optimal: Indexed and associative arrays
Covers: a=(x y z), "${a[@]}", ${#a[@]}, ${a[0]}, ${a[-1]}, a+=(x), mapfile -t, declare -A, ${map[$key]:-default}, ${!map[@]}, "${a[@]}" vs ${a[*]}, filtering with [[ == pattern ]].
Strict mode, used on every page in this section. The second line, set -euo pipefail, makes bash stop on mistakes instead of carrying on:
| Option | Means |
|---|---|
-e |
exit as soon as a command fails (with some exceptions, see Handle Command Failures) |
-u |
treat an unset variable as an error, instead of silently using empty text |
-o pipefail |
a pipeline fails if any command in it fails, not just the last one |
Put it right after the shebang line #!/usr/bin/env bash in every script you write.
An array is a numbered list. One variable holds many items, each with a position starting at 0:
| Form | Gives |
|---|---|
hosts=(web-1 web-2 db-1) |
create it |
"${hosts[@]}" |
every item, each as its own word |
${#hosts[@]} |
how many items |
${hosts[0]}, ${hosts[-1]} |
the first, the last (bash 4.3+) |
hosts+=(lb-1) |
add to the end |
mapfile -t hosts < file |
one item per line of the file (-t drops the newline) |
Always loop with "${hosts[@]}". With the quotes, each item stays exactly as it was stored, even with spaces. Without them, items are split at spaces like any unquoted variable. "${hosts[*]}" is different again: it joins all items into one single string, using the first character of IFS between them, which is handy for printing a list with commas.
An associative array maps keys to values. declare -A role creates one, and role[web-1]=web sets a value. Keys can be any text:
| Form | Gives |
|---|---|
${role[$h]} |
the value for key $h |
${role[$h]:-unknown} |
the value, or unknown if there is none |
${!role[@]} |
all the keys |
[[ -v role[$h] ]] |
true if the key exists |
Like awk arrays, the keys come out in no fixed order, so sort the output.
Filtering. There is no built-in filter, but a loop with a pattern test is short: for h in "${hosts[@]}"; do [[ $h == web-* ]] && echo "$h"; done. Under set -e, a failing && test inside a loop is fine, because it is part of a condition.
Walking through the code. The # Setup: line only writes hosts.txt, so skip past it.
mapfileloads 4 hosts; the count, first and last are printed.hosts+=(lb-1)makes it 5.- The role loop prints
host roleandsortorders it;lb-1has no role, so it printsunknown. - The filter loop prints the two web hosts.
printf '[%s]\n'prints each argument in brackets, which shows how many items each form produces.
Edge cases. mapfile and declare -A need bash 4 or newer; macOS still ships bash 3.2, so install a newer bash there or avoid them. Arrays cannot be exported to child processes; pass them as arguments, ./check.sh "${hosts[@]}", instead. Blank lines in the file become empty items, so clean the file or skip -z items.
#!/usr/bin/env bash
set -euo pipefail
# Setup: write the host list in a fresh temporary folder
cd "$(mktemp -d)"
printf '%s\n' web-1 web-2 db-1 cache-1 > hosts.txt
mapfile -t hosts < hosts.txt
echo "== loaded =="
echo "count: ${#hosts[@]}, first: ${hosts[0]}, last: ${hosts[-1]}"
hosts+=(lb-1)
echo "after adding lb-1: ${#hosts[@]} hosts"
declare -A role=([web-1]=web [web-2]=web [db-1]=database [cache-1]=cache)
echo "== host and role =="
for h in "${hosts[@]}"; do
echo "$h ${role[$h]:-unknown}"
done | sort
echo "== only web hosts =="
for h in "${hosts[@]}"; do
[[ $h == web-* ]] && echo "$h"
done
echo "== quoting arrays =="
names=("my server" db)
printf '[%s]\n' "${names[@]}"
echo "unquoted gives:"
printf '[%s]\n' ${names[@]}
echo "joined with commas: $(IFS=,; echo "${hosts[*]}")"Interview follow-ups
Run a check on every host and collect the failures in a second array.
Start with
failed=(), then for each host run the check in anif:if ! ping -c1 -W1 "$h" > /dev/null 2>&1; then failed+=("$h"); fi. After the loop,${#failed[@]}tells you how many failed, andprintf '%s\n' "${failed[@]}"lists them. Finish with(( ${#failed[@]} == 0 )) || exit 1so the script's exit code reflects the result. The Health-Check URLs page uses exactly this pattern.
Frequently asked questions
Quoted, "${a[@]}" gives each item as a separate word, exactly as stored, which is what you want for loops and for passing items to commands. "${a[*]}" gives all items as one word, joined with the first character of IFS (a space by default). Unquoted, both forms split every item at spaces, which breaks items like my server. Use "${a[@]}" almost always, and "${a[*]}" only to build a display string.
macOS ships bash 3.2 from 2007, because newer versions use a licence Apple does not ship. Associative arrays, mapfile, negative indexes and ${var^^} all need bash 4 or newer. Install a current bash with Homebrew and start scripts with #!/usr/bin/env bash, so the newer one in your PATH is used. Check with bash --version and echo "$BASH_VERSION" inside the script.
Pass the items as arguments: check_hosts "${hosts[@]}", and inside the function use "$@" as the list, or copy it with local list=("$@"). Bash 4.3+ also has name references, local -n arr=$1, which let a function work on an array by its name, including changing it. Arguments are simpler and work everywhere.