CODEWITHSUNDEEP
arraysbashloopsscriptingvariable-assignment
TheDailyToast
TheDailyToast

Reputation: 1363

Creating an array from a text file in Bash

A script takes a URL, parses it for the required fields, and redirects its output to be saved in a file, file.txt. The output is saved on a new line each time a field has been found.

file.txt

A Cat
A Dog
A Mouse 
etc...

I want to take file.txt and create an array from it in a new script, where every line gets to be its own string variable in the array. So far I have tried: 

#!/bin/bash

filename=file.txt
declare -a myArray
myArray=(`cat "$filename"`)

for (( i = 0 ; i < 9 ; i++))
do
  echo "Element [$i]: ${myArray[$i]}"
done

When I run this script, whitespace results in words getting split and instead of getting

Desired output

Element [0]: A Cat 
Element [1]: A Dog 
etc...

I end up getting this:

Actual output

Element [0]: A 
Element [1]: Cat 
Element [2]: A
Element [3]: Dog 
etc...

How can I adjust the loop below such that the entire string on each line will correspond one-to-one with each variable in the array?

Upvotes: 126

Views: 184047

Answers (7)

Chris Reid
Chris Reid

Reputation: 488

Make sure set the Internal File Separator (IFS) variable to $'\n' so that it does not put each word into a new array entry.

#!/bin/bash

# move all 2020 - 2022 movies to /backup/movies
# put list into file 1 line per dir

# dirs are  "movie name (year)/"
ls | egrep 202[0-2]  > 2020_movies.txt

OLDIFS=${IFS}
  
IFS=$'\n'    #fix separator

declare -a MOVIES  # array for dir names

MOVIES=( $( cat "${1}" ) )  // load into array 

for M in ${MOVIES[@]} ; do
        echo "[${M}]"
        if [ -d "${M}" ] ; then  # if dir name

                mv -v "$M" /backup/movies/
        fi

done

IFS=${OLDIFS}  # restore standard separators
               # not essential as IFS reverts when script ends

#END

Upvotes: 0

codeforester
codeforester

Reputation: 42999

mapfile and readarray (which are synonymous) are available in Bash version 4 and above. If you have an older version of Bash, you can use a loop to read the file into an array:

arr=()
while IFS= read -r line; do
  arr+=("$line")
done < file

In case the file has an incomplete (missing newline) last line, you could use this alternative:

arr=()
while IFS= read -r line || [[ "$line" ]]; do
  arr+=("$line")
done < file

Related:

Upvotes: 47

glenn jackman
glenn jackman

Reputation: 246807

Use the mapfile command:

mapfile -t myArray < file.txt

The error is using for -- the idiomatic way to loop over lines of a file is:

while IFS= read -r line; do echo ">>$line<<"; done < file.txt

See BashFAQ/005 for more details.

Upvotes: 156

dosentmatter
dosentmatter

Reputation: 1624

This answer says to use

mapfile -t myArray < file.txt

I made a shim for mapfile if you want to use mapfile on bash < 4.x for whatever reason. It uses the existing mapfile command if you are on bash >= 4.x

Currently, only options -d and -t work. But that should be enough for that command above. I've only tested on macOS. On macOS Sierra 10.12.6, the system bash is 3.2.57(1)-release. So the shim can come in handy. You can also just update your bash with homebrew, build bash yourself, etc.

It uses this technique to set variables up one call stack.

Upvotes: 0

Cameron Lowell Palmer
Cameron Lowell Palmer

Reputation: 22245

Use mapfile or read -a

Always check your code using shellcheck. It will often give you the correct answer. In this case SC2207 covers reading a file that either has space separated or newline separated values into an array.

Don't do this

array=( $(mycommand) )

Files with values separated by newlines

mapfile -t array < <(mycommand)

Files with values separated by spaces

IFS=" " read -r -a array <<< "$(mycommand)"

The shellcheck page will give you the rationale why this is considered best practice.

Upvotes: 11

Prateek Joshi
Prateek Joshi

Reputation: 4067

You can simply read each line from the file and assign it to an array.

#!/bin/bash
i=0
while read line 
do
        arr[$i]="$line"
        i=$((i+1))
done < file.txt

Upvotes: 5

Jahid
Jahid

Reputation: 22428

You can do this too:

oldIFS="$IFS"
IFS=$'\n' arr=($(<file))
IFS="$oldIFS"
echo "${arr[1]}" # It will print `A Dog`.

Note:

Filename expansion still occurs. For example, if there's a line with a literal * it will expand to all the files in current folder. So use it only if your file is free of this kind of scenario.

Upvotes: 11

Related Questions