CODEWITHSUNDEEP
hadoopnamenode
Bargitta
Bargitta

Reputation: 2406

hadoop 3.1.2 ./start-all.sh error, syntax error near unexpected token `<'

I'm running hadoop 3.1.2 on mac, and when executing ./start-all.sh, I got error saying

Starting namenodes on [localhost]
/usr/local/Cellar/hadoop/3.1.2/libexec/bin/../libexec/hadoop-functions.sh: line 398: syntax error near unexpected token `<'
/usr/local/Cellar/hadoop/3.1.2/libexec/bin/../libexec/hadoop-functions.sh: line 398: `  done < <(for text in "${input[@]}"; do'
/usr/local/Cellar/hadoop/3.1.2/libexec/bin/../libexec/hadoop-config.sh: line 70: hadoop_deprecate_envvar: command not found
/usr/local/Cellar/hadoop/3.1.2/libexec/bin/../libexec/hadoop-config.sh: line 87: hadoop_bootstrap: command not found
/usr/local/Cellar/hadoop/3.1.2/libexec/bin/../libexec/hadoop-config.sh: line 104: hadoop_parse_args: command not found
/usr/local/Cellar/hadoop/3.1.2/libexec/bin/../libexec/hadoop-config.sh: line 105: shift: : numeric argument required
/usr/local/Cellar/hadoop/3.1.2/libexec/bin/../libexec/hadoop-config.sh: line 244: hadoop_need_reexec: command not found
/usr/local/Cellar/hadoop/3.1.2/libexec/bin/../libexec/hadoop-config.sh: line 252: hadoop_verify_user_perm: command not found

I open the hadoop-functions.sh and find below info in line 398:

done < <(for text in "${input[@]}"; do
    echo "${text}"
  done | sort)

any idea how to fix that?

Upvotes: 1

Views: 611

Answers (1)

tk421
tk421

Reputation: 5957

You found a bug although it's not likely to get resolved soon. MacOS runs a bash 3.x but this syntax works on most modern Linuxes which run bash with a version 4.x.

According to the Bash Manual:Process Substitution <(cmd(s) gets treated as a file so you can interpret this as:

for text in "${input[@]}"; do
  echo "${text}"
done | sort > /tmp/results

while read -r line; do
  tmpa[${counter}]=${line}
  ((counter=counter+1))
  IFS='@' read -ra brup <<< "${line}"
  option="${brup[0]}"
  if [[ ${#option} -gt ${maxoptsize} ]]; then
    maxoptsize=${#option}
  fi
done < /tmp/results

Your options are:

  • install a later version of bash
  • rewrite the 2 instances of < <( in hadoop-functions.sh which are the only instances you'd hit of this construct.

References

Upvotes: 2

Related Questions