How can I break a variable definition across multiple lines in a Makefile without spaces?

Question

Consider the following Makefile

CP = .:${HADOOP_HOME}/share/hadoop/common/lib/hadoop-auth-2.2.0.jar:\
${HADOOP_HOME}/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar:\
${HADOOP_HOME}/share/hadoop/common/hadoop-common-2.2.0.jar:\
${HADOOP_HOME}/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar:\
${HADOOP_HOME}/share/hadoop/mapreduce/lib/hadoop-annotations-2.2.0.jar\

all:
    echo $(CP)

The output of running make is

.:/home/hduser/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/hadoop-auth-2.2.0.jar: /home/hduser/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar: /home/hduser/Hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar: /home/hduser/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar: /home/hduser/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.2.0.jar

Observe that there are spaces after each :.

Is there a way to define the variable CP with the line breaks, but without the extraneous space substituting every newline?

Answer 1

The simplest solution is to use $\<newline> to split the line. Using the following fake paths for brevity:

CP = one$\
     two$\
     three$\
     four

all:
    echo $(CP)

The output will be "onetwothreefour" with no spaces. This is because GNU Make will replace backslash-newline-whitespace with a single space, making the assignment to CP be equivalent to:

CP = one$ two$ three$ four

From https://www.gnu.org/software/make/manual/html_node/Reference.html#Reference: "A dollar sign followed by a character other than a dollar sign, open-parenthesis or open-brace treats that single character as the variable name." So the $<space> pairs are expansions of the variable whose name is a single space character. Since this variable is not defined by default, it will expand to the empty string.

Note that the variable CP will still contain the $<space> pairs until it is expanded. Most of the time, this doesn't matter, but if your makefile depends on using $(value CP) to process the underlying (unexpanded) value, the above technique may provide surprising results.

Also, the recently released GNU Make 4.3 now explicitly documents this technique for splitting lines (https://www.gnu.org/software/make/manual/make.html#Splitting-Lines):

Splitting Without Adding Whitespace

If you need to split a line but do not want any whitespace added, you can utilize a subtle trick: replace your backslash/newline pairs with the three characters dollar sign/backslash/newline:

var := one$\
       word

After make removes the backslash/newline and condenses the following line into a single space, this is equivalent to:

var := one$ word

Then make will perform variable expansion. The variable reference ‘$ ’ refers to a variable with the one-character name “ ” (space) which does not exist, and so expands to the empty string, giving a final assignment which is the equivalent of:

var := oneword

Another technique is to launder the value of CP. This requires GNU Make 3.80 or above because it relies on $(value) and $(eval). It makes use of a few variables and functions for the laundering process.

First, define the variable empty to be the empty string and the variables space and newline to contain a literal space and newline, respectively:

empty :=

space := $(empty) $(empty)

define newline


endef

Next, use $(eval) to programmatically create a recursively expanded variable with a given value:

# Define recursively expanded variable $1 with value $2.
defineVar = $(eval define $1$(newline)$2$(newline)endef)

And define the function resubst to repeated substitute one string with another:

# Replace $1 with $2 in $3 until no more changes are made.
resubst = $\
  $(if $(findstring $1,$3),$\
       $(call resubst,$\
              $1,$\
              $2,$\
              $(subst $1,$2,$3)),$\
       $3)

These are sufficient to define functions two-dimensionally with arbitrary newlines and indentation. The general method comprises three steps:

1. Prepend a newline to the function body;
2. Repeatedly replace newline-space pairs with newlines;
3. Remove all newlines.

The function def removes the newline/whitespace line continuations from the $(value) of a recursively expanded variable defined via define/endef:

# $1 - name of function to redefine as a normalized single-line function.
def = $\
  $(call defineVar,$\
         $1,$\
         $(subst $(newline),$\
                 $(empty),$\
                 $(call resubst,$\
                        $(newline)$(space),$\
                        $(newline),$\
                        $(newline)$(value $1))))

Now def may be used to post-process a recursively expanded variable. For example:

define CP
  one
  two
  three
  four
endef
$(call def,CP)

Now, $(value CP) will return the desired onetwothreefour.

The above is a distillation from my article "GNU Make line continuations": http://drmikehenry.com/gnu-make-line-continuations/

Answer 2

I made it similar to @goodguy5

CP:=${HADOOP_HOME}/share/hadoop/common/lib/hadoop-auth-2.2.0.jar
CP+=$(CP)${HADOOP_HOME}/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar
CP+=$(CP)${HADOOP_HOME}/share/hadoop/common/hadoop-common-2.2.0.jar
CP+=$(CP)${HADOOP_HOME}/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar
CP+=$(CP)${HADOOP_HOME}/share/hadoop/mapreduce/lib/hadoop-annotations-2.2.0.jar

Answer 3

It’s impossible to prevent backslash-newline from becoming a space, and it’s clumsy and error-prone to try to remove the spaces afterwards (what if there are supposed to be spaces?), but you can remove each as it’s produced. This has the significant advantage of working anywhere, even inside function calls. The trick is to embed the space produced in an expression that expands to nothing.

$(call foo) with empty/undefined foo would work, but we can do better: variable names can contain spaces in (GNU) Make. It’s hard to assign to them, but we don’t want to anyway. So then we can shorten it to $(a b) or even $(a ); a backslash-newline will be turned into a space before the lookup. But even a single space works:

foo=bar$(\
)baz

Finally, the parentheses may be omitted for a single-character variable name:

foo=bar$\
baz

…which finally looks like we are (fully) escaping the newline rather than using it somehow. So long as no one assigns to “ ” (which is even crazier than using it!), anyway.

Answer 4

Sorry to necro this a little bit, but I think this will give you what you're looking for:

CP = .:${HADOOP_HOME}/share/hadoop/common/lib/hadoop-auth-2.2.0.jar:
CP:=$(CP)${HADOOP_HOME}/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar:
CP:=$(CP)${HADOOP_HOME}/share/hadoop/common/hadoop-common-2.2.0.jar:
CP:=$(CP)${HADOOP_HOME}/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar:
CP:=$(CP)${HADOOP_HOME}/share/hadoop/mapreduce/lib/hadoop-annotations-2.2.0.jar

This should append the next line onto the value of CP, somewhat different than "+=", which adds a space.

Answer 5

Not really; the backslash-newline combination is defined to produce a space. However, if you are using GNU Make, you could simply $(subst : ,:,$(CP)).