CODEWITHSUNDEEP
variablesawknewline
xine78
xine78

Reputation: 35

How to prevent new line when using awk

I have seen several variations of this question, but none of the answers are helping for my particular scenario.

I am trying to load some files, adding a column for filename. This works fine only if I put the filename as the first column. If I put the filename column at the end (where I want it) it creates a new line between $0 and the rest of the print that I am unable to stop.

for f in "${FILE_LIST[@]}"
do
awk '{ print FILENAME,"\t",$0 } ' ${DEST_DIR_FILES}/$f > tmp ## this one works
awk '{ print $0,"\t",FILENAME } ' ${DEST_DIR_FILES}/$f > tmp ## this one does not work
mv tmp ${DEST_DIR_FILES}/$f
done > output

Example data:

-- I'm starting with this:

A       B       C
aaaa    bbbb    cccc
1111    2222    3333

-- I want this (new column with filename):

A       B       C       FILENAME
aaaa    bbbb    cccc    FILENAME
1111    2222    3333    FILENAME

-- I'm getting this (\t and filename on new line):

A       B       C
        FILENAME
aaaa    bbbb    cccc
        FILENAME
1111    2222    3333
        FILENAME

Bonus question I'm using a variable to pass the filename, but it is putting the whole path. What is the best way to only print the filename (without path) ~OR~ strip out the file path using a variable that holds the path?

Upvotes: 1

Views: 1827

Answers (1)

Chris Seymour
Chris Seymour

Reputation: 85775

It's almost certainly a line endings issues as your awk script a syntactically correct. I suspect your files in "${FILE_LIST[@]}" came from a Windows box and have \r\n line endings. To confirm the line endings for a given file you can run the file command on each file i.e. file filename:

# create a test file
$ echo test > foo      

# use unix2dos to convert to Windows style line endings
$ unix2dos foo       
unix2dos: converting file foo to DOS format ...

# Use file to confirm line endings
$ file foo       
foo: ASCII text, with CRLF line terminators

# Convert back to Unix style line endings
$ dos2unix foo
dos2unix: converting file foo to Unix format ...

$ file foo
foo: ASCII text

To convert your files to Unix style line endings \n run the following command:

$ for "f" in "${FILE_LIST[@]}"; do; dos2unix "$f"; done

Explanation:

When the FILENAME is the first string on the line the carriage returns \r effectively does nothing as we are already at the start of the line. When we try to print FILENAME after any other characters we see the effects that we are brought to the start of the next line, the TAB is printed then the FILENAME.

Side note:

Awk has the variable OFS for setting the output field separator so: 

$ awk '{print $0,"\t",FILENAME}' file

Can be rewritten as:

$ awk '{print $0,FILENAME}' OFS='\t' file

Bonus Answer

The best way I.M.O to strip the path of file is to use the utility basename:

$ basename /tmp/foo
foo

Using command substitution:

$ awk '{print FILENAME}' $(basename /tmp/foo)
foo

Upvotes: 2

Related Questions