CODEWITHSUNDEEP
bash
Marko
Marko

Reputation: 397

Loop for deleting first line of Multiple Files using Bash Script

Just new to Bash scripting and programming in general. I would like to automate the deletion of the first line of multiple .data files in a directory. My script is as follows:

#!/bin/bash
for f in *.data ;
do tail -n +2 $f | echo "processing $f";
done

I get the echo message but when I cat the file nothing has changed. Any ideas?

Thanks in advance

Upvotes: 10

Views: 14112

Answers (4)

devnull
devnull

Reputation: 123478

I get the echo message but when I cat the file nothing has changed.

Because simply tailing wouldn't change the file.

You could use sed to modify the files in-place with the first line excluded. Saying

sed -i '1d' *.data

would delete the first line from all .data files.

EDIT: BSD sed (on OSX) would expect an argument to -i, so you can either specify an extension to backup older files, or to edit the files in-place, say:

sed -i '' '1d' *.data

Upvotes: 24

gniourf_gniourf
gniourf_gniourf

Reputation: 46823

ed is the standard editor:

shopt -s nullglob
for f in *.data; do
    echo "Processing file \`$f'"
    ed -s -- "$f" < <( printf '%s\n' "1d" "wq" )
done

The shopt -s nullglob is here just because you should always use this when using globs, especially in a script: it will make globs expand to nothing if there are no matches; you don't want to run commands with uncontrolled arguments.

Next, we loop on all your files, and use ed with the commands:

  • 1: go to first line
  • d: delete that line
  • wq: write and quit

Options for ed:

  • -s: tells ed to shut up! we don't want ed to print its junk on our screen.
  • --: end of options: this will make your script much more robust, in case a file name starts with a hypen: in this case, the hyphen will confuse ed trying to process it as an option. With --, ed knows that there are no more options after that and will happily process any files, even those starting with a hyphen.

Upvotes: 0

John Zwinck
John Zwinck

Reputation: 249153

I'd do it this way:

#!/usr/bin/env bash
set -eu
for f in *.data; do
  echo "processing $f"
  tail -n +2 "$f" | sponge "$f"
done

If you don't have sponge you can get it in the moreutils package.

The quotes around the filename are important--they will make it work with filenames containing spaces. And the env thing at the top is so that people can set which Bash interpreter they want to use via their PATH, in case someone has a non-default one. The set -eu makes Bash exit if an error occurs, which is usually safer.

Upvotes: 0

pfnuesel
pfnuesel

Reputation: 15310

You are not changing the file itself. By using tail you simply read the file and print parts of it to stdout (the terminal), you have to redirect that output to a temporary file and then overwrite the original file with the temporary one.

#!/usr/bin/env bash
for f in *.data; do
    tail -n +2 "$f" > "${f}".tmp && mv "${f}".tmp "$f"
    echo "Processing $f"
done

Moreover it's not clear what you'd like to achieve with the echo command. Why do you use a pipe (|) there?

will give you an easier way to achieve this. See devnull's answer.

Upvotes: 4

Related Questions