CODEWITHSUNDEEP
stringbashreplace
kev
kev

Reputation: 2881

Edit a text file line by line in bash

I have a requirements.txt file that has the following lines-

PyMySQL==0.9.3
botocore==1.12.196
boto3==1.9.188

I need to edit this file in-place using bash to remove all lines containing boto3 and botocore. So far, I've come up with-

while read a ; do echo ${a//boto3==1.9.188/} ; done < requirements.txt > requirements.txt.t ; mv requirements.txt{.t,}

.. which successfully removes the line containing boto3==1.9.188. However, this version number could be anything, say 1.10.122.

How do I generalize the script above to remove all lines containing boto3 and botocore strings? Or is there a better way to do this? Thanks.

Upvotes: 0

Views: 682

Answers (4)

WaltDe
WaltDe

Reputation: 1832

Use perl:

I like perl in this case because

  1. You don't need to worry about file manipulation
  2. You can easily make a backup of the file as well.

Command:

$ perl -ni -e '/\b(boto3|botocore)\b/ || print;' requirements.txt

Breakdown of options

  • -n iterate over lines in the file
  • -i edit the file in place.
  • -i.orig edit the file in place and make a backup with the extension '.orig'
  • -e execute the perl command

/\b(boto3|botocore)\b/ || print;

This reads if the line doesn't match a word with boto3 or botocore then print. The \b indicates a word boundary.

Example:

$ cat requirements.txt
PyMySQL==0.9.3
botocore==1.12.196
boto3==1.9.188
$ perl -ni -e '/\b(boto3|botocore)\b/ || print;' requirements.txt
$ cat requirements.txt
PyMySQL==0.9.3
$

Example with making backup of file called "requirements.txt.orig"

$ cat requirements.txt
PyMySQL==0.9.3
botocore==1.12.196
boto3==1.9.188
$ perl -ni.orig -e '/\b(boto3|botocore)\b/ || print;' requirements.txt
$ cat requirements.txt
PyMySQL==0.9.3
$ cat requirements.txt.orig
PyMySQL==0.9.3
botocore==1.12.196
boto3==1.9.188

Upvotes: 0

chepner
chepner

Reputation: 530853

Use ed:

printf 'g/botocore/d\ng/boto3/d\nwq\n' | ed requirements.txt

ed works by reading a set of commands (terminated by newlines) from standard input and applying them to the file named by its argument. The commands should look familiar if you are familiar with sed (indeed, sed is a stream editor based on ed).

  • g/botocore/d selects each line matching the regular expression botocore and applies the d (delete) command to each.
  • g/boto3/d does the same for lines matching boto3.
  • wq saves the changes and quotes.

Upvotes: 1

cdhowie
cdhowie

Reputation: 168958

grep -vE '^(botocore|boto3)\W' requirements.txt > requirements.txt.new && \
    mv requirements.txt.new requirements.txt

Explanation:

  • -v tells grep to output lines that don't match the pattern, and skip lines that do match.
  • -E tells grep to allow extended regular expressions, which we use for alternation.
  • ^ anchors the pattern to the beginning of the line so that it won't match foobotocore==1.2.3.
  • The (x|y) construct matches either x or y. (To add a third package, just add another | to create (x|y|z).)
  • \W in the pattern matches a "non-word character" so that the pattern won't match botocorefoo==1.2.3.
  • && only invokes the mv command if grep was successful and matched at least one line (prevent clobbering the whole file).

Upvotes: 2

Davide
Davide

Reputation: 2124

Use awk

awk '!/(botocore|boto3)/' requirements.txt > requirements.txt.t && mv requirements.txt.t requirements.txt

Upvotes: 2

Related Questions