CODEWITHSUNDEEP
regexbashgrep
user3162968
user3162968

Reputation: 1046

Bash, grep between two lines with specified string

Example:

a43
test1
abc
cvb
bnm
test2
kfo

I need all lines between test1 and test2. Normal grep does not work in this case. Do you have any propositions?

Upvotes: 78

Views: 108610

Answers (9)

Arav Stark
Arav Stark

Reputation: 13

I wrote a script which even works for the lines that might have any special character.

#!/bin/bash
if  [ "$#" -ne 1 ]; then
    echo "Usage: $0 <logfile_name>"
    exit 1
fi

log_file="$1"

if [ ! -f "$log_file" ]; then
    echo "File not found: $log_file"
    exit 1
fi

# User Input
read -p "Enter 1st string: " str1
read -p "Enter 2nd string: " str2

# Escape special characters in str1 and str2 for sed
escaped_str1=$(sed 's/[^^]/[&]/g; s/\^/\\^/g' <<< "$str1")
escaped_str2=$(sed 's/[^^]/[&]/g; s/\^/\\^/g' <<< "$str2")

# Check if str1 and str2 exist in the file
if ! grep -q "$escaped_str1" "$log_file"; then
    echo -e "\e[31mString \"$str1\" not found in \"$log_file\"\e[0m"
    exit 1
fi

if ! grep -q "$escaped_str2" "$log_file"; then
    echo -e "\e[31mString \"$str2\" not found in \"$log_file\"\e[0m"
    exit 1
fi

# Using sed to extract lines between str1 and str2
if [[ "$escaped_str1" < "$escaped_str2" ]]; then
    echo -e "\e[31mError: Second string \"$str2\" appears before first string \"$str1\" in the file $log_file\e[0m"
    echo -e "\e[32mSwapping strings: \"$str1\" and \"$str2\"\e[0m"
    tmp="$escaped_str1"
    escaped_str1="$escaped_str2"
    escaped_str2="$tmp"
    
fi

sed -n "/$escaped_str1/,/$escaped_str2/p" "$log_file"

This script accepts a file as an argument and prompts the user to enter two lines of strings one by one and extracts the lines in between them.

Note: the input lines of strings are also included.

Upvotes: 0

Brad Parks
Brad Parks

Reputation: 71961

The following script wraps up this process. More details in this similar StackOverflow post

get_text.sh

#!/usr/bin/env bash
function show_help()
{
  ME=$(basename "$0")
  IT=$(cat <<EOF

  $ME: extracts lines in a file between two tags

  usage: FILENAME {TAG_PREFIX|START_TAG} {END_TAG}

  examples:
    $ME 1.txt AA     => extracts lines in file 1.txt between AA_START and AA_END
    $ME 1.txt AA BB  => extracts lines in file 1.txt between AA and BB
EOF
)
  echo "$IT"
  echo
  exit
}

if [ "$1" == "help" ]
then
  show_help
fi
if [ -z "$2" ]
then
  show_help
fi

function doMain()
{
  FILENAME=$1
  if [ ! -f $FILENAME ]; then
      echo "File not found: $FILENAME"
      exit;
  fi

  if [ -z "$3" ]
  then
    START_TAG=$2_START
    END_TAG=$2_END
  else
    START_TAG=$2
    END_TAG=$3
  fi

  CMD="cat $FILENAME | awk '/$START_TAG/{f=1;next} /$END_TAG/{f=0} f'"
  eval $CMD
}

doMain "$@"

Upvotes: 0

Dan
Dan

Reputation: 11

To make it more deterministic and not having to worry about size of file, use the wc -l and cut the output.

grep -Awc -l test.txt|cut -d" " -f1 test1 test.txt | grep -Bwc -l test.txt|cut -d" " -f1 test2

To make it easier to read, assign it to a variable first.

fsize=wc -l test.txt|cut -d" " -f1; grep -A$fsize test1 test.txt | grep -B$fsize test2

Upvotes: 1

Jotne
Jotne

Reputation: 41446

Print from test1 to test2 (Trigger lines included)

awk '/test1/{f=1} /test2/{f=0;print} f'
awk '/test1/{f=1} f; /test2/{f=0}' 
awk '/test1/,/test2/'


test1
abc
cvb
bnm
test2

Prints data between test1 to test2 (Trigger lines excluded)

awk '/test1/{f=1;next} /test2/{f=0} f' 
awk '/test2/{f=0} f; /test1/{f=1}' 


abc
cvb
bnm

Upvotes: 80

Tweeks
Tweeks

Reputation: 19

The answer by PratPor above:

cat test.txt | grep -A10 test1 | grep -B10 test2

is cool.. but if you don't know the file length:

cat test.txt | grep -A1000 test1 | grep -B1000 test2

Not deterministic, but not too bad. Anyone have better (more deterministic)?

Upvotes: 0

pratpor
pratpor

Reputation: 2104

You can do something like this too. Lets say you this file test.txt with content:

a43
test1
abc
cvb
bnm
test2
kfo

You can do

cat test.txt | grep -A10 test1 | grep -B10 test2

where -A<n> is to get you n lines after your match in the file and -B<n> is to give you n lines before the match. You just have to make sure that n > number of expected lines between test1 and test2. Or you can give it large enough to reach EOF.

Result:

test1
abc
cvb
bnm
test2

Upvotes: 0

Avinash Raj
Avinash Raj

Reputation: 174696

Yep, normal grep won't do this. But grep with -P parameter will do this job.

$ grep -ozP '(?s)test1\n\K.*?(?=\ntest2)' file
abc
cvb
bnm

\K discards the previously matched characters from printing at the final and the positive lookahead (?=\ntest2) asserts that the match must be followed by a \n newline character and then test2 string.

Upvotes: 8

devnull
devnull

Reputation: 123448

You could use sed:

sed -n '/test1/,/test2/p' filename

In order to exclude the lines containing test1 and test2, say:

sed -n '/test1/,/test2/{/test1/b;/test2/b;p}' filename

Upvotes: 59

philshem
philshem

Reputation: 25331

If you can only use grep:

grep -A100000 test1 file.txt | grep -B100000 test2 > new.txt

grep -A and then a number gets the lines after the matching string, and grep -B gets the lines before the matching string. The number, 100000 in this case, has to be large enough to include all lines before and after.

If you don't want to include test1 and test2, then you can remove them afterwards by grep -v, which prints everything except the matching line(s):

egrep -v "test1|test2" new.txt > newer.txt

or everything in one line:

grep -A100000 test1 file.txt | grep -B100000 test2 | egrep -v "test1|test2" > new.txt

Upvotes: 15

Related Questions