CODEWITHSUNDEEP
linuxbashshell
Yamuna_dhungana
Yamuna_dhungana

Reputation: 663

How to store chunk of lines in variable in shell

I have a text file with 10000+ lines with chr and values per line. I need this information to run a program (called TOOL below). The text file mylist19.list looks like this:

chr1 1 12
chr1 14 34
chr1 44 65
chr1 48 88
chr1 88 93
chr1 122 144

I wan to run this in a batch of 99 lines. So this is what I did:

file="/scratch/mylist19.list" 

start=1
diff=99
LASTline=$(cat $file | wc -l)
while [ ${start} -le ${LASTline} ]; do
    CHUNK=sed -n `expr ${start},${diff}p` ${file}
    echo ${CHUNK}
    #TOOL --use ${CHUNK}
    start=$((${start}+${diff}))
done < ${file}

So how do I store the output of sed -n expr ${start},${diff}p;${LASTline}q ${file} in CHUNK so I can use it in the TOOL command? Also, I would like to include $LASTline in the last chunk.

Upvotes: 1

Views: 272

Answers (2)

lbal
lbal

Reputation: 291

With sed, just a few changes to your script:

file="/scratch/mylist19.list"

start=1
diff=99
end=0
LASTline=$(cat $file | wc -l)
while [ ${start} -le ${LASTline} ]; do
    end=$((${end}+${diff}))
    CHUNK=$(sed -n "${start},${end}p" ${file})
    echo ${CHUNK}
    #TOOL --use ${CHUNK}
    start=$((${start}+${diff}))
done

CHUNK+=$'\n'"$LASTline"

echo "${CHUNK}"

Upvotes: 0

Jetchisel
Jetchisel

Reputation: 7791

I'm not sure about sed but since this is tagged as bash here is a solution without sed which might do what you wanted.

#!/usr/bin/env bash

diff=99
file="/scratch/mylist19.list" 

while mapfile -n "$diff" -t array && (( ${#array[*]} )); do
  chunk=("${array[@]}" "$file")
  echo "${chunk[@]}"
  echo tool --use "${chunk[@]}"
done < "$file"

Also, I would like to include $LASTline in the last chunk.

Something like

#!/usr/bin/env bash

diff=99
file="/scratch/mylist19.list"
total=$(wc -l < "$file")

while mapfile -n "$diff" -t array && (( ${#array[*]} )); do
  (( count += ${#array[*]} ))

  if (( total != count )); then
    chunk=("${array[@]}" "$file") 
  else
    chunk=("${array[@]}" "$file" "$total")
  fi

  echo "${chunk[@]}"
  echo tool --use "${chunk[@]}"
done < "$file"

Upvotes: 2

Related Questions