CODEWITHSUNDEEP
bashshellawksedgrep
Fizi
Fizi

Reputation: 1861

Bash: pass multiline output of grep to another command that loops

I have the following command to grep the kafka topics 

kafka-topics --list --zookeeper localhost:2181 | grep repartition

This give me a shell output with multiple topics 

dev-ALPHA_CLUSTER-investment.ed.store.alpha_cluster-repartition
dev-CUSTOM_GROUP-KSTREAM-REDUCE-STATE-STORE-0000000003-repartition
dev-CUSTOM_GROUP-investment.ed.store.custom_group-repartition

I need to provide these topics one by one to another command

kafka-topics --zookeeper localhost:2181 --alter --config cleanup.policy=compact --topic TOPIC_NAME

where TOPIC_NAME is the topic(s) from the previous grep command. I was wondering if there is a way to combine them such that if the grep has some results (topics separated by newline) then the other command is executed in a loop with TOPIC_NAME being each topic returned by grep

Upvotes: 1

Views: 293

Answers (3)

ssemilla
ssemilla

Reputation: 3970

Use xargs

kafka-topics --list --zookeeper localhost:2181 | grep repartition | xargs -i kafka-topics --zookeeper localhost:2181 --alter --config cleanup.policy=compact --topic {}

Upvotes: 1

Walter A
Walter A

Reputation: 20022

You can use xargs -n1 for executing the one parameter at a time:

zookeeper="--zookeeper localhost:2181"
kafka-topics --list ${zookeeper} | grep repartition |
   xargs -n1 kafka-topics ${zookeeper} --alter --config cleanup.policy=compact --topic

Upvotes: 2

Martin Urbanec
Martin Urbanec

Reputation: 436

kafka-topics --list --zookeeper localhost:2181 | grep repartition | sed 's/^/kafka-topics --zookeeper localhost:2181 --alter --config cleanup.policy=compact --topic /g'

The sed-part of the command is prepending kafka-topics --zookeeper localhost:2181 --alter --config cleanup.policy=compact --topic (including the space at the end) to output of kafka-topics --list --zookeeper localhost:2181. You can either redirect those commands to a file or redirect them to bash to run them directly.

Upvotes: 0

Related Questions