Configure flume in shell/bash script - avoid interactive flume shell console

Question

The normal way you configure flume is via flume master web console, way easy to talk about it here.

OR

via interactive flume shell console, follow the steps below:

1. $ flume shell   //this brings you to the interactive flume shell console
2. in the interactive console,connect flume-master-node   // this connects you to flume-master
3. in the interactive console, run "exec unconfig your_node"  // this unconfig all flume configuration 
4. in the interactive console, run "exec config your_node new_config"    // apply new flume configuration
5. quit  // exit the the interactive console

So far so good.

Then I am trying to write a bash script for my flume configuration. so I want to squeeze 1,2,3,4,5 into a bash and it runs automatically every time without intervention, something like this:

/usr/bin/flume shell << EOF         #line1
connect $FLUME_MASTER               #line2

exec unconfig your_node             #line3
exec config your_node new_config    #line4

quit                                #line5

EOF                                 #line6

But everytime I run this bash script it always stops at #line1 and brings me to interactive flume shell console instead of running it sliently in a non-iteractive mode.

Does anybody know how to ignore the interactive mode and just run it sliently?

Answer 1

Somehow followed flume user guide flume/UserGuide_using_the_flume_command_shell:

echo "connect localhost:35873\ngetconfigs\nquit" | flume shell -q 

this doesn't work if you put it in a bash script. But I got way with it by doing this:

/usr/bin/flume shell -q << EOF 

connect localhost:35873

getconfigs

quit

EOF

Answer 2

Try the -q option to flume to see if that helps.