CODEWITHSUNDEEP
sqoopsqoop2
Nishikant Jain
Nishikant Jain

Reputation: 141

passing mysql properties via sqoop eval

sqoop eval command :

sqoop eval --connect 'jdbc:mysql://<connection url>' --driver com.mysql.jdbc.Driver --query "select max(rdate) from test.sqoop_test"

gives me output:

Warning: /usr/hdp/2.3.2.0-2950/accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. Warning: /usr/hdp/2.3.2.0-2950/zookeeper does not exist! Accumulo imports will fail. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. 16/10/05 18:38:17 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.3.2.0-2950 16/10/05 18:38:17 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 16/10/05 18:38:17 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time. 16/10/05 18:38:17 INFO manager.SqlManager: Using default fetchSize of 1000 -------------- | max(rdate) | -------------- | 2014-01-25 |

but i want output without warning and table boundries like:

max(rdate) 2014-01-25

i basically want to store this output to a file. thanks in advance

Upvotes: 0

Views: 625

Answers (3)

Sathiyan S
Sathiyan S

Reputation: 1023

We have two ways to get the query results:

  1. The other way is you can write to HDFS by importing query results(--target-dir /path) and read from there.

  2. You can change the file system option in sqoop command to store the results from import query, So the idea behind is you importing data to local file system rather HDFS.

eg: sqoop import -fs local -jt local --connect "connection string" --username root --password root query "Select * from table" --target-dir /home/output

https://sqoop.apache.org/docs/1.4.0-incubating/SqoopUserGuide.html#id1762587

Upvotes: 0

Farslan
Farslan

Reputation: 106

You can create a .sh file , write your sqoop commands into it, then run it as shell_file_name.sh > your_output_file.txt

Upvotes: 0

Dev
Dev

Reputation: 13753

You can perform Sqoop Import operation to save output in HDFS.

Warnings are straight forward.

  • You can set $ACCUMULO_HOME, $ZOOKEEPER_HOME if available.

  • You can set --connection-manager corresponding to Mysql

  • For the sake of security, It's recommended to use -P for password rather than writing in command.

These are not errors, you can live with these warnings.

Upvotes: 0

Related Questions