CODEWITHSUNDEEP
mysqlhadoophdfsrelational-databasesqoop
sola.carol
sola.carol

Reputation: 47

Best ways to do select in hdfs?

I have a hadoop environment with 1 master and 4 nodes where I am saving all the data of a mysql application with sqoop

I need to access this data saved in hadoop through the web of the application, in other words: If the user makes a record with a date before 6 months I want the application to select in hadoop data.

They are relational data, mysql. I do not have to do any great analysis.

Thank you in advance

Upvotes: 1

Views: 579

Answers (1)

Dev
Dev

Reputation: 13753

As I understand your question, you are importing data from MySQL to HDFS using sqoop.

Now you want to perform some query over this data in HDFS.

You can do this using Hive. You can perform HQL (similar to SQL) on your data.

You can import your data directly from MySQL to Hive using sqoop. Now you have table in Hive similar to MySQL. You can perform any query over it.

Sample command:

sqoop import \
--connect 'jdbc:mysql://myhost:3306/classicmodels' \
--driver com.mysql.jdbc.Driver \
--username root \
--password root \
--table abc \
--target-dir /user/dev/db/sqoop/temp_81323/ \
--hive-import \
--hive-table hive_abc \
--null-string '\\N' \
--null-non-string '\\N' \
--verbose

Check sqoop documentation for more details.

Upvotes: 1

Related Questions