CODEWITHSUNDEEP
hadoopgoogle-cloud-platformsqoopgoogle-cloud-dataproc
Joseph N
Joseph N

Reputation: 560

Issues while submitting job to dataproc sqoop

I am very new to apache sqoop. I have created dataproc cluster in GCP and trying to submit sqoop job using below command:

gcloud dataproc jobs submit hadoop --cluster cluster123 --class=org.apache.sqoop.Sqoop --import -Dmapreduce.job.user.classpath.first=true --jars=gs://bucket123/sqoop-1.4.7-hadoop260.jar,gs://bucket123/mssql-jdbc-8.4.1.jre11.jar --connect="jdbc:sqlserver://server_ip:1433/db_name" --username=sql_user --password-file=gs://bucket123/password.txt --target-dir gs://bucket123/mssql-output-folder/ --table=table123

Getting below error:

    ERROR: (gcloud.dataproc.jobs.submit.hadoop) unrecognized arguments:
  --import (did you mean '--format'?)
  -Dmapreduce.job.user.classpath.first=true
  --connect=jdbc:sqlserver://server_ip:1433/table123 (did you mean '--project'?)
  --username=sqlserver (did you mean '--cluster'?)
  --password-file=gs://bucket123/password.txt (did you mean '--flags-file'?)
  --target-dir
  gs://bucket123/mssql-output-folder/
  --table=test1 (did you mean '--labels'?)
  To search the help text of gcloud commands, run:
  gcloud help -- SEARCH_TERMS

Upvotes: 2

Views: 578

Answers (1)

Gaurangi Saxena
Gaurangi Saxena

Reputation: 236

Before adding job arguments in the command, you need to put '--'.

Try this:

gcloud dataproc jobs submit hadoop \
  --cluster cluster123 \
  --class=org.apache.sqoop.Sqoop \
  --jars=gs://bucket123/sqoop-1.4.7-hadoop260.jar,gs://bucket123/mssql-jdbc-8.4.1.jre11.jar \
  -- \
  --import -Dmapreduce.job.user.classpath.first=true --connect="jdbc:sqlserver://server_ip:1433/db_name" --username=sql_user --password-file=gs://bucket123/password.txt --target-dir gs://bucket123/mssql-output-folder/ --table=table123

Upvotes: 2

Related Questions