CODEWITHSUNDEEP
pythonhadoophiveclouderajaydebeapi
Pavan Sai Aravala
Pavan Sai Aravala

Reputation: 49

How to load csv file into hive table using python on local windows machine

We have enterprise hadoop cluster installed on linux servers in our organisation. I am trying to insert csv file into one of our hive tables. I have csv file in my local windows machine. I am using python and using jaydebeapi package to connect to hive through jdbc. While I am using "LOAD DATA LOCAL..." command in code, it is throwing error related to paths not legal file path. How to resolve this issue? Below is the error I am getting.

Note: I am not going to use hdfs commands manually to copy csv file to hdfs and then load data. I want to load it directly from my windows machine to hive table.

com.cloudera.hiveserver2.support.exceptions.ErrorException

Error while compiling statement: FAILED: SemanticException [Error 10028]: Line 1:23 Path is not legal

I am using below piece of code for loading data.

query="""LOAD DATA LOCAL INPATH 'C:/Users/Docs/file.csv' INTO TABLE default.logstable"""
conn = getConnection()
curr = conn.cursor()
curr.execute(query)

connection was already setup, select queries are working fine.

Upvotes: 0

Views: 56

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191681

C:/ is not a legal Unix filepath.

You would have to use file://c:/Users/Docs/file.csv

Otherwise, yes, upload the CSV directly to HDFS, then define an EXTERNAL TABLE over it.

Upvotes: 0

Related Questions