CODEWITHSUNDEEP
pythonazure-cognitive-services
JohnMartin186
JohnMartin186

Reputation: 19

CSV ingestion error in Azure Data Bricks - File read error

My task is to perform data ingestion of CSV file using python. I am using the following code block and getting error of file not found eventhough the location is right.

spark.read.format("Filelocation") \
  .option("file.format", "csv") \
  .option("file.schemaLocation", "checkpoint") \
  .load("source_data") \
  .writeStream \
  .option("mergeSchema", "true") \
  .option("My_Checkpoint", "Checkpoint path") \
  .start("target")

I have no clue of the error. Any help is appreciated.

Upvotes: 0

Views: 59

Answers (1)

Sairam Tadepalli
Sairam Tadepalli

Reputation: 1683

It's a simple read statement error that I have observed from your code block.

spark.readStream.format("Filelocation")

Use the above code block in the first line instead of spark.read.format("Filelocation") and remaining statements are as it is.

Upvotes: 1

Related Questions