CODEWITHSUNDEEP
scalaapache-sparkdatabricks
Artem
Artem

Reputation: 1217

How to use file from Databricks FileStore

Trying to use a .dat file for ip lookup. File is on Databricks file store from Scala code:

  def getCountryCode(ip: String) {
     val filePath = "FileStore/maxmind/GeoIPCountry.dat"
     val ipLookups = new IpLookups(geoFile = Option(new File(filePath)),
  ispFile = None, orgFile = None, domainFile = None, memCache = false, lruCache = 0)

     val location = ipLookups.performLookups(ip)._1.head
     println(location.countryCode)
  }

I am getting an exception:

java.io.FileNotFoundException: FileStore/maxmind/GeoIPCountry.dat (No such file or directory)

Method works on local environment with relative/absolute paths

Upvotes: 2

Views: 2018

Answers (1)

Alper t. Turker
Alper t. Turker

Reputation: 35219

Use fully qualified path to the file on the FUSE mounted file system:

val filePath = "/dbfs/FileStore/maxmind/GeoIPCountry.dat"

Reference: Using Local File I/O APIs.

Upvotes: 4

Related Questions