CODEWITHSUNDEEP
scalaapache-sparkread-eval-print-loop
mut1na
mut1na

Reputation: 795

loading resources in spark-shell

normally, loading resources in the Scala REPL is done like this:

getClass().getClassLoader().getResource("/resource-file")

see here

but this doesn't find resources from jars I load using the usual startup

spark-shell --jars list-of-jars

How are resources loaded in spark-shell? (am I referencing the wrong ClassLoader?)

Upvotes: 2

Views: 1383

Answers (1)

zsxwing
zsxwing

Reputation: 20836

Please remove the prefix "/". I tested in Spark shell and both getClass().getClassLoader().getResource("resource-file") and Thread.currentThread().getContextClassLoader().getResource("resource-file") worked. However, I would recommend using Thread.currentThread().getContextClassLoader() since it doesn't rely on what getClass() returns.

Upvotes: 2

Related Questions