Adding dependency to Apache Toree fails on import

Question

After adding a dependency in my Jupyter notebook using Apache Toree Scala kernel, for example vegas:

%AddDeps org.vegas-viz vegas_2.11 0.3.11 --transitive

Then when I try to import something from this JAR, I get a long stacktrace. In the notebook, I do:

import vegas._
import vegas.render.WindowRenderer._

The command line (where I started Jupyter) shows a long stacktrace:

17/09/25 12:41:16 ERROR OneForOneStrategy: object Predef does not have a member classOf
scala.reflect.internal.FatalError: object Predef does not have a member classOf
    at scala.reflect.internal.Definitions$DefinitionsClass.scala$reflect$internal$Definitions$DefinitionsClass$$fatalMissingSymbol(Definitions.scala:1186)
    at scala.reflect.internal.Definitions$DefinitionsClass.getMember(Definitions.scala:1203)
    at scala.reflect.internal.Definitions$DefinitionsClass.getMemberMethod(Definitions.scala:1238)
    at scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.Predef_classOf$lzycompute(Definitions.scala:1469)
    at scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.Predef_classOf(Definitions.scala:1469)
    at scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.isPredefClassOf(Definitions.scala:1459)
    at scala.tools.nsc.typechecker.Typers$Typer.typedIdent$2(Typers.scala:4885)
    at scala.tools.nsc.typechecker.Typers$Typer.typedIdentOrWildcard$1(Typers.scala:4908)
    at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5340)
    at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5360)
    at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5396)
    at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5423)
    at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5370)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5374)
    at scala.tools.nsc.interpreter.ReplGlobal$$anon$1$$anon$2.typed(ReplGlobal.scala:36)
    at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5472)
    at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5480)
    at scala.tools.nsc.typechecker.Typers$Typer.typedPackageDef$1(Typers.scala:5012)
    at scala.tools.nsc.typechecker.Typers$Typer.typedMemberDef$1(Typers.scala:5312)
    at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5359)
    at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5396)
    at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5423)
    at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5370)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5374)
    at scala.tools.nsc.interpreter.ReplGlobal$$anon$1$$anon$2.typed(ReplGlobal.scala:36)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5448)
    at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3.apply(Analyzer.scala:102)
    at scala.tools.nsc.Global$GlobalPhase$$anonfun$applyPhase$1.apply$mcV$sp(Global.scala:440)
    at scala.tools.nsc.Global$GlobalPhase.withCurrentUnit(Global.scala:431)
    at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:440)
    at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3$$anonfun$run$1.apply(Analyzer.scala:94)
    at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3$$anonfun$run$1.apply(Analyzer.scala:93)
    at scala.collection.Iterator$class.foreach(Iterator.scala:893)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
    at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3.run(Analyzer.scala:93)
    at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1501)
    at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1486)
    at scala.tools.nsc.Global$Run.compileSources(Global.scala:1481)
    at scala.tools.nsc.interpreter.IMain.compileSourcesKeepingRun(IMain.scala:435)
    at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compileAndSaveRun(IMain.scala:855)
    at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compile(IMain.scala:813)
    at scala.tools.nsc.interpreter.IMain$Request.compile$lzycompute(IMain.scala:1002)
    at scala.tools.nsc.interpreter.IMain$Request.compile(IMain.scala:997)
    at scala.tools.nsc.interpreter.IMain.compile(IMain.scala:579)
    at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:567)
    at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
    at org.apache.toree.kernel.interpreter.scala.ScalaInterpreterSpecific$$anonfun$interpretAddTask$1$$anonfun$apply$3.apply(ScalaInterpreterSpecific.scala:368)
    at org.apache.toree.kernel.interpreter.scala.ScalaInterpreterSpecific$$anonfun$interpretAddTask$1$$anonfun$apply$3.apply(ScalaInterpreterSpecific.scala:363)
    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
    at scala.Console$.withErr(Console.scala:80)
    at org.apache.toree.global.StreamState$$anonfun$1$$anonfun$apply$1.apply(StreamState.scala:73)
    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
    at scala.Console$.withOut(Console.scala:53)
    at org.apache.toree.global.StreamState$$anonfun$1.apply(StreamState.scala:72)
    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
    at scala.Console$.withIn(Console.scala:124)
    at org.apache.toree.global.StreamState$.withStreams(StreamState.scala:71)
    at org.apache.toree.kernel.interpreter.scala.ScalaInterpreterSpecific$$anonfun$interpretAddTask$1.apply(ScalaInterpreterSpecific.scala:362)
    at org.apache.toree.kernel.interpreter.scala.ScalaInterpreterSpecific$$anonfun$interpretAddTask$1.apply(ScalaInterpreterSpecific.scala:362)
    at org.apache.toree.utils.TaskManager$$anonfun$add$2$$anon$1.run(TaskManager.scala:140)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

I'm using Jupyter 4.3.0 on Linux Mint 18.1 (based on Ubuntu 16.04) and Apache Toree version:

VERSION: 0.2.0.dev1-incubating
COMMIT: 9b577f19df83

I suspect the error I get may be related to an incompatibility between Apache Toree and the scala version I have... Or something... Any suggestions on how to proceed with this? Many thanks in advance :-)

Answer 1

You are right this is due to dependency conflict. I was facing the same problem. It was due to conflict between scala-reflect-2.11.6 and scala-reflect-2.11.8. I had to manually install vegas-viz and its dependencies to local maven repo. Then added vegas-viz as sparks.jars.pacakages in configuration file instead of loading it on the run. Or you can exclude of the version.