CODEWITHSUNDEEP
scalasbt
Minh Thai
Minh Thai

Reputation: 578

SBT refuses to continue downloading

SBT stucks for an hour at downloadinng a library and showing speed = 0, here is the output:

sbt kafka/compile [process_args] java_version = '8'
# Executing command line: java
-XX:ReservedCodeCacheSize=128m
-Xms2g
-Xmx4g
-Xss4m
-XX:+UseG1GC
-XX:MaxMetaspaceSize=1g
-jar /usr/local/Cellar/sbt/1.2.8/libexec/bin/sbt-launch.jar kafka/compile

[info] Loading global plugins from /Users/minhthai/.sbt/1.0/plugins [info] Loading settings for project kafka_spark_streaming-build from assembly.sbt,plugins.sbt ... [info] Loading project definition from /Users/minhthai/coding/kafka_spark_streaming/project [info] Loading settings for project kafka_spark_streaming from build.sbt ... [info] Set current project to kafka_spark_streaming (in build file:/Users/minhthai/coding/kafka_spark_streaming/) [info] Fetching artifacts of kafka https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.11.12/scala-reflect-2.11.12.jar
   33.6% [...       ] 1.5 MiB (0 B / s)

Here is my build file

ThisBuild / scalaVersion := "2.11.12"
ThisBuild / version := "0.0.1"

lazy val spark = project
  .settings(
    assembly / mainClass := Some("Main"),
    assembly / assemblyJarName := "spark.jar",
    libraryDependencies ++= Seq(
      "org.apache.spark" %% "spark-sql" % "2.4.3",
      "org.apache.spark" %% "spark-sql-kafka-0-10" % "2.4.3" % "provided"
    )
  )

lazy val kafka = project
  .settings(
    libraryDependencies ++= Seq(
      "org.apache.kafka" % "kafka-clients" % "2.2.0"
    )
  )

I tried removing the cache at ~/.ivy2 and run again but the message is exactly the same, the download stops at 33.6%. I can still download other library and can manually download this jar file in browser.

So what can I do to resolve this issue? If no, is there a way to add this jar file manually?

Upvotes: 0

Views: 155

Answers (2)

Minh Thai
Minh Thai

Reputation: 578

I found the problem and it is silly. I use coursier plugin to handle dependencies and I should have deleted the cache of coursier instead of ivy2. On Mac, it is at ~/Library/Caches/Coursier/v1 (doc).

Upvotes: 0

Gal Naor
Gal Naor

Reputation: 2397

Sbt has those kinds of downloading problems...restarting the process (Ctrl+C) and running again might help.

If it doesn't work, you can add the jar manually like this:

libraryDependencies += "org.scala-lang" % "scala-reflect" % "2.11.12" from "http://central.maven.org/maven2/org/scala-lang/scala-reflect/2.11.12/scala-reflect-2.11.12.jar"

You can look at the sbt official documentation here.

Upvotes: 1

Related Questions