CODEWITHSUNDEEP
scalajvmsbtjrebel
Johnny Everson
Johnny Everson

Reputation: 8601

Will running everything from RAM disk speed up scala compile time?

Scenario:

The machine I use for development have 32Gb of DDR3 RAM, i7 3770, SSD. The project is large, Scala compiles fast most of the time during incremental compilation but sometimes a single change leads to recompilation of hundreds of files, it then take some time to compile all and some good time for jrebel to reload all changed files.

Question:

Will putting everything on a RAMFS (Mac) make compile and jrebel reload significantly faster?

My plan was to put everything directly related to the project in a RAMFS partition ( .ivy, project source, .sbt, maybe even copy JDK. etc). I would create a script to do all that in the boot or manually, that won't be a problem. Also, I would setup file sync tasks, so, losing a change won't be a concern in case of a OS failure.

Updates:

  1. log says around 400 among java and scala sources are compiled after a clean.
  2. after changing a file in a core module, it recompiles 130 files in 50s.
  3. jrebel takes 72s to reload after #1 and 50s after #2
  4. adding -Drebel.check_class_hash=true made jrebel reload instantaneous after #2.

I am quite happy with these results, but still interested on how to make scala compilation even faster, since cpu usage gets at most 70% for just about 5 seconds in compilation process that takes 170s, overall cpu usage during the compilation is 20%.

UPDATE:

After putting JVM, source, .ivy2 and .sbt folders on RAMDISK, I noticed a small improvement on compile time only: from 132s to 122s ( after a clean). So, not worth the trouble.

NOTE:

That is excluding the dependency resolution, since I using this approach to avoid losing dependency resolution after a clean.

Upvotes: 11

Views: 2504

Answers (3)

Oliver Shaw
Oliver Shaw

Reputation: 5412

There's often very little point in a RAM disk if you're working on Linux or OSX. Those OS's cache the files anyway.

https://unix.stackexchange.com/a/66402/141286

Upvotes: 1

Francois G
Francois G

Reputation: 11985

I have no idea what speedup you can expect with a Mac, but I have seen speedups on Linux compiling the Scala compiler itself that are encouraging enough to try. My report (warning : quite Linux-specific) is there.

Upvotes: 2

Anton Arhipov
Anton Arhipov

Reputation: 6591

You can try setting a VM argument -Drebel.check_class_hash=true which will check the checksum before reloading the classes.

Upvotes: 2

Related Questions