Keeping the bytecode of inline methods in resulting JAR

Question

I'm not a Scala dev by any means (rather Dart and Rust one), but I need to use some library written in Scala in my Java project.

I can build the JAR from that library and use it as Maven dependency, import classes from it (despite the somewhat mangled names), but there's one thing, which prevents me from using this lib fully. Most of the classes there abound with methods marked with "inline" modifier. These methods look useful to me and I'd like to have the ability to call them freely.

But because of their inline nature they're optimized away from resulting .class files during build, being written into .tasty files only, that are ignored by Java compiler, consequently they become inaccessible to Java code.

So what I need, is to get these methods in Java accessible bytecode, either by disabling the inlining at all on compiler level, or by keeping them inline but additionally compiling their bodies into .class files.

I tried to use some related scalac CLI options, like -opt:* and -Yno-inline, but with no luck, Scala 3 doesn't provide any of them anymore.

Then I asked ChatGPT, it suggested to write Scala compiler plugin, which programmatically changes some related flag in AST tree, but the provided code didn't compile due to missing imports and I'm unable to find them in scala3-compiler.

I tried to google some theory about writing Scala compiler plugins, but also with no luck.

So my question is pretty simple - how to forcibly include the inlined methods into bytecode?

Thanks.

Answer 1

It's not always possible to forcibly include inline methods into bytecode because in Scala 3 macros are implemented with inline. But let's assume we're talking about "ordinary" inline methods (not macros).

In principle, you can remove inline not only with a compiler plugin (at compile time) but also transforming sources (at pre-compile-time) e.g. with Scalameta.

See https://github.com/DmytroMitin/scalametademo

build.sbt

lazy val scala2V = "2.13.13"
lazy val scala3V = "3.4.1"

lazy val javaProj = project
  .settings(
    scalaVersion := scala3V,
    libraryDependencies ++= Seq(
      "org.lichess" %% "scalalib-core" % "11.1.5",
    ),
    resolvers ++= Seq(
      "lila-maven" at "https://raw.githubusercontent.com/ornicar/lila-maven/master",
    ),
  )

//lazy val before = project
//  .settings(
//    scalaVersion := scala3V
//  )
//
//lazy val after = project
//  .settings(
//    scalaVersion := scala3V
//  )

lazy val transform = project
  .settings(
    scalaVersion := scala2V,
    libraryDependencies ++= Seq(
      "com.lihaoyi" %% "os-lib" % "0.10.0",
      "org.scalameta" %% "scalameta" % "4.9.3",
      "org.eclipse.jgit" % "org.eclipse.jgit" % "6.9.0.202403050737-r",
    ),
  )

transform/src/main/scala/Main.scala

import org.eclipse.jgit.api.Git

object Main extends App {
  private val root = os.pwd
  private val before = root / "before"
  private val after = root / "after"
  private val javaProjName = "javaProj"
  private val javaProj = root / javaProjName
  private val javaProjLib = javaProj / "lib"
  private val jarName = "scalachess_3-16.0.5.jar"

  println("cleaning...")

  os.remove.all(before)
  os.remove.all(after)
  os.remove.all(javaProjLib)
  os.proc("sbt", s"$javaProjName/clean").call(cwd = root, stdout = os.Inherit)

  println("cloning...")

  Git.cloneRepository()
    .setURI("https://github.com/lichess-org/scalachess.git")
    .setDirectory(before.toIO)
    .call()

  println("transforming (removing inline)...")

  FileTransformer.transform(before = before, after = after)

  println("packaging...")

  os.proc("sbt", "scalachess/package").call(cwd = after, stdout = os.Inherit)

  println("copying...")

  os.copy(
    from = after / "core" / "target" / "scala-3.4.1" / jarName,
    to = javaProjLib / jarName,
    createFolders = true,
  )

  println("running...")

  os.proc("sbt", s"$javaProjName/runMain JMain").call(cwd = root, stdout = os.Inherit)

  println("done")
}

transform/src/main/scala/FileTransformer.scala

object FileTransformer {
  def transform(before: os.Path, after: os.Path): Unit = {
    os.walk.attrs(
        before,
        skip = (path, _) => path.segments.contains("target")
      )
      .filter { case (_, info) => info.isFile }
      .foreach { case (path, _) =>
        val path1 = after / path.relativeTo(before)

        if (path.last.endsWith(".scala")) {
          val str = os.read(path)
          val str1 = TreeTransformer.transform(str)
          os.write.over(path1, str1, createFolders = true)
        } else os.copy.over(from = path, to = path1, createFolders = true)
      }
  }
}

transform/src/main/scala/TreeTransformer.scala

import scala.meta._

object TreeTransformer {
  def transform(str: String): String = {
    implicit val dialect: Dialect = dialects.Scala3
    val tree = str.parse[Source].get
    val tree1 = removeInlineTransformer(tree)
    tree1.syntax
  }

  private val removeInlineTransformer: Transformer = new Transformer {
    private def removeInlineMod(mods: List[Mod]): List[Mod] =
      mods.filter(_.isNot[Mod.Inline])

    override def apply(tree: Tree): Tree = {
      def removeInline(
                        mods: List[Mod],
                        ename: Term.Name,
                        tparams: Type.ParamClause,
                        paramss: List[Term.ParamClause],
                        tpeopt: Option[Type],
                        expr: Term,
                      ): Tree = {
        val mods1 = removeInlineMod(mods)

        val paramss1 = paramss.map(_.map {
          case param"..$paramMods $name: $paramTpeopt = $expropt" =>
            val paramMods1 = removeInlineMod(paramMods)
            param"..$paramMods1 $name: $paramTpeopt = $expropt"
        })

        q"..$mods1 def $ename[..$tparams](...$paramss1): $tpeopt = $expr"
      }

      def ignoreMacros(
                       mods: List[Mod],
                       ename: Term.Name,
                       tparams: Type.ParamClause,
                       paramss: List[Term.ParamClause],
                       tpeopt: Option[Type],
                       expr: Term,
                     ): Tree = {
        expr match {
          case _: Term.SplicedMacroExpr => tree
          case _ => removeInline(mods, ename, tparams, paramss, tpeopt, expr)
        }
      }

      val tree1 = tree match {
        case q"..$mods def $ename: $tpeopt = $expr" =>
          ignoreMacros(mods, ename, Nil, Nil, tpeopt, expr)
        case q"..$mods def $ename[..$tparams](...$paramss): $tpeopt = $expr" =>
          ignoreMacros(mods, ename, tparams, paramss, tpeopt, expr)
        case _ => tree
      }

      super.apply(tree1)
    }
  }
}

javaProj/src/main/java/JMain.java

public class JMain {
    public static void main(String[] args) {
        System.out.println(
                chess.Square$package.Square$.MODULE$.fromKey("a1") // Some(0)
        );
    }
}

You can execute sbt transform/run and this will clone Scalachess library (into before), transform its sources in core removing inline (into after), package new edited Scalachess jar, put it into javaProj/lib and run JMain in javaProj.

So we forcibly include the method fromKey into bytecode.

JDK 21, sbt 1.9.9.

Answer 2

You imply that you have some knowledge of Scala, so one option is to write your own Scala wrapper for that library:

1. Define a Java interface for the functions you want to use from your Java code

2. Implement the Java interface in Scala using the Scala library

3. Import this new library into your Java code and use the Java interface

Stage 1 avoids the problem with mangled names because you will be using a pure Java Interface at stage 3.

Stage 2 will have access to the inline functions in the Scala code and can make them available from Java.