CODEWITHSUNDEEP
algorithmscalacode-generation
flowit
flowit

Reputation: 1442

Permute function calls

I want to run a list of transformations on a dataset. Input and output of those transformations are type-wise always the same, so I can choose an order arbitrarily. In the end, I want to evaluate how the different orders perform (quality of the output data, computing performance and so on). So I give a file to the program where each line contains a function identifier and I run it like this:

// read lines of file into plan
for (x <- plan) {

  val temp : myDataType

  x match {
    case "Func_A" => temp = myData.map(y => funcA(y))
    case "Func_B" => temp = myData.map(y => funcB(y))
    ...
  }

  myData = temp
}

myDataType is an immutable collection which forces me to use these temporary variables. plan can contain possibly 20 lines/function identifiers which would lead to a lot of copy operations. So I am looking for a better solution. My dream would be to chain all the map functions like this, where write is just a function to collect statistics and write them to disc, no temporary variables needed:

plan match {
  case "Plan_A" => myData.map(x => funcB(x)).map(x => funcA(x)).map(...).write
  case "Plan_B" => myData.map(x => funcA(x)).map(...).map(x => funcA(x)).write
  ...
}

However, in this case, I would need to know all (at least those I want to try) permutations in advance, and there will be a lot (20+ functions). I also looked into compile time/runtime code generation but it looks a bit like overkill to me.

So maybe you have some ideas how to solve this a bit more elegant.

Upvotes: 0

Views: 70

Answers (1)

Iulian
Iulian

Reputation: 1536

I see two things to be addressed here.

  1. Chaining functions together
  2. Getting a sequence of functions

Note: For simplicity I am assuming operating on Int, and therefore functions of type Int => Int. You said you are mapping to the same type.

1. Chaining

Assuming you have a sequence of functions and some data, you can do the following: 

...
val plan: Seq[Int => Int] = ... // see my second point
...
val myData = List(1, 2, 3, 4) // immutable List

// Can use compose instead of andThen
val compositeFunc = plan.foldLeft((i: Int) => i)(_ andThen _)
val newData = myData map compositeFunc

Note you can compose the functions with andThen or compose (https://twitter.github.io/scala_school/pattern-matching-and-functional-composition.html)

2. Getting a sequence of functions (plan)

You can do this any number of ways:

  • Define them in a map somewhere in your code
  • Define them in code and select them using reflection
  • Compile them at runtime

Static definition in a map

class MyFunctions {
  def f1(i: Int): Int = i * 2
  def f2(i: Int): Int = i * i
  def f3(i: Int): Int = i * i * i

  // Hardcoded map from string to function name
  // Avoids any reflection
  val funMap = Map[String, (Int) => Int](
    "f1" -> f1,
    "f2" -> f2,
    "f3" -> f3
  )
}

val funcs = new MyFunctions

val myData = List(1, 2, 3, 4) // immutable List

// Assume you read these from your file
val planFromFile = List("f1", "f3", "f2") // String function names

// Get functions using the hardcoded map
val plan = planFromFile map (name => funcs.funMap(name))

// Can use compose instead of andThen
val compositeFunc = plan.foldLeft((i: Int) => i)(_ andThen _)

// Map the function composition to your data
val newData = myData map compositeFunc

Using reflection

import scala.reflect.runtime.{universe => ru}

class MyFunctions {
  def f1(i: Int): Int = i * 2
  def f2(i: Int): Int = i * i
  def f3(i: Int): Int = i * i * i
}

val funcs = new MyFunctions

val myData = List(1, 2, 3, 4) // immutable List

// Assume you read these from your file
val planFromFile = List("f1", "f3", "f2") // String function names

// Acts as a function wrapper that wraps a MethodMirror
// Note that all functions in Scala are objects ((Int => Int) is shorthand for Function1 ...)
class WrappedFunction(mirror: reflect.runtime.universe.MethodMirror) extends (Int => Int) {
  override def apply(v1: Int): Int = mirror(v1).asInstanceOf[Int]
}

// Returns function wrappers for each string
// Note for simplicity there is no code dealing with missing function , errors etc.
def getFunctions(names: Seq[String]): Seq[Int => Int] =
  names.map(s => new WrappedFunction(ru.runtimeMirror(funcs.getClass.getClassLoader)
    .reflect(funcs)
    .reflectMethod(ru.typeOf[MyFunctions]
      .decl(ru.TermName(s)).asMethod)))


val reflectedFunctions = getFunctions(planFromFile)

// Compose them from the generated functions
val compositeFunc2 = reflectedFunctions.foldLeft((i: Int) => i)(_ andThen _)

// Generate data
val newData2 = myData map compositeFunc2

I have not talked about compiling at runtime and I am assuming this is not your use case.

You can also combine the approaches and generate the map using reflection.

EDIT Obviously you can also use pattern matching instead of that map.. something like this:

def fromName(name: String): Int => Int = name match {
    case "f1" => f1
    case "f2" => f2
    case "f3" => f3
    case _ => throw new IllegalArgumentException("booooom!")
}

For more information on reflection see the docs: http://docs.scala-lang.org/overviews/reflection/overview.html

Upvotes: 2

Related Questions