CODEWITHSUNDEEP
scalaconstructornull
pferrel
pferrel

Reputation: 5702

Scala constructor with default values

I want to slap a read and/or write trait on the Store below. To do that I need to have default null-like values for the Schemas or functions, then check them at runtime. This is to allow a read, read+write, and write only Store. The traits will define reader and/or writer and that is all.

This would looks like:

//read only, exception if someone tried to write with it
val rStore = new Store[SomeType](sc, readSchema) with SomeReadTrait
//write only, this would leave inSchema as null-ish so I can check if readfrom is called.
val wStore = new Store[SomeType](sc, outSchema = writeSchema) with SomeWriteTrait
// read/write
val rwStore = new Store[SomeType](sc, readSchema, writeSchema) with SomeReadTrait with SomeWriteTrait
// another way to do read/write
val rwStore = new Store[SomeType](sc, readSchema, writeSchema) with SomeReadWriteTrait

But I can't figure out how to create a default null-ish value in my constructor and a default null-ish value for an internal function def.

I might be able to handle this with three abstract types, but is there a way to do something like the following to keep the code DRY. I don't care if the check is for the reader/writer function (to see if a non null-ish one was supplied by the trait) or the Schema (to see which were given non null-ish values when constructed) or both.

abstract class Store[T](sc: SparkContext, inSchema: Schema = ????, outSchema: Schema = ????){
  /** Reader function that reads from the location into the T and returns it, supplied by a Read trait*/
  def reader: (SparkContext, Schema, String) => T = ????
  /** Writer function, supplied in a Write trait */
  def writer: (SparkContext, Schema, String, T) => ????
  def readFrom(source: String): T = {
    if (reader != ????) reader(sc, inSchema, source) else throw new IllegalArgumentException("Store: readFrom called but there is no reader function has been defined")
  }
  def writeTo(collection: T, dest: String): Unit = {
    if (writer != ????) writer(sc, outSchema, dest, collection) else throw new IllegalArgumentException("Store: writeTo has been called but there is no writer function")
  }
}

Upvotes: 1

Views: 314

Answers (2)

pferrel
pferrel

Reputation: 5702

With @DCKing's help I think this is what I needed. No abstract class--all traits. There is a root trait that defines the attributes and one method for readFrom or writeTo. The extended trait defines the type read or written and the reader or writer. Now to extend this we just create a new read or write trait for a specific data type, only one protected function to write. Quite DRY. Using String as a standin for various types. 

trait RT[T]{
    val sc: String
    val inSchema: String
    protected def reader(sc: String, s: String, source: String): T
    def readFrom(source: String): T = reader(sc, inSchema, source)

}

trait StringReadTrait extends RT[String]{
    protected def reader(sc: String, s: String, source: String): String = {println("StringReadTrait#reader called, sc: "+sc+" s: "+s+" source: "+source); "StringReadTrait#reader"}
}

trait WT[T]{
    val sc: String
    val outSchema: String
    protected def writer(sc: String, s: String, dest: String, collection: String): Unit
    def writeTo(collection: String, dest: String) = writer(sc, outSchema, dest, collection)
}

trait StringWriteTrait extends WT[String]{
    protected def writer(sc: String, s: String, dest: String, collection: String) = {println("writer called, sc: "+sc+" s: "+s+" dest: "+dest+" collection: "+collection)}
}

trait StringReadWriteTrait extends StringReadTrait with StringWriteTrait

class ReadStore(val inSchema: String, val sc: String) extends StringReadTrait
class WriteStore(val outSchema: String, val sc: String) extends StringWriteTrait
class ReadWriteStore(val inSchema: String, val outSchema: String, val sc: String) extends StringReadWriteTrait

Upvotes: 0

DCKing
DCKing

Reputation: 4283

This is a weird way to accomplish what you want. If the outSchema is only relevant when a WriteTrait is mixed in, and an inSchema only when a ReadTrait is mixed in, then why are they not part of the traits but part of some other class? If you make them part of the trait, you can have the compile ensure everything is okay instead of throwing IllegalArgumentExceptions around.

 trait SomeReadTrait[T] {
    val inSchema: Schema
    val context: SparkContext
    def reader(sc: SparkContext, s: Schema, source: String): T = ???
    def readFrom(source: String): T = reader(context, inSchema, source)
 }

 trait SomeWriteTrait[T] {
    val outSchema: Schema
    val context: SparkContext
    def writer(sc: SparkContext, s: Schema, dest: String, collection: T) = ???
    def writeTo(collection: T, dest: String) = writer(context, outSchema, dest, collection)
 }

 trait SomeReadWriteTrait[T] extends SomeReadTrait[T] with SomeWriteTrait[T]

 class ReadStore[T](val context: SparkContext, val inSchema: Schema) extends SomeReadTrait[T]
 class WriteStore[T](val context: SparkContext, val outSchema: Schema) extends SomeWriteTrait[T]
 class ReadWriteStore[T](val context: SparkContext, val inSchema: Schema, val outSchema: Schema) extends SomeReadWriteTrait[T]

Isn't this what you want?

Upvotes: 3

Related Questions