CODEWITHSUNDEEP
scalabinary-serialization
Will I Am
Will I Am

Reputation: 2672

Scala bitfields serialization

I am extremely new at Scala and I'm getting confused by the bit manipulation features. I am hoping someone can point me in the right direction?

I have a byte array defined with the following bit fields:

0-3 - magic number
  4 - version
5-7 - payload length in bytes
8-X - payload, of variable length, as indicated in bits 5-7

I would like to serialize this back and forth to a structure such as:

MagicNumber: Integer
Version: Integer
Length: Integer
payload: Array[Byte]

How do you deal with bits in this situation optimally? Most of the examples I've seen deal with higher level serialization, such as JSON. I am trying to serialize and deserialize TCP binary data in this case.

Upvotes: 3

Views: 547

Answers (2)

Alexey Romanov
Alexey Romanov

Reputation: 170735

I'd look at scodec. Based on the UDP example, it should be something like (untested):

import scodec.bits.{ BitVector, ByteVector }
import scodec.codecs._

case class Datagram(
  magicNumber: Int,
  version: Byte,
  payload: ByteVector)

object Datagram {
  implicit val codec: Codec[Datagram] = {
    ("magic_number" | int32 ) ::
    ("version" | byte ) ::
    variableSizeBytes(int(3),
      ("payload" | bytes ))
  }.as[Datagram]
}

Upvotes: 4

dk14
dk14

Reputation: 22374

You can use Scala Pickling or POF or Google Protobuf, but if your format is so restricted, the simplest way is to write your own serializer:

case class Data(magicNumber: Int, version: Int, payload: Array[Byte])

def serialize(data: Stream[Data]): Stream[Byte] = 
   data.flatMap(x => 
     Array((x.magicNumber << 4 | x.version << 3 | x.payload.length).toByte) ++ x.payload)

@scala.annotation.tailrec
def deserialize(binary: Stream[Byte], acc: Stream[Data] = Stream[Data]()): Stream[Data] =   
   if(binary.nonEmpty) {
     val magicNumber = binary.head >> 4 
     val version = (binary.head & 0x08) >>3 
     val size = binary.head & 0x07
     val data = Data(magicNumber, version, ByteVector(binary.tail.take(size).toArray)) 
     deserialize(binary.drop(size + 1), acc ++ Stream(data)) 
   } else acc

Or you can use Scodec library (this option is better because you will have automatical value range check):

Sbt:

  libraryDependencies += "org.typelevel" %% "scodec-core" % "1.3.0"

Codec:

  case class Data(magicNumber: Int, version: Int, payload: ByteVector)
  val codec = (uint(4) :: uint(1) :: variableSizeBytes(uint(3), bytes)).as[Data]

Use:

  val encoded = codec.encode(Data(2, 1, bin"01010101".bytes)).fold(sys.error, _.toByteArray)
  val decoded = codec.decode(BitVector(encoded)).fold(sys.error, _._2)

Upvotes: 7

Related Questions