CODEWITHSUNDEEP
scalaapache-spark
Mary Jane
Mary Jane

Reputation: 139

Scala: what is a CompactBuffer?

I am trying to figure out the meaning of CompactBuffer. Is it the same as iterator?

Please explain the differences.

Upvotes: 4

Views: 6748

Answers (1)

omrsin
omrsin

Reputation: 578

According to Spark's documentation, it is an alternative to ArrayBuffer that results in better performance because it allocates less memory.

Here is an extract of the documentation of the CompactBuffer class:

/**
* An append-only buffer similar to ArrayBuffer, but more memory-efficient for small buffers.
* ArrayBuffer always allocates an Object array to store the data, with 16 entries by default,
* so it has about 80-100 bytes of overhead. In contrast, CompactBuffer can keep up to two
* elements in fields of the main object, and only allocates an Array[AnyRef] if there are more
* entries than that. This makes it more efficient for operations like groupBy where we expect
* some keys to have very few elements.
*/

Upvotes: 6

Related Questions