CODEWITHSUNDEEP
ocaml
Ulrar
Ulrar

Reputation: 983

Split a string in chunks

Since I don't see anything for that in core or batteries, I'm looking for an efficient way of splitting a Bytes (or string, or whatever is convenient) in chunks of about ~65k (I'm using a short as my length header, it's for sending over tcp). Tried writing the function myself but I'm having trouble with it, so any help would be appreciated, or pointers to an existing function would be fine too. Thanks !

Upvotes: 0

Views: 303

Answers (2)

Chris
Chris

Reputation: 36496

You might also handle this lazily using the Seq module.

# let split_bytes n b =
    let len = Bytes.length b in
    let rec aux chunk () =
      if chunk * n >= len then
        Seq.Nil
      else
        let start_pos = chunk * n in
        Seq.Cons (
          Bytes.sub b start_pos (min n (len - start_pos)),
          aux (chunk + 1)
        )
    in
    aux 0;;
val split_bytes : int -> bytes -> bytes Seq.t = <fun>
# "Hello world foo bar wooble"
  |> Bytes.of_string
  |> split_bytes 5
  |> List.of_seq;;
- : bytes list =
[Bytes.of_string "Hello"; Bytes.of_string " worl";
 Bytes.of_string "d foo"; Bytes.of_string " bar ";
 Bytes.of_string "woobl"; Bytes.of_string "e"]

Upvotes: 0

Jeffrey Scofield
Jeffrey Scofield

Reputation: 66818

Here's a function to split a bytes value into a list of bytes of size 65534 (except for the last of course).

let split_bytes b =
    let rec isplit sofar ib =
        let iblen = Bytes.length ib in
        if iblen > 65534 then
            let chunk = Bytes.sub ib 0 65534 in
            let rest = Bytes.sub ib 65534 (iblen - 65534) in
            isplit (chunk :: sofar) rest
        else
            ib :: sofar
    in
    List.rev (isplit [] b)

(It's worth thinking about passing the entire bytes value around in your code along with a count telling how much has been written so far. This would avoid the copying required to make the smaller chunks.)

Upvotes: 2

Related Questions