CODEWITHSUNDEEP
common-lispsbcl
Ken
Ken

Reputation: 571

How to convert byte array to string in Common Lisp?

I'm calling a funny API that returns a byte array, but I want a text stream. Is there an easy way to get a text stream from a byte array? For now I just threw together:

(defun bytearray-to-string (bytes)
  (let ((str (make-string (length bytes))))
    (loop for byte across bytes
       for i from 0
       do (setf (aref str i) (code-char byte)))
    str))

and then wrap the result in with-input-from-string, but that can't be the best way. (Plus, it's horribly inefficient.)

In this case, I know it's always ASCII, so interpreting it as either ASCII or UTF-8 would be fine. I'm using Unicode-aware SBCL, but I'd prefer a portable (even ASCII-only) solution to a SBCL-Unicode-specific one.

Upvotes: 25

Views: 13762

Answers (6)

HD.
HD.

Reputation: 2155

I say go with the proposed flexistream or babel solutions.

But just for completeness and the benefit of future googlers arriving at this page I want to mention sbcl's own sb-ext:octets-to-string:

   SB-EXT:OCTETS-TO-STRING is an external symbol in #<PACKAGE "SB-EXT">.
   Function: #<FUNCTION SB-EXT:OCTETS-TO-STRING>
   Its associated name (as in FUNCTION-LAMBDA-EXPRESSION) is
     SB-EXT:OCTETS-TO-STRING.
   The function's arguments are:  (VECTOR &KEY (EXTERNAL-FORMAT DEFAULT) (START 0)
                                          END)
   Its defined argument types are:
     ((VECTOR (UNSIGNED-BYTE 8)) &KEY (:EXTERNAL-FORMAT T) (:START T) (:END T))
   Its result type is:
     *

Upvotes: 7

Vatine
Vatine

Reputation: 21258

If you don't have to worry about UTF-8 encoding (that, essentially, means "just plain ASCII"), you may be able to use MAP:

(map 'string #'code-char #(72 101 108 108 111))

Upvotes: 16

David Lichteblau
David Lichteblau

Reputation: 3561

There are two portable libraries for this conversion:

  • flexi-streams, already mentioned in another answer.

    This library is older and has more features, in particular the extensible streams.

  • Babel, a library specificially for character encoding and decoding

    The main advantage of Babel over flexi-streams is speed.

For best performance, use Babel if it has the features you need, and fall back to flexi-streams otherwise. Below a (slighly unscientific) microbenchmark illustrating the speed difference.

For this test case, Babel is 337 times faster and needs 200 times less memory.

(asdf:operate 'asdf:load-op :flexi-streams)
(asdf:operate 'asdf:load-op :babel)

(defun flexi-streams-test (bytes n)
  (loop
     repeat n
     collect (flexi-streams:octets-to-string bytes :external-format :utf-8)))

(defun babel-test (bytes n)
  (loop
     repeat n
     collect (babel:octets-to-string bytes :encoding :utf-8)))

(defun test (&optional (data #(72 101 108 108 111))
                       (n 10000))
  (let* ((ub8-vector (coerce data '(simple-array (unsigned-byte 8) (*))))
         (result1 (time (flexi-streams-test ub8-vector n)))
         (result2 (time (babel-test ub8-vector n))))
    (assert (equal result1 result2))))

#|
CL-USER> (test)
Evaluation took:
  1.348 seconds of real time
  1.328083 seconds of user run time
  0.020002 seconds of system run time
  [Run times include 0.12 seconds GC run time.]
  0 calls to %EVAL
  0 page faults and
  126,402,160 bytes consed.
Evaluation took:
  0.004 seconds of real time
  0.004 seconds of user run time
  0.0 seconds of system run time
  0 calls to %EVAL
  0 page faults and
  635,232 bytes consed.
|#

Upvotes: 24

dmitry_vk
dmitry_vk

Reputation: 4469

FLEXI-STREAMS (http://weitz.de/flexi-streams/) has portable conversion function

(flexi-streams:octets-to-string #(72 101 108 108 111) :external-format :utf-8)

=>

"Hello"

Or, if you want a stream:

(flexi-streams:make-flexi-stream
   (flexi-streams:make-in-memory-input-stream
      #(72 101 108 108 111))
   :external-format :utf-8)

will return a stream that reads the text from byte-vector

Upvotes: 33

Rainer Joswig
Rainer Joswig

Reputation: 139261

SBCL supports the so-called Gray Streams. These are extensible streams based on CLOS classes and generic functions. You could create a text stream subclass that gets the characters from the byte array.

Upvotes: 3

Charlie Martin
Charlie Martin

Reputation: 112366

Try the FORMAT function. (FORMAT NIL ...) returns the results as a string.

Upvotes: 0

Related Questions