CODEWITHSUNDEEP
common-lispffisbcl
xiepan
xiepan

Reputation: 783

Is it possible passing bytes vector to C function without copying in SBCL

I'm trying to call C functions from SBCL with its FFI facility. The C functions require buffer of bytes (unsigned char) and process the bytes data. Unfortunately, in most cases the buffer is quite large. I'm wondering if I already have a Lisp bytes vector (a simple-array with element-type '(unsigned-byte 8)), is it possible just pass it to the C functions, rather than allocate alien buffers and copy the bytes there. I think this is a frequently recurring scenario, maybe SBCL team already cover that.

I look into the SBCL manual, it only states that sb-alien:c-string type might be passed directly to C functions without copying, and it requires the character be `base-char' (in the range 0 ~ 127 with unicode support). Wouldn't it be nice that we can also pass byte vectors without copying?

Upvotes: 5

Views: 203

Answers (2)

Tomas Zellerin
Tomas Zellerin

Reputation: 426

Technically, it is possible - at least if the vector is a simple octet array - using sb-kernel:get-lisp-obj-address, and you would also need sb-sys:with-pinned-objects probably. Other question is how wise idea it is (portability, internal changes in array layout in sbcl...).

Dirty example (ignores pinning, long vs. int, ...):

Let us make a function that would use the Lisp array:

extern char test(const char *t, int idx)
{
  return t[idx];
}

and then (after making and loading shared library) we can pass an address of a vector as an integer:

(sb-alien:alien-funcall
 (sb-c-call:extern-alien "test"
                         (function sb-alien:int sb-alien:unsigned-long sb-alien:unsigned-long))
 (sb-kernel:get-lisp-obj-address (make-array 5 :element-type '(unsigned-byte 8) :initial-contents '(10 20 30 40 50)))
 3)

would return third element (not fourth as would be zero based). The object is a bit more complicated. For example, you get array size (tagged) at -7 address.

Upvotes: 4

ignis volens
ignis volens

Reputation: 9252

I don't know whether this is possible in SBCL. To make it possible it would at least be necessary to allocate arrays in such a way that the GC doesn't move them. The SBCL manual contains a reference to a macro sb-sys:with-pinned-objects which might do this, but there doesn't seem to be any other information there.

However when I've needed to do something like this I've done it backwards: create a suitable object at the C level, and then wrap it with Lisp code to access it. The advantage of this is that you know how to deal with it from C, the disadvantage is that it's not an array, so you have to write your own access functions: things like read-sequence aren't going to work and so on.

I ended up using structures which wrapped alien vectors: here is a chunk of the code I used to deal with arrays of signed bytes:

(deftype index ()
  `(integer 0 (,most-positive-fixnum)))

...

(defstruct (foreign-octet-vector
            (:constructor %make-foreign-octet-vector (octets size)))
  (octets (error "no")
   :type (alien (* (signed 8)))
   :read-only t)
  (size 0
   :type index
   :read-only t))

(defun make-foreign-octet-vector (size)
  (%make-foreign-octet-vector
   (make-alien (signed 8) size)
   size))

(defun free-foreign-octet-vector (v)
  (free-alien (foreign-octet-vector-octets v))
  nil)

(declaim (inline fov-ref (setf fov-ref)))

(defun fov-ref (v n)
  (declare (type foreign-octet-vector v)
           (type index n))
  (if (< n (foreign-octet-vector-size v))
      (deref (foreign-octet-vector-octets v) n)
      (error "out of range")))

(defun (setf fov-ref) (value v n)
  (declare (type foreign-octet-vector v)
           (type index n)
           (type (signed-byte 8) value))
  (if (< n (foreign-octet-vector-size v))
      (setf (deref (foreign-octet-vector-octets v) n) value)
      (error "out of range")))

On top of this there was a fairly obvious macro using unwind-protect to make sure things got deallocated.

I don't know if this is either idiosyncratic SBCL code, or actually correct in general, but it worked for me. It may be that CFFI provides some better interface to all this.

Upvotes: 3

Related Questions