CODEWITHSUNDEEP
pythoncythonmemoryview
wvxvw
wvxvw

Reputation: 9469

How do I cast a slice of memoryview into C string (unsigned char*)?

Below is the code where I'm having this problem:

cpdef object encode_file(object fin, str fout):
    if not PyObject_CheckBuffer(fin):
        raise TypeError("fin must follow the buffer protocol")


    cdef Py_buffer in_view
    cdef int ret_code = PyObject_GetBuffer(fin, &in_view, PyBUF_SIMPLE)
    if ret_code < 0:
        raise TypeError("Couldn't get buffer from fin")

    cdef bytes py_filename = fout.encode()
    cdef char* cy_filename = py_filename
    cdef bytes py_mode = "w".encode()
    cdef const char* mode = py_mode
    cdef FILE* fd = fopen(<const char*>py_filename, <const char*>mode)
    if <size_t>fd == 0:
        raise FileNotFoundError(fout)

    cdef unsigned char out_buff[256]
    cdef size_t written = 0
    cdef size_t total_written = 0
    cdef size_t used = 0
    cdef size_t total_used = 0
    cdef size_t pad_start = 80

    cdef unsigned char[:] char_view = fin
    cdef unsigned char* char_slice

    while total_used < <size_t>in_view.len:
        char_view = char_view[used:]
        # This is the place where I get the error
        char_slice = char_view.buf
        used = encode_buffer(
            char_slice,
            in_view.len - used,
            out_buff,
            256,
            pad_start,
            80,
            &written,
        )
        pad_start = 80 - used % 80
        total_written += written
        total_used += used
        if fwrite(out_buff, sizeof(char), used, fd) != used:
            fclose(fd)
            raise Exception(
                "Couldn't write to file: {}. Bytes written: {}".format(
                    fout, total_used,
                ),
            )

    fclose(fd)

    print "used: {}, written: {}".format(used, total_written)
    return total_written

This may be a bit too much code for a simple example, but it's really not that much if you think about it. The part before the loop deals with filtering out all kinds of edge cases - they are of no interest for this question. The only important part is that the first argument must implement buffer protocol and the second argument is a file name.

So, in order to write to a file, I want to take a slice of memory view, and then pass it to a C function that expects a pointer to unsigned char. For the life of me, I cannot figure how to do this using Cython... I tried all sorts of permutations of the code above, but, in most cases I'm getting

Storing unsafe C derivative of temporary Python reference

Without any hints of what it was trying to generate.

There's also some duplication in the code above since I couldn't figure out how to use in_view.buf[x] and have it have the type that I need. I left it here just to show that I tried that too.

The answer given to the similar question doesn't work because Cython memory views have bugs. I'd appreciate a different answer.

Upvotes: 3

Views: 1087

Answers (1)

DavidW
DavidW

Reputation: 30888

Given that you don't seem to be able to use memoryviews because the data is read-only you could use the Py_Buffer object instead. The data is stored as a void* in in_view.buf. Cast it to a const char* with <const char*>(in_view.buf). You can get the nth element by simple pointer arithmetic (i.e. just add n to that value).

Since you've used PyBuf_SIMPLE you know the item size is 1 and the array is contiguous but in more complicated cases you might have to worry about this.

Upvotes: 1

Related Questions