Merging 13 bits array into an array of unsigned char

Question

I'm writing an algorithm that compresses data (LZSS) and it requires me to have two 13-bit values which I'll have to later merge together.

In some cases, however, I don't need 13 bits; 8 are enough. For this purpose I have a structure like this:

typedef struct pattern
{
    char is_compressed:1; //flag
    short index :13; //first value
    short length :13;  //second value
    unsigned char c;   //is 8 bits are enough, use this instead
} Pattern;

I therefore have an array of these structures, and each structure can either contain the two 13-bit values or an 8-bit value.

I am now looping over this array, and my objective is to merge all these bits together.

I easily calculated the total number of bits used and the number of arrays of unsigned chars (8 bits) needed in order to store all the values:

int compressed = 0, plain = 0;
  //count is the amount of patterns i have and p is the array of patterns (the structures)
for (int i = 0; i < count; i++)
{
    if (p[i]->is_compressed)
        compressed++;
    else
        plain++;
}
  //this stores the number of bits used in the pattern (13 for length and 13 for the index or 8 for the plain uchar)
int tot_bits = compressed * 26 + plain * 8;
  //since we can only write a minimum of 8 bits, we calculate how many arrays are needed to store the bits
int nr_of_arrays = (tot_bits % 8 == 0) ? tot_bits / 8 : (tot_bits / 8) + 1;
  //we allocate the needed memory for the array of unsigned chars that will contain, concatenated, all the bits
unsigned char* uc = (unsigned char*) malloc(nr_of_arrays * sizeof(unsigned char));

After allocating the memory for the array I'm going to fill, I simply loop through the array of structures and recognize whether the structure I'm looking at contains the two 13-bit values or just the 8-bit one

for (int i = 0; i < count; i++)
{
    if (p->is_compressed)
    {
        //The structure contains the two 13 bits value
    } 
    else
    {
        //The structure only contains the 8 bits value
    }
}

Here I'm stuck and can't seem to figure out a proper way of getting the job done.

Does anybody of you know how to implement that part there?

---

A practical example would be:

pattern 1 contains the 2 13-bit values:

1111 1111 1111 1
0000 0000 0000 0

pattern 2 contains the 8-bit value

1010 1010

total bits: 34
number of arrays required: 5 (that will waste 6 bits)

resulting array is:

[0] 1111 1111
[1] 1111 1000
[2] 0000 0000
[3] 0010 1010
[4] 1000 0000 (the remaining 6 bits are set to 0)

Answer 1

The problem intrigued me. Here's a possible implementation of "by using a lot of bitwise operations":

/* A writable bit string, with an indicator of the next available bit */
struct bitbuffer {
    uint8_t *bytes;
    size_t next_bit;
};

/*
 * writes the bits represented by the given pattern to the next available
 * positions in the specified bit buffer
 */
void write_bits(struct bitbuffer *buffer, Pattern *pattern) {
    /* The index of the byte containing the next available bit */
    size_t next_byte = buffer->next_bit / 8;
    /* the number of bits already used in the next available byte */
    unsigned bits_used = buffer->next_bit % 8;

    if (pattern->is_compressed) {
        /* assemble the bits to write in a 32-bit block */
        uint32_t bits = pattern->index << 13 + pattern->length;

        if (bits_used == 7) {
            /* special case: the bits to write will span 5 bytes */

            /* the first bit written will be the last in the current byte */
            uint8_t first_bit = bits >> 25;

            buffer->bytes[next_byte] |= first_bit;

            /* write the next 8 bits to the next byte */
            buffer->bytes[++next_byte] = (bits >> 17) & 0xFF;

            /* align the tail of the bit block with the buffer*/
            bits <<= 7;
        } else {

            /* the first bits written will fill out the current byte */
            uint8_t first_bits = (bits >> (18 + bits_used)) & 0xFF;

            buffer->bytes[next_byte] |= first_bits;

            /* align the tail of the bit block with the buffer*/
            bits <<= (6 - bits_used);
        }

        /*
         * Write the remainder of the bit block to the buffer,
         * most-significant bits first. Three (more) bytes will be modified.
         */
        buffer->bytes[++next_byte] = (bits >> 16) & 0xFF;
        buffer->bytes[++next_byte] = (bits >>  8) & 0xFF;
        buffer->bytes[++next_byte] =  bits        & 0xFF;

        /* update the buffer's index of the next available bit */
        buffer->next_bit += 26;
    } else {  /* the pattern is not compressed */
        if (bits_used) {
            /* the bits to write will span two bytes in the buffer */
            buffer->bytes[next_byte] |= (pattern->c >> bits_used);
            buffer[++next_byte] = (pattern->c << bits_used) & 0xFF;
        } else {
            /* the bits to write exactly fill the next buffer byte */
            buffer->bytes[next_byte] = pattern->c;
        }

        /* update the buffer's index of the next available bit */
        buffer->next_bit += 8;
    }
}

Answer 2

One way to do that is to write bytes one by one and keep track of partial bytes as you write.

You need a pointer to your char array, and an integer to keep track of how many bits you wrote to the last byte. Every time you write bits, you check how many bits you can write to the last byte, and you write these bits accordingly (ex: if there is 5 bits free, you shift your next value by 3 and add it to the last byte). Every time a byte is complete, you increment your array pointer and reset your bit tracker.

A clean way to implement this would be to write functions like :

void BitWriter_init( char *myArray );
void BitWriter_write( int theBitsToWrite, int howManyBits );

Now you just have to figure out how to implement these functions, or use any other method of your choice.