CODEWITHSUNDEEP
cdebuggingbuffer-overflowinteger-overflow
Mark Ezberg
Mark Ezberg

Reputation: 655

Integer overflow while malloc memory

We have struct image pointer:

struct image {
uint16_t size_x;
uint16_t size_y;
struct pixel *px;
};

and:

  img->px = malloc(sizeof(struct pixel) * height * width);

where pixel:

struct pixel {
   uint8_t red;
   uint8_t green;
   uint8_t blue;
   uint8_t alpha;
   }

where width & height:

  long height = strtol(height_arg, &end_ptr, 10);
  long width = strtol(width_arg, &end_ptr, 10);

So, since we use malloc() to allocate memory and it uses size_t for allocating. Since we multiplying height and width which are long typed to allocate memory, is there integer overflow expected? If, yes then how to handle it?

Later on we iterate over picture and color it:

  for (int i = 0; i < img->size_y; i++) {
      for (int j = 0; j < img->size_x; j++) {
         image_data[i][j].red = palette[0].red;
         image_data[i][j].green = palette[0].green;
         image_data[i][j].blue = palette[0].blue;
         image_data[i][j].alpha = 0xff;
      }
  }

where

 img->size_x = width;
 img->size_y = height;

Upvotes: 1

Views: 1158

Answers (2)

klutt
klutt

Reputation: 31296

Yes, there is a risk.

You can check it like this:

struct pixel *px;    

if (height > SIZE_MAX / width || sizeof *px  > SIZE_MAX / (width * height)) {
    // Handle overflow
} 

px = malloc(sizeof *px * width * height);

if(!px) {
    // Handle allocation error
}

img->px = px;

Upvotes: 2

Adalcar
Adalcar

Reputation: 1458

Integer overflow may happen at every point of this program:

strtol("9999999999", &end_ptr, 10) -> will overflow and return LONG_MAX
height * width -> can overflow and return a negative value (since those are signed longs)
sizeof(struct pixel) * height * width 
-> can return a bigger value than UNSIGNED_MAX, wrapping around to a smaller value than expected

Malloc can allocate some very large data segments, and may not fail immediately, so you should seriously consider verifying your results before allocating.

Upvotes: 2

Related Questions