Python PIL efficienty glueing images together, while keeping image optimizations

Question

TL;DR : When concatenating 10mb worth of images into one large image, resulting image is 1GB worth of memory, before I save/optimize it to disk. How can make this in-memory size smaller?

I am working on a project where I am taking a list of lists of Python Pil image objects (image tiles), and gluing them together to:

1. Generate a list of images that have been concatenated together into columns
2. Taking #1, and making a full blown image out of all the tiles

This post has been great at providing a function that accomplishes 1&2 by

1. Figuring out the final image size
2. Creating a blank canvas for images to be added to
3. Adding all the images, in a sequence, to canvas we just generated

However, the issue I am encountering with the code:

1. The size of the original objects in the list of lists, is ~50mb.
2. When I do the first past over the list of lists of image object, to generated list of images that are columns, the memory increases by 1gb... And when I make the final image, the memory increases by another 1gb.

Since the resulting image is 105,985 x 2560 pixels... the 1gb is somewhat expected ((105984*2560)*3 /1024 /1024) [~800mb]

My hunch is that the canvases that are being created, are non-optimized, hence, take up a bit of space (pixels * 3 bytes), but the image tile objects I am trying to paste onto canvas, are optimized for size.

Hence my question - utilizing PIL/Python3, is there a better way to concatenate images together, keeping their original sizes/optimizations? After I do process image/re-optimize it via

.save(DiskLocation, optimize=True, quality=94)

The resulting image is ~30 MB (which is, roughly the size of the original list of lists containing PIL objects)

For reference, from the post linked above, this is the function that I use to concatenate images together:

from PIL import Image

#written by teekarna
# https://stackoverflow.com/questions/30227466/combine-several-images-horizontally-with-python

def append_images(images, direction='horizontal',
                  bg_color=(255,255,255), aligment='center'):
    """
    Appends images in horizontal/vertical direction.

    Args:
        images: List of PIL images
        direction: direction of concatenation, 'horizontal' or 'vertical'
        bg_color: Background color (default: white)
        aligment: alignment mode if images need padding;
           'left', 'right', 'top', 'bottom', or 'center'

    Returns:
        Concatenated image as a new PIL image object.
    """
    widths, heights = zip(*(i.size for i in images))

    if direction=='horizontal':
        new_width = sum(widths)
        new_height = max(heights)
    else:
        new_width = max(widths)
        new_height = sum(heights)

    new_im = Image.new('RGB', (new_width, new_height), color=bg_color)


    offset = 0
    for im in images:
        if direction=='horizontal':
            y = 0
            if aligment == 'center':
                y = int((new_height - im.size[1])/2)
            elif aligment == 'bottom':
                y = new_height - im.size[1]
            new_im.paste(im, (offset, y))
            offset += im.size[0]
        else:
            x = 0
            if aligment == 'center':
                x = int((new_width - im.size[0])/2)
            elif aligment == 'right':
                x = new_width - im.size[0]
            new_im.paste(im, (x, offset))
            offset += im.size[1]

    return new_im

Answer 1

While I do not have explanation for what was causing my runaway memory issue, I was able to tack on some code that seemed to have fixed the issue.

For each tile that I am trying to glue together, I ran a 'resizing' script (below). Somehow, this fixed the issue I was having ¯\ (ツ) /¯

Resize images script:

def resize_image(image, ImageQualityReduction = .78125):
    #resize image, by percentage
    width, height = image.size
    #print(width,height)
    new_width =  int(round(width * ImageQualityReduction))
    new_height = int(round(height * ImageQualityReduction))
    resized_image =  image.resize((new_width, new_height), Image.ANTIALIAS)
    return resized_image#, new_width, new_height