A program that cuts GIFs into frames and converts them into pixels is using too much memory

Question

I wrote a code which splits a gif-image into frames. Then, they are scaled-down into smaller ones, the resolution is 128 * 128 pixels. Then they combine a full frame with a specific resolution.

The getFrames (String path) method gets the GIF file along the path from the jar and creates image frames.

The getResizedimages (…) method cuts the frame into smaller 128 by 128 images.

The getPixels (Bufferedlmage image) method gets an array of pixels from an image.

The problem is that during the execution of this code (via the test method), a very very large amount of memory is used even at the stage of execution of the getFrames (…) method or the getFramesRaw (…) method.

import com.sun.imageio.plugins.gif.GIFImageReader;
import com.sun.imageio.plugins.gif.GIFImageReaderSpi;
import org.apache.commons.io.FileUtils;
import org.bukkit.map.MapPalette;

import javax.imageio.ImageIO;
import javax.imageio.ImageReader;
import javax.imageio.stream.ImageInputStream;
import java.awt.*;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;

public class Gif {

static void test(String path, int height, int width) {
    List<BufferedImage> images = getFrames(path);
    for (int x = 0; x < width; x ++) for (int y = 0; y < height; y ++)
        getResizedFrames(images, width, height, x, y).forEach(image -> getPixels(image));
}

@SuppressWarnings("deprecation")
public static byte[] getPixels(BufferedImage image) {
    int pixelCount = image.getWidth() * image.getHeight();
    int[] pixels = new int[pixelCount];
    image.getRGB(0, 0, image.getWidth(), image.getHeight(), pixels, 0, image.getWidth());
    byte[] colors = new byte[pixelCount];
    for (int i = 0; i < pixelCount; i++)
        colors[i] = MapPalette.matchColor(new Color(pixels[i], true));
    return colors;
}

public static BufferedImage getScaledImage(BufferedImage image) {
    BufferedImage newImg = new BufferedImage(128, 128, 3);
    Graphics2D g2 = newImg.createGraphics();
    g2.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR);
    g2.drawImage(image, 0, 0, 128, 128, null);
    g2.dispose();
    return newImg;
}

public static BufferedImage cropImage(BufferedImage completeImg, int width, int height, int xCoord, int yCoord) {
    int coordWidth = completeImg.getWidth() / width;
    int coordHeight = completeImg.getHeight() / height;
    BufferedImage croppedImg = new BufferedImage(completeImg.getWidth(), completeImg.getHeight(), 1);
    Graphics2D g2 = (Graphics2D)croppedImg.getGraphics();
    g2.drawImage(completeImg, 0, 0, completeImg.getWidth(), completeImg.getHeight(), null);
    g2.dispose();
    BufferedImage image = croppedImg.getSubimage(coordWidth * xCoord, coordHeight * yCoord, coordWidth, coordHeight);
    return getScaledImage(image);
}

public static java.util.List<BufferedImage> getFramesRaw(String path) {
    java.util.List<BufferedImage> frames = new ArrayList<>();
    try {
        File file = File.createTempFile("data", ".gif");
        InputStream stream = Main.class.getResourceAsStream(path);
        FileUtils.copyInputStreamToFile(stream, file);
        stream.close();

        ImageInputStream inputStream = ImageIO.createImageInputStream(file);
        ImageReader ir = new GIFImageReader(new GIFImageReaderSpi());
        ir.setInput(inputStream);

        int number = ir.getNumImages(true);
        for (int i = 0; i < number; i++) frames.add(ir.read(i));
        inputStream.close();
        System.gc();
    } catch (Exception e) {
        e.printStackTrace();
    }
    return frames;
}

public static java.util.List<BufferedImage> getFrames(String path) {
    java.util.List<BufferedImage> copies = new ArrayList<>();
    java.util.List<BufferedImage> frames = getFramesRaw(path);
    copies.add(frames.remove(0));
    int width = copies.get(0).getWidth(), height = copies.get(0).getHeight();
    for (BufferedImage frame : frames) {
        BufferedImage img = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
        Graphics g = img.getGraphics();
        g.drawImage(copies.get(copies.size()-1),0,0,null);
        g.drawImage(frame,0,0,null);
        copies.add(img);
    }
    return copies;
}

public static java.util.List<BufferedImage> getResizedFrames(java.util.List<BufferedImage> frames, int width, int height, int x, int y) {
    List<BufferedImage> copies = new ArrayList<>();
    for (BufferedImage image : frames)
        copies.add(cropImage(image, width, height, width - 1 - x, height - 1 - y));
    return copies;
}

}

I need your help.

Answer 1

The reason your program uses a lot of memory, is because your code operates at all the frames at each step (in many small loops). All the decoded image data for all the frames will be held in memory at the same time, resulting in unnecessary memory usage. While this may feel logical, it is for sure not efficient.

Instead, create one loop that does all the needed operations for a single frame, before moving on to the next.

Something like this:

public static void test(String path) throws IOException {
    List<BufferedImage> thumbnails = new ArrayList<>();

    readFrames(path, image -> thumbnails.add(getScaledImage(image)));
}

private static void readFrames(String path, Consumer<BufferedImage> forEachFrame) throws IOException {
    try (ImageInputStream inputStream = ImageIO.createImageInputStream(GifSplitter.class.getResourceAsStream(path))) {
        Iterator<ImageReader> readers = ImageIO.getImageReaders(inputStream); // ImageIO detects format, no need to hardcode GIF plugin
        if (!readers.hasNext()) {
            return;
        }

        ImageReader ir = readers.next();
        ir.setInput(inputStream);

        // For GIF format (which does not contain frame count in header),
        // it's more efficient to just read each frame until IOOBE occurs,
        // instead of invoking getNumImages(true)
        int i = 0;
        while (i >= 0) {
            try {
                BufferedImage image = ir.read(i++);

                // crop, scale, etc, for a SINGLE image
                forEachFrame.accept(image);
            }
            catch (IndexOutOfBoundsException endOfGif) {
                // No more frames
                break;
            }
        }

        ir.dispose();
    }
}