How can I improve performance of drawing of BufferedImage, created by ComponentColorModel in Java language?

Question

I'm developing a Java application, that captures video from dvi frame grabber. I want to render the frames, that I get from this frame grabber in the real time. But I faced the problem with the productivity - the rendering was very slow, my computer managed to render only 5-6 frames per second. Though the device was able to capture up to 40 frames per second. I profiled my code and found that drawImage method was relatively slow. If I called drawImage method with the BMP image, I loaded via ImageIO.readImage, than it took 20 ms to draw it using drawImage. With the images, I got from the frame grabber, It takes more than 100 ms to draw them. I studied the code of the frame grabber library, and found that the BufferedImage was created in the following way -

    ColorModel cm;
    if (format == PixelFormat.RGB24) {
       cm = new ComponentColorModel(
            ColorSpace.getInstance(ColorSpace.CS_sRGB), new int[] {8,8,8},
            false, false, Transparency.OPAQUE, DataBuffer.TYPE_BYTE);
    } else {
        throw new UnsupportedOperationException();
    }
    SampleModel sm = cm.createCompatibleSampleModel(width, height);
    DataBuffer db = new DataBufferByte(pixels, length);
    WritableRaster raster = Raster.createWritableRaster(sm, db, null);
    return new BufferedImage(cm, raster, false, null);

pixels is an array of bytes, that is provided by the frame grabber. Is there any way to create BufferedImage differentelely, to make the call to drawImage method faster. I know about ColorSpaces, I do not need to preserve the correct color space. Speed matters much more.

Thanks in advance

Answer 1

I had an issue with jai's tiff implementation for the ImageIO API which sounds similar (I don't know if this will help or not).

Basically, it will convert the color model to a "single pixel packed sample model" :P

This is not my code, I take no credit for it, I found it on the net some time ago and I'm afraid I don't recall where (I did try searching for it, but didn't find a suitable reference.

/*******************************************************************************
 *
 * It seems that SinglePixelPackedSampleModel is the only fast mode when a
 * color profile is converted. This is probably a bug (that has nothing to do
 * with bugs 4886071 and 4705399).
 * Note that grayscale images (TYPE_GRAY) are not converted.
 *
 ******************************************************************************/
public static BufferedImage convertToSinglePixelPackedSampleModel(BufferedImage image) {

    long time = System.currentTimeMillis();

    WritableRaster sourceRaster = image.getRaster();
    ColorModel colorModel = image.getColorModel();
    ICC_ColorSpace colorSpace = (ICC_ColorSpace) colorModel.getColorSpace();
    final SampleModel ssmd = sourceRaster.getSampleModel();

    if (colorSpace.getType() == ColorSpace.TYPE_GRAY) {

        logger.info(">>>> TYPE_GRAY, not converting");

    } else if (!(ssmd instanceof PixelInterleavedSampleModel)) {

        logger.info(">>>> sourceSampleModel is " + ssmd.getClass() + ", not converting");

    } else {

        PixelInterleavedSampleModel sourceSampleModel = (PixelInterleavedSampleModel) ssmd;
        int[] bitMasks = new int[]{0x00ff0000, 0x0000ff00, 0x000000ff};

        SinglePixelPackedSampleModel sampleModel = new SinglePixelPackedSampleModel(DataBuffer.TYPE_INT, image.getWidth(),
                image.getHeight(), bitMasks);

        WritableRaster destRaster = Raster.createWritableRaster(sampleModel, null);
        DataBufferInt destDataBuffer = (DataBufferInt) destRaster.getDataBuffer();
        int[] destBuffer = destDataBuffer.getData();
        int[] bandOffsets = sourceSampleModel.getBandOffsets();

        for (int i = 0; i < bandOffsets.length; i++) {
            bandOffsets[i] += ((-sourceRaster.getSampleModelTranslateX() * sourceSampleModel.getPixelStride())
                    - (sourceRaster.getSampleModelTranslateY() * sourceSampleModel.getScanlineStride()));
        }

        DataBuffer sourceDataBuffer = sourceRaster.getDataBuffer();

        if (sourceDataBuffer instanceof DataBufferUShort) {

            convertUShortDataBuffer(image, (DataBufferUShort) sourceDataBuffer, sourceSampleModel, bandOffsets, destBuffer);

        } else if (sourceDataBuffer instanceof DataBufferByte) {

            convertByteDataBuffer(image, (DataBufferByte) sourceDataBuffer, sourceSampleModel, bandOffsets, destBuffer);

        } else {

            throw new IllegalArgumentException("Cannot deal with " + sourceDataBuffer.getClass());

        }

        String sourceProfileName = getICCProfileName(colorSpace.getProfile());

        if (sourceProfileName.equals("Nikon sRGB 4.0.0.3001")) {
            logger.warn(">>>> Workaround #1094403: using sRGB instead of " + sourceProfileName);
            colorSpace = new ICC_ColorSpace(ICC_Profile.getInstance(ColorSpace.CS_LINEAR_RGB));
        }

        colorModel = new DirectColorModel(colorSpace, 24, bitMasks[0], bitMasks[1], bitMasks[2], 0, false, DataBuffer.TYPE_INT);
        image = new BufferedImage(colorModel, destRaster, false, null);
    }

    time = System.currentTimeMillis() - time;
    logger.info(">>>> convertToSinglePixelPackedSampleModel() completed ok in " + time + " msec");

    return image;
}

/**
 * @param image
 * @param sourceDataBuffer
 * @param sourceSampleModel
 * @param bandOffsets
 * @param destBuffer
 */
protected static void convertByteDataBuffer(BufferedImage image, DataBufferByte sourceDataBuffer,
        PixelInterleavedSampleModel sourceSampleModel, int[] bandOffsets, int[] destBuffer) {
    int base = 0;
    int i = 0;
    byte[] sourceBuffer = sourceDataBuffer.getData();
    int pixelStride = sourceSampleModel.getPixelStride();

    for (int y = 0; y < image.getHeight(); y++) {
        int j = base;

        for (int x = 0; x < image.getWidth(); x++) {
            int r = (sourceBuffer[j + bandOffsets[0]] & 0xff);
            int g = (sourceBuffer[j + bandOffsets[1]] & 0xff);
            int b = (sourceBuffer[j + bandOffsets[2]] & 0xff);

            destBuffer[i++] = (r << 16) | (g << 8) | b;
            j += pixelStride;
        }

        base += sourceSampleModel.getScanlineStride();
    }
}

protected static void convertUShortDataBuffer(BufferedImage image, DataBufferUShort sourceDataBuffer,
        PixelInterleavedSampleModel sourceSampleModel, int[] bandOffsets, int[] destBuffer) {
    int base = 0;
    int i = 0;
    short[] sourceBuffer = sourceDataBuffer.getData();

    for (int y = 0; y < image.getHeight(); y++) {
        int j = base;

        for (int x = 0; x < image.getWidth(); x++) {
            int r = (sourceBuffer[j + bandOffsets[0]] & 0xffff) >> 8;
            int g = (sourceBuffer[j + bandOffsets[1]] & 0xffff) >> 8;
            int b = (sourceBuffer[j + bandOffsets[2]] & 0xffff) >> 8;

            destBuffer[i++] = (r << 16) | (g << 8) | b;
            j += 3;
        }

        base += sourceSampleModel.getScanlineStride();
    }
}

//    public static ICC_Profile getICCProfile(RenderedImage image) {
//
//        ColorSpace colorSpace = image.getColorModel().getColorSpace();
//
//        if (colorSpace instanceof ICC_ColorSpace) {
//
//            ICC_ColorSpace iccColorSpace = (ICC_ColorSpace) colorSpace;
//
//            return iccColorSpace.getProfile();
//
//        }
//
//        return null;
//
//    }

public static String getICCProfileName(ICC_Profile profile) {

    if (profile == null) {
        return null;
    }

    byte[] xx = profile.getData(ICC_Profile.icSigProfileDescriptionTag);
    int offset = 12;
    int count;

    for (count = 1; xx[offset + count] != 0; count++) {
        ;
    }

    return new String(xx, 0, offset, count);

}

Basically, just call .convertToSinglePixelPackedSampleModel(image).

I took the rendering time of a (relatively large TIFF image) down from minutes to less then a few seconds :P

ps - I think this where I found the original code http://www.koders.com/java/fidFE1D69AFE6930A514D5E189310AB10A3DFD43F78.aspx