Unknown bytes while reading an image from ImageIO after sending it with ImageIO

Question

While transfering images over the network using sockets, I had a -for me- strange issue:
When I wrote images to the OutputStream of one socket with ImageIO.write() and read the same images from the InputStream of the other socket with ImageIO.read() I noticed, that 16 bytes per image were sent more than read. To be able to send multiple images in a row I had to read these bytes after every call of ImageIO.read() to not receive null because the input could not be parsed.

Does anybody know, why this is so and what these bytes are?

In this piece of code I have extracted the issue:

public class Test implements Runnable
{
    public static final int COUNT = 5;

    public void run()
    {
        try(ServerSocket server = new ServerSocket(3040))
        {
            Socket client = server.accept();
            for(int i = 0; i < COUNT; i++)
            {
                final BufferedImage image = readImage(client.getInputStream());
                System.out.println(image);
            }
        } 
        catch(IOException e)
        {
            e.printStackTrace();
        }
    }

    private BufferedImage readImage(InputStream stream) throws IOException
    {
        BufferedImage image = ImageIO.read(stream);

        dontKnowWhy(stream);

        return image;
    }

    private void dontKnowWhy(InputStream stream) throws IOException
    {
        stream.read(new byte[16]);
    }

    public static void main(String... args)
    {
        new Thread(new Test()).start();

        try(Socket server = new Socket("localhost", 3040))
        {
            for(int i = 0; i < COUNT; i++)
            {
                BufferedImage image = new BufferedImage(300, 300, BufferedImage.TYPE_INT_ARGB); //
                int[] vals = new int[image.getWidth() * image.getHeight()];                     //
                Arrays.fill(vals, new Random().nextInt());                                      // Create random image
                image.setRGB(0, 0, image.getWidth(), image.getHeight(), vals, 0, 1);            //

                ImageIO.write(image, "png", server.getOutputStream());  //send image to server

                long time = System.currentTimeMillis();             //
                while(time + 1000 > System.currentTimeMillis());    //wait a second
            }
        }
        catch(IOException e)
        {
            e.printStackTrace();
        }

    }
}

I am glad about any answers, already thank you!

Answer 1

The "extra" bytes you see, is not read, simply because they are not needed to correctly decode the image (they are, however, most likely needed to form a fully compliant file in the chosen file format, so they are not just random "garbage" bytes).

For any given ImageIO plugin, the number of bytes left in the stream after a read may be 0, 16or any other number. It might depend on the format, the writer that wrote it, the reader, the number of images in the input, the metadata in the file, etc. In other words, relying on this behavior would be an error.

The easies way to fix this, is to prepend each image with a byte count, containing the length of the output image. This typically means you need to buffer the response on the client, to either a ByteArrayOutputStream (in-memory) or a FileOutputStream (disk).

The client then needs to read the byte count for the image, and make sure you skip any remaining bytes after the read. This can be accomplished by wrapping the input (see FilterInputStream) and keep track of the byte count internally.

(You can also read all the bytes up front, and wrapping them in a ByteArrayInputStream, before passing the data to ImageIO.read(), which is simpler but does more in-memory buffering).

After this, the client is ready do start over, by reading a new byte count, and a new image.

Another approach if you'd like less buffering on the server, could be to implement something like HTTP chunked transfer encoding, where you have multiple smaller blocks (chunks) sent to the client for each image, each prepended with its own byte count. You would need to handle the last chunk of each image especially, or insert special delimiter chunks to mark end of stream or start of a new stream.

Code below implements the buffering approach on the server, while using direct reading on the client.

Server:

DataOutputStream stream = new DataOutputStream(server.getOutputStream());

ByteArrayOutputStream buffer = new ByteArrayOutputStream();
for (...) {
    buffer.reset();
    ImageIO.write(image, "png", buffer);

    stream.writeInt(buffer.size());
    buffer.writeTo(stream); // Send image to server
}

Client:

DataInputStream stream = new DataInputStream(client.getInputStream());

for (...) {
    int size = stream.readInt();

    try (InputStream imageData = new SubStream(stream, size)) {
        return ImageIO.read(imageData);
    }
    // Note: imageData implicitly closed using try-with-resources
}

...

// Util class
private static final class SubStream extends FilterInputStream {
    private final long length;
    private long pos;

    public SubStream(final InputStream stream, final long length) {
        super(stream);

        this.length = length;
    }

    @Override
    public boolean markSupported() {
        return false;
    }

    @Override
    public int available() throws IOException {
        return (int) Math.min(super.available(), length - pos);
    }

    @Override
    public int read() throws IOException {
        if (pos++ >= length) {
            return -1;
        }

        return super.read();
    }

    @Override
    public int read(byte[] b, int off, int len) throws IOException {
        if (pos >= length) {
            return -1;
        }

        int count = super.read(b, off, (int) Math.min(len, length - pos));

        if (count < 0) {
            return -1;
        }

        pos += count;

        return count;
    }

    @Override
    public long skip(long n) throws IOException {
        if (pos >= length) {
            return -1;
        }

        long skipped = super.skip(Math.min(n, length - pos));

        if (skipped < 0) {
            return -1;
        }

        pos += skipped;

        return skipped;
    }

    @Override
    public void close() throws IOException {
        // Don't close wrapped stream, just consume any bytes left
        while (pos < length) {
            skip(length - pos);
        }
    }
}