CODEWITHSUNDEEP
delphiresizejpeg
Alex
Alex

Reputation: 57

Resize huge jpeg using no memory

I need to resize huge (up to 30000x30000) JPEG files using no RAM, speed doesn't matter, is there any way to do so? I tried different libraries (nativejpg and others) but they use all free RAM and and crash with errors like "Out of memory" or "Not enough storage is available to process this command". I even tried command line utility imagemagick, but it also uses gigabytes of memory.

Upvotes: 1

Views: 1679

Answers (3)

jcupitt
jcupitt

Reputation: 11200

You can do this with imagemagick if you turn on libjpeg shrink-on-load. Try:

$ identify big.jpg 
big.jpg JPEG 30000x30000 30000x30000+0+0 8-bit sRGB 128MB 0.000u 0:00.000
$ time convert -define jpeg:size=2500x2500 big.jpg -resize 2500x2500 small.jpg
real    0m3.169s
user    0m2.999s
sys 0m0.159s
peak mem: 170MB

How this works: libjpeg has a great shrink-on-load feature. When you open an image, you can ask the library to downsample by x2, x4 or x8 during the loading process -- the library then just decodes part of each DCT block.

However, this feature must be enabled when the image is opened, you can't set it later. So convert needs a hint that when it opens big.jpg, it only needs to get an image of at least size 2500x2500 (your target size). Now all -resize is doing is shrinking a 3800x3800 pixel image down to 2500x2500, a pretty easy operation. You'll only need 1/64th of the CPU and memory.

As @mark-setchell said above, vipsthumbnail is even faster:

$ time vipsthumbnail big.jpg -s 2500 -o small.jpg --vips-leak
memory: high-water mark 29.93 MB
real    0m2.362s
user    0m2.873s
sys 0m0.082s

Though the speedup is not very dramatic, since both systems are really just resizing 3800 -> 2500.

If you try tif instead, you do see a large difference, since there's no shrink-on-load trick you can use:

$ identify 360mp.tif 
360mp.tif TIFF 18000x18000 18000x18000+0+0 8-bit sRGB 972MB 0.000u 0:00.000
$ time convert 360mp.tif -resize 2500 x.tif
peak mem: 2.8 GB
real    0m8.397s
user    0m25.508s
sys 0m1.648s
$ time vipsthumbnail 360mp.tif -o x.tif -s 2500 --vips-leak
memory: high-water mark 122.08 MB
real    0m2.583s
user    0m9.012s
sys 0m0.308s

Now vipsthumbnail is about 4x faster and needs only 1/20th of the memory.

Upvotes: 2

Dalija Prasnikar
Dalija Prasnikar

Reputation: 28512

With built in Delphi Jpeg support you can load large jpeg image resampled to smaller size while loading without excessive usage of RAM.

Jpeg image Scale property can have following values jsFullSize, jsHalf, jsQuarter, jsEighth

procedure ScaleJpg(const Source, Dest: string);
var
  SourceImg, DestImg: TJPEGImage;
  Bmp: TBitmap;
begin
  Bmp := TBitmap.Create;
  try
    SourceImg := TJPEGImage.Create;
    try
      SourceImg.Scale := jsEighth;
      SourceImg.LoadFromFile(Source);
      Bmp.Width := SourceImg.Width;
      Bmp.Height := SourceImg.Height;
      Bmp.Canvas.Draw(0, 0, SourceImg);
    finally
      SourceImg.Free;
    end;
    DestImg := TJPEGImage.Create;
    try
      DestImg := TJPEGImage.Create;
      DestImg.Assign(Bmp);
      DestImg.SaveToFile(Dest);
    finally
      DestImg.Free;
    end;
  finally
    Bmp.Free;
  end;
end;

Once you have roughly rescaled image to size that can be comfortably processed in memory you can apply ordinary scaling algorithms to get the actual image size you want.

Upvotes: 1

Mark Setchell
Mark Setchell

Reputation: 207660

I would suggest you have a look at vips. It is documented here.

I can create a 10000x10000 image of noise like this with ImageMagick

convert -size 10000x10000! xc:gray50 +noise poisson image.jpg

and check it is the correct size like this:

identify image.jpg
image.jpg JPEG 10000x10000 10000x10000+0+0 8-bit sRGB 154.9MB 0.000u

I can now use vips to resize the 10000x10000 image down to 2500x2500 like this

time vipsthumbnail image.jpg -s 2500 -o small.jpg --vips-leak
memory: high-water mark 20.48 MB

real    0m1.974s
user    0m2.158s
sys     0m0.096s

Note the memory usage peaked at just 20MB

Check the result like this with ImageMagick

identify result.jpg
result.jpg JPEG 2500x2500 2500x2500+0+0 8-bit sRGB 1.33MB 0.000u 0:00.000

Have a look at the Technical Note too, regarding performance and memory usage - here.

You can also call it from C as well as the command line.

Upvotes: 3

Related Questions