CODEWITHSUNDEEP
c++opencvimage-processinglarge-data
Freewister
Freewister

Reputation: 200

Can not load large image with opencv

I try to load an image with opencv that has the size 100.000 * 15.000 pixels and a file size of 4.305 kb. If i load this image with the following code:

   cv::Mat reallyBigImage = cv::imread("reallyBigImage.png");

i get the following error:

Error message

I have a pc with 32 gb ram and compile this program in 64 bit. This should be enough for an image of this size. Is it possible to load such an large image as whole to opencv?

Here is where my program breaks in assembler. The arrow indicates the specific location. This snippet is from the memcpy.asm.

    CopyUp:
    cmp     r8, 128
    jbe     XmmCopySmall

    bt      __favor, __FAVOR_ENFSTRG ; check for ENFSTRG (enhanced fast strings)
    jnc     XmmCopyUp               ; If Enhanced Fast String not available, use XMM

    ; use Enhanced Fast Strings
    ; but first align the destination dst to 16 byte alignment
    mov     rax, r11                ; return original destination pointer
    mov     r11, rdi                ; save rdi in r11
    mov     rdi, rcx                ; move destination pointer to rdi
    mov     rcx, r8                 ; move length to rcx
    mov     r8, rsi                 ; save rsi in r8
    mov     rsi, r10                ; move source pointer to rsi
    -->rep     movsb                   ; copy source to destination buffer
    mov     rsi, r8                 ; restore rsi
    mov     rdi, r11                ; restore rdi
    ret

Upvotes: 1

Views: 1399

Answers (3)

Mark Setchell
Mark Setchell

Reputation: 208087

May I commend libvips to you if you are dealing with large images? It is available for Linux, macOS and Windows - for free, from here.

If you have a 100,000 x 15,000 PNG image and you want to extract a region 2000x1000 starting at a point (100,100), you would do:

time vips crop test.png excerpt.png 2000 1000 100 100 --vips-leak

Sample Output

memory: high-water mark 192.29 MB

real    0m1.771s
user    0m0.826s
sys     0m0.082s

As you can see, it only used 200MB of RAM and took around 1s. The time and --vips-leak were for demonstration purposes only - you can omit them normally.

Upvotes: 1

Dan Mašek
Dan Mašek

Reputation: 19071

Running some local tests, this can be reproduced with OpenCV 3.1, but not with 3.2 or 3.3. That suggests it was a bug (along with the fact that you're getting an access violation in the first place).

Specifically issue #6317.

This was fixed prior to release of OpenCV 3.2, which would match the above observations. Hence, the best option would be to upgrade your copy of OpenCV.

Upvotes: 2

Scott Mudge
Scott Mudge

Reputation: 977

That image is 6 GB raw data (100,000 * 15,000 * 4 Bytes). Typically Windows limits the maximum heap size of a process much below that. But even if you increased that limit, you're going to encounter memory fragmentation (being unable to write to a single, 6GB contiguous block of memory), which OpenCV's allocator likely cannot handle.

Which is why you are getting access violation error during writing the image data to memory. It's attempting to write into a memory location it is not allowed/supposed to be writing in.

You're either going to have to split the image up into multiple ROIs/sub-images to avoid fragmentation, or write your own allocator that allocates the image in memory in smaller chunks.

Upvotes: 1

Related Questions