CODEWITHSUNDEEP
emscriptenwebassembly
Brandon Petty
Brandon Petty

Reputation: 635

Why in WebAssembly does ALLOW_MEMORY_GROWTH=1 fail while TOTAL_MEMORY=512MB succeeds?

I have an image processing Wasm project that applies different binarization algorithms to a given image. One of those algorithms was producing this error when I would run it:

Uncaught abort("Cannot enlarge memory arrays to size 17100800 bytes (OOM). Either
(1) compile with  -s TOTAL_MEMORY=X  with X higher than the current value 16777216,
(2) compile with  -s ALLOW_MEMORY_GROWTH=1  which allows increasing the size at runtime, or
(3) if you want malloc to return NULL (0)
instead of this abort, compile with  -s ABORTING_MALLOC=0 ") at Error

After compiling with "-s ALLOW_MEMORY_GROWTH=1", the algorithm did not error out in Chrome, but turned the image black. Running it a second time in a row gave this error:

Uncaught RuntimeError: memory access out of bounds

On the first run in Edge 42.17134.1.0 I get this error:

SCRIPT5147: The ArrayBuffer is detached.

Interesting enough, removing that option and replacing it with "-s TOTAL_MEMORY=512MB" fixed the problem.

So my question is this: Isn't ALLOW_MEMORY_GROWTH=1 suppose to, at runtime, expand heap memory for me automatically and almost be a replacement for TOTAL_MEMORY? I hate the idea of locking down at compile time the max memory my app can use. Tested with Chrome 73.0.3683.103/74.0.3729.108 (Official Build) (64-bit) and Firefox 66.0.3 (64-bit).

Edit I was able to isolate the issue with the following code.

Compiled for debug:

em++ -O0 -o ./dist/doxaWasm.js doxaWasm.cpp -std=c++1z -s WASM=1 -s NO_EXIT_RUNTIME=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['writeAsciiToMemory']" -g4 --source-map-base http://localhost:8080/ -s ALLOW_MEMORY_GROWTH=1

C++ Code Snippet:

extern "C"
{
    void EMSCRIPTEN_KEEPALIVE Initialize(uint8_t* data, const int width, const int height)
    {
        // Large enough to force a reallocation.
        // If there is no reallocation, everything works.
        std::vector<int64_t> integral_image1(width*height*10);

        data[0] = 123;
    }
}

Javascript Code Snippet:

...
var size = width * height;
var heapPtr = Module._malloc(size);
var data = new Uint8ClampedArray(Module.HEAPU8.buffer, heapPtr, size);
... // Populate 'data' with 8bit grayscale based on canvas ImageData
Module._Initialize(data.byteOffset, width, height);
console.log(data[0]); // Equals 123 if there is no reallocation

Upvotes: 8

Views: 8303

Answers (1)

Brandon Petty
Brandon Petty

Reputation: 635

I figured out what was happening. It was due to my own naïveté of the subject. The Uint8ClampedArray I was creating was acting as a window to the underlying memory, shaping it into a form that I could access. But upon reallocation, that HEAPU8 buffer is changing, causing it to "detach" from my array.

One would then create a mapped array before the Wasm method call to pass data to the function, or create a mapped array after the Wasm call to read data from the function. In my example above I would need two of them, one before and one after since the array created before may become detached.

...
var size = width * height;
var heapPtr = Module._malloc(size);
var input = new Uint8ClampedArray(Module.HEAPU8.buffer, heapPtr, size);
... // Populate 'data' with 8bit grayscale based on canvas ImageData
Module._Initialize(heapPtr, width, height);
var output = new Uint8ClampedArray(Module.HEAPU8.buffer, heapPtr, size);
console.log(output[0]);

I hate answering my own questions... but hopefully this saves someone some headache if they run into this. Any maybe someone with more expertise on the subject could clarify any part of this answer that may need it.

Upvotes: 7

Related Questions