Reputation: 781
I am having an issue of my Javascript application on our deployment server. It processes a few large CSVs (about 3 GB in size) by means of loading into memory (it is necessary because there are interdependencies between them).
However, the application crashes with follow:
[3910795:0x7f2d1c144000] 153589 ms: Scavenge (interleaved) 15.0 (17.9) -> 14.4 (16.1) MB, pooled: 2 MB, 487.30 / 0.00 ms (average mu = 0.962, current mu = 0.962) allocation failure;
<--- Last few GCs --->
[3910795:0x7f35941ef000] 118015 ms: Scavenge (interleaved) 10.3 (14.1) -> 9.6 (14.1) MB, pooled: 2 MB, 0.37 / 0.00 ms (average mu = 0.988, current mu = 0.988) task;
[3910795:0x7f35941ef000] 151219 ms: Mark-Compact (reduce) 9.8 (14.1) -> 9.5 (11.1) MB, pooled: 0 MB, 32232.90 / 0.01 ms (+ 151.2 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 32418 ms) (average mu = 0.778, c
<--- JS stacktrace --->
<--- JS stacktrace --->
FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
1: 0xe3811e node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
1: 0xe3811e node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
2: 0x125faf0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
2: 0x125faf0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
3: 0x125fdc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
4: 0x148d805 [node]
3: 0x125fdc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
5: 0x14fadce [node]
4: 0x148d805 [node]
5: 0x14fadce [node]
6: 0x14a4ab0 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::internal::GarbageCollectionReason, char const*) [node]
7: 0x14a6629 [node]
8: 0x14a9ab8 [node]
9: 0x1ce6351 [node]
I am already running the application with node --max-semi-space-size=1024 --max-old-space-size=65536
and the memory allocation has never reached 64 GiB, while on my local workstation it doesn't crash even I have only allocated 32 GiB to max-old-space-size
.
How can I prevent it from crashing?
Upvotes: 0
Views: 52