CODEWITHSUNDEEP
azureazure-data-lakeu-sql
Nemanja Vrndic
Nemanja Vrndic

Reputation: 11

E_RUNTIME_SYSTEM_ERROR on Azure Data Lake Analytics

I am doing some benchmark on Azure Data Lake Analytics. After running a query during an execution phase Combine_partition, I got error:

ERROR E_RUNTIME_SYSTEM_ERROR MESSAGE A system error has been reported [VirtualAlloc returned 0]

It looks like out of memory exceptions. Any idea?

Error

Upvotes: 1

Views: 213

Answers (2)

Nemanja Vrndic
Nemanja Vrndic

Reputation: 11

Issue solved.

Vertex ran out of the memory, so I changed my query, eliminating GROUP BY using WINDOWING functions on same data and partitions.

Tnx for help.

Upvotes: 0

Michael Rys
Michael Rys

Reputation: 6684

This looks like your join may have received too much data and ran out of the 6GB memory that a vertex get.

The suggestion is to distribute the data or operation better by either having a better distribution strategy for that join (e.g., avoiding data skew where one bucket is too large), use a different operation that provides you with better scale, or add some data hints that "lures" the optimizer to make better partitioning decisions.

Upvotes: 1

Related Questions