CODEWITHSUNDEEP
javaorc
Ruben Hartenstein
Ruben Hartenstein

Reputation: 1

Apache ORC buffer size too small

I face the attached problem when reading an orc file: error msg

Is it possible to change this buffer size of 65536 to the needed one of 1817279? Which configuration values do I have to adapt in order to set this value?

I did not find the correct configuration value in the documentation.

Upvotes: 0

Views: 42

Answers (1)

吴振威
吴振威

Reputation: 1

The exceeding buffer-size is related to an issue with the HDFS Erasure Coding (EC) file-encoding. See this issue at Apache ORC:

It could be traced back to the Hadoop HDFS bug:

So, check your Hadoop HDFS version if it is affected by this bug.

Upvotes: 0

Related Questions