CODEWITHSUNDEEP
hawq
Chunling Wang
Chunling Wang

Reputation: 21

How to configure HAWQ memory?

Can I configure the amount of memory available to all HAWQ segment instances and the amount of memory available for each segment?

Upvotes: 0

Views: 225

Answers (6)

huan zhang
huan zhang

Reputation: 11

@ztao, virtual segments are not shared by queries. For one query, it asks virtual segments from RM, and return virtual segments to RM when it finishes.

Upvotes: 0

ztao1987
ztao1987

Reputation: 71

@Wen Lin, you mentioned "one query may request many virtual segments", so one virtual segment is also shared by many queries, right? Then all shared queries will share the memory quota, which is 128mb by default?

Upvotes: 0

Wen Lin
Wen Lin

Reputation: 21

hawq_rm_stmt_vseg_memory is the memory quota(size) of one virtual segment, the default value is 128mb. That means, the memory size of one virtual segment is 128mb, one query may request many virtual segments.

Upvotes: 0

ztao1987
ztao1987

Reputation: 71

The virtual segment memory usage is set by hawq_rm_stmt_vseg_memory, which calculates the total memory for all forked QE. Since different query statements may resides on the same vseg, so hawq_rm_stmt_vseg_memory will be shared across different queries.

Upvotes: 0

huan zhang
huan zhang

Reputation: 11

In Apache Hawq, virtual segments are used as containers of the executors. As a result, the memory used by queries is controlled by the number of virtual segments. You can use GUC hawq_rm_memory_limit_perseg to control the total memory size of each host(segment instance) and to control the memory size of a virtual segment, you can create your own resqueue with specified memsize for each container(256M as default).

Upvotes: 0

Jon Roberts
Jon Roberts

Reputation: 2106

This is covered in the system requirements.

Upvotes: 1

Related Questions