CODEWITHSUNDEEP
ignite
serdroid
serdroid

Reputation: 166

Why ignite does not load all records into cache when server node count increased?

I have developed ignite sample data grid application with third party persistence. I have an entity named UserInfo. I have generated 3602000 records and added them into database. I have 64 GB ram on my machine. I am running n server nodes and after cluster initialized run a client node to trigger loading caches. When I run one or two server nodes and then load cache everything is fine. All records are loaded into the cache. When I run 4 server nodes approximately half of the records are loaded into the cache. There are a lot of free memory on the machine. I have created a github repo for sample application https://github.com/serdroid/userinfo .

Below free command output and ignitevisorcmd output for both cases.

before starting servers

free --giga
              total        used        free      shared  buff/cache   available
Mem:             65           8          52           0           4          56

2 server nodes running with -Xmx4G

Loaded 3602000 keys with backups in 255911ms.

Free memory

free --giga
              total        used        free      shared  buff/cache   available
Mem:             65          20          38           0           6          45

Cache details

Nodes for: info.serdroid.userinfo.grid.model.UserInfo(@c0)
+================================================================================================================+
|       Node ID8(@), IP        | CPUs | Heap Used | CPU Load |   Up Time    |         Size         | Hi/Mi/Rd/Wr |
+================================================================================================================+
| 800C684F(@n1), 10.251.74.157 | 4    | 87.97 %   | 0.40 %   | 00:13:07.851 | Total: 1837114       | Hi: 0       |
|                              |      |           |          |              |   Heap: 0            | Mi: 0       |
|                              |      |           |          |              |   Off-Heap: 1837114  | Rd: 0       |
|                              |      |           |          |              |   Off-Heap Memory: 0 | Wr: 0       |
+------------------------------+------+-----------+----------+--------------+----------------------+-------------+
| 40FD7F83(@n0), 10.251.74.157 | 4    | 86.30 %   | 0.40 %   | 00:13:25.431 | Total: 1764886       | Hi: 0       |
|                              |      |           |          |              |   Heap: 0            | Mi: 0       |
|                              |      |           |          |              |   Off-Heap: 1764886  | Rd: 0       |
|                              |      |           |          |              |   Off-Heap Memory: 0 | Wr: 0       |
+----------------------------------------------------------------------------------------------------------------+

4 server nodes running with -Xmx4G

Loaded 1805474 keys with backups in 98203ms.

Free memory

free --giga
              total        used        free      shared  buff/cache   available
Mem:             65          23          35           0           6          41

Cache details

Nodes for: info.serdroid.userinfo.grid.model.UserInfo(@c0)
+================================================================================================================+
|       Node ID8(@), IP        | CPUs | Heap Used | CPU Load |   Up Time    |         Size         | Hi/Mi/Rd/Wr |
+================================================================================================================+
| 9B36B2E9(@n2), 10.251.74.157 | 4    | 65.52 %   | 0.27 %   | 00:14:58.116 | Total: 534454        | Hi: 0       |
|                              |      |           |          |              |   Heap: 0            | Mi: 0       |
|                              |      |           |          |              |   Off-Heap: 534454   | Rd: 0       |
|                              |      |           |          |              |   Off-Heap Memory: 0 | Wr: 0       |
+------------------------------+------+-----------+----------+--------------+----------------------+-------------+
| 1222A1A4(@n0), 10.251.74.157 | 4    | 39.65 %   | 2.33 %   | 00:15:06.421 | Total: 389344        | Hi: 0       |
|                              |      |           |          |              |   Heap: 0            | Mi: 0       |
|                              |      |           |          |              |   Off-Heap: 389344   | Rd: 0       |
|                              |      |           |          |              |   Off-Heap Memory: 0 | Wr: 0       |
+------------------------------+------+-----------+----------+--------------+----------------------+-------------+
| 8699AB89(@n1), 10.251.74.157 | 4    | 62.77 %   | 0.23 %   | 00:15:01.325 | Total: 441069        | Hi: 0       |
|                              |      |           |          |              |   Heap: 0            | Mi: 0       |
|                              |      |           |          |              |   Off-Heap: 441069   | Rd: 0       |
|                              |      |           |          |              |   Off-Heap Memory: 0 | Wr: 0       |
+------------------------------+------+-----------+----------+--------------+----------------------+-------------+
| 8B43AF4C(@n3), 10.251.74.157 | 4    | 65.44 %   | 1.73 %   | 00:14:55.624 | Total: 440607        | Hi: 0       |
|                              |      |           |          |              |   Heap: 0            | Mi: 0       |
|                              |      |           |          |              |   Off-Heap: 440607   | Rd: 0       |
|                              |      |           |          |              |   Off-Heap Memory: 0 | Wr: 0       |
+----------------------------------------------------------------------------------------------------------------+

Upvotes: 0

Views: 986

Answers (1)

Alexey Popov
Alexey Popov

Reputation: 181

  1. Apache Ignite uses off-heap memory to store your data. Please configure (enlarge) data region as it is described at https://apacheignite.readme.io/docs/memory-configuration.

  2. Apache Ignite works in a Lazy way for loading data into memory from a store by design. You can force data reload by Cache.loadCache(null) manually, or you can use EventType.EVT_NODE_JOINED local event as a trigger. Please see https://apacheignite.readme.io/docs/events for more details.

Upvotes: 0

Related Questions