CODEWITHSUNDEEP
javaignite
Matthew McPeak
Matthew McPeak

Reputation: 17924

Why is my Apache Ignite persistent store remaining locked?

I have an application that uses Apache Ignite as a single node cluster. That is, Ignite is started and stopped by the application and its lifecycle matches the application.

The Ignite caches have both persistent storage and read-through enabled. So,

  1. First call to cache.get() --> nothing in cache, reads through via CacheAdapter to backing store (an Oracle database) and loads the entry into the cache.
  2. Subsequent call to cache.get() -> returns from cache
  3. Application bounces, the call cache.get() --> returns from cache (from persistent store). Key point is that it does not read through to the backing store after an application bounce!

All this seems to be working wonderfully. Here is my problem: sometimes (fairly often) when the application is bounced or redeployed, the persistent store data directory remains locked as far as Apache Ignite is concerned. As a result, Ignite silently creates a new, empty persistent store, which defeats the whole purpose of persistent storage.

Our data directory currently looks like this:

-sh-4.2$ pwd
/myapp/oracle/user_projects/domains/myapp_inst/igniteCache/myserver/data
-sh-4.2$ ls -ltr
total 48
drwxr-x---. 10 mygroup mygroup 4096 Jan  9 16:25 node00-7c02d9ab-7ef3-4d01-8ebd-d1184fa281a2
drwxr-x---. 10 mygroup mygroup 4096 Jan  9 17:28 node01-06a5d059-1b62-4ac2-84bf-5325deac8138
drwxr-x---. 10 mygroup mygroup 4096 Feb  4 15:39 node02-ba720d52-570a-448c-9109-75687ee664e7
drwxr-x---. 10 mygroup mygroup 4096 Feb  5 12:16 node03-d2c521aa-e0ee-471b-ad16-08af382a1e3d
drwxr-x---. 10 mygroup mygroup 4096 Feb  6 17:44 node04-505e754e-a3d3-48b1-a759-d5ec8867dc96
drwxr-x---. 10 mygroup mygroup 4096 Feb  6 18:23 node05-ec0f89e3-bfe5-4bb6-87da-302951439f66
drwxr-x---. 10 mygroup mygroup 4096 Feb 13 15:41 node06-5b9dc33a-42f6-4f1c-8d57-14d5ddb30dc5
drwxr-x---. 10 mygroup mygroup 4096 Feb 13 15:43 node07-47aacde5-2598-4a85-9383-761e569bb1d1
drwxr-x---. 10 mygroup mygroup 4096 Feb 13 15:47 node08-0374a51d-4b90-4e5a-9465-adabc900ea0b
drwxr-x---. 10 mygroup mygroup 4096 Feb 13 15:52 node09-610c69c6-35e0-4d74-90db-6be09bb77659
-rw-r-----.  1 mygroup mygroup   41 Feb 13 17:00 lock
drwxr-x---. 10 mygroup mygroup 4096 Feb 13 17:00 node10-f115d95b-7b76-41e0-bd32-966030331c9c

There should only be one "nodeXX..." directory above.

Questions

  1. Is there anything I can do my application to ensure that, whatever the failure, the persistent store locks are released?
  2. Is there anything I can do on application startup to tell Ignite to break or delete the lock and not create a new, empty store every time?
  3. Is there anything I can do in my application to tell Ignite that this persistent store is local to the node and that, therefore, locks aren't necessary?

Really, anything I can do to get around this behavior would be helpful.

Upvotes: 0

Views: 741

Answers (1)

Michael
Michael

Reputation: 650

Please set Ignite ConsistentId: igniteCfg.setConsistentId("My_Node_Id" + id); https://apacheignite.readme.io/docs/distributed-persistent-store#section-persistence-path-management

in this case, ignite will try to acquire file lock and wait for 10 seconds, if it fails to acquire it, it won't create a new folder, but fail with an exception.

Upvotes: 2

Related Questions