CODEWITHSUNDEEP
iossqliteswiftcore-data
zono
zono

Reputation: 8584

CoreData import overload memory

I have some master data, which are JSON files and I need to import by using CoreData. Transaction point is when all masters insert normally. However, the amount of use of memory is increasing until commit. The app crash due to the overload memory before commit.

I'm thinking two solutions like below.

  1. Use SQLite's function like saving data to file temporary. (I'm not sure whether the function exists or not.)

  2. Use two DB file. One is for operation and another one is temporal. If import is done normally, replace from temporal to the operation DB.

But still not sure about it. Does anyone know any solutions in this case? Thanks.

Update1:

So you're importing into an empty data store, or you're adding to existing data that you need to revert to on failure?

In my case, this is the process below.

  1. Delete existing data. (All tables)

  2. Importing into an empty data.

  3. After master data is done, commit it. If something error happen, rollback transaction. The deleting and importing are reverted so that to ensure consistency between master data.

Also, why do much data on a mobile device?

My customer's need.

Update2:

I tried disable undoManager but the result was not effected so much. I do not need "undo" but I need rollback. In this case, setUndoManager is available...?

[[self managedObjectContext] setUndoManager:nil] ;

Upvotes: 0

Views: 88

Answers (1)

Nicolas Buquet
Nicolas Buquet

Reputation: 3955

CoreData is memory hungry.

Its context (the scratchpad) is stored in memory.

If you can work with 2 db files, store your new data in a new file and save and flush regularly (like every 1000 records, depending on your record's size) to maintain your memory consumption acceptable. if there is an error at import, simply close and delete the new db file.

You can also work on the initial db file with adding a field to your records that can be a "validated": boolean or a "timestamp": date. The new records have a validated filed to false. If you encounter an error, delete all records that are not "validated" (or on date, delete all records whose "timestamp" is after the date of the beginning of the import).

But, in the second solution, you need to not delete all records at once, because if you make a request to get all records whose "validated" is false, all your new records will be instantiated in scratchpad (that's the way CoreData works) and that will again consume a lot of memory.

Upvotes: 2

Related Questions