CODEWITHSUNDEEP
javamongodbmorphia
user1883212
user1883212

Reputation: 7859

Morphia for Mongo DB, and low performance code

I'm copying a Mongo DB collection from a database to another, using Morphia in that way:

List<User> users = datastoreSource.find(User.class).asList();
datastoreDest.save(users);

This code works, however it takes hours to perform. I have a collection with 3 million documents in it and I would expect to copy it in few minutes. Can someone help me to improve that code?

Upvotes: 0

Views: 547

Answers (1)

xeraa
xeraa

Reputation: 10859

I'm amazed this is working at all with 3 million documents. First you are loading them all into memory and then you are writing them back.

I would try reading a batch of data (say 100) with .order("_id").limit(100).skip(i*100) and bulk write it back.

And it would be interesting:

  • How much data is that actually?
  • How much memory do you have available? Do you start swapping?
  • How much is "hours"? In the area of two or ten?

Upvotes: 1

Related Questions