CODEWITHSUNDEEP
javamongodbmigrationdatabase-migrationmorphia
Philipp Jahoda
Philipp Jahoda

Reputation: 51431

How to handle query migration in MongoDB / Morphia?

I recently had to introduce a change in my data structure regarding the "users" collection which resulted in a migration from the following simplified example:

{ 
   "name": "John",
   "emails" : [ "[email protected]", "[email protected]" ]
}

to this:

{ 
   "name": "John",
   "emailAddresses" : [ 
       { 
         "email" : "[email protected]",
         "verified" : true
       },
       { 
         "email" : "[email protected]",
         "verified" : true
       }
    ]
}

So the "emails" field was changed from being a simple String array to an array of complex objects, each having a field "email" and "verified". In addition to that, it's name changed to "emailAddresses".

I am aware of how to migrate my data concerning loading and storage with changed data models using the annotations provided by Morphia such as @PostLoad or @NotSaved.

My problem arises with queries. Originally, I would query for users by their email address like this:

Query<User> q = dataStore.createQuery(User.class);
q.filter("emails", email);
User u = q.get();

I know that I simply have to adapt the query to the following to support my new data structure:

Query<User> q = dataStore.createQuery(User.class);
q.filter("emailAddresses.email", email);
User u = q.get();

That's fine. The problem is that some of the documents in my collection are still stored in the "old" way, and some are already stored in the "new" way. If I only use the "new" way, I am not going to be able to find older documents.

The question is:

How can I create a query that considers both old and new data structures when (like in this example) querying by email address?

The best thing I could come up with so far is to use "or" queries and query for both fields with validation disabled (exception thrown otherwise):

Query<User> q = dataStore.createQuery(User.class);
q.disableValidation();
q.or(q.criteria("emails").equal(email), q.criteria("emailAddresses.email").equal(email)); 
User u = q.get();

However, this seems quite cumbersome and possibly bad for performance. I was wondering if there is a better way to approach this problem?

Upvotes: 2

Views: 381

Answers (1)

Nic Cottrell
Nic Cottrell

Reputation: 9695

You don't have to use only the Morphia annotations. You can use the MongoDB command line to run update queries. You can do quite complex logic in JavaScript to lookup and transform documents to the new format. You can run it only on documents that still have the old 'emails' field if you have an index on that field and millions of documents

Upvotes: 1

Related Questions