CODEWITHSUNDEEP
nhibernateduplicatesintersect
Henry H
Henry H

Reputation: 105

Avoiding duplicate entries in NHibernate

I am trying to store messages that I receive in a database using NHibernate. However, there is a possibility that the same message is received twice, and in that case I do not want to save the duplicate in the database. My first thought was to do the following:

// in SaveRange(IEnumerable<Message> messages
var alreadyStoredMessages = session.Query<Message>().Intersect(messages);

var newMessages = messages.Except(alreadyStoredMessages);

However, it seems that NHibernate does not support Intersect so this will result in an exception. I know I could always fetch all the messages, convert them to a list or array, and then do the intersect, but it wouldn't be very effective.

The Message class implements IEquatable and it also overrides GetHashCode() and Equals(object obj). The equality depends on several properties (timestamp, several strings, etc).

Upvotes: 0

Views: 718

Answers (1)

Firo
Firo

Reputation: 30813

if all new messages come at once use a filter:

var alreadyStoredMessages = session.QueryOver<Message>()
    .WhereRestrictionOn(m => m.timestamp).In(messages.Select(m => m.timeStamp))
    .AsEnumerable()
    .Intersect(messages);

var newMessages = messages.Except(alreadyStoredMessages).ToList();

assuming duplicate messages are following short after each other: Hold a buffer of last n received messages and look into them.

var lastMessages = new Queue<Message>(100);
while(true)
{
    var message = GetNextMessage();
    if (!lastMessages.Contains(message))
    {
        lastMessages.Enqueue(message);
        session.Save(message);
        if (lastMessages.Count >= 100);
            lastMessages.Dequeue();
    }
}

Upvotes: 1

Related Questions