Reputation: 1507

MongoDB .NET driver and text search

I am using this MongoDB driver: https://mongodb.github.io/mongo-csharp-driver/ and I would like to search using a text index, which (I think) is created on all text fields like so:

{
    "_fts" : "text",
    "_ftsx" : 1
}

I am using linq queries to filter the data, example:

MongoClient client = new MongoClient(_mongoConnectionString);
IMongoDatabase mongoDatabase = client.GetDatabase(DatabaseName);
var aCollection = mongoDatabase.GetCollection<MyTypeSerializable>(CollectionName);

IMongoQueryable<MyTypeSerializable> queryable = aCollection.AsQueryable()
                .Where(e=> e.Field == 1);
var result = queryable.ToList();

How do I utilize the text search using this method?

Upvotes: 3

Answers (4)

The Smallest

Reputation: 5773

Searching for solution I found FilterDefinition<T>.Inject() extension method. So we can go deeper and create one more extension on IMongoQueryable<T>:

public static class MongoQueryableFullTextExtensions
{
    public static IMongoQueryable<T> WhereText<T>(this IMongoQueryable<T> query, string search)
    {
        var filter = Builders<T>.Filter.Text(search);
        return query.Where(_ => filter.Inject());
    }
}

And use it like this:

IMongoDatabase database = GetMyDatabase();

var results = database
    .GetCollection<Blog>("Blogs")
    .AsQueryable()
    .WhereText("stackoverflow")
    .Take(10)
    .ToArray();

Hope this helps somebody :)

Upvotes: 15

Tolga Kabadurmus

Reputation: 1

It's possible to modify MongoDb driver source code. Let me explain it to you:

Consider "PredicateTranslator" does not convert linq Expression into "$text" query. However there is a Text() method "FilterDefinitionBuilder" class, "PredicateTranslator" does not know the entity class property has a text search index.
You have to mark entity class property (a condition in predicate statement) with an Attribute. The attribute works to remark property has a full text search index.
From now on "PredicateTranslator" class knows the property has a full text search index with this Attribute "PredicateTranslator"

Let me show you some code:

At MongoDB.Bson project create an Attribute as shown below:

[AttributeUsage(AttributeTargets.Property | AttributeTargets.Field)] public class BsonFullTextSearchAttribute : Attribute { }

At your entity class property place the "BsonFullTextSearchAttribute" Attribute as shown below:

public class History 
{
    [MongoDB.Bson.Serialization.Attributes.BsonFullTextSearch]
    public string ObjectJSON { get; set; }
}

At MongoDB.Driver.Linq.Translators.QueryableTranslator.cs

Add a field that keeps entity class type in Expression>, as shown below:
```
private Type _sourceObjectTypeInExpression;
```

Add a method to get entity class type as shown below:

private void GetObjectType(Expression node)
{
    if (node.Type != null && node.Type.GenericTypeArguments != null && node.Type.GenericTypeArguments.Length > 0)
    {
        this._sourceObjectTypeInExpression = node.Type.GenericTypeArguments[0]; 
    }
 }

Replace "public static QueryableTranslation Translate()" method as shown below:

public static QueryableTranslation Translate(Expression node, IBsonSerializerRegistry serializerRegistry, ExpressionTranslationOptions translationOptions)
{
var translator = new QueryableTranslator(serializerRegistry, translationOptions);
translator.GetObjectType(node);
translator.Translate(node);

var outputType = translator._outputSerializer.ValueType;
var modelType = typeof(AggregateQueryableExecutionModel<>).MakeGenericType(outputType);
var modelTypeInfo = modelType.GetTypeInfo();
var outputSerializerInterfaceType = typeof(IBsonSerializer<>).MakeGenericType(new[] { outputType });
var constructorParameterTypes = new Type[] { typeof(IEnumerable<BsonDocument>), outputSerializerInterfaceType };
var constructorInfo = modelTypeInfo.GetConstructors(BindingFlags.Instance | BindingFlags.NonPublic)
    .Where(c => c.GetParameters().Select(p => p.ParameterType).SequenceEqual(constructorParameterTypes))
    .Single();
var constructorParameters = new object[] { translator._stages, translator._outputSerializer };
var model = (QueryableExecutionModel)constructorInfo.Invoke(constructorParameters);

return new QueryableTranslation(model, translator._resultTransformer);
}

In TranslateWhere() method pass "_sourceObjectTypeInExpression" field to PredicateTranslator.Translate() static method

var predicateValue = PredicateTranslator.Translate(node.Predicate, _serializerRegistry, this._sourceObjectTypeInExpression);

B. MongoDB.Driver.Linq.Translators.PredicateTranslator.cs - Add a field: "private Type sourceObjectTypeInExpression = null;"

- Replace constructor as shown below (there has to be only one constructor);
    private PredicateTranslator(Type _sourceObjectTypeInExpression)
    {
        this.sourceObjectTypeInExpression = _sourceObjectTypeInExpression;
    }

- Replace function "public static BsonDocument Translate(Expression node, IBsonSerializerRegistry serializerRegistry)" as shown below;
    public static BsonDocument Translate(Expression node, IBsonSerializerRegistry serializerRegistry, Type sourceObjectTypeInExpression)
    {
        var translator = new PredicateTranslator(sourceObjectTypeInExpression);
        node = FieldExpressionFlattener.FlattenFields(node);
        return translator.Translate(node)
            .Render(serializerRegistry.GetSerializer<BsonDocument>(), serializerRegistry);
    }

- Add these lines for reflection cache:
    #region FullTextSearch
    private static readonly object mSysncFullTextSearchObjectCache = new object();
    private static ConcurrentDictionary<string, List<string>> _fullTextSearchObjectCache = null;
    private static ConcurrentDictionary<string, List<string>> FullTextSearchObjectCache
    {
        get
        {
            if (_fullTextSearchObjectCache == null)
            {
                lock (mSysncFullTextSearchObjectCache)
                {
                    try
                    {
                        if (_fullTextSearchObjectCache == null)
                        {
                            _fullTextSearchObjectCache = new ConcurrentDictionary<string, List<string>>();
                        }
                    }
                    finally
                    {
                        Monitor.PulseAll(mSysncFullTextSearchObjectCache);
                    }
                }
            }

            return _fullTextSearchObjectCache;
        }
    }

    private bool IsFullTextSearchProp(Type entityType, string propName)
    {
        bool retVal = false;
        string entityName = entityType.Name;

        this.SetObject2FullTextSearchObjectCache(entityType);
        if (FullTextSearchObjectCache.ContainsKey(entityName))
        {
            List<string> x = FullTextSearchObjectCache[entityName];
            retVal = x.Any(p => p == propName);
        }

        return retVal;
    }

    private void SetObject2FullTextSearchObjectCache(Type entityType)
    {
        string entityName = entityType.Name;

        if (!FullTextSearchObjectCache.ContainsKey(entityName))
        {
            List<string> retVal = new List<string>();

            PropertyInfo[] currentProperties = entityType.GetProperties(BindingFlags.Public | BindingFlags.Instance);
            foreach (PropertyInfo tmp in currentProperties)
            {
                var attributes = tmp.GetCustomAttributes();
                BsonFullTextSearchAttribute x = (BsonFullTextSearchAttribute)attributes.FirstOrDefault(a => typeof(BsonFullTextSearchAttribute) == a.GetType());
                if (x != null)
                {
                    retVal.Add(tmp.Name);
                }
            }

            FieldInfo[] currentFields = entityType.GetFields(BindingFlags.Public | BindingFlags.Instance);
            foreach (FieldInfo tmp in currentFields)
            {
                var attributes = tmp.GetCustomAttributes();
                BsonFullTextSearchAttribute x = (BsonFullTextSearchAttribute)attributes.FirstOrDefault(a => typeof(BsonFullTextSearchAttribute) == a.GetType());
                if (x != null)
                {
                    retVal.Add(tmp.Name);
                }
            }

            FullTextSearchObjectCache.AddOrUpdate(entityName, retVal, (k, v) => v);
        }
    }
    #endregion

- Replace "switch (operatorType)" switch in "private FilterDefinition<BsonDocument> TranslateComparison(Expression variableExpression, ExpressionType operatorType, ConstantExpression constantExpression)" function as shown below;
    bool isFullTextSearchProp = this.IsFullTextSearchProp(this.sourceObjectTypeInExpression, fieldExpression.FieldName);
    switch (operatorType)
    {
        case ExpressionType.Equal:
            if (!isFullTextSearchProp)
            {
                return __builder.Eq(fieldExpression.FieldName, serializedValue);
            }
            else
            {
                return __builder.Text(serializedValue.ToString());
            }
        case ExpressionType.GreaterThan: return __builder.Gt(fieldExpression.FieldName, serializedValue);
        case ExpressionType.GreaterThanOrEqual: return __builder.Gte(fieldExpression.FieldName, serializedValue);
        case ExpressionType.LessThan: return __builder.Lt(fieldExpression.FieldName, serializedValue);
        case ExpressionType.LessThanOrEqual: return __builder.Lte(fieldExpression.FieldName, serializedValue);
        case ExpressionType.NotEqual:
            if (!isFullTextSearchProp)
            {
                return __builder.Ne(fieldExpression.FieldName, serializedValue);
            }
            else
            {
                throw new ApplicationException(string.Format("Cannot use \"NotEqual\" on FullTextSearch property: \"{0}\"", fieldExpression.FieldName));
            }
    }

- Replace "switch (methodCallExpression.Method.Name)" switch in "private FilterDefinition<BsonDocument> TranslateStringQuery(MethodCallExpression methodCallExpression)" function as shown below;
    bool isFullTextSearchProp = this.IsFullTextSearchProp(this.sourceObjectTypeInExpression, tmpFieldExpression.FieldName);
    var pattern = Regex.Escape((string)constantExpression.Value);
    if (!isFullTextSearchProp)
    {
        switch (methodCallExpression.Method.Name)
        {
            case "Contains": pattern = ".*" + pattern + ".*"; break;
            case "EndsWith": pattern = ".*" + pattern; break;
            case "StartsWith": pattern = pattern + ".*"; break; // query optimizer will use index for rooted regular expressions
            default: return null;
        }

        var caseInsensitive = false;
        MethodCallExpression stringMethodCallExpression;
        while ((stringMethodCallExpression = stringExpression as MethodCallExpression) != null)
        {
            var trimStart = false;
            var trimEnd = false;
            Expression trimCharsExpression = null;
            switch (stringMethodCallExpression.Method.Name)
            {
                case "ToLower":
                case "ToLowerInvariant":
                case "ToUpper":
                case "ToUpperInvariant":
                    caseInsensitive = true;
                    break;
                case "Trim":
                    trimStart = true;
                    trimEnd = true;
                    trimCharsExpression = stringMethodCallExpression.Arguments.FirstOrDefault();
                    break;
                case "TrimEnd":
                    trimEnd = true;
                    trimCharsExpression = stringMethodCallExpression.Arguments.First();
                    break;
                case "TrimStart":
                    trimStart = true;
                    trimCharsExpression = stringMethodCallExpression.Arguments.First();
                    break;
                default:
                    return null;
            }

            if (trimStart || trimEnd)
            {
                var trimCharsPattern = GetTrimCharsPattern(trimCharsExpression);
                if (trimCharsPattern == null)
                {
                    return null;
                }

                if (trimStart)
                {
                    pattern = trimCharsPattern + pattern;
                }
                if (trimEnd)
                {
                    pattern = pattern + trimCharsPattern;
                }
            }

            stringExpression = stringMethodCallExpression.Object;
        }

        pattern = "^" + pattern + "$";
        if (pattern.StartsWith("^.*"))
        {
            pattern = pattern.Substring(3);
        }
        if (pattern.EndsWith(".*$"))
        {
            pattern = pattern.Substring(0, pattern.Length - 3);
        }

        var fieldExpression = GetFieldExpression(stringExpression);
        var options = caseInsensitive ? "is" : "s";
        return __builder.Regex(fieldExpression.FieldName, new BsonRegularExpression(pattern, options));
    }
    else
    {
        return __builder.Text(pattern);
    }

Upvotes: 0

Kevin Smith

Reputation: 14436

Looking at the PredicateTranslator within the C# MongoDB driver there isn't any expression that gets converted in to a text query. So you won't be able to achive a text query using a linq query.

However you could try just doing a text search with the Builder<>:

MongoClient client = new MongoClient(_mongoConnectionString);
IMongoDatabase mongoDatabase = client.GetDatabase(DatabaseName);
var aCollection = mongoDatabase.GetCollection<MyTypeSerializable>(CollectionName);

var cursor = await aCollection.FindAsync(Builders<MyTypeSerializable>.Filter.Text("search"));

var results = await cursor.ToListAsync();

Details about the text filter is here https://docs.mongodb.com/manual/reference/operator/query/text/

Upvotes: 3

Vitaliy Kalinin

Reputation: 1871

How about:

IMongoQueryable<MyTypeSerializable> queryable = aCollection
.AsQueryable()
.Where(e=> e.Field.Contains("term"));

Upvotes: -1

MongoDB .NET driver and text search

Answers (4)

Related Questions