CODEWITHSUNDEEP
elasticsearchnest
jhilden
jhilden

Reputation: 12439

ElasticSearch Nest: AutoMap with DynamicTemplates

I'm trying use a dynamic template in ES such that all string fields are multifields. I also want to apply some specific mappings to certain fields.

Take the following example class:

[ElasticsearchType(Name = "sample1")]
public class Sample1
{
    public string ID { get; set; }

    [String(Index = FieldIndexOption.No)]
    public string DoNotIndex { get; set; }

    public string MultiField1 { get; set; }

    public string MultiField2 { get; set; }
}

I want to then create the dynamic template and apply the mapping to DoNotIndex using the following command:

_client.Map<Sample1>(m => m
  .AutoMap()
  .DynamicTemplates(dt=> dt
      .DynamicTemplate("all_strings_multifields", t => t
        .MatchMappingType("string")
        .Mapping(tm => tm
            .String(mf => mf
                .Index(FieldIndexOption.Analyzed)
                .Fields(mff => mff
                    .String(s => s
                        .Name("raw")
                        .Index(FieldIndexOption.NotAnalyzed)
                    )
                )
            )
        )
    )
    )
)
.VerifySuccessfulResponse();

The result is:

{
  "test1": {
    "mappings": {
      "sample1": {
        "dynamic_templates": [
          {
            "all_strings_multifields": {
              "match_mapping_type": "string",
              "mapping": {
                "fields": {
                  "raw": {
                    "type": "string",
                    "index": "not_analyzed"
                  }
                },
                "index": "analyzed",
                "type": "string"
              }
            }
          }
        ],
        "properties": {
          "doNotIndex": {
            "type": "keyword",
            "index": false
          },
          "iD": {
            "type": "text"
          },
          "multiField1": {
            "type": "text"
          },
          "multiField2": {
            "type": "text"
          }
        }
      }
    }
  }
}

Results

you'll see that the DoNotIndex property is indeed correct, but the multifield1 and multifield2 are not correct (they are not multi fields).

Workaround

I know I can "fix" this by NOT doing the AutoMap() and instead specifying each of the special indexes but there are a lot of fields and that isn't as clean of a solution.

Can I do AutoMap with DynamicTemplates?

Upvotes: 0

Views: 2197

Answers (1)

Russ Cam
Russ Cam

Reputation: 125528

Dynamic templates only apply to fields that are dynamically added to a mapping, so properties explicitly mapped with .AutoMap() will not be affected by dynamic mapping.

There is however a way to apply conventions to explicit mappings with NEST using the visitor pattern. It looks like you're using Elasticsearch 5.0, so you should use the text and keyword mappings.

First define a visitor

[ElasticsearchType(Name = "sample1")]
public class Sample1
{
    public string ID { get; set; }

    [Keyword(Index = false)]
    public string DoNotIndex { get; set; }

    public string MultiField1 { get; set; }

    public string MultiField2 { get; set; }
}

public class AllStringsMultiFieldsVisitor : NoopPropertyVisitor
{
    public override void Visit(ITextProperty type, PropertyInfo propertyInfo, ElasticsearchPropertyAttributeBase attribute)
    {
        // if a custom attribute has been applied, let it take precedence
        if (propertyInfo.GetCustomAttribute<ElasticsearchPropertyAttributeBase>() == null)
        {
            type.Fields = new Properties
            {
                {
                    "raw", new KeywordProperty()
                }
            };
        }

        base.Visit(type, propertyInfo, attribute);
    }
}

Then pass an instance of the visitor to .AutoMap()

client.Map<Sample1>(m => m
    .AutoMap(new AllStringsMultiFieldsVisitor())
    .DynamicTemplates(dt => dt
        .DynamicTemplate("all_strings_multifields", t => t
            .MatchMappingType("text")
            .Mapping(tm => tm
                .Text(mf => mf
                    .Index(true)
                    .Fields(mff => mff
                        .Keyword(s => s
                            .Name("raw")
                        )
                    )
                )
            )
        )
    )
);

produces

{
  "dynamic_templates": [
    {
      "all_strings_multifields": {
        "match_mapping_type": "text",
        "mapping": {
          "type": "text",
          "fields": {
            "raw": {
              "type": "keyword"
            }
          },
          "index": true
        }
      }
    }
  ],
  "properties": {
    "iD": {
      "fields": {
        "raw": {
          "type": "keyword"
        }
      },
      "type": "text"
    },
    "doNotIndex": {
      "type": "keyword",
      "index": false
    },
    "multiField1": {
      "fields": {
        "raw": {
          "type": "keyword"
        }
      },
      "type": "text"
    },
    "multiField2": {
      "fields": {
        "raw": {
          "type": "keyword"
        }
      },
      "type": "text"
    }
  }
}

I should point out however that the default automapping for a C# string property in NEST 5.0 is to map it as a text field with a keyword sub field with ignore_above:256. NEST 5.0 was released to nuget earlier this week

client.Map<Sample1>(m => m
    .AutoMap()
    .DynamicTemplates(dt => dt
        .DynamicTemplate("all_strings_multifields", t => t
            .MatchMappingType("text")
            .Mapping(tm => tm
                .Text(mf => mf
                    .Index(true)
                    .Fields(mff => mff
                        .Keyword(s => s
                            .Name("raw")
                        )
                    )
                )
            )
        )
    )
);

produces

{
  "dynamic_templates": [
    {
      "all_strings_multifields": {
        "match_mapping_type": "text",
        "mapping": {
          "type": "text",
          "fields": {
            "raw": {
              "type": "keyword"
            }
          },
          "index": true
        }
      }
    }
  ],
  "properties": {
    "iD": {
      "fields": {
        "keyword": {
          "ignore_above": 256,
          "type": "keyword"
        }
      },
      "type": "text"
    },
    "doNotIndex": {
      "type": "keyword",
      "index": false
    },
    "multiField1": {
      "fields": {
        "keyword": {
          "ignore_above": 256,
          "type": "keyword"
        }
      },
      "type": "text"
    },
    "multiField2": {
      "fields": {
        "keyword": {
          "ignore_above": 256,
          "type": "keyword"
        }
      },
      "type": "text"
    }
  }
}

Upvotes: 4

Related Questions