Elasticsearch NEST PUT Mapping to Add Field / Property

Question

I am using the NEST Suggest.Completion query to provide suggestive search. I already have data in the index and wanted to add a new field "IsActive" to allow me to disable certain documents from appearing in the suggest.

I thought the NEST Map<> method would add the new field to all existing documents in the index when run, but it does not. Is there some way to make it work like that?

I am using Elasticsearch 6.8.0.

My Object with the new field

[ElasticsearchType(
       IdProperty = "search"
   )]
public class SearchCompletion
{
    public string search { get; set; }


    /// <summary>
    /// Use this field for aggregations and sorts
    /// </summary>
    [Keyword]
    public string search_keyword { get; set; }

    public bool isActive { get; set; }  // <---- This is the new field

    /// <summary>
    /// To use for sorting results
    /// since you can't sort by the Completionfield.Weight 
    /// property for some reason
    /// </summary>
    public int weight { get; set; }

    public CompletionField suggest { get; set; }
}

Method to Re-Apply Mapping

    public static void MapSearchCompletions(ElasticClient client, string index)
    {
        var mapResponse = client.Map<SearchCompletion>(m => m
            .Index(index)
            .AutoMap()
       ); //re-apply the index mapping
    } 

The PUT request

 PUT /local.project.tests.searchcompletions/searchcompletion/_mapping

{
    "properties": {
        "search": {
            "type": "text",
            "fields": {
                "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                }
            }
        },
        "search_keyword": {
            "type": "keyword"
        },
        "isActive": {
            "type": "boolean"
        },
        "weight": {
            "type": "integer"
        },
        "suggest": {
            "type": "completion"
        }
    }
}

The result of querying the index after the mapping

GET /local.project.tests.searchcompletions/searchcompletion/_search

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 1,
    "hits": [
      {
        "_index": "local.project.tests.searchcompletions",
        "_type": "searchcompletion",
        "_id": "the",
        "_score": 1,
        "_source": {
          "search": "the",
          "search_keyword": "the",
          "weight": 1,
          "suggest": {
            "input": [
              "the",
              "the"
            ],
            "weight": 1
          }
        }
      }
    ]
  }
}

Answer 1

Yes, updating mapping won't change existing documents. To do so, you can use update_by_query API.

var updateByQueryResponse = await client.UpdateByQueryAsync<Document>(u => u
    .Query(q => q.MatchAll())
    .Script("ctx._source.isActive = true")
    .Refresh());

Here is the full example:

class Program
{
    public class Document
    {
        public int Id { get; set; }
        public bool IsActive { get; set; }
    }

    static async Task Main(string[] args)
    {
        var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
        var connectionSettings = new ConnectionSettings(pool);
        connectionSettings.DefaultIndex("documents");

        var client = new ElasticClient(connectionSettings);

        var deleteIndexResponse = await client.Indices.DeleteAsync("documents");
        var createIndexResponse = await client.Indices.CreateAsync("documents", d => d
            .Map(m => m.AutoMap<Document>()));

        var indexDocument = await client.IndexDocumentAsync(new Document {Id = 1});

        var refreshAsync = client.Indices.RefreshAsync();

        var putMappingResponse = await client.MapAsync<Document>(m => m
            .AutoMap());

        var updateByQueryResponse = await client.UpdateByQueryAsync<Document>(u => u
            .Query(q => q.MatchAll())
            .Script("ctx._source.isActive = true")
            .Refresh());

        var response = await client.GetAsync<Document>(1);

        Console.WriteLine(response.Source.IsActive);
    }
}

Prints:

True

Hope that helps.