CODEWITHSUNDEEP
kibana
peter.murray.rust
peter.murray.rust

Reputation: 38071

how to set fielddata=true in kibana

I am new to Kibana, have data loaded into Elastic 5.0.0-alpha3 and am using Kibana 5.0.0-alpha3 to Visualise. I can display some numeric fields as histograms but when I want to use text fields I get:

Visualize: Fielddata is disabled on text fields by default. Set fielddata=true on [publisher] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.

I am warned that the data (names of Publishers) may have been analyzed into subfields, but I'd like to display anyway.

How do I set fielddata=true?

EDIT: recent issues on Kibana github suggest that this is a new feature in 5.0.0 and is still awaiting an answer!

EDIT (following @Val's answer, and asking for Elastic newbie help, and hoping others will find it useful). The ingest script was:

fs = require('fs')

var elasticsearch = require('elasticsearch');
var client = new elasticsearch.Client({
 host: 'localhost:9200',
 log: 'trace'
});

fs.readFile('/Users/pm286/workspace/cmdev/getpapers/20160602/crossref_results.json', (err, data) => {
  if (err) throw err;
   document = JSON.parse(data)
  document = JSON.parse(data)

  for(i=0;i<document.length;i++) {
      client.create({
          index: 'index',
          type: 'type',
          body: document[i]
          })
      }
  });

How do I include @Val's approach in this?

Upvotes: 66

Views: 141247

Answers (7)

addin
addin

Reputation: 146

PUT your_index/_mapping
{
  "properties": {
    "your_field": { 
      "type":     "text",
      "fielddata": true
    }
  }
}

source

Upvotes: 0

Giuseppe Sanna
Giuseppe Sanna

Reputation: 51

what is the field "your_type"?

PUT your_index/_mapping/your_type

Upvotes: 4

Ahmed
Ahmed

Reputation: 519

If you are coming from the book "ElasticsearchThe Definitive Guide", try changing this

"terms" : { "field" : "interests" },

to

"terms" : { "field" : "interests.keyword" },

So, the code to run will become;

GET /megacorp/employee/_search
{
  "aggs": {
    "all_interests": {
      "terms": { "field": "interests.keyword"}
    }
  }
}

Upvotes: 28

dpp.2325
dpp.2325

Reputation: 131

Enable fielddata on an existing text field, this is required to do aggregation on this field

PUT megacorp/_mapping/employee
{
  "properties": {
    "interests": { 
      "type":     "text",
      "fielddata": true
    }
  }
}

Upvotes: 2

Kris Holt
Kris Holt

Reputation: 81

This code fixes this problem.

PUT megacorp/_mapping/employee
{
   "employee": {
      "properties": {
        "interests": {
          "type": "text",
          "fielddata": true
        }
      }
   }
}

So this code will run thereafter:

GET /megacorp/employee/_search
    {
      "aggs": {
        "all_interests": {
          "terms": { "field": "interests"}
        }
      }
    }

Upvotes: 8

Patrick B
Patrick B

Reputation: 391

Since you're using Elastic 5.x (5.2 is out now as I write this), you should instead use the new keyword support instead of enabling fielddata on an indexed field.

https://www.elastic.co/guide/en/elasticsearch/reference/5.2/fielddata.html provides good information on the pros and cons and on how to set this. From the page:

PUT my_index
{
  "mappings": {
    "my_type": {
      "properties": {
        "my_field": { 
          "type": "text",
          "fields": {
            "keyword": { 
              "type": "keyword"
            }
          }
        }
      }
    }
  }
}

You then use the 'my_field' field for searches and the 'my_field.keyword' field for aggregations, sorting, or in scripts.

my_field.keyword is what you would use within Kibana / Grafana.

Upvotes: 39

Val
Val

Reputation: 217564

In your ES mapping, you need to set fielddata:true in your publisher field:

PUT your_index/_mapping/your_type
{
   "your_type": {
      "properties": {
        "publisher": {
          "type": "text",
          "fielddata": true
        }
      }
   }
}

You'll need to reindex your data after making this change, but afterwards Kibana won't complain anymore.

UPDATE

You can either execute the above query in the Sense UI or through curl

curl -XPUT http://localhost:9200/index -d '{
  "mappings": {
    "type": {
      "properties": {
        "publisher": {
          "type": "text",
          "fielddata": true
        }
      }
    }
  }
}'

Or you can also execute it in your Javascript file just before creating your document:

client.indices.create({
  index: 'index',
  body: {
      "mappings": {
        "type": {
          "properties": {
            "publisher": {
              "type": "text",
              "fielddata": true
            }
          }
        }
      }
    }
});

Upvotes: 60

Related Questions