CODEWITHSUNDEEP
elasticsearchlogstashfilebeatmetricbeatelastic-beats
DevBot
DevBot

Reputation: 509

Elastic Beats - Changing the Field Type of Default Fields in Beats Documents?

I'm still fairly new to the Elastic Stack and I'm still not seeing the entire picture from what I'm reading on this topic.

Let's say I'm using the latest versions of Filebeat or Metricbeat for example, and pushing that data to Logstash output, (which is then configured to push to ES). I want an "out of the box" field from one of these beats to have its field type changed (example: change beat.hostname from it's current default "text" type to "keyword"), what is the best place/practice for configuring this? This kind of change is something I would want consistent across multiple hosts running the same Beat.

Upvotes: 0

Views: 643

Answers (3)

Zuhair
Zuhair

Reputation: 287

The other answers are correct but I did the below in Dev console to update the message field from text to text & keyword

PUT /index_name/_mapping
{
  "properties": {
    "message": {
        "type": "match_only_text",
        "fields": {
          "keyword": { 
            "type":  "keyword",
             "ignore_above": 10000
          }
        }
      }
  }
}

Upvotes: 0

chandraP
chandraP

Reputation: 81

Agreed with @xeraa. It is not advised to change the default template since that field might be used in any default visualizations.

Create a new template, you can have multiple templates for the same index pattern. All the mappings will be merged.The order of the merging can be controlled using the order parameter, with lower order being applied first, and higher orders overriding them.

For your case, probably create a multi-field for any field that needs to be changed. Eg: As shown here create a new keyword multifield, then you can refer the new field as

fieldname.raw

.

"properties": {
      "city": {
        "type": "text",
        "fields": {
          "raw": { 
            "type":  "keyword"
          }
        }
      }
    }

Upvotes: 0

xeraa
xeraa

Reputation: 10859

I wouldn't change any existing fields since Kibana is building a lot of visualizations, dashboards, SIEM,... on the exptected fields + data types.

Instead extend (add, don't change) the default mapping if needed. On top of the default index template, you can add your own and they will be merged. Adding more fields will require some more disk space (and probably memory when loading), but it should be manageable and avoids a lot of drawbacks of other approaches.

Upvotes: 1

Related Questions