elasticsearch: multiple types in a single index

Question

I'm trying to create multiple types in a single index. For example I'm trying to create two types (host,post) in ytb index in order to create parent-child relationship between of them.

PUT /ytb
{
  "mappings": {
      "post": {
          "_parent": {
              "type": "host" 
            },
          "properties":{
            "@timestamp": {
                  "type": "date"
              },
            "indexed": {
                  "type": "date"
              },
              "n_comments": {
                  "type": "long"
              }, 
              "n_harvested": {
                  "type": "long"
              }, 
              "n_likes": {
                  "type": "long"
              },
              "network": {
                  "fields": {
                      "keyword": {
                          "ignore_above": 256, 
                          "type": "keyword"
                      }
                  }, 
                  "type": "text"
              },
              "parent_id": {
                  "fields": {
                      "keyword": {
                          "ignore_above": 256, 
                          "type": "keyword"
                      }
                  }, 
                  "type": "text"
              }, 
              "post_dbid": {
                  "type": "long"
              }, 
              "post_id": {
                  "fields": {
                      "keyword": {
                          "ignore_above": 256, 
                          "type": "keyword"
                      }
                  }, 
                  "type": "text"
              }, 
              "post_netid": {
                  "fields": {
                      "keyword": {
                          "ignore_above": 256, 
                          "type": "keyword"
                      }
                  }, 
                  "type": "text"
              }, 
              "published": {
                  "type": "date"
              }
          }
      },
      "host": {
          "properties": {
              "@timestamp": {
                  "type": "date"
              }, 
              "@version": {
                  "fields": {
                      "keyword": {
                          "ignore_above": 256, 
                          "type": "keyword"
                      }
                  }, 
                  "type": "text"
              }, 
              "country": {
                  "fields": {
                      "keyword": {
                          "ignore_above": 256, 
                          "type": "keyword"
                      }
                  }, 
                  "type": "text"
              }, 
              "host_dbid": {
                  "type": "long"
              }, 
              "host_id": {
                  "fields": {
                      "keyword": {
                          "ignore_above": 256, 
                          "type": "keyword"
                      }
                  }, 
                  "type": "text"
              }, 
              "host_netid": {
                  "fields": {
                      "keyword": {
                          "ignore_above": 256, 
                          "type": "keyword"
                      }
                  }, 
                  "type": "text"
              }, 
              "id": {
                  "fields": {
                      "keyword": {
                          "ignore_above": 256, 
                          "type": "keyword"
                      }
                  }, 
                  "type": "text"
              }, 
              "indexed": {
                  "type": "date"
              }, 
              "language": {
                  "fields": {
                      "keyword": {
                          "ignore_above": 256, 
                          "type": "keyword"
                      }
                  }, 
                  "type": "text"
              },
              "name": {
                  "fields": {
                      "keyword": {
                          "ignore_above": 256, 
                          "type": "keyword"
                      }
                  }, 
                  "type": "text"
              }, 
              "vertical": {
                  "fields": {
                      "keyword": {
                          "ignore_above": 256, 
                          "type": "keyword"
                      }
                  }, 
                  "type": "text"
              }
          }
      }
  }
}

but I'm getting this error:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Rejecting mapping update to [ytb] as the final mapping would have more than 1 type: [post, host]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Rejecting mapping update to [ytb] as the final mapping would have more than 1 type: [post, host]"
  },
  "status": 400
}

UPDATE: Elasticsearch version: 6.3.0

Answer 1

If you have ES 5.6 or more recent, you need to read this. To sum it up, mapping types are going to be removed and only one type per index is going to be the norm as of ES 6 onwards.

To answer your question in the comment, I know of another Kibana-like tool (a Kibana fork actually) that knows how to handle JOINs and relational data. It is called Kibi by Siren Solutions. Also read the blog announcement.

UPDATE:

The Kibi project doesn't seem to be maintained anymore. If you want to do JOINs now, you might want to invest some time into learning the new ES|QL language that allows you to do search-time enrichments (very similar to JOINs)

Answer 2

This blog is very detailed for reasons, disadvantages and workarounds: https://www.elastic.co/guide/en/elasticsearch/reference/6.1/removal-of-types.html

To summarize:

• reason: different types in the same index is not independent. Common fields in these types should have the same data type, because they are backed by the same Lucene field internally.
• disadvantages: two types with few common fields will cause sparse data storage.
• workaround: define a custom type having all the fields in the two types, with a custom type(or something like this), and CRUD using the custom type field.