CODEWITHSUNDEEP
pythonelasticsearch
kevin.w.johnson
kevin.w.johnson

Reputation: 1794

Elasticsearch multi search not returning correct number of results

I'm using Elasticsearch's multi search to run a list of queries through the python library. It seems to be working... mostly. The problem is that if I send up 200 queries, it returns 100 results (no errors). I've tried running even 20 queries and still get 10 results. For some reason it seems to always return only half of the queries. Which also means I don't know which query the results correlate to.

The queries all work individually but would take too long. There's no mention in the docs of the possibility of the response not containing the same number of queries so I'm not sure where to go here. I've stepped into the code as much as the python library will let me to verify all of the queries are being sent up and it seems to work as expected.

Thanks for any help or a push in the right direction!

Edit: Here's an example of the query I use.

   {  
   "filter":{  
      "and":[  
         {  
            "not":{  
               "term":{  
                  "uuid":"60507f9e-01c1-11e5-a369-34363bd16ec4"
               }
            }
         },
         {  
            "term":{  
               "brand_id":22212
            }
         }
      ]
   },
   "query":{  
      "bool":{  
         "minimum_should_match":1,
         "should":[  
            {  
               "match":{  
                  "name":{  
                     "query":"spork",
                     "boost":3
                  }
               }
            },
            {  
               "match":{  
                  "categories":{  
                     "slop":10,
                     "type":"phrase",
                     "query":"household plates, bowls, cups & flatware"
                  }
               }
            }
         ],
         "must_not":[  

         ],
         "must":[  
            {  
               "match":{  
                  "name_analyzed":{  
                     "boost":4,
                     "query":"spork",
                     "type":"phrase",
                     "slop":15
                  }
               }
            }
         ]
      }
   }
}

I create a list of however many queries like these I'd like to run and execute them as so, as per the python elasticsearch documentation:

elasticsearch.msearch(list_of_search_queries)

Upvotes: 8

Views: 2297

Answers (1)

kevin.w.johnson
kevin.w.johnson

Reputation: 1794

Whew! That was annoying.

So the caveat between search and msearch is that elasticsearch expects two lines per query. The first line indicates the index (can be left blank if you've already specified it) and the second line is your actual query.

So when creating my list of queries, I just append {} right before it as so:

list_of_queries = []
for thing in list_of_things:
    list_of_queries.append({})
    list_of_queries.append(thing.get_query())

Then it works!

Upvotes: 16

Related Questions