CODEWITHSUNDEEP
jsoncurlelasticsearch
Kausty
Kausty

Reputation: 859

Elasticsearch query to retrieve particular _source value for a particular type for all ID's

Is there a way in CURL/JSON to retrieve the source values of a particular type for all the ID's that exists in the index? So for example, in the sample data below, I need to get the _id_batch (source value) for all the id's of the type (document). The _source_include works at the ID level but I wanted to retrieve all the values for _source_include for all the ID's of the type in the index.

Once I get there, I would then like to limit the data in the output so give me the _id_batch only for those records which have maybe something like, "location":"west palm beach".

Something like (* is just for example to select all the values):

curl -X GET "http://<IP Server>:9200/<index_name>/document/*/_source?_source_include=_id_batch" -d "
{
"from": 0,
"fields":[],
"size":50,
"sort":[]
}
" > "C:\Users\user\Downloads\output.json"

Sample Data:

URL: http://IPServer:9200/index_name/document

{
  "_index" : "<index_name>",
  "_type" : "document",
  "_id" : "11",
  "_version" : 1,
  "found" : true,
  "_source":{"_id_batch":1001688,"_id_document":11,"name":"xxx","location":"west palm beach","tweet":"0.0"}
}

{
  "_index" : "<index_name>",
  "_type" : "document",
  "_id" : "12",
  "_version" : 1,
  "found" : true,
  "_source":{"_id_batch":1001689,"_id_document":12,"name":"yyy","location":"west palm beach","tweet":"0.0"}
}

Expected Output:

{"_id_batch":1001688}
{"_id_batch":1001689}

Sample Query (using command line hence the ""):

curl -X GET "http://<IP Server>:9200/<index_name>/document/11/_source?_source_include=_id_batch" -d "
{
    "from": 0,
    "fields":[],
    "size":50,
    "sort":[]
}
" > "C:\Users\user\Downloads\output.json"

Sample Output in the JSON file:

{"_id_batch":1001688}

Tried the call -

curl -XPOST "http://<IP Server>:9200/<index_name>/document/_search" -d "{"query" : { "match_all" : {} },"size" : <LARGE_NUM>,"_source" : [ "_id_batch" ]}"

Got the Error -

{"error":"SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[dyYdbJLMQeC5XvXE0wwA9Q][<index_name>][0]: RemoteTransportException[[<server>][inet[/<IP Server>:9300]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[<index_name>][0]: query[ConstantScore(*:*)],from[-1],size[50]: Parse Failure [Failed to parse source [{query : { match_all : {} },size : 50,_source : [ _id_batch ]}]]]; nested: JsonParseException[Unrecognized token '_id_batch': was expecting ('true', 'false' or 'null')\n at [Source: UNKNOWN; line: 1, column: 61]]; }{[dyYdbJLMQeC5XvXE0wwA9Q][<index_name>][1]: RemoteTransportException[[<server>][inet[/<IP Server>:9300]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[<index_name>][1]: query[ConstantScore(*:*)],from[-1],size[50]: Parse Failure [Failed to parse source [{query : { match_all : {} },size : 50,_source : [ _id_batch ]}]]]; nested: JsonParseException[Unrecognized token '_id_batch': was expecting ('true', 'false' or 'null')\n at [Source: UNKNOWN; line: 1, column: 61]]; }{[31mSI_d3SPmTf7cU1dSNlQ][<index_name>][2]: RemoteTransportException[[D1-C04-WAPP02][inet[/<IP>:9300]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[<index_name>][2]: query[ConstantScore(*:*)],from[-1],size[50]: Parse Failure [Failed to parse source [{query : { match_all : {} },size : 50,_source : [ _id_batch ]}]]]; nested: JsonParseException[Unrecognized token '_id_batch': was expecting ('true', 'false' or 'null')\n at [Source: UNKNOWN; line: 1, column: 61]]; }{[q2tWPLiCQR-O2zXRZBuQYg][<index_name>][3]: SearchParseException[[<index_name>][3]: query[ConstantScore(*:*)],from[-1],size[50]: Parse Failure [Failed to parse source [{query : { match_all : {} },size : 50,_source : [ _id_batch ]}]]]; nested: JsonParseException[Unrecognized token '_id_batch': was expecting ('true', 'false' or 'null')\n at [Source: [B@4fccd33b; line: 1, column: 61]]; }{[q2tWPLiCQR-O2zXRZBuQYg][<index_name>][4]: SearchParseException[[<index_name>][4]: query[ConstantScore(*:*)],from[-1],size[50]: Parse Failure [Failed to parse source [{query : { match_all : {} },size : 50,_source : [ _id_batch ]}]]]; nested: JsonParseException[Unrecognized token '_id_batch': was expecting ('true', 'false' or 'null')\n at [Source: [B@4fccd33b; line: 1, column: 61]]; }]","status":400}

Upvotes: 0

Views: 1542

Answers (1)

eemp
eemp

Reputation: 1166

I think you want something like this:

curl -XPOST "http://<IP Server>:9200/<index_name>/document/_search" -d '{
    "query" : { "match_all" : {} },
    "size" : <LARGE_NUM>,
    "_source" : [ "_id_batch" ]
}'

where LARGE_NUM is a large number >> count of all records of type 'document'

So you are specifying the type or mapping ('document') via the endpoint you are hitting and you are querying for all Elasticsearch records for that mapping.

Upvotes: 1

Related Questions