CODEWITHSUNDEEP
amazon-web-serviceselasticsearchgroovyamazon-elasticsearch
sidd607
sidd607

Reputation: 1409

Scripting in AWS elasticsearch

I read online about scripting in aws elacticsearch service. It said that AWS ES doesn't support dynamic scripting so I am writing aggregations using scripts stored in my disk. I wrote the following query

{
    "query":{
        "match_all":{}
    },
    "aggs":{
        "inBoundRecieved":{
            "scripted_metric":{
                "init_script":{
                    "file": "init.groovy"
                },
                "map_script": {
                    "file": "map.groovy"
                },
                "combine_script": {
                    "file":"comb.groovy"
                },
                "params":{
                    "field":"call_direction"
                },
                "reduce_script": {
                    "file": "red.groovy"
                }
            }
        }
    }

}

But I keep getting this error. 

Parse Failure [Unknown key for a START_OBJECT in [inBoundRecieved]: [init_script]

I have searched a lot online but couldn't find a good solution. Full Error ->

{ 
 "error" : "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[M-Sp4ZKmQCW0C4Ph2FIA1Q][plivoredshift][0]:                 RemoteTransportException[[Merlin][inet[/x.x.x.x:y]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[plivoredshift][0]: query[ConstantScore(*:*)],  from[-1],size[-1]: Parse Failure [Failed to parse source [{    \"query\":{        \"match_all\":{}    },    \"aggs\":{        \"inBoundRecieved\":{                       \"scripted_metric\":{                \"init_script\":{                    \"file\": \"init.groovy\"                },                \"map_script\": {                    \"file\": \"map.groovy\"                },                \"combine_script\": {                    \"file\":\"comb.groovy\"                },                \"params\":  {                    \"field\":\"call_direction\"                },                \"reduce_script\": {                    \"file\": \"red.groovy\"                       }            }        }    }}]]]; nested: SearchParseException[[plivoredshift][0]: query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Unknown key for a          START_OBJECT in [inBoundRecieved]: [init_script].]]; }{[M-Sp4ZKmQCW0C4Ph2FIA1Q][plivoredshift][1]: RemoteTransportException[[Merlin][inet[/x.x.x.x:y]][indices:data/read/ search[phase/query]]]; nested: SearchParseException[[plivoredshift][1]: query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Failed to parse source [{             \"query\":{        \"match_all\":{}    },    \"aggs\":{        \"inBoundRecieved\":{            \"scripted_metric\":{                \"init_script\":{                    \"file\": \"init.groovy\"                },                \"map_script\": {                    \"file\": \"map.groovy\"                },                                \"combine_script\": {                    \"file\":\"comb.groovy\"                },                \"params\":{                    \"field\":                             \"call_direction\"                },                \"reduce_script\": {                    \"file\": \"red.groovy\"                }            }        }    }}]]];     nested: SearchParseException[[plivoredshift][1]: query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Unknown key for a START_OBJECT in [inBoundRecieved]:         [init_script].]]; }{[M-Sp4ZKmQCW0C4Ph2FIA1Q][plivoredshift][2]: RemoteTransportException[[Merlin][inet[/x.x.x.x:y]][indices:data/read/search[phase/query]]]; nested:      SearchParseException[[plivoredshift][2]: query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Failed to parse source [{    \"query\":{        \"match_all\":{}     },    \"aggs\":{        \"inBoundRecieved\":{            \"scripted_metric\":{                \"init_script\":{                    \"file\": \"init.                      groovy\"                },                \"map_script\": {                    \"file\": \"map.groovy\"                },                \"combine_script\":              {                    \"file\":\"comb.groovy\"                },                \"params\":{                    \"field\":\"call_direction\"                               },                \"reduce_script\": {                    \"file\": \"red.groovy\"                }            }        }    }}]]]; nested:                               SearchParseException[[plivoredshift][2]: query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Unknown key for a START_OBJECT in [inBoundRecieved]: [init_script].  ]]; }{[M-Sp4ZKmQCW0C4Ph2FIA1Q][plivoredshift][3]: RemoteTransportException[[Merlin][inet[/x.x.x.x:y]][indices:data/read/search[phase/query]]]; nested:                    SearchParseException[[plivoredshift][3]: query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Failed to parse source [{    \"query\":{        \"match_all\":{}     },    \"aggs\":{        \"inBoundRecieved\":{            \"scripted_metric\":{                \"init_script\":{                    \"file\": \"init.                      groovy\"                },                \"map_script\": {                    \"file\": \"map.groovy\"                },                \"combine_script\":              {                    \"file\":\"comb.groovy\"                },                \"params\":{                    \"field\":\"call_direction\"                               },                \"reduce_script\": {                    \"file\": \"red.groovy\"                }            }        }    }}]]]; nested:                               SearchParseException[[plivoredshift][3]: query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Unknown key for a START_OBJECT in [inBoundRecieved]: [init_script].
"status":400
}

Here are my scripts init.groovy

_agg['transactions'] = []

map.groovy

if (doc['call_direction']=="inbound" {_agg.transactions.add(1)} else {_agg.transactions.add(0)}

comb.groovy

inBoundRecieved=0; for( t in _agg.transactions) {inBoundRecieved+=t}; return inBoundRecieved

red.groovy

inBoundRecieved=0; for( a in _aggs) {inBoundRecieved += a}; return inBoundRecieved

I have been following this tutorial from the ElacticSearch website

Upvotes: 3

Views: 4312

Answers (1)

user1383029
user1383029

Reputation: 2125

Looks like scripting can be used in AWS elasticsearch service for version 5 now:

http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-supported-resources.html

https://forums.aws.amazon.com/thread.jspa?threadID=217896&start=25&tstart=0

Upvotes: 4

Related Questions