CODEWITHSUNDEEP
elasticsearchcurl
khinester
khinester

Reputation: 3520

escape triple quotes in curl correctly

I have the following curl request

curl -H "Content-Type: application/json" -X POST http://localhost:9200/_reindex\?wait_for_completion\=true -d '{"source": {"index": "analytics-prod-2019.12.30", "size":1000 }, "dest": {"index": "analytics-prod-2019.12"}, "conflicts": "proceed", "script": { "lang": "painless","source: """ctx._source.index = ctx._index; def eventData = ctx._source["event.data"]; if(eventData != null) { eventData.remove("realmDb.size"); eventData.remove("realmDb.format"); eventData.remove("realmDb.contents"); }""" } }'

but this fails with the following error:

{"error":{"root_cause":[{"type":"x_content_parse_exception","reason":"[1:166] [script] failed to parse object"}],"type":"x_content_parse_exception","reason":"[1:166] [reindex] failed to parse field [script]","caused_by":{"type":"x_content_parse_exception","reason":"[1:166] [script] failed to parse object","caused_by":{"type":"json_parse_exception","reason":"Unexpected character ('\"' (code 34)): was expecting a colon to separate field name and value\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@51c48433; line: 1, column: 177]"}}},"status":400}

if i remove the script field from the request this works just fine:

curl -H "Content-Type: application/json" -X POST http://localhost:9200/_reindex\?wait_for_completion\=true -d '{"source":{"index":"analytics-prod-2019.12.30","size":1000},"dest":{"index":"test-index"},"conflicts":"proceed"}}'

using the kibana UI works fine.

what is the correct way to run this in curl?

Upvotes: 4

Views: 1596

Answers (3)

khinester
khinester

Reputation: 3520


curl -XPOST "http://elasticsearch:9200/_reindex?requests_per_second=115&wait_for_completion=true" -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "analytics-prod-2019.12.30",
    "size": 1000
  },
  "dest": {
    "index": "analytics-prod-2019.12"
  },
  "script": {
    "lang": "painless",   
    "source": "      ctx._source.index = ctx._index;\n      def eventData = ctx._source[\"event.data\"];\n      if (eventData != null) {\n        eventData.remove(\"realmDb.size\");\n        eventData.remove(\"realmDb.format\");\n        eventData.remove(\"realmDb.contents\");\n      }"
  }
}'

had to escape \"

Upvotes: 2

Joe - Check out my books
Joe - Check out my books

Reputation: 16905

Your source was missing a double quote: enter image description here

Corrected:

curl -H "Content-Type: application/json" \
     -X POST http://localhost:9200/_reindex\?wait_for_completion\=true \
     -d '{"source": {"index": "analytics-prod-2019.12.30", "size":1000 }, "dest": {"index": "analytics-prod-2019.12"}, "conflicts": "proceed", "script": { "lang": "painless","source": "ctx._source.index = ctx._index; def eventData = ctx._source[\"event.data\"]; if (eventData != null) { eventData.remove(\"realmDb.size\"); eventData.remove(\"realmDb.format\"); eventData.remove(\"realmDb.contents\"); }" } }'

You can either use single quotes like @Zsolt pointed out but even Kibana itself, when clicking "Copy as cURL", uses escaped double quotes.

Upvotes: 2

zsltg
zsltg

Reputation: 773

Use a single " to surround your script value and \u0027 to escape in your Painless script.

curl -H "Content-Type: application/json" -X POST http://localhost:9200/_reindex\?wait_for_completion\=true -d '
{
  "source": {
    "index": "analytics-prod-2019.12.30",
    "size": 1000
  },
  "dest": {
    "index": "analytics-prod-2019.12"
  },
  "conflicts": "proceed",
  "script": {
    "lang": "painless",
    "source": "ctx._source.index = ctx._index; def eventData = ctx._source[\u0027event.data\u0027]; if(eventData != null) { eventData.remove(\u0027realmDb.size\u0027); eventData.remove(\u0027realmDb.format\u0027); eventData.remove(\u0027realmDb.contents\u0027);"
  }
}
'

You can also see an example of this here, click on the Copy as cURL link and review the example in that format.

Upvotes: 3

Related Questions