Reputation: 37
I want to setup kafka druid ingestion but Even after configuring in common.runtime.properties and adding druid - kafka indexing service it s still giving me error . Please help me out with this. My data is in csv format.
{
"type": "kafka",
"spec": {
"dataSchema": {
"dataSource": "london_crime_by_lsoa",
"parser": {
"type": "string",
"parseSpec": {
"format": "csv",
"dimensionsSpec": {
"dimensions": [
"lsoa_code",
"borough",
"major_category",
"minor_category",
{
"name": "value",
"type": "long"
},
{
"name": "year",
"type": "long"
},
{
"name": "month",
"type": "long"
}
]
},
"timestampSpec": {
"column": "year",
"format": "auto"
},
"columns": [
"lsoa_code",
"borough",
"major_category",
"minor_category",
"value",
"year",
"month"
]
}
},
"metricsSpec": [],
"granularitySpec": {
"type": "uniform",
"segmentGranularity": "year",
"queryGranularity": "NONE",
"rollup": false
}
},
"ioConfig": {
"topic": "london_crime_by_lsoa",
"taskDuration": "PT10M",
"useEarliestOffset": "true",
"consumerProperties": {
"bootstrap.servers": "localhost:9092"
}
},
"tuningConfig": {
"type": "kafka",
"maxRowsPerSegment": 500000
}
}
}
After executing this command:
curl -XPOST -H'Content-Type: application/json' -d @quickstart/tutorial/crime_supervisor.json http://localhost:8090/druid/indexer/v1/supervisor
I am getting this error :
{"error":"Instantiation of [simple type, class org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorSpec] value failed: dataSchema"}
Upvotes: 1
Views: 1245
Reputation: 2452
I think it's a problem with the way you specified spec in your JSON.
You have to specify dataSchema
directly in your JSON instead of as a child attribute of spec
.
Here's the format you should follow:
{
"type": "kafka",
"dataSchema": {},
"tuningConfig": {},
"ioConfig": {}
}
Upvotes: 1