CODEWITHSUNDEEP
pipelinehere-olp
Venkata Kalyan
Venkata Kalyan

Reputation: 11

Pipeline configuration in HERE OLP

I configured a new pipeline in HERE OLP. Having issues while running it. If anybody has past experience dealing with here olp pipeline, please advise. 

2019-08-16 18:21:00.946 [SIGTERM handler] level=ERROR org.apache.spark.executor.CoarseGrainedExecutorBackend  - RECEIVED SIGNAL TERM // PipeLineId=63e07977-ba0a-4997-a633-1b578ae86ce1 Realm=olp-ford DeploymentId=4832de82-afb9-4f03-b92a-a6ba12eb9116 VersionId=0bc6df20-8873-4195-bfdb-87271a95fdba
3-4195-bfdb-87271a95fdba
com.here.platform.data.processing.exception.ContextRuntimeException:  // Task=seedmap cat=highways l=adas-attributes p=18939748
    at com.here.platform.data.processing.exception.ContextRuntimeException$.apply(ContextRuntimeException.scala:46)
    at com.here.platform.data.processing.exception.package$.wrapInTryWithContext(package.scala:109)
    at com.here.platform.data.processing.driver.impl.executors.DirectCompileFnUtils$$anonfun$applyMappingFn$1$$anonfun$apply$5.apply(DirectCompileFnUtils.scala:83)
    at com.here.platform.data.processing.driver.impl.executors.DirectCompileFnUtils$$anonfun$applyMappingFn$1$$anonfun$apply$5.apply(DirectCompileFnUtils.scala:82)
    at com.here.platform.data.processing.logging.Implicits$LogContextKeyWrapper$.withChild$extension(Implicits.scala:46)
    at com.here.platform.data.processing.driver.impl.executors.DirectCompileFnUtils$$anonfun$applyMappingFn$1.apply(DirectCompileFnUtils.scala:82)
    at com.here.platform.data.processing.driver.impl.executors.DirectCompileFnUtils$$anonfun$applyMappingFn$1.apply(DirectCompileFnUtils.scala:81)
    at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
    at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
    at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:434)
    at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
    at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
    at org.apache.spark.scheduler.Task.run(Task.scala:99)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: requirement failed: Compile function returned partition key not compatible with the layer: Key('output,'seed-map,18939748)
    at scala.Predef$.require(Predef.scala:224)
    at com.here.platform.data.processing.driver.impl.executors.ExecutorUtils$.verifyPartitionKey(ExecutorUtils.scala:291)
    at com.here.platform.data.processing.driver.impl.executors.DirectCompileFnUtils$$anonfun$applyMappingFn$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$7.apply(DirectCompileFnUtils.scala:85)
    at com.here.platform.data.processing.driver.impl.executors.DirectCompileFnUtils$$anonfun$applyMappingFn$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$7.apply(DirectCompileFnUtils.scala:85)
    at scala.collection.immutable.List.foreach(List.scala:381)
    at com.here.platform.data.processing.driver.impl.executors.DirectCompileFnUtils$$anonfun$applyMappingFn$1$$anonfun$apply$5$$anonfun$apply$6.apply(DirectCompileFnUtils.scala:85)
    at com.here.platform.data.processing.driver.impl.executors.DirectCompileFnUtils$$anonfun$applyMappingFn$1$$anonfun$apply$5$$anonfun$apply$6.apply(DirectCompileFnUtils.scala:83)
    at scala.util.Try$.apply(Try.scala:192)
    at com.here.platform.data.processing.exception.package$.wrapInTryWithContext(package.scala:100)
    ... 17 common frames omitted

Upvotes: -1

Views: 233

Answers (2)

PTT
PTT

Reputation: 566

Also check Partition Type and Zoom level of layer.

Upvotes: 0

Andriy Budzinskyy
Andriy Budzinskyy

Reputation: 2001

The problem might be related to the partition type of layer. Make sure the returned key has the correct Partition.Name type. If your layer is generically partitioned you have to build the partition name with Generic(), and not HereTile().

Upvotes: 0

Related Questions