CODEWITHSUNDEEP
springhadoopspring-xd
Rodrigo
Rodrigo

Reputation: 1

spring xd stream definition dynamic parameters

We need to use the HDFS sink to store some data in HDFS. When creating the stream definition, I can use the "directory" property to specify the hdfs directory where we want to save the file. However, in our use case we want the directory to be dynamic based on the timestamp. I was thinking if it's possible to maybe use some SpEL in the directory property of the HDFS sink to maybe extract the timestamp from the header?

Thanks!!

Upvotes: 0

Views: 457

Answers (1)

mfisher
mfisher

Reputation: 506

Rodrigo already knows this, but for anyone else checking out this thread, see the explanation of the new functionality by searching for "partitionPath" under this section of the reference doc: https://github.com/spring-projects/spring-xd/wiki/Sinks#hadoop-hdfs

Upvotes: 1

Related Questions