CODEWITHSUNDEEP
hadoopapache-flink
Dolphin
Dolphin

Reputation: 38671

how to store apache flink checkpoint in nfs filesystem

I am pulling data stream from RabbitMQ using Apache Flink 1.10.0, now I am using default checkpoint config in memory. Now to make it recovery when task manager restart, I need to store the state and checkpoint in filesystem, the all demo tell should using "hdfs://namenode:4000/....", but now I have no HDFS cluster, my Apache Flink is running in kubernetes cluster, how to store my check point in filesystem?

I read the docs of Apache Flink and tell me it support:

how to config flink to using NFS to store checkpoint and state? I search from internete and find no story about this solution.

Upvotes: 0

Views: 1311

Answers (1)

David Anderson
David Anderson

Reputation: 43499

To use NFS for checkpointing with Flink you should specify a checkpoint directory using a file: URI that is accessible from every node in the cluster (the job manager and all task managers need to have access using the same URI).

So, for example, you might mount your NFS volume at /data/flink/checkpoints on each machine, and then specify

state.checkpoints.dir: file:///data/flink/checkpoints

Upvotes: 1

Related Questions