CODEWITHSUNDEEP
apache-flinkflink-streaming
Ankit
Ankit

Reputation: 317

Flink v6 rest api : how to get all the finished/completed jobs

In Flink v-1.4, there was a rest api to get all the finished/completed jobs :

/joboverview/completed

As i can see there is no such api in v-1.6, how can i get ONLY the finished/completed jobs.

Upvotes: 1

Views: 944

Answers (1)

David Anderson
David Anderson

Reputation: 43707

You can do this by configuring and running a History Server.

In flink-conf.yaml you'll find a section for the History Server. To test this locally I tried these settings:

#==============================================================================
# HistoryServer
#==============================================================================

# The HistoryServer is started and stopped via bin/historyserver.sh (start|stop)

# Directory to upload completed jobs to. Add this directory to the list of
# monitored directories of the HistoryServer as well (see below).
jobmanager.archive.fs.dir: file:///tmp/completed-jobs/

# The address under which the web-based HistoryServer listens.
#historyserver.web.address: 0.0.0.0

# The port under which the web-based HistoryServer listens.
#historyserver.web.port: 8082

# Comma separated list of directories to monitor for completed jobs.
historyserver.archive.fs.dir: file:///tmp/completed-jobs/

# Interval in milliseconds for refreshing the monitored directories.
historyserver.archive.fs.refresh-interval: 10000

I created /tmp/completed-jobs, restarted my cluster, and started the history server, after which I was able to see completed jobs at http://localhost:8082 (html) and at http://localhost:8082/jobs/overview (json).

See the list of available requests for more info on the API.

Upvotes: 1

Related Questions