CODEWITHSUNDEEP
hadoopapache-flinkflink-streaming
codeX
codeX

Reputation: 5428

Apache Flink : Available task slots 0

I have started a Flink Cluster via start-cluster.bat in the windows terminal.

Starting a local cluster with one JobManager process and one TaskManager process. You can terminate the processes via CTRL-C in the spawned shell windows. Web interface by default on http://localhost:8081/.

I was able to access the localhost URL but the Available Task Slots is given as 0. By default, it is expected to have one task (as configured in yamlfile). Anyone had a similar issue? enter image description here

Upvotes: 1

Views: 3752

Answers (3)

leechor
leechor

Reputation: 26

add these parameters to flink-conf.yml

taskmanager.resource-id: tm-resource-1
taskmanager.memory.network.min: 166429984b 
taskmanager.cpu.cores: 4.0 
taskmanager.memory.task.off-heap.size: 0b 
taskmanager.memory.jvm-metaspace.size: 268435456b 
external-resources: none 
taskmanager.memory.jvm-overhead.min: 214748368b 
taskmanager.memory.framework.off-heap.size: 134217728b 
taskmanager.memory.network.max: 166429984b 
taskmanager.memory.framework.heap.size: 134217728b 
taskmanager.memory.managed.size: 665719939b 
taskmanager.memory.task.heap.size: 563714445b 
taskmanager.numberOfTaskSlots: 20 
taskmanager.memory.jvm-overhead.max: 214748368b

Upvotes: 0

Yanwt
Yanwt

Reputation: 27

One possible solution:
Attach the remote JVM Debugger to TaskManager.
or: suspend=n My Case:

        FLINK_PROPERTIES=
    jobmanager.rpc.address: jobmanager
    taskmanager.memory.process.size: 2728m
    taskmanager.memory.flink.size: 2280m
    taskmanager.numberOfTaskSlots: 4
    env.java.opts: "-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=*:5005"
    parallelism.default: 4

Before:
The JobManager and TaskManager can be started normally with memory correctly configured. TaskManager log:

Starting taskexecutor as a console application on host bb166efde8f4.

Listening for transport dt_socket at address: 5005

After submitting the Job, it tries to create for about 3 minutes. And then fails with NoResourceAvailableException. Also, the dashboard show available task slot is 0.

Solution:

  • Set Remote JVM Debug in IDEA as follows:
    -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 attach to remote jvm at localhost 5005
  • launch debugger.
  • submit the job.
    Or you can set suspend=n instead of suspend=y in env var.
    Now the available task slot is 4.

Upvotes: 0

keven
keven

Reputation: 46

Have your processes started yet? It is possible that your JobManager startup failed. It is also possible that your system is running out of checks.

Upvotes: 0

Related Questions