CODEWITHSUNDEEP
node.jsffmpegfluent-ffmpegsigkill
jaimish11
jaimish11

Reputation: 564

FFMPEG randomly gets killed with SIGKILL on fluent-ffmpeg

I'm working on an Electron app with a Node backend and a React frontend.

I'm using fluent-ffmpeg to stream data from an external RTSP feed and render the feed on my app. The feed renders and displays fine but it's randomly killed after a few minutes. (sometimes within seconds, it's unreliable)

This is the error message I get: Error: ffmpeg was killed with signal SIGKILL

I printed out the stderr variable within one of the library's core files and this is the error that is thrown whenever a SIGKILL occurs:

[rtsp @ 0x7fc9ad80be00] RTP: missed 1 packets
frame= 4292 fps= 24 q=24.8 size=N/A time=00:02:58.21 bitrate=N/A dup=55 drop=5 speed=   1x
[rtsp @ 0x7fc9ad80be00] max delay reached. need to consume packet
[rtsp @ 0x7fc9ad80be00] RTP: missed 1 packets
frame= 4310 fps= 24 q=24.8 size=N/A time=00:02:58.96 bitrate=N/A dup=56 drop=5 speed=1.01x
frame= 4322 fps= 24 q=24.8 size=N/A time=00:02:59.46 bitrate=N/A dup=56 drop=5 speed=1.01x
frame= 4334 fps= 24 q=24.8 size=N/A time=00:02:59.95 bitrate=N/A dup=56 drop=5 speed=1.01x
frame= 4347 fps= 24 q=24.8 size=N/A time=00:03:00.49 bitrate=N/A dup=56 drop=5 speed=1.01x
frame= 4360 fps= 24 q=24.8 size=N/A time=00:03:01.03 bitrate=N/A dup=56 drop=5 speed=1.01x
frame= 4372 fps= 24 q=24.8 size=N/A time=00:03:01.53 bitrate=N/A dup=56 drop=5 speed=1.01x
frame= 4384 fps= 24 q=24.8 size=N/A time=00:03:02.03 bitrate=N/A dup=56 drop=5 speed=1.01x
frame= 4396 fps= 24 q=24.8 size=N/A time=00:03:02.53 bitrate=N/A dup=56 drop=5 speed=1.01x
frame= 4408 fps= 24 q=24.8 size=N/A time=00:03:03.03 bitrate=N/A dup=56 drop=5 speed=1.01x
frame= 4419 fps= 24 q=24.8 size=N/A time=00:03:03.48 bitrate=N/A dup=56 drop=5 speed=   1x
frame= 4432 fps= 24 q=24.8 size=N/A time=00:03:04.02 bitrate=N/A dup=56 drop=5 speed=1.01x
frame= 4445 fps= 24 q=24.8 size=N/A time=00:03:04.56 bitrate=N/A dup=56 drop=5 speed=1.01x
frame= 4457 fps= 24 q=24.8 size=N/A time=00:03:05.06 bitrate=N/A dup=56 drop=5 speed=1.01x
frame= 4469 fps= 24 q=24.8 size=N/A time=00:03:05.56 bitrate=N/A dup=56 drop=5 speed=1.01x
frame= 4480 fps= 24 q=24.8 size=N/A time=00:03:06.02 bitrate=N/A dup=56 drop=5 speed=   1x
frame= 4497 fps= 24 q=24.8 size=N/A time=00:03:06.72 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4512 fps= 24 q=24.8 size=N/A time=00:03:07.34 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4524 fps= 24 q=24.8 size=N/A time=00:03:07.84 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4537 fps= 24 q=24.8 size=N/A time=00:03:08.38 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4549 fps= 24 q=24.8 size=N/A time=00:03:08.88 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4561 fps= 24 q=24.8 size=N/A time=00:03:09.38 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4573 fps= 24 q=24.8 size=N/A time=00:03:09.88 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4585 fps= 24 q=24.8 size=N/A time=00:03:10.38 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4597 fps= 24 q=24.8 size=N/A time=00:03:10.87 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4609 fps= 24 q=24.8 size=N/A time=00:03:11.37 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4622 fps= 24 q=24.8 size=N/A time=00:03:11.91 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4634 fps= 24 q=24.8 size=N/A time=00:03:12.41 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4646 fps= 24 q=24.8 size=N/A time=00:03:12.91 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4658 fps= 24 q=24.8 size=N/A time=00:03:13.41 bitrate=N/A dup=57 drop=5 speed=   1x
frame= 4671 fps= 24 q=24.8 size=N/A time=00:03:13.95 bitrate=N/A dup=57 drop=5 speed=   1x
frame= 4687 fps= 24 q=24.8 size=N/A time=00:03:14.61 bitrate=N/A dup=57 drop=5 speed=   1x
frame= 4699 fps= 24 q=24.8 size=N/A time=00:03:15.11 bitrate=N/A dup=57 drop=5 speed=   1x
frame= 4712 fps= 24 q=24.8 size=N/A time=00:03:15.65 bitrate=N/A dup=57 drop=5 speed=1.01x
frame= 4722 fps= 24 q=24.8 size=N/A time=00:03:16.06 bitrate=N/A dup=57 drop=5 speed=   1x
frame= 4738 fps= 24 q=24.8 size=N/A time=00:03:16.73 bitrate=N/A dup=57 drop=5 speed=   1x
frame= 4750 fps= 24 q=24.8 size=N/A time=00:03:17.23 bitrate=N/A dup=57 drop=5 speed=   1x
frame= 4762 fps= 24 q=24.8 size=N/A time=00:03:17.73 bitrate=N/A dup=57 drop=5 speed=   1x
frame= 4774 fps= 24 q=24.8 size=N/A time=00:03:18.22 bitrate=N/A dup=57 drop=5 speed=   1x
frame= 4790 fps= 24 q=24.8 size=N/A time=00:03:18.89 bitrate=N/A dup=58 drop=5 speed=   1x
frame= 4803 fps= 24 q=24.8 size=N/A time=00:03:19.43 bitrate=N/A dup=58 drop=5 speed=1.01x
frame= 4815 fps= 24 q=24.8 size=N/A time=00:03:19.93 bitrate=N/A dup=58 drop=5 speed=   1x
frame= 4827 fps= 24 q=24.8 size=N/A time=00:03:20.42 bitrate=N/A dup=58 drop=5 speed=   1x
frame= 4839 fps= 24 q=24.8 size=N/A time=00:03:20.92 bitrate=N/A dup=58 drop=5 speed=   1x
[rtsp @ 0x7fc9ad80be00] max delay reached. need to consume packet
[rtsp @ 0x7fc9ad80be00] RTP: missed 1 packets
frame= 4853 fps= 24 q=24.8 size=N/A time=00:03:21.50 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 4869 fps= 24 q=24.8 size=N/A time=00:03:22.17 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 4881 fps= 24 q=24.8 size=N/A time=00:03:22.67 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 4894 fps= 24 q=24.8 size=N/A time=00:03:23.21 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 4907 fps= 24 q=24.8 size=N/A time=00:03:23.75 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 4919 fps= 24 q=24.8 size=N/A time=00:03:24.24 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 4931 fps= 24 q=24.8 size=N/A time=00:03:24.74 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 4943 fps= 24 q=24.8 size=N/A time=00:03:25.24 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 4956 fps= 24 q=24.8 size=N/A time=00:03:25.78 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 4968 fps= 24 q=24.8 size=N/A time=00:03:26.28 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 4981 fps= 24 q=24.8 size=N/A time=00:03:26.82 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 4992 fps= 24 q=24.8 size=N/A time=00:03:27.28 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 5003 fps= 24 q=24.8 size=N/A time=00:03:27.73 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 5018 fps= 24 q=24.8 size=N/A time=00:03:28.35 bitrate=N/A dup=59 drop=5 speed=   1x
frame= 5028 fps= 24 q=24.8 size=N/A time=00:03:28.77 bitrate=N/A dup=59 drop=5 speed=   1x
[rtsp @ 0x7fc9ad80be00] max delay reached. need to consume packet
[rtsp @ 0x7fc9ad80be00] RTP: missed 1 packets
[rtsp @ 0x7fc9ad80be00] max delay reached. need to consume packet
[rtsp @ 0x7fc9ad80be00] RTP: missed 1 packets
frame= 5043 fps= 24 q=24.8 size=N/A time=00:03:29.39 bitrate=N/A dup=61 drop=5 speed=   1x
frame= 5055 fps= 24 q=24.8 size=N/A time=00:03:29.89 bitrate=N/A dup=61 drop=5 speed=   1x
frame= 5068 fps= 24 q=24.8 size=N/A time=00:03:30.43 bitrate=N/A dup=61 drop=5 speed=   1x
frame= 5081 fps= 24 q=24.8 size=N/A time=00:03:30.97 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5093 fps= 24 q=24.8 size=N/A time=00:03:31.47 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5106 fps= 24 q=24.8 size=N/A time=00:03:32.01 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5118 fps= 24 q=24.8 size=N/A time=00:03:32.51 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5129 fps= 24 q=24.8 size=N/A time=00:03:32.96 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5142 fps= 24 q=24.8 size=N/A time=00:03:33.50 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5153 fps= 24 q=24.8 size=N/A time=00:03:33.96 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5167 fps= 24 q=24.8 size=N/A time=00:03:34.54 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5179 fps= 24 q=24.8 size=N/A time=00:03:35.04 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5191 fps= 24 q=24.8 size=N/A time=00:03:35.54 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5203 fps= 24 q=24.8 size=N/A time=00:03:36.04 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5215 fps= 24 q=24.8 size=N/A time=00:03:36.53 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5230 fps= 24 q=24.8 size=N/A time=00:03:37.16 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5242 fps= 24 q=24.8 size=N/A time=00:03:37.66 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5254 fps= 24 q=24.8 size=N/A time=00:03:38.15 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5267 fps= 24 q=24.8 size=N/A time=00:03:38.69 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5279 fps= 24 q=24.8 size=N/A time=00:03:39.19 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5291 fps= 24 q=24.8 size=N/A time=00:03:39.69 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5303 fps= 24 q=24.8 size=N/A time=00:03:40.19 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5315 fps= 24 q=24.8 size=N/A time=00:03:40.69 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5330 fps= 24 q=24.8 size=N/A time=00:03:41.31 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5342 fps= 24 q=24.8 size=N/A time=00:03:41.81 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5354 fps= 24 q=24.8 size=N/A time=00:03:42.31 bitrate=N/A dup=62 drop=5 speed=   1x
frame= 5368 fps= 24 q=24.8 size=N/A time=00:03:42.89 bitrate=N/A dup=63 drop=5 speed=   1x
frame= 5380 fps= 24 q=24.8 size=N/A time=00:03:43.39 bitrate=N/A dup=63 drop=5 speed=   1x
frame= 5392 fps= 24 q=24.8 size=N/A time=00:03:43.88 bitrate=N/A dup=63 drop=5 speed=   1x
  1. Can I fix this problem? Is there something I can do to make this more reliable?
  2. If not, then how can I work around this problem so that I have a stable data ingress pipeline?

Upvotes: 4

Views: 3729

Answers (1)

Ramon
Ramon

Reputation: 1465

I'm not sure about your environment, but I have solved this issue in a Docker container running in Kubernetes.

In K8s, you're obviously able to set resource limits. You don't have to, but it's good practice.

It was hard to tell, but K8s killed the FFMPEG process in my container, without killing or restarting the container itself. This was all due to resource limits that were set too low. I have found that in my case, these resource limits were around the absolute minimum:

resources:
  requests:
    cpu: 200m
    memory: 512Mi
  limits:
    cpu: 1800m
    memory: 2Gi

Where memory is the most critical one here. I could render small, low resolution video's in a container with 512Mi memory, but rendering 1080p video's needed the 2Gi limit.

Hope this helps anyone.

Upvotes: 2

Related Questions