How to drop frames to limit memory usage (or keep latency low)?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

How to drop frames to limit memory usage (or keep latency low)?

Adam Nielsen
Hi all,

I am transcoding footage from a security camera from MJPEG into MPEG2,
but the machine I'm doing it on can't quite manage it in real-time.
It's pegged at 100% CPU usage but after 10 minutes it exhausts the
available memory and the output video becomes full of visual artefacts.

As far as I can tell, it's not a memory leak but rather the input
frames from the camera being buffered, because the transcoding is
happening slower than real-time.  So I am looking for some way to
prevent this buffer from growing so large that it causes problems.

Since it's security camera footage, I don't mind if a few frames are
dropped here and there.  If I could set a memory limit and have frames
dropped once this limit is reached, that would be good.  Alternatively a
low latency/realtime option that dropped frames if the output was
getting too far behind the input would perhaps be even better.

I've had a look through the manpage but I can't find any options that
can do this.  The -r and -vsync options seem to only drop frames to
match a constant framerate (rather than trying to minimise latency) and
-frame_drop_threshold looks to be similar, keeping up with timestamps
rather than affecting latency.

Is there some option that will effectively drop frames in order to
prevent the latency from becoming too great?

This is what I'm running now:

    ffmpeg -overrun_nonfatal 1 -y -i udp://1.2.3.4
      -c mpeg2video -b:v 2M -vf transpose=2
      -preset ultrafast -an -r 30
      out.mkv

Many thanks,
Adam.


_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: How to drop frames to limit memory usage (or keep latency low)?

Victor Carneiro Fardim
Hi, Adam.

I don't see any way to do automatic frame skipping in FFmpeg.

What you could do is reduce the output framerate by using the `-r (framerate)` or the `fps` video filter, which will reduce the encoding load on the CPU.

Adjust this until the average encoding throughput is 1.00x (that means, when the average number of frames encoded per second is equal to the specified framerate, and your average CPU load is below ~90% just to be safe).

Also, I'm sure `mpeg2video` does not support encoding presets. If you want fast encoding, you may want to try `libx264` with `-preset ultrafast`, `-tune stillimage` and `-deblock 0:0`, since that can be faster than MPEG-2 depending on your CPU architecture, and maybe more efficient.

If available on your platform, you may also want to try hardware-accelerated H 264 encoding APIs. You can find some info here: https://trac.ffmpeg.org/wiki/HWAccelIntro (note: you may have to compile a custom build of FFmpeg to enable this feature, as pre-built ones aren't usually compiled for it.

Victor Carneiro
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: How to drop frames to limit memory usage (or keep latency low)?

Adam Nielsen
Hi Victor,

Many thanks for your reply!

> What you could do is reduce the output framerate by using the `-r
> (framerate)` or the `fps` video filter, which will reduce the
> encoding load on the CPU.

This looks like the best option for me.  With fps=15, removing the
rotation filter, and adding your -tune stillimage suggestion, I can get
15fps transcoding at 1.05x realtime.  I imagine this will slow down and
start using more memory for buffering if there's a lot of movement in
the image, but being slightly faster than realtime means it should
catch up again once the movement settles down.

> Also, I'm sure `mpeg2video` does not support encoding presets. If you
> want fast encoding, you may want to try `libx264` with `-preset
> ultrafast`, `-tune stillimage` and `-deblock 0:0`, since that can be
> faster than MPEG-2 depending on your CPU architecture, and maybe more
> efficient.

I tried this however it makes no difference to the encoding speed on
the Intel NUC platform I'm using (Intel i3 @ 2.1GHz).  I did end up
switching to libx264 now that I realise it's no slower, as this matches
the format of the other files I'm getting from my other cameras,
however there was unfortunately no speed increase just from changing
the codec.  The stillimage tuning option did provide a noticeable speed
boost though.

> If available on your platform, you may also want to try
> hardware-accelerated H 264 encoding APIs. You can find some info
> here: https://trac.ffmpeg.org/wiki/HWAccelIntro (note: you may have
> to compile a custom build of FFmpeg to enable this feature, as
> pre-built ones aren't usually compiled for it.

I have thought about this as the video is coming from a Raspberry Pi,
which apparently has hardware H264 encoding now.  I am not sure if it
has enough power to encode two H264 streams at the same time (both the
Pi camera and the MJPEG USB camera that is the root cause of this issue)
but if it can, that will probably be the best option.

Thanks again for your help!

Cheers,
Adam.


_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: How to drop frames to limit memory usage (or keep latency low)?

Carl Zwanzig
In reply to this post by Adam Nielsen
On 4/3/2018 8:55 PM, Adam Nielsen wrote:

> I've had a look through the manpage but I can't find any options that
> can do this.  The -r and -vsync options seem to only drop frames to
> match a constant framerate (rather than trying to minimise latency) and
> -frame_drop_threshold looks to be similar, keeping up with timestamps
> rather than affecting latency.

Do you -need- full-rate motion?

It sounds like you need to drop frames to match a constant FR that you then
transcode. You might be able to use the framestep filter to discard every
other frame, transcode, then use -r to set the desired output rate which
will dup frames to fill the time. (I have not tried this myself.)

There's a some vaguely apropos answer regarding framestep at
https://video.stackexchange.com/questions/16552/4x-resample-videoframes-using-ffmpeg.

Later,

z!
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: How to drop frames to limit memory usage (or keep latency low)?

Adam Nielsen
> Do you -need- full-rate motion?

No, but it would be nice if I could automatically get a variable
framerate based on whatever the CPU can handle.

> It sounds like you need to drop frames to match a constant FR that you then
> transcode. You might be able to use the framestep filter to discard every
> other frame, transcode, then use -r to set the desired output rate which
> will dup frames to fill the time. (I have not tried this myself.)

This seems to be the best solution.  Since my last post I haven't run
into a single memory issue since dropping back to 15fps instead of
30fps (although I'm using "-vf fps=15" instead).  Problem solved!

Cheers,
Adam.


_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".