Decoding frames as fast as possible

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Decoding frames as fast as possible

Evan
Hi everyone,

I'm developping a media player featuring advanced video/frame processing.
To be able to seek to any particular frame, I need to know, for each frame,
its presentation time and if available, if it's a key frame or not. The
latter is optional, really.

To do that I build an "index" once the media is valid and opened.
It's a basically a loop, fetching a packet, checking if it's the video
stream I'm interested in, feeding the codec until he can produces a frame
or reaching the end of the file.

This is perfectly working but it's way too long for me, like 20 seconds for
a ~800MB mp4/h264 file.
I'm considering the following options:
- building the index in the background while playing the media: I guess
I'll have to copy the decoder context to not mix both processes but I
expect having playback (stuttering) issues.
- use multiple threads: from what I read, I can only use one thread for a
stream. I may eventually use two threads, one for video and one for audio
but I want to speed up video decoding fisrt.
- use the GPU to decode frames faster: while it sounds like a good idea, I
have to add two constraints: I'm on Windows 7 (and soon, Windows 10) and I
want a generic approach so not H.264-only.
- decode only packets: it's tempting but I think I will either loose
timestamp information or precision (because of I, P, B frames.
- use an option (av_dict_set) to do a dummy frame decoding: is there
anything like that?
- use reference counting to avoid costly frame allocations: I already tried
that but didn't see any difference.

For now, I will measure time elapsed in each piece of code in order to
pinpoint lengthy operations.
If you guys have any hint, I'd be glad to hear them.

Thanks.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Decoding frames as fast as possible

Carl Eugen Hoyos-2
2018-07-03 10:23 GMT+02:00, Evan <[hidden email]>:

> I'm developping a media player featuring advanced video/frame
> processing. To be able to seek to any particular frame, I need to

(After writing the answer below: Please understand that in the
general case, you cannot seek to any particular frame, this is
how video codecs work.)

> know, for each frame, its presentation time and if available, if it's
> a key frame or not. The latter is optional, really.
>
> To do that I build an "index" once the media is valid and opened.

In general, this is not considered a good idea (for the reason that
you found out and that - imo obviously - cannot be solved: No
matter how fast your hardware is there will always be a longer
video or a video with higher resolution).
If you absolutely have to build an index, do it for shown frames
at playback time.

You can of course only parse the video (instead of decoding it)
but remember that for "real" video codecs, the information you
get has limited relevance - just let FFmpeg do the seeking...

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Decoding frames as fast as possible

rogerdpack2
In reply to this post by Evan
Does ffprobe take 20s?

On Tue, Jul 3, 2018 at 2:23 AM, Evan <[hidden email]> wrote:

> Hi everyone,
>
> I'm developping a media player featuring advanced video/frame processing.
> To be able to seek to any particular frame, I need to know, for each frame,
> its presentation time and if available, if it's a key frame or not. The
> latter is optional, really.
>
> To do that I build an "index" once the media is valid and opened.
> It's a basically a loop, fetching a packet, checking if it's the video
> stream I'm interested in, feeding the codec until he can produces a frame
> or reaching the end of the file.
>
> This is perfectly working but it's way too long for me, like 20 seconds for
> a ~800MB mp4/h264 file.
> I'm considering the following options:
> - building the index in the background while playing the media: I guess
> I'll have to copy the decoder context to not mix both processes but I
> expect having playback (stuttering) issues.
> - use multiple threads: from what I read, I can only use one thread for a
> stream. I may eventually use two threads, one for video and one for audio
> but I want to speed up video decoding fisrt.
> - use the GPU to decode frames faster: while it sounds like a good idea, I
> have to add two constraints: I'm on Windows 7 (and soon, Windows 10) and I
> want a generic approach so not H.264-only.
> - decode only packets: it's tempting but I think I will either loose
> timestamp information or precision (because of I, P, B frames.
> - use an option (av_dict_set) to do a dummy frame decoding: is there
> anything like that?
> - use reference counting to avoid costly frame allocations: I already tried
> that but didn't see any difference.
>
> For now, I will measure time elapsed in each piece of code in order to
> pinpoint lengthy operations.
> If you guys have any hint, I'd be glad to hear them.
>
> Thanks.
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Decoding frames as fast as possible

rogerdpack2
On Tue, Jul 24, 2018 at 2:50 PM, √Čvan <[hidden email]> wrote:
> On Mon, Jul 23, 2018 at 8:45 AM, Roger Pack wrote:
>> Does ffprobe take 20s?
>
> I just tested and "ffprobe.exe -show_entries frame=pkt_pts_time
> bbb_sunflower_1080p_30fps_normal.mp4" does take time to complete.

I also saw this recently:
https://stackoverflow.com/questions/51547517/using-ffprobe-to-get-number-of-keyframes-in-raw-avi-file-without-processing-en#comment90288838_51547517
(hint: consider asking on superuser and/or av.stackexchange)
Anyway my current hypothesis is that ffmpeg has no way to "skip
reading" input as it were...

> I'm using ffprobe version 4.0 [...] built with gcc 7.3.0 (GCC), x64, win32
> zeranoe version.
> From my test, ffprobes takes like 3 minutes and a half when printing the
> result to the console and a minute and a half when output is redirected to a
> file.
>
> My test video is "Big Buck Bunny" 1080p 30FPS MP4 version, md5 sum
> babbe51b47146187a66632daa791484b.
>
> I can send you my test program but I used ffmpeg (and ffprobe) source code
> to write it.
> Moreover, all CPUs are used at 100% when "building the index" so my program
> seems efficient CPU-side.
>
> Regards
>
> PS : I'm not subscribed to the list so please CC me.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".