Why does '-c:v copy' decode video frames?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Why does '-c:v copy' decode video frames?

Yukun Guo
Hi,

I'm trying to transmux an H.264 movie in FLV container into raw H.264 in
Annex.B format:

    ffmpeg -i movie.flv -c:v copy -an -bsf:v h264_mp4toannexb movie.264

The original video seems problematic, so FFmpeg outputs several lines of
"co located POCs
unavailable" error. The error message is printed in
`h264_direct.c/ff_h264_direct_ref_list_init`,
which is further called by `h264dec.c/h264_decode_frame`. What confuses me
is why does the copy
codec still try to decode H.264 frames and even do motion vector
prediction? If I understand
correctly, FFmpeg can simply extract SPS/PPS from
AVCDecoderConfigurationRecord and copy slice data
verbatim. Besides that, I find `h264_decode_frame` is only called sparsely,
not as many as the
actual frame number.

Could any FFmpeg experts explain why this occurs?
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Why does '-c:v copy' decode video frames?

Moritz Barsnick
On Wed, Feb 20, 2019 at 14:28:12 +0800, Yukun Guo wrote:
> I'm trying to transmux an H.264 movie in FLV container into raw H.264 in
> Annex.B format:
>
>     ffmpeg -i movie.flv -c:v copy -an -bsf:v h264_mp4toannexb movie.264
[...]
> which is further called by `h264dec.c/h264_decode_frame`. What
> confuses me is why does the copy codec still try to decode H.264
> frames and even do motion vector prediction? If I understand
[...]
> Could any FFmpeg experts explain why this occurs?

I'm no expert, but a simple backtrace shows me:

Breakpoint 1, h264_decode_frame (avctx=0xa21ac80, data=0xa221c00, got_frame=0xbfffe930, avpkt=0xa221e00) at src/libavcodec/h264dec.c:986
986         if (avctx->codec_id == AV_CODEC_ID_IMM5) {
(gdb) bt
#0  h264_decode_frame (avctx=0xa21ac80, data=0xa221c00, got_frame=0xbfffe930, avpkt=0xa221e00) at src/libavcodec/h264dec.c:986
#1  0x08531e43 in decode_simple_internal (avctx=0xa221e00, frame=0x98c6980) at src/libavcodec/decode.c:433
#2  0x0853009a in decode_simple_receive_frame (avctx=<optimized out>, frame=<optimized out>) at src/libavcodec/decode.c:629
#3  decode_receive_frame_internal (avctx=<optimized out>, frame=<optimized out>) at src/libavcodec/decode.c:647
#4  avcodec_send_packet (avctx=0xa21ac80, avpkt=0xbfffe9d0) at src/libavcodec/decode.c:705
#5  0x084774af in try_decode_frame (s=0xa221e00, st=0x98c6980, avpkt=0xa21ac80, options=0xa21c340) at src/libavformat/utils.c:3079
#6  0x08474220 in avformat_find_stream_info (ic=0xa219740, options=0xa21c340) at src/libavformat/utils.c:3900
#7  0x08055aa0 in open_input_file. () at src/fftools/ffmpeg_opt.c:1126
#8  0x08054cf3 in open_files (l=<optimized out>, inout=<optimized out>, open_file=<optimized out>) at src/fftools/ffmpeg_opt.c:3273
#9  ffmpeg_parse_options (argc=8, argv=0xbffff4b4) at src/fftools/ffmpeg_opt.c:3313
#10 0x0806fff3 in main (argc=8, argv=0xbffff4b4) at src/fftools/ffmpeg.c:4869
(gdb)

libavformat is probing the stream, as far as I can tell, and using the
h264dec functions for this. (It also does so if you give it "-f mp4
-c:v h264".)

Moritz
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Why does '-c:v copy' decode video frames?

Gyan
In reply to this post by Yukun Guo


On 20-02-2019 11:58 AM, Yukun Guo wrote:

> Hi,
>
> I'm trying to transmux an H.264 movie in FLV container into raw H.264 in
> Annex.B format:
>
>      ffmpeg -i movie.flv -c:v copy -an -bsf:v h264_mp4toannexb movie.264
>
> The original video seems problematic, so FFmpeg outputs several lines of
> "co located POCs
> unavailable" error. The error message is printed in
> `h264_direct.c/ff_h264_direct_ref_list_init`,
> which is further called by `h264dec.c/h264_decode_frame`. What confuses me
> is why does the copy
> codec still try to decode H.264 frames and even do motion vector
> prediction? If I understand
> correctly, FFmpeg can simply extract SPS/PPS from
> AVCDecoderConfigurationRecord and copy slice data
> verbatim. Besides that, I find `h264_decode_frame` is only called sparsely,
> not as many as the
> actual frame number.
>
> Could any FFmpeg experts explain why this occurs?
FFmpeg opens the decoder to decode the codec config and populate stream
parameters. Muxers, in general, read these parameters, to generate headers.

Gyan
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Why does '-c:v copy' decode video frames?

kumowoon1025
In reply to this post by Yukun Guo
> The original video seems problematic, so FFmpeg outputs several lines of
> "co located POCs
> unavailable" error. The error message is printed in
> `h264_direct.c/ff_h264_direct_ref_list_init`,
> which is further called by `h264dec.c/h264_decode_frame`. What confuses me
> is why does the copy
> codec still try to decode H.264 frames and even do motion vector
> prediction?

It doesn’t. And as you noted it doesn’t make sense for it to be doing any prediction with no picture, it is just calculating the mv. If slice data partitioning is used (or maybe regardless, don’t remember) the equation for motion vector needs at least the field order list and sub/mb partition indexes for the current and colocated mbs, so parsing the parameter sets can get pretty involved even if the stream isn’t actually decoded.

>  If I understand
> correctly, FFmpeg can simply extract SPS/PPS from
> AVCDecoderConfigurationRecord and copy slice data
> verbatim.
I guess this is the “extract” phase of that process, except it’s not liking the input bitstream.

> Besides that, I find `h264_decode_frame` is only called sparsely,
> not as many as the
> actual frame number.
I bet the ones it is not called on correspond with nal_ref_idc == 0
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Why does '-c:v copy' decode video frames?

Carl Eugen Hoyos-2
2019-02-20 10:32 GMT+01:00, Ted Park <[hidden email]>:
>> The original video seems problematic, so FFmpeg outputs
>> several lines of "co located POCs unavailable" error. The
>> error message is printed in `h264_direct.c/ff_h264_direct_ref_list_init`,
>> which is further called by `h264dec.c/h264_decode_frame`.
>> What confuses me is why does the copy codec still try
>> to decode H.264 frames and even do motion vector prediction?
>
> It doesn’t.

I may misunderstand this but in any case as Moritz explained
libavformat will use the decoder to know the properties of the
stream (you cannot avoid this), the properties are typically
necessary for the muxer to actually implement "-c copy" (even
if not needed in all cases).
Not all frames are decoded but some frames in the
beginning of the stream (for some codecs).

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Why does '-c:v copy' decode video frames?

Yukun Guo
Thank you for the very helpful replies!

I was busy experimenting with FFmpeg yesterday but got lost in the
complicated codebase. So far I have found FFmpeg keeps decoding frames
until it has collected enough information. In my particular case, it
is the third condition in this while loop
https://github.com/FFmpeg/FFmpeg/blob/n3.4.2/libavformat/utils.c#L3002
evaluating to false that ends the slice decoding.

...the properties are typically necessary for the muxer to actually
> implement "-c copy" (even if not needed in all cases).
>

This is exactly what I missed: the muxer is designed for general
purpose so it will go through all the steps. For the same reason
`ffmpeg -i src.flv -c copy -f flv dst.flv` doesn't mean a file copy.

...it is just calculating the mv. If slice data partitioning is
> used (or maybe regardless, don’t remember) the equation for motion
> vector needs at least the field order list and sub/mb partition
> indexes for the current and colocated mbs
>

I still can't understand this. Why does FFmpeg need MVs? My guess is
calculating MVs is just a byproduct of extracting the metadata.

On Thu, 21 Feb 2019 at 07:21, Carl Eugen Hoyos <[hidden email]> wrote:

> 2019-02-20 10:32 GMT+01:00, Ted Park <[hidden email]>:
> >> The original video seems problematic, so FFmpeg outputs
> >> several lines of "co located POCs unavailable" error. The
> >> error message is printed in
> `h264_direct.c/ff_h264_direct_ref_list_init`,
> >> which is further called by `h264dec.c/h264_decode_frame`.
> >> What confuses me is why does the copy codec still try
> >> to decode H.264 frames and even do motion vector prediction?
> >
> > It doesn’t.
>
> I may misunderstand this but in any case as Moritz explained
> libavformat will use the decoder to know the properties of the
> stream (you cannot avoid this), the properties are typically
> necessary for the muxer to actually implement "-c copy" (even
> if not needed in all cases).
> Not all frames are decoded but some frames in the
> beginning of the stream (for some codecs).
>
> Carl Eugen
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Why does '-c:v copy' decode video frames?

Carl Eugen Hoyos-2
2019-02-22 18:28 GMT+01:00, Yukun Guo <[hidden email]>:

> I still can't understand this. Why does FFmpeg need
> MVs? My guess is calculating MVs is just a
> byproduct of extracting the metadata.

The decoder does not know that it was only called to
provide some information about the stream, not the
actual decoded data.

Please do not top-post here, Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".