The h264_cuvid decoder would output more frames than the number of origin video frame.

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

The h264_cuvid decoder would output more frames than the number of origin video frame.

linecorp baek
I'm used to use ffmpeg with nvidia Gpu devices for video transcoding work.
I found issue that when I try to decode a video using h264_cuvid, the
number of output frames are more than the number of origin video frames.

I guess, It issue would be caused ffmpeg side. So I tried some tests and
Please see below results.

[Test environments]
 1. Sample Video
  - 3,600 nb_frame and 1920x1080 60fps video. (A number is increasing
video.)
  - baseline profile and h264 coded video.
  - Youtube link, but it is not mine. (
https://www.youtube.com/watch?v=jEVewMQ4iu8)
 2. Machine Spec
  - CPU: Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz x40
  - Mem: 128GB
  - GPU: Tesla P4 x2, NVIDIA-SMI 396.26, nvcc V9.2.88 (Release)
 3. Use ffmpeg version 3.4

[Test]
 1. Video decoding using h264_cuvid on FFmpeg
  - using `ffmpeg -c:v h264_cuvid -i baseline-numbered.mp4 -f rawvideo
-pix_fmt nv12 ./output.ffmpeg.nv12`
  - 4 frames were duplicated. (11209881600 bytes output, 3604 frames)
  - Snapshot:
https://drive.google.com/open?id=1sfpjtOMMhLkXD72abHyLNY0FdOVS1xl9

 2. Video decoding using AppDec Sample code in Nvidia Video SDK.
  - using the source code of "{VIDEO_SDK_BASE}/Samples/AppDecode/AppDec"
  - 0 frames were duplicated. (11197440000 bytes output, 3600 frames)
  - Snapshot:
https://drive.google.com/file/d/1Ww9P0Vhd2K76xsyRKkPHf1Yis3-HuMN-/view?usp=sharing

As I knew that FFmpeg h264_cuvid module was used same code of Nvidia Video
SDK code. So I thought ffmpeg seems occur creating duplicated frame or
something wrong. So I wonder this happening is caused by ffmpeg or not.

In order words,
For getting the same result on ffmpeg, would I ask you this issue
validation by anyone who can help me out?

Thank you.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: The h264_cuvid decoder would output more frames than the number of origin video frame.

Moritz Barsnick
On Wed, Aug 08, 2018 at 16:18:53 +0900, linecorp baek wrote:

> (A number is increasing video.)

I don't understand this sentence.

>   - Youtube link, but it is not mine. (https://www.youtube.com/watch?v=jEVewMQ4iu8)

Is that the source of your input? How did you download it? youtube-dl?
If so, which format did you choose?

>  3. Use ffmpeg version 3.4

Why? A lot of development has happened since. Please use the very
latest release (4.0.2), or, even better, a build from recent git
master. Links to binaries are provided on the ffmpeg home page.

>  1. Video decoding using h264_cuvid on FFmpeg
>   - using `ffmpeg -c:v h264_cuvid -i baseline-numbered.mp4 -f rawvideo -pix_fmt nv12 ./output.ffmpeg.nv12`

Please also provide the complete, uncut console output of your ffmpeg
command(s).

>   - 4 frames were duplicated. (11209881600 bytes output, 3604 frames)
[...]
>   - 0 frames were duplicated. (11197440000 bytes output, 3600 frames)

I have an assumption, regarding VFR vs. CFR, but it could be
undermined by some facts about the input.

Cheers,
Moritz
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: The h264_cuvid decoder would output more frames than the number of origin video frame.

linecorp baek
Hello Mr. Moritz.
Thanks for your reply.

I apologize for providing insufficient information in first mail.
In summary of this reply, when FFmpeg version 4.0.2 is used and the input
video is decoded with the h264_cuvid decoder,
more frames are output than the origin video.

I would appreciate if you are checking the following information.
I have written some related topics with [ ] mark.
Please find [ ] mark so that read easier. :)

[Answering about your mentions]
> Is that the source of your input? How did you download it? youtube-dl?
If so, which format did you choose?

I used the 4K Video downloader program to get an MP4 file from Youtube.
And then, I changed video profile from High profile to baseline profile in
order to remove B-frames.
   (--> ffmpeg -i youtube-download.mp4 -c:v libx264 -vprofile baseline
./baseline-numbered.mp4)
Here is ffprobe result about an input video.

~/test/180808$ ffprobe -show_streams -show_format -print_format json
baseline-numbered.mp4
{
    "streams": [
        {
            "index": 0,
            "codec_name": "h264",
            "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
            "profile": "Constrained Baseline",
            "codec_type": "video",
            "codec_time_base": "1/120",
            "codec_tag_string": "avc1",
            "codec_tag": "0x31637661",
            "width": 1920,
            "height": 1080,
            "coded_width": 1920,
            "coded_height": 1080,
            "has_b_frames": 0,
            "sample_aspect_ratio": "1:1",
            "display_aspect_ratio": "16:9",
            "pix_fmt": "yuv420p",
            "level": 42,
            "chroma_location": "left",
            "refs": 1,
            "is_avc": "true",
            "nal_length_size": "4",
            "r_frame_rate": "60/1",
            "avg_frame_rate": "60/1",
            "time_base": "1/15360",
            "start_pts": 0,
            "start_time": "0.000000",
            "duration_ts": 921600,
            "duration": "60.000000",
            "bit_rate": "1298712",
            "bits_per_raw_sample": "8",
            "nb_frames": "3600",
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0
            },
            "tags": {
                "language": "und",
                "handler_name": "VideoHandler"
            }
        },
        {
            "index": 1,
            "codec_name": "aac",
            "codec_long_name": "AAC (Advanced Audio Coding)",
            "profile": "LC",
            "codec_type": "audio",
            "codec_time_base": "1/44100",
            "codec_tag_string": "mp4a",
            "codec_tag": "0x6134706d",
            "sample_fmt": "fltp",
            "sample_rate": "44100",
            "channels": 2,
            "channel_layout": "stereo",
            "bits_per_sample": 0,
            "r_frame_rate": "0/0",
            "avg_frame_rate": "0/0",
            "time_base": "1/44100",
            "start_pts": 0,
            "start_time": "0.000000",
            "duration_ts": 2649087,
            "duration": "60.070000",
            "bit_rate": "134109",
            "max_bit_rate": "134109",
            "nb_frames": "2588",
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0
            },
            "tags": {
                "language": "und",
                "handler_name": "SoundHandler"
            }
        }
    ],
    "format": {
        "filename": "baseline-numbered.mp4",
        "nb_streams": 2,
        "nb_programs": 0,
        "format_name": "mov,mp4,m4a,3gp,3g2,mj2",
        "format_long_name": "QuickTime / MOV",
        "start_time": "0.000000",
        "duration": "60.094000",
        "size": "10818961",
        "bit_rate": "1440271",
        "probe_score": 100,
        "tags": {
            "major_brand": "isom",
            "minor_version": "512",
            "compatible_brands": "isomiso2avc1mp41",
            "encoder": "Lavf57.83.100"
        }
    }
}


> Why? A lot of development has happened since. Please use the very
latest release (4.0.2), or, even better, a build from recent git
master. Links to binaries are provided on the ffmpeg home page.

As your recommend, I installed ffmpeg 4.0.2 version on CentOS 7.4 after I
got your reply.

[Test results of h264_cuvid decoding]
  1. decoding with ffmeg h264_cuvid to nv12 raw format.

[~/pkg/licoder-ffmpeg/ffmpeg-n4.0.2]$ ./ffmpeg -c:v h264_cuvid -i
baseline-numbered.mp4 -f rawvideo -pix_fmt nv12 ./output.ffmpeg402.nv12
ffmpeg version 4.0.2 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-16)
  configuration: --prefix=/home1/irteam/apps/ffmpeg/ffmpeg_build_n4.0.2
--extra-cflags='-I/home1/irteam/apps/ffmpeg/ffmpeg_build_n4.0.2/include
-I/usr/local/cuda/include'
--extra-ldflags='-L/home1/irteam/apps/ffmpeg/ffmpeg_build_n4.0.2/lib -ldl
-L/usr/local/cuda/lib64'
--bindir=/home1/irteam/apps/ffmpeg/ffmpeg_bin_n4.0.2
--pkg-config-flags=--static --enable-gpl --enable-nonfree
--enable-libfdk-aac --enable-libmp3lame --enable-libopus --enable-libvorbis
--enable-libvpx --enable-libx264 --enable-libx265 --enable-cuda
--enable-cuvid --enable-nvenc --enable-libnpp
  libavutil      56. 14.100 / 56. 14.100
  libavcodec     58. 18.100 / 58. 18.100
  libavformat    58. 12.100 / 58. 12.100
  libavdevice    58.  3.100 / 58.  3.100
  libavfilter     7. 16.100 /  7. 16.100
  libswscale      5.  1.100 /  5.  1.100
  libswresample   3.  1.100 /  3.  1.100
  libpostproc    55.  1.100 / 55.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'baseline-numbered.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.83.100
  Duration: 00:01:00.09, start: 0.000000, bitrate: 1440 kb/s
    Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 /
0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 1298 kb/s, 60 fps, 60
tbr, 15360 tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz,
stereo, fltp, 134 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_cuvid) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, rawvideo, to './output.ffmpeg402.nv12':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.12.100
    Stream #0:0(und): Video: rawvideo (NV12 / 0x3231564E), nv12, 1920x1080
[SAR 1:1 DAR 16:9], q=2-31, 1492992 kb/s, 60 fps, 60 tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.18.100 rawvideo
frame= 3604 fps=282 q=-0.0 Lsize=10947150kB time=00:01:00.06
bitrate=1492992.0kbits/s dup=4 drop=0 speed= 4.7x
video:10947150kB audio:0kB subtitle:0kB other streams:0kB global
headers:0kB muxing overhead: 0.000000%


 2. checking the output file size.
-rw-r--r--  1 irteam irteam 11209881600  8월  8 19:05 output.ffmpeg402.nv12

11209881600 bytes -> it means that is 3604 frames. But the nb_frames of
input video is 3600.

I'm not a good English speaker.
So you might feel uncomfortable to read this text. :( sorry.


I thought this duplicating frame issue seems not a Nvidia decoder problem.
Could you please check this issue for solving duplication problem?

Thank you!
Baek.

2018-08-08 16:51 GMT+09:00 Moritz Barsnick <[hidden email]>:

> On Wed, Aug 08, 2018 at 16:18:53 +0900, linecorp baek wrote:
>
> > (A number is increasing video.)
>
> I don't understand this sentence.
>
> >   - Youtube link, but it is not mine. (https://www.youtube.com/
> watch?v=jEVewMQ4iu8)
>
> Is that the source of your input? How did you download it? youtube-dl?
> If so, which format did you choose?
>
> >  3. Use ffmpeg version 3.4
>
> Why? A lot of development has happened since. Please use the very
> latest release (4.0.2), or, even better, a build from recent git
> master. Links to binaries are provided on the ffmpeg home page.
>
> >  1. Video decoding using h264_cuvid on FFmpeg
> >   - using `ffmpeg -c:v h264_cuvid -i baseline-numbered.mp4 -f rawvideo
> -pix_fmt nv12 ./output.ffmpeg.nv12`
>
> Please also provide the complete, uncut console output of your ffmpeg
> command(s).
>
> >   - 4 frames were duplicated. (11209881600 bytes output, 3604 frames)
> [...]
> >   - 0 frames were duplicated. (11197440000 bytes output, 3600 frames)
>
> I have an assumption, regarding VFR vs. CFR, but it could be
> undermined by some facts about the input.
>
> Cheers,
> Moritz
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".