Compiled ffmpeg - x264 - instability when using -pix_fmt yuv420p

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Compiled ffmpeg - x264 - instability when using -pix_fmt yuv420p

Jonathan Niquet
Hey there,

We've been using John Van Sickle's ffmpeg static build for quite a while
and it's worked wonderfully for us.

However, we recently had to incorporate aac into our build.

I then started to statically compile every library we needed and proceed to
successfully build FFmpeg.

This is on Linux, centos 7.6.
This was my config arguments:

--extra-ldflags="-L$/home/jon/ffmpegDependancies/lib"
--extra-ldflags="-L$/usr/local/lib"
--extra-cflags="-I/home/jon/ffmpegDependancies/include"
--prefix="/home/jon/ffmpegBuild" --extra-libs="-lpthread -lm -fopenmp"
--enable-gpl --enable-version3 --enable-nonfree
--pkg-config-flags="--static" --enable-static --disable-shared
--enable-debug --disable-ffplay --disable-indev=sndio
--disable-outdev=sndio --enable-libfdk-aac --enable-frei0r --enable-libaom
--enable-libfribidi  --enable-libass --enable-libvpx --enable-libx264
--enable-libwebp --enable-libxml2 --enable-fontconfig --enable-gray
--enable-libfreetype --enable-libzvbi --enable-libvidstab --enable-libvmaf
--enable-libvorbis --enable-libmp3lame --enable-libopus
--enable-libopenjpeg --enable-libzimg --enable-libspeex --enable-libx265
--enable-gmp --enable-libvo-amrwbenc --enable-libopencore-amrwb
--enable-libopencore-amrnb --enable-libsoxr --enable-libxvid

However, in a random fashion, whenever we use "*-pix_fmt yuv420p*" when
encoding libx264
(*-c:v libx264 -c:a copy -crf 20 -pix_fmt yuv420p -vf
colormatrix=bt601:bt709*), FFmpeg will randomly segfault between 40% to 45%
of the time. Using the same command 20 times in a row, on the same
computer, will literally give 11 completion and 9 segfaults. I've tried
recompiling the whole things on a virtual machine specially setup for this
task (blank, minimal install) and the exact same behavior is observed.

I've tried to strace the process, I didn't notice anything that sounds like
a solution.
From John's static build, I only removed 3 libraries: gnutls, librubberband
and libtheora.

I'm thinking it might be a problem with my x264 compilation, the randomness
of the things might point to threading issues, but I'm not too sure what to
look at at this point.

Anyone experienced something similar before or would have clues to help me
debug this hellish instability?

Thanks!
[image: RODEO FX] <https://www.rodeofx.com/>
[image: Facebook] <https://www.facebook.com/rodeofx/> [image: Twitter]
<https://twitter.com/FXRodeo> [image: LinkedIn]
<https://www.linkedin.com/company/251532/admin/> [image: Vimeo]
<https://vimeo.com/rodeofx> [image: Instagram]
<https://www.instagram.com/rodeofxstyle/>
*Jonathan Niquet <https://ca.linkedin.com/in/jonathan-niquet-62b37239>*
Application Support Specialist
*[hidden email] <[hidden email]> *
514.397.9999 ext.599
*www.rodeofx.com <https://www.rodeofx.com/>*
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Compiled ffmpeg - x264 - instability when using -pix_fmt yuv420p

Jonathan Niquet
just realized i forgot some important information:
this is for ffmpeg 4.1.1 but I also tried 4.2.1 and the exact same behavior
was observed:

And here's the log:

jniquet@mtl-rmw001l bin $ ./ffmpeg -start_number 1160 -v 48 -y -r 23.98 -t
0.0417014178482 -i
/rdo/staging/rdo_graph_core/rdo_graph_core_3852239/tmpHVIGIE/102024_035_comp_v007.1160.exr
-timecode 00:00:00:00 -c:v libx264 -c:a copy -crf 20 -pix_fmt yuv420p -vf
colormatrix=bt601:bt709
/rdo/staging/rdo_graph_core/rdo_graph_core_3852239/tmpjxs_EA/split_8.mov
ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-36)
  configuration: --extra-ldflags='-L$/home/jon/ffmpegDependancies/lib'
--extra-ldflags='-L$/usr/local/lib'
--extra-cflags=-I/home/jon/ffmpegDependancies/include
--prefix=/home/jon/ffmpegBuild --extra-libs='-lpthread -lm -fopenmp'
--enable-gpl --enable-version3 --enable-nonfree --pkg-config-flags=--static
--enable-static --disable-shared --disable-debug --disable-ffplay
--disable-indev=sndio --disable-outdev=sndio --enable-libfdk-aac
--enable-frei0r --enable-libaom --enable-libfribidi --enable-libass
--enable-libvpx --enable-libx264 --enable-libwebp --enable-libxml2
--enable-fontconfig --enable-gray --enable-libfreetype --enable-libzvbi
--enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libmp3lame
--enable-libopus --enable-libopenjpeg --enable-libzimg --enable-libspeex
--enable-libx265 --enable-gmp --enable-libvo-amrwbenc
--enable-libopencore-amrwb --enable-libopencore-amrnb --enable-libsoxr
--enable-libxvid
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Splitting the commandline.
Reading option '-start_number' ... matched as AVOption 'start_number' with
argument '1160'.
Reading option '-v' ... matched as option 'v' (set logging level) with
argument '48'.
Reading option '-y' ... matched as option 'y' (overwrite output files) with
argument '1'.
Reading option '-r' ... matched as option 'r' (set frame rate (Hz value,
fraction or abbreviation)) with argument '23.98'.
Reading option '-t' ... matched as option 't' (record or transcode
"duration" seconds of audio/video) with argument '0.0417014178482'.
Reading option '-i' ... matched as input url with argument
'/rdo/staging/rdo_graph_core/rdo_graph_core_3852239/tmpHVIGIE/102024_035_comp_v007.1160.exr'.
Reading option '-timecode' ... matched as option 'timecode' (set initial
TimeCode value.) with argument '00:00:00:00'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument
'libx264'.
Reading option '-c:a' ... matched as option 'c' (codec name) with argument
'copy'.
Reading option '-crf' ... matched as AVOption 'crf' with argument '20'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel
format) with argument 'yuv420p'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with
argument 'colormatrix=bt601:bt709'.
Reading option
'/rdo/staging/rdo_graph_core/rdo_graph_core_3852239/tmpjxs_EA/split_8.mov'
... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument 48.
Applying option y (overwrite output files) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url
/rdo/staging/rdo_graph_core/rdo_graph_core_3852239/tmpHVIGIE/102024_035_comp_v007.1160.exr.
Applying option r (set frame rate (Hz value, fraction or abbreviation))
with argument 23.98.
Applying option t (record or transcode "duration" seconds of audio/video)
with argument 0.0417014178482.
Successfully parsed a group of options.
Opening an input file:
/rdo/staging/rdo_graph_core/rdo_graph_core_3852239/tmpHVIGIE/102024_035_comp_v007.1160.exr.
[NULL @ 0x428e580] Opening
'/rdo/staging/rdo_graph_core/rdo_graph_core_3852239/tmpHVIGIE/102024_035_comp_v007.1160.exr'
for reading
[file @ 0x428ef00] Setting default whitelist 'file,crypto'
[exr_pipe @ 0x428e580] Format exr_pipe probed with size=2048 and score=51
[exr_pipe @ 0x428e580] Before avformat_find_stream_info() pos: 0 bytes
read:32768 seeks:0 nb_streams:1
[exr_pipe @ 0x428e580] parser not found for codec exr, packets or times may
be invalid.
    Last message repeated 1 times
[exr @ 0x4290480] line order: 0.
[exr_pipe @ 0x428e580] Probe buffer size limit of 5000000 bytes reached
[exr_pipe @ 0x428e580] Stream #0: not enough frames to estimate rate;
consider increasing probesize
[exr_pipe @ 0x428e580] After avformat_find_stream_info() pos: 7699076 bytes
read:7699076 seeks:0 frames:1
Input #0, exr_pipe, from
'/rdo/staging/rdo_graph_core/rdo_graph_core_3852239/tmpHVIGIE/102024_035_comp_v007.1160.exr':
  Duration: N/A, bitrate: N/A
    Stream #0:0, 1, 1/25: Video: exr, 1 reference frame, rgb48le, 1920x1080
[SAR 1:1 DAR 16:9], 0/1, 25 tbr, 25 tbn, 25 tbc
Successfully opened the file.
Parsing a group of options: output url
/rdo/staging/rdo_graph_core/rdo_graph_core_3852239/tmpjxs_EA/split_8.mov.
Applying option timecode (set initial TimeCode value.) with argument
00:00:00:00.
Applying option c:v (codec name) with argument libx264.
Applying option c:a (codec name) with argument copy.
Applying option pix_fmt (set pixel format) with argument yuv420p.
Applying option vf (set video filters) with argument
colormatrix=bt601:bt709.
Successfully parsed a group of options.
Opening an output file:
/rdo/staging/rdo_graph_core/rdo_graph_core_3852239/tmpjxs_EA/split_8.mov.
[file @ 0x4297080] Setting default whitelist 'file,crypto'
Successfully opened the file.
detected 56 logical cores
Stream mapping:
  Stream #0:0 -> #0:0 (exr (native) -> h264 (libx264))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per
stream)
    Last message repeated 1 times
[exr @ 0x4294000] line order: 0.
[Parsed_colormatrix_0 @ 0x44f9a00] Setting 'src' to value 'bt601'
[Parsed_colormatrix_0 @ 0x44f9a00] Setting 'dst' to value 'bt709'
[graph 0 input from stream 0:0 @ 0x44f9f00] Setting 'video_size' to value
'1920x1080'
[graph 0 input from stream 0:0 @ 0x44f9f00] Setting 'pix_fmt' to value '35'
[graph 0 input from stream 0:0 @ 0x44f9f00] Setting 'time_base' to value
'50/1199'
[graph 0 input from stream 0:0 @ 0x44f9f00] Setting 'pixel_aspect' to value
'1/1'
[graph 0 input from stream 0:0 @ 0x44f9f00] Setting 'sws_param' to value
'flags=2'
[graph 0 input from stream 0:0 @ 0x44f9f00] Setting 'frame_rate' to value
'1199/50'
[graph 0 input from stream 0:0 @ 0x44f9f00] w:1920 h:1080 pixfmt:rgb48le
tb:50/1199 fr:1199/50 sar:1/1 sws_param:flags=2
[format @ 0x44fb380] Setting 'pix_fmts' to value 'yuv420p'
[auto_scaler_0 @ 0x44fb7c0] Setting 'flags' to value 'bicubic'
[auto_scaler_0 @ 0x44fb7c0] w:iw h:ih flags:'bicubic' interl:0
[trim_in_0_0 @ 0x44fa640] auto-inserting filter 'auto_scaler_0' between the
filter 'graph 0 input from stream 0:0' and the filter 'trim_in_0_0'
[AVFilterGraph @ 0x4293a00] query_formats: 5 queried, 3 merged, 1 already
done, 0 delayed
[auto_scaler_0 @ 0x44fb7c0] w:1920 h:1080 fmt:rgb48le sar:1/1 -> w:1920
h:1080 fmt:yuv420p sar:1/1 flags:0x4
[Parsed_colormatrix_0 @ 0x44f9a00] bt601 -> bt709
[libx264 @ 0x42927c0] using mv_range_thread = 24
[libx264 @ 0x42927c0] using SAR=1/1
[libx264 @ 0x42927c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
AVX FMA3 BMI2 AVX2
[libx264 @ 0x42927c0] profile High, level 4.0, 4:2:0, 8-bit
[libx264 @ 0x42927c0] 264 - core 158 r2984 3759fcb - H.264/MPEG-4 AVC codec
- Copyleft 2003-2019 - http://www.videolan.org/x264.html - options: cabac=1
ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00
mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11
fast_pskip=1 chroma_qp_offset=-2 threads=34 lookahead_threads=5
sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0
constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1
weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=23 scenecut=40
intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=20.0 qcomp=0.60 qpmin=0
qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mov, to
'/rdo/staging/rdo_graph_core/rdo_graph_core_3852239/tmpjxs_EA/split_8.mov':
  Metadata:
    timecode        : 00:00:00:00
    encoder         : Lavf58.20.100
    Stream #0:0, 0, 1/19184: Video: h264 (libx264), 1 reference frame (avc1
/ 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 0/1, q=-1--1, 0.04
fps, 19184 tbn, 23.98 tbc
    Metadata:
      encoder         : Lavc58.35.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Clipping frame in rate conversion by 0.000008
Segmentation fault (core dumped)

When it segfault, it always segfault there.

[image: RODEO FX] <https://www.rodeofx.com/>
[image: Facebook] <https://www.facebook.com/rodeofx/> [image: Twitter]
<https://twitter.com/FXRodeo> [image: LinkedIn]
<https://www.linkedin.com/company/251532/admin/> [image: Vimeo]
<https://vimeo.com/rodeofx> [image: Instagram]
<https://www.instagram.com/rodeofxstyle/>
*Jonathan Niquet <https://ca.linkedin.com/in/jonathan-niquet-62b37239>*
Application Support Specialist
*[hidden email] <[hidden email]> *
514.397.9999 ext.599
*www.rodeofx.com <https://www.rodeofx.com/>*


On Thu, Sep 12, 2019 at 2:40 PM Jonathan Niquet <[hidden email]> wrote:

> Hey there,
>
> We've been using John Van Sickle's ffmpeg static build for quite a while
> and it's worked wonderfully for us.
>
> However, we recently had to incorporate aac into our build.
>
> I then started to statically compile every library we needed and proceed
> to successfully build FFmpeg.
>
> This is on Linux, centos 7.6.
> This was my config arguments:
>
> --extra-ldflags="-L$/home/jon/ffmpegDependancies/lib"
> --extra-ldflags="-L$/usr/local/lib"
> --extra-cflags="-I/home/jon/ffmpegDependancies/include"
> --prefix="/home/jon/ffmpegBuild" --extra-libs="-lpthread -lm -fopenmp"
> --enable-gpl --enable-version3 --enable-nonfree
> --pkg-config-flags="--static" --enable-static --disable-shared
> --enable-debug --disable-ffplay --disable-indev=sndio
> --disable-outdev=sndio --enable-libfdk-aac --enable-frei0r --enable-libaom
> --enable-libfribidi  --enable-libass --enable-libvpx --enable-libx264
> --enable-libwebp --enable-libxml2 --enable-fontconfig --enable-gray
> --enable-libfreetype --enable-libzvbi --enable-libvidstab --enable-libvmaf
> --enable-libvorbis --enable-libmp3lame --enable-libopus
> --enable-libopenjpeg --enable-libzimg --enable-libspeex --enable-libx265
> --enable-gmp --enable-libvo-amrwbenc --enable-libopencore-amrwb
> --enable-libopencore-amrnb --enable-libsoxr --enable-libxvid
>
> However, in a random fashion, whenever we use "*-pix_fmt yuv420p*" when
> encoding libx264
> (*-c:v libx264 -c:a copy -crf 20 -pix_fmt yuv420p -vf
> colormatrix=bt601:bt709*), FFmpeg will randomly segfault between 40% to
> 45% of the time. Using the same command 20 times in a row, on the same
> computer, will literally give 11 completion and 9 segfaults. I've tried
> recompiling the whole things on a virtual machine specially setup for this
> task (blank, minimal install) and the exact same behavior is observed.
>
> I've tried to strace the process, I didn't notice anything that sounds
> like a solution.
> From John's static build, I only removed 3 libraries: gnutls,
> librubberband and libtheora.
>
> I'm thinking it might be a problem with my x264 compilation, the
> randomness of the things might point to threading issues, but I'm not too
> sure what to look at at this point.
>
> Anyone experienced something similar before or would have clues to help me
> debug this hellish instability?
>
> Thanks!
> [image: RODEO FX] <https://www.rodeofx.com/>
> [image: Facebook] <https://www.facebook.com/rodeofx/> [image: Twitter]
> <https://twitter.com/FXRodeo> [image: LinkedIn]
> <https://www.linkedin.com/company/251532/admin/> [image: Vimeo]
> <https://vimeo.com/rodeofx> [image: Instagram]
> <https://www.instagram.com/rodeofxstyle/>
> *Jonathan Niquet <https://ca.linkedin.com/in/jonathan-niquet-62b37239>*
> Application Support Specialist
> *[hidden email] <[hidden email]> *
> 514.397.9999 ext.599
> *www.rodeofx.com <https://www.rodeofx.com/>*
>
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Compiled ffmpeg - x264 - instability when using -pix_fmt yuv420p

Carl Eugen Hoyos-2
Am Do., 12. Sept. 2019 um 21:15 Uhr schrieb Jonathan Niquet
<[hidden email]>:

> just realized i forgot some important information:
> this is for ffmpeg 4.1.1 but I also tried 4.2.1 and the exact same behavior
> was observed:

But only current FFmpeg git head is supported.

Backtrace, disassembly and register dump missing.

Please find out what top-posting means and avoid it here.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".