Robustness on processing transport streams (TS)?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Robustness on processing transport streams (TS)?

Uwe Freese
Hello,

how robust and precise especially regarding a/v sync is ffmpeg when
processing transport streams (*.ts) files from TV recordings?

Some programs (e.g. Avidemux on Linux) have problems (crash) with some
*.ts files I have, and I thought it maybe is a good idea to first
convert my input to a "raw" file with:

ffmpeg -i input.ts -map 0:v:0 -c:v huffyuv -map 0:a -c:a pcm_s16le -sn
temp.mkv

and use that hopefully "corrected" version for further processing (and
encoding).


Question is, does ffmpeg a good job detecting gaps and errors and fill
out the produces audio streams with silence etc, so they stay in sync?

I mean, when I play whatever corrupt file in VLC, it does a good job in
this. How is it in ffmpeg?

Any other program that someone suggests to (pre)process and "correct"
such ts files under Linux?


Some example output I had for a TV recording (which is not from a bad
reception or so - it's totally ok when playing it back):

$ ffmpeg -i input.ts -map 0:v:0 -c:v huffyuv -map 0:a -c:a pcm_s16le -sn
temp.mkv
ffmpeg version 3.2.12-1~deb9u1 Copyright (c) 2000-2018 the FFmpeg developers
   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
   configuration: --prefix=/usr --extra-version='1~deb9u1'
--toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu
--incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping
--enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa
--enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca
--enable-libcdio --enable-libebur128 --enable-libflite
--enable-libfontconfig --enable-libfreetype --enable-libfribidi
--enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg
--enable-libopenmpt --enable-libopus --enable-libpulse
--enable-librubberband --enable-libshine --enable-libsnappy
--enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora
--enable-libtwolame --enable-libvorbis --enable-libvpx
--enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid
--enable-libzmq --enable-libzvbi --enable-omx --enable-openal
--enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883
--enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264
--enable-shared
   libavutil      55. 34.101 / 55. 34.101
   libavcodec     57. 64.101 / 57. 64.101
   libavformat    57. 56.101 / 57. 56.101
   libavdevice    57.  1.100 / 57.  1.100
   libavfilter     6. 65.100 /  6. 65.100
   libavresample   3.  1.  0 /  3.  1.  0
   libswscale      4.  2.100 /  4.  2.100
   libswresample   2.  3.100 /  2.  3.100
   libpostproc    54.  1.100 / 54.  1.100
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] decode_slice_header error
[h264 @ 0x558e3216ee00] no frame!
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] decode_slice_header error
[h264 @ 0x558e3216ee00] no frame!
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] decode_slice_header error
[h264 @ 0x558e3216ee00] no frame!
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] decode_slice_header error
[h264 @ 0x558e3216ee00] no frame!
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] decode_slice_header error
[h264 @ 0x558e3216ee00] no frame!
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] decode_slice_header error
[h264 @ 0x558e3216ee00] no frame!
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] decode_slice_header error
[h264 @ 0x558e3216ee00] no frame!
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] decode_slice_header error
[h264 @ 0x558e3216ee00] no frame!
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] decode_slice_header error
[h264 @ 0x558e3216ee00] no frame!
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] SPS unavailable in decode_picture_timing
[h264 @ 0x558e3216ee00] non-existing PPS 0 referenced
[h264 @ 0x558e3216ee00] decode_slice_header error
[h264 @ 0x558e3216ee00] no frame!
[mpegts @ 0x558e32145f60] PES packet size mismatch
     Last message repeated 3 times
[mpegts @ 0x558e32145f60] Could not find codec parameters for stream 0
(Unknown: none ([5][0][0][0] / 0x0005)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize'
options
[mpegts @ 0x558e32145f60] Could not find codec parameters for stream 1
(Unknown: none ([11][0][0][0] / 0x000B)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize'
options
[mpegts @ 0x558e32145f60] Could not find codec parameters for stream 2
(Unknown: none ([12][0][0][0] / 0x000C)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize'
options
[mpegts @ 0x558e32145f60] Could not find codec parameters for stream 9
(Unknown: none ([11][0][0][0] / 0x000B)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize'
options
Input #0, mpegts, from 'Ohnsorg-Theater - Die Kartenlegerin.ts':
   Duration: 03:00:01.26, start: 25628.222800, bitrate: 13827 kb/s
   Program 10325
   Program 10326
   Program 10327
     Stream #0:0[0x6ea]: Unknown: none ([5][0][0][0] / 0x0005)
     Stream #0:1[0x87b]: Unknown: none ([11][0][0][0] / 0x000B)
     Stream #0:2[0xa74]: Unknown: none ([12][0][0][0] / 0x000C)
     Stream #0:3[0x1465]: Video: h264 (High) ([27][0][0][0] / 0x001B),
yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k
tbn, 100 tbc
     Stream #0:4[0x1466](deu): Audio: mp2 ([3][0][0][0] / 0x0003), 48000
Hz, stereo, s16p, 192 kb/s
     Stream #0:5[0x1467](mis): Audio: mp2 ([3][0][0][0] / 0x0003), 48000
Hz, stereo, s16p, 192 kb/s
     Stream #0:6[0x1468](deu): Subtitle: dvb_teletext ([6][0][0][0] /
0x0006)
     Stream #0:7[0x1469](deu): Subtitle: dvb_subtitle ([6][0][0][0] /
0x0006) (hearing impaired)
     Stream #0:8[0x146a](deu): Audio: ac3 ([6][0][0][0] / 0x0006), 48000
Hz, stereo, fltp, 448 kb/s
     Stream #0:9[0x1499]: Unknown: none ([11][0][0][0] / 0x000B)
   Program 10328
   Program 10329
   Program 10330
   Program 10331
[huffyuv @ 0x558e3241e960] using huffyuv 2.2.0 or newer interlacing flag
[huffyuv @ 0x558e32527940] using huffyuv 2.2.0 or newer interlacing flag
[huffyuv @ 0x558e32630820] using huffyuv 2.2.0 or newer interlacing flag
[huffyuv @ 0x558e32739700] using huffyuv 2.2.0 or newer interlacing flag
[huffyuv @ 0x558e321fe620] using huffyuv 2.2.0 or newer interlacing flag
Output #0, matroska, to 'video_encode_input.mkv':
   Metadata:
     encoder         : Lavf57.56.101
     Stream #0:0: Video: huffyuv (HFYU / 0x55594648), yuv422p, 1280x720
[SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 50 fps, 1k tbn, 50 tbc
     Metadata:
       encoder         : Lavc57.64.101 huffyuv
     Stream #0:1(deu): Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000
Hz, stereo, s16, 1536 kb/s
     Metadata:
       encoder         : Lavc57.64.101 pcm_s16le
     Stream #0:2(mis): Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000
Hz, stereo, s16, 1536 kb/s
     Metadata:
       encoder         : Lavc57.64.101 pcm_s16le
     Stream #0:3(deu): Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000
Hz, stereo, s16, 1536 kb/s
     Metadata:
       encoder         : Lavc57.64.101 pcm_s16le
Stream mapping:
   Stream #0:3 -> #0:0 (h264 (native) -> huffyuv (native))
   Stream #0:4 -> #0:1 (mp2 (native) -> pcm_s16le (native))
   Stream #0:5 -> #0:2 (mp2 (native) -> pcm_s16le (native))
   Stream #0:8 -> #0:3 (ac3 (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
[h264 @ 0x558e321b08c0] co located POCs unavailable
[h264 @ 0x558e321b3060] co located POCs unavailable
[h264 @ 0x558e323c7200] co located POCs unavailable
[h264 @ 0x558e323e2aa0] co located POCs unavailable
[mpegts @ 0x558e32145f60] PES packet size mismatch:15:18.54
bitrate=291523.3kbits/s speed=3.17x
[h264 @ 0x558e323e2aa0] concealing 2831 DC, 2831 AC, 2831 MV errors in P
frame
[ac3 @ 0x558e3220b520] exponent -1 is out-of-range
[ac3 @ 0x558e3220b520] error decoding the audio block
[ac3 @ 0x558e3220b520] frame sync error
Error while decoding stream #0:8: Invalid data found when processing input
[h264 @ 0x558e323fe500] co located POCs unavailable
[h264 @ 0x558e321b08c0] co located POCs unavailable
[h264 @ 0x558e321b3060] co located POCs unavailable
[h264 @ 0x558e323c7200] co located POCs unavailable
[h264 @ 0x558e323e2aa0] co located POCs unavailable
[h264 @ 0x558e323fe500] co located POCs unavailable
[h264 @ 0x558e321b08c0] co located POCs unavailable
[mpegts @ 0x558e32145f60] PES packet size mismatch
     Last message repeated 2 times
[h264 @ 0x558e321b3060] concealing 1505 DC, 1505 AC, 1505 MV errors in P
frame
[ac3 @ 0x558e3220b520] exponent 25 is out-of-range
[ac3 @ 0x558e3220b520] error decoding the audio block
[ac3 @ 0x558e3220b520] frame sync error
Error while decoding stream #0:8: Invalid data found when processing input
Past duration 0.813881 too large
[mp2 @ 0x558e321fdd00] Header missing
Error while decoding stream #0:4: Invalid data found when processing input
[mp2 @ 0x558e3218aca0] Header missing
Error while decoding stream #0:5: Invalid data found when processing input
[mpegts @ 0x558e32145f60] PES packet size mismatch:15:19.92
bitrate=291724.4kbits/s speed=3.17x
     Last message repeated 1 times
[h264 @ 0x558e323fe500] concealing 3071 DC, 3071 AC, 3071 MV errors in B
frame
[h264 @ 0x558e321b08c0] co located POCs unavailable
[h264 @ 0x558e321b3060] co located POCs unavailable
[mpegts @ 0x558e32145f60] PES packet size mismatch
[ac3 @ 0x558e3220b520] frame sync error
Error while decoding stream #0:8: Invalid data found when processing input
[h264 @ 0x558e323c7200] co located POCs unavailable
[h264 @ 0x558e323e2aa0] co located POCs unavailable
[mp2 @ 0x558e321fdd00] Header missing
Error while decoding stream #0:4: Invalid data found when processing input
[mp2 @ 0x558e3218aca0] Header missing
Error while decoding stream #0:5: Invalid data found when processing input

...

Regards,
Uwe

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Robustness on processing transport streams (TS)?

Carl Eugen Hoyos-2
2018-12-27 15:14 GMT+01:00, Uwe Freese <[hidden email]>:

> how robust and precise especially regarding a/v sync is ffmpeg when
> processing transport streams (*.ts) files from TV recordings?

I have seen many samples that can be transcoded in-sync with FFmpeg.

> Some programs (e.g. Avidemux on Linux) have problems (crash)

If you have an input file that crashes FFmpeg, please share it!

> with some *.ts files I have, and I thought it maybe is a good idea
> to first convert my input to a "raw" file with:
>
> ffmpeg -i input.ts -map 0:v:0 -c:v huffyuv -map 0:a -c:a pcm_s16le
> -sn temp.mkv

I would assume that this makes possible problems with A/V sync
immutable. Always use the original input file for transcoding.

A long time back, there was ProjectX to fix transport streams, I
believe there is another program now (suggested by the HandBrake
user support) but I forgot its name.

To "insert silence", FFmpeg needs the "-async 1" option or the
corresponding filter chain.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Robustness on processing transport streams (TS)?

Uwe Freese
Hello,
> If you have an input file that crashes FFmpeg, please share it!
It was only crashing Avidemux when seeking to the end. - "Sorry". ;-)
> A long time back, there was ProjectX to fix transport streams, I
> believe there is another program now (suggested by the HandBrake
> user support) but I forgot its name.

ProjectX doesn't work with HD. Under Win****, I used "TSDoctor", but I'm
now working on Linux only and didn't find anything appropriate.

But it seems ffmpeg is then enough for this purpose (I'm not expecting
"very damaged" recordings).

> To "insert silence", FFmpeg needs the "-async 1" option or the
> corresponding filter chain.

Very good hint, thanks!

Just a small point I'm curious about. Help says:

-async             <float>      ....A... simplified 1 parameter audio
timestamp matching, 0(disabled), 1(filling and trimming), >1(maximum
stretch/squeeze in samples per second) (from INT_MIN to INT_MAX) (default 0)

Why does it say "float" and at the end "from INT_MIN to INT_MAX"? It's
also defined like that in the code in libswresample/options.c, also for
other parameters.

{"async"                , "simplified 1 parameter audio timestamp
matching, 0(disabled), 1(filling and trimming), >1(maximum
stretch/squeeze in samples per second)"
                                                         ,
OFFSET(async)          , AV_OPT_TYPE_FLOAT ,{.dbl=0                    
}, INT_MIN, INT_MAX   , PARAM },

A bug, or how does it make sense to use INT border values with a float?

Regards,
Uwe

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Robustness on processing transport streams (TS)?

Carl Eugen Hoyos-2
2018-12-27 22:21 GMT+01:00, Uwe Freese <[hidden email]>:

>> A long time back, there was ProjectX to fix transport streams, I
>> believe there is another program now (suggested by the HandBrake
>> user support) but I forgot its name.
>
> ProjectX doesn't work with HD.

HD should have no issues, h264 support is very limited iirc.

> Under Win****, I used "TSDoctor", but I'm now working on
> Linux only and didn't find anything appropriate.

That's probably what I read about.

> But it seems ffmpeg is then enough for this purpose (I'm not expecting
> "very damaged" recordings).
>
>> To "insert silence", FFmpeg needs the "-async 1" option or the
>> corresponding filter chain.
>
> Very good hint, thanks!
>
> Just a small point I'm curious about. Help says:
>
> -async             <float>      ....A... simplified 1 parameter audio
> timestamp matching, 0(disabled), 1(filling and trimming), >1(maximum
> stretch/squeeze in samples per second) (from INT_MIN to INT_MAX) (default 0)
>
> Why does it say "float" and at the end "from INT_MIN to INT_MAX"? It's
> also defined like that in the code in libswresample/options.c, also for
> other parameters.
>
> {"async"                , "simplified 1 parameter audio timestamp
> matching, 0(disabled), 1(filling and trimming), >1(maximum
> stretch/squeeze in samples per second)"
>                                                          ,
> OFFSET(async)          , AV_OPT_TYPE_FLOAT ,{.dbl=0
> }, INT_MIN, INT_MAX   , PARAM },
>
> A bug, or how does it make sense to use INT border values with a float?

I don't know but I believe this values are already too large for real
use cases.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Robustness on processing transport streams (TS)?

Moritz Barsnick
In reply to this post by Uwe Freese
On Thu, Dec 27, 2018 at 22:21:26 +0100, Uwe Freese wrote:
> -async             <float>      ....A... simplified 1 parameter audio
> timestamp matching, 0(disabled), 1(filling and trimming), >1(maximum
> stretch/squeeze in samples per second) (from INT_MIN to INT_MAX) (default 0)

That's the swresample option, not the ffmpeg option. "swr simple 1
parameter async, similar to ffmpegs -async"

> Why does it say "float" and at the end "from INT_MIN to INT_MAX"? It's
> also defined like that in the code in libswresample/options.c, also for
> other parameters.

*shrug*

ffmpeg's "-async" option is defined differently.

Moritz
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".