On the recovery behavior of HLS muxer when slaved to tee'd fifo muxers in ffmpeg.

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

On the recovery behavior of HLS muxer when slaved to tee'd fifo muxers in ffmpeg.

Brainiarc7
Four months ago, I posted a ticket on trac, #8387, see
https://trac.ffmpeg.org/ticket/8387

Which described an anomaly with the recovery behavior of the underlying HLS
muxers called up via the tee's fifo slaves. More details in the ticket.

What I've discovered by extensively studying this behavior is that when
dealing with fragmented mp4, the HLS muxer (as configured above) will
definitely fail to re-create the init.mp4 file should the target path (on a
local or remote filesystem) go missing.

This is still present with ffmpeg git head, compiled a few hours ago.

Case in point: Simulating failures with unstable NFS mounts, using the same
recovery options for fifo work flawlessly for DASH where the init segments
per variant are successfully re-created but the same doesn't carry over for
HLS.

What gives?

Are there specific tee and fifo muxer settings I can try out to resolve
this?

Thanks,

Dennis.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: On the recovery behavior of HLS muxer when slaved to tee'd fifo muxers in ffmpeg.

Brainiarc7
On Wed, 25 Mar 2020 at 08:27, Dennis Mungai <[hidden email]> wrote:

> Four months ago, I posted a ticket on trac, #8387, see
> https://trac.ffmpeg.org/ticket/8387
>
> Which described an anomaly with the recovery behavior of the underlying
> HLS muxers called up via the tee's fifo slaves. More details in the ticket.
>
> What I've discovered by extensively studying this behavior is that when
> dealing with fragmented mp4, the HLS muxer (as configured above) will
> definitely fail to re-create the init.mp4 file should the target path (on a
> local or remote filesystem) go missing.
>
> This is still present with ffmpeg git head, compiled a few hours ago.
>
> Case in point: Simulating failures with unstable NFS mounts, using the
> same recovery options for fifo work flawlessly for DASH where the init
> segments per variant are successfully re-created but the same doesn't carry
> over for HLS.
>
> What gives?
>
> Are there specific tee and fifo muxer settings I can try out to resolve
> this?
>
> Thanks,
>
> Dennis.
>

Update: A bit too soon, perhaps I should also update the subject.
DASH demonstrates something similar:
In effect, the muxer tries to re-create the init segment(s) for each
variant stream(s), but fails with the ominous message:

"Application provided invalid, non monotonically increasing dts to muxer in
stream 0"
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: On the recovery behavior of HLS muxer when slaved to tee'd fifo muxers in ffmpeg.

Brainiarc7
On Wed, 25 Mar 2020 at 08:32, Dennis Mungai <[hidden email]> wrote:

> On Wed, 25 Mar 2020 at 08:27, Dennis Mungai <[hidden email]> wrote:
>
>> Four months ago, I posted a ticket on trac, #8387, see
>> https://trac.ffmpeg.org/ticket/8387
>>
>> Which described an anomaly with the recovery behavior of the underlying
>> HLS muxers called up via the tee's fifo slaves. More details in the ticket.
>>
>> What I've discovered by extensively studying this behavior is that when
>> dealing with fragmented mp4, the HLS muxer (as configured above) will
>> definitely fail to re-create the init.mp4 file should the target path (on a
>> local or remote filesystem) go missing.
>>
>> This is still present with ffmpeg git head, compiled a few hours ago.
>>
>> Case in point: Simulating failures with unstable NFS mounts, using the
>> same recovery options for fifo work flawlessly for DASH where the init
>> segments per variant are successfully re-created but the same doesn't carry
>> over for HLS.
>>
>> What gives?
>>
>> Are there specific tee and fifo muxer settings I can try out to resolve
>> this?
>>
>> Thanks,
>>
>> Dennis.
>>
>
> Update: A bit too soon, perhaps I should also update the subject.
> DASH demonstrates something similar:
> In effect, the muxer tries to re-create the init segment(s) for each
> variant stream(s), but fails with the ominous message:
>
> "Application provided invalid, non monotonically increasing dts to muxer
> in stream 0"
>
>

Update: I fixed the issue.

For NFS mounts, use these options in fstab:

$server_ip:/$nfs_mount /$mount_point nfs
timeo=21,fg,nofail,retrans=600,actimeo=3,vers=4,soft,x-systemd.automount,x-systemd.mount-timeout=infinity,retry=10000_netdev
0 0

Sneaky details are in the systemd NFS guide:
https://www.freedesktop.org/software/systemd/man/systemd.mount.html

And for FFmpeg, configure the muxer options as shown:

(a). For fifo muxers called up by tee, pass these options:
use_fifo=1:fifo_options=queue_size=90000\\:attempt_recovery=1\\:drop_pkts_on_overflow=1
(b). For both HLS and DASH, pass the option ignore_io_errors=1

This was tested with ffmpeg 1:N transcode to multiple NFS mounted
directories for each output, and can confirm clean recovery when NFS
server(s) are restarted.
I cannot (yet) explain why this setup works differently from a similar case
but with local filesystems, but as tested over the last two days, this is
what I got to work with no issues.

Leaving this here incase someone else runs into the same issue with NFS and
transparent tee/fifo recovery with ffmpeg and HLS output.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".