Quantcast

Audio/video stream are out of sync?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Audio/video stream are out of sync?

christina zou
Hi,

I am trying to stream audio and video from my Pi to Youtube Live using
FFMPEG. My stream's audio and video are not syncing, because the audio is
perfectly realtime, but the video skips forward frequently.

Here is an example streamed video:
https://www.youtube.com/watch?v=Bz12pDbay0U

My Pi's upload speed is 5 Mbps and I'm using no more than 30% of CPU.

If I write to a FLV file instead of streaming to Youtube, the audio/video
sync is perfect.

I have tried params like -async 2, -vsync 2, and -re, but no improvement.
Any suggestions to improve the sync?


*My command:*

*sudo rm temp_audio.v*

*sudo rm temp_video.h264*

*mkfifo temp_audio.v*

*mkfifo temp_video.h264*

*arecord -Dmic_sv -c2 -r48000 -fS32_LE -twav temp_audio.v & \*

*raspivid -fps 10 -v -b 3000000 -o temp_video.h264 -t 0 & \*

~/special/ffmpeg/ffmpeg* \*

*    -framerate 10 \*

*    -re \*

*    -i temp_video.h264 \*

*    -i temp_audio.v \*

*    -ab 32k \*

*    -ac 2 \*

*    -c:v copy \*

*    -c:a aac \*

*    -report \*

*    -f flv **rtmp://209.85.230.23/live2/KEY*
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Audio/video stream are out of sync?

Carl Eugen Hoyos-2
2017-02-08 22:57 GMT+01:00 christina zou <[hidden email]>:

> *arecord -Dmic_sv -c2 -r48000 -fS32_LE -twav temp_audio.v & \*

Why?
(See below)

> *raspivid -fps 10 -v -b 3000000 -o temp_video.h264 -t 0 & \*

Is this not possible with ffmpeg alone?

> ~/special/ffmpeg/ffmpeg* \*
> *    -framerate 10 \*
> *    -re \*
> *    -i temp_video.h264 \*
> *    -i temp_audio.v \*

(Complete, uncut console output missing.)
How are these two input streams supposed to be synced?

Normally, you would use (for example) alsa and v4l2 input
and hope that the two drivers both provide wallclock timestamps.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Audio/video stream are out of sync?

christina zou
Carl,

I spent a lot of time trying to use ffmpeg alone (alsa/v4l2), but it
produces extreme stuttering on the audio track. Using my approach of
writing to named pipes and sending those into ffmpeg, I add a bit of
latency but ensure that there's no loss of frames. This was the approach
suggested to me by the ffmpeg IRC channel. I'm on a Pi Zero (single core),
by the way.

Is there any way to align the two input streams' timestamps, if they are
both being read from named pipes?

Here is my full console output: http://pastebin.com/phmiphaL

Thanks,
Christina



On Wed, Feb 8, 2017 at 3:02 PM, Carl Eugen Hoyos <[hidden email]> wrote:

> 2017-02-08 22:57 GMT+01:00 christina zou <[hidden email]>:
>
> > *arecord -Dmic_sv -c2 -r48000 -fS32_LE -twav temp_audio.v & \*
>
> Why?
> (See below)
>
> > *raspivid -fps 10 -v -b 3000000 -o temp_video.h264 -t 0 & \*
>
> Is this not possible with ffmpeg alone?
>
> > ~/special/ffmpeg/ffmpeg* \*
> > *    -framerate 10 \*
> > *    -re \*
> > *    -i temp_video.h264 \*
> > *    -i temp_audio.v \*
>
> (Complete, uncut console output missing.)
> How are these two input streams supposed to be synced?
>
> Normally, you would use (for example) alsa and v4l2 input
> and hope that the two drivers both provide wallclock timestamps.
>
> Carl Eugen
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Audio/video stream are out of sync?

christina zou
I've made some progress thanks to advice from the IRC channel. I increased
my pipe max buffer size as high as I could:

sysctl fs.pipe-max-size=1048576

I also lowered my video resolution and bitrate.

Now, with the below command I've managed 3 minutes of fully synced
audio/video livestream to Youtube from my Pi Zero. At the 3 minute mark,
there's a slight desync, and the audio goes about 0.5s ahead of the video.

That's my next question. I don't mind if there's a stutter every 3 minutes,
but I need the A/V sync to recover after that. So if I lose a few audio
frames, I need the video to catch up (or audio to slow down). Any
suggestions about how to do this (either ffmpeg param, or script)?

Here's my current command:

sysctl fs.pipe-max-size=1048576

*sudo rm temp_audio.v*

*sudo rm temp_video.h264*

*mkfifo temp_audio.v*

*mkfifo temp_video.h264*

*arecord -Dmic_sv -c2 -r48000 -fS32_LE -twav temp_audio.v & \*

*raspivid *-w *640* -h *480** -fps 10 -v -b 1000000 -o temp_video.h264 -t 0
& \*

~/special/ffmpeg/ffmpeg* \*

*    -framerate 10 \*

*    -i temp_video.h264 \*

*    -i temp_audio.v \*

*    -ab 24k \*

*    -c:v copy \*

*    -c:a aac \*

*    -report \*

*    -f flv **rtmp://209.85.230.23/live2/KEY*

On Wed, Feb 8, 2017 at 3:50 PM, christina zou <[hidden email]>
wrote:

> Carl,
>
> I spent a lot of time trying to use ffmpeg alone (alsa/v4l2), but it
> produces extreme stuttering on the audio track. Using my approach of
> writing to named pipes and sending those into ffmpeg, I add a bit of
> latency but ensure that there's no loss of frames. This was the approach
> suggested to me by the ffmpeg IRC channel. I'm on a Pi Zero (single core),
> by the way.
>
> Is there any way to align the two input streams' timestamps, if they are
> both being read from named pipes?
>
> Here is my full console output: http://pastebin.com/phmiphaL
>
> Thanks,
> Christina
>
>
>
> On Wed, Feb 8, 2017 at 3:02 PM, Carl Eugen Hoyos <[hidden email]>
> wrote:
>
>> 2017-02-08 22:57 GMT+01:00 christina zou <[hidden email]>:
>>
>> > *arecord -Dmic_sv -c2 -r48000 -fS32_LE -twav temp_audio.v & \*
>>
>> Why?
>> (See below)
>>
>> > *raspivid -fps 10 -v -b 3000000 -o temp_video.h264 -t 0 & \*
>>
>> Is this not possible with ffmpeg alone?
>>
>> > ~/special/ffmpeg/ffmpeg* \*
>> > *    -framerate 10 \*
>> > *    -re \*
>> > *    -i temp_video.h264 \*
>> > *    -i temp_audio.v \*
>>
>> (Complete, uncut console output missing.)
>> How are these two input streams supposed to be synced?
>>
>> Normally, you would use (for example) alsa and v4l2 input
>> and hope that the two drivers both provide wallclock timestamps.
>>
>> Carl Eugen
>> _______________________________________________
>> ffmpeg-user mailing list
>> [hidden email]
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>>
>> To unsubscribe, visit link above, or email
>> [hidden email] with subject "unsubscribe".
>
>
>
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Audio/video stream are out of sync?

Erik Dobberkau
Am Donnerstag, 9. Februar 2017 schrieb christina zou :

> I've made some progress thanks to advice from the IRC channel. I increased
> my pipe max buffer size as high as I could:
>
> sysctl fs.pipe-max-size=1048576
>
> I also lowered my video resolution and bitrate.
>
> Now, with the below command I've managed 3 minutes of fully synced
> audio/video livestream to Youtube from my Pi Zero. At the 3 minute mark,
> there's a slight desync, and the audio goes about 0.5s ahead of the video.
>
> That's my next question. I don't mind if there's a stutter every 3 minutes,
> but I need the A/V sync to recover after that. So if I lose a few audio
> frames, I need the video to catch up (or audio to slow down). Any
> suggestions about how to do this (either ffmpeg param, or script)?
>
> Here's my current command:
>
> sysctl fs.pipe-max-size=1048576
>
> *sudo rm temp_audio.v*
>
> *sudo rm temp_video.h264*
>
> *mkfifo temp_audio.v*
>
> *mkfifo temp_video.h264*
>
> *arecord -Dmic_sv -c2 -r48000 -fS32_LE -twav temp_audio.v & \*
>
> *raspivid *-w *640* -h *480** -fps 10 -v -b 1000000 -o temp_video.h264 -t 0
> & \*
>
> ~/special/ffmpeg/ffmpeg* \*
>
> *    -framerate 10 \*
>
> *    -i temp_video.h264 \*
>
> *    -i temp_audio.v \*
>
> *    -ab 24k \*
>
> *    -c:v copy \*
>
> *    -c:a aac \*
>
> *    -report \*
>
> *    -f flv **rtmp://209.85.230.23/live2/KEY*
>
> On Wed, Feb 8, 2017 at 3:50 PM, christina zou <[hidden email]
> <javascript:;>>
> wrote:
>
> > Carl,
> >
> > I spent a lot of time trying to use ffmpeg alone (alsa/v4l2), but it
> > produces extreme stuttering on the audio track. Using my approach of
> > writing to named pipes and sending those into ffmpeg, I add a bit of
> > latency but ensure that there's no loss of frames. This was the approach
> > suggested to me by the ffmpeg IRC channel. I'm on a Pi Zero (single
> core),
> > by the way.
> >
> > Is there any way to align the two input streams' timestamps, if they are
> > both being read from named pipes?
> >
> > Here is my full console output: http://pastebin.com/phmiphaL
> >
> > Thanks,
> > Christina
> >
> >
> >
> > On Wed, Feb 8, 2017 at 3:02 PM, Carl Eugen Hoyos <[hidden email]
> <javascript:;>>
> > wrote:
> >
> >> 2017-02-08 22:57 GMT+01:00 christina zou <[hidden email]
> <javascript:;>>:
> >>
> >> > *arecord -Dmic_sv -c2 -r48000 -fS32_LE -twav temp_audio.v & \*
> >>
> >> Why?
> >> (See below)
> >>
> >> > *raspivid -fps 10 -v -b 3000000 -o temp_video.h264 -t 0 & \*
> >>
> >> Is this not possible with ffmpeg alone?
> >>
> >> > ~/special/ffmpeg/ffmpeg* \*
> >> > *    -framerate 10 \*
> >> > *    -re \*
> >> > *    -i temp_video.h264 \*
> >> > *    -i temp_audio.v \*
> >>
> >> (Complete, uncut console output missing.)
> >> How are these two input streams supposed to be synced?
> >>
> >> Normally, you would use (for example) alsa and v4l2 input
> >> and hope that the two drivers both provide wallclock timestamps.
> >>
> >> Carl Eugen
>

My suspicion is the transcoder the stream is being fed to doesn't cope well
with the framerate. In my experience it's most stable when using either 25
or 30fps with a keyframe interval of 50 or 60 frames maximum. Also best to
use standard bitrates, especially with audio, to avoid drifting.

Erik
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Audio/video stream are out of sync?

christina zou
Thanks for the reply. I am using a little wifi module on my Pi Zero and
want to send the minimum amount of data so I don't experience buffering and
frame drops. So I am using 10 fps. I don't think my setup could handle
25-30 fps?

Should I be adding in a param related to keyframes, though?

It is indeed my audio that drifted out of sync after 3 minutes, so I will
try adjusting the bitrate back to 32K.

On Wed, Feb 8, 2017 at 11:31 PM, Erik Dobberkau <[hidden email]>
wrote:

> Am Donnerstag, 9. Februar 2017 schrieb christina zou :
>
> > I've made some progress thanks to advice from the IRC channel. I
> increased
> > my pipe max buffer size as high as I could:
> >
> > sysctl fs.pipe-max-size=1048576
> >
> > I also lowered my video resolution and bitrate.
> >
> > Now, with the below command I've managed 3 minutes of fully synced
> > audio/video livestream to Youtube from my Pi Zero. At the 3 minute mark,
> > there's a slight desync, and the audio goes about 0.5s ahead of the
> video.
> >
> > That's my next question. I don't mind if there's a stutter every 3
> minutes,
> > but I need the A/V sync to recover after that. So if I lose a few audio
> > frames, I need the video to catch up (or audio to slow down). Any
> > suggestions about how to do this (either ffmpeg param, or script)?
> >
> > Here's my current command:
> >
> > sysctl fs.pipe-max-size=1048576
> >
> > *sudo rm temp_audio.v*
> >
> > *sudo rm temp_video.h264*
> >
> > *mkfifo temp_audio.v*
> >
> > *mkfifo temp_video.h264*
> >
> > *arecord -Dmic_sv -c2 -r48000 -fS32_LE -twav temp_audio.v & \*
> >
> > *raspivid *-w *640* -h *480** -fps 10 -v -b 1000000 -o temp_video.h264
> -t 0
> > & \*
> >
> > ~/special/ffmpeg/ffmpeg* \*
> >
> > *    -framerate 10 \*
> >
> > *    -i temp_video.h264 \*
> >
> > *    -i temp_audio.v \*
> >
> > *    -ab 24k \*
> >
> > *    -c:v copy \*
> >
> > *    -c:a aac \*
> >
> > *    -report \*
> >
> > *    -f flv **rtmp://209.85.230.23/live2/KEY*
> >
> > On Wed, Feb 8, 2017 at 3:50 PM, christina zou <[hidden email]
> > <javascript:;>>
> > wrote:
> >
> > > Carl,
> > >
> > > I spent a lot of time trying to use ffmpeg alone (alsa/v4l2), but it
> > > produces extreme stuttering on the audio track. Using my approach of
> > > writing to named pipes and sending those into ffmpeg, I add a bit of
> > > latency but ensure that there's no loss of frames. This was the
> approach
> > > suggested to me by the ffmpeg IRC channel. I'm on a Pi Zero (single
> > core),
> > > by the way.
> > >
> > > Is there any way to align the two input streams' timestamps, if they
> are
> > > both being read from named pipes?
> > >
> > > Here is my full console output: http://pastebin.com/phmiphaL
> > >
> > > Thanks,
> > > Christina
> > >
> > >
> > >
> > > On Wed, Feb 8, 2017 at 3:02 PM, Carl Eugen Hoyos <[hidden email]
> > <javascript:;>>
> > > wrote:
> > >
> > >> 2017-02-08 22:57 GMT+01:00 christina zou <[hidden email]
> > <javascript:;>>:
> > >>
> > >> > *arecord -Dmic_sv -c2 -r48000 -fS32_LE -twav temp_audio.v & \*
> > >>
> > >> Why?
> > >> (See below)
> > >>
> > >> > *raspivid -fps 10 -v -b 3000000 -o temp_video.h264 -t 0 & \*
> > >>
> > >> Is this not possible with ffmpeg alone?
> > >>
> > >> > ~/special/ffmpeg/ffmpeg* \*
> > >> > *    -framerate 10 \*
> > >> > *    -re \*
> > >> > *    -i temp_video.h264 \*
> > >> > *    -i temp_audio.v \*
> > >>
> > >> (Complete, uncut console output missing.)
> > >> How are these two input streams supposed to be synced?
> > >>
> > >> Normally, you would use (for example) alsa and v4l2 input
> > >> and hope that the two drivers both provide wallclock timestamps.
> > >>
> > >> Carl Eugen
> >
>
> My suspicion is the transcoder the stream is being fed to doesn't cope well
> with the framerate. In my experience it's most stable when using either 25
> or 30fps with a keyframe interval of 50 or 60 frames maximum. Also best to
> use standard bitrates, especially with audio, to avoid drifting.
>
> Erik
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
>
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Audio/video stream are out of sync?

Carl Eugen Hoyos-2
In reply to this post by christina zou
2017-02-09 7:02 GMT+01:00 christina zou <[hidden email]>:

> Now, with the below command I've managed 3 minutes of fully synced
> audio/video livestream to Youtube from my Pi Zero. At the 3 minute mark,
> there's a slight desync, and the audio goes about 0.5s ahead of the video.

That's what I would have expected.

> That's my next question. I don't mind if there's a stutter every 3 minutes,
> but I need the A/V sync to recover after that.

How would that (for a theoretical device) be possible?
You do not provide input timestamps, so A/V sync is not possible.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Audio/video stream are out of sync?

Carl Eugen Hoyos-2
In reply to this post by christina zou
2017-02-09 0:50 GMT+01:00 christina zou <[hidden email]>:
> I spent a lot of time trying to use ffmpeg alone (alsa/v4l2), but it
> produces extreme stuttering on the audio track.

Command line and complete, uncut console output (here on the
mailing list!) missing.

Please do not top-post here, Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Loading...