pipe from /dev/video to /dev/video adding a delay - syncing issues

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

pipe from /dev/video to /dev/video adding a delay - syncing issues

Javier Infante Porro
Hello,

I am trying to create a video device using another video device, adding a 20 seconds extra delay.
Also, i will create on the fly a custom countdown video before my delayed stream starts.











I create my virtual video device using v4l2loopback:

modprobe v4l2loopback exclusive_caps=1 video_nr=20 card_label="LIVE"







The command writting to file works pretty good (an initial 20 seconds countdown with bg.jpg background):



ffmpeg -loglevel 43 -y -loop 1 -r 28 -t 20 -i bg.jpg -f v4l2 -thread_queue_size 512 -i /dev/video0 -filter_complex "[0:v:0]setpts=N/(28*TB),drawtext=fontfile='/usr/share/fonts/truetype/msttcorefonts/Georgia.ttf':fontcolor=yellow:fontsize=40:x=(w-text_w)/2:y=(h-text_h)/2:text='%{eif\:20-t\:d}'[count]; [count] [1:v:0] concat=n=2:v=1:a=0:unsafe=1 [v];[v]scale=852:480[out]" -map "[out]" -an -pix_fmt yuv420p -c:v libx264 -preset ultrafast s1.mp4


But whenever i try to push the stream back to the the /dev/video20 device, it works, but the countdown goes real fast (like 10x faster), and then video tries to catch up and keeps around a 2 second delay:





ffmpeg -loglevel 43 -y -loop 1 -r 28 -t 20 -i bg.jpg -f v4l2 -thread_queue_size 512 -i /dev/video0 -filter_complex "[0:v:0]setpts=N/(28*TB),drawtext=fontfile='/usr/share/fonts/truetype/msttcorefonts/Georgia.ttf':fontcolor=yellow:fontsize=40:x=(w-text_w)/2:y=(h-text_h)/2:text='%{eif\:20-t\:d}'[count]; [count] [1:v:0] concat=n=2:v=1:a=0:unsafe=1 [v];[v]scale=852:480[out]" -map "[out]" -an -pix_fmt yuv420p -f v4l2 -r 28 /dev/video20


Any ideas on how to output as v4l2 correctly or achiving what i am trying using another approach?

Thanks in advance!


--
Jabi
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: pipe from /dev/video to /dev/video adding a delay - syncing issues

Moritz Barsnick
On Wed, Jun 26, 2019 at 21:17:11 +0200, Javier Infante Porro wrote:
> I am trying to create a video device using another video device, adding a 20 seconds extra delay.
> Also, i will create on the fly a custom countdown video before my delayed stream starts.
[...]
> But whenever i try to push the stream back to the the /dev/video20
> device, it works, but the countdown goes real fast (like 10x faster),
> and then video tries to catch up and keeps around a 2 second delay:
>
> ffmpeg -loglevel 43 -y -loop 1 -r 28 -t 20 -i bg.jpg -f v4l2 -thread_queue_size 512 -i /dev/video0 -filter_complex "[0:v:0]setpts=N/(28*TB),drawtext=fontfile='/usr/share/fonts/truetype/msttcorefonts/Georgia.ttf':fontcolor=yellow:fontsize=40:x=(w-text_w)/2:y=(h-text_h)/2:text='%{eif\:20-t\:d}'[count]; [count] [1:v:0] concat=n=2:v=1:a=0:unsafe=1 [v];[v]scale=852:480[out]" -map "[out]" -an -pix_fmt yuv420p -f v4l2 -r 28 /dev/video20

The input image is processed "as fast as possible" by ffmpeg, while the
v4l2 input stream is "realtime" by nature.

It should suffice to add "-re" as an input option to the first input:

$ ffmpeg -loglevel 43 -y -re -loop 1 -r 28 -t 20 -i bg.jpg -f v4l2 [...]
                         ^^^

Hope this helps,
Moritz
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: pipe from /dev/video to /dev/video adding a delay - syncing issues

Nicolas George
In reply to this post by Javier Infante Porro
Javier Infante Porro (12019-06-26):
> But whenever i try to push the stream back to the the /dev/video20
> device, it works, but the countdown goes real fast (like 10x faster),
> and then video tries to catch up and keeps around a 2 second delay:

For a regular countdown, you need an accurate timestamp. You are killing
your timestamps with the setpts filter: why do you do it?

Regards,

--
  Nicolas George

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: pipe from /dev/video to /dev/video adding a delay - syncing issues

Javier Infante Porro
In reply to this post by Moritz Barsnick
----- Mensaje original -----

> De: "Moritz Barsnick" <[hidden email]>
> Para: "FFmpeg user discussions" <[hidden email]>
> Enviados: Jueves, 27 de Junio 2019 9:27:04
> Asunto: Re: [FFmpeg-user] pipe from /dev/video to /dev/video adding a delay - syncing issues
>
> On Wed, Jun 26, 2019 at 21:17:11 +0200, Javier Infante Porro wrote:
> > I am trying to create a video device using another video device, adding a
> > 20 seconds extra delay.
> > Also, i will create on the fly a custom countdown video before my delayed
> > stream starts.
> [...]
> > But whenever i try to push the stream back to the the /dev/video20
> > device, it works, but the countdown goes real fast (like 10x faster),
> > and then video tries to catch up and keeps around a 2 second delay:
> >
> > ffmpeg -loglevel 43 -y -loop 1 -r 28 -t 20 -i bg.jpg -f v4l2
> > -thread_queue_size 512 -i /dev/video0 -filter_complex
> > "[0:v:0]setpts=N/(28*TB),drawtext=fontfile='/usr/share/fonts/truetype/msttcorefonts/Georgia.ttf':fontcolor=yellow:fontsize=40:x=(w-text_w)/2:y=(h-text_h)/2:text='%{eif\:20-t\:d}'[count];
> > [count] [1:v:0] concat=n=2:v=1:a=0:unsafe=1 [v];[v]scale=852:480[out]"
> > -map "[out]" -an -pix_fmt yuv420p -f v4l2 -r 28 /dev/video20
>
> The input image is processed "as fast as possible" by ffmpeg, while the
> v4l2 input stream is "realtime" by nature.
>
> It should suffice to add "-re" as an input option to the first input:
>
> $ ffmpeg -loglevel 43 -y -re -loop 1 -r 28 -t 20 -i bg.jpg -f v4l2 [...]
>

Yes! It works as you are describing with the countdown.
Thank you very much Moritz!

So I have a countdown, but when it's done, the video source tries to catch up  with "reality" really fast...
I have been trying to play with the setpts filter, but still no luck...


TIME=5
ffmpeg -y -re \
  -loop 1 -r 28 -t $TIME -i bg.jpg \
  -f v4l2 -thread_queue_size 512 -framerate 28 -i /dev/video30 \
  -filter_complex \
   "[0:v:0]drawtext=fontfile='/usr/share/fonts/truetype/msttcorefonts/Georgia.ttf':fontcolor=yellow:fontsize=40:x=(w-text_w)/2:y=(h-text_h)/2:text=\'%{eif\:$TIME+1-t\:d}\'[count]; \
   [1:v:0]fps=28,setpts=PTS+$TIME/TB[live];[count] [live] concat=n=2:v=1:a=0:unsafe=1 [v];[v]scale=852:480[out]" \
  -map "[out]" -an -pix_fmt yuv420p -f v4l2 -r 28 /dev/video20


Any ideas?



_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: pipe from /dev/video to /dev/video adding a delay - syncing issues

Javier Infante Porro
In reply to this post by Nicolas George


----- Mensaje original -----

> De: "Nicolas George" <[hidden email]>
> Para: "FFmpeg user questions" <[hidden email]>
> Enviados: Jueves, 27 de Junio 2019 10:35:04
> Asunto: Re: [FFmpeg-user] pipe from /dev/video to /dev/video adding a delay - syncing issues
>
> Javier Infante Porro (12019-06-26):
> > But whenever i try to push the stream back to the the /dev/video20
> > device, it works, but the countdown goes real fast (like 10x faster),
> > and then video tries to catch up and keeps around a 2 second delay:
>
> For a regular countdown, you need an accurate timestamp. You are killing
> your timestamps with the setpts filter: why do you do it?

I am not using it anymore... it's something i forgot inside the filter chain...

Right now i am using the -re switch (as Moritz suggest me in this thread), but whenever the countdown finishes, thy video from /dev/video0 tries to "recover" the lost time...

I am guessing it has sometyhing to do with the "-ts  parameter on the v4l2 input or maybe using the pts filter... but i am out of luck so far.


TIME=5
ffmpeg -y -re \
  -loop 1 -r 28 -t $TIME -i bg.jpg \
  -f v4l2 -thread_queue_size 512 -framerate 28 -i /dev/video30 \
  -filter_complex \
   "[0:v:0]drawtext=fontfile='/usr/share/fonts/truetype/msttcorefonts/Georgia.ttf':fontcolor=yellow:fontsize=40=(w-text_w)/2:y=(h-text_h)/2:text=\'%{eif\:$TIME+1-t\}\'[count]; \
   [1:v:0]fps=28,setpts=PTS+$TIME/TB[live];[count] [live] concat=n=2:v=1:a=0:unsafe=1 [v];[v]scale=852:480[out]" \
  -map "[out]" -an -pix_fmt yuv420p -f v4l2 -r 28 /dev/video20


Any ideas on what am I missing?



--
Jabi
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: pipe from /dev/video to /dev/video adding a delay - syncing issues

Nicolas George
Javier Infante Porro (12019-06-27):
> > For a regular countdown, you need an accurate timestamp. You are killing
> > your timestamps with the setpts filter: why do you do it?
> I am not using it anymore... it's something i forgot inside the filter chain...

That does not answer the question about why you put it in the first
place.

> Right now i am using the -re switch (as Moritz suggest me in this thread),

That was a mistake. Devices already are timed.

> I am guessing it has sometyhing to do with the "-ts  parameter on the v4l2 input

You did not set the -ts option on the v4l input. If you want efficient
help here, you need to be very careful when showing your command lines.

> or maybe using the pts filter... but i am out of luck so far.

As I told you, no setpts filter, you are killing the timestamps you
need.

Regards,

--
  Nicolas George

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: pipe from /dev/video to /dev/video adding a delay - syncing issues

Javier Infante Porro


----- Mensaje original -----

> De: "Nicolas George" <[hidden email]>
> Para: "FFmpeg user questions" <[hidden email]>
> Enviados: Jueves, 27 de Junio 2019 11:23:10
> Asunto: Re: [FFmpeg-user] pipe from /dev/video to /dev/video adding a delay - syncing issues
>
> Javier Infante Porro (12019-06-27):
> > > For a regular countdown, you need an accurate timestamp. You are killing
> > > your timestamps with the setpts filter: why do you do it?
> > I am not using it anymore... it's something i forgot inside the filter
> > chain...
>
> That does not answer the question about why you put it in the first
> place.

Yes sorry, I put it trying to make it work before I realize the "-re" option.
I was using setpts=N/(28*TB) as a filter on the countdown input, trying to "force" de video to use 28 fps. (huge mistake i guess)



> > Right now i am using the -re switch (as Moritz suggest me in this thread),
>
> That was a mistake. Devices already are timed.

 

> > I am guessing it has sometyhing to do with the "-ts  parameter on the v4l2
> > input
>
> You did not set the -ts option on the v4l input. If you want efficient
> help here, you need to be very careful when showing your command lines.
>
> > or maybe using the pts filter... but i am out of luck so far.
>
> As I told you, no setpts filter, you are killing the timestamps you
> need.

Ok, thank you for your advice.

I have seen in the log v4l2 detects monotonic timestamps:
[video4linux2,v4l2 @ 0x56150895f400] Detected monotonic timestamps, converting

So i have tried:

TIME=5
ffmpeg -loglevel 48 -y -re -loop 1 -r 28 -t $TIME -i bg.jpg -f v4l2 -thread_queue_size 512 -ts mono2abs -framerate 28 -i /dev/video30 -filter_complex \
"[0:v:0]drawtext=fontfile='/usr/share/fonts/truetype/msttcorefonts/Georgia.ttf':fontcolor=yellow:fontsize=40:x=(w-text_w)/2:y=(h-text_h)/2:text=\'%{eif\:$TIME+1-t\:d}\'[count]; \
   [1:v:0]fps=28[live];[count] [live] concat=n=2:v=1:a=0:unsafe=1 [v];[v]scale=852:480[out]"   -map "[out]" -an -pix_fmt yuv420p -f v4l2 -r 28 /dev/video20


But the result its the same with and without "-ts": The nice 5 seconds timeout, a few real-fast frames, and the a ~2 seconds delayed-time video (when it should be 5 seconds delayed).


And again, using the fps filter on the 1:v:0 input is useless, as i get the same output:

TIME=5
ffmpeg -loglevel 48 -y -re -loop 1 -r 28 -t $TIME -i bg.jpg -f v4l2 -thread_queue_size 512 -ts mono2abs -framerate 28 -i /dev/video30 -filter_complex \
"[0:v:0]drawtext=fontfile='/usr/share/fonts/truetype/msttcorefonts/Georgia.ttf':fontcolor=yellow:fontsize=40:x=(w-text_w)/2:y=(h-text_h)/2:text=\'%{eif\:$TIME+1-t\:d}\'[count]; \
   [count] [1:v:0] concat=n=2:v=1:a=0:unsafe=1 [v];[v]scale=852:480[out]" -map "[out]" -an -pix_fmt yuv420p -f v4l2 -r 28 /dev/video20


Anyway, thanks for your time and will keep investigating (and keeping away from setpts filter)
All thought i am running out of ideas...

Greets!


_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".