Audio-Video Synchronization and Fastest Encoding

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Audio-Video Synchronization and Fastest Encoding

Stephan Monecke
Hi together,


I have a weak computer (i3 something) connected to a HDMI-grabber
(Magewell USB Capture HDMI PLUS, acts like a webcam) and audio via
line over the microphone port.

I want to merge those two streams together with as little audio-video
offset as possible and as light as possible on the CPU BUT I need
multiple other programs to be able to read the HDMI-grabber
simultaneously (I hence somehow need to mirror /dev/video0).

For the latter, I currently use an instance of the nginx-rtmp plugin
as a local video relay -- I might as well have a look at the
v4l2loopback module but I don't know about the performance or
side-effects so far.

I now have the following questions:

  1. What is the best way to automatically produce synchronized audio
and video? Is there some resource someone can point me at? I currently
use `-itsoffset` with empirical value for the video and HOPE it to be
constant. Post-production would not be a problem as long its
automatable and command line.

  2. How can I send the video to the local nginx as light as possible
on the CPU?

When using the GPU via

ffmpeg
    -vaapi_device /dev/dri/renderD128
    -r 60
    -i /dev/video0
    -vf 'format=nv12,hwupload'
    -c:v h264_vaapi
    -r 25
    -profile high
    -threads 1
    -an
    -f flv rtmp://localhost/live

I have around 70-80 % CPU usage on the respective core but the
audio-video offset seems to vary about a second between multiple
recordings. When I use

ffmpeg
    -r 60
    -i /dev/video0
    -preset ultrafast
    -c:v libx264
    -r 25
    -threads 1
    -an
    -f flv rtmp://localhost/live

I have dangerous 100 % CPU utilization on the respective core but did
not notice the offset variation so far. Is there something lighter?

  3. Is there a completely different more sane approach?


Thanks a lot for any help!

Stephan
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Audio-Video Synchronization and Fastest Encoding

Stephan Monecke
Hi,

in case I've been unclear, I would like to clarify two things.
Question 2 should be

> 2. "How can I send a rawvideo to a rtmp-server as light as possible
on the CPU?"

as well as add, that the structure of the current setup are two ffmpeg
instances (question 2 is regarding the first) as follows:

1. ffmpeg [...] -r 60 -i /dev/video0 [...] -r 25 -an -f flv
rtmp://localhost/live
2. ffmpeg \
    -use_wallclock_as_timestamps 1 \
    -fflags +genpts \
    -max_delay 2000000 \
    -thread_queue_size 1024 \
    -i rtmp://localhost/live \
    -max_delay 2000000 \
    -thread_queue_size 1024 \
    -itsoffset 4.8 \
    -f pulse \
    -i "alsa_input.pci-0000_00_1f.3.analog-stereo" \
    -af "aresample=async=1" \
    -codec:a aac \
    -b:a 384k \
    -ar 48000 \
    -vcodec copy \
    -tune zerolatency \
    -map 0:v -map 1:a \
    -max_muxing_queue_size 99999 \
    -f flv
    outfile.flv

Thanks a lot!

Am Do., 4. Juni 2020 um 17:11 Uhr schrieb Stephan Monecke
<[hidden email]>:

>
> Hi together,
>
>
> I have a weak computer (i3 something) connected to a HDMI-grabber
> (Magewell USB Capture HDMI PLUS, acts like a webcam) and audio via
> line over the microphone port.
>
> I want to merge those two streams together with as little audio-video
> offset as possible and as light as possible on the CPU BUT I need
> multiple other programs to be able to read the HDMI-grabber
> simultaneously (I hence somehow need to mirror /dev/video0).
>
> For the latter, I currently use an instance of the nginx-rtmp plugin
> as a local video relay -- I might as well have a look at the
> v4l2loopback module but I don't know about the performance or
> side-effects so far.
>
> I now have the following questions:
>
>   1. What is the best way to automatically produce synchronized audio
> and video? Is there some resource someone can point me at? I currently
> use `-itsoffset` with empirical value for the video and HOPE it to be
> constant. Post-production would not be a problem as long its
> automatable and command line.
>
>   2. How can I send the video to the local nginx as light as possible
> on the CPU?
>
> When using the GPU via
>
> ffmpeg
>     -vaapi_device /dev/dri/renderD128
>     -r 60
>     -i /dev/video0
>     -vf 'format=nv12,hwupload'
>     -c:v h264_vaapi
>     -r 25
>     -profile high
>     -threads 1
>     -an
>     -f flv rtmp://localhost/live
>
> I have around 70-80 % CPU usage on the respective core but the
> audio-video offset seems to vary about a second between multiple
> recordings. When I use
>
> ffmpeg
>     -r 60
>     -i /dev/video0
>     -preset ultrafast
>     -c:v libx264
>     -r 25
>     -threads 1
>     -an
>     -f flv rtmp://localhost/live
>
> I have dangerous 100 % CPU utilization on the respective core but did
> not notice the offset variation so far. Is there something lighter?
>
>   3. Is there a completely different more sane approach?
>
>
> Thanks a lot for any help!
>
> Stephan
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".