Problem with combining a video and an audio stream from USB device

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem with combining a video and an audio stream from USB device

FHoevi
Dear all,

I hope you could provide a hint to my problem: I have two USB devices
attached to an RPi, both show up as usual as /dev/video0. Here's some
additional info coming from two command line inputs:

 

Device 1, video only (attached to an RPi4):

"ffmpeg -f v4l2 -list_formats all -i /dev/video0" reports

[video4linux2,v4l2 @ 0xe5e1c0] Compressed:       mjpeg :
Motion-JPEG : 1280x720 640x480 320x240

 

"v4l2-ctl --list-formats-ext" reports

ioctl: VIDIOC_ENUM_FMT

        Type: Video Capture

 

        [0]: 'MJPG' (Motion-JPEG, compressed)

                Size: Discrete 1280x720

                        Interval: Stepwise 0.033s - 0.033s with step 0.000s
(30.000-30.000 fps)

                Size: Discrete 640x480

                        Interval: Stepwise 0.033s - 0.033s with step 0.000s
(30.000-30.000 fps)

                Size: Discrete 320x240

                        Interval: Stepwise 0.033s - 0.033s with step 0.000s
(30.000-30.000 fps)

 

Does work: ffmpeg -f v4l2 -i /dev/video0 -vcodec h264_omx -preset ultrafast
-tune zerolatency -g 300 -b:v 1M -mpegts_service_type
advanced_codec_digital_hdtv -f mpegts udp://OtherMachine:Port?pkt_size=1316

 

Device 2, video and audio (attached to an RPi3, but does not work either on
the RPi4):

"ffmpeg -f v4l2 -list_formats all -i /dev/video0" reports

[video4linux2,v4l2 @ 0x2c41210] Compressed:       mjpeg :
Motion-JPEG : 1920x1080 1280x720

 

"v4l2-ctl --list-formats-ext" reports

ioctl: VIDIOC_ENUM_FMT

               Index       : 0

               Type        : Video Capture

               Pixel Format: 'MJPG' (compressed)

               Name        : Motion-JPEG

                              Size: Discrete 1920x1080

                                            Interval: Discrete 0.033s
(30.000 fps)

                                            Interval: Discrete 0.067s
(15.000 fps)

                              Size: Discrete 1280x720

                                            Interval: Discrete 0.033s
(30.000 fps)

                                            Interval: Discrete 0.067s
(15.000 fps)

 

After quite some tedious work and way too many hours I got this running:

 

Video only: ffmpeg -f v4l2 -input_format mjpeg -i /dev/video0 -c:v copy
-preset ultrafast -tune zerolatency -g 300 -f matroska
udp://OtherMachine:Port?pkt_size=1316

Does not work at all: ffmpeg -f v4l2 -input_format mjpeg -i /dev/video0 -c:v
copy -preset ultrafast -tune zerolatency -g 300 -f mpegts
udp://OtherMachine:Port?pkt_size=1316, on "OtherMachine" I do see that there
is an incoming data stream via VLC, but it could not be digested properly.

Audio only: ffmpeg -f alsa -thread_queue_size 1024 -i plughw:1 -c:a mp2 -ac
2 -ar 44100 -preset ultrafast -tune zerolatency -b:a 128K -f mpegts
udp://OtherMachine:Port?pkt_size=1316

 

But this does not work either:

ffmpeg -f v4l2 -input_format mjpeg -i /dev/video0 -f alsa -thread_queue_size
1024 -i plughw:1 -c:v copy -c:a mp2 -ac 2 -ar 44100 -preset ultrafast -tune
zerolatency -g 300  -b:a 128K -f mpegts
udp://OtherMachine:Port?pkt_size=1316

 

Could you please provide a hint on how to get these two streams for device 2
working together? Both of them come from the same hardware/device, my guess
is that the MJPG video stream is somehow not fully compliant with the mpegts
standard (like it is for device 1) since it works with matroska, but not
with mpegts. Could that be? What needs to be done in that case?

 

Thanks in advance and kind regards

FHoevi

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Problem with combining a video and an audio stream from USB device

Carl Eugen Hoyos-2


> Am 01.02.2020 um 17:18 schrieb FHoevi <[hidden email]>:
>
> since it works with matroska, but not
> with mpegts. Could that be? What needs to be done in that case?

You cannot put mjpeg into mpegts, this is not a limitation of FFmpeg.

For future question: Please understand that we require all users to test current FFmpeg git head and provide the command tested together with the complete, uncut console output because it makes understanding issues like yours infinitely easier.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Problem with combining a video and an audio stream from USB device

FHoevi
Dear Carl Eugen,
thanks for your immediate reply. I got Raspbian Buster freshly installed with ffmpeg included. Building all this from Git head code base would potentially be one of the next steps but only if anything else won't work. All streams below do show no errors at all, therefore I have decided to not disclose them, mea culpa. And, sorry, I'm certainly not a video expert like you obviously are and thus haven't been aware of the fact that mjpeg and mpegts may not work with each other by design. I simply would like to use what I have in the most efficient way and just tried to get potential aid by ffmpeg experts.

Perhaps let me ask this question: Given that I have an incoming mjpeg video stream as well as an incoming mp2 audio stream which kind of container format (ok, it's obviously not mpegts) is the most appropriate one for combined streaming across my LAN or even into a pipe for further processing? Believe me, I tried my very best over a couple of hours to find out how to proceed but with no success. At least to my humble knowledge there is nothing such like a table providing answers to questions of that kind.

Perhaps another hint, with the same kind of hardware setup I can do this

cvlc -vvv v4l2:///dev/video0 --input-slave=alsa://plughw:1,0 --sout='#transcode{acodec=mpga,ab=128}:std{access=http,mux=asf,dst=:Port}'

So, here my understanding is that video gets passed on unchanged (mjpeg) and audio gets transcoded via vlc's mpga which presumably corresponds to mp2 for ffmpeg. The container format is asf, but I was not able to get that running with ffmpeg for no obvious reason. Anyway, picking up this vlc broadcast stream via <a href="http://StreamingMachine:Port">http://StreamingMachine:Port on any other machine in my network is working well. But how to achieve that with ffmpeg directly and potentially not as http:// but udp:// or pipe stream?

Thx in advance
FHoevi


_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".