FFmpeg: how to output over HTTP

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

FFmpeg: how to output over HTTP

Glenn W
I am trying to send FFmpeg output over HTTP stream to a web server backend.

The FFmpeg client is sitting behind a NAT so I need to stream outbound to
server, rather than have server request stream.

So far, while testing on my local network I am using these commands, which
works, but uses TCP rather than HTTP. I need to use HTTP only.

From Client:

`ffmpeg -re -i hq-video.mp4 -c:v libx264 -an -bsf h264_mp4toannexb -f
mpegts -chunked_post 1 http://0.0.0.0:5558/mpegts`

Using Wireshark, I see that the output is being sent still over TCP, rather
than HTTP.

Please help me understand what I can do to force HTTP output. The desired
behavior is to send HTTP output direct to server (without server first
sending out request to client) using PUT or POST headers.

Thank You,

Glenn W
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Moritz Barsnick
On Thu, Mar 14, 2019 at 17:22:54 -0400, Glenn W wrote:
> So far, while testing on my local network I am using these commands, which
> works, but uses TCP rather than HTTP. I need to use HTTP only.

That's a bit of an ambiguous statement. TCP is layer 3, HTTP is layer
7, and uses TCP underneath.

> `ffmpeg -re -i hq-video.mp4 -c:v libx264 -an -bsf h264_mp4toannexb -f
> mpegts -chunked_post 1 http://0.0.0.0:5558/mpegts`

Please always show us your complete, uncut console output together with
the command line.

I didn't know you could connect to 0.0.0.0- :-)

> Using Wireshark, I see that the output is being sent still over TCP, rather
> than HTTP.

I tried this:
$ ffmpeg -listen 1 -i http://localhost:8888/video -f null -
$ ffmpeg -f lavfi -i testsrc -t 1.55 -f mpegts http://0.0.0.0:8888/video

and my Wireshark tell me it's using HTTP:
> POST /video HTTP/1.1
> Transfer-Encoding: chunked
> User-Agent: Lavf/58.26.101
> Accept: */*
> Connection: close
> Host: 0.0.0.0:8888
> Icy-MetaData: 1
>
< HTTP/1.1 200 OK
< Content-Type: application/octet-stream
< Transfer-Encoding: chunked
<
> [... bin data follows]

Works perfectly for me.

Moritz
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Glenn W
Thanks Moritz,

I tried to replicate your command, however, I am still only seeing TCP
packets (rather than HTTP packets) in my Wireshark output.

To be thorough, I have included my FFmpeg version commands for comparison.

The full commands I am running can be seen below :

From Client:

`ffmpeg -re -i hq-video.mp4 -c:v libx264 -an -f mpegts
http://0.0.0.0:5558/video`

ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.1 --enable-shared
--enable-pthreads --enable-version3 --enable-hardcoded-tables
--enable-avresample --cc=clang
--host-cflags='-I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include
-I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include/darwin'
--host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl
--enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus
--enable-librubberband --enable-libsnappy --enable-libtesseract
--enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264
--enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig
--enable-libfreetype --enable-frei0r --enable-libass
--enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg
--enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack
--disable-indev=jack --enable-libaom --enable-libsoxr
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100


From Server:

`ffmpeg -listen 1 -i http://0.0.0.0:5558/video -f null - `

ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 6.4.0 (Alpine 6.4.0)
  configuration: --disable-debug --disable-doc --disable-ffplay
--enable-ffmpeg --enable-protocol=rtp --enable-protocol=udp
--enable-protocol=file --enable-protocol=crypto --enable-protocol=data
--enable-encoder=mp4 --enable-encoder=rtp --enable-decoder=rtp
--enable-encoder='rawvideo,libx264' --enable-decoder=h264
--enable-encoder=h264 --enable-muxer=segment
--enable-muxer='stream_segment,ssegment' --enable-muxer='rawvideo,mp4'
--enable-muxer=rtsp --enable-muxer=h264 --enable-demuxer=rawvideo
--enable-demuxer=mov --enable-demuxer=h264 --enable-demuxer=rtsp
--enable-parser=h264 --enable-parser=mpeg4 --enable-avcodec
--enable-avformat --enable-avfilter --enable-gpl --enable-small
--enable-libx264 --enable-nonfree --enable-openssl
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100


Server is running inside alpine docker container. I have also tried running
outside docker container on my local machine and still do not see any HTTP
traffic.


On Fri, Mar 15, 2019 at 7:27 AM Moritz Barsnick <[hidden email]> wrote:

> On Thu, Mar 14, 2019 at 17:22:54 -0400, Glenn W wrote:
> > So far, while testing on my local network I am using these commands,
> which
> > works, but uses TCP rather than HTTP. I need to use HTTP only.
>
> That's a bit of an ambiguous statement. TCP is layer 3, HTTP is layer
> 7, and uses TCP underneath.
>
> > `ffmpeg -re -i hq-video.mp4 -c:v libx264 -an -bsf h264_mp4toannexb -f
> > mpegts -chunked_post 1 http://0.0.0.0:5558/mpegts`
> <http://0.0.0.0:5558/mpegts>
>
> Please always show us your complete, uncut console output together with
> the command line.
>
> I didn't know you could connect to 0.0.0.0- :-)
>
> > Using Wireshark, I see that the output is being sent still over TCP,
> rather
> > than HTTP.
>
> I tried this:
> $ ffmpeg -listen 1 -i http://localhost:8888/video -f null -
> $ ffmpeg -f lavfi -i testsrc -t 1.55 -f mpegts http://0.0.0.0:8888/video
>
> and my Wireshark tell me it's using HTTP:
> > POST /video HTTP/1.1
> > Transfer-Encoding: chunked
> > User-Agent: Lavf/58.26.101
> > Accept: */*
> > Connection: close
> > Host: 0.0.0.0:8888
> > Icy-MetaData: 1
> >
> < HTTP/1.1 200 OK
> < Content-Type: application/octet-stream
> < Transfer-Encoding: chunked
> <
> > [... bin data follows]
>
> Works perfectly for me.
>
> Moritz
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Glenn W
To further make clear,

I understand and acknowledge that HTTP uses TCP, of course. My desired
behavior is simply to encapsulate the TCP packets with HTTP formatting and
send with POST/PUT headers to server.  The HTTP encapsulation is essential
for my use case.

I am able to make connection just fine, and send video data. It is just
using TCP for transport rather than encapsulating with TCP with HTTP.

Is there some 3rd party library I might be missing here?


On Mon, Mar 18, 2019 at 2:21 PM wolfe.t.glenn <[hidden email]>
wrote:

> Thanks Moritz,
>
> I tried to replicate your command, however, I am still only seeing TCP
> packets (rather than HTTP packets) in my Wireshark output.
>
> To be thorough, I have included my FFmpeg version commands for comparison.
>
> The full commands I am running can be seen below :
>
> From Client:
>
> `ffmpeg -re -i hq-video.mp4 -c:v libx264 -an -f mpegts
> http://0.0.0.0:5558/video` <http://0.0.0.0:5558/video>
>
> ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
>   built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
>   configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.1 --enable-shared
> --enable-pthreads --enable-version3 --enable-hardcoded-tables
> --enable-avresample --cc=clang
> --host-cflags='-I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include
> -I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include/darwin'
> --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl
> --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus
> --enable-librubberband --enable-libsnappy --enable-libtesseract
> --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264
> --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig
> --enable-libfreetype --enable-frei0r --enable-libass
> --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg
> --enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack
> --disable-indev=jack --enable-libaom --enable-libsoxr
>   libavutil      56. 22.100 / 56. 22.100
>   libavcodec     58. 35.100 / 58. 35.100
>   libavformat    58. 20.100 / 58. 20.100
>   libavdevice    58.  5.100 / 58.  5.100
>   libavfilter     7. 40.101 /  7. 40.101
>   libavresample   4.  0.  0 /  4.  0.  0
>   libswscale      5.  3.100 /  5.  3.100
>   libswresample   3.  3.100 /  3.  3.100
>   libpostproc    55.  3.100 / 55.  3.100
>
>
> From Server:
>
> `ffmpeg -listen 1 -i http://0.0.0.0:5558/video -f null - `
>
> ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
>   built with gcc 6.4.0 (Alpine 6.4.0)
>   configuration: --disable-debug --disable-doc --disable-ffplay
> --enable-ffmpeg --enable-protocol=rtp --enable-protocol=udp
> --enable-protocol=file --enable-protocol=crypto --enable-protocol=data
> --enable-encoder=mp4 --enable-encoder=rtp --enable-decoder=rtp
> --enable-encoder='rawvideo,libx264' --enable-decoder=h264
> --enable-encoder=h264 --enable-muxer=segment
> --enable-muxer='stream_segment,ssegment' --enable-muxer='rawvideo,mp4'
> --enable-muxer=rtsp --enable-muxer=h264 --enable-demuxer=rawvideo
> --enable-demuxer=mov --enable-demuxer=h264 --enable-demuxer=rtsp
> --enable-parser=h264 --enable-parser=mpeg4 --enable-avcodec
> --enable-avformat --enable-avfilter --enable-gpl --enable-small
> --enable-libx264 --enable-nonfree --enable-openssl
>   libavutil      56. 22.100 / 56. 22.100
>   libavcodec     58. 35.100 / 58. 35.100
>   libavformat    58. 20.100 / 58. 20.100
>   libavdevice    58.  5.100 / 58.  5.100
>   libavfilter     7. 40.101 /  7. 40.101
>   libswscale      5.  3.100 /  5.  3.100
>   libswresample   3.  3.100 /  3.  3.100
>   libpostproc    55.  3.100 / 55.  3.100
>
>
> Server is running inside alpine docker container. I have also tried
> running outside docker container on my local machine and still do not see
> any HTTP traffic.
>
>
> On Fri, Mar 15, 2019 at 7:27 AM Moritz Barsnick <[hidden email]> wrote:
>
>> On Thu, Mar 14, 2019 at 17:22:54 -0400, Glenn W wrote:
>> > So far, while testing on my local network I am using these commands,
>> which
>> > works, but uses TCP rather than HTTP. I need to use HTTP only.
>>
>> That's a bit of an ambiguous statement. TCP is layer 3, HTTP is layer
>> 7, and uses TCP underneath.
>>
>> > `ffmpeg -re -i hq-video.mp4 -c:v libx264 -an -bsf h264_mp4toannexb -f
>> > mpegts -chunked_post 1 http://0.0.0.0:5558/mpegts`
>> <http://0.0.0.0:5558/mpegts>
>>
>> Please always show us your complete, uncut console output together with
>> the command line.
>>
>> I didn't know you could connect to 0.0.0.0- :-)
>>
>> > Using Wireshark, I see that the output is being sent still over TCP,
>> rather
>> > than HTTP.
>>
>> I tried this:
>> $ ffmpeg -listen 1 -i http://localhost:8888/video -f null -
>> $ ffmpeg -f lavfi -i testsrc -t 1.55 -f mpegts http://0.0.0.0:8888/video
>>
>> and my Wireshark tell me it's using HTTP:
>> > POST /video HTTP/1.1
>> > Transfer-Encoding: chunked
>> > User-Agent: Lavf/58.26.101
>> > Accept: */*
>> > Connection: close
>> > Host: 0.0.0.0:8888
>> > Icy-MetaData: 1
>> >
>> < HTTP/1.1 200 OK
>> < Content-Type: application/octet-stream
>> < Transfer-Encoding: chunked
>> <
>> > [... bin data follows]
>>
>> Works perfectly for me.
>>
>> Moritz
>> _______________________________________________
>> ffmpeg-user mailing list
>> [hidden email]
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>>
>> To unsubscribe, visit link above, or email
>> [hidden email] with subject "unsubscribe".
>
>
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Moritz Barsnick
Hi Glenn,

On Mon, Mar 18, 2019 at 14:27:43 -0400, wolfe.t.glenn wrote:
> Is there some 3rd party library I might be missing here?

I don't think so.

Can you please
- post the actual complete console output (not just the version info),
- send me a PCAP file as captured with Wireshark?

Thanks,
Moritz
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Glenn W
Thanks Moritz,

Please see the below full console output and attached PCAP file:

From Client (sending) side: https://ghostbin.com/paste/zs8yg

From Server (receiving) side: https://ghostbin.com/paste/xf27e

Best,
Glenn W

On Tue, Mar 19, 2019 at 8:27 AM Moritz Barsnick <[hidden email]> wrote:

> Hi Glenn,
>
> On Mon, Mar 18, 2019 at 14:27:43 -0400, wolfe.t.glenn wrote:
> > Is there some 3rd party library I might be missing here?
>
> I don't think so.
>
> Can you please
> - post the actual complete console output (not just the version info),
> - send me a PCAP file as captured with Wireshark?
>
> Thanks,
> Moritz
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".

ffmpeg-mpegts-http-test.pcapng (510K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Carl Eugen Hoyos-2
2019-03-19 18:46 GMT+01:00, Glenn W <[hidden email]>:

> Please see the below full console output

Unfortunately not;-(

Anyway, this was certainly transmitted via http.

The more important question is if there is an issue:
> Segmentation fault: 11

This normally indicates a (important) bug.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

sirk98 qwerty89
Hello friends,

I reach this talk because I try also to use FFMpeg as transcoder for live
streams over HTTP , but I can't find the right way .
 with this command ffmpeg is sending data to a server that is listening on
port and dir 5558/video, but I can't understand what kind of configuration
web server like nginx needs to have .
ffmpeg -re -i hq-video.mp4 -c:v libx264 -an -f mpegts
http://0.0.0.0:5558/video

nginx wait on port 5558 and dir video and redirect content to client in
some way . Someone can me explain if ffmpeg have some new muxer for this or
the right way to use for this things, I transcode a live stream in h265 +
ac3 codec over mpgets so rtmp module is not valid as solution, and I also
want to use a solution that not produce a list of file like HLS index .

any hints is really appreciate,
best regards


Il giorno mar 19 mar 2019 alle ore 22:49 Carl Eugen Hoyos <
[hidden email]> ha scritto:

> 2019-03-19 18:46 GMT+01:00, Glenn W <[hidden email]>:
>
> > Please see the below full console output
>
> Unfortunately not;-(
>
> Anyway, this was certainly transmitted via http.
>
> The more important question is if there is an issue:
> > Segmentation fault: 11
>
> This normally indicates a (important) bug.
>
> Carl Eugen
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Glenn W
Carl,

> Anyway, this was certainly transmitted via http.

What is your evidence for this claim? Where do you see any HTTP headers? I
do not see any in my Wireshark output nor do I see any indication of HTTP
protocol being used.

> The more important question is if there is an issue: Segmentation Fault 11

This is not an issue. I can segment the mpegts chunks just fine. This error
is coming due to the nature of how I am running my server side `-f null -`
to output null to stdin (not how I would normally) for the sole purpose of
testing network transport.




On Tue, Mar 19, 2019 at 6:26 PM sirk98 qwerty89 <[hidden email]>
wrote:

> Hello friends,
>
> I reach this talk because I try also to use FFMpeg as transcoder for live
> streams over HTTP , but I can't find the right way .
>  with this command ffmpeg is sending data to a server that is listening on
> port and dir 5558/video, but I can't understand what kind of configuration
> web server like nginx needs to have .
> ffmpeg -re -i hq-video.mp4 -c:v libx264 -an -f mpegts
> http://0.0.0.0:5558/video
>
> nginx wait on port 5558 and dir video and redirect content to client in
> some way . Someone can me explain if ffmpeg have some new muxer for this or
> the right way to use for this things, I transcode a live stream in h265 +
> ac3 codec over mpgets so rtmp module is not valid as solution, and I also
> want to use a solution that not produce a list of file like HLS index .
>
> any hints is really appreciate,
> best regards
>
>
> Il giorno mar 19 mar 2019 alle ore 22:49 Carl Eugen Hoyos <
> [hidden email]> ha scritto:
>
> > 2019-03-19 18:46 GMT+01:00, Glenn W <[hidden email]>:
> >
> > > Please see the below full console output
> >
> > Unfortunately not;-(
> >
> > Anyway, this was certainly transmitted via http.
> >
> > The more important question is if there is an issue:
> > > Segmentation fault: 11
> >
> > This normally indicates a (important) bug.
> >
> > Carl Eugen
> > _______________________________________________
> > ffmpeg-user mailing list
> > [hidden email]
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >
> > To unsubscribe, visit link above, or email
> > [hidden email] with subject "unsubscribe".
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Moritz Barsnick
In reply to this post by Carl Eugen Hoyos-2
On Tue, Mar 19, 2019 at 22:49:32 +0100, Carl Eugen Hoyos wrote:
> 2019-03-19 18:46 GMT+01:00, Glenn W <[hidden email]>:
>
> > Please see the below full console output
>
> Unfortunately not;-(

Too bad it'ss in an external link (which tends to disappear after
weeks).

> Anyway, this was certainly transmitted via http.

According to the ffmpeg logs - yes, obviously.
According to the PCAP: hmm.
Well, the command lines show port 5558:

> chill@life ~$ ffmpeg -listen 1 -i http://localhost:5558/video -f null -

but I don't see a single packet related to port 5558 in that PCAP file.
(Filter: tcp.port == 5888). I wonder what was captured - probably eth0
(or whatever the physical interface is called) and not lo (to which
localhost is bound). No wonder you, Glenn, don't see HTTP traffic if
you don't actually capture the ffmpeg traffic...

Moritz
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Glenn W
Thank you Moritz,

I did not know that I was running Wireshark on the wrong network interface
- that makes a lot more sense now.

Please see the new attached PCAP file with a sample of the correct
Wireshark output bound to localhost. (had to gzip this one since it was
pretty big) ;

I do see some VERY sporadic HTTP POST headers, I could only find 1 out of
over 6000 and it came at the very end (see Frame 6137 of the sample).
Further, it appears that Wireshark does not even recognize this POST as
HTTP protocol. I am wondering if this is the expected behavior? It seems
very strange to me.

BTW I could actually share a small sample of the actual output since it was
so big (ffmpeg user mail list would not allow) - but there were 5000+
packets before that looked similar with zero HTTP headers whatsoever.

Since my video server (receiver) sits behind an HTTP load balancer, it will
need to see an HTTP route in order receive the packets from the FFmpeg
client (sender).

If I can get an HTTP route through to the receiver, I am thinking I could
use the below to maintain a persisted connection so that rest of data
packets can just be sent over TCP:

*multiple_requests*
*Use persistent connections if set to 1, default is 0.*

Please advise. Really appreciate your help with debugging this issue.

Best,
Glenn W

On Tue, Mar 19, 2019 at 7:33 PM Moritz Barsnick <[hidden email]> wrote:

> On Tue, Mar 19, 2019 at 22:49:32 +0100, Carl Eugen Hoyos wrote:
> > 2019-03-19 18:46 GMT+01:00, Glenn W <[hidden email]>:
> >
> > > Please see the below full console output
> >
> > Unfortunately not;-(
>
> Too bad it'ss in an external link (which tends to disappear after
> weeks).
>
> > Anyway, this was certainly transmitted via http.
>
> According to the ffmpeg logs - yes, obviously.
> According to the PCAP: hmm.
> Well, the command lines show port 5558:
>
> > chill@life ~$ ffmpeg -listen 1 -i http://localhost:5558/video -f null -
>
> but I don't see a single packet related to port 5558 in that PCAP file.
> (Filter: tcp.port == 5888). I wonder what was captured - probably eth0
> (or whatever the physical interface is called) and not lo (to which
> localhost is bound). No wonder you, Glenn, don't see HTTP traffic if
> you don't actually capture the ffmpeg traffic...
>
> Moritz
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".

ffmpeg-abridged-packet-sampling.pcapng.gz (782K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Moritz Barsnick
On Tue, Mar 19, 2019 at 21:05:19 -0400, Glenn W wrote:
> Please see the new attached PCAP file with a sample of the correct
> Wireshark output bound to localhost. (had to gzip this one since it was
> pretty big)

Unlikle the relatively tiny logs, you could upload such binaries to a
file sharer of your choice. ;-) (They are probably not of much interest
to the world once this issue is solved.)

> I do see some VERY sporadic HTTP POST headers, I could only find 1 out of
> over 6000 and it came at the very end (see Frame 6137 of the sample).

I only see 139 packets in the attached trace.

> Further, it appears that Wireshark does not even recognize this POST as
> HTTP protocol. I am wondering if this is the expected behavior? It seems
> very strange to me.

The content looks somewhat like only MPEG-TS, from somewhere in the
middle of a stream. (I extracted the payload and tried to have ffmpeg
swallow it.) I would guess that this trace was started *after* the HTTP
connection was established.

> BTW I could actually share a small sample of the actual output since it was
> so big (ffmpeg user mail list would not allow) - but there were 5000+
> packets before that looked similar with zero HTTP headers whatsoever.

File share host. ;-)

> Since my video server (receiver) sits behind an HTTP load balancer, it will
> need to see an HTTP route in order receive the packets from the FFmpeg
> client (sender).

Does it also expect particular ports? It will need to be configured to
understand the same ports, right?

> If I can get an HTTP route through to the receiver, I am thinking I could
> use the below to maintain a persisted connection so that rest of data
> packets can just be sent over TCP:

I'm totally convinced that if you use ffmpeg the way you are doing, you
use HTTP only.

> *multiple_requests*
> *Use persistent connections if set to 1, default is 0.*

This is only a single connection stream, I doubt this applies (but I'm
not totally sure). It won't be a tunnel which you can reuse. But once
your load balancer accepts these connections, you can do it as often as
you like anyway.

Moritz
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

kumowoon1025
In reply to this post by Glenn W
> Wireshark does not even recognize this POST as
> HTTP protocol.

IIRC, if you do not use heuristic sub-dissectors first, you will have to add port 5558 to http protocol settings. Don't see any headers though… Maybe you can check the logs on the receiving end?
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Moritz Barsnick
On Wed, Mar 20, 2019 at 06:56:38 -0400, Ted Park wrote:
> IIRC, if you do not use heuristic sub-dissectors first, you will have
> to add port 5558 to http protocol settings. Don't see any headers
> though… Maybe you can check the logs on the receiving end?

In my trace which I had analyzed as a proof of concept, HTTP was
perfectly recognized. I had used port 8888, while my Wireshark's HTTP
protocol seems to be configured to
"80,3128,3132,5985,8080,8088,11371,1900,2869,2710". So Wireshark should
be smart enough...

If I take this PCAP, and add 5558 to Wireshark's config, there are no
headers either. They weren't even captured, as I assumed in my other
mail.

Moritz
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

kumowoon1025
> In my trace which I had analyzed as a proof of concept, HTTP was
> perfectly recognized. I had used port 8888, while my Wireshark's HTTP
> protocol seems to be configured to
> "80,3128,3132,5985,8080,8088,11371,1900,2869,2710". So Wireshark should
> be smart enough...

You must be right, the only thing I can think of is if the capture was started after ffmpeg was run and only the chunked content was captured that can’t be decoded without the headers.

>> The more important question is if there is an issue: Segmentation Fault 11
>
> This is not an issue. I can segment the mpegts chunks just fine. This error
> is coming due to the nature of how I am running my server side `-f null -`
> to output null to stdin (not how I would normally) for the sole purpose of
> testing network transport.

You misunderstand, the message isn’t regarding segmenting the stream, it might indicate an issue within ffmpeg itself
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

andrei ka
remarque: ppl who send ismv via http to unified streaming servers
(apache/nginx) just put http url as output, it works.

On Wed, Mar 20, 2019 at 1:20 PM Ted Park <[hidden email]> wrote:

> > In my trace which I had analyzed as a proof of concept, HTTP was
> > perfectly recognized. I had used port 8888, while my Wireshark's HTTP
> > protocol seems to be configured to
> > "80,3128,3132,5985,8080,8088,11371,1900,2869,2710". So Wireshark should
> > be smart enough...
>
> You must be right, the only thing I can think of is if the capture was
> started after ffmpeg was run and only the chunked content was captured that
> can’t be decoded without the headers.
>
> >> The more important question is if there is an issue: Segmentation Fault
> 11
> >
> > This is not an issue. I can segment the mpegts chunks just fine. This
> error
> > is coming due to the nature of how I am running my server side `-f null
> -`
> > to output null to stdin (not how I would normally) for the sole purpose
> of
> > testing network transport.
>
> You misunderstand, the message isn’t regarding segmenting the stream, it
> might indicate an issue within ffmpeg itself
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Glenn W
In reply to this post by kumowoon1025
Thanks Moritz,

> File share host. ;-)

So true. Please see below link for more full example log from my local
tests:

https://drive.google.com/file/d/1HMS64eBDpTRlFwVarz8G75IffHzgqvCV/view?usp=sharing

> I would guess that this trace was started *after* the HTTP
connection was established.

This time I definitely started the trace before I even started either
ffmpeg command in my terminal and still see similar behavior (many TCP
connections before any HTTP header). You can still see how the POSTS are
disparately spread throughout the packets. I'm confused about this
behavior.

> I'm totally convinced that if you use ffmpeg the way you are doing, you
use HTTP only.

So are you thinking all those TCP [PSH, ACK] packets are contained within a
given HTTP POST?

If you look at the first few packets, I see that FFmpeg first sends a [ SYN
], followed by a [ SYN, ACK ] response from server, followed by an [ ACK ]
from client. At this point, should it not initiate HTTP protocol with
header and endpoint? Instead it starts sending TCP [PSH, ACK]'s.

Perhaps I am confused how this should work.  What ends up happening if I
point this at my HTTP load balancer is the message is never passed to the
backend service properly. It never is able to establish a connection.

> Does it also expect particular ports? It will need to be configured to
understand the same ports, right?

I use an nginx http ingress to load balance http headers to respective
backend services listening on TCP ports on various pods. In this case, my
ingress would load balance HTTP <an-example-endpoint>/video to port 5558 on
whichever pod. That pod will be running the same listen command.

*UPDATE: *

After digging into this more, I think I found what was going wrong! Looking
at my nginx-controller logs, I see that I am getting an error because the
`client intended to send too large chunked body` :

```
2019/03/20 16:02:41 [warn] 324#324: *3468886 a client request body is
buffered to a temporary file /tmp/client-body/0000000009, client:
10.52.0.1, server: live.nycv.biz, request: "POST /push HTTP/1.1", host: "
live.nycv.biz"
2019/03/20 16:02:42 [error] 324#324: *3468886 client intended to send too
large chunked body: 1046500+32768 bytes, client: 10.52.0.1, server:
live.nycv.biz, request: "POST /push HTTP/1.1", host: "live.nycv.biz"
```

As before, when I was sending to server, after about a minute of sending
the video, I would see an error on the client side from FFmpeg:

```
av_interleaved_write_frame(): Broken pipeB time=00:00:35.10
bitrate=17993.7kbits/s speed=0.399x
Error writing trailer of http://live.nycv.biz/push: Broken pipe
```

Now, setting my nginx configuration to not check for the body size by
setting max to 0 in the ingress configuration template, I tried again:

(Side note: Is there any way to tell FFmpeg to set a maximum on the chunked
bodies before it sends the HTTP POST? )

*From Client (sending side) : *

chill@life ~$ ffmpeg -re -i hq-video.mp4 -c:v libx264 -an -f mpegts
http://live.nycv.biz/push
ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.1 --enable-shared
--enable-pthreads --enable-version3 --enable-hardcoded-tables
--enable-avresample --cc=clang
--host-cflags='-I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include
-I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include/darwin'
--host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl
--enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus
--enable-librubberband --enable-libsnappy --enable-libtesseract
--enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264
--enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig
--enable-libfreetype --enable-frei0r --enable-libass
--enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg
--enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack
--disable-indev=jack --enable-libaom --enable-libsoxr
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'hq-video.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2019-02-11T22:01:43.000000Z
    location        : +40.7298-073.9904/
    location-eng    : +40.7298-073.9904/
    com.android.version: 9
    com.android.capture.fps: 30.000000
  Duration: 00:01:42.85, start: 0.000000, bitrate: 42251 kb/s
    Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc,
bt470bg/bt470bg/smpte170m), 3840x2160, 42033 kb/s, SAR 1:1 DAR 16:9, 29.95
fps, 30 tbr, 90k tbn, 180k tbc (default)
    Metadata:
      creation_time   : 2019-02-11T22:01:43.000000Z
      handler_name    : VideoHandle
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz,
stereo, fltp, 156 kb/s (default)
    Metadata:
      creation_time   : 2019-02-11T22:01:43.000000Z
      handler_name    : SoundHandle
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x7fe0e4803e00] using SAR=1/1
[libx264 @ 0x7fe0e4803e00] using cpu capabilities: MMX2 SSE2Fast SSSE3
SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7fe0e4803e00] profile High, level 5.1
Output #0, mpegts, to 'http://live.nycv.biz/push':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    com.android.capture.fps: 30.000000
    location        : +40.7298-073.9904/
    location-eng    : +40.7298-073.9904/
    com.android.version: 9
    encoder         : Lavf58.20.100
    Stream #0:0(eng): Video: h264 (libx264), yuvj420p(pc), 3840x2160 [SAR
1:1 DAR 16:9], q=-1--1, 30 fps, 90k tbn, 30 tbc (default)
    Metadata:
      creation_time   : 2019-02-11T22:01:43.000000Z
      handler_name    : VideoHandle
      encoder         : Lavc58.35.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
frame= 3080 fps= 11 q=-1.0 Lsize=  254544kB time=00:01:42.76
bitrate=20290.8kbits/s speed=0.379x
video:235782kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: 7.957043%
[libx264 @ 0x7fe0e4803e00] frame I:13    Avg QP:20.17  size:930179
[libx264 @ 0x7fe0e4803e00] frame P:992   Avg QP:23.19  size:154103
[libx264 @ 0x7fe0e4803e00] frame B:2075  Avg QP:27.27  size: 36857
[libx264 @ 0x7fe0e4803e00] consecutive B-frames:  9.0%  2.2%  3.7% 85.1%
[libx264 @ 0x7fe0e4803e00] mb I  I16..4:  0.9% 86.0% 13.1%
[libx264 @ 0x7fe0e4803e00] mb P  I16..4:  0.1%  4.5%  0.3%  P16..4: 48.4%
9.9%  8.0%  0.0%  0.0%    skip:28.8%
[libx264 @ 0x7fe0e4803e00] mb B  I16..4:  0.0%  1.5%  0.1%  B16..8: 42.0%
1.4%  0.4%  direct: 1.0%  skip:53.7%  L0:42.0% L1:56.0% BI: 2.1%
[libx264 @ 0x7fe0e4803e00] 8x8 transform intra:92.0% inter:60.7%
[libx264 @ 0x7fe0e4803e00] coded y,uvDC,uvAC intra: 91.2% 56.8% 13.3%
inter: 14.0% 8.8% 0.2%
[libx264 @ 0x7fe0e4803e00] i16 v,h,dc,p: 17% 29% 23% 30%
[libx264 @ 0x7fe0e4803e00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 16% 26%  6%
7%  8%  8%  7%  9%
[libx264 @ 0x7fe0e4803e00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 22% 12%  7%
10%  8%  9%  7%  8%
[libx264 @ 0x7fe0e4803e00] i8c dc,h,v,p: 59% 21% 17%  3%
[libx264 @ 0x7fe0e4803e00] Weighted P-Frames: Y:0.7% UV:0.0%
[libx264 @ 0x7fe0e4803e00] ref P L0: 69.0% 13.9% 14.2%  2.8%  0.0%
[libx264 @ 0x7fe0e4803e00] ref B L0: 93.7%  5.6%  0.8%
[libx264 @ 0x7fe0e4803e00] ref B L1: 94.1%  5.9%
[libx264 @ 0x7fe0e4803e00] kb/s:18777.01

Looks like everything went through alright from client perspective, *no
more `broken pipe error`*.

*However, *

From Server Side (receiving):

I am still not getting any data through:

Inside my kubernetes pod:

```
/ # . scripts/ffmpeg-listen.sh
current time @ 1553099678
ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 6.4.0 (Alpine 6.4.0)
  configuration: --disable-debug --disable-doc --disable-ffplay
--enable-ffmpeg --enable-protocol=rtp --enable-protocol=udp
--enable-protocol=file --enable-protocol=crypto --enable-protocol=data
--enable-encoder=mp4 --enable-encoder=rtp --enable-decoder=rtp
--enable-encoder='rawvideo,libx264' --enable-decoder=h264
--enable-encoder=h264 --enable-muxer=segment
--enable-muxer='stream_segment,ssegment' --enable-muxer='rawvideo,mp4'
--enable-muxer=rtsp --enable-muxer=h264 --enable-demuxer=rawvideo
--enable-demuxer=mov --enable-demuxer=h264 --enable-demuxer=rtsp
--enable-parser=h264 --enable-parser=mpeg4 --enable-avcodec
--enable-avformat --enable-avfilter --enable-gpl --enable-small
--enable-libx264 --enable-nonfree --enable-openssl
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging
level) with argument 'debug'.
Reading option '-listen' ... matched as AVOption 'listen' with argument '1'.
Reading option '-i' ... matched as input url with argument '
<a href="http://0.0.0.0:5558'">http://0.0.0.0:5558'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument
'h264'.
Reading option '-r' ... matched as option 'r' (set frame rate (Hz value,
fraction or abbreviation)) with argument '30'.
Reading option '-flags' ... matched as AVOption 'flags' with argument
'+cgop'.
Reading option '-g' ... matched as AVOption 'g' with argument '30'.
Reading option '-hls_segment_filename' ... matched as AVOption
'hls_segment_filename' with argument '/fuse/tmp/file%03d.ts'.
Reading option '-hls_time' ... matched as AVOption 'hls_time' with argument
'1'.
Reading option '/fuse/tmp/out.m3u8' ... matched as output url.
Reading option '-y' ... matched as option 'y' (overwrite output files) with
argument '1'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Applying option y (overwrite output files) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url http://0.0.0.0:5558.
Successfully parsed a group of options.
Opening an input file: http://0.0.0.0:5558.
[NULL @ 0x5610666ab240] Opening '<a href="http://0.0.0.0:5558'">http://0.0.0.0:5558' for reading
[http @ 0x5610666abc00] Setting default whitelist
'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
<a href="http://0.0.0.0:5558:">http://0.0.0.0:5558: End of file
```

With no indication that any data actually made it though to the pod,
however, I did notice that the EOF did come about the exact same time that
the client side process finished, which is quite peculiar.


And finally, here are the logs from the nginx-controller:

```
10.52.0.1 - [10.52.0.1] - - [20/Mar/2019:16:50:47 +0000] "POST /push
HTTP/1.1" 499 0 "-" "Lavf/58.20.100" 260727893 270.979
[default-live-rtsp-in-5558] 10.52.0.32:5558 0 0.001 -
8a27399690c628bd357ccf7216bf4aa6
```

Simply a POST with no error... this would indicate strangely that a single
chunked POST is coming with after all the packets have been sent, which is
a bit crazy. I would much rather it break it up into many chunked POSTs.


This is quite a mystery to me what is going wrong. I really appreciate your
help in getting to the bottom of this.

Best,
Glenn W

On Wed, Mar 20, 2019 at 8:20 AM Ted Park <[hidden email]> wrote:

> > In my trace which I had analyzed as a proof of concept, HTTP was
> > perfectly recognized. I had used port 8888, while my Wireshark's HTTP
> > protocol seems to be configured to
> > "80,3128,3132,5985,8080,8088,11371,1900,2869,2710". So Wireshark should
> > be smart enough...
>
> You must be right, the only thing I can think of is if the capture was
> started after ffmpeg was run and only the chunked content was captured that
> can’t be decoded without the headers.
>
> >> The more important question is if there is an issue: Segmentation Fault
> 11
> >
> > This is not an issue. I can segment the mpegts chunks just fine. This
> error
> > is coming due to the nature of how I am running my server side `-f null
> -`
> > to output null to stdin (not how I would normally) for the sole purpose
> of
> > testing network transport.
>
> You misunderstand, the message isn’t regarding segmenting the stream, it
> might indicate an issue within ffmpeg itself
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

andrei ka
btw, did you set client_max_body_size  in nginx.conf ? something like
1000000M :-)

On Wed, Mar 20, 2019 at 6:03 PM Glenn W <[hidden email]> wrote:

> Thanks Moritz,
>
> > File share host. ;-)
>
> So true. Please see below link for more full example log from my local
> tests:
>
>
> https://drive.google.com/file/d/1HMS64eBDpTRlFwVarz8G75IffHzgqvCV/view?usp=sharing
>
> > I would guess that this trace was started *after* the HTTP
> connection was established.
>
> This time I definitely started the trace before I even started either
> ffmpeg command in my terminal and still see similar behavior (many TCP
> connections before any HTTP header). You can still see how the POSTS are
> disparately spread throughout the packets. I'm confused about this
> behavior.
>
> > I'm totally convinced that if you use ffmpeg the way you are doing, you
> use HTTP only.
>
> So are you thinking all those TCP [PSH, ACK] packets are contained within a
> given HTTP POST?
>
> If you look at the first few packets, I see that FFmpeg first sends a [ SYN
> ], followed by a [ SYN, ACK ] response from server, followed by an [ ACK ]
> from client. At this point, should it not initiate HTTP protocol with
> header and endpoint? Instead it starts sending TCP [PSH, ACK]'s.
>
> Perhaps I am confused how this should work.  What ends up happening if I
> point this at my HTTP load balancer is the message is never passed to the
> backend service properly. It never is able to establish a connection.
>
> > Does it also expect particular ports? It will need to be configured to
> understand the same ports, right?
>
> I use an nginx http ingress to load balance http headers to respective
> backend services listening on TCP ports on various pods. In this case, my
> ingress would load balance HTTP <an-example-endpoint>/video to port 5558 on
> whichever pod. That pod will be running the same listen command.
>
> *UPDATE: *
>
> After digging into this more, I think I found what was going wrong! Looking
> at my nginx-controller logs, I see that I am getting an error because the
> `client intended to send too large chunked body` :
>
> ```
> 2019/03/20 16:02:41 [warn] 324#324: *3468886 a client request body is
> buffered to a temporary file /tmp/client-body/0000000009, client:
> 10.52.0.1, server: live.nycv.biz, request: "POST /push HTTP/1.1", host: "
> live.nycv.biz"
> 2019/03/20 16:02:42 [error] 324#324: *3468886 client intended to send too
> large chunked body: 1046500+32768 bytes, client: 10.52.0.1, server:
> live.nycv.biz, request: "POST /push HTTP/1.1", host: "live.nycv.biz"
> ```
>
> As before, when I was sending to server, after about a minute of sending
> the video, I would see an error on the client side from FFmpeg:
>
> ```
> av_interleaved_write_frame(): Broken pipeB time=00:00:35.10
> bitrate=17993.7kbits/s speed=0.399x
> Error writing trailer of http://live.nycv.biz/push: Broken pipe
> ```
>
> Now, setting my nginx configuration to not check for the body size by
> setting max to 0 in the ingress configuration template, I tried again:
>
> (Side note: Is there any way to tell FFmpeg to set a maximum on the chunked
> bodies before it sends the HTTP POST? )
>
> *From Client (sending side) : *
>
> chill@life ~$ ffmpeg -re -i hq-video.mp4 -c:v libx264 -an -f mpegts
> http://live.nycv.biz/push
> ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
>   built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
>   configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.1 --enable-shared
> --enable-pthreads --enable-version3 --enable-hardcoded-tables
> --enable-avresample --cc=clang
>
> --host-cflags='-I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include
>
> -I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include/darwin'
> --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl
> --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus
> --enable-librubberband --enable-libsnappy --enable-libtesseract
> --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264
> --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig
> --enable-libfreetype --enable-frei0r --enable-libass
> --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg
> --enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack
> --disable-indev=jack --enable-libaom --enable-libsoxr
>   libavutil      56. 22.100 / 56. 22.100
>   libavcodec     58. 35.100 / 58. 35.100
>   libavformat    58. 20.100 / 58. 20.100
>   libavdevice    58.  5.100 / 58.  5.100
>   libavfilter     7. 40.101 /  7. 40.101
>   libavresample   4.  0.  0 /  4.  0.  0
>   libswscale      5.  3.100 /  5.  3.100
>   libswresample   3.  3.100 /  3.  3.100
>   libpostproc    55.  3.100 / 55.  3.100
> Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'hq-video.mp4':
>   Metadata:
>     major_brand     : mp42
>     minor_version   : 0
>     compatible_brands: isommp42
>     creation_time   : 2019-02-11T22:01:43.000000Z
>     location        : +40.7298-073.9904/
>     location-eng    : +40.7298-073.9904/
>     com.android.version: 9
>     com.android.capture.fps: 30.000000
>   Duration: 00:01:42.85, start: 0.000000, bitrate: 42251 kb/s
>     Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc,
> bt470bg/bt470bg/smpte170m), 3840x2160, 42033 kb/s, SAR 1:1 DAR 16:9, 29.95
> fps, 30 tbr, 90k tbn, 180k tbc (default)
>     Metadata:
>       creation_time   : 2019-02-11T22:01:43.000000Z
>       handler_name    : VideoHandle
>     Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz,
> stereo, fltp, 156 kb/s (default)
>     Metadata:
>       creation_time   : 2019-02-11T22:01:43.000000Z
>       handler_name    : SoundHandle
> Stream mapping:
>   Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
> Press [q] to stop, [?] for help
> [libx264 @ 0x7fe0e4803e00] using SAR=1/1
> [libx264 @ 0x7fe0e4803e00] using cpu capabilities: MMX2 SSE2Fast SSSE3
> SSE4.2 AVX FMA3 BMI2 AVX2
> [libx264 @ 0x7fe0e4803e00] profile High, level 5.1
> Output #0, mpegts, to 'http://live.nycv.biz/push':
>   Metadata:
>     major_brand     : mp42
>     minor_version   : 0
>     compatible_brands: isommp42
>     com.android.capture.fps: 30.000000
>     location        : +40.7298-073.9904/
>     location-eng    : +40.7298-073.9904/
>     com.android.version: 9
>     encoder         : Lavf58.20.100
>     Stream #0:0(eng): Video: h264 (libx264), yuvj420p(pc), 3840x2160 [SAR
> 1:1 DAR 16:9], q=-1--1, 30 fps, 90k tbn, 30 tbc (default)
>     Metadata:
>       creation_time   : 2019-02-11T22:01:43.000000Z
>       handler_name    : VideoHandle
>       encoder         : Lavc58.35.100 libx264
>     Side data:
>       cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
> frame= 3080 fps= 11 q=-1.0 Lsize=  254544kB time=00:01:42.76
> bitrate=20290.8kbits/s speed=0.379x
> video:235782kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
> muxing overhead: 7.957043%
> [libx264 @ 0x7fe0e4803e00] frame I:13    Avg QP:20.17  size:930179
> [libx264 @ 0x7fe0e4803e00] frame P:992   Avg QP:23.19  size:154103
> [libx264 @ 0x7fe0e4803e00] frame B:2075  Avg QP:27.27  size: 36857
> [libx264 @ 0x7fe0e4803e00] consecutive B-frames:  9.0%  2.2%  3.7% 85.1%
> [libx264 @ 0x7fe0e4803e00] mb I  I16..4:  0.9% 86.0% 13.1%
> [libx264 @ 0x7fe0e4803e00] mb P  I16..4:  0.1%  4.5%  0.3%  P16..4: 48.4%
> 9.9%  8.0%  0.0%  0.0%    skip:28.8%
> [libx264 @ 0x7fe0e4803e00] mb B  I16..4:  0.0%  1.5%  0.1%  B16..8: 42.0%
> 1.4%  0.4%  direct: 1.0%  skip:53.7%  L0:42.0% L1:56.0% BI: 2.1%
> [libx264 @ 0x7fe0e4803e00] 8x8 transform intra:92.0% inter:60.7%
> [libx264 @ 0x7fe0e4803e00] coded y,uvDC,uvAC intra: 91.2% 56.8% 13.3%
> inter: 14.0% 8.8% 0.2%
> [libx264 @ 0x7fe0e4803e00] i16 v,h,dc,p: 17% 29% 23% 30%
> [libx264 @ 0x7fe0e4803e00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 16% 26%  6%
> 7%  8%  8%  7%  9%
> [libx264 @ 0x7fe0e4803e00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 22% 12%  7%
> 10%  8%  9%  7%  8%
> [libx264 @ 0x7fe0e4803e00] i8c dc,h,v,p: 59% 21% 17%  3%
> [libx264 @ 0x7fe0e4803e00] Weighted P-Frames: Y:0.7% UV:0.0%
> [libx264 @ 0x7fe0e4803e00] ref P L0: 69.0% 13.9% 14.2%  2.8%  0.0%
> [libx264 @ 0x7fe0e4803e00] ref B L0: 93.7%  5.6%  0.8%
> [libx264 @ 0x7fe0e4803e00] ref B L1: 94.1%  5.9%
> [libx264 @ 0x7fe0e4803e00] kb/s:18777.01
>
> Looks like everything went through alright from client perspective, *no
> more `broken pipe error`*.
>
> *However, *
>
> From Server Side (receiving):
>
> I am still not getting any data through:
>
> Inside my kubernetes pod:
>
> ```
> / # . scripts/ffmpeg-listen.sh
> current time @ 1553099678
> ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
>   built with gcc 6.4.0 (Alpine 6.4.0)
>   configuration: --disable-debug --disable-doc --disable-ffplay
> --enable-ffmpeg --enable-protocol=rtp --enable-protocol=udp
> --enable-protocol=file --enable-protocol=crypto --enable-protocol=data
> --enable-encoder=mp4 --enable-encoder=rtp --enable-decoder=rtp
> --enable-encoder='rawvideo,libx264' --enable-decoder=h264
> --enable-encoder=h264 --enable-muxer=segment
> --enable-muxer='stream_segment,ssegment' --enable-muxer='rawvideo,mp4'
> --enable-muxer=rtsp --enable-muxer=h264 --enable-demuxer=rawvideo
> --enable-demuxer=mov --enable-demuxer=h264 --enable-demuxer=rtsp
> --enable-parser=h264 --enable-parser=mpeg4 --enable-avcodec
> --enable-avformat --enable-avfilter --enable-gpl --enable-small
> --enable-libx264 --enable-nonfree --enable-openssl
>   libavutil      56. 22.100 / 56. 22.100
>   libavcodec     58. 35.100 / 58. 35.100
>   libavformat    58. 20.100 / 58. 20.100
>   libavdevice    58.  5.100 / 58.  5.100
>   libavfilter     7. 40.101 /  7. 40.101
>   libswscale      5.  3.100 /  5.  3.100
>   libswresample   3.  3.100 /  3.  3.100
>   libpostproc    55.  3.100 / 55.  3.100
> Splitting the commandline.
> Reading option '-loglevel' ... matched as option 'loglevel' (set logging
> level) with argument 'debug'.
> Reading option '-listen' ... matched as AVOption 'listen' with argument
> '1'.
> Reading option '-i' ... matched as input url with argument '
> <a href="http://0.0.0.0:5558'">http://0.0.0.0:5558'.
> Reading option '-c:v' ... matched as option 'c' (codec name) with argument
> 'h264'.
> Reading option '-r' ... matched as option 'r' (set frame rate (Hz value,
> fraction or abbreviation)) with argument '30'.
> Reading option '-flags' ... matched as AVOption 'flags' with argument
> '+cgop'.
> Reading option '-g' ... matched as AVOption 'g' with argument '30'.
> Reading option '-hls_segment_filename' ... matched as AVOption
> 'hls_segment_filename' with argument '/fuse/tmp/file%03d.ts'.
> Reading option '-hls_time' ... matched as AVOption 'hls_time' with argument
> '1'.
> Reading option '/fuse/tmp/out.m3u8' ... matched as output url.
> Reading option '-y' ... matched as option 'y' (overwrite output files) with
> argument '1'.
> Finished splitting the commandline.
> Parsing a group of options: global .
> Applying option loglevel (set logging level) with argument debug.
> Applying option y (overwrite output files) with argument 1.
> Successfully parsed a group of options.
> Parsing a group of options: input url http://0.0.0.0:5558.
> Successfully parsed a group of options.
> Opening an input file: http://0.0.0.0:5558.
> [NULL @ 0x5610666ab240] Opening '<a href="http://0.0.0.0:5558'">http://0.0.0.0:5558' for reading
> [http @ 0x5610666abc00] Setting default whitelist
> 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
> <a href="http://0.0.0.0:5558:">http://0.0.0.0:5558: End of file
> ```
>
> With no indication that any data actually made it though to the pod,
> however, I did notice that the EOF did come about the exact same time that
> the client side process finished, which is quite peculiar.
>
>
> And finally, here are the logs from the nginx-controller:
>
> ```
> 10.52.0.1 - [10.52.0.1] - - [20/Mar/2019:16:50:47 +0000] "POST /push
> HTTP/1.1" 499 0 "-" "Lavf/58.20.100" 260727893 270.979
> [default-live-rtsp-in-5558] 10.52.0.32:5558 0 0.001 -
> 8a27399690c628bd357ccf7216bf4aa6
> ```
>
> Simply a POST with no error... this would indicate strangely that a single
> chunked POST is coming with after all the packets have been sent, which is
> a bit crazy. I would much rather it break it up into many chunked POSTs.
>
>
> This is quite a mystery to me what is going wrong. I really appreciate your
> help in getting to the bottom of this.
>
> Best,
> Glenn W
>
> On Wed, Mar 20, 2019 at 8:20 AM Ted Park <[hidden email]> wrote:
>
> > > In my trace which I had analyzed as a proof of concept, HTTP was
> > > perfectly recognized. I had used port 8888, while my Wireshark's HTTP
> > > protocol seems to be configured to
> > > "80,3128,3132,5985,8080,8088,11371,1900,2869,2710". So Wireshark should
> > > be smart enough...
> >
> > You must be right, the only thing I can think of is if the capture was
> > started after ffmpeg was run and only the chunked content was captured
> that
> > can’t be decoded without the headers.
> >
> > >> The more important question is if there is an issue: Segmentation
> Fault
> > 11
> > >
> > > This is not an issue. I can segment the mpegts chunks just fine. This
> > error
> > > is coming due to the nature of how I am running my server side `-f null
> > -`
> > > to output null to stdin (not how I would normally) for the sole purpose
> > of
> > > testing network transport.
> >
> > You misunderstand, the message isn’t regarding segmenting the stream, it
> > might indicate an issue within ffmpeg itself
> > _______________________________________________
> > ffmpeg-user mailing list
> > [hidden email]
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >
> > To unsubscribe, visit link above, or email
> > [hidden email] with subject "unsubscribe".
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Glenn W
> btw, did you set client_max_body_size  in nginx.conf ? something like
1000000M :-)

Yes, see:

 "Now, setting my nginx configuration to not check for the body size by
setting max to 0 in the ingress configuration template, I tried again: "

I set to zero so as to forgo the body size check altogether, as can be seen
here:
http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size

That is how I got to not throw same error.

However, as described, I am still experiencing problem ingesting the
packets.



On Wed, Mar 20, 2019 at 1:32 PM andrei ka <[hidden email]> wrote:

> btw, did you set client_max_body_size  in nginx.conf ? something like
> 1000000M :-)
>
> On Wed, Mar 20, 2019 at 6:03 PM Glenn W <[hidden email]> wrote:
>
> > Thanks Moritz,
> >
> > > File share host. ;-)
> >
> > So true. Please see below link for more full example log from my local
> > tests:
> >
> >
> >
> https://drive.google.com/file/d/1HMS64eBDpTRlFwVarz8G75IffHzgqvCV/view?usp=sharing
> >
> > > I would guess that this trace was started *after* the HTTP
> > connection was established.
> >
> > This time I definitely started the trace before I even started either
> > ffmpeg command in my terminal and still see similar behavior (many TCP
> > connections before any HTTP header). You can still see how the POSTS are
> > disparately spread throughout the packets. I'm confused about this
> > behavior.
> >
> > > I'm totally convinced that if you use ffmpeg the way you are doing, you
> > use HTTP only.
> >
> > So are you thinking all those TCP [PSH, ACK] packets are contained
> within a
> > given HTTP POST?
> >
> > If you look at the first few packets, I see that FFmpeg first sends a [
> SYN
> > ], followed by a [ SYN, ACK ] response from server, followed by an [ ACK
> ]
> > from client. At this point, should it not initiate HTTP protocol with
> > header and endpoint? Instead it starts sending TCP [PSH, ACK]'s.
> >
> > Perhaps I am confused how this should work.  What ends up happening if I
> > point this at my HTTP load balancer is the message is never passed to the
> > backend service properly. It never is able to establish a connection.
> >
> > > Does it also expect particular ports? It will need to be configured to
> > understand the same ports, right?
> >
> > I use an nginx http ingress to load balance http headers to respective
> > backend services listening on TCP ports on various pods. In this case, my
> > ingress would load balance HTTP <an-example-endpoint>/video to port 5558
> on
> > whichever pod. That pod will be running the same listen command.
> >
> > *UPDATE: *
> >
> > After digging into this more, I think I found what was going wrong!
> Looking
> > at my nginx-controller logs, I see that I am getting an error because the
> > `client intended to send too large chunked body` :
> >
> > ```
> > 2019/03/20 16:02:41 [warn] 324#324: *3468886 a client request body is
> > buffered to a temporary file /tmp/client-body/0000000009, client:
> > 10.52.0.1, server: live.nycv.biz, request: "POST /push HTTP/1.1", host:
> "
> > live.nycv.biz"
> > 2019/03/20 16:02:42 [error] 324#324: *3468886 client intended to send too
> > large chunked body: 1046500+32768 bytes, client: 10.52.0.1, server:
> > live.nycv.biz, request: "POST /push HTTP/1.1", host: "live.nycv.biz"
> > ```
> >
> > As before, when I was sending to server, after about a minute of sending
> > the video, I would see an error on the client side from FFmpeg:
> >
> > ```
> > av_interleaved_write_frame(): Broken pipeB time=00:00:35.10
> > bitrate=17993.7kbits/s speed=0.399x
> > Error writing trailer of http://live.nycv.biz/push: Broken pipe
> > ```
> >
> > Now, setting my nginx configuration to not check for the body size by
> > setting max to 0 in the ingress configuration template, I tried again:
> >
> > (Side note: Is there any way to tell FFmpeg to set a maximum on the
> chunked
> > bodies before it sends the HTTP POST? )
> >
> > *From Client (sending side) : *
> >
> > chill@life ~$ ffmpeg -re -i hq-video.mp4 -c:v libx264 -an -f mpegts
> > http://live.nycv.biz/push
> > ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
> >   built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
> >   configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.1 --enable-shared
> > --enable-pthreads --enable-version3 --enable-hardcoded-tables
> > --enable-avresample --cc=clang
> >
> >
> --host-cflags='-I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include
> >
> >
> -I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include/darwin'
> > --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl
> > --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus
> > --enable-librubberband --enable-libsnappy --enable-libtesseract
> > --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264
> > --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig
> > --enable-libfreetype --enable-frei0r --enable-libass
> > --enable-libopencore-amrnb --enable-libopencore-amrwb
> --enable-libopenjpeg
> > --enable-librtmp --enable-libspeex --enable-videotoolbox
> --disable-libjack
> > --disable-indev=jack --enable-libaom --enable-libsoxr
> >   libavutil      56. 22.100 / 56. 22.100
> >   libavcodec     58. 35.100 / 58. 35.100
> >   libavformat    58. 20.100 / 58. 20.100
> >   libavdevice    58.  5.100 / 58.  5.100
> >   libavfilter     7. 40.101 /  7. 40.101
> >   libavresample   4.  0.  0 /  4.  0.  0
> >   libswscale      5.  3.100 /  5.  3.100
> >   libswresample   3.  3.100 /  3.  3.100
> >   libpostproc    55.  3.100 / 55.  3.100
> > Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'hq-video.mp4':
> >   Metadata:
> >     major_brand     : mp42
> >     minor_version   : 0
> >     compatible_brands: isommp42
> >     creation_time   : 2019-02-11T22:01:43.000000Z
> >     location        : +40.7298-073.9904/
> >     location-eng    : +40.7298-073.9904/
> >     com.android.version: 9
> >     com.android.capture.fps: 30.000000
> >   Duration: 00:01:42.85, start: 0.000000, bitrate: 42251 kb/s
> >     Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661),
> yuvj420p(pc,
> > bt470bg/bt470bg/smpte170m), 3840x2160, 42033 kb/s, SAR 1:1 DAR 16:9,
> 29.95
> > fps, 30 tbr, 90k tbn, 180k tbc (default)
> >     Metadata:
> >       creation_time   : 2019-02-11T22:01:43.000000Z
> >       handler_name    : VideoHandle
> >     Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz,
> > stereo, fltp, 156 kb/s (default)
> >     Metadata:
> >       creation_time   : 2019-02-11T22:01:43.000000Z
> >       handler_name    : SoundHandle
> > Stream mapping:
> >   Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
> > Press [q] to stop, [?] for help
> > [libx264 @ 0x7fe0e4803e00] using SAR=1/1
> > [libx264 @ 0x7fe0e4803e00] using cpu capabilities: MMX2 SSE2Fast SSSE3
> > SSE4.2 AVX FMA3 BMI2 AVX2
> > [libx264 @ 0x7fe0e4803e00] profile High, level 5.1
> > Output #0, mpegts, to 'http://live.nycv.biz/push':
> >   Metadata:
> >     major_brand     : mp42
> >     minor_version   : 0
> >     compatible_brands: isommp42
> >     com.android.capture.fps: 30.000000
> >     location        : +40.7298-073.9904/
> >     location-eng    : +40.7298-073.9904/
> >     com.android.version: 9
> >     encoder         : Lavf58.20.100
> >     Stream #0:0(eng): Video: h264 (libx264), yuvj420p(pc), 3840x2160 [SAR
> > 1:1 DAR 16:9], q=-1--1, 30 fps, 90k tbn, 30 tbc (default)
> >     Metadata:
> >       creation_time   : 2019-02-11T22:01:43.000000Z
> >       handler_name    : VideoHandle
> >       encoder         : Lavc58.35.100 libx264
> >     Side data:
> >       cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
> > frame= 3080 fps= 11 q=-1.0 Lsize=  254544kB time=00:01:42.76
> > bitrate=20290.8kbits/s speed=0.379x
> > video:235782kB audio:0kB subtitle:0kB other streams:0kB global
> headers:0kB
> > muxing overhead: 7.957043%
> > [libx264 @ 0x7fe0e4803e00] frame I:13    Avg QP:20.17  size:930179
> > [libx264 @ 0x7fe0e4803e00] frame P:992   Avg QP:23.19  size:154103
> > [libx264 @ 0x7fe0e4803e00] frame B:2075  Avg QP:27.27  size: 36857
> > [libx264 @ 0x7fe0e4803e00] consecutive B-frames:  9.0%  2.2%  3.7% 85.1%
> > [libx264 @ 0x7fe0e4803e00] mb I  I16..4:  0.9% 86.0% 13.1%
> > [libx264 @ 0x7fe0e4803e00] mb P  I16..4:  0.1%  4.5%  0.3%  P16..4: 48.4%
> > 9.9%  8.0%  0.0%  0.0%    skip:28.8%
> > [libx264 @ 0x7fe0e4803e00] mb B  I16..4:  0.0%  1.5%  0.1%  B16..8: 42.0%
> > 1.4%  0.4%  direct: 1.0%  skip:53.7%  L0:42.0% L1:56.0% BI: 2.1%
> > [libx264 @ 0x7fe0e4803e00] 8x8 transform intra:92.0% inter:60.7%
> > [libx264 @ 0x7fe0e4803e00] coded y,uvDC,uvAC intra: 91.2% 56.8% 13.3%
> > inter: 14.0% 8.8% 0.2%
> > [libx264 @ 0x7fe0e4803e00] i16 v,h,dc,p: 17% 29% 23% 30%
> > [libx264 @ 0x7fe0e4803e00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 16% 26%  6%
> > 7%  8%  8%  7%  9%
> > [libx264 @ 0x7fe0e4803e00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 22% 12%  7%
> > 10%  8%  9%  7%  8%
> > [libx264 @ 0x7fe0e4803e00] i8c dc,h,v,p: 59% 21% 17%  3%
> > [libx264 @ 0x7fe0e4803e00] Weighted P-Frames: Y:0.7% UV:0.0%
> > [libx264 @ 0x7fe0e4803e00] ref P L0: 69.0% 13.9% 14.2%  2.8%  0.0%
> > [libx264 @ 0x7fe0e4803e00] ref B L0: 93.7%  5.6%  0.8%
> > [libx264 @ 0x7fe0e4803e00] ref B L1: 94.1%  5.9%
> > [libx264 @ 0x7fe0e4803e00] kb/s:18777.01
> >
> > Looks like everything went through alright from client perspective, *no
> > more `broken pipe error`*.
> >
> > *However, *
> >
> > From Server Side (receiving):
> >
> > I am still not getting any data through:
> >
> > Inside my kubernetes pod:
> >
> > ```
> > / # . scripts/ffmpeg-listen.sh
> > current time @ 1553099678
> > ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
> >   built with gcc 6.4.0 (Alpine 6.4.0)
> >   configuration: --disable-debug --disable-doc --disable-ffplay
> > --enable-ffmpeg --enable-protocol=rtp --enable-protocol=udp
> > --enable-protocol=file --enable-protocol=crypto --enable-protocol=data
> > --enable-encoder=mp4 --enable-encoder=rtp --enable-decoder=rtp
> > --enable-encoder='rawvideo,libx264' --enable-decoder=h264
> > --enable-encoder=h264 --enable-muxer=segment
> > --enable-muxer='stream_segment,ssegment' --enable-muxer='rawvideo,mp4'
> > --enable-muxer=rtsp --enable-muxer=h264 --enable-demuxer=rawvideo
> > --enable-demuxer=mov --enable-demuxer=h264 --enable-demuxer=rtsp
> > --enable-parser=h264 --enable-parser=mpeg4 --enable-avcodec
> > --enable-avformat --enable-avfilter --enable-gpl --enable-small
> > --enable-libx264 --enable-nonfree --enable-openssl
> >   libavutil      56. 22.100 / 56. 22.100
> >   libavcodec     58. 35.100 / 58. 35.100
> >   libavformat    58. 20.100 / 58. 20.100
> >   libavdevice    58.  5.100 / 58.  5.100
> >   libavfilter     7. 40.101 /  7. 40.101
> >   libswscale      5.  3.100 /  5.  3.100
> >   libswresample   3.  3.100 /  3.  3.100
> >   libpostproc    55.  3.100 / 55.  3.100
> > Splitting the commandline.
> > Reading option '-loglevel' ... matched as option 'loglevel' (set logging
> > level) with argument 'debug'.
> > Reading option '-listen' ... matched as AVOption 'listen' with argument
> > '1'.
> > Reading option '-i' ... matched as input url with argument '
> > <a href="http://0.0.0.0:5558'">http://0.0.0.0:5558'.
> > Reading option '-c:v' ... matched as option 'c' (codec name) with
> argument
> > 'h264'.
> > Reading option '-r' ... matched as option 'r' (set frame rate (Hz value,
> > fraction or abbreviation)) with argument '30'.
> > Reading option '-flags' ... matched as AVOption 'flags' with argument
> > '+cgop'.
> > Reading option '-g' ... matched as AVOption 'g' with argument '30'.
> > Reading option '-hls_segment_filename' ... matched as AVOption
> > 'hls_segment_filename' with argument '/fuse/tmp/file%03d.ts'.
> > Reading option '-hls_time' ... matched as AVOption 'hls_time' with
> argument
> > '1'.
> > Reading option '/fuse/tmp/out.m3u8' ... matched as output url.
> > Reading option '-y' ... matched as option 'y' (overwrite output files)
> with
> > argument '1'.
> > Finished splitting the commandline.
> > Parsing a group of options: global .
> > Applying option loglevel (set logging level) with argument debug.
> > Applying option y (overwrite output files) with argument 1.
> > Successfully parsed a group of options.
> > Parsing a group of options: input url http://0.0.0.0:5558.
> > Successfully parsed a group of options.
> > Opening an input file: http://0.0.0.0:5558.
> > [NULL @ 0x5610666ab240] Opening '<a href="http://0.0.0.0:5558'">http://0.0.0.0:5558' for reading
> > [http @ 0x5610666abc00] Setting default whitelist
> > 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
> > <a href="http://0.0.0.0:5558:">http://0.0.0.0:5558: End of file
> > ```
> >
> > With no indication that any data actually made it though to the pod,
> > however, I did notice that the EOF did come about the exact same time
> that
> > the client side process finished, which is quite peculiar.
> >
> >
> > And finally, here are the logs from the nginx-controller:
> >
> > ```
> > 10.52.0.1 - [10.52.0.1] - - [20/Mar/2019:16:50:47 +0000] "POST /push
> > HTTP/1.1" 499 0 "-" "Lavf/58.20.100" 260727893 270.979
> > [default-live-rtsp-in-5558] 10.52.0.32:5558 0 0.001 -
> > 8a27399690c628bd357ccf7216bf4aa6
> > ```
> >
> > Simply a POST with no error... this would indicate strangely that a
> single
> > chunked POST is coming with after all the packets have been sent, which
> is
> > a bit crazy. I would much rather it break it up into many chunked POSTs.
> >
> >
> > This is quite a mystery to me what is going wrong. I really appreciate
> your
> > help in getting to the bottom of this.
> >
> > Best,
> > Glenn W
> >
> > On Wed, Mar 20, 2019 at 8:20 AM Ted Park <[hidden email]> wrote:
> >
> > > > In my trace which I had analyzed as a proof of concept, HTTP was
> > > > perfectly recognized. I had used port 8888, while my Wireshark's HTTP
> > > > protocol seems to be configured to
> > > > "80,3128,3132,5985,8080,8088,11371,1900,2869,2710". So Wireshark
> should
> > > > be smart enough...
> > >
> > > You must be right, the only thing I can think of is if the capture was
> > > started after ffmpeg was run and only the chunked content was captured
> > that
> > > can’t be decoded without the headers.
> > >
> > > >> The more important question is if there is an issue: Segmentation
> > Fault
> > > 11
> > > >
> > > > This is not an issue. I can segment the mpegts chunks just fine. This
> > > error
> > > > is coming due to the nature of how I am running my server side `-f
> null
> > > -`
> > > > to output null to stdin (not how I would normally) for the sole
> purpose
> > > of
> > > > testing network transport.
> > >
> > > You misunderstand, the message isn’t regarding segmenting the stream,
> it
> > > might indicate an issue within ffmpeg itself
> > > _______________________________________________
> > > ffmpeg-user mailing list
> > > [hidden email]
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > >
> > > To unsubscribe, visit link above, or email
> > > [hidden email] with subject "unsubscribe".
> > _______________________________________________
> > ffmpeg-user mailing list
> > [hidden email]
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >
> > To unsubscribe, visit link above, or email
> > [hidden email] with subject "unsubscribe".
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: FFmpeg: how to output over HTTP

Glenn W
For reference, here are the results of my latest attempt to send chunked
data over HTTP to cloud endpoint behind HTTP Nginx Load Balancer:

See earlier posts for details on configuration and other errors which I
have resolved.

TLDR; Connection is being made, and server (receiving side) is getting an
EOF, but seemingly no data is getting through. I am receiving no errors
from client (sender) nor from my ingress controller (load balancer).

*Wireshark PCAP Logs:*

https://drive.google.com/file/d/1EdPmStjajZ6rvVFcN7upTT8ym6mSMIaU/view?usp=sharing

*From Client (sending) side:*

```
chill@life ~$ ffmpeg -re -i hq-video.mp4 -c:v libx264 -s 1280x720 -an -f
mpegts http://live.nycv.biz/push
ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.1 --enable-shared
--enable-pthreads --enable-version3 --enable-hardcoded-tables
--enable-avresample --cc=clang
--host-cflags='-I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include
-I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include/darwin'
--host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl
--enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus
--enable-librubberband --enable-libsnappy --enable-libtesseract
--enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264
--enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig
--enable-libfreetype --enable-frei0r --enable-libass
--enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg
--enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack
--disable-indev=jack --enable-libaom --enable-libsoxr
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'hq-video.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2019-02-11T22:01:43.000000Z
    location        : +40.7298-073.9904/
    location-eng    : +40.7298-073.9904/
    com.android.version: 9
    com.android.capture.fps: 30.000000
  Duration: 00:01:42.85, start: 0.000000, bitrate: 42251 kb/s
    Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc,
bt470bg/bt470bg/smpte170m), 3840x2160, 42033 kb/s, SAR 1:1 DAR 16:9, 29.95
fps, 30 tbr, 90k tbn, 180k tbc (default)
    Metadata:
      creation_time   : 2019-02-11T22:01:43.000000Z
      handler_name    : VideoHandle
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz,
stereo, fltp, 156 kb/s (default)
    Metadata:
      creation_time   : 2019-02-11T22:01:43.000000Z
      handler_name    : SoundHandle
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[swscaler @ 0x7faf93313200] deprecated pixel format used, make sure you did
set range correctly
[libx264 @ 0x7faf92810a00] using SAR=1/1
[libx264 @ 0x7faf92810a00] using cpu capabilities: MMX2 SSE2Fast SSSE3
SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7faf92810a00] profile High, level 3.1
Output #0, mpegts, to 'http://live.nycv.biz/push':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    com.android.capture.fps: 30.000000
    location        : +40.7298-073.9904/
    location-eng    : +40.7298-073.9904/
    com.android.version: 9
    encoder         : Lavf58.20.100
    Stream #0:0(eng): Video: h264 (libx264), yuvj420p(pc), 1280x720 [SAR
1:1 DAR 16:9], q=-1--1, 30 fps, 90k tbn, 30 tbc (default)
    Metadata:
      creation_time   : 2019-02-11T22:01:43.000000Z
      handler_name    : VideoHandle
      encoder         : Lavc58.35.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
frame= 3080 fps= 30 q=-1.0 Lsize=   26613kB time=00:01:42.76
bitrate=2121.5kbits/s speed=0.989x
video:24332kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: 9.375980%
[libx264 @ 0x7faf92810a00] frame I:13    Avg QP:20.26  size:217796
[libx264 @ 0x7faf92810a00] frame P:947   Avg QP:23.33  size: 18634
[libx264 @ 0x7faf92810a00] frame B:2120  Avg QP:28.06  size:  2094
[libx264 @ 0x7faf92810a00] consecutive B-frames:  5.6%  5.3%  7.2% 81.8%
[libx264 @ 0x7faf92810a00] mb I  I16..4:  6.2% 29.5% 64.3%
[libx264 @ 0x7faf92810a00] mb P  I16..4:  0.1%  1.1%  0.6%  P16..4: 35.2%
9.8% 10.5%  0.0%  0.0%    skip:42.8%
[libx264 @ 0x7faf92810a00] mb B  I16..4:  0.0%  0.1%  0.0%  B16..8: 21.7%
1.3%  0.5%  direct: 0.6%  skip:75.7%  L0:31.6% L1:64.4% BI: 4.0%
[libx264 @ 0x7faf92810a00] 8x8 transform intra:49.9% inter:55.7%
[libx264 @ 0x7faf92810a00] coded y,uvDC,uvAC intra: 90.1% 70.4% 45.4%
inter: 9.2% 5.3% 0.7%
[libx264 @ 0x7faf92810a00] i16 v,h,dc,p: 10% 31%  8% 51%
[libx264 @ 0x7faf92810a00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 18% 19%  7%
8% 10%  8%  8%  9%
[libx264 @ 0x7faf92810a00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 31% 11%  6%
6%  7%  7%  6%  8%
[libx264 @ 0x7faf92810a00] i8c dc,h,v,p: 47% 29% 18%  6%
[libx264 @ 0x7faf92810a00] Weighted P-Frames: Y:1.0% UV:0.0%
[libx264 @ 0x7faf92810a00] ref P L0: 68.4% 12.5% 13.7%  5.3%  0.1%
[libx264 @ 0x7faf92810a00] ref B L0: 90.0%  7.4%  2.6%
[libx264 @ 0x7faf92810a00] ref B L1: 94.4%  5.6%
[libx264 @ 0x7faf92810a00] kb/s:1937.73
```

*From Server (receiving) side:*

```
/scripts # . ffmpeg-listen.sh
current time @ 1553109598
ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 6.4.0 (Alpine 6.4.0)
  configuration: --disable-debug --disable-doc --disable-ffplay
--enable-ffmpeg --enable-protocol=rtp --enable-protocol=udp
--enable-protocol=file --enable-protocol=crypto --enable-protocol=data
--enable-encoder=mp4 --enable-encoder=rtp --enable-decoder=rtp
--enable-encoder='rawvideo,libx264' --enable-decoder=h264
--enable-encoder=h264 --enable-muxer=segment
--enable-muxer='stream_segment,ssegment' --enable-muxer='rawvideo,mp4'
--enable-muxer=rtsp --enable-muxer=h264 --enable-demuxer=rawvideo
--enable-demuxer=mov --enable-demuxer=h264 --enable-demuxer=rtsp
--enable-parser=h264 --enable-parser=mpeg4 --enable-avcodec
--enable-avformat --enable-avfilter --enable-gpl --enable-small
--enable-libx264 --enable-nonfree --enable-openssl
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging
level) with argument 'debug'.
Reading option '-listen' ... matched as AVOption 'listen' with argument '1'.
Reading option '-i' ... matched as input url with argument '
http://0.0.0.0:5558/push'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument
'h264'.
Reading option '-r' ... matched as option 'r' (set frame rate (Hz value,
fraction or abbreviation)) with argument '30'.
Reading option 'test.mpegts' ... matched as output url.
Reading option '-y' ... matched as option 'y' (overwrite output files) with
argument '1'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Applying option y (overwrite output files) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url http://0.0.0.0:5558/push.
Successfully parsed a group of options.
Opening an input file: http://0.0.0.0:5558/push.
[NULL @ 0x55959ceed080] Opening 'http://0.0.0.0:5558/push' for reading
[http @ 0x55959ceed9c0] Setting default whitelist
'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
http://0.0.0.0:5558/push: End of file
```

*From Ingress Controller Logs*:

```
2019/03/20 19:21:06 [warn] 37#37: *32269 a client request body is buffered
to a temporary file /tmp/client-body/0000000005, client: 10.52.0.1, server:
live.nycv.biz, request: "POST /push HTTP/1.1", host: "live.nycv.biz"
10.52.0.1 - [10.52.0.1] - - [20/Mar/2019:19:22:48 +0000] "POST /push
HTTP/1.1" 499 0 "-" "Lavf/58.20.100" 27276563 103.995
[default-live-rtsp-in-5558] 10.52.0.32:5558 0 0.000 -
7798306d3848eee1b4837971f1167a69
```

I'm really confused what's happening because it seems like it should be
working.

Thanks again, really appreciate all the help.

Best,
Glenn W

On Wed, Mar 20, 2019 at 2:31 PM Glenn W <[hidden email]> wrote:

> > btw, did you set client_max_body_size  in nginx.conf ? something like
> 1000000M :-)
>
> Yes, see:
>
>  "Now, setting my nginx configuration to not check for the body size by
> setting max to 0 in the ingress configuration template, I tried again: "
>
> I set to zero so as to forgo the body size check altogether, as can be
> seen here:
> http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size
>
> That is how I got to not throw same error.
>
> However, as described, I am still experiencing problem ingesting the
> packets.
>
>
>
> On Wed, Mar 20, 2019 at 1:32 PM andrei ka <[hidden email]> wrote:
>
>> btw, did you set client_max_body_size  in nginx.conf ? something like
>> 1000000M :-)
>>
>> On Wed, Mar 20, 2019 at 6:03 PM Glenn W <[hidden email]> wrote:
>>
>> > Thanks Moritz,
>> >
>> > > File share host. ;-)
>> >
>> > So true. Please see below link for more full example log from my local
>> > tests:
>> >
>> >
>> >
>> https://drive.google.com/file/d/1HMS64eBDpTRlFwVarz8G75IffHzgqvCV/view?usp=sharing
>> >
>> > > I would guess that this trace was started *after* the HTTP
>> > connection was established.
>> >
>> > This time I definitely started the trace before I even started either
>> > ffmpeg command in my terminal and still see similar behavior (many TCP
>> > connections before any HTTP header). You can still see how the POSTS are
>> > disparately spread throughout the packets. I'm confused about this
>> > behavior.
>> >
>> > > I'm totally convinced that if you use ffmpeg the way you are doing,
>> you
>> > use HTTP only.
>> >
>> > So are you thinking all those TCP [PSH, ACK] packets are contained
>> within a
>> > given HTTP POST?
>> >
>> > If you look at the first few packets, I see that FFmpeg first sends a [
>> SYN
>> > ], followed by a [ SYN, ACK ] response from server, followed by an [
>> ACK ]
>> > from client. At this point, should it not initiate HTTP protocol with
>> > header and endpoint? Instead it starts sending TCP [PSH, ACK]'s.
>> >
>> > Perhaps I am confused how this should work.  What ends up happening if I
>> > point this at my HTTP load balancer is the message is never passed to
>> the
>> > backend service properly. It never is able to establish a connection.
>> >
>> > > Does it also expect particular ports? It will need to be configured to
>> > understand the same ports, right?
>> >
>> > I use an nginx http ingress to load balance http headers to respective
>> > backend services listening on TCP ports on various pods. In this case,
>> my
>> > ingress would load balance HTTP <an-example-endpoint>/video to port
>> 5558 on
>> > whichever pod. That pod will be running the same listen command.
>> >
>> > *UPDATE: *
>> >
>> > After digging into this more, I think I found what was going wrong!
>> Looking
>> > at my nginx-controller logs, I see that I am getting an error because
>> the
>> > `client intended to send too large chunked body` :
>> >
>> > ```
>> > 2019/03/20 16:02:41 [warn] 324#324: *3468886 a client request body is
>> > buffered to a temporary file /tmp/client-body/0000000009, client:
>> > 10.52.0.1, server: live.nycv.biz, request: "POST /push HTTP/1.1",
>> host: "
>> > live.nycv.biz"
>> > 2019/03/20 16:02:42 [error] 324#324: *3468886 client intended to send
>> too
>> > large chunked body: 1046500+32768 bytes, client: 10.52.0.1, server:
>> > live.nycv.biz, request: "POST /push HTTP/1.1", host: "live.nycv.biz"
>> > ```
>> >
>> > As before, when I was sending to server, after about a minute of sending
>> > the video, I would see an error on the client side from FFmpeg:
>> >
>> > ```
>> > av_interleaved_write_frame(): Broken pipeB time=00:00:35.10
>> > bitrate=17993.7kbits/s speed=0.399x
>> > Error writing trailer of http://live.nycv.biz/push: Broken pipe
>> > ```
>> >
>> > Now, setting my nginx configuration to not check for the body size by
>> > setting max to 0 in the ingress configuration template, I tried again:
>> >
>> > (Side note: Is there any way to tell FFmpeg to set a maximum on the
>> chunked
>> > bodies before it sends the HTTP POST? )
>> >
>> > *From Client (sending side) : *
>> >
>> > chill@life ~$ ffmpeg -re -i hq-video.mp4 -c:v libx264 -an -f mpegts
>> > http://live.nycv.biz/push
>> > ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
>> >   built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
>> >   configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.1 --enable-shared
>> > --enable-pthreads --enable-version3 --enable-hardcoded-tables
>> > --enable-avresample --cc=clang
>> >
>> >
>> --host-cflags='-I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include
>> >
>> >
>> -I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include/darwin'
>> > --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl
>> > --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus
>> > --enable-librubberband --enable-libsnappy --enable-libtesseract
>> > --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264
>> > --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig
>> > --enable-libfreetype --enable-frei0r --enable-libass
>> > --enable-libopencore-amrnb --enable-libopencore-amrwb
>> --enable-libopenjpeg
>> > --enable-librtmp --enable-libspeex --enable-videotoolbox
>> --disable-libjack
>> > --disable-indev=jack --enable-libaom --enable-libsoxr
>> >   libavutil      56. 22.100 / 56. 22.100
>> >   libavcodec     58. 35.100 / 58. 35.100
>> >   libavformat    58. 20.100 / 58. 20.100
>> >   libavdevice    58.  5.100 / 58.  5.100
>> >   libavfilter     7. 40.101 /  7. 40.101
>> >   libavresample   4.  0.  0 /  4.  0.  0
>> >   libswscale      5.  3.100 /  5.  3.100
>> >   libswresample   3.  3.100 /  3.  3.100
>> >   libpostproc    55.  3.100 / 55.  3.100
>> > Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'hq-video.mp4':
>> >   Metadata:
>> >     major_brand     : mp42
>> >     minor_version   : 0
>> >     compatible_brands: isommp42
>> >     creation_time   : 2019-02-11T22:01:43.000000Z
>> >     location        : +40.7298-073.9904/
>> >     location-eng    : +40.7298-073.9904/
>> >     com.android.version: 9
>> >     com.android.capture.fps: 30.000000
>> >   Duration: 00:01:42.85, start: 0.000000, bitrate: 42251 kb/s
>> >     Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661),
>> yuvj420p(pc,
>> > bt470bg/bt470bg/smpte170m), 3840x2160, 42033 kb/s, SAR 1:1 DAR 16:9,
>> 29.95
>> > fps, 30 tbr, 90k tbn, 180k tbc (default)
>> >     Metadata:
>> >       creation_time   : 2019-02-11T22:01:43.000000Z
>> >       handler_name    : VideoHandle
>> >     Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz,
>> > stereo, fltp, 156 kb/s (default)
>> >     Metadata:
>> >       creation_time   : 2019-02-11T22:01:43.000000Z
>> >       handler_name    : SoundHandle
>> > Stream mapping:
>> >   Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
>> > Press [q] to stop, [?] for help
>> > [libx264 @ 0x7fe0e4803e00] using SAR=1/1
>> > [libx264 @ 0x7fe0e4803e00] using cpu capabilities: MMX2 SSE2Fast SSSE3
>> > SSE4.2 AVX FMA3 BMI2 AVX2
>> > [libx264 @ 0x7fe0e4803e00] profile High, level 5.1
>> > Output #0, mpegts, to 'http://live.nycv.biz/push':
>> >   Metadata:
>> >     major_brand     : mp42
>> >     minor_version   : 0
>> >     compatible_brands: isommp42
>> >     com.android.capture.fps: 30.000000
>> >     location        : +40.7298-073.9904/
>> >     location-eng    : +40.7298-073.9904/
>> >     com.android.version: 9
>> >     encoder         : Lavf58.20.100
>> >     Stream #0:0(eng): Video: h264 (libx264), yuvj420p(pc), 3840x2160
>> [SAR
>> > 1:1 DAR 16:9], q=-1--1, 30 fps, 90k tbn, 30 tbc (default)
>> >     Metadata:
>> >       creation_time   : 2019-02-11T22:01:43.000000Z
>> >       handler_name    : VideoHandle
>> >       encoder         : Lavc58.35.100 libx264
>> >     Side data:
>> >       cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
>> > frame= 3080 fps= 11 q=-1.0 Lsize=  254544kB time=00:01:42.76
>> > bitrate=20290.8kbits/s speed=0.379x
>> > video:235782kB audio:0kB subtitle:0kB other streams:0kB global
>> headers:0kB
>> > muxing overhead: 7.957043%
>> > [libx264 @ 0x7fe0e4803e00] frame I:13    Avg QP:20.17  size:930179
>> > [libx264 @ 0x7fe0e4803e00] frame P:992   Avg QP:23.19  size:154103
>> > [libx264 @ 0x7fe0e4803e00] frame B:2075  Avg QP:27.27  size: 36857
>> > [libx264 @ 0x7fe0e4803e00] consecutive B-frames:  9.0%  2.2%  3.7% 85.1%
>> > [libx264 @ 0x7fe0e4803e00] mb I  I16..4:  0.9% 86.0% 13.1%
>> > [libx264 @ 0x7fe0e4803e00] mb P  I16..4:  0.1%  4.5%  0.3%  P16..4:
>> 48.4%
>> > 9.9%  8.0%  0.0%  0.0%    skip:28.8%
>> > [libx264 @ 0x7fe0e4803e00] mb B  I16..4:  0.0%  1.5%  0.1%  B16..8:
>> 42.0%
>> > 1.4%  0.4%  direct: 1.0%  skip:53.7%  L0:42.0% L1:56.0% BI: 2.1%
>> > [libx264 @ 0x7fe0e4803e00] 8x8 transform intra:92.0% inter:60.7%
>> > [libx264 @ 0x7fe0e4803e00] coded y,uvDC,uvAC intra: 91.2% 56.8% 13.3%
>> > inter: 14.0% 8.8% 0.2%
>> > [libx264 @ 0x7fe0e4803e00] i16 v,h,dc,p: 17% 29% 23% 30%
>> > [libx264 @ 0x7fe0e4803e00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 16% 26%
>> 6%
>> > 7%  8%  8%  7%  9%
>> > [libx264 @ 0x7fe0e4803e00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 22% 12%
>> 7%
>> > 10%  8%  9%  7%  8%
>> > [libx264 @ 0x7fe0e4803e00] i8c dc,h,v,p: 59% 21% 17%  3%
>> > [libx264 @ 0x7fe0e4803e00] Weighted P-Frames: Y:0.7% UV:0.0%
>> > [libx264 @ 0x7fe0e4803e00] ref P L0: 69.0% 13.9% 14.2%  2.8%  0.0%
>> > [libx264 @ 0x7fe0e4803e00] ref B L0: 93.7%  5.6%  0.8%
>> > [libx264 @ 0x7fe0e4803e00] ref B L1: 94.1%  5.9%
>> > [libx264 @ 0x7fe0e4803e00] kb/s:18777.01
>> >
>> > Looks like everything went through alright from client perspective, *no
>> > more `broken pipe error`*.
>> >
>> > *However, *
>> >
>> > From Server Side (receiving):
>> >
>> > I am still not getting any data through:
>> >
>> > Inside my kubernetes pod:
>> >
>> > ```
>> > / # . scripts/ffmpeg-listen.sh
>> > current time @ 1553099678
>> > ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
>> >   built with gcc 6.4.0 (Alpine 6.4.0)
>> >   configuration: --disable-debug --disable-doc --disable-ffplay
>> > --enable-ffmpeg --enable-protocol=rtp --enable-protocol=udp
>> > --enable-protocol=file --enable-protocol=crypto --enable-protocol=data
>> > --enable-encoder=mp4 --enable-encoder=rtp --enable-decoder=rtp
>> > --enable-encoder='rawvideo,libx264' --enable-decoder=h264
>> > --enable-encoder=h264 --enable-muxer=segment
>> > --enable-muxer='stream_segment,ssegment' --enable-muxer='rawvideo,mp4'
>> > --enable-muxer=rtsp --enable-muxer=h264 --enable-demuxer=rawvideo
>> > --enable-demuxer=mov --enable-demuxer=h264 --enable-demuxer=rtsp
>> > --enable-parser=h264 --enable-parser=mpeg4 --enable-avcodec
>> > --enable-avformat --enable-avfilter --enable-gpl --enable-small
>> > --enable-libx264 --enable-nonfree --enable-openssl
>> >   libavutil      56. 22.100 / 56. 22.100
>> >   libavcodec     58. 35.100 / 58. 35.100
>> >   libavformat    58. 20.100 / 58. 20.100
>> >   libavdevice    58.  5.100 / 58.  5.100
>> >   libavfilter     7. 40.101 /  7. 40.101
>> >   libswscale      5.  3.100 /  5.  3.100
>> >   libswresample   3.  3.100 /  3.  3.100
>> >   libpostproc    55.  3.100 / 55.  3.100
>> > Splitting the commandline.
>> > Reading option '-loglevel' ... matched as option 'loglevel' (set logging
>> > level) with argument 'debug'.
>> > Reading option '-listen' ... matched as AVOption 'listen' with argument
>> > '1'.
>> > Reading option '-i' ... matched as input url with argument '
>> > <a href="http://0.0.0.0:5558'">http://0.0.0.0:5558'.
>> > Reading option '-c:v' ... matched as option 'c' (codec name) with
>> argument
>> > 'h264'.
>> > Reading option '-r' ... matched as option 'r' (set frame rate (Hz value,
>> > fraction or abbreviation)) with argument '30'.
>> > Reading option '-flags' ... matched as AVOption 'flags' with argument
>> > '+cgop'.
>> > Reading option '-g' ... matched as AVOption 'g' with argument '30'.
>> > Reading option '-hls_segment_filename' ... matched as AVOption
>> > 'hls_segment_filename' with argument '/fuse/tmp/file%03d.ts'.
>> > Reading option '-hls_time' ... matched as AVOption 'hls_time' with
>> argument
>> > '1'.
>> > Reading option '/fuse/tmp/out.m3u8' ... matched as output url.
>> > Reading option '-y' ... matched as option 'y' (overwrite output files)
>> with
>> > argument '1'.
>> > Finished splitting the commandline.
>> > Parsing a group of options: global .
>> > Applying option loglevel (set logging level) with argument debug.
>> > Applying option y (overwrite output files) with argument 1.
>> > Successfully parsed a group of options.
>> > Parsing a group of options: input url http://0.0.0.0:5558.
>> > Successfully parsed a group of options.
>> > Opening an input file: http://0.0.0.0:5558.
>> > [NULL @ 0x5610666ab240] Opening '<a href="http://0.0.0.0:5558'">http://0.0.0.0:5558' for reading
>> > [http @ 0x5610666abc00] Setting default whitelist
>> > 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
>> > <a href="http://0.0.0.0:5558:">http://0.0.0.0:5558: End of file
>> > ```
>> >
>> > With no indication that any data actually made it though to the pod,
>> > however, I did notice that the EOF did come about the exact same time
>> that
>> > the client side process finished, which is quite peculiar.
>> >
>> >
>> > And finally, here are the logs from the nginx-controller:
>> >
>> > ```
>> > 10.52.0.1 - [10.52.0.1] - - [20/Mar/2019:16:50:47 +0000] "POST /push
>> > HTTP/1.1" 499 0 "-" "Lavf/58.20.100" 260727893 270.979
>> > [default-live-rtsp-in-5558] 10.52.0.32:5558 0 0.001 -
>> > 8a27399690c628bd357ccf7216bf4aa6
>> > ```
>> >
>> > Simply a POST with no error... this would indicate strangely that a
>> single
>> > chunked POST is coming with after all the packets have been sent, which
>> is
>> > a bit crazy. I would much rather it break it up into many chunked POSTs.
>> >
>> >
>> > This is quite a mystery to me what is going wrong. I really appreciate
>> your
>> > help in getting to the bottom of this.
>> >
>> > Best,
>> > Glenn W
>> >
>> > On Wed, Mar 20, 2019 at 8:20 AM Ted Park <[hidden email]>
>> wrote:
>> >
>> > > > In my trace which I had analyzed as a proof of concept, HTTP was
>> > > > perfectly recognized. I had used port 8888, while my Wireshark's
>> HTTP
>> > > > protocol seems to be configured to
>> > > > "80,3128,3132,5985,8080,8088,11371,1900,2869,2710". So Wireshark
>> should
>> > > > be smart enough...
>> > >
>> > > You must be right, the only thing I can think of is if the capture was
>> > > started after ffmpeg was run and only the chunked content was captured
>> > that
>> > > can’t be decoded without the headers.
>> > >
>> > > >> The more important question is if there is an issue: Segmentation
>> > Fault
>> > > 11
>> > > >
>> > > > This is not an issue. I can segment the mpegts chunks just fine.
>> This
>> > > error
>> > > > is coming due to the nature of how I am running my server side `-f
>> null
>> > > -`
>> > > > to output null to stdin (not how I would normally) for the sole
>> purpose
>> > > of
>> > > > testing network transport.
>> > >
>> > > You misunderstand, the message isn’t regarding segmenting the stream,
>> it
>> > > might indicate an issue within ffmpeg itself
>> > > _______________________________________________
>> > > ffmpeg-user mailing list
>> > > [hidden email]
>> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>> > >
>> > > To unsubscribe, visit link above, or email
>> > > [hidden email] with subject "unsubscribe".
>> > _______________________________________________
>> > ffmpeg-user mailing list
>> > [hidden email]
>> > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>> >
>> > To unsubscribe, visit link above, or email
>> > [hidden email] with subject "unsubscribe".
>> _______________________________________________
>> ffmpeg-user mailing list
>> [hidden email]
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>>
>> To unsubscribe, visit link above, or email
>> [hidden email] with subject "unsubscribe".
>
>
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
12