Receive network stream and drop without decoding

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Receive network stream and drop without decoding

Andrius Narbutas
Hello, trying to do something like "DASH load generator" for testing.
Trying to simulate DASH client on unencrypted stream (using public
stream in examples, should be accessible from anywhere):

ffplay -nodisp -vst 3 -an -i
'http://download.tsi.telecom-paristech.fr/gpac/dataset/dash/uhd/dashevc-ondemand-4s/dashevc-ondemand-4s-p60.mpd'

  - this works and displays no video, but eats 40-50% of CPU core.
Recompiled latest git pull with debug symbols, ran perf, top 4 CPU users:
9.01% ff_h264_decode_mb_cabac
6.66% decode_cabac_residual_nondc_internal.isra.4
6.16% loop_filter
5.19% get_cabac

So, it is clearly doing decoding even with -nodisp and -vn options (-vn
makes no difference).

On IRC i got suggestion to use ffmpeg instead of ffplay:
ffmpeg/bin/ffmpeg -hide_banner -vn -an -i
http://download.tsi.telecom-paristech.fr/gpac/dataset/dash/uhd/dashevc-ondemand-4s/dashevc-ondemand-4s-p60.mpd 
-f null -
..then it fails with:
Output #0, null, to 'pipe:':
Output file #0 does not contain any stream
If i skip -vn and -an - it works without problems (i see traffic
flowing) but eats even more CPU (~70%)

So far this is not suitable as "fake client" as server can handle
thousands clients, but "client server" can barely handle 100 connections.

Is there any way to tell ffmpeg to receive stream (network traffic) and
discard it immediately, without decoding?
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Receive network stream and drop without decoding

Carl Eugen Hoyos-2
Am Mo., 15. Juni 2020 um 22:36 Uhr schrieb Andrius Narbutas
<[hidden email]>:

> Is there any way to tell ffmpeg to receive stream (network traffic) and
> discard it immediately, without decoding?

One example is:
$ ffmpeg -i input -map 0:a:0 -c copy -f null -

Other possibilities like -map 0:0 exist.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Receive network stream and drop without decoding

Andrius Narbutas
On 2020-06-15 23:58, Carl Eugen Hoyos wrote:
> $ ffmpeg -i input -map 0:a:0 -c copy -f null -

ffmpeg -i
http://download.tsi.telecom-paristech.fr/gpac/dataset/dash/uhd/dashevc-ondemand-4s/dashevc-ondemand-4s-p60.mpd 
-map 0:a:0 -c copy -f null -
<...>
size=N/A time=00:02:31.80 bitrate=N/A speed= 165x
video:0kB audio:1195kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: unknown
...and quits
So, it just pulls one DASH audio chunk into null sink and exits (no
re-request, also note video:0kB)

> Other possibilities like -map 0:0 exist.

Slightly better as now it takes video, too:
ffmpeg -i
http://download.tsi.telecom-paristech.fr/gpac/dataset/dash/uhd/dashevc-ondemand-4s/dashevc-ondemand-4s-p60.mpd 
-map 0:0 -c copy -f null -
<...>
frame= 4320 fps=197 q=-1.0 Lsize=N/A time=00:02:19.93 bitrate=N/A
speed=6.37x
video:24094kB audio:0kB subtitle:0kB other streams:0kB global
headers:0kB muxing overhead: unknown
Now it pulls one chunk of video (but not audio stream), pushes to null
sink and exits.

With -map 0:a:0 -map 0:v:0 ir pulls audio and video, but again only one
chunk, does not follow stream. With -v debug it does not tell story why
it decided to not take next chunk...

CPU load during this one step is low, so probably no decoding done.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Receive network stream and drop without decoding

Carl Eugen Hoyos-2
Am Mo., 15. Juni 2020 um 23:39 Uhr schrieb Andrius Narbutas
<[hidden email]>:

>
> On 2020-06-15 23:58, Carl Eugen Hoyos wrote:
> > $ ffmpeg -i input -map 0:a:0 -c copy -f null -
>
> ffmpeg -i
> http://download.tsi.telecom-paristech.fr/gpac/dataset/dash/uhd/dashevc-ondemand-4s/dashevc-ondemand-4s-p60.mpd
> -map 0:a:0 -c copy -f null -
> <...>
> size=N/A time=00:02:31.80 bitrate=N/A speed= 165x
> video:0kB audio:1195kB subtitle:0kB other streams:0kB global headers:0kB
> muxing overhead: unknown
> ...and quits

Drop the map option.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Receive network stream and drop without decoding

Andrius Narbutas
On 2020-06-16 01:22, Carl Eugen Hoyos wrote:
> Drop the map option.

Well, not working as expected :(

While this is kinda-working on this DASH example (it receives stream at
max speed with low CPU load) - it does not work as "client" at the same
time and fails badly on live streams as it tries to get stream as fast
as possible.
In other words - ffplay "plays" stream at 100% of speed, requests new
packet, gets it, "plays" again and so on. Server live stream time is
going together with client, no desync, all DASH segments are provided
without problems.
ffmpeg tries to do something with stream fast as possible, so only
network is limiting factor on recorded streams (i tested few, sometimes
it goes up to 30x realtime). On live streams it fails quickly as server
do not have stream in future (as ffmpeg requests new chunk in 1 second,
instead of 2 seconds, for example). This is also a problem on recorded
stream, as every client tries to take full record at maximum speed - far
from real client which requests each chunk at realtime.

Example live stream (takes a while, depending on network speed, with
~1.5x speedup it takes ~40 seconds) which will fail because ffmpeg will
requests chunks from future:
ffmpeg -i
"http://download.tsi.telecom-paristech.fr/gpac/dataset/dash/uhd/dashevc-live-bs-2s/dashevc-live-bs-2s-p60.mpd"
-c copy -f null -
<...>
[http @ 0000020b46696500] HTTP error 404 Not Found
[dash @ 0000020b45a49300] Failed to open fragment of playlist 0

I can't find any option to limit ffmpeg speed (if it can work at exactly
1x speed this may solve this problem), so maybe modifying ffplay is easier?
There is decoder_decode_frame function in ffplay, what if i do immediate
'return 0' from it?
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Receive network stream and drop without decoding

Andrius Narbutas
In reply to this post by Carl Eugen Hoyos-2
Looks like there is a way to sync ffmpeg with stream speed, so it does
not try to go as fast as possible - it's "-re".
So far so good, 1080p DASH live stream works with below 1%/core:

ffmpeg -re -i "input_stream" -c copy -map 0:v:6 -f null -

I see network traffic (10Mbps) and, based on consumption - can fit
probably thousand or more copies to make proper load generator for server.
perf report shows that most load comes from libc functions, so probably
nothing to optimize on ffmpeg side already (network traffic).

So far - so good, thanks to all who helped! ;-)
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".