Slow seeking for GB-scale MOVs?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Slow seeking for GB-scale MOVs?

John Hawkinson
When I try to seek within a 5 GB MOV file (24 hours 12 minutes of video; someone screwed up) over http, it seems that ffmpeg downloads a lot more data than is necessary, and so is fairly slow to start (delay of minutes on a medium-speed network connection), seemingly unnecessarily so. In particular, it seems that in the search for the moov atom, it may end up downloading the final 69 MB of the file, and that takes quite a while.

(I'm not quite sure of this -- the http traces from libavformat appear to indicate the start of HTTP byte-range queries, but don't indicate  the end of such a transfer. For instance, iniially there is a GET for "Range: bytes=0-" and shortly thereafter there is a GET for "Range: bytes=6050885555-" and it's clear that the first could not have downloaded the entire 5 GB file before turning to the second. So maybe I am missing when http transfer stops and maybe that matters here.)

I spent a while staring at libavformat/mov.c and I wasn't able to discern exactly how/when it decides to do this seeking, although empirically it would appear that the offset it begins to retrieve is a function of the 'sz' reported in the 'mdat' trace line, and it finds the 'moov' atom quite quickly. Yet there is long delay (again, a minute-ish, depending on connection speed) before any playback starts. There are a few stages along the way, with a lot of AvIndex output about halfway before playback begins, but that don't seem correlated with http GETs.

My example is trying to seek 30 minutes into http://mediadownload.iqm2.com/CambridgeMA/2288.mpeg4 , the 5 GB video.
So, for instance:

bash$ ffplay -ss 30:00 -loglevel trace http://mediadownload.iqm2.com/CambridgeMA/2288.mpeg4
...
2020-06-07 18:56:30.112945500     nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0  
2020-06-07 18:56:30.112945500 Probing mov,mp4,m4a,3gp,3g2,mj2 score:100 size:2048
2020-06-07 18:56:30.127791500 Probing mp3 score:1 size:2048
2020-06-07 18:56:30.129018500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
2020-06-07 18:56:30.129069500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'ftyp' parent:'root' sz: 32 8 6124146041
2020-06-07 18:56:30.129287500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] ISO: File Type Major Brand: isom
2020-06-07 18:56:30.129304500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'mdat' parent:'root' sz: 6050885515 48 6124146041
...
2020-06-07 18:56:30.172001500 [http @ 0x7f9252c2e700] request: GET /CambridgeMA/2288.mpeg4 HTTP/1.1
2020-06-07 18:56:30.172002500 User-Agent: Lavf/58.29.100
2020-06-07 18:56:30.172002500 Accept: */*
2020-06-07 18:56:30.172002500 Range: bytes=6050885555-
2020-06-07 18:56:30.172003500 Connection: close
2020-06-07 18:56:30.172003500 Host: mediadownload.iqm2.com
2020-06-07 18:56:30.172003500 Icy-MetaData: 1
2020-06-07 18:56:30.172003500
2020-06-07 18:56:30.172004500
2020-06-07 18:56:30.177046500     nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0  
2020-06-07 18:56:30.177046500 [http @ 0x7f9252c2e700] header='HTTP/1.1 206 Partial Content'
2020-06-07 18:56:30.195744500 [http @ 0x7f9252c2e700] http_code=206
2020-06-07 18:56:30.195745500 [http @ 0x7f9252c2e700] header='Server: Apache'
2020-06-07 18:56:30.195765500 [http @ 0x7f9252c2e700] header='Accept-Ranges: bytes'
2020-06-07 18:56:30.195766500 [http @ 0x7f9252c2e700] header='Content-Disposition: attachment'
2020-06-07 18:56:30.195766500 [http @ 0x7f9252c2e700] header='Content-Type: application/octet-stream'
2020-06-07 18:56:30.195767500 [http @ 0x7f9252c2e700] header='Content-Range: bytes 6050885555-6124146040/6124146041'
2020-06-07 18:56:30.195768500 [http @ 0x7f9252c2e700] header='Age: 35458'
2020-06-07 18:56:30.195773500 [http @ 0x7f9252c2e700] header='Date: Sun, 07 Jun 2020 22:56:30 GMT'
2020-06-07 18:56:30.195774500 [http @ 0x7f9252c2e700] header='Last-Modified: Wed, 03 Jun 2020 15:15:47 GMT'
2020-06-07 18:56:30.195779500 [http @ 0x7f9252c2e700] header='X-LLID: 0194500ec1fe9a05fa8c3f10d9a198d9'
2020-06-07 18:56:30.195784500 [http @ 0x7f9252c2e700] header='Content-Length: 73260486'
2020-06-07 18:56:30.195788500 [http @ 0x7f9252c2e700] header='Connection: close'
2020-06-07 18:56:30.195792500 [http @ 0x7f9252c2e700] header=''
2020-06-07 18:56:30.201315500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'moov' parent:'root' sz: 73260486 6050885563 6124146041
2020-06-07 18:56:30.201338500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'mvhd' parent:'moov' sz: 108 8 73260478
2020-06-07 18:56:30.201339500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] time scale = 1000
2020-06-07 18:56:30.201369500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'trak' parent:'moov' sz: 46942399 116 73260478
2020-06-07 18:56:30.201475500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'tkhd' parent:'trak' sz: 92 8 46942391
2020-06-07 18:56:30.201487500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'edts' parent:'trak' sz: 36 100 46942391
2020-06-07 18:56:30.201492500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'elst' parent:'edts' sz: 28 8 28
2020-06-07 18:56:30.201519500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] track[0].edit_count = 1
2020-06-07 18:56:30.201520500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] duration=87144443 time=1024 rate=1.000000
2020-06-07 18:56:30.201520500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'mdia' parent:'trak' sz: 46942263 136 46942391
2020-06-07 18:56:30.201526500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'mdhd' parent:'mdia' sz: 44 8 46942255
2020-06-07 18:56:30.201549500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'hdlr' parent:'mdia' sz: 45 52 46942255
2020-06-07 18:56:30.201550500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] ctype=[0][0][0][0]
2020-06-07 18:56:30.201550500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] stype=soun
2020-06-07 18:56:30.201557500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'minf' parent:'mdia' sz: 46942166 97 46942255
2020-06-07 18:56:30.201571500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'smhd' parent:'minf' sz: 16 8 46942158
2020-06-07 18:56:30.201579500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'dinf' parent:'minf' sz: 36 24 46942158
2020-06-07 18:56:30.201584500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'dref' parent:'dinf' sz: 28 8 28
2020-06-07 18:56:30.201593500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] Unknown dref type 0x206c7275 size 12
2020-06-07 18:56:30.201598500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'stbl' parent:'minf' sz: 46942106 60 46942158
2020-06-07 18:56:30.201606500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'stsd' parent:'stbl' sz: 106 8 46942098
2020-06-07 18:56:30.201629500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] size=90 4CC=mp4a codec_type=1
2020-06-07 18:56:30.201638500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] audio channels 2
2020-06-07 18:56:30.201639500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] version =0, isom =1
2020-06-07 18:56:30.201647500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'esds' parent:'stsd' sz: 54 8 54
2020-06-07 18:56:30.201659500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] MPEG-4 description: tag=0x03 len=37
2020-06-07 18:56:30.201665500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] MPEG-4 description: tag=0x04 len=23
2020-06-07 18:56:30.201670500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] esds object type id 0x40
2020-06-07 18:56:30.201675500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] MPEG-4 description: tag=0x05 len=5
2020-06-07 18:56:30.201680500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] Specific MPEG-4 header len=5
2020-06-07 18:56:30.201697500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] mp4a config channels 2 obj 2 ext obj 5 sample rate 44100 ext sample rate 0
2020-06-07 18:56:30.201708500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'stts' parent:'stbl' sz: 24 114 46942098
2020-06-07 18:56:30.201713500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] track[0].stts.entries = 1
2020-06-07 18:56:30.201718500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] sample_count=3752999, sample_duration=1024
2020-06-07 18:56:30.201727500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'stsc' parent:'stbl' sz: 14501248 138 46942098
2020-06-07 18:56:30.201732500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] track[0].stsc.entries = 1208436
2020-06-07 18:56:30.209085500     nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0  
...
2020-06-07 18:58:00.634529500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] AVIndex stream 0, sample 0, offset 30, dts 0, size 145, distance 0, keyframe 1
2020-06-07 18:58:00.634529500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] AVIndex stream 0, sample 1, offset c1, dts 1024, size 129, distance 0, keyframe 1
2020-06-07 18:58:00.634530500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] AVIndex stream 0, sample 2, offset 5735, dts 2048, size 142, distance 0, keyframe 1
...
2020-06-07 18:59:47.780440500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] AVIndex stream 1, sample 2178584, offset 168a90a60, dts 7842902400, size 964, distance 34, keyframe 0


That is, we began at 18:56:30, a lot of these

    nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0

liens ntil 18:58:00, then many many

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7ffb46000000] AVIndex stream 0, sample 0, offset 30, dts 0, size 145, distance 0, keyframe 1

lines until 18:59:47, and then playback begins within a second. At that point thhere are some more http queries, including starting at byte 48 at 18:59:48,
and then byte 126003831 within the same second.

The complete uncut output at the defaul loglevel is uninteresting.
The above is excerpted from loglevel trace, which gives nearly 1 GB of output and is probably not helpful here in the entirety.

A lightly edited version and prematurely truncated version of the trace output follows, with prefixed timestamps (via tai64n/tai64nlocal).


Anyhow, is there a way to reduce how much ffmpeg downloads when it goes for the moov atom, and am I correct in thinking that that is the issue here?

Thanks.


p.s.: I'm not 100% sure this HTTP resource cited above will persist. It's possible someone may decide to correct the errantly recorded 24+ hours of video (it should have been only 9) and so the problem may go away or shift.

--
[hidden email]
John Hawkinson

2020-06-07 18:56:29.836799500 ffplay version 4.2.2 Copyright (c) 2003-2019 the FFmpeg developers
2020-06-07 18:56:29.836800500   built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
2020-06-07 18:56:29.836801500   configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/jdk1.8.0_74.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/jdk1.8.0_74.jdk/Contents/Home/include/darwin' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack --enable-libzvbi
2020-06-07 18:56:29.836855500   libavutil      56. 31.100 / 56. 31.100
2020-06-07 18:56:29.836855500   libavcodec     58. 54.100 / 58. 54.100
2020-06-07 18:56:29.836856500   libavformat    58. 29.100 / 58. 29.100
2020-06-07 18:56:29.836856500   libavdevice    58.  8.100 / 58.  8.100
2020-06-07 18:56:29.836856500   libavfilter     7. 57.100 /  7. 57.100
2020-06-07 18:56:29.836863500   libavresample   4.  0.  0 /  4.  0.  0
2020-06-07 18:56:29.836864500   libswscale      5.  5.100 /  5.  5.100
2020-06-07 18:56:29.836869500   libswresample   3.  5.100 /  3.  5.100
2020-06-07 18:56:29.836874500   libpostproc    55.  5.100 / 55.  5.100
2020-06-07 18:56:29.987504500 Initialized metal renderer.
2020-06-07 18:56:29.996615500 [NULL @ 0x7f9253000600] Opening 'http://mediadownload.iqm2.com/CambridgeMA/2288.mpeg4' for reading
2020-06-07 18:56:29.996616500 [http @ 0x7f9252c2e700] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
2020-06-07 18:56:30.057426500 [tcp @ 0x7f9252f34d40] Original list of addresses:
2020-06-07 18:56:30.057448500 [tcp @ 0x7f9252f34d40] Address 69.164.46.0 port 80
2020-06-07 18:56:30.057457500 [tcp @ 0x7f9252f34d40] Address 69.164.46.128 port 80
2020-06-07 18:56:30.057462500 [tcp @ 0x7f9252f34d40] Interleaved list of addresses:
2020-06-07 18:56:30.057472500 [tcp @ 0x7f9252f34d40] Address 69.164.46.0 port 80
2020-06-07 18:56:30.057476500 [tcp @ 0x7f9252f34d40] Address 69.164.46.128 port 80
2020-06-07 18:56:30.057485500 [tcp @ 0x7f9252f34d40] Starting connection attempt to 69.164.46.0 port 80
2020-06-07 18:56:30.082411500 [tcp @ 0x7f9252f34d40] Successfully connected to 69.164.46.0 port 80
2020-06-07 18:56:30.082499500 [http @ 0x7f9252c2e700] request: GET /CambridgeMA/2288.mpeg4 HTTP/1.1
2020-06-07 18:56:30.082499500 User-Agent: Lavf/58.29.100
2020-06-07 18:56:30.082500500 Accept: */*
2020-06-07 18:56:30.082500500 Range: bytes=0-
2020-06-07 18:56:30.082500500 Connection: close
2020-06-07 18:56:30.082501500 Host: mediadownload.iqm2.com
2020-06-07 18:56:30.082501500 Icy-MetaData: 1
2020-06-07 18:56:30.082501500
2020-06-07 18:56:30.082501500
2020-06-07 18:56:30.108572500 [http @ 0x7f9252c2e700] header='HTTP/1.1 206 Partial Content'
2020-06-07 18:56:30.108587500 [http @ 0x7f9252c2e700] http_code=206
2020-06-07 18:56:30.108591500 [http @ 0x7f9252c2e700] header='Server: Apache'
2020-06-07 18:56:30.108594500 [http @ 0x7f9252c2e700] header='Accept-Ranges: bytes'
2020-06-07 18:56:30.108598500 [http @ 0x7f9252c2e700] header='Content-Disposition: attachment'
2020-06-07 18:56:30.108601500 [http @ 0x7f9252c2e700] header='Content-Type: application/octet-stream'
2020-06-07 18:56:30.108605500 [http @ 0x7f9252c2e700] header='Age: 35458'
2020-06-07 18:56:30.108608500 [http @ 0x7f9252c2e700] header='Date: Sun, 07 Jun 2020 22:56:30 GMT'
2020-06-07 18:56:30.108612500 [http @ 0x7f9252c2e700] header='Last-Modified: Wed, 03 Jun 2020 15:15:47 GMT'
2020-06-07 18:56:30.108615500 [http @ 0x7f9252c2e700] header='Content-Range: bytes 0-6124146040/6124146041'
2020-06-07 18:56:30.108624500 [http @ 0x7f9252c2e700] header='X-LLID: fe2d4871173adb88805ac55d4a741bb8'
2020-06-07 18:56:30.108624500 [http @ 0x7f9252c2e700] header='Content-Length: 6124146041'
2020-06-07 18:56:30.108643500 [http @ 0x7f9252c2e700] header='Connection: close'
2020-06-07 18:56:30.108644500 [http @ 0x7f9252c2e700] header=''
2020-06-07 18:56:30.112945500     nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0  
Probing mov,mp4,m4a,3gp,3g2,mj2 score:100 size:2048
2020-06-07 18:56:30.127791500 Probing mp3 score:1 size:2048
2020-06-07 18:56:30.129018500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
2020-06-07 18:56:30.129069500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'ftyp' parent:'root' sz: 32 8 6124146041
2020-06-07 18:56:30.129287500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] ISO: File Type Major Brand: isom
2020-06-07 18:56:30.129304500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'mdat' parent:'root' sz: 6050885515 48 6124146041
2020-06-07 18:56:30.130446500 [tcp @ 0x7f9252e321c0] Original list of addresses:
2020-06-07 18:56:30.130461500 [tcp @ 0x7f9252e321c0] Address 69.164.46.0 port 80
2020-06-07 18:56:30.130462500 [tcp @ 0x7f9252e321c0] Address 69.164.46.128 port 80
2020-06-07 18:56:30.130462500 [tcp @ 0x7f9252e321c0] Interleaved list of addresses:
2020-06-07 18:56:30.130468500 [tcp @ 0x7f9252e321c0] Address 69.164.46.0 port 80
2020-06-07 18:56:30.130476500 [tcp @ 0x7f9252e321c0] Address 69.164.46.128 port 80
2020-06-07 18:56:30.130481500 [tcp @ 0x7f9252e321c0] Starting connection attempt to 69.164.46.0 port 80
2020-06-07 18:56:30.145192500     nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0  
[tcp @ 0x7f9252e321c0] Successfully connected to 69.164.46.0 port 80
2020-06-07 18:56:30.172001500 [http @ 0x7f9252c2e700] request: GET /CambridgeMA/2288.mpeg4 HTTP/1.1
2020-06-07 18:56:30.172002500 User-Agent: Lavf/58.29.100
2020-06-07 18:56:30.172002500 Accept: */*
2020-06-07 18:56:30.172002500 Range: bytes=6050885555-
2020-06-07 18:56:30.172003500 Connection: close
2020-06-07 18:56:30.172003500 Host: mediadownload.iqm2.com
2020-06-07 18:56:30.172003500 Icy-MetaData: 1
2020-06-07 18:56:30.172003500
2020-06-07 18:56:30.172004500
2020-06-07 18:56:30.177046500     nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0  
[http @ 0x7f9252c2e700] header='HTTP/1.1 206 Partial Content'
2020-06-07 18:56:30.195744500 [http @ 0x7f9252c2e700] http_code=206
2020-06-07 18:56:30.195745500 [http @ 0x7f9252c2e700] header='Server: Apache'
2020-06-07 18:56:30.195765500 [http @ 0x7f9252c2e700] header='Accept-Ranges: bytes'
2020-06-07 18:56:30.195766500 [http @ 0x7f9252c2e700] header='Content-Disposition: attachment'
2020-06-07 18:56:30.195766500 [http @ 0x7f9252c2e700] header='Content-Type: application/octet-stream'
2020-06-07 18:56:30.195767500 [http @ 0x7f9252c2e700] header='Content-Range: bytes 6050885555-6124146040/6124146041'
2020-06-07 18:56:30.195768500 [http @ 0x7f9252c2e700] header='Age: 35458'
2020-06-07 18:56:30.195773500 [http @ 0x7f9252c2e700] header='Date: Sun, 07 Jun 2020 22:56:30 GMT'
2020-06-07 18:56:30.195774500 [http @ 0x7f9252c2e700] header='Last-Modified: Wed, 03 Jun 2020 15:15:47 GMT'
2020-06-07 18:56:30.195779500 [http @ 0x7f9252c2e700] header='X-LLID: 0194500ec1fe9a05fa8c3f10d9a198d9'
2020-06-07 18:56:30.195784500 [http @ 0x7f9252c2e700] header='Content-Length: 73260486'
2020-06-07 18:56:30.195788500 [http @ 0x7f9252c2e700] header='Connection: close'
2020-06-07 18:56:30.195792500 [http @ 0x7f9252c2e700] header=''
2020-06-07 18:56:30.201315500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'moov' parent:'root' sz: 73260486 6050885563 6124146041
2020-06-07 18:56:30.201338500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'mvhd' parent:'moov' sz: 108 8 73260478
2020-06-07 18:56:30.201339500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] time scale = 1000
2020-06-07 18:56:30.201369500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'trak' parent:'moov' sz: 46942399 116 73260478
2020-06-07 18:56:30.201475500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'tkhd' parent:'trak' sz: 92 8 46942391
2020-06-07 18:56:30.201487500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'edts' parent:'trak' sz: 36 100 46942391
2020-06-07 18:56:30.201492500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'elst' parent:'edts' sz: 28 8 28
2020-06-07 18:56:30.201519500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] track[0].edit_count = 1
2020-06-07 18:56:30.201520500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] duration=87144443 time=1024 rate=1.000000
2020-06-07 18:56:30.201520500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'mdia' parent:'trak' sz: 46942263 136 46942391
2020-06-07 18:56:30.201526500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'mdhd' parent:'mdia' sz: 44 8 46942255
2020-06-07 18:56:30.201549500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'hdlr' parent:'mdia' sz: 45 52 46942255
2020-06-07 18:56:30.201550500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] ctype=[0][0][0][0]
2020-06-07 18:56:30.201550500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] stype=soun
2020-06-07 18:56:30.201557500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'minf' parent:'mdia' sz: 46942166 97 46942255
2020-06-07 18:56:30.201571500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'smhd' parent:'minf' sz: 16 8 46942158
2020-06-07 18:56:30.201579500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'dinf' parent:'minf' sz: 36 24 46942158
2020-06-07 18:56:30.201584500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'dref' parent:'dinf' sz: 28 8 28
2020-06-07 18:56:30.201593500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] Unknown dref type 0x206c7275 size 12
2020-06-07 18:56:30.201598500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'stbl' parent:'minf' sz: 46942106 60 46942158
2020-06-07 18:56:30.201606500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'stsd' parent:'stbl' sz: 106 8 46942098
2020-06-07 18:56:30.201629500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] size=90 4CC=mp4a codec_type=1
2020-06-07 18:56:30.201638500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] audio channels 2
2020-06-07 18:56:30.201639500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] version =0, isom =1
2020-06-07 18:56:30.201647500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'esds' parent:'stsd' sz: 54 8 54
2020-06-07 18:56:30.201659500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] MPEG-4 description: tag=0x03 len=37
2020-06-07 18:56:30.201665500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] MPEG-4 description: tag=0x04 len=23
2020-06-07 18:56:30.201670500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] esds object type id 0x40
2020-06-07 18:56:30.201675500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] MPEG-4 description: tag=0x05 len=5
2020-06-07 18:56:30.201680500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] Specific MPEG-4 header len=5
2020-06-07 18:56:30.201697500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] mp4a config channels 2 obj 2 ext obj 5 sample rate 44100 ext sample rate 0
2020-06-07 18:56:30.201708500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'stts' parent:'stbl' sz: 24 114 46942098
2020-06-07 18:56:30.201713500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] track[0].stts.entries = 1
2020-06-07 18:56:30.201718500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] sample_count=3752999, sample_duration=1024
2020-06-07 18:56:30.201727500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'stsc' parent:'stbl' sz: 14501248 138 46942098
2020-06-07 18:56:30.201732500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] track[0].stsc.entries = 1208436
2020-06-07 18:56:30.209085500     nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0  
    nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0  
    nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0  
    nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0  
...
nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0  
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] type:'stsz' parent:'stbl' sz: 15012016 14501386 46942098
2020-06-07 18:56:56.305727500 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9253000600] sample_size = 0 sample_count = 3752999
2020-06-07 18:56:56.333577500     nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0  
    nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0  
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Slow seeking for GB-scale MOVs?

John Hawkinson
Any thoughts on this issue, or how best to narrow or debug it? (Platform is MacOS if that's helpful, but I could do so from Linux is that's easier.)

Thank you!

--
[hidden email]
John Hawkinson


John Hawkinson <[hidden email]> wrote on Sun,  7 Jun 2020
at 19:26:26 EDT in <[hidden email]>:

> Date: Sun, 7 Jun 2020 19:26:26 -0400
> From: John Hawkinson <[hidden email]>
> To: [hidden email]
> Subject: Slow seeking for GB-scale MOVs?
> Message-ID: <[hidden email]>
>
> When I try to seek within a 5 GB MOV file (24 hours 12 minutes of video; someone screwed up) over http, it seems that ffmpeg downloads a lot more data than is necessary, and so is fairly slow to start (delay of minutes on a medium-speed network connection), seemingly unnecessarily so. In particular, it seems that in the search for the moov atom, it may end up downloading the final 69 MB of the file, and that takes quite a while.
>
> (I'm not quite sure of this -- the http traces from libavformat appear to indicate the start of HTTP byte-range queries, but don't indicate  the end of such a transfer. For instance, iniially there is a GET for "Range: bytes=0-" and shortly thereafter there is a GET for "Range: bytes=6050885555-" and it's clear that the first could not have downloaded the entire 5 GB file before turning to the second. So maybe I am missing when http transfer stops and maybe that matters here.)
>
> I spent a while staring at libavformat/mov.c and I wasn't able to discern exactly how/when it decides to do this seeking, although empirically it would appear that the offset it begins to retrieve is a function of the 'sz' reported in the 'mdat' trace line, and it finds the 'moov' atom quite quickly. Yet there is long delay (again, a minute-ish, depending on connection speed) before any playback starts. There are a few stages along the way, with a lot of AvIndex output about halfway before playback begins, but that don't seem correlated with http GETs.
>
> My example is trying to seek 30 minutes into http://mediadownload.iqm2.com/CambridgeMA/2288.mpeg4 , the 5 GB video.
> So, for instance:
>
> bash$ ffplay -ss 30:00 -loglevel trace http://mediadownload.iqm2.com/CambridgeMA/2288.mpeg4
...

>
> The complete uncut output at the defaul loglevel is uninteresting.
> The above is excerpted from loglevel trace, which gives nearly 1 GB of output and is probably not helpful here in the entirety.
>
> A lightly edited version and prematurely truncated version of the trace output follows, with prefixed timestamps (via tai64n/tai64nlocal).
>
>
> Anyhow, is there a way to reduce how much ffmpeg downloads when it goes for the moov atom, and am I correct in thinking that that is the issue here?
>
> Thanks.
>
>
> p.s.: I'm not 100% sure this HTTP resource cited above will persist. It's possible someone may decide to correct the errantly recorded 24+ hours of video (it should have been only 9) and so the problem may go away or shift.
>
> --
> [hidden email]
> John Hawkinson
...
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".