Compression is a lot smaller

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Compression is a lot smaller

Cecil Westerhof-3
Through scripts I use ffmpeg for cutting and compressing videos and
sometimes adding a watermark. Nothing fancy, but handy.

In the past the original file was between 4 to 13 times bigger as the
compressed file. Today I compressed a few files again. The compression
was almost non existing to less as three.

454 -> 415
503 -> 230
490 -> 254
462 -> 157
497 -> 197
454 -> 174

Some logging of the worst (the first):
    nice -n 10 ionice -c3 ffmpeg -y -i 00039.MTS -vcodec libx264 -crf 23 -acodec libmp3lame -preset veryfast 00039.mp4
    ffmpeg version 4.1.6-1~deb10u1 Copyright (c) 2000-2020 the FFmpeg developers
      built with gcc 8 (Debian 8.3.0-6)
    .
    .
    .
      libavutil      56. 22.100 / 56. 22.100
      libavcodec     58. 35.100 / 58. 35.100
      libavformat    58. 20.100 / 58. 20.100
      libavdevice    58.  5.100 / 58.  5.100
      libavfilter     7. 40.101 /  7. 40.101
      libavresample   4.  0.  0 /  4.  0.  0
      libswscale      5.  3.100 /  5.  3.100
      libswresample   3.  3.100 /  3.  3.100
      libpostproc    55.  3.100 / 55.  3.100
    Input #0, mpegts, from '00039.MTS':
      Duration: 00:02:49.02, start: 1.040000, bitrate: 22533 kb/s
      Program 1
        Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc
        Stream #0:1[0x1100]: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 256 kb/s
        Stream #0:2[0x1200]: Subtitle: hdmv_pgs_subtitle ([144][0][0][0] / 0x0090), 1920x1080
    Stream mapping:
      Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
      Stream #0:1 -> #0:1 (ac3 (native) -> mp3 (libmp3lame))
    Press [q] to stop, [?] for help
    [libx264 @ 0x563820eef380] using SAR=1/1
    [libx264 @ 0x563820eef380] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX XOP FMA3 BMI1
    [libx264 @ 0x563820eef380] profile High, level 4.0
    [libx264 @ 0x563820eef380] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=6 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
    Output #0, mp4, to '00039.mp4':
      Metadata:
        encoder         : Lavf58.20.100
        Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 25 fps, 12800 tbn, 25 tbc
        Metadata:
          encoder         : Lavc58.35.100 libx264
        Side data:
          cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
        Stream #0:1: Audio: mp3 (libmp3lame) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp
        Metadata:
          encoder         : Lavc58.35.100 libmp3lame
    .
    .
    .
    video:421739kB audio:2642kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.027043%
    [libx264 @ 0x563820eef380] frame I:23    Avg QP:24.52  size:417042
    [libx264 @ 0x563820eef380] frame P:1621  Avg QP:26.67  size:165193
    [libx264 @ 0x563820eef380] frame B:2581  Avg QP:30.40  size: 59857
    [libx264 @ 0x563820eef380] consecutive B-frames:  8.0% 29.4%  6.3% 56.2%
    [libx264 @ 0x563820eef380] mb I  I16..4:  3.2% 19.9% 76.8%
    [libx264 @ 0x563820eef380] mb P  I16..4:  1.1%  5.9%  3.6%  P16..4: 45.3% 16.5% 19.4%  0.0%  0.0%    skip: 8.3%
    [libx264 @ 0x563820eef380] mb B  I16..4:  0.2%  0.8%  0.1%  B16..8: 20.0%  9.5%  3.8%  direct:24.2%  skip:41.5%  L0:20.1% L1:34.8% BI:45.1%
    [libx264 @ 0x563820eef380] 8x8 transform intra:54.1% inter:41.3%
    [libx264 @ 0x563820eef380] coded y,uvDC,uvAC intra: 84.1% 74.6% 39.9% inter: 43.9% 18.0% 3.8%
    [libx264 @ 0x563820eef380] i16 v,h,dc,p: 38% 29% 21% 12%
    [libx264 @ 0x563820eef380] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 22% 23%  5%  5%  5%  6%  5%  9%
    [libx264 @ 0x563820eef380] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 36% 14%  5%  6%  5%  8%  5% 11%
    [libx264 @ 0x563820eef380] i8c dc,h,v,p: 45% 25% 19% 12%
    [libx264 @ 0x563820eef380] Weighted P-Frames: Y:5.9% UV:2.7%
    [libx264 @ 0x563820eef380] kb/s:20443.06

What could be happening here?
When I need to provide more information: let me know.

--
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Compression is a lot smaller

Simon
On Sat, Aug 15, 2020 at 5:01 AM Cecil Westerhof <[hidden email]> wrote:

> Through scripts I use ffmpeg for cutting and compressing videos and
> sometimes adding a watermark. Nothing fancy, but handy.
>
> In the past the original file was between 4 to 13 times bigger as the
> compressed file. Today I compressed a few files again. The compression
> was almost non existing to less as three.
>

[...]


>

What could be happening here?


The most immediate thought is that zoom improved their compression
algorithm and now there's not so much room left for a secondary
improvement.

You might want to take a look at the original, perhaps using ffprobe, to
determine what codec was used, and what bitrate it started out with.


>
>
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Compression is a lot smaller

Cecil Westerhof-3
Simon Roberts <[hidden email]> writes:

> On Sat, Aug 15, 2020 at 5:01 AM Cecil Westerhof <[hidden email]> wrote:
>
>> Through scripts I use ffmpeg for cutting and compressing videos and
>> sometimes adding a watermark. Nothing fancy, but handy.
>>
>> In the past the original file was between 4 to 13 times bigger as the
>> compressed file. Today I compressed a few files again. The compression
>> was almost non existing to less as three.
>>
>
> [...]
>
>
>>
>
> What could be happening here?
>
>
> The most immediate thought is that zoom improved their compression
> algorithm and now there's not so much room left for a secondary
> improvement.
>
> You might want to take a look at the original, perhaps using ffprobe, to
> determine what codec was used, and what bitrate it started out with.

No, that is not the case. These where videos I shot with my Sony
handycam. So uncompressed files.
But maybe I should have described it a bit clearer.

But there is nothing inherently wrong with the way I call ffmpeg?

--
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Compression is a lot smaller

Alexander Strasser
Hi Cecil!

On 2020-08-15 16:25 +0200, Cecil Westerhof wrote:

> Simon Roberts <[hidden email]> writes:
>
> > On Sat, Aug 15, 2020 at 5:01 AM Cecil Westerhof <[hidden email]> wrote:
> >
> >> Through scripts I use ffmpeg for cutting and compressing videos and
> >> sometimes adding a watermark. Nothing fancy, but handy.
> >>
> >> In the past the original file was between 4 to 13 times bigger as the
> >> compressed file. Today I compressed a few files again. The compression
> >> was almost non existing to less as three.
> >>
> >
> > [...]
> >
> >
> >>
> >
> > What could be happening here?
> >
> >
> > The most immediate thought is that zoom improved their compression
> > algorithm and now there's not so much room left for a secondary
> > improvement.
> >
> > You might want to take a look at the original, perhaps using ffprobe, to
> > determine what codec was used, and what bitrate it started out with.
>
> No, that is not the case. These where videos I shot with my Sony
> handycam. So uncompressed files.
> But maybe I should have described it a bit clearer.

Did I misread your logs?

|    Input #0, mpegts, from '00039.MTS':
|      Duration: 00:02:49.02, start: 1.040000, bitrate: 22533 kb/s
|      Program 1
|        Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9],
|+25 fps, 50 tbr, 90k tbn, 50 tbc
|        Stream #0:1[0x1100]: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 256 kb/s
|        Stream #0:2[0x1200]: Subtitle: hdmv_pgs_subtitle ([144][0][0][0] / 0x0090), 1920x1080
|    Stream mapping:
|      Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))

The input doesn't look uncompressed to me.


[...]

  Alexander
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Compression is a lot smaller

Cecil Westerhof-3
Alexander Strasser <[hidden email]> writes:

> On 2020-08-15 16:25 +0200, Cecil Westerhof wrote:
>> Simon Roberts <[hidden email]> writes:
>>
>> > On Sat, Aug 15, 2020 at 5:01 AM Cecil Westerhof <[hidden email]> wrote:
>> >
>> >> Through scripts I use ffmpeg for cutting and compressing videos and
>> >> sometimes adding a watermark. Nothing fancy, but handy.
>> >>
>> >> In the past the original file was between 4 to 13 times bigger as the
>> >> compressed file. Today I compressed a few files again. The compression
>> >> was almost non existing to less as three.
>> >>
>> >
>> > [...]
>> >
>> >
>> >>
>> >
>> > What could be happening here?
>> >
>> >
>> > The most immediate thought is that zoom improved their compression
>> > algorithm and now there's not so much room left for a secondary
>> > improvement.
>> >
>> > You might want to take a look at the original, perhaps using ffprobe, to
>> > determine what codec was used, and what bitrate it started out with.
>>
>> No, that is not the case. These where videos I shot with my Sony
>> handycam. So uncompressed files.
>> But maybe I should have described it a bit clearer.
>
> Did I misread your logs?
>
> |    Input #0, mpegts, from '00039.MTS':
> |      Duration: 00:02:49.02, start: 1.040000, bitrate: 22533 kb/s
> |      Program 1
> |        Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9],
> |+25 fps, 50 tbr, 90k tbn, 50 tbc
> |        Stream #0:1[0x1100]: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 256 kb/s
> |        Stream #0:2[0x1200]: Subtitle: hdmv_pgs_subtitle ([144][0][0][0] / 0x0090), 1920x1080
> |    Stream mapping:
> |      Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
>
> The input doesn't look uncompressed to me.

I do not have much knowledge of video formats, but this was really
shot with a Sony. Maybe that the Sony compresses the video itself?
It is a HDR-CX320.

--
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Compression is a lot smaller

Moritz Barsnick
On Mon, Aug 17, 2020 at 13:54:33 +0200, Cecil Westerhof wrote:
> > The input doesn't look uncompressed to me.
>
> I do not have much knowledge of video formats, but this was really
> shot with a Sony. Maybe that the Sony compresses the video itself?
> It is a HDR-CX320.

You won't see use of uncompressed video on storage, especially with
modern resolutions and bit-depths. The storage just cannot handle it.

So the recorder has a HW chip which does compression on the fly.

> 454 -> 415
> 503 -> 230
> 490 -> 254
> 462 -> 157
> 497 -> 197
> 454 -> 174

Assuming the camera used the same settings for each recording:
Compression results (size reduction) depend on the content as well. The
Sony may be compressing nearly static videos inefficiently, while
ffmpeg/x264 optimized those well. At the same time, Sony may be
compressing videos with movement, shakiness, or even noise, quite well,
and ffmpeg cannot improve on that.

This is just speculation, but could be an explanation.

It might be interesting to look at the original and resulting
bandwidths as well (i.e. video size in relation to duration).

Cheers,
Moritz
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Compression is a lot smaller

Carl Eugen Hoyos-2
In reply to this post by Cecil Westerhof-3
Am Sa., 15. Aug. 2020 um 13:01 Uhr schrieb Cecil Westerhof <[hidden email]>:
>
> In the past the original file was between 4 to 13 times bigger as the
> compressed file.

As I tried to explain in the other thread:
Even neglecting the fact that you seem to believe your camera
does an uncompressed recording (it does not), size is by itself
no useful metric for any modern video codec (modern: Anything
since the early nineties). The oldest "modern" video codec (from
1991) can easily produce significantly smaller files.
Only if you take file size and quality together can you get a metric
that may make sense. Note that only subjective blind tests are
really meaningful, everything automated (like SSIM and PSNR)
does not measure what the human eye sees - but is probably
better than no quality metric at all.
A particular quality setting at encoding time does not give you a
particular comparable quality of the output file.

And just to repeat what was already written:
FFmpeg does not compress in your command line, x264 does
the compression.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Compression is a lot smaller

pdr0
In reply to this post by Cecil Westerhof-3
Cecil Westerhof-3 wrote
> yuv420p(top first)

Maybe partially related - your camera files are encoded interlaced TFF (the
content might not be) , but your commandline specifies progressive encoding.
This has other implications in the way other programs / players handle the
file if the content is interlaced

add
 -flags +ildct+ilme -x264opts tff=1






--
Sent from: http://www.ffmpeg-archive.org/
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".