Lossless encoding to ffv1 changes frames md5

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Lossless encoding to ffv1 changes frames md5

Alberto Mattea
Hi all, I'm trying to losslessly convert some videos from grassvalley hqx to ffv1.
Unfortunately it seems the conversion is not really lossless, as the frames md5 changes after the conversion.
Here's what I'm doing:

- Calculate md5 of the original video frames

$ ffmpeg -i 01.avi -f framemd5 orig.framemd5
ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
  libavutil      56. 36.101 / 56. 36.101
  libavcodec     58. 65.100 / 58. 65.100
  libavformat    58. 35.101 / 58. 35.101
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 68.100 /  7. 68.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, avi, from '01.avi':
  Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
    Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (hqx (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, framemd5, to 'orig.framemd5':
  Metadata:
    encoder         : Lavf58.35.101
    Stream #0:0: Video: rawvideo (Y3[10][16] / 0x100A3359), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 1566720 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.65.100 rawvideo
frame=22610 fps= 50 q=-0.0 Lsize=    1767kB time=00:15:04.40 bitrate=  16.0kbits/s speed=1.98x    
video:172966500kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

- Convert

$ ffmpeg -i 01.avi -c:v ffv1 01ffv1.mkv
ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
  libavutil      56. 36.101 / 56. 36.101
  libavcodec     58. 65.100 / 58. 65.100
  libavformat    58. 35.101 / 58. 35.101
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 68.100 /  7. 68.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, avi, from '01.avi':
  Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
    Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (hqx (native) -> ffv1 (native))
Press [q] to stop, [?] for help
[ffv1 @ 0x6e1ce80] bits_per_raw_sample > 8, forcing range coder
Output #0, matroska, to '01ffv1.mkv':
  Metadata:
    encoder         : Lavf58.35.101
    Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 200 kb/s, 25 fps, 1k tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.65.100 ffv1
frame=22610 fps= 17 q=-0.0 Lsize=35631195kB time=00:15:04.36 bitrate=322759.1kbits/s speed=0.691x    
video:35630872kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000908%

- Calculate md5 of the converted frames (I've stopped it after a few hundred frames, just to check the beginning)

$ ffmpeg -i 01ffv1.mkv -f framemd5 converted.framemd5      
ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
  libavutil      56. 36.101 / 56. 36.101
  libavcodec     58. 65.100 / 58. 65.100
  libavformat    58. 35.101 / 58. 35.101
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 68.100 /  7. 68.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, matroska,webm, from '01ffv1.mkv':
  Metadata:
    ENCODER         : Lavf58.35.101
  Duration: 00:15:04.40, start: 0.000000, bitrate: 322745 kb/s
    Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p10le, 1632x1200, SAR 1:1 DAR 34:25, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      ENCODER         : Lavc58.65.100 ffv1
      DURATION        : 00:15:04.400000000
Stream mapping:
  Stream #0:0 -> #0:0 (ffv1 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, framemd5, to 'converted.framemd5':
  Metadata:
    encoder         : Lavf58.35.101
    Stream #0:0: Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 979200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      DURATION        : 00:15:04.400000000
      encoder         : Lavc58.65.100 rawvideo
frame=  406 fps= 31 q=-0.0 Lsize=      32kB time=00:00:16.24 bitrate=  16.1kbits/s speed=1.23x    
video:3105900kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

- Compare hashes

orig.framemd5:
#format: frame checksums
#version: 2
#hash: MD5
#software: Lavf58.35.101
#tb 0: 1/25
#media_type 0: video
#codec_id 0: rawvideo
#dimensions 0: 1632x1200
#sar 0: 1/1
#stream#, dts,        pts, duration,     size, hash
0,          0,          0,        1,  7833600, 39d64300c8b749b0776423a80746d8dc
0,          1,          1,        1,  7833600, 12e7317d49727740187d2e23439d87d8
0,          2,          2,        1,  7833600, 12e7317d49727740187d2e23439d87d8
0,          3,          3,        1,  7833600, 0b3b0136b008ec9d67e900595be0c923
0,          4,          4,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
0,          5,          5,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
0,          6,          6,        1,  7833600, 1632e4b3ba7dc28e9dde698f3b7ec9e7
0,          7,          7,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
0,          8,          8,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
0,          9,          9,        1,  7833600, 612c13b4824bd427faa51efa7a09660d
...

converted.framemd5:
#format: frame checksums
#version: 2
#hash: MD5
#software: Lavf58.35.101
#tb 0: 1/25
#media_type 0: video
#codec_id 0: rawvideo
#dimensions 0: 1632x1200
#sar 0: 1/1
#stream#, dts,        pts, duration,     size, hash
0,          0,          0,        1,  7833600, 3ed693fc02ba36cca92a2153afa960ac
0,          1,          1,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
0,          2,          2,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
0,          3,          3,        1,  7833600, 1fc392823ebc14001ee0e44c68d710e6
0,          4,          4,        1,  7833600, d63335f1c34132680e571549da51eac7
0,          5,          5,        1,  7833600, d63335f1c34132680e571549da51eac7
0,          6,          6,        1,  7833600, 09304bca4578458c353ecc119de0bd87
0,          7,          7,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
0,          8,          8,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
0,          9,          9,        1,  7833600, 78b22236d03517e5ceea3bd390ac2d0b
...

Any idea of why this is happening?


_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Lossless encoding to ffv1 changes frames md5

Carl Eugen Hoyos-2
Am Di., 17. Dez. 2019 um 14:18 Uhr schrieb Alberto Mattea <[hidden email]>:

>
> Hi all, I'm trying to losslessly convert some videos from grassvalley hqx to ffv1.
> Unfortunately it seems the conversion is not really lossless, as the frames md5 changes after the conversion.
> Here's what I'm doing:
>
> - Calculate md5 of the original video frames
>
> $ ffmpeg -i 01.avi -f framemd5 orig.framemd5
> ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
>   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
>   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
>   libavutil      56. 36.101 / 56. 36.101
>   libavcodec     58. 65.100 / 58. 65.100
>   libavformat    58. 35.101 / 58. 35.101
>   libavdevice    58.  9.101 / 58.  9.101
>   libavfilter     7. 68.100 /  7. 68.100
>   libswscale      5.  6.100 /  5.  6.100
>   libswresample   3.  6.100 /  3.  6.100
>   libpostproc    55.  6.100 / 55.  6.100
> Input #0, avi, from '01.avi':
>   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
>     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
> Stream mapping:
>   Stream #0:0 -> #0:0 (hqx (native) -> rawvideo (native))
> Press [q] to stop, [?] for help
> Output #0, framemd5, to 'orig.framemd5':
>   Metadata:
>     encoder         : Lavf58.35.101
>     Stream #0:0: Video: rawvideo (Y3[10][16] / 0x100A3359), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 1566720 kb/s, 25 fps, 25 tbn, 25 tbc
>     Metadata:
>       encoder         : Lavc58.65.100 rawvideo
> frame=22610 fps= 50 q=-0.0 Lsize=    1767kB time=00:15:04.40 bitrate=  16.0kbits/s speed=1.98x
> video:172966500kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
>
> - Convert
>
> $ ffmpeg -i 01.avi -c:v ffv1 01ffv1.mkv
> ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
>   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
>   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
>   libavutil      56. 36.101 / 56. 36.101
>   libavcodec     58. 65.100 / 58. 65.100
>   libavformat    58. 35.101 / 58. 35.101
>   libavdevice    58.  9.101 / 58.  9.101
>   libavfilter     7. 68.100 /  7. 68.100
>   libswscale      5.  6.100 /  5.  6.100
>   libswresample   3.  6.100 /  3.  6.100
>   libpostproc    55.  6.100 / 55.  6.100
> Input #0, avi, from '01.avi':
>   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
>     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
> Stream mapping:
>   Stream #0:0 -> #0:0 (hqx (native) -> ffv1 (native))
> Press [q] to stop, [?] for help
> [ffv1 @ 0x6e1ce80] bits_per_raw_sample > 8, forcing range coder
> Output #0, matroska, to '01ffv1.mkv':
>   Metadata:
>     encoder         : Lavf58.35.101
>     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 200 kb/s, 25 fps, 1k tbn, 25 tbc
>     Metadata:
>       encoder         : Lavc58.65.100 ffv1
> frame=22610 fps= 17 q=-0.0 Lsize=35631195kB time=00:15:04.36 bitrate=322759.1kbits/s speed=0.691x
> video:35630872kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000908%
>
> - Calculate md5 of the converted frames (I've stopped it after a few hundred frames, just to check the beginning)
>
> $ ffmpeg -i 01ffv1.mkv -f framemd5 converted.framemd5
> ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
>   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
>   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
>   libavutil      56. 36.101 / 56. 36.101
>   libavcodec     58. 65.100 / 58. 65.100
>   libavformat    58. 35.101 / 58. 35.101
>   libavdevice    58.  9.101 / 58.  9.101
>   libavfilter     7. 68.100 /  7. 68.100
>   libswscale      5.  6.100 /  5.  6.100
>   libswresample   3.  6.100 /  3.  6.100
>   libpostproc    55.  6.100 / 55.  6.100
> Input #0, matroska,webm, from '01ffv1.mkv':
>   Metadata:
>     ENCODER         : Lavf58.35.101
>   Duration: 00:15:04.40, start: 0.000000, bitrate: 322745 kb/s
>     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p10le, 1632x1200, SAR 1:1 DAR 34:25, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
>     Metadata:
>       ENCODER         : Lavc58.65.100 ffv1
>       DURATION        : 00:15:04.400000000
> Stream mapping:
>   Stream #0:0 -> #0:0 (ffv1 (native) -> rawvideo (native))
> Press [q] to stop, [?] for help
> Output #0, framemd5, to 'converted.framemd5':
>   Metadata:
>     encoder         : Lavf58.35.101
>     Stream #0:0: Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 979200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
>     Metadata:
>       DURATION        : 00:15:04.400000000
>       encoder         : Lavc58.65.100 rawvideo
> frame=  406 fps= 31 q=-0.0 Lsize=      32kB time=00:00:16.24 bitrate=  16.1kbits/s speed=1.23x
> video:3105900kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
>
> - Compare hashes
>
> orig.framemd5:
> #format: frame checksums
> #version: 2
> #hash: MD5
> #software: Lavf58.35.101
> #tb 0: 1/25
> #media_type 0: video
> #codec_id 0: rawvideo
> #dimensions 0: 1632x1200
> #sar 0: 1/1
> #stream#, dts,        pts, duration,     size, hash
> 0,          0,          0,        1,  7833600, 39d64300c8b749b0776423a80746d8dc
> 0,          1,          1,        1,  7833600, 12e7317d49727740187d2e23439d87d8
> 0,          2,          2,        1,  7833600, 12e7317d49727740187d2e23439d87d8
> 0,          3,          3,        1,  7833600, 0b3b0136b008ec9d67e900595be0c923
> 0,          4,          4,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
> 0,          5,          5,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
> 0,          6,          6,        1,  7833600, 1632e4b3ba7dc28e9dde698f3b7ec9e7
> 0,          7,          7,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
> 0,          8,          8,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
> 0,          9,          9,        1,  7833600, 612c13b4824bd427faa51efa7a09660d
> ...
>
> converted.framemd5:
> #format: frame checksums
> #version: 2
> #hash: MD5
> #software: Lavf58.35.101
> #tb 0: 1/25
> #media_type 0: video
> #codec_id 0: rawvideo
> #dimensions 0: 1632x1200
> #sar 0: 1/1
> #stream#, dts,        pts, duration,     size, hash
> 0,          0,          0,        1,  7833600, 3ed693fc02ba36cca92a2153afa960ac
> 0,          1,          1,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
> 0,          2,          2,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
> 0,          3,          3,        1,  7833600, 1fc392823ebc14001ee0e44c68d710e6
> 0,          4,          4,        1,  7833600, d63335f1c34132680e571549da51eac7
> 0,          5,          5,        1,  7833600, d63335f1c34132680e571549da51eac7
> 0,          6,          6,        1,  7833600, 09304bca4578458c353ecc119de0bd87
> 0,          7,          7,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
> 0,          8,          8,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
> 0,          9,          9,        1,  7833600, 78b22236d03517e5ceea3bd390ac2d0b
> ...
>
> Any idea of why this is happening?

The input has 10 bit precision, therefore only 10 bit are encoded.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Lossless encoding to ffv1 changes frames md5

Carl Eugen Hoyos-2
Am Di., 17. Dez. 2019 um 14:26 Uhr schrieb Carl Eugen Hoyos
<[hidden email]>:

>
> Am Di., 17. Dez. 2019 um 14:18 Uhr schrieb Alberto Mattea <[hidden email]>:
> >
> > Hi all, I'm trying to losslessly convert some videos from grassvalley hqx to ffv1.
> > Unfortunately it seems the conversion is not really lossless, as the frames md5 changes after the conversion.
> > Here's what I'm doing:
> >
> > - Calculate md5 of the original video frames
> >
> > $ ffmpeg -i 01.avi -f framemd5 orig.framemd5
> > ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
> >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> >   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
> >   libavutil      56. 36.101 / 56. 36.101
> >   libavcodec     58. 65.100 / 58. 65.100
> >   libavformat    58. 35.101 / 58. 35.101
> >   libavdevice    58.  9.101 / 58.  9.101
> >   libavfilter     7. 68.100 /  7. 68.100
> >   libswscale      5.  6.100 /  5.  6.100
> >   libswresample   3.  6.100 /  3.  6.100
> >   libpostproc    55.  6.100 / 55.  6.100
> > Input #0, avi, from '01.avi':
> >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
> >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
> > Stream mapping:
> >   Stream #0:0 -> #0:0 (hqx (native) -> rawvideo (native))
> > Press [q] to stop, [?] for help
> > Output #0, framemd5, to 'orig.framemd5':
> >   Metadata:
> >     encoder         : Lavf58.35.101
> >     Stream #0:0: Video: rawvideo (Y3[10][16] / 0x100A3359), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 1566720 kb/s, 25 fps, 25 tbn, 25 tbc
> >     Metadata:
> >       encoder         : Lavc58.65.100 rawvideo
> > frame=22610 fps= 50 q=-0.0 Lsize=    1767kB time=00:15:04.40 bitrate=  16.0kbits/s speed=1.98x
> > video:172966500kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
> >
> > - Convert
> >
> > $ ffmpeg -i 01.avi -c:v ffv1 01ffv1.mkv
> > ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
> >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> >   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
> >   libavutil      56. 36.101 / 56. 36.101
> >   libavcodec     58. 65.100 / 58. 65.100
> >   libavformat    58. 35.101 / 58. 35.101
> >   libavdevice    58.  9.101 / 58.  9.101
> >   libavfilter     7. 68.100 /  7. 68.100
> >   libswscale      5.  6.100 /  5.  6.100
> >   libswresample   3.  6.100 /  3.  6.100
> >   libpostproc    55.  6.100 / 55.  6.100
> > Input #0, avi, from '01.avi':
> >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
> >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
> > Stream mapping:
> >   Stream #0:0 -> #0:0 (hqx (native) -> ffv1 (native))
> > Press [q] to stop, [?] for help
> > [ffv1 @ 0x6e1ce80] bits_per_raw_sample > 8, forcing range coder
> > Output #0, matroska, to '01ffv1.mkv':
> >   Metadata:
> >     encoder         : Lavf58.35.101
> >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 200 kb/s, 25 fps, 1k tbn, 25 tbc
> >     Metadata:
> >       encoder         : Lavc58.65.100 ffv1
> > frame=22610 fps= 17 q=-0.0 Lsize=35631195kB time=00:15:04.36 bitrate=322759.1kbits/s speed=0.691x
> > video:35630872kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000908%
> >
> > - Calculate md5 of the converted frames (I've stopped it after a few hundred frames, just to check the beginning)
> >
> > $ ffmpeg -i 01ffv1.mkv -f framemd5 converted.framemd5
> > ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
> >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> >   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
> >   libavutil      56. 36.101 / 56. 36.101
> >   libavcodec     58. 65.100 / 58. 65.100
> >   libavformat    58. 35.101 / 58. 35.101
> >   libavdevice    58.  9.101 / 58.  9.101
> >   libavfilter     7. 68.100 /  7. 68.100
> >   libswscale      5.  6.100 /  5.  6.100
> >   libswresample   3.  6.100 /  3.  6.100
> >   libpostproc    55.  6.100 / 55.  6.100
> > Input #0, matroska,webm, from '01ffv1.mkv':
> >   Metadata:
> >     ENCODER         : Lavf58.35.101
> >   Duration: 00:15:04.40, start: 0.000000, bitrate: 322745 kb/s
> >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p10le, 1632x1200, SAR 1:1 DAR 34:25, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
> >     Metadata:
> >       ENCODER         : Lavc58.65.100 ffv1
> >       DURATION        : 00:15:04.400000000
> > Stream mapping:
> >   Stream #0:0 -> #0:0 (ffv1 (native) -> rawvideo (native))
> > Press [q] to stop, [?] for help
> > Output #0, framemd5, to 'converted.framemd5':
> >   Metadata:
> >     encoder         : Lavf58.35.101
> >     Stream #0:0: Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 979200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
> >     Metadata:
> >       DURATION        : 00:15:04.400000000
> >       encoder         : Lavc58.65.100 rawvideo
> > frame=  406 fps= 31 q=-0.0 Lsize=      32kB time=00:00:16.24 bitrate=  16.1kbits/s speed=1.23x
> > video:3105900kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
> >
> > - Compare hashes
> >
> > orig.framemd5:
> > #format: frame checksums
> > #version: 2
> > #hash: MD5
> > #software: Lavf58.35.101
> > #tb 0: 1/25
> > #media_type 0: video
> > #codec_id 0: rawvideo
> > #dimensions 0: 1632x1200
> > #sar 0: 1/1
> > #stream#, dts,        pts, duration,     size, hash
> > 0,          0,          0,        1,  7833600, 39d64300c8b749b0776423a80746d8dc
> > 0,          1,          1,        1,  7833600, 12e7317d49727740187d2e23439d87d8
> > 0,          2,          2,        1,  7833600, 12e7317d49727740187d2e23439d87d8
> > 0,          3,          3,        1,  7833600, 0b3b0136b008ec9d67e900595be0c923
> > 0,          4,          4,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
> > 0,          5,          5,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
> > 0,          6,          6,        1,  7833600, 1632e4b3ba7dc28e9dde698f3b7ec9e7
> > 0,          7,          7,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
> > 0,          8,          8,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
> > 0,          9,          9,        1,  7833600, 612c13b4824bd427faa51efa7a09660d
> > ...
> >
> > converted.framemd5:
> > #format: frame checksums
> > #version: 2
> > #hash: MD5
> > #software: Lavf58.35.101
> > #tb 0: 1/25
> > #media_type 0: video
> > #codec_id 0: rawvideo
> > #dimensions 0: 1632x1200
> > #sar 0: 1/1
> > #stream#, dts,        pts, duration,     size, hash
> > 0,          0,          0,        1,  7833600, 3ed693fc02ba36cca92a2153afa960ac
> > 0,          1,          1,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
> > 0,          2,          2,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
> > 0,          3,          3,        1,  7833600, 1fc392823ebc14001ee0e44c68d710e6
> > 0,          4,          4,        1,  7833600, d63335f1c34132680e571549da51eac7
> > 0,          5,          5,        1,  7833600, d63335f1c34132680e571549da51eac7
> > 0,          6,          6,        1,  7833600, 09304bca4578458c353ecc119de0bd87
> > 0,          7,          7,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
> > 0,          8,          8,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
> > 0,          9,          9,        1,  7833600, 78b22236d03517e5ceea3bd390ac2d0b
> > ...
> >
> > Any idea of why this is happening?
>
> The input has 10 bit precision, therefore only 10 bit are encoded.

The more difficult question is how big hqx precision really is:
The original commit specified 10 bit, but a comment in the code (same commit)
indicates 12 bit.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Lossless encoding to ffv1 changes frames md5

Alberto Mattea
In data martedì 17 dicembre 2019 14:33:47 CET, Carl Eugen Hoyos ha scritto:

> Am Di., 17. Dez. 2019 um 14:26 Uhr schrieb Carl Eugen Hoyos
> <[hidden email]>:
> >
> > Am Di., 17. Dez. 2019 um 14:18 Uhr schrieb Alberto Mattea <[hidden email]>:
> > >
> > > Hi all, I'm trying to losslessly convert some videos from grassvalley hqx to ffv1.
> > > Unfortunately it seems the conversion is not really lossless, as the frames md5 changes after the conversion.
> > > Here's what I'm doing:
> > >
> > > - Calculate md5 of the original video frames
> > >
> > > $ ffmpeg -i 01.avi -f framemd5 orig.framemd5
> > > ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
> > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> > >   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
> > >   libavutil      56. 36.101 / 56. 36.101
> > >   libavcodec     58. 65.100 / 58. 65.100
> > >   libavformat    58. 35.101 / 58. 35.101
> > >   libavdevice    58.  9.101 / 58.  9.101
> > >   libavfilter     7. 68.100 /  7. 68.100
> > >   libswscale      5.  6.100 /  5.  6.100
> > >   libswresample   3.  6.100 /  3.  6.100
> > >   libpostproc    55.  6.100 / 55.  6.100
> > > Input #0, avi, from '01.avi':
> > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
> > >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
> > > Stream mapping:
> > >   Stream #0:0 -> #0:0 (hqx (native) -> rawvideo (native))
> > > Press [q] to stop, [?] for help
> > > Output #0, framemd5, to 'orig.framemd5':
> > >   Metadata:
> > >     encoder         : Lavf58.35.101
> > >     Stream #0:0: Video: rawvideo (Y3[10][16] / 0x100A3359), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 1566720 kb/s, 25 fps, 25 tbn, 25 tbc
> > >     Metadata:
> > >       encoder         : Lavc58.65.100 rawvideo
> > > frame=22610 fps= 50 q=-0.0 Lsize=    1767kB time=00:15:04.40 bitrate=  16.0kbits/s speed=1.98x
> > > video:172966500kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
> > >
> > > - Convert
> > >
> > > $ ffmpeg -i 01.avi -c:v ffv1 01ffv1.mkv
> > > ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
> > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> > >   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
> > >   libavutil      56. 36.101 / 56. 36.101
> > >   libavcodec     58. 65.100 / 58. 65.100
> > >   libavformat    58. 35.101 / 58. 35.101
> > >   libavdevice    58.  9.101 / 58.  9.101
> > >   libavfilter     7. 68.100 /  7. 68.100
> > >   libswscale      5.  6.100 /  5.  6.100
> > >   libswresample   3.  6.100 /  3.  6.100
> > >   libpostproc    55.  6.100 / 55.  6.100
> > > Input #0, avi, from '01.avi':
> > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
> > >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
> > > Stream mapping:
> > >   Stream #0:0 -> #0:0 (hqx (native) -> ffv1 (native))
> > > Press [q] to stop, [?] for help
> > > [ffv1 @ 0x6e1ce80] bits_per_raw_sample > 8, forcing range coder
> > > Output #0, matroska, to '01ffv1.mkv':
> > >   Metadata:
> > >     encoder         : Lavf58.35.101
> > >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 200 kb/s, 25 fps, 1k tbn, 25 tbc
> > >     Metadata:
> > >       encoder         : Lavc58.65.100 ffv1
> > > frame=22610 fps= 17 q=-0.0 Lsize=35631195kB time=00:15:04.36 bitrate=322759.1kbits/s speed=0.691x
> > > video:35630872kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000908%
> > >
> > > - Calculate md5 of the converted frames (I've stopped it after a few hundred frames, just to check the beginning)
> > >
> > > $ ffmpeg -i 01ffv1.mkv -f framemd5 converted.framemd5
> > > ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
> > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> > >   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
> > >   libavutil      56. 36.101 / 56. 36.101
> > >   libavcodec     58. 65.100 / 58. 65.100
> > >   libavformat    58. 35.101 / 58. 35.101
> > >   libavdevice    58.  9.101 / 58.  9.101
> > >   libavfilter     7. 68.100 /  7. 68.100
> > >   libswscale      5.  6.100 /  5.  6.100
> > >   libswresample   3.  6.100 /  3.  6.100
> > >   libpostproc    55.  6.100 / 55.  6.100
> > > Input #0, matroska,webm, from '01ffv1.mkv':
> > >   Metadata:
> > >     ENCODER         : Lavf58.35.101
> > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 322745 kb/s
> > >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p10le, 1632x1200, SAR 1:1 DAR 34:25, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
> > >     Metadata:
> > >       ENCODER         : Lavc58.65.100 ffv1
> > >       DURATION        : 00:15:04.400000000
> > > Stream mapping:
> > >   Stream #0:0 -> #0:0 (ffv1 (native) -> rawvideo (native))
> > > Press [q] to stop, [?] for help
> > > Output #0, framemd5, to 'converted.framemd5':
> > >   Metadata:
> > >     encoder         : Lavf58.35.101
> > >     Stream #0:0: Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 979200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
> > >     Metadata:
> > >       DURATION        : 00:15:04.400000000
> > >       encoder         : Lavc58.65.100 rawvideo
> > > frame=  406 fps= 31 q=-0.0 Lsize=      32kB time=00:00:16.24 bitrate=  16.1kbits/s speed=1.23x
> > > video:3105900kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
> > >
> > > - Compare hashes
> > >
> > > orig.framemd5:
> > > #format: frame checksums
> > > #version: 2
> > > #hash: MD5
> > > #software: Lavf58.35.101
> > > #tb 0: 1/25
> > > #media_type 0: video
> > > #codec_id 0: rawvideo
> > > #dimensions 0: 1632x1200
> > > #sar 0: 1/1
> > > #stream#, dts,        pts, duration,     size, hash
> > > 0,          0,          0,        1,  7833600, 39d64300c8b749b0776423a80746d8dc
> > > 0,          1,          1,        1,  7833600, 12e7317d49727740187d2e23439d87d8
> > > 0,          2,          2,        1,  7833600, 12e7317d49727740187d2e23439d87d8
> > > 0,          3,          3,        1,  7833600, 0b3b0136b008ec9d67e900595be0c923
> > > 0,          4,          4,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
> > > 0,          5,          5,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
> > > 0,          6,          6,        1,  7833600, 1632e4b3ba7dc28e9dde698f3b7ec9e7
> > > 0,          7,          7,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
> > > 0,          8,          8,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
> > > 0,          9,          9,        1,  7833600, 612c13b4824bd427faa51efa7a09660d
> > > ...
> > >
> > > converted.framemd5:
> > > #format: frame checksums
> > > #version: 2
> > > #hash: MD5
> > > #software: Lavf58.35.101
> > > #tb 0: 1/25
> > > #media_type 0: video
> > > #codec_id 0: rawvideo
> > > #dimensions 0: 1632x1200
> > > #sar 0: 1/1
> > > #stream#, dts,        pts, duration,     size, hash
> > > 0,          0,          0,        1,  7833600, 3ed693fc02ba36cca92a2153afa960ac
> > > 0,          1,          1,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
> > > 0,          2,          2,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
> > > 0,          3,          3,        1,  7833600, 1fc392823ebc14001ee0e44c68d710e6
> > > 0,          4,          4,        1,  7833600, d63335f1c34132680e571549da51eac7
> > > 0,          5,          5,        1,  7833600, d63335f1c34132680e571549da51eac7
> > > 0,          6,          6,        1,  7833600, 09304bca4578458c353ecc119de0bd87
> > > 0,          7,          7,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
> > > 0,          8,          8,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
> > > 0,          9,          9,        1,  7833600, 78b22236d03517e5ceea3bd390ac2d0b
> > > ...
> > >
> > > Any idea of why this is happening?
> >
> > The input has 10 bit precision, therefore only 10 bit are encoded.
>
> The more difficult question is how big hqx precision really is:
> The original commit specified 10 bit, but a comment in the code (same commit)
> indicates 12 bit.
>
> Carl Eugen

Hey, thanks for the instant answer.
So if I understand correctly no information is being lost, it's just that the 10 bits are represented over <10 bits plus 6 bits of padding> instead of over <16-bits where the least significant 6 bits always happen to be zero>. Is it correct?
I'm assuming there is padding since if I extract a raw yuv frame from the hqx and the ffv1 they are both the size you would expect if they were 16-bit.

Alberto




_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Lossless encoding to ffv1 changes frames md5

Paul B Mahol
On 12/17/19, Alberto Mattea <[hidden email]> wrote:

> In data martedì 17 dicembre 2019 14:33:47 CET, Carl Eugen Hoyos ha scritto:
>> Am Di., 17. Dez. 2019 um 14:26 Uhr schrieb Carl Eugen Hoyos
>> <[hidden email]>:
>> >
>> > Am Di., 17. Dez. 2019 um 14:18 Uhr schrieb Alberto Mattea
>> > <[hidden email]>:
>> > >
>> > > Hi all, I'm trying to losslessly convert some videos from grassvalley
>> > > hqx to ffv1.
>> > > Unfortunately it seems the conversion is not really lossless, as the
>> > > frames md5 changes after the conversion.
>> > > Here's what I'm doing:
>> > >
>> > > - Calculate md5 of the original video frames
>> > >
>> > > $ ffmpeg -i 01.avi -f framemd5 orig.framemd5
>> > > ffmpeg version N-51066-ge52070e89c-static
>> > > https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg
>> > > developers
>> > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
>> > >   configuration: --enable-gpl --enable-version3 --enable-static
>> > > --disable-debug --disable-ffplay --disable-indev=sndio
>> > > --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r
>> > > --enable-gnutls --enable-gmp --enable-libgme --enable-gray
>> > > --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf
>> > > --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb
>> > > --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband
>> > > --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis
>> > > --enable-libopus --enable-libtheora --enable-libvidstab
>> > > --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp
>> > > --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d
>> > > --enable-libxvid --enable-libzvbi --enable-libzimg
>> > >   libavutil      56. 36.101 / 56. 36.101
>> > >   libavcodec     58. 65.100 / 58. 65.100
>> > >   libavformat    58. 35.101 / 58. 35.101
>> > >   libavdevice    58.  9.101 / 58.  9.101
>> > >   libavfilter     7. 68.100 /  7. 68.100
>> > >   libswscale      5.  6.100 /  5.  6.100
>> > >   libswresample   3.  6.100 /  3.  6.100
>> > >   libpostproc    55.  6.100 / 55.  6.100
>> > > Input #0, avi, from '01.avi':
>> > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
>> > >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc,
>> > > progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25
>> > > tbr, 25 tbn, 25 tbc
>> > > Stream mapping:
>> > >   Stream #0:0 -> #0:0 (hqx (native) -> rawvideo (native))
>> > > Press [q] to stop, [?] for help
>> > > Output #0, framemd5, to 'orig.framemd5':
>> > >   Metadata:
>> > >     encoder         : Lavf58.35.101
>> > >     Stream #0:0: Video: rawvideo (Y3[10][16] / 0x100A3359),
>> > > yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 1566720
>> > > kb/s, 25 fps, 25 tbn, 25 tbc
>> > >     Metadata:
>> > >       encoder         : Lavc58.65.100 rawvideo
>> > > frame=22610 fps= 50 q=-0.0 Lsize=    1767kB time=00:15:04.40 bitrate=
>> > > 16.0kbits/s speed=1.98x
>> > > video:172966500kB audio:0kB subtitle:0kB other streams:0kB global
>> > > headers:0kB muxing overhead: unknown
>> > >
>> > > - Convert
>> > >
>> > > $ ffmpeg -i 01.avi -c:v ffv1 01ffv1.mkv
>> > > ffmpeg version N-51066-ge52070e89c-static
>> > > https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg
>> > > developers
>> > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
>> > >   configuration: --enable-gpl --enable-version3 --enable-static
>> > > --disable-debug --disable-ffplay --disable-indev=sndio
>> > > --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r
>> > > --enable-gnutls --enable-gmp --enable-libgme --enable-gray
>> > > --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf
>> > > --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb
>> > > --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband
>> > > --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis
>> > > --enable-libopus --enable-libtheora --enable-libvidstab
>> > > --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp
>> > > --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d
>> > > --enable-libxvid --enable-libzvbi --enable-libzimg
>> > >   libavutil      56. 36.101 / 56. 36.101
>> > >   libavcodec     58. 65.100 / 58. 65.100
>> > >   libavformat    58. 35.101 / 58. 35.101
>> > >   libavdevice    58.  9.101 / 58.  9.101
>> > >   libavfilter     7. 68.100 /  7. 68.100
>> > >   libswscale      5.  6.100 /  5.  6.100
>> > >   libswresample   3.  6.100 /  3.  6.100
>> > >   libpostproc    55.  6.100 / 55.  6.100
>> > > Input #0, avi, from '01.avi':
>> > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
>> > >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc,
>> > > progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25
>> > > tbr, 25 tbn, 25 tbc
>> > > Stream mapping:
>> > >   Stream #0:0 -> #0:0 (hqx (native) -> ffv1 (native))
>> > > Press [q] to stop, [?] for help
>> > > [ffv1 @ 0x6e1ce80] bits_per_raw_sample > 8, forcing range coder
>> > > Output #0, matroska, to '01ffv1.mkv':
>> > >   Metadata:
>> > >     encoder         : Lavf58.35.101
>> > >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p16le(10 bpc),
>> > > 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 200 kb/s, 25 fps, 1k tbn, 25
>> > > tbc
>> > >     Metadata:
>> > >       encoder         : Lavc58.65.100 ffv1
>> > > frame=22610 fps= 17 q=-0.0 Lsize=35631195kB time=00:15:04.36
>> > > bitrate=322759.1kbits/s speed=0.691x
>> > > video:35630872kB audio:0kB subtitle:0kB other streams:0kB global
>> > > headers:0kB muxing overhead: 0.000908%
>> > >
>> > > - Calculate md5 of the converted frames (I've stopped it after a few
>> > > hundred frames, just to check the beginning)
>> > >
>> > > $ ffmpeg -i 01ffv1.mkv -f framemd5 converted.framemd5
>> > > ffmpeg version N-51066-ge52070e89c-static
>> > > https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg
>> > > developers
>> > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
>> > >   configuration: --enable-gpl --enable-version3 --enable-static
>> > > --disable-debug --disable-ffplay --disable-indev=sndio
>> > > --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r
>> > > --enable-gnutls --enable-gmp --enable-libgme --enable-gray
>> > > --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf
>> > > --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb
>> > > --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband
>> > > --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis
>> > > --enable-libopus --enable-libtheora --enable-libvidstab
>> > > --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp
>> > > --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d
>> > > --enable-libxvid --enable-libzvbi --enable-libzimg
>> > >   libavutil      56. 36.101 / 56. 36.101
>> > >   libavcodec     58. 65.100 / 58. 65.100
>> > >   libavformat    58. 35.101 / 58. 35.101
>> > >   libavdevice    58.  9.101 / 58.  9.101
>> > >   libavfilter     7. 68.100 /  7. 68.100
>> > >   libswscale      5.  6.100 /  5.  6.100
>> > >   libswresample   3.  6.100 /  3.  6.100
>> > >   libpostproc    55.  6.100 / 55.  6.100
>> > > Input #0, matroska,webm, from '01ffv1.mkv':
>> > >   Metadata:
>> > >     ENCODER         : Lavf58.35.101
>> > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 322745 kb/s
>> > >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p10le,
>> > > 1632x1200, SAR 1:1 DAR 34:25, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
>> > >     Metadata:
>> > >       ENCODER         : Lavc58.65.100 ffv1
>> > >       DURATION        : 00:15:04.400000000
>> > > Stream mapping:
>> > >   Stream #0:0 -> #0:0 (ffv1 (native) -> rawvideo (native))
>> > > Press [q] to stop, [?] for help
>> > > Output #0, framemd5, to 'converted.framemd5':
>> > >   Metadata:
>> > >     encoder         : Lavf58.35.101
>> > >     Stream #0:0: Video: rawvideo (Y3[10][10] / 0xA0A3359),
>> > > yuv422p10le, 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 979200 kb/s, 25
>> > > fps, 25 tbn, 25 tbc (default)
>> > >     Metadata:
>> > >       DURATION        : 00:15:04.400000000
>> > >       encoder         : Lavc58.65.100 rawvideo
>> > > frame=  406 fps= 31 q=-0.0 Lsize=      32kB time=00:00:16.24 bitrate=
>> > > 16.1kbits/s speed=1.23x
>> > > video:3105900kB audio:0kB subtitle:0kB other streams:0kB global
>> > > headers:0kB muxing overhead: unknown
>> > >
>> > > - Compare hashes
>> > >
>> > > orig.framemd5:
>> > > #format: frame checksums
>> > > #version: 2
>> > > #hash: MD5
>> > > #software: Lavf58.35.101
>> > > #tb 0: 1/25
>> > > #media_type 0: video
>> > > #codec_id 0: rawvideo
>> > > #dimensions 0: 1632x1200
>> > > #sar 0: 1/1
>> > > #stream#, dts,        pts, duration,     size, hash
>> > > 0,          0,          0,        1,  7833600,
>> > > 39d64300c8b749b0776423a80746d8dc
>> > > 0,          1,          1,        1,  7833600,
>> > > 12e7317d49727740187d2e23439d87d8
>> > > 0,          2,          2,        1,  7833600,
>> > > 12e7317d49727740187d2e23439d87d8
>> > > 0,          3,          3,        1,  7833600,
>> > > 0b3b0136b008ec9d67e900595be0c923
>> > > 0,          4,          4,        1,  7833600,
>> > > 97cd9b7689d1f49d1e1988a3613ffa72
>> > > 0,          5,          5,        1,  7833600,
>> > > 97cd9b7689d1f49d1e1988a3613ffa72
>> > > 0,          6,          6,        1,  7833600,
>> > > 1632e4b3ba7dc28e9dde698f3b7ec9e7
>> > > 0,          7,          7,        1,  7833600,
>> > > 796bda137d768f4ad31eb7166ea16447
>> > > 0,          8,          8,        1,  7833600,
>> > > 796bda137d768f4ad31eb7166ea16447
>> > > 0,          9,          9,        1,  7833600,
>> > > 612c13b4824bd427faa51efa7a09660d
>> > > ...
>> > >
>> > > converted.framemd5:
>> > > #format: frame checksums
>> > > #version: 2
>> > > #hash: MD5
>> > > #software: Lavf58.35.101
>> > > #tb 0: 1/25
>> > > #media_type 0: video
>> > > #codec_id 0: rawvideo
>> > > #dimensions 0: 1632x1200
>> > > #sar 0: 1/1
>> > > #stream#, dts,        pts, duration,     size, hash
>> > > 0,          0,          0,        1,  7833600,
>> > > 3ed693fc02ba36cca92a2153afa960ac
>> > > 0,          1,          1,        1,  7833600,
>> > > 23d7482899eb8c854b26444dcdf38db4
>> > > 0,          2,          2,        1,  7833600,
>> > > 23d7482899eb8c854b26444dcdf38db4
>> > > 0,          3,          3,        1,  7833600,
>> > > 1fc392823ebc14001ee0e44c68d710e6
>> > > 0,          4,          4,        1,  7833600,
>> > > d63335f1c34132680e571549da51eac7
>> > > 0,          5,          5,        1,  7833600,
>> > > d63335f1c34132680e571549da51eac7
>> > > 0,          6,          6,        1,  7833600,
>> > > 09304bca4578458c353ecc119de0bd87
>> > > 0,          7,          7,        1,  7833600,
>> > > 0ec1e49008d1f70385dc4a1636559a30
>> > > 0,          8,          8,        1,  7833600,
>> > > 0ec1e49008d1f70385dc4a1636559a30
>> > > 0,          9,          9,        1,  7833600,
>> > > 78b22236d03517e5ceea3bd390ac2d0b
>> > > ...
>> > >
>> > > Any idea of why this is happening?
>> >
>> > The input has 10 bit precision, therefore only 10 bit are encoded.
>>
>> The more difficult question is how big hqx precision really is:
>> The original commit specified 10 bit, but a comment in the code (same
>> commit)
>> indicates 12 bit.
>>
>> Carl Eugen
>
> Hey, thanks for the instant answer.
> So if I understand correctly no information is being lost, it's just that
> the 10 bits are represented over <10 bits plus 6 bits of padding> instead of
> over <16-bits where the least significant 6 bits always happen to be zero>.
> Is it correct?
> I'm assuming there is padding since if I extract a raw yuv frame from the
> hqx and the ffv1 they are both the size you would expect if they were
> 16-bit.

Thing is decoder's output should be changed so it will be 12/10 bit
pixel format.
Instead of using 16bit format. Assuming 4 upper bits are not having useful info.

>
> Alberto
>
>
>
>
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Lossless encoding to ffv1 changes frames md5

Paul B Mahol
In reply to this post by Carl Eugen Hoyos-2
On 12/17/19, Carl Eugen Hoyos <[hidden email]> wrote:

> Am Di., 17. Dez. 2019 um 14:26 Uhr schrieb Carl Eugen Hoyos
> <[hidden email]>:
>>
>> Am Di., 17. Dez. 2019 um 14:18 Uhr schrieb Alberto Mattea
>> <[hidden email]>:
>> >
>> > Hi all, I'm trying to losslessly convert some videos from grassvalley
>> > hqx to ffv1.
>> > Unfortunately it seems the conversion is not really lossless, as the
>> > frames md5 changes after the conversion.
>> > Here's what I'm doing:
>> >
>> > - Calculate md5 of the original video frames
>> >
>> > $ ffmpeg -i 01.avi -f framemd5 orig.framemd5
>> > ffmpeg version N-51066-ge52070e89c-static
>> > https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg
>> > developers
>> >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
>> >   configuration: --enable-gpl --enable-version3 --enable-static
>> > --disable-debug --disable-ffplay --disable-indev=sndio
>> > --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r
>> > --enable-gnutls --enable-gmp --enable-libgme --enable-gray
>> > --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf
>> > --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb
>> > --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband
>> > --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis
>> > --enable-libopus --enable-libtheora --enable-libvidstab
>> > --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp
>> > --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d
>> > --enable-libxvid --enable-libzvbi --enable-libzimg
>> >   libavutil      56. 36.101 / 56. 36.101
>> >   libavcodec     58. 65.100 / 58. 65.100
>> >   libavformat    58. 35.101 / 58. 35.101
>> >   libavdevice    58.  9.101 / 58.  9.101
>> >   libavfilter     7. 68.100 /  7. 68.100
>> >   libswscale      5.  6.100 /  5.  6.100
>> >   libswresample   3.  6.100 /  3.  6.100
>> >   libpostproc    55.  6.100 / 55.  6.100
>> > Input #0, avi, from '01.avi':
>> >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
>> >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc,
>> > progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25
>> > tbr, 25 tbn, 25 tbc
>> > Stream mapping:
>> >   Stream #0:0 -> #0:0 (hqx (native) -> rawvideo (native))
>> > Press [q] to stop, [?] for help
>> > Output #0, framemd5, to 'orig.framemd5':
>> >   Metadata:
>> >     encoder         : Lavf58.35.101
>> >     Stream #0:0: Video: rawvideo (Y3[10][16] / 0x100A3359),
>> > yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 1566720
>> > kb/s, 25 fps, 25 tbn, 25 tbc
>> >     Metadata:
>> >       encoder         : Lavc58.65.100 rawvideo
>> > frame=22610 fps= 50 q=-0.0 Lsize=    1767kB time=00:15:04.40 bitrate=
>> > 16.0kbits/s speed=1.98x
>> > video:172966500kB audio:0kB subtitle:0kB other streams:0kB global
>> > headers:0kB muxing overhead: unknown
>> >
>> > - Convert
>> >
>> > $ ffmpeg -i 01.avi -c:v ffv1 01ffv1.mkv
>> > ffmpeg version N-51066-ge52070e89c-static
>> > https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg
>> > developers
>> >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
>> >   configuration: --enable-gpl --enable-version3 --enable-static
>> > --disable-debug --disable-ffplay --disable-indev=sndio
>> > --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r
>> > --enable-gnutls --enable-gmp --enable-libgme --enable-gray
>> > --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf
>> > --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb
>> > --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband
>> > --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis
>> > --enable-libopus --enable-libtheora --enable-libvidstab
>> > --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp
>> > --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d
>> > --enable-libxvid --enable-libzvbi --enable-libzimg
>> >   libavutil      56. 36.101 / 56. 36.101
>> >   libavcodec     58. 65.100 / 58. 65.100
>> >   libavformat    58. 35.101 / 58. 35.101
>> >   libavdevice    58.  9.101 / 58.  9.101
>> >   libavfilter     7. 68.100 /  7. 68.100
>> >   libswscale      5.  6.100 /  5.  6.100
>> >   libswresample   3.  6.100 /  3.  6.100
>> >   libpostproc    55.  6.100 / 55.  6.100
>> > Input #0, avi, from '01.avi':
>> >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
>> >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc,
>> > progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25
>> > tbr, 25 tbn, 25 tbc
>> > Stream mapping:
>> >   Stream #0:0 -> #0:0 (hqx (native) -> ffv1 (native))
>> > Press [q] to stop, [?] for help
>> > [ffv1 @ 0x6e1ce80] bits_per_raw_sample > 8, forcing range coder
>> > Output #0, matroska, to '01ffv1.mkv':
>> >   Metadata:
>> >     encoder         : Lavf58.35.101
>> >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p16le(10 bpc),
>> > 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 200 kb/s, 25 fps, 1k tbn, 25 tbc
>> >     Metadata:
>> >       encoder         : Lavc58.65.100 ffv1
>> > frame=22610 fps= 17 q=-0.0 Lsize=35631195kB time=00:15:04.36
>> > bitrate=322759.1kbits/s speed=0.691x
>> > video:35630872kB audio:0kB subtitle:0kB other streams:0kB global
>> > headers:0kB muxing overhead: 0.000908%
>> >
>> > - Calculate md5 of the converted frames (I've stopped it after a few
>> > hundred frames, just to check the beginning)
>> >
>> > $ ffmpeg -i 01ffv1.mkv -f framemd5 converted.framemd5
>> > ffmpeg version N-51066-ge52070e89c-static
>> > https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg
>> > developers
>> >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
>> >   configuration: --enable-gpl --enable-version3 --enable-static
>> > --disable-debug --disable-ffplay --disable-indev=sndio
>> > --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r
>> > --enable-gnutls --enable-gmp --enable-libgme --enable-gray
>> > --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf
>> > --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb
>> > --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband
>> > --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis
>> > --enable-libopus --enable-libtheora --enable-libvidstab
>> > --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp
>> > --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d
>> > --enable-libxvid --enable-libzvbi --enable-libzimg
>> >   libavutil      56. 36.101 / 56. 36.101
>> >   libavcodec     58. 65.100 / 58. 65.100
>> >   libavformat    58. 35.101 / 58. 35.101
>> >   libavdevice    58.  9.101 / 58.  9.101
>> >   libavfilter     7. 68.100 /  7. 68.100
>> >   libswscale      5.  6.100 /  5.  6.100
>> >   libswresample   3.  6.100 /  3.  6.100
>> >   libpostproc    55.  6.100 / 55.  6.100
>> > Input #0, matroska,webm, from '01ffv1.mkv':
>> >   Metadata:
>> >     ENCODER         : Lavf58.35.101
>> >   Duration: 00:15:04.40, start: 0.000000, bitrate: 322745 kb/s
>> >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p10le,
>> > 1632x1200, SAR 1:1 DAR 34:25, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
>> >     Metadata:
>> >       ENCODER         : Lavc58.65.100 ffv1
>> >       DURATION        : 00:15:04.400000000
>> > Stream mapping:
>> >   Stream #0:0 -> #0:0 (ffv1 (native) -> rawvideo (native))
>> > Press [q] to stop, [?] for help
>> > Output #0, framemd5, to 'converted.framemd5':
>> >   Metadata:
>> >     encoder         : Lavf58.35.101
>> >     Stream #0:0: Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le,
>> > 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 979200 kb/s, 25 fps, 25 tbn, 25
>> > tbc (default)
>> >     Metadata:
>> >       DURATION        : 00:15:04.400000000
>> >       encoder         : Lavc58.65.100 rawvideo
>> > frame=  406 fps= 31 q=-0.0 Lsize=      32kB time=00:00:16.24 bitrate=
>> > 16.1kbits/s speed=1.23x
>> > video:3105900kB audio:0kB subtitle:0kB other streams:0kB global
>> > headers:0kB muxing overhead: unknown
>> >
>> > - Compare hashes
>> >
>> > orig.framemd5:
>> > #format: frame checksums
>> > #version: 2
>> > #hash: MD5
>> > #software: Lavf58.35.101
>> > #tb 0: 1/25
>> > #media_type 0: video
>> > #codec_id 0: rawvideo
>> > #dimensions 0: 1632x1200
>> > #sar 0: 1/1
>> > #stream#, dts,        pts, duration,     size, hash
>> > 0,          0,          0,        1,  7833600,
>> > 39d64300c8b749b0776423a80746d8dc
>> > 0,          1,          1,        1,  7833600,
>> > 12e7317d49727740187d2e23439d87d8
>> > 0,          2,          2,        1,  7833600,
>> > 12e7317d49727740187d2e23439d87d8
>> > 0,          3,          3,        1,  7833600,
>> > 0b3b0136b008ec9d67e900595be0c923
>> > 0,          4,          4,        1,  7833600,
>> > 97cd9b7689d1f49d1e1988a3613ffa72
>> > 0,          5,          5,        1,  7833600,
>> > 97cd9b7689d1f49d1e1988a3613ffa72
>> > 0,          6,          6,        1,  7833600,
>> > 1632e4b3ba7dc28e9dde698f3b7ec9e7
>> > 0,          7,          7,        1,  7833600,
>> > 796bda137d768f4ad31eb7166ea16447
>> > 0,          8,          8,        1,  7833600,
>> > 796bda137d768f4ad31eb7166ea16447
>> > 0,          9,          9,        1,  7833600,
>> > 612c13b4824bd427faa51efa7a09660d
>> > ...
>> >
>> > converted.framemd5:
>> > #format: frame checksums
>> > #version: 2
>> > #hash: MD5
>> > #software: Lavf58.35.101
>> > #tb 0: 1/25
>> > #media_type 0: video
>> > #codec_id 0: rawvideo
>> > #dimensions 0: 1632x1200
>> > #sar 0: 1/1
>> > #stream#, dts,        pts, duration,     size, hash
>> > 0,          0,          0,        1,  7833600,
>> > 3ed693fc02ba36cca92a2153afa960ac
>> > 0,          1,          1,        1,  7833600,
>> > 23d7482899eb8c854b26444dcdf38db4
>> > 0,          2,          2,        1,  7833600,
>> > 23d7482899eb8c854b26444dcdf38db4
>> > 0,          3,          3,        1,  7833600,
>> > 1fc392823ebc14001ee0e44c68d710e6
>> > 0,          4,          4,        1,  7833600,
>> > d63335f1c34132680e571549da51eac7
>> > 0,          5,          5,        1,  7833600,
>> > d63335f1c34132680e571549da51eac7
>> > 0,          6,          6,        1,  7833600,
>> > 09304bca4578458c353ecc119de0bd87
>> > 0,          7,          7,        1,  7833600,
>> > 0ec1e49008d1f70385dc4a1636559a30
>> > 0,          8,          8,        1,  7833600,
>> > 0ec1e49008d1f70385dc4a1636559a30
>> > 0,          9,          9,        1,  7833600,
>> > 78b22236d03517e5ceea3bd390ac2d0b
>> > ...
>> >
>> > Any idea of why this is happening?
>>
>> The input has 10 bit precision, therefore only 10 bit are encoded.
>
> The more difficult question is how big hqx precision really is:
> The original commit specified 10 bit, but a comment in the code (same
> commit)
> indicates 12 bit.

Idct output is padded by 4 bits with upper bits of 12bit idct output.

> Carl Eugen
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Lossless encoding to ffv1 changes frames md5

Carl Eugen Hoyos-2
In reply to this post by Alberto Mattea
Am Di., 17. Dez. 2019 um 14:48 Uhr schrieb Alberto Mattea <[hidden email]>:

>
> In data martedì 17 dicembre 2019 14:33:47 CET, Carl Eugen Hoyos ha scritto:
> > Am Di., 17. Dez. 2019 um 14:26 Uhr schrieb Carl Eugen Hoyos
> > <[hidden email]>:
> > >
> > > Am Di., 17. Dez. 2019 um 14:18 Uhr schrieb Alberto Mattea <[hidden email]>:
> > > >
> > > > Hi all, I'm trying to losslessly convert some videos from grassvalley hqx to ffv1.
> > > > Unfortunately it seems the conversion is not really lossless, as the frames md5 changes after the conversion.
> > > > Here's what I'm doing:
> > > >
> > > > - Calculate md5 of the original video frames
> > > >
> > > > $ ffmpeg -i 01.avi -f framemd5 orig.framemd5
> > > > ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
> > > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> > > >   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
> > > >   libavutil      56. 36.101 / 56. 36.101
> > > >   libavcodec     58. 65.100 / 58. 65.100
> > > >   libavformat    58. 35.101 / 58. 35.101
> > > >   libavdevice    58.  9.101 / 58.  9.101
> > > >   libavfilter     7. 68.100 /  7. 68.100
> > > >   libswscale      5.  6.100 /  5.  6.100
> > > >   libswresample   3.  6.100 /  3.  6.100
> > > >   libpostproc    55.  6.100 / 55.  6.100
> > > > Input #0, avi, from '01.avi':
> > > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
> > > >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
> > > > Stream mapping:
> > > >   Stream #0:0 -> #0:0 (hqx (native) -> rawvideo (native))
> > > > Press [q] to stop, [?] for help
> > > > Output #0, framemd5, to 'orig.framemd5':
> > > >   Metadata:
> > > >     encoder         : Lavf58.35.101
> > > >     Stream #0:0: Video: rawvideo (Y3[10][16] / 0x100A3359), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 1566720 kb/s, 25 fps, 25 tbn, 25 tbc
> > > >     Metadata:
> > > >       encoder         : Lavc58.65.100 rawvideo
> > > > frame=22610 fps= 50 q=-0.0 Lsize=    1767kB time=00:15:04.40 bitrate=  16.0kbits/s speed=1.98x
> > > > video:172966500kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
> > > >
> > > > - Convert
> > > >
> > > > $ ffmpeg -i 01.avi -c:v ffv1 01ffv1.mkv
> > > > ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
> > > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> > > >   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
> > > >   libavutil      56. 36.101 / 56. 36.101
> > > >   libavcodec     58. 65.100 / 58. 65.100
> > > >   libavformat    58. 35.101 / 58. 35.101
> > > >   libavdevice    58.  9.101 / 58.  9.101
> > > >   libavfilter     7. 68.100 /  7. 68.100
> > > >   libswscale      5.  6.100 /  5.  6.100
> > > >   libswresample   3.  6.100 /  3.  6.100
> > > >   libpostproc    55.  6.100 / 55.  6.100
> > > > Input #0, avi, from '01.avi':
> > > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
> > > >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
> > > > Stream mapping:
> > > >   Stream #0:0 -> #0:0 (hqx (native) -> ffv1 (native))
> > > > Press [q] to stop, [?] for help
> > > > [ffv1 @ 0x6e1ce80] bits_per_raw_sample > 8, forcing range coder
> > > > Output #0, matroska, to '01ffv1.mkv':
> > > >   Metadata:
> > > >     encoder         : Lavf58.35.101
> > > >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 200 kb/s, 25 fps, 1k tbn, 25 tbc
> > > >     Metadata:
> > > >       encoder         : Lavc58.65.100 ffv1
> > > > frame=22610 fps= 17 q=-0.0 Lsize=35631195kB time=00:15:04.36 bitrate=322759.1kbits/s speed=0.691x
> > > > video:35630872kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000908%
> > > >
> > > > - Calculate md5 of the converted frames (I've stopped it after a few hundred frames, just to check the beginning)
> > > >
> > > > $ ffmpeg -i 01ffv1.mkv -f framemd5 converted.framemd5
> > > > ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
> > > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> > > >   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
> > > >   libavutil      56. 36.101 / 56. 36.101
> > > >   libavcodec     58. 65.100 / 58. 65.100
> > > >   libavformat    58. 35.101 / 58. 35.101
> > > >   libavdevice    58.  9.101 / 58.  9.101
> > > >   libavfilter     7. 68.100 /  7. 68.100
> > > >   libswscale      5.  6.100 /  5.  6.100
> > > >   libswresample   3.  6.100 /  3.  6.100
> > > >   libpostproc    55.  6.100 / 55.  6.100
> > > > Input #0, matroska,webm, from '01ffv1.mkv':
> > > >   Metadata:
> > > >     ENCODER         : Lavf58.35.101
> > > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 322745 kb/s
> > > >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p10le, 1632x1200, SAR 1:1 DAR 34:25, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
> > > >     Metadata:
> > > >       ENCODER         : Lavc58.65.100 ffv1
> > > >       DURATION        : 00:15:04.400000000
> > > > Stream mapping:
> > > >   Stream #0:0 -> #0:0 (ffv1 (native) -> rawvideo (native))
> > > > Press [q] to stop, [?] for help
> > > > Output #0, framemd5, to 'converted.framemd5':
> > > >   Metadata:
> > > >     encoder         : Lavf58.35.101
> > > >     Stream #0:0: Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 979200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
> > > >     Metadata:
> > > >       DURATION        : 00:15:04.400000000
> > > >       encoder         : Lavc58.65.100 rawvideo
> > > > frame=  406 fps= 31 q=-0.0 Lsize=      32kB time=00:00:16.24 bitrate=  16.1kbits/s speed=1.23x
> > > > video:3105900kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
> > > >
> > > > - Compare hashes
> > > >
> > > > orig.framemd5:
> > > > #format: frame checksums
> > > > #version: 2
> > > > #hash: MD5
> > > > #software: Lavf58.35.101
> > > > #tb 0: 1/25
> > > > #media_type 0: video
> > > > #codec_id 0: rawvideo
> > > > #dimensions 0: 1632x1200
> > > > #sar 0: 1/1
> > > > #stream#, dts,        pts, duration,     size, hash
> > > > 0,          0,          0,        1,  7833600, 39d64300c8b749b0776423a80746d8dc
> > > > 0,          1,          1,        1,  7833600, 12e7317d49727740187d2e23439d87d8
> > > > 0,          2,          2,        1,  7833600, 12e7317d49727740187d2e23439d87d8
> > > > 0,          3,          3,        1,  7833600, 0b3b0136b008ec9d67e900595be0c923
> > > > 0,          4,          4,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
> > > > 0,          5,          5,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
> > > > 0,          6,          6,        1,  7833600, 1632e4b3ba7dc28e9dde698f3b7ec9e7
> > > > 0,          7,          7,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
> > > > 0,          8,          8,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
> > > > 0,          9,          9,        1,  7833600, 612c13b4824bd427faa51efa7a09660d
> > > > ...
> > > >
> > > > converted.framemd5:
> > > > #format: frame checksums
> > > > #version: 2
> > > > #hash: MD5
> > > > #software: Lavf58.35.101
> > > > #tb 0: 1/25
> > > > #media_type 0: video
> > > > #codec_id 0: rawvideo
> > > > #dimensions 0: 1632x1200
> > > > #sar 0: 1/1
> > > > #stream#, dts,        pts, duration,     size, hash
> > > > 0,          0,          0,        1,  7833600, 3ed693fc02ba36cca92a2153afa960ac
> > > > 0,          1,          1,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
> > > > 0,          2,          2,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
> > > > 0,          3,          3,        1,  7833600, 1fc392823ebc14001ee0e44c68d710e6
> > > > 0,          4,          4,        1,  7833600, d63335f1c34132680e571549da51eac7
> > > > 0,          5,          5,        1,  7833600, d63335f1c34132680e571549da51eac7
> > > > 0,          6,          6,        1,  7833600, 09304bca4578458c353ecc119de0bd87
> > > > 0,          7,          7,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
> > > > 0,          8,          8,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
> > > > 0,          9,          9,        1,  7833600, 78b22236d03517e5ceea3bd390ac2d0b
> > > > ...
> > > >
> > > > Any idea of why this is happening?
> > >
> > > The input has 10 bit precision, therefore only 10 bit are encoded.
> >
> > The more difficult question is how big hqx precision really is:
> > The original commit specified 10 bit, but a comment in the code (same commit)
> > indicates 12 bit.
> >
> > Carl Eugen
>
> Hey, thanks for the instant answer.
> So if I understand correctly no information is being lost, it's just that the 10 bits are
> represented over <10 bits plus 6 bits of padding> instead of over <16-bits where
> the least significant 6 bits always happen to be zero>. Is it correct?

I don't think they are zero (and this is not necessary for them to be
"no information").

> I'm assuming there is padding since if I extract a raw yuv frame from the hqx and the ffv1
> they are both the size you would expect if they were 16-bit.

But this is not necessarily related.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Lossless encoding to ffv1 changes frames md5

Alberto Mattea
In reply to this post by Paul B Mahol
In data martedì 17 dicembre 2019 14:53:46 CET, Paul B Mahol ha scritto:

> On 12/17/19, Alberto Mattea <[hidden email]> wrote:
> > In data martedì 17 dicembre 2019 14:33:47 CET, Carl Eugen Hoyos ha scritto:
> >> Am Di., 17. Dez. 2019 um 14:26 Uhr schrieb Carl Eugen Hoyos
> >> <[hidden email]>:
> >> >
> >> > Am Di., 17. Dez. 2019 um 14:18 Uhr schrieb Alberto Mattea
> >> > <[hidden email]>:
> >> > >
> >> > > Hi all, I'm trying to losslessly convert some videos from grassvalley
> >> > > hqx to ffv1.
> >> > > Unfortunately it seems the conversion is not really lossless, as the
> >> > > frames md5 changes after the conversion.
> >> > > Here's what I'm doing:
> >> > >
> >> > > - Calculate md5 of the original video frames
> >> > >
> >> > > $ ffmpeg -i 01.avi -f framemd5 orig.framemd5
> >> > > ffmpeg version N-51066-ge52070e89c-static
> >> > > https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg
> >> > > developers
> >> > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> >> > >   configuration: --enable-gpl --enable-version3 --enable-static
> >> > > --disable-debug --disable-ffplay --disable-indev=sndio
> >> > > --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r
> >> > > --enable-gnutls --enable-gmp --enable-libgme --enable-gray
> >> > > --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf
> >> > > --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb
> >> > > --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband
> >> > > --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis
> >> > > --enable-libopus --enable-libtheora --enable-libvidstab
> >> > > --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp
> >> > > --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d
> >> > > --enable-libxvid --enable-libzvbi --enable-libzimg
> >> > >   libavutil      56. 36.101 / 56. 36.101
> >> > >   libavcodec     58. 65.100 / 58. 65.100
> >> > >   libavformat    58. 35.101 / 58. 35.101
> >> > >   libavdevice    58.  9.101 / 58.  9.101
> >> > >   libavfilter     7. 68.100 /  7. 68.100
> >> > >   libswscale      5.  6.100 /  5.  6.100
> >> > >   libswresample   3.  6.100 /  3.  6.100
> >> > >   libpostproc    55.  6.100 / 55.  6.100
> >> > > Input #0, avi, from '01.avi':
> >> > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
> >> > >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc,
> >> > > progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25
> >> > > tbr, 25 tbn, 25 tbc
> >> > > Stream mapping:
> >> > >   Stream #0:0 -> #0:0 (hqx (native) -> rawvideo (native))
> >> > > Press [q] to stop, [?] for help
> >> > > Output #0, framemd5, to 'orig.framemd5':
> >> > >   Metadata:
> >> > >     encoder         : Lavf58.35.101
> >> > >     Stream #0:0: Video: rawvideo (Y3[10][16] / 0x100A3359),
> >> > > yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 1566720
> >> > > kb/s, 25 fps, 25 tbn, 25 tbc
> >> > >     Metadata:
> >> > >       encoder         : Lavc58.65.100 rawvideo
> >> > > frame=22610 fps= 50 q=-0.0 Lsize=    1767kB time=00:15:04.40 bitrate=
> >> > > 16.0kbits/s speed=1.98x
> >> > > video:172966500kB audio:0kB subtitle:0kB other streams:0kB global
> >> > > headers:0kB muxing overhead: unknown
> >> > >
> >> > > - Convert
> >> > >
> >> > > $ ffmpeg -i 01.avi -c:v ffv1 01ffv1.mkv
> >> > > ffmpeg version N-51066-ge52070e89c-static
> >> > > https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg
> >> > > developers
> >> > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> >> > >   configuration: --enable-gpl --enable-version3 --enable-static
> >> > > --disable-debug --disable-ffplay --disable-indev=sndio
> >> > > --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r
> >> > > --enable-gnutls --enable-gmp --enable-libgme --enable-gray
> >> > > --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf
> >> > > --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb
> >> > > --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband
> >> > > --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis
> >> > > --enable-libopus --enable-libtheora --enable-libvidstab
> >> > > --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp
> >> > > --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d
> >> > > --enable-libxvid --enable-libzvbi --enable-libzimg
> >> > >   libavutil      56. 36.101 / 56. 36.101
> >> > >   libavcodec     58. 65.100 / 58. 65.100
> >> > >   libavformat    58. 35.101 / 58. 35.101
> >> > >   libavdevice    58.  9.101 / 58.  9.101
> >> > >   libavfilter     7. 68.100 /  7. 68.100
> >> > >   libswscale      5.  6.100 /  5.  6.100
> >> > >   libswresample   3.  6.100 /  3.  6.100
> >> > >   libpostproc    55.  6.100 / 55.  6.100
> >> > > Input #0, avi, from '01.avi':
> >> > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
> >> > >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc,
> >> > > progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25
> >> > > tbr, 25 tbn, 25 tbc
> >> > > Stream mapping:
> >> > >   Stream #0:0 -> #0:0 (hqx (native) -> ffv1 (native))
> >> > > Press [q] to stop, [?] for help
> >> > > [ffv1 @ 0x6e1ce80] bits_per_raw_sample > 8, forcing range coder
> >> > > Output #0, matroska, to '01ffv1.mkv':
> >> > >   Metadata:
> >> > >     encoder         : Lavf58.35.101
> >> > >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p16le(10 bpc),
> >> > > 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 200 kb/s, 25 fps, 1k tbn, 25
> >> > > tbc
> >> > >     Metadata:
> >> > >       encoder         : Lavc58.65.100 ffv1
> >> > > frame=22610 fps= 17 q=-0.0 Lsize=35631195kB time=00:15:04.36
> >> > > bitrate=322759.1kbits/s speed=0.691x
> >> > > video:35630872kB audio:0kB subtitle:0kB other streams:0kB global
> >> > > headers:0kB muxing overhead: 0.000908%
> >> > >
> >> > > - Calculate md5 of the converted frames (I've stopped it after a few
> >> > > hundred frames, just to check the beginning)
> >> > >
> >> > > $ ffmpeg -i 01ffv1.mkv -f framemd5 converted.framemd5
> >> > > ffmpeg version N-51066-ge52070e89c-static
> >> > > https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg
> >> > > developers
> >> > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> >> > >   configuration: --enable-gpl --enable-version3 --enable-static
> >> > > --disable-debug --disable-ffplay --disable-indev=sndio
> >> > > --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r
> >> > > --enable-gnutls --enable-gmp --enable-libgme --enable-gray
> >> > > --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf
> >> > > --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb
> >> > > --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband
> >> > > --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis
> >> > > --enable-libopus --enable-libtheora --enable-libvidstab
> >> > > --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp
> >> > > --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d
> >> > > --enable-libxvid --enable-libzvbi --enable-libzimg
> >> > >   libavutil      56. 36.101 / 56. 36.101
> >> > >   libavcodec     58. 65.100 / 58. 65.100
> >> > >   libavformat    58. 35.101 / 58. 35.101
> >> > >   libavdevice    58.  9.101 / 58.  9.101
> >> > >   libavfilter     7. 68.100 /  7. 68.100
> >> > >   libswscale      5.  6.100 /  5.  6.100
> >> > >   libswresample   3.  6.100 /  3.  6.100
> >> > >   libpostproc    55.  6.100 / 55.  6.100
> >> > > Input #0, matroska,webm, from '01ffv1.mkv':
> >> > >   Metadata:
> >> > >     ENCODER         : Lavf58.35.101
> >> > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 322745 kb/s
> >> > >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p10le,
> >> > > 1632x1200, SAR 1:1 DAR 34:25, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
> >> > >     Metadata:
> >> > >       ENCODER         : Lavc58.65.100 ffv1
> >> > >       DURATION        : 00:15:04.400000000
> >> > > Stream mapping:
> >> > >   Stream #0:0 -> #0:0 (ffv1 (native) -> rawvideo (native))
> >> > > Press [q] to stop, [?] for help
> >> > > Output #0, framemd5, to 'converted.framemd5':
> >> > >   Metadata:
> >> > >     encoder         : Lavf58.35.101
> >> > >     Stream #0:0: Video: rawvideo (Y3[10][10] / 0xA0A3359),
> >> > > yuv422p10le, 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 979200 kb/s, 25
> >> > > fps, 25 tbn, 25 tbc (default)
> >> > >     Metadata:
> >> > >       DURATION        : 00:15:04.400000000
> >> > >       encoder         : Lavc58.65.100 rawvideo
> >> > > frame=  406 fps= 31 q=-0.0 Lsize=      32kB time=00:00:16.24 bitrate=
> >> > > 16.1kbits/s speed=1.23x
> >> > > video:3105900kB audio:0kB subtitle:0kB other streams:0kB global
> >> > > headers:0kB muxing overhead: unknown
> >> > >
> >> > > - Compare hashes
> >> > >
> >> > > orig.framemd5:
> >> > > #format: frame checksums
> >> > > #version: 2
> >> > > #hash: MD5
> >> > > #software: Lavf58.35.101
> >> > > #tb 0: 1/25
> >> > > #media_type 0: video
> >> > > #codec_id 0: rawvideo
> >> > > #dimensions 0: 1632x1200
> >> > > #sar 0: 1/1
> >> > > #stream#, dts,        pts, duration,     size, hash
> >> > > 0,          0,          0,        1,  7833600,
> >> > > 39d64300c8b749b0776423a80746d8dc
> >> > > 0,          1,          1,        1,  7833600,
> >> > > 12e7317d49727740187d2e23439d87d8
> >> > > 0,          2,          2,        1,  7833600,
> >> > > 12e7317d49727740187d2e23439d87d8
> >> > > 0,          3,          3,        1,  7833600,
> >> > > 0b3b0136b008ec9d67e900595be0c923
> >> > > 0,          4,          4,        1,  7833600,
> >> > > 97cd9b7689d1f49d1e1988a3613ffa72
> >> > > 0,          5,          5,        1,  7833600,
> >> > > 97cd9b7689d1f49d1e1988a3613ffa72
> >> > > 0,          6,          6,        1,  7833600,
> >> > > 1632e4b3ba7dc28e9dde698f3b7ec9e7
> >> > > 0,          7,          7,        1,  7833600,
> >> > > 796bda137d768f4ad31eb7166ea16447
> >> > > 0,          8,          8,        1,  7833600,
> >> > > 796bda137d768f4ad31eb7166ea16447
> >> > > 0,          9,          9,        1,  7833600,
> >> > > 612c13b4824bd427faa51efa7a09660d
> >> > > ...
> >> > >
> >> > > converted.framemd5:
> >> > > #format: frame checksums
> >> > > #version: 2
> >> > > #hash: MD5
> >> > > #software: Lavf58.35.101
> >> > > #tb 0: 1/25
> >> > > #media_type 0: video
> >> > > #codec_id 0: rawvideo
> >> > > #dimensions 0: 1632x1200
> >> > > #sar 0: 1/1
> >> > > #stream#, dts,        pts, duration,     size, hash
> >> > > 0,          0,          0,        1,  7833600,
> >> > > 3ed693fc02ba36cca92a2153afa960ac
> >> > > 0,          1,          1,        1,  7833600,
> >> > > 23d7482899eb8c854b26444dcdf38db4
> >> > > 0,          2,          2,        1,  7833600,
> >> > > 23d7482899eb8c854b26444dcdf38db4
> >> > > 0,          3,          3,        1,  7833600,
> >> > > 1fc392823ebc14001ee0e44c68d710e6
> >> > > 0,          4,          4,        1,  7833600,
> >> > > d63335f1c34132680e571549da51eac7
> >> > > 0,          5,          5,        1,  7833600,
> >> > > d63335f1c34132680e571549da51eac7
> >> > > 0,          6,          6,        1,  7833600,
> >> > > 09304bca4578458c353ecc119de0bd87
> >> > > 0,          7,          7,        1,  7833600,
> >> > > 0ec1e49008d1f70385dc4a1636559a30
> >> > > 0,          8,          8,        1,  7833600,
> >> > > 0ec1e49008d1f70385dc4a1636559a30
> >> > > 0,          9,          9,        1,  7833600,
> >> > > 78b22236d03517e5ceea3bd390ac2d0b
> >> > > ...
> >> > >
> >> > > Any idea of why this is happening?
> >> >
> >> > The input has 10 bit precision, therefore only 10 bit are encoded.
> >>
> >> The more difficult question is how big hqx precision really is:
> >> The original commit specified 10 bit, but a comment in the code (same
> >> commit)
> >> indicates 12 bit.
> >>
> >> Carl Eugen
> >
> > Hey, thanks for the instant answer.
> > So if I understand correctly no information is being lost, it's just that
> > the 10 bits are represented over <10 bits plus 6 bits of padding> instead of
> > over <16-bits where the least significant 6 bits always happen to be zero>.
> > Is it correct?
> > I'm assuming there is padding since if I extract a raw yuv frame from the
> > hqx and the ffv1 they are both the size you would expect if they were
> > 16-bit.
>
> Thing is decoder's output should be changed so it will be 12/10 bit
> pixel format.
> Instead of using 16bit format. Assuming 4 upper bits are not having useful info.
>

Aaah I understand now. I tried forcing ffmpeg to keep all the 16 bits (by first converting to rawvideo and then encoding from that to ffv1) and now the md5 matches. I think I'm going to do this until hqx outputs in 10/12 bits format just to be sure (to be able to perform the md5 check).

As a side note, I tried using the mpdecimate filter to remove the duplicate frames (the videos are 16 -> 25fps conversions) but it changes the md5 (of corresponding frames) again, checking the source it seems it supports 8-bit only?


_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Lossless encoding to ffv1 changes frames md5

Alberto Mattea
In reply to this post by Carl Eugen Hoyos-2
In data martedì 17 dicembre 2019 15:29:16 CET, Carl Eugen Hoyos ha scritto:

> Am Di., 17. Dez. 2019 um 14:48 Uhr schrieb Alberto Mattea <[hidden email]>:
> >
> > In data martedì 17 dicembre 2019 14:33:47 CET, Carl Eugen Hoyos ha scritto:
> > > Am Di., 17. Dez. 2019 um 14:26 Uhr schrieb Carl Eugen Hoyos
> > > <[hidden email]>:
> > > >
> > > > Am Di., 17. Dez. 2019 um 14:18 Uhr schrieb Alberto Mattea <[hidden email]>:
> > > > >
> > > > > Hi all, I'm trying to losslessly convert some videos from grassvalley hqx to ffv1.
> > > > > Unfortunately it seems the conversion is not really lossless, as the frames md5 changes after the conversion.
> > > > > Here's what I'm doing:
> > > > >
> > > > > - Calculate md5 of the original video frames
> > > > >
> > > > > $ ffmpeg -i 01.avi -f framemd5 orig.framemd5
> > > > > ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
> > > > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> > > > >   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
> > > > >   libavutil      56. 36.101 / 56. 36.101
> > > > >   libavcodec     58. 65.100 / 58. 65.100
> > > > >   libavformat    58. 35.101 / 58. 35.101
> > > > >   libavdevice    58.  9.101 / 58.  9.101
> > > > >   libavfilter     7. 68.100 /  7. 68.100
> > > > >   libswscale      5.  6.100 /  5.  6.100
> > > > >   libswresample   3.  6.100 /  3.  6.100
> > > > >   libpostproc    55.  6.100 / 55.  6.100
> > > > > Input #0, avi, from '01.avi':
> > > > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
> > > > >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
> > > > > Stream mapping:
> > > > >   Stream #0:0 -> #0:0 (hqx (native) -> rawvideo (native))
> > > > > Press [q] to stop, [?] for help
> > > > > Output #0, framemd5, to 'orig.framemd5':
> > > > >   Metadata:
> > > > >     encoder         : Lavf58.35.101
> > > > >     Stream #0:0: Video: rawvideo (Y3[10][16] / 0x100A3359), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 1566720 kb/s, 25 fps, 25 tbn, 25 tbc
> > > > >     Metadata:
> > > > >       encoder         : Lavc58.65.100 rawvideo
> > > > > frame=22610 fps= 50 q=-0.0 Lsize=    1767kB time=00:15:04.40 bitrate=  16.0kbits/s speed=1.98x
> > > > > video:172966500kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
> > > > >
> > > > > - Convert
> > > > >
> > > > > $ ffmpeg -i 01.avi -c:v ffv1 01ffv1.mkv
> > > > > ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
> > > > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> > > > >   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
> > > > >   libavutil      56. 36.101 / 56. 36.101
> > > > >   libavcodec     58. 65.100 / 58. 65.100
> > > > >   libavformat    58. 35.101 / 58. 35.101
> > > > >   libavdevice    58.  9.101 / 58.  9.101
> > > > >   libavfilter     7. 68.100 /  7. 68.100
> > > > >   libswscale      5.  6.100 /  5.  6.100
> > > > >   libswresample   3.  6.100 /  3.  6.100
> > > > >   libpostproc    55.  6.100 / 55.  6.100
> > > > > Input #0, avi, from '01.avi':
> > > > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
> > > > >     Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
> > > > > Stream mapping:
> > > > >   Stream #0:0 -> #0:0 (hqx (native) -> ffv1 (native))
> > > > > Press [q] to stop, [?] for help
> > > > > [ffv1 @ 0x6e1ce80] bits_per_raw_sample > 8, forcing range coder
> > > > > Output #0, matroska, to '01ffv1.mkv':
> > > > >   Metadata:
> > > > >     encoder         : Lavf58.35.101
> > > > >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 200 kb/s, 25 fps, 1k tbn, 25 tbc
> > > > >     Metadata:
> > > > >       encoder         : Lavc58.65.100 ffv1
> > > > > frame=22610 fps= 17 q=-0.0 Lsize=35631195kB time=00:15:04.36 bitrate=322759.1kbits/s speed=0.691x
> > > > > video:35630872kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000908%
> > > > >
> > > > > - Calculate md5 of the converted frames (I've stopped it after a few hundred frames, just to check the beginning)
> > > > >
> > > > > $ ffmpeg -i 01ffv1.mkv -f framemd5 converted.framemd5
> > > > > ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
> > > > >   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
> > > > >   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
> > > > >   libavutil      56. 36.101 / 56. 36.101
> > > > >   libavcodec     58. 65.100 / 58. 65.100
> > > > >   libavformat    58. 35.101 / 58. 35.101
> > > > >   libavdevice    58.  9.101 / 58.  9.101
> > > > >   libavfilter     7. 68.100 /  7. 68.100
> > > > >   libswscale      5.  6.100 /  5.  6.100
> > > > >   libswresample   3.  6.100 /  3.  6.100
> > > > >   libpostproc    55.  6.100 / 55.  6.100
> > > > > Input #0, matroska,webm, from '01ffv1.mkv':
> > > > >   Metadata:
> > > > >     ENCODER         : Lavf58.35.101
> > > > >   Duration: 00:15:04.40, start: 0.000000, bitrate: 322745 kb/s
> > > > >     Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p10le, 1632x1200, SAR 1:1 DAR 34:25, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
> > > > >     Metadata:
> > > > >       ENCODER         : Lavc58.65.100 ffv1
> > > > >       DURATION        : 00:15:04.400000000
> > > > > Stream mapping:
> > > > >   Stream #0:0 -> #0:0 (ffv1 (native) -> rawvideo (native))
> > > > > Press [q] to stop, [?] for help
> > > > > Output #0, framemd5, to 'converted.framemd5':
> > > > >   Metadata:
> > > > >     encoder         : Lavf58.35.101
> > > > >     Stream #0:0: Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 979200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
> > > > >     Metadata:
> > > > >       DURATION        : 00:15:04.400000000
> > > > >       encoder         : Lavc58.65.100 rawvideo
> > > > > frame=  406 fps= 31 q=-0.0 Lsize=      32kB time=00:00:16.24 bitrate=  16.1kbits/s speed=1.23x
> > > > > video:3105900kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
> > > > >
> > > > > - Compare hashes
> > > > >
> > > > > orig.framemd5:
> > > > > #format: frame checksums
> > > > > #version: 2
> > > > > #hash: MD5
> > > > > #software: Lavf58.35.101
> > > > > #tb 0: 1/25
> > > > > #media_type 0: video
> > > > > #codec_id 0: rawvideo
> > > > > #dimensions 0: 1632x1200
> > > > > #sar 0: 1/1
> > > > > #stream#, dts,        pts, duration,     size, hash
> > > > > 0,          0,          0,        1,  7833600, 39d64300c8b749b0776423a80746d8dc
> > > > > 0,          1,          1,        1,  7833600, 12e7317d49727740187d2e23439d87d8
> > > > > 0,          2,          2,        1,  7833600, 12e7317d49727740187d2e23439d87d8
> > > > > 0,          3,          3,        1,  7833600, 0b3b0136b008ec9d67e900595be0c923
> > > > > 0,          4,          4,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
> > > > > 0,          5,          5,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
> > > > > 0,          6,          6,        1,  7833600, 1632e4b3ba7dc28e9dde698f3b7ec9e7
> > > > > 0,          7,          7,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
> > > > > 0,          8,          8,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
> > > > > 0,          9,          9,        1,  7833600, 612c13b4824bd427faa51efa7a09660d
> > > > > ...
> > > > >
> > > > > converted.framemd5:
> > > > > #format: frame checksums
> > > > > #version: 2
> > > > > #hash: MD5
> > > > > #software: Lavf58.35.101
> > > > > #tb 0: 1/25
> > > > > #media_type 0: video
> > > > > #codec_id 0: rawvideo
> > > > > #dimensions 0: 1632x1200
> > > > > #sar 0: 1/1
> > > > > #stream#, dts,        pts, duration,     size, hash
> > > > > 0,          0,          0,        1,  7833600, 3ed693fc02ba36cca92a2153afa960ac
> > > > > 0,          1,          1,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
> > > > > 0,          2,          2,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
> > > > > 0,          3,          3,        1,  7833600, 1fc392823ebc14001ee0e44c68d710e6
> > > > > 0,          4,          4,        1,  7833600, d63335f1c34132680e571549da51eac7
> > > > > 0,          5,          5,        1,  7833600, d63335f1c34132680e571549da51eac7
> > > > > 0,          6,          6,        1,  7833600, 09304bca4578458c353ecc119de0bd87
> > > > > 0,          7,          7,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
> > > > > 0,          8,          8,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
> > > > > 0,          9,          9,        1,  7833600, 78b22236d03517e5ceea3bd390ac2d0b
> > > > > ...
> > > > >
> > > > > Any idea of why this is happening?
> > > >
> > > > The input has 10 bit precision, therefore only 10 bit are encoded.
> > >
> > > The more difficult question is how big hqx precision really is:
> > > The original commit specified 10 bit, but a comment in the code (same commit)
> > > indicates 12 bit.
> > >
> > > Carl Eugen
> >
> > Hey, thanks for the instant answer.
> > So if I understand correctly no information is being lost, it's just that the 10 bits are
> > represented over <10 bits plus 6 bits of padding> instead of over <16-bits where
> > the least significant 6 bits always happen to be zero>. Is it correct?
>
> I don't think they are zero (and this is not necessary for them to be
> "no information").

Thanks for the clarification, I had a too simplistic view of this.

>
> > I'm assuming there is padding since if I extract a raw yuv frame from the hqx and the ffv1
> > they are both the size you would expect if they were 16-bit.
>
> But this is not necessarily related.

Got it

>
> Carl Eugen




_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Lossless encoding to ffv1 changes frames md5

Moritz Barsnick
In reply to this post by Alberto Mattea
On Tue, Dec 17, 2019 at 16:04:02 +0100, Alberto Mattea wrote:
> As a side note, I tried using the mpdecimate filter to remove the
> duplicate frames (the videos are 16 -> 25fps conversions) but it
> changes the md5 (of corresponding frames) again, checking the source
> it seems it supports 8-bit only?

That's correct, mpdecimate currently only supports (taken from the
source code):

AV_PIX_FMT_YUV444P
AV_PIX_FMT_YUV422P,
AV_PIX_FMT_YUV420P,
AV_PIX_FMT_YUV411P,
AV_PIX_FMT_YUV410P,
AV_PIX_FMT_YUV440P,
AV_PIX_FMT_YUVJ444P,
AV_PIX_FMT_YUVJ422P,
AV_PIX_FMT_YUVJ420P
AV_PIX_FMT_YUVJ440P,
AV_PIX_FMT_YUVA420P,
AV_PIX_FMT_GBRP,
AV_PIX_FMT_YUVA444P,
AV_PIX_FMT_YUVA422P

Cheers,
Moritz
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".