Question about macroblocks in soft telecined video

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Question about macroblocks in soft telecined video

Mark Filipak
I can't answer this for myself because I don't have the tools needed to probe into undecoded
macroblocks (at least, I don't think I have the tools).

I would guess that, for an undecoded video that's soft telecined (i.e. @24/1.001 FPS), the interlace
in the macroblocks is field-based (i.e. the same as if @30/1.001 FPS), not frame-based (i.e. the
same as if @24 FPS). Specifically, for YCbCr420, blocks 5 (Cb) & 6 (Cr) are deinterlaced into top &
bottom fields (in the top-half & bottom-half of the blocks) rather than being progressive.

Is my guess correct?

Thanks so much!
- Mark.
--
Racism is like roaches. When you switch on the light, they scurry.
But if you don't switch on the light, you don't know they're there.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Question about macroblocks in soft telecined video

Carl Eugen Hoyos-2
Am So., 6. Sept. 2020 um 06:20 Uhr schrieb Mark Filipak
<[hidden email]>:

> I would guess that, for an undecoded video that's soft telecined (i.e. @24/1.001 FPS),
> the interlace in the macroblocks is field-based (i.e. the same as if @30/1.001 FPS),
> not frame-based (i.e. the same as if @24 FPS).

This does not make sense.

> Specifically, for YCbCr420, blocks 5 (Cb) & 6 (Cr) are deinterlaced into top & bottom
> fields (in the top-half & bottom-half of the blocks) rather than being progressive.

This seems more difficult to understand but I don't think it makes sense either.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Question about macroblocks in soft telecined video

Mark Filipak
On 09/06/2020 02:26 AM, Carl Eugen Hoyos wrote:
> Am So., 6. Sept. 2020 um 06:20 Uhr schrieb Mark Filipak
> <[hidden email]>:
>
>> I would guess that, for an undecoded video that's soft telecined (i.e. @24/1.001 FPS),
>> the interlace in the macroblocks is field-based (i.e. the same as if @30/1.001 FPS),
>> not frame-based (i.e. the same as if @24 FPS).
>
> This does not make sense.

Okay. I followed what I wrote with an example (below), so I'll go with the example because...
examples are usually easier to understand than are abstractions.

>> Specifically, for YCbCr420, blocks 5 (Cb) & 6 (Cr) are deinterlaced into top & bottom
>> fields (in the top-half & bottom-half of the blocks) rather than being progressive.
>
> This seems more difficult to understand but I don't think it makes sense either.

A YCbCr420 macroblock contains 6 blocks: blocks 1..4, which are 256 samples of Y (luminance) at full
resolution, and blocks 5+6, which are each 64 samples of chrominance: Cb & Cr, at 1/4 resolution.

I'm happy to I explain.

For undecoded, frame-based (so-called "progressive") macroblocks, the Y-blocks (1..4) are 4-way
interleaved in the stream (2x2 samples/quad, 4x4 quads/block, 2x1 blocks/field, and 1x2
fields/frame) -- [1] -- and must be deinterleaved by the decoder to arrive at whole and unbroken
(concurrent) frames. Staying with frame-based macroblock, the Cb & Cr blocks (5 & 6) are entirely
uninterleaved because 64 samples (remember, 1/4 resolution) fit in 64 bytes.

[1] I reserve the word "interlace" for whole sample rows, only, not parts of sample rows.
Frame-based macroblocks are never deinterlaced when they're encoded. Consequently, they don't have
to be interlaced when they're decoded because they're already interlaced.

For undecoded, field-based (so-called "interlaced") macroblocks, in addition to the 4-way
interleaves, the 2x2 quads between blocks 1 & 3 (and also between blocks 2 & 4) are deinterlaced in
the stream and must be interlaced by the decoder to arrive at whole and unbroken frames. The
equivalent deinterlace for chrominance is that the rows in top half of block 5 (Cb) are deinterlaced
with the rows in the bottom half of block 5, and must be interlaced by the decoder. Likewise for
block 6 (Cr).

You see, to you guys, all that's important is that frames come out of the decoder as whole, unbroken
streams. But to anyone who examines disc files (VOBs), the stuff on the disc is all we have to work
with when trying to figure out a strategy for identifying the nature of the source. And the nature
of the source is important.

Okay, to proceed. Soft telecined video is actually 23/1.001 frames per second of video even though
the metadata tells the decoder to produce 30/1.001 FPS. Of course, the metadata is the key to how
the stream 'teaches' the decoder how to telecine. MPV is smart enough to recognize 23/1.001 FPS data
and to ignore the metadata and to play at 23/1.001 FPS. Ffmpeg can do the same thing (and thereby
eliminate the need to transcode), but the ffmpeg user has to tell ffmpeg to do it.

Okay, so what does this have to do with macroblocks? Well, I'm writing a video glossary and I want
it to be complete. For example, think back to the controversy we've had regarding the meaning of
"interlace". From your perspective, interlace is function that the decoder performs. From my
perspective, interlace is a condition of the stream. From your perspective, it seems like I don't
know what I'm talking about. From my perspective, it seems like ffmpeg developers are using sloppy
nomenclature. We have both been right and wrong.

Studying macroblocks has shown me what your perspective is. To you, interlacing is what the decoder
must do. Of course that's correct. To me (prior to studying macroblocks), interlacing was an
architectural feature of the stream (program or transport), and saying that fields that are clearly
deinterlaced in the stream are 'interlaced' just didn't make sense. Of course, your perspective is
after decoding while my perspective is prior to decoding (because discs contain undecoded
macroblocks!). I hope I've made myself clearly understood.

By the way, I have made pictures of all this stuff. Would you like to see them?

--
Racism is like roaches. When you switch on the light, they scurry.
But if you don't switch on the light, you don't know they're there.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Question about macroblocks in soft telecined video

Carl Eugen Hoyos-2
Am So., 6. Sept. 2020 um 09:28 Uhr schrieb Mark Filipak
<[hidden email]>:

[...]

> Soft telecined video is actually 23/1.001 frames per second of video
> even though the metadata tells the decoder to produce 30/1.001 FPS.

On the FFmpeg user mailing list, "decoder" and "metadata" have
relatively strict meanings. Given these meanings, what you write
is simply wrong.
(Apart from the fact that telecined content does not necessarily
have a framerate of 24000/1001, the whole point is that it can
have any framerate, in the case of ntsc crts any framerate smaller
than 30000/1001.)

> Of course, the metadata is the key to how the stream 'teaches' the
> decoder how to telecine.

But since we don't want to telecine, this is irrelevant: We don't have
access to an ntsc crt. If you decide to telecine, this cannot be done
in the (FFmpeg) decoder, you need a (FFmpeg) video filter.

> MPV is smart enough to recognize 23/1.001 FPS data and to ignore
> the metadata and to play at 23/1.001 FPS.

> Ffmpeg can do the same thing (and thereby eliminate the need to
> transcode)

Telecine and transcoding do not depend on each-other so this is
highly misleading.

> but the ffmpeg user has to tell ffmpeg to do it.

No, in general, this is not true.
(Command line and complete, uncut console output missing.)

A few random remarks:
You provided some definitions (actually claims) in your email without
explaining in which domain you believe they are valid. They are not
valid in the general case (and not valid in most domains).
I believe you somewhere mention that you want to detect soft-
telecined content. This is trivial and is not related to macroblocks.
Your definition of our different interpretations of "interlaced" is
completely backwards and wrong: A digital video stream can be
encoded (!, not decoded as you imply) using encoding techniques
meant for interlaced content. The decoder has to detect this and
comply with it but this has no relevance for a user (it for many years
was very important for FFmpeg developers). For display, it is
very important to know if the actual content is interlaced or not.
Most video players take the first information to decide but this is
not valid in general (or, using your wording, "sloppy").
In both views, this is not necessarily related to macroblocks.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Question about macroblocks in soft telecined video

Mark Filipak
On 09/06/2020 12:07 PM, Carl Eugen Hoyos wrote:

> Am So., 6. Sept. 2020 um 09:28 Uhr schrieb Mark Filipak
> <[hidden email]>:
>
> [...]
>
>> Soft telecined video is actually 23/1.001 frames per second of video
>> even though the metadata tells the decoder to produce 30/1.001 FPS.
>
> On the FFmpeg user mailing list, "decoder" and "metadata" have
> relatively strict meanings.

Your point is a 'lateral arabesque'. Argue the issues please. Don't bring up something unrelated in
an attempt to discredit what I wrote.

> Given these meanings, what you write
> is simply wrong.
> (Apart from the fact that telecined content does not necessarily
> have a framerate of 24000/1001, ...

That's an ad homonym attack. I didn't say "telecined". I said "soft telecined". *Soft telecined*
frames *do* have an FPS of 24/1.001.

>... the whole point is that it can
> have any framerate, in the case of ntsc crts any framerate smaller
> than 30000/1001.)

CRTs? What does the display device have to do with anything? (And NTSC CRTs definitely do have
30/1.001 Hz frame rate. Solely 30/1.001 Hz frame rate. Otherwise, they're not NTSC.)

>> Of course, the metadata is the key to how the stream 'teaches' the
>> decoder how to telecine.
>
> But since we don't want to telecine, this is irrelevant: We don't have
> access to an ntsc crt. If you decide to telecine, this cannot be done
> in the (FFmpeg) decoder, you need a (FFmpeg) video filter.

You continue to fail to understand the issue. I'm addressing undecoded video, not decoded video. Why
am I addressing undecoded video? Because that's what's on a DVD or BD. You don't care about that
because you (ffmpeg developers) get undecoded frames from the decoder. We (ffmpeg users) need to
detect what's on discs because that's important.

>> MPV is smart enough to recognize 23/1.001 FPS data and to ignore
>> the metadata and to play at 23/1.001 FPS.
>
>> Ffmpeg can do the same thing (and thereby eliminate the need to
>> transcode)
>
> Telecine and transcoding do not depend on each-other so this is
> highly misleading.

They do depend on each other. Users transcode soft telecined video to get 24FPS because they think
that's what they need to do. Instead, all they need to do is force 24/1.001 -- at least, that's what
I think, but I'm unsure because ffmpeg is so poorly documented.

>> but the ffmpeg user has to tell ffmpeg to do it.
>
> No, in general, this is not true.
> (Command line and complete, uncut console output missing.)

I don't know how to respond to that. Console output doesn't have anything to do with adequate
documentation. The issue isn't a command line mistake. The issue is: How does ffmpeg work?

> A few random remarks:
> You provided some definitions (actually claims) in your email without
> explaining in which domain you believe they are valid. ...

The 'domain' is undecoded video as found on DVDs & BDs.

>... They are not
> valid in the general case (and not valid in most domains).
> I believe you somewhere mention that you want to detect soft-
> telecined content. This is trivial and is not related to macroblocks.

So-called "progressive" video -- I prefer "concurrent" -- is contained in interlaced data structures
within the macroblocks. Hard telecined video (which may or may not be concurrent depending on
whether it comes from cinema or from TV) is contained in deinterlaced data within the macroblocks
that the decoder interlaces to make frames as part of the decoding process. My question has to do
with whether, for soft telecine, the fields in the chrominance blocks of undecoded macroblocks is
deinterlaced or interlaced.

> Your definition of our different interpretations of "interlaced" is
> completely backwards and wrong: ...

If you read H.222 and H.262 carefully, you will find that so-called "interlaced" video is not called
"interlaced". It's called "interlace" (no "d"). It's called "interlace" because interlace is what
the decoder does. If you read H.222 and H.262 carefully, you will find that the undecoded data is,
in fact, deinterlaced (in other words: The undecoded data is in fields, not frames).

Interlaced frames is NOT what's found in undecoded video. What's found in undecoded video are
deinterlaced fields. It is you who have it backwards. And that backwardness is what confused me and
what confuses others.

>... A digital video stream can be
> encoded (!, not decoded as you imply) using encoding techniques
> meant for interlaced content. The decoder has to detect this and
> comply with it but this has no relevance for a user (it for many years
> was very important for FFmpeg developers). For display, it is
> very important to know if the actual content is interlaced or not.
> Most video players take the first information to decide but this is
> not valid in general (or, using your wording, "sloppy").
> In both views, this is not necessarily related to macroblocks.

It is absolutely intimately related to macroblocks and the data structures they contain.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Question about macroblocks in soft telecined video

Carl Eugen Hoyos-2
Am So., 6. Sept. 2020 um 19:07 Uhr schrieb Mark Filipak
<[hidden email]>:

> > (Apart from the fact that telecined content does not necessarily

Sorry for the typo here, this should have said "soft telecined".

> > have a framerate of 24000/1001, ...
>
> That's an ad homonym attack.

lol

> I didn't say "telecined". I said "soft telecined". *Soft telecined*
> frames *do* have an FPS of 24/1.001.

Not necessarily.

> >... the whole point is that it can
> > have any framerate, in the case of ntsc crts any framerate smaller
> > than 30000/1001.)
>
> CRTs? What does the display device have to do with anything?

Because they are the reasons soft-telecine in vob streams was
invented - you should ignore the feature if you don't use one.

> (And NTSC CRTs definitely do have 30/1.001 Hz frame rate.

Yes, that's my point (although soft telecine was also necessary
for pal tv's if your input was < 25fps).

> Solely 30/1.001 Hz frame rate. Otherwise, they're not NTSC.)
>
> >> Of course, the metadata is the key to how the stream 'teaches' the
> >> decoder how to telecine.
> >
> > But since we don't want to telecine, this is irrelevant: We don't have
> > access to an ntsc crt. If you decide to telecine, this cannot be done
> > in the (FFmpeg) decoder, you need a (FFmpeg) video filter.
>
> You continue to fail to understand the issue.

No, you fail to understand that this is the FFmpeg user mailing list.

> I'm addressing undecoded video, not decoded video. Why
> am I addressing undecoded video? Because that's what's on a
> DVD or BD.

> You don't care about that because you (ffmpeg developers)

(The FFmpeg developers care a lot about all these things. But
they are mostly off-topic on this mailing list.)

> get undecoded frames from the decoder.

No. We provide these decoded frames to our users.

> We (ffmpeg users) need to
> detect what's on discs because that's important.

Which you can do to some extent.

If you really believe that there is an information missing, please
provide a sample stream and tell us what the command line
utility does not tell you about it (note though that in the past many
developers argued that we should print less information, not more).

> >> MPV is smart enough to recognize 23/1.001 FPS data and to ignore
> >> the metadata and to play at 23/1.001 FPS.
> >
> >> Ffmpeg can do the same thing (and thereby eliminate the need to
> >> transcode)
> >
> > Telecine and transcoding do not depend on each-other so this is
> > highly misleading.
>
> They do depend on each other. Users transcode soft telecined video to
> get 24FPS because they think that's what they need to do. Instead, all
> they need to do is force 24/1.001

This does not sound correct to me.

> -- at least, that's what I think, but I'm unsure because ffmpeg is so
> poorly documented.

That may or may not be true but I wonder how it is related.

> >> but the ffmpeg user has to tell ffmpeg to do it.
> >
> > No, in general, this is not true.
> > (Command line and complete, uncut console output missing.)
>
> I don't know how to respond to that. Console output doesn't have
> anything to do with adequate documentation.

No, only with posts on this mailing list.

> The issue isn't a command line mistake. The issue is: How does ffmpeg work?

I can often answer this question for specific use cases.

> > A few random remarks:
> > You provided some definitions (actually claims) in your email without
> > explaining in which domain you believe they are valid. ...
>
> The 'domain' is undecoded video as found on DVDs & BDs.

These are two (very) different domains with very different (and
incompatible) definitions.

> >... They are not
> > valid in the general case (and not valid in most domains).
> > I believe you somewhere mention that you want to detect soft-
> > telecined content. This is trivial and is not related to macroblocks.
>
> So-called "progressive" video -- I prefer "concurrent" -- is contained in interlaced data structures
> within the macroblocks. Hard telecined video (which may or may not be concurrent depending on
> whether it comes from cinema or from TV) is contained in deinterlaced data within the macroblocks
> that the decoder interlaces to make frames as part of the decoding process. My question has to do
> with whether, for soft telecine, the fields in the chrominance blocks of undecoded macroblocks is
> deinterlaced or interlaced.

As said before: I don't think this makes sense (not even for mpeg2 video).

> > Your definition of our different interpretations of "interlaced" is
> > completely backwards and wrong: ...
>
> If you read H.222 and H.262 carefully

Given that reading these two is uncommon on the development mailing list
(longtime bugs that nobody wants to attack depend on them), I fear we will
not be able to discuss them here.

Carl Eugen

PS: I hope people will forgive me - contrary to my first mail, this one
does contain an ad-hominem attack.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Question about macroblocks in soft telecined video

Mark Filipak
On 09/06/2020 03:31 PM, Carl Eugen Hoyos wrote:
> Am So., 6. Sept. 2020 um 19:07 Uhr schrieb Mark Filipak

First, Carl Eugen, could you fix your email client so that it doesn't echo people's email addresses
in the clear?

-snip-
>> So-called "progressive" video -- I prefer "concurrent" -- is contained in interlaced data structures
>> within the macroblocks. Hard telecined video (which may or may not be concurrent depending on
>> whether it comes from cinema or from TV) is contained in deinterlaced data within the macroblocks
>> that the decoder interlaces to make frames as part of the decoding process. My question has to do
>> with whether, for soft telecine, the fields in the chrominance blocks of undecoded macroblocks is
>> deinterlaced or interlaced.
>
> As said before: I don't think this makes sense (not even for mpeg2 video).

I offered to post pictures but you didn't respond. I really like pictures. Pictures don't generally
require explanations. Written text in lieu of pictures is counterproductive. The more that is
written, the more that can be wrong or misunderstood. ffmpeg documentation doesn't have (avoids?)
pictures. Why is that?
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".