ffprobe -show_frames and coded_picture_number

classic Classic list List threaded Threaded
35 messages Options
12
Reply | Threaded
Open this post in threaded view
|

ffprobe -show_frames and coded_picture_number

Richard H Lee
I use ffprobe -show_frames to find out the exact frame rate of videos
that stray away from 24000/1001 fps before conversion to 25 fps so that
I can get accurate a/v sync when I speed up the audio.

I basically use pkt_pts_time / coded_picture_number of the very last
frame for the overall frame rate.

The problem is that on large files, it can take quite a while to dump
all the frame data.

Is it possible to just dump the last few frames / seconds of the file?
Or would this affect coded_picture_number? (I'm not sure if
coded_picture_number is counted by keeping track of frames/packets or is
stored within the packet itself.)
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

thljcl
The frame rate of your video source is 24000/1001 FPS. You want to convert it to 25 FPS and still make sure that the audio is in sync with the video? Is that what you want?
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Richard H Lee
On 09/08/2013 19:48, thljcl wrote:
> The frame rate of your video source is 24000/1001 FPS. You want to convert it
> to 25 FPS and still make sure that the audio is in sync with the video? Is
> that what you want?

No, I don't have any problems with conversion and synchronization at
24000/1001 fps. You can just speed up the audio by 1001/960 to get it to
match 25 fps.

The problem is that many media is not exactly 24000/1001 fps . Thus to
find the exact frame rate run ffprobe -show_frames on the file, and
divide the coded_picture_number by the pkt_pts_time of the last frame.

The thing is that ffprobe -show_frames can take quite a while especially
on a slower machine as it has to go through every frame.

This leads me to my question: it is possible to save time by getting the
coded_picture_number and the pkt_pts_time fields of the last frame
without having to run through all the other preceding frames?
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

thljcl
What you are saying is that some sources do not have constant frame rate of 24000/1001 FPS despite having the “average frame rate” close to 24000/1001 FPS, thus speeding up the audio would cause the video and audio to be out of sync.
The truth is, to me, regardless of the original frame rate of the source, I do not need to speed up or slow down the audio in order to keep the audio and video in sync. I do understand that speeding up the audio is the standard practice being used by television stations in PAL regions to keep the audio and video in sync from the source of 24 FPS. Such practice makes sense before digital age; without re-encoding the video/audio, speeding up the audio would keep video and audio in sync. With ffmpeg, however, there is no need to change the playback speed to keep the audio and video in sync. Let me briefly explains how the video filter “fps” works.
Basically, with -vf "fps=25" as an output option, you can change the frame rate to constant frame rate of 25 FPS; ffmpeg would duplicate frames if the source is of lower frame rate; ffmpeg would drop frames if the source is of higher frame rates. In fact, that’s how I create a slideshow which consists a lot of duplicated frames (slides) as I wrote at http://ffmpeg-users.933282.n4.nabble.com/Create-slideshow-with-ffmpeg-from-still-images-td4660277.html. You can read those posts as reference. By duplicating and dropping frames, there would be minimal change of playback speed, subject to the round-off error. In practice, there would be no need for the change of audio speed or to actually know how the frame rate of the source varies over time.
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Robert Krüger-2
On Sat, Aug 10, 2013 at 5:43 AM, thljcl <[hidden email]> wrote:

> What you are saying is that some sources do not have constant frame rate of
> 24000/1001 FPS despite having the “average frame rate” close to 24000/1001
> FPS, thus speeding up the audio would cause the video and audio to be out of
> sync.
> The truth is, to me, regardless of the original frame rate of the source, I
> do not need to speed up or slow down the audio in order to keep the audio
> and video in sync. I do understand that speeding up the audio is the
> standard practice being used by television stations in PAL regions to keep
> the audio and video in sync from the source of 24 FPS. Such practice makes
> sense before digital age; without re-encoding the video/audio, speeding up
> the audio would keep video and audio in sync. With ffmpeg, however, there is
> no need to change the playback speed to keep the audio and video in sync.
> Let me briefly explains how the video filter “fps” works.
> Basically, with -vf "fps=25" as an output option, you can change the frame
> rate to constant frame rate of 25 FPS; ffmpeg would duplicate frames if the
> source is of lower frame rate; ffmpeg would drop frames if the source is of
> higher frame rates. In fact, that’s how I create a slideshow which consists
> a lot of duplicated frames (slides) as I wrote at
> http://ffmpeg-users.933282.n4.nabble.com/Create-slideshow-with-ffmpeg-from-still-images-td4660277.html.
> You can read those posts as reference. By duplicating and dropping frames,
> there would be minimal change of playback speed, subject to the round-off
> error. In practice, there would be no need for the change of audio speed or
> to actually know how the frame rate of the source varies over time.
>

this is bad advice if your input has more or less constant frame rate
and you care about smooth motion as you will have visible degradation
of perceived quality when changing frame rates that way.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Nicolas George-2
In reply to this post by thljcl
Le duodi 22 thermidor, an CCXXI, thljcl a écrit :
> Such practice makes
> sense before digital age; without re-encoding the video/audio, speeding up
> the audio would keep video and audio in sync. With ffmpeg, however, there is
> no need to change the playback speed to keep the audio and video in sync.
> Let me briefly explains how the video filter “fps” works.

You obviously never watched a DVD that was converted from NTSC to PAL that
way, or you would not dare make such a statement: they just look completely
ugly.

To Richard H Lee: if you follow that advice, you will get a video that has a
hiccup every second, very noticeable in regular motion scenes such as
tracking shots. A good way to give your audience a headache.

Also, this apply both to analog and digital, it makes no difference with
regard to frame rate.

Regards,

--
  Nicolas George

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Paul B Mahol
In reply to this post by Richard H Lee
On 8/9/13, Richard H Lee <[hidden email]> wrote:
> I use ffprobe -show_frames to find out the exact frame rate of videos
> that stray away from 24000/1001 fps before conversion to 25 fps so that
> I can get accurate a/v sync when I speed up the audio.
>
> I basically use pkt_pts_time / coded_picture_number of the very last
> frame for the overall frame rate.
>
> The problem is that on large files, it can take quite a while to dump
> all the frame data.

Mainly because you need to decode whole file (at least currently).

>
> Is it possible to just dump the last few frames / seconds of the file?

Yes, by seeking to such location.

> Or would this affect coded_picture_number? (I'm not sure if
> coded_picture_number is counted by keeping track of frames/packets or is
> stored within the packet itself.)

It looks like coded_picture_number is increased by one when decoding
(and only for few codecs).
So its not stored in bitstream, and skiping decoding would change it.

You can try this anyway and report outcome.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

thljcl
In reply to this post by Robert Krüger-2
I do have experiences over changing frame rate from 25 FPS, 29.97 FPS, and 30 FPS to 24 FPS using the method I described. As far as my eyes are concerned, I do not have the so-called “unrealistic motion” issues. Some people have believed that higher frame rate is required for more realistic motion.
Without regards to human eyes or perceptions, there is no such thing as “absolute motion” as Laws of Physics are equivalent to all inertial frame of references; depending on which postulate you use to derive the theory of special relativity, such a statement could be regarded as either “consequence” or “postulate”. That said, human visual perception has its own properties; what we see does not necessarily represents the actual motion relative to us as an observer. Generally, our eyes take time to capture visual information; hence our visual perception is actually formed by information we received from different time. That’s why we observe the effect of “motion blurring” from “fast-moving object”. In a way of speaking, “motion blurring” effect we observe is similar to the camera, which also requires time to capture electromagnetic radiation from different time. Due to our day-to-day experience, it’s hence more realistic to have “motion blurring” effect from fast-moving object to our human eyes.
Generally speaking, human eyes are able to differentiate 12-15 images every second by perceiving them individually. In other words, the frame rate of 24 FPS, 25 FPS, 29.97 FPS, 30 FPS, or 60 FPS all create the illusion of motion picture, which we cannot actually know the frame rate using our naked eyes. Theoretically speaking, if a camera is able to capture high-resolution image at a very high frame rate on high-speed moving object, the “motion blurring” effect would be lessened or disappear for individual frames. Hence, dropping the frames to achieve lower frame rate of 24 FPS would actually result in “less realistic” motion. It’s possible to add “special effect” with software to make the motion “more realistic” to our eyes. That said, we were talking about perhaps from 300 FPS to 24 FPS. For the common frame rate used by different standard such as “24 FPS”, “25 FPS”, “29.97 FPS, etc., conversion between them would almost certainly would not result any difference to human naked eyes by duplicating and dropping frames.
As a matter of fact, most LCD displays can only have up to 60 Hz of refresh rate. That means, even if the video does have very high frame rate, it would become somewhat unrealistic to human eyes because of the lack of “motion blurring” effect and the low refresh rate of the display. Only when both the display has similarly high refresh rate and at the same time the video does have very high frame rate, the properties of human visual system would create “motion blurring” effect just as we actually see the actual motion of the object.
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Richard H Lee
In reply to this post by Paul B Mahol
On 10/08/2013 13:48, Paul B Mahol wrote:
> It looks like coded_picture_number is increased by one when decoding
> (and only for few codecs).
> So its not stored in bitstream, and skiping decoding would change it.

OK, I see thanks.

> You can try this anyway and report outcome.

Unfortunately, there does not seems to be an option for ffprobe to dump
frames for a certain point. There is a seek_point option for the movie
source, but this appears only to be for overlays.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Nicolas George-2
In reply to this post by thljcl
Le tridi 23 thermidor, an CCXXI, thljcl a écrit :
> I do have experiences over changing frame rate from 25 FPS, 29.97 FPS, and 30
> FPS to 24 FPS using the method I described.

That is very different from 24 -> 25 fps, because in one case the extra
frame comes at 5~6 Hz while in the other case it comes at ~1 Hz. An extra
frame at 5~6 Hz is barely noticeable, while an extra frame at 0.1~2 Hz is
very noticeable.

<snip>

How you can have so much knowledge and fail so utterly to actually
understand it is beyond me.

Regards,

--
  Nicolas George

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

thljcl
Nicolas George-2 wrote
That is very different from 24 -> 25 fps, because in one case the extra
frame comes at 5~6 Hz while in the other case it comes at ~1 Hz. An extra
frame at 5~6 Hz is barely noticeable, while an extra frame at 0.1~2 Hz is
very noticeable.

<snip>

How you can have so much knowledge and fail so utterly to actually
understand it is beyond me.

Regards,

--
  Nicolas George

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user


signature.asc (205 bytes) <http://ffmpeg-users.933282.n4.nabble.com/attachment/4660624/0/signature.asc>
Hm… Well, let’s put it to actual test, which should be a much better option than believing in either you or me. Do you currently have a source of 24 FPS which you think that it has “high speed motion”? If you seriously think that using the method I describe would lead to “unrealistic motion”, can you share such a source with me and allow me to do the encoding process? After doing the encoding process, I will upload the output file to SkyDrive and share it temporarily publicly. We can then all use our eyes to make our judgment. I do believe that an actual experiment would help me understand better. Thank you.

Oh, yes… Please make sure that the source you use is not too big or too lengthy. After all, it’s just meant to prove the hypothesis. I would prefer your source to have less than 10 minutes in length.
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Phil Rhodes
Just duplicating frames is unlikely to produce results anyone wants to watch. Duplicating one frame per second to go from 24 (or nearly 24) to 25 was occasionally done in days of yore when people wanted to edit film-originated material on PAL video. It looks very rough.

Properly done frame rate conversions that are intended to be of high quality, regardless of the desired change in rate, are invariably done with motion compensated interpolation. Of course this is not always an exact science either, but it's about the best option and properly done it is generally fairly close to invisible on most subjects I don't know if ffmpeg is capable of doing it.

I don't think there's really any serious argument about any of this, is there?

P
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Robert Krüger-2
On Sun, Aug 11, 2013 at 9:53 AM, Phil Rhodes <[hidden email]> wrote:
> Just duplicating frames is unlikely to produce results anyone wants to watch. Duplicating one frame per second to go from 24 (or nearly 24) to 25 was occasionally done in days of yore when people wanted to edit film-originated material on PAL video. It looks very rough.
>
> Properly done frame rate conversions that are intended to be of high quality, regardless of the desired change in rate, are invariably done with motion compensated interpolation. Of course this is not always an exact science either, but it's about the best option and properly done it is generally fairly close to invisible on most subjects I don't know if ffmpeg is capable of doing it.

No, there is currently no way to change frame rate via motion
interpolation (in some programs this is called "optical flow") in
ffmpeg.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Nicolas George-2
In reply to this post by thljcl
Le tridi 23 thermidor, an CCXXI, thljcl a écrit :
> Hm… Well, let’s put it to actual test, which should be a much better option
> than believing in either you or me.

The sliding coloured band of testsrc is already enough to see the problem,
although slightly due to the gradient. It is even more noticeable with a
more contrasted source, such as one produced by drawtext:

ffmpeg -lavfi 'color=black:r=24:d=20,drawtext=fontfile=sans:text=!:fontcolor=white:fontsize=1000:y=-10:x=t*24-200,fps=fps=25' -preset ultrafast -y t.mkv

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Nicolas George-2
In reply to this post by Robert Krüger-2
Le quartidi 24 thermidor, an CCXXI, Robert Krüger a écrit :
> No, there is currently no way to change frame rate via motion
> interpolation (in some programs this is called "optical flow") in
> ffmpeg.

There could be, and it is a worthy project. But still, even with
interpolation, converting between two frame rated differing by 0.1~2 Hz will
cause visible hiccups.

The same applies to scaling: take an image with sharp edges, it will look
uglier if you scale it by a few pixels than if you scale it a lot.

Regards,

--
  Nicolas George

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Phil Rhodes


> But still, even with interpolation, converting between 
> two frame rated differing by 0.1~2 Hz will cause visible hiccups.

That's not always the case, although of course it's tricky. There are commercial products and hardware devices which will do it. They end up creating effectively all of the output frames using optical flow, presumably using algorithms tweaked to the specific case of that particular set of frame rates.

But yes, there are special considerations.

P
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Andy Furniss-2
In reply to this post by thljcl
thljcl wrote:

> Hm… Well, let’s put it to actual test, which should be a much better
> option than believing in either you or me. Do you currently have a
> source of 24 FPS which you think that it has “high speed motion”? If
> you seriously think that using the method I describe would lead to
> “unrealistic motion”, can you share such a source with me and allow
> me to do the encoding process? After doing the encoding process, I
> will upload the output file to SkyDrive and share it temporarily
> publicly. We can then all use our eyes to make our judgment. I do
> believe that an actual experiment would help me understand better.
> Thank you.

Remember also that the 24 -> 25 case under discussion is going to show
better on a 50Hz display so when you say "We can then all use our eyes"
a lot of people will be viewing on a monitor at 60Hz that likely won't
do 50Hz.


_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Andy Furniss-2
In reply to this post by thljcl
thljcl wrote:
> I do have experiences over changing frame rate from 25 FPS, 29.97
> FPS, and 30 FPS to 24 FPS using the method I described. As far as my
> eyes are concerned, I do not have the so-called “unrealistic motion”
> issues. Some people have believed that higher frame rate is required
> for more realistic motion. Without regards to human eyes or
> perceptions

Well you totally do not consider the aspect of display technology and
the issue of film judder caused by the need to refresh > framerate to
avoid flicker perception.

Of course film makers know the limitations of there media and do use
blur and tricks, including just not attempting some shots to alleviate
judder perception. This does not mean that this is ideal.

I could link to a Poynton here but I thought you would want something
more recent :-)

http://www.bbc.co.uk/rd/publications/whitepaper209



_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Nicolas George-2
In reply to this post by Andy Furniss-2
Le quartidi 24 thermidor, an CCXXI, Andy Furniss a écrit :
> Remember also that the 24 -> 25 case under discussion is going to show
> better on a 50Hz display so when you say "We can then all use our eyes"
> a lot of people will be viewing on a monitor at 60Hz that likely won't
> do 50Hz.

Good remark. On a 60 Hz display, I re-ran the drawtext scrolling I posted a
few hours ago doing a 29->30 conversion instead 24->25, and it is even more
visible.

Regards,

--
  Nicolas George

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: ffprobe -show_frames and coded_picture_number

Phil Rhodes


> a lot of people will be viewing on a monitor at 60Hz that likely won't
> do 50Hz.


Bear in mind that almost no computer video playback software will show video frame-for-frame even if the frame rate of the pictures matches the monitor's refresh rate precisely. They may happen to do so, perhaps even for quite extended periods of time, but eventually it'll always glitch as the clocks on the graphics and sound hardware run in and out of phase.

P
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
12