Inaccurate trimming with re-encoding

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Inaccurate trimming with re-encoding

cmharwood89
Hi all,

I'm a fairly new ffmpeg user, but I've found it really helpful for processing videos as part of my graduate research. I've got a mess of videos (recordings of experiments from different angles) that I'm trying to synchronize. I coded my data acquisition system such that an LED turns on when I begin collecting data, giving me a visual indicator of where I can sync the videos, which should be accurate to within ~1/2 frame.

So I've got a script to trim my videos to the frames at which I see the LED turn on (in HH:MM:SS[.ms] format). I tried it with -vcodec copy, but (predictably) I found my outputs began at the nearest I-frame, up to 1s away from my desired trim point.

My understanding is that, if I re-encode, I should be able to trim at exactly the desired time and place an I-frame at the beginning of the saved output. An example command is using a gopro video shot at 59.94 fps, with h.264 codec:

ffmpeg -t 30 -i GOPR0103.mp4 -vcodec libx264 –threads 0 -preset ultrafast -an -y -ss 00:03:09[.606] Run041_UW1.avi

Here's where I'm confused. If I execute the above command, I do get a trimmed output, but the resulting video starts 40 frames away from the desired frame. In contrast, if I use -vcodec copy on the same file, the output is only off by ~26 frames.

What am I doing wrong here? Is there any way for me to get frame-accurate splitting? I'm okay with re-encoding if that's what it takes. I have about 800 videos to chew through, but I can run my lab workstation for a week straight to get it done.

Thanks very much!
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

thljcl
Did you try to cut out a portion of video from a longer video?
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

Mark Himsley-2
In reply to this post by cmharwood89
The square brackets around the decimal parts of seconds is to indicate to
you that they are optional. Remove the square brackets from your command.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

cmharwood89
@thljcl - Yes, I'm cutting a section from a longer video. I have an camera running continuously during physics experiments and I need to segment the master video into individual movies for each experimental condition.

@Mark Himsley - I know that the square brackets are optional, but I need the millisecond resolution to precisely locate the frame I want. I need to cut the video at exactly the frame where an LED turns on. From my reading, it looks like there's not a way to specify a cutting point by frame number, so the HH:MM:SS[.ms] format is the only way to get the precision I need.

Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

cmharwood89
Or am I misunderstanding what you're saying? Do you mean to include the milliseconds, but without the square brackets? I thought you were suggesting to truncate the time down to HH:MM:SS format.
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

thljcl
You do know that how small the video can be cut out from the source depends on your camera frame rate, do you not? I do not think that there is a consumer-application camera that can take 1000 high-resolution pictures per second. The maximum frame rate supported by H.264 is 172 FPS. For normal playback though, it would be necessary to convert the frame rate to 24 FPS, 25 FPS, or 30 FPS for smooth playback. Storing video in 172 FPS is mainly for editing purposes and being kept as master copy.
Yes, it’s possible to cut out the video from the source based on frame numbers, assuming that your video source is of constant frame rate. Only with constant frame rate, you can then tell the length of video by counting the number of available frames. Is cutting out a portion of video based on number of frames is what you are looking for? In fact, that’s the most accurate we can get in terms of cutting out portions of video from a longer video.
Tell me exactly if I get your question right. If you do want to know how to cut the video based on the number of frames, I would tell you later on this forum.
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

Dave P-3
In reply to this post by cmharwood89
On Aug 5, 2013 10:26 AM, "cmharwood89" <[hidden email]> wrote:
>
> @thljcl - Yes, I'm cutting a section from a longer video. I have an camera
> running continuously during physics experiments and I need to segment the
> master video into individual movies for each experimental condition.
>
> @Mark Himsley - I know that the square brackets are optional, but I need
the
> millisecond resolution to precisely locate the frame I want. I need to cut
> the video at exactly the frame where an LED turns on. From my reading, it
> looks like there's not a way to specify a cutting point by frame number,
so
> the HH:MM:SS[.ms] format is the only way to get the precision I need.

The select filter mentioned in this recent thread might help:
lists.ffmpeg.org/pipermail/ffmpeg-user/2013-July/016291.html
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

cmharwood89
In reply to this post by thljcl
I'm working with a 60 FPS (or 59.94) video, though the camera is ostensibly able to capture up to 240 FPS (under which codec I'm not sure). Later on, I may be capturing video at 1000+ FPS, but I'll be using machine-vision cameras with their own controllers; I doubt I'll need to do the same kind of post-processing with those.

I've been using AVIdemux to seek the exact frame I want, and then pulling the time marker in HH:MM:SS.ms format from the AVIdemux GUI, which I then use with '-ss' command in ffmpeg to get trimming. Mark's tip to remove the square brackets seemed to do the trick, because my output begins with the exact frame I picked in AVIdemux in my test case.

The duration of the cut video is less important; I do not need the trimmed video to be a specific number of frames. I do need it to begin at a specific frame in the master video. My method for doing this is a bit convoluted, so if specifying a frame number to begin cutting at is more accurate, I'd really appreciate any tips on how to do that.

I think that in the future, I'll buck up and get nicer cameras that have millisecond-precision triggering so I don't have to trim the extra dead time from each video.

Thanks so much!
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

thljcl
You said that you were already able to get the job done but still want to know the way to cut a video from a specific frame number?
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

cmharwood89
I'm able to get it done using time markers, yes. However, if cutting from a frame number is a more precise method, I'd like to do that instead.


On Mon, Aug 5, 2013 at 10:37 AM, thljcl [via FFmpeg-users] <[hidden email]> wrote:
You said that you were already able to get the job done but still want to know the way to cut a video from a specific frame number?


If you reply to this email, your message will be added to the discussion below:
http://ffmpeg-users.933282.n4.nabble.com/Inaccurate-trimming-with-re-encoding-tp4660437p4660465.html
To unsubscribe from Inaccurate trimming with re-encoding, click here.
NAML



--
Casey M. Harwood
Ph.D. Candidate
NSF GRFP Fellow
Naval Architecture & Marine Engineering
University of Michigan
Ann Arbor, MI
360-589-9626
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

thljcl
Does your source have one or more audio tracks? You do not know of the codec which is the source is encoded in?
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

James Board
In reply to this post by cmharwood89
I'm able to do frame-exact editing with ffmpeg.  It's described in this thread:


    http://lists.ffmpeg.org/pipermail/ffmpeg-user/2013-July/016291.html

The command is something like this (thanks to Carl Lindqvist)
ffmpeg -i SOURCEFILE.avi -vf
select='-between(n,0,100)-between(n,130,200)',setpts=N/FRAME_RATE/TB
-c:v libx264 -preset veryfast -b:v 700k -profile:v main -af
aselect='-between(n,0,100)-between(n,130,200)',asetpts=N/SR/TB -c:a
libvo_aacenc -b:a 96k -ac 2 OUT.mp4
I verified that the frames are exactly what I wanted by checking the md5sums of each frame.

My input avi file used a raw codec with no inter-frame compression.  I'm not sure if frame-exact editing will work when the input file is compressed with mpeg-like codecs.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

thljcl
In reply to this post by cmharwood89


Well, I hope the above slideshow I created can help you.
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

cmharwood89
@ James,

Thanks for the referral. I checked out the link and I'll probably use the filter eventually. For the time being, I'm making-do with time-indexed trimming.

@thljcl,

Wow, thanks for putting together those slides! That looks like a very helpful solution. I see the value of changing to 1fps, because I could essentially use time as a dummy variable for frame number, but I don't understand how that would allow me to trim at non keyframes.

From my (limited) understanding, there simply is no way to trim at a P-frame without decompressing into a raw format first, trimming at the shiny new keyframe, and then recompressing back into my desired. Right now, I'm having pretty good results doing it this way. If I change the framerate to 1fps, it's not going to redistribute keyframes, right? If I choose a cutting point, say at frame 300, but the nearest I-frame is at 295, then my output video will be trimmed at 5 frames/5 seconds from the desired cut-point.

For reference, my input and desired output are:
INPUT:
h264 codec @59.94 fps, 1920x1080 res; AAC audio codec; MP4 container

OUTPUT
h264 codec @59.94 fps, 1920x1080 res; NO AUDIO; AVI container

I want the output video to be encoded with no subjectively observable losses.  Computing time and disk space are not a concern, but video quality and accurate trimming are absolutely necessary.

My original problem has been solved (by removing the square brackets surrounding my millisecond count in the -ss argument), but I'm definitely interested in making my approach smarter- i.e. avoiding a brute-force complete encoding of the video. I like the speed and losslessness of container-only manipulation and cutting at keyframes, and I like the accuracy of a decompress-then-cut approach. My new challenge is finding a way to combine the two.
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

thljcl
This post was updated on .
cmharwood89 wrote
@thljcl,

Wow, thanks for putting together those slides! That looks like a very helpful solution. I see the value of changing to 1fps, because I could essentially use time as a dummy variable for frame number, but I don't understand how that would allow me to trim at non keyframes.

From my (limited) understanding, there simply is no way to trim at a P-frame without decompressing into a raw format first, trimming at the shiny new keyframe, and then recompressing back into my desired. Right now, I'm having pretty good results doing it this way. If I change the framerate to 1fps, it's not going to redistribute keyframes, right? If I choose a cutting point, say at frame 300, but the nearest I-frame is at 295, then my output video will be trimmed at 5 frames/5 seconds from the desired cut-point.

For reference, my input and desired output are:
INPUT:
h264 codec @59.94 fps, 1920x1080 res; AAC audio codec; MP4 container

OUTPUT
h264 codec @59.94 fps, 1920x1080 res; NO AUDIO; AVI container

I want the output video to be encoded with no subjectively observable losses.  Computing time and disk space are not a concern, but video quality and accurate trimming are absolutely necessary.

My original problem has been solved (by removing the square brackets surrounding my millisecond count in the -ss argument), but I'm definitely interested in making my approach smarter- i.e. avoiding a brute-force complete encoding of the video. I like the speed and losslessness of container-only manipulation and cutting at keyframes, and I like the accuracy of a decompress-then-cut approach. My new challenge is finding a way to combine the two.
In H.264, or any other codecs, there is a compression technique called inter-frame compression. Because of inter-frame compression, typically you cannot cut H.264 video stream based on frame numbers without round-off error. You’re right, round-off error ‘usually’ resulted in inaccurate trimming; instead of cutting at exactly the frame number you specified, what you got was usually with round-off error. In fact, that’s exactly what happened if you use mkvtoolnix directly to cut the video into pieces.
However, the method I described in the slideshow did not have such issue. The round-off error problem was put away because of the constant frame rate of 1 FPS. No, it would not get to the nearest key frame; it would get to exactly the frame or ‘time’ you specified. If you think about it, it’s pretty much like how to extract all frames (no matter whether or not they are key frames) to individual images; to be in more details, it actually involves both decoding and encoding process.
The typical lossy encoding process would result in the loss of quality. Strictly speaking, it’s not possible for absolutely accurate trimming to happen without re-encoding process for the source with inter-frame compression. However, what really happens here is can be thought to be “lossless encoding”; but it’s not to create a video file of “lossless compression” but just cut a portion of video out. As you may already know -codec “copy” is to instruct the encoder the copy the stream without re-encoding. But you can also say re-encoding is also happening; it’s just that no change of information would occur in direct stream copying. To be more specific, you just copy a single frame from the H.264 source; if that happens to be non-key frame from the source, it would be the key frame for the output file; but this is strictly stream-copying process, thus no mathematical loss of information would occur in this process; it’s copying data from multiple frames from the source to be a single key frame in the output file.
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

cmharwood89
thljcl wrote
In H.264, or any other codecs, there is a compression technique called inter-frame compression. Because of inter-frame compression, typically you cannot cut H.264 video stream based on frame numbers without round-off error. You’re right, round-off error ‘usually’ resulted in inaccurate trimming; instead of cutting at exactly the frame number you specified, what you got was usually with round-off error. In fact, that’s exactly what happened if you use mkvtoolnix directly to cut the video into pieces.
However, the method I described in the slideshow did not have such issue. The round-off error problem was put away because of the constant frame rate of 1 FPS. No, it would not get to the nearest key frame; it would get to exactly the frame or ‘time’ you specified. If you think about it, it’s pretty much like how to extract all frames (no matter whether or not they are key frames) to individual images; to be in more details, it actually involves both decoding and encoding process.
The typical lossy encoding process would result in the loss of quality. Strictly speaking, it’s not possible for absolutely accurate trimming to happen without re-encoding process for the source with inter-frame compression. However, what really happens here is can be thought to be “lossless encoding”; but it’s not to create a video file of “lossless compression” but just cut a portion of video out. As you may already know -codec “copy” is to instruct the encoder the copy the stream without re-encoding. But you can also say re-encoding is also happening; it’s just that no change of information would occur in direct stream copying. To be more specific, you just copy a single frame from the H.264 source; if that happens to be non-key frame from the source, it would be the key frame for the output file; but this is strictly stream-copying process, thus no mathematical loss of information would occur in this process; it’s copying data from multiple frames from the source to be a single key frame in the output file.
I stand corrected. Thanks a ton for taking the time to explain that. I'm definitely going to try this out. It looks like mkvtoolnix has a command line execution, so I should be able to execute everything programmatically.
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

thljcl
In reply to this post by thljcl
thljcl wrote
In H.264, or any other codecs, there is a compression technique called inter-frame compression. Because of inter-frame compression, typically you cannot cut H.264 video stream based on frame numbers without round-off error. You’re right, round-off error ‘usually’ resulted in inaccurate trimming; instead of cutting at exactly the frame number you specified, what you got was usually with round-off error. In fact, that’s exactly what happened if you use mkvtoolnix directly to cut the video into pieces.
However, the method I described in the slideshow did not have such issue. The round-off error problem was put away because of the constant frame rate of 1 FPS. No, it would not get to the nearest key frame; it would get to exactly the frame or ‘time’ you specified. If you think about it, it’s pretty much like how to extract all frames (no matter whether or not they are key frames) to individual images; to be in more details, it actually involves both decoding and encoding process.
The typical lossy encoding process would result in the loss of quality. Strictly speaking, it’s not possible for absolutely accurate trimming to happen without re-encoding process for the source with inter-frame compression. However, what really happens here is can be thought to be “lossless encoding”; but it’s not to create a video file of “lossless compression” but just cut a portion of video out. As you may already know -codec “copy” is to instruct the encoder the copy the stream without re-encoding. But you can also say re-encoding is also happening; it’s just that no change of information would occur in direct stream copying. To be more specific, you just copy a single frame from the H.264 source; if that happens to be non-key frame from the source, it would be the key frame for the output file; but this is strictly stream-copying process, thus no mathematical loss of information would occur in this process; it’s copying data from multiple frames from the source to be a single key frame in the output file.
While I use mkvtoolnix to manipulate the MKV container, if you like your video/audio track to be stored in other containers, if you use the -codec “copy” to do that; that’s subject to limitation of the containers; other containers may not have supported the as many codecs as MKV does. Not only the video/audio streams would be copied, the stored container information would be copied as well, subject to the container’s limitation. If not, as I said earlier, the output file cannot be played correctly because h.264 elementary stream does not store frame rate information.
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

thljcl
In reply to this post by cmharwood89
cmharwood89 wrote
I stand corrected. Thanks a ton for taking the time to explain that. I'm definitely going to try this out. It looks like mkvtoolnix has a command line execution, so I should be able to execute everything programmatically.
I typically write batch file to let ffmpeg execute a list of encoding jobs, instead of writing a line of command directly in command processor. In my previous posts of “Create slideshow with ffmpeg from still images” at http://ffmpeg-users.933282.n4.nabble.com/Create-slideshow-with-ffmpeg-from-still-images-td4660277.html, I even published the batch files I created. As promised, I did remove the slideshow with slides of varying length together with the batch file from SkyDrive.
I did try to include mkvtoolnix command in my batch file. Theoretically, it should work. In practice, when it comes to creating batch file, I did make careless mistakes quite often. The more complex the batch file got, the harder for me to iron out the problems. I use the GUI front end because of the ease of use. The visual feedback does help to reduce careless mistakes. On the other hand, I do appreciate automated processes, if not I would not prefer creating batch files for different tasks. So, at the end, I use the combination of command-line UI and GUI. If you are a person who is extremely careful and rarely make careless mistakes, I would admit that doing job programmatically sounds a lot better.
GUI front end for x264 does exist, such as HandBrake. HandBrake even has a command-line front end. But I eventually choose ffmpeg (as x264 front end) instead of HandBrake because of the number of options I got from ffmpeg. I do like more functionalities, features, or options, whatever you might call them. I do not necessarily prefer one over another; it really depends on the usage scenarios. I still keep HandBrake around.
Reply | Threaded
Open this post in threaded view
|

Re: Inaccurate trimming with re-encoding

Mark Himsley-2
In reply to this post by cmharwood89
I said "The square brackets around the decimal parts of seconds is to
indicate to you that they are optional."

What I mean is, the format is HH:MM:SS[.ms], so you can specify 01:02:03
for hours, minutes and seconds, or 01:02:03.48 for hours minutes, seconds
and decimal parts of seconds.

You said your command was "ffmpeg -t 30 -i GOPR0103.mp4 -vcodec libx264
–threads 0 -preset ultrafast
-an -y -ss 00:03:09[.606] Run041_UW1.avi"

I said "Remove the square brackets from your command."

What I mean is, try your command as:

ffmpeg -t 30 -i GOPR0103.mp4 -vcodec libx264 –threads 0 -preset ultrafast
-an -y -ss 00:03:09.606 Run041_UW1.avi

See. No square brackets.



On 5 August 2013 15:03, cmharwood89 <[hidden email]> wrote:

> Or am I misunderstanding what you're saying? Do you mean to include the
> milliseconds, but without the square brackets? I thought you were
> suggesting
> to truncate the time down to HH:MM:SS format.
>
>
>
> --
> View this message in context:
> http://ffmpeg-users.933282.n4.nabble.com/Inaccurate-trimming-with-re-encoding-tp4660437p4660459.html
> Sent from the FFmpeg-users mailing list archive at Nabble.com.
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>



--
Mark Himsley
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user