Comparing audio to sync video

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Comparing audio to sync video

Joshua Grauman
Hi all,

I have two video cameras recording an event 30 minutes long. I hit start
on the two cameras at different times. I would like to figure out a
command-line script that can basically diff the audio to figure out the
difference between the start times and align the videos. The audio won't
be exactly identical as they are two different cameras, different mics,
different locations, but they are both recording an interview, so the
audio should be able to be synced. Any ideas?

Josh
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Comparing audio to sync video

Carl Eugen Hoyos-2
2018-12-18 12:36 GMT+01:00, Joshua Grauman <[hidden email]>:

> I have two video cameras recording an event 30 minutes long. I hit start
> on the two cameras at different times. I would like to figure out a
> command-line script that can basically diff the audio to figure out the
> difference between the start times and align the videos. The audio won't
> be exactly identical as they are two different cameras, different mics,
> different locations, but they are both recording an interview, so the
> audio should be able to be synced. Any ideas?

Are you asking how to find out the delay for one pair of videos?
You could use a filter chain and audio visualization to see the
delay.

Or are you searching for an automated process to get two independent
video in-sync?

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Comparing audio to sync video

Carl Zwanzig
On 12/18/2018 10:27 AM, Carl Eugen Hoyos wrote:
> 2018-12-18 12:36 GMT+01:00, Joshua Grauman <[hidden email]>:
>
>> I have two video cameras recording an event 30 minutes long.[...] The audio won't
>> be exactly identical as they are two different cameras, different mics,
>> different locations, but they are both recording an interview, so the
>> audio should be able to be synced.

So, effectively you have two -different- videos. To a human, they may sound
and look mostly the same, but really they're different. "be able to be
synced" does not necessarily imply "easy to be done" here. (If they had the
same audio feed, it would be much easier. Heck, if they both recorded real
timecode, it would be almost trivial.)

> Or are you searching for an automated process to get two independent
> video in-sync?

If that's the goal, there needs to be a readily identifiable sound or visual
marker at the start of the videos as a sync point, then a process to -find-
those and get a timecode. The process also needs to know what that marker is
to look for it. There's a non-trivial amount of processing involved to find
and make the identification.

If this is an occasional task, you're better off opening the two videos in
an editor and sliding the timelines around so they match.

Later,

z!
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Comparing audio to sync video

Joshua Grauman
On Tue, 18 Dec 2018, Carl Zwanzig wrote:

>>  2018-12-18 12:36 GMT+01:00, Joshua Grauman <[hidden email]>:
>>
>>>  I have two video cameras recording an event 30 minutes long.[...] The
>>>  audio won't
>>>  be exactly identical as they are two different cameras, different mics,
>>>  different locations, but they are both recording an interview, so the
>>>  audio should be able to be synced.
>
> So, effectively you have two -different- videos. To a human, they may sound
> and look mostly the same, but really they're different. "be able to be
> synced" does not necessarily imply "easy to be done" here. (If they had the
> same audio feed, it would be much easier. Heck, if they both recorded real
> timecode, it would be almost trivial.)
>
>>  Or are you searching for an automated process to get two independent
>>  video in-sync?
>
> If that's the goal, there needs to be a readily identifiable sound or visual
> marker at the start of the videos as a sync point, then a process to -find-
> those and get a timecode. The process also needs to know what that marker is
> to look for it. There's a non-trivial amount of processing involved to find
> and make the identification.
>
> If this is an occasional task, you're better off opening the two videos in an
> editor and sliding the timelines around so they match.

Yes, two different videos but which should be strongly correlated. I found
some code to compute cross-correlation which should work, see my previous
post. Thanks for the suggestion about an easy to find marker, such as a
'clap' or something, that could also come in handy.

And sorry for top-posting in my last post. Bad habit...

Josh
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Comparing audio to sync video

Michael Koch
In reply to this post by Joshua Grauman

> I have two video cameras recording an event 30 minutes long. I hit
> start on the two cameras at different times. I would like to figure
> out a command-line script that can basically diff the audio to figure
> out the difference between the start times and align the videos. The
> audio won't be exactly identical as they are two different cameras,
> different mics, different locations, but they are both recording an
> interview, so the audio should be able to be synced. Any ideas?

I think to automatically find the best-fit audio delay between the two
tracks you would need to calculate a cross correlation of the two audio
tracks. As far as I know this doesn't exist in ffmpeg. But it would be
nice to have.

Michael

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Comparing audio to sync video

Joshua Grauman
Yes, exactly. Thanks for the mathematical name for what I'm trying to do.
I don't necessarily need to do it in ffmpeg, so I found this, which should
be easy to adapt for my needs (includes sample C code).

http://paulbourke.net/miscellaneous/correlate

Josh

>
>>  I have two video cameras recording an event 30 minutes long. I hit start
>>  on the two cameras at different times. I would like to figure out a
>>  command-line script that can basically diff the audio to figure out the
>>  difference between the start times and align the videos. The audio won't
>>  be exactly identical as they are two different cameras, different mics,
>>  different locations, but they are both recording an interview, so the
>>  audio should be able to be synced. Any ideas?
>
> I think to automatically find the best-fit audio delay between the two tracks
> you would need to calculate a cross correlation of the two audio tracks. As
> far as I know this doesn't exist in ffmpeg. But it would be nice to have.
>
> Michael
>
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
>
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".