Muxing multiple files and concatenating those outputs

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Muxing multiple files and concatenating those outputs

Greg Oliver
Hi,

We currently use ffmpeg to combine 2 streams (telephony codecs) into
mp3/wav/flac, etc..  This is no problem - we use:

ffmpeg -ar 8000 -y -f mulaw -i calls/example.g711a.ulaw -ar 8000 -f mulaw
-i calls/example.g711b.ulaw -filter_complex "[0:a][1:a]amerge[aout]" -map
"[aout]" output.wav

If we have multiple legs of a recording, we just use linux `cat' to
concatenate them prior to ffmpeg processing (we have raw audio frames
without headers, etc, so this works fine).

Due to Covid, almost all of our customers have moved their call centers to
employee homes, so we have mixed codec calls and the necessity has arose to
generate audio files accordingly.  It is easily accomplished with multiple
temporary files, but I would like to clean it up and have been unsuccessful
in coming up with a single string to accomplish it.

Example using these files (suffix denotes codec - a and b are each side of
a call):

18_17_248.g729a
18_17_248.g729b
19_18_440.g711a
19_18_440.g711b
20_01_886.g729a
20_01_886.g729b

Current method to concatenate and transcode to wav putting caller on left
and callee on right:

ffmpeg -f g729 -i 18_17_248.g729a -f g729 -i 18_17_248.g729b
-filter_complex "[0:a][1:]amerge[aout]" -map "[aout]" out_1.wav

ffmpeg -ar 8000 -y -f mulaw -i 19_18_440.g711a -ar 8000 -f mulaw -i
19_18_440.g711b -filter_complex "[0:a][1:a]amerge[aout]" -map "[aout]"
out_2.wav

ffmpeg -f g729 -i 20_01_886.g729a -f g729 -i 20_01_886.g729b
-filter_complex "[0:a][1:]amerge[aout]" -map "[aout]" out_3.wav

ffmpeg out_1.wav out_2.wav out_3.wav final.wav

My goal is to syntactically accomplish this with one ffmpeg string (not
pipes, ; or && bash syntax).

I would appreciate any insight - I have tried everything I can find / think
of without success.

TiA,

Greg
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Muxing multiple files and concatenating those outputs

kumowoon1025
Hi,

> Example using these files (suffix denotes codec - a and b are each side of
> a call):
>
> 18_17_248.g729a
> 18_17_248.g729b
> 19_18_440.g711a
> 19_18_440.g711b
> 20_01_886.g729a
> 20_01_886.g729b
>
That is really confusing...

> Current method to concatenate and transcode to wav putting caller on left
> and callee on right:
>
> ffmpeg -f g729 -i 18_17_248.g729a -f g729 -i 18_17_248.g729b
> -filter_complex "[0:a][1:]amerge[aout]" -map "[aout]" out_1.wav
>
> ffmpeg -ar 8000 -y -f mulaw -i 19_18_440.g711a -ar 8000 -f mulaw -i
> 19_18_440.g711b -filter_complex "[0:a][1:a]amerge[aout]" -map "[aout]"
> out_2.wav
>
> ffmpeg -f g729 -i 20_01_886.g729a -f g729 -i 20_01_886.g729b
> -filter_complex "[0:a][1:]amerge[aout]" -map "[aout]" out_3.wav
>
> ffmpeg out_1.wav out_2.wav out_3.wav final.wav
>
> My goal is to syntactically accomplish this with one ffmpeg string (not
> pipes, ; or && bash syntax).
>
> I would appreciate any insight - I have tried everything I can find / think
> of without success.
>
The final command is supposed to concatenate the out_?.wav files into the final.wav file right?

I think the way you have them written, it makes it easy to combine it into one command, though I have to wonder what benefit (if any) you seek to get from it. Because, if you're scripting it, it really shouldn't matter, and if you're doing it by hand, doesn't it make one big command that has to be repeated if it fails? Anyhow,

> ffmpeg -f g729 -i 18_17_248.g729a -f g729 -i 18_17_248.g729b -ar 8000 -y -f mulaw -i 19_18_440.g711a -ar 8000 -f mulaw -i
> 19_18_440.g711b -f g729 -i 20_01_886.g729a -f g729 -i 20_01_886.g729b -filter_complex '[0][1]amerge,afifo[l1];[2][3]amerge,afifo[l2];[4][5]amerge,afifo[l3];[l1][l2][l3]concat=v=0:a=1:n=3' <insert output settings and output uri here>

I think this will do what you want, the individual input pads to amerge and/or fifo might not be needed.

Regards,
Ted Park

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Muxing multiple files and concatenating those outputs

Greg Oliver
On Tue, Apr 14, 2020 at 3:30 PM Ted Park <[hidden email]> wrote:

> Hi,
>
> > Example using these files (suffix denotes codec - a and b are each side
> of
> > a call):
> >
> > 18_17_248.g729a
> > 18_17_248.g729b
> > 19_18_440.g711a
> > 19_18_440.g711b
> > 20_01_886.g729a
> > 20_01_886.g729b
> >
> That is really confusing...
>

Yeah, when we record the calls, the directory structure preceeding those
names is YYYY/MM/DD/HH and the filenames are MIN_SEC_MSEC.codec(side)



> > Current method to concatenate and transcode to wav putting caller on left
> > and callee on right:
> >
> > ffmpeg -f g729 -i 18_17_248.g729a -f g729 -i 18_17_248.g729b
> > -filter_complex "[0:a][1:]amerge[aout]" -map "[aout]" out_1.wav
> >
> > ffmpeg -ar 8000 -y -f mulaw -i 19_18_440.g711a -ar 8000 -f mulaw -i
> > 19_18_440.g711b -filter_complex "[0:a][1:a]amerge[aout]" -map "[aout]"
> > out_2.wav
> >
> > ffmpeg -f g729 -i 20_01_886.g729a -f g729 -i 20_01_886.g729b
> > -filter_complex "[0:a][1:]amerge[aout]" -map "[aout]" out_3.wav
> >
> > ffmpeg out_1.wav out_2.wav out_3.wav final.wav
> >
> > My goal is to syntactically accomplish this with one ffmpeg string (not
> > pipes, ; or && bash syntax).
> >
> > I would appreciate any insight - I have tried everything I can find /
> think
> > of without success.
> >
> The final command is supposed to concatenate the out_?.wav files into the
> final.wav file right?
>
> I think the way you have them written, it makes it easy to combine it into
> one command, though I have to wonder what benefit (if any) you seek to get
> from it. Because, if you're scripting it, it really shouldn't matter, and
> if you're doing it by hand, doesn't it make one big command that has to be
> repeated if it fails? Anyhow,
>
> > ffmpeg -f g729 -i 18_17_248.g729a -f g729 -i 18_17_248.g729b -ar 8000 -y
> -f mulaw -i 19_18_440.g711a -ar 8000 -f mulaw -i
> > 19_18_440.g711b -f g729 -i 20_01_886.g729a -f g729 -i 20_01_886.g729b
> -filter_complex
> '[0][1]amerge,afifo[l1];[2][3]amerge,afifo[l2];[4][5]amerge,afifo[l3];[l1][l2][l3]concat=v=0:a=1:n=3'
> <insert output settings and output uri here>
>
> I think this will do what you want, the individual input pads to amerge
> and/or fifo might not be needed.
>

Yeah, the only issue is that if there are 10 legs of a call (think patient
calling a hospital and asking for neurology, getting the neurology
receptionist and then being transferred to a doctor who does not answer and
rings back to the receptionist who does not answer and then finally
arriving back at the main switchboard and the cycle starts over).  We may
have quite a few legs, so we must mux each leg prior to the concatenation.

I was really looking for just syntax to group commands, so I could use the
merge filter output as direct input for the concatenation.  The man page
and web searches I tried came up empty, but I figured someone may know some
magic sauce I could not find.

Thanks!



> Regards,
> Ted Park
>
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Muxing multiple files and concatenating those outputs

kumowoon1025
Hi,

> Yeah, when we record the calls, the directory structure preceeding those
> names is YYYY/MM/DD/HH and the filenames are MIN_SEC_MSEC.codec(side)


I mean some phones specifically put g729a, and I assume it's the same for g729b. So I started  imagining files ending all over the place g729aa, g729ba, etc.

> I was really looking for just syntax to group commands, so I could use the
> merge filter output as direct input for the concatenation.

If you mean the afifo I inserted, you could probably get rid of those if the machine's fast enough, or the calls short enough. Or a different approach might be padding each file in the same call or using the cue filter, and mixing them all. But grouping commands isn't really a thing that ffmpeg does, except for things like image sequences, preparing segmented delivery media.

> The man page and web searches I tried came up empty, but I figured someone may know some magic sauce I could not find.


Maybe you've been looking for the wrong terms? It sounds to me like what you are looking for is closer to shell features, like parameter substitution, filename generation, etc.

If portability is not an issue, some shells have more features than others, though there's a different learning curve to each one.

Or there's always xargs. Yeah probably xargs. I can't really tell but the associated filenames seem pretty much arbitrary. Do you parse the date/time in the filename to find which ones to put together? Or is there a call log to reference?

Regards,
Ted Park

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Muxing multiple files and concatenating those outputs

Greg Oliver
On Wed, Apr 15, 2020 at 5:43 AM Ted Park <[hidden email]> wrote:

> Hi,
>
> > Yeah, when we record the calls, the directory structure preceeding those
> > names is YYYY/MM/DD/HH and the filenames are MIN_SEC_MSEC.codec(side)
>
>
> I mean some phones specifically put g729a, and I assume it's the same for
> g729b. So I started  imagining files ending all over the place g729aa,
> g729ba, etc.
>

Ahhh, I see what you are referring to now :)

g729b is just an annex to a that only allows for more header enhancements -
the raw codec audio is the same though, so we just write g729 for both.



> > I was really looking for just syntax to group commands, so I could use
> the
> > merge filter output as direct input for the concatenation.
>
> If you mean the afifo I inserted, you could probably get rid of those if
> the machine's fast enough, or the calls short enough. Or a different
> approach might be padding each file in the same call or using the cue
> filter, and mixing them all. But grouping commands isn't really a thing
> that ffmpeg does, except for things like image sequences, preparing
> segmented delivery media.
>

I see the restraints I am up against now and I think what I am trying is
just not possible, so I will take the path of least resistance and keep it
multi-lined.  To clarify, the reason I need to merge channels prior is that
the 2 sides do not always exactly line up (there almost always is a
fraction of a second less on one side of the packet stream) and it causes
conversation sync/skew if they are concatenated either prior or
simultaneously with the merging.



> > The man page and web searches I tried came up empty, but I figured
> someone may know some magic sauce I could not find.
>
>
> Maybe you've been looking for the wrong terms? It sounds to me like what
> you are looking for is closer to shell features, like parameter
> substitution, filename generation, etc.
>

Yeah, I am currently using mktemp, etc..  I was just hoping to simplify it
because IO is the limiting factor when it comes to call recording, so
minimizing it is always a prioriry for me.



> If portability is not an issue, some shells have more features than
> others, though there's a different learning curve to each one.
>
> Or there's always xargs. Yeah probably xargs. I can't really tell but the
> associated filenames seem pretty much arbitrary. Do you parse the date/time
> in the filename to find which ones to put together? Or is there a call log
> to reference?
>
> Regards,
> Ted Park
>

Thanks again for all of your input Ted.

-Greg
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Muxing multiple files and concatenating those outputs

Nicolas George
In reply to this post by Greg Oliver
Greg Oliver (12020-04-14):

> We currently use ffmpeg to combine 2 streams (telephony codecs) into
> mp3/wav/flac, etc..  This is no problem - we use:
>
> ffmpeg -ar 8000 -y -f mulaw -i calls/example.g711a.ulaw -ar 8000 -f mulaw
> -i calls/example.g711b.ulaw -filter_complex "[0:a][1:a]amerge[aout]" -map
> "[aout]" output.wav
>
> If we have multiple legs of a recording, we just use linux `cat' to
> concatenate them prior to ffmpeg processing (we have raw audio frames
> without headers, etc, so this works fine).
>
> Due to Covid, almost all of our customers have moved their call centers to
> employee homes, so we have mixed codec calls and the necessity has arose to
> generate audio files accordingly.  It is easily accomplished with multiple
> temporary files, but I would like to clean it up and have been unsuccessful
> in coming up with a single string to accomplish it.
>
> Example using these files (suffix denotes codec - a and b are each side of
> a call):
>
> 18_17_248.g729a
> 18_17_248.g729b
> 19_18_440.g711a
> 19_18_440.g711b
> 20_01_886.g729a
> 20_01_886.g729b
>
> Current method to concatenate and transcode to wav putting caller on left
> and callee on right:
>
> ffmpeg -f g729 -i 18_17_248.g729a -f g729 -i 18_17_248.g729b
> -filter_complex "[0:a][1:]amerge[aout]" -map "[aout]" out_1.wav
>
> ffmpeg -ar 8000 -y -f mulaw -i 19_18_440.g711a -ar 8000 -f mulaw -i
> 19_18_440.g711b -filter_complex "[0:a][1:a]amerge[aout]" -map "[aout]"
> out_2.wav
>
> ffmpeg -f g729 -i 20_01_886.g729a -f g729 -i 20_01_886.g729b
> -filter_complex "[0:a][1:]amerge[aout]" -map "[aout]" out_3.wav
>
> ffmpeg out_1.wav out_2.wav out_3.wav final.wav
>
> My goal is to syntactically accomplish this with one ffmpeg string (not
> pipes, ; or && bash syntax).
>
> I would appreciate any insight - I have tried everything I can find / think
> of without success.
I suspect what you need is the concat filter. But you were not clear
enough about what exactly you are trying to achieve. Maybe make diagram?

Regards,

--
  Nicolas George

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Muxing multiple files and concatenating those outputs

Greg Oliver
On Thu, Apr 16, 2020 at 10:19 AM Nicolas George <[hidden email]> wrote:

> Greg Oliver (12020-04-14):
> > We currently use ffmpeg to combine 2 streams (telephony codecs) into
> > mp3/wav/flac, etc..  This is no problem - we use:
> >
> > ffmpeg -ar 8000 -y -f mulaw -i calls/example.g711a.ulaw -ar 8000 -f mulaw
> > -i calls/example.g711b.ulaw -filter_complex "[0:a][1:a]amerge[aout]" -map
> > "[aout]" output.wav
> >
> > If we have multiple legs of a recording, we just use linux `cat' to
> > concatenate them prior to ffmpeg processing (we have raw audio frames
> > without headers, etc, so this works fine).
> >
> > Due to Covid, almost all of our customers have moved their call centers
> to
> > employee homes, so we have mixed codec calls and the necessity has arose
> to
> > generate audio files accordingly.  It is easily accomplished with
> multiple
> > temporary files, but I would like to clean it up and have been
> unsuccessful
> > in coming up with a single string to accomplish it.
> >
> > Example using these files (suffix denotes codec - a and b are each side
> of
> > a call):
> >
> > 18_17_248.g729a
> > 18_17_248.g729b
> > 19_18_440.g711a
> > 19_18_440.g711b
> > 20_01_886.g729a
> > 20_01_886.g729b
> >
> > Current method to concatenate and transcode to wav putting caller on left
> > and callee on right:
> >
> > ffmpeg -f g729 -i 18_17_248.g729a -f g729 -i 18_17_248.g729b
> > -filter_complex "[0:a][1:]amerge[aout]" -map "[aout]" out_1.wav
> >
> > ffmpeg -ar 8000 -y -f mulaw -i 19_18_440.g711a -ar 8000 -f mulaw -i
> > 19_18_440.g711b -filter_complex "[0:a][1:a]amerge[aout]" -map "[aout]"
> > out_2.wav
> >
> > ffmpeg -f g729 -i 20_01_886.g729a -f g729 -i 20_01_886.g729b
> > -filter_complex "[0:a][1:]amerge[aout]" -map "[aout]" out_3.wav
> >
> > ffmpeg out_1.wav out_2.wav out_3.wav final.wav
> >
> > My goal is to syntactically accomplish this with one ffmpeg string (not
> > pipes, ; or && bash syntax).
> >
> > I would appreciate any insight - I have tried everything I can find /
> think
> > of without success.
>
> I suspect what you need is the concat filter. But you were not clear
> enough about what exactly you are trying to achieve. Maybe make diagram?
>
> Regards,
>
> --
>   Nicolas George


I ended up just keeping it as a script and getting the developers to
incorporate it into our code.  Thanks though!  Since it is a script, a
single bash call is not too intrusive.  If we had some C programmers, I
would have them use the ffmpeg libraries directly, but they are all just
java heads..

-Greg
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Muxing multiple files and concatenating those outputs

kumowoon1025
Hi,

> I ended up just keeping it as a script and getting the developers to
> incorporate it into our code.  Thanks though!  Since it is a script, a
> single bash call is not too intrusive.  If we had some C programmers, I
> would have them use the ffmpeg libraries directly, but they are all just
> java heads..

So is the problem basically solved now? If I had known this zombie apocalypse-like status quo was going to maintain for this long I would have recommended seeing if your VAR or sales rep if you have an account with the vendor for your call management system to see if you can get a license for doing this automatically (call recording's probably a feature? Just not activated I'm thiinking?)

Regards,
Ted Park
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Muxing multiple files and concatenating those outputs

Greg Oliver
On Tue, Apr 21, 2020 at 9:40 PM Edward Park <[hidden email]> wrote:

> Hi,
>
> > I ended up just keeping it as a script and getting the developers to
> > incorporate it into our code.  Thanks though!  Since it is a script, a
> > single bash call is not too intrusive.  If we had some C programmers, I
> > would have them use the ffmpeg libraries directly, but they are all just
> > java heads..
>
> So is the problem basically solved now? If I had known this zombie
> apocalypse-like status quo was going to maintain for this long I would have
> recommended seeing if your VAR or sales rep if you have an account with the
> vendor for your call management system to see if you can get a license for
> doing this automatically (call recording's probably a feature? Just not
> activated I'm thiinking?)
>
> Regards,
> Ted Park


We are the call recording vendor :)

Yes, it is solved (basically just kept it like it was prior to y email to
the list).  Thanks everyone for their input.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".