Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Jim DeLaHunt
Hello, folks:

I'm working with a video production company that is thinking about using
ffmpeg for a field production. They set up a 4k camera on top of a
building (have electricity, but limited internet), and they need to
capture 6-12 hours of 4k 29.92fps video from that camera reliably and
with very low quality loss. We know we can use ffmpeg to excerpt and
zoom in on parts of the main video afterwards, if the recording system
doesn't crash in the middle of the shoot, and if the main video retains
quality.

Vendor A seems to be saying, buy our expensive system, and it will
capture the video and put it on a file server which your computer can
access. But my boss is saying, can't ffmpeg just capture and write out
the video, and save us the expense of Vendor A?

We tried a simple experiment. We set up a 4K camera in the office. A
straightforward ffmpeg invocation did capture a couple of minutes of
video. But there were nasty artifacts, such as 25% of the pixels being
corrupted in the first second or so, and occasional speckles of
corruption. This tells me that reliable capture by ffmpeg is easier than
"impossible", but harder than "works without even trying".

Is anyone on this list capturing hours of 4k video, reliably and with
low quality loss?  Or, has anyone tried it and discovered that it's
really, really hard to do it well?  Is ffmpeg up to the task of running
for 12 hours of capturing 4k video without crashing? What bottlenecks or
weak points in our capture card or computer do we need to watch out for?

I looked through the archives of /ffmpeg-user/. The most relevant thread
I found was this: "4K 60Hz Directshow Video Capture"[1] (Feb 2018). The
conclusion I drew from this thread was circumstantial evidence that my
task might be reasonable for short periods, and no evidence about longer
periods like 12 hours. a) they are trying to capture 60Hz, and we only
want 30Hz. b) it really matters that the capture card exports something
which is accepted by the ffmpeg code you use. c) disk I/O is a primary
bottleneck, CPU speed is a secondary bottleneck. d) the right GPU card
can do the encoding, relieving the CPU speed bottleneck.

The drawback of /ffmpeg-user/ is that generally people post here about
problems, not about successes. If there are people quietly having
success with ffmpeg for 12 hours continuously or ffmpeg for 4K video at
29.92 fps, I'd love to hear it.

[1] <https://ffmpeg.org/pipermail/ffmpeg-user/2018-February/038895.html>

--
     --Jim DeLaHunt, [hidden email]     http://blog.jdlh.com/ (http://jdlh.com/)
       multilingual websites consultant

       355-1027 Davie St, Vancouver BC V6E 4L2, Canada
          Canada mobile +1-604-376-8953

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Carl Eugen Hoyos-2
2018-06-28 21:58 GMT+02:00, Jim DeLaHunt <[hidden email]>:

> We tried a simple experiment. We set up a 4K camera in the
> office. A straightforward ffmpeg invocation did capture a
> couple of minutes of video. But there were nasty artifacts

Command line and complete, uncut console outupt missing.

In this case, you should also tell us which operating system
you have, which cpu and which graphics-card (if applicable).

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Jim DeLaHunt
On 2018-06-28 14:21, Carl Eugen Hoyos wrote:

> 2018-06-28 21:58 GMT+02:00, Jim DeLaHunt <[hidden email]>:
>
>> We tried a simple experiment. We set up a 4K camera in the
>> office. A straightforward ffmpeg invocation did capture a
>> couple of minutes of video. But there were nasty artifacts
> Command line and complete, uncut console outupt missing.
>
> In this case, you should also tell us which operating system
> you have, which cpu and which graphics-card (if applicable).

You are missing the question.

I am not asking for help to diagnose that simple experiment. I am asking
if others have had experience with the top-level goal: capturing hours
of 4k video, reliably and with low loss, using ffmpeg.

I can use whatever capture card, operating system, CPU, and graphics
card are necessary to build a system that works, and costs less than
what Vendor A would charge us for their closed system.

--
     --Jim DeLaHunt, [hidden email]     http://blog.jdlh.com/ 
(http://jdlh.com/)
       multilingual websites consultant

       355-1027 Davie St, Vancouver BC V6E 4L2, Canada
          Canada mobile +1-604-376-8953
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

rogerdpack2
It "should" work assuming your transcoding/disk can keep up with realtime
...

On Thu, Jun 28, 2018 at 3:50 PM, Jim DeLaHunt <[hidden email]>
wrote:

> On 2018-06-28 14:21, Carl Eugen Hoyos wrote:
>
> 2018-06-28 21:58 GMT+02:00, Jim DeLaHunt <[hidden email]>:
>>
>> We tried a simple experiment. We set up a 4K camera in the
>>> office. A straightforward ffmpeg invocation did capture a
>>> couple of minutes of video. But there were nasty artifacts
>>>
>> Command line and complete, uncut console outupt missing.
>>
>> In this case, you should also tell us which operating system
>> you have, which cpu and which graphics-card (if applicable).
>>
>
> You are missing the question.
>
> I am not asking for help to diagnose that simple experiment. I am asking
> if others have had experience with the top-level goal: capturing hours of
> 4k video, reliably and with low loss, using ffmpeg.
>
> I can use whatever capture card, operating system, CPU, and graphics card
> are necessary to build a system that works, and costs less than what Vendor
> A would charge us for their closed system.
>
> --
>     --Jim DeLaHunt, [hidden email]     http://blog.jdlh.com/ (
> http://jdlh.com/)
>       multilingual websites consultant
>
>       355-1027 Davie St, Vancouver BC V6E 4L2, Canada
>          Canada mobile +1-604-376-8953
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
>
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Another Sillyname
Yes, but honestly I think you are over simplifying it......

For example, are you intending to capturing scenes with high activity
that requires a much higher bandwidth and transcoding
capability?

Anyone can capture and transcode to say H265 even, 4K@60FPS if you're
just capturing a black backdrop.

The capability exists and as you've stated the existing solutions can
be crazy expensive, but if you're going to homebrew a solution you
need to do some work/testing to make sure what YOU want to capture can
be done within YOUR budget.





On 23 July 2018 at 08:42, Roger Pack <[hidden email]> wrote:

> It "should" work assuming your transcoding/disk can keep up with realtime
> ...
>
> On Thu, Jun 28, 2018 at 3:50 PM, Jim DeLaHunt <[hidden email]>
> wrote:
>
>> On 2018-06-28 14:21, Carl Eugen Hoyos wrote:
>>
>> 2018-06-28 21:58 GMT+02:00, Jim DeLaHunt <[hidden email]>:
>>>
>>> We tried a simple experiment. We set up a 4K camera in the
>>>> office. A straightforward ffmpeg invocation did capture a
>>>> couple of minutes of video. But there were nasty artifacts
>>>>
>>> Command line and complete, uncut console outupt missing.
>>>
>>> In this case, you should also tell us which operating system
>>> you have, which cpu and which graphics-card (if applicable).
>>>
>>
>> You are missing the question.
>>
>> I am not asking for help to diagnose that simple experiment. I am asking
>> if others have had experience with the top-level goal: capturing hours of
>> 4k video, reliably and with low loss, using ffmpeg.
>>
>> I can use whatever capture card, operating system, CPU, and graphics card
>> are necessary to build a system that works, and costs less than what Vendor
>> A would charge us for their closed system.
>>
>> --
>>     --Jim DeLaHunt, [hidden email]     http://blog.jdlh.com/ (
>> http://jdlh.com/)
>>       multilingual websites consultant
>>
>>       355-1027 Davie St, Vancouver BC V6E 4L2, Canada
>>          Canada mobile +1-604-376-8953
>> _______________________________________________
>> ffmpeg-user mailing list
>> [hidden email]
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>>
>> To unsubscribe, visit link above, or email
>> [hidden email] with subject "unsubscribe".
>>
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Rafael Lima
In reply to this post by Jim DeLaHunt
>
>  They set up a 4k camera on top of a building (have electricity, but
> limited internet),


4K on limited internet? is just in my mind that those two words doesn't fit
together?


What do you mean by " capture 6-12 hours of 4k 29.92fps video from that
camera"? Is the camera streaming the video somehow and you just need to
store it? If it is your only limitation are the badwith and storage as
ffmpeg doesn't need to process nothing and it could be done with any stream
saver program




--
--
Rafael Lima
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Jim DeLaHunt

On 2018-07-23 10:25, Rafael Lima wrote:

>>   They set up a 4k camera on top of a building (have electricity, but
>> limited internet),
>
> 4K on limited internet? is just in my mind that those two words doesn't fit
> together?
>
>
> What do you mean by " capture 6-12 hours of 4k 29.92fps video from that
> camera"? Is the camera streaming the video somehow and you just need to
> store it?

Thank you for your reply, Rafael.

I'm sorry if my original message wasn't clear. I understand why "4K" and
"limited internet" might not fit together. The missing part is "enough
terabytes of reasonably fast SSD storage on the server to which the
camera is attached to hold the video".

So yes, I "just" need to store it. And I need to store every frame of 4K
at 29.92fps. And I need the capture system to not run out of memory, or
crash, during the 12-hour session.  And if an individual part of the
capture system could crash more often than, say, during 1% of the
12-hour sessions, then I need some kind of redundancy to allow another
system to capture if the primary system has failed.

On 2018-07-23 10:25, Rafael Lima wrote:
> ...If it is your only limitation are the [bandwith] and storage as
> ffmpeg doesn't need to process nothing and it could be done with any stream
> saver program

This sounds reassuring, as if this isn't such a hard task after all. But
I don't hear you saying that you know of someone who has done it.

On 2018-07-23 00:42, Roger Pack wrote:
> It "should" work assuming your transcoding/disk can keep up with realtime
> ...

Thank you, Roger. I am hearing you say "should work", not "did work, for
a real-life case I know".

On 2018-07-23 05:42, Another Sillyname wrote:
> Yes, but honestly I think you [Roger Pack] are over simplifying it......
>
> For example, are you intending to capturing scenes with high activity
> that requires a much higher bandwidth and transcoding capability? ...
> The capability exists and as you've stated the existing solutions can
> be crazy expensive, but if you're going to homebrew a solution you
> need to do some work/testing to make sure what YOU want to capture can
> be done within YOUR budget.

Thank you, "Another Sillyname". This is what I'm concerned with: that it
might work, might not, and the only way to gather data is to try my own
experiments.

My goal with the question was to limit the range of possibilities, by
having someone come forward with concrete experiences. If I hear, "we
tried it, it was crazy hard", maybe I should tell my boss to pay for the
crazy expensive commercial solutions. If I hear, "we tried it, it was
pretty easy", maybe I encourage my boss to let me homebrew a solution.

In either case, we will have to test it.

Thank you all for your replies,
        —Jim DeLaHunt, Vancouver, Canada

--

--
     --Jim DeLaHunt, [hidden email]     http://blog.jdlh.com/ (http://jdlh.com/)
       multilingual websites consultant

       355-1027 Davie St, Vancouver BC V6E 4L2, Canada
          Canada mobile +1-604-376-8953

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Rafael Lima
Jim,

Teorically there is no limitation on ffmpeg that would make it fail because
of the long running time... there are many stream serveres built with
ffmpeg running on 24hours basis.

You had an experience and you didn't like the result, what on your result
made you conclude that was ffmpeg fault? How to make sure the loss is not
related to the camera, to the connection between server and camera, to
server limitations... have you had a sucessfull test with the same
configuration just changing the ffmpeg to one propretary system?

I believe most of people who answered before were believing you need to
compress the data before store, since 1h of 4K can lead to some terabytes
of data if not compressed


On Wed, Jul 25, 2018 at 5:34 PM, Jim DeLaHunt <[hidden email]>
wrote:

>
> On 2018-07-23 10:25, Rafael Lima wrote:
>
>>   They set up a 4k camera on top of a building (have electricity, but
>>> limited internet),
>>>
>>
>> 4K on limited internet? is just in my mind that those two words doesn't
>> fit
>> together?
>>
>>
>> What do you mean by " capture 6-12 hours of 4k 29.92fps video from that
>> camera"? Is the camera streaming the video somehow and you just need to
>> store it?
>>
>
> Thank you for your reply, Rafael.
>
> I'm sorry if my original message wasn't clear. I understand why "4K" and
> "limited internet" might not fit together. The missing part is "enough
> terabytes of reasonably fast SSD storage on the server to which the camera
> is attached to hold the video".
>
> So yes, I "just" need to store it. And I need to store every frame of 4K
> at 29.92fps. And I need the capture system to not run out of memory, or
> crash, during the 12-hour session.  And if an individual part of the
> capture system could crash more often than, say, during 1% of the 12-hour
> sessions, then I need some kind of redundancy to allow another system to
> capture if the primary system has failed.
>
> On 2018-07-23 10:25, Rafael Lima wrote:
>
>> ...If it is your only limitation are the [bandwith] and storage as
>> ffmpeg doesn't need to process nothing and it could be done with any
>> stream
>> saver program
>>
>
> This sounds reassuring, as if this isn't such a hard task after all. But I
> don't hear you saying that you know of someone who has done it.
>
> On 2018-07-23 00:42, Roger Pack wrote:
>
>> It "should" work assuming your transcoding/disk can keep up with realtime
>> ...
>>
>
> Thank you, Roger. I am hearing you say "should work", not "did work, for a
> real-life case I know".
>
> On 2018-07-23 05:42, Another Sillyname wrote:
>
>> Yes, but honestly I think you [Roger Pack] are over simplifying it......
>>
>> For example, are you intending to capturing scenes with high activity
>> that requires a much higher bandwidth and transcoding capability? ...
>> The capability exists and as you've stated the existing solutions can
>> be crazy expensive, but if you're going to homebrew a solution you
>> need to do some work/testing to make sure what YOU want to capture can
>> be done within YOUR budget.
>>
>
> Thank you, "Another Sillyname". This is what I'm concerned with: that it
> might work, might not, and the only way to gather data is to try my own
> experiments.
>
> My goal with the question was to limit the range of possibilities, by
> having someone come forward with concrete experiences. If I hear, "we tried
> it, it was crazy hard", maybe I should tell my boss to pay for the crazy
> expensive commercial solutions. If I hear, "we tried it, it was pretty
> easy", maybe I encourage my boss to let me homebrew a solution.
>
> In either case, we will have to test it.
>
> Thank you all for your replies,
>        —Jim DeLaHunt, Vancouver, Canada
>
> --
>
> --
>     --Jim DeLaHunt, [hidden email]     http://blog.jdlh.com/ (
> http://jdlh.com/)
>       multilingual websites consultant
>
>       355-1027 Davie St, Vancouver BC V6E 4L2, Canada
>          Canada mobile +1-604-376-8953
>
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
>



--
--
Rafael Lima
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Dragan Randjelovic
In reply to this post by Jim DeLaHunt
Recommendation:

-several ssd drives in raid 0 configuration ex. Samsung EVO PRO ( allows
for capacity and speed, and longer read/write life-cycle)  Calculate the
space you need based on codec selection or god forbid based on that long 4K
raw video :)
-use thunderbolt port straight from camera or capture trough some interface
like Black Magic 4K studio pro (preferably over sdi interface) and/or BM
intensity (sdi/hdmi -> thunderbolt.->PC)

use either software coming with the capturing board (which will allow you
to test performance metrics of critical components and show if you are
short on something, offer you a few codec for trans-coding on the fly
etc..) or use ffmpeg with manners ( blackmagic is supported with a caveat -
non-free license, in other words you have to compile ffmpeg for yourself)
Try not to record raw but to transcode on the fly or make use of camera's
potentially built in encoder. Now here is a critical part! Do you have a
reference signal? If not camera can become your reference clock. Rethink
and interconnect reference signal between capturing adapter and camera.
Moreover it's up to you to select the codec of choice ex. AVC intra wont
blow up your core i7 4th gen or newer box. As far as refresh rate value
goes, make sure that both camera and capturing adapter support desired one.
Capture long and prosper!

P>S. Oh sorry I just read top of the house part. I guess 100m is enough.
Forget about clock and use SDI over coax.

This looks like off topic isn't it ?

Regards.
Dragan


On Thu, Jun 28, 2018 at 9:58 PM, Jim DeLaHunt <[hidden email]>
wrote:

> Hello, folks:
>
> I'm working with a video production company that is thinking about using
> ffmpeg for a field production. They set up a 4k camera on top of a building
> (have electricity, but limited internet), and they need to capture 6-12
> hours of 4k 29.92fps video from that camera reliably and with very low
> quality loss. We know we can use ffmpeg to excerpt and zoom in on parts of
> the main video afterwards, if the recording system doesn't crash in the
> middle of the shoot, and if the main video retains quality.
>
> Vendor A seems to be saying, buy our expensive system, and it will capture
> the video and put it on a file server which your computer can access. But
> my boss is saying, can't ffmpeg just capture and write out the video, and
> save us the expense of Vendor A?
>
> We tried a simple experiment. We set up a 4K camera in the office. A
> straightforward ffmpeg invocation did capture a couple of minutes of video.
> But there were nasty artifacts, such as 25% of the pixels being corrupted
> in the first second or so, and occasional speckles of corruption. This
> tells me that reliable capture by ffmpeg is easier than "impossible", but
> harder than "works without even trying".
>
> Is anyone on this list capturing hours of 4k video, reliably and with low
> quality loss?  Or, has anyone tried it and discovered that it's really,
> really hard to do it well?  Is ffmpeg up to the task of running for 12
> hours of capturing 4k video without crashing? What bottlenecks or weak
> points in our capture card or computer do we need to watch out for?
>
> I looked through the archives of /ffmpeg-user/. The most relevant thread I
> found was this: "4K 60Hz Directshow Video Capture"[1] (Feb 2018). The
> conclusion I drew from this thread was circumstantial evidence that my task
> might be reasonable for short periods, and no evidence about longer periods
> like 12 hours. a) they are trying to capture 60Hz, and we only want 30Hz.
> b) it really matters that the capture card exports something which is
> accepted by the ffmpeg code you use. c) disk I/O is a primary bottleneck,
> CPU speed is a secondary bottleneck. d) the right GPU card can do the
> encoding, relieving the CPU speed bottleneck.
>
> The drawback of /ffmpeg-user/ is that generally people post here about
> problems, not about successes. If there are people quietly having success
> with ffmpeg for 12 hours continuously or ffmpeg for 4K video at 29.92 fps,
> I'd love to hear it.
>
> [1] <https://ffmpeg.org/pipermail/ffmpeg-user/2018-February/038895.html>
>
> --
>     --Jim DeLaHunt, [hidden email]     http://blog.jdlh.com/ (
> http://jdlh.com/)
>       multilingual websites consultant
>
>       355-1027 Davie St, Vancouver BC V6E 4L2, Canada
> <https://maps.google.com/?q=355-1027+Davie+St,+Vancouver+BC+V6E+4L2,+Canada&entry=gmail&source=g>
>          Canada mobile +1-604-376-8953
>
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Jim DeLaHunt
In reply to this post by Rafael Lima

On 2018-07-25 16:01, Rafael Lima wrote:
> Jim,
>
> Teorically there is no limitation on ffmpeg that would make it fail because
> of the long running time... there are many stream serveres built with
> ffmpeg running on 24hours basis.
Thank you, Rafael. It's helpful to have anecdotes like this.  It would
be even more helpful to have specific testimonials, such as "we use
ffmpeg to do ABC, we run ffmpeg instances 24hours continuously without
error."

I'm not surprised to hear that ffmpeg has no designed limitation that
would make it fail when running a long time. There could, however, be
_bugs_ that would make it fail. That's why testimonials are helpful.

But what I'm taking from this thread is: should work in theory, no-one
saying they have experience with it failing, a few saying they have
experience with it working.
> You had an experience and you didn't like the result, what on your result
> made you conclude that was ffmpeg fault?
I don't conclude that ffmpeg was at fault. That experiment is just a
data point to say that setting up such a system is not incredibly simple
and reliable.
> How to make sure the loss is not
> related to the camera, to the connection between server and camera, to
> server limitations... have you had a sucessfull test with the same
> configuration just changing the ffmpeg to one propretary system?
>
> I believe most of people who answered before were believing you need to
> compress the data before store, since 1h of 4K can lead to some terabytes
> of data if not compressed
Again, I'm not asking people to design my system for me. My goal with
the question was to limit the range of possibilities, by having someone
come forward with concrete experiences.

In any case, we will have to test it. Thank you for your reply,

       —Jim DeLaHunt, Vancouver, Canada

--
     --Jim DeLaHunt, [hidden email]     http://blog.jdlh.com/ (http://jdlh.com/)
       multilingual websites consultant

       355-1027 Davie St, Vancouver BC V6E 4L2, Canada
          Canada mobile +1-604-376-8953

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Jim DeLaHunt
In reply to this post by Dragan Randjelovic
On 2018-07-25 18:52, Dragan Randjelovic wrote:

> Recommendation:
>
> -several ssd drives in raid 0 configuration ...[snip]...
> -use thunderbolt port straight from camera or capture trough some interface
> like Black Magic 4K studio pro (preferably over sdi interface) ...[snip]...
>
> use either software coming with the capturing board  ...[snip]...
> or use ffmpeg with manners  ...[snip]...
>
> Try not to record raw but to transcode on the fly or make use of camera's
> potentially built in encoder.  ...[snip]...
>
> Do you have a reference signal?  ...[snip]...
> As far as refresh rate value goes, make sure that both camera and capturing adapter support desired one.
>
> Capture long and prosper!
>
> P>S. Oh sorry I just read top of the house part. I guess 100m is enough.
> Forget about clock and use SDI over coax.
>
> This looks like off topic isn't it ?
Thank you, Dragan. Off-topic or not, it's very helpful suggestions as I
report to my boss about building the capture system.

Actually, it makes me inclined to tell my boss to hire a proper digital
video expert to design the capture system, or to pay for the proprietary
capture system. I'm just a software engineer. I can figure out these
video factors, but maybe my boss doesn't want to pay me to learn all of
that.

--
     --Jim DeLaHunt, [hidden email]     http://blog.jdlh.com/ (http://jdlh.com/)
       multilingual websites consultant

       355-1027 Davie St, Vancouver BC V6E 4L2, Canada
          Canada mobile +1-604-376-8953

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Bryce McLeod
In reply to this post by Jim DeLaHunt
Its not 4k but on one of our systems we regularly capture on the same
machine from an sdi source 8-12hours of 3 x 1080p25 streams in lossess h264
using ffmpeg, all 3 outputs going to spinning disk on 5 year old hardware.
There is enough headroom that a 4th would be easily possible with newer
hardware, so I'd say 4k is well doable (though it would make sense to use
ssds)


--
Bryce McLeod

Codory Software Limited
email: [hidden email]
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Jim DeLaHunt
On 2018-07-31 02:43, Bryce McLeod wrote:
> Its not 4k but on one of our systems we regularly capture on the same
> machine from an sdi source 8-12hours of 3 x 1080p25 streams in lossess h264
> using ffmpeg, all 3 outputs going to spinning disk on 5 year old hardware.
> There is enough headroom that a 4th would be easily possible with newer
> hardware, so I'd say 4k is well doable (though it would make sense to use
> ssds)
Thank you, Bryce. This is exactly the sort of first-hand report from the
field I was hoping to get. You are awesome!

--
     --Jim DeLaHunt, [hidden email]     http://blog.jdlh.com/ (http://jdlh.com/)
       multilingual websites consultant

       355-1027 Davie St, Vancouver BC V6E 4L2, Canada
          Canada mobile +1-604-376-8953

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Carl Eugen Hoyos-2
In reply to this post by Rafael Lima
2018-07-26 1:01 GMT+02:00, Rafael Lima <[hidden email]>:

> Teorically there is no limitation on ffmpeg that would make
> it fail because of the long running time.

I don't think this is entirely correct, I believe it depends
on the specific parameters.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

rogerdpack2
In reply to this post by Jim DeLaHunt
On Wed, Jul 25, 2018 at 2:34 PM, Jim DeLaHunt <[hidden email]> wrote:

>
> On 2018-07-23 10:25, Rafael Lima wrote:
>>>
>>>   They set up a 4k camera on top of a building (have electricity, but
>>> limited internet),
>>
>>
>> 4K on limited internet? is just in my mind that those two words doesn't
>> fit
>> together?
>>
>>
>> What do you mean by " capture 6-12 hours of 4k 29.92fps video from that
>> camera"? Is the camera streaming the video somehow and you just need to
>> store it?
>
>
> Thank you for your reply, Rafael.
>
> I'm sorry if my original message wasn't clear. I understand why "4K" and
> "limited internet" might not fit together. The missing part is "enough
> terabytes of reasonably fast SSD storage on the server to which the camera
> is attached to hold the video".
>
> So yes, I "just" need to store it. And I need to store every frame of 4K at
> 29.92fps. And I need the capture system to not run out of memory, or crash,
> during the 12-hour session.  And if an individual part of the capture system
> could crash more often than, say, during 1% of the 12-hour sessions, then I
> need some kind of redundancy to allow another system to capture if the
> primary system has failed.
>
> On 2018-07-23 10:25, Rafael Lima wrote:
>>
>> ...If it is your only limitation are the [bandwith] and storage as
>> ffmpeg doesn't need to process nothing and it could be done with any
>> stream
>> saver program
>
>
> This sounds reassuring, as if this isn't such a hard task after all. But I
> don't hear you saying that you know of someone who has done it.
>
> On 2018-07-23 00:42, Roger Pack wrote:
>>
>> It "should" work assuming your transcoding/disk can keep up with realtime
>> ...
>
>
> Thank you, Roger. I am hearing you say "should work", not "did work, for a
> real-life case I know".

Try it out and let us know.  Another thing that might help is some
kind of "gnu" encoding.
I'm not sure if libx264 even on "ultrafast" will be able to handle it, GL!
-Roger-

>
> On 2018-07-23 05:42, Another Sillyname wrote:
>>
>> Yes, but honestly I think you [Roger Pack] are over simplifying it......
>>
>> For example, are you intending to capturing scenes with high activity
>> that requires a much higher bandwidth and transcoding capability? ...
>> The capability exists and as you've stated the existing solutions can
>> be crazy expensive, but if you're going to homebrew a solution you
>> need to do some work/testing to make sure what YOU want to capture can
>> be done within YOUR budget.
>
>
> Thank you, "Another Sillyname". This is what I'm concerned with: that it
> might work, might not, and the only way to gather data is to try my own
> experiments.
>
> My goal with the question was to limit the range of possibilities, by having
> someone come forward with concrete experiences. If I hear, "we tried it, it
> was crazy hard", maybe I should tell my boss to pay for the crazy expensive
> commercial solutions. If I hear, "we tried it, it was pretty easy", maybe I
> encourage my boss to let me homebrew a solution.
>
> In either case, we will have to test it.
>
> Thank you all for your replies,
>        —Jim DeLaHunt, Vancouver, Canada
>
> --
>
> --
>     --Jim DeLaHunt, [hidden email]     http://blog.jdlh.com/
> (http://jdlh.com/)
>       multilingual websites consultant
>
>       355-1027 Davie St, Vancouver BC V6E 4L2, Canada
>          Canada mobile +1-604-376-8953
>
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Gabriel Balaich
In reply to this post by Jim DeLaHunt
>
> You are missing the question.
>
> I am not asking for help to diagnose that simple experiment. I am asking
> if others have had experience with the top-level goal: capturing hours
> of 4k video, reliably and with low loss, using ffmpeg.
>
> I can use whatever capture card, operating system, CPU, and graphics
> card are necessary to build a system that works, and costs less than
> what Vendor A would charge us for their closed system.
>

I capture 4K60 and 1080P60 simultaneously using one instance of FFmpeg
with no artifacting (ever) or perceivable loss in quality for 7-8 hours at
a time
consistently.

No crazy hardware - a few capture cards (HD60 Pro + Magewell Pro Capture
HDMI 4K Plus), 6800K, 16GB of RAM, GTX 1050 (handles encoding), and
an Intel 600P NVME SSD for storage.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

rogerdpack2
On Tue, Aug 7, 2018 at 3:15 PM, Gabriel Balaich <[hidden email]> wrote:

>>
>> You are missing the question.
>>
>> I am not asking for help to diagnose that simple experiment. I am asking
>> if others have had experience with the top-level goal: capturing hours
>> of 4k video, reliably and with low loss, using ffmpeg.
>>
>> I can use whatever capture card, operating system, CPU, and graphics
>> card are necessary to build a system that works, and costs less than
>> what Vendor A would charge us for their closed system.
>>
>
> I capture 4K60 and 1080P60 simultaneously using one instance of FFmpeg
> with no artifacting (ever) or perceivable loss in quality for 7-8 hours at
> a time
> consistently.
>
> No crazy hardware - a few capture cards (HD60 Pro + Magewell Pro Capture
> HDMI 4K Plus), 6800K, 16GB of RAM, GTX 1050 (handles encoding), and
> an Intel 600P NVME SSD for storage.

Nice.  What codec and/or command line?

Also if anybody's desperate for a "software" solution I might be able
to provide a new video codec that "uses a lot of space but is
screaming fast" let me know if x264 ultrafast isn't fast enough.
Cheers!
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Anyone having success capturing hours of 4k video, reliably and with low loss, using ffmpeg?

Gabriel Balaich
>
> Nice.  What codec and/or command line?
>
> Also if anybody's desperate for a "software" solution I might be able
> to provide a new video codec that "uses a lot of space but is
> screaming fast" let me know if x264 ultrafast isn't fast enough.
> Cheers!
>

Kind of a huge command:
 ffmpeg -loglevel error -stats -y -thread_queue_size 9999 -indexmem 9999
-guess_layout_max 0 -f dshow -rtbufsize 2147.48M ^
-i audio="Analog (3+4) (RME Fireface UC)" ^
-thread_queue_size 9999 -indexmem 9999 -guess_layout_max 0 -f dshow
-rtbufsize 2147.48M ^
-i audio="ADAT (3+4) (RME Fireface UC)" ^
-thread_queue_size 9999 -indexmem 9999 -guess_layout_max 0 -f dshow
-video_size 3840x2160 -rtbufsize 2147.48M ^
-framerate 120 -pixel_format nv12 -i video="Video (00 Pro Capture HDMI
4K+)":audio="Audio (00 Pro Capture HDMI 4K+)" ^
-thread_queue_size 9999 -indexmem 9999 -guess_layout_max 0 -f dshow
-video_size 1920x1080 -rtbufsize 2147.48M ^
-framerate 60 -pixel_format yuyv422 -i video="Game Capture HD60 Pro (Video)
(#01)":audio="ADAT (5+6) (RME Fireface UC)" ^
-thread_queue_size 9999 -indexmem 9999 -r 25 -f lavfi -rtbufsize 2147.48M
-i color=c=black:s=50x50 ^
-map 4,0 -map 0 -c:v libx264 -r 25 -rc-lookahead 50 -forced-idr 1
-strict_gop 1 -sc_threshold 0 -flags +cgop ^
-force_key_frames expr:gte(t,n_forced*2) -preset ultrafast -pix_fmt nv12
-b:v 16K -minrate 16K -maxrate 16K -bufsize 16k ^
-c:a aac -ar 44100 -b:a 384k -ac 2 -vf "fps=25" -af "aresample=async=250"
-vsync 1 -ss 00:00:02.456 ^
-max_muxing_queue_size 9999 -f segment -segment_time 600 -segment_wrap 9
-reset_timestamps 1 ^
-segment_format_options max_delay=0
C:\Users\djcim\Videos\PS4\Theirs\TPS%02d.ts ^
-map 4,1 -map 1 -c:v libx264 -r 25 -rc-lookahead 50 -forced-idr 1
-strict_gop 1 -sc_threshold 0 -flags +cgop ^
-force_key_frames expr:gte(t,n_forced*2) -preset ultrafast -pix_fmt nv12
-b:v 16K -minrate 16K -maxrate 16K -bufsize 16k ^
-c:a aac -ar 44100 -b:a 384k -ac 2 -vf "fps=25" -af "aresample=async=250"
-vsync 1 -ss 00:00:01.840 ^
-max_muxing_queue_size 9999 -f segment -segment_time 600 -segment_wrap 9
-reset_timestamps 1 ^
-segment_format_options max_delay=0
C:\Users\djcim\Videos\PS4\Soundboard\SPS%02d.ts ^
-map 2:0,2:1 -map 2:1 -c:v h264_nvenc -r 60 -rc-lookahead 120 -forced-idr 1
-strict_gop 1 -sc_threshold 0 -flags +cgop ^
-force_key_frames expr:gte(t,n_forced*2) -preset: llhp -pix_fmt nv12 -b:v
250M -minrate 250M -maxrate 250M -bufsize 250M ^
-c:a aac -ar 44100 -b:a 384k -ac 2 -af "aresample=async=250" -vsync 1 -ss
00:00:01.096 ^
-max_muxing_queue_size 9999 -f segment -segment_time 600 -segment_wrap 9
-reset_timestamps 1 ^
-segment_format_options max_delay=0 C:\Users\djcim\Videos\PS4\PS4\PS%02d.ts
^
-map 3:0,3:1 -map 3:1 -c:v h264_nvenc -r 60 -rc-lookahead 120 -forced-idr 1
-strict_gop 1 -sc_threshold 0 -flags +cgop ^
-force_key_frames expr:gte(t,n_forced*2) -preset: llhp -pix_fmt yuv420p
-b:v 40M -minrate 40M -maxrate 40M -bufsize 40M ^
-c:a aac -ar 44100 -b:a 384k -ac 2 -af "pan=mono|c0=c0, adelay=120|120,
aresample=async=250" -vsync 1 ^
-max_muxing_queue_size 9999 -f segment -segment_time 600 -segment_wrap 9
-reset_timestamps 1 ^
-segment_format_options max_delay=0
C:\Users\djcim\Videos\PS4\Camera\CPS%02d.ts

Writing to 4 files at once, 2 video (1080P60 + 4K60), and 2 "audio".
All four files are technically video, needed to generate blank video
and link to desired audio so parting would stay in sync with the 2
actual video files.

Many options and the file formats are put in place to ensure
synchronized and consistent parting across all four files. When I
want to record longer files I typically just increase the segment
time as opposed to entirely removing parting options. That being
said if your goal is to simply record one continuous 4K29.97 file
and nothing else with the equipment I have this would suffice:

-thread_queue_size 9999 -indexmem 9999 -f dshow -video_size 3840x2160
-rtbufsize 2147.48M -framerate 30000/1001 ^
-pixel_format nv12 -i video="Video (00 Pro Capture HDMI 4K+)" -map 0 -c:v
h264_nvenc -preset: llhp -pix_fmt nv12 ^
-b:v 250M -minrate 250M -maxrate 250M -bufsize 250M -max_muxing_queue_size
9999 Ouput.mp4

Encoding 4k60 with my 6800K immediately pegs all cores, even
the 7900X in my other PC can't handle it. NVENC seems to be
the only reasonable option, may be different for 4k30 though.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".