Overlay performance

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Overlay performance

Darrin Smith
Are there any "tricks" to improve the performance of merging an image
(overlay) with a video?

I have a video snippet (cropped from a larger video) that I then add an
overlay to so additional data is added in the video. The png I use is the
same size as the video source.  I typically notice a 3X time to merge the
png and the video as compared to the length of the video clip. So, if the
clip is 5 seconds long, it normally takes FFMpeg 15 seconds to merge them
together to create a final video. I'm using a Pixel 3XL. Not THE fastest
out there, but certainly in the upper tier.

Is there anything that can be done to improve the performance?

Thanks.

Darrin



[image: Mailtrack]
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&>
Sender
notified by
Mailtrack
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&>
08/26/19,
10:15:02 AM
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Overlay performance

Devin Heitmueller
On Mon, Aug 26, 2019 at 11:28 AM Darrin Smith <[hidden email]> wrote:

>
> Are there any "tricks" to improve the performance of merging an image
> (overlay) with a video?
>
> I have a video snippet (cropped from a larger video) that I then add an
> overlay to so additional data is added in the video. The png I use is the
> same size as the video source.  I typically notice a 3X time to merge the
> png and the video as compared to the length of the video clip. So, if the
> clip is 5 seconds long, it normally takes FFMpeg 15 seconds to merge them
> together to create a final video. I'm using a Pixel 3XL. Not THE fastest
> out there, but certainly in the upper tier.

The overlay filter isn't really designed to run on slow ARM CPUs (i.e.
blending not really optimized).  Usually you would do this sort of
compositing further down the pipeline in OpenGL on an embedded target.

Does the PNG really need to be the same size, or did you just do that
because it was convenient?  If the latter, see if you can just blend
the region you care about (potentially using multiple overlay filters
if there are a couple of regions).  In general, anything that isnt'
hardware accelerated and touches every pixel in every video frame is
going to run very poorly on an ARM target.

Also, might be worth dumping out the pipeline and making sure you're
not getting some unexpected YUV->RGB->YUV colorspace conversion in the
middle.

Devin

--
Devin J. Heitmueller - Kernel Labs
http://www.kernellabs.com
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Overlay performance

Carl Eugen Hoyos-2
In reply to this post by Darrin Smith



> Am 26.08.2019 um 17:20 schrieb Darrin Smith <[hidden email]>:
>
> Are there any "tricks" to improve the performance of merging an image
> (overlay) with a video?

Command line and complete, uncut console output missing.


> I have a video snippet (cropped from a larger video) that I then add an
> overlay to so additional data is added in the video.


> The png I use is the
> same size as the video source.

Using a format with a different pix_fmt will likely speed up the process.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Overlay performance

Darrin Smith
In reply to this post by Darrin Smith
I was asked for the output and for the original command string. Here they
are.

Note that I first use FFMpeg to trim the video to a 5 second clip, then I
add the overlay to that. The clipping is very fast. The merging of the png
with the video though is very slow.

I've read a few comments that I should make the png smaller than the video
content size. I assume the image will auto-stretch to be the size of the
video?  I need output on all four corners.

Here is the output:

2019-08-27 09:18:21.228 20948-21800/com.android.myapp D/CCodec:
allocate(c2.qti.avc.decoder)

2019-08-27 09:18:21.230 20948-21800/com.android.myapp I/Codec2Client:
Creating a Codec2 client to service "default"

2019-08-27 09:18:21.248 20948-21800/com.android.myapp I/Codec2Client:
Client to Codec2 service "default" created

2019-08-27 09:18:21.256 20948-21800/com.android.myapp I/CCodec: setting up
'default' as default (vendor) store

2019-08-27 09:18:21.271 20948-21800/com.android.myapp I/CCodec: Created
component [c2.qti.avc.decoder]

2019-08-27 09:18:21.271 20948-21800/com.android.myapp D/CCodecConfig: read
media type: video/avc

2019-08-27 09:18:21.272 20948-21800/com.android.myapp
D/ReflectedParamUpdater: extent() != 1 for single value type:
output.buffers.pool-ids.values

2019-08-27 09:18:21.274 20948-21800/com.android.myapp D/CCodecConfig:
ignoring local param raw.size (0xd2001800) as it is already supported

2019-08-27 09:18:21.274 20948-21800/com.android.myapp D/CCodecConfig:
ignoring local param raw.color (0xd2001809) as it is already supported

2019-08-27 09:18:21.274 20948-21800/com.android.myapp D/CCodecConfig:
ignoring local param raw.hdr-static-info (0xd200180a) as it is already
supported

2019-08-27 09:18:21.279 20948-21800/com.android.myapp I/CCodecConfig: query
failed after returning 17 values (BAD_INDEX)

2019-08-27 09:18:21.279 20948-21800/com.android.myapp D/CCodecConfig: c2
config is Dict {

      c2::i32 algo.priority.value = -1

      c2::float algo.rate.value = 4.2039e-44

      c2::u32 algo.secure-mode.value = 0

      c2::float coded.frame-rate.value = 30

      c2::u32 coded.pl.level = 20480

      c2::u32 coded.pl.profile = 20480

      c2::u32 coded.vui.color.matrix = 0

      c2::u32 coded.vui.color.primaries = 0

      c2::u32 coded.vui.color.range = 0

      c2::u32 coded.vui.color.transfer = 0

      c2::u32 default.color.matrix = 0

      c2::u32 default.color.primaries = 3

      c2::u32 default.color.range = 2

      c2::u32 default.color.transfer = 0

      c2::u32 input.buffers.max-size.value = 13271040

      c2::u32 input.delay.value = 4

      string input.media-type.value = "video/avc"

      c2::u32 output.delay.value = 18

      string output.media-type.value = "video/raw"

      c2::u32 raw.color.matrix = 0

      c2::u32 raw.color.primaries = 0

      c2::u32 raw.color.range = 0

      c2::u32 raw.color.transfer = 0

      c2::float raw.hdr-static-info.mastering.blue.x = 1.4013e-45

      c2::float raw.hdr-static-info.mastering.blue.y = 1.4013e-45

      c2::float raw.hdr-stati

2019-08-27 09:18:21.281 20948-21800/com.android.myapp W/ColorUtils:
expected specified color aspects (0:0:0:0)

2019-08-27 09:18:21.290 20948-20994/com.android.myapp D/FA: Connected to
remote service

2019-08-27 09:18:21.300 20948-21799/com.android.myapp D/SurfaceUtils:
connecting to surface 0x6fd5c08010, reason connectToSurface

2019-08-27 09:18:21.300 20948-21799/com.android.myapp I/MediaCodec:
[c2.qti.avc.decoder] setting surface generation to 21450756

2019-08-27 09:18:21.300 20948-21799/com.android.myapp D/SurfaceUtils:
disconnecting from surface 0x6fd5c08010, reason connectToSurface(reconnect)

2019-08-27 09:18:21.300 20948-21799/com.android.myapp D/SurfaceUtils:
connecting to surface 0x6fd5c08010, reason connectToSurface(reconnect)

2019-08-27 09:18:21.301 20948-21800/com.android.myapp D/CCodecConfig: no c2
equivalents for csd-1

2019-08-27 09:18:21.301 20948-21800/com.android.myapp D/CCodecConfig: no c2
equivalents for native-window

2019-08-27 09:18:21.302 20948-21800/com.android.myapp D/CCodecConfig: c2
config is Dict {

      c2::i32 algo.priority.value = 0

      c2::float algo.rate.value = 4.2039e-44

      c2::u32 algo.secure-mode.value = 0

      c2::float coded.frame-rate.value = 30

      c2::u32 coded.pl.level = 20480

      c2::u32 coded.pl.profile = 20480

      c2::u32 coded.vui.color.matrix = 0

      c2::u32 coded.vui.color.primaries = 0

      c2::u32 coded.vui.color.range = 0

      c2::u32 coded.vui.color.transfer = 0

      c2::u32 default.color.matrix = 0

      c2::u32 default.color.primaries = 3

      c2::u32 default.color.range = 2

      c2::u32 default.color.transfer = 0

      c2::u32 input.buffers.max-size.value = 3317760

      c2::u32 input.delay.value = 4

      string input.media-type.value = "video/avc"

      c2::u32 output.delay.value = 18

      string output.media-type.value = "video/raw"

      c2::u32 raw.color.matrix = 0

      c2::u32 raw.color.primaries = 0

      c2::u32 raw.color.range = 0

      c2::u32 raw.color.transfer = 0

      c2::float raw.hdr-static-info.mastering.blue.x = 1.4013e-45

      c2::float raw.hdr-static-info.mastering.blue.y = 1.4013e-45

      c2::float raw.hdr-static-

2019-08-27 09:18:21.302 20948-21800/com.android.myapp W/Codec2Client: query
-- param skipped: index = 1107298332.

2019-08-27 09:18:21.302 20948-21800/com.android.myapp D/CCodec: client
requested max input size 458635, which is smaller than what component
recommended (3317760); overriding with component recommendation.

2019-08-27 09:18:21.302 20948-21800/com.android.myapp W/CCodec: This
behavior is subject to change. It is recommended that app developers double
check whether the requested max input size is in reasonable range.

2019-08-27 09:18:21.302 20948-21800/com.android.myapp D/CCodec: setup
formats input: AMessage(what = 0x00000000) = {

      int32_t feature-secure-playback = 0

      int32_t frame-rate = 30

      int32_t height = 1080

      int32_t level = 1

      int32_t max-input-size = 3317760

      string mime = "video/avc"

      int32_t priority = 0

      int32_t profile = 1

      int32_t width = 1920

      Rect crop(0, 0, 1919, 1079)

    } and output: AMessage(what = 0x00000000) = {

      int32_t android._video-scaling = 1

      Rect crop(0, 0, 1919, 1079)

      int32_t color-standard = 1

      int32_t color-range = 2

      int32_t color-transfer = 3

      int32_t android._dataspace = 260

      int32_t width = 1920

      int32_t feature-secure-playback = 0

      int32_t frame-rate = 30

      int32_t height = 1080

      int32_t max-height = 1080

      int32_t max-width = 1920

      string mime = "video/raw"

      int32_t priority = 0

      int32_t rotation-degrees = 0

      Buffer hdr-static-info = {

        00000000:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
................

        00000010:  00 00 00 00 00 00 00 00  00
.........

      }

      int32_t android._color-format

2019-08-27 09:18:21.324 20948-21800/com.android.myapp W/Codec2Client: query
-- param skipped: index = 1342179345.

2019-08-27 09:18:21.324 20948-21800/com.android.myapp W/Codec2Client: query
-- param skipped: index = 2415921170.

2019-08-27 09:18:21.324 20948-21800/com.android.myapp W/Codec2Client: query
-- param skipped: index = 1610614798.

2019-08-27 09:18:21.325 20948-21800/com.android.myapp
D/CCodecBufferChannel: [c2.qti.avc.decoder#435] Query input allocators
returned 0 params => BAD_INDEX (6)

2019-08-27 09:18:21.327 20948-21800/com.android.myapp
D/CCodecBufferChannel: [c2.qti.avc.decoder#435] Created input block pool
with allocatorID 16 => poolID 19 - OK (0)

2019-08-27 09:18:21.327 20948-21800/com.android.myapp
D/CCodecBufferChannel: [c2.qti.avc.decoder#435] Query output allocators
returned 0 params => BAD_INDEX (6)

2019-08-27 09:18:21.327 20948-21800/com.android.myapp
D/CCodecBufferChannel: [c2.qti.avc.decoder#435] Query output surface
allocator returned 0 params => BAD_INDEX (6)

2019-08-27 09:18:21.327 20948-21800/com.android.myapp
I/CCodecBufferChannel: [c2.qti.avc.decoder#435] Created output block pool
with allocatorID 18 => poolID 187 - OK

2019-08-27 09:18:21.327 20948-21800/com.android.myapp
D/CCodecBufferChannel: [c2.qti.avc.decoder#435] Configured output block
pool ids 187 => OK

2019-08-27 09:18:21.328 20948-21800/com.android.myapp
D/Codec2-block_helper: remote graphic buffer migration 0/0

2019-08-27 09:18:21.328 20948-21800/com.android.myapp D/Codec2Client:
generation remote change 21450756

2019-08-27 09:18:21.348 20948-21042/com.android.myapp D/BufferPoolAccessor:
bufferpool2 0x6fd6dc9c20 : 0(0 size) total buffers - 0(0 size) used buffers
- 0/8 (recycle/alloc) - 8/40 (fetch/transfer)

2019-08-27 09:18:21.348 20948-21042/com.android.myapp D/BufferPoolAccessor:
Destruction - bufferpool2 0x6fd6dc9c20 cached: 0/0M, 0/0% in use; allocs:
8, 0% recycled; transfers: 40, 80% unfetced

2019-08-27 09:18:21.355 20948-21805/com.android.myapp D/CCodec:
allocate(c2.android.aac.decoder)

2019-08-27 09:18:21.355 20948-21805/com.android.myapp I/Codec2Client:
Creating a Codec2 client to service "default"

2019-08-27 09:18:21.357 20948-21805/com.android.myapp I/Codec2Client:
Client to Codec2 service "default" created

2019-08-27 09:18:21.357 20948-21805/com.android.myapp I/CCodec: setting up
'default' as default (vendor) store

2019-08-27 09:18:21.364 20948-21042/com.android.myapp D/PipelineWatcher:
onInputBufferReleased: frameIndex not found (1); ignored

2019-08-27 09:18:21.377 20948-21805/com.android.myapp I/CCodec: Created
component [c2.android.aac.decoder]

2019-08-27 09:18:21.377 20948-21805/com.android.myapp D/CCodecConfig: read
media type: audio/mp4a-latm

2019-08-27 09:18:21.378 20948-21805/com.android.myapp
D/ReflectedParamUpdater: extent() != 1 for single value type:
algo.buffers.max-count.values

2019-08-27 09:18:21.379 20948-21805/com.android.myapp
D/ReflectedParamUpdater: extent() != 1 for single value type:
output.subscribed-indices.values

2019-08-27 09:18:21.379 20948-21805/com.android.myapp
D/ReflectedParamUpdater: extent() != 1 for single value type:
input.buffers.allocator-ids.values

2019-08-27 09:18:21.379 20948-21805/com.android.myapp
D/ReflectedParamUpdater: extent() != 1 for single value type:
output.buffers.allocator-ids.values

2019-08-27 09:18:21.379 20948-21805/com.android.myapp
D/ReflectedParamUpdater: extent() != 1 for single value type:
algo.buffers.allocator-ids.values

2019-08-27 09:18:21.379 20948-21805/com.android.myapp
D/ReflectedParamUpdater: extent() != 1 for single value type:
output.buffers.pool-ids.values

2019-08-27 09:18:21.379 20948-21805/com.android.myapp
D/ReflectedParamUpdater: extent() != 1 for single value type:
algo.buffers.pool-ids.values

2019-08-27 09:18:21.380 20948-21800/com.android.myapp D/CCodecConfig: c2
config is Dict {

      c2::i32 algo.priority.value = 0

      c2::float algo.rate.value = 4.2039e-44

      c2::u32 algo.secure-mode.value = 0

      c2::float coded.frame-rate.value = 30

      c2::u32 coded.pl.level = 20480

      c2::u32 coded.pl.profile = 20480

      c2::u32 coded.vui.color.matrix = 0

      c2::u32 coded.vui.color.primaries = 0

      c2::u32 coded.vui.color.range = 0

      c2::u32 coded.vui.color.transfer = 0

      c2::u32 default.color.matrix = 0

      c2::u32 default.color.primaries = 3

      c2::u32 default.color.range = 2

      c2::u32 default.color.transfer = 0

      c2::u32 input.buffers.max-size.value = 3317760

      c2::u32 input.delay.value = 4

      string input.media-type.value = "video/avc"

      c2::u32 output.delay.value = 18

      string output.media-type.value = "video/raw"

      c2::u32 raw.color.matrix = 3

      c2::u32 raw.color.primaries = 3

      c2::u32 raw.color.range = 1

      c2::u32 raw.color.transfer = 3

      c2::u32 raw.crop.height = 1080

      c2::u32 raw.crop.left = 0

      c2::u32 raw.crop.top = 0

      c2::u32 raw.crop.width = 1920

      c2::float raw.hdr-static-info

2019-08-27 09:18:21.380 20948-21800/com.android.myapp
D/CCodecBufferChannel: [c2.qti.avc.decoder#435] onWorkDone: output format
changed to AMessage(what = 0x00000000) = {

      int32_t android._video-scaling = 1

      Rect crop(0, 0, 1919, 1079)

      int32_t color-standard = 2

      int32_t color-range = 1

      int32_t color-transfer = 3

      int32_t android._dataspace = 257

      int32_t width = 1920

      int32_t feature-secure-playback = 0

      int32_t frame-rate = 30

      int32_t height = 1080

      int32_t max-height = 1080

      int32_t max-width = 1920

      string mime = "video/raw"

      int32_t priority = 0

      int32_t rotation-degrees = 0

      Buffer hdr-static-info = {

        00000000:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
................

        00000010:  00 00 00 00 00 00 00 00  00
.........

      }

      int32_t android._color-format = 2130708361

      int32_t color-format = 2130708361

    }

2019-08-27 09:18:21.380 20948-21805/com.android.myapp I/CCodecConfig: query
failed after returning 16 values (BAD_INDEX)

2019-08-27 09:18:21.380 20948-21805/com.android.myapp D/CCodecConfig: c2
config is Dict {

      c2::u32 coded.aac-packaging.value = 0

      c2::u32 coded.bitrate.value = 64000

      c2::u32 coded.pl.level = 0

      c2::u32 coded.pl.profile = 8192

      c2::float coding.drc.attenuation-factor.value = 1

      c2::float coding.drc.boost-factor.value = 1

      c2::i32 coding.drc.compression-mode.value = 3

      c2::i32 coding.drc.effect-type.value = 3

      c2::float coding.drc.encoded-level.value = 0.25

      c2::float coding.drc.reference-level.value = -16

      c2::u32 input.buffers.max-size.value = 8192

      c2::u32 input.delay.value = 0

      string input.media-type.value = "audio/mp4a-latm"

      c2::u32 output.delay.value = 2

      string output.media-type.value = "audio/raw"

      c2::u32 raw.channel-count.value = 1

      c2::u32 raw.sample-rate.value = 44100

    }

2019-08-27 09:18:21.381 20948-21805/com.android.myapp D/CCodecConfig:
config failed => CORRUPTED

2019-08-27 09:18:21.382 20948-21805/com.android.myapp D/CCodecConfig: c2
config is Dict {

      c2::u32 coded.aac-packaging.value = 0

      c2::u32 coded.bitrate.value = 64000

      c2::u32 coded.pl.level = 0

      c2::u32 coded.pl.profile = 8192

      c2::float coding.drc.attenuation-factor.value = 1

      c2::float coding.drc.boost-factor.value = 1

      c2::i32 coding.drc.compression-mode.value = 3

      c2::i32 coding.drc.effect-type.value = 3

      c2::float coding.drc.encoded-level.value = 0.25

      c2::float coding.drc.reference-level.value = -16

      c2::u32 input.buffers.max-size.value = 8192

      c2::u32 input.delay.value = 0

      string input.media-type.value = "audio/mp4a-latm"

      c2::u32 output.delay.value = 2

      string output.media-type.value = "audio/raw"

      c2::u32 raw.channel-count.value = 1

      c2::u32 raw.sample-rate.value = 8000

    }

2019-08-27 09:18:21.382 20948-21805/com.android.myapp W/Codec2Client: query
-- param skipped: index = 1107298332.

2019-08-27 09:18:21.382 20948-21805/com.android.myapp D/CCodec: client
requested max input size 309, which is smaller than what component
recommended (8192); overriding with component recommendation.

2019-08-27 09:18:21.382 20948-21805/com.android.myapp W/CCodec: This
behavior is subject to change. It is recommended that app developers double
check whether the requested max input size is in reasonable range.

2019-08-27 09:18:21.382 20948-21805/com.android.myapp D/CCodec: setup
formats input: AMessage(what = 0x00000000) = {

      int32_t channel-count = 1

      int32_t level = 0

      int32_t max-input-size = 8192

      string mime = "audio/mp4a-latm"

      int32_t profile = 2

      int32_t sample-rate = 8000

    } and output: AMessage(what = 0x00000000) = {

      int32_t channel-count = 1

      string mime = "audio/raw"

      int32_t sample-rate = 8000

    }

2019-08-27 09:18:21.387 20948-21805/com.android.myapp W/Codec2Client: query
-- param skipped: index = 1342179345.

2019-08-27 09:18:21.387 20948-21805/com.android.myapp W/Codec2Client: query
-- param skipped: index = 2415921170.

2019-08-27 09:18:21.387 20948-21805/com.android.myapp W/Codec2Client: query
-- param skipped: index = 1610614798.

2019-08-27 09:18:21.388 20948-21805/com.android.myapp
D/CCodecBufferChannel: [c2.android.aac.decoder#110] Created input block
pool with allocatorID 16 => poolID 20 - OK (0)

2019-08-27 09:18:21.392 20948-21042/com.android.myapp D/BufferPoolAccessor:
bufferpool2 0x6fd518f020 : 0(0 size) total buffers - 0(0 size) used buffers
- 1/7 (recycle/alloc) - 6/15 (fetch/transfer)

2019-08-27 09:18:21.393 20948-21042/com.android.myapp D/BufferPoolAccessor:
Destruction - bufferpool2 0x6fd518f020 cached: 0/0M, 0/0% in use; allocs:
7, 14% recycled; transfers: 15, 60% unfetced

2019-08-27 09:18:21.393 20948-21805/com.android.myapp
I/CCodecBufferChannel: [c2.android.aac.decoder#110] Created output block
pool with allocatorID 16 => poolID 179 - OK

2019-08-27 09:18:21.393 20948-21805/com.android.myapp
D/CCodecBufferChannel: [c2.android.aac.decoder#110] Configured output block
pool ids 179 => OK

2019-08-27 09:18:21.436 20948-21797/com.android.myapp E/AudioTrack:
Discontinuity detected [expected 128000, got 459375]

2019-08-27 09:18:21.438 20948-21799/com.android.myapp D/SurfaceUtils:
connecting to surface 0x6fc5b0b010, reason connectToSurface

2019-08-27 09:18:21.438 20948-21799/com.android.myapp I/MediaCodec:
[c2.qti.avc.decoder] setting surface generation to 21450757

2019-08-27 09:18:21.438 20948-21799/com.android.myapp D/SurfaceUtils:
disconnecting from surface 0x6fc5b0b010, reason connectToSurface(reconnect)

2019-08-27 09:18:21.438 20948-21799/com.android.myapp D/SurfaceUtils:
connecting to surface 0x6fc5b0b010, reason connectToSurface(reconnect)

2019-08-27 09:18:21.440 20948-21799/com.android.myapp
D/Codec2-block_helper: remote graphic buffer migration 14/14

2019-08-27 09:18:21.440 20948-21799/com.android.myapp D/Codec2Client:
generation remote change 21450757

2019-08-27 09:18:21.444 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 18 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 19 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 20 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 21 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 22 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 23 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 24 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 25 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 26 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 27 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 28 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 29 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 30 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 31 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 32 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 33 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 34 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 35 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 36 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.445 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 37 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 38 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 39 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 40 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 41 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 42 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 43 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 44 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 45 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 46 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 47 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 48 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 49 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 50 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 51 is
not owned by the producer (state = FREE)

2019-08-27 09:18:21.446 20948-21799/com.android.myapp
E/BufferQueueProducer: [SurfaceTexture-1-20948-1] detachBuffer: slot 52 is
not owned by the producer (state = FREE)





Here is the command string for the merging


val cmd = "-y -i $inputFileUrl -i $transparencyFullUrl -filter_complex
[1]format=bgra,colorchannelmixer=aa=1,rotate=0:c=black@0:ow=rotw(0):oh=roth(0)[overlayImage];[0][overlayImage]overlay=(main_w-overlay_w)/2:(main_h-overlay_h)/2
$outputPathAndName"
val command = cmd.split(" ".toRegex()).dropLastWhile { it.isEmpty()
}.toTypedArray()


try {
    FFmpeg.getInstance(context).execute(command, object :
ExecuteBinaryResponseHandler() {

...







[image: Mailtrack]
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&>
Sender
notified by
Mailtrack
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&>
08/27/19,
09:28:25 AM

On Mon, Aug 26, 2019 at 10:20 AM Darrin Smith <[hidden email]> wrote:

> Are there any "tricks" to improve the performance of merging an image
> (overlay) with a video?
>
> I have a video snippet (cropped from a larger video) that I then add an
> overlay to so additional data is added in the video. The png I use is the
> same size as the video source.  I typically notice a 3X time to merge the
> png and the video as compared to the length of the video clip. So, if the
> clip is 5 seconds long, it normally takes FFMpeg 15 seconds to merge them
> together to create a final video. I'm using a Pixel 3XL. Not THE fastest
> out there, but certainly in the upper tier.
>
> Is there anything that can be done to improve the performance?
>
> Thanks.
>
> Darrin
>
>
>
> [image: Mailtrack]
> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&> Sender
> notified by
> Mailtrack
> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&> 08/26/19,
> 10:15:02 AM
>
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Overlay performance

Paul B Mahol
>
> Here is the command string for the merging
>
>
> val cmd = "-y -i $inputFileUrl -i $transparencyFullUrl -filter_complex
> [1]format=bgra,colorchannelmixer=aa=1,rotate=0:c=black@0
> :ow=rotw(0):oh=roth(0)[overlayImage];[0][overlayImage]overlay=(main_w-overlay_w)/2:(main_h-overlay_h)/2
> $outputPathAndName"
> val command = cmd.split(" ".toRegex()).dropLastWhile { it.isEmpty()
> }.toTypedArray()
>

colorchannelmixer=aa=1 is here doing nothing.
Have you blindly copied this command from somewhere?

To make current command to process faster get better hardware, seriously.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Overlay performance

Darrin Smith
I did copy the command, but there is no way to get better hardware. This is
for an app where the user may have even lesser hardware (cell phones) than
what was used so I'm hoping to optimize it as much as possible.

Thank you for your feedback anything else you can think of?

On Tue, Aug 27, 2019, 10:57 AM Paul B Mahol <[hidden email]> wrote:

> >
> > Here is the command string for the merging
> >
> >
> > val cmd = "-y -i $inputFileUrl -i $transparencyFullUrl -filter_complex
> > [1]format=bgra,colorchannelmixer=aa=1,rotate=0:c=black@0
> >
> :ow=rotw(0):oh=roth(0)[overlayImage];[0][overlayImage]overlay=(main_w-overlay_w)/2:(main_h-overlay_h)/2
> > $outputPathAndName"
> > val command = cmd.split(" ".toRegex()).dropLastWhile { it.isEmpty()
> > }.toTypedArray()
> >
>
> colorchannelmixer=aa=1 is here doing nothing.
> Have you blindly copied this command from somewhere?
>
> To make current command to process faster get better hardware, seriously.
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Overlay performance

Carl Zwanzig
On 8/27/2019 9:04 AM, Darrin Smith wrote:
> I did copy the command, but there is no way to get better hardware. This is
> for an app where the user may have even lesser hardware (cell phones) than
> what was used so I'm hoping to optimize it as much as possible.

Try using a cropped overlay (not full size)- if most of the full-size
overlay is transparent, there's no point in processing that part, let ffmpeg
ignore those pixels.

Make sure the color spaces and formats are identical.

Remove the math from the filter complex; do it in the outside script (but
the next will take care of this).

Drop any filters/options that aren't actually necessary (i.e. rotate=0 and
all the rotate options); if you're not rotating, don't include them at all.

Consider using an uncompressed overlay (so it doesn't need to be decoded).


Basically, optimize all of the operations and remove redundant or
non-operational ones.


> Thank you for your feedback anything else you can think of?
Please stop top-posting on this list.

Later,

z!

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Overlay performance

Paul B Mahol
On Tue, Aug 27, 2019 at 6:17 PM Carl Zwanzig <[hidden email]> wrote:

> On 8/27/2019 9:04 AM, Darrin Smith wrote:
> > I did copy the command, but there is no way to get better hardware. This
> is
> > for an app where the user may have even lesser hardware (cell phones)
> than
> > what was used so I'm hoping to optimize it as much as possible.
>
> Try using a cropped overlay (not full size)- if most of the full-size
> overlay is transparent, there's no point in processing that part, let
> ffmpeg
> ignore those pixels.
>
>
Yea, that is sane approach.


> Make sure the color spaces and formats are identical.
>
> Remove the math from the filter complex; do it in the outside script (but
> the next will take care of this).
>
> Drop any filters/options that aren't actually necessary (i.e. rotate=0 and
> all the rotate options); if you're not rotating, don't include them at all.
>
> Consider using an uncompressed overlay (so it doesn't need to be decoded).
>
>
> Basically, optimize all of the operations and remove redundant or
> non-operational ones.
>
>
> > Thank you for your feedback anything else you can think of?
> Please stop top-posting on this list.
>
> Later,
>
> z!
>
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Overlay performance

Darrin Smith
Really appreciate the feedback.

One question though so I understand correctly.

It was suggested to use a cropped overlay. Does FFMpeg auto scale up then?

I need overlay content on all four corners of the video. What I did was
create on png that is the same size as the video source resolution. If I
crop everything out but the text I need, then FFMpeg, by default, will
stretch that to the corners of the video? I had not read that anywhere so I
was just wanting to make sure that I understood correctly.

Again, thank you for your help here.



[image: Mailtrack]
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&>
Sender
notified by
Mailtrack
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&>
08/27/19,
11:46:35 AM

On Tue, Aug 27, 2019 at 11:21 AM Paul B Mahol <[hidden email]> wrote:

> On Tue, Aug 27, 2019 at 6:17 PM Carl Zwanzig <[hidden email]> wrote:
>
> > On 8/27/2019 9:04 AM, Darrin Smith wrote:
> > > I did copy the command, but there is no way to get better hardware.
> This
> > is
> > > for an app where the user may have even lesser hardware (cell phones)
> > than
> > > what was used so I'm hoping to optimize it as much as possible.
> >
> > Try using a cropped overlay (not full size)- if most of the full-size
> > overlay is transparent, there's no point in processing that part, let
> > ffmpeg
> > ignore those pixels.
> >
> >
> Yea, that is sane approach.
>
>
> > Make sure the color spaces and formats are identical.
> >
> > Remove the math from the filter complex; do it in the outside script (but
> > the next will take care of this).
> >
> > Drop any filters/options that aren't actually necessary (i.e. rotate=0
> and
> > all the rotate options); if you're not rotating, don't include them at
> all.
> >
> > Consider using an uncompressed overlay (so it doesn't need to be
> decoded).
> >
> >
> > Basically, optimize all of the operations and remove redundant or
> > non-operational ones.
> >
> >
> > > Thank you for your feedback anything else you can think of?
> > Please stop top-posting on this list.
> >
> > Later,
> >
> > z!
> >
> > _______________________________________________
> > ffmpeg-user mailing list
> > [hidden email]
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >
> > To unsubscribe, visit link above, or email
> > [hidden email] with subject "unsubscribe".
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Overlay performance

Micael Silva
On Tue, 27 Aug 2019, 13:50 Darrin Smith, <[hidden email]> wrote:

> Really appreciate the feedback.
>
> One question though so I understand correctly.
>
> It was suggested to use a cropped overlay. Does FFMpeg auto scale up then?
>
> I need overlay content on all four corners of the video. What I did was
> create on png that is the same size as the video source resolution. If I
> crop everything out but the text I need, then FFMpeg, by default, will
> stretch that to the corners of the video? I had not read that anywhere so I
> was just wanting to make sure that I understood correctly.
>
> Again, thank you for your help here.
>
>
>
> [image: Mailtrack]
> <
> https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&
> >
> Sender
> notified by
> Mailtrack
> <
> https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&
> >
> 08/27/19,
> 11:46:35 AM
>
> On Tue, Aug 27, 2019 at 11:21 AM Paul B Mahol <[hidden email]> wrote:
>
> > On Tue, Aug 27, 2019 at 6:17 PM Carl Zwanzig <[hidden email]> wrote:
> >
> > > On 8/27/2019 9:04 AM, Darrin Smith wrote:
> > > > I did copy the command, but there is no way to get better hardware.
> > This
> > > is
> > > > for an app where the user may have even lesser hardware (cell phones)
> > > than
> > > > what was used so I'm hoping to optimize it as much as possible.
> > >
> > > Try using a cropped overlay (not full size)- if most of the full-size
> > > overlay is transparent, there's no point in processing that part, let
> > > ffmpeg
> > > ignore those pixels.
> > >
> > >
> > Yea, that is sane approach.
> >
> >
> > > Make sure the color spaces and formats are identical.
> > >
> > > Remove the math from the filter complex; do it in the outside script
> (but
> > > the next will take care of this).
> > >
> > > Drop any filters/options that aren't actually necessary (i.e. rotate=0
> > and
> > > all the rotate options); if you're not rotating, don't include them at
> > all.
> > >
> > > Consider using an uncompressed overlay (so it doesn't need to be
> > decoded).
> > >
> > >
> > > Basically, optimize all of the operations and remove redundant or
> > > non-operational ones.
> > >
> > >
> > > > Thank you for your feedback anything else you can think of?
> > > Please stop top-posting on this list.
> > >
> > > Later,
> > >
> > > z!
> > >
> > >


No, it doesn't. Overlay doesn't scale or stretch by itself. It is overlayed
pixel by pixel, if the source image changes to position or size the result
image can change just because of that.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Overlay performance

Carl Zwanzig
In reply to this post by Darrin Smith
On 8/27/2019 9:49 AM, Darrin Smith wrote:
> I need overlay content on all four corners of the video. What I did was
> create on png that is the same size as the video source resolution.

Ah, that was not mentioned (details are important). If it has to be
full-size, do whatever you can to minimize other processing/conversion, such
as using the same color space as the video.

You need to try a bunch of options to see what works the best for the target
platform.

Please stop top-posting, if you're not sure what that is.... google....

Later,

z!
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".