Invalid NAL unit size, S3 Invalid data found when processing input

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Invalid NAL unit size, S3 Invalid data found when processing input

Christoph Burgmair
Hi,
yesterday at 11:20 CET we discovered problems with our AWS batch transcoding.
The EC2 machines that execute the transcoding tasks have problems accessing the mp4 source files by streaming directly from S3.
These streaming / access problems result in FFmpeg transcoding problems.
This is the FFmpeg command we used:

ffmpeg -y -i "$(aws s3 presign s3://<bucket-name-hidden>/VX-926874.mp4)" -an -codec:v libx264 -profile:v main -preset veryslow -sws_flags lanczos -s:v 640x360 -aspect 16:9 -b:v 500k -maxrate:v 500k -bufsize:v 500k -keyint_min 50 -g 50 -x264opts nal-hrd=cbr:pic-struct:no-scenecut:colorprim=bt709:transfer=bt709:colormatrix=bt709:open_gop=0:ref=4 -pix_fmt yuv420p -an -async 1 -metadata:s language=zxx -movflags faststart -loglevel verbose /out/aedaa15d-1b6b-4178-ab5d-d6d845dab205_2019-08-2609:25:10.922542_868803588-402356695-74297555.mp4

Here are the resulting logs:

...
frame=26164 fps= 77 q=27.0 size= 63488kB time=00:17:23.64 bitrate= 498.3kbits/s speed=3.07x

frame=26188 fps= 76 q=27.0 size= 63744kB time=00:17:24.60 bitrate= 499.9kbits/s speed=3.02x

[tls @ 0x1c10fc0] error:00000000:lib(0):func(0):reason(0)

[h264 @ 0x1e58840] Invalid NAL unit size (63816 > 51433).

[h264 @ 0x1e58840] Error splitting the input into NAL units.

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x1c05380] stream 0, offset 0xaaa1ca47: partial file

https://<bucket-name-hidden>.s3.amazonaws.com/VX-926874.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Date=20190826T092511Z&X-Amz-SignedHeaders=host&X-Amz-Security-Token=<tocken-hidden>: Invalid data found when processing input

frame=26208 fps= 40 q=26.0 size= 63744kB time=00:17:25.40 bitrate= 499.5kbits/s speed=1.61x

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x1c05380] stream 0, offset 0xaaa375a4: partial file

https://<bucket-name-hidden>.s3.amazonaws.com/VX-926874.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Date=20190826T092511Z&X-Amz-SignedHeaders=host&X-Amz-Security-Token=F<tocken-hidden>: Invalid data found when processing input

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x1c05380] stream 0, offset 0xaaa46722: partial file



First, the transcoding runs quite smoothly, but after a while we geht the error "Invalid NAL unit size (63816 > 51433).“ as seen above.
The strange thing is that this error only happens very rarely. Usually everything is fine and the files are fully transcoded. But sometimes we get these errors which result in no fully transcoded output files.
From AWS side everything was ok.

I hope someone can help me here.

Best regards,
Christoph
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Invalid NAL unit size, S3 Invalid data found when processing input

Carl Eugen Hoyos-2
Am Di., 27. Aug. 2019 um 18:02 Uhr schrieb Christoph Burgmair
<[hidden email]>:

> ffmpeg -y -i "$(aws s3 presign s3://<bucket-name-hidden>/VX-926874.mp4)" -an -codec:v libx264 -profile:v main -preset veryslow -sws_flags lanczos -s:v 640x360 -aspect 16:9 -b:v 500k -maxrate:v 500k -bufsize:v 500k -keyint_min 50 -g 50 -x264opts nal-hrd=cbr:pic-struct:no-scenecut:colorprim=bt709:transfer=bt709:colormatrix=bt709:open_gop=0:ref=4 -pix_fmt yuv420p -an -async 1 -metadata:s language=zxx -movflags faststart -loglevel verbose /out/aedaa15d-1b6b-4178-ab5d-d6d845dab205_2019-08-2609:25:10.922542_868803588-402356695-74297555.mp4
>
> Here are the resulting logs:

Sadly not;-(

Please provide an input file that allows to reproduce the issue.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".