drawtext, suggestion for improvement

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

drawtext, suggestion for improvement

Michael Koch
Hi,

I have a suggestion for improvement for the drawtext filter.
If the text is read from a file, it must be in UTF-8 format. In Windows,
many text editors (including Notepad) insert a 3-byte "Byte order mark"
at the beginning of the file. This is described here:
https://en.wikipedia.org/wiki/UTF-8#Byte_order_mark

The problem is that drawtext prints these 3 bytes as non-printable
characters (empty rectangle). You can see an example screenshot here in
the Zeranoe discussion forum:
https://ffmpeg.zeranoe.com/forum/viewtopic.php?f=7&t=7813

My suggestion is to let the drawtext filter check the first three bytes
and if they are EF BB BF, then don't print them.

Michael
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: drawtext, suggestion for improvement

Carl Eugen Hoyos-2
Am Do., 26. März 2020 um 19:34 Uhr schrieb Michael Koch
<[hidden email]>:

> I have a suggestion for improvement for the drawtext filter.
> If the text is read from a file, it must be in UTF-8 format. In Windows,
> many text editors (including Notepad) insert a 3-byte "Byte order mark"
> at the beginning of the file. This is described here:
> https://en.wikipedia.org/wiki/UTF-8#Byte_order_mark
>
> The problem is that drawtext prints these 3 bytes as non-printable
> characters (empty rectangle). You can see an example screenshot here in
> the Zeranoe discussion forum:
> https://ffmpeg.zeranoe.com/forum/viewtopic.php?f=7&t=7813
>
> My suggestion is to let the drawtext filter check the first three bytes
> and if they are EF BB BF, then don't print them.

This bug report is - sadly - missing the (simplified) command line
you tested together with its complete, uncut console output and
- equally important - a simple input file that allows to reproduce.

Or to say it differently: We already have a very large number of
confirmed bugs, most of them show all of the above: Why should
anybody spend time to find out how to reproduce or don't you
agree that this time is better spend on reports that contain all
information?

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: drawtext, suggestion for improvement

Michael Koch
Am 26.03.2020 um 19:59 schrieb Carl Eugen Hoyos:

> Am Do., 26. März 2020 um 19:34 Uhr schrieb Michael Koch
> <[hidden email]>:
>
>> I have a suggestion for improvement for the drawtext filter.
>> If the text is read from a file, it must be in UTF-8 format. In Windows,
>> many text editors (including Notepad) insert a 3-byte "Byte order mark"
>> at the beginning of the file. This is described here:
>> https://en.wikipedia.org/wiki/UTF-8#Byte_order_mark
>>
>> The problem is that drawtext prints these 3 bytes as non-printable
>> characters (empty rectangle). You can see an example screenshot here in
>> the Zeranoe discussion forum:
>> https://ffmpeg.zeranoe.com/forum/viewtopic.php?f=7&t=7813
>>
>> My suggestion is to let the drawtext filter check the first three bytes
>> and if they are EF BB BF, then don't print them.
> This bug report is - sadly - missing the (simplified) command line
> you tested together with its complete, uncut console output and
> - equally important - a simple input file that allows to reproduce.

How to reproduce:
Use Windows Notepad to create a UTF-8 text file which contains only the
word "test".
This is how the file looks in a hex editor:
EF BB BF 74 65 73 74

Now run this command line and have a look at the output image.

C:\Users\mKoch\Desktop>c:\ffmpeg\ffmpeg -f lavfi -i color=c=white -vf
drawtext=t
extfile=test.txt -frames 1 -y out.png
ffmpeg version git-2020-03-23-ba698a2 Copyright (c) 2000-2020 the FFmpeg
develop
ers
   built with gcc 9.2.1 (GCC) 20200122
   configuration: --enable-gpl --enable-version3 --enable-sdl2
--enable-fontconfi
g --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d
--enable-libb
luray --enable-libfreetype --enable-libmp3lame
--enable-libopencore-amrnb --enab
le-libopencore-amrwb --enable-libopenjpeg --enable-libopus
--enable-libshine --e
nable-libsnappy --enable-libsoxr --enable-libsrt --enable-libtheora
--enable-lib
twolame --enable-libvpx --enable-libwavpack --enable-libwebp
--enable-libx264 --
enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma
--enable-zlib --e
nable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc
--enabl
e-libmysofa --enable-libspeex --enable-libxvid --enable-libaom
--enable-libmfx -
-enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-d3d11va
--enable-nv
enc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
--enable
-amf
   libavutil      56. 42.101 / 56. 42.101
   libavcodec     58. 76.100 / 58. 76.100
   libavformat    58. 42.100 / 58. 42.100
   libavdevice    58.  9.103 / 58.  9.103
   libavfilter     7. 77.100 /  7. 77.100
   libswscale      5.  6.101 /  5.  6.101
   libswresample   3.  6.100 /  3.  6.100
   libpostproc    55.  6.100 / 55.  6.100
Input #0, lavfi, from 'color=c=white':
   Duration: N/A, start: 0.000000, bitrate: N/A
     Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240
[SAR 1:1
DAR 4:3], 25 tbr, 25 tbn, 25 tbc
Stream mapping:
   Stream #0:0 -> #0:0 (rawvideo (native) -> png (native))
Press [q] to stop, [?] for help
Fontconfig error: Cannot load default config file
[Parsed_drawtext_0 @ 00000000005b2440] Using
"C:/Windows/fonts/KievitOffcPro.ttf
"
Output #0, image2, to 'out.png':
   Metadata:
     encoder         : Lavf58.42.100
     Stream #0:0: Video: png, rgb24, 320x240 [SAR 1:1 DAR 4:3], q=2-31,
200 kb/s,
  25 fps, 25 tbn, 25 tbc
     Metadata:
       encoder         : Lavc58.76.100 png
frame=    1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
speed=   2x

video:1kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing ove
rhead: unknown

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: drawtext, suggestion for improvement

Reino Wijnsma
Hello Michael,

On 2020-03-26T20:28:02+0100, Michael Koch <[hidden email]> wrote:

> How to reproduce:
> Use Windows Notepad to create a UTF-8 text file which contains only the word "test".
> This is how the file looks in a hex editor:
> EF BB BF 74 65 73 74
>
> Now run this command line and have a look at the output image.
>
> C:\Users\mKoch\Desktop>c:\ffmpeg\ffmpeg -f lavfi -i color=c=white -vf drawtext=textfile=test.txt -frames 1 -y out.png
> ffmpeg version git-2020-03-23-ba698a2 Copyright (c) 2000-2020 the FFmpeg developers
>   built with gcc 9.2.1 (GCC) 20200122
>   [...]
Your FFmpeg binary looks very recent, but what about the FontConfig library you compiled this FFmpeg binary with?

I was only able to reproduce this with one of the very first FFmpeg binaries I compiled myself (N-86393, dated 2017-06-06 and compiled with FontConfig 2.12.1).
The next FFmpeg binary I compiled (N-86763, dated 2017-07-12 and compiled with FontConfig 2.12.4), and all that come after it, don't show this behaviour.

I think updating FontConfig (and recompiling FFmpeg) would most likely solve your problem.

-- Reino
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: drawtext, suggestion for improvement

Michael Koch
Am 27.03.2020 um 22:58 schrieb Reino Wijnsma:

> Hello Michael,
>
> On 2020-03-26T20:28:02+0100, Michael Koch <[hidden email]> wrote:
>> How to reproduce:
>> Use Windows Notepad to create a UTF-8 text file which contains only the word "test".
>> This is how the file looks in a hex editor:
>> EF BB BF 74 65 73 74
>>
>> Now run this command line and have a look at the output image.
>>
>> C:\Users\mKoch\Desktop>c:\ffmpeg\ffmpeg -f lavfi -i color=c=white -vf drawtext=textfile=test.txt -frames 1 -y out.png
>> ffmpeg version git-2020-03-23-ba698a2 Copyright (c) 2000-2020 the FFmpeg developers
>>    built with gcc 9.2.1 (GCC) 20200122
>>    [...]
> Your FFmpeg binary looks very recent, but what about the FontConfig library you compiled this FFmpeg binary with?

I didn't compile it myself, it's the Windows version from Zeranoe.

Michael
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: drawtext, suggestion for improvement

Reino Wijnsma
On 2020-03-27T23:25:09+0100, Michael Koch <[hidden email]> wrote:
> Am 27.03.2020 um 22:58 schrieb Reino Wijnsma:
>> Your FFmpeg binary looks very recent, but what about the FontConfig library you compiled this FFmpeg binary with?
> I didn't compile it myself, it's the Windows version from Zeranoe.
Then Kyle knows what to do. If he's not subscribed to this list, then you might want to reach out to him.

-- Reino
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: drawtext, suggestion for improvement

kumowoon1025
In reply to this post by Reino Wijnsma
Hi,

> On Mar 27, 2020, at 17:58, Reino Wijnsma <[hidden email]> wrote:
>
> Hello Michael,
>
> On 2020-03-26T20:28:02+0100, Michael Koch <[hidden email]> wrote:
>> How to reproduce:
>> Use Windows Notepad to create a UTF-8 text file which contains only the word "test".
>> This is how the file looks in a hex editor:
>> EF BB BF 74 65 73 74
>>
>> Now run this command line and have a look at the output image.
>>
>> C:\Users\mKoch\Desktop>c:\ffmpeg\ffmpeg -f lavfi -i color=c=white -vf drawtext=textfile=test.txt -frames 1 -y out.png
>> ffmpeg version git-2020-03-23-ba698a2 Copyright (c) 2000-2020 the FFmpeg developers
>>  built with gcc 9.2.1 (GCC) 20200122
>>  [...]
> Your FFmpeg binary looks very recent, but what about the FontConfig library you compiled this FFmpeg binary with?
>
> I was only able to reproduce this with one of the very first FFmpeg binaries I compiled myself (N-86393, dated 2017-06-06 and compiled with FontConfig 2.12.1).
> The next FFmpeg binary I compiled (N-86763, dated 2017-07-12 and compiled with FontConfig 2.12.4), and all that come after it, don't show this behavior.
>
> I think updating FontConfig (and recompiling FFmpeg) would most likely solve your problem.
>
> -- Reino
>

That is bizarre, is there no way to change that behavior (in Notepad?) A byte order mark in UTF-8 is a bit confounding, I wonder if there is some history behind it.

Manually entering the code point in a UTF-8 document goes so far as to crash the character viewer pane on my Mac, I’ll have to test on more. In the drawtext filter it seems to depend on the actual font and its glyph table, some of them render it as literally a “zero-width nonbreaking space” which is indistinguishable to it not being there and some fonts do the missing glyph box thing.

Regards,
Ted Park

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: drawtext, suggestion for improvement

Michael Koch
Am 28.03.2020 um 07:22 schrieb Ted Park:

> Hi,
>
>> On Mar 27, 2020, at 17:58, Reino Wijnsma <[hidden email]> wrote:
>>
>> Hello Michael,
>>
>> On 2020-03-26T20:28:02+0100, Michael Koch <[hidden email]> wrote:
>>> How to reproduce:
>>> Use Windows Notepad to create a UTF-8 text file which contains only the word "test".
>>> This is how the file looks in a hex editor:
>>> EF BB BF 74 65 73 74
>>>
>>> Now run this command line and have a look at the output image.
>>>
>>> C:\Users\mKoch\Desktop>c:\ffmpeg\ffmpeg -f lavfi -i color=c=white -vf drawtext=textfile=test.txt -frames 1 -y out.png
>>> ffmpeg version git-2020-03-23-ba698a2 Copyright (c) 2000-2020 the FFmpeg developers
>>>   built with gcc 9.2.1 (GCC) 20200122
>>>   [...]
>> Your FFmpeg binary looks very recent, but what about the FontConfig library you compiled this FFmpeg binary with?
>>
>> I was only able to reproduce this with one of the very first FFmpeg binaries I compiled myself (N-86393, dated 2017-06-06 and compiled with FontConfig 2.12.1).
>> The next FFmpeg binary I compiled (N-86763, dated 2017-07-12 and compiled with FontConfig 2.12.4), and all that come after it, don't show this behavior.
>>
>> I think updating FontConfig (and recompiling FFmpeg) would most likely solve your problem.
>>
>> -- Reino
>>
> That is bizarre, is there no way to change that behavior (in Notepad?) A byte order mark in UTF-8 is a bit confounding, I wonder if there is some history behind it.
>
> Manually entering the code point in a UTF-8 document goes so far as to crash the character viewer pane on my Mac, I’ll have to test on more. In the drawtext filter it seems to depend on the actual font and its glyph table, some of them render it as literally a “zero-width nonbreaking space” which is indistinguishable to it not being there and some fonts do the missing glyph box thing.

I just saw that in a newer version of Notepad (on Windows 10) you can
select between "UTF-8" and "UTF-8 with BOM", and choosing "UTF-8" does
indeed solve the problem. The older Notepad version on my Windows 7
computer doesn't have this selection and always writes the BOM at the
beginning of the file.

Michael

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: drawtext, suggestion for improvement

Moritz Barsnick
On Sat, Mar 28, 2020 at 09:08:25 +0100, Michael Koch wrote:
> I just saw that in a newer version of Notepad (on Windows 10) you can
> select between "UTF-8" and "UTF-8 with BOM", and choosing "UTF-8" does
> indeed solve the problem. The older Notepad version on my Windows 7
> computer doesn't have this selection and always writes the BOM at the
> beginning of the file.

Oh, then Notepad has become a bit less braindead than it had been the
last 30 years.

Does it still open files with Unix line-endings displayed as one long
line?

Sorry for the rant,
Moritz
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: drawtext, suggestion for improvement

Moritz Barsnick
In reply to this post by Reino Wijnsma
On Fri, Mar 27, 2020 at 23:43:50 +0100, Reino Wijnsma wrote:
> On 2020-03-27T23:25:09+0100, Michael Koch <[hidden email]> wrote:
> > Am 27.03.2020 um 22:58 schrieb Reino Wijnsma:
> >> Your FFmpeg binary looks very recent, but what about the FontConfig library you compiled this FFmpeg binary with?
> > I didn't compile it myself, it's the Windows version from Zeranoe.
> Then Kyle knows what to do. If he's not subscribed to this list, then you might want to reach out to him.

The zeranow ffmpeg build is compiled and linked with Fontconfig
2.13.92, so that shouldn't be the issue.

Moritz
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".