Weird characters in created passlogfile on Windows

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Weird characters in created passlogfile on Windows

Thierry Lelégard
Hello,

I found a strange problem with ffmpeg on Windows (not tested yet on Linux)
with the -passlogfile option.

Consider a file path containing non ASCII characters like è (e grave) used
in all file specifications (in a directory name for instance). There are
three different use cases: input file, output file, pass log file.

In the first two cases, the non-ASCII characters in file names are processed
correctly. But in the case of the pass log file, the file name is transformed
into some weird sequence (typical UTF-8 sequence). The strange thing is that
ffmpeg creates two files, the pass log file and the output file, but processes
the characters in the file name quite differently. The way the two files are
created must be completely different.

See the following example:

D:\Public\Videos\TEMP>ls -l h*
-rw-rw-rw-   1 user     group    18800000 Oct 15 22:50 hè.ts

There is one single file. Let's transcode it that way;

D:\Public\Videos\TEMP>ffmpeg -i hè.ts -pass 1 -passlogfile hè hè.mpg
ffmpeg version N-56060-gbcd1c20 Copyright (c) 2000-2013 the FFmpeg developers
   built on Sep  6 2013 00:49:05 with gcc 4.7.3 (GCC)
   configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetype
--enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex
--enable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
   libavutil      52. 43.100 / 52. 43.100
   libavcodec     55. 31.101 / 55. 31.101
   libavformat    55. 16.101 / 55. 16.101
   libavdevice    55.  3.100 / 55.  3.100
   libavfilter     3. 83.104 /  3. 83.104
   libswscale      2.  5.100 /  2.  5.100
   libswresample   0. 17.103 /  0. 17.103
   libpostproc    52.  3.100 / 52.  3.100
[....]

Now look at the result:

D:\Public\Videos\TEMP>ls -l h*
-rw-rw-rw-   1 user     group      258179 Oct 15 22:50 hè-0.log
-rw-rw-rw-   1 user     group     5261312 Oct 15 22:50 hè.mpg
-rw-rw-rw-   1 user     group    18800000 Oct 15 22:50 hè.ts

The command contains "-passlogfile hè hè.mpg" but the "hè" in the two
created files resulted in two different encodings.

The problem is that the specified file name on the command line does NOT
correspond to the actual file name in the file system. One may say that
we don't care since this is a temporary log file.

But there is a real problem when the non-ASCII character is in the name
of the directory for all files. In that case, ffmpeg transforms the
directory name and the resulting path is invalid since the transformed
path uses a non-existent directory.

Error resulting from option -passlogfile D:\Public\Videos\DVD\hè\fflog
(assuming that the directory D:\Public\Videos\DVD\hè does exist and
contains all files);

Cannot write log file 'D:\Public\Videos\DVD\hè\fflog-0.log' for pass-1 encoding: No such file or directory

Interestingly, the characters in the error message are correct. So, you do
not understand why this failed. This is only after a test with an accent
in the file name but none in the directory name that I discovered the
transformation.

Is this known? Shall I open a bug report for that?

Best regards,
-Thierry

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: Weird characters in created passlogfile on Windows

Tom Evans
On Tue, Oct 15, 2013 at 10:15 PM, Thierry Lelégard
<[hidden email]> wrote:
> Hello,
>
> I found a strange problem with ffmpeg on Windows (not tested yet on Linux)
> with the -passlogfile option.
>
> Consider a file path containing non ASCII characters like è (e grave) used
> in all file specifications (in a directory name for instance). There are
> three different use cases: input file, output file, pass log file.

All three work correctly under FreeBSD:

> $ ffmpeg -y -i ../hè.mkv -t 60 -pass 1 -passlogfile hè hè.ts >/dev/null 2>&1 && ls -l
total 1
-rw-r--r--  1 tom  wheel   202078 16 Oct 09:05 hè-0.log
-rw-r--r--  1 tom  wheel  5054756 16 Oct 09:05 hè.ts

Cheers

Tom
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: Weird characters in created passlogfile on Windows

jackyr

Le 16/10/2013 11:06, Tom Evans a écrit :

> On Tue, Oct 15, 2013 at 10:15 PM, Thierry Lelégard
> <[hidden email]> wrote:
>> Hello,
>>
>> I found a strange problem with ffmpeg on Windows (not tested yet on Linux)
>> with the -passlogfile option.
>>
>> Consider a file path containing non ASCII characters like è (e grave) used
>> in all file specifications (in a directory name for instance). There are
>> three different use cases: input file, output file, pass log file.
> All three work correctly under FreeBSD:
>
>> $ ffmpeg -y -i ../hè.mkv -t 60 -pass 1 -passlogfile hè hè.ts >/dev/null 2>&1 && ls -l
> total 1
> -rw-r--r--  1 tom  wheel   202078 16 Oct 09:05 hè-0.log
> -rw-r--r--  1 tom  wheel  5054756 16 Oct 09:05 hè.ts
>
> Cheers
>
> Tom
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Hi
I havent tested with your particular case but it seems to be related to
windows dos page
  (characters dos convertion page : chcp)
I had same problem running batch file  " ffmpeg  -list_devices true -f
dshow -i dummy"
i solved it that way
    on dos screen
       undestand which active page (characters table)
       type    chcp  and note the given value (mine is 850 )
       type    set chcp 1252  (this enlarge ascci characters page to latin )

      go back to the default page when you end up the full session
   hope this help
regards
jacky

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: Weird characters in created passlogfile on Windows

Thierry Lelegard

> De: "jacky" <[hidden email]>
>
> I havent tested with your particular case but it seems to be related
> to windows dos page (characters dos convertion page : chcp)

I understand what you mean but this cannot be the right explanation.

I wrote that the same sequence of characters was used three times
on the command line: ffmpeg -i hè.ts -pass 1 -passlogfile hè hè.mpg

However, only the sequence for -passlogfile is interpreted differently:

-rw-rw-rw-   1 user     group      258179 Oct 15 22:50 hè-0.log
-rw-rw-rw-   1 user     group     5261312 Oct 15 22:50 hè.mpg
-rw-rw-rw-   1 user     group    18800000 Oct 15 22:50 hè.ts

If your explanation were right, the output file hè.mpg would also have
been created with the same gotcha. And the input file hè.ts would not
even have been found in the first place.

So, ffmpeg treats command line arguments and code pages quite properly.
There is something specific in the creation of the passlogfile.

-Thierry
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: Weird characters in created passlogfile on Windows

Rodney Baker
On Wed, 16 Oct 2013 14:51:02 Thierry Lelegard wrote:

> > De: "jacky" <[hidden email]>
> >
> > I havent tested with your particular case but it seems to be related
> > to windows dos page (characters dos convertion page : chcp)
>
> I understand what you mean but this cannot be the right explanation.
>
> I wrote that the same sequence of characters was used three times
> on the command line: ffmpeg -i hè.ts -pass 1 -passlogfile hè hè.mpg
>
> However, only the sequence for -passlogfile is interpreted differently:
>
> -rw-rw-rw-   1 user     group      258179 Oct 15 22:50 hè-0.log
> -rw-rw-rw-   1 user     group     5261312 Oct 15 22:50 hè.mpg
> -rw-rw-rw-   1 user     group    18800000 Oct 15 22:50 hè.ts
>
> If your explanation were right, the output file hè.mpg would also have
> been created with the same gotcha. And the input file hè.ts would not
> even have been found in the first place.
>
> So, ffmpeg treats command line arguments and code pages quite properly.
> There is something specific in the creation of the passlogfile.
>
> -Thierry
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user

Thierry,

It might be worth asking this question on the ffmpeg-devel mailing list. If
you do, make sure you include the full command line and full, uncut console
output, plus as much detail as possible to reproduce the problem (the way
you've described it here is probably OK). Note that most developers use Linux
though, so be prepared to assist with testing any fixes and provide additional
feedback.

If you know enough to propose a patch it would be welcomed I'm sure. You can
submit the proposed patch to the ffmpeg-devel mailing list.

Rodney.

--
==============================================================
Rodney Baker VK5ZTV
[hidden email]
==============================================================
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: Weird characters in created passlogfile on Windows

Carl Eugen Hoyos
Rodney Baker <rodney.baker <at> iinet.net.au> writes:

> It might be worth asking this question on the
> ffmpeg-devel mailing list.

(I did not really read this thread.)
Please only do that if you plan to work on a fix yourself,
bug reports should never be sent to ffmpeg-devel.

Carl Eugen

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|

Re: Weird characters in created passlogfile on Windows

Thierry Lelegard
> De: "Carl Eugen Hoyos" <[hidden email]>
>
> > It might be worth asking this question on the
> > ffmpeg-devel mailing list.
>
> (I did not really read this thread.)
> Please only do that if you plan to work on a fix yourself,
> bug reports should never be sent to ffmpeg-devel.

You are right. I will submit a bug report to FFmpeg Trac system
when I have time.

-Thierry
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user