[Templates] PROCESS and file encoding

[Olivier Salaün - CRU]

Hello,

We are using TT2 as the template mecanism for the Sympa mailing list
software (http://www.sympa.org). We have faced some miss-interpretation
encoding problems while using TT2. I think TT2 should provide some new
options to allow correct interpretation of file encoding, as explained
below.

We are using a main TT2 template that includes many PROCESS directives to
include other files and build an HTML file. Some of these included files
are encoded using different charset. TT2 syntax does not provide a way to
specify what is the file encoding, so it is well interpreted.

Perl 5.8 now provides an I/O layers that can transparently perform
character encoding to or from the perl encoding. That's amazingly
convenient because one you tell perl what is the input/output file
encoding supposed to be, you don't have to decode/recode yourself. See
http://search.cpan.org/~nwclark/perl-5.8.8/lib/open.pm

If I read the TT2 documentation and part of the code carefully, there is
no way to tell Template::Provider::_load() to use a provided encoding
while opening the template file. Therefore there are many cases where the
input file may be miss-interpreted.

Currently the code looks like that :
          if (open(FH, $name)) {
             my $text = <FH>;
             $text = $self->_decode_unicode($text) if $self->{ UNICODE };

The open() call should take a third argument to define the file encoding :
    open FH, '<:utf8', $name;
OR
    open FH, '<:encoding(iso-8859-1)', $name;

The other solution is to use binmode as follows :
    open FH, $name;
    binmode FH, ':utf8';

Both Template::process() and the PROCESS directive require an encoding
parameter that would go all the way to Template::Provider::_load().

Would the developers consider adding this feature to TT2 ?