[Templates] Apache::Template, Plugins and UTF-8 OH MY!

[Clinton Gormley]

Hi Todd

I'm not sure if this is the problem, but it may be at least PART of the
problem.

As I understand it, TT either needs to be told explicitly to use UTF8 in
the process() call, or it needs a UTF8-BOM at the beginning of the
template to know that it should treat the template (and the included
UTF8 data) as UTF8.

Basically, I got some fairly funky renderings until I added the BOM.

I have a little script which I use to add the BOM (once and only once)
to all of my templates - may be of use to you (works in linux, will need
a few changes to work on Windows):


#!/usr/bin/perl
use strict;
use warnings FATAL => 'all';

our $root = '/PATH/TO/TEMPLATES';
our $bom  = "\x{EF}\x{BB}\x{BF}";

process_dir($root);
$| = 1;

sub process_dir {
    my $dir   = shift;
    my @files = glob( $dir . "/*" );
    foreach my $file (@files) {
        if ( -f $file && $file =~ /\.tt$/ ) {
            process_file($file);
        }
        elsif ( -d $file && $file !~ m|/\.svn| ) {
            process_dir($file);
        }
    }
}

sub process_file {
    my $name = my $file = shift;
    $name =~ s/^$root//;
    print sprintf( "Processing : %-50s", $name );
    local ( *FH, $/ );
    open( FH, '<:bytes', $file )
        or die "can't open $file: $!";
    my $a = <FH>;
    close FH;

    my $b = $a;
    $a =~ s/$bom//g;
    $a = $bom . $a;
    if ( $a ne $b ) {
        open( FH, '>:bytes', $file )
            or die "can't write to $file : $!";
        print FH $a;
        close FH
            or die "can't close file $file";
        print " ...Updated\n";
    }
    else {
        print "\n";
    }
}