Mailing List Archive: Catalyst: Users

I18N with quotation marks

Index | Next | Previous | View Threaded

ton.voon at opsera

Jul 1, 2009, 10:22 AM

Post #1 of 11 (884 views)
Permalink

I18N with quotation marks

Hi!

I wanted to find out how other people are handling this problem.

I am localising our app, which consists of strings in html and in
dynamic javascript snippets. However, if the translated value contains
quotations (such as: s'il vous plait), then it could break the HTML:

<select value='[% c.loc("Please select one") %]'>

or the javascript:

alert('[% c.loc("Please select one") %]');

We also sometimes use double quotes for attributes instead of single
quotes.

What is the best practise? Always run c.loc() through a filter to
convert to HTML entities? (Although in FF3.0 alert('Impossible
d'exécuter snmpget pour tester la connexion'); does not give the
single quote).

I was considering creating methods of c.hloc() (for a html
environment) and c.jloc() (for a javascript environment), but then the
xgettext.pl helper does not look for these method names.

Ton

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

moseley at hank

Jul 1, 2009, 10:55 AM

Post #2 of 11 (837 views)
Permalink

Re: I18N with quotation marks [In reply to]

On Wed, Jul 1, 2009 at 10:22 AM, Ton Voon <ton.voon[at]opsera.com> wrote:

> Hi!
>
> I wanted to find out how other people are handling this problem.
>
> I am localising our app, which consists of strings in html and in dynamic
> javascript snippets. However, if the translated value contains quotations
> (such as: s'il vous plait), then it could break the HTML:
>
> <select value='[% c.loc("Please select one") %]'>

I would think the correct approach would be a filter:

<select value='[% c.loc("Please select one") | html %]'>

but the default html filter only escapes double quotes.

--
Bill Moseley
moseley[at]hank.org

christian at lackas

Jul 1, 2009, 11:47 AM

Post #3 of 11 (835 views)
Permalink

Re: I18N with quotation marks [In reply to]

* Bill Moseley <moseley[at]hank.org> [090701 19:58]:

Hi Ton,

> On Wed, Jul 1, 2009 at 10:22 AM, Ton Voon <ton.voon[at]opsera.com> wrote:
> > I am localising our app, which consists of strings in html and in dynamic
> > javascript snippets. However, if the translated value contains quotations
> > (such as: s'il vous plait), then it could break the HTML:
> > <select value='[% c.loc("Please select one") %]'>
> I would think the correct approach would be a filter:
> <select value='[% c.loc("Please select one") | html %]'>

you can try the 'xml' filter?

http://template-toolkit.org/docs/manual/Filters.html#section_xml
Same as the html filter, but adds ' which is the fifth XML
built-in entity.

or html_entity (have not used that yet). Above manual page also contains
all other built-in filters. And you can also add your own:

http://template-toolkit.org/docs/modules/Template/Filters.html

Christian

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

larryl at emailplus

Jul 1, 2009, 11:51 AM

Post #4 of 11 (837 views)
Permalink

Re: I18N with quotation marks [In reply to]

Hi Ton -

> However, if the translated value contains quotations (such as: s'il
> vous plait), then it could break the HTML:
>
> <select value='[% c.loc("Please select one") %]'>
>
> or the javascript:
>
> alert('[% c.loc("Please select one") %]');

We create some custom scalar ops in a subclass of Catalyst::View::TT
(code below) that let you do:

<select value="[% c.loc("Please select one").escape_dq %]">

alert('[% c.loc("Please select one").escape_sq %]');

HTH,
Larry

--------------------------------------------

package Platform::View::TT;

# This is "MyApp/View/TT.pm":

use strict;
use warnings;

use base 'My::Catalyst::View::TT';

1;

--------------------------------------------

package My::Catalyst::View::TT;

use strict;
use warnings;

use base 'Catalyst::View::TT';

$Template::Stash::SCALAR_OPS->{escape_q} = sub {
my $s = shift;
$s =~ s/\\/\\\\/g;
$s =~ s/"/\\"/g;
$s =~ s/'/\\'/g;
return $s;
};

$Template::Stash::SCALAR_OPS->{escape_dq} = sub {
my $s = shift;
$s =~ s/\\/\\\\/g;
$s =~ s/"/\\"/g;
return $s;
};

$Template::Stash::SCALAR_OPS->{escape_sq} = sub {
my $s = shift;
$s =~ s/\\/\\\\/g;
$s =~ s/'/\\'/g;
return $s;
};

1;

--------------------------------------------

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

larryl at emailplus

Jul 1, 2009, 11:57 AM

Post #5 of 11 (836 views)
Permalink

Re: I18N with quotation marks [In reply to]

> We create some custom scalar ops in a subclass of Catalyst::View::TT
> (code below) that let you do:
>
> <select value="[% c.loc("Please select one").escape_dq %]">

Actually, escape_dq won't work here:

<select value="[% c.loc("Please select one").escape_dq %]">

but it will work if for some reason you have a double-quote delimited
JavaScript string.
Will probably need to convert the double quotes to %22 instead.
Sorry...

> alert('[% c.loc("Please select one").escape_sq %]');

This one does work though...

Larry

> --------------------------------------------
>
> package Platform::View::TT;
>
> # This is "MyApp/View/TT.pm":
>
> use strict;
> use warnings;
>
> use base 'My::Catalyst::View::TT';
>
> 1;
>
> --------------------------------------------
>
> package My::Catalyst::View::TT;
>
> use strict;
> use warnings;
>
> use base 'Catalyst::View::TT';
>
> $Template::Stash::SCALAR_OPS->{escape_q} = sub {
> my $s = shift;
> $s =~ s/\\/\\\\/g;
> $s =~ s/"/\\"/g;
> $s =~ s/'/\\'/g;
> return $s;
> };
>
> $Template::Stash::SCALAR_OPS->{escape_dq} = sub {
> my $s = shift;
> $s =~ s/\\/\\\\/g;
> $s =~ s/"/\\"/g;
> return $s;
> };
>
> $Template::Stash::SCALAR_OPS->{escape_sq} = sub {
> my $s = shift;
> $s =~ s/\\/\\\\/g;
> $s =~ s/'/\\'/g;
> return $s;
> };
>
> 1;
>
> --------------------------------------------
>
> _______________________________________________
> List: Catalyst[at]lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive:
> http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

ton.voon at opsera

Jul 2, 2009, 1:35 AM

Post #6 of 11 (810 views)
Permalink

Re: I18N with quotation marks [In reply to]

On 1 Jul 2009, at 18:22, Ton Voon wrote:

> I am localising our app, which consists of strings in html and in
> dynamic javascript snippets. However, if the translated value
> contains quotations (such as: s'il vous plait), then it could break
> the HTML:
>
> <select value='[% c.loc("Please select one") %]'>
>
> or the javascript:
>
> alert('[% c.loc("Please select one") %]');
>
> We also sometimes use double quotes for attributes instead of single
> quotes.
>
> What is the best practise? Always run c.loc() through a filter to
> convert to HTML entities? (Although in FF3.0 alert('Impossible
> d'exécuter snmpget pour tester la connexion'); does not give the
> single quote).
>
> I was considering creating methods of c.hloc() (for a html
> environment) and c.jloc() (for a javascript environment), but then
> the xgettext.pl helper does not look for these method names.

Hi!

Thanks for all the responses.

I think I now realise it depends on the context of the output so,
given that the translated string is "as-is" (without any markup or
html elements), then some filtering is required based on where the
translated value belongs.

This is my current thinking:

For HTML text, you should pass through the html filter, eg:

<p>[.% c.loc("Some text that might have < or > in it") | html %]</p>

For HTML elements, you should use double quotes for quoting attributes
and then pass the string through the html filter, eg,

<select value="[.% c.loc("May have some single or double quotes in") |
html %]">

For javascript in <script> blocks, you should use single quotes for
the string value and pass through an escape_js filter, eg:

<script>
var string = '[.% c.loc("May have single quotes or \ in it") |
escape_js %]';
</script>

For javascript in HTML elements, you should use double quotes for
quoting the attributes and single quotes for the javascript strings
and pass through the escape_js filter and the html filter, eg:

<select onclick=" alert('[.% c.loc("May have all sorts of things in
it") | escape_js | html %]') ">

The escape_js filter is defined as (From Larry Leszcznski's example):

$Template::Stash::SCALAR_OPS->{escape_js} = sub {
my $s = shift;
$s =~ s/\\/\\\\/g;
$s =~ s/'/\\'/g;
return $s;
};

Does everyone agree this makes sense? If so, any objections if I add
this to http://dev.catalystframework.org/wiki/best_practices?

Ton

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

jlmartinez-lists-catalyst at capside

Jul 3, 2009, 3:27 AM

Post #7 of 11 (793 views)
Permalink

Re: I18N with quotation marks [In reply to]

Ton Voon escribió:
> For javascript in <script> blocks, you should use single quotes for the
> string value and pass through an escape_js filter, eg:
>
> <script>
> var string = '[.% c.loc("May have single quotes or \ in it") | escape_js
> %]';
> </script>

Instead of forcing yourself to use single quoted strings in javascript,
you can escape single quotes AND double quotes :)

<script>
alert('I\'m a string with \\ and lots of \"things\"');
alert("I\'m a string with \\ and lots of \"things\"");
</script>

return the same output.

And to make it more solid...

You would expect that:

<script>
alert('I\'m a </script> string');
</script>

would show you a nice alert. You're wrong :) At least FF3 and IE fail. I
suppose that it's very normal (because the browser's parser understands
nothing about the string context of the javascript, and thinks the
<script> tag ends just in the middle of your script.

The solution is as easy as to "hide" the script tag from the parser.
<script>
alert('I\'m a <\/script> string');
</script>

Note: I don't know if it's better to escape all "/", or all "</" or just
"</script>" instances in the string. Any thoughts?

> $Template::Stash::SCALAR_OPS->{escape_js} = sub {
> my $s = shift;
> $s =~ s/\\/\\\\/g;
> $s =~ s/'/\\'/g;
> return $s;
> };

Maybe it's more efficient to do this in one pass?
$s =~ s/(\\|'|"|\/)/\\$1/g;

Just my 2 cents,

Jose Luis Martinez
jlmartinez[at]capside.com

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

ton.voon at opsera

Jul 8, 2009, 6:59 AM

Post #8 of 11 (692 views)
Permalink

Re: I18N with quotation marks [In reply to]

On 2 Jul 2009, at 09:35, Ton Voon wrote:

> Does everyone agree this makes sense? If so, any objections if I add
> this to http://dev.catalystframework.org/wiki/best_practices?

I've added a section at http://dev.catalystframework.org/wiki/best_practices
now.

This includes Jose Luis Martinez's suggestions for the
escape_js_string routine, implemented as a TT filter.

Final question: How do you internationalise a bit of text that does
want some markup within it? For instance, I want something that outputs:

Click <a href="/about">here</a> for the about page.

If I do something like:

link = '<a href="/about">' _ c.loc("here") _ '</a>';
c.loc("Click %1 for the about page", link)

Then I cannot filter through html for the 2nd loc output.

Is there a nicer way?

Ton

gunnarstrand at yahoo

Jul 8, 2009, 8:28 AM

Post #9 of 11 (689 views)
Permalink

Re: I18N with quotation marks [In reply to]

Ton Voon skrev:
>
> On 2 Jul 2009, at 09:35, Ton Voon wrote:
>
>> Does everyone agree this makes sense? If so, any objections if I add
>> this to http://dev.catalystframework.org/wiki/best_practices?
>
> I've added a section at
> http://dev.catalystframework.org/wiki/best_practices now.
>
> This includes Jose Luis Martinez's suggestions for the
> escape_js_string routine, implemented as a TT filter.
>
> Final question: How do you internationalise a bit of text that does
> want some markup within it? For instance, I want something that outputs:
>
> Click <a href="/about">here</a> for the about page.
>
> If I do something like:
>
> link = '<a href="/about">' _ c.loc("here") _ '</a>';
> c.loc("Click %1 for the about page", link)
>
> Then I cannot filter through html for the 2nd loc output.
>
> Is there a nicer way?
Wouldn't you need to send every part of the text through c.loc
individually? I guess something like this:

c.loc("Click ") _ link _ c.loc(" for the about page")

Not so nice maybe, but I guess it should work?

KR,
Gunnar
>
> Ton

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

cosimo at streppone

Jul 8, 2009, 8:58 AM

Post #10 of 11 (692 views)
Permalink

Re: I18N with quotation marks [In reply to]

Gunnar Strand <gunnarstrand[at]yahoo.com> wrote:

> Ton Voon skrev:
>>
>> Final question: How do you internationalise a bit of text that does
>> want some markup within it? For instance, I want something that outputs:
>>
>> Click <a href="/about">here</a> for the about page.
>
> Wouldn't you need to send every part of the text through c.loc
> individually? I guess something like this:
>
> c.loc("Click ") _ link _ c.loc(" for the about page")

If you collect all your i18n messages into .po files,
that are worked on by translators, they have little or no
context information, so they are going to have a really
hard time figuring out the sense of words.

We internationalized a dynamic non-catalyst website in
19 languages now, and we found the following, in gettext notation,
to be the best for front-end and back-end developers
and translators.

_('Click <a href="[_1]">here</a> for the about page')

So,

1) Keep the markup. It's ugly, but to us it's slightly better than
the alternatives.

2) Variables as variables. This also conserves strings
if/when you're changing URLs or variable content.
Example:

"You can upload up to [_1] Mb of pictures."

As the string embeds the variable, your string won't need
to be translated again if you change your upload limit.

3) Don't break sentences. In general, the longer, the better.
In general.

--
Cosimo

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

gunnarstrand at yahoo

Jul 8, 2009, 10:32 PM

Post #11 of 11 (683 views)
Permalink

Re: I18N with quotation marks [In reply to]

Cosimo Streppone skrev:
> Gunnar Strand <gunnarstrand[at]yahoo.com> wrote:
>
>> Ton Voon skrev:
>>>
>>> Final question: How do you internationalise a bit of text that does
>>> want some markup within it? For instance, I want something that
>>> outputs:
>>>
>>> Click <a href="/about">here</a> for the about page.
>>
>> Wouldn't you need to send every part of the text through c.loc
>> individually? I guess something like this:
>>
>> c.loc("Click ") _ link _ c.loc(" for the about page")
>
> If you collect all your i18n messages into .po files,
> that are worked on by translators, they have little or no
> context information, so they are going to have a really
> hard time figuring out the sense of words.
>
> We internationalized a dynamic non-catalyst website in
> 19 languages now, and we found the following, in gettext notation,
> to be the best for front-end and back-end developers
> and translators.
>
> _('Click <a href="[_1]">here</a> for the about page')
>
> So,
>
> 1) Keep the markup. It's ugly, but to us it's slightly better than
> the alternatives.
>
> 2) Variables as variables. This also conserves strings
> if/when you're changing URLs or variable content.
> Example:
>
> "You can upload up to [_1] Mb of pictures."
>
> As the string embeds the variable, your string won't need
> to be translated again if you change your upload limit.
>
> 3) Don't break sentences. In general, the longer, the better.
> In general.
I agree completely, and I may have misunderstood the question, but I
interpreted it that the I18Ned text would be put through an HTML filter.
In that case the above example of having markup inside the text sent to
the localizer will not work.

I personally prefer to use library routines for creating html markup
instead of hard coding data. I am using Maketext for my project (I'm not
very experienced yet), but it seems that an alternative would be to
extend the language file with HTML support in it to produce
corresponding output:

_('Click [link,_1,here] for the about page')

This would be prettier (IMHO) and allows the rendering mechanism for
links to be put in one place.

KR,
Gunnar

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com