Mailing List Archive: ModPerl: ASP

saving decoded unicode value to Session causes error

Index | Next | Previous | View Threaded

dba at richyen

Aug 28, 2007, 4:19 PM

Post #1 of 4 (3741 views)
Permalink

saving decoded unicode value to Session causes error

Hi,

Wondering if anyone has ever encountered this issue.

Trying to internationalize my site, so at every input, I pass it
through Encode::decode_utf8.

However, when I attempt to put the values into Session, I get the
following message:
> Errors Output
> Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
> 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, <GEN10> line 21. , /usr/
> local/perl/lib/site_perl/5.8.7/Apache/ASP.pm line 1521
> Debug Output
> Use of uninitialized value in string ne at ../global/lib/
> lib_filesystem.pm line 41.
> Using an array as a reference is deprecated at ../global/lib/
> lib_peer_review.pm line 389, line 21.
> Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
> 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, line 21. , /usr/local/perl/
> lib/site_perl/5.8.7/Apache/ASP.pm line 1521

An example is if I use the capital Omega character (option-z on mac
keyboard), this error occurs when attempting to store into Session:

> my $secret_answer = Encode::decode_utf8($main::Request->Form
> ('secret_answer'));
> $main::Session->{secret_answer} = $secret_answer;

I've discovered that if I put the decoded value into a hash, and then
put the hash into Session, it behaves normally. Also, if I don't
decode at all, it behaves normally also.

Would anyone know why this happens, or how to fix it?

Thanks!
--Richard

---------------------------------------------------------------------
To unsubscribe, e-mail: asp-unsubscribe[at]perl.apache.org
For additional commands, e-mail: asp-help[at]perl.apache.org

tsirkin at gmail

Aug 29, 2007, 12:28 AM

Post #2 of 4 (3585 views)
Permalink

Re: saving decoded unicode value to Session causes error [In reply to]

It looks like that SDBM_File file handler or something else in it's
internals does not
know that you are storing (writing) utf8 records into it.
I don't know how to force it to know this :( maybe some SDBM_File expert
here
will know.
However the workaround for this is to NOT mark the string you get as utf8
i.e.
don't do Encode::decode_utf8. Just set your pages/server to send to the
browser
header saying that you need utf8 as request charset and you will be fine.

<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
will do this or
$Response->AddHeader()

What this solution does NOT do - it does not validate the input to be legal
utf8.
But most of the time you will be fine.
Hope this helps.
Evgeny
**
On 8/29/07, Richard Yen <dba[at]richyen.com> wrote:
>
> Hi,
>
> Wondering if anyone has ever encountered this issue.
>
> Trying to internationalize my site, so at every input, I pass it
> through Encode::decode_utf8.
>
> However, when I attempt to put the values into Session, I get the
> following message:
> > Errors Output
> > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
> > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, <GEN10> line 21. , /usr/
> > local/perl/lib/site_perl/5.8.7/Apache/ASP.pm line 1521
> > Debug Output
> > Use of uninitialized value in string ne at ../global/lib/
> > lib_filesystem.pm line 41.
> > Using an array as a reference is deprecated at ../global/lib/
> > lib_peer_review.pm line 389, line 21.
> > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
> > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, line 21. , /usr/local/perl/
> > lib/site_perl/5.8.7/Apache/ASP.pm line 1521
>
> An example is if I use the capital Omega character (option-z on mac
> keyboard), this error occurs when attempting to store into Session:
>
> > my $secret_answer = Encode::decode_utf8($main::Request->Form
> > ('secret_answer'));
> > $main::Session->{secret_answer} = $secret_answer;
>
> I've discovered that if I put the decoded value into a hash, and then
> put the hash into Session, it behaves normally. Also, if I don't
> decode at all, it behaves normally also.
>
> Would anyone know why this happens, or how to fix it?
>
> Thanks!
> --Richard
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: asp-unsubscribe[at]perl.apache.org
> For additional commands, e-mail: asp-help[at]perl.apache.org
>
>

dba at richyen

Aug 29, 2007, 8:20 AM

Post #3 of 4 (3594 views)
Permalink

Re: saving decoded unicode value to Session causes error [In reply to]

On Aug 29, 2007, at 12:28 AM, Tsirkin Evgeny wrote:

> It looks like that SDBM_File file handler or something else in it's
> internals does not
> know that you are storing (writing) utf8 records into it.
> I don't know how to force it to know this :( maybe some SDBM_File
> expert here
> will know.
Hmm, I thought that perhaps it was something with Apache::ASP, and
that SDBM_File would be a dependency that was being passed bad data?

> However the workaround for this is to NOT mark the string you get
> as utf8 i.e.
> don't do Encode::decode_utf8. Just set your pages/server to send to
> the browser
> header saying that you need utf8 as request charset and you will be
> fine.
>
> <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
> will do this or
> $Response->AddHeader()
I suppose I misunderstood the purpose of this <meta> tag...so not
only output, but inputs get encoded as UTF-8 here? Cool stuff....
When, then, would I need to use decode_utf8? I suppose it's for file-
reading or command-line input then?

> What this solution does NOT do - it does not validate the input to
> be legal utf8.
True. Will keep that in mind.

> But most of the time you will be fine.
> Hope this helps.
> Evgeny
Thanks!
--Richard

>
>
> On 8/29/07, Richard Yen <dba[at]richyen.com> wrote: Hi,
>
> Wondering if anyone has ever encountered this issue.
>
> Trying to internationalize my site, so at every input, I pass it
> through Encode::decode_utf8.
>
> However, when I attempt to put the values into Session, I get the
> following message:
> > Errors Output
> > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
> > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, <GEN10> line 21. , /usr/
> > local/perl/lib/site_perl/5.8.7/Apache/ASP.pm line 1521
> > Debug Output
> > Use of uninitialized value in string ne at ../global/lib/
> > lib_filesystem.pm line 41.
> > Using an array as a reference is deprecated at ../global/lib/
> > lib_peer_review.pm line 389, line 21.
> > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
> > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, line 21. , /usr/local/perl/
> > lib/site_perl/5.8.7/Apache/ASP.pm line 1521
>
> An example is if I use the capital Omega character (option-z on mac
> keyboard), this error occurs when attempting to store into Session:
>
> > my $secret_answer = Encode::decode_utf8($main::Request->Form
> > ('secret_answer'));
> > $main::Session->{secret_answer} = $secret_answer;
>
> I've discovered that if I put the decoded value into a hash, and then
> put the hash into Session, it behaves normally. Also, if I don't
> decode at all, it behaves normally also.
>
> Would anyone know why this happens, or how to fix it?
>
> Thanks!
> --Richard
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: asp-unsubscribe[at]perl.apache.org
> For additional commands, e-mail: asp-help[at]perl.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: asp-unsubscribe[at]perl.apache.org
For additional commands, e-mail: asp-help[at]perl.apache.org

tsirkin at gmail

Aug 30, 2007, 12:46 AM

Post #4 of 4 (3586 views)
Permalink

Re: saving decoded unicode value to Session causes error [In reply to]

Hi Richard!
First of all i need to apologize:
It is possible that i gave you a bad advice.
I am not a Apache::ASP developer, just a user that is monitoring the list
to follow the development (if any is making place ,i hope :) ) and
I have tried to help .As NOT a developer i hope it is allowed for me to
make
mistakes.

I will try to explain :
If you setup your server and/or application to send the header saying that
the
data is utf8 encoded then what you get from client will also be utf8
encoded.
That IS right .So,if you just use
$Request->Form()
to get the data posted you will get the utf8 encoded data and you probably
WILL be fine ,the same as i have said in the prev. reply.
However there IS a problem:
The string you will get from $Request object although utf8 encoded will NOT
be
marked as utf8 encoded .In other words - you get the data fine and if you
for example
store it in DBMS you are fine,but perl does not know that it is utf8
encoded.For
perl the data is just an array of bytes
Why does it matter for perl to mark strings as utf8?If you try to use
regular expr.
or some string functions (like length ,i think) it may not work because perl
will
assume to be character 8 bit long which not always true for utf8.
So ,my advise of just NOT using Encode::decode_utf8 may cause actually
problems
if you try to do some processing on the data you get.
Actually, i personally do exactly what i have advised to you :
Taking the data just like i get it from $Request and store it.
In the rare cases when i need to do some processing one the localized
data (like regular expressions match ) i am marking it as utf8.

> Hmm, I thought that perhaps it was something with Apache::ASP, and
> that SDBM_File would be a dependency that was being passed bad data?

I have made a test and it worked just fine,here is the code:
<html>
<head>
</head>
<body>
<form action="" method=post>
<input type=text name=v>
<input type=submit>
</form>
</body>
</html>
<%

use Encode;
my $v=Encode::decode_utf8($Request->Form('v'));
$Session->{v}=$v;
print $Session->{v};
I am also using SDBM_File .Thought i am obviously have another version of
perl :(
Why i think that this an SDBM issue?
The error you are getting :
"Wide character in subroutine entry"
means that you are trying to output/call function with wide utf8 character
while this file handler is not setup to get this kind of data .
A quick google search for "MLDBM utf8" approves this. :(
So i guess you need a workaround or just try to use another DBM.

> I suppose I misunderstood the purpose of this <meta> tag...so not
> only output, but inputs get encoded as UTF-8 here? Cool stuff....

Sometimes the meta tag get ignored and you need the header,also make sure
your
server does not send such a header by his own (Apache sometimes does).

> When, then, would I need to use decode_utf8? I suppose it's for file-
> reading or command-line input then?

If you need perl to know it is utf8 ,such as some of string manipulations.
Again,You may also try another DBM ,change your StateDB setting .
Maybe this is the best solution .
Evgeny.

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com