Juergen.Herrmann at XLhost
Jan 8, 2009, 2:28 AM
Post #4 of 4
(1134 views)
Permalink
|
|
Re: automagic bome header at start of utf16 content?
[In reply to]
|
|
On Thu, January 8, 2009 11:04, Andreas Jung wrote:
> On 08.01.2009 10:33 Uhr, Jürgen Herrmann wrote:
>> i already sent the request directly to the zope server
>> omitting our apache proxy and monitored traffic with wireshark. the
>> com header comes from zope. i did not find anything in zope's code
>> that heuristically finds out this is utf16 content and prepends the
>> BOM header. so i'm a bit confused where zope takes it's wisdom from :)
>> anybody?
>
> I can not remember having seen any kind of code with the Zope core
> setting the BOM. We have code in the pagetemplate implementation
> interpreting a BOM but I have doubt that Zope sends a BOM out by itself
> (especially not for utf-16).
>
> Andreas
i wrote a small python script to check this out, content:
request = container.REQUEST
RESPONSE = request.RESPONSE
RESPONSE.setHeader('Content-Type', 'x-bom-test')
RESPONSE.setHeader('Content-Disposition', 'attachment; filename=bom_test.dat')
ustring = u'sgh sdgh\ns\xf6\xe4\xe4gddp\xe4s\n\u8a0a\u4ee5\u53ca\u76f8\u95dc\u7db2\u7d61\u670d\u52d9'
return ustring.encode('utf16')
here's what wireshark captured:
0000 00 16 17 1e 26 c6 00 1d 09 b8 cf cb 08 00 45 00 ....&... ......E.
0010 01 46 93 51 40 00 40 06 21 f6 c0 a8 01 79 c0 a8 .F.Q@.@. !....y..
0020 01 a1 1f 91 0d 2f a9 b9 5b 33 55 8e 5f 1f 50 18 ...../.. [.3U._.P.
0030 1d 50 d1 0b 00 00 48 54 54 50 2f 31 2e 31 20 32 .P....HT TP/1.1 2
0040 30 30 20 4f 4b 0d 0a 53 65 72 76 65 72 3a 20 5a 00 OK..S erver: Z
0050 6f 70 65 2f 28 5a 6f 70 65 20 32 2e 31 30 2e 35 ope/(Zop e 2.10.5
0060 2d 66 69 6e 61 6c 2c 20 70 79 74 68 6f 6e 20 32 -final, python 2
0070 2e 34 2e 34 2c 20 6c 69 6e 75 78 32 29 20 5a 53 .4.4, li nux2) ZS
0080 65 72 76 65 72 2f 31 2e 31 0d 0a 44 61 74 65 3a erver/1. 1..Date:
0090 20 54 68 75 2c 20 30 38 20 4a 61 6e 20 32 30 30 Thu, 08 Jan 200
00a0 39 20 31 30 3a 32 30 3a 34 35 20 47 4d 54 0d 0a 9 10:20: 45 GMT..
00b0 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 68 3a 20 Content- Length:
00c0 36 30 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 60..Cont ent-Type
00d0 3a 20 78 2d 62 6f 6d 2d 74 65 73 74 0d 0a 43 6f : x-bom- test..Co
00e0 6e 74 65 6e 74 2d 44 69 73 70 6f 73 69 74 69 6f ntent-Di spositio
00f0 6e 3a 20 61 74 74 61 63 68 6d 65 6e 74 3b 20 66 n: attac hment; f
0100 69 6c 65 6e 61 6d 65 3d 62 6f 6d 5f 74 65 73 74 ilename= bom_test
0110 2e 64 61 74 0d 0a 0d 0a ff fe 73 00 67 00 68 00 .dat.... ..s.g.h.
0120 20 00 73 00 64 00 67 00 68 00 0a 00 73 00 f6 00 .s.d.g. h...s...
0130 e4 00 e4 00 67 00 64 00 64 00 70 00 e4 00 73 00 ....g.d. d.p...s.
0140 0a 00 0a 8a e5 4e ca 53 f8 76 dc 95 b2 7d 61 7d .....N.S .v...}a}
0150 0d 67 d9 52 .g.R
look at offset 0x0119...
ok, time to look at repr(ustring.encode('utf16')):
'\xff\xfes\x00g\x00h\x00 \x00s\x00d\x00g\x00h\x00\n\x00s\x00\xf6\x00\xe4\x00'\
'\xe4\x00g\x00d\x00d\x00p\x00\xe4\x00s\x00\n\x00\n\x8a\xe5N\xcaS\xf8v\xdc\x95'\
'\xb2}a}\rg\xd9R'
bam!
i din't exepct that encoding in utf16 would add a bom header by itself...
sorry for posting so lenghty, thought that it might be interesting for
people having to deal with utf16...
best regards,
jürgen herrmann
--
>> XLhost.de - eXperts in Linux hosting ® <<
XLhost.de GmbH
Jürgen Herrmann, Geschäftsführer
Boelckestrasse 21, 93051 Regensburg, Germany
Geschäftsführer: Volker Geith, Jürgen Herrmann
Registriert unter: HRB9918
Umsatzsteuer-Identifikationsnummer: DE245931218
Fon: +49 (0)700 XLHOSTDE [0700 95467833]
Fax: +49 (0)700 XLHOSTDE [0700 95467833]
WEB: http://www.XLhost.de
IRC: #XLhost [at] irc
_______________________________________________
Zope maillist - Zope [at] zope
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
|
lists.vcf
