Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Zope: Dev

XML-RPC does not work under the paster process

 

 

Zope dev RSS feed   Index | Next | Previous | View Threaded


michael at d2m

Mar 15, 2009, 7:29 AM

Post #1 of 7 (1054 views)
Permalink
XML-RPC does not work under the paster process

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

When using paste.httpserver instead of twisted.wsgi server
zope.publisher.xmlrpc.XMLRPCRequest.processInputs() hangs when reading 0
Bytes from the request (wsgi.input <socket._fileobject object>).

As i found the zope.publisher.http.HTTPInputStream.readlines() signature
was changed 3 years ago
from readlines(self, hint=None)
to readlines(self, hint=0)
http://svn.zope.org/Zope3/trunk/src/zope/publisher/http.py?rev=66941&r1=66940&r2=66941

Do you think we could revert the change?

Bugs reported:
zopeproject: https://bugs.launchpad.net/zope3/+bug/283089
grokproject: https://bugs.launchpad.net/grok/+bug/332063

Regards
Michael

- --
http://blog.d2m.at
http://planetzope.org

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJvRDVl0uAvQJUKVYRAuMZAJ0eZBKjoInmx5CigwgjUf2gDpQyXQCfTBh5
IVbnn5i1v5WAwGPJxUJWCZs=
=+cQ+
-----END PGP SIGNATURE-----

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


michael at d2m

Mar 18, 2009, 5:12 AM

Post #2 of 7 (989 views)
Permalink
Re: XML-RPC does not work under the paster process [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Michael Haubenwallner wrote:
> When using paste.httpserver instead of twisted.wsgi server
> zope.publisher.xmlrpc.XMLRPCRequest.processInputs() hangs when reading 0
> Bytes from the request (wsgi.input <socket._fileobject object>).
>
> As i found the zope.publisher.http.HTTPInputStream.readlines() signature
> was changed 3 years ago
> from readlines(self, hint=None)
> to readlines(self, hint=0)
> http://svn.zope.org/Zope3/trunk/src/zope/publisher/http.py?rev=66941&r1=66940&r2=66941
>
> Do you think we could revert the change?
>
> Bugs reported:
> zopeproject: https://bugs.launchpad.net/zope3/+bug/283089
> grokproject: https://bugs.launchpad.net/grok/+bug/332063
>

Bump - maybe my description was too complicated, but the issue is serious:

XMLRPC is broken at least with paste.httpserver

Any input on the implications of reverting the change in revision #66941?

Regards
Michael

- --
http://blog.d2m.at
http://planetzope.org

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJwOU0l0uAvQJUKVYRAnILAKCOSnA3YfUXilqoTD74/ukemGPkyACgil0O
1F0ZmR11vhodC33S6Wcg0z0=
=JX17
-----END PGP SIGNATURE-----

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


dev at projekt01

Mar 18, 2009, 5:18 AM

Post #3 of 7 (999 views)
Permalink
Re: XML-RPC does not work under the paster process [In reply to]

Hi Michael

> Betreff: Re: [Zope-dev] XML-RPC does not work under the paster process
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Michael Haubenwallner wrote:
> > When using paste.httpserver instead of twisted.wsgi server
> > zope.publisher.xmlrpc.XMLRPCRequest.processInputs() hangs
> when reading
> > 0 Bytes from the request (wsgi.input <socket._fileobject object>).
> >
> > As i found the zope.publisher.http.HTTPInputStream.readlines()
> > signature was changed 3 years ago from readlines(self, hint=None)
> > to readlines(self, hint=0)
> >
> http://svn.zope.org/Zope3/trunk/src/zope/publisher/http.py?rev=66941&r
> > 1=66940&r2=66941
> >
> > Do you think we could revert the change?
> >
> > Bugs reported:
> > zopeproject: https://bugs.launchpad.net/zope3/+bug/283089
> > grokproject: https://bugs.launchpad.net/grok/+bug/332063
> >
>
> Bump - maybe my description was too complicated, but the
> issue is serious:
>
> XMLRPC is broken at least with paste.httpserver
>
> Any input on the implications of reverting the change in
> revision #66941?

We should also add a good test case where we can see what
happens.

Regards
Roger Ineichen

> Regards
> Michael
>
> - --
> http://blog.d2m.at
> http://planetzope.org
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFJwOU0l0uAvQJUKVYRAnILAKCOSnA3YfUXilqoTD74/ukemGPkyACgil0O
> 1F0ZmR11vhodC33S6Wcg0z0=
> =JX17
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Zope-Dev maillist - Zope-Dev [at] zope
> http://mail.zope.org/mailman/listinfo/zope-dev
> ** No cross posts or HTML encoding! ** (Related lists -
> http://mail.zope.org/mailman/listinfo/zope-announce
> http://mail.zope.org/mailman/listinfo/zope )
>

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


ct at gocept

Mar 21, 2009, 3:07 AM

Post #4 of 7 (962 views)
Permalink
Re: XML-RPC does not work under the paster process [In reply to]

On Wed, 2009-03-18 at 13:12 +0100, Michael Haubenwallner wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Michael Haubenwallner wrote:
> > When using paste.httpserver instead of twisted.wsgi server
> > zope.publisher.xmlrpc.XMLRPCRequest.processInputs() hangs when reading 0
> > Bytes from the request (wsgi.input <socket._fileobject object>).
> >
> > As i found the zope.publisher.http.HTTPInputStream.readlines() signature
> > was changed 3 years ago
> > from readlines(self, hint=None)
> > to readlines(self, hint=0)
> > http://svn.zope.org/Zope3/trunk/src/zope/publisher/http.py?rev=66941&r1=66940&r2=66941
> >
> > Do you think we could revert the change?
> >
> > Bugs reported:
> > zopeproject: https://bugs.launchpad.net/zope3/+bug/283089
> > grokproject: https://bugs.launchpad.net/grok/+bug/332063
> >
>
> Bump - maybe my description was too complicated, but the issue is serious:
>
> XMLRPC is broken at least with paste.httpserver
>
> Any input on the implications of reverting the change in revision #66941?

I've looked at the various APIs of actual objects that need to consume
the size hint parameter (cStringIO, StringIO, file, socket._fileobject).

All of them should be fine getting a 0 as an argument, but file and
cStringIO will break if they get `None` instead of 0:

Type | readlines() | readlines(None) | readlines(0)
------------------------+-------------+-----------------+--------------
file | works | TypeError | works
StringIO | works | works | works
cStringIO | works | TypeError | works
socket._fileobject [1]_ | works | works | works

.. [1] I've only looked at the code and reasoned about this.

What I'm puzzled about is that the commit message of 66491 mentions
compatibility to StringIO but that would have worked just fine with
`None`. So this either actually means "compatibility with file" or
"compatibility with cStringIO".

Nevertheless: passing in 0 seems to be the safe bet in general and I
wonder what is actually breaking. I haven't seen a traceback be attached
at either bug. Also, those bugs should probably be marked as duplicates.

Christian

--
Christian Theune · ct [at] gocept
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1
Zope and Plone consulting and development
--
Christian Theune · ct [at] gocept
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1
Zope and Plone consulting and development
Attachments: signature.asc (0.19 KB)


michael at d2m

May 25, 2009, 8:08 AM

Post #5 of 7 (746 views)
Permalink
Re: XML-RPC does not work under the paster process [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Christian Theune wrote:
> On Wed, 2009-03-18 at 13:12 +0100, Michael Haubenwallner wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Michael Haubenwallner wrote:
>>> When using paste.httpserver instead of twisted.wsgi server
>>> zope.publisher.xmlrpc.XMLRPCRequest.processInputs() hangs when reading 0
>>> Bytes from the request (wsgi.input <socket._fileobject object>).
>>>
>>> As i found the zope.publisher.http.HTTPInputStream.readlines() signature
>>> was changed 3 years ago
>>> from readlines(self, hint=None)
>>> to readlines(self, hint=0)
>>> http://svn.zope.org/Zope3/trunk/src/zope/publisher/http.py?rev=66941&r1=66940&r2=66941
>>>
>>> Do you think we could revert the change?
>>>
>>> Bugs reported:
>>> zopeproject: https://bugs.launchpad.net/zope3/+bug/283089
>>> grokproject: https://bugs.launchpad.net/grok/+bug/332063
>>>
>> Bump - maybe my description was too complicated, but the issue is serious:
>>
>> XMLRPC is broken at least with paste.httpserver
>>
>> Any input on the implications of reverting the change in revision #66941?
>
> I've looked at the various APIs of actual objects that need to consume
> the size hint parameter (cStringIO, StringIO, file, socket._fileobject).
>
> All of them should be fine getting a 0 as an argument, but file and
> cStringIO will break if they get `None` instead of 0:
>
> Type | readlines() | readlines(None) | readlines(0)
> ------------------------+-------------+-----------------+--------------
> file | works | TypeError | works
> StringIO | works | works | works
> cStringIO | works | TypeError | works
> socket._fileobject [1]_ | works | works | works
>
> .. [1] I've only looked at the code and reasoned about this.
>
> What I'm puzzled about is that the commit message of 66491 mentions
> compatibility to StringIO but that would have worked just fine with
> `None`. So this either actually means "compatibility with file" or
> "compatibility with cStringIO".
>
> Nevertheless: passing in 0 seems to be the safe bet in general and I
> wonder what is actually breaking. I haven't seen a traceback be attached
> at either bug. Also, those bugs should probably be marked as duplicates.
>

You are right, looking at the problem again i suggest to add this change
to zope.publisher.xmlrpc

- --- src/zope/publisher/xmlrpc.py (revision 100356)
+++ src/zope/publisher/xmlrpc.py (working copy)
@@ -46,10 +46,14 @@
'See IPublisherRequest'
# Parse the request XML structure

- - # XXX using readlines() instead of lines()
- - # as twisted's BufferedStream sends back
- - # an empty stream here for read() (bug)
- - lines = ''.join(self._body_instream.readlines())
+ # XXX using readline() instead of readlines()
+ # as readlines() is not working with
+ # paster.httpserver
+ line = 1
+ lines = ''
+ while line != '':
+ line = self._body_instream.readline()
+ lines += line
self._args, function = xmlrpclib.loads(lines)


The fix was proposed by jens Klein, see
https://bugs.launchpad.net/zope3/+bug/283089/comments/3

If noone objects i will add it to the zope.publisher 3.4 branch
and make a release (3.4.8). The fix is needed at least for Grok to make
XMLRPC work with paster.httpserver.

Regards
Michael

- --
http://blog.d2m.at
http://planetzope.org

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKGrR4l0uAvQJUKVYRAh1hAKCJDDSM/MDeuuzOn9jddnZWsfI2IACgnWOi
lMnwK3vbJcAYu5miFKXjAH8=
=Fxqf
-----END PGP SIGNATURE-----

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


ct at gocept

May 25, 2009, 11:22 PM

Post #6 of 7 (728 views)
Permalink
Re: XML-RPC does not work under the paster process [In reply to]

On Mon, 2009-05-25 at 17:08 +0200, Michael Haubenwallner wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Christian Theune wrote:
> > [...]
> >
> > Nevertheless: passing in 0 seems to be the safe bet in general and I
> > wonder what is actually breaking. I haven't seen a traceback be attached
> > at either bug. Also, those bugs should probably be marked as duplicates.
> >
>
> You are right, looking at the problem again i suggest to add this change
> to zope.publisher.xmlrpc

Uhhhm. Why are we changing this? IMHO passing 0 should work and I don't
see what the actual breakage is.

> - --- src/zope/publisher/xmlrpc.py (revision 100356)
> +++ src/zope/publisher/xmlrpc.py (working copy)
> @@ -46,10 +46,14 @@
> 'See IPublisherRequest'
> # Parse the request XML structure
>
> - - # XXX using readlines() instead of lines()
> - - # as twisted's BufferedStream sends back
> - - # an empty stream here for read() (bug)
> - - lines = ''.join(self._body_instream.readlines())
> + # XXX using readline() instead of readlines()
> + # as readlines() is not working with
> + # paster.httpserver
> + line = 1
> + lines = ''
> + while line != '':
> + line = self._body_instream.readline()
> + lines += line
> self._args, function = xmlrpclib.loads(lines)

I'm fine with the patch. I just don't understand why we need it. :/

Here's a small readability suggestion:

lines = ''
while True:
line = self._body_instream.readline()
if not line:
break
lines += line

I think assigning assigning an unused value can lead to confusions and
the head-controlled loop obfuscates whats actually going on.

Cheers,
Christian

--
Christian Theune · ct [at] gocept
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1
Zope and Plone consulting and development
Attachments: signature.asc (0.19 KB)


michael at d2m

May 26, 2009, 12:05 AM

Post #7 of 7 (744 views)
Permalink
Re: XML-RPC does not work under the paster process [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Christian Theune wrote:
> On Mon, 2009-05-25 at 17:08 +0200, Michael Haubenwallner wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Christian Theune wrote:
>>> [...]
>>>
>>> Nevertheless: passing in 0 seems to be the safe bet in general and I
>>> wonder what is actually breaking. I haven't seen a traceback be attached
>>> at either bug. Also, those bugs should probably be marked as duplicates.
>>>
>> You are right, looking at the problem again i suggest to add this change
>> to zope.publisher.xmlrpc
>
> Uhhhm. Why are we changing this? IMHO passing 0 should work and I don't
> see what the actual breakage is.
>

There is another bugfix related to paster.httpserver in zope.publisher
3.5.3 (2008-06-20)

Bugs fixed:

* It turns out that some Web servers (Paste for example) do not send
the EOF character after the data has been transmitted and the read() of
the cached stream simply hangs if no expected content length has been
specified.

Could be the same reason as with XMLRPC

>> - --- src/zope/publisher/xmlrpc.py (revision 100356)
>> +++ src/zope/publisher/xmlrpc.py (working copy)
>> @@ -46,10 +46,14 @@
>> 'See IPublisherRequest'
>> # Parse the request XML structure
>>
>> - - # XXX using readlines() instead of lines()
>> - - # as twisted's BufferedStream sends back
>> - - # an empty stream here for read() (bug)
>> - - lines = ''.join(self._body_instream.readlines())
>> + # XXX using readline() instead of readlines()
>> + # as readlines() is not working with
>> + # paster.httpserver
>> + line = 1
>> + lines = ''
>> + while line != '':
>> + line = self._body_instream.readline()
>> + lines += line
>> self._args, function = xmlrpclib.loads(lines)
>
> I'm fine with the patch. I just don't understand why we need it. :/
>
> Here's a small readability suggestion:
>
> lines = ''
> while True:
> line = self._body_instream.readline()
> if not line:
> break
> lines += line
>
> I think assigning assigning an unused value can lead to confusions and
> the head-controlled loop obfuscates whats actually going on.
>

Thanks for your suggestion, will use it.

Regards
Michael

- --
http://blog.d2m.at
http://planetzope.org

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKG5S6l0uAvQJUKVYRAlajAJ4+TXIVNtFBzwN5F253rqfc2wqH0QCeJvCq
CMumYwCnLWCeCKNocCmkbp0=
=X7uK
-----END PGP SIGNATURE-----

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Zope dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.