Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Zope: Dev

Re: zope.publisher 3.5 branch has code/behavior not a part of subsequent releases

 

 

Zope dev RSS feed   Index | Next | Previous | View Threaded


gary.poster at gmail

Aug 24, 2009, 12:32 PM

Post #1 of 6 (753 views)
Permalink
Re: zope.publisher 3.5 branch has code/behavior not a part of subsequent releases

Hm. I sent this from the wrong account, so it didn't make it to the
zope-dev list. I'm also adding an additional bit of war story at the
end.


On Aug 24, 2009, at 11:16 AM, Gary Poster wrote:

> Hi Tres
>
> I made a 3.5.8 release of the zope.publisher 3.5 branch for a reason
> unimportant to this email (I backported a change to trunk that was
> discussed on the mailing list). It looks like you made a 3.5.7 with
> the following change (omitting tests and the like).
>
> 98932 tseaver # Python 2.6 notices QS-on-POST,
> which breaks BBB for us.
> 98932 tseaver # Suppress that.
> 98932 tseaver if 'QUERY_STRING' in self._environ:
> 98932 tseaver env = self._environ
> 98932 tseaver env['X-POST-QUERY_STRING'] =
> env.pop('QUERY_STRING')
>
> I can handle adjusting to this change, if it is appropriate, but my
> concern is that it is not in trunk or any subsequent major release
> (3.6, 3.7, 3.8) of zope.publisher. Therefore, if I change my code
> to use my updated 3.5 release, which I had hoped would be a
> conservative update, I have to change in a currently forward-
> incompatible way.
>
> What's the story here? Is 3.5.7 a brownbag that needs to have its
> changes aborted in a subsequent release in that branch? Or are
> those legitimate changes that need to be forward ported?
>
> FWIW, we (Launchpad) also care about this case of a POST with other
> pertinent, separate data in the query string, and the behavior you
> implement here would be fine if it is necessary for Py 2.6 as your
> comment describes.

Also FWIW, this change breaks many of our forms that were explicitly
constructing form actions that included the current query string. I'm
not sure how that could be avoided, although the fix might have been
simpler if there were always an X-ORIGINAL-QUERY_STRING or something
else.

If I were not already behind, I would investigate to understand the
Python 2.6 problem better and see what other frameworks are doing
here. I understand from conversations with other engineers that at
least some Django developers are accustomed to always having access to
the query string on the request object, whether the method were get or
post or whatever else.

>
> Gary

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


tseaver at palladion

Aug 24, 2009, 2:27 PM

Post #2 of 6 (706 views)
Permalink
Re: zope.publisher 3.5 branch has code/behavior not a part of subsequent releases [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Gary Poster wrote:
> Hm. I sent this from the wrong account, so it didn't make it to the
> zope-dev list. I'm also adding an additional bit of war story at the
> end.
>
>
> On Aug 24, 2009, at 11:16 AM, Gary Poster wrote:
>
>> Hi Tres
>>
>> I made a 3.5.8 release of the zope.publisher 3.5 branch for a reason
>> unimportant to this email (I backported a change to trunk that was
>> discussed on the mailing list). It looks like you made a 3.5.7 with
>> the following change (omitting tests and the like).
>>
>> 98932 tseaver # Python 2.6 notices QS-on-POST,
>> which breaks BBB for us.
>> 98932 tseaver # Suppress that.
>> 98932 tseaver if 'QUERY_STRING' in self._environ:
>> 98932 tseaver env = self._environ
>> 98932 tseaver env['X-POST-QUERY_STRING'] =
>> env.pop('QUERY_STRING')
>>
>> I can handle adjusting to this change, if it is appropriate, but my
>> concern is that it is not in trunk or any subsequent major release
>> (3.6, 3.7, 3.8) of zope.publisher. Therefore, if I change my code
>> to use my updated 3.5 release, which I had hoped would be a
>> conservative update, I have to change in a currently forward-
>> incompatible way.
>>
>> What's the story here? Is 3.5.7 a brownbag that needs to have its
>> changes aborted in a subsequent release in that branch? Or are
>> those legitimate changes that need to be forward ported?
>>
>> FWIW, we (Launchpad) also care about this case of a POST with other
>> pertinent, separate data in the query string, and the behavior you
>> implement here would be fine if it is necessary for Py 2.6 as your
>> comment describes.
>
> Also FWIW, this change breaks many of our forms that were explicitly
> constructing form actions that included the current query string. I'm
> not sure how that could be avoided, although the fix might have been
> simpler if there were always an X-ORIGINAL-QUERY_STRING or something
> else.
>
> If I were not already behind, I would investigate to understand the
> Python 2.6 problem better and see what other frameworks are doing
> here. I understand from conversations with other engineers that at
> least some Django developers are accustomed to always having access to
> the query string on the request object, whether the method were get or
> post or whatever else.

It is *essential* for correct operation that QUERY_STRING values *not*
be admixed with POSTed form values. I don't really care how we resolve
your issue, as long as we do not end up in a case where the values in
the query string get mingled into the form data: for instance, we could
hand a QUERY_STRING-free copy of the environment to the cgi.FieldStorage
machinery.

Whatever gets done needs to leave the existing test in place::

self.assertEqual(dict(request.form), dict(x='1', y='2'))

for a request whose QUERY_STRING was 'a=5&b:int=6', but which posted the
'x' and 'y' values.



Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tseaver [at] palladion
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKkwXA+gerLs4ltQ4RAhVKAKDY5LuTUshFf1ihKTQXpJjyxvIeeACglpu8
FZSMcnQjaiOyax9ziOAlFNE=
=qJS/
-----END PGP SIGNATURE-----
_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


dev at projekt01

Aug 24, 2009, 3:02 PM

Post #3 of 6 (704 views)
Permalink
Re: zope.publisher 3.5 branch has code/behavior not a part of subsequent releases [In reply to]

Hi Tres

> Betreff: Re: [Zope-dev] zope.publisher 3.5 branch has
> code/behavior not a part of subsequent releases

[...]

> > If I were not already behind, I would investigate to understand the
> > Python 2.6 problem better and see what other frameworks are doing
> > here. I understand from conversations with other engineers that at
> > least some Django developers are accustomed to always
> having access to
> > the query string on the request object, whether the method
> were get or
> > post or whatever else.
>
> It is *essential* for correct operation that QUERY_STRING
> values *not* be admixed with POSTed form values. I don't
> really care how we resolve your issue, as long as we do not
> end up in a case where the values in the query string get
> mingled into the form data: for instance, we could hand a
> QUERY_STRING-free copy of the environment to the
> cgi.FieldStorage machinery.

As far as I understand, you are saying that it is essential
that posted data and a query string should be separated
for processing in python libraries e.g. FieldStorage or so.

But this doesn't mean both values could not end in the
request.form dict right?

> Whatever gets done needs to leave the existing test in place::
>
> self.assertEqual(dict(request.form), dict(x='1', y='2'))
>
> for a request whose QUERY_STRING was 'a=5&b:int=6', but which
> posted the 'x' and 'y' values.

Was this supported before your changes? Is this a new feature
you decided to add? What's the reason for this? Can you point
me to more infos?

Regards
Roger Ineichen

> Tres.
> - --
> ===================================================================
> Tres Seaver +1 540-429-0999 tseaver [at] palladion
> Palladion Software "Excellence by Design" http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFKkwXA+gerLs4ltQ4RAhVKAKDY5LuTUshFf1ihKTQXpJjyxvIeeACglpu8
> FZSMcnQjaiOyax9ziOAlFNE=
> =qJS/
> -----END PGP SIGNATURE-----
> _______________________________________________
> Zope-Dev maillist - Zope-Dev [at] zope
> http://mail.zope.org/mailman/listinfo/zope-dev
> ** No cross posts or HTML encoding! ** (Related lists -
> http://mail.zope.org/mailman/listinfo/zope-announce
> http://mail.zope.org/mailman/listinfo/zope )
>

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


gary.poster at gmail

Aug 24, 2009, 5:27 PM

Post #4 of 6 (700 views)
Permalink
Re: zope.publisher 3.5 branch has code/behavior not a part of subsequent releases [In reply to]

On Aug 24, 2009, at 5:27 PM, Tres Seaver wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Gary Poster wrote:
>> Hm. I sent this from the wrong account, so it didn't make it to the
>> zope-dev list. I'm also adding an additional bit of war story at the
>> end.
>>
>>
>> On Aug 24, 2009, at 11:16 AM, Gary Poster wrote:
>>
>>> Hi Tres
>>>
>>> I made a 3.5.8 release of the zope.publisher 3.5 branch for a reason
>>> unimportant to this email (I backported a change to trunk that was
>>> discussed on the mailing list). It looks like you made a 3.5.7 with
>>> the following change (omitting tests and the like).
>>>
>>> 98932 tseaver # Python 2.6 notices QS-on-POST,
>>> which breaks BBB for us.
>>> 98932 tseaver # Suppress that.
>>> 98932 tseaver if 'QUERY_STRING' in self._environ:
>>> 98932 tseaver env = self._environ
>>> 98932 tseaver env['X-POST-QUERY_STRING'] =
>>> env.pop('QUERY_STRING')
>>>
>>> I can handle adjusting to this change, if it is appropriate, but my
>>> concern is that it is not in trunk or any subsequent major release
>>> (3.6, 3.7, 3.8) of zope.publisher. Therefore, if I change my code
>>> to use my updated 3.5 release, which I had hoped would be a
>>> conservative update, I have to change in a currently forward-
>>> incompatible way.
>>>
>>> What's the story here? Is 3.5.7 a brownbag that needs to have its
>>> changes aborted in a subsequent release in that branch? Or are
>>> those legitimate changes that need to be forward ported?
>>>
>>> FWIW, we (Launchpad) also care about this case of a POST with other
>>> pertinent, separate data in the query string, and the behavior you
>>> implement here would be fine if it is necessary for Py 2.6 as your
>>> comment describes.
>>
>> Also FWIW, this change breaks many of our forms that were explicitly
>> constructing form actions that included the current query string.
>> I'm
>> not sure how that could be avoided, although the fix might have been
>> simpler if there were always an X-ORIGINAL-QUERY_STRING or something
>> else.
>>
>> If I were not already behind, I would investigate to understand the
>> Python 2.6 problem better and see what other frameworks are doing
>> here. I understand from conversations with other engineers that at
>> least some Django developers are accustomed to always having access
>> to
>> the query string on the request object, whether the method were get
>> or
>> post or whatever else.
>
> It is *essential* for correct operation that QUERY_STRING values *not*
> be admixed with POSTed form values. I don't really care how we
> resolve
> your issue, as long as we do not end up in a case where the values in
> the query string get mingled into the form data: for instance, we
> could
> hand a QUERY_STRING-free copy of the environment to the
> cgi.FieldStorage
> machinery.
>
> Whatever gets done needs to leave the existing test in place::
>
> self.assertEqual(dict(request.form), dict(x='1', y='2'))
>
> for a request whose QUERY_STRING was 'a=5&b:int=6', but which posted
> the
> 'x' and 'y' values.

I'm good with that.

My additional constraint would be that

self.assertEqual(request.get('QUERY_STRING'), 'a=5&b:int=6')

for the same request. Hiding the QUERY_STRING causes us some
unpleasant and subtle functional test pain.

Passing a QUERY_STRING-free copy to cgi.FieldStorage as you suggest
seems like a good way to go to me.

Unless someone beats me to it or stops me, I'll plan to make this go
in trunk and in the 3.5 branch, and make a 3.5.9 release.

(If someone *were* to beat me to it, that would be awesome. I'm kinda
swamped.)

Thanks

Gary
_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


gary.poster at gmail

Aug 24, 2009, 5:31 PM

Post #5 of 6 (699 views)
Permalink
Re: zope.publisher 3.5 branch has code/behavior not a part of subsequent releases [In reply to]

On Aug 24, 2009, at 6:02 PM, Roger Ineichen wrote:

> Hi Tres
>
>> Betreff: Re: [Zope-dev] zope.publisher 3.5 branch has
>> code/behavior not a part of subsequent releases
>
> [...]
>
>>> If I were not already behind, I would investigate to understand the
>>> Python 2.6 problem better and see what other frameworks are doing
>>> here. I understand from conversations with other engineers that at
>>> least some Django developers are accustomed to always
>> having access to
>>> the query string on the request object, whether the method
>> were get or
>>> post or whatever else.
>>
>> It is *essential* for correct operation that QUERY_STRING
>> values *not* be admixed with POSTed form values. I don't
>> really care how we resolve your issue, as long as we do not
>> end up in a case where the values in the query string get
>> mingled into the form data: for instance, we could hand a
>> QUERY_STRING-free copy of the environment to the
>> cgi.FieldStorage machinery.
>
> As far as I understand, you are saying that it is essential
> that posted data and a query string should be separated
> for processing in python libraries e.g. FieldStorage or so.
>
> But this doesn't mean both values could not end in the
> request.form dict right?

right, that's what he wants, and that's the pre-Py 2.6 behavior.

>
>> Whatever gets done needs to leave the existing test in place::
>>
>> self.assertEqual(dict(request.form), dict(x='1', y='2'))
>>
>> for a request whose QUERY_STRING was 'a=5&b:int=6', but which
>> posted the 'x' and 'y' values.
>
> Was this supported before your changes? Is this a new feature
> you decided to add? What's the reason for this? Can you point
> me to more infos?

The constraint is an old behavior.

The solution in 3.5.8 and 3.5.9 is a pretty big behavior change if you
are paying attention to the query string during POSTs.

Maybe http://bugs.python.org/issue1817 gives you the information you
want?

Gary

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


dev at projekt01

Aug 25, 2009, 4:59 AM

Post #6 of 6 (687 views)
Permalink
Re: zope.publisher 3.5 branch has code/behavior not a part of subsequent releases [In reply to]

Hi Gary

> Betreff: Re: AW: [Zope-dev] zope.publisher 3.5 branch has
> code/behavior not a part of subsequent releases
>
>
> On Aug 24, 2009, at 6:02 PM, Roger Ineichen wrote:
>
> > Hi Tres
> >
> >> Betreff: Re: [Zope-dev] zope.publisher 3.5 branch has
> code/behavior
> >> not a part of subsequent releases
> >
> > [...]
> >
> >>> If I were not already behind, I would investigate to
> understand the
> >>> Python 2.6 problem better and see what other frameworks are doing
> >>> here. I understand from conversations with other
> engineers that at
> >>> least some Django developers are accustomed to always
> >> having access to
> >>> the query string on the request object, whether the method
> >> were get or
> >>> post or whatever else.
> >>
> >> It is *essential* for correct operation that QUERY_STRING values
> >> *not* be admixed with POSTed form values. I don't really
> care how we
> >> resolve your issue, as long as we do not end up in a case
> where the
> >> values in the query string get mingled into the form data: for
> >> instance, we could hand a QUERY_STRING-free copy of the
> environment
> >> to the cgi.FieldStorage machinery.
> >
> > As far as I understand, you are saying that it is essential that
> > posted data and a query string should be separated for
> processing in
> > python libraries e.g. FieldStorage or so.
> >
> > But this doesn't mean both values could not end in the request.form
> > dict right?
>
> right, that's what he wants, and that's the pre-Py 2.6 behavior.
>
> >
> >> Whatever gets done needs to leave the existing test in place::
> >>
> >> self.assertEqual(dict(request.form), dict(x='1', y='2'))
> >>
> >> for a request whose QUERY_STRING was 'a=5&b:int=6', but
> which posted
> >> the 'x' and 'y' values.
> >
> > Was this supported before your changes? Is this a new feature you
> > decided to add? What's the reason for this? Can you point
> me to more
> > infos?
>
> The constraint is an old behavior.
>
> The solution in 3.5.8 and 3.5.9 is a pretty big behavior
> change if you are paying attention to the query string during POSTs.

Ah, this sounds good. We often use query string urls for simulate a post
request e.g. foo.html?form.action.remove=1&id=123 which forces to process
the button action in z3c.form within a value 123 for the id field etc.

I was afraid that this isn't working anymore if such query values
will not longer work as request.form key/values

But as far as I understand it's only a problem with POST and query strings
and does not affect the above usecase.


> Maybe http://bugs.python.org/issue1817 gives you the
> information you want?

Thanks a lot

Roger Ineichen

> Gary
>
>

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Zope dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.