Mailing List Archive: Python: Bugs

[issue1424152] urllib/urllib2: HTTPS over (Squid) Proxy fails

Index | Next | Previous | View Threaded

report at bugs

Jun 15, 2008, 6:56 AM

Post #1 of 11 (302 views)
Permalink

[issue1424152] urllib/urllib2: HTTPS over (Squid) Proxy fails

Mathias Behrle <mathias.behrle[at]gmx.de> added the comment:

Just supporting, that proxy support over https is widely used and
important. I am wondering, why this request is open for more than 2 years.
Mathias

----------
nosy: +matb

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue1424152>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 15, 2008, 6:57 AM

Post #2 of 11 (297 views)
Permalink

[issue1424152] urllib/urllib2: HTTPS over (Squid) Proxy fails [In reply to]

Martin Wilck <martin.wilck[at]fujitsu-siemens.com> added the comment:

I am not in my office. I'll be back on June 30, 2008.
In urgent cases, please contact:
Peter Pols <peter.pols[at]fujitsu-siemens.com> or
Gerhard Wichert <gerhard.wichert[at]fujitsu-siemens.com>

Best regards
Martin Wilck

Added file: http://bugs.python.org/file10632/unnamed

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue1424152>
_______________________________________

Attachments:

unnamed (0.42 KB)

report at bugs

Jun 27, 2008, 4:25 AM

Post #3 of 11 (264 views)
Permalink

[issue1424152] urllib/urllib2: HTTPS over (Squid) Proxy fails [In reply to]

Martijn Pieters <mj[at]python.org> added the comment:

I've created a python 2.4 compatible distribution of the python 2.5
urllib2 and httplib modules with the patch attached to this issue. This
has proven especially useful in zc.buildout or other setuptools
environments:

http://pypi.python.org/pypi/httpsproxy_urllib2

----------
nosy: +mjpieters

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue1424152>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2008, 3:22 PM

Post #4 of 11 (260 views)
Permalink

[issue1424152] urllib/urllib2: HTTPS over (Squid) Proxy fails [In reply to]

nfl <nflpython[at]irto.hu> added the comment:

Hi,

looks like the attached patch only works if I use
urllib2.ProxyHandler({'https' : proxy}) at the start, but not when I use
request.set_proxy(proxy, 'https') per request. I tested with Python
2.5.1 on Win32.

----------
nosy: +nfl

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue1424152>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2008, 4:09 PM

Post #5 of 11 (260 views)
Permalink

[issue1424152] urllib/urllib2: HTTPS over (Squid) Proxy fails [In reply to]

Christopher Li <python[at]chrisli.org> added the comment:

Hi NL,

Can you please try this patch on top of the original patch?

Thanks

Chris

--- .pc/fix-up/urllib2.py 2008-02-06 01:13:10.000000000 -0800
+++ urllib2.py 2008-06-27 15:45:34.000000000 -0700
@@ -253,7 +253,7 @@ class Request:
return self.__r_host

def set_proxy(self, host, type):
- if self.type == 'https' and not self._tunnel_host:
+ if self.get_type() == 'https' and not self._tunnel_host:
self._tunnel_host = self.host
else:
self.type = type

On Fri, Jun 27, 2008 at 3:22 PM, nfl <report[at]bugs.python.org> wrote:
>
> nfl <nflpython[at]irto.hu> added the comment:
>
> Hi,
>
> looks like the attached patch only works if I use
> urllib2.ProxyHandler({'https' : proxy}) at the start, but not when I use
> request.set_proxy(proxy, 'https') per request. I tested with Python
> 2.5.1 on Win32.
>
> ----------
> nosy: +nfl
>
> _______________________________________
> Python tracker <report[at]bugs.python.org>
> <http://bugs.python.org/issue1424152>
> _______________________________________
>

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue1424152>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 30, 2008, 1:49 AM

Post #6 of 11 (249 views)
Permalink

[issue1424152] urllib/urllib2: HTTPS over (Squid) Proxy fails [In reply to]

Nagy Ferenc László <nflpython[at]irto.hu> added the comment:

> Can you please try this patch on top of the original patch?

It makes an infinite loop:

Traceback (most recent call last):
File "F:\nfl\proxyhttps\test.py", line 8, in <module>
response = urllib2.urlopen(req)
File "C:\Python25\lib\urllib2.py", line 121, in urlopen
return _opener.open(url, data)
File "C:\Python25\lib\urllib2.py", line 377, in open
req = meth(req)
File "C:\Python25\lib\urllib2.py", line 1037, in do_request_
scheme, sel = splittype(request.get_selector())
File "C:\Python25\lib\urllib2.py", line 253, in get_selector
return self.__r_host
File "C:\Python25\lib\urllib2.py", line 215, in __getattr__
return getattr(self, attr)
File "C:\Python25\lib\urllib2.py", line 215, in __getattr__
return getattr(self, attr)
File "C:\Python25\lib\urllib2.py", line 215, in __getattr__
return getattr(self, attr)
File "C:\Python25\lib\urllib2.py", line 215, in __getattr__
return getattr(self, attr)
File "C:\Python25\lib\urllib2.py", line 215, in __getattr__
return getattr(self, attr)
...

The test program was:

import urllib2

targeturl = 'https://www.paypal.com/'
proxyhost = 'proxy.xxxxxxxx.hu:3128'

req = urllib2.Request(targeturl)
req.set_proxy(proxyhost, 'https')
response = urllib2.urlopen(req)
print response.info()

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue1424152>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jul 3, 2008, 10:30 AM

Post #7 of 11 (234 views)
Permalink

[issue1424152] urllib/urllib2: HTTPS over (Squid) Proxy fails [In reply to]

Changes by Facundo Batista <facundo[at]taniquetil.com.ar>:

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue1424152>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jul 3, 2008, 10:30 AM

Post #8 of 11 (234 views)
Permalink

[issue1424152] urllib/urllib2: HTTPS over (Squid) Proxy fails [In reply to]

Changes by Facundo Batista <facundo[at]taniquetil.com.ar>:

Removed file: http://bugs.python.org/file10632/unnamed

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue1424152>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jul 3, 2008, 10:33 AM

Post #9 of 11 (234 views)
Permalink

[issue1424152] urllib/urllib2: HTTPS over (Squid) Proxy fails [In reply to]

Facundo Batista <facundo[at]taniquetil.com.ar> added the comment:

I see that you're working on a final solution, this is great!

What we would need also to incorporate this to the trunk is a patch for
the test suite. Senthil, could you handle this?

----------
assignee: -> facundobatista
nosy: +facundobatista

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue1424152>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jul 7, 2008, 8:42 PM

Post #10 of 11 (214 views)
Permalink

[issue1424152] urllib/urllib2: HTTPS over (Squid) Proxy fails [In reply to]

Senthil <orsenthil[at]gmail.com> added the comment:

>> Senthil, could you handle this?

Sure, I shall take this up, Facundo.

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue1424152>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jul 10, 2008, 3:04 PM

Post #11 of 11 (210 views)
Permalink

[issue1424152] urllib/urllib2: HTTPS over (Squid) Proxy fails [In reply to]

Christopher Li <python[at]chrisli.org> added the comment:

Hi,

After a closer look of your testing program. I believe that your testing
program is not correct. It might be the urllib way of doing things. But
that is not the urllib2 way of doing it. You should add a proxy_handler
rather than directly set_proxy on request objects.

The reason is that, you mess req directly. The urllib2 will not handle
correctly if the request needs rebuild and forward to other handlers.
One example is if the request needs redirect, I believe your test program
will not work.

So the right way of doing things, IMHO, is just adding a proxy handler
which contain the proxy you want. Then that proxy handler will just
automatically set_proxy() to every request it goes out. It even works
if the request needs to be rebuilt.

Then my patch should just work in that case. Try to make it work
for your test script will make things unnecessary complicated.
That defeats the benefit of the handler call chain in urllib2.

Does it make sense to you?

Thanks,

Chris

On Mon, Jun 30, 2008 at 1:49 AM, Nagy Ferenc László
<report[at]bugs.python.org> wrote:
> The test program was:
>
> import urllib2
>
> targeturl = 'https://www.paypal.com/'
> proxyhost = 'proxy.xxxxxxxx.hu:3128'
>
> req = urllib2.Request(targeturl)
> req.set_proxy(proxyhost, 'https')
> response = urllib2.urlopen(req)
> print response.info()

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue1424152>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com