Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Python

Waiting for receiving data

 

 

Python python RSS feed   Index | Next | Previous | View Threaded


mail at anjanesh

Nov 23, 2009, 12:30 PM

Post #1 of 5 (195 views)
Permalink
Waiting for receiving data

fp = urllib.urlopen(url)
data = fp.read()

Retrieving XML data via an XML service API.
Very often network gets stuck in between. No errors / exceptions.

CTRL+C

File "get-xml.py", line 32, in <module>
fp = urllib.urlopen(url)
File "/usr/lib/python2.6/urllib.py", line 87, in urlopen
return opener.open(url)
File "/usr/lib/python2.6/urllib.py", line 206, in open
return getattr(self, name)(url)
File "/usr/lib/python2.6/urllib.py", line 348, in open_http
errcode, errmsg, headers = h.getreply()
File "/usr/lib/python2.6/httplib.py", line 1048, in getreply
response = self._conn.getresponse()
File "/usr/lib/python2.6/httplib.py", line 974, in getresponse
response.begin()
File "/usr/lib/python2.6/httplib.py", line 391, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.6/httplib.py", line 349, in _read_status
line = self.fp.readline()
File "/usr/lib/python2.6/socket.py", line 397, in readline
data = recv(1)
KeyboardInterrupt

Is there I can do to try something else if its taking too long to
retrieve from the network ? Like kill previous attempt and retry ?

Thanks
Anjanesh Lekshmnarayanan
--
http://mail.python.org/mailman/listinfo/python-list


geraldwalkerx at gmail

Nov 23, 2009, 5:53 PM

Post #2 of 5 (191 views)
Permalink
Re: Waiting for receiving data [In reply to]

Anjanesh Lekshminarayanan wrote:
> fp = urllib.urlopen(url)
> data = fp.read()
>
> Retrieving XML data via an XML service API.
> Very often network gets stuck in between. No errors / exceptions.
>
> CTRL+C
>
> File "get-xml.py", line 32, in <module>
> fp = urllib.urlopen(url)
> File "/usr/lib/python2.6/urllib.py", line 87, in urlopen
> return opener.open(url)
> File "/usr/lib/python2.6/urllib.py", line 206, in open
> return getattr(self, name)(url)
> File "/usr/lib/python2.6/urllib.py", line 348, in open_http
> errcode, errmsg, headers = h.getreply()
> File "/usr/lib/python2.6/httplib.py", line 1048, in getreply
> response = self._conn.getresponse()
> File "/usr/lib/python2.6/httplib.py", line 974, in getresponse
> response.begin()
> File "/usr/lib/python2.6/httplib.py", line 391, in begin
> version, status, reason = self._read_status()
> File "/usr/lib/python2.6/httplib.py", line 349, in _read_status
> line = self.fp.readline()
> File "/usr/lib/python2.6/socket.py", line 397, in readline
> data = recv(1)
> KeyboardInterrupt
>
> Is there I can do to try something else if its taking too long to
> retrieve from the network ? Like kill previous attempt and retry ?
>
> Thanks
> Anjanesh Lekshmnarayanan



import socket
from urllib2 import urlopen

# A one-hundredths of a second (0.01) timeout before socket throws
# an exception to demonstrate catching the timeout.
# Obviously, this you will set this greater than 0.01 in real life.
socket.setdefaulttimeout(0.01)

# example xml feed
xml_source = "http://mlb.mlb.com/partnerxml/gen/news/rss/mlb.xml"

try:
data = urlopen(xml_source)
except urllib2.URLError, e:
print 'URLError = ' + str(e.reason)



--
http://mail.python.org/mailman/listinfo/python-list


geraldwalkerx at gmail

Nov 23, 2009, 6:01 PM

Post #3 of 5 (180 views)
Permalink
Re: Waiting for receiving data [In reply to]

>
> import socket
> from urllib2 import urlopen
>
> # A one-hundredths of a second (0.01) timeout before socket throws
> # an exception to demonstrate catching the timeout.
> # Obviously, this you will set this greater than 0.01 in real life.
> socket.setdefaulttimeout(0.01)
>
> # example xml feed
> xml_source = "http://mlb.mlb.com/partnerxml/gen/news/rss/mlb.xml"
>
> try:
> data = urlopen(xml_source)
> except urllib2.URLError, e:
> print 'URLError = ' + str(e.reason)

Also, if you are using multiple threads to retrieve the xml source(s)
and any thread blocks due to network problems, the thread can go way by
itself after the default timeout expires.


--
http://mail.python.org/mailman/listinfo/python-list


geraldwalkerx at gmail

Nov 23, 2009, 6:06 PM

Post #4 of 5 (184 views)
Permalink
Re: Waiting for receiving data [In reply to]

>
> Also, if you are using multiple threads to retrieve the xml source(s)
> and any thread blocks due to network problems, the thread can go way by
> itself after the default timeout expires.
>
>

Typo, edited for clarity:
That is: "..the thread can go *away* by itself after the default timeout
expires." You don't need to explicitly kill it.


--
http://mail.python.org/mailman/listinfo/python-list


geraldwalkerx at gmail

Nov 23, 2009, 6:15 PM

Post #5 of 5 (182 views)
Permalink
Re: Waiting for receiving data [In reply to]

>
> import socket
> from urllib2 import urlopen
>
> # A one-hundredths of a second (0.01) timeout before socket throws
> # an exception to demonstrate catching the timeout.
> # Obviously, this you will set this greater than 0.01 in real life.
> socket.setdefaulttimeout(0.01)
>
> # example xml feed
> xml_source = "http://mlb.mlb.com/partnerxml/gen/news/rss/mlb.xml"
>
> try:
> data = urlopen(xml_source)
> except urllib2.URLError, e:
> print 'URLError = ' + str(e.reason)

Oops, the above doesn't run. This version works:

import socket
import urllib2

# A one-hundredths of a second (0.01) timeout before socket throws
# an exception to demonstrate catching the timeout.
# Obviously, this you will set this greater than 0.01 in real life.
socket.setdefaulttimeout(0.01)

# example xml feed
xml_source = "http://mlb.mlb.com/partnerxml/gen/news/rss/mlb.xml"

try:
data = urllib2.urlopen(xml_source)
except urllib2.URLError, e:
print 'URLError = ' + str(e.reason)

--
http://mail.python.org/mailman/listinfo/python-list

Python python RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.