Mailing List Archive: Python: Python

Newbie question about scrapy tutorial

Index | Next | Previous | View Threaded

azeynel1 at gmail

Nov 20, 2009, 8:46 AM

Post #1 of 4 (239 views)
Permalink

Newbie question about scrapy tutorial

Hello,

I have a question about scrapy tutorial and I put 2 questions in the
scrapy group

http://groups.google.com/group/scrapy-users/t/d5afae7d88672e02
http://groups.google.com/group/scrapy-users/t/4d52fa8c589a412a

but there were no answers. Can anyone here help me? The spider works
fine but I couldn't figure out where the items file is so that I can
print it to a file to transfer it to my database.

The same question in stackoverflow
http://stackoverflow.com/questions/1771151/newbie-q-about-scrapy-pipeline-py

Thanks!
--
http://mail.python.org/mailman/listinfo/python-list

joost at h-labahn

Nov 21, 2009, 8:21 AM

Post #2 of 4 (206 views)
Permalink

Re: Newbie question about scrapy tutorial [In reply to]

Sorry,

i have no idea what scrapy is - but i see, that you try to use
idle with pipes or anything like that. That cannot work.
idle doesn't even handle __file__ correctly. Use a full-fledged
python IDE (PyDev under Eclipse leaves very few wishes open) or
test in a python interpreter shell.

Good luck, DreiJane
--
http://mail.python.org/mailman/listinfo/python-list

azeynel1 at gmail

Nov 21, 2009, 11:00 AM

Post #3 of 4 (204 views)
Permalink

Re: Newbie question about scrapy tutorial [In reply to]

Now I use EditpadPro and IDLE which appears to be adequate for
beginning level. But PyDev looks fun, I'll try it.

By the way, I realized that the spider sends the scraped data to the
pipelines.py. Now everything is working.

On Nov 21, 11:21 am, DreiJane <jo...@h-labahn.de> wrote:
> Sorry,
>
> i have no idea what scrapy is - but i see, that you try to use
> idle with pipes or anything like that. That cannot work.
> idle doesn't even handle __file__ correctly. Use a full-fledged
> python IDE (PyDev under Eclipse leaves very few wishes open) or
> test in a python interpreter shell.
>
> Good luck, DreiJane

--
http://mail.python.org/mailman/listinfo/python-list

cousinstanley at gmail

Nov 21, 2009, 11:10 AM

Post #4 of 4 (212 views)
Permalink

Re: Newbie question about scrapy tutorial [In reply to]

> i have no idea what scrapy is
> ....

http://scrapy.org/

Scrapy is a fast high-level screen scraping and web
crawling framework, used to crawl websites and extract
structured data from their pages. It can be used for
a wide range of purposes, from data mining to monitoring
and automated testing.

Also, previously unknown to me
before google-izing ....

--
Stanley C. Kitching
Human Being
Phoenix, Arizona

--
http://mail.python.org/mailman/listinfo/python-list

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads