Gossamer Forum
Home : Products : Gossamer Links : Development, Plugins and Globals :

Spider Plugin adds all links.

Quote Reply
Spider Plugin adds all links.
I can start my spider with telnet login, but i can't Pause, Restart, Shutdown, with the links in admin panel.

Rules that I have defined are working fine (if I test it with 'Execute Code'),

but many links there spidered and indexed and I can nothing do against it.

Quote Reply
Re: Spider Plugin adds all links. In reply to
Kann denn niemand sagen, wie man diesen Spider sinnvoll einsetzt? Ich will doch nur Seiten aus einer bestimmten Zielgruppe finden und pro Domain nur eine Seite.


-------------------
Heiko Mentzel
http://findgay.net/
-------------------
Quote Reply
Re: Spider Plugin adds all links. In reply to
Maybe you run the spider.pl in another user so user apache can not kill the spider.pl from the web

Quote Reply
Re: Spider Plugin adds all links. In reply to
My apache doesn't kill my spider. The spider spiders all links (equal what I do), I have to kill the spider per telnet.

First the spider spiders all links, that I alredy have, than more and more it is spidering all undesirable links.

Rules have no effect.

-------------------
Heiko Mentzel
http://findgay.net/
-------------------
Quote Reply
Re: Spider Plugin adds all links. In reply to
Are you using spider 2.1? the the stop and pause from web issues were a problem with the 2.0 version, but should be fixed vith 2.1. It works for me anyway.

The rules are really hard to figure out. I think it is time that someone from the staff explains this to us, using examples. It seems that everybody has problems with it.
Quote Reply
Re: Spider Plugin adds all links. In reply to
Yes, I'm using Spider 2.1, but the stop and pause from web doesn't work. I put any links in the queue and than I start the spider.pl via telnet. the process and the cgi's will be started as the same owner.

I've a notion that the staff the has same problems to configure the rules. Perhaps it doesn't work correctly.

Whatever I might do, the the spider adds all unrelated links.

-------------------
Heiko Mentzel
http://findgay.net/
-------------------