Hi,

A couple of questions about the spider:

1. Can you tell me if the spider / links SQL combination could be used create a search engine? ie. the links directory 'only' contains spidered links? What I'm thinking is of a linksSQL directory that technically has no categories(?) but it also has a searh box which users can search with and return any matching spidered results, spanning accross various pages. If not, perhaps you could offer another suggestion? I'm running a unix based server.

2. can you limit the spider to search only limited domain suffixes? ie. if i wanted to only spider sites in new zealand can it only search on '.nz' domains, or if I only wanted to spider school web sites in new zealand can I limit it to spidering sites with '.school.nz' suffixs.

Hi,

1. Yes, if you had Links SQL and the spider installed, you could use only the search.cgi and in the search_results.html template only print spider_hits and spider_results.

2. Yes, there is a powerful filtering system that lets you choose what is indexed and what is spidered. See:

http://gossamer-threads.com/...er/rules_screen.html

for an example of what you can do. We designed this for people wanting to build targeted indexes, not an index of the entire web, as most people wont have the server resources neccessary to do that.

Cheers,

Alex

--
Gossamer Threads Inc.