Hey all,

Also playing with the spider plugins and the not so easy to understand documentation...

I queued a concurrent site of mine and can see the result of the indexing now...

I have -> tons of robots.txt to validate
-> tons of empty "description" field links to validate
-> tons of empty "keywords" field links to validate
-> tons of empty "keywords & description" fields links to validate

Then, when I will find the way to shoot all those empty or robot.txt related files, I will stay with a good big bulk of link to validate...

1st I need to to the step I just talk about and then I would like to find a way to compare the links still in my indexed spider with my own current of "normal" links in my own directory and shoot the duplicates from the spider...

uhh

Another thing... anybody figured out how to spider it's own indexed links database(from the normal index) add a keyword field and parse the missing info "keywords" into that field easiely?

Well, still lots of dark zones for me, but I think we'll face them ... Thanks for your help...

Cheers