I have been playing around with the spider 2.1 for some time now. The installation worked fine, the status works, I can stop an pause the spider from web. Which is all fine, BUT:
The reason why I bought the spider was the promise that you can use spider rules to get only the specific pages that YOU want in your database.
I can not get this to work. I made a rule to get only the pages with a very rare word in the data. I knew that this word was only on two or three of the pages on my site. The I spidererd my site. - the rule didn't work at all, all pages on my site came up for validation!
So please... what are we doing wrong? Can anyone give some instructions on how to use the spider rules?
The reason why I bought the spider was the promise that you can use spider rules to get only the specific pages that YOU want in your database.
I can not get this to work. I made a rule to get only the pages with a very rare word in the data. I knew that this word was only on two or three of the pages on my site. The I spidererd my site. - the rule didn't work at all, all pages on my site came up for validation!
So please... what are we doing wrong? Can anyone give some instructions on how to use the spider rules?