Second question,
Some links come back with:
The following URL's have been grabbed, but were decided that they were not suitable for spidering;
What if I want to spider it anyway, there is no option or way of doing it???
Thanks,
Also, as a future update, how hard would it be to have an admin function that once you have spidered all the links, you can click a check box or something and then click add, and it will add all of the clicked spidered links into the category you selected instead of individually???
</not a clue>
Some links come back with:
The following URL's have been grabbed, but were decided that they were not suitable for spidering;
What if I want to spider it anyway, there is no option or way of doing it???
Thanks,
Also, as a future update, how hard would it be to have an admin function that once you have spidered all the links, you can click a check box or something and then click add, and it will add all of the clicked spidered links into the category you selected instead of individually???
</not a clue>
