We are working on just that, a targeted spider. It's about three weeks away from completetion (we've been sidetracked with another project).
Basically you can seed it with some initial links, and define a set of rules for what is a link you want to follow, then let it go search the web.
We'll post more details as they become available.
Cheers,
Alex
Basically you can seed it with some initial links, and define a set of rules for what is a link you want to follow, then let it go search the web.
We'll post more details as they become available.
Cheers,
Alex