I am using the Spider plugin and i need to know how to check if the downloaded links arent already in my database.
Also, is there anyway to make Spider check automatically if it is a duplicated link before downloading it?
Can this be done, or it is necesary to make some special modifications?

I would apreciate if someone helps.

thanks in advance

jml