another possibility would be to run a script that converted all of:

www.domain.com
www.domain.com/
www.domain.com/index.html
www.domain.com/index.php
http://www.doamin.com

to

http://www.domain.com/

if think this is normally better than specifying http://www.domain.com/index.html mainly because the owner might change .html to .htm but they are always going to make sure that www.domain.com works.

If a script was run that updated all of these then duplicate check would work fine.

http://www.ASciFi.com/ - The Science Fiction Portal