Ok, feature is added :) It now gives you the option to either Use LWP::Simple (which doesn't obey robots.txt), or LWP::RobotUA, which does obey robots.txt. You can also define a 2 second delay between requests if you want (slows the process down, but limits the chance of your server getting banned)
Version 1.2
I'm uploading it to the accounts area now, so anyone wishing to upgrade, please login at http://new.linkssql.net/page.php?page=account
Cheers
Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Version 1.2
I'm uploading it to the accounts area now, so anyone wishing to upgrade, please login at http://new.linkssql.net/page.php?page=account
Cheers
Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!