Gossamer Forum
Quote Reply
Duplicate_Check
Hello Andy,

I test Duplicate_Check script

It is possible to check the following copy ( / ) ?

http://www.mysite.com

with

http://www.mysite.com/

Thank you

Mick
Quote Reply
Re: [mick31] Duplicate_Check In reply to
I'm sure I added this in a release version... but maybe I'm just going mad Crazy

Try replacing the following line in admin/Plugins/Duplicate_Check.pm;

Code:
$chk_dup = $db_con->count ( { URL => $IN->param('URL') } );

with;

Code:
my $url = $IN->param('URL');
my $cond = GT::SQL::Condition->new('URL','LIKE',"$url%");
$chk_dup = $db_con->count ( $cond );

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Duplicate_Check In reply to
Thank you Andy,

It is good

I downloaded script yesterday.

Admin = > Plugins

It is perhaps not up to date?

Still Thank you.

Mick

Quote Reply
Re: [mick31] Duplicate_Check In reply to
Nah, I just checked the copy on my computer, and that didn't have the extra part in either. I'll get it added in again, and update it in the download area.

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Duplicate_Check In reply to
Andy, I made the changes above and got a 500 error when adding a duplicate link (only occurred with or without the trailing slash, can't remember which exactly).

The server error log detailed a malformed header problem so from searching on here I found a similar problem was fixed by adding this before the print statement in the code:

Code:
print $IN->header();


It fixed my problems anyway so I share it here in case it can help anyone else.
Quote Reply
Re: [aus_dave] Duplicate_Check In reply to
Mmmm...odd. I'll have to get that fixed up too. Not sure how that managed to slip through though.. as this plugin has been out for ages Frown

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Duplicate_Check In reply to
Code:
I test Duplicate_Check script

It is possible to check the following copy ( / ) ?



Hello Andy

It is possible to obtain the checking with Spider ?
Currently, it is Duplicate_Check which makes the checking.

That done more work, because it checks after the validation of the link.

Thank you for your assistance.


Mick
Quote Reply
Re: [mick31] Duplicate_Check In reply to
Hi. I'm not quite sure what you mean. My Spider already checks for duplicate links (it shows an error in red if the link already exists in your database).

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Duplicate_Check In reply to
Hello Andy,

I noticed that script does not check:

http://www.site.com
and
http://www.site.com/ --> ( / )

An idea?

Mick
Quote Reply
Re: [mick31] Duplicate_Check In reply to
How about replacing;

Code:
my $url = $IN->param('URL');
my $cond = GT::SQL::Condition->new('URL','LIKE',"$url%");
$chk_dup = $db_con->count ( $cond );

...with;

Code:
my $url = $IN->param('URL');
my $url2 = $url;
$url2 =~ s/www\.//i;
my $url3 = $url;
$url3 =~ s,(/|\\)$,,i;

my $cond1 = GT::SQL::Condition->new('URL','LIKE',"$url%");
my $chk_dup1 = $db_con->count ( $cond1 );

my $cond2 = GT::SQL::Condition->new('URL','LIKE',"$url2%");
my $chk_dup2 = $db_con->count ( $cond2 );

my $cond2 = GT::SQL::Condition->new('URL','LIKE',"$url3%");
my $chk_dup3 = $db_con->count ( $cond3 );

$chk_dup1 ? $chk_dup = $chk_dup1 : '';
$chk_dup2 ? $chk_dup = $chk_dup2 : '';
$chk_dup3 ? $chk_dup = $chk_dup3 : '';

Untested, but it should work.

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Duplicate_Check In reply to
There are a few dubious bits in that code. Why would a URL have a backslash at the end, as suggested by this line:

Code:
$url3 =~ s,(/|\\)$,,i;

You can't have case sensitive and case insensitive slashes either so the "i" modifier is redundant Wink

It might be better written like this:

Code:
my $url = $IN->param('URL');
my $bit = ($url =~ m!(?:http://)?(?:www\.)?(.+)!)[0];
my $chk_dup;
if ($db->count(GT::SQL::Condition->new('URL','LIKE',"$bit%"))) {
$chk_dup = 1;
}

(That regex might be iffy as I rushes a bit).

Regarding the last part....well it's overly lengthy doing 3 checks like this:

Code:
$chk_dup1 ? $chk_dup = $chk_dup1 : '';
$chk_dup2 ? $chk_dup = $chk_dup2 : '';
$chk_dup3 ? $chk_dup = $chk_dup3 : '';

The order is slightly wrong too...it would normally be:

Code:
$chk_dup = $chk_dup1 ? $chk_dup1 : '';
$chk_dup = $chk_dup2 ? $chk_dup2 : '';
$chk_dup = $chk_dup3 ? $chk_dup3 : '';

But even nicer would be:

Code:
$chk_dup = $chk_dup1 || $chk_dup2 || $chk_dup3 || 0;

...but that's all by the by if you use the example above which saves doing 3 queries.
Quote Reply
Re: [Coombes..] Duplicate_Check In reply to
Quote:
There are a few dubious bits in that code. Why would a URL have a backslash at the end, as suggested by this line:

You would be suprised what people do to try and get past duplicate checking ;)

Quote:
You can't have case sensitive and case insensitive slashes either so the "i" modifier is redundant

Good point Blush

Quote:
The order is slightly wrong too...it would normally be:

Not really.

if $chk_dup1 then $chk_dup = $chk_dup1 or do nothing
if $chk_dup2 then $chk_dup = $chk_dup2 or do nothing
if $chk_dup3 then $chk_dup = $chk_dup3 or do nothing

Code:
$chk_dup = $chk_dup1 || $chk_dup2 || $chk_dup3 || 0;

You are correct here though, as this is most definatly shorter (I was rushing);

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Duplicate_Check In reply to
Quote:
You would be suprised what people do to try and get past duplicate checking ;)

If you look to grab the main domain name from the input then you needn't worry about all the obscurities that they may try fool the script with.

Quote:
Not really.

if $chk_dup1 then $chk_dup = $chk_dup1 or do nothing
if $chk_dup2 then $chk_dup = $chk_dup2 or do nothing
if $chk_dup3 then $chk_dup = $chk_dup3 or do nothing

I know how you are thinking but you aren't really doing nothing, you are using '' in a void context....it is just floating in your script. If you turn on warnings you'll get an error about a constant being used in void context.
Quote Reply
Re: [Andy] Duplicate_Check In reply to
Andy,

I do not understand the discussion with Coombes.

Which code to use? You can summarize the code?

Thank you for your assistance

Mick
Quote Reply
Re: [Andy] Duplicate_Check In reply to
Andy,

The problem is with Spider and not with Duplicate_Check.

Duplicate_Check seems very well to check the validation.

Where to modify the code in Spider ?

Thank you for your assistance.

Mick
Quote Reply
Re: [mick31] Duplicate_Check In reply to
Sorry, I got confused Crazy

In /admin/Plugins/Spider.pm, on about line 401, change;

Code:
# see if it exists, and if so, show a WARNING!
my $db_con = $DB->table("Links");
my $chk_dup = $db_con->count ( { URL => $_ } );
if ($chk_dup) {
$status = "<font color=red>WARNING: Link already exists!</font>";
} else {
$status = "<font color=blue>Ok....couldn't find in database...</font>";
}

...to....

Code:
my $db_con = $DB->table("Links");

my $url = $_;
my $url2 = $_;
$url2 =~ s/www\.//i;
my $url3 = $_;
$url3 =~ s,/$,,;

my $cond1 = GT::SQL::Condition->new('URL','LIKE',"$url%");
my $chk_dup1 = $db_con->count ( $cond1 );
my $cond2 = GT::SQL::Condition->new('URL','LIKE',"$url2%");
my $chk_dup2 = $db_con->count ( $cond2 );
my $cond2 = GT::SQL::Condition->new('URL','LIKE',"$url3%");
my $chk_dup3 = $db_con->count ( $cond3 );

# see if it exists, and if so, show a WARNING!
if ($chk_dup1 || $chk_dup2 || $chk_dup3) {
$status = "<font color=red>WARNING: Link already exists!</font>";
} else {
$status = "<font color=blue>Ok....couldn't find in database...</font>";
}

...also, at the top of the file, just below;

Code:
use CGI::Carp qw(fatalsToBrowser);

...you need to add;

Code:
use GT::SQL::Condition;

Hope that helps/works Smile

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!

Last edited by:

Andy: Dec 17, 2003, 3:21 AM
Quote Reply
Re: [Andy] Duplicate_Check In reply to
Hello Andy,

I have the following error.

Unable to load plugin: Spider (Compilation failed in require at admin.cgi line 205.
) at admin.cgi line 207.


An idea?

Thank you for your assistance

Mick

Last edited by:

mick31: Dec 17, 2003, 3:18 AM
Quote Reply
Re: [mick31] Duplicate_Check In reply to
Mmm...please try the edited version of the mod a couple of posts up ( http://www.gossamer-threads.com/perl/gforum/gforum.cgi?post=258270#258270 ).

Does that work?

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Duplicate_Check In reply to
Not, it does not work.
An idea?

Mick
Quote Reply
Re: [mick31] Duplicate_Check In reply to
Can you PM me over the Spider.pm file, so I can take a look at how you modified it?

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!