Gossamer Forum
Home : General : Perl Programming :

Whys this take so long to run?

Quote Reply
Whys this take so long to run?
Hi. Can anyone give me any pointers as to why this is so slow to run the updates?

Code:
open(GRAB, "Hotel_Description_Cut_1.txt") or die "Error reading Hotel_Description_Cut_1.txt. Reason: $!";
my ($old_id,$image_name,$field);
my $i = 0;
my $count = 0;
while (<GRAB>) {

chomp;

my $_update;
my @cut = split /\|/, $_;

# skip the beginning line....
if (/HotelID/) { next; }

#print "$count: HotelID: " . $cut[0] . "\n";

unless ($count % 10) { print "> $count\n"; }

my $hotel_id = $cut[0];
my $desc = $cut[1];

undef @cut;

$count++;

if ($DB->table('Links')->count( { HotelID => $hotel_id } )) {
$DB->table('Links')->update( { Description => $desc } , { HotelID => $hotel_id } ) or die $GT::SQL::error;
}

}
close(GRAB);

It is taking about 1 minute to complete 160 entries, which is far too slow. There are almost 50,000 to run... which would take about 5 1/2 hours!

I just can't see why its going so slow.

Anyone got any ideas? Unsure

Cheers

Andy (mod)
andy@ultranerds.co.uk


IMPORTANT: I've now moved to ultranerds.co.uk, and the .com will no longer work!
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package (plugins total "value" $3,325 & rising, for just $350)| GLinks ULTRA Package PRO (plugins total "value" $5,625 & rising, for just $500)
Support Forum | Links SQL Plugins | DMOZ Dumps | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Compare our different Plugin packages *new* Free CSS Templates
Quote Reply
Re: [Andy] Whys this take so long to run? In reply to
I'd love to help but my upstairs neighbor has been playing this horrible Indian music on her stereo for the past 2 hours and it's making my head hurt.

Philip
------------------
Limecat is not pleased.
Quote Reply
Re: [Andy] Whys this take so long to run? In reply to
I don't know how, or why this is possible... but using the below code seems to have solved the problem;

Code:
open(GRAB, "Hotel_Description.txt") or die "Error reading Hotel_Description.txt. Reason: $!";
my ($old_id,$image_name,$field);
my $i = 0;
my $count = 0;
my $pushed;
my @_ids;
while (<GRAB>) {

chomp;

my $_update;
my @cut = split /\|/, $_;

# skip the beginning line....
if (/HotelID/) { next; }

push(@_ids,$cut[0]);
$pushed->{$cut[0]} = $cut[1];

unless($count % 1000) { print "Reading IDs group: $count \n"; }

$count++;

}
close(GRAB);


my $count = 0;
foreach (@_ids) {

if ($DB->table('Links')->count( { ID => $_ } )) {
$DB->table('Links')->update( { Description => $pushed->{$_} } , { ID => $_ } ) or die $GT::SQL::error;
}
unless($count % 100) { print "Writing: $count \n"; }
$count++;

}

It now completes in about 5 minutes! Cool

Cheers

Andy (mod)
andy@ultranerds.co.uk


IMPORTANT: I've now moved to ultranerds.co.uk, and the .com will no longer work!
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package (plugins total "value" $3,325 & rising, for just $350)| GLinks ULTRA Package PRO (plugins total "value" $5,625 & rising, for just $500)
Support Forum | Links SQL Plugins | DMOZ Dumps | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Compare our different Plugin packages *new* Free CSS Templates