Gossamer Forum
Home : Products : DBMan : Customization :

Verify Records Exist

Quote Reply
Verify Records Exist
I implemented the following code to verify all accounts had a corresponding record in the database. However when I try to use it - it does not complete. It gets as far as "The following users do not have records in the database:" and stops - the footer does not even load to complete the page.

Any ideas? Here is the code:
Code:
sub html_verify_accounts {
# --------------------------------------------------------

&html_print_headers;
$page_title = "Accounts Without Records";
&html_page_top;

open (PASSWD, "<$auth_pw_file") or &cgierr("unable to open password file. Reason: $!\n");
@passwds = <PASSWD>;
close PASSWD;

foreach $pass (@passwds) {
@data = split ":",$pass;
open (DB, "<$db_file_name") or &cgierr("unable to open database: $db_file_name.\nReason: $!");
@lines = <DB>;
close DB;

$found = 0;
foreach $line (@lines) {
if ($line =~ /^#/) { next; }
if ($line =~ /^\s*$/) { next; }
chomp ($line);
@values = &split_decode($line);

if ($values[$auth_user_field] eq $data[0]) {
$found = 1;
last;
}
}
unless ($found == 1) {
push (@list,$data[0]);
}
}
unless ($list[0]) {
print "All users have records in the database";
}
else {
print "The following users do not have records in the database:<BR>";
foreach $user (@list) {
print "$user<BR>";
}
}
&html_footer;
&html_page_bottom;
}
--------------------
donm


Quote Reply
Re: Verify Records Exist In reply to
Looks like you have not defined $user properly in your codes. This seems to be the wrong scalar to use in this sub.

Try using $list.

Try changing the following codes:

Code:

foreach $user (@list) {
print "$user<BR>";
}


to the following:

Code:

foreach $list (@list) {
print "$list<BR>";
}


Regards,

Eliot Lee
Quote Reply
Re: Verify Records Exist In reply to
Eliot - tried that - same result. Page stops at same place.

-----------------
donm

Quote Reply
Re: Verify Records Exist In reply to
Don,
My syntax is exactly as yours, 'cept mine works. Only thing I could think of is JPD's ref to the
In Reply To:
"user-friendly" html.pl
at her latest http://www.gossamer-threads.com/...m12/HTML/001076.html

Does that help or hurt?

JR

Quote Reply
Re: Verify Records Exist In reply to
Yep...I quickly installed the codes and it works fine for me as well.

Regards,

Eliot Lee
Quote Reply
Re: Verify Records Exist In reply to
When did you start having problems with this? It was working before, right? Is there anything you did to change the script?


JPD
Quote Reply
Re: Verify Records Exist In reply to
Yes it was working before. The change we made was to add the new permission to this database that you and I worked through on a different database.

The verify is working fine on two of the three databases - the only one it will not work on is this one, which totally has me baffled as this one is just a copy of the other two with "color" changes.

------------------
donm

Quote Reply
Re: Verify Records Exist In reply to
Are you sure your $auth_user_field variable is set correctly in this database? That's the only thing I can think of that might be a problem.


JPD
Quote Reply
Re: Verify Records Exist In reply to
Yes, the $auth_user_field is the same as the other two databases that are currently working. As I stated in my earlier post all of the files are exact copies of one another for each database. The only differences are in the appearance - colors. The one we are having problems with has the most records over 1,600 - while the other two between 500-600.

The largest one was working fine - it finally just stopped working in the past few days. After adding the new permissions and after going over 1,600 records. It's almost as if it times out and doesn't finish loading the page. When page finally loads after 5-10min - the header "Accounts without Records" appears but there are no records and the &footer is missing.

--------------------
donm

Quote Reply
Re: Verify Records Exist In reply to
Could be a CPU problem...try taking the codes out and creating a separate .cgi script...Then execute the script via telnet....

Code:
perl check.cgi > check.log
This will create a log file of the results.

Regards,

Eliot Lee
Quote Reply
Re: Verify Records Exist In reply to
If the only difference is the size of the .pass file, that could be what the problem is.

I just got another idea that might make things work a little faster.

Replace your subroutine with the following:

Code:

sub html_verify_accounts {
# --------------------------------------------------------
&html_print_headers;
$page_title = "Accounts Without Records";
&html_page_top;
open (DB, "<$db_file_name") or &cgierr("unable to open database: $db_file_name.\nReason: $!");
@lines = <DB>;
close DB;

foreach $line (@lines) {
if ($line =~ /^#/) { next; }
if ($line =~ /^\s*$/) { next; }
chomp ($line);
@values = &split_decode($line);
$user{$values[$auth_user_field]} = 1;
}

open (PASSWD, "<$auth_pw_file") or &cgierr("unable to open password file. Reason: $!\n");
@passwds = <PASSWD>;
close PASSWD;
foreach $pass (@passwds) {
@data = split ":",$pass;
unless ($user{$data[0]}) {
push (@list,$data[0]);
}
}
unless ($list[0]) {
print "All users have records in the database";
}
else {
print "The following users do not have records in the database:<BR>";
foreach $user (@list) {
print "$user<BR>";
}
}
&html_footer;
&html_page_bottom;
}
This way, it only reads and the .db file once instead of 1600 times! Smile


JPD
Quote Reply
Re: Verify Records Exist In reply to
WOW - That did it ! It also now returns the records in less than 3 seconds. It must have been the size of the pass file?

Once again - THANKS CAROL !!! :-)
-------------------------
donm

Quote Reply
Re: Verify Records Exist In reply to
You're welcome. Smile

Yeah, I'm sure the problem was because of the size of the .pass file. With the old code, for each user, the database would open, split-decode each line and compare the userid field to see if it belonged to the user. As I said, that meant that the .db file was opened 1600 times. And, if there was a record for each user, there would be 1600+1599+1598+1597...+1 times that split/decode was invoked. That takes a lot of time!

I've just recently started thinking along the lines of hashes more than arrays, which is why the new code works so much faster.

This code goes through the .db file once and creates a hash like $user{'username'} and sets the value of the element to 1. Then it goes through the .pass file once, checking each one to see if there is a hash element that matches the .pass file username. That means that, with 1600 names in the .pass file, it looks at a maximum of 3200 file lines. Much quicker! Smile

I keep learning all the time. Sometimes I think I should delete some of the old stuff. I'm almost embarrassed that I wrote some of these things before I knew what I was doing. Blush Then again, it's possible that in a couple of years I'll be embarrassed about the stuff I wrote today. Smile

JPD
Quote Reply
Re: Verify Records Exist In reply to
Nice catch, JPDeni....

Codes work great.

Regards,

Eliot Lee
Quote Reply
Re: Verify Records Exist In reply to
Thank you, Eliot.

There's almost always more than one way to skin a (oops, my feline friend is sleeping on the couch. If I finish that line, she just might wake up and be offended! Wink)



JPD