Gossamer Forum
Home : Products : DBMan : Customization :

NEW MOD: Count Records in DB After xx Minutes

Quote Reply
NEW MOD: Count Records in DB After xx Minutes
I have created a new mod based on the 'Display # of Records' Mod. This new mod will count the number of records after a specified time ($update_time). This will reduce the load on your server and the time it takes to load the page with the &num_records on it. It reads the count from a file instead of counting every record in your DB again and again, (oh yeah - and again).

$update_time should depend on how busy your DB is and how exactly accurate you want your count. It only counts when someone goes to the page that calls the &num_records AND if time is greater than $update_time. I have mine set to 60 minutes. I know I have about 50 visitors an hour that will go to my database and search, but maybe only 2-4 an hour add an entry. This means after the first visitor activates the NEWCOUNT, the next 49 people won't have to wait for a count because it's already in count.txt, and I have over 25,000 records in my DB!

To make this work, create 2 new files called time.txt (put 955955955 in it) and count.txt (put 0 in it). Upload them in ASCII where your db.cgi is in. You DON'T have to chmod them.

In default.cfg add:

Code:
$update_time = "1800";
# Number of seconds between update of count.txt
# 1800 = 30 minutes, 3600 = 1 hour, 14400 = 4 hours

In db.cgi add new subroutines (REPLACE 'sub num_records' if you have the 'Display # of Records' Mod installed):

Code:
sub num_records {
#-----------------------------------------------
$current_time = time();

#open last time records were counted
open (LASTTIME,"<time.txt") or &cgierr("error in num_records. unable to open database: $db_file_name.\nReason: $!");
if ($db_use_flock) { flock(LASTTIME, 2) or &cgierr("unable to get exclusive lock on $db_file_name.\nReason: $!"); }
$last = <LASTTIME>;
close (LASTTIME);
$doit = $current_time - $last;

#check if it's time to count again
if ($doit > $update_time) {
#if so, overwrite time.txt with new time
open (NEWTIME,">time.txt") or &cgierr("error in num_records. unable to open database: time.txt.\nReason: $!");
if ($db_use_flock) { flock(NEWTIME, 2) or &cgierr("unable to get exclusive lock on time.txt.\nReason: $!"); }
print NEWTIME "$current_time";
close (NEWTIME);
#run sub to count records
&thecount
}
#opens count file and retrieves count
open (COUNT,"<count.txt") or &cgierr("error in num_records. unable to open file: count.txt.\nReason: $!");
if ($db_use_flock) { flock(COUNT, 2) or &cgierr("unable to get exclusive lock on count.txt.\nReason: $!"); }
$count = <COUNT>;
close (COUNT);
print $count; #prints count from count.txt
}

sub thecount {
#-----------------------------------------------
my $count = 0;

#opens db and counts records
open (DB, "<$db_file_name") or &cgierr("error in thecount. unable to open database: $db_file_name.\nReason: $!");
if ($db_use_flock) { flock(DB, 2) or &cgierr("unable to get exclusive lock on $db_file_name.\nReason: $!"); }
LINE: while (<DB> ) {
foreach (LINE) {
++$count;
}
}
close DB;

#opens count file and replaces old count with new count
open (NEWCOUNT,">count.txt") or &cgierr("error in thecount. unable to open file: count.txt.\nReason: $!");
if ($db_use_flock) { flock(NEWCOUNT, 2) or &cgierr("unable to get exclusive lock on count.txt.\nReason: $!"); }
print NEWCOUNT "$count";
close (NEWCOUNT);
}

In html.pl add where you want to show the number of records:

Code:
|; &num_records print qq|

Anything to take the stress off of a server helps!

Have a day!
AJ

[This message has been edited by TheFew (edited May 02, 2000).]
Quote Reply
Re: NEW MOD: Count Records in DB After xx Minutes In reply to
Excellent!

Be sure that you submit this to the Resource Center so it won't get lost in the shuffle. I'm sure a lot of people will want to use it.


------------------
JPD






Quote Reply
Re: NEW MOD: Count Records in DB After xx Minutes In reply to
To make sure I understand this correctly, if someone searches the database and it's been more than xx seconds/minutes, then a count is made? Makes sense. Smile

Another variation of this is whenever a record is added or deleted from the database, the count of records would be updated. That way, if no one changes the database for days on end, you'd save that much more bandwidth by never updating it over those days when the database isn't changed.

Great idea! Good job!!

--Lee
Quote Reply
Re: NEW MOD: Count Records in DB After xx Minutes In reply to
Definetely a good idea. I'll see about adding the code for this and the people that want the mod can choose how they want it to update the count.txt! I'll post it here in this thread when I finish it.

AJ
Quote Reply
Re: NEW MOD: Count Records in DB After xx Minutes In reply to
Cool...its nice mod.
I think it would be nice if the hits counter could do the same thing as well

Smile

Quote Reply
Re: NEW MOD: Count Records in DB After xx Minutes In reply to
Johnsonny, what would you like the hits counter to do?


------------------
JPD






Quote Reply
Re: NEW MOD: Count Records in DB After xx Minutes In reply to
 Smile
Actually,i would like to do the same thing like "Count Records in DB After xx Minutes mod" for the hits counter. But it would be a bit more.

Let me explian a bit clear. The top ten hits counter mod basically depends on how many times of specific page like html_long_record, people have viewed. But it seems like its not so accuracy somehow because user can go "front and back" to view taht page in order to increase the "hits" of the counters. So,it would be nice if the hits counter could do the same thing that "Count Records in DB After xx Minutes" mod have done. But it might be a bit complicated because i would like the hits counter will be considered as "a hit" per a person wthin certain period. For example, if i set the hits counter to count the hit of a record within 24hours, when a person search for that record, it would count "1 time". However within 24 hours, if that person view that record again, it wont count for it again. Of couse, if the other person view the same record,it would still count for it but again, still "one person one hit within 24hours"
Is it possible to do it??

Thanks Smile
Quote Reply
Re: NEW MOD: Count Records in DB After xx Minutes In reply to
Well, that would mean that the hits counter would have to be completely scrapped and rewritten, if it could be done at all.

First, it would require that all users who view records be logged in. Is this the case in your database? Or do you allow default users to view?

Second, instead of the quick incrementing of the counter, it would require adding the userid, the date and the time to the hit counter file, after checking to see if this user had already viewed the record and, if he had, when.

It would slow down things considerably, especially with your top ten list -- which would also have to be completely rewritten.


------------------
JPD






Quote Reply
Re: NEW MOD: Count Records in DB After xx Minutes In reply to
I believe it's possible, but it will need brighter brains than mine. Smile

As I understand it, it would require the use of session cookies. In a single session, when a user views a record for the first time, a cookie is written that says that particular record has been viewed. If they hit reload or browse the record again in the same session, a cookie check is made, and if that particular record has been viewed, the counter isn't incremented. But, any other new record which hasn't been viewed yet would still incriment.

This can also be done over a time period; all you'd do is create a cookie and make it expire after the time delay you'd like. It would work the same, but instead of the cookie clearing when they close and restart their browser, it would stay active until the expiration date.

I'll look into it; I've never been terribly creative with cookies. But if not me, perhaps oldmoney or another code guru can riddle this one out...

To the Mystery Machine!!

--Lee
Quote Reply
Re: NEW MOD: Count Records in DB After xx Minutes In reply to
I guess I never think about cookies because I don't work with them. Also, not everyone has cookies enabled in their browsers. I'm not sure if a non-enabled user would increment the hit list multiple times or not at all.


------------------
JPD






Quote Reply
Re: NEW MOD: Count Records in DB After xx Minutes In reply to
How about creating an ip.txt file and write the IP address of the visitor and the record number that they looked at. The file would look like:

IP | Record | Unix Time
129.1.4.56|179|957416259
158.96.252.41|26|957417789
129.1.4.56|15|957417905

Note that 129.1.4.56 is listed twice, but the person viewed different records. Then if they look at it again within a certain amount of time, the program will check the ip.txt file for matching information of the IP address and record number. If a match is found, don't add to the count. If it does match, count it as a hit on the record. After the certain amount of time, the script would delete that entry from the ip.txt.

I know it can be done. I have the topsites.cgi program with the "no-cheat" addon. The no-cheat creates an ip.txt file that records when someone votes for a site on the topsites list. If they try to vote again within 2 hours, it won't record the vote. After 2 hours (what I have it set for) it goes and removes that IP address from ip.txt.

Better and more reliable than cookies! Anyone up to coding it? I may try in a day or 2. I'm kind of swamped right now!

Semper Fi,
AJ
Quote Reply
Re: NEW MOD: Count Records in DB After xx Minutes In reply to
Go for it. I've been trying to get something else accomplished for several weeks now and I'd really like to get it done. Smile


------------------
JPD






Quote Reply
Re: NEW MOD: Count Records in DB After xx Minutes In reply to
Thanks for all of sugguestion.

The sugguestion from TheFew would be pretty nice if it can creat an ip.txt file and write the IP address of the visitor and the record number that they looked at.

Could any one know how to do it??

Smile
Quote Reply
Re: NEW MOD: Count Records in DB After xx Minutes In reply to
JPDeni, leisurelee, johnsonny, and others...

I have created a new thread for the ip record counter/top 10 mod that we have been discussing above. Check it out at:

http://www.gossamer-threads.com/...m12/HTML/002878.html

Also, there is a better "Count DB Records" Mod I made at this thread: http://www.gossamer-threads.com/...m12/HTML/002855.html Everyone should use it instead of the one at the top of this thread.

Semper Fi,
AJ