Hello y'all...

I was wondering what things people have done to limit SPAMMING of their databases. I have installed the following in one of my databases that will be publicly accessible for people to register and contribute data:

1) Bad Link Filter Mod
2) IP Address Stamp (Still working on it)

Do you have any other ideas or mods that would help to limit SPAMMING? (One flaw in DBMAN is that for public databases, it lacks the verification process available through LINKS. But, the nice thing about DBMAN is that it allows adding, deleting, and modifying records on-the-fly.)

One idea I have is to create a temporary database that will hold temp records submitted by potential employers or web visitors. An email message would be sent to the administrator notifying them that a record is waiting validation. Then the administrator can validate records. This would happen similar to how records are added into the database in LINKS.

I like the features of DBMAN over LINKS in terms of Classified Ads, so that is why I am posting this message in DBMAN Discussion Forum to hear people's ideas. I am hoping that Carol is a little more understanding than some other moderators in this site who tend to aggressively moderate discussions related to cross-over ideas between LINKS and DBMAN.

Thank you.

Best regards,

------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us

[This message has been edited by Eliot (edited August 06, 1999).]

[This message has been edited by Eliot (edited August 06, 1999).]

I'm not sure what you mean when you refer to "spamming." Do you mean people who add massive numbers of bogus data? One thing I do is require login of everyone and, with using the password lookup mod I have the email address of everyone.

Regarding verification (which I think is the same thing that I think of as validation of a record), I think I mentioned to someone else that I have written some validation code. The problem is that there are so many possible permutations of options that I would have to write a mod for each one. I just haven't had the time.

What my validation routine does is to set a field -- "Validated" -- to "No" when a record is added by anyone other than an admin. This setting is done within db.cgi, so the user can't just change the html. I have a similar thing in sub view_records, setting

$in{'Validated'} = "Yes";

if the user is not an admin, so that only validated records are returned from a search.

There's an "admin only" link in the footer -- Validate -- which automatically does a search for non-validated records and results in a form. Here's where the different permutations take effect.

If I want to either send canned email to those whose records are validated or deleted or I don't want to send any email, there are two radio buttons -- Validate and Delete -- next to each record. I select one of those and click one button to mass validate or delete records in one fell swoop.

If I want to edit the email before it gets sent out, I have a radio button for Validate and a radio button for Delete. There are two buttons at the bottom of the form -- to delete the selected record or validate the selected record. Either way, after the procedure is done, there is a form with a textarea to write an email to the user. I usually have some preliminary text in there that I can edit if I want to.

There's another permutation where validation of records allows editing of the email but deleting does not. (Allows for mass deletes, instead of one at a time.) And there could also be editing of the email for deleted records, but not for validated ones, which would allow for mass validations.

(See why I haven't written a mod yet?

You could do the temp .db file thing if you wanted to. It might be more work, though.

I think it's very valuable to discuss crossover ideas between Links and DBMan. Each can take features of the other. The only thing I don't want is folks asking why they had problems building pages in Links, or something that is solely connected with the other script.


------------------
JPD

Carol,

I totally forgot about the email address for registering to my database. That is another way. I did mean posting negative or mass postings as "SPAMMING". Although that does not protect the database 100% because people could use fradulant email addresses.

Basically, I would like to apply the "Permission" type of mod that you assisted me with in terms of posting email addresses or using the send_email_form with Validating the records. I would like the records added to the database but only viewable in the search results when I select the Validate button.

But I was unsure if I needed to write a lot of other code rather than copying the Permission codes.

If you or anyone can assist me with this process, I would greatly appreciate it.

Thanks so much for allowing me to post this discussion in your Forum. Some people are a bit snotty about DBMAN. It seems to be an inappropriate word over at the LINKS forums. I really think that people should look more globally and see what potentials there are to integrate aspects of each script into a killer program that accomodates different needs.

Regards,

------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us

[This message has been edited by Eliot (edited August 06, 1999).]

I completely agree about open discussions about the two scripts. Of course, I haven't even visited a Links forum for several months, so I'm not sure what's going on over there. But each has benefits that the other doesn't have and I've sure stolen a lot of code from Links for DBMan use!

Regarding fraudulent email addresses, my new password lookup mod prevents fraudulent email addresses by generating a password that is sent to the email address. If the address isn't any good, they won't get their password and can't add records.

I've been thinking of trying a "banned email" and even a "banned domain" mod. It's still in the inner recesses of my brain, though, so it will be a while before it's ready.

Can you give me a detailed description of what you want to do?

------------------
JPD

Regarding the security issue -- you said that you might want a previously "bad" user to be able to still view records, but not to be able to add. You could easily change his permissions in the Admin page. Then you wouldn't need to deal with a mod at all.

Regarding validation -- Looks like you have issues 1 and 2 covered.

It's best, I think to have the "Validated" field be a radio field and not a checkbox. That way there is something in the field no matter what.

Set up the field to default to "No."

In sub add_record (db.cgi), after


add


That keeps the user from being able to change the value of the field by downloading the html.

In sub view_records (db.cgi), right at the beginning, add


That allows only validated records to be seen in a search.

By your description of #4, it seems that you want to send out a "canned" email -- one that you won't be editing before it goes out. Let's see if I can figure out a way to give you pointers without giving you the whole code.

Make a subroutine in html.pl, sub sub html_validate_form.

Add

$in{'Validated'} = "No";

to the beginning of the subroutine.

Then copy html_delete_form and paste it into your new subroutine.

In the place where you print out the "delete" checkbox, change that code to


This will give you radio buttons by each record that you can use to decide whether you want to validate or delete the records.

Change the submit button "name" to

"validate_records"

Copy sub html_delete_success and paste it, changing the name to html_validate_success. (You'll want to change the wording, too.)

In db.cgi, sub main, add an "elsif" statement, for $in{'validate_records'}, sending the script to &validate_records (don't forget the "s" at the end of that, or it'll go to the wrong subroutine!) if the user has admin permission.

sub validate_records is a combination of sub_delete_records and a little bit of sub modify_record. You wanna give it a try? I can just copy the code I have in a script, if you'd like.

This is where you would also send out your email messages.

At the end of sub validate_records, send the script to &html_validate_success, along with a $message variable that lists the keys of the records validated and deleted.

This should get you started. Let me know if you have any problems or don't understand what I mean by my instructions above.



------------------
JPD







[This message has been edited by JPDeni (edited August 07, 1999).]

Carol,

I did get your new password lookup to work. You brought up another good point...If they use a fraudalent email then they will never gain access. That is a nice feature. It will take some getting used to with opening a database to the general public. As you know, I have been working on other databases using DBMAN where no one except myself as Administrator and Data Managers are able to add, modify, or delete records. So, this is something new. But I am glad that I installed your new password mod.

The email ban would be something nice, too. Let's say that a "registered user" signs up with a legitimate email address, but chooses to post mass records that do not pertain to my database, which is "Classified Ads". It would be nice to add a mod that would ban the user from posting ads in the future, but to not be a total jerk, still allow the individual to view ads in the database.

I apologize...I think I should have created two Topics/Threads since these are two totally separate topics (1) Enhancing Security of DBMAN, and 2) Validating Records). I did search DBMAN Discussion Forum and I could not find any topics that directly related to validation of records.)

What I would really like to concentrate on is the validation of records mod....

Here is what I'd like to see happen:

1) Registered User is able to add, modify, and delete ONLY their ads. (I have set this up properly and it is working just fine.)

2) An email is sent to me to notify that a new record is waiting to be validated. (I have this working just fine, but I will most likely have to modify the print statements to put a message that pertains to this step.)

3) The record is put on "hold" until I, as the administrator, add the record by clicking on a Validate checkbox. When I click on the checkbox and submit the form, the record is open for public viewing.

4) An email message is sent to the registered user notifying them that their record has been "added". (Again I have the email notification working.) Added in terms that their record is now viewable.

That should be about it. If I can get this to work, I will totally forget about using LINKS for this particular project.

Any suggestions or code snippets would be very appreciated. I would not bother writing a completely different mod for me, just point me in the right direction (that way I can learn more about Perl).

BTW: The LINKS Discussion Forum is a very scary place these days...You don't want to visit it. It pales in comparision to your Forum, which is a nice place to hang out and get help on problems. That is all I can say without flaming people.

Thanks.

Regards,


------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us

[This message has been edited by Eliot (edited August 07, 1999).]

Thanks, Carol.

I will try your suggestions. This is the amount of coding I expected. (Not more or less than...a good thing.)

I may need assistance with the sub validate_records, but let me tinker with the script first to see if I can do it.

I will let you know either way if I can get this going.

Thanks again for taking the time to provide me those pointers. They are enough to get me started.

Regards,

------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us

JPDeni,

Hi there. Well, I added in your suggestion. And it doesn't seem to work. One of the bugs I found was that when I went to the Browse Categories page, the "new" record showed.

Also, I was unable to find the new record to validate.

I guess I may be missing a small, but a big part of the puzzle...When you mentioned adding a field known as Validate as a radio button. Did you mean that I add another field in the %db_def?? I did add Validate as a FieldName into the %db_radio_fields.

Also, would you be willing to post the &validate_records sub-routine. May be that is another source of the problem. What I did was copy the delete_records sub routine and I replaced the delete variables with validate.

Example:


TO


Most of the web forms (including validate_search and validate_form) are coming up fine, which means that the problem has to be with properly setting up that "Validate" field and the validate_records sub-routine.

Thanks for your continued support and guidance.

Regards,

------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us

  

------------------
JPD







[This message has been edited by JPDeni (edited August 07, 1999).]

Thanks, Carol for the codes. I think that is all I am missing. I figured out that I need to add a Validate field in the db_def
array. I did edit links in the Public Main Menu and sub html_footer sub-routine for the List All and List New links. When I added &Validate=Yes, none of the non-validated records showed. Also in the search form, I attempted to search for records recently added, but not validated and I got no search results. I will also add that code to the Browse Categories sub-routine (browse_cat).

All in all, I think it is the &validate_records sub-routine for the default.cgi file that I am missing.

It is coming along very nicely...thanks to you.

Update: Hmmm...It is not working. I will review my html.pl tomorrow and see if I can't debug it.

Also, I don't know if I mentioned that I created two other sub-routines:

1) validate_search - a copy of delete_search with changed codes to reflect validate_records versus delete_records.

2) validate_success - a copy of delete_success with changed codes to reflect response for successfully validation of records.

Could the addition of the first sub-routine conflict with the validate_form sub-routine??


Regards,

------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us

[This message has been edited by Eliot (edited August 08, 1999).]

[This message has been edited by Eliot (edited August 08, 1999).]

Thanks Eliot nice thread and should prove usefull towards info on my Links2 and DBman Mods on the other side of the tracks at:
http://www.gossamer-threads.com/scripts/forum/resources/Forum3/HTML/002677.html
Titled:
"Has anyone Mod Links2 to be updated using DBman on the User side for Modify, etc. "

This manner of posting seems to have worked nice for both Links2 and DBman "cross-mods"

TimRyan

TimRyan,

No problem. I think you are right. Cross-over mods are best discussed in their primary forum, in this case, DBMAN Discussion, but then can be referred in other forums. That way the Threads don't get compacted with "useless" discussions. They can stay focused and on-task.

JPDeni,

Hmm...I've looked over the validate_** sub-routines in the html.pl file, and I can't see anything wrong with the coding. I think that there may be some bugs in the default.cgi file since when I do try the validate_search, the information passes to validate_form sub-routine, but no records show. Also, I only get Error with no error message.
That is odd. I'll keep plugging away at it.
If you see any syntax errors or possible changes to the validate_records sub-routine that you provided, please let me know.

Thanks.

Regards,

------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us

I copied sub validate_records directly from a script I'm using. The only changes I made were that I added the comment lines about sending emails. (The one I copied it from doesn't send emails at all.)

I hadn't seen some of your comments in earlier posts.


That is completely unnecessary. All you need is a link like this:

<A HREF="$db_script_link_url&validate_form=1">Validate</A>

sub html_validate_form should do the searching for you.

I don't know if it would conflict, but I do know it's unnecessary.

I do get a little confused because you have changed some file names and I'm used to the old ones. It took me a while to figure out which file was default.cgi, for example.

It may be best if I start another thread and just post all of the subroutines there that are working for me. Then you can compare what I have to what you have.


------------------
JPD

That may be a good idea to start another thread, but may be reference that new thread in this thread, so that people can refer to both threads (if that is not too much trouble). I am sorry about the name change. The reason that I use index.cgi rather than default.cgi is that my server only allows the following default files:

index.cgi
index.html
index.htm
index.shtml
index.shtm

Sorry about that. I will try to take out the validate_search and change the link for validate_form to see what that does.

Thanks.

Regards,


------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us

Another suggestion, Carol...(that may help Forum Participants) is using the Forum Announcement function in UBB to post a few announcements, such as "Unofficial Mods" or "Official Mods". You could post links to unofficial mods that you or other programmers have worked on. These could be links to good Threads that outline functional mods. The Official Mod Announcement could be a referral to the Modification section of the Resource Center.

Forum Announcements are a great way to post common resources, information, etc. that will assist Forum Participants and Moderators to keep track of information more efficiently and effectively.

Also, since Forum Participants cannot respond to the Forum Announcements, it will keep a clean post for participants to refer to.



Regards,

------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us

[This message has been edited by Eliot (edited August 08, 1999).]

I was thinking about something like that. I'll ask Alex if he thinks this is a good idea.


------------------
JPD

Carol,

Okay...I am about to hit the wall. I gave it my best effort. I did notice that I needed to change the "Validated" variable in &validate_records you specified to "Validate" since that is the variable I am using in both index.cgi (default.cgi) and html.pl. I have saved the following files on my server. When you get a chance, I would appreciate your assistance in de-bugging them....

Complete index.cgi (default.cgi):

http://anthrotech.com/cgibin/classifieds/indexcgi.txt

Partial index.cgi (default.cgi):
(ONLY Includes the validate_records sub-routine)

http://anthrotech.com/cgibin/classifieds/validate_records.txt

Complete html.pl:

http://anthrotech.com/cgibin/classifieds/htmlpl.txt

Partial html.pl:
(ONLY Includes the html_validate_form sub-routine)

http://anthrotech.com/cgibin/classifieds/validate_form.txt

Thanks a lot.

Regards,


------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us

I'm not really sure what I'm looking for, since I'm not really sure what's going wrong. However, I saw something that might be a part of it.

You said:

So the field, as defined in your .cfg file is Validate? If so, you need to change it in sub html_validate_form, too.

That's all I see.

Are you getting syntax errors?

------------------
JPD

Hi, Carol.

Ah, ha! I see the error. I did not change the "Validated" to "Validate" in the validate_form. I guess I needed a second set of eyes to look at the script.


However, it still is not working. Here is the output I receive when I click on Validate Records link:

http://www.anthrotech.com/...ssifieds/output.html

Hopefully, this will shed some light.

Regards,



------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us

Remember when I said, several posts ago that you needed to copy sub html_delete_form? You didn't.

You need to add:


After

$in{'Validate'} = "No";

in sub html_validate_form. That's why you're not getting anything. It's not doing the search.
----------
JPD

[This message has been edited by JPDeni (edited August 08, 1999).]

Yea!



Finally! One of the things I am going to have get used to is that when I add new mods, they may or may not interact with older records in the database. One of the other problems was that I thought that the records currently with "No" as their validation default would show up, but since I changed the add_record process and added the valdiate_records, it ignored all the old records. I just added a dummy record and the process worked just fine!

Thanks again for your help, Carol. We all need a second set of eyes to exam and debug scripts.

Anyway, there are some things that I need to clean up such as the Browse Categories listing. But I won't be able to re-visit this project until next weekend. It is back to the grind at my real job....We are working on another DBMAN based project...our employee directory. I don't think that it will be as complicated as the Classified Ads.

I may pick your brain if I come accross any problems next weekend...or earlier if problems arise with the employee directory.



Regards,

------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us