Gossamer Forum
Home : Products : DBMan : Customization :

CGI error

Quote Reply
CGI error
I am having an error problem and wonder if anyone else has ever had the problem and could offer any suggestions. The database works fine all day, then sometime during the night someone fills out the add_form page and takes the database offline. The error that is sent after it goes down is:


"CGI Error
The specified CGI application misbehaved by not returning a complete set of HTTP headers. The headers it did return are:"


It doesn't say anything else - I have the debugging turned off so the students don't see it.

Then to fix the problem, I have found all I have to do is replace the default.db file with a new empty file and it all works again. I have tried to set the database up again from a clean unaltered copy of dbman 3 times, but the same problem occurs. I have other dbman databases running on the server and they run just fine. I don't know what I could have done differently.

It is a student evaluation form - and it goes down every night after midnight but before 7am. I tried to think of a time reason for it, but I can't - I don't have scheduled back-ups during the week.

Any information would be greatly appreciated. I will continue to search through the archives here.

Thank you in advance for any thoughts.

Amy Shelton
Quote Reply
Re: CGI error In reply to
Amy, this is, without a doubt, the oddest problem that I have seen with DBMan. You win the prize!!! Smile

It's very good that you have other databases running that don't have this problem. It allows for some checking.

Is there anything at all in the problem database that isn't in any of the others? Or anything the problem database doesn't have that all the others do?


------------------
JPD






Quote Reply
Re: CGI error In reply to
JPD,

Exactly my approach! What is different.

The only thing I can think of that is different is that the ID field is not required, but it is still there were the user can't see it to keep a count on the database.
I am only using the add_form, no one has any other rights except to add.

I didn't change the code much from the original. Just the fields in .cfg and the html to display the fields on the add_form. I removed the footer too, so it wouldn't print any options at the bottom. (I am using your user friendly version of html.pl)

Authentification is not required and flock is on. The students access the page through a URL http://venus.atlantic.edu/cgi-shl/survey/db.cgi?add_form=1

They fill in the form and submit it. They then go to the add_success page that I changed the html on. Then they go to a second database to add an entry for a drawing.

I didn't think any of that stuff would bring it down, but maybe it would.

I'm sure I will have to go in in the morning to clear out the file again. I am thinking on it tonight and sending test forms in via different ISP's to see if one of those is knocking it, like AOL (no offense, I just have a lot of problems with students using AOL) in combination with IE and Netscape browsers. I've set the database up four times from scratch.

Thanks for your reply. I was kinda hoping someone had this problem before.

Amy

[This message has been edited by shelton (edited April 07, 2000).]
Quote Reply
Re: CGI error In reply to
 
Quote:
I have found all I have to do is replace the default.db file with a new empty file and it all works again.
Just to make sure I follow, are you first checking the default.db file before replacing it to see if there are any suspiscious entries?

I read some warnings just recently that one of the 'hottest' methods of hacking is corrupting programs through mischievous form entries. I think it had to do with entering JavaScript snippets and things that may or may not be visible after the fact, but would wreak havoc upon processing. The suggested methods of combatting this included setting maxlength's on input areas as short as possible (i.e. 15 characters to not allow more than a <script></script> ) and using proper field validations.

Hope this helps,
Dan
Quote Reply
Re: CGI error In reply to
Thanks Dan.

Yes, I am checking the content of the file first. I will double check and look for the javascript, I didn't see it the first time, but I will check again.

Maybe someone is passing code and changing the permissions on the file. I never thought to check this. Thanks!

I will do as you suggest. I will check all the maxlengths and the validation, but I think I did minimize them as much as possible and there are two text boxes, so snippets of code can be still be put in.

Amy
Quote Reply
Re: CGI error In reply to
Hi Amy,

You're welcome. No guarantees that's the answer, but it couldn't hurt to keep an eye on. It turns out I still have the full article I was referring to above. Here it is, reprinted from the Jayde.com newsletter:

Quote:
Does Your Site Have 3 Basic Security Measures? By Gim Yeap

In recent weeks, attacks on prominent sites such as Yahoo and Ebay
have brought home a very pressing point - site security. Anywhere
you have a dynamically-generated page, you could be open to attacks
where malicious HTML is embedded into your pages. Your pages could
be rewritten to substitute your customers' names with "Dummy." Or,
credit card information could be intercepted and sent to a secret
depository for later use.

What can we do about this?

There are many methods by which a hacker may attack or take control
of a site. I am focusing this discussion on attacks that come via
form input. That is, anywhere you have input coming in from your
web user, e.g. a registration form, user login or even a search on
your site. Scripts could be sent to your server by entering
< script> some malicious code < /script> in your input fields.
The following are steps you can take to minimise the risk of
this happening. These measures will not make your site hacker-proof
(no site can be if a hacker really has it in for you), but it can
make it less of an easy target.

STEP 1: PLACE CHARACTER LIMITS ON YOUR INPUTS

You do this by adding the "maxlength" attribute into your text
input tags.

e.g. < input type="text" name="firstname" maxlength="15">

The example above restricts the user to a 15 character input for
that field. The "< script>" and "< /script>" tags alone will take
17 characters so the smaller you limit your "maxlength" attribute
to, the harder it will be to include rogue codes in your inputs.
Of course, you must ensure that you impose a suitable limit so that
actual input from your valid users will not be excluded.


STEP 2: FILTERING YOUR DATA

All data received from your site should be filtered, you can either
filter your data when it comes into your server as user input, or
when it goes out as results for your user's browser. Whether you
should filter input or output, depends on your site and its
requirements, there is a good discussion on this at:
http://www.cert.org/tech_tips/malicious_code_mitigation.html/

Filters can be written in any language, here is an example in Perl:

# This function checks the input, $firstname, for the following
# symbols ;<>?*/'&$!#()[]{}:"'
# and tells the user to re-enter his/her firstname if any of the
# symbols is found.

if($firstname =~ /([;<>?\*\/'&\$!#\(\)\[\]\{\}:'"])/) {

print p('Invalid input found, please use only alphanumerical
input. Please re-enter your FIRSTNAME');

}

You can see this script at work on our site :
http://www.payingads.com/freesignup.html .


STEP 3: SETTING THE CHARACTER ENCODING

Some HTML editors already set this while it creates a page, but
those of you who have older HTML editors or like me, like to code
the page from scratch will need to include the following line in
our HTML pages:
< META http-equiv="Content-Type" content="text/html;
charset=IS0-8859-1">

It should go as high as possible on your webpage, I normally place
it just after the < /head> tag, before the < title> tag. This META
tag tells the browser to use the "ISO-8859-1" character set, which
is suitable for most Western European languages, rather than let the
browser choose it's own character encoding, which may or may not be
ISO-8859-1.

Why is it important to explicitly set it?

The character encoding basically tells browsers how to display a
particular character. For example, in the ISO-8859-1character set,
"&#65" represents the letter "A" while "&#169" represents the
copyright symbol "©" (You can try this out by typing < p>&#65< /p>
or < p>&#169< /p> in a html file then call it up on a browser).
Some character sets, have more than one representation for special
characters such as "<" or ">", so your filter program may not toss
out all the representations of the character you have asked it to
exclude. So when it serves a new page back to the browser, the
browser, because it has not been told what encoding to use, can
still read the malicious script intact.

So there you have it, 3 steps that should be incorporated into
every website. Use them as a base to further build on. Because
every site is different, you (or the security consultant you hire)
will need to assess your site's own vulnerabilities and implement
appropriate security measures. To do this you need to take into
account your site's risk factor, your budget and your available
resources.

On a final note, I'd like to stress the importance of keeping up
with the latest threats and developments in site security. A good
site for checking out security alerts is the CERT Coordination
Center http://www.cert.org/nav/index.html or better yet sign up
for their Security Advisory that is sent via email.

=============================================

Gim Yeap is host and founder of PayingAds.com, an online trading
post for webmasters looking for paid advertising for their sites
or wanting to sell ad-space. Gim is also editor and writer for the
site's free e-zine, The PayingAds Gazette, a newsletter for
webmasters and online marketers.

=============================================

Expert in your field? We're looking for ARTICLES on web site design
and development, promotion, positioning, and e-commerce. Articles
should be in the 700 to 800 word range and include a 2 line promotion
of the author's site. Send submissions to: mailto:info@jayde.com

=============================================

Dan
Quote Reply
Re: CGI error In reply to
Couldn't help but to respond to make a small correction regarding META tags Smile


quote:
---------------
< META http-equiv="Content-Type" content="text/html; charset=IS0-8859-1">

It should go as high as possible on your webpage, I normally place it just after the < /head>

--------------

META tags should be placed WITHIN the header tags after <HEAD> and prior to </HEAD> to be read properly by browsers and search engines.

Quote Reply
Re: CGI error In reply to
Just to let anyone interested know, I found the problem and hopefully fixed it (knock on wood here).

What I believe was the problem is this:

When I use the 'ID' field to count the records, but I don't make it a required field - the following portion of db.cgi has to be changed:

# We keep checking for the next available key, or until we've tried 50 times
# after which we give up.
while ($status eq "duplicate key error" and $db_key_track) {
return "duplicate key error" if ($counter++ > 50);
$in{$db_key}++;
$status = &validate_record;
}

If we leave it like it is then when it gets to record 52 it gives up and won't add any more records because it has reached its limit of 51.

so I changed the one line to:


if ($counter++ > 1000);


and it seems to work just fine. I don't anticipate getting any more than 900 records, so I hope this works.

Thanks for all your suggestions!

Amy

Quote Reply
Re: CGI error In reply to
Ah-hah! That would probably do it. It was just coincidence that it happened in the early hours of the morning. (That was my major confusion. I couldn't figure out what the time of day had to do with anything.)

Is your ID field your key field? It should be required so that the counter keeps track of the number of records.


------------------
JPD






Quote Reply
Re: CGI error In reply to
Yes, the ID field is the key field so that the counter will keep track of the number of records. I can't require it because I don't want it visible to the people filling out the form. I tried requiring it, but leaving it off the form, but then it returns "can't leave the ID field blank". So I don't require it, but it still keeps the count.

Yeah, the time of day was wierd - it made me think it was system related, not code related. Today I saw the pattern! It errored every time it had stored 51 records, so I made it do it, searched the code for '50' and fixed it. A light bulb went on!

Thanks! Have a great evening! I know I'm due a hot fudge sundae after this one!

Amy
Quote Reply
Re: CGI error In reply to
You can make it a required field, but make it hidden on the form:

Code:
<input type="hidden" name="ID" value="$rec{'ID'}">

If you're using the auto generate feature, just set the field length to -1.


------------------
JPD






Quote Reply
Re: CGI error In reply to
OH! Never thought of that, though I use the hidden fields on web pages that query the database. I added it to html_add_form with the rest of the hiddens.

Thanks! :-)

Amy