Gossamer Forum
Home : Products : Gossamer Links : Pre Sales :

Urgent MySQL Problem...

Quote Reply
Urgent MySQL Problem...
Links has been working great with MySQL until today... when trying to access anything to do with MySQL I get the following error....

Software error:
DBSQL (25808): Fatal Error: Unable to connect to the SQL server. Reason: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (111) at /home/sites/site1/web/cgi-bin/links/jump.cgi line 65
For help, please send mail to the webmaster (admin), giving this error message and the time and date of the error. [Tue Jan 23 15:33:52 2001] jump.cgi: DBSQL (25808): Fatal Error: Unable to connect to the SQL server. Reason: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (111) at /home/sites/site1/web/cgi-bin/links/jump.cgi line 65

What do I have to do to fix this?

Quote Reply
Re: Urgent MySQL Problem... In reply to
Shut down and restart the mysql server. You might need to _KILL_ it, since you can't connect without the .sock file.

If you can't manage to _KILL_ the server and restart it, then you'll have to restart the physical server.

Obviously you are not on a virtual host, or you shouldn't be seeing that error -- your ISP has dropped the ball and their database server is dead :)

PUGDOGŪ
PUGDOGŪ Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Urgent MySQL Problem... In reply to
Okay, here is our problem. Bear with me :)

We were running 3.22.22 I think. Since I am remotely (in terms of location and hands-on work) involved with our server, etc. I was left out of the communication loop, and someone (supposed professionals) attempted to install the latest version (3.22.32 or something I guess) because the people at home weren't sure we had MySQL installed (wouldn't this guy have been able to tell upon trying to install over an old version?).

Ever since then, we've had these problems. Is it possible that we somehow have 2 instances of MySQL going? Or could it just be a password issue with MySQL (a new passsword that is different from the old password)? Or should I attempt a shutdown and restart -- may as well if it can't hurt, right?

If it matters, this is what happens when I try to access MySQL from telnet:

[root mysql]# mysql
ERROR 2002: Can't connect to local MySQL server through socket '/var/lib/mysql/)

People at home will be calling the appropriate pros tomorrow... I am just hoping I might be able to reduce downtime by getting things going in the interim.

If you think I might as well give restarting a try, would you be able to instruct me as to the steps to do this?

Thanks for the help!

Quote Reply
Re: Urgent MySQL Problem... In reply to
Are you on a RAQ ??

PUGDOGŪ
PUGDOGŪ Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Urgent MySQL Problem... In reply to
Yeah, Raq4. How'd you know?

Quote Reply
Re: Urgent MySQL Problem... In reply to
By the location of the mysql.sock file.

It's non-standard on a RAQ.

You need to backtrack what was done. You probably don't have two version running, but what probably happened is that the new install killed the old install but left the old install running.

Try _KILL_ ing the running server, and then restarting it. If you can shutdown and reboot the whole server, that should be even better over all.


PUGDOGŪ
PUGDOGŪ Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Urgent MySQL Problem... In reply to
How do I kill it and get it going?

I can REBOOT the server via the server administration panel... but might this cause any irreparable damage?

Quote Reply
Re: Urgent MySQL Problem... In reply to
No, it shouldn't. It should close all processes and restart. Any damage would have been caused by the new install over the old one. Without the mysql.sock file, the servera can't accept any connections, so you can try to kill any running mysql process -- you need to be root. Perhaps the admin panel (which I think is a modified version of Webmin) will let you list all running processes. If it does, you can selectively kill any that are 'mysql'.

You can't shut down the server if you can't connect. The only options are to _KILL_ it, or to reboot the whole machine. You can try to _KILL_ it first. If you can, then restart it, and see if it works. Usually, that is all that is needed.

PUGDOGŪ
PUGDOGŪ Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Urgent MySQL Problem... In reply to
What I meant was, can any damage be caused by rebooting the whole machine?

Also, what is the command to KILL?

Thanks for your help and patience.

Quote Reply
Re: Urgent MySQL Problem... In reply to
I had a problem getting the forum to respond for awhile. Don't know which end it was on.

Rebooting the server _can_ make problems that are already there evident, such as if the boot configuration is not correct. My feeling is that it's always better to find that out when you are planning on working on the server, than during a crunch time.

Can rebooting itself cause damage? No.

The big risk is that the server will shut itself down (only happened once to me) or that when the server re-boots, you'll find that your network configurations or other configurations were wrong (or temporary) and need to be reset. But it won't cause your system to explode or anything :)

I would suggest if the pros are looking at the system, make sure they reboot it so you know it can be rebooted.

_KILL_ is a command you can execute as root. If you have a command panel like in Webmin, you click the process and terminate or end it.

do a ps aux to see your running processes

Then, kill PID (whatever the number for mysql is) to stop that process.

Do a ps aux again, to make sure that it's really all died off.

Then restart it with safe-mysqld command as per your system.

PUGDOGŪ
PUGDOGŪ Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Urgent MySQL Problem... In reply to
Hi,

If you are still stuck try:

1. ps aux | grep mysql | grep -v grep

and see if mysql is running. If it is, you should see at least three entries there. If you don't see anything then mysql is not running.

2. /etc/rc.d/init.d/mysql start

This should start mysql running. If you see something like "Starting Mysql .. Mysql ended" then there is something wrong with how mysql is installed, however running:

3. safe_mysqld &

Could get it started ok.

You should definately have something monitoring your mysql installation and restarting it as neccessary. Also, I would make sure that your mysql is setup properly and not running as root, but rather as user mysql. This can cause problems when upgrading though if the old version was running as root (default install on old mysql's) and the new version is running as user mysql (default isntall on new mysql's).

You shouldn't need to reboot the server.

Hope that helps,

Alex

--
Gossamer Threads Inc.
Quote Reply
Re: Urgent MySQL Problem... In reply to
Alex,

I've had trouble trying to kill MySQl -- even as root. When the mysql.sock file is erased for some reason, MySQl can still be running, or in memory as a real, or locked process (at least with soloaris 7). The only way to get rid of it is to _KILL_ it (often multiple times). Why problems occur with the mysql.sock file, I don't know... but it seems one really big cause is the upgrade/install of a new version without taking down the old version first.

My suggestion to reboot was tempered by the reported problem as well as the history of the new install. Very often people forget to try a reboot, when they are expecting problems, and when they do it not expecting problems, all of a sudden everything they did since the last reboot doesn't quite come back properly.

As for "monitoring MySQL" doesn't the use of the start up scripts cause the original start up process to make sure the processes it started (such as send mail, etc) get restarted if they fail? This is another reason to reboot and make sure everything works.



PUGDOGŪ
PUGDOGŪ Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Urgent MySQL Problem... In reply to
Hi,

The error message 'Can't connect to MySQL throught socket '...'' 99% of the time means MySQL is not running.

If MySQL is running and you get this, then most likely MySQL has locked up, or less likely your mysql client tools are using a differt .sock file then your MySQL server. This can happen when you upgrade the server, but not the client tools (rpm upgrades), and things have changed.

To kill mysql, you can do a `killall mysqld` or to forcefully shut it down `killall -9 mysqld`. If you need the -9, then you should run isamchk to check your tables as the -9 is equivalant to just shutting the power off. MySQL has no chance to close tables and clean things up properly.

Cheers,

Alex

--
Gossamer Threads Inc.
Quote Reply
Re: Urgent MySQL Problem... In reply to
Thanks. I keep forgetting to add a kill level, since if you do that, as you pointed out, you can do some "damage" you have to try to correct later. After doing hard-kills, it's really a good idea to reboot to make sure all your services come back. It's possible to screw things up even in Unix, when you use kill.

So, to summarize slightly:

the first thing to do is check to see if MySQl is running ... (several ways to do that were given).

The next thing is to try to kill it if it is, since it's doing something unexpected. The Key here, is that the history of this incident included an upgrade where the installers didn't know MySQL was already installed. This can cause the mysql.sock file to "disapper" either physically, or in assumed location.

Next, if the above is successful, restarting MySQL with the 'safe' option is the best.

IMHO: from experience bringing the servers up and down with and without new software installs, it's always a good idea to make sure the server reboots the way you expect it to while you are thinking about it. That way, if you reboot, and mysql doesn't start running automatically, it's on your mind to fix it.

The only problems/surprises I've had have been when I rebooted for some reason (sometimes over 6 or 8 months apart) and things don't work right. That's a lot of time to try to remember all the changes you made.

It's just a CYA gesture when you make changes to the boot-sequence such as adding or updating services or background processes, to make sure they all come back as expected, and haven't gone away.

I realize my experience is somewhat limited compared to the trial-by-fire you've been getting, but have hit my share of problems and worked out solutions that let me sleep at night :) Also, I have little RAQ experience, although now that SUN is selling them, I might get more. And, I do try to make the footnote when my experience is on Solaris and might not port to Linux or BSD.

Anyway, to some it might appear we are arguing.... (I'm really sensitive to this lately, perhaps too sensitive, adolescents change your views of life greatly), but much of what we learned comes from experience, and trying to put all that experience into a few short steps or a sequence people can follow that will always work, is difficult to impossible.

So, readers of this thread should take all the suggestions into consideration, and keep notes of what works on their system.

Unlike windows, *nix doesn't give you many reasons, chances or opportunities to suffer through a reboot :) That is both good and bad... since it's not uncommon for a year to go by between reboots, and all of a sudden your finely tuned system isn't working the way it was before.

(Heck, my last reboot I took my ISP's sub-net down, <G>.... I had a bad IP on my machine that was hijacking a load of traffic.... It was 7+ months between reboots. No one was looking for that!)

PUGDOGŪ
PUGDOGŪ Enterprises, Inc.
FAQ: http://pugdog.com/FAQ