Mailing List Archive: DBMail: users

DBMAIL grinds to a craw in production

Index | Next | Previous | View Threaded

khanh.truong at 1800GOTJUNK

May 13, 2008, 8:46 AM

Post #1 of 18 (776 views)
Permalink

DBMAIL grinds to a craw in production

Hi,

We've been testing DBMail for a few months now and did a test launch yesterday. We have about 850 pop3/imap users accessing through the webmail, Outlook and mobile devices. Things were fine when we tested with a few users, but after we put it in production - we get about a hundred concurrent connections - MySQL just grinds to a halt. I've tweaked so many settings but nothing helped. I think the problem might be because the DBMail database isn't indexed properly or has no indexing at all... I haven't verified yet.

My question is: does the schema that comes with DBMail already have indexing defined in it or do I have to do my own indexing and optimization?

Our server is a dual 64bit xeon with 4GB of RAM on a RAID1 SCSI
FreeBSD 7
MySQL 5.1

Some one on this list mentioned that they run DBMail with 5000 users no problem so I'm wondering why ours crawed to a stop.

Thanks

Khanh
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

dbmail at bobich

May 13, 2008, 9:07 AM

Post #2 of 18 (750 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

On Tue, 13 May 2008, khanh.truong[at]1800GOTJUNK.COM wrote:

> We've been testing DBMail for a few months now and did a test launch
> yesterday. We have about 850 pop3/imap users accessing through the webmail,
> Outlook and mobile devices. Things were fine when we tested with a few users,
> but after we put it in production - we get about a hundred concurrent
> connections - MySQL just grinds to a halt. I've tweaked so many settings but
> nothing helped. I think the problem might be because the DBMail database
> isn't indexed properly or has no indexing at all... I haven't verified yet.
>
> My question is: does the schema that comes with DBMail already have indexing
> defined in it or do I have to do my own indexing and optimization?

There are some indices. Whether there are indices for every query that the
system could throw at the DB, I don't know.

You could, of course, apply the slow-query-log, and get it to also log all
queries that run unindexed.

The chances are that checking this would have taken less time than appling DB
engine tweaks and typing up the email. :^)

Is the problem CPU or I/O bound?

> Our server is a dual 64bit xeon with 4GB of RAM on a RAID1 SCSI
> FreeBSD 7
> MySQL 5.1
>
> Some one on this list mentioned that they run DBMail with 5000 users no
> problem so I'm wondering why ours crawed to a stop.

If you are seeing lots of connections, it sounds like something is running
unindexed. Get the slow/unindexed query logging going and take it from there.
850 users, even heavy ones, shouldn't be a problem on the sort of hardware you
have.

Gordan
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

aleksander at krediidiinfo

May 13, 2008, 9:07 AM

Post #3 of 18 (750 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

khanh.truong[at]1800GOTJUNK.COM wrote:
> Hi,
>
> We've been testing DBMail for a few months now and did a test launch
> yesterday. We have about 850 pop3/imap users accessing through the
> webmail, Outlook and mobile devices. Things were fine when we tested
> with a few users, but after we put it in production - we get about a
> hundred concurrent connections - MySQL just grinds to a halt.

The fact the default connection limit in mysql is 100 is not to blame here?

Anyway, enable slow query logging in mysql and have a look at it. Make
sure there are no errors in the mail or mysql log.

> My question is: does the schema that comes with DBMail already have
> indexing defined in it or do I have to do my own indexing and
> optimization?

The indexes are already there.

HTH,

--
Aleksander Kamenik
System Administrator
Krediidiinfo AS
an Experian Company
Phone: +372 665 9649
Email: aleksander[at]krediidiinfo.ee

http://www.krediidiinfo.ee/
http://www.experiangroup.com/
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

lists.hzfw4 at liquidbytes

May 13, 2008, 9:10 AM

Post #4 of 18 (750 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

khanh.truong[at]1800GOTJUNK.COM wrote:
> My question is: does the schema that comes with DBMail already have indexing defined in it or do I have to do my own indexing and optimization?

Yes, there are indexes - I just checked it with my db (the structure
works, so it should not be too outdated). But especially the messages
table seems to be over-indexed. That costs storage as well as
insert/update performance. No problem with my little mail server, but at
a larger scale it might cause trouble.

You should also think about installing a MySQL cluster and/or use MySQL
Proxy. Caching will help a lot with read performance.

Michael
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

dbmail at bobich

May 13, 2008, 9:16 AM

Post #5 of 18 (748 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

On Tue, 13 May 2008, Michael Mayer wrote:

> khanh.truong[at]1800GOTJUNK.COM wrote:
>> My question is: does the schema that comes with DBMail already have
>> indexing defined in it or do I have to do my own indexing and optimization?
>
> Yes, there are indexes - I just checked it with my db (the structure works,
> so it should not be too outdated). But especially the messages table seems to
> be over-indexed. That costs storage as well as insert/update performance. No
> problem with my little mail server, but at a larger scale it might cause
> trouble.

Unlikely, unless you have more writes than reads, which would be rather
unusual for most applications. :)

> You should also think about installing a MySQL cluster and/or use MySQL
> Proxy. Caching will help a lot with read performance.

I'd avoid MySQL cluster - last time I checked it it was slow to the pain,
not to mention the distinctly dodgy recovery modes when a node goes down.
Round-robin replication is probably a better solution if you need to scale
to multiple servers. Or you could just go PostgreSQL and do proper star
topology replication with a bit of pl/perl and trigger magic.

Anyway, the original poster said there is only one server being used, so
all this replication/clustering is irrelevant in this particular case.

Gordan
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

lists.hzfw4 at liquidbytes

May 13, 2008, 9:54 AM

Post #6 of 18 (749 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

dbmail[at]bobich.net wrote:
> Unlikely, unless you have more writes than reads, which would be rather
> unusual for most applications. :)

An index on a field with a selectivity of more than 15% can perform
worse than a full table scan in most cases. That affects read
performance as well as disk I/O and locks the table/rows for write
access. Also, there is an index on mailbox_idnr plus one on mailbox_idnr
AND status (in that order), which doesn't make sense, if the index is
not a hash but a btree. That is btw the only index type for InnoDB. So
the case is clear.

> Anyway, the original poster said there is only one server being used, so
> all this replication/clustering is irrelevant in this particular case.

A cluster always is the last thing that helps. I did not recommend that.
But a cache is for sure not the worst idea one could come up with ;)

Michael
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

dbmail at bobich

May 13, 2008, 10:04 AM

Post #7 of 18 (748 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

On Tue, 13 May 2008, Michael Mayer wrote:

> dbmail[at]bobich.net wrote:
>> Unlikely, unless you have more writes than reads, which would be rather
>> unusual for most applications. :)
>
> An index on a field with a selectivity of more than 15% can perform worse
> than a full table scan in most cases. That affects read performance as well
> as disk I/O and locks the table/rows for write access. Also, there is an
> index on mailbox_idnr plus one on mailbox_idnr AND status (in that order),
> which doesn't make sense, if the index is not a hash but a btree. That is btw
> the only index type for InnoDB. So the case is clear.

The case isn't clear until you can post a direct comparison where a particular
query frequently used by the system runs faster without some of the said
indices than it does with them. Remember that the DB engine is free to decide
what indices to use and which to ignore. An unused index wastes little other
than additional space on disk.

>> Anyway, the original poster said there is only one server being used, so all
>> this replication/clustering is irrelevant in this particular case.
>
> A cluster always is the last thing that helps. I did not recommend that. But
> a cache is for sure not the worst idea one could come up with ;)

Absolutely agreed. And for the sake of enabling a switch in my.cnf it's worth
trying, if it's not enabled already.

Gordan
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

paul at nfg

May 13, 2008, 10:14 AM

Post #8 of 18 (749 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

Michael Mayer wrote:
> But especially the messages
> table seems to be over-indexed.

Please propose a better index setup.

--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

lists.hzfw4 at liquidbytes

May 13, 2008, 10:29 AM

Post #9 of 18 (749 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

Paul J Stevens wrote:
> Michael Mayer wrote:
>> But especially the messages
>> table seems to be over-indexed.
>
> Please propose a better index setup.

As I said, I would check the performance without mailbox_idnr_index in
the first step, because the column is already contained in
mailbox_status - the optimizer should notice that. I do not have large
enough tables with real-world data to perform tests - so any
recommendation is pure theory at the moment.

Michael

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

aleksander at krediidiinfo

May 13, 2008, 11:38 AM

Post #10 of 18 (752 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

dbmail[at]bobich.net wrote:
> On Tue, 13 May 2008, Michael Mayer wrote:
>> A cluster always is the last thing that helps. I did not recommend
>> that. But a cache is for sure not the worst idea one could come up
>> with ;)
>
> Absolutely agreed. And for the sake of enabling a switch in my.cnf it's
> worth trying, if it's not enabled already.

For the GP, by cache we mean:

http://dev.mysql.com/doc/refman/5.0/en/query-cache.html

HTH,

--
Aleksander Kamenik
System Administrator
Krediidiinfo AS
an Experian Company
Phone: +372 665 9649
Email: aleksander[at]krediidiinfo.ee

http://www.krediidiinfo.ee/
http://www.experiangroup.com/
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

josh at worldhosting

May 13, 2008, 3:33 PM

Post #11 of 18 (746 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

Hi,

I found with our installation that 100 concurrent connections isn't
unusual, especially during peak usage times. We've set our max
connections right up to compensate.

I would recommend getting the innotop program, as you can see realtime
what queries are taking a long time to execute.

I've attached the my.cnf we're using in a hope that my config tweaks can
help others. For comparison my hardware is 3Gb RAM, Celeron 3GHz on
linux 2.6.18 and MySQL 5.0.32 - and currently I have 135 connections and
it's still half an hour before everyone starts work. Here we have 4,400
accounts on the server.

Another recommendation I have is to have a second server purely for
backup, have it as a read-only slave and take it offline every day to
perform the daily backup dump, then bring it back online and let it sync
up to the main server. Then the time, disk and cpu it takes to backup
doesn't affect the speed of the main server.

Disclaimer: I'm not a database expert, my tweaks aren't perfect but they
work for me from the information I've gleaned from many mailing lists.

# The MySQL database server configuration file.
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock

[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0

[mysqld]
#
# * Basic Settings
#
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
language = /usr/share/mysql/english
skip-external-locking
skip-name-resolve
bind-address = 0.0.0.0
#
# * Fine Tuning
#
key_buffer = 1024M
key_buffer_size = 16M
max_allowed_packet = 128M
table_cache = 2048
sort_buffer_size = 32M
read_buffer_size = 8M
read_rnd_buffer_size = 62M
myisam_sort_buffer_size = 64M
thread_concurrency = 4

thread_stack = 128K
thread_cache_size = 8
max_connections = 1800

query_cache_limit = 5M
query_cache_size = 512M

log_bin = /storage/binlog/mysql-bin.log

expire_logs_days = 5
max_binlog_size = 100M

skip-bdb

innodb_buffer_pool_size=100M
innodb_log_buffer_size=8M
innodb_file_per_table

[mysqldump]
quick
quote-names
max_allowed_packet = 128M

[mysql]
#no-auto-rehash # faster start of mysql but no tab completition

[isamchk]
key_buffer = 16M

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

jesse at kci

May 13, 2008, 3:44 PM

Post #12 of 18 (747 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

I don't know if the original problem here was actually tracked to the
database or just guessing there, but another thing to check is logging.
Eg. if you have log level 5 turned on and your syslog isn't set to
buffer writes, you will likely notice the performance hit. :)

--
Jesse Norell
Kentec Communications, Inc.
jesse[at]kci.net
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

dbmail at bobich

May 13, 2008, 3:47 PM

Post #13 of 18 (746 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

> I've attached the my.cnf we're using in a hope that my config tweaks can
> help others. For comparison my hardware is 3Gb RAM, Celeron 3GHz on
> linux 2.6.18 and MySQL 5.0.32 - and currently I have 135 connections and
> it's still half an hour before everyone starts work. Here we have 4,400
> accounts on the server.

[...]

> myisam_sort_buffer_size = 64M

This, obviously, won't have any effect if you are using the default InnoDB
storage engine, it'll at best just waste your memory.

> key_buffer = 1024M
> key_buffer_size = 16M

I'm not 100% sure, but I think these are MyISAM only. I think the
roughly equivalent InnoDB setting is:

innodb_buffer_pool = 1024M

> innodb_buffer_pool_size=100M

This should probably be about 1/2 if total memory if the machine is a
dedicated InnoDB database server or about 1/4 if it's also running all the
other mail services (webmail, dbmail, etc.).

Another thing that is worth doing is bypassing OS caching for table access
because MySQL does it's own caching, and is much more efficient than the
generic OS algorithm:

innodb_flush_method=O_DIRECT

Gordan
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

maenaka at pluto

May 13, 2008, 5:38 PM

Post #14 of 18 (741 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

Hi Khanh

I don't know the exact scenario how your MySQL halts though, it may be
caused by the bug in the MySQL's query optimizer.

If you see a query "SELECT messageblk, is_header FROM dbmail_messageblks
WHERE physmessage_id = NNN ORDER BY messageblk_idnr" doing a full table
scan, the bug is your cause, and the attached patch (against 2.2 branch)
should fix.

Note that how the bug affecting can be seen in the attached mysql
session log. In which, the 2nd "EXPLAIN SELECT ..." chose the incorrect
key requiring a full table scan.

FreeBSD 7.0-BETA2 as of Nov. 3 2007
MySQL 5.1.23-rc as of Dec. 14 2007
DBMail 2.2 as of Dec. 13 2007

--
UEMURA (fka. MAENAKA) Tetsuya <maenaka[at]pluto.dti.ne.jp>

Posted on Tue, 13 May 2008 08:46:59 -0700
by author khanh.truong <khanh.truong[at]1800GOTJUNK.COM>
> Hi,
>
> We've been testing DBMail for a few months now and did a test launch yesterday. We have about 850 pop3/imap users accessing through the webmail, Outlook and mobile devices. Things were fine when we tested with a few users, but after we put it in production - we get about a hundred concurrent connections - MySQL just grinds to a halt. I've tweaked so many settings but nothing helped. I think the problem might be because the DBMail database isn't indexed properly or has no indexing at all... I haven't verified yet.
>
> My question is: does the schema that comes with DBMail already have indexing defined in it or do I have to do my own indexing and optimization?
>
> Our server is a dual 64bit xeon with 4GB of RAM on a RAID1 SCSI
> FreeBSD 7
> MySQL 5.1
>
> Some one on this list mentioned that they run DBMail with 5000 users no problem so I'm wondering why ours crawed to a stop.
>
> Thanks
>
> Khanh
> _______________________________________________
> DBmail mailing list
> DBmail[at]dbmail.org
> https://mailman.fastxs.nl/mailman/listinfo/dbmail

Attachments:

patch-dbmail-message.c (0.58 KB)

mysql.log (3.00 KB)

vulture at netvulture

May 14, 2008, 10:39 PM

Post #15 of 18 (719 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

Do Not use Mysql 5.1.23-rc on FreeBSD from ports/packages!!! Use 5.1.22.
I ran into an issue where order by's were causing data from the result
sets to be dropped. This was causing a lot of issues with dbmail imapd
clients. There was a bug open on an issue with this with MySQL team.
Essentially something in the ports build process for 5.1.23 was
corrupting the sorting.

I always compile from source and I think that I had that issue with a
manual compile as well on 5.1.23.

I am running 7-STABLE with MySQL 5.1.22-rc built from ports.

To build the older version from ports modify the version number in the
Makefile to 5.1.22 and use the following for the distinfo file

MD5 (mysql-5.1.22-rc.tar.gz) = 87be813d1b2d76b1c756de0a16fa29fb
SHA256 (mysql-5.1.22-rc.tar.gz) =
1bce64c7e8fe476828644cbc05157fc84fe02e9c13379d1611fbd7d27c0721ec
SHA1 (mysql-5.1.22-rc.tar.gz) = 97c1396e0f0561485df120cce2092192aecd12a0

My Mysql cconfig is as follows and is probably pretty bad, but access
times are pretty fast for my few users. I have 8 GB of ram and quad
opterons, so you may need to drop some of the numbers down

-BEGIN-
[client]
#password = your_password
port = 3306
socket = /tmp/mysql.sock

[mysqld]
port=3306
socket=/tmp/mysql.sock
myisam_recover
tmpdir=/usr/mysql.tmp
server-id=1
log-bin=mysql-bin
log-slow-queries=/var/log/mysqld-slow.log
log-output=FILE
character-set-server=utf8
innodb_force_recovery=0
innodb_file_per_table
skip-locking
default_table_type=INNODB
thread_concurrency=64
max_connections=512
max_allowed_packet=16M
#table_cache=1024
key_buffer_size=2G
innodb_buffer_pool_size=4G
innodb_additional_mem_pool_size=256M
innodb_autoinc_lock_mode=2
innodb_commit_concurrency=0
innodb_concurrency_tickets=32
innodb_flush_method=O_DIRECT
innodb_thread_concurrency=32
innodb_log_buffer_size=16M
innodb_flush_log_at_trx_commit=2
table_cache=1024
thread_cache=32
query_cache_size=256M
sort_buffer_size=256M
read_buffer_size=256M
read_rnd_buffer_size=256M
myisam_sort_buffer_size=256M

[mysqldump]
quick
max_allowed_packet = 16M

[isamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 256M
write_buffer = 256M

[myisamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 256M
write_buffer = 256M

[mysqlhotcopy]
interactive-timeout

[mysqld_safe]
# Increase the amount of open files allowed per process. Warning: Make
# sure you have set the global system limit high enough! The high value
# is required for a large number of opened tables
open-files-limit = 3072
-END-

Also - My custom kernel config has to following non-standard options

options HZ=2000
options SCHED_ULE # ULE scheduler
options PREEMPTION # Enable kernel thread preemption
options FULL_PREEMPTION # Enable kernel thread preemption
options IPI_PREEMPTION # Enable kernel thread preemption
options SMP

last pid: 77763; load averages: 0.12, 0.19,
0.15
up 1+05:13:58 22:38:57
187 processes: 1 running, 186 sleeping
CPU states: 0.2% user, 0.0% nice, 0.0% system, 0.2% interrupt, 99.6%
idle
Mem: 5365M Active, 805M Inact, 450M Wired, 293M Cache, 214M Buf, 216M Free
Swap: 8192M Total, 388M Used, 7804M Free, 4% Inuse

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
30517 nobody 1 4 0 55548K 16612K sbwait 0 1:23 0.29%
dbmail-imapd
965 mysql 41 4 0 7216M 4605M sbwait 2 37:53 0.00% mysqld

Just a little bit of ram in use on mysql!

Hope that helps.

-Jon

UEMURA (fka. MAENAKA) Tetsuya wrote:
> Hi Khanh
>
> I don't know the exact scenario how your MySQL halts though, it may be
> caused by the bug in the MySQL's query optimizer.
>
> If you see a query "SELECT messageblk, is_header FROM dbmail_messageblks
> WHERE physmessage_id = NNN ORDER BY messageblk_idnr" doing a full table
> scan, the bug is your cause, and the attached patch (against 2.2 branch)
> should fix.
>
> Note that how the bug affecting can be seen in the attached mysql
> session log. In which, the 2nd "EXPLAIN SELECT ..." chose the incorrect
> key requiring a full table scan.
>
> FreeBSD 7.0-BETA2 as of Nov. 3 2007
> MySQL 5.1.23-rc as of Dec. 14 2007
> DBMail 2.2 as of Dec. 13 2007
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> DBmail mailing list
> DBmail[at]dbmail.org
> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

dbmail at tech

May 16, 2008, 3:16 AM

Post #16 of 18 (704 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

> I'm not 100% sure, but I think these are MyISAM only. I think the
> roughly equivalent InnoDB setting is:
>
> innodb_buffer_pool = 1024M
It is myisam only, use buffer_pool instead.
> Another thing that is worth doing is bypassing OS caching for table
> access because MySQL does it's own caching, and is much more efficient
> than the generic OS algorithm:
>
> innodb_flush_method=O_DIRECT
As long as you have a decent raid card with a bbu. You should be careful
with these options, they can actually reduce performance, its best to
benchmark and see which performs best on your hardware.

> max_connections = 1800
You might want to watch out for the max connections - can you server
actually handle 1800 connections? remember for each connection, you'll
have all the relevant buffers/memory in place. You'll probably find your
server runs out of memory and swaps before it hits 1800 connections.

Its worth looking at innodb_flush_log_at_trx_commit=0/2
- http://dev.mysql.com/doc/refman/5.0/en/innodb-parameters.html

It'll reduce your disk i/o slightly for log writes. You may want to
separate your data from your logs and os. Logs are sequential, data is
random - you can optimise each accordingly.

Also, to answer the first question about the performance - it may be
worth looking towards imapproxy - it helps reuse existing connections,
rather than opening a new one each time (particularly with webmail) and
waiting for the connection to timeout - it might help with your scenario.

Paul is working on the new threading model, which will help dbmail's
performance dramatically. Look at the dev list for more info.

Regards,

SG
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

khanh.truong at 1800GOTJUNK

May 16, 2008, 9:06 AM

Post #17 of 18 (704 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

Thanks Jon and all others for your great suggestions.

As Jon said, I'm drawing towards the conclusion that the issue was with some bug or incompatibility between FreeBSD 7 and MySql 5.1. FreeBSD 7 has a new scheduler but by default it still uses the older one. Things seemed to be much better after we recompiled the kernel with the new scheduler.

>Also - My custom kernel config has to following non-standard options

>options HZ=2000
>options SCHED_ULE # ULE scheduler
>options PREEMPTION # Enable kernel thread preemption
>options FULL_PREEMPTION # Enable kernel thread preemption
>options IPI_PREEMPTION # Enable kernel thread preemption
>options SMP

Khanh
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

vulture at netvulture

May 16, 2008, 5:58 PM

Post #18 of 18 (694 views)
Permalink

Re: DBMAIL grinds to a craw in production [In reply to]

Also make sure that your MySQL is not 5.1.23 from packages or ports. I
had an issue with the sorting of large result sets losing rows. I had
the issue on both i386 and amd64. DBMail did all kinds a weird things
such as messages not showing up at all when headers were downloaded.

I'm glad that the new kernel is working better.

-Jon

khanh.truong[at]1800GOTJUNK.COM wrote:
> Thanks Jon and all others for your great suggestions.
>
> As Jon said, I'm drawing towards the conclusion that the issue was with some bug or incompatibility between FreeBSD 7 and MySql 5.1. FreeBSD 7 has a new scheduler but by default it still uses the older one. Things seemed to be much better after we recompiled the kernel with the new scheduler.
>
>
>> Also - My custom kernel config has to following non-standard options
>>
>
>
>> options HZ=2000
>> options SCHED_ULE # ULE scheduler
>> options PREEMPTION # Enable kernel thread preemption
>> options FULL_PREEMPTION # Enable kernel thread preemption
>> options IPI_PREEMPTION # Enable kernel thread preemption
>> options SMP
>>
>
> Khanh
> _______________________________________________
> DBmail mailing list
> DBmail[at]dbmail.org
> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>
>

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com