pwadas at jewish
Nov 10, 2009, 4:01 AM
Post #1 of 7
(958 views)
Permalink
|
Well, after some tests for a few months on selected accounts with dbmail
2.3.6, I'm afraid
I must go back to previous solution for now.
To summarize experience:
+ dbmail offers uncomparable BETTER per-account performance, including imap
and pop3 mailboxes
containing serveral gigabytes of messages, in hundreds/thousands accounts.
This is probably the most
important advantage, which actually lead me to dbmail generally.
+ migration FROM any other pop3/imap suite to dbmail is quite easy, and does
not
generate much work and end-user problems, ability to set-up a fixed UIDL in
some dbmail table
with some self-made perl pop3 script, grabbed from old pop3 generated
variables is a HUGE,
I mean really HUGE handicap, allowing to avoid re-downloading gigabytes of
messages.
+ integration with sendmail.org's sendmail is easy achievable in debian
through slight modification
in debian sensible-mda helper executable.
+ it was quite easy to migrate users continously (not thousands or users
all-at-once). I setup
source of information (actually an sql table), and then on MTA
(sensible-mta) I put sql query,
resulting "ok, user is already migrated to dbmail, use dbmail-deliver" OR
"user is not migrated yet,
use procmail/maildrop/other file-based MDA). On the pop3/imap I set-up
dovecot-proxy, which
used the same table. So finally I had three pop3 concurrently running -
previously used
pop3/imap daemon bound to internal IP address A, dbmail daemons bound to
internal IP address B,
and dovecot proxy bound to external address X.
In the effect, I achieved an MTA able to decide which MDA should be used
per-user, and pop3/imap
proxy able to decide where forward connection, also per-user, and both
(MTA/POP+IMAP) used
the same, one source of decision-critical information. Simple, and
effective.
however
- dbmail daemons segfaults sometimes, enough often to force me to create a
little script, making
sure the daemons are re-started in an instant (running in background,
started on gnu screen utility)
probably better solution, until segfaults are code-fixed, would be starting
dbmail from xinetd,
if possible (?), but I didn't dive into this.
-----------
#!/bin/bash
#[ `ps ax|egrep dbmail-imapd\|dbmail-pop3d\|dbmail-lmtpd|wc -l` -ne 3 ] &&
# /etc/init.d/dbmail restart
if ( [ `ps --no-headers -C dbmail-pop3d,dbmail-imapd,dbmail-lmtpd|wc -l`
-ne 3 ] ) ; then
echo dbmail-just-restarted | mail -s dbmail-restart
admin [at] mysite
/etc/init.d/dbmail restart
else
echo "DBMAIL: `date` alright"
fi
-----------
I really regret I don't have enough time to debug/track/post to the list
each issue :(
Probably most of them are already fixed in development branch, not in last
release unfortunately :(
- there are still some problems on vacuuming/cleaning up the database, at
least with postgresql 8.4+
Even if messages are properly deleted and removed from database (they don't
remains as orphaned parts),
with thousands of mailboxes and hundreds of gigabytes of messages, nigtly
(or weekly) vacuuming
took so long, that sometimes is still in progress when the next, or
after-next business day starts,
interfering with overall server machines performance. IMHO the main problem
is famous
"delete-yes-I-am-sure-then-purge-deleted-yes-I-am-sure-then-cleanup-db'
feature.
In my environment all MUA's (including webmail), used has a IMAP "Trash", or
local pop3 trash,
which actually makes real-unrecoverable-drop-into-space quite hard. With
thousands deletions per-day,
this really impact used space. Users are aware they're responsible for
mistaken deletions, and even
if they're not, there's still their problem. Such situations happen rarely,
there are daily backups,
and this does not explain usage of gigabytes (performance impact) of space
under "what-if-user-is-fool".
Disks are cheap, actually, but from administrative point of view, managing
tens/hundreds of gigabytes
of unneeded data in backups, various local-and-remote trashes, and more
various yes-really-purge,
yes-really-cleanup features, is quite annoying.
IMHO, the solution is to disable, or get-off "really-purge" server features.
The vacuuming issues are discussed below.
- finally, the effective decision is to back-off from dbmail for now, and
this is mainly
because quota and size limits management complexity. Limits for users in my
environments are
counted globally, without differentiating between email/ftp/www/sql space
usage, and dbmail's
construction really makes it harder. Will probably back to dbmail with next
release.
With the vacuuming/cleanup issues (probably existing with mysql too (?), I
really think, the
solution is to make possible per-user or per-group different database
storage.
I'd gracefully pay connection's performance cost, if I could have separate
database for
each user, or (even better), a separate database per posix group.
At database (or ldap) level, this could be achieved with some table
specifying
WHICH database (or schema with postgres ?) keeps database for this user. A
posix or non-posix
group of users would simply have the same setting for "which-db".
This would simplify per-user (per-group) backups, space usage limits (with
dbmail's internal
quotas, or external FS-based quotas, combined with posix extended ACLs for
database files),
and solve cleanup/vacuuming issues. With milions of users it is no problem
to vacuum, or
backup, or space-count, or purge-everything-and-start-again per-group (or
per-user).
IMHO Connection performance cost is really low cost, in comparison to cost
of extremely
complicated global (dbmail + various file resources) diskspace usage
counting, extremly complicated
giant all-users-in-one backups and restore operation, and annoying
vacuuming/cleanup.
There's no , and probably won't be, a need for browsing
all-universe-of-our-users messages set.
More - actually messages size, stored in db, is definitely not the all
storage user's messages use,
in some cases indexes size may be significant, probably hard to count
per-user. With different
database storage, simple count of total space used by user's (or group's) db
is simple
and effective, may be combined with posix extended acls and kernel's quota.
Already implemented message-size information is of course useful, and needed
- at least
for enduser informational purposes. But hard to use in addition to
global-count usage and limits.
Note - I'm not saying, that dbmail system has to do something with the rest
of environment.
It's messaging (imap/pop3) system, and not a hosting system, or file-usage
counting system.
But behaviour, which avoids complicating the rest of the system is worthful,
at least for the
sake of dbmail's popularity. Even, if messaging system has nothing to do
with webserver subsystem,
internet domain management subsystem, and so on, in real life 90% of cases,
the same physical person, (administrator), is responsible for the whole
stuff. In small-and-medium-businesses , even if 5 admins are in business,
each one of them has its part of 24/7 time of duty, when he's responsible
for functioning of the whole stuff. Even, if there are specialized admins
for each service,
quotas/filesysytem/ftp admin won't be happy if db admin will say "sorry, I
don't care about how you
include db usage in your quota stats".
To close - I have some problem dbmail-export utility, seems it generates
mbox file
with invalid structure, for the first look mbox file seems to be ok, but MUA
(pine), and
old pop3/imap server complains about invalid format, I'll post if I find
what's actually wrong.
Hope to back to dbmail soon :)
Regards,
DT
--
View this message in context: http://old.nabble.com/%3A--tp26282237p26282237.html
Sent from the dbmail users mailing list archive at Nabble.com.
_______________________________________________
DBmail mailing list
DBmail [at] dbmail
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
|
