Mailing List Archive: DBMail: users

dbmail 3.1.2 status

Index | Next | Previous | View Threaded

h.reindl at thelounge

Aug 9, 2013, 5:38 PM

Post #1 of 11 (25 views)
Permalink

dbmail 3.1.2 status

after a week running dbmail 3.1 a short report

* memory usage: perfect, dbmail-imapd currentyl 133 MB
and sometimes it goes below 100 MB
so no leaks and dail restarts needed
_____________________________________________

some small issues with no idea how to debug because
it happens not often enough to enable debug-logs in
production or not possible to reproduce with my clients

* IMAP / Tunderbird: in the message-list randomly empty subject
it is present in the preview
mostly folder-options -> rapair let it appear in the list
sometimes this is need twice, i can find no logic there
maybe some timing/race-conditions, hard to debug

* Apple Mail - i hate it
POP3 as well as IMAP until now 3 messages which are
reported to me with empy or crippeled body
AFAIK all times multipart-messages
in all cases no problem in roundcube-mail or Thunderbird

* twice one thread appeared to consume 100% CPU without
lead to troubles for active users and SIGTERM ignroed
_____________________________________________

[root [at] mai:~]$ systemctl status dbmail-imapd.service
dbmail-imapd.service - DBMail IMAP Server
Loaded: loaded (/usr/lib/systemd/system/dbmail-imapd.service; enabled)
Active: active (running) since Di 2013-08-06 14:08:19 CEST; 3 days ago
Main PID: 5921 (dbmail-imapd)
CGroup: name=systemd:/system/dbmail-imapd.service
└─5921 /usr/sbin/dbmail-imapd -D

Attachments:

signature.asc (0.26 KB)

paul at nfg

Aug 11, 2013, 2:11 PM

Post #2 of 11 (15 views)
Permalink

Re: dbmail 3.1.2 status [In reply to]

Harald,

thanks for reporting.

On 2013-08-10 02:38, Reindl Harald wrote:
> after a week running dbmail 3.1 a short report
>
> * memory usage: perfect, dbmail-imapd currentyl 133 MB
> and sometimes it goes below 100 MB
> so no leaks and dail restarts needed

I see the same, and still improving in the master branch. And it's
not only the memory footprint that improves because of this, but also
speed to some extend.

> some small issues with no idea how to debug because
> it happens not often enough to enable debug-logs in
> production or not possible to reproduce with my clients

If it's impossible to reproduce reliably, it's difficult to debug. I run
into smaller and bigger problems all the time, mostly from testing with
the stuff in test-scripts. Also, adding new IMAP capabilities like I've
been busy doing since 3.1.2 exposes older code to review and
refactoring. I finally have a simple but effective framework in place
for automatic schema migrations - very cool. So, when adding a rather
complex capability like CONDSTORE (rfc4551) only takes a day or two, it
gives me a nice fuzzy feeling about the state of the code. I must be
doing something right. Of course that feeling disappears as I stumble on
older code that doesn't look quite right...

> * IMAP / Tunderbird: in the message-list randomly empty subject
> it is present in the preview

I've seen it, right around 3.1.0 or 3.1.1, but not since...

> mostly folder-options -> rapair let it appear in the list
> sometimes this is need twice, i can find no logic there
> maybe some timing/race-conditions, hard to debug
>
> * Apple Mail - i hate it
> POP3 as well as IMAP until now 3 messages which are
> reported to me with empy or crippeled body
> AFAIK all times multipart-messages

That is possibly a combination of the exact IMAP commands that expose a
bug in GMime. Only way to know for sure is by running the IMAP commands
Apple uses to fetch a message, against the exact MIME structure of the
message involved.

And even updating GMime is not guaranteed to fix things, since I've
worked around problems and limitations in GMime in the past, rather than
getting them fixed or dealt with in GMime proper.

GMime is really nice and it's author very helpful. It does however
depend on downstream users like dbmail for battle-testing some parts of
the code.

> in all cases no problem in roundcube-mail or Thunderbird

Good. I really love how much more responsive especially imap now is,
compared with 3.0.2.

> * twice one thread appeared to consume 100% CPU without
> lead to troubles for active users and SIGTERM ignroed

I would try to strace the thread involved. Maybe that will give me some
hint as to where it's happening in the code.

top -bH -n 1 -p <PID>

should give you a listing of the threads, with process-ids per thread.
Maybe one particular thread is causing havoc, which you can then strace.

> _____________________________________________
>
> [root [at] mai:~]$ systemctl status dbmail-imapd.service
> dbmail-imapd.service - DBMail IMAP Server
> Loaded: loaded (/usr/lib/systemd/system/dbmail-imapd.service;
> enabled)
> Active: active (running) since Di 2013-08-06 14:08:19 CEST; 3 days
> ago
> Main PID: 5921 (dbmail-imapd)
> CGroup: name=systemd:/system/dbmail-imapd.service
> └─5921 /usr/sbin/dbmail-imapd -D

--
________________________________________________________________
Paul J Stevens pjstevns @ gmail, twitter, skype, linkedin

* Premium Hosting Services and Web Application Consultancy *

www.nfg.nl/info [at] nfg/+31.85.877.99.97
________________________________________________________________
_______________________________________________
DBmail mailing list
DBmail [at] dbmail
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

harald.leithner at itronic

Aug 12, 2013, 2:33 AM

Post #3 of 11 (12 views)
Permalink

Re: dbmail 3.1.2 status [In reply to]

Hi,

my 2 cents,

Am 11.08.2013, 23:11 Uhr, schrieb Paul J Stevens <paul [at] nfg>:

>
> Harald,
>
> thanks for reporting.
>
> On 2013-08-10 02:38, Reindl Harald wrote:
>> after a week running dbmail 3.1 a short report
>>
>> * memory usage: perfect, dbmail-imapd currentyl 133 MB
>> and sometimes it goes below 100 MB
>> so no leaks and dail restarts needed
>
> I see the same, and still improving in the master branch. And it's
> not only the memory footprint that improves because of this, but also
> speed to some extend.
>

Uptime 4 Days consumes 882M mb memory, it leaks about 200 MB per day. But
performance is great.

>
>> some small issues with no idea how to debug because
>> it happens not often enough to enable debug-logs in
>> production or not possible to reproduce with my clients
>
> If it's impossible to reproduce reliably, it's difficult to debug. I run
> into smaller and bigger problems all the time, mostly from testing with
> the stuff in test-scripts. Also, adding new IMAP capabilities like I've
> been busy doing since 3.1.2 exposes older code to review and
> refactoring. I finally have a simple but effective framework in place
> for automatic schema migrations - very cool. So, when adding a rather
> complex capability like CONDSTORE (rfc4551) only takes a day or two, it
> gives me a nice fuzzy feeling about the state of the code. I must be
> doing something right. Of course that feeling disappears as I stumble on
> older code that doesn't look quite right...

Its really great that you add new stuff, do you know a table which client
supports with imap extension?

Any thing new on RFC6154, if I understood it correctly you can extend the
users table with the 7 Fields and set the SPECIAL-USE defaults in
dbmail.conf.

>
>> * IMAP / Tunderbird: in the message-list randomly empty subject
>> it is present in the preview
>
> I've seen it, right around 3.1.0 or 3.1.1, but not since...

No problems with this.

>
>> mostly folder-options -> rapair let it appear in the list
>> sometimes this is need twice, i can find no logic there
>> maybe some timing/race-conditions, hard to debug
>>
>> * Apple Mail - i hate it
>> POP3 as well as IMAP until now 3 messages which are
>> reported to me with empy or crippeled body
>> AFAIK all times multipart-messages
>
> That is possibly a combination of the exact IMAP commands that expose a
> bug in GMime. Only way to know for sure is by running the IMAP commands
> Apple uses to fetch a message, against the exact MIME structure of the
> message involved.
>
> And even updating GMime is not guaranteed to fix things, since I've
> worked around problems and limitations in GMime in the past, rather than
> getting them fixed or dealt with in GMime proper.
>
> GMime is really nice and it's author very helpful. It does however
> depend on downstream users like dbmail for battle-testing some parts of
> the code.

My 10 Apple Mail clients don't have this problem.

>
>> in all cases no problem in roundcube-mail or Thunderbird
>
> Good. I really love how much more responsive especially imap now is,
> compared with 3.0.2.
>
>> * twice one thread appeared to consume 100% CPU without
>> lead to troubles for active users and SIGTERM ignroed
>
> I would try to strace the thread involved. Maybe that will give me some
> hint as to where it's happening in the code.
>
> top -bH -n 1 -p <PID>
>
> should give you a listing of the threads, with process-ids per thread.
> Maybe one particular thread is causing havoc, which you can then strace.
>
>

I would upgrade use SIGHUP to debug, but no time atm and my customers
should have a short period of time with a great working mail server
without experiments ;-)
>
>
>
>> _____________________________________________
>>
>> [root [at] mai:~]$ systemctl status dbmail-imapd.service
>> dbmail-imapd.service - DBMail IMAP Server
>> Loaded: loaded (/usr/lib/systemd/system/dbmail-imapd.service;
>> enabled)
>> Active: active (running) since Di 2013-08-06 14:08:19 CEST; 3 days
>> ago
>> Main PID: 5921 (dbmail-imapd)
>> CGroup: name=systemd:/system/dbmail-imapd.service
>> └─5921 /usr/sbin/dbmail-imapd -D
>

thx

--
Harald Leithner

ITronic
Vogelweidplatz 12, 1150 Wien, Austria
Tel: +43-1-786 23 88
Fax: +43-1-98 52 077
Mobil: +43-699-123 78 4 78
Mail: leithner [at] itronic | itronic.at
_______________________________________________
DBmail mailing list
DBmail [at] dbmail
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

mysql.jorge at decimal

Aug 12, 2013, 2:35 AM

Post #4 of 11 (12 views)
Permalink

Re: dbmail 3.1.2 status [In reply to]

Tell me,

What's your configuration?

- total users
- +/- simultaneous connections per minute
- DB size on this moment

Just want to compare with my setup

(
- 1800 users
- 250/300 simultaneous connections
- 185GB right now
)

> -----Original Message-----
> From: dbmail-bounces [at] dbmail [mailto:dbmail-bounces [at] dbmail] On
> Behalf Of Harald Leithner
> Sent: segunda-feira, 12 de Agosto de 2013 10:33
> To: DBMail mailinglist
> Subject: Re: [Dbmail] dbmail 3.1.2 status
>
> Hi,
>
> my 2 cents,
>
> Am 11.08.2013, 23:11 Uhr, schrieb Paul J Stevens <paul [at] nfg>:
>
> >
> > Harald,
> >
> > thanks for reporting.
> >
> > On 2013-08-10 02:38, Reindl Harald wrote:
> >> after a week running dbmail 3.1 a short report
> >>
> >> * memory usage: perfect, dbmail-imapd currentyl 133 MB
> >> and sometimes it goes below 100 MB
> >> so no leaks and dail restarts needed
> >
> > I see the same, and still improving in the master branch. And it's
> not
> > only the memory footprint that improves because of this, but also
> > speed to some extend.
> >
>
> Uptime 4 Days consumes 882M mb memory, it leaks about 200 MB per day.
> But performance is great.
>
> >
> >> some small issues with no idea how to debug because
> >> it happens not often enough to enable debug-logs in
> >> production or not possible to reproduce with my clients
> >
> > If it's impossible to reproduce reliably, it's difficult to debug. I
> run
> > into smaller and bigger problems all the time, mostly from testing
> with
> > the stuff in test-scripts. Also, adding new IMAP capabilities like
> I've
> > been busy doing since 3.1.2 exposes older code to review and
> > refactoring. I finally have a simple but effective framework in place
> > for automatic schema migrations - very cool. So, when adding a rather
> > complex capability like CONDSTORE (rfc4551) only takes a day or two,
> it
> > gives me a nice fuzzy feeling about the state of the code. I must be
> > doing something right. Of course that feeling disappears as I stumble
> on
> > older code that doesn't look quite right...
>
> Its really great that you add new stuff, do you know a table which
> client
> supports with imap extension?
>
> Any thing new on RFC6154, if I understood it correctly you can extend
> the
> users table with the 7 Fields and set the SPECIAL-USE defaults in
> dbmail.conf.
>
>
>
> >
> >> * IMAP / Tunderbird: in the message-list randomly empty subject
> >> it is present in the preview
> >
> > I've seen it, right around 3.1.0 or 3.1.1, but not since...
>
> No problems with this.
>
> >
> >> mostly folder-options -> rapair let it appear in the list
> >> sometimes this is need twice, i can find no logic there
> >> maybe some timing/race-conditions, hard to debug
> >>
> >> * Apple Mail - i hate it
> >> POP3 as well as IMAP until now 3 messages which are
> >> reported to me with empy or crippeled body
> >> AFAIK all times multipart-messages
> >
> > That is possibly a combination of the exact IMAP commands that expose
> a
> > bug in GMime. Only way to know for sure is by running the IMAP
> commands
> > Apple uses to fetch a message, against the exact MIME structure of
> the
> > message involved.
> >
> > And even updating GMime is not guaranteed to fix things, since I've
> > worked around problems and limitations in GMime in the past, rather
> than
> > getting them fixed or dealt with in GMime proper.
> >
> > GMime is really nice and it's author very helpful. It does however
> > depend on downstream users like dbmail for battle-testing some parts
> of
> > the code.
>
> My 10 Apple Mail clients don't have this problem.
>
> >
> >> in all cases no problem in roundcube-mail or Thunderbird
> >
> > Good. I really love how much more responsive especially imap now is,
> > compared with 3.0.2.
> >
> >> * twice one thread appeared to consume 100% CPU without
> >> lead to troubles for active users and SIGTERM ignroed
> >
> > I would try to strace the thread involved. Maybe that will give me
> some
> > hint as to where it's happening in the code.
> >
> > top -bH -n 1 -p <PID>
> >
> > should give you a listing of the threads, with process-ids per
> thread.
> > Maybe one particular thread is causing havoc, which you can then
> strace.
> >
> >
>
> I would upgrade use SIGHUP to debug, but no time atm and my customers
> should have a short period of time with a great working mail server
> without experiments ;-)
> >
> >
> >
> >> _____________________________________________
> >>
> >> [root [at] mai:~]$ systemctl status dbmail-imapd.service
> >> dbmail-imapd.service - DBMail IMAP Server
> >> Loaded: loaded (/usr/lib/systemd/system/dbmail-imapd.service;
> >> enabled)
> >> Active: active (running) since Di 2013-08-06 14:08:19 CEST; 3
> days
> >> ago
> >> Main PID: 5921 (dbmail-imapd)
> >> CGroup: name=systemd:/system/dbmail-imapd.service
> >> └─5921 /usr/sbin/dbmail-imapd -D
> >
>
>
> thx
>
> --
> Harald Leithner
>
> ITronic
> Vogelweidplatz 12, 1150 Wien, Austria
> Tel: +43-1-786 23 88
> Fax: +43-1-98 52 077
> Mobil: +43-699-123 78 4 78
> Mail: leithner [at] itronic | itronic.at
> _______________________________________________
> DBmail mailing list
> DBmail [at] dbmail
> http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

_______________________________________________
DBmail mailing list
DBmail [at] dbmail
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

harald.leithner at itronic

Aug 12, 2013, 2:44 AM

Post #5 of 11 (12 views)
Permalink

Re: dbmail 3.1.2 status [In reply to]

300 users
80 connections on Imap, most users use pop3
50 GB

Am 12.08.2013, 11:35 Uhr, schrieb Jorge Bastos <mysql.jorge [at] decimal>:

> Tell me,
>
> What's your configuration?
>
> - total users
> - +/- simultaneous connections per minute
> - DB size on this moment
>
>
> Just want to compare with my setup
>
> (
> - 1800 users
> - 250/300 simultaneous connections
> - 185GB right now
> )
>
>
>> -----Original Message-----
>> From: dbmail-bounces [at] dbmail [mailto:dbmail-bounces [at] dbmail] On
>> Behalf Of Harald Leithner
>> Sent: segunda-feira, 12 de Agosto de 2013 10:33
>> To: DBMail mailinglist
>> Subject: Re: [Dbmail] dbmail 3.1.2 status
>>
>> Hi,
>>
>> my 2 cents,
>>
>> Am 11.08.2013, 23:11 Uhr, schrieb Paul J Stevens <paul [at] nfg>:
>>
>> >
>> > Harald,
>> >
>> > thanks for reporting.
>> >
>> > On 2013-08-10 02:38, Reindl Harald wrote:
>> >> after a week running dbmail 3.1 a short report
>> >>
>> >> * memory usage: perfect, dbmail-imapd currentyl 133 MB
>> >> and sometimes it goes below 100 MB
>> >> so no leaks and dail restarts needed
>> >
>> > I see the same, and still improving in the master branch. And it's
>> not
>> > only the memory footprint that improves because of this, but also
>> > speed to some extend.
>> >
>>
>> Uptime 4 Days consumes 882M mb memory, it leaks about 200 MB per day.
>> But performance is great.
>>
>> >
>> >> some small issues with no idea how to debug because
>> >> it happens not often enough to enable debug-logs in
>> >> production or not possible to reproduce with my clients
>> >
>> > If it's impossible to reproduce reliably, it's difficult to debug. I
>> run
>> > into smaller and bigger problems all the time, mostly from testing
>> with
>> > the stuff in test-scripts. Also, adding new IMAP capabilities like
>> I've
>> > been busy doing since 3.1.2 exposes older code to review and
>> > refactoring. I finally have a simple but effective framework in place
>> > for automatic schema migrations - very cool. So, when adding a rather
>> > complex capability like CONDSTORE (rfc4551) only takes a day or two,
>> it
>> > gives me a nice fuzzy feeling about the state of the code. I must be
>> > doing something right. Of course that feeling disappears as I stumble
>> on
>> > older code that doesn't look quite right...
>>
>> Its really great that you add new stuff, do you know a table which
>> client
>> supports with imap extension?
>>
>> Any thing new on RFC6154, if I understood it correctly you can extend
>> the
>> users table with the 7 Fields and set the SPECIAL-USE defaults in
>> dbmail.conf.
>>
>>
>>
>> >
>> >> * IMAP / Tunderbird: in the message-list randomly empty subject
>> >> it is present in the preview
>> >
>> > I've seen it, right around 3.1.0 or 3.1.1, but not since...
>>
>> No problems with this.
>>
>> >
>> >> mostly folder-options -> rapair let it appear in the list
>> >> sometimes this is need twice, i can find no logic there
>> >> maybe some timing/race-conditions, hard to debug
>> >>
>> >> * Apple Mail - i hate it
>> >> POP3 as well as IMAP until now 3 messages which are
>> >> reported to me with empy or crippeled body
>> >> AFAIK all times multipart-messages
>> >
>> > That is possibly a combination of the exact IMAP commands that expose
>> a
>> > bug in GMime. Only way to know for sure is by running the IMAP
>> commands
>> > Apple uses to fetch a message, against the exact MIME structure of
>> the
>> > message involved.
>> >
>> > And even updating GMime is not guaranteed to fix things, since I've
>> > worked around problems and limitations in GMime in the past, rather
>> than
>> > getting them fixed or dealt with in GMime proper.
>> >
>> > GMime is really nice and it's author very helpful. It does however
>> > depend on downstream users like dbmail for battle-testing some parts
>> of
>> > the code.
>>
>> My 10 Apple Mail clients don't have this problem.
>>
>> >
>> >> in all cases no problem in roundcube-mail or Thunderbird
>> >
>> > Good. I really love how much more responsive especially imap now is,
>> > compared with 3.0.2.
>> >
>> >> * twice one thread appeared to consume 100% CPU without
>> >> lead to troubles for active users and SIGTERM ignroed
>> >
>> > I would try to strace the thread involved. Maybe that will give me
>> some
>> > hint as to where it's happening in the code.
>> >
>> > top -bH -n 1 -p <PID>
>> >
>> > should give you a listing of the threads, with process-ids per
>> thread.
>> > Maybe one particular thread is causing havoc, which you can then
>> strace.
>> >
>> >
>>
>> I would upgrade use SIGHUP to debug, but no time atm and my customers
>> should have a short period of time with a great working mail server
>> without experiments ;-)
>> >
>> >
>> >
>> >> _____________________________________________
>> >>
>> >> [root [at] mai:~]$ systemctl status dbmail-imapd.service
>> >> dbmail-imapd.service - DBMail IMAP Server
>> >> Loaded: loaded (/usr/lib/systemd/system/dbmail-imapd.service;
>> >> enabled)
>> >> Active: active (running) since Di 2013-08-06 14:08:19 CEST; 3
>> days
>> >> ago
>> >> Main PID: 5921 (dbmail-imapd)
>> >> CGroup: name=systemd:/system/dbmail-imapd.service
>> >> └─5921 /usr/sbin/dbmail-imapd -D
>> >
>>
>>
>> thx
>>
>> --
>> Harald Leithner
>>
>> ITronic
>> Vogelweidplatz 12, 1150 Wien, Austria
>> Tel: +43-1-786 23 88
>> Fax: +43-1-98 52 077
>> Mobil: +43-699-123 78 4 78
>> Mail: leithner [at] itronic | itronic.at
>> _______________________________________________
>> DBmail mailing list
>> DBmail [at] dbmail
>> http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
>
> _______________________________________________
> DBmail mailing list
> DBmail [at] dbmail
> http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

--
Harald Leithner

ITronic
Vogelweidplatz 12, 1150 Wien, Austria
Tel: +43-1-786 23 88
Fax: +43-1-98 52 077
Mobil: +43-699-123 78 4 78
Mail: leithner [at] itronic | itronic.at
_______________________________________________
DBmail mailing list
DBmail [at] dbmail
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

paul at nfg

Aug 12, 2013, 2:46 AM

Post #6 of 11 (12 views)
Permalink

Re: dbmail 3.1.2 status [In reply to]

On 08/12/2013 11:33 AM, Harald Leithner wrote:

> Its really great that you add new stuff, do you know a table which
> client supports with imap extension?

If you find such a table, please share!

> Any thing new on RFC6154, if I understood it correctly you can extend
> the users table with the 7 Fields and set the SPECIAL-USE defaults in
> dbmail.conf.

thanks for reminding me. Working on CONDSTORE I almost forgot.

rfc6154 extends rfc5258: the extended list command. Those two, plus
QRESYNC (5162) are on my TODO before 3.2.0

--
________________________________________________________________
Paul J Stevens pjstevns @ gmail, twitter, skype, linkedin

* Premium Hosting Services and Web Application Consultancy *

www.nfg.nl/info [at] nfg/+31.85.877.99.97
________________________________________________________________
_______________________________________________
DBmail mailing list
DBmail [at] dbmail
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

h.reindl at thelounge

Aug 12, 2013, 6:16 AM

Post #7 of 11 (12 views)
Permalink

Re: dbmail 3.1.2 status [In reply to]

Am 12.08.2013 11:33, schrieb Harald Leithner:
>>> * memory usage: perfect, dbmail-imapd currentyl 133 MB
>>> and sometimes it goes below 100 MB
>>> so no leaks and dail restarts needed
>>
>> I see the same, and still improving in the master branch. And it's
>> not only the memory footprint that improves because of this, but also
>> speed to some extend.
>
> Uptime 4 Days consumes 882M mb memory, it leaks about 200 MB per day. But performance is great.

uhm did you build it with 'jemalloc' which my builds are?

>>> * IMAP / Tunderbird: in the message-list randomly empty subject
>>> it is present in the preview
>>
>> I've seen it, right around 3.1.0 or 3.1.1, but not since...
>
> No problems with this

there are hours where it does not happen and sometimes it
is really frustrating - maybe it has something to do
with the fact taht 99% of my mails are sieve-filtered
in folders and INBOX is meaningsless here?

>>> mostly folder-options -> rapair let it appear in the list
>>> sometimes this is need twice, i can find no logic there
>>> maybe some timing/race-conditions, hard to debug
>>>
>>> * Apple Mail - i hate it
>>> POP3 as well as IMAP until now 3 messages which are
>>> reported to me with empy or crippeled body
>>> AFAIK all times multipart-messages
>>
> My 10 Apple Mail clients don't have this problem

i had only 3 cases from which i know
so it is not a permanently problem which makes it hard to debug

BTW:
with 3.1.2 we are down at 150 queries per second while 3.0.x
raised up to 1200 qps - AFAIR i made notice last year of this
on the list and now it's perfect again

Attachments:

signature.asc (0.26 KB)

harald.leithner at itronic

Aug 12, 2013, 6:30 AM

Post #8 of 11 (12 views)
Permalink

Re: dbmail 3.1.2 status [In reply to]

Am 12.08.2013, 15:16 Uhr, schrieb Reindl Harald <h.reindl [at] thelounge>:

>
>
> Am 12.08.2013 11:33, schrieb Harald Leithner:
>>>> * memory usage: perfect, dbmail-imapd currentyl 133 MB
>>>> and sometimes it goes below 100 MB
>>>> so no leaks and dail restarts needed
>>>
>>> I see the same, and still improving in the master branch. And it's
>>> not only the memory footprint that improves because of this, but also
>>> speed to some extend.
>>
>> Uptime 4 Days consumes 882M mb memory, it leaks about 200 MB per day.
>> But performance is great.
>
> uhm did you build it with 'jemalloc' which my builds are?

yes build with jemalloc

I'm not sure if EPOLL has something todo with it but I have

export EVENT_NOEPOLL=yes

activated. Maybe it has something todo with SSL?

>
>>>> * IMAP / Tunderbird: in the message-list randomly empty subject
>>>> it is present in the preview
>>>
>>> I've seen it, right around 3.1.0 or 3.1.1, but not since...
>>
>> No problems with this
>
> there are hours where it does not happen and sometimes it
> is really frustrating - maybe it has something to do
> with the fact taht 99% of my mails are sieve-filtered
> in folders and INBOX is meaningsless here?
>
>>>> mostly folder-options -> rapair let it appear in the list
>>>> sometimes this is need twice, i can find no logic there
>>>> maybe some timing/race-conditions, hard to debug
>>>>
>>>> * Apple Mail - i hate it
>>>> POP3 as well as IMAP until now 3 messages which are
>>>> reported to me with empy or crippeled body
>>>> AFAIK all times multipart-messages
>>>
>> My 10 Apple Mail clients don't have this problem
>
> i had only 3 cases from which i know
> so it is not a permanently problem which makes it hard to debug
>
> BTW:
> with 3.1.2 we are down at 150 queries per second while 3.0.x
> raised up to 1200 qps - AFAIR i made notice last year of this
> on the list and now it's perfect again
>

--
Harald Leithner

ITronic
Vogelweidplatz 12, 1150 Wien, Austria
Tel: +43-1-786 23 88
Fax: +43-1-98 52 077
Mobil: +43-699-123 78 4 78
Mail: leithner [at] itronic | itronic.at
_______________________________________________
DBmail mailing list
DBmail [at] dbmail
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

h.reindl at thelounge

Aug 12, 2013, 6:34 AM

Post #9 of 11 (12 views)
Permalink

Re: dbmail 3.1.2 status [In reply to]

Am 12.08.2013 15:30, schrieb Harald Leithner:
> Am 12.08.2013, 15:16 Uhr, schrieb Reindl Harald <h.reindl [at] thelounge>:
>> Am 12.08.2013 11:33, schrieb Harald Leithner:
>>>>> * memory usage: perfect, dbmail-imapd currentyl 133 MB
>>>>> and sometimes it goes below 100 MB
>>>>> so no leaks and dail restarts needed
>>>>
>>>> I see the same, and still improving in the master branch. And it's
>>>> not only the memory footprint that improves because of this, but also
>>>> speed to some extend.
>>>
>>> Uptime 4 Days consumes 882M mb memory, it leaks about 200 MB per day. But performance is great.
>>
>> uhm did you build it with 'jemalloc' which my builds are?
>
> yes build with jemalloc
>
> I'm not sure if EPOLL has something todo with it but I have
> export EVENT_NOEPOLL=yes

good question - no idea

> Maybe it has something todo with SSL?

that sounds like a possible reason

we have dovecot in front of pop3d/imapd as proxy because dbmail 2.x
did not support TSL or even CRAM-MD5 and it's also a lot less pain
having the same auth-backend with the same configurations for
postfix SASL which supports dovecot instead cyrus

most likely caused by the proxy in front re-connects from
mobile clients are buffered without touch dbmail in any way

Attachments:

signature.asc (0.26 KB)

paul at nfg

Aug 12, 2013, 7:33 AM

Post #10 of 11 (12 views)
Permalink

Re: dbmail 3.1.2 status [In reply to]

On 08/12/2013 03:30 PM, Harald Leithner wrote:
>> uhm did you build it with 'jemalloc' which my builds are?
>
> yes build with jemalloc

That won't affect any kind of leakage. Jemalloc is a *fast* allocator
though.

>
> I'm not sure if EPOLL has something todo with it but I have
>
> export EVENT_NOEPOLL=yes
>
> activated. Maybe it has something todo with SSL?

No, it has to do with libevent. Apparently some people experience linux
kernel bugs with epoll. EVENT_NOEPOLL disables epoll.

--
________________________________________________________________
Paul J Stevens pjstevns @ gmail, twitter, skype, linkedin

* Premium Hosting Services and Web Application Consultancy *

www.nfg.nl/info [at] nfg/+31.85.877.99.97
________________________________________________________________
_______________________________________________
DBmail mailing list
DBmail [at] dbmail
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

h.reindl at thelounge

Aug 12, 2013, 7:35 AM

Post #11 of 11 (12 views)
Permalink

Re: dbmail 3.1.2 status [In reply to]

Am 12.08.2013 16:33, schrieb Paul J Stevens:
> On 08/12/2013 03:30 PM, Harald Leithner wrote:
>>> uhm did you build it with 'jemalloc' which my builds are?
>>
>> yes build with jemalloc
>
> That won't affect any kind of leakage. Jemalloc is a *fast* allocator
> though.
>
>>
>> I'm not sure if EPOLL has something todo with it but I have
>>
>> export EVENT_NOEPOLL=yes
>>
>> activated. Maybe it has something todo with SSL?
>
> No, it has to do with libevent. Apparently some people experience linux
> kernel bugs with epoll. EVENT_NOEPOLL disables epoll

so kernel / libevent could be relevant?

[root [at] mai:~]$ rpm -q kernel
kernel-3.10.5-101.fc18.x86_64

[root [at] mai:~]$ rpm -q libevent
libevent-2.0.21-4.fc18.20130729.rh.x86_64

Attachments:

signature.asc (0.26 KB)

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads