Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: DBMail: users

status of dbmail from git

  

 
DBMail users RSS feed   Index | Next | Previous | View Threaded


aptem at ngs

Feb 12, 2009, 4:39 AM

Post #1 of 25 (1389 views)
Permalink
status of dbmail from git
Hello again.

Dbmail from git is broken or it should perform functions correctly?
I've got garbage in headers and bodies.
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


paul at nfg

Feb 12, 2009, 5:21 AM

Post #2 of 25 (1343 views)
Permalink
Re: status of dbmail from git [In reply to]
Artem,

there is a reason for doing releases, you know.

if you don't know how to debug this beast, try running dbmail-2.2.11
which is well-tested production code.

code from the git trees is unstable development code. Given the massive
changes in some of the libraries we consume (esp libzdb, gmime), and the
major bugs in some databases (esp mysql-5.1) I can not help you without
further information.



Artem Bokhan wrote:
> Hello again.
>
> Dbmail from git is broken or it should perform functions correctly?
> I've got garbage in headers and bodies.
> _______________________________________________
> DBmail mailing list
> DBmail[at]dbmail.org
> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>


--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


APTEM at ngs

Feb 12, 2009, 6:53 AM

Post #3 of 25 (1343 views)
Permalink
Re: status of dbmail from git [In reply to]
Paul, all I want to know - are you trying to keep git version
consistent? In other words, do you accept/intrested in bug reports?
In some future I will need 2.3.x/2.4.x for production, not 2.2.x, so I
want to help you (and myself) with testing (if you are intrested in
that, of course).

What versions of glibc/gmime/libzdb are known to work with git version?

Paul J Stevens :
> Artem,
>
> there is a reason for doing releases, you know.
>
> if you don't know how to debug this beast, try running dbmail-2.2.11
> which is well-tested production code.
>
> code from the git trees is unstable development code. Given the massive
> changes in some of the libraries we consume (esp libzdb, gmime), and the
> major bugs in some databases (esp mysql-5.1) I can not help you without
> further information.
>
>
>
> Artem Bokhan wrote:
>
>> Hello again.
>>
>> Dbmail from git is broken or it should perform functions correctly?
>> I've got garbage in headers and bodies.
>> _______________________________________________
>> DBmail mailing list
>> DBmail[at]dbmail.org
>> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>>
>>
>
>
>

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


paul at nfg

Feb 12, 2009, 10:16 AM

Post #4 of 25 (1342 views)
Permalink
Re: status of dbmail from git [In reply to]
Bokhan Artem wrote:
> Paul, all I want to know - are you trying to keep git version
> consistent? In other words, do you accept/intrested in bug reports?
> In some future I will need 2.3.x/2.4.x for production, not 2.2.x, so I
> want to help you (and myself) with testing (if you are intrested in
> that, of course).

Don't get me wrong. Of course I'm interested in any and all help in
testing and validating 2.3.x so we can get ready for 2.4.0

But what doesn't help is undocumented reports saying "it's broken".

So: if you run into a problem please be specific, and if at all
possible, please provide steps to reproduce it.

>
> What versions of glibc/gmime/libzdb are known to work with git version?

GLibc is not a problem.

Gmime-2.2.22
libzdb-2.3

should work. gmime-2.3 and gmime-2.4 have not been tested yet.

Main development platforms are debian/lenny and ubuntu/hardy - though
I'm currently switching to ubunty/intrepid. Both i386 and amd64 archs
are tested extensively before releases. Same for all three database
backends.


So please do report problems, but do expect problems and be willing to
drill down on them.

file_logging_levels=511 and gdb are your friends.

thanks.


>
> Paul J Stevens :
>> Artem,
>>
>> there is a reason for doing releases, you know.
>>
>> if you don't know how to debug this beast, try running dbmail-2.2.11
>> which is well-tested production code.
>>
>> code from the git trees is unstable development code. Given the massive
>> changes in some of the libraries we consume (esp libzdb, gmime), and the
>> major bugs in some databases (esp mysql-5.1) I can not help you without
>> further information.
>>
>>
>>
>> Artem Bokhan wrote:
>>
>>> Hello again.
>>>
>>> Dbmail from git is broken or it should perform functions correctly?
>>> I've got garbage in headers and bodies.
>>> _______________________________________________
>>> DBmail mailing list
>>> DBmail[at]dbmail.org
>>> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>>>
>>>
>>
>>
>>
>
> _______________________________________________
> DBmail mailing list
> DBmail[at]dbmail.org
> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>


--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


aptem at ngs

Feb 13, 2009, 2:49 AM

Post #5 of 25 (1332 views)
Permalink
Re: status of dbmail from git [In reply to]
> GLibc is not a problem.
>
As I understood 2.14.x is needed to compile gmime-2.2
> Gmime-2.2.22
> libzdb-2.3
>
>
Yep, now dbmail (git) works. The problem was with libzdb-2.2
I will ask some more questions in this thread if you do not mind.

I tried to send message with 4mb attachment, and dbmail-lmtpd took about
16 cpu (3ghz pentium d) seconds to recieve it. Also I saw a lot of
syscalls/small reads with strace. One byte for one read? =/

I want to run dbmail on copy of real traffic to look for dbmail's
behavior, but it is not possible, too much cpu is used.

_____

gettimeofday({1234517829, 711207}, NULL) = 0
epoll_ctl(13, EPOLL_CTL_ADD, 17, {EPOLLIN, {u32=134817768,
u64=134817768}}) = 0
rt_sigprocmask(SIG_BLOCK, [HUP INT PIPE], NULL, 8) = 0
rt_sigaction(SIGINT, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGHUP, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGPIPE, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
gettimeofday({1234517829, 711545}, NULL) = 0
gettimeofday({1234517829, 711580}, NULL) = 0
rt_sigprocmask(SIG_UNBLOCK, [HUP INT PIPE], NULL, 8) = 0
epoll_wait(13, {{EPOLLIN, {u32=134817768, u64=134817768}}}, 1024,
300000) = 1
rt_sigprocmask(SIG_BLOCK, [HUP INT PIPE], NULL, 8) = 0
rt_sigaction(SIGINT, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGHUP, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGPIPE, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
epoll_ctl(13, EPOLL_CTL_DEL, 17, {EPOLLIN, {u32=134817768,
u64=134817768}}) = 0
gettimeofday({1234517829, 711905}, NULL) = 0
read(17, "/", 1) = 1
read(17, "/", 1) = 1
read(17, "+", 1) = 1
read(17, "A", 1) = 1
read(17, "f", 1) = 1
read(17, "d", 1) = 1
read(17, "o", 1) = 1
read(17, "g", 1) = 1
read(17, "d", 1) = 1
read(17, "w", 1) = 1
read(17, "a", 1) = 1
read(17, "A", 1) = 1
read(17, "f", 1) = 1
read(17, "b", 1) = 1
read(17, "g", 1) = 1
read(17, "g", 1) = 1
read(17, "d", 1) = 1
read(17, "m", 1) = 1
read(17, "y", 1) = 1
read(17, "L", 1) = 1
read(17, "x", 1) = 1
read(17, "u", 1) = 1
read(17, "i", 1) = 1
read(17, "R", 1) = 1
read(17, "/", 1) = 1
read(17, "v", 1) = 1
read(17, "/", 1) = 1
read(17, "/", 1) = 1
read(17, "U", 1) = 1
read(17, "I", 1) = 1
read(17, "v", 1) = 1
read(17, "D", 1) = 1
read(17, "6", 1) = 1
read(17, "I", 1) = 1
read(17, "n", 1) = 1
read(17, "+", 1) = 1
read(17, "/", 1) = 1
read(17, "/", 1) = 1
read(17, "9", 1) = 1
read(17, "a", 1) = 1
read(17, "6", 1) = 1
read(17, "A", 1) = 1
read(17, "t", 1) = 1
read(17, "0", 1) = 1
read(17, "+", 1) = 1
read(17, "P", 1) = 1
read(17, "+", 1) = 1
read(17, "L", 1) = 1
read(17, "2", 1) = 1
read(17, "D", 1) = 1
read(17, "P", 1) = 1
read(17, "A", 1) = 1
read(17, "i", 1) = 1
read(17, "k", 1) = 1
read(17, "X", 1) = 1
read(17, "a", 1) = 1
read(17, "Z", 1) = 1
read(17, "j", 1) = 1
read(17, "v", 1) = 1
read(17, "Y", 1) = 1
read(17, "c", 1) = 1
read(17, "w", 1) = 1
read(17, "K", 1) = 1
read(17, "L", 1) = 1
read(17, "2", 1) = 1
read(17, "D", 1) = 1
read(17, "P", 1) = 1
read(17, "A", 1) = 1
read(17, "i", 1) = 1
read(17, "k", 1) = 1
read(17, "W", 1) = 1
read(17, "4", 1) = 1
read(17, "\r", 1) = 1
read(17, "\n", 1) = 1
gettimeofday({1234517829, 717062}, NULL) = 0
epoll_ctl(13, EPOLL_CTL_ADD, 17, {EPOLLIN, {u32=134817768,
u64=134817768}}) = 0
rt_sigprocmask(SIG_BLOCK, [HUP INT PIPE], NULL, 8) = 0
rt_sigaction(SIGINT, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGHUP, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGPIPE, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
gettimeofday({1234517829, 717475}, NULL) = 0
gettimeofday({1234517829, 717512}, NULL) = 0
rt_sigprocmask(SIG_UNBLOCK, [HUP INT PIPE], NULL, 8) = 0
epoll_wait(13, {{EPOLLIN, {u32=134817768, u64=134817768}}}, 1024,
300000) = 1
rt_sigprocmask(SIG_BLOCK, [HUP INT PIPE], NULL, 8) = 0
rt_sigaction(SIGINT, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGHUP, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGPIPE, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
epoll_ctl(13, EPOLL_CTL_DEL, 17, {EPOLLIN, {u32=134817768,
u64=134817768}}) = 0
gettimeofday({1234517829, 717902}, NULL) = 0
read(17, "Z", 1) = 1
read(17, "j", 1) = 1
read(17, "v", 1) = 1
read(17, "Y", 1) = 1
read(17, "c", 1) = 1
read(17, "w", 1) = 1
read(17, "K", 1) = 1
read(17, "L", 1) = 1
read(17, "2", 1) = 1
read(17, "I", 1) = 1
read(17, "1", 1) = 1
read(17, "N", 1) = 1
read(17, "l", 1) = 1
read(17, "A", 1) = 1
read(17, "+", 1) = 1
read(17, "3", 1) = 1
read(17, "8", 1) = 1
read(17, "4", 1) = 1
read(17, "v", 1) = 1
read(17, "W", 1) = 1
read(17, "j", 1) = 1
read(17, "U", 1) = 1
read(17, "X", 1) = 1
read(17, "a", 1) = 1
read(17, "6", 1) = 1
read(17, "H", 1) = 1
read(17, "n", 1) = 1
read(17, "+", 1) = 1
read(17, "/", 1) = 1
read(17, "/", 1) = 1
read(17, "9", 1) = 1
read(17, "W", 1) = 1
read(17, "j", 1) = 1
read(17, "X", 1) = 1
read(17, "W", 1) = 1
read(17, "U", 1) = 1
read(17, "j", 1) = 1
read(17, "X", 1) = 1
read(17, "3", 1) = 1
read(17, "a", 1) = 1
read(17, "u", 1) = 1
read(17, "Q", 1) = 1
read(17, "g", 1) = 1
read(17, "A", 1) = 1
read(17, "A", 1) = 1
read(17, "A", 1) = 1
read(17, "D", 1) = 1
read(17, "z", 1) = 1
read(17, "p", 1) = 1
read(17, "W", 1) = 1
read(17, "a", 1) = 1
read(17, "l", 1) = 1
read(17, "X", 1) = 1
read(17, "o", 1) = 1
read(17, "1", 1) = 1
read(17, "N", 1) = 1
read(17, "l", 1) = 1
read(17, "I", 1) = 1
read(17, "v", 1) = 1
read(17, "W", 1) = 1
read(17, "j", 1) = 1
read(17, "U", 1) = 1
read(17, "W", 1) = 1
read(17, "4", 1) = 1
read(17, "6", 1) = 1
read(17, "F", 1) = 1
read(17, "v", 1) = 1
read(17, "+", 1) = 1
read(17, "/", 1) = 1
read(17, "/", 1) = 1
read(17, "+", 1) = 1
read(17, "N", 1) = 1
read(17, "\r", 1) = 1
read(17, "\n", 1) = 1
gettimeofday({1234517829, 722558}, NULL) = 0
epoll_ctl(13, EPOLL_CTL_ADD, 17, {EPOLLIN, {u32=134817768,
u64=134817768}}) = 0
rt_sigprocmask(SIG_BLOCK, [HUP INT PIPE], NULL, 8) = 0
rt_sigaction(SIGINT, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGHUP, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGPIPE, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
gettimeofday({1234517829, 723104}, NULL) = 0
gettimeofday({1234517829, 723262}, NULL) = 0
rt_sigprocmask(SIG_UNBLOCK, [HUP INT PIPE], NULL, 8) = 0
epoll_wait(13, {{EPOLLIN, {u32=134817768, u64=134817768}}}, 1024,
300000) = 1
rt_sigprocmask(SIG_BLOCK, [HUP INT PIPE], NULL, 8) = 0
rt_sigaction(SIGINT, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGHUP, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGPIPE, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
epoll_ctl(13, EPOLL_CTL_DEL, 17, {EPOLLIN, {u32=134817768,
u64=134817768}}) = 0
gettimeofday({1234517829, 723956}, NULL) = 0
read(17, "d", 1) = 1
read(17, "Z", 1) = 1
read(17, "S", 1) = 1
read(17, "N", 1) = 1
read(17, "f", 1) = 1
read(17, "b", 1) = 1
read(17, "i", 1) = 1
read(17, "5", 1) = 1
read(17, "C", 1) = 1
read(17, "A", 1) = 1
read(17, "A", 1) = 1
read(17, "A", 1) = 1
read(17, "A", 1) = 1
read(17, "P", 1) = 1
read(17, "O", 1) = 1
read(17, "l", 1) = 1
read(17, "Z", 1) = 1
read(17, "q", 1) = 1
read(17, "U", 1) = 1
read(17, "z", 1) = 1
read(17, "0", 1) = 1
read(17, "o", 1) = 1
read(17, "p", 1) = 1
read(17, "V", 1) = 1
read(17, "u", 1) = 1
read(17, "D", 1) = 1
read(17, "P", 1) = 1
read(17, "A", 1) = 1
read(17, "i", 1) = 1
read(17, "k", 1) = 1
read(17, "X", 1) = 1
read(17, "a", 1) = 1
read(17, "6", 1) = 1
read(17, "K", 1) = 1
read(17, "V", 1) = 1
read(17, "z", 1) = 1
read(17, "+", 1) = 1
read(17, "P", 1) = 1
read(17, "+", 1) = 1
read(17, "L", 1) = 1
read(17, "2", 1) = 1
read(17, "I", 1) = 1
read(17, "1", 1) = 1
read(17, "F", 1) = 1
read(17, "u", 1) = 1
read(17, "O", 1) = 1
read(17, "j", 1) = 1
read(17, "T", 1) = 1
read(17, "C", 1) = 1
read(17, "Q", 1) = 1
read(17, "A", 1) = 1
read(17, "A", 1) = 1
read(17, "D", 1) = 1
read(17, "7", 1) = 1
read(17, "f", 1) = 1
read(17, "A", 1) = 1
read(17, "U", 1) = 1
read(17, "I", 1) = 1
read(17, "1", 1) = 1
read(17, "F", 1) = 1
read(17, "2", 1) = 1
read(17, "u", 1) = 1
read(17, "j", 1) = 1
read(17, "H", 1) = 1
read(17, "C", 1) = 1
read(17, "Q", 1) = 1
read(17, "A", 1) = 1
read(17, "A", 1) = 1
read(17, "D", 1) = 1
read(17, "7", 1) = 1
read(17, "f", 1) = 1
read(17, "A", 1) = 1
read(17, "\r", 1) = 1
read(17, "\n", 1) = 1
gettimeofday({1234517829, 729687}, NULL) = 0
epoll_ctl(13, EPOLL_CTL_ADD, 17, {EPOLLIN, {u32=134817768,
u64=134817768}}) = 0
rt_sigprocmask(SIG_BLOCK, [HUP INT PIPE], NULL, 8) = 0
rt_sigaction(SIGINT, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGHUP, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGPIPE, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
gettimeofday({1234517829, 729941}, NULL) = 0
gettimeofday({1234517829, 729976}, NULL) = 0
rt_sigprocmask(SIG_UNBLOCK, [HUP INT PIPE], NULL, 8) = 0
epoll_wait(13, {{EPOLLIN, {u32=134817768, u64=134817768}}}, 1024,
300000) = 1
rt_sigprocmask(SIG_BLOCK, [HUP INT PIPE], NULL, 8) = 0
rt_sigaction(SIGINT, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGHUP, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
rt_sigaction(SIGPIPE, {0xb7d25d9c, [HUP INT PIPE], SA_RESTART}, NULL, 8) = 0
epoll_ctl(13, EPOLL_CTL_DEL, 17, {EPOLLIN, {u32=134817768,
u64=134817768}}) = 0
gettimeofday({1234517829, 730559}, NULL) = 0
read(17, "W", 1) = 1
read(17, "u", 1) = 1
read(17, "i", 1) = 1
read(17, "G", 1) = 1
read(17, "c", 1) = 1
read(17, "/", 1) = 1
read(17, "j", 1) = 1
read(17, "/", 1) = 1
read(17, "i", 1) = 1
read(17, "/", 1) = 1
read(17, "D", 1) = 1
read(17, "r", 1) = 1
read(17, "B", 1) = 1
read(17, "G", 1) = 1
read(17, "a", 1) = 1
read(17, "D", 1) = 1
read(17, "w", 1) = 1
read(17, "w", 1) = 1
read(17, "J", 1) = 1
read(17, "m", 1) = 1
read(17, "g", 1) = 1
read(17, "/", 1) = 1
read(17, "t", 1) = 1
read(17, "A", 1) = 1
read(17, "c", 1) = 1
read(17, "z", 1) = 1
read(17, "S", 1) = 1
read(17, "N", 1) = 1
read(17, "R", 1) = 1
read(17, "d", 1) = 1
read(17, "r", 1) = 1
read(17, "o", 1) = 1
read(17, "q", 1) = 1
read(17, "A", 1) = 1
read(17, "k", 1) = 1
read(17, "A", 1) = 1
read(17, "A", 1) = 1
read(17, "A", 1) = 1
read(17, "+", 1) = 1
read(17, "3", 1) = 1
read(17, "w", 1) = 1
read(17, "D", 1) = 1
read(17, "P", 1) = 1
read(17, "S", 1) = 1
read(17, "i", 1) = 1
read(17, "l", 1) = 1



_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


paul at nfg

Feb 13, 2009, 3:17 AM

Post #6 of 25 (1332 views)
Permalink
Re: status of dbmail from git [In reply to]
Artem Bokhan wrote:
>
>> GLibc is not a problem.
>>
> As I understood 2.14.x is needed to compile gmime-2.2
>> Gmime-2.2.22
>> libzdb-2.3
>>
>>
> Yep, now dbmail (git) works. The problem was with libzdb-2.2

great.

> I will ask some more questions in this thread if you do not mind.
>
> I tried to send message with 4mb attachment, and dbmail-lmtpd took about
> 16 cpu (3ghz pentium d) seconds to recieve it. Also I saw a lot of
> syscalls/small reads with strace. One byte for one read? =/

That's correct. In the lmtp context, there is no prior notification of
the number of octects that need to be read from the network. So we take
it one-by-one, until we read '\r\n\.\r\n' that marks the end of the
message.

In the contect of imap-append for example the message size *is*
indicated beforehand, so we can do more efficient reads.

If you can think of, or direct me to a better approach for doing async
network reads in lmtp, please do share. I'm sure it's possible. But I
havent really dug into this yet.


--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


aptem at ngs

Feb 13, 2009, 3:57 AM

Post #7 of 25 (1333 views)
Permalink
Re: status of dbmail from git [In reply to]
Paul J Stevens пишет:
>
> If you can think of, or direct me to a better approach for doing async
> network reads in lmtp, please do share. I'm sure it's possible. But I
> havent really dug into this yet.
>
>
>
Although I'm not sure this can help you, the smallest example of fast
and simple lmtp server it's possible to find with postfix,
/src/smtpstone/smtp-sink.c


NAME
smtp-sink - multi-threaded SMTP/LMTP test server

SYNOPSIS
smtp-sink [options] [inet:][host]:port backlog

smtp-sink [options] unix:pathname backlog

DESCRIPTION
smtp-sink listens on the named host (or address) and port. It takes
SMTP messages from the network and throws them away. The purpose is to
measure client performance, not protocol compliance.

smtp-sink may also be configured to capture each mail delivery transac‐
tion to file. Since disk latencies are large compared to network
delays, this mode of operation can reduce the maximal performance by
several orders of magnitude.

Connections can be accepted on IPv4 or IPv6 endpoints, or on UNIX-
domain sockets. IPv4 and IPv6 are the default. This program is the
complement of the smtp-source(1) program.

Note: this is an unsupported test program. No attempt is made to main‐
tain compatibility between successive versions.

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


paul at nfg

Feb 13, 2009, 4:50 AM

Post #8 of 25 (1333 views)
Permalink
Re: status of dbmail from git [In reply to]
I looked at the code for smtp-sink.c.

It does the same as dbmail-lmtp: read one char at a time. Only
difference is that it uses getc(2) rather than read(2).



Artem Bokhan wrote:
> Paul J Stevens пишет:
>>
>> If you can think of, or direct me to a better approach for doing async
>> network reads in lmtp, please do share. I'm sure it's possible. But I
>> havent really dug into this yet.
>>
>>
>>
> Although I'm not sure this can help you, the smallest example of fast
> and simple lmtp server it's possible to find with postfix,
> /src/smtpstone/smtp-sink.c
>
>
> NAME
> smtp-sink - multi-threaded SMTP/LMTP test server
>
> SYNOPSIS
> smtp-sink [options] [inet:][host]:port backlog
>
> smtp-sink [options] unix:pathname backlog
>
> DESCRIPTION
> smtp-sink listens on the named host (or address) and port. It takes
> SMTP messages from the network and throws them away. The purpose is to
> measure client performance, not protocol compliance.
>
> smtp-sink may also be configured to capture each mail delivery transac‐
> tion to file. Since disk latencies are large compared to network
> delays, this mode of operation can reduce the maximal performance by
> several orders of magnitude.
>
> Connections can be accepted on IPv4 or IPv6 endpoints, or on UNIX-
> domain sockets. IPv4 and IPv6 are the default. This program is the
> complement of the smtp-source(1) program.
>
> Note: this is an unsupported test program. No attempt is made to main‐
> tain compatibility between successive versions.
>
> _______________________________________________
> DBmail mailing list
> DBmail[at]dbmail.org
> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>


--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


aptem at ngs

Feb 13, 2009, 5:09 AM

Post #9 of 25 (1333 views)
Permalink
Re: status of dbmail from git [In reply to]
Paul J Stevens пишет:
> I looked at the code for smtp-sink.c.
>
> It does the same as dbmail-lmtp: read one char at a time. Only
> difference is that it uses getc(2) rather than read(2).
>
But it looks like it works with 4k blocks. It's 4096 times faster :)

read(5, "AiAC0AIgA/AD8AIgBABC4AIgBfAC0AOw"..., 4096) = 4096
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
read(5, "MDEIAAEx\r\nADEAMwA2ABAEMAAwADIAFA"..., 4096) = 4096
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
read(5, "ICAgICANAAA4NjAwMjExMTAzMjM5CgAA"..., 4096) = 4096
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
read(5, "EwMzI0MQgAADAwMTMyLTE1DQAAODYw\r\n"..., 4096) = 4096
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
read(5, "MDcu\r\nMTk4NiAMAAAgMTEuMTAuMTk4Ni"..., 4096) = 4096
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
read(5, "AAYDAACYLwAAegMAAAswAADtAwAAmzAA"..., 4096) = 4096
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
read(5, "hQUGhvdG9GaXgAQkFTSUMASFBS\r\nUE9G"..., 4096) = 4096
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
read(5, "\r\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 4096) = 4096
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
read(5, "AAAAEwEAAA8AAAATAQAAAAAAAAAAAAAA"..., 4096) = 4096
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
read(5, "wAAAAAACAAAMCyQCEA6RQB\r\nQQsAfgIK"..., 4096) = 4096
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
read(5, "gABgAfAAAa4kA2AIBPAkE2AABM7UAcAA"..., 4096) = 4096
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
read(5, "AAD9AAoAKgACACMADAAAAP0ACgAqAAMA"..., 4096) = 4096


>
>
> Artem Bokhan wrote:
>
>> Paul J Stevens пишет:
>>
>>> If you can think of, or direct me to a better approach for doing async
>>> network reads in lmtp, please do share. I'm sure it's possible. But I
>>> havent really dug into this yet.
>>>
>>>
>>>
>>>
>> Although I'm not sure this can help you, the smallest example of fast
>> and simple lmtp server it's possible to find with postfix,
>> /src/smtpstone/smtp-sink.c
>>
>>
>> NAME
>> smtp-sink - multi-threaded SMTP/LMTP test server
>>
>> SYNOPSIS
>> smtp-sink [options] [inet:][host]:port backlog
>>
>> smtp-sink [options] unix:pathname backlog
>>
>> DESCRIPTION
>> smtp-sink listens on the named host (or address) and port. It takes
>> SMTP messages from the network and throws them away. The purpose is to
>> measure client performance, not protocol compliance.
>>
>> smtp-sink may also be configured to capture each mail delivery transac‐
>> tion to file. Since disk latencies are large compared to network
>> delays, this mode of operation can reduce the maximal performance by
>> several orders of magnitude.
>>
>> Connections can be accepted on IPv4 or IPv6 endpoints, or on UNIX-
>> domain sockets. IPv4 and IPv6 are the default. This program is the
>> complement of the smtp-source(1) program.
>>
>> Note: this is an unsupported test program. No attempt is made to main‐
>> tain compatibility between successive versions.
>>
>> _______________________________________________
>> DBmail mailing list
>> DBmail[at]dbmail.org
>> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>>
>>
>
>
>

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


aptem at ngs

Feb 13, 2009, 5:19 AM

Post #10 of 25 (1332 views)
Permalink
Re: status of dbmail from git [In reply to]
That was strace -f -p 4676 2>&1 | grep -E "^read|^poll"

Artem Bokhan пишет:
> Paul J Stevens пишет:
>> I looked at the code for smtp-sink.c.
>>
>> It does the same as dbmail-lmtp: read one char at a time. Only
>> difference is that it uses getc(2) rather than read(2).
>>
> But it looks like it works with 4k blocks. It's 4096 times faster :)
>
> read(5, "AiAC0AIgA/AD8AIgBABC4AIgBfAC0AOw"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
> read(5, "MDEIAAEx\r\nADEAMwA2ABAEMAAwADIAFA"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
> read(5, "ICAgICANAAA4NjAwMjExMTAzMjM5CgAA"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


paul at nfg

Feb 13, 2009, 5:52 AM

Post #11 of 25 (1333 views)
Permalink
Re: status of dbmail from git [In reply to]
You are correct. I didn't look too well. They do buffered reads.
I did those as well some time ago (between 2.3.2 and 2.3.5) but undid
them because I suspected some unwanted side effects. Turned out the
side-effects weren't caused by the IO buffering, so maybe that change
should be reverted somehow....



Artem Bokhan wrote:
> That was strace -f -p 4676 2>&1 | grep -E "^read|^poll"
>
> Artem Bokhan пишет:
>> Paul J Stevens пишет:
>>> I looked at the code for smtp-sink.c.
>>>
>>> It does the same as dbmail-lmtp: read one char at a time. Only
>>> difference is that it uses getc(2) rather than read(2).
>>>
>> But it looks like it works with 4k blocks. It's 4096 times faster :)
>>
>> read(5, "AiAC0AIgA/AD8AIgBABC4AIgBfAC0AOw"..., 4096) = 4096
>> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
>> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
>> read(5, "MDEIAAEx\r\nADEAMwA2ABAEMAAwADIAFA"..., 4096) = 4096
>> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
>> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
>> read(5, "ICAgICANAAA4NjAwMjExMTAzMjM5CgAA"..., 4096) = 4096
>> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
>> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
>
> _______________________________________________
> DBmail mailing list
> DBmail[at]dbmail.org
> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>


--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


paul at nfg

Feb 13, 2009, 5:54 AM

Post #12 of 25 (1332 views)
Permalink
Re: status of dbmail from git [In reply to]
Artem Bokhan wrote:
>
> strace -c -f ./dbmail-lmtpd -f ../etc/dbmail.conf
> % time seconds usecs/call calls errors syscall
> ------ ----------- ----------- --------- --------- ----------------
> 82.25 73.907258 13 5828219 read
> 5.71 5.128893 11 466512 rt_sigaction

Jikes! Point taken!

--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


aptem at ngs

Feb 13, 2009, 5:56 AM

Post #13 of 25 (1333 views)
Permalink
Re: status of dbmail from git [In reply to]
Results of receiving of the same letter with smtp-sink and dbmail-lmtpd

strace -c -f ./smtp-sink -L -d /tmp/pool2/%Y/%m/%d/%H/%M:%S- -u root
0.0.0.0:24 1024
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
63.81 0.446398 218 2046 epoll_wait
9.79 0.068460 11 6139 time
8.21 0.057440 9 6390 poll
7.43 0.052006 37 1392 write
5.89 0.041224 13 3205 read
4.59 0.032137 10 3201 gettimeofday
0.05 0.000338 19 18 1 open
0.03 0.000224 9 26 old_mmap
0.03 0.000187 8 23 close
0.03 0.000178 36 5 mkdir
0.02 0.000154 154 1 execve
0.02 0.000151 17 9 5 stat64
0.01 0.000093 5 18 fcntl64
0.01 0.000090 5 17 fstat64
0.01 0.000085 12 7 munmap
0.01 0.000074 7 10 10 access
0.01 0.000069 12 6 socket
0.01 0.000055 14 4 4 connect
0.01 0.000052 13 4 mmap2
0.00 0.000029 10 3 epoll_ctl
0.00 0.000024 24 1 listen
0.00 0.000020 5 4 _llseek
0.00 0.000017 17 1 accept
0.00 0.000015 5 3 brk
0.00 0.000014 7 2 setsockopt
0.00 0.000009 9 1 epoll_create
0.00 0.000009 9 1 bind
0.00 0.000006 6 1 uname
0.00 0.000006 6 1 setgroups32
0.00 0.000005 5 1 setrlimit
0.00 0.000005 5 1 rt_sigaction
0.00 0.000005 5 1 getrlimit
0.00 0.000005 5 1 geteuid32
0.00 0.000005 5 1 setuid32
0.00 0.000005 5 1 setgid32
0.00 0.000005 5 1 set_thread_area
------ ----------- ----------- --------- --------- ----------------
100.00 0.699599 22546 20 total

strace -c -f ./dbmail-lmtpd -f ../etc/dbmail.conf
Process 3678 attached
Process 3679 attached
Process 3678 detached
Process 3679 detached
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
82.25 73.907258 13 5828219 read
5.71 5.128893 11 466512 rt_sigaction
5.19 4.666613 15 311262 gettimeofday
2.31 2.072720 9 233256 rt_sigprocmask
2.13 1.916936 12 155486 epoll_ctl
1.19 1.068105 14 78296 write
0.68 0.611118 7 90235 _llseek
0.48 0.430506 6 77751 epoll_wait
0.01 0.012321 385 32 munmap
0.01 0.010507 184 57 mremap
0.01 0.006720 22 310 254 open
0.01 0.005247 9 593 poll
0.01 0.004626 75 62 1 close
0.00 0.003993 3993 1 execve
0.00 0.001638 10 163 120 stat64
0.00 0.000983 12 83 old_mmap
0.00 0.000514 47 11 5 connect
0.00 0.000404 8 51 fcntl64
0.00 0.000372 12 32 mmap2
0.00 0.000332 6 58 fstat64
0.00 0.000269 269 1 setsid
0.00 0.000253 127 2 clone
0.00 0.000232 21 11 5 setsockopt
0.00 0.000230 230 1 set_thread_area
0.00 0.000227 8 29 29 access
0.00 0.000205 21 10 time
0.00 0.000144 12 12 socket
0.00 0.000085 9 9 brk
0.00 0.000070 35 2 unlink
0.00 0.000067 8 8 send
0.00 0.000025 25 1 listen
0.00 0.000023 6 4 futex
0.00 0.000019 19 1 accept
0.00 0.000019 19 1 socketpair
0.00 0.000018 6 3 3 ioctl
0.00 0.000016 16 1 epoll_create
0.00 0.000015 15 1 shutdown
0.00 0.000013 13 1 chdir
0.00 0.000013 7 2 uname
0.00 0.000012 6 2 getrlimit
0.00 0.000011 6 2 getsid
0.00 0.000011 11 1 bind
0.00 0.000010 10 1 chmod
0.00 0.000009 9 1 _sysctl
0.00 0.000006 6 1 umask
0.00 0.000005 5 1 1 kill
0.00 0.000005 5 1 setuid32
0.00 0.000005 5 1 setgid32
0.00 0.000004 4 1 set_tid_address
------ ----------- ----------- --------- --------- ----------------
100.00 89.851827 7242582 418 total




Artem Bokhan пишет:
> Paul J Stevens пишет:
>> I looked at the code for smtp-sink.c.
>>
>> It does the same as dbmail-lmtp: read one char at a time. Only
>> difference is that it uses getc(2) rather than read(2).
>>
> But it looks like it works with 4k blocks. It's 4096 times faster :)
>
> read(5, "AiAC0AIgA/AD8AIgBABC4AIgBfAC0AOw"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
> read(5, "MDEIAAEx\r\nADEAMwA2ABAEMAAwADIAFA"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
> read(5, "ICAgICANAAA4NjAwMjExMTAzMjM5CgAA"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
> read(5, "EwMzI0MQgAADAwMTMyLTE1DQAAODYw\r\n"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
> read(5, "MDcu\r\nMTk4NiAMAAAgMTEuMTAuMTk4Ni"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
> read(5, "AAYDAACYLwAAegMAAAswAADtAwAAmzAA"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
> read(5, "hQUGhvdG9GaXgAQkFTSUMASFBS\r\nUE9G"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
> read(5, "\r\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
> read(5, "AAAAEwEAAA8AAAATAQAAAAAAAAAAAAAA"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
> read(5, "wAAAAAACAAAMCyQCEA6RQB\r\nQQsAfgIK"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
> read(5, "gABgAfAAAa4kA2AIBPAkE2AABM7UAcAA"..., 4096) = 4096
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
> read(5, "AAD9AAoAKgACACMADAAAAP0ACgAqAAMA"..., 4096) = 4096
>
>
>>
>>
>> Artem Bokhan wrote:
>>
>>> Paul J Stevens пишет:
>>>
>>>> If you can think of, or direct me to a better approach for doing async
>>>> network reads in lmtp, please do share. I'm sure it's possible. But I
>>>> havent really dug into this yet.
>>>>
>>>>
>>>>
>>> Although I'm not sure this can help you, the smallest example of fast
>>> and simple lmtp server it's possible to find with postfix,
>>> /src/smtpstone/smtp-sink.c
>>>
>>>
>>> NAME
>>> smtp-sink - multi-threaded SMTP/LMTP test server
>>>
>>> SYNOPSIS
>>> smtp-sink [options] [inet:][host]:port backlog
>>>
>>> smtp-sink [options] unix:pathname backlog
>>>
>>> DESCRIPTION
>>> smtp-sink listens on the named host (or address) and port. It takes
>>> SMTP messages from the network and throws them away. The purpose is to
>>> measure client performance, not protocol compliance.
>>>
>>> smtp-sink may also be configured to capture each mail delivery transac‐
>>> tion to file. Since disk latencies are large compared to network
>>> delays, this mode of operation can reduce the maximal performance by
>>> several orders of magnitude.
>>>
>>> Connections can be accepted on IPv4 or IPv6 endpoints, or on UNIX-
>>> domain sockets. IPv4 and IPv6 are the default. This program is the
>>> complement of the smtp-source(1) program.
>>>
>>> Note: this is an unsupported test program. No attempt is made to main‐
>>> tain compatibility between successive versions.
>>>
>>> _______________________________________________
>>> DBmail mailing list
>>> DBmail[at]dbmail.org
>>> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>>>
>>>
>>
>>
>>
>
> _______________________________________________
> DBmail mailing list
> DBmail[at]dbmail.org
> https://mailman.fastxs.nl/mailman/listinfo/dbmail

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


APTEM at ngs

Feb 13, 2009, 6:57 AM

Post #14 of 25 (1328 views)
Permalink
Re: status of dbmail from git [In reply to]
Paul J Stevens пишет:
> so maybe that change should be reverted somehow....
>
That would allow to start a test process :)
>
>
> Artem Bokhan wrote:
>
>> That was strace -f -p 4676 2>&1 | grep -E "^read|^poll"
>>
>> Artem Bokhan пишет:
>>
>>> Paul J Stevens пишет:
>>>
>>>> I looked at the code for smtp-sink.c.
>>>>
>>>> It does the same as dbmail-lmtp: read one char at a time. Only
>>>> difference is that it uses getc(2) rather than read(2).
>>>>
>>>>
>>> But it looks like it works with 4k blocks. It's 4096 times faster :)
>>>
>>> read(5, "AiAC0AIgA/AD8AIgBABC4AIgBfAC0AOw"..., 4096) = 4096
>>> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
>>> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
>>> read(5, "MDEIAAEx\r\nADEAMwA2ABAEMAAwADIAFA"..., 4096) = 4096
>>> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
>>> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
>>> read(5, "ICAgICANAAA4NjAwMjExMTAzMjM5CgAA"..., 4096) = 4096
>>> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 0) = 1
>>> poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 100000) = 1
>>>
>> _______________________________________________
>> DBmail mailing list
>> DBmail[at]dbmail.org
>> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>>
>>
>
>
>

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


paul at nfg

Feb 13, 2009, 8:10 AM

Post #15 of 25 (1327 views)
Permalink
Re: status of dbmail from git [In reply to]
Bokhan Artem wrote:
> Paul J Stevens пишет:
>> so maybe that change should be reverted somehow....
>>
> That would allow to start a test process :)

I've begun refactoring ci_read/ci_readln to use a read-buffer.

Stay tuned...


--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


APTEM at ngs

Feb 13, 2009, 9:50 AM

Post #16 of 25 (1328 views)
Permalink
Re: status of dbmail from git [In reply to]
> I will ask some more questions in this thread if you do not mind.

What is with memory handling? Looks like it just does not free. In the
next example I send 100Mb message in cycle.

while [ 1 ] ; do ps up 4598 && ./smtp-source -l 100000000 -m 1 -t e[at]mail
-c -L -S "dbmail test" -s 1 localhost:24 ; done

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
nobody 4598 0.0 0.1 8816 2428 ? S 23:10 0:00
./dbmail-lmtpd -f ../etc/dbmail.conf
1

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
nobody 4598 80.1 9.4 236568 196304 ? S 23:10 4:04
./dbmail-lmtpd -f ../etc/dbmail.conf
1

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
nobody 4598 77.8 14.1 367644 292748 ? S 23:10 8:13
./dbmail-lmtpd -f ../etc/dbmail.conf
1

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
nobody 4598 83.2 18.7 498720 389188 ? S 23:10 12:15
./dbmail-lmtpd -f ../etc/dbmail.conf
1

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
nobody 4598 86.2 23.4 629796 485624 ? S 23:10 16:20
./dbmail-lmtpd -f ../etc/dbmail.conf
1

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
nobody 4598 88.1 28.0 760872 582060 ? S 23:10 20:25
./dbmail-lmtpd -f ../etc/dbmail.conf
1

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
nobody 4598 89.3 32.7 891948 678496 ? S 23:10 24:29
./dbmail-lmtpd -f ../etc/dbmail.conf


_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


paul at nfg

Feb 13, 2009, 12:46 PM

Post #17 of 25 (1326 views)
Permalink
Re: status of dbmail from git [In reply to]
Bokhan Artem wrote:
>> I will ask some more questions in this thread if you do not mind.
>
> What is with memory handling? Looks like it just does not free. In the
> next example I send 100Mb message in cycle.
>
> while [ 1 ] ; do ps up 4598 && ./smtp-source -l 100000000 -m 1 -t e[at]mail
> -c -L -S "dbmail test" -s 1 localhost:24 ; done

nice test.

>
> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> nobody 4598 0.0 0.1 8816 2428 ? S 23:10 0:00
> ../dbmail-lmtpd -f ../etc/dbmail.conf
> 1

Someone should run that through valgrind!

I'll focus on the buffered reads for now, if you don't mind.

I wasn't aware of those tools in the postfix codetree.

Thanks.

--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


mysql.jorge at decimal

Feb 13, 2009, 12:58 PM

Post #18 of 25 (1323 views)
Permalink
RE: status of dbmail from git [In reply to]
> > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
> COMMAND
> > nobody 4598 0.0 0.1 8816 2428 ? S 23:10 0:00
> > ../dbmail-lmtpd -f ../etc/dbmail.conf
> > 1
>
> Someone should run that through valgrind!
>

Paul, this may be related to my problem, can I provide you such information?
Just run it in valgrind, and let it run for some time and sent you the info?

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


paul at nfg

Feb 14, 2009, 6:37 AM

Post #19 of 25 (1303 views)
Permalink
Re: status of dbmail from git [In reply to]
Jorge Bastos wrote:
>>> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
>> COMMAND
>>> nobody 4598 0.0 0.1 8816 2428 ? S 23:10 0:00
>>> ../dbmail-lmtpd -f ../etc/dbmail.conf
>>> 1
>> Someone should run that through valgrind!
>>
>
> Paul, this may be related to my problem, can I provide you such information?

Really? No Jorge, I seriously doubt the reading from the network has
anything to do with leaking filehandles.

--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


mysql.jorge at decimal

Feb 14, 2009, 11:34 AM

Post #20 of 25 (1301 views)
Permalink
RE: status of dbmail from git [In reply to]
> Paul, this may be related to my problem, can I provide you such
> information?
>
> Really? No Jorge, I seriously doubt the reading from the network has
> anything to do with leaking filehandles.
>

Hum I thouth it could be the same :P

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


APTEM at ngs

Feb 22, 2009, 6:38 AM

Post #21 of 25 (1250 views)
Permalink
Re: status of dbmail from git [In reply to]
> I will ask some more questions in this thread if you do not mind.

As I see, UIDs are sequential in context of the hole server. Are there
any ideas/plans to make UIDs sequential in context of a mailbox? It is
almost impossible for me to migrate from one type of numeration to
another, too many users.

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


APTEM at ngs

Feb 22, 2009, 7:01 AM

Post #22 of 25 (1250 views)
Permalink
Re: status of dbmail from git [In reply to]
> As I see, UIDs are sequential in context of the hole server.
"whole", sorry :)


_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


paul at nfg

Feb 22, 2009, 7:39 AM

Post #23 of 25 (1249 views)
Permalink
Re: status of dbmail from git [In reply to]
Bokhan Artem wrote:
>> I will ask some more questions in this thread if you do not mind.
>
> As I see, UIDs are sequential in context of the hole server. Are there
> any ideas/plans to make UIDs sequential in context of a mailbox? It is
> almost impossible for me to migrate from one type of numeration to
> another, too many users.

This has been discussed several times. There are no plans in that
direction however that I'm aware of.



--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


APTEM at ngs

Feb 22, 2009, 9:04 PM

Post #24 of 25 (1246 views)
Permalink
Re: status of dbmail from git [In reply to]
Paul J Stevens :
>
> If you can think of, or direct me to a better approach for doing async
> network reads in lmtp, please do share. I'm sure it's possible. But I
> havent really dug into this yet.
>
>
>

Paul, I thought about nginx
(http://sysoev.ru/nginx/nginx-0.7.37.tar.gz), it is widely used as http
server, but also has functions of smtp proxy (to save cpu,
processess/threads of mta; nginx uses single-process model). Main goals
of nginx are high perfomance and very small memory usage, about 3mb per
10000 tcp connections, it supports kqueue, epoll, rt signals, /dev/poll,
event ports, select and poll methods.

It uses recvfrom syscall with epoll. May be you are intrested in my
information.

recvfrom(25, " }\r\n "..., 4096, 0, NULL, NULL) = 4096
recvfrom(25, "/p><p /><p"..., 4096, 0, NULL, NULL) = 4096
recvfrom(25, "0=B8 =D1=8"..., 4096, 0, NULL, NULL) = 4096
recvfrom(25, "2=D1=80=D1"..., 4096, 0, NULL, NULL) = 2192
recvfrom(25, 0x575be0, 4096, 0, 0, 0) = -1 EAGAIN (Resource
temporarily unavailable)
recvfrom(68, "DATA\r\n", 4096, 0, NULL, NULL) = 6


_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail


paul at nfg

Feb 23, 2009, 9:15 AM

Post #25 of 25 (1243 views)
Permalink
Re: status of dbmail from git [In reply to]
Bokhan Artem wrote:
> Paul J Stevens :
>>
>> If you can think of, or direct me to a better approach for doing async
>> network reads in lmtp, please do share. I'm sure it's possible. But I
>> havent really dug into this yet.
>>
>>
>>
>
> Paul, I thought about nginx
> (http://sysoev.ru/nginx/nginx-0.7.37.tar.gz), it is widely used as http
> server, but also has functions of smtp proxy (to save cpu,
> processess/threads of mta; nginx uses single-process model). Main goals
> of nginx are high perfomance and very small memory usage, about 3mb per
> 10000 tcp connections, it supports kqueue, epoll, rt signals, /dev/poll,
> event ports, select and poll methods.

Mmm, pretty much like dbmail does also, using libevent. I haven't really
looked into recvfrom. Looks to me like it's much the same as read(2).

>
> It uses recvfrom syscall with epoll. May be you are intrested in my
> information.
>
> recvfrom(25, " }\r\n "..., 4096, 0, NULL, NULL) = 4096
> recvfrom(25, "/p><p /><p"..., 4096, 0, NULL, NULL) = 4096
> recvfrom(25, "0=B8 =D1=8"..., 4096, 0, NULL, NULL) = 4096
> recvfrom(25, "2=D1=80=D1"..., 4096, 0, NULL, NULL) = 2192
> recvfrom(25, 0x575be0, 4096, 0, 0, 0) = -1 EAGAIN (Resource
> temporarily unavailable)
> recvfrom(68, "DATA\r\n", 4096, 0, NULL, NULL) = 6
>
>
> _______________________________________________
> DBmail mailing list
> DBmail[at]dbmail.org
> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>


--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail

DBMail users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.