Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Qmail: users

Outgoing SMTP logging

 

 

Qmail users RSS feed   Index | Next | Previous | View Threaded


jsteele1997 at gmail

Feb 18, 2009, 11:15 AM

Post #1 of 7 (2607 views)
Permalink
Outgoing SMTP logging

Hello,

I am working on migrating my company's outgoing mail servers from
Windows servers running Microsoft's SMTP service to machines running
qmail. These servers relay notification (e.g. "Your order has been
shipped") mail from our application servers to our customers.

I have gotten mail flow working properly with qmail, but there is one
thing that I need that I cannot figure out. A few times per month,
there will be a customer who claims not to have received an email from
our system, and I go to the Microsoft SMTP service log file to
retrieve the log of the outgoing SMTP conversation that proves that we
sent the message, including what our server said, and what their
server said in response (less the DATA portion). I provide the log to
the customer so that they can determine the problem on their end.

How do I make qmail log the outgoing SMTP conversation?

Thank you,
Jim


jms1 at jms1

Feb 18, 2009, 12:30 PM

Post #2 of 7 (2485 views)
Permalink
Re: Outgoing SMTP logging [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2009-02-18, at 1415, jim steele wrote:
>
> I have gotten mail flow working properly with qmail, but there is one
> thing that I need that I cannot figure out. A few times per month,
> there will be a customer who claims not to have received an email from
> our system, and I go to the Microsoft SMTP service log file to
> retrieve the log of the outgoing SMTP conversation that proves that we
> sent the message, including what our server said, and what their
> server said in response (less the DATA portion). I provide the log to
> the customer so that they can determine the problem on their end.
>
> How do I make qmail log the outgoing SMTP conversation?

qmail doesn't log the entire conversation. the qmail-send log contains
a record of every delivery attempt, including the envelope sender and
recipient, as well as the remote server's response after the end of
the DATA transaction.

the logs are not all one-line entries, and no one line contains all of
the information you need. plus if the server happens to be sending
more than one message at a time, the log lines for the various
messages will be interspersed with each other. the "mtrack" script on
my web site may prove helpful in reading the log- it reads the log
file and prints the lines back out, grouped by the message they
pertain to. i wrote it when i was running a busy ISP and needed to
track my own log files.

for example, here's a snippet of my own log file, as parsed through
mtrack. i sent a message to this list earlier today, and when djb's
"secretary" sent back the challenge message, i replied to it. while
this was happening, there were two or three other messages being
processed at the same time, and mtrack showed them as separate groups
of lines.

@40000000499bcc6333b4cf0c new msg 2982375
@40000000499bcc6333b4f234 info msg 2982375: bytes 3401 from <jms1 [at] jms1
> qp 27472 uid 162
@40000000499bcc6333b4fdec starting delivery 28713: msg 2982375 to
remote qmail [at] list
@40000000499bcc66048f1b4c delivery 28713: success:
131.193.36.27_accepted_message./
Remote_host_said:_250_ok_1234947623_qp_71040/
@40000000499bcc66048fafbc end msg 2982375

@40000000499bccb60b8d9694 new msg 2982375
@40000000499bccb60b8db5d4 info msg 2982375: bytes 7019 from <jms1 [at] jms1
> qp 27505 uid 162
@40000000499bccb60b8dc574 starting delivery 28715: msg 2982375 to
remote list-qmail-notbulkmail-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx [at] list
@40000000499bccb727bd06ec delivery 28715: success:
131.193.36.27_accepted_message./
Remote_host_said:_250_ok_1234947704_qp_75254/
@40000000499bccb727bd9b5c end msg 2982375

http://qmail.jms1.net/scripts/#mtrack if you're interested.

it's certainly possible to generate more detailed logging if you need
it, however this would involve patching (modifying the source code.)
it's an interesting idea, and i may write something like this in the
future (i'm working on other things at the moment) however if it's
something you need now (i.e. if the information in the qmail-send logs
isn't enough) then contact me off-list and we can talk about it.


- ----------------------------------------------------------------
| John M. Simpson --- KG4ZOW --- Programmer At Large |
| http://www.jms1.net/ <jms1 [at] jms1> |
- ----------------------------------------------------------------
| http://video.google.com/videoplay?docid=-1656880303867390173 |
- ----------------------------------------------------------------





-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iEYEARECAAYFAkmcb8kACgkQj42MmpAUrRopSACghqR8xXGnIe4EiU91Y67Swluv
ezkAniUdEqqKfZfJV5bLQaDnpw8vZ3Hc
=Ob0g
-----END PGP SIGNATURE-----


kyle-qmail at memoryhole

Feb 18, 2009, 12:58 PM

Post #3 of 7 (2484 views)
Permalink
Re: Outgoing SMTP logging [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wednesday, February 18 at 11:15 AM, quoth jim steele:
> I have gotten mail flow working properly with qmail, but there is
> one thing that I need that I cannot figure out. A few times per
> month, there will be a customer who claims not to have received an
> email from our system, and I go to the Microsoft SMTP service log
> file to retrieve the log of the outgoing SMTP conversation that
> proves that we sent the message, including what our server said, and
> what their server said in response (less the DATA portion). I
> provide the log to the customer so that they can determine the
> problem on their end.
>
> How do I make qmail log the outgoing SMTP conversation?

There's no *easy* way to record the entire outgoing conversation.
Seriously, for most folks, that would be gigabytes if not terabytes of
data every month, even without the DATA portion of the conversation.

However, qmail *does* log every mail delivery, including what the
remote server said in response to the DATA transmission. For example:

delivery 1222: success:
194.70.142.30_accepted_message./Remote_host_said:_250_ok:__Message_93918201_accepted/

That tells me that for delivery attempt 1222, the IP address
194.70.142.30 accepted the email with the response "250 ok: Message
93918201 accepted". You can match that up with the delivery details
based on the attempt number, e.g.:

starting delivery 1222: msg 64027 to remote mutt-users [at] mutt

More detail can be had by looking for the message's id number (note
that this number is reused frequently):

info msg 64027: bytes 3247 from <user [at] domain> qp 18214 uid
64020

In all the situations where I've had to prove that a message was
accepted, this was sufficient.

If you REALLY need the full SMTP transmission, you're going to have to
patch qmail-remote to log that information.

~Kyle
- --
However many holy words you read, However many you speak, what good
will they do you If you do not act on upon them?
-- Buddha
-----BEGIN PGP SIGNATURE-----
Comment: Thank you for using encryption!

iEYEARECAAYFAkmcdooACgkQBkIOoMqOI17U9QCgvNBSbu3cietblL45Akj0Ll15
/6EAoK/ZZd9djn00q1motuikkzV2XQLw
=svSK
-----END PGP SIGNATURE-----


feh at fehcom

Feb 18, 2009, 1:41 PM

Post #4 of 7 (2505 views)
Permalink
Re: Outgoing SMTP logging [In reply to]

Hi,

simply to check, whether (and when) the outgoing mail was deliverd, you can
use my small PERL program qmFind. It is part of the NEWANALYSE package
(allowing to step back in the past to check for mails from particular
sender/recipients).

regards.
--eh.

--On Mittwoch, 18. Februar 2009 11:15 -0800 jim steele
<jsteele1997 [at] gmail> wrote:

> Hello,
>
> I am working on migrating my company's outgoing mail servers from
> Windows servers running Microsoft's SMTP service to machines running
> qmail. These servers relay notification (e.g. "Your order has been
> shipped") mail from our application servers to our customers.
>
> I have gotten mail flow working properly with qmail, but there is one
> thing that I need that I cannot figure out. A few times per month,
> there will be a customer who claims not to have received an email from
> our system, and I go to the Microsoft SMTP service log file to
> retrieve the log of the outgoing SMTP conversation that proves that we
> sent the message, including what our server said, and what their
> server said in response (less the DATA portion). I provide the log to
> the customer so that they can determine the problem on their end.
>
> How do I make qmail log the outgoing SMTP conversation?
>
> Thank you,
> Jim
>


jsteele1997 at gmail

Feb 18, 2009, 2:20 PM

Post #5 of 7 (2487 views)
Permalink
Re: Outgoing SMTP logging [In reply to]

On Wed, Feb 18, 2009 at 12:58 PM, Kyle Wheeler
<kyle-qmail [at] memoryhole> wrote:
> There's no *easy* way to record the entire outgoing conversation.
> Seriously, for most folks, that would be gigabytes if not terabytes of
> data every month, even without the DATA portion of the conversation.
>
> However, qmail *does* log every mail delivery, including what the
> remote server said in response to the DATA transmission. For example:
>
> delivery 1222: success:
> 194.70.142.30_accepted_message./Remote_host_said:_250_ok:__Message_93918201_accepted/

Thanks for your quick reply. The logs are actually about 700 megabytes
per month, and compress to about 50 megabytes. I'm sorry to hear that
this isn't possible with qmail, unfortunately I'll have to stick with
the Microsoft SMTP service.

Thank you,
Jim


search-web-for-address at pyropus

Feb 18, 2009, 6:32 PM

Post #6 of 7 (2496 views)
Permalink
Re: Outgoing SMTP logging [In reply to]

jim steele <jsteele1997 [at] gmail> wrote:
> >
> > However, qmail *does* log every mail delivery, including what the
> > remote server said in response to the DATA transmission. For example:
> >
> > delivery 1222: success:
> > 194.70.142.30_accepted_message./Remote_host_said:_250_ok:__Message_93918201_accepted/
>
> Thanks for your quick reply. The logs are actually about 700 megabytes
> per month, and compress to about 50 megabytes.

Doesn't sound like you're handling a lot of mail, then.

> I'm sorry to hear that this isn't possible with qmail, unfortunately I'll
> have to stick with the Microsoft SMTP service.

He didn't say it wasn't possible with qmail (very little is impossible with
qmail, because it's so modular); he was merely trying to see if the default
logging functionality would suffice for your needs.

So which part of the SMTP conversation do you want to see end up logged, then?
The above shows the response to the end of the DATA phase.

Charles
--
--------------------------------------------------------------------------
Charles Cazabon
GPL'ed software available at: http://pyropus.ca/software/
Read http://pyropus.ca/personal/writings/12-steps-to-qmail-list-bliss.html
--------------------------------------------------------------------------


shepherdzw at gmail

Feb 18, 2009, 11:34 PM

Post #7 of 7 (2492 views)
Permalink
Re: Outgoing SMTP logging [In reply to]

On Thu, Feb 19, 2009 at 4:32 AM, Charles Cazabon <
search-web-for-address [at] pyropus> wrote:

> jim steele <jsteele1997 [at] gmail> wrote:
> > >
> > > However, qmail *does* log every mail delivery, including what the
> > > remote server said in response to the DATA transmission. For example:
> > >
> > > delivery 1222: success:
> > >
> 194.70.142.30_accepted_message./Remote_host_said:_250_ok:__Message_93918201_accepted/
> >
> > Thanks for your quick reply. The logs are actually about 700 megabytes
> > per month, and compress to about 50 megabytes.
>

Get isoqlog my friend html based logging system...
http://www.enderunix.org/isoqlog/

It has a better logging by day,month,year or sender and by domain.

If you need more reports then customise it and develop it to suit yourself .
I did the same but have never released it on the public domain but i might
give it to you if you tell me what is missing from the current stable
release after you have used it.


>
>
> Doesn't sound like you're handling a lot of mail, then.
>
> > I'm sorry to hear that this isn't possible with qmail, unfortunately I'll
> > have to stick with the Microsoft SMTP service.
>
> He didn't say it wasn't possible with qmail (very little is impossible with
> qmail, because it's so modular); he was merely trying to see if the default
> logging functionality would suffice for your needs.
>
> So which part of the SMTP conversation do you want to see end up logged,
> then?
> The above shows the response to the end of the DATA phase.
>
> Charles
> --
> --------------------------------------------------------------------------
> Charles Cazabon
> GPL'ed software available at: http://pyropus.ca/software/
> Read http://pyropus.ca/personal/writings/12-steps-to-qmail-list-bliss.html
> --------------------------------------------------------------------------
>



--
Shepherd Nhongo

Do not Quee mail with SENDMAIL, send mail with QMAIL

Mobile +267 74476040

Qmail users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.