Mailing List Archive: exim: users

OT: Extracting information from an email

Index | Next | Previous | View Threaded

steve at syscall

May 8, 2008, 6:32 AM

Post #1 of 12 (528 views)
Permalink

OT: Extracting information from an email

Hi All

This isn't strictly about Exim, but there are so many knowledgeable
people on this list that I'm sure someone can point me in the correct
direction. My apologies is this is to far off topic.

I want remote applications to pass information between themselves. The
data will be in XML format (no-brainer these days). For various reasons
the communications link between the two system can not be guaranteed, so
rather than use some direct link protocol I want to send the data via
email. It may be that some human intermediary will be part of the
communications chain too. So the system has to cope with data being
forwarded and the like.

For security reasons I only want to process the email if it was signed
by a known GnuPG key. I don't want to trust the From header as these
are easy to forge.

I plan to set up a pipe delivery from exim to a script I'll write. The
script will need to extract the various mime parts for later processing.
munpack(1) does a nice job of extracting the various mime parts for me,
but I am having a problem validating the email's signature.

At the moment I'm testing this all out by hand using Evolution as my
MUA, but I want the system to cope whatever MUA is used (Evolution,
Outlook, JavaMail, mutt, pine, you name it).

What I'm lacking is a nice guide on how to check signed emails. There
must be a standard, something I'm missing because mutt, Evolution,
Outlook, ... can all interoperate. Can some one please put me out of my
misery and point me at the appropriate documentation.

Thanks
Steve
--
Steve Dobson

I am not an Economist. I am an honest man!
-- Paul McCracken

Attachments:

signature.asc (0.18 KB)

peter at bowyer

May 8, 2008, 6:40 AM

Post #2 of 12 (514 views)
Permalink

Re: OT: Extracting information from an email [In reply to]

On 08/05/2008, Steve Dobson <steve[at]syscall.org.uk> wrote:

> I want remote applications to pass information between themselves. The
> data will be in XML format (no-brainer these days). For various reasons
> the communications link between the two system can not be guaranteed, so
> rather than use some direct link protocol I want to send the data via
> email. It may be that some human intermediary will be part of the
> communications chain too. So the system has to cope with data being
> forwarded and the like.

Rather than answer your question, can I suggest an alternative
approach - how about Amazon SQS? It's a reliable message-queue service
'in the sky' designed for exactly the sort of thing you're describing,
with a web services API and toolkits in various languages. Other
services are probably available.

Just a thought...

Peter

--
Peter Bowyer
Email: peter[at]bowyer.org
Follow me on Twitter: twitter.com/peeebeee

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

jpm at retail-sc

May 8, 2008, 7:32 AM

Post #3 of 12 (517 views)
Permalink

Re: OT: Extracting information from an email [In reply to]

On Thu May 08 2008 at 15:32:02 CEST, Steve Dobson wrote:

> I want remote applications to pass information between themselves. The
> data will be in XML format (no-brainer these days).

SOAP with an SMTP binding, perhaps?

> What I'm lacking is a nice guide on how to check signed emails.

Signed how? S/MIME or Gnu PG (PGP) ? You ought to find plenty of Perl
modules on CPAN for doing that.

-JP

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

steve at syscall

May 8, 2008, 9:58 AM

Post #4 of 12 (512 views)
Permalink

Re: OT: Extracting information from an email [In reply to]

Peter

On Thu, 2008-05-08 at 14:40 +0100, Peter Bowyer wrote:
> On 08/05/2008, Steve Dobson <steve[at]syscall.org.uk> wrote:
> Rather than answer your question, can I suggest an alternative
> approach - how about Amazon SQS? It's a reliable message-queue service
> 'in the sky' designed for exactly the sort of thing you're describing,
> with a web services API and toolkits in various languages. Other
> services are probably available.
>
> Just a thought...

Thanks for the thought but it doesn't work. For a start it is
infinitely more expensive than e-mail. E-mail is free, Amazon SQS
charges $0.000001 per request. Okay, trivial I grant you but why should
I pay anything if there is a cheaper alternative. We are talking market
forces here.

You're also assuming that the server will have access to the net. Not
guaranteed. Brain dead polices are to blame, but I have to live with
them. Amazon SQS states no special firewall configurations but it does
require an Internet connection. How would this work on a site that
doesn't allow any outbound connection to all but a few machines?

After long discussions with the client I came to the conclusion that
e-mails were the only way to go, because attaching a spreadsheet to an
e-mail is the current manual way to transport this data. The only thing
I know is that e-mail works.

The Excel files are way bigger than they need to be for the data being
transmitted. So by using e-mails and can use the current transport
system for my data transfer. E-mails work as a universal transport
system. If the site really doesn't allow direct Internet connection
then some poor loser can save the attachment and walk the data file over
to the server on a USB key.

I didn't want to put all that in the original posting as this is an OT
thread.

Any chance you could point me to the documentation now?

Steve

--
Steve Dobson

I want to see people using Perl to glue things together creatively, not
just technically but also socially.
-- Larry Wall in <199702111730.JAA28598[at]wall.org>

Attachments:

signature.asc (0.18 KB)

steve at syscall

May 8, 2008, 10:01 AM

Post #5 of 12 (515 views)
Permalink

Re: OT: Extracting information from an email [In reply to]

Hi JP

On Thu, 2008-05-08 at 16:32 +0200, Jan-Piet Mens wrote:
> On Thu May 08 2008 at 15:32:02 CEST, Steve Dobson wrote:
>
> > I want remote applications to pass information between themselves. The
> > data will be in XML format (no-brainer these days).
>
> SOAP with an SMTP binding, perhaps?

No. Java.

> > What I'm lacking is a nice guide on how to check signed emails.
>
> Signed how? S/MIME or Gnu PG (PGP) ?

I use GnuPG but I would like to cope with S/MIME if I can too.

> You ought to find plenty of Perl modules on CPAN for doing that.

There may well be. But that means I will have to learn Perl and I
really, really don't want to do that. I don't like a language where
source code can look like random typings from a far from infinite number
of monkeys.

Steve

--
Steve Dobson

Long life is in store for you.

Attachments:

signature.asc (0.18 KB)

jpm at retail-sc

May 8, 2008, 10:36 PM

Post #6 of 12 (508 views)
Permalink

Re: OT: Extracting information from an email [In reply to]

On Thu May 08 2008 at 19:01:34 CEST, Steve Dobson wrote:

> No. Java.

;-)

> There may well be. But that means I will have to learn Perl and I

Will C do? If so, check cryptlib, an incredible powerful library with a
huge and quite good manual. Not easy to get started with, but it'll
probably give you what you want.
http://www.cs.auckland.ac.nz/~pgut001/cryptlib/

Regards,
-JP

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

peter at bowyer

May 9, 2008, 1:05 AM

Post #7 of 12 (498 views)
Permalink

Re: OT: Extracting information from an email [In reply to]

On 08/05/2008, Steve Dobson <steve[at]syscall.org.uk> wrote:
> Peter
>
> On Thu, 2008-05-08 at 14:40 +0100, Peter Bowyer wrote:
> > On 08/05/2008, Steve Dobson <steve[at]syscall.org.uk> wrote:
> > Rather than answer your question, can I suggest an alternative
> > approach - how about Amazon SQS? It's a reliable message-queue service
> > 'in the sky' designed for exactly the sort of thing you're describing,
> > with a web services API and toolkits in various languages. Other
> > services are probably available.
> >
> > Just a thought...

> I didn't want to put all that in the original posting as this is an OT
> thread.

Fair enough, but the probability of getting a suggestion that won't
work or that you've already discounted increases with the amount of
information you don't disclose.... one of the benefits of asking a
question with full disclosure is that you might just hear a few things
you hadn't thought of.... but hey, it's your call.

> Any chance you could point me to the documentation now?

A bit harsh, surely?

Peter

--
Peter Bowyer
Email: peter[at]bowyer.org
Follow me on Twitter: twitter.com/peeebeee

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

hagen at rz

May 9, 2008, 1:34 AM

Post #8 of 12 (499 views)
Permalink

Re: OT: Extracting information from an email [In reply to]

Steve Dobson schrieb:
> Hi JP
>
> On Thu, 2008-05-08 at 16:32 +0200, Jan-Piet Mens wrote:
>> On Thu May 08 2008 at 15:32:02 CEST, Steve Dobson wrote:
>>
>>> I want remote applications to pass information between themselves. The
>>> data will be in XML format (no-brainer these days).
>> SOAP with an SMTP binding, perhaps?
>
> No. Java.
"SOAP with an SMTP binding" is basically "sending XML via SMTP". It has
no relationship to any specific programming-language.
Given your misinterpretation it sounds like you have to learn quite a
bit of background-knowledge before developing a solution for your customer.
For example have a look at "Enterprise Service Bus". Basically you can
run something like Amazon SQS in your local network. But then you'll
realize that they offer VERY good pricing. It is business we are talking
about and running your own (reliable) infrastructure is likely to be
much more expensive....
But now we are REALLY OT, answers off-list, please.

--
CU,
Patrick.

Attachments:

smime.p7s (5.75 KB)

a.phillips at met

May 9, 2008, 1:49 AM

Post #9 of 12 (499 views)
Permalink

Re: OT: Extracting information from an email [In reply to]

>>>>> "Steve" == Steve Dobson <steve[at]syscall.org.uk> writes:

Steve> I want remote applications to pass information between
Steve> themselves. The data will be in XML format (no-brainer
Steve> these days). For various reasons the communications link
Steve> between the two system can not be guaranteed, so rather
Steve> than use some direct link protocol I want to send the data
Steve> via email. It may be that some human intermediary will be
Steve> part of the communications chain too. So the system has to
Steve> cope with data being forwarded and the like.

Steve> For security reasons I only want to process the email if it
Steve> was signed by a known GnuPG key. I don't want to trust the
Steve> From header as these are easy to forge.

I've done this for receiving meteorological data not using our "usual
channels" (direct links between countries). I wasn't keen to do so
given the importance of the data and that email transmission isn't
necessarily reliable (and to be trusted) but the powers that be
decided that it had to be :-(

Basically a forward file pipes the message into a perl program that
checks the received headers added by our gateway to verify the ip
address/helo information and, in addition, other mail headers as
necessary. It has the option to run gpg on the message but getting
other places to sign messages is a nightmare so its never been used.

Anyway, thanks to CPAN it was relatively easy to code in perl.

Sincerely,

Adrian Phillips

--
Who really wrote the works of William Shakespeare ?
http://www.pbs.org/wgbh/pages/frontline/shakespeare/

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

steve at syscall

May 9, 2008, 2:12 AM

Post #10 of 12 (501 views)
Permalink

Re: OT: Extracting information from an email [In reply to]

Hi Peter

On Fri, 2008-05-09 at 09:05 +0100, Peter Bowyer wrote:
> On 08/05/2008, Steve Dobson <steve[at]syscall.org.uk> wrote:
> > On Thu, 2008-05-08 at 14:40 +0100, Peter Bowyer wrote:
> > > Rather than answer your question, can I suggest an alternative
> > > approach - how about Amazon SQS? It's a reliable message-queue service
> > > 'in the sky' designed for exactly the sort of thing you're describing,
> > > with a web services API and toolkits in various languages. Other
> > > services are probably available.
> > >
> > > Just a thought...
>
> > I didn't want to put all that in the original posting as this is an OT
> > thread.
>
> Fair enough, but the probability of getting a suggestion that won't
> work or that you've already discounted increases with the amount of
> information you don't disclose.... one of the benefits of asking a
> question with full disclosure is that you might just hear a few things
> you hadn't thought of.... but hey, it's your call.

Oh I agree. I took a look at Amazon SQS as I wasn't aware of it. I'm
glad you told me about it even though I can't use it this time. One
never knows for the future.

I am not so arrogant to think that my reasoning could be in error.
There maybe some really nice technology out there of which I am unaware
which will solve all my problems, if so I welcome the information.
With the information I originally gave Amazon SQS was a good and valid
solution, and I do appreciate it.

> > Any chance you could point me to the documentation now?
>
> A bit harsh, surely?

I'm surprised you think so. If I offended my apologies.

But you said above "[r]ather than answer your question", which I took to
mean that you could have answered but you choose not to. If you are now
withholding that information I find that "[a] bit harsh".

Steve
--
Steve Dobson

The best way to accelerate a Macintoy is at 9.8 meters per second per
second.

Attachments:

signature.asc (0.18 KB)

peter at bowyer

May 9, 2008, 2:18 AM

Post #11 of 12 (498 views)
Permalink

Re: OT: Extracting information from an email [In reply to]

On 09/05/2008, Steve Dobson <steve[at]syscall.org.uk> wrote:
> Hi Peter
>
> But you said above "[r]ather than answer your question", which I took to
> mean that you could have answered but you choose not to. If you are now
> withholding that information I find that "[a] bit harsh".

No, not witholding anything - you mis-interpreted my 'rather than...' line.

No harm, no foul...

Peter

--
Peter Bowyer
Email: peter[at]bowyer.org
Follow me on Twitter: twitter.com/peeebeee

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

graeme at graemef

May 9, 2008, 2:45 AM

Post #12 of 12 (497 views)
Permalink

Re: OT: Extracting information from an email [In reply to]

Hi

On Thu, 2008-05-08 at 18:01 +0100, Steve Dobson wrote:
> I use GnuPG but I would like to cope with S/MIME if I can too.

http://www.bouncycastle.org/wiki/display/JA1/Frequently+Asked+Questions

I'm no Java person, but I can Google. (Cue Marc Bolan and T-Rex's "I
love to Google"!).

Looks like bouncycastle will do PGP and S/Mime for you. Enjoy!

Graeme (veering, for once, dangerously off topic).

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com