I myself use Gossamer Mail with everything under SSL. With modern systems, the performance impact of having to compress and decompress everything is minimal. My personal setup is Gossamer Mail under mod_perl, with mod_gzip and mod_ssl. Combine this with pop3 over ssl for any remote mail, and you have a fairly secure mail setup (assuming your server is secure).
Adrian
Adrian