Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: DBMail: users

request for extension: "domain" and "customers" table

 

 

DBMail users RSS feed   Index | Next | Previous | View Threaded


michael.monnerie at is

Jun 24, 2009, 12:47 PM

Post #1 of 7 (396 views)
Permalink
request for extension: "domain" and "customers" table

I think I've said this around 2006 already, but maybe time has come to
suggest it again. As far as I read this mailing list, most people use
dbmail to run several (if not lots of) domains. We are in the same
situation, a small ISP running some hundred domains.

I've extended dbmail since before we started, in order to have an extra
"domains" and "customers" table. This has big advantages. But first,
I'll list our tables (named zmi_* to distinct from dbmail builtin
tables):

# \d dbmail_users
Spalte | Typ | Attribute
-----------------+-----------------------------+------------------
user_idnr | bigint | not null default
userid | character varying(100) | not null
passwd | character varying(34) | not null
client_idnr | bigint | not null default 0
maxmail_size | bigint | not null default 0
curmail_size | bigint | not null default 0
maxsieve_size | bigint | not null default 0
cursieve_size | bigint | not null default 0
encryption_type | character varying(20) | not null default ''
last_login | timestamp without time zone | not null
user_realname | character varying(100) |
domain_idnr | bigint | not null
visible | boolean | not null default true
active | boolean | not null default true
Indexe:
»dbmail_users_pkey« PRIMARY KEY, btree (user_idnr)
»dbmail_users_name_idx« UNIQUE, btree (userid)
»dbmail_users_2« btree (lower(userid::text))
»dbmail_users_domain« btree (domain_idnr, userid) CLUSTER
(not used by us; could be removed)

client_idnr is from dbmail, but nowhere used; could be deleted
user_realname is to store the name of a person. We use that in order for
support to be easier ("Mr. Ahmed Turfan has a problem")
domain_idnr is a link to the zmi_domains table
visible defines if that user is visible in the web interface (used to
suppress display of spamtraps)
active can be set false to disable a user (e.g. did not pay, etc.)

# \d zmi_domains
Spalte | Typ | Attribute
-----------------+------------------------+----------------
domain_idnr | bigint | not null default
nextval('zmi_domains_domain_idnr_seq'::regclass)
domain | character varying(200) | not null
maxaccounts | bigint | not null default 0
userquota | bigint | not null default 100
domainquotasoft | bigint | not null default 100
domainquotahard | bigint | not null default 100
comment | text |
pridomain_idnr | bigint |
cust_idnr | bigint | not null default 0
active | boolean | not null default true
Indexe:
»zmidom_pk« PRIMARY KEY, btree ("domain")
»zmi_domains_domain_idnr_key« UNIQUE, btree (domain_idnr) CLUSTER
Fremdschlüssel-Constraints:
»zmi_domains_cust_idnr_fkey« FOREIGN KEY (cust_idnr) REFERENCES
zmi_customers(cust_idnr)
»zmi_domains_pridomain_idnr_fkey« FOREIGN KEY (pridomain_idnr)
REFERENCES zmi_domains(domain_idnr)

domain_idnr is simply a SERIAL
domain is the name of the domain
maxaccounts sets a limit on how many addresses a customer can define via
the web interface
userquota sets the default quota a user in that domain receives
domainquota(hard|soft) sets a limit for that domain as a whole. So you
can say "that domain can have 5GB storage, each user max. 2GB", just to
be sure that the 20 users of that domain cannot get 40GB storage
occupied.
comment: internal comment for special setups (human readable)
pridomain_idnr: this is the magic. normally NULL, but if a domain is an
aliasdomain, contains the domain_idnr of the primary domain. This way
it's easy to setup domain1.at and define that domain2.at and domain3.at
are "aliasdomains". This means there can't be users in domain2/3, but a
defined address x[at]domain1.at also works with x[at]domain2/3.at; this makes
management a lot easier for aliases, and also simplifies MTA setup.
cust_idnr: to which customer does this domain belong?
active: is this domain able to receive mail, are it's users able to
login?

# \d zmi_customers
Spalte | Typ | Attribute
---------------+------------------------+----------------------
cust_idnr | bigint | not null default
nextval('zmi_customers_cust_idnr_seq'::regclass)
intname | character varying(200) |
name1 | character varying(200) | not null
name2 | character varying(200) |
addr1 | character varying(200) |
addr2 | character varying(200) |
zip | character varying(20) |
city | character varying(200) |
uid | character varying(30) |
maxaccounts | bigint | not null default 0
domainquota | bigint | not null default 100
custquotasoft | bigint | not null default 100
custquotahard | bigint | not null default 100
comment | text |
maxdomains | bigint | not null default 1
active | boolean | not null default true
Indexe:
»zmicust_pk« PRIMARY KEY, btree (cust_idnr)
»zmi_customers_intname« btree (intname, name1, name2) CLUSTER

cust_idnr is a SERIAL
intname: how is that customer called. We use it to encode resellers
names there, and sort in the web app by this key. This makes support for
resellers more easy.
name/addr/zip/city are customers data
uid is the TAX ID in the european union, needed for billing
maxaccounts: defines the max. number of *users* for this customer, for a
sum of all domains of that customer.
domainquota: defines default domainquota for new domains
custquotasoft/hard: max. disk space allowed for this customer for all
domains and all users in sum. Just to ensure a max. disk space.
comment is an internal, human readable reminder for specialities.
maxdomains: how many domains can this customer have?
active: false if they didn't pay, or moved to somewhere else

A bit off topic: Then I also have a contactperson table:
# \d zmi_contactperson
Tabelle »public.zmi_contactperson«
Spalte | Typ | Attribute
-----------------+--------+-----------
domain_idnr | bigint | not null
user_idnr | bigint | not null
informtype_idnr | bigint | not null
Indexe:
»zmi_contactperson_pkey« PRIMARY KEY, btree (domain_idnr, user_idnr,
informtype_idnr)
Fremdschlüssel-Constraints:
»zmi_contactperson_domain_idnr_fkey« FOREIGN KEY (domain_idnr)
REFERENCES zmi_domains(domain_idnr) ON UPDATE CASCADE ON DELETE CASCADE
»zmi_contactperson_informtype_idnr_fkey« FOREIGN KEY
(informtype_idnr) REFERENCES zmi_informtypes(informtype_idnr) ON UPDATE
CASCADE ON DELETE CASCADE
»zmi_contactperson_user_idnr_fkey« FOREIGN KEY (user_idnr)
REFERENCES dbmail_users(user_idnr) ON UPDATE CASCADE ON DELETE CASCADE

This defines a list of "who is to be informed of an event of type X for
a domain?". Example: technical maintenance, sales e-mail, etc.
informtype_idnr is a simple integer, it's meaning defined in the web
app. (Currently under development)

The advantages of the domains/customers table:
- List of domains easy to retrieve for postfix:
virtual_mailbox_domains = pgsql:/etc/postfix/domains.sql
query = SELECT domain FROM zmi_domains WHERE domain='%s'

- Postfix easily rewrite aliasdomains, so that user[at]aliasdomain goes in
as user[at]primarydomain into dbmail:
recipient_canonical_maps = pgsql:/etc/postfix/recipients-rewrite.sql
query = SELECT '%u@' || domain
FROM zmi_domains
WHERE domain_idnr = ( SELECT pridomain_idnr FROM zmi_domains pridom
WHERE pridom.domain='%d')

- The query to find if an e-mail is accepted on our system looks a bit
complicated, but it catches all possible forms of aliases:
# WHERE userid = '%u[at]domain' searches for a user
# WHERE alias = '@domain' searches a domain catch-all
# WHERE alias = '%u[at]domain' search an alias
# WHERE alias = '%u@' search a server catch-all (postmaster@)
query = SELECT 1
FROM dbmail_users, dbmail_aliases
WHERE
userid = (
SELECT '%u@' || domain
FROM zmi_domains
WHERE domain_idnr = ( SELECT COALESCE(pridom.pridomain_idnr,
pridom.domain_idnr) FROM zmi_domains pridom WHERE
pridom.domain=substring('%s' from position('@' in '%s')+1))
)
OR
alias = (
SELECT '@' || domain
FROM zmi_domains
WHERE domain_idnr = ( SELECT COALESCE(pridom.pridomain_idnr,
pridom.domain_idnr) FROM zmi_domains pridom WHERE
pridom.domain=substring('%s' from position('@' in '%s')+1))
)
OR
alias = (
SELECT '%u@' || domain
FROM zmi_domains
WHERE domain_idnr = ( SELECT COALESCE(pridom.pridomain_idnr,
pridom.domain_idnr) FROM zmi_domains pridom WHERE
pridom.domain=substring('%s' from position('@' in '%s')+1))
)
OR
alias = '%u@'
LIMIT 1

- we especially appreciate the postmaster@, webmaster@ etc. aliases, so
for each customer we automatically have the "important" aliases routed
directly into our mailbox, as we provide that service for them. Those
who want to do that themselves, just define postmaster[at]mydomain.com

- It helps stop backscatter easily, because you don't accept e-mail for
aliases that don't exist, even in alias domains.

- Alias domains never need to be checked really, as postfix rewrites
them to user[at]primarydomain itself

- It's easy to see which domains are just aliases. This makes displaying
on the web interface much more fun.

Disadvantage:
- DB schema change necessary
- at least two more tables (domains, customers)

All this might be biased for our purposes, but I guess there are lots of
features many of you would appreciate. For example, the per-domain
and/or per-customer quotas. Those are currently not implemented in
dbmail, but that shouldn't be too hard. We use is from the web interface
for a simple display. The customer gets a call in case of heavy use :-)

PS: we've also modified dbmail_aliases to include the "domain_idnr",
"visible" and "active" fields.

mfg zmi
--
// Michael Monnerie, Ing.BSc ----- http://it-management.at
// Tel: 0660 / 415 65 31 .network.your.ideas.
// PGP Key: "curl -s http://zmi.at/zmi.asc | gpg --import"
// Fingerprint: AC19 F9D5 36ED CD8A EF38 500E CE14 91F7 1C12 09B4
// Keyserver: wwwkeys.eu.pgp.net Key-ID: 1C1209B4



_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail


michael.monnerie at is

Jun 26, 2009, 4:25 AM

Post #2 of 7 (364 views)
Permalink
Re: request for extension: "domain" and "customers" table [In reply to]

On Mittwoch 24 Juni 2009 Michael Monnerie wrote:
> I've extended dbmail since before we started, in order to have an
> extra "domains" and "customers" table. This has big advantages.

Nobody to comment on this? Too complicated, too harsh, or maybe just a
stupid idea of me?

mfg zmi
--
// Michael Monnerie, Ing.BSc ----- http://it-management.at
// Tel: 0660 / 415 65 31 .network.your.ideas.
// PGP Key: "curl -s http://zmi.at/zmi.asc | gpg --import"
// Fingerprint: AC19 F9D5 36ED CD8A EF38 500E CE14 91F7 1C12 09B4
// Keyserver: wwwkeys.eu.pgp.net Key-ID: 1C1209B4

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail


paul at nfg

Jun 26, 2009, 4:59 AM

Post #3 of 7 (365 views)
Permalink
Re: request for extension: "domain" and "customers" table [In reply to]

Michael Monnerie wrote:
> On Mittwoch 24 Juni 2009 Michael Monnerie wrote:
>> I've extended dbmail since before we started, in order to have an
>> extra "domains" and "customers" table. This has big advantages.
>
> Nobody to comment on this? Too complicated, too harsh, or maybe just a
> stupid idea of me?

None of the above. But your business logic is not my business logic.
Even we do very much the same in our main installation, we use LDAP to
extend the user model, so the extra tables would be redundant for us.
Also, I fail to understand how the additional tables effect dbmail
internals. My business logic builds on ldap, and doesn't require any
tweaking of the dbmail code or tables. Most of the delivery process is
handled by postfix talking to ldap - no dbmail involved. All the lmtp
server needs to be able to do is route a message (that already was
verified as destined for a valid user) to the correct user's inbox (or
sieve script).

So your idea as merit, but is more an illustration of the cool things
you can do *with* dbmail, rather than a program of what we might do *to*
dbmail.



--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail


michael.monnerie at is

Jun 26, 2009, 6:30 AM

Post #4 of 7 (361 views)
Permalink
Re: request for extension: "domain" and "customers" table [In reply to]

On Freitag 26 Juni 2009 Paul J Stevens wrote:
> Even we do very much the same in our main installation, we use LDAP
> to extend the user model, so the extra tables would be redundant for
> us. Also, I fail to understand how the additional tables effect
> dbmail internals. My business logic builds on ldap, and doesn't
> require any tweaking of the dbmail code or tables. Most of the
> delivery process is handled by postfix talking to ldap - no dbmail
> involved. All the lmtp server needs to be able to do is route a
> message (that already was verified as destined for a valid user) to
> the correct user's inbox (or sieve script).

Can you post your LDAP schema? I guess I really should change and use
that. I didn't until now, as I can't see the advantage of LDAP over
dbmail (for us). But as you're using it that way, I'd like to change to
the same model in order to have the same usage in dbmail-2.3, once we
switch.

And if others you the same scheme, it'l be easier to understand each
other or workout features. I've recently expressed our will to provide
our web interface. I'm in the process to do that, but it's tricky as
it's heavily using our internals. So using a "standard model" should be
less pain for the 2.3 adopted version.

I hope you're able to post your LDAP schema (confidential?), and that I
can adopt to use it. Then I could start trying out dbmail-2.3 with that
schema already, so migration should be easier also.

mfg zmi
--
// Michael Monnerie, Ing.BSc ----- http://it-management.at
// Tel: 0660 / 415 65 31 .network.your.ideas.
// PGP Key: "curl -s http://zmi.at/zmi.asc | gpg --import"
// Fingerprint: AC19 F9D5 36ED CD8A EF38 500E CE14 91F7 1C12 09B4
// Keyserver: wwwkeys.eu.pgp.net Key-ID: 1C1209B4

_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail


paul at nfg

Jun 26, 2009, 6:54 AM

Post #5 of 7 (362 views)
Permalink
Re: request for extension: "domain" and "customers" table [In reply to]

Michael Monnerie wrote:
> On Freitag 26 Juni 2009 Paul J Stevens wrote:
>> Even we do very much the same in our main installation, we use LDAP
>> to extend the user model, so the extra tables would be redundant for
>> us. Also, I fail to understand how the additional tables effect
>> dbmail internals. My business logic builds on ldap, and doesn't
>> require any tweaking of the dbmail code or tables. Most of the
>> delivery process is handled by postfix talking to ldap - no dbmail
>> involved. All the lmtp server needs to be able to do is route a
>> message (that already was verified as destined for a valid user) to
>> the correct user's inbox (or sieve script).
>
> Can you post your LDAP schema? I guess I really should change and use
> that. I didn't until now, as I can't see the advantage of LDAP over
> dbmail (for us). But as you're using it that way, I'd like to change to
> the same model in order to have the same usage in dbmail-2.3, once we
> switch.

I don't follow. It's been in dbmail forever. Take a look at dbmail.schema.

To summarize:

objectclass ( 1.3.6.1.4.1.12340.6.2.2.1 NAME 'dbmailUser'
DESC 'DBMail-LDAP User' SUP top AUXILIARY
MUST ( uid $ mail )
MAY ( userPassword $ uidNumber $ gidNumber $ mailQuota $
mailForwardingAddress $ mailHost $
mailAlternateAddress $ dbmailUID $ dbmailGID $
deliveryMode $ accountStatus ) )

objectclass ( 1.3.6.1.4.1.12340.6.2.2.2 NAME 'dbmailForwardingAddress'
DESC 'DBMail-LDAP Forwarding Address' SUP top AUXILIARY
MUST ( mail $ mailForwardingAddress ) )

objectclass ( 1.3.6.1.4.1.12340.6.2.2.3 NAME 'dbmailDomain'
DESC 'DBMail-LDAP Virtual Domain' SUP top STRUCTURAL
MUST ( mailDomain )
MAY ( userPassword $ mailQuota $ mailForwardingAddress $
mailHost $ mailCluster ) )


> And if others you the same scheme, it'l be easier to understand each
> other or workout features. I've recently expressed our will to provide
> our web interface. I'm in the process to do that, but it's tricky as
> it's heavily using our internals. So using a "standard model" should be
> less pain for the 2.3 adopted version.
>
> I hope you're able to post your LDAP schema (confidential?), and that I
> can adopt to use it. Then I could start trying out dbmail-2.3 with that
> schema already, so migration should be easier also.

There is nothing 2.3-ish about any of this.

--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail


michael.monnerie at is

Jun 26, 2009, 9:23 AM

Post #6 of 7 (362 views)
Permalink
Re: request for extension: "domain" and "customers" table [In reply to]

On Freitag 26 Juni 2009 Paul J Stevens wrote:
> I don't follow. It's been in dbmail forever. Take a look at
> dbmail.schema.

OK, I never looked so far, as LDAP was never really thought about. I
looked once into it, but didn't find a nice tool to insert our data in
an easy way (like an editor, webpage, whatever). Is there something to
start with? Of course I want to adopt our web interface to use that, but
it will take some time.

> To summarize:
> DESC 'DBMail-LDAP User' SUP top AUXILIARY
> 'dbmailForwardingAddress' DESC 'DBMail-LDAP Forwarding Address' SUP
> DESC 'DBMail-LDAP Virtual Domain' SUP top STRUCTURAL

Hm. I understand this are 3 objects user/alias/domain, but that's only
part of the story. There need to be objects where users and companies
are defined, and how they are connected, where the uidNumber is defined,
the mailCluster, etc.
I don't really have the full picture of LDAP, only know it's some
directory like Novell's NDS used to be (which I loved a lot). Now I
looked into dbmail.schema, it says it needs
# - core.schema
# - cosine.schema
# - nis.schema
Is this everything you use? I found a company definition there, but are
other modifications to make? Might sound stupid, but it's a PITA to
start something new. Is there a simple way to copy existing dbmail users
into the LDAP schema? I start fresh, so I'd like to copy everything from
SQL to LDAP.

> > I hope you're able to post your LDAP schema (confidential?), and
> > that I can adopt to use it. Then I could start trying out
> > dbmail-2.3 with that schema already, so migration should be easier
> > also.
> There is nothing 2.3-ish about any of this.

I didn't mean that, rather that I want to
1) copy SQL users to LDAP
2) setup dbmail-2.3
3) config dbmail-2.3 to use LDAP

and the old 2.2 should stay with SQL until I fully trust my LDAP setup.
Like this I also have an easy transition from SQL to LDAP, so there is
nothing to break on changing the existing server.

mfg zmi
--
// Michael Monnerie, Ing.BSc ----- http://it-management.at
// Tel: 0660 / 415 65 31 .network.your.ideas.
// PGP Key: "curl -s http://zmi.at/zmi.asc | gpg --import"
// Fingerprint: AC19 F9D5 36ED CD8A EF38 500E CE14 91F7 1C12 09B4
// Keyserver: wwwkeys.eu.pgp.net Key-ID: 1C1209B4
Attachments: signature.asc (0.19 KB)


paul at nfg

Jun 27, 2009, 7:38 AM

Post #7 of 7 (357 views)
Permalink
Re: request for extension: "domain" and "customers" table [In reply to]

Michael Monnerie wrote:
> On Freitag 26 Juni 2009 Paul J Stevens wrote:
>> I don't follow. It's been in dbmail forever. Take a look at
>> dbmail.schema.
>
> OK, I never looked so far, as LDAP was never really thought about. I
> looked once into it, but didn't find a nice tool to insert our data in
> an easy way (like an editor, webpage, whatever). Is there something to
> start with? Of course I want to adopt our web interface to use that, but
> it will take some time.

phpldapadmin comes to mind. I've used cpu (cpu.sf.net) a lot for
managing users from the commandline. But lately I've written a set of
shell scripts acting as wrappers around ldapmodify that allow me to
setup users/domains/aliases/forward/transports etc etc really easy.
These shell tools are also wrapped in a simple set of python classes
that are exposed as xmlrpc objects for remote management from a pylons
web-interface.


>
>> To summarize:
>> DESC 'DBMail-LDAP User' SUP top AUXILIARY
>> 'dbmailForwardingAddress' DESC 'DBMail-LDAP Forwarding Address' SUP
>> DESC 'DBMail-LDAP Virtual Domain' SUP top STRUCTURAL
>
> Hm. I understand this are 3 objects user/alias/domain, but that's only
> part of the story. There need to be objects where users and companies
> are defined, and how they are connected, where the uidNumber is defined,
> the mailCluster, etc.

You can used OU (organisationalUnit) type tree nodes to map out your
accounting logic.

Users come in flavors. For shell users I use the following list of
objectClasses: posixAccount, account, shadowAccount, dbmailUser, top

But for email-only users the list is different:
top, account, dbmailUser

For example a simple email-only user might look like:

dn: uid=someuser[at]foobar.com,mailDomain=foobar.com,ou=MailDomains,
dc=foobar,dc=com
objectClass: top
objectClass: account
objectClass: dbmailUser
objectClass: amavisAccount
uid: someuser[at]foobar.com
mail: someuser[at]foobar.com
uidNumber: 12345
gidNumber: 12345
mailQuota: 200000000

but you can make the ldap-tree as deeply nested as you like, assigning
management permissions to certain objects in the tree, allowing them to
manage specific sub-trees.


> I don't really have the full picture of LDAP, only know it's some
> directory like Novell's NDS used to be (which I loved a lot). Now I
> looked into dbmail.schema, it says it needs
> # - core.schema
> # - cosine.schema
> # - nis.schema
> Is this everything you use? I found a company definition there, but are
> other modifications to make? Might sound stupid, but it's a PITA to
> start something new. Is there a simple way to copy existing dbmail users
> into the LDAP schema? I start fresh, so I'd like to copy everything from
> SQL to LDAP.

The schema files you need all depend on the ldap design you come up
with. You can re-use existing schemas, or even build your own like I did
for dbmail. All you need is an OID which you can request for free from
IANA. (1.3.6.1.4.1.12340 is *mine*).

I'm not aware of an easy way to migrate. When we moved to ldap, all our
users where stored in /etc/passwd, and cpu did all the work. The main
problem will be the password. I don't think ldap enforces a certain kind
of encryption. The standard is SHA1, but crypt is also supported, and
probably other types as well. Really depends on the ldap server you use.


--
________________________________________________________________
Paul Stevens paul at nfg.nl
NET FACILITIES GROUP GPG/PGP: 1024D/11F8CD31
The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail[at]dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

DBMail users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.