alanr at bell-labs
Jul 16, 1999, 5:31 AM
Post #11 of 25
"Stephen C. Tweedie" wrote:
> On Fri, 9 Jul 1999 14:18:29 +0200, Lars Marowsky-Bree <lmb [at] teuto>
> > In a redundant/failover system, there are often lots of
> > configuration files which need to be synchronised. While we are all
> > anxiously awaiting Stephens cluster layer, it should be possible to
> > implement something a little less sophisticated right now.
> Actually, what I was planning in the long term was simply to have a
> layer on top of a cluster-wide database, accessed in exactly the same
> simple manner as libdbm (with the possible addition of a transaction
> This can work very well indeed. libdbm is an extremely simply but
> versatile API. You can store information about the config files in
> the dbm "repository" by using hash keys such as "config#0", "config#1"
> etc to contain values of the form "/etc/hosts=HOSTS", which associates
> a config file with a key into the database. Looking up keys
> "HOSTS#0", "HOSTS#1" etc. will return the successive chunks of the
> stored file, say 1K at a time.
This is close, but not quite right. You have to have Config#0:HOSTS#0 (or
something similar) as the keys in order to have transactional semanatics.
> It's a really simple model with lots of advantages. First, the libdbm
> API exists today, and has loads of implementations. Secondly, NIS
> *already* has the ability to export libdbm files around a cluster and
> to maintain cluster synchronisation of the files. (The one thing it
> doesn't give is transactional updates, but that can be done via
> versioning, adding a version name to the hash keys and updating the
> "valid" version as a single key update.)
I'm concerned about what seem to be notable disadvantages to NIS (not DBM) for
the cluster as well.
First, a machine can't be bound to more than one NIS domain
at a time. This means that each cluster would have to have it's own NIS
domain, but cannot bind to the cluster's NIS domain - because binding to the NIS
domain is already used for things like password and group file access, which are
usually handled according to local site policy, and not directly associated with
cluster membership -- in fact can't be used for cluster memebership. Having it
bind to the cluster domain would mean that each cluster had it's own independent
password, group, etc file. This in turn means that you cannot use the yp
library functions for accessing the data. I'm not sure if a machine can be a
slave server for more than one NIS domain at a time. If it can, then all may be
well (and this item is a red herring), but if it can't, then there are problems
afoot, because a high-availability server can't be an NIS client (because then
the NIS server becomes a single point of failure).
Second, NIS is unreliable in practice for distributing data. It has been my
experience that one NIS map or another gets gets trashed, and has to be
redistributed several times a year. This is not where you want to start for a
high-availability file distribution model.
Third, NIS has the wrong file distribution model for a high-availability
environment. In NIS, there is a single master server which is a single point of
failure for distributing updates in the NIS world. I suppose you could move the
NIS master around the cluster as part of the recovery process, but the
complexity of this as part of the cluster's basic services makes me more than a
bit nervous. It seems to me that this adds unnecessary complexity to basic
cluster services in order to cover up what is basically the wrong underlying
file distribution model.
> This sort of model would let you start coding a libdbm-to-config-file
> maintenance layer right now, independently of any other coding going
> on. If somebody wants to come up with a better, transactional,
> distributed version of libdbm later then the code can be ported
> forward with minimal changes.
And, after all is said and done, it still doesn't get the data where you need it
(in files), and you have to then write a piece of code which still has to
checksum the file and have transactional installation properties anyway.
I would offer the opinion that this doesn't sound particularly simple, elegant
or reliable when all the details involved are considered.
-- Alan Robertson
alanr [at] bell-labs