Mailing List Archive: OpenStack: Dev

[Keystone] Quotas: LDAP Help

Index | Next | Previous | View Threaded

everett.toews at cybera

Jul 16, 2012, 4:31 PM

Post #1 of 12 (308 views)
Permalink

[Keystone] Quotas: LDAP Help

Hi All,

I've got a working implementation of quotas in Keystone. However it's only
working for the KVS and SQL backends right now and I need it to work with
LDAP before submitting it for review. I have limited experience with LDAP
and only from an ops perspective, I've never developed any application code
against an LDAP backed app, so I'm here asking for help.

My original plan was to just piggy back on the metadata code in the LDAP
backend (like I did with SQL). But, as you can see from get_metadata [1]
and create_metadata [2], it's not really there. Since that's not possible
I'll need to build something myself but I'm not too sure what's the best
way to go about doing that. Based on a bit of research, I've come up with a
couple of options.

Option 1 - Separate Quota ou

Looking at ldap/core.py, I could create a new QuotaApi class with the fields

DEFAULT_OU = 'ou=Quota'
DEFAULT_STRUCTURAL_CLASSES = []
DEFAULT_OBJECTCLASS = '???'
DEFAULT_ID_ATTR = 'cn'
DEFAULT_MEMBER_ATTRIBUTE = 'cn'
options_name = 'quota'
attribute_mapping = {'quota': 'cn'}
model = models.Quota

The idea being that quota information is an ou associated with a tenant
(somehow). I'm not sure how best to store the quota data itself in this
case. Could it just be stored as JSON in the cn? I'm not sure if that's a
good idea or a bad idea but I suspect bad...

Option 2 - Metadata Attribute on Tenant

Quotas are just an attribute of a Tenant so why not just add a single
'quotas' attribute to the Tenant ou. Then the quotas JSON could be stored
in this attribute. This seems like a simple and straight-forward solution
but I don't know how to add this attribute via an objectclass to Tenant.

How would I add a quotas attribute to the Tenant ou?
How would I reference that attribute?
Is there an existing attribute on Tenant where I could reasonably store the
quotas JSON instead of adding another one?

Thoughts or feedback on these options? Are there any other options I'm
missing?

Thanks,
Everett

[1]
https://github.com/openstack/keystone/blob/master/keystone/identity/backends/ldap/core.py#L140-147
[2]
https://github.com/openstack/keystone/blob/master/keystone/identity/backends/ldap/core.py#L205-206

ayoung at redhat

Jul 16, 2012, 6:20 PM

Post #2 of 12 (290 views)
Permalink

Re: [Keystone] Quotas: LDAP Help [In reply to]

On 07/16/2012 07:31 PM, Everett Toews wrote:
> Hi All,
>
> I've got a working implementation of quotas in Keystone. However it's
> only working for the KVS and SQL backends right now and I need it to
> work with LDAP before submitting it for review. I have limited
> experience with LDAP and only from an ops perspective, I've never
> developed any application code against an LDAP backed app, so I'm here
> asking for help.
>
> My original plan was to just piggy back on the metadata code in the
> LDAP backend (like I did with SQL). But, as you can see from
> get_metadata [1] and create_metadata [2], it's not really there. Since
> that's not possible I'll need to build something myself but I'm not
> too sure what's the best way to go about doing that. Based on a bit of
> research, I've come up with a couple of options.
>
> Option 1 - Separate Quota ou
>
> Looking at ldap/core.py, I could create a new QuotaApi class with the
> fields
>
> DEFAULT_OU = 'ou=Quota'
> DEFAULT_STRUCTURAL_CLASSES = []
> DEFAULT_OBJECTCLASS = '???'
> DEFAULT_ID_ATTR = 'cn'
> DEFAULT_MEMBER_ATTRIBUTE = 'cn'
> options_name = 'quota'
> attribute_mapping = {'quota': 'cn'}
> model = models.Quota
>
> The idea being that quota information is an ou associated with a
> tenant (somehow). I'm not sure how best to store the quota data itself
> in this case. Could it just be stored as JSON in the cn? I'm not sure
> if that's a good idea or a bad idea but I suspect bad...

That does not sound right.

>
> Option 2 - Metadata Attribute on Tenant
>
> Quotas are just an attribute of a Tenant so why not just add a single
> 'quotas' attribute to the Tenant ou. Then the quotas JSON could be
> stored in this attribute. This seems like a simple and
> straight-forward solution but I don't know how to add this attribute
> via an objectclass to Tenant.
>
> How would I add a quotas attribute to the Tenant ou?
> How would I reference that attribute?
> Is there an existing attribute on Tenant where I could reasonably
> store the quotas JSON instead of adding another one?

Usually a Quota is a limitation on a resource. I suspect that the
problem here is we have not nailed down the resource objects that you
would then apply a quota to. If, for example, we were talking about
disk quotas, we could look at the LDAP schema's that are in place for
disks, automount, and so forth. For network or CPU quotas, the concepts
don't really exist.

My immediate thought is that maybe these things are not really Keystone
quantities to manage. Nova has the database that deals with the actual
quantities of disk and so forth. BUt I know that LDAP is the system of
record for Hosts in many systems, so the Data from LDAP needs to feed
into Nova somehow....

Can you post your code to a GIthub repo and send out a link to the
commit so that I could take a look? It would be much more clear to
discuss with actual code in front of me.

>
> Thoughts or feedback on these options? Are there any other options I'm
> missing?
>
> Thanks,
> Everett
>
>
> [1]
> https://github.com/openstack/keystone/blob/master/keystone/identity/backends/ldap/core.py#L140-147
> [2]
> https://github.com/openstack/keystone/blob/master/keystone/identity/backends/ldap/core.py#L205-206

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

everett.toews at cybera

Jul 17, 2012, 8:18 AM

Post #3 of 12 (291 views)
Permalink

Re: [Keystone] Quotas: LDAP Help [In reply to]

On Mon, Jul 16, 2012 at 7:20 PM, Adam Young <ayoung [at] redhat> wrote:

>
> Usually a Quota is a limitation on a resource. I suspect that the problem
> here is we have not nailed down the resource objects that you would then
> apply a quota to. If, for example, we were talking about disk quotas, we
> could look at the LDAP schema's that are in place for disks, automount, and
> so forth. For network or CPU quotas, the concepts don't really exist.
>
> My immediate thought is that maybe these things are not really Keystone
> quantities to manage. Nova has the database that deals with the actual
> quantities of disk and so forth. BUt I know that LDAP is the system of
> record for Hosts in many systems, so the Data from LDAP needs to feed into
> Nova somehow....
>

I led the session on quotas at the Folsom Summit where the consensus was
that, because this data was tied to Tenants, it should be stored in
Keystone. We've also discussed it on the mailing list at
http://markmail.org/message/vlk6otl2yggjeouc and
http://markmail.org/message/7agsnjo3n4il56ar (where you'll find links to
the Summit etherpad, spec, and blueprint).

I searched around a bit for an objectclass that handled generic quotas but
couldn't find one. I really wouldn't want us to write our own objectclass
either as it's simply not flexible enough. I don't think we want to
necessarily nail down the resource objects we want to apply a quota to.
Each OpenStack project is going to need its own quotas and I suspect there
are going to be many additions to those quotas over the next 2 years so we
something that can handle anything.

If we just store some JSON in the backend then that will meet our needs
nicely. This is how "metadata" is stored in the SQL backend. I simply
reused that and it was pretty effective.

> Can you post your code to a GIthub repo and send out a link to the commit
> so that I could take a look? It would be much more clear to discuss with
> actual code in front of me.

My branch is at https://github.com/everett-toews/keystone/tree/quotas
The SQL implementation is at
https://github.com/everett-toews/keystone/blob/quotas/keystone/identity/backends/sql.py

Everett

ayoung at redhat

Jul 17, 2012, 10:20 AM

Post #4 of 12 (292 views)
Permalink

Re: [Keystone] Quotas: LDAP Help [In reply to]

On 07/17/2012 11:18 AM, Everett Toews wrote:
> On Mon, Jul 16, 2012 at 7:20 PM, Adam Young <ayoung [at] redhat
> <mailto:ayoung [at] redhat>> wrote:
>
>
> Usually a Quota is a limitation on a resource. I suspect that the
> problem here is we have not nailed down the resource objects that
> you would then apply a quota to. If, for example, we were talking
> about disk quotas, we could look at the LDAP schema's that are in
> place for disks, automount, and so forth. For network or CPU
> quotas, the concepts don't really exist.
>
> My immediate thought is that maybe these things are not really
> Keystone quantities to manage. Nova has the database that deals
> with the actual quantities of disk and so forth. BUt I know that
> LDAP is the system of record for Hosts in many systems, so the
> Data from LDAP needs to feed into Nova somehow....
>
>
> I led the session on quotas at the Folsom Summit where the consensus
> was that, because this data was tied to Tenants, it should be stored
> in Keystone. We've also discussed it on the mailing list at
> http://markmail.org/message/vlk6otl2yggjeouc and
> http://markmail.org/message/7agsnjo3n4il56ar (where you'll find links
> to the Summit etherpad, spec, and blueprint).
>
> I searched around a bit for an objectclass that handled generic quotas
> but couldn't find one. I really wouldn't want us to write our own
> objectclass either as it's simply not flexible enough. I don't think
> we want to necessarily nail down the resource objects we want to apply
> a quota to. Each OpenStack project is going to need its own quotas and
> I suspect there are going to be many additions to those quotas over
> the next 2 years so we something that can handle anything.
>
> If we just store some JSON in the backend then that will meet our
> needs nicely. This is how "metadata" is stored in the SQL backend. I
> simply reused that and it was pretty effective.
>
> Can you post your code to a GIthub repo and send out a link to the
> commit so that I could take a look? It would be much more clear
> to discuss with actual code in front of me.
>
> My branch is at https://github.com/everett-toews/keystone/tree/quotas
> The SQL implementation is at
> https://github.com/everett-toews/keystone/blob/quotas/keystone/identity/backends/sql.py
>
> Everett

I haven't been thinking about quotas, so bear with me here. A few thoughts:

Certain deployments might not be able to touch the LDAP backend. I am
thinking specifically where there is a corporate AD/LDAP server. I tried
to keep the scheme dependency simple enough that it could be layered
onto a read-only scenario. If we put quotas into LDAP, it might break
on those deployments.

I'm trying to get an approach to Federation and delegation for
Keystone. Imagine where a company has a signing certificate that allows
it validate the users for only their own tenancy. In these cases, the
end user organization then would have the ability to control their own
quotas.

One reason to do the PKI signed tokens was to minimize the number of
calls going to Keystone. This adds yet another one. The quotas could
be added to the signed auth token, but it is already huge, I'd hate to
cram more data in there.

A serialized object block defers a lot of problems, but they will bite
us later. For example, there is already a ticket for "enabled" which
does not have the text string normalized. Doing one for Quotas misses
validation of both resource names and the units used. For example, are
disk quotas in bytes, megabytes, or Gigs? Are all of those acceptable?
If so, how do we define which one to use. I realize you might have
nailed this down in your code, or at least design, but then the code is
the contract.

This design seems to address some of those issues.
http://wiki.openstack.org/Boson

I can see that we don't want to define them in the Nova database, as
Swift might not have access to that, and swift is going to be one of the
primary consumers of Quotas. I am Assuming Quantum will have them as well.

As you are aware, there is no metadata storage in the LDAP driver,
instead it is generated from the tenant and role information on the
fly. There is no place to store metadata in "groupOfNames" which is the
lowest( common denominator) grouping used for Tenants. Probably the
most correct thing to do would be to use a "seeAlso" that points to
where the quota data is stored.

rlane at wikimedia

Jul 17, 2012, 11:42 AM

Post #5 of 12 (288 views)
Permalink

Re: [Keystone] Quotas: LDAP Help [In reply to]

> I haven't been thinking about quotas, so bear with me here. A few thoughts:
>
> Certain deployments might not be able to touch the LDAP backend. I am
> thinking specifically where there is a corporate AD/LDAP server. I tried to
> keep the scheme dependency simple enough that it could be layered onto a
> read-only scenario. If we put quotas into LDAP, it might break on those
> deployments.
>

Many, many deployments won't be able to. Applications should generally
assume they are read-only in regards to LDAP.

> I can see that we don't want to define them in the Nova database, as Swift
> might not have access to that, and swift is going to be one of the primary
> consumers of Quotas. I am Assuming Quantum will have them as well.
>
> As you are aware, there is no metadata storage in the LDAP driver, instead
> it is generated from the tenant and role information on the fly. There is
> no place to store metadata in "groupOfNames" which is the lowest( common
> denominator) grouping used for Tenants. Probably the most correct thing to
> do would be to use a "seeAlso" that points to where the quota data is
> stored.
>

Let's try not to force things into attributes if possible.

When LDAP is used, is the SQL backend not used at all? Why not store
quota info in Keystone's SQL backend, but pull user info from LDAP,
when enabled?

We should only consider storing something in LDAP if it's going to be
reused by other applications. LDAP has a strict schema for exactly
this purpose. If the quota information isn't directly usable by other
applications we shouldn't store it in LDAP.

Many applications with an LDAP backend also have an SQL backend, and
use the SQL as primary storage for most things, and as a cache for
LDAP, if it's used. I think this is likely a sane approach here, as
well.

- Ryan

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

ayoung at redhat

Jul 17, 2012, 11:56 AM

Post #6 of 12 (285 views)
Permalink

Re: [Keystone] Quotas: LDAP Help [In reply to]

On 07/17/2012 02:42 PM, Ryan Lane wrote:
>> I haven't been thinking about quotas, so bear with me here. A few thoughts:
>>
>> Certain deployments might not be able to touch the LDAP backend. I am
>> thinking specifically where there is a corporate AD/LDAP server. I tried to
>> keep the scheme dependency simple enough that it could be layered onto a
>> read-only scenario. If we put quotas into LDAP, it might break on those
>> deployments.
>>
> Many, many deployments won't be able to. Applications should generally
> assume they are read-only in regards to LDAP.
>
>> I can see that we don't want to define them in the Nova database, as Swift
>> might not have access to that, and swift is going to be one of the primary
>> consumers of Quotas. I am Assuming Quantum will have them as well.
>>
>> As you are aware, there is no metadata storage in the LDAP driver, instead
>> it is generated from the tenant and role information on the fly. There is
>> no place to store metadata in "groupOfNames" which is the lowest( common
>> denominator) grouping used for Tenants. Probably the most correct thing to
>> do would be to use a "seeAlso" that points to where the quota data is
>> stored.
>>
> Let's try not to force things into attributes if possible.
>
> When LDAP is used, is the SQL backend not used at all? Why not store
> quota info in Keystone's SQL backend, but pull user info from LDAP,
> when enabled?
>
> We should only consider storing something in LDAP if it's going to be
> reused by other applications. LDAP has a strict schema for exactly
> this purpose. If the quota information isn't directly usable by other
> applications we shouldn't store it in LDAP.
>
> Many applications with an LDAP backend also have an SQL backend, and
> use the SQL as primary storage for most things, and as a cache for
> LDAP, if it's used. I think this is likely a sane approach here, as
> well.
>
> - Ryan

Yes, it is possible to use LDAP for Identity and SQL for the other
things, like Tokens and Policy. Quotas could be done the same way. You
just have to extract the Quotas calls out of the Identity Provider. It
might make sense to go in Policy, or into its own driver.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

Tim.Bell at cern

Jul 17, 2012, 2:16 PM

Post #7 of 12 (298 views)
Permalink

Re: [Keystone] Quotas: LDAP Help [In reply to]

+1 The corporate LDAP should be read-only for a source of user, roles and
attributes. Updating the corporate LDAP is not an option in many
environments which can significantly benefit from the structured directory
information available.

Thus, at minimum, allow a r/o LDAP and local DB store for any openstack
specific information that needs updating.

Tim

> -----Original Message-----
> From: openstack-bounces+tim.bell=cern.ch [at] lists
> [mailto:openstack-bounces+tim.bell=cern.ch [at] lists] On Behalf
> Of Ryan Lane
> Sent: 17 July 2012 20:43
> To: Adam Young
> Cc: Joseph Heck; openstack
> Subject: Re: [Openstack] [Keystone] Quotas: LDAP Help
>
> > I haven't been thinking about quotas, so bear with me here. A few
> thoughts:
> >
> > Certain deployments might not be able to touch the LDAP backend. I am
> > thinking specifically where there is a corporate AD/LDAP server. I
> > tried to keep the scheme dependency simple enough that it could be
> > layered onto a read-only scenario. If we put quotas into LDAP, it
> > might break on those deployments.
> >
>
> Many, many deployments won't be able to. Applications should generally
> assume they are read-only in regards to LDAP.
>
> > I can see that we don't want to define them in the Nova database, as
> > Swift might not have access to that, and swift is going to be one of
> > the primary consumers of Quotas. I am Assuming Quantum will have them
> as well.
> >
> > As you are aware, there is no metadata storage in the LDAP driver,
> > instead it is generated from the tenant and role information on the
> > fly. There is no place to store metadata in "groupOfNames" which is
> > the lowest( common
> > denominator) grouping used for Tenants. Probably the most correct
> > thing to do would be to use a "seeAlso" that points to where the
> > quota data is stored.
> >
>
> Let's try not to force things into attributes if possible.
>
> When LDAP is used, is the SQL backend not used at all? Why not store quota
> info in Keystone's SQL backend, but pull user info from LDAP, when
enabled?
>
> We should only consider storing something in LDAP if it's going to be
reused
> by other applications. LDAP has a strict schema for exactly this purpose.
If the
> quota information isn't directly usable by other applications we shouldn't
> store it in LDAP.
>
> Many applications with an LDAP backend also have an SQL backend, and use
> the SQL as primary storage for most things, and as a cache for LDAP, if
it's
> used. I think this is likely a sane approach here, as well.
>
> - Ryan
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

Attachments:

smime.p7s (5.08 KB)

heckj at mac

Jul 17, 2012, 2:33 PM

Post #8 of 12 (290 views)
Permalink

Re: [Keystone] Quotas: LDAP Help [In reply to]

That's the general area I was going to head with the Active Directory backend I'm hacking on. Chris Hoge of UOregon presented today (@ OSCON) on a local keystone hack that they did to enable LDAP AuthN + a fail back to SQL based system for their scientific computing cluster - follows a very similar model.

-joe

On Jul 17, 2012, at 2:16 PM, Tim Bell <Tim.Bell [at] cern> wrote:
> +1 The corporate LDAP should be read-only for a source of user, roles and
> attributes. Updating the corporate LDAP is not an option in many
> environments which can significantly benefit from the structured directory
> information available.
>
> Thus, at minimum, allow a r/o LDAP and local DB store for any openstack
> specific information that needs updating.
>
> Tim
>
>> -----Original Message-----
>> From: openstack-bounces+tim.bell=cern.ch [at] lists
>> [mailto:openstack-bounces+tim.bell=cern.ch [at] lists] On Behalf
>> Of Ryan Lane
>> Sent: 17 July 2012 20:43
>> To: Adam Young
>> Cc: Joseph Heck; openstack
>> Subject: Re: [Openstack] [Keystone] Quotas: LDAP Help
>>
>>> I haven't been thinking about quotas, so bear with me here. A few
>> thoughts:
>>>
>>> Certain deployments might not be able to touch the LDAP backend. I am
>>> thinking specifically where there is a corporate AD/LDAP server. I
>>> tried to keep the scheme dependency simple enough that it could be
>>> layered onto a read-only scenario. If we put quotas into LDAP, it
>>> might break on those deployments.
>>>
>>
>> Many, many deployments won't be able to. Applications should generally
>> assume they are read-only in regards to LDAP.
>>
>>> I can see that we don't want to define them in the Nova database, as
>>> Swift might not have access to that, and swift is going to be one of
>>> the primary consumers of Quotas. I am Assuming Quantum will have them
>> as well.
>>>
>>> As you are aware, there is no metadata storage in the LDAP driver,
>>> instead it is generated from the tenant and role information on the
>>> fly. There is no place to store metadata in "groupOfNames" which is
>>> the lowest( common
>>> denominator) grouping used for Tenants. Probably the most correct
>>> thing to do would be to use a "seeAlso" that points to where the
>>> quota data is stored.
>>>
>>
>> Let's try not to force things into attributes if possible.
>>
>> When LDAP is used, is the SQL backend not used at all? Why not store quota
>> info in Keystone's SQL backend, but pull user info from LDAP, when
> enabled?
>>
>> We should only consider storing something in LDAP if it's going to be
> reused
>> by other applications. LDAP has a strict schema for exactly this purpose.
> If the
>> quota information isn't directly usable by other applications we shouldn't
>> store it in LDAP.
>>
>> Many applications with an LDAP backend also have an SQL backend, and use
>> the SQL as primary storage for most things, and as a cache for LDAP, if
> it's
>> used. I think this is likely a sane approach here, as well.
>>
>> - Ryan
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack [at] lists
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

everett.toews at cybera

Jul 17, 2012, 2:44 PM

Post #9 of 12 (303 views)
Permalink

Re: [Keystone] Quotas: LDAP Help [In reply to]

On Tue, Jul 17, 2012 at 12:56 PM, Adam Young <ayoung [at] redhat> wrote:

> Yes, it is possible to use LDAP for Identity and SQL for the other things,
> like Tokens and Policy. Quotas could be done the same way. You just have
> to extract the Quotas calls out of the Identity Provider. It might make
> sense to go in Policy, or into its own driver.
>

Yep. I was thinking along the same lines. I think we could reasonably
sidestep the issue by storing the quotas in the SQL database. It probably
belongs in its own driver as I wouldn't want to pollute the Policy driver.

But, at this point, I think we need to revisit the question...do we want
quotas in Keystone?

This is the first I've heard of Boson, which makes sense as Kevin only
recently sent an email to the list. It looks like a much more comprehensive
and scalable solution to quotas. The fact that it would be cell aware is a
big benefit. Plus I would not want to see 3 quota implementations running
around in OpenStack (the current Nova impl., my Keystone impl., and Boson).
I would be willing to step aside to make way for Boson, if it is something
that would become a reality.

I'll respond to Kevin's thread on Boson...

Everett

iartarisi at suse

Jul 25, 2012, 7:19 AM

Post #10 of 12 (261 views)
Permalink

Re: [Keystone] Quotas: LDAP Help [In reply to]

On 07/17/2012 11:33 PM, Joseph Heck wrote:
> That's the general area I was going to head with the Active Directory backend I'm hacking on. Chris Hoge of UOregon presented today (@ OSCON) on a local keystone hack that they did to enable LDAP AuthN + a fail back to SQL based system for their scientific computing cluster - follows a very similar model.
>
> -joe
>
> On Jul 17, 2012, at 2:16 PM, Tim Bell <Tim.Bell [at] cern> wrote:
>> +1 The corporate LDAP should be read-only for a source of user, roles and
>> attributes. Updating the corporate LDAP is not an option in many
>> environments which can significantly benefit from the structured directory
>> information available.
>>
>> Thus, at minimum, allow a r/o LDAP and local DB store for any openstack
>> specific information that needs updating.
>>
>> Tim
>>
>>> -----Original Message-----
>>> From: openstack-bounces+tim.bell=cern.ch [at] lists
>>> [mailto:openstack-bounces+tim.bell=cern.ch [at] lists] On Behalf
>>> Of Ryan Lane
>>> Sent: 17 July 2012 20:43
>>> To: Adam Young
>>> Cc: Joseph Heck; openstack
>>> Subject: Re: [Openstack] [Keystone] Quotas: LDAP Help
>>>
>>>> I haven't been thinking about quotas, so bear with me here. A few
>>> thoughts:
>>>> Certain deployments might not be able to touch the LDAP backend. I am
>>>> thinking specifically where there is a corporate AD/LDAP server. I
>>>> tried to keep the scheme dependency simple enough that it could be
>>>> layered onto a read-only scenario. If we put quotas into LDAP, it
>>>> might break on those deployments.
>>>>
>>> Many, many deployments won't be able to. Applications should generally
>>> assume they are read-only in regards to LDAP.
>>>
>>>> I can see that we don't want to define them in the Nova database, as
>>>> Swift might not have access to that, and swift is going to be one of
>>>> the primary consumers of Quotas. I am Assuming Quantum will have them
>>> as well.
>>>> As you are aware, there is no metadata storage in the LDAP driver,
>>>> instead it is generated from the tenant and role information on the
>>>> fly. There is no place to store metadata in "groupOfNames" which is
>>>> the lowest( common
>>>> denominator) grouping used for Tenants. Probably the most correct
>>>> thing to do would be to use a "seeAlso" that points to where the
>>>> quota data is stored.
>>>>
>>> Let's try not to force things into attributes if possible.
>>>
>>> When LDAP is used, is the SQL backend not used at all? Why not store quota
>>> info in Keystone's SQL backend, but pull user info from LDAP, when
>> enabled?
>>> We should only consider storing something in LDAP if it's going to be
>> reused
>>> by other applications. LDAP has a strict schema for exactly this purpose.
>> If the
>>> quota information isn't directly usable by other applications we shouldn't
>>> store it in LDAP.
>>>
>>> Many applications with an LDAP backend also have an SQL backend, and use
>>> the SQL as primary storage for most things, and as a cache for LDAP, if
>> it's
>>> used. I think this is likely a sane approach here, as well.
>>>
>>> - Ryan
>>>
>>> _______________________________________________
>>>

Hi,

I just wanted to add a bit to this thread. We're currently working on a
hybrid backend between LDAP and SQL. I have a working version for a
specific setup in which the user accounts are stored in LDAP, but
tenants and roles are all stored in SQL together with other openstack
user accounts such as the nova admin account.

I basically just Frankensteined the two backends together for user
processing and left everything else to be handled by the SQL backend.
I'd like to hear other people's opinion on this or alternative
implementations.

https://gist.github.com/3176390

-Ionuț

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

ayoung at redhat

Jul 25, 2012, 8:32 AM

Post #11 of 12 (264 views)
Permalink

Re: [Keystone] Quotas: LDAP Help [In reply to]

On 07/25/2012 10:19 AM, Ionuț Arțăriși wrote:
> On 07/17/2012 11:33 PM, Joseph Heck wrote:
>> That's the general area I was going to head with the Active Directory
>> backend I'm hacking on. Chris Hoge of UOregon presented today (@
>> OSCON) on a local keystone hack that they did to enable LDAP AuthN +
>> a fail back to SQL based system for their scientific computing
>> cluster - follows a very similar model.
>>
>> -joe
>>
>> On Jul 17, 2012, at 2:16 PM, Tim Bell <Tim.Bell [at] cern> wrote:
>>> +1 The corporate LDAP should be read-only for a source of user,
>>> roles and
>>> attributes. Updating the corporate LDAP is not an option in many
>>> environments which can significantly benefit from the structured
>>> directory
>>> information available.
>>>
>>> Thus, at minimum, allow a r/o LDAP and local DB store for any openstack
>>> specific information that needs updating.
>>>
>>> Tim
>>>
>>>> -----Original Message-----
>>>> From: openstack-bounces+tim.bell=cern.ch [at] lists
>>>> [mailto:openstack-bounces+tim.bell=cern.ch [at] lists] On
>>>> Behalf
>>>> Of Ryan Lane
>>>> Sent: 17 July 2012 20:43
>>>> To: Adam Young
>>>> Cc: Joseph Heck; openstack
>>>> Subject: Re: [Openstack] [Keystone] Quotas: LDAP Help
>>>>
>>>>> I haven't been thinking about quotas, so bear with me here. A few
>>>> thoughts:
>>>>> Certain deployments might not be able to touch the LDAP backend.
>>>>> I am
>>>>> thinking specifically where there is a corporate AD/LDAP server. I
>>>>> tried to keep the scheme dependency simple enough that it could be
>>>>> layered onto a read-only scenario. If we put quotas into LDAP, it
>>>>> might break on those deployments.
>>>>>
>>>> Many, many deployments won't be able to. Applications should generally
>>>> assume they are read-only in regards to LDAP.
>>>>
>>>>> I can see that we don't want to define them in the Nova database, as
>>>>> Swift might not have access to that, and swift is going to be one of
>>>>> the primary consumers of Quotas. I am Assuming Quantum will have
>>>>> them
>>>> as well.
>>>>> As you are aware, there is no metadata storage in the LDAP driver,
>>>>> instead it is generated from the tenant and role information on the
>>>>> fly. There is no place to store metadata in "groupOfNames" which is
>>>>> the lowest( common
>>>>> denominator) grouping used for Tenants. Probably the most correct
>>>>> thing to do would be to use a "seeAlso" that points to where the
>>>>> quota data is stored.
>>>>>
>>>> Let's try not to force things into attributes if possible.
>>>>
>>>> When LDAP is used, is the SQL backend not used at all? Why not
>>>> store quota
>>>> info in Keystone's SQL backend, but pull user info from LDAP, when
>>> enabled?
>>>> We should only consider storing something in LDAP if it's going to be
>>> reused
>>>> by other applications. LDAP has a strict schema for exactly this
>>>> purpose.
>>> If the
>>>> quota information isn't directly usable by other applications we
>>>> shouldn't
>>>> store it in LDAP.
>>>>
>>>> Many applications with an LDAP backend also have an SQL backend,
>>>> and use
>>>> the SQL as primary storage for most things, and as a cache for
>>>> LDAP, if
>>> it's
>>>> used. I think this is likely a sane approach here, as well.
>>>>
>>>> - Ryan
>>>>
>>>> _______________________________________________
>>>>
>
> Hi,
>
> I just wanted to add a bit to this thread. We're currently working on
> a hybrid backend between LDAP and SQL. I have a working version for a
> specific setup in which the user accounts are stored in LDAP, but
> tenants and roles are all stored in SQL together with other openstack
> user accounts such as the nova admin account.
>
> I basically just Frankensteined the two backends together for user
> processing and left everything else to be handled by the SQL backend.
> I'd like to hear other people's opinion on this or alternative
> implementations.

Are tenants completely in the SQL DB? If so, how to you list tenants
for a given user?

Do you copy users from LDAP to SQL for anything?

>
> https://gist.github.com/3176390
>
> -Ionuț
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

iartarisi at suse

Aug 14, 2012, 6:05 AM

Post #12 of 12 (228 views)
Permalink

Re: [Keystone] Quotas: LDAP Help [In reply to]

On 07/25/2012 05:32 PM, Adam Young wrote:
> On 07/25/2012 10:19 AM, Ionuț Arțăriși wrote:
>>
>> Hi,
>>
>> I just wanted to add a bit to this thread. We're currently working on
>> a hybrid backend between LDAP and SQL. I have a working version for a
>> specific setup in which the user accounts are stored in LDAP, but
>> tenants and roles are all stored in SQL together with other openstack
>> user accounts such as the nova admin account.
>>
>> I basically just Frankensteined the two backends together for user
>> processing and left everything else to be handled by the SQL backend.
>> I'd like to hear other people's opinion on this or alternative
>> implementations.
>
> Are tenants completely in the SQL DB? If so, how to you list tenants
> for a given user?
>
> Do you copy users from LDAP to SQL for anything?

Urgh, sorry for the late answer.

Tenats are all in the SQL DB and no users are copied from LDAP to SQL.

For listing tenants for a given user, right now we have a hacky
get_tenants_for_user method which can be configured/rewritten by the
sysadmin. We have a sample method which adds a pre-configured tenant to
the existing list of tenants (from SQL) for usecases like: make all LDAP
users part of tenant X.

-Ionuț

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads