Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: OpenStack: Dev

[Swift][Keystone] Swift Quotas

 

 

OpenStack dev RSS feed   Index | Next | Previous | View Threaded


everett.toews at cybera

May 1, 2012, 1:04 PM

Post #1 of 10 (339 views)
Permalink
[Swift][Keystone] Swift Quotas

Hi All,

I led the session [1] on Swift Quotas at the Summit and I'd appreciate some
feedback on the updated spec [2] and blueprint [3]. I also have a couple of
design level questions.

1. Should we store the Swift quota data in Keystone?

One of the ideas that came out of the session was that we could store the
Swift quota data in Keystone. I took a look at Keystone and it appears the
best place for this would be in the metada table but it appears to me that
it's only accessible via the User Metadata Extension API in Keystone. Is it
feasible to store the Swift quota data in Keystone this way? Should we?

On a related note, during the Pluggable Tenant Data session [4] led by Phil
Day something similar was being discussed for moving Nova quotas to
Keystone.

2. Where should the code for Swift quotas live?

It was suggested during the session that this code could live in a
middleware for Swift. Seems like a reasonable approach to me. I've taken a
look at some of the code in /swift/common/middleware and it looks
relatively straight-forward. Any thoughts or suggestions on implementing
this as middleware?

Regards,
Everett

[1] http://etherpad.openstack.org/SwiftQuotas
[2] http://wiki.openstack.org/SwiftQuotas
[3] https://blueprints.launchpad.net/swift/+spec/storage-quotas
[4] http://etherpad.openstack.org/FolsomPluggableTenantData


dougw at sdsc

May 3, 2012, 6:53 AM

Post #2 of 10 (318 views)
Permalink
Re: [Swift][Keystone] Swift Quotas [In reply to]

On Tue, 01 May 2012, Everett Toews wrote:

> 2. Where should the code for Swift quotas live?
>
> It was suggested during the session that this code could live in a
> middleware for Swift. Seems like a reasonable approach to me. I've taken a
> look at some of the code in /swift/common/middleware and it looks
> relatively straight-forward. Any thoughts or suggestions on implementing
> this as middleware?

I think a quota implementation should be split into two separate
middleware pieces; one for quota discovery and one for quota enforcement.
The discovery middleware could place the quota settings for a given tenant
into a set of X-Quota-* headers. The enforcement middleware would then
check for any existing X-Quota-* headers and take action if they were
present. This would allow a deployment to use a custom authentication
system or a new quota enforcement method more easily than if it were
combined in a single middleware.

Thanks,

Doug

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


kevin.mitchell at rackspace

May 3, 2012, 8:35 AM

Post #3 of 10 (317 views)
Permalink
Re: [Swift][Keystone] Swift Quotas [In reply to]

I missed the first post(s) in this thread, but I should probably put out
there that I'm currently working on refactoring quotas in Nova; see:

* https://blueprints.launchpad.net/nova/+spec/quota-refactor
* https://github.com/klmitch/nova/tree/quota-atomicity
* https://review.openstack.org/#/c/6774
* https://review.openstack.org/#/c/7048

To get a sense of what I'm doing. I've also been thinking about the
constraints of an external quota manager, but haven't gotten much
further than some kind of RPC or REST-based API.

(Note I haven't been strongly considering integrating this with Keystone
for a couple of reasons: 1. I tend to prefer the UNIX paradigm of doing
one thing well; 2. I want to ensure that this external quota manager is
usable for those who choose to use something other than Keystone.)

Feel free to ask me questions; I'm sure there's a lot of stuff I've
thought of that may not be obvious from the above references, and your
questions will probably help me articulate it better :)
--
Kevin L. Mitchell <kevin.mitchell [at] rackspace>


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


everett.toews at cybera

May 3, 2012, 3:58 PM

Post #4 of 10 (315 views)
Permalink
Re: [Swift][Keystone] Swift Quotas [In reply to]

Since storing the quota data in Keystone is a prereq for do the quotas in
Swift, I'm starting there. After digging through the Keystone code a bit
I've identified at least one issue with storing the quota data per tenant
for the SQL backend.

In the metadata table both tenant_id and user_id are primary keys. So the
question is, *within the current design* how do you store some piece of
metadata per tenant only?

Storing the same quota data per user for a tenant doesn't work because of
the many-to-many relationship between users and tenants.

Another option is to use a static user_id for rows where you want to store
metadata per tenant.
e.g. user_id='per_tenant_metadata',
tenant_id='55b6d515e00e48c38e2c92d27dc5c03e', data='{"quota": ...}' (I'll
figure out what the JSON looks like later)

If you retrieved the quotas via SQL it would look something like,
select data from metadata where user_id='per_tenant_metadata' and
tenant_id='55b6d515e00e48c38e2c92d27dc5c03e';

This works but doesn't feel like the cleanest solution. Any thoughts on
this approach?

Of course, stepping outside the current design would yield more options but
I wanted to look for a solution within the current design first.

Everett

On Tue, May 1, 2012 at 2:04 PM, Everett Toews <everett.toews [at] cybera>wrote:

> Hi All,
>
> I led the session [1] on Swift Quotas at the Summit and I'd appreciate
> some feedback on the updated spec [2] and blueprint [3]. I also have a
> couple of design level questions.
>
> 1. Should we store the Swift quota data in Keystone?
>
> One of the ideas that came out of the session was that we could store the
> Swift quota data in Keystone. I took a look at Keystone and it appears the
> best place for this would be in the metada table but it appears to me that
> it's only accessible via the User Metadata Extension API in Keystone. Is it
> feasible to store the Swift quota data in Keystone this way? Should we?
>
> On a related note, during the Pluggable Tenant Data session [4] led by
> Phil Day something similar was being discussed for moving Nova quotas to
> Keystone.
>
> 2. Where should the code for Swift quotas live?
>
> It was suggested during the session that this code could live in a
> middleware for Swift. Seems like a reasonable approach to me. I've taken a
> look at some of the code in /swift/common/middleware and it looks
> relatively straight-forward. Any thoughts or suggestions on implementing
> this as middleware?
>
> Regards,
> Everett
>
> [1] http://etherpad.openstack.org/SwiftQuotas
> [2] http://wiki.openstack.org/SwiftQuotas
> [3] https://blueprints.launchpad.net/swift/+spec/storage-quotas
> [4] http://etherpad.openstack.org/FolsomPluggableTenantData
>


endre.karlson at gmail

May 4, 2012, 12:25 AM

Post #5 of 10 (315 views)
Permalink
Re: [Swift][Keystone] Swift Quotas [In reply to]

Should you mix this into Keystone ? Seems kind of wrong to mix identity
manager with Quotas?

2012/5/4 Everett Toews <everett.toews [at] cybera>

> Since storing the quota data in Keystone is a prereq for do the quotas in
> Swift, I'm starting there. After digging through the Keystone code a bit
> I've identified at least one issue with storing the quota data per tenant
> for the SQL backend.
>
> In the metadata table both tenant_id and user_id are primary keys. So the
> question is, *within the current design* how do you store some piece of
> metadata per tenant only?
>
> Storing the same quota data per user for a tenant doesn't work because of
> the many-to-many relationship between users and tenants.
>
> Another option is to use a static user_id for rows where you want to store
> metadata per tenant.
> e.g. user_id='per_tenant_metadata',
> tenant_id='55b6d515e00e48c38e2c92d27dc5c03e', data='{"quota": ...}' (I'll
> figure out what the JSON looks like later)
>
> If you retrieved the quotas via SQL it would look something like,
> select data from metadata where user_id='per_tenant_metadata' and
> tenant_id='55b6d515e00e48c38e2c92d27dc5c03e';
>
> This works but doesn't feel like the cleanest solution. Any thoughts on
> this approach?
>
> Of course, stepping outside the current design would yield more options
> but I wanted to look for a solution within the current design first.
>
> Everett
>
> On Tue, May 1, 2012 at 2:04 PM, Everett Toews <everett.toews [at] cybera>wrote:
>
>> Hi All,
>>
>> I led the session [1] on Swift Quotas at the Summit and I'd appreciate
>> some feedback on the updated spec [2] and blueprint [3]. I also have a
>> couple of design level questions.
>>
>> 1. Should we store the Swift quota data in Keystone?
>>
>> One of the ideas that came out of the session was that we could store the
>> Swift quota data in Keystone. I took a look at Keystone and it appears the
>> best place for this would be in the metada table but it appears to me that
>> it's only accessible via the User Metadata Extension API in Keystone. Is it
>> feasible to store the Swift quota data in Keystone this way? Should we?
>>
>> On a related note, during the Pluggable Tenant Data session [4] led by
>> Phil Day something similar was being discussed for moving Nova quotas to
>> Keystone.
>>
>> 2. Where should the code for Swift quotas live?
>>
>> It was suggested during the session that this code could live in a
>> middleware for Swift. Seems like a reasonable approach to me. I've taken a
>> look at some of the code in /swift/common/middleware and it looks
>> relatively straight-forward. Any thoughts or suggestions on implementing
>> this as middleware?
>>
>> Regards,
>> Everett
>>
>> [1] http://etherpad.openstack.org/SwiftQuotas
>> [2] http://wiki.openstack.org/SwiftQuotas
>> [3] https://blueprints.launchpad.net/swift/+spec/storage-quotas
>> [4] http://etherpad.openstack.org/FolsomPluggableTenantData
>>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>


eglynn at redhat

May 4, 2012, 2:10 AM

Post #6 of 10 (314 views)
Permalink
Re: [Swift][Keystone] Swift Quotas [In reply to]

> Should you mix this into Keystone ? Seems kind of wrong to mix
> identity manager with Quotas?


This was discussed at several sessions the design summit, so I
brought it up at the keystone 'state of the nation' session to
get a feel for the keystone community's disposition to the idea.

There was in-principle agreement that storing such data in the
keystone backend is a reasonable approach (well, no strong objections
in any case).

The idea is that the determining if a user is over-quota is related
to authorization in the sense that it determines whether a user
normally authorized to consume a resource is allowed to do so in
this particular case.

>From an implementation point, it would also aid greatly in allowing
the nova quotas infrastructure to be promoted to openstack-common
and hence be reused by other projects such as glance.

Cheers,
Eoghan

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


jason.rouault at hp

May 4, 2012, 7:14 AM

Post #7 of 10 (321 views)
Permalink
Re: [Swift][Keystone] Swift Quotas [In reply to]

IMHO, if it is a quota related to a tenant or user, then managing it in
Keystone makes sense.

Jason

-----Original Message-----
From: openstack-bounces+jason.rouault=hp.com [at] lists
[mailto:openstack-bounces+jason.rouault=hp.com [at] lists] On
Behalf Of Eoghan Glynn
Sent: Friday, May 04, 2012 3:11 AM
To: Endre Karlson
Cc: openstack [at] lists
Subject: Re: [Openstack] [Swift][Keystone] Swift Quotas



> Should you mix this into Keystone ? Seems kind of wrong to mix
> identity manager with Quotas?


This was discussed at several sessions the design summit, so I brought it up
at the keystone 'state of the nation' session to get a feel for the keystone
community's disposition to the idea.

There was in-principle agreement that storing such data in the keystone
backend is a reasonable approach (well, no strong objections in any case).

The idea is that the determining if a user is over-quota is related to
authorization in the sense that it determines whether a user normally
authorized to consume a resource is allowed to do so in this particular
case.

>From an implementation point, it would also aid greatly in allowing
the nova quotas infrastructure to be promoted to openstack-common and hence
be reused by other projects such as glance.

Cheers,
Eoghan

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Attachments: smime.p7s (4.74 KB)


jaypipes at gmail

May 4, 2012, 9:02 AM

Post #8 of 10 (314 views)
Permalink
Re: [Swift][Keystone] Swift Quotas [In reply to]

On 05/04/2012 10:14 AM, Rouault, Jason (Cloud Services) wrote:
> IMHO, if it is a quota related to a tenant or user, then managing it in
> Keystone makes sense.

I think, as Eoghan mentioned, there was pretty broad agreement that
*storing* quota information and even the CRUD operations on it were
appropriate for Keystone to do, but the actual enforcement of quotas
should be left in individual projects.

Best,
-jay

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


doug.hellmann at dreamhost

May 4, 2012, 10:11 AM

Post #9 of 10 (309 views)
Permalink
Re: [Swift][Keystone] Swift Quotas [In reply to]

On Fri, May 4, 2012 at 12:02 PM, Jay Pipes <jaypipes [at] gmail> wrote:

> On 05/04/2012 10:14 AM, Rouault, Jason (Cloud Services) wrote:
>
>> IMHO, if it is a quota related to a tenant or user, then managing it in
>> Keystone makes sense.
>>
>
> I think, as Eoghan mentioned, there was pretty broad agreement that
> *storing* quota information and even the CRUD operations on it were
> appropriate for Keystone to do, but the actual enforcement of quotas should
> be left in individual projects.
>

That's how I remember it, too.

Doug


everett.toews at cybera

May 4, 2012, 10:28 AM

Post #10 of 10 (321 views)
Permalink
Re: [Swift][Keystone] Swift Quotas [In reply to]

Yep, storing the quota info is all I'm trying to achieve with Keystone. For
Swift, the enforcement would happen within Swift (middleware).

I see it as being roughly analogous to roles in Keystone. The roles are
stored in Keystone but enforced by the individual projects.

Everett

On Fri, May 4, 2012 at 10:02 AM, Jay Pipes <jaypipes [at] gmail> wrote:

> On 05/04/2012 10:14 AM, Rouault, Jason (Cloud Services) wrote:
>
>> IMHO, if it is a quota related to a tenant or user, then managing it in
>> Keystone makes sense.
>>
>
> I think, as Eoghan mentioned, there was pretty broad agreement that
> *storing* quota information and even the CRUD operations on it were
> appropriate for Keystone to do, but the actual enforcement of quotas should
> be left in individual projects.
>
> Best,
> -jay
>
>
> ______________________________**_________________
> Mailing list: https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> More help : https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp>
>

OpenStack dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.