Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: OpenStack: Dev

db & notification support for API extension?

 

 

OpenStack dev RSS feed   Index | Next | Previous | View Threaded


abogott at wikimedia

Mar 8, 2012, 9:53 AM

Post #1 of 7 (325 views)
Permalink
db & notification support for API extension?

I'm working on an API and implementation to support the creation of
filesystems that are shared among Nova instances.

http://wiki.openstack.org/SharedFS

My hope is to keep this API isolated from core Nova code, partly to
avoid stepping on toes and partly because I hope to be able to drop it
into an existing essex install. There are two things I need which I
know how to do within Nova but am not clear on how to do without modding
Nova code:

1) DB support

I need a database table to keep track of some filesystem metadata.
My current implementation adds the table via
nova/db/sqlalchemy/migrate_repo... but is it really necessary to
coordinate this table with the rest of Nova? Would it be reasonable to
maintain the table independently within the extension code? And, if so,
are there any existing extensions that do something like this?

2) Responding to nova events

In order to manage access permissions, I need to hook the creation
and deletion of instances. The easiest way to do this is by hooking
into existing nova functions; a more isolated way is to listen on the
queue for creation/deletion notifications. My question about that
latter method is: who will listen? Should my API extension just spin
up a daemon and leave it to run forever? Should I create a full-blown
service for this?

I can think of various approaches to both these problems, but I welcome
your suggestions. Thank you!

-Andrew


johannes at erdfelt

Mar 8, 2012, 10:04 AM

Post #2 of 7 (324 views)
Permalink
Re: db & notification support for API extension? [In reply to]

On Thu, Mar 08, 2012, Andrew Bogott <abogott [at] wikimedia> wrote:
> 1) DB support
>
> I need a database table to keep track of some filesystem
> metadata. My current implementation adds the table via
> nova/db/sqlalchemy/migrate_repo... but is it really necessary to
> coordinate this table with the rest of Nova? Would it be reasonable
> to maintain the table independently within the extension code? And,
> if so, are there any existing extensions that do something like
> this?

This is another area where sqlalchemy-migrate is lacking. It makes it
very difficult to maintain out-of-tree migrations.

I started on a port to alembic, which has a migration model that would
make this easy to maintain in a fork.

It's incomplete and three is over a month old now, but for reference:

https://github.com/jerdfelt/nova/tree/alembic

It probably wouldn't help drop-in plugins. Assuming it only creates new
tables and doesn't touch existing ones, you might be able to maintain
migrations separately.

JE


nathanael.i.burton at gmail

Apr 25, 2012, 2:48 PM

Post #3 of 7 (324 views)
Permalink
Re: db & notification support for API extension? [In reply to]

On Thu, Mar 8, 2012 at 11:53 AM, Andrew Bogott <abogott [at] wikimedia> wrote:
>    I'm working on an API and implementation to support the creation of
> filesystems that are shared among Nova instances.
>
> http://wiki.openstack.org/SharedFS
>
>    My hope is to keep this API isolated from core Nova code, partly to avoid
> stepping on toes and partly because I hope to be able to drop it into an
> existing essex install.  There are two things I need which I know how to do
> within Nova but am not clear on how to do without modding Nova code:
>
> 1)  DB support
>
>    I need a database table to keep track of some filesystem metadata.  My
> current implementation adds the table via nova/db/sqlalchemy/migrate_repo...
> but is it really necessary to coordinate this table with the rest of Nova?
>  Would it be reasonable to maintain the table independently within the
> extension code?  And, if so, are there any existing extensions that do
> something like this?

Have you determined a cleaner way of doing this? I have the same
issues as you when writing API extensions.

Nate

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


abogott at wikimedia

Apr 25, 2012, 3:04 PM

Post #4 of 7 (326 views)
Permalink
Re: db & notification support for API extension? [In reply to]

On 4/25/12 4:48 PM, Nathanael Burton wrote:
> On Thu, Mar 8, 2012 at 11:53 AM, Andrew Bogott<abogott [at] wikimedia> wrote:
>> I'm working on an API and implementation to support the creation of
>> filesystems that are shared among Nova instances.
>>
>> http://wiki.openstack.org/SharedFS
>>
>> My hope is to keep this API isolated from core Nova code, partly to avoid
>> stepping on toes and partly because I hope to be able to drop it into an
>> existing essex install. There are two things I need which I know how to do
>> within Nova but am not clear on how to do without modding Nova code:
>>
>> 1) DB support
>>
>> I need a database table to keep track of some filesystem metadata. My
>> current implementation adds the table via nova/db/sqlalchemy/migrate_repo...
>> but is it really necessary to coordinate this table with the rest of Nova?
>> Would it be reasonable to maintain the table independently within the
>> extension code? And, if so, are there any existing extensions that do
>> something like this?
> Have you determined a cleaner way of doing this? I have the same
> issues as you when writing API extensions.
Nate --

The short answer is: I'm sure that it's straightforward to create a
'private' table which doesn't collide with existing nova tables, but I
have yet to do so.

The longer answer is: Everything about that thread is now rolled into
the topic of 'the plugin framework' which we discussed at the design
summit and which I'm currently devoted to. Please consider adding your
use cases to the wiki page at http://wiki.openstack.org/novaplugin, and
let me know if you would like me to add you to the list of people I cc:
when looking for opinions and/or reporting progress.

-Andrew

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


nathanael.i.burton at gmail

Apr 25, 2012, 3:18 PM

Post #5 of 7 (328 views)
Permalink
Re: db & notification support for API extension? [In reply to]

On Wed, Apr 25, 2012 at 6:04 PM, Andrew Bogott <abogott [at] wikimedia> wrote:
> Nate --
>
> The short answer is:  I'm sure that it's straightforward to create a
> 'private' table which doesn't collide with existing nova tables, but I have
> yet to do so.
>
> The longer answer is:  Everything about that thread is now rolled into the
> topic of 'the plugin framework' which we discussed at the design summit and
> which I'm currently devoted to.  Please consider adding your use cases to
> the wiki page at http://wiki.openstack.org/novaplugin, and let me know if
> you would like me to add you to the list of people I cc: when looking for
> opinions and/or reporting progress.
>
> -Andrew

Wish I would have made it to that session, but that sounds exactly
like what I'm looking for. I'll follow and contribute ideas as
necessary.

Thanks.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


harlowja at yahoo-inc

Apr 25, 2012, 5:29 PM

Post #6 of 7 (324 views)
Permalink
Re: db & notification support for API extension? [In reply to]

Has there been any investigation into using already existing plugin frameworks and just use those (and/or make those better)?

Just from a quick google search:

http://wehart.blogspot.com/2009/01/python-plugin-frameworks.html

It might be useful to see if we can find one that is generic and well-supported and already works "good enough"??

On 4/25/12 3:04 PM, "Andrew Bogott" <abogott [at] wikimedia> wrote:

On 4/25/12 4:48 PM, Nathanael Burton wrote:
> On Thu, Mar 8, 2012 at 11:53 AM, Andrew Bogott<abogott [at] wikimedia> wrote:
>> I'm working on an API and implementation to support the creation of
>> filesystems that are shared among Nova instances.
>>
>> http://wiki.openstack.org/SharedFS
>>
>> My hope is to keep this API isolated from core Nova code, partly to avoid
>> stepping on toes and partly because I hope to be able to drop it into an
>> existing essex install. There are two things I need which I know how to do
>> within Nova but am not clear on how to do without modding Nova code:
>>
>> 1) DB support
>>
>> I need a database table to keep track of some filesystem metadata. My
>> current implementation adds the table via nova/db/sqlalchemy/migrate_repo...
>> but is it really necessary to coordinate this table with the rest of Nova?
>> Would it be reasonable to maintain the table independently within the
>> extension code? And, if so, are there any existing extensions that do
>> something like this?
> Have you determined a cleaner way of doing this? I have the same
> issues as you when writing API extensions.
Nate --

The short answer is: I'm sure that it's straightforward to create a
'private' table which doesn't collide with existing nova tables, but I
have yet to do so.

The longer answer is: Everything about that thread is now rolled into
the topic of 'the plugin framework' which we discussed at the design
summit and which I'm currently devoted to. Please consider adding your
use cases to the wiki page at http://wiki.openstack.org/novaplugin, and
let me know if you would like me to add you to the list of people I cc:
when looking for opinions and/or reporting progress.

-Andrew

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


doug.hellmann at dreamhost

Apr 26, 2012, 6:30 AM

Post #7 of 7 (323 views)
Permalink
Re: db & notification support for API extension? [In reply to]

On Wed, Apr 25, 2012 at 6:04 PM, Andrew Bogott <abogott [at] wikimedia>wrote:

> On 4/25/12 4:48 PM, Nathanael Burton wrote:
>
>> On Thu, Mar 8, 2012 at 11:53 AM, Andrew Bogott<abogott [at] wikimedia>
>> wrote:
>>
>>> I'm working on an API and implementation to support the creation of
>>> filesystems that are shared among Nova instances.
>>>
>>> http://wiki.openstack.org/**SharedFS<http://wiki.openstack.org/SharedFS>
>>>
>>> My hope is to keep this API isolated from core Nova code, partly to
>>> avoid
>>> stepping on toes and partly because I hope to be able to drop it into an
>>> existing essex install. There are two things I need which I know how to
>>> do
>>> within Nova but am not clear on how to do without modding Nova code:
>>>
>>> 1) DB support
>>>
>>> I need a database table to keep track of some filesystem metadata. My
>>> current implementation adds the table via nova/db/sqlalchemy/migrate_**
>>> repo...
>>> but is it really necessary to coordinate this table with the rest of
>>> Nova?
>>> Would it be reasonable to maintain the table independently within the
>>> extension code? And, if so, are there any existing extensions that do
>>> something like this?
>>>
>> Have you determined a cleaner way of doing this? I have the same
>> issues as you when writing API extensions.
>>
> Nate --
>
> The short answer is: I'm sure that it's straightforward to create a
> 'private' table which doesn't collide with existing nova tables, but I have
> yet to do so.
>
> The longer answer is: Everything about that thread is now rolled into the
> topic of 'the plugin framework' which we discussed at the design summit and
> which I'm currently devoted to. Please consider adding your use cases to
> the wiki page at http://wiki.openstack.org/**novaplugin<http://wiki.openstack.org/novaplugin>,
> and let me know if you would like me to add you to the list of people I cc:
> when looking for opinions and/or reporting progress.


The wiki page says that a plugin may want to "Access the Nova database" but
that phrasing is a little vague. Does it mean "Read and write data to its
own tables in the Nova database" (as mentioned later on the page) or "Read
data from the Nova tables" or even "Write data to the Nova tables"?

I assume if we're talking about Nova tables, access would be through the
existing core classes in Nova that manage those tables, rather than
manipulating them directly. Should that be stated explicitly?


>
>
> -Andrew
>
>
> ______________________________**_________________
> Mailing list: https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> More help : https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp>
>

OpenStack dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.