Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: OpenStack: Dev

RFC - dynamically loading virt drivers

 

 

OpenStack dev RSS feed   Index | Next | Previous | View Threaded


sdague at linux

May 17, 2012, 1:52 PM

Post #1 of 8 (479 views)
Permalink
RFC - dynamically loading virt drivers

Rationale: nova loads drivers for various subsystems in very different
ways, and not all of them are truly dynamic (i.e. a new driver could be
fully self contained and not have to change a core source code file or
monkey patch to load itself). I'd like to get all the driver plug points
fully dynamic, and eventually a common pattern and loading mechanism is
used on all of them, from a consistency perspective.

I've got a first attempt at refactoring the nova.virt.connection module
to stop having a pre-defined list of virt drivers as strings, and
instead load modules dynamically using the the
nova.openstack.commons.importutils. This is in a branch on github -
https://github.com/sdague/nova

The changed files are:
git diff master | diffstat -w 20
tests/test_virt_driver_loader.py | 54 ++++++++++
utils.py | 3
virt/connection.py | 51 ++-------
virt/fake.py | 4
virt/libvirt/__init__.py | 1
5 files changed, 75 insertions(+), 38 deletions(-)

This passes the unit test battery on fake and libvirt drivers (* on
fake) Because the strings passed to connection_type do map to module
names, this shouldn't cause any issues with existing configurations.

The xenapi and vmwareapi modules probably just need a similar
__init__.py addition to make get_connection available on module load.
Baremetal will be slightly more work (but not much) because it had some
additional setup in nova/virt/connection.py.

What I'm mostly looking for is comments on approach. Is importutils the
prefered way to go about this (which is the nova.volume approach) now,
or should this be using utils.LazyPluggable as is in nova.db.api, or
some other approach entirely? Comments, redirections, appreciated.

* one test fails for Fake in test_virt_drivers, but only when it's run
as the full unit test, not when run on it's own. It looks like it has to
do with FakeConnection.instance() caching, which actually confuses me a
bit, as I would have assumed one unit test file couldn't affect another
(i.e. they started a clean env each time).

-Sean

--
Sean Dague
IBM Linux Technology Center
email: sdague [at] linux
alt-email: sldague [at] us


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


vishvananda at gmail

May 17, 2012, 3:38 PM

Post #2 of 8 (457 views)
Permalink
Re: RFC - dynamically loading virt drivers [In reply to]

On May 17, 2012, at 1:52 PM, Sean Dague wrote:
>
> What I'm mostly looking for is comments on approach. Is importutils the prefered way to go about this (which is the nova.volume approach) now, or should this be using utils.LazyPluggable as is in nova.db.api, or some other approach entirely? Comments, redirections, appreciated.

-1 to LazyPluggable

So we already have plugabillity by just specifying a different compute_driver config option. I don't like that we defer another level in compute and call get_connection. IMO the best cleanup would be to remove the get_connection altogether and just construct the driver directly based on compute_driver.

The main issue with changing this is breaking existing installs.

So I guess this would be my strategy:

a) remove get_connection from the drivers (and just have it construct the 'connection' class directly)
b) modify the global get_connection to construct the drivers for backwards compatibilty
c) modify the documentation to suggest changing drivers by specifying the full path to the driver instead of connection_type
d) rename the connection classes to something reasonable representing drivers (libvirt.driver:LibvirtDriver() vs libvirt.connection.LibvirtConnection)
e) bonus points if it could be done with a short path for ease of use (compute_driver=libvirt.LibvirtDriver vs compute_driver=nova.virt.libvirt.driver.LibvirtDriver)

>
> * one test fails for Fake in test_virt_drivers, but only when it's run as the full unit test, not when run on it's own. It looks like it has to do with FakeConnection.instance() caching, which actually confuses me a bit, as I would have assumed one unit test file couldn't affect another (i.e. they started a clean env each time).

Generally breakage like this is due to some global state that is not cleaned up, so if FakeConnection is caching globally, then this could happen.

Vish


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


jaypipes at gmail

May 18, 2012, 11:14 AM

Post #3 of 8 (455 views)
Permalink
Re: RFC - dynamically loading virt drivers [In reply to]

On 05/17/2012 06:38 PM, Vishvananda Ishaya wrote:
> On May 17, 2012, at 1:52 PM, Sean Dague wrote:
>>
>> What I'm mostly looking for is comments on approach. Is importutils the prefered way to go about this (which is the nova.volume approach) now, or should this be using utils.LazyPluggable as is in nova.db.api, or some other approach entirely? Comments, redirections, appreciated.
>
> -1 to LazyPluggable
>
> So we already have plugabillity by just specifying a different compute_driver config option. I don't like that we defer another level in compute and call get_connection. IMO the best cleanup would be to remove the get_connection altogether and just construct the driver directly based on compute_driver.
>
> The main issue with changing this is breaking existing installs.
>
> So I guess this would be my strategy:
>
> a) remove get_connection from the drivers (and just have it construct the 'connection' class directly)
> b) modify the global get_connection to construct the drivers for backwards compatibilty
> c) modify the documentation to suggest changing drivers by specifying the full path to the driver instead of connection_type
> d) rename the connection classes to something reasonable representing drivers (libvirt.driver:LibvirtDriver() vs libvirt.connection.LibvirtConnection)
> e) bonus points if it could be done with a short path for ease of use (compute_driver=libvirt.LibvirtDriver vs compute_driver=nova.virt.libvirt.driver.LibvirtDriver)

+1

-jay

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


sdague at linux

May 18, 2012, 12:08 PM

Post #4 of 8 (462 views)
Permalink
Re: RFC - dynamically loading virt drivers [In reply to]

On 05/17/2012 06:38 PM, Vishvananda Ishaya wrote:
>
> So we already have plugabillity by just specifying a different compute_driver config option. I don't like that we defer another level in compute and call get_connection. IMO the best cleanup would be to remove the get_connection altogether and just construct the driver directly based on compute_driver.
>
> The main issue with changing this is breaking existing installs.
>
> So I guess this would be my strategy:
>
> a) remove get_connection from the drivers (and just have it construct the 'connection' class directly)
> b) modify the global get_connection to construct the drivers for backwards compatibilty
> c) modify the documentation to suggest changing drivers by specifying the full path to the driver instead of connection_type
> d) rename the connection classes to something reasonable representing drivers (libvirt.driver:LibvirtDriver() vs libvirt.connection.LibvirtConnection)
> e) bonus points if it could be done with a short path for ease of use (compute_driver=libvirt.LibvirtDriver vs compute_driver=nova.virt.libvirt.driver.LibvirtDriver)

On point c), is the long term view that .conf options are going to
specify full class names? It seems like this actually gets kind of
confusing to admins.


What are your thoughts on the following approach, which is related, but
a little different?

a) have compute_driver take a module name in nova.virt. which is loaded
with some standard construction method that all drivers would implement
in their __init__.py. Match all existing module names to connection_type
names current in use. Basically just jump to e, but also make all
drivers conform some factory interface so "libvirt" is actually enough
to get you nova.virt.libvirt.connect()

b) if compute_driver is not specified, use connection_type, but spit out
a deprecation warning that the option is going away. (Removed fully in
G). Because compute_drivers map to existing connection_types this just
works with only a little refactoring in the drivers.

c) remove nova/virt/connection.py

The end result is that every driver is a self contained subdir in
nova/virt/DRIVERNAME/.

>> * one test fails for Fake in test_virt_drivers, but only when it's run as the full unit test, not when run on it's own. It looks like it has to do with FakeConnection.instance() caching, which actually confuses me a bit, as I would have assumed one unit test file couldn't affect another (i.e. they started a clean env each time).
>
> Generally breakage like this is due to some global state that is not cleaned up, so if FakeConnection is caching globally, then this could happen.

It is keeping global state, I'll look at fixing that independently.

-Sean

--
Sean Dague
IBM Linux Technology Center
email: sdague [at] linux
alt-email: sldague [at] us


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


doug.hellmann at dreamhost

May 18, 2012, 1:49 PM

Post #5 of 8 (456 views)
Permalink
Re: RFC - dynamically loading virt drivers [In reply to]

On Fri, May 18, 2012 at 3:08 PM, Sean Dague <sdague [at] linux>wrote:

> On 05/17/2012 06:38 PM, Vishvananda Ishaya wrote:
> >
>
>> So we already have plugabillity by just specifying a different
>> compute_driver config option. I don't like that we defer another level in
>> compute and call get_connection. IMO the best cleanup would be to remove
>> the get_connection altogether and just construct the driver directly based
>> on compute_driver.
>>
>> The main issue with changing this is breaking existing installs.
>>
>> So I guess this would be my strategy:
>>
>> a) remove get_connection from the drivers (and just have it construct the
>> 'connection' class directly)
>> b) modify the global get_connection to construct the drivers for
>> backwards compatibilty
>> c) modify the documentation to suggest changing drivers by specifying the
>> full path to the driver instead of connection_type
>> d) rename the connection classes to something reasonable representing
>> drivers (libvirt.driver:LibvirtDriver(**) vs libvirt.connection.**
>> LibvirtConnection)
>> e) bonus points if it could be done with a short path for ease of use
>> (compute_driver=libvirt.**LibvirtDriver vs compute_driver=nova.virt.**
>> libvirt.driver.LibvirtDriver)
>>
>
> On point c), is the long term view that .conf options are going to specify
> full class names? It seems like this actually gets kind of confusing to
> admins.
>
>
> What are your thoughts on the following approach, which is related, but a
> little different?
>
> a) have compute_driver take a module name in nova.virt. which is loaded
> with some standard construction method that all drivers would implement in
> their __init__.py. Match all existing module names to connection_type names
> current in use. Basically just jump to e, but also make all drivers conform
> some factory interface so "libvirt" is actually enough to get you
> nova.virt.libvirt.connect()
>

Andrew Bogott is working on a common plugin architecture. Under that system
plugins will have well-known, but short names and be loaded using
setuptools entry points (allowing them to be named independently of their
code/filesystem layout and packaged and installed separately from
nova). Could the drivers be loaded from these plugins?


>
> b) if compute_driver is not specified, use connection_type, but spit out a
> deprecation warning that the option is going away. (Removed fully in G).
> Because compute_drivers map to existing connection_types this just works
> with only a little refactoring in the drivers.
>
> c) remove nova/virt/connection.py
>
> The end result is that every driver is a self contained subdir in
> nova/virt/DRIVERNAME/.
>
>
> * one test fails for Fake in test_virt_drivers, but only when it's run as
>>> the full unit test, not when run on it's own. It looks like it has to do
>>> with FakeConnection.instance() caching, which actually confuses me a bit,
>>> as I would have assumed one unit test file couldn't affect another (i.e.
>>> they started a clean env each time).
>>>
>>
>> Generally breakage like this is due to some global state that is not
>> cleaned up, so if FakeConnection is caching globally, then this could
>> happen.
>>
>
> It is keeping global state, I'll look at fixing that independently.
>
>
> -Sean
>
> --
> Sean Dague
> IBM Linux Technology Center
> email: sdague [at] linux
> alt-email: sldague [at] us
>
>
> ______________________________**_________________
> Mailing list: https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> More help : https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp>
>


sdague at linux

May 21, 2012, 11:57 AM

Post #6 of 8 (450 views)
Permalink
Re: RFC - dynamically loading virt drivers [In reply to]

On 05/17/2012 06:38 PM, Vishvananda Ishaya wrote:
> The main issue with changing this is breaking existing installs.
>
> So I guess this would be my strategy:
>
> a) remove get_connection from the drivers (and just have it construct the 'connection' class directly)

I'm starting down this path, but a lot of the drivers do substantial
amount of FLAG handling in get_connection before they create the driver
objects.

Do you want to move that FLAGS handling into the object constructor, or
keep having some factory layer to manipulate the flags before we
construct the driver?

-Sean

--
Sean Dague
IBM Linux Technology Center
email: sdague [at] linux
alt-email: sldague [at] us


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


akirayoshiyama at gmail

May 21, 2012, 7:59 PM

Post #7 of 8 (454 views)
Permalink
Re: RFC - dynamically loading virt drivers [In reply to]

Hi,

FYI, I have a patch for nova-cactus to import only one virt driver.
http://www.debian.or.jp/~yosshy/nova-cactus-patches/1004.txt

Regards,
Akira Yoshiyama
2012/05/18 6:12 "Sean Dague" <sdague [at] linux>:

> Rationale: nova loads drivers for various subsystems in very different
> ways, and not all of them are truly dynamic (i.e. a new driver could be
> fully self contained and not have to change a core source code file or
> monkey patch to load itself). I'd like to get all the driver plug points
> fully dynamic, and eventually a common pattern and loading mechanism is
> used on all of them, from a consistency perspective.
>
> I've got a first attempt at refactoring the nova.virt.connection module to
> stop having a pre-defined list of virt drivers as strings, and instead load
> modules dynamically using the the nova.openstack.commons.**importutils.
> This is in a branch on github - https://github.com/sdague/nova
>
> The changed files are:
> git diff master | diffstat -w 20
> tests/test_virt_driver_loader.**py | 54 ++++++++++
> utils.py | 3
> virt/connection.py | 51 ++-------
> virt/fake.py | 4
> virt/libvirt/__init__.py | 1
> 5 files changed, 75 insertions(+), 38 deletions(-)
>
> This passes the unit test battery on fake and libvirt drivers (* on fake)
> Because the strings passed to connection_type do map to module names, this
> shouldn't cause any issues with existing configurations.
>
> The xenapi and vmwareapi modules probably just need a similar __init__.py
> addition to make get_connection available on module load. Baremetal will be
> slightly more work (but not much) because it had some additional setup in
> nova/virt/connection.py.
>
> What I'm mostly looking for is comments on approach. Is importutils the
> prefered way to go about this (which is the nova.volume approach) now, or
> should this be using utils.LazyPluggable as is in nova.db.api, or some
> other approach entirely? Comments, redirections, appreciated.
>
> * one test fails for Fake in test_virt_drivers, but only when it's run as
> the full unit test, not when run on it's own. It looks like it has to do
> with FakeConnection.instance() caching, which actually confuses me a bit,
> as I would have assumed one unit test file couldn't affect another (i.e.
> they started a clean env each time).
>
> -Sean
>
> --
> Sean Dague
> IBM Linux Technology Center
> email: sdague [at] linux
> alt-email: sldague [at] us
>
>
> ______________________________**_________________
> Mailing list: https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> More help : https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp>
>


sdague at linux

May 30, 2012, 1:17 PM

Post #8 of 8 (427 views)
Permalink
Re: RFC - dynamically loading virt drivers [In reply to]

On 05/18/2012 02:14 PM, Jay Pipes wrote:
> On 05/17/2012 06:38 PM, Vishvananda Ishaya wrote:
>> On May 17, 2012, at 1:52 PM, Sean Dague wrote:
<snip>
>> So I guess this would be my strategy:
>>
>> a) remove get_connection from the drivers (and just have it construct
>> the 'connection' class directly)
>> b) modify the global get_connection to construct the drivers for
>> backwards compatibilty
>> c) modify the documentation to suggest changing drivers by specifying
>> the full path to the driver instead of connection_type
>> d) rename the connection classes to something reasonable representing
>> drivers (libvirt.driver:LibvirtDriver() vs
>> libvirt.connection.LibvirtConnection)
>> e) bonus points if it could be done with a short path for ease of use
>> (compute_driver=libvirt.LibvirtDriver vs
>> compute_driver=nova.virt.libvirt.driver.LibvirtDriver)

A review which covers a, b, and part of d is up at -
https://review.openstack.org/#/c/7930/. I'd like to get comments on the
work so far, and handle c, the rest of d, and e in a second round once a
version of this lands.

Reviews appreciated.

-Sean

--
Sean Dague
IBM Linux Technology Center
email: sdague [at] linux
alt-email: sldague [at] us


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

OpenStack dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.