Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: OpenStack: Dev

[Nova] Instance Type Extra Specs clarifications

  

 
OpenStack dev RSS feed   Index | Next | Previous | View Threaded


patrick.michel.petit at gmail

Aug 24, 2012, 6:12 AM

Post #1 of 21 (303 views)
Permalink
[Nova] Instance Type Extra Specs clarifications
Hi,

Could someone give a practical overview of how configuring and using the
instance type extra specs extension capability introduced in Folsom?

If how extending an instance type is relatively clear.

Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key
<cpu_arch> --value <'s== x86_64'>

The principles of capability advertising is less clearer. Is it assumed
that the key/value pairs are always declared statically as flags in
nova.conf of the compute node, or can they be generated dynamically and if
so, who would that be? And also, are the keys completely free form strings
or strings that are known (reserved) by Nova?

Thanks in advance for clarifying this.

Patrick


dkang at isi

Aug 24, 2012, 8:34 AM

Post #2 of 21 (288 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
Parick,

We are using the feature in Bare-metal machine provisioning.
Some keys are automatically generated by nova-compute.
For example, "hypervisor_type", "hypervisor_version", etc. fields are automatically
put into capabilities by nova-compute (in the case of libvirt).
So, you don't need to specify that.
But, if you want to add custom fields, you should put them into nova.conf file of
the nova-compute node.

Since the new key are put into 'capabilities',
the new key must be different from any other keys in the 'capabilities'.
If that uniqueness is enforced, it can be any string, I believe.

Thanks,
David

----------------------
Dr. Dong-In "David" Kang
Computer Scientist
USC/ISI

----- Original Message -----
> Hi,
>
>
> Could someone give a practical overview of how configuring and using
> the instance type extra specs extension capability introduced in
> Folsom?
>
>
> If how extending an instance type is relatively clear.
>
>
> Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key
> <cpu_arch> --value <'s== x86_64'>
>
>
> The principles of capability advertising is less clearer. Is it
> assumed that the key/value pairs are always declared statically as
> flags in nova.conf of the compute node, or can they be generated
> dynamically and if so, who would that be? And also, are the keys
> completely free form strings or strings that are known (reserved) by
> Nova?
>
>
> Thanks in advance for clarifying this.
>
>
> Patrick
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


donald.d.dugger at intel

Aug 24, 2012, 8:45 AM

Post #3 of 21 (288 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
Patrick-

We've enhanced `nova-manage' to manipulate the `extra_specs' entries, c.f. https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value, You can add an `extra_specs' key/value pair to a flavor with the command:

nova-manage instance_type add_key m1.humongous cpu_type itanium

And delete a key/value pair with the command:

nova-manage instance_type del_key m1.humongous cpu_type

We're in the process of enhancing `python-novaclient' and Horizon with similar capabilities and hope to have them ready for the Folsom release.

Currently, there's no hook to set `extra_specs' through the `nova.conf' file, the mechanism is to dynamically add the `extra_specs' key/values after the administrator has created a flavor.

Currently, the keys are completely free form but there are some issues with that so that should change. Checkout the bug:

https://bugs.launchpad.net/nova/+bug/1039386

Based upon that bug we need to put some sort of scope on the keys to indicate which components a key applies to. I'm in favor of adding a new column to the `extra_specs' table that would explicitly set the scope but an alternative would be to encode the scope into the key itself, something like `TrustedFilter:trust' to indicate that the `trust' key only applies to the `TrustedFilter' scheduler component. Feel free to chime in on the BZ entry on how to specify the scope, once we decide on how to deal with this I'll create a patch to handle it.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786

From: openstack-bounces+donald.d.dugger=intel.com [at] lists [mailto:openstack-bounces+donald.d.dugger=intel.com [at] lists] On Behalf Of Patrick Petit
Sent: Friday, August 24, 2012 7:13 AM
To: openstack [at] lists (openstack [at] lists)
Subject: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hi,

Could someone give a practical overview of how configuring and using the instance type extra specs extension capability introduced in Folsom?

If how extending an instance type is relatively clear.

Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key <cpu_arch> --value <'s== x86_64'>

The principles of capability advertising is less clearer. Is it assumed that the key/value pairs are always declared statically as flags in nova.conf of the compute node, or can they be generated dynamically and if so, who would that be? And also, are the keys completely free form strings or strings that are known (reserved) by Nova?

Thanks in advance for clarifying this.

Patrick


jsuh at isi

Aug 24, 2012, 9:38 AM

Post #4 of 21 (295 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
Patrick,

Once a new item (key and value pair) is added to the capabilities, it can be compared against extra_specs. The extra_specs can be populated in instance_type_extra_specs table. The items in the extra_specs can start with one of the keywords for operations such as ">=" and "s==". For example, if "ngpus: 4" is populated in capability, extra_specs of ">= 2" will choose the host. If the extra_specs is ">= 5", the host will not be chosen. If no keyword is found at the beginning of the extra_specs (with the latest change in upstream code), the given string is directly compared with capability. For example, if "fpu" is given as extra_specs, the capability must be "fpu" to be selected.

If more clarification is needed, please let us know.

Thanks,

Joseph

----- Original Message -----
From: "David Kang" <dkang [at] isi>
To: "Patrick Petit" <patrick.michel.petit [at] gmail>
Cc: "openstack [at] lists (openstack [at] lists)" <openstack [at] lists>
Sent: Friday, August 24, 2012 11:34:11 AM
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications


Parick,

We are using the feature in Bare-metal machine provisioning.
Some keys are automatically generated by nova-compute.
For example, "hypervisor_type", "hypervisor_version", etc. fields are automatically
put into capabilities by nova-compute (in the case of libvirt).
So, you don't need to specify that.
But, if you want to add custom fields, you should put them into nova.conf file of
the nova-compute node.

Since the new key are put into 'capabilities',
the new key must be different from any other keys in the 'capabilities'.
If that uniqueness is enforced, it can be any string, I believe.

Thanks,
David

----------------------
Dr. Dong-In "David" Kang
Computer Scientist
USC/ISI

----- Original Message -----
> Hi,
>
>
> Could someone give a practical overview of how configuring and using
> the instance type extra specs extension capability introduced in
> Folsom?
>
>
> If how extending an instance type is relatively clear.
>
>
> Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key
> <cpu_arch> --value <'s== x86_64'>
>
>
> The principles of capability advertising is less clearer. Is it
> assumed that the key/value pairs are always declared statically as
> flags in nova.conf of the compute node, or can they be generated
> dynamically and if so, who would that be? And also, are the keys
> completely free form strings or strings that are known (reserved) by
> Nova?
>
>
> Thanks in advance for clarifying this.
>
>
> Patrick
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


patrick.michel.petit at gmail

Aug 24, 2012, 4:37 PM

Post #5 of 21 (283 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
Hi Joseph and All,

You are pointing the root cause of my question. The question is about how to specify capabilities for a compute node so that they can be compared with the extra specs. I think how to define extra specs in a flavor is clear enough.

So, some capabilities are standard and are generated dynamically. Others are not known or not captured by the system and so not standard (yet), like the GPUs, and therefore must be specified somehow. Today, the somehow, as I understand things, is to add key/value pairs in nova.conf when/if it is supported.

I wanted to make sure i understand the basic principals.

Now, this in my opinion poses couple problems and/or call for additional questions:

What is the naming convention to add capabilities in nova.conf? I would suppose that any key/value pair cannot be taken for a capability.

How to avoid name clashing with standard capability? At the very least one should have an option to print them out (in nova-manage?). Even a simple written list would be helpful.

But, are we really comfortable with the idea to define static capabilities in nova.conf? that's putting a lot of burden on config management. Also, not standard doesn't imply static.

We can certainly live we that for now, But eventually, i think we'll need some sort of an extension mechanism so that providers can generate whatever capabilities they want using their own plugin? Note that capabilities could be software related too.

What do you think?
Best,
Patrick

Envoyé de mon iPad

Le 24 août 2012 à 18:38, Joseph Suh <jsuh [at] isi> a écrit :

> Patrick,
>
> Once a new item (key and value pair) is added to the capabilities, it can be compared against extra_specs. The extra_specs can be populated in instance_type_extra_specs table. The items in the extra_specs can start with one of the keywords for operations such as ">=" and "s==". For example, if "ngpus: 4" is populated in capability, extra_specs of ">= 2" will choose the host. If the extra_specs is ">= 5", the host will not be chosen. If no keyword is found at the beginning of the extra_specs (with the latest change in upstream code), the given string is directly compared with capability. For example, if "fpu" is given as extra_specs, the capability must be "fpu" to be selected.
>
> If more clarification is needed, please let us know.
>
> Thanks,
>
> Joseph
>
> ----- Original Message -----
> From: "David Kang" <dkang [at] isi>
> To: "Patrick Petit" <patrick.michel.petit [at] gmail>
> Cc: "openstack [at] lists (openstack [at] lists)" <openstack [at] lists>
> Sent: Friday, August 24, 2012 11:34:11 AM
> Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
>
>
> Parick,
>
> We are using the feature in Bare-metal machine provisioning.
> Some keys are automatically generated by nova-compute.
> For example, "hypervisor_type", "hypervisor_version", etc. fields are automatically
> put into capabilities by nova-compute (in the case of libvirt).
> So, you don't need to specify that.
> But, if you want to add custom fields, you should put them into nova.conf file of
> the nova-compute node.
>
> Since the new key are put into 'capabilities',
> the new key must be different from any other keys in the 'capabilities'.
> If that uniqueness is enforced, it can be any string, I believe.
>
> Thanks,
> David
>
> ----------------------
> Dr. Dong-In "David" Kang
> Computer Scientist
> USC/ISI
>
> ----- Original Message -----
>> Hi,
>>
>>
>> Could someone give a practical overview of how configuring and using
>> the instance type extra specs extension capability introduced in
>> Folsom?
>>
>>
>> If how extending an instance type is relatively clear.
>>
>>
>> Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key
>> <cpu_arch> --value <'s== x86_64'>
>>
>>
>> The principles of capability advertising is less clearer. Is it
>> assumed that the key/value pairs are always declared statically as
>> flags in nova.conf of the compute node, or can they be generated
>> dynamically and if so, who would that be? And also, are the keys
>> completely free form strings or strings that are known (reserved) by
>> Nova?
>>
>>
>> Thanks in advance for clarifying this.
>>
>>
>> Patrick
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack [at] lists
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


jsuh at isi

Aug 24, 2012, 6:39 PM

Post #6 of 21 (284 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
Patrick,

That's a good point. I think the issue is already being discussed at https://bugs.launchpad.net/nova/+bug/1039386 as Don Dugger pointed out.

That being said, as answers to some of your questions: yes, any key/value pair can be used and it is user's (in this case, system admin's) responsibility to avoid conflict at this time. The scope we originally thought was pretty much static like the number of GPUs, but there is no reason why it should be static as some features can change dynamically. I'd encourage you to propose a blueprint if you can. We can also consider the feature, but our team needs to discuss it before we can commit to it.

Thanks,

Joseph

----- Original Message -----
From: "Patrick Petit" <patrick.michel.petit [at] gmail>
To: "Joseph Suh" <jsuh [at] isi>
Cc: "openstack [at] lists (openstack [at] lists)" <openstack [at] lists>, "David Kang" <dkang [at] isi>
Sent: Friday, August 24, 2012 7:37:31 PM
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hi Joseph and All,

You are pointing the root cause of my question. The question is about how to specify capabilities for a compute node so that they can be compared with the extra specs. I think how to define extra specs in a flavor is clear enough.

So, some capabilities are standard and are generated dynamically. Others are not known or not captured by the system and so not standard (yet), like the GPUs, and therefore must be specified somehow. Today, the somehow, as I understand things, is to add key/value pairs in nova.conf when/if it is supported.

I wanted to make sure i understand the basic principals.

Now, this in my opinion poses couple problems and/or call for additional questions:

What is the naming convention to add capabilities in nova.conf? I would suppose that any key/value pair cannot be taken for a capability.

How to avoid name clashing with standard capability? At the very least one should have an option to print them out (in nova-manage?). Even a simple written list would be helpful.

But, are we really comfortable with the idea to define static capabilities in nova.conf? that's putting a lot of burden on config management. Also, not standard doesn't imply static.

We can certainly live we that for now, But eventually, i think we'll need some sort of an extension mechanism so that providers can generate whatever capabilities they want using their own plugin? Note that capabilities could be software related too.

What do you think?
Best,
Patrick

Envoyé de mon iPad

Le 24 août 2012 à 18:38, Joseph Suh <jsuh [at] isi> a écrit :

> Patrick,
>
> Once a new item (key and value pair) is added to the capabilities, it can be compared against extra_specs. The extra_specs can be populated in instance_type_extra_specs table. The items in the extra_specs can start with one of the keywords for operations such as ">=" and "s==". For example, if "ngpus: 4" is populated in capability, extra_specs of ">= 2" will choose the host. If the extra_specs is ">= 5", the host will not be chosen. If no keyword is found at the beginning of the extra_specs (with the latest change in upstream code), the given string is directly compared with capability. For example, if "fpu" is given as extra_specs, the capability must be "fpu" to be selected.
>
> If more clarification is needed, please let us know.
>
> Thanks,
>
> Joseph
>
> ----- Original Message -----
> From: "David Kang" <dkang [at] isi>
> To: "Patrick Petit" <patrick.michel.petit [at] gmail>
> Cc: "openstack [at] lists (openstack [at] lists)" <openstack [at] lists>
> Sent: Friday, August 24, 2012 11:34:11 AM
> Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
>
>
> Parick,
>
> We are using the feature in Bare-metal machine provisioning.
> Some keys are automatically generated by nova-compute.
> For example, "hypervisor_type", "hypervisor_version", etc. fields are automatically
> put into capabilities by nova-compute (in the case of libvirt).
> So, you don't need to specify that.
> But, if you want to add custom fields, you should put them into nova.conf file of
> the nova-compute node.
>
> Since the new key are put into 'capabilities',
> the new key must be different from any other keys in the 'capabilities'.
> If that uniqueness is enforced, it can be any string, I believe.
>
> Thanks,
> David
>
> ----------------------
> Dr. Dong-In "David" Kang
> Computer Scientist
> USC/ISI
>
> ----- Original Message -----
>> Hi,
>>
>>
>> Could someone give a practical overview of how configuring and using
>> the instance type extra specs extension capability introduced in
>> Folsom?
>>
>>
>> If how extending an instance type is relatively clear.
>>
>>
>> Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key
>> <cpu_arch> --value <'s== x86_64'>
>>
>>
>> The principles of capability advertising is less clearer. Is it
>> assumed that the key/value pairs are always declared statically as
>> flags in nova.conf of the compute node, or can they be generated
>> dynamically and if so, who would that be? And also, are the keys
>> completely free form strings or strings that are known (reserved) by
>> Nova?
>>
>>
>> Thanks in advance for clarifying this.
>>
>>
>> Patrick
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack [at] lists
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


malini.k.bhandaru at intel

Aug 24, 2012, 7:00 PM

Post #7 of 21 (288 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
There are two entry points
1) the filters
2) the flavors

From a UI perspective it feels safer to let the user select extra spec keys and values from drop-down lists (of pre-defined/registered keys and respective values) to avoid typographic errors
This would then introduce a dependency, with needing to define the keys and values apriori

It is good to keep user interfaces simple and reduce error (simple in this case comes at the expense of implementation effort)

For the namespace, scope, which hopefully should reduce the likelihood of name clashes
A python package type approach would make sense. a.b.c.myKey=myValue


-----Original Message-----
From: openstack-bounces+malini.k.bhandaru=intel.com [at] lists [mailto:openstack-bounces+malini.k.bhandaru=intel.com [at] lists] On Behalf Of Joseph Suh
Sent: Friday, August 24, 2012 6:40 PM
To: Patrick Petit
Cc: openstack [at] lists (openstack [at] lists)
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Patrick,

That's a good point. I think the issue is already being discussed at https://bugs.launchpad.net/nova/+bug/1039386 as Don Dugger pointed out.

That being said, as answers to some of your questions: yes, any key/value pair can be used and it is user's (in this case, system admin's) responsibility to avoid conflict at this time. The scope we originally thought was pretty much static like the number of GPUs, but there is no reason why it should be static as some features can change dynamically. I'd encourage you to propose a blueprint if you can. We can also consider the feature, but our team needs to discuss it before we can commit to it.

Thanks,

Joseph

----- Original Message -----
From: "Patrick Petit" <patrick.michel.petit [at] gmail>
To: "Joseph Suh" <jsuh [at] isi>
Cc: "openstack [at] lists (openstack [at] lists)" <openstack [at] lists>, "David Kang" <dkang [at] isi>
Sent: Friday, August 24, 2012 7:37:31 PM
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hi Joseph and All,

You are pointing the root cause of my question. The question is about how to specify capabilities for a compute node so that they can be compared with the extra specs. I think how to define extra specs in a flavor is clear enough.

So, some capabilities are standard and are generated dynamically. Others are not known or not captured by the system and so not standard (yet), like the GPUs, and therefore must be specified somehow. Today, the somehow, as I understand things, is to add key/value pairs in nova.conf when/if it is supported.

I wanted to make sure i understand the basic principals.

Now, this in my opinion poses couple problems and/or call for additional questions:

What is the naming convention to add capabilities in nova.conf? I would suppose that any key/value pair cannot be taken for a capability.

How to avoid name clashing with standard capability? At the very least one should have an option to print them out (in nova-manage?). Even a simple written list would be helpful.

But, are we really comfortable with the idea to define static capabilities in nova.conf? that's putting a lot of burden on config management. Also, not standard doesn't imply static.

We can certainly live we that for now, But eventually, i think we'll need some sort of an extension mechanism so that providers can generate whatever capabilities they want using their own plugin? Note that capabilities could be software related too.

What do you think?
Best,
Patrick

Envoyé de mon iPad

Le 24 août 2012 à 18:38, Joseph Suh <jsuh [at] isi> a écrit :

> Patrick,
>
> Once a new item (key and value pair) is added to the capabilities, it can be compared against extra_specs. The extra_specs can be populated in instance_type_extra_specs table. The items in the extra_specs can start with one of the keywords for operations such as ">=" and "s==". For example, if "ngpus: 4" is populated in capability, extra_specs of ">= 2" will choose the host. If the extra_specs is ">= 5", the host will not be chosen. If no keyword is found at the beginning of the extra_specs (with the latest change in upstream code), the given string is directly compared with capability. For example, if "fpu" is given as extra_specs, the capability must be "fpu" to be selected.
>
> If more clarification is needed, please let us know.
>
> Thanks,
>
> Joseph
>
> ----- Original Message -----
> From: "David Kang" <dkang [at] isi>
> To: "Patrick Petit" <patrick.michel.petit [at] gmail>
> Cc: "openstack [at] lists (openstack [at] lists)"
> <openstack [at] lists>
> Sent: Friday, August 24, 2012 11:34:11 AM
> Subject: Re: [Openstack] [Nova] Instance Type Extra Specs
> clarifications
>
>
> Parick,
>
> We are using the feature in Bare-metal machine provisioning.
> Some keys are automatically generated by nova-compute.
> For example, "hypervisor_type", "hypervisor_version", etc. fields are
> automatically put into capabilities by nova-compute (in the case of libvirt).
> So, you don't need to specify that.
> But, if you want to add custom fields, you should put them into
> nova.conf file of the nova-compute node.
>
> Since the new key are put into 'capabilities', the new key must be
> different from any other keys in the 'capabilities'.
> If that uniqueness is enforced, it can be any string, I believe.
>
> Thanks,
> David
>
> ----------------------
> Dr. Dong-In "David" Kang
> Computer Scientist
> USC/ISI
>
> ----- Original Message -----
>> Hi,
>>
>>
>> Could someone give a practical overview of how configuring and using
>> the instance type extra specs extension capability introduced in
>> Folsom?
>>
>>
>> If how extending an instance type is relatively clear.
>>
>>
>> Eg.: #nova-manage instance_type set_key --name=<my.instancetype>
>> --key <cpu_arch> --value <'s== x86_64'>
>>
>>
>> The principles of capability advertising is less clearer. Is it
>> assumed that the key/value pairs are always declared statically as
>> flags in nova.conf of the compute node, or can they be generated
>> dynamically and if so, who would that be? And also, are the keys
>> completely free form strings or strings that are known (reserved) by
>> Nova?
>>
>>
>> Thanks in advance for clarifying this.
>>
>>
>> Patrick
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack Post to :
>> openstack [at] lists Unsubscribe :
>> https://launchpad.net/~openstack More help :
>> https://help.launchpad.net/ListHelp
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


vishvananda at gmail

Aug 24, 2012, 7:10 PM

Post #8 of 21 (284 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
Folsom also supports setting key values for things like capabilities via host aggregates. There is a filter[1] that matches the extra specs by exact comparison just like was done for capabilities before the last patch. The new extra specs matching should be added to it. These capabilities can be set dynamically by administrators so it directly supports the use case below.

[1] https://github.com/openstack/nova/blob/master/nova/scheduler/filters/aggregate_instance_extra_specs.py

On Aug 24, 2012, at 6:39 PM, Joseph Suh <jsuh [at] isi> wrote:

> Patrick,
>
> That's a good point. I think the issue is already being discussed at https://bugs.launchpad.net/nova/+bug/1039386 as Don Dugger pointed out.
>
> That being said, as answers to some of your questions: yes, any key/value pair can be used and it is user's (in this case, system admin's) responsibility to avoid conflict at this time. The scope we originally thought was pretty much static like the number of GPUs, but there is no reason why it should be static as some features can change dynamically. I'd encourage you to propose a blueprint if you can. We can also consider the feature, but our team needs to discuss it before we can commit to it.
>
> Thanks,
>
> Joseph
>
> ----- Original Message -----
> From: "Patrick Petit" <patrick.michel.petit [at] gmail>
> To: "Joseph Suh" <jsuh [at] isi>
> Cc: "openstack [at] lists (openstack [at] lists)" <openstack [at] lists>, "David Kang" <dkang [at] isi>
> Sent: Friday, August 24, 2012 7:37:31 PM
> Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
>
> Hi Joseph and All,
>
> You are pointing the root cause of my question. The question is about how to specify capabilities for a compute node so that they can be compared with the extra specs. I think how to define extra specs in a flavor is clear enough.
>
> So, some capabilities are standard and are generated dynamically. Others are not known or not captured by the system and so not standard (yet), like the GPUs, and therefore must be specified somehow. Today, the somehow, as I understand things, is to add key/value pairs in nova.conf when/if it is supported.
>
> I wanted to make sure i understand the basic principals.
>
> Now, this in my opinion poses couple problems and/or call for additional questions:
>
> What is the naming convention to add capabilities in nova.conf? I would suppose that any key/value pair cannot be taken for a capability.
>
> How to avoid name clashing with standard capability? At the very least one should have an option to print them out (in nova-manage?). Even a simple written list would be helpful.
>
> But, are we really comfortable with the idea to define static capabilities in nova.conf? that's putting a lot of burden on config management. Also, not standard doesn't imply static.
>
> We can certainly live we that for now, But eventually, i think we'll need some sort of an extension mechanism so that providers can generate whatever capabilities they want using their own plugin? Note that capabilities could be software related too.
>
> What do you think?
> Best,
> Patrick
>
> Envoyé de mon iPad
>
> Le 24 août 2012 à 18:38, Joseph Suh <jsuh [at] isi> a écrit :
>
>> Patrick,
>>
>> Once a new item (key and value pair) is added to the capabilities, it can be compared against extra_specs. The extra_specs can be populated in instance_type_extra_specs table. The items in the extra_specs can start with one of the keywords for operations such as ">=" and "s==". For example, if "ngpus: 4" is populated in capability, extra_specs of ">= 2" will choose the host. If the extra_specs is ">= 5", the host will not be chosen. If no keyword is found at the beginning of the extra_specs (with the latest change in upstream code), the given string is directly compared with capability. For example, if "fpu" is given as extra_specs, the capability must be "fpu" to be selected.
>>
>> If more clarification is needed, please let us know.
>>
>> Thanks,
>>
>> Joseph
>>
>> ----- Original Message -----
>> From: "David Kang" <dkang [at] isi>
>> To: "Patrick Petit" <patrick.michel.petit [at] gmail>
>> Cc: "openstack [at] lists (openstack [at] lists)" <openstack [at] lists>
>> Sent: Friday, August 24, 2012 11:34:11 AM
>> Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
>>
>>
>> Parick,
>>
>> We are using the feature in Bare-metal machine provisioning.
>> Some keys are automatically generated by nova-compute.
>> For example, "hypervisor_type", "hypervisor_version", etc. fields are automatically
>> put into capabilities by nova-compute (in the case of libvirt).
>> So, you don't need to specify that.
>> But, if you want to add custom fields, you should put them into nova.conf file of
>> the nova-compute node.
>>
>> Since the new key are put into 'capabilities',
>> the new key must be different from any other keys in the 'capabilities'.
>> If that uniqueness is enforced, it can be any string, I believe.
>>
>> Thanks,
>> David
>>
>> ----------------------
>> Dr. Dong-In "David" Kang
>> Computer Scientist
>> USC/ISI
>>
>> ----- Original Message -----
>>> Hi,
>>>
>>>
>>> Could someone give a practical overview of how configuring and using
>>> the instance type extra specs extension capability introduced in
>>> Folsom?
>>>
>>>
>>> If how extending an instance type is relatively clear.
>>>
>>>
>>> Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key
>>> <cpu_arch> --value <'s== x86_64'>
>>>
>>>
>>> The principles of capability advertising is less clearer. Is it
>>> assumed that the key/value pairs are always declared statically as
>>> flags in nova.conf of the compute node, or can they be generated
>>> dynamically and if so, who would that be? And also, are the keys
>>> completely free form strings or strings that are known (reserved) by
>>> Nova?
>>>
>>>
>>> Thanks in advance for clarifying this.
>>>
>>>
>>> Patrick
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack [at] lists
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help : https://help.launchpad.net/ListHelp
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack [at] lists
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


jsuh at isi

Aug 25, 2012, 3:48 AM

Post #9 of 21 (290 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
A bug fix patch that adds the extra specs matching against host aggregate is under review at https://review.openstack.org/#/c/11449/.

Thanks,

Joseph

----- Original Message -----
From: "Vishvananda Ishaya" <vishvananda [at] gmail>
To: "Joseph Suh" <jsuh [at] isi>
Cc: "Patrick Petit" <patrick.michel.petit [at] gmail>, "openstack [at] lists (openstack [at] lists)" <openstack [at] lists>
Sent: Friday, August 24, 2012 10:10:05 PM
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Folsom also supports setting key values for things like capabilities via host aggregates. There is a filter[1] that matches the extra specs by exact comparison just like was done for capabilities before the last patch. The new extra specs matching should be added to it. These capabilities can be set dynamically by administrators so it directly supports the use case below.

[1] https://github.com/openstack/nova/blob/master/nova/scheduler/filters/aggregate_instance_extra_specs.py

On Aug 24, 2012, at 6:39 PM, Joseph Suh <jsuh [at] isi> wrote:

> Patrick,
>
> That's a good point. I think the issue is already being discussed at https://bugs.launchpad.net/nova/+bug/1039386 as Don Dugger pointed out.
>
> That being said, as answers to some of your questions: yes, any key/value pair can be used and it is user's (in this case, system admin's) responsibility to avoid conflict at this time. The scope we originally thought was pretty much static like the number of GPUs, but there is no reason why it should be static as some features can change dynamically. I'd encourage you to propose a blueprint if you can. We can also consider the feature, but our team needs to discuss it before we can commit to it.
>
> Thanks,
>
> Joseph
>
> ----- Original Message -----
> From: "Patrick Petit" <patrick.michel.petit [at] gmail>
> To: "Joseph Suh" <jsuh [at] isi>
> Cc: "openstack [at] lists (openstack [at] lists)" <openstack [at] lists>, "David Kang" <dkang [at] isi>
> Sent: Friday, August 24, 2012 7:37:31 PM
> Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
>
> Hi Joseph and All,
>
> You are pointing the root cause of my question. The question is about how to specify capabilities for a compute node so that they can be compared with the extra specs. I think how to define extra specs in a flavor is clear enough.
>
> So, some capabilities are standard and are generated dynamically. Others are not known or not captured by the system and so not standard (yet), like the GPUs, and therefore must be specified somehow. Today, the somehow, as I understand things, is to add key/value pairs in nova.conf when/if it is supported.
>
> I wanted to make sure i understand the basic principals.
>
> Now, this in my opinion poses couple problems and/or call for additional questions:
>
> What is the naming convention to add capabilities in nova.conf? I would suppose that any key/value pair cannot be taken for a capability.
>
> How to avoid name clashing with standard capability? At the very least one should have an option to print them out (in nova-manage?). Even a simple written list would be helpful.
>
> But, are we really comfortable with the idea to define static capabilities in nova.conf? that's putting a lot of burden on config management. Also, not standard doesn't imply static.
>
> We can certainly live we that for now, But eventually, i think we'll need some sort of an extension mechanism so that providers can generate whatever capabilities they want using their own plugin? Note that capabilities could be software related too.
>
> What do you think?
> Best,
> Patrick
>
> Envoyé de mon iPad
>
> Le 24 août 2012 à 18:38, Joseph Suh <jsuh [at] isi> a écrit :
>
>> Patrick,
>>
>> Once a new item (key and value pair) is added to the capabilities, it can be compared against extra_specs. The extra_specs can be populated in instance_type_extra_specs table. The items in the extra_specs can start with one of the keywords for operations such as ">=" and "s==". For example, if "ngpus: 4" is populated in capability, extra_specs of ">= 2" will choose the host. If the extra_specs is ">= 5", the host will not be chosen. If no keyword is found at the beginning of the extra_specs (with the latest change in upstream code), the given string is directly compared with capability. For example, if "fpu" is given as extra_specs, the capability must be "fpu" to be selected.
>>
>> If more clarification is needed, please let us know.
>>
>> Thanks,
>>
>> Joseph
>>
>> ----- Original Message -----
>> From: "David Kang" <dkang [at] isi>
>> To: "Patrick Petit" <patrick.michel.petit [at] gmail>
>> Cc: "openstack [at] lists (openstack [at] lists)" <openstack [at] lists>
>> Sent: Friday, August 24, 2012 11:34:11 AM
>> Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
>>
>>
>> Parick,
>>
>> We are using the feature in Bare-metal machine provisioning.
>> Some keys are automatically generated by nova-compute.
>> For example, "hypervisor_type", "hypervisor_version", etc. fields are automatically
>> put into capabilities by nova-compute (in the case of libvirt).
>> So, you don't need to specify that.
>> But, if you want to add custom fields, you should put them into nova.conf file of
>> the nova-compute node.
>>
>> Since the new key are put into 'capabilities',
>> the new key must be different from any other keys in the 'capabilities'.
>> If that uniqueness is enforced, it can be any string, I believe.
>>
>> Thanks,
>> David
>>
>> ----------------------
>> Dr. Dong-In "David" Kang
>> Computer Scientist
>> USC/ISI
>>
>> ----- Original Message -----
>>> Hi,
>>>
>>>
>>> Could someone give a practical overview of how configuring and using
>>> the instance type extra specs extension capability introduced in
>>> Folsom?
>>>
>>>
>>> If how extending an instance type is relatively clear.
>>>
>>>
>>> Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key
>>> <cpu_arch> --value <'s== x86_64'>
>>>
>>>
>>> The principles of capability advertising is less clearer. Is it
>>> assumed that the key/value pairs are always declared statically as
>>> flags in nova.conf of the compute node, or can they be generated
>>> dynamically and if so, who would that be? And also, are the keys
>>> completely free form strings or strings that are known (reserved) by
>>> Nova?
>>>
>>>
>>> Thanks in advance for clarifying this.
>>>
>>>
>>> Patrick
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack [at] lists
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help : https://help.launchpad.net/ListHelp
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack [at] lists
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


patrick.michel.petit at gmail

Aug 28, 2012, 8:02 AM

Post #10 of 21 (262 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
Hi Don,

I added a comment in https://bugs.launchpad.net/nova/+bug/1039386 regarding
your point.
Best regards,
Patrick

2012/8/24 Dugger, Donald D <donald.d.dugger [at] intel>

> Patrick-****
>
> ** **
>
> We’ve enhanced `nova-manage’ to manipulate the `extra_specs’ entries, c.f.
> https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value,
> You can add an `extra_specs’ key/value pair to a flavor with the command:
> ****
>
> ** **
>
> nova-manage instance_type add_key m1.humongous cpu_type
> itanium****
>
> ** **
>
> And delete a key/value pair with the command:****
>
> ** **
>
> nova-manage instance_type del_key m1.humongous cpu_type***
> *
>
> ** **
>
> We’re in the process of enhancing `python-novaclient’ and Horizon with
> similar capabilities and hope to have them ready for the Folsom release.**
> **
>
> ** **
>
> Currently, there’s no hook to set `extra_specs’ through the `nova.conf’
> file, the mechanism is to dynamically add the `extra_specs’ key/values
> after the administrator has created a flavor.****
>
> ** **
>
> Currently, the keys are completely free form but there are some issues
> with that so that should change. Checkout the bug:****
>
> ** **
>
> https://bugs.launchpad.net/nova/+bug/1039386****
>
> ** **
>
> Based upon that bug we need to put some sort of scope on the keys to
> indicate which components a key applies to. I’m in favor of adding a new
> column to the `extra_specs’ table that would explicitly set the scope but
> an alternative would be to encode the scope into the key itself, something
> like `TrustedFilter:trust’ to indicate that the `trust’ key only applies
> to the `TrustedFilter’ scheduler component. Feel free to chime in on the
> BZ entry on how to specify the scope, once we decide on how to deal with
> this I’ll create a patch to handle it.****
>
> ** **
>
> --****
>
> Don Dugger****
>
> "Censeo Toto nos in Kansa esse decisse." - D. Gale****
>
> Ph: 303/443-3786****
>
> ** **
>
> *From:* openstack-bounces+donald.d.dugger=intel.com [at] lists[mailto:
> openstack-bounces+donald.d.dugger=intel.com [at] lists] *On
> Behalf Of *Patrick Petit
> *Sent:* Friday, August 24, 2012 7:13 AM
> *To:* openstack [at] lists (openstack [at] lists)
> *Subject:* [Openstack] [Nova] Instance Type Extra Specs clarifications****
>
> ** **
>
> Hi,****
>
> ** **
>
> Could someone give a practical overview of how configuring and using the
> instance type extra specs extension capability introduced in Folsom?****
>
> ** **
>
> If how extending an instance type is relatively clear.****
>
> ** **
>
> Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key
> <cpu_arch> --value <'s== x86_64'> ****
>
> ** **
>
> The principles of capability advertising is less clearer. Is it assumed
> that the key/value pairs are always declared statically as flags in
> nova.conf of the compute node, or can they be generated dynamically and if
> so, who would that be? And also, are the keys completely free form strings
> or strings that are known (reserved) by Nova?****
>
> ** **
>
> Thanks in advance for clarifying this.****
>
> ** **
>
> Patrick****
>



--
*"Give me a place to stand, and I shall move the earth with a lever"*


vbannai at gmail

Sep 7, 2012, 1:20 PM

Post #11 of 21 (240 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
Hello all,

I am part of the SF south bay meetup group and trying to add a Disk I/O QoS
feature which is based on the blkiotune in libvirt.
We would like to add flavor types in which we specify the blkiotune in the
create flavor screen. After reviewing the discussions and some emails it
appears that it makes sense to use the "instance_type_extra_specs" to add
the blkiotune as a key/value pair instead of extending the "instance_type"
table in nova db.

I am able to use nova-manage to create instance type and use "set_key" to
add extra specs. The set_key seems to make a direct call to the db to
insert the keys whereas the instance_type create takes the more traditional
path through the flavomanage controller. However I notice that there is no
documentation on how to add the extra_specs keys using a RESTful api. Is
this something still in discussions?

Thanks
Vinay

On Tue, Aug 28, 2012 at 8:02 AM, Patrick Petit <
patrick.michel.petit [at] gmail> wrote:

> Hi Don,
>
> I added a comment in https://bugs.launchpad.net/nova/+bug/1039386regarding your point.
> Best regards,
> Patrick
>
> 2012/8/24 Dugger, Donald D <donald.d.dugger [at] intel>
>
> Patrick-****
>>
>> ** **
>>
>> We’ve enhanced `nova-manage’ to manipulate the `extra_specs’ entries,
>> c.f. https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value,
>> You can add an `extra_specs’ key/value pair to a flavor with the command:
>> ****
>>
>> ** **
>>
>> nova-manage instance_type add_key m1.humongous cpu_type
>> itanium****
>>
>> ** **
>>
>> And delete a key/value pair with the command:****
>>
>> ** **
>>
>> nova-manage instance_type del_key m1.humongous cpu_type**
>> **
>>
>> ** **
>>
>> We’re in the process of enhancing `python-novaclient’ and Horizon with
>> similar capabilities and hope to have them ready for the Folsom release.*
>> ***
>>
>> ** **
>>
>> Currently, there’s no hook to set `extra_specs’ through the `nova.conf’
>> file, the mechanism is to dynamically add the `extra_specs’ key/values
>> after the administrator has created a flavor.****
>>
>> ** **
>>
>> Currently, the keys are completely free form but there are some issues
>> with that so that should change. Checkout the bug:****
>>
>> ** **
>>
>> https://bugs.launchpad.net/nova/+bug/1039386****
>>
>> ** **
>>
>> Based upon that bug we need to put some sort of scope on the keys to
>> indicate which components a key applies to. I’m in favor of adding a new
>> column to the `extra_specs’ table that would explicitly set the scope but
>> an alternative would be to encode the scope into the key itself, something
>> like `TrustedFilter:trust’ to indicate that the `trust’ key only applies
>> to the `TrustedFilter’ scheduler component. Feel free to chime in on the
>> BZ entry on how to specify the scope, once we decide on how to deal with
>> this I’ll create a patch to handle it.****
>>
>> ** **
>>
>> --****
>>
>> Don Dugger****
>>
>> "Censeo Toto nos in Kansa esse decisse." - D. Gale****
>>
>> Ph: 303/443-3786****
>>
>> ** **
>>
>> *From:* openstack-bounces+donald.d.dugger=intel.com [at] lists[mailto:
>> openstack-bounces+donald.d.dugger=intel.com [at] lists] *On
>> Behalf Of *Patrick Petit
>> *Sent:* Friday, August 24, 2012 7:13 AM
>> *To:* openstack [at] lists (openstack [at] lists)
>> *Subject:* [Openstack] [Nova] Instance Type Extra Specs clarifications***
>> *
>>
>> ** **
>>
>> Hi,****
>>
>> ** **
>>
>> Could someone give a practical overview of how configuring and using the
>> instance type extra specs extension capability introduced in Folsom?****
>>
>> ** **
>>
>> If how extending an instance type is relatively clear.****
>>
>> ** **
>>
>> Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key
>> <cpu_arch> --value <'s== x86_64'> ****
>>
>> ** **
>>
>> The principles of capability advertising is less clearer. Is it assumed
>> that the key/value pairs are always declared statically as flags in
>> nova.conf of the compute node, or can they be generated dynamically and if
>> so, who would that be? And also, are the keys completely free form strings
>> or strings that are known (reserved) by Nova?****
>>
>> ** **
>>
>> Thanks in advance for clarifying this.****
>>
>> ** **
>>
>> Patrick****
>>
>
>
>
> --
> *"Give me a place to stand, and I shall move the earth with a lever"*
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>


--
Vinay Bannai
Email: vbannai [at] gmail
Google Voice: 415 938 7576


donald.d.dugger at intel

Sep 7, 2012, 1:27 PM

Post #12 of 21 (241 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
Well, Yunhong added the API to allow you to update the extra specs table so he should be able to give you the details on that (he's in China, he might not get back to you until next week).

Also, make sure you add a scope (where scope is a string followed by a `:' at the beginning of the key) to whatever key you are adding to the extra specs table, otherwise your key will create problems with some of the scheduler filters.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786

From: Vinay Bannai [mailto:vbannai [at] gmail]
Sent: Friday, September 07, 2012 2:20 PM
To: Patrick Petit
Cc: Dugger, Donald D; openstack [at] lists (openstack [at] lists)
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hello all,

I am part of the SF south bay meetup group and trying to add a Disk I/O QoS feature which is based on the blkiotune in libvirt.
We would like to add flavor types in which we specify the blkiotune in the create flavor screen. After reviewing the discussions and some emails it appears that it makes sense to use the "instance_type_extra_specs" to add the blkiotune as a key/value pair instead of extending the "instance_type" table in nova db.

I am able to use nova-manage to create instance type and use "set_key" to add extra specs. The set_key seems to make a direct call to the db to insert the keys whereas the instance_type create takes the more traditional path through the flavomanage controller. However I notice that there is no documentation on how to add the extra_specs keys using a RESTful api. Is this something still in discussions?

Thanks
Vinay

On Tue, Aug 28, 2012 at 8:02 AM, Patrick Petit <patrick.michel.petit [at] gmail<mailto:patrick.michel.petit [at] gmail>> wrote:
Hi Don,

I added a comment in https://bugs.launchpad.net/nova/+bug/1039386 regarding your point.
Best regards,
Patrick
2012/8/24 Dugger, Donald D <donald.d.dugger [at] intel<mailto:donald.d.dugger [at] intel>>

Patrick-

We've enhanced `nova-manage' to manipulate the `extra_specs' entries, c.f. https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value, You can add an `extra_specs' key/value pair to a flavor with the command:

nova-manage instance_type add_key m1.humongous cpu_type itanium

And delete a key/value pair with the command:

nova-manage instance_type del_key m1.humongous cpu_type

We're in the process of enhancing `python-novaclient' and Horizon with similar capabilities and hope to have them ready for the Folsom release.

Currently, there's no hook to set `extra_specs' through the `nova.conf' file, the mechanism is to dynamically add the `extra_specs' key/values after the administrator has created a flavor.

Currently, the keys are completely free form but there are some issues with that so that should change. Checkout the bug:

https://bugs.launchpad.net/nova/+bug/1039386

Based upon that bug we need to put some sort of scope on the keys to indicate which components a key applies to. I'm in favor of adding a new column to the `extra_specs' table that would explicitly set the scope but an alternative would be to encode the scope into the key itself, something like `TrustedFilter:trust' to indicate that the `trust' key only applies to the `TrustedFilter' scheduler component. Feel free to chime in on the BZ entry on how to specify the scope, once we decide on how to deal with this I'll create a patch to handle it.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786<tel:303%2F443-3786>

From: openstack-bounces+donald.d.dugger=intel.com [at] lists<mailto:intel.com [at] lists> [mailto:openstack-bounces+donald.d.dugger<mailto:openstack-bounces%2Bdonald.d.dugger>=intel.com [at] lists<mailto:intel.com [at] lists>] On Behalf Of Patrick Petit
Sent: Friday, August 24, 2012 7:13 AM
To: openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hi,

Could someone give a practical overview of how configuring and using the instance type extra specs extension capability introduced in Folsom?

If how extending an instance type is relatively clear.

Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key <cpu_arch> --value <'s== x86_64'>

The principles of capability advertising is less clearer. Is it assumed that the key/value pairs are always declared statically as flags in nova.conf of the compute node, or can they be generated dynamically and if so, who would that be? And also, are the keys completely free form strings or strings that are known (reserved) by Nova?

Thanks in advance for clarifying this.

Patrick



--
"Give me a place to stand, and I shall move the earth with a lever"

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists<mailto:openstack [at] lists>
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp



--
Vinay Bannai
Email: vbannai [at] gmail<mailto:vbannai [at] gmail>
Google Voice: 415 938 7576


vbannai at gmail

Sep 7, 2012, 1:40 PM

Post #13 of 21 (240 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
Yes, the intent has been to use something along the lines of
"OS-FLV-EXTRA-DATA:disk_qos"

This leads to another question which I saw being discussed earlier in the
email thread. Are we looking for consistency in adding the extra specs? For
the most obvious ones it may be worth while to come up with a standardized
convention for the names.

I was looking at the code and it appears that the extra_specs and the
instance_types creation don't have similar code flow. When you delete a
instance_type that you created with extra_specs (using the set_key method
in nova-manage), the extra_specs are not cleaned up. I would have thought
that they should go away too, right? Unless I understood the concept of
extra_specs wrong.

Vinay



On Fri, Sep 7, 2012 at 1:27 PM, Dugger, Donald D
<donald.d.dugger [at] intel>wrote:

> Well, Yunhong added the API to allow you to update the extra specs table
> so he should be able to give you the details on that (he’s in China, he
> might not get back to you until next week).****
>
> ** **
>
> Also, make sure you add a scope (where scope is a string followed by a `:’
> at the beginning of the key) to whatever key you are adding to the extra
> specs table, otherwise your key will create problems with some of the
> scheduler filters.****
>
> ** **
>
> --****
>
> Don Dugger****
>
> "Censeo Toto nos in Kansa esse decisse." - D. Gale****
>
> Ph: 303/443-3786****
>
> ** **
>
> *From:* Vinay Bannai [mailto:vbannai [at] gmail]
> *Sent:* Friday, September 07, 2012 2:20 PM
> *To:* Patrick Petit
> *Cc:* Dugger, Donald D; openstack [at] lists (
> openstack [at] lists)
> *Subject:* Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
> ****
>
> ** **
>
> Hello all,****
>
> ** **
>
> I am part of the SF south bay meetup group and trying to add a Disk I/O
> QoS feature which is based on the blkiotune in libvirt. ****
>
> We would like to add flavor types in which we specify the blkiotune in the
> create flavor screen. After reviewing the discussions and some emails it
> appears that it makes sense to use the "instance_type_extra_specs" to add
> the blkiotune as a key/value pair instead of extending the "instance_type"
> table in nova db. ****
>
> ** **
>
> I am able to use nova-manage to create instance type and use "set_key" to
> add extra specs. The set_key seems to make a direct call to the db to
> insert the keys whereas the instance_type create takes the more traditional
> path through the flavomanage controller. However I notice that there is no
> documentation on how to add the extra_specs keys using a RESTful api. Is
> this something still in discussions? ****
>
> ** **
>
> Thanks****
>
> Vinay****
>
> ** **
>
> On Tue, Aug 28, 2012 at 8:02 AM, Patrick Petit <
> patrick.michel.petit [at] gmail> wrote:****
>
> Hi Don,****
>
> ** **
>
> I added a comment in https://bugs.launchpad.net/nova/+bug/1039386regarding your point.
> ****
>
> Best regards,****
>
> Patrick****
>
> 2012/8/24 Dugger, Donald D <donald.d.dugger [at] intel>****
>
> ** **
>
> Patrick-****
>
> ****
>
> We’ve enhanced `nova-manage’ to manipulate the `extra_specs’ entries, c.f.
> https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value,
> You can add an `extra_specs’ key/value pair to a flavor with the command:
> ****
>
> ****
>
> nova-manage instance_type add_key m1.humongous cpu_type
> itanium****
>
> ****
>
> And delete a key/value pair with the command:****
>
> ****
>
> nova-manage instance_type del_key m1.humongous cpu_type***
> *
>
> ****
>
> We’re in the process of enhancing `python-novaclient’ and Horizon with
> similar capabilities and hope to have them ready for the Folsom release.**
> **
>
> ****
>
> Currently, there’s no hook to set `extra_specs’ through the `nova.conf’
> file, the mechanism is to dynamically add the `extra_specs’ key/values
> after the administrator has created a flavor.****
>
> ****
>
> Currently, the keys are completely free form but there are some issues
> with that so that should change. Checkout the bug:****
>
> ****
>
> https://bugs.launchpad.net/nova/+bug/1039386****
>
> ****
>
> Based upon that bug we need to put some sort of scope on the keys to
> indicate which components a key applies to. I’m in favor of adding a new
> column to the `extra_specs’ table that would explicitly set the scope but
> an alternative would be to encode the scope into the key itself, something
> like `TrustedFilter:trust’ to indicate that the `trust’ key only applies
> to the `TrustedFilter’ scheduler component. Feel free to chime in on the
> BZ entry on how to specify the scope, once we decide on how to deal with
> this I’ll create a patch to handle it.****
>
> ****
>
> --****
>
> Don Dugger****
>
> "Censeo Toto nos in Kansa esse decisse." - D. Gale****
>
> Ph: 303/443-3786****
>
> ****
>
> *From:* openstack-bounces+donald.d.dugger=intel.com [at] lists[mailto:
> openstack-bounces+donald.d.dugger=intel.com [at] lists] *On
> Behalf Of *Patrick Petit
> *Sent:* Friday, August 24, 2012 7:13 AM
> *To:* openstack [at] lists (openstack [at] lists)
> *Subject:* [Openstack] [Nova] Instance Type Extra Specs clarifications****
>
> ****
>
> Hi,****
>
> ****
>
> Could someone give a practical overview of how configuring and using the
> instance type extra specs extension capability introduced in Folsom?****
>
> ****
>
> If how extending an instance type is relatively clear.****
>
> ****
>
> Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key
> <cpu_arch> --value <'s== x86_64'> ****
>
> ****
>
> The principles of capability advertising is less clearer. Is it assumed
> that the key/value pairs are always declared statically as flags in
> nova.conf of the compute node, or can they be generated dynamically and if
> so, who would that be? And also, are the keys completely free form strings
> or strings that are known (reserved) by Nova?****
>
> ****
>
> Thanks in advance for clarifying this.****
>
> ****
>
> Patrick****
>
>
>
> ****
>
> ** **
>
> --
> *"Give me a place to stand, and I shall move the earth with a lever"*****
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp****
>
>
>
> ****
>
> ** **
>
> --
> Vinay Bannai
> Email: vbannai [at] gmail
> Google Voice: 415 938 7576****
>



--
Vinay Bannai
Email: vbannai [at] gmail
Google Voice: 415 938 7576


donald.d.dugger at intel

Sep 7, 2012, 2:17 PM

Post #14 of 21 (240 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
OS-FLV-EXTRA-DATA: Do you really need a 17 character scope? Your current scope (I assume it's a derivative of OpenStack Flavor Extra Data) is really just redundant info, we know it's extra data for a flavor since we're talking about the `extra_specs' table which is used for flavors. I would think something like `disk:qos' would be sufficient, informative and succinct.

Naming consistency: I don't see a real problem here. As long as a unique scope is used then the actual key names can be arbitrary. Since they are limited to a specific scope there is no danger of overlap so no need for a naming convention.

Deletion issues: You raise a good point and I think we should address this. When a flavor is deleted any `extra_specs' entries for that flavor should be deleted also. Not a major issue but it would avoid surprises where sale `extra_specs' could be associated with a new flavor that re-used an ID.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786

From: Vinay Bannai [mailto:vbannai [at] gmail]
Sent: Friday, September 07, 2012 2:41 PM
To: Dugger, Donald D
Cc: Patrick Petit; Jiang, Yunhong; openstack [at] lists (openstack [at] lists)
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Yes, the intent has been to use something along the lines of
"OS-FLV-EXTRA-DATA:disk_qos"

This leads to another question which I saw being discussed earlier in the email thread. Are we looking for consistency in adding the extra specs? For the most obvious ones it may be worth while to come up with a standardized convention for the names.

I was looking at the code and it appears that the extra_specs and the instance_types creation don't have similar code flow. When you delete a instance_type that you created with extra_specs (using the set_key method in nova-manage), the extra_specs are not cleaned up. I would have thought that they should go away too, right? Unless I understood the concept of extra_specs wrong.

Vinay


On Fri, Sep 7, 2012 at 1:27 PM, Dugger, Donald D <donald.d.dugger [at] intel<mailto:donald.d.dugger [at] intel>> wrote:
Well, Yunhong added the API to allow you to update the extra specs table so he should be able to give you the details on that (he's in China, he might not get back to you until next week).

Also, make sure you add a scope (where scope is a string followed by a `:' at the beginning of the key) to whatever key you are adding to the extra specs table, otherwise your key will create problems with some of the scheduler filters.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786<tel:303%2F443-3786>

From: Vinay Bannai [mailto:vbannai [at] gmail<mailto:vbannai [at] gmail>]
Sent: Friday, September 07, 2012 2:20 PM
To: Patrick Petit
Cc: Dugger, Donald D; openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hello all,

I am part of the SF south bay meetup group and trying to add a Disk I/O QoS feature which is based on the blkiotune in libvirt.
We would like to add flavor types in which we specify the blkiotune in the create flavor screen. After reviewing the discussions and some emails it appears that it makes sense to use the "instance_type_extra_specs" to add the blkiotune as a key/value pair instead of extending the "instance_type" table in nova db.

I am able to use nova-manage to create instance type and use "set_key" to add extra specs. The set_key seems to make a direct call to the db to insert the keys whereas the instance_type create takes the more traditional path through the flavomanage controller. However I notice that there is no documentation on how to add the extra_specs keys using a RESTful api. Is this something still in discussions?

Thanks
Vinay

On Tue, Aug 28, 2012 at 8:02 AM, Patrick Petit <patrick.michel.petit [at] gmail<mailto:patrick.michel.petit [at] gmail>> wrote:
Hi Don,

I added a comment in https://bugs.launchpad.net/nova/+bug/1039386 regarding your point.
Best regards,
Patrick
2012/8/24 Dugger, Donald D <donald.d.dugger [at] intel<mailto:donald.d.dugger [at] intel>>

Patrick-

We've enhanced `nova-manage' to manipulate the `extra_specs' entries, c.f. https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value, You can add an `extra_specs' key/value pair to a flavor with the command:

nova-manage instance_type add_key m1.humongous cpu_type itanium

And delete a key/value pair with the command:

nova-manage instance_type del_key m1.humongous cpu_type

We're in the process of enhancing `python-novaclient' and Horizon with similar capabilities and hope to have them ready for the Folsom release.

Currently, there's no hook to set `extra_specs' through the `nova.conf' file, the mechanism is to dynamically add the `extra_specs' key/values after the administrator has created a flavor.

Currently, the keys are completely free form but there are some issues with that so that should change. Checkout the bug:

https://bugs.launchpad.net/nova/+bug/1039386

Based upon that bug we need to put some sort of scope on the keys to indicate which components a key applies to. I'm in favor of adding a new column to the `extra_specs' table that would explicitly set the scope but an alternative would be to encode the scope into the key itself, something like `TrustedFilter:trust' to indicate that the `trust' key only applies to the `TrustedFilter' scheduler component. Feel free to chime in on the BZ entry on how to specify the scope, once we decide on how to deal with this I'll create a patch to handle it.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786<tel:303%2F443-3786>

From: openstack-bounces+donald.d.dugger=intel.com [at] lists<mailto:intel.com [at] lists> [mailto:openstack-bounces+donald.d.dugger<mailto:openstack-bounces%2Bdonald.d.dugger>=intel.com [at] lists<mailto:intel.com [at] lists>] On Behalf Of Patrick Petit
Sent: Friday, August 24, 2012 7:13 AM
To: openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hi,

Could someone give a practical overview of how configuring and using the instance type extra specs extension capability introduced in Folsom?

If how extending an instance type is relatively clear.

Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key <cpu_arch> --value <'s== x86_64'>

The principles of capability advertising is less clearer. Is it assumed that the key/value pairs are always declared statically as flags in nova.conf of the compute node, or can they be generated dynamically and if so, who would that be? And also, are the keys completely free form strings or strings that are known (reserved) by Nova?

Thanks in advance for clarifying this.

Patrick



--
"Give me a place to stand, and I shall move the earth with a lever"

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists<mailto:openstack [at] lists>
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp



--
Vinay Bannai
Email: vbannai [at] gmail<mailto:vbannai [at] gmail>
Google Voice: 415 938 7576<tel:415%20938%207576>



--
Vinay Bannai
Email: vbannai [at] gmail<mailto:vbannai [at] gmail>
Google Voice: 415 938 7576


yunhong.jiang at intel

Sep 8, 2012, 6:42 AM

Post #15 of 21 (242 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
The extra_specs do have the RESTful interface as "/flavors/flavor_type/os-extra_specs". You can refer to https://review.openstack.org/#/c/10768/ for some changes. (I didn't update the patch because I'm recently busy on another task, and will restore that patch when I finish current task).

Thanks
--jyh

From: Dugger, Donald D
Sent: Saturday, September 08, 2012 4:28 AM
To: Vinay Bannai; Patrick Petit; Jiang, Yunhong
Cc: openstack [at] lists (openstack [at] lists)
Subject: RE: [Openstack] [Nova] Instance Type Extra Specs clarifications

Well, Yunhong added the API to allow you to update the extra specs table so he should be able to give you the details on that (he's in China, he might not get back to you until next week).

Also, make sure you add a scope (where scope is a string followed by a `:' at the beginning of the key) to whatever key you are adding to the extra specs table, otherwise your key will create problems with some of the scheduler filters.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786

From: Vinay Bannai [mailto:vbannai [at] gmail]<mailto:[mailto:vbannai [at] gmail]>
Sent: Friday, September 07, 2012 2:20 PM
To: Patrick Petit
Cc: Dugger, Donald D; openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hello all,

I am part of the SF south bay meetup group and trying to add a Disk I/O QoS feature which is based on the blkiotune in libvirt.
We would like to add flavor types in which we specify the blkiotune in the create flavor screen. After reviewing the discussions and some emails it appears that it makes sense to use the "instance_type_extra_specs" to add the blkiotune as a key/value pair instead of extending the "instance_type" table in nova db.

I am able to use nova-manage to create instance type and use "set_key" to add extra specs. The set_key seems to make a direct call to the db to insert the keys whereas the instance_type create takes the more traditional path through the flavomanage controller. However I notice that there is no documentation on how to add the extra_specs keys using a RESTful api. Is this something still in discussions?

Thanks
Vinay

On Tue, Aug 28, 2012 at 8:02 AM, Patrick Petit <patrick.michel.petit [at] gmail<mailto:patrick.michel.petit [at] gmail>> wrote:
Hi Don,

I added a comment in https://bugs.launchpad.net/nova/+bug/1039386 regarding your point.
Best regards,
Patrick
2012/8/24 Dugger, Donald D <donald.d.dugger [at] intel<mailto:donald.d.dugger [at] intel>>

Patrick-

We've enhanced `nova-manage' to manipulate the `extra_specs' entries, c.f. https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value, You can add an `extra_specs' key/value pair to a flavor with the command:

nova-manage instance_type add_key m1.humongous cpu_type itanium

And delete a key/value pair with the command:

nova-manage instance_type del_key m1.humongous cpu_type

We're in the process of enhancing `python-novaclient' and Horizon with similar capabilities and hope to have them ready for the Folsom release.

Currently, there's no hook to set `extra_specs' through the `nova.conf' file, the mechanism is to dynamically add the `extra_specs' key/values after the administrator has created a flavor.

Currently, the keys are completely free form but there are some issues with that so that should change. Checkout the bug:

https://bugs.launchpad.net/nova/+bug/1039386

Based upon that bug we need to put some sort of scope on the keys to indicate which components a key applies to. I'm in favor of adding a new column to the `extra_specs' table that would explicitly set the scope but an alternative would be to encode the scope into the key itself, something like `TrustedFilter:trust' to indicate that the `trust' key only applies to the `TrustedFilter' scheduler component. Feel free to chime in on the BZ entry on how to specify the scope, once we decide on how to deal with this I'll create a patch to handle it.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786<tel:303%2F443-3786>

From: openstack-bounces+donald.d.dugger=intel.com [at] lists<mailto:intel.com [at] lists> [mailto:openstack-bounces+donald.d.dugger<mailto:openstack-bounces%2Bdonald.d.dugger>=intel.com [at] lists<mailto:intel.com [at] lists>] On Behalf Of Patrick Petit
Sent: Friday, August 24, 2012 7:13 AM
To: openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hi,

Could someone give a practical overview of how configuring and using the instance type extra specs extension capability introduced in Folsom?

If how extending an instance type is relatively clear.

Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key <cpu_arch> --value <'s== x86_64'>

The principles of capability advertising is less clearer. Is it assumed that the key/value pairs are always declared statically as flags in nova.conf of the compute node, or can they be generated dynamically and if so, who would that be? And also, are the keys completely free form strings or strings that are known (reserved) by Nova?

Thanks in advance for clarifying this.

Patrick



--
"Give me a place to stand, and I shall move the earth with a lever"

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists<mailto:openstack [at] lists>
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp



--
Vinay Bannai
Email: vbannai [at] gmail<mailto:vbannai [at] gmail>
Google Voice: 415 938 7576


yunhong.jiang at intel

Sep 8, 2012, 6:53 AM

Post #16 of 21 (242 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
For the Deletion issue, it should be ok. Followed is the code in nova/db/sqlalchemy/api.py to delete the instance type, which will delete the extra spec table for the corresponding extra_specs also.

Thanks
--jyh

def instance_type_destroy(context, name):
"""Marks specific instance_type as deleted"""
session = get_session()
with session.begin():
instance_type_ref = instance_type_get_by_name(context, name,
session=session)
instance_type_id = instance_type_ref['id']
session.query(models.InstanceTypes).\
filter_by(id=instance_type_id).\
update({'deleted': True,
'deleted_at': timeutils.utcnow(),
'updated_at': literal_column('updated_at')})
session.query(models.InstanceTypeExtraSpecs).\
filter_by(instance_type_id=instance_type_id).\
update({'deleted': True,
'deleted_at': timeutils.utcnow(),
'updated_at': literal_column('updated_at')})

From: Dugger, Donald D
Sent: Saturday, September 08, 2012 5:18 AM
To: Vinay Bannai
Cc: Patrick Petit; Jiang, Yunhong; openstack [at] lists (openstack [at] lists)
Subject: RE: [Openstack] [Nova] Instance Type Extra Specs clarifications

OS-FLV-EXTRA-DATA: Do you really need a 17 character scope? Your current scope (I assume it's a derivative of OpenStack Flavor Extra Data) is really just redundant info, we know it's extra data for a flavor since we're talking about the `extra_specs' table which is used for flavors. I would think something like `disk:qos' would be sufficient, informative and succinct.

Naming consistency: I don't see a real problem here. As long as a unique scope is used then the actual key names can be arbitrary. Since they are limited to a specific scope there is no danger of overlap so no need for a naming convention.

Deletion issues: You raise a good point and I think we should address this. When a flavor is deleted any `extra_specs' entries for that flavor should be deleted also. Not a major issue but it would avoid surprises where sale `extra_specs' could be associated with a new flavor that re-used an ID.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786

From: Vinay Bannai [mailto:vbannai [at] gmail]<mailto:[mailto:vbannai [at] gmail]>
Sent: Friday, September 07, 2012 2:41 PM
To: Dugger, Donald D
Cc: Patrick Petit; Jiang, Yunhong; openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Yes, the intent has been to use something along the lines of
"OS-FLV-EXTRA-DATA:disk_qos"

This leads to another question which I saw being discussed earlier in the email thread. Are we looking for consistency in adding the extra specs? For the most obvious ones it may be worth while to come up with a standardized convention for the names.

I was looking at the code and it appears that the extra_specs and the instance_types creation don't have similar code flow. When you delete a instance_type that you created with extra_specs (using the set_key method in nova-manage), the extra_specs are not cleaned up. I would have thought that they should go away too, right? Unless I understood the concept of extra_specs wrong.

Vinay


On Fri, Sep 7, 2012 at 1:27 PM, Dugger, Donald D <donald.d.dugger [at] intel<mailto:donald.d.dugger [at] intel>> wrote:
Well, Yunhong added the API to allow you to update the extra specs table so he should be able to give you the details on that (he's in China, he might not get back to you until next week).

Also, make sure you add a scope (where scope is a string followed by a `:' at the beginning of the key) to whatever key you are adding to the extra specs table, otherwise your key will create problems with some of the scheduler filters.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786<tel:303%2F443-3786>

From: Vinay Bannai [mailto:vbannai [at] gmail<mailto:vbannai [at] gmail>]
Sent: Friday, September 07, 2012 2:20 PM
To: Patrick Petit
Cc: Dugger, Donald D; openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hello all,

I am part of the SF south bay meetup group and trying to add a Disk I/O QoS feature which is based on the blkiotune in libvirt.
We would like to add flavor types in which we specify the blkiotune in the create flavor screen. After reviewing the discussions and some emails it appears that it makes sense to use the "instance_type_extra_specs" to add the blkiotune as a key/value pair instead of extending the "instance_type" table in nova db.

I am able to use nova-manage to create instance type and use "set_key" to add extra specs. The set_key seems to make a direct call to the db to insert the keys whereas the instance_type create takes the more traditional path through the flavomanage controller. However I notice that there is no documentation on how to add the extra_specs keys using a RESTful api. Is this something still in discussions?

Thanks
Vinay

On Tue, Aug 28, 2012 at 8:02 AM, Patrick Petit <patrick.michel.petit [at] gmail<mailto:patrick.michel.petit [at] gmail>> wrote:
Hi Don,

I added a comment in https://bugs.launchpad.net/nova/+bug/1039386 regarding your point.
Best regards,
Patrick
2012/8/24 Dugger, Donald D <donald.d.dugger [at] intel<mailto:donald.d.dugger [at] intel>>

Patrick-

We've enhanced `nova-manage' to manipulate the `extra_specs' entries, c.f. https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value, You can add an `extra_specs' key/value pair to a flavor with the command:

nova-manage instance_type add_key m1.humongous cpu_type itanium

And delete a key/value pair with the command:

nova-manage instance_type del_key m1.humongous cpu_type

We're in the process of enhancing `python-novaclient' and Horizon with similar capabilities and hope to have them ready for the Folsom release.

Currently, there's no hook to set `extra_specs' through the `nova.conf' file, the mechanism is to dynamically add the `extra_specs' key/values after the administrator has created a flavor.

Currently, the keys are completely free form but there are some issues with that so that should change. Checkout the bug:

https://bugs.launchpad.net/nova/+bug/1039386

Based upon that bug we need to put some sort of scope on the keys to indicate which components a key applies to. I'm in favor of adding a new column to the `extra_specs' table that would explicitly set the scope but an alternative would be to encode the scope into the key itself, something like `TrustedFilter:trust' to indicate that the `trust' key only applies to the `TrustedFilter' scheduler component. Feel free to chime in on the BZ entry on how to specify the scope, once we decide on how to deal with this I'll create a patch to handle it.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786<tel:303%2F443-3786>

From: openstack-bounces+donald.d.dugger=intel.com [at] lists<mailto:intel.com [at] lists> [mailto:openstack-bounces+donald.d.dugger<mailto:openstack-bounces%2Bdonald.d.dugger>=intel.com [at] lists<mailto:intel.com [at] lists>] On Behalf Of Patrick Petit
Sent: Friday, August 24, 2012 7:13 AM
To: openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hi,

Could someone give a practical overview of how configuring and using the instance type extra specs extension capability introduced in Folsom?

If how extending an instance type is relatively clear.

Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key <cpu_arch> --value <'s== x86_64'>

The principles of capability advertising is less clearer. Is it assumed that the key/value pairs are always declared statically as flags in nova.conf of the compute node, or can they be generated dynamically and if so, who would that be? And also, are the keys completely free form strings or strings that are known (reserved) by Nova?

Thanks in advance for clarifying this.

Patrick



--
"Give me a place to stand, and I shall move the earth with a lever"

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists<mailto:openstack [at] lists>
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp



--
Vinay Bannai
Email: vbannai [at] gmail<mailto:vbannai [at] gmail>
Google Voice: 415 938 7576<tel:415%20938%207576>



--
Vinay Bannai
Email: vbannai [at] gmail<mailto:vbannai [at] gmail>
Google Voice: 415 938 7576


vbannai at gmail

Sep 8, 2012, 12:02 PM

Post #17 of 21 (240 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
Thanks Yunhong,

I picked your changes for flavors.py from your review to give it a try in
my setup. I wrote a little python script to create a flavor id and set_key
using the new RESTful apis that you defined. After creating the flavor,
when I try to set the key I am getting error message from the nova side.
Do I need pick to any changes from nova side to make it work? I can send
you my sample code and error message privately or put it on the mailing
list if that helps.

Vinay

On Sat, Sep 8, 2012 at 6:42 AM, Jiang, Yunhong <yunhong.jiang [at] intel>wrote:

> The extra_specs do have the RESTful interface as
> “/flavors/flavor_type/os-extra_specs”. You can refer to
> https://review.openstack.org/#/c/10768/ for some changes. (I didn’t
> update the patch because I’m recently busy on another task, and will
> restore that patch when I finish current task).****
>
> ** **
>
> Thanks****
>
> --jyh****
>
> ** **
>
> *From:* Dugger, Donald D
> *Sent:* Saturday, September 08, 2012 4:28 AM
> *To:* Vinay Bannai; Patrick Petit; Jiang, Yunhong
> *Cc:* openstack [at] lists (openstack [at] lists)
> *Subject:* RE: [Openstack] [Nova] Instance Type Extra Specs clarifications
> ****
>
> ** **
>
> Well, Yunhong added the API to allow you to update the extra specs table
> so he should be able to give you the details on that (he’s in China, he
> might not get back to you until next week).****
>
> ** **
>
> Also, make sure you add a scope (where scope is a string followed by a `:’
> at the beginning of the key) to whatever key you are adding to the extra
> specs table, otherwise your key will create problems with some of the
> scheduler filters.****
>
> ** **
>
> --****
>
> Don Dugger****
>
> "Censeo Toto nos in Kansa esse decisse." - D. Gale****
>
> Ph: 303/443-3786****
>
> ** **
>
> *From:* Vinay Bannai [mailto:vbannai [at] gmail]
> *Sent:* Friday, September 07, 2012 2:20 PM
> *To:* Patrick Petit
> *Cc:* Dugger, Donald D; openstack [at] lists (
> openstack [at] lists)
> *Subject:* Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
> ****
>
> ** **
>
> Hello all,****
>
> ** **
>
> I am part of the SF south bay meetup group and trying to add a Disk I/O
> QoS feature which is based on the blkiotune in libvirt. ****
>
> We would like to add flavor types in which we specify the blkiotune in the
> create flavor screen. After reviewing the discussions and some emails it
> appears that it makes sense to use the "instance_type_extra_specs" to add
> the blkiotune as a key/value pair instead of extending the "instance_type"
> table in nova db. ****
>
> ** **
>
> I am able to use nova-manage to create instance type and use "set_key" to
> add extra specs. The set_key seems to make a direct call to the db to
> insert the keys whereas the instance_type create takes the more traditional
> path through the flavomanage controller. However I notice that there is no
> documentation on how to add the extra_specs keys using a RESTful api. Is
> this something still in discussions? ****
>
> ** **
>
> Thanks****
>
> Vinay****
>
> ** **
>
> On Tue, Aug 28, 2012 at 8:02 AM, Patrick Petit <
> patrick.michel.petit [at] gmail> wrote:****
>
> Hi Don,****
>
> ** **
>
> I added a comment in https://bugs.launchpad.net/nova/+bug/1039386regarding your point.
> ****
>
> Best regards,****
>
> Patrick****
>
> 2012/8/24 Dugger, Donald D <donald.d.dugger [at] intel>****
>
> ** **
>
> Patrick-****
>
> ****
>
> We’ve enhanced `nova-manage’ to manipulate the `extra_specs’ entries, c.f.
> https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value,
> You can add an `extra_specs’ key/value pair to a flavor with the command:
> ****
>
> ****
>
> nova-manage instance_type add_key m1.humongous cpu_type
> itanium****
>
> ****
>
> And delete a key/value pair with the command:****
>
> ****
>
> nova-manage instance_type del_key m1.humongous cpu_type***
> *
>
> ****
>
> We’re in the process of enhancing `python-novaclient’ and Horizon with
> similar capabilities and hope to have them ready for the Folsom release.**
> **
>
> ****
>
> Currently, there’s no hook to set `extra_specs’ through the `nova.conf’
> file, the mechanism is to dynamically add the `extra_specs’ key/values
> after the administrator has created a flavor.****
>
> ****
>
> Currently, the keys are completely free form but there are some issues
> with that so that should change. Checkout the bug:****
>
> ****
>
> https://bugs.launchpad.net/nova/+bug/1039386****
>
> ****
>
> Based upon that bug we need to put some sort of scope on the keys to
> indicate which components a key applies to. I’m in favor of adding a new
> column to the `extra_specs’ table that would explicitly set the scope but
> an alternative would be to encode the scope into the key itself, something
> like `TrustedFilter:trust’ to indicate that the `trust’ key only applies
> to the `TrustedFilter’ scheduler component. Feel free to chime in on the
> BZ entry on how to specify the scope, once we decide on how to deal with
> this I’ll create a patch to handle it.****
>
> ****
>
> --****
>
> Don Dugger****
>
> "Censeo Toto nos in Kansa esse decisse." - D. Gale****
>
> Ph: 303/443-3786****
>
> ****
>
> *From:* openstack-bounces+donald.d.dugger=intel.com [at] lists[mailto:
> openstack-bounces+donald.d.dugger=intel.com [at] lists] *On
> Behalf Of *Patrick Petit
> *Sent:* Friday, August 24, 2012 7:13 AM
> *To:* openstack [at] lists (openstack [at] lists)
> *Subject:* [Openstack] [Nova] Instance Type Extra Specs clarifications****
>
> ****
>
> Hi,****
>
> ****
>
> Could someone give a practical overview of how configuring and using the
> instance type extra specs extension capability introduced in Folsom?****
>
> ****
>
> If how extending an instance type is relatively clear.****
>
> ****
>
> Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key
> <cpu_arch> --value <'s== x86_64'> ****
>
> ****
>
> The principles of capability advertising is less clearer. Is it assumed
> that the key/value pairs are always declared statically as flags in
> nova.conf of the compute node, or can they be generated dynamically and if
> so, who would that be? And also, are the keys completely free form strings
> or strings that are known (reserved) by Nova?****
>
> ****
>
> Thanks in advance for clarifying this.****
>
> ****
>
> Patrick****
>
>
>
> ****
>
> ** **
>
> --
> *"Give me a place to stand, and I shall move the earth with a lever"*****
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp****
>
>
>
> ****
>
> ** **
>
> --
> Vinay Bannai
> Email: vbannai [at] gmail
> Google Voice: 415 938 7576****
>



--
Vinay Bannai
Email: vbannai [at] gmail
Google Voice: 415 938 7576


vbannai at gmail

Sep 8, 2012, 3:26 PM

Post #18 of 21 (240 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
Sorry Yunhong, it was my mistake. I was able to get it to work with your
changes.
I am a little confused by the terminology of "instance_id" and "flavor_id"
in the tables. Either their meaning is crossed or I am not reading it
right.

Vinay

On Sat, Sep 8, 2012 at 12:02 PM, Vinay Bannai <vbannai [at] gmail> wrote:

> Thanks Yunhong,
>
> I picked your changes for flavors.py from your review to give it a try in
> my setup. I wrote a little python script to create a flavor id and set_key
> using the new RESTful apis that you defined. After creating the flavor,
> when I try to set the key I am getting error message from the nova side.
> Do I need pick to any changes from nova side to make it work? I can send
> you my sample code and error message privately or put it on the mailing
> list if that helps.
>
> Vinay
>
>
> On Sat, Sep 8, 2012 at 6:42 AM, Jiang, Yunhong <yunhong.jiang [at] intel>wrote:
>
>> The extra_specs do have the RESTful interface as
>> “/flavors/flavor_type/os-extra_specs”. You can refer to
>> https://review.openstack.org/#/c/10768/ for some changes. (I didn’t
>> update the patch because I’m recently busy on another task, and will
>> restore that patch when I finish current task).****
>>
>> ** **
>>
>> Thanks****
>>
>> --jyh****
>>
>> ** **
>>
>> *From:* Dugger, Donald D
>> *Sent:* Saturday, September 08, 2012 4:28 AM
>> *To:* Vinay Bannai; Patrick Petit; Jiang, Yunhong
>> *Cc:* openstack [at] lists (openstack [at] lists)
>> *Subject:* RE: [Openstack] [Nova] Instance Type Extra Specs
>> clarifications****
>>
>> ** **
>>
>> Well, Yunhong added the API to allow you to update the extra specs table
>> so he should be able to give you the details on that (he’s in China, he
>> might not get back to you until next week).****
>>
>> ** **
>>
>> Also, make sure you add a scope (where scope is a string followed by a
>> `:’ at the beginning of the key) to whatever key you are adding to the
>> extra specs table, otherwise your key will create problems with some of the
>> scheduler filters.****
>>
>> ** **
>>
>> --****
>>
>> Don Dugger****
>>
>> "Censeo Toto nos in Kansa esse decisse." - D. Gale****
>>
>> Ph: 303/443-3786****
>>
>> ** **
>>
>> *From:* Vinay Bannai [mailto:vbannai [at] gmail]
>> *Sent:* Friday, September 07, 2012 2:20 PM
>> *To:* Patrick Petit
>> *Cc:* Dugger, Donald D; openstack [at] lists (
>> openstack [at] lists)
>> *Subject:* Re: [Openstack] [Nova] Instance Type Extra Specs
>> clarifications****
>>
>> ** **
>>
>> Hello all,****
>>
>> ** **
>>
>> I am part of the SF south bay meetup group and trying to add a Disk I/O
>> QoS feature which is based on the blkiotune in libvirt. ****
>>
>> We would like to add flavor types in which we specify the blkiotune in
>> the create flavor screen. After reviewing the discussions and some emails
>> it appears that it makes sense to use the "instance_type_extra_specs" to
>> add the blkiotune as a key/value pair instead of extending the
>> "instance_type" table in nova db. ****
>>
>> ** **
>>
>> I am able to use nova-manage to create instance type and use "set_key" to
>> add extra specs. The set_key seems to make a direct call to the db to
>> insert the keys whereas the instance_type create takes the more traditional
>> path through the flavomanage controller. However I notice that there is no
>> documentation on how to add the extra_specs keys using a RESTful api. Is
>> this something still in discussions? ****
>>
>> ** **
>>
>> Thanks****
>>
>> Vinay****
>>
>> ** **
>>
>> On Tue, Aug 28, 2012 at 8:02 AM, Patrick Petit <
>> patrick.michel.petit [at] gmail> wrote:****
>>
>> Hi Don,****
>>
>> ** **
>>
>> I added a comment in https://bugs.launchpad.net/nova/+bug/1039386regarding your point.
>> ****
>>
>> Best regards,****
>>
>> Patrick****
>>
>> 2012/8/24 Dugger, Donald D <donald.d.dugger [at] intel>****
>>
>> ** **
>>
>> Patrick-****
>>
>> ****
>>
>> We’ve enhanced `nova-manage’ to manipulate the `extra_specs’ entries,
>> c.f. https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value,
>> You can add an `extra_specs’ key/value pair to a flavor with the command:
>> ****
>>
>> ****
>>
>> nova-manage instance_type add_key m1.humongous cpu_type
>> itanium****
>>
>> ****
>>
>> And delete a key/value pair with the command:****
>>
>> ****
>>
>> nova-manage instance_type del_key m1.humongous cpu_type**
>> **
>>
>> ****
>>
>> We’re in the process of enhancing `python-novaclient’ and Horizon with
>> similar capabilities and hope to have them ready for the Folsom release.*
>> ***
>>
>> ****
>>
>> Currently, there’s no hook to set `extra_specs’ through the `nova.conf’
>> file, the mechanism is to dynamically add the `extra_specs’ key/values
>> after the administrator has created a flavor.****
>>
>> ****
>>
>> Currently, the keys are completely free form but there are some issues
>> with that so that should change. Checkout the bug:****
>>
>> ****
>>
>> https://bugs.launchpad.net/nova/+bug/1039386****
>>
>> ****
>>
>> Based upon that bug we need to put some sort of scope on the keys to
>> indicate which components a key applies to. I’m in favor of adding a new
>> column to the `extra_specs’ table that would explicitly set the scope but
>> an alternative would be to encode the scope into the key itself, something
>> like `TrustedFilter:trust’ to indicate that the `trust’ key only applies
>> to the `TrustedFilter’ scheduler component. Feel free to chime in on the
>> BZ entry on how to specify the scope, once we decide on how to deal with
>> this I’ll create a patch to handle it.****
>>
>> ****
>>
>> --****
>>
>> Don Dugger****
>>
>> "Censeo Toto nos in Kansa esse decisse." - D. Gale****
>>
>> Ph: 303/443-3786****
>>
>> ****
>>
>> *From:* openstack-bounces+donald.d.dugger=intel.com [at] lists[mailto:
>> openstack-bounces+donald.d.dugger=intel.com [at] lists] *On
>> Behalf Of *Patrick Petit
>> *Sent:* Friday, August 24, 2012 7:13 AM
>> *To:* openstack [at] lists (openstack [at] lists)
>> *Subject:* [Openstack] [Nova] Instance Type Extra Specs clarifications***
>> *
>>
>> ****
>>
>> Hi,****
>>
>> ****
>>
>> Could someone give a practical overview of how configuring and using the
>> instance type extra specs extension capability introduced in Folsom?****
>>
>> ****
>>
>> If how extending an instance type is relatively clear.****
>>
>> ****
>>
>> Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key
>> <cpu_arch> --value <'s== x86_64'> ****
>>
>> ****
>>
>> The principles of capability advertising is less clearer. Is it assumed
>> that the key/value pairs are always declared statically as flags in
>> nova.conf of the compute node, or can they be generated dynamically and if
>> so, who would that be? And also, are the keys completely free form strings
>> or strings that are known (reserved) by Nova?****
>>
>> ****
>>
>> Thanks in advance for clarifying this.****
>>
>> ****
>>
>> Patrick****
>>
>>
>>
>> ****
>>
>> ** **
>>
>> --
>> *"Give me a place to stand, and I shall move the earth with a lever"*****
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack [at] lists
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp****
>>
>>
>>
>> ****
>>
>> ** **
>>
>> --
>> Vinay Bannai
>> Email: vbannai [at] gmail
>> Google Voice: 415 938 7576****
>>
>
>
>
> --
> Vinay Bannai
> Email: vbannai [at] gmail
> Google Voice: 415 938 7576
>
>


--
Vinay Bannai
Email: vbannai [at] gmail
Google Voice: 415 938 7576


yunhong.jiang at intel

Sep 9, 2012, 7:31 AM

Post #19 of 21 (241 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
Glad that the patch works for you.

For the instance_id and flavor_id, I'm not sure which table are you talking about. But I think flavor_id is same as "instance_type_id", while instance_id is purely an id for an instance, which is usually created based on an instance_type (or, flavor).

Thanks
--jyh

From: Vinay Bannai [mailto:vbannai [at] gmail]
Sent: Sunday, September 09, 2012 6:26 AM
To: Jiang, Yunhong
Cc: Dugger, Donald D; Patrick Petit; openstack [at] lists (openstack [at] lists)
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Sorry Yunhong, it was my mistake. I was able to get it to work with your changes.
I am a little confused by the terminology of "instance_id" and "flavor_id" in the tables. Either their meaning is crossed or I am not reading it right.

Vinay

On Sat, Sep 8, 2012 at 12:02 PM, Vinay Bannai <vbannai [at] gmail<mailto:vbannai [at] gmail>> wrote:
Thanks Yunhong,

I picked your changes for flavors.py from your review to give it a try in my setup. I wrote a little python script to create a flavor id and set_key using the new RESTful apis that you defined. After creating the flavor, when I try to set the key I am getting error message from the nova side.
Do I need pick to any changes from nova side to make it work? I can send you my sample code and error message privately or put it on the mailing list if that helps.

Vinay

On Sat, Sep 8, 2012 at 6:42 AM, Jiang, Yunhong <yunhong.jiang [at] intel<mailto:yunhong.jiang [at] intel>> wrote:
The extra_specs do have the RESTful interface as "/flavors/flavor_type/os-extra_specs". You can refer to https://review.openstack.org/#/c/10768/ for some changes. (I didn't update the patch because I'm recently busy on another task, and will restore that patch when I finish current task).

Thanks
--jyh

From: Dugger, Donald D
Sent: Saturday, September 08, 2012 4:28 AM
To: Vinay Bannai; Patrick Petit; Jiang, Yunhong
Cc: openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: RE: [Openstack] [Nova] Instance Type Extra Specs clarifications

Well, Yunhong added the API to allow you to update the extra specs table so he should be able to give you the details on that (he's in China, he might not get back to you until next week).

Also, make sure you add a scope (where scope is a string followed by a `:' at the beginning of the key) to whatever key you are adding to the extra specs table, otherwise your key will create problems with some of the scheduler filters.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786<tel:303%2F443-3786>

From: Vinay Bannai [mailto:vbannai [at] gmail]<mailto:[mailto:vbannai [at] gmail]>
Sent: Friday, September 07, 2012 2:20 PM
To: Patrick Petit
Cc: Dugger, Donald D; openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hello all,

I am part of the SF south bay meetup group and trying to add a Disk I/O QoS feature which is based on the blkiotune in libvirt.
We would like to add flavor types in which we specify the blkiotune in the create flavor screen. After reviewing the discussions and some emails it appears that it makes sense to use the "instance_type_extra_specs" to add the blkiotune as a key/value pair instead of extending the "instance_type" table in nova db.

I am able to use nova-manage to create instance type and use "set_key" to add extra specs. The set_key seems to make a direct call to the db to insert the keys whereas the instance_type create takes the more traditional path through the flavomanage controller. However I notice that there is no documentation on how to add the extra_specs keys using a RESTful api. Is this something still in discussions?

Thanks
Vinay

On Tue, Aug 28, 2012 at 8:02 AM, Patrick Petit <patrick.michel.petit [at] gmail<mailto:patrick.michel.petit [at] gmail>> wrote:
Hi Don,

I added a comment in https://bugs.launchpad.net/nova/+bug/1039386 regarding your point.
Best regards,
Patrick
2012/8/24 Dugger, Donald D <donald.d.dugger [at] intel<mailto:donald.d.dugger [at] intel>>

Patrick-

We've enhanced `nova-manage' to manipulate the `extra_specs' entries, c.f. https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value, You can add an `extra_specs' key/value pair to a flavor with the command:

nova-manage instance_type add_key m1.humongous cpu_type itanium

And delete a key/value pair with the command:

nova-manage instance_type del_key m1.humongous cpu_type

We're in the process of enhancing `python-novaclient' and Horizon with similar capabilities and hope to have them ready for the Folsom release.

Currently, there's no hook to set `extra_specs' through the `nova.conf' file, the mechanism is to dynamically add the `extra_specs' key/values after the administrator has created a flavor.

Currently, the keys are completely free form but there are some issues with that so that should change. Checkout the bug:

https://bugs.launchpad.net/nova/+bug/1039386

Based upon that bug we need to put some sort of scope on the keys to indicate which components a key applies to. I'm in favor of adding a new column to the `extra_specs' table that would explicitly set the scope but an alternative would be to encode the scope into the key itself, something like `TrustedFilter:trust' to indicate that the `trust' key only applies to the `TrustedFilter' scheduler component. Feel free to chime in on the BZ entry on how to specify the scope, once we decide on how to deal with this I'll create a patch to handle it.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786<tel:303%2F443-3786>

From: openstack-bounces+donald.d.dugger=intel.com [at] lists<mailto:intel.com [at] lists> [mailto:openstack-bounces+donald.d.dugger<mailto:openstack-bounces%2Bdonald.d.dugger>=intel.com [at] lists<mailto:intel.com [at] lists>] On Behalf Of Patrick Petit
Sent: Friday, August 24, 2012 7:13 AM
To: openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hi,

Could someone give a practical overview of how configuring and using the instance type extra specs extension capability introduced in Folsom?

If how extending an instance type is relatively clear.

Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key <cpu_arch> --value <'s== x86_64'>

The principles of capability advertising is less clearer. Is it assumed that the key/value pairs are always declared statically as flags in nova.conf of the compute node, or can they be generated dynamically and if so, who would that be? And also, are the keys completely free form strings or strings that are known (reserved) by Nova?

Thanks in advance for clarifying this.

Patrick



--
"Give me a place to stand, and I shall move the earth with a lever"

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists<mailto:openstack [at] lists>
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp



--
Vinay Bannai
Email: vbannai [at] gmail<mailto:vbannai [at] gmail>
Google Voice: 415 938 7576<tel:415%20938%207576>



--
Vinay Bannai
Email: vbannai [at] gmail<mailto:vbannai [at] gmail>
Google Voice: 415 938 7576<tel:415%20938%207576>



--
Vinay Bannai
Email: vbannai [at] gmail<mailto:vbannai [at] gmail>
Google Voice: 415 938 7576


vbannai at gmail

Sep 9, 2012, 3:56 PM

Post #20 of 21 (238 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
I am talking about the "instance_type_extra_specs" table. The instance_id
and the flavor_id seem a little confusing. I saw an email thread from you
from couple of weeks ago and you seem raise the same issue.

On Sun, Sep 9, 2012 at 7:31 AM, Jiang, Yunhong <yunhong.jiang [at] intel>wrote:

> Glad that the patch works for you.****
>
> ** **
>
> For the instance_id and flavor_id, I’m not sure which table are you
> talking about. But I think flavor_id is same as “instance_type_id”, while
> instance_id is purely an id for an instance, which is usually created based
> on an instance_type (or, flavor).****
>
> ** **
>
> Thanks****
>
> --jyh****
>
> ** **
>
> *From:* Vinay Bannai [mailto:vbannai [at] gmail]
> *Sent:* Sunday, September 09, 2012 6:26 AM
> *To:* Jiang, Yunhong
> *Cc:* Dugger, Donald D; Patrick Petit; openstack [at] lists (
> openstack [at] lists)
>
> *Subject:* Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
> ****
>
> ** **
>
> Sorry Yunhong, it was my mistake. I was able to get it to work with your
> changes. ****
>
> I am a little confused by the terminology of "instance_id" and "flavor_id"
> in the tables. Either their meaning is crossed or I am not reading it
> right. ****
>
> ** **
>
> Vinay
> ****
>
> On Sat, Sep 8, 2012 at 12:02 PM, Vinay Bannai <vbannai [at] gmail> wrote:**
> **
>
> Thanks Yunhong,****
>
> ** **
>
> I picked your changes for flavors.py from your review to give it a try in
> my setup. I wrote a little python script to create a flavor id and set_key
> using the new RESTful apis that you defined. After creating the flavor,
> when I try to set the key I am getting error message from the nova side. *
> ***
>
> Do I need pick to any changes from nova side to make it work? I can send
> you my sample code and error message privately or put it on the mailing
> list if that helps. ****
>
> ** **
>
> Vinay****
>
> ** **
>
> On Sat, Sep 8, 2012 at 6:42 AM, Jiang, Yunhong <yunhong.jiang [at] intel>
> wrote:****
>
> The extra_specs do have the RESTful interface as
> “/flavors/flavor_type/os-extra_specs”. You can refer to
> https://review.openstack.org/#/c/10768/ for some changes. (I didn’t
> update the patch because I’m recently busy on another task, and will
> restore that patch when I finish current task).****
>
> ****
>
> Thanks****
>
> --jyh****
>
> ****
>
> *From:* Dugger, Donald D
> *Sent:* Saturday, September 08, 2012 4:28 AM
> *To:* Vinay Bannai; Patrick Petit; Jiang, Yunhong
> *Cc:* openstack [at] lists (openstack [at] lists)
> *Subject:* RE: [Openstack] [Nova] Instance Type Extra Specs clarifications
> ****
>
> ****
>
> Well, Yunhong added the API to allow you to update the extra specs table
> so he should be able to give you the details on that (he’s in China, he
> might not get back to you until next week).****
>
> ****
>
> Also, make sure you add a scope (where scope is a string followed by a `:’
> at the beginning of the key) to whatever key you are adding to the extra
> specs table, otherwise your key will create problems with some of the
> scheduler filters.****
>
> ****
>
> --****
>
> Don Dugger****
>
> "Censeo Toto nos in Kansa esse decisse." - D. Gale****
>
> Ph: 303/443-3786****
>
> ****
>
> *From:* Vinay Bannai [mailto:vbannai [at] gmail]
> *Sent:* Friday, September 07, 2012 2:20 PM
> *To:* Patrick Petit
> *Cc:* Dugger, Donald D; openstack [at] lists (
> openstack [at] lists)
> *Subject:* Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
> ****
>
> ****
>
> Hello all,****
>
> ****
>
> I am part of the SF south bay meetup group and trying to add a Disk I/O
> QoS feature which is based on the blkiotune in libvirt. ****
>
> We would like to add flavor types in which we specify the blkiotune in the
> create flavor screen. After reviewing the discussions and some emails it
> appears that it makes sense to use the "instance_type_extra_specs" to add
> the blkiotune as a key/value pair instead of extending the "instance_type"
> table in nova db. ****
>
> ****
>
> I am able to use nova-manage to create instance type and use "set_key" to
> add extra specs. The set_key seems to make a direct call to the db to
> insert the keys whereas the instance_type create takes the more traditional
> path through the flavomanage controller. However I notice that there is no
> documentation on how to add the extra_specs keys using a RESTful api. Is
> this something still in discussions? ****
>
> ****
>
> Thanks****
>
> Vinay****
>
> ****
>
> On Tue, Aug 28, 2012 at 8:02 AM, Patrick Petit <
> patrick.michel.petit [at] gmail> wrote:****
>
> Hi Don,****
>
> ****
>
> I added a comment in https://bugs.launchpad.net/nova/+bug/1039386regarding your point.
> ****
>
> Best regards,****
>
> Patrick****
>
> 2012/8/24 Dugger, Donald D <donald.d.dugger [at] intel>****
>
> ****
>
> Patrick-****
>
> ****
>
> We’ve enhanced `nova-manage’ to manipulate the `extra_specs’ entries, c.f.
> https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value,
> You can add an `extra_specs’ key/value pair to a flavor with the command:
> ****
>
> ****
>
> nova-manage instance_type add_key m1.humongous cpu_type
> itanium****
>
> ****
>
> And delete a key/value pair with the command:****
>
> ****
>
> nova-manage instance_type del_key m1.humongous cpu_type***
> *
>
> ****
>
> We’re in the process of enhancing `python-novaclient’ and Horizon with
> similar capabilities and hope to have them ready for the Folsom release.**
> **
>
> ****
>
> Currently, there’s no hook to set `extra_specs’ through the `nova.conf’
> file, the mechanism is to dynamically add the `extra_specs’ key/values
> after the administrator has created a flavor.****
>
> ****
>
> Currently, the keys are completely free form but there are some issues
> with that so that should change. Checkout the bug:****
>
> ****
>
> https://bugs.launchpad.net/nova/+bug/1039386****
>
> ****
>
> Based upon that bug we need to put some sort of scope on the keys to
> indicate which components a key applies to. I’m in favor of adding a new
> column to the `extra_specs’ table that would explicitly set the scope but
> an alternative would be to encode the scope into the key itself, something
> like `TrustedFilter:trust’ to indicate that the `trust’ key only applies
> to the `TrustedFilter’ scheduler component. Feel free to chime in on the
> BZ entry on how to specify the scope, once we decide on how to deal with
> this I’ll create a patch to handle it.****
>
> ****
>
> --****
>
> Don Dugger****
>
> "Censeo Toto nos in Kansa esse decisse." - D. Gale****
>
> Ph: 303/443-3786****
>
> ****
>
> *From:* openstack-bounces+donald.d.dugger=intel.com [at] lists[mailto:
> openstack-bounces+donald.d.dugger=intel.com [at] lists] *On
> Behalf Of *Patrick Petit
> *Sent:* Friday, August 24, 2012 7:13 AM
> *To:* openstack [at] lists (openstack [at] lists)
> *Subject:* [Openstack] [Nova] Instance Type Extra Specs clarifications****
>
> ****
>
> Hi,****
>
> ****
>
> Could someone give a practical overview of how configuring and using the
> instance type extra specs extension capability introduced in Folsom?****
>
> ****
>
> If how extending an instance type is relatively clear.****
>
> ****
>
> Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key
> <cpu_arch> --value <'s== x86_64'> ****
>
> ****
>
> The principles of capability advertising is less clearer. Is it assumed
> that the key/value pairs are always declared statically as flags in
> nova.conf of the compute node, or can they be generated dynamically and if
> so, who would that be? And also, are the keys completely free form strings
> or strings that are known (reserved) by Nova?****
>
> ****
>
> Thanks in advance for clarifying this.****
>
> ****
>
> Patrick****
>
>
>
> ****
>
> ****
>
> --
> *"Give me a place to stand, and I shall move the earth with a lever"*****
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp****
>
>
>
> ****
>
> ****
>
> --
> Vinay Bannai
> Email: vbannai [at] gmail
> Google Voice: 415 938 7576****
>
>
>
> ****
>
> ** **
>
> --
> Vinay Bannai
> Email: vbannai [at] gmail
> Google Voice: 415 938 7576****
>
>
>
> ****
>
> ** **
>
> --
> Vinay Bannai
> Email: vbannai [at] gmail
> Google Voice: 415 938 7576****
>



--
Vinay Bannai
Email: vbannai [at] gmail
Google Voice: 415 938 7576


yunhong.jiang at intel

Sep 9, 2012, 5:21 PM

Post #21 of 21 (241 views)
Permalink
Re: [Nova] Instance Type Extra Specs clarifications [In reply to]
According to followed code, seems there is no instance_id?

class InstanceTypeExtraSpecs(BASE, NovaBase):
"""Represents additional specs as key/value pairs for an instance_type"""
__tablename__ = 'instance_type_extra_specs'
id = Column(Integer, primary_key=True)
key = Column(String(255))
value = Column(String(255))
instance_type_id = Column(Integer, ForeignKey('instance_types.id'),
nullable=False)
instance_type = relationship(InstanceTypes, backref="extra_specs",
foreign_keys=instance_type_id,
primaryjoin='and_('
'InstanceTypeExtraSpecs.instance_type_id == InstanceTypes.id,'
'InstanceTypeExtraSpecs.deleted == False)')

Thanks
--jyh

From: Vinay Bannai [mailto:vbannai [at] gmail]
Sent: Monday, September 10, 2012 6:56 AM
To: Jiang, Yunhong
Cc: Dugger, Donald D; Patrick Petit; openstack [at] lists (openstack [at] lists)
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

I am talking about the "instance_type_extra_specs" table. The instance_id and the flavor_id seem a little confusing. I saw an email thread from you from couple of weeks ago and you seem raise the same issue.
On Sun, Sep 9, 2012 at 7:31 AM, Jiang, Yunhong <yunhong.jiang [at] intel<mailto:yunhong.jiang [at] intel>> wrote:
Glad that the patch works for you.

For the instance_id and flavor_id, I'm not sure which table are you talking about. But I think flavor_id is same as "instance_type_id", while instance_id is purely an id for an instance, which is usually created based on an instance_type (or, flavor).

Thanks
--jyh

From: Vinay Bannai [mailto:vbannai [at] gmail<mailto:vbannai [at] gmail>]
Sent: Sunday, September 09, 2012 6:26 AM
To: Jiang, Yunhong
Cc: Dugger, Donald D; Patrick Petit; openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)

Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Sorry Yunhong, it was my mistake. I was able to get it to work with your changes.
I am a little confused by the terminology of "instance_id" and "flavor_id" in the tables. Either their meaning is crossed or I am not reading it right.

Vinay

On Sat, Sep 8, 2012 at 12:02 PM, Vinay Bannai <vbannai [at] gmail<mailto:vbannai [at] gmail>> wrote:
Thanks Yunhong,

I picked your changes for flavors.py from your review to give it a try in my setup. I wrote a little python script to create a flavor id and set_key using the new RESTful apis that you defined. After creating the flavor, when I try to set the key I am getting error message from the nova side.
Do I need pick to any changes from nova side to make it work? I can send you my sample code and error message privately or put it on the mailing list if that helps.

Vinay

On Sat, Sep 8, 2012 at 6:42 AM, Jiang, Yunhong <yunhong.jiang [at] intel<mailto:yunhong.jiang [at] intel>> wrote:
The extra_specs do have the RESTful interface as "/flavors/flavor_type/os-extra_specs". You can refer to https://review.openstack.org/#/c/10768/ for some changes. (I didn't update the patch because I'm recently busy on another task, and will restore that patch when I finish current task).

Thanks
--jyh

From: Dugger, Donald D
Sent: Saturday, September 08, 2012 4:28 AM
To: Vinay Bannai; Patrick Petit; Jiang, Yunhong
Cc: openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: RE: [Openstack] [Nova] Instance Type Extra Specs clarifications

Well, Yunhong added the API to allow you to update the extra specs table so he should be able to give you the details on that (he's in China, he might not get back to you until next week).

Also, make sure you add a scope (where scope is a string followed by a `:' at the beginning of the key) to whatever key you are adding to the extra specs table, otherwise your key will create problems with some of the scheduler filters.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786<tel:303%2F443-3786>

From: Vinay Bannai [mailto:vbannai [at] gmail]<mailto:[mailto:vbannai [at] gmail]>
Sent: Friday, September 07, 2012 2:20 PM
To: Patrick Petit
Cc: Dugger, Donald D; openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hello all,

I am part of the SF south bay meetup group and trying to add a Disk I/O QoS feature which is based on the blkiotune in libvirt.
We would like to add flavor types in which we specify the blkiotune in the create flavor screen. After reviewing the discussions and some emails it appears that it makes sense to use the "instance_type_extra_specs" to add the blkiotune as a key/value pair instead of extending the "instance_type" table in nova db.

I am able to use nova-manage to create instance type and use "set_key" to add extra specs. The set_key seems to make a direct call to the db to insert the keys whereas the instance_type create takes the more traditional path through the flavomanage controller. However I notice that there is no documentation on how to add the extra_specs keys using a RESTful api. Is this something still in discussions?

Thanks
Vinay

On Tue, Aug 28, 2012 at 8:02 AM, Patrick Petit <patrick.michel.petit [at] gmail<mailto:patrick.michel.petit [at] gmail>> wrote:
Hi Don,

I added a comment in https://bugs.launchpad.net/nova/+bug/1039386 regarding your point.
Best regards,
Patrick
2012/8/24 Dugger, Donald D <donald.d.dugger [at] intel<mailto:donald.d.dugger [at] intel>>

Patrick-

We've enhanced `nova-manage' to manipulate the `extra_specs' entries, c.f. https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value, You can add an `extra_specs' key/value pair to a flavor with the command:

nova-manage instance_type add_key m1.humongous cpu_type itanium

And delete a key/value pair with the command:

nova-manage instance_type del_key m1.humongous cpu_type

We're in the process of enhancing `python-novaclient' and Horizon with similar capabilities and hope to have them ready for the Folsom release.

Currently, there's no hook to set `extra_specs' through the `nova.conf' file, the mechanism is to dynamically add the `extra_specs' key/values after the administrator has created a flavor.

Currently, the keys are completely free form but there are some issues with that so that should change. Checkout the bug:

https://bugs.launchpad.net/nova/+bug/1039386

Based upon that bug we need to put some sort of scope on the keys to indicate which components a key applies to. I'm in favor of adding a new column to the `extra_specs' table that would explicitly set the scope but an alternative would be to encode the scope into the key itself, something like `TrustedFilter:trust' to indicate that the `trust' key only applies to the `TrustedFilter' scheduler component. Feel free to chime in on the BZ entry on how to specify the scope, once we decide on how to deal with this I'll create a patch to handle it.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786<tel:303%2F443-3786>

From: openstack-bounces+donald.d.dugger=intel.com [at] lists<mailto:intel.com [at] lists> [mailto:openstack-bounces+donald.d.dugger<mailto:openstack-bounces%2Bdonald.d.dugger>=intel.com [at] lists<mailto:intel.com [at] lists>] On Behalf Of Patrick Petit
Sent: Friday, August 24, 2012 7:13 AM
To: openstack [at] lists<mailto:openstack [at] lists> (openstack [at] lists<mailto:openstack [at] lists>)
Subject: [Openstack] [Nova] Instance Type Extra Specs clarifications

Hi,

Could someone give a practical overview of how configuring and using the instance type extra specs extension capability introduced in Folsom?

If how extending an instance type is relatively clear.

Eg.: #nova-manage instance_type set_key --name=<my.instancetype> --key <cpu_arch> --value <'s== x86_64'>

The principles of capability advertising is less clearer. Is it assumed that the key/value pairs are always declared statically as flags in nova.conf of the compute node, or can they be generated dynamically and if so, who would that be? And also, are the keys completely free form strings or strings that are known (reserved) by Nova?

Thanks in advance for clarifying this.

Patrick



--
"Give me a place to stand, and I shall move the earth with a lever"

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists<mailto:openstack [at] lists>
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp



--
Vinay Bannai
Email: vbannai [at] gmail<mailto:vbannai [at] gmail>
Google Voice: 415 938 7576<tel:415%20938%207576>



--
Vinay Bannai
Email: vbannai [at] gmail<mailto:vbannai [at] gmail>
Google Voice: 415 938 7576<tel:415%20938%207576>



--
Vinay Bannai
Email: vbannai [at] gmail<mailto:vbannai [at] gmail>
Google Voice: 415 938 7576<tel:415%20938%207576>



--
Vinay Bannai
Email: vbannai [at] gmail<mailto:vbannai [at] gmail>
Google Voice: 415 938 7576

OpenStack dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.