Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: OpenStack: Dev

Energy efficiency

 

 

OpenStack dev RSS feed   Index | Next | Previous | View Threaded


semyazz at gmail

Apr 29, 2012, 9:28 AM

Post #1 of 9 (382 views)
Permalink
Energy efficiency

Hi,

Me and my colleague are doing research about openstack and energy
efficiency during part of our master thesis about cloud computing. And
mayby we would like to write something inside nova-scheduler to dynamically
manage vms from cloud administrator's point of view. the general idea is to
automate process of vm migration to suite current policy. For example, we
have 10 servers in cloud with nova-compute, each is capable of running 5
vm. I'd like to run 20 vms. Aaccording to current nova-scheduler (filters),
each server will run 2 VMs, but it would be cheaper (this is policy defined
by administrator) if we run all of them on just 4 servers. of course, cloud
has to keep proper QoS rate (response time etc.). This is general idea.

Energy efficiency is really popular topic, when we talk about
servers,datacenters and virtualization, but I can't find any papers about
it in context of openstack. Are there any projects doing such researches or
articles? in fact it would be really surprising if there is nothing about
energy efficiency in context of openstack.

Cheers,


hitesh.wadekar at gmail

Apr 29, 2012, 9:56 AM

Post #2 of 9 (376 views)
Permalink
Re: Energy efficiency [In reply to]

Hi Szymon,

This is a really great that you are working for energy efficiency
datacenter. whatever the topics you mentioned that really affects for
efficient data center. I would like to add some points on this. You should
consider Quantum project to deploy and research for datacenter. Distributed
networks play an important role in the data center. May be you can consider
OpenVswitch + OpenStack-Quantum+ OpenStack-Nova combination infrastructure.

Here are some news that I came across, Clarkson University is going to
build energy efficient datacenter at near New York. You can see this news
at
http://northcountrynow.com/business/clarkson-takes-partners-develop-efficient-data-centers-050575.
As far as I know that Prof. Jeanna Matthews is working for it.

I am not sure whether they are doing this for OpenStack Cloud. But, You can
ask them what exactly they are doing. You can contact these persons, they
might redirect you to respective person. I am putting them in CC

1. Dr. Todd Deshane,
2. Mr. Patrick wilbur

For Quantum, may be Dan Wendlandt and me can help you to see what parameter
you should consider for distributed network Data Center.

Its really great that somebody working on OpenStack efficient Data Center.

In the mean time, If I find some articles on it then I will forward to you.

Thanks,
Hitesh Wadekar


On Sun, Apr 29, 2012 at 9:58 PM, Szymon Grzybowski <semyazz [at] gmail>wrote:

> Hi,
>
> Me and my colleague are doing research about openstack and energy
> efficiency during part of our master thesis about cloud computing. And
> mayby we would like to write something inside nova-scheduler to dynamically
> manage vms from cloud administrator's point of view. the general idea is to
> automate process of vm migration to suite current policy. For example, we
> have 10 servers in cloud with nova-compute, each is capable of running 5
> vm. I'd like to run 20 vms. Aaccording to current nova-scheduler (filters),
> each server will run 2 VMs, but it would be cheaper (this is policy defined
> by administrator) if we run all of them on just 4 servers. of course, cloud
> has to keep proper QoS rate (response time etc.). This is general idea.
>
> Energy efficiency is really popular topic, when we talk about
> servers,datacenters and virtualization, but I can't find any papers about
> it in context of openstack. Are there any projects doing such researches or
> articles? in fact it would be really surprising if there is nothing about
> energy efficiency in context of openstack.
>
> Cheers,
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>


aquette.dev at gmail

Apr 30, 2012, 12:26 AM

Post #3 of 9 (372 views)
Permalink
Re: Energy efficiency [In reply to]

Hi Simon and Hitesh,

I've started a spec on cloud PM some times ago, that you may be interesting
in reading:
https://blueprints.launchpad.net/ubuntu/+spec/servercloud-p-cloud-power-management

It's incomplete, but most ideas are there.
The base ideas are that:
- you can't just consider power efficiency, without power availability and
cost,
- PUE must also be considered for the whole infrastructure. So it's not
just about optimizing 1 server load, but also which server (WRT for example
UPS efficiency, or its power status, Ie online/on battery),
- using the deployed power protection system maximize visibility and
features, by using the data from the power devices,
- the NUT (networkupstools.org) can provide both this protection and
visibility, by supporting all types of power devices out there (UPS, PDU,
server power supplies, and soon smart meters).

FYI, NUT is also bundled in all Linux, available for all systems, and
embedded in tons of NAS and appliances found in DC...

>From my first discussions with Jay, it seemed that local integration of NUT
in Nova + a new scheduler (that has the global visibility) is the way to go.

I also came to the conclusion that it was worth creating a power QOS API,
that could be supported by any cloud OS, and implemented by internal or 3rd
party solution.

Cheers,
Arnaud
(Sent from eeePad...)
Le 29 avr. 2012 19:00, "hitesh wadekar" <hitesh.wadekar [at] gmail> a écrit :

> Hi Szymon,
>
> This is a really great that you are working for energy efficiency
> datacenter. whatever the topics you mentioned that really affects for
> efficient data center. I would like to add some points on this. You should
> consider Quantum project to deploy and research for datacenter. Distributed
> networks play an important role in the data center. May be you can consider
> OpenVswitch + OpenStack-Quantum+ OpenStack-Nova combination infrastructure.
>
> Here are some news that I came across, Clarkson University is going to
> build energy efficient datacenter at near New York. You can see this news
> at
> http://northcountrynow.com/business/clarkson-takes-partners-develop-efficient-data-centers-050575.
> As far as I know that Prof. Jeanna Matthews is working for it.
>
> I am not sure whether they are doing this for OpenStack Cloud. But, You
> can ask them what exactly they are doing. You can contact these persons,
> they might redirect you to respective person. I am putting them in CC
>
> 1. Dr. Todd Deshane,
> 2. Mr. Patrick wilbur
>
> For Quantum, may be Dan Wendlandt and me can help you to see what
> parameter you should consider for distributed network Data Center.
>
> Its really great that somebody working on OpenStack efficient Data Center.
>
> In the mean time, If I find some articles on it then I will forward to you.
>
> Thanks,
> Hitesh Wadekar
>
>
> On Sun, Apr 29, 2012 at 9:58 PM, Szymon Grzybowski <semyazz [at] gmail>wrote:
>
>> Hi,
>>
>> Me and my colleague are doing research about openstack and energy
>> efficiency during part of our master thesis about cloud computing. And
>> mayby we would like to write something inside nova-scheduler to dynamically
>> manage vms from cloud administrator's point of view. the general idea is to
>> automate process of vm migration to suite current policy. For example, we
>> have 10 servers in cloud with nova-compute, each is capable of running 5
>> vm. I'd like to run 20 vms. Aaccording to current nova-scheduler (filters),
>> each server will run 2 VMs, but it would be cheaper (this is policy defined
>> by administrator) if we run all of them on just 4 servers. of course, cloud
>> has to keep proper QoS rate (response time etc.). This is general idea.
>>
>> Energy efficiency is really popular topic, when we talk about
>> servers,datacenters and virtualization, but I can't find any papers about
>> it in context of openstack. Are there any projects doing such researches or
>> articles? in fact it would be really surprising if there is nothing about
>> energy efficiency in context of openstack.
>>
>> Cheers,
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack [at] lists
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>>
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>


pednape at gmail

Apr 30, 2012, 2:48 AM

Post #4 of 9 (368 views)
Permalink
Re: Energy efficiency [In reply to]

Hi all,

CLUES team in Polytechnic University of Valencia in Spain have done
some work about energy management:

http://www.grycap.upv.es/clues/eng/index.php

Cheers

On Mon, Apr 30, 2012 at 9:26 AM, Arnaud Quette <aquette.dev [at] gmail> wrote:
> Hi Simon and Hitesh,
>
> I've started a spec on cloud PM some times ago, that you may be interesting
> in reading:
> https://blueprints.launchpad.net/ubuntu/+spec/servercloud-p-cloud-power-management
>
> It's incomplete, but most ideas are there.
> The base ideas are that:
> - you can't just consider power efficiency, without power availability and
> cost,
> - PUE must also be considered for the whole infrastructure. So it's not just
> about optimizing 1 server load, but also which server (WRT for example UPS
> efficiency, or its power status, Ie online/on battery),
> - using the deployed power protection system maximize visibility and
> features, by using the data from the power devices,
> - the NUT (networkupstools.org) can provide both this protection and
> visibility, by supporting all types of power devices out there (UPS, PDU,
> server power supplies, and soon smart meters).
>
> FYI, NUT is also bundled in all Linux, available for all systems, and
> embedded in tons of NAS and appliances found in DC...
>
> From my first discussions with Jay, it seemed that local integration of NUT
> in Nova + a new scheduler (that has the global visibility) is the way to go.
>
> I also came to the conclusion that it was worth creating a power QOS API,
> that could be supported by any cloud OS, and implemented by internal or 3rd
> party solution.
>
> Cheers,
> Arnaud
> (Sent from eeePad...)
>
> Le 29 avr. 2012 19:00, "hitesh wadekar" <hitesh.wadekar [at] gmail> a ťcrit†:
>
>> Hi Szymon,
>>
>> This is a really great that you are working for energy efficiency
>> datacenter. whatever the topics you mentioned that really affects for
>> efficient data center. I would like to add some points on this. You should
>> consider Quantum project to deploy and research for datacenter. Distributed
>> networks play an important role in the data center. May be you can consider
>> OpenVswitch + OpenStack-Quantum+ OpenStack-Nova†combination infrastructure.
>>
>> Here are some news that I came across, Clarkson University is going to
>> build energy efficient datacenter at near New York. You can see this news at
>> http://northcountrynow.com/business/clarkson-takes-partners-develop-efficient-data-centers-050575.
>> As far as I know that Prof. Jeanna Matthews is working for it.
>>
>> I am not sure whether they are doing this for OpenStack Cloud. But, You
>> can ask them what exactly they are doing. You can contact these persons,
>> they might redirect you to respective person. I am putting them in CC
>>
>> 1. Dr. Todd Deshane,
>> 2. Mr. Patrick wilbur
>>
>> For Quantum, may be Dan Wendlandt and me can help you to see what
>> parameter you should consider for distributed network Data Center.
>>
>> Its really great that somebody working on OpenStack efficient Data Center.
>>
>> In the mean time, If I find some articles on it then I will forward to
>> you.
>>
>> Thanks,
>> Hitesh Wadekar
>>
>>
>> On Sun, Apr 29, 2012 at 9:58 PM, Szymon Grzybowski <semyazz [at] gmail>
>> wrote:
>>>
>>> Hi,
>>>
>>> Me and my colleague are doing research about openstack and energy
>>> efficiency during part of our master thesis about cloud computing. And mayby
>>> we would like to write something inside nova-scheduler to dynamically manage
>>> vms from cloud administrator's point of view. the general idea is to
>>> automate process of vm migration to suite current policy. For example, we
>>> have 10 servers in cloud with nova-compute, each is capable of running 5 vm.
>>> I'd like to run 20 vms. Aaccording to current nova-scheduler (filters), each
>>> server will run 2 VMs, but it would be cheaper (this is policy defined by
>>> administrator) if we run all of them on just 4 servers. of course, cloud has
>>> to keep proper QoS rate (response time etc.). This is general idea.
>>>
>>> Energy efficiency is really popular topic, when we talk about
>>> servers,datacenters and virtualization, but I can't find any papers about it
>>> in context of openstack. Are there any projects doing such researches or
>>> articles? in fact it would be really surprising if there is nothing about
>>> energy efficiency in context of openstack.
>>>
>>> Cheers,
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to † † : openstack [at] lists
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help † : https://help.launchpad.net/ListHelp
>>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to † † : openstack [at] lists
>> Unsubscribe : https://launchpad.net/~openstack
>> More help † : https://help.launchpad.net/ListHelp
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to † † : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help † : https://help.launchpad.net/ListHelp
>

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


yorik.sar at gmail

May 3, 2012, 1:46 AM

Post #5 of 9 (363 views)
Permalink
Re: Energy efficiency [In reply to]

Just note that since Essex release Nova by default use fill-first cost
function, meaning that nodes with less free RAM will be preferred for
new instances.

Kind regards, Yuriy.


On Sun, Apr 29, 2012 at 8:28 PM, Szymon Grzybowski <semyazz [at] gmail> wrote:
> Hi,
>
> Me and my colleague are doing research about openstack and energy efficiency
> during part of our master thesis about cloud computing. And mayby we would
> like to write something inside nova-scheduler to dynamically manage vms from
> cloud administrator's point of view. the general idea is to automate process
> of vm migration to suite current policy. For example, we have 10 servers in
> cloud with nova-compute, each is capable of running 5 vm. I'd like to run 20
> vms. Aaccording to current nova-scheduler (filters), each server will run 2
> VMs, but it would be cheaper (this is policy defined by administrator) if we
> run all of them on just 4 servers. of course, cloud has to keep proper QoS
> rate (response time etc.). This is general idea.
>
> Energy efficiency is really popular topic, when we talk about
> servers,datacenters and virtualization, but I can't find any papers about it
> in context of openstack. Are there any projects doing such researches or
> articles? in fact it would be really surprising if there is nothing about
> energy efficiency in context of openstack.
>
> Cheers,
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


lorin at nimbisservices

May 3, 2012, 9:05 AM

Post #6 of 9 (369 views)
Permalink
Re: Energy efficiency [In reply to]

Yuriy:


On May 3, 2012, at 4:46 AM, Yuriy Taraday wrote:

> Just note that since Essex release Nova by default use fill-first cost
> function, meaning that nodes with less free RAM will be preferred for
> new instances.
>
> Kind regards, Yuriy.
>

I thought the default behavior in essex was spread-first:

From:

https://github.com/openstack/nova/blob/stable/essex/nova/scheduler/least_cost.py#L41

cfg.FloatOpt('compute_fill_first_cost_fn_weight',
default=-1.0,
help='How much weight to give the fill-first cost function. '
'A negative value will reverse behavior: '
'e.g. spread-first'),
]




Take care,

Lorin
--
Lorin Hochstein
Lead Architect - Cloud Services
Nimbis Services, Inc.
www.nimbisservices.com
Attachments: smime.p7s (4.78 KB)


yorik.sar at gmail

May 3, 2012, 9:15 AM

Post #7 of 9 (365 views)
Permalink
Re: Energy efficiency [In reply to]

Fill-first cost function returns the amount of free RAM. By default it
is negated (multiplied by -1.0), so the less free RAM is better.

I think, this is a bit misguiding, but was changed right before Essex
(see https://bugs.launchpad.net/nova/+bug/965732 ).

Kind regards, Yuriy.


On Thu, May 3, 2012 at 8:05 PM, Lorin Hochstein
<lorin [at] nimbisservices> wrote:
> Yuriy:
>
>
> On May 3, 2012, at 4:46 AM, Yuriy Taraday wrote:
>
> Just note that since Essex release Nova by default use fill-first cost
> function, meaning that nodes with less free RAM will be preferred for
> new instances.
>
> Kind regards, Yuriy.
>
>
> I thought the default behavior in essex was spread-first:
>
> From:
>
> https://github.com/openstack/nova/blob/stable/essex/nova/scheduler/least_cost.py#L41
>
>     cfg.FloatOpt('compute_fill_first_cost_fn_weight',
>              default=-1.0,
>                help='How much weight to give the fill-first cost function. '
>                     'A negative value will reverse behavior: '
>                     'e.g. spread-first'),
>     ]
>
>
>
>
> Take care,
>
> Lorin
> --
> Lorin Hochstein
> Lead Architect - Cloud Services
> Nimbis Services, Inc.
> www.nimbisservices.com
>
>

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


jaypipes at gmail

May 3, 2012, 11:19 AM

Post #8 of 9 (355 views)
Permalink
Re: Energy efficiency [In reply to]

On 05/03/2012 12:05 PM, Lorin Hochstein wrote:
> On May 3, 2012, at 4:46 AM, Yuriy Taraday wrote:
>
>> Just note that since Essex release Nova by default use fill-first cost
>> function, meaning that nodes with less free RAM will be preferred for
>> new instances.
>>
>> Kind regards, Yuriy.
>>
>
> I thought the default behavior in essex was spread-first:
>
> From:
>
> https://github.com/openstack/nova/blob/stable/essex/nova/scheduler/least_cost.py#L41
>
> cfg.FloatOpt('compute_fill_first_cost_fn_weight',
> default=-1.0,
> help='How much weight to give the fill-first cost function. '
> 'A negative value will reverse behavior: '
> 'e.g. spread-first'),
> ]

Hi Lorin!

The behaviour actually changed late in Essex:

https://github.com/openstack/nova/commit/d4d081ad4cb028f9f9d389e0d7d4af3873c94cb6

Best,
-jay

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


yorik.sar at gmail

May 4, 2012, 3:44 AM

Post #9 of 9 (354 views)
Permalink
Re: Energy efficiency [In reply to]

Sorry, my bad.
The more free RAM is better (cost is smaller) thanks to this -1.0. So
the default behavior is spread-first indeed. To make it fill-first,
you can add flag --compute_fill_first_cost_fn_weight=1.0 to scheduler
node.

Kind regards, Yuriy.


On Thu, May 3, 2012 at 8:15 PM, Yuriy Taraday <yorik.sar [at] gmail> wrote:
> Fill-first cost function returns the amount of free RAM. By default it
> is negated (multiplied by -1.0), so the less free RAM is better.
>
> I think, this is a bit misguiding, but was changed right before Essex
> (see https://bugs.launchpad.net/nova/+bug/965732 ).
>
> Kind regards, Yuriy.
>
>
> On Thu, May 3, 2012 at 8:05 PM, Lorin Hochstein
> <lorin [at] nimbisservices> wrote:
>> Yuriy:
>>
>>
>> On May 3, 2012, at 4:46 AM, Yuriy Taraday wrote:
>>
>> Just note that since Essex release Nova by default use fill-first cost
>> function, meaning that nodes with less free RAM will be preferred for
>> new instances.
>>
>> Kind regards, Yuriy.
>>
>>
>> I thought the default behavior in essex was spread-first:
>>
>> From:
>>
>> https://github.com/openstack/nova/blob/stable/essex/nova/scheduler/least_cost.py#L41
>>
>>     cfg.FloatOpt('compute_fill_first_cost_fn_weight',
>>              default=-1.0,
>>                help='How much weight to give the fill-first cost function. '
>>                     'A negative value will reverse behavior: '
>>                     'e.g. spread-first'),
>>     ]
>>
>>
>>
>>
>> Take care,
>>
>> Lorin
>> --
>> Lorin Hochstein
>> Lead Architect - Cloud Services
>> Nimbis Services, Inc.
>> www.nimbisservices.com
>>
>>

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

OpenStack dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.