Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Netapp: toasters

De-dup'ing Primary Storage

 

 

Netapp toasters RSS feed   Index | Next | Previous | View Threaded


hollandwl at gmail

Feb 13, 2008, 3:50 AM

Post #1 of 13 (428 views)
Permalink
De-dup'ing Primary Storage

To those of you that have implemented Nearstore and A-SIS on your primary
storage:

1. Have you seen any difference in overall filer performance?
2. If you have LUNs, how are your space savings on those volumes?

I know that enabling Nearstore does some system tweaking in the background
to increase the number of concurrent backup streams that can be running, but
I don't know what else it tweaks that may adversely affect performance of a
primary storage system. Afterall, it was originally designed to run as a
secondary storage platform.


Jochen.Willeke at wincor-nixdorf

Feb 13, 2008, 5:00 AM

Post #2 of 13 (418 views)
Permalink
RE: De-dup'ing Primary Storage [In reply to]

Hi Bill,

we have ASIS on two of our 6070's for some time. We did only few tests
but could not see any direct impact on the filers performance.
We have not yet used it for LUNs so far, but would be interessted in
experience with ASIS on LUNs as well.

Regards

Jochen

________________________________

From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Bill Holland
Sent: Wednesday, February 13, 2008 12:50 PM
To: toasters[at]mathworks.com
Subject: De-dup'ing Primary Storage


To those of you that have implemented Nearstore and A-SIS on your
primary storage:

1. Have you seen any difference in overall filer performance?
2. If you have LUNs, how are your space savings on those volumes?

I know that enabling Nearstore does some system tweaking in the
background to increase the number of concurrent backup streams that can
be running, but I don't know what else it tweaks that may adversely
affect performance of a primary storage system. Afterall, it was
originally designed to run as a secondary storage platform.

--
Wincor Nixdorf International GmbH
Sitz der Gesellschaft: Paderborn
Registergericht Paderborn HRB 3507
Geschäftsführer: Eckard Heidloff (Vorsitzender), Stefan Auerbach, Dr. Jürgen Wunram
Vorsitzender des Aufsichtsrats: Karl-Heinz Stiller
Steuernummer: 339/5884/0020 - Ust-ID Nr.: DE812927716 - WEEE-Reg.-Nr. DE44477193

Diese E-Mail enthält vertrauliche Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet.

This e-mail may contain confidential information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden.


gdekhayser at voyantinc

Feb 13, 2008, 6:19 AM

Post #3 of 13 (416 views)
Permalink
RE: De-dup'ing Primary Storage [In reply to]

I have a client de-duping ~15 iSCSI LUNs in the same volume (vmware c:
drives), getting great data reduction, with no perceivable performance
impact.



-Glenn (the other one)



From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Willeke, Jochen
Sent: Wednesday, February 13, 2008 8:01 AM
To: Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage



Hi Bill,



we have ASIS on two of our 6070's for some time. We did only few tests
but could not see any direct impact on the filers performance.

We have not yet used it for LUNs so far, but would be interessted in
experience with ASIS on LUNs as well.



Regards



Jochen



________________________________

From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Bill Holland
Sent: Wednesday, February 13, 2008 12:50 PM
To: toasters[at]mathworks.com
Subject: De-dup'ing Primary Storage

To those of you that have implemented Nearstore and A-SIS on your
primary storage:



1. Have you seen any difference in overall filer performance?

2. If you have LUNs, how are your space savings on those volumes?



I know that enabling Nearstore does some system tweaking in the
background to increase the number of concurrent backup streams that can
be running, but I don't know what else it tweaks that may adversely
affect performance of a primary storage system. Afterall, it was
originally designed to run as a secondary storage platform.

________________________________

Wincor Nixdorf International GmbH
Sitz der Gesellschaft: Paderborn
Registergericht Paderborn HRB 3507
Geschftsfhrer: Eckard Heidloff (Vorsitzender), Stefan Auerbach, Dr.
Jrgen Wunram
Vorsitzender des Aufsichtsrats: Karl-Heinz Stiller
Steuernummer: 339/5884/0020 - Ust-ID Nr.: DE812927716 - WEEE-Reg.-Nr.
DE44477193

Diese E-Mail enthlt vertrauliche Informationen. Wenn Sie nicht der
richtige Adressat sind oder diese E-Mail irrtmlich erhalten haben,
informieren Sie bitte sofort den Absender und vernichten Sie diese
E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser
E-Mail ist nicht gestattet.

This e-mail may contain confidential information. If you are not the
intended recipient (or have received this e-mail in error) please notify
the sender immediately and destroy this e-mail. Any unauthorised
copying, disclosure or distribution of the material in this e-mail is
strictly forbidden.


gdekhayser at voyantinc

Feb 13, 2008, 6:29 AM

Post #4 of 13 (417 views)
Permalink
RE: De-dup'ing Primary Storage [In reply to]

I would be interested in knowing the impact of a-sis on cache hit %
though. Since you're essentially changing pointers to other instances
of blocks, those de-duped blocks wouldn't be located along with the
other original blocks, so it could affect the performance of onTap's
read-ahead algorithm.



From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Willeke, Jochen
Sent: Wednesday, February 13, 2008 8:01 AM
To: Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage



Hi Bill,



we have ASIS on two of our 6070's for some time. We did only few tests
but could not see any direct impact on the filers performance.

We have not yet used it for LUNs so far, but would be interessted in
experience with ASIS on LUNs as well.



Regards



Jochen



________________________________

From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Bill Holland
Sent: Wednesday, February 13, 2008 12:50 PM
To: toasters[at]mathworks.com
Subject: De-dup'ing Primary Storage

To those of you that have implemented Nearstore and A-SIS on your
primary storage:



1. Have you seen any difference in overall filer performance?

2. If you have LUNs, how are your space savings on those volumes?



I know that enabling Nearstore does some system tweaking in the
background to increase the number of concurrent backup streams that can
be running, but I don't know what else it tweaks that may adversely
affect performance of a primary storage system. Afterall, it was
originally designed to run as a secondary storage platform.

________________________________

Wincor Nixdorf International GmbH
Sitz der Gesellschaft: Paderborn
Registergericht Paderborn HRB 3507
Geschftsfhrer: Eckard Heidloff (Vorsitzender), Stefan Auerbach, Dr.
Jrgen Wunram
Vorsitzender des Aufsichtsrats: Karl-Heinz Stiller
Steuernummer: 339/5884/0020 - Ust-ID Nr.: DE812927716 - WEEE-Reg.-Nr.
DE44477193

Diese E-Mail enthlt vertrauliche Informationen. Wenn Sie nicht der
richtige Adressat sind oder diese E-Mail irrtmlich erhalten haben,
informieren Sie bitte sofort den Absender und vernichten Sie diese
E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser
E-Mail ist nicht gestattet.

This e-mail may contain confidential information. If you are not the
intended recipient (or have received this e-mail in error) please notify
the sender immediately and destroy this e-mail. Any unauthorised
copying, disclosure or distribution of the material in this e-mail is
strictly forbidden.


ggwalker at mindspring

Feb 13, 2008, 7:42 AM

Post #5 of 13 (416 views)
Permalink
RE: De-dup'ing Primary Storage [In reply to]

We've been doing some VMWare testing with FCP LUNs and A-SIS.



We saw a reduction from 471GB to 21GB with only about a 7% reduction in
performance. More than a fair trade-off in my opinion.



Our testing could have had impact on the performance more than the
de-dupe, however.



Glenn



________________________________

From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Bill Holland
Sent: Wednesday, February 13, 2008 6:50 AM
To: toasters[at]mathworks.com
Subject: De-dup'ing Primary Storage



To those of you that have implemented Nearstore and A-SIS on your
primary storage:



1. Have you seen any difference in overall filer performance?

2. If you have LUNs, how are your space savings on those volumes?



I know that enabling Nearstore does some system tweaking in the
background to increase the number of concurrent backup streams that can
be running, but I don't know what else it tweaks that may adversely
affect performance of a primary storage system. Afterall, it was
originally designed to run as a secondary storage platform.


daniel.keisling at austin

Feb 13, 2008, 9:36 AM

Post #6 of 13 (414 views)
Permalink
RE: De-dup'ing Primary Storage [In reply to]

Thanks for the stats, I'll be de-duping VMWare data soon too.

My NetApp storage tech says that read cache peformance will increase
since you're reducing the total number of actual blocks. I have not
heard of any performance degradations with A-SIS, other than filer
overhead (CPU) when SIS is actually de-duplicating. My A-SIS schedules
are during off-peak hours so it's not a concern of mine.

Daniel

________________________________

From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Glenn Walker
Sent: Wednesday, February 13, 2008 9:42 AM
To: Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage



We've been doing some VMWare testing with FCP LUNs and A-SIS.



We saw a reduction from 471GB to 21GB with only about a 7% reduction in
performance. More than a fair trade-off in my opinion.



Our testing could have had impact on the performance more than the
de-dupe, however.



Glenn



________________________________

From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Bill Holland
Sent: Wednesday, February 13, 2008 6:50 AM
To: toasters[at]mathworks.com
Subject: De-dup'ing Primary Storage



To those of you that have implemented Nearstore and A-SIS on your
primary storage:



1. Have you seen any difference in overall filer performance?

2. If you have LUNs, how are your space savings on those volumes?



I know that enabling Nearstore does some system tweaking in the
background to increase the number of concurrent backup streams that can
be running, but I don't know what else it tweaks that may adversely
affect performance of a primary storage system. Afterall, it was
originally designed to run as a secondary storage platform.


______________________________________________________________________
This email transmission and any documents, files or previous email
messages attached to it may contain information that is confidential or
legally privileged. If you are not the intended recipient or a person
responsible for delivering this transmission to the intended recipient,
you are hereby notified that you must not read this transmission and
that any disclosure, copying, printing, distribution or use of this
transmission is strictly prohibited. If you have received this transmission
in error, please immediately notify the sender by telephone or return email
and delete the original transmission and its attachments without reading
or saving in any manner.


ggwalker at mindspring

Feb 13, 2008, 9:49 AM

Post #7 of 13 (413 views)
Permalink
RE: De-dup'ing Primary Storage [In reply to]

Just keep in mind that we were using IOMeter to test, not actual real
world workloads - we were also very much disk bound in our testing. If
you are neither using IOMeter (ie, a real-world workload) nor disk
bound, I can't foresee it being a huge problem.



We'll be testing with NFS dedupe a bit later today and I'll gladly share
that info if you want. Same number of disks, so same stipulations
exist.



And because the question did come up, the VMWare server guy didn't build
the VMDKs with thin provisioning, so it may have also impacted the
testing.



Glenn



________________________________

From: Daniel Keisling [mailto:daniel.keisling[at]austin.ppdi.com]
Sent: Wednesday, February 13, 2008 12:37 PM
To: Glenn Walker; Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage



Thanks for the stats, I'll be de-duping VMWare data soon too.



My NetApp storage tech says that read cache peformance will increase
since you're reducing the total number of actual blocks. I have not
heard of any performance degradations with A-SIS, other than filer
overhead (CPU) when SIS is actually de-duplicating. My A-SIS schedules
are during off-peak hours so it's not a concern of mine.


Daniel



________________________________

From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Glenn Walker
Sent: Wednesday, February 13, 2008 9:42 AM
To: Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage

We've been doing some VMWare testing with FCP LUNs and A-SIS.



We saw a reduction from 471GB to 21GB with only about a 7% reduction in
performance. More than a fair trade-off in my opinion.



Our testing could have had impact on the performance more than the
de-dupe, however.



Glenn



________________________________

From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Bill Holland
Sent: Wednesday, February 13, 2008 6:50 AM
To: toasters[at]mathworks.com
Subject: De-dup'ing Primary Storage



To those of you that have implemented Nearstore and A-SIS on your
primary storage:



1. Have you seen any difference in overall filer performance?

2. If you have LUNs, how are your space savings on those volumes?



I know that enabling Nearstore does some system tweaking in the
background to increase the number of concurrent backup streams that can
be running, but I don't know what else it tweaks that may adversely
affect performance of a primary storage system. Afterall, it was
originally designed to run as a secondary storage platform.





______________________________________________________________________
This email transmission and any documents, files or previous email
messages attached to it may contain information that is confidential or
legally privileged. If you are not the intended recipient or a person
responsible for delivering this transmission to the intended recipient,
you are hereby notified that you must not read this transmission and
that any disclosure, copying, printing, distribution or use of this
transmission is strictly prohibited. If you have received this
transmission
in error, please immediately notify the sender by telephone or return
email
and delete the original transmission and its attachments without reading
or saving in any manner.


jeff.mery at ni

Feb 13, 2008, 11:48 AM

Post #8 of 13 (412 views)
Permalink
RE: De-dup'ing Primary Storage [In reply to]

Like the others, thanks for the info. At one point, A-SIS wasn't
supported in true "production" roles, meaning in a Tier-1 use case. Has
that changed recently? If so, I think I need to make some phone calls =).

On the thin provisioning - ESX doesn't support thin-provisioned VMDK's
yet. Workstation does that by default, but I'm not sure about VMware
Server. Hopefully ESX is coming soon though. I would still imagine a
fair amount of reclaimed space once ESX does come through. While you'll
no longer de-dupe the white space in a VMDK, I'd imagine you'll still
de-dupe all the base OS commonalities.

Jeff Mery - MCSE, MCP
National Instruments

-------------------------------------------------------------------------
"Allow me to extol the virtues of the Net Fairy, and of all the fantastic
dorks that make the nice packets go from here to there. Amen."
TB - Penny Arcade
-------------------------------------------------------------------------



From:
"Glenn Walker" <ggwalker[at]mindspring.com>
To:
"Daniel Keisling" <daniel.keisling[at]austin.ppdi.com>, "Bill Holland"
<hollandwl[at]gmail.com>, <toasters[at]mathworks.com>
Date:
02/13/2008 12:37 PM
Subject:
RE: De-dup'ing Primary Storage



Just keep in mind that we were using IOMeter to test, not actual real
world workloads – we were also very much disk bound in our testing. If
you are neither using IOMeter (ie, a real-world workload) nor disk bound,
I can’t foresee it being a huge problem.

We’ll be testing with NFS dedupe a bit later today and I’ll gladly share
that info if you want. Same number of disks, so same stipulations exist.

And because the question did come up, the VMWare server guy didn’t build
the VMDKs with thin provisioning, so it may have also impacted the
testing.

Glenn


From: Daniel Keisling [mailto:daniel.keisling[at]austin.ppdi.com]
Sent: Wednesday, February 13, 2008 12:37 PM
To: Glenn Walker; Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage

Thanks for the stats, I'll be de-duping VMWare data soon too.

My NetApp storage tech says that read cache peformance will increase since
you're reducing the total number of actual blocks. I have not heard of
any performance degradations with A-SIS, other than filer overhead (CPU)
when SIS is actually de-duplicating. My A-SIS schedules are during
off-peak hours so it's not a concern of mine.

Daniel


From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Glenn Walker
Sent: Wednesday, February 13, 2008 9:42 AM
To: Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage
We’ve been doing some VMWare testing with FCP LUNs and A-SIS.

We saw a reduction from 471GB to 21GB with only about a 7% reduction in
performance. More than a fair trade-off in my opinion.

Our testing could have had impact on the performance more than the
de-dupe, however.

Glenn


From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Bill Holland
Sent: Wednesday, February 13, 2008 6:50 AM
To: toasters[at]mathworks.com
Subject: De-dup'ing Primary Storage

To those of you that have implemented Nearstore and A-SIS on your primary
storage:

1. Have you seen any difference in overall filer performance?
2. If you have LUNs, how are your space savings on those volumes?

I know that enabling Nearstore does some system tweaking in the background
to increase the number of concurrent backup streams that can be running,
but I don't know what else it tweaks that may adversely affect performance
of a primary storage system. Afterall, it was originally designed to run
as a secondary storage platform.




______________________________________________________________________
This email transmission and any documents, files or previous email
messages attached to it may contain information that is confidential or
legally privileged. If you are not the intended recipient or a person
responsible for delivering this transmission to the intended recipient,
you are hereby notified that you must not read this transmission and
that any disclosure, copying, printing, distribution or use of this
transmission is strictly prohibited. If you have received this
transmission
in error, please immediately notify the sender by telephone or return
email
and delete the original transmission and its attachments without reading
or saving in any manner.


ggwalker at mindspring

Feb 13, 2008, 12:08 PM

Post #9 of 13 (412 views)
Permalink
RE: De-dup'ing Primary Storage [In reply to]

Supportability for ESX I cannot say, but the NetApp VMWare whitepaper
tells you how to set up thin-provisioned VMDKs. Why wouldn't you
de-dupe the white space in the VMDK? If all zeros are written, then it
should de-dupe just fine ;)



Supportability for A-SIS is another thing I cannot say for sure, but I
know it works :-) The only documentation I can find (up to 7.2.4)
states something about it only being supported for NetBackup on NetApp.



________________________________

From: jeff.mery[at]ni.com [mailto:jeff.mery[at]ni.com]
Sent: Wednesday, February 13, 2008 2:48 PM
To: Glenn Walker
Cc: Daniel Keisling; Bill Holland; owner-toasters[at]mathworks.com;
toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage




Like the others, thanks for the info. At one point, A-SIS wasn't
supported in true "production" roles, meaning in a Tier-1 use case. Has
that changed recently? If so, I think I need to make some phone calls
=).

On the thin provisioning - ESX doesn't support thin-provisioned VMDK's
yet. Workstation does that by default, but I'm not sure about VMware
Server. Hopefully ESX is coming soon though. I would still imagine a
fair amount of reclaimed space once ESX does come through. While you'll
no longer de-dupe the white space in a VMDK, I'd imagine you'll still
de-dupe all the base OS commonalities.

Jeff Mery - MCSE, MCP
National Instruments

------------------------------------------------------------------------
-
"Allow me to extol the virtues of the Net Fairy, and of all the
fantastic
dorks that make the nice packets go from here to there. Amen."
TB - Penny Arcade
------------------------------------------------------------------------
-



From:

"Glenn Walker" <ggwalker[at]mindspring.com>

To:

"Daniel Keisling" <daniel.keisling[at]austin.ppdi.com>, "Bill Holland"
<hollandwl[at]gmail.com>, <toasters[at]mathworks.com>

Date:

02/13/2008 12:37 PM

Subject:

RE: De-dup'ing Primary Storage



________________________________




Just keep in mind that we were using IOMeter to test, not actual real
world workloads - we were also very much disk bound in our testing. If
you are neither using IOMeter (ie, a real-world workload) nor disk
bound, I can't foresee it being a huge problem.

We'll be testing with NFS dedupe a bit later today and I'll gladly share
that info if you want. Same number of disks, so same stipulations
exist.

And because the question did come up, the VMWare server guy didn't build
the VMDKs with thin provisioning, so it may have also impacted the
testing.

Glenn




________________________________


From: Daniel Keisling [mailto:daniel.keisling[at]austin.ppdi.com
<mailto:daniel.keisling[at]austin.ppdi.com> ]
Sent: Wednesday, February 13, 2008 12:37 PM
To: Glenn Walker; Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage

Thanks for the stats, I'll be de-duping VMWare data soon too.

My NetApp storage tech says that read cache peformance will increase
since you're reducing the total number of actual blocks. I have not
heard of any performance degradations with A-SIS, other than filer
overhead (CPU) when SIS is actually de-duplicating. My A-SIS schedules
are during off-peak hours so it's not a concern of mine.

Daniel




________________________________


From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com
<mailto:owner-toasters[at]mathworks.com> ] On Behalf Of Glenn Walker
Sent: Wednesday, February 13, 2008 9:42 AM
To: Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage
We've been doing some VMWare testing with FCP LUNs and A-SIS.

We saw a reduction from 471GB to 21GB with only about a 7% reduction in
performance. More than a fair trade-off in my opinion.

Our testing could have had impact on the performance more than the
de-dupe, however.

Glenn




________________________________


From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com
<mailto:owner-toasters[at]mathworks.com> ] On Behalf Of Bill Holland
Sent: Wednesday, February 13, 2008 6:50 AM
To: toasters[at]mathworks.com
Subject: De-dup'ing Primary Storage

To those of you that have implemented Nearstore and A-SIS on your
primary storage:

1. Have you seen any difference in overall filer performance?
2. If you have LUNs, how are your space savings on those volumes?

I know that enabling Nearstore does some system tweaking in the
background to increase the number of concurrent backup streams that can
be running, but I don't know what else it tweaks that may adversely
affect performance of a primary storage system. Afterall, it was
originally designed to run as a secondary storage platform.




______________________________________________________________________
This email transmission and any documents, files or previous email
messages attached to it may contain information that is confidential or
legally privileged. If you are not the intended recipient or a person
responsible for delivering this transmission to the intended recipient,
you are hereby notified that you must not read this transmission and
that any disclosure, copying, printing, distribution or use of this
transmission is strictly prohibited. If you have received this
transmission
in error, please immediately notify the sender by telephone or return
email
and delete the original transmission and its attachments without reading
or saving in any manner.


Peter.Learmonth at netapp

Feb 13, 2008, 1:16 PM

Post #10 of 13 (412 views)
Permalink
RE: De-dup'ing Primary Storage [In reply to]

Hi Jeff!
Make your calls! ;-)

ESX does support thin-provisioned VMDKs. They are default in NFS
datastores. You have to use vmkfstools from the command line to create
thin-provisioned VMDKs in VMFS. Either way, when you clone a thin VM in
VirtualCenter, it makes a thick clone.

There is also thin provisioning on the NetApp side, which some customers
are also using successfully (at least one that I know of for over a
year).

Enjoy!

Peter

________________________________

From: jeff.mery[at]ni.com [mailto:jeff.mery[at]ni.com]
Sent: Wednesday, February 13, 2008 11:48 AM
To: Glenn Walker
Cc: Daniel Keisling; Bill Holland; owner-toasters[at]mathworks.com;
toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage



Like the others, thanks for the info. At one point, A-SIS wasn't
supported in true "production" roles, meaning in a Tier-1 use case. Has
that changed recently? If so, I think I need to make some phone calls
=).

On the thin provisioning - ESX doesn't support thin-provisioned VMDK's
yet. Workstation does that by default, but I'm not sure about VMware
Server. Hopefully ESX is coming soon though. I would still imagine a
fair amount of reclaimed space once ESX does come through. While you'll
no longer de-dupe the white space in a VMDK, I'd imagine you'll still
de-dupe all the base OS commonalities.

Jeff Mery - MCSE, MCP
National Instruments

------------------------------------------------------------------------
-
"Allow me to extol the virtues of the Net Fairy, and of all the
fantastic
dorks that make the nice packets go from here to there. Amen."
TB - Penny Arcade
------------------------------------------------------------------------
-



From: "Glenn Walker" <ggwalker[at]mindspring.com>
To: "Daniel Keisling" <daniel.keisling[at]austin.ppdi.com>, "Bill
Holland" <hollandwl[at]gmail.com>, <toasters[at]mathworks.com>
Date: 02/13/2008 12:37 PM
Subject: RE: De-dup'ing Primary Storage

________________________________




Just keep in mind that we were using IOMeter to test, not actual real
world workloads - we were also very much disk bound in our testing. If
you are neither using IOMeter (ie, a real-world workload) nor disk
bound, I can't foresee it being a huge problem.

We'll be testing with NFS dedupe a bit later today and I'll gladly share
that info if you want. Same number of disks, so same stipulations
exist.

And because the question did come up, the VMWare server guy didn't build
the VMDKs with thin provisioning, so it may have also impacted the
testing.

Glenn


________________________________


From: Daniel Keisling [mailto:daniel.keisling[at]austin.ppdi.com
<mailto:daniel.keisling[at]austin.ppdi.com> ]
Sent: Wednesday, February 13, 2008 12:37 PM
To: Glenn Walker; Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage

Thanks for the stats, I'll be de-duping VMWare data soon too.

My NetApp storage tech says that read cache peformance will increase
since you're reducing the total number of actual blocks. I have not
heard of any performance degradations with A-SIS, other than filer
overhead (CPU) when SIS is actually de-duplicating. My A-SIS schedules
are during off-peak hours so it's not a concern of mine.

Daniel


________________________________


From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com
<mailto:owner-toasters[at]mathworks.com> ] On Behalf Of Glenn Walker
Sent: Wednesday, February 13, 2008 9:42 AM
To: Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage
We've been doing some VMWare testing with FCP LUNs and A-SIS.

We saw a reduction from 471GB to 21GB with only about a 7% reduction in
performance. More than a fair trade-off in my opinion.

Our testing could have had impact on the performance more than the
de-dupe, however.

Glenn


________________________________


From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com
<mailto:owner-toasters[at]mathworks.com> ] On Behalf Of Bill Holland
Sent: Wednesday, February 13, 2008 6:50 AM
To: toasters[at]mathworks.com
Subject: De-dup'ing Primary Storage

To those of you that have implemented Nearstore and A-SIS on your
primary storage:

1. Have you seen any difference in overall filer performance?
2. If you have LUNs, how are your space savings on those volumes?

I know that enabling Nearstore does some system tweaking in the
background to increase the number of concurrent backup streams that can
be running, but I don't know what else it tweaks that may adversely
affect performance of a primary storage system. Afterall, it was
originally designed to run as a secondary storage platform.




______________________________________________________________________
This email transmission and any documents, files or previous email
messages attached to it may contain information that is confidential or
legally privileged. If you are not the intended recipient or a person
responsible for delivering this transmission to the intended recipient,
you are hereby notified that you must not read this transmission and
that any disclosure, copying, printing, distribution or use of this
transmission is strictly prohibited. If you have received this
transmission
in error, please immediately notify the sender by telephone or return
email
and delete the original transmission and its attachments without reading
or saving in any manner.


jack1729 at gmail

Feb 13, 2008, 6:50 PM

Post #11 of 13 (410 views)
Permalink
Re: De-dup'ing Primary Storage [In reply to]

Ditto. We are seeing ~65% savings on vmware luns....serving about 60 luns.
no performance impacts during the day slight (~3%) when actually deduping.

Jack
Glenn Dekhayser wrote:
>
> I have a client de-duping ~15 iSCSI LUNs in the same volume (vmware c:
> drives), getting great data reduction, with no perceivable performance
> impact.
>
>
>
> -Glenn (the other one)
>
>
>
> *From:* owner-toasters[at]mathworks.com
> [mailto:owner-toasters[at]mathworks.com] *On Behalf Of *Willeke, Jochen
> *Sent:* Wednesday, February 13, 2008 8:01 AM
> *To:* Bill Holland; toasters[at]mathworks.com
> *Subject:* RE: De-dup'ing Primary Storage
>
>
>
> Hi Bill,
>
>
>
> we have ASIS on two of our 6070's for some time. We did only few tests
> but could not see any direct impact on the filers performance.
>
> We have not yet used it for LUNs so far, but would be interessted in
> experience with ASIS on LUNs as well.
>
>
>
> Regards
>
>
>
> Jochen
>
>
>
> ------------------------------------------------------------------------
>
> *From:* owner-toasters[at]mathworks.com
> [mailto:owner-toasters[at]mathworks.com] *On Behalf Of *Bill Holland
> *Sent:* Wednesday, February 13, 2008 12:50 PM
> *To:* toasters[at]mathworks.com
> *Subject:* De-dup'ing Primary Storage
>
> To those of you that have implemented Nearstore and A-SIS on your
> primary storage:
>
>
>
> 1. Have you seen any difference in overall filer performance?
>
> 2. If you have LUNs, how are your space savings on those volumes?
>
>
>
> I know that enabling Nearstore does some system tweaking in the
> background to increase the number of concurrent backup streams that
> can be running, but I don't know what else it tweaks that may
> adversely affect performance of a primary storage system. Afterall,
> it was originally designed to run as a secondary storage platform.
>
> ------------------------------------------------------------------------
>
> Wincor Nixdorf International GmbH
> Sitz der Gesellschaft: Paderborn
> Registergericht Paderborn HRB 3507
> Geschftsfhrer: Eckard Heidloff (Vorsitzender), Stefan Auerbach, Dr.
> Jrgen Wunram
> Vorsitzender des Aufsichtsrats: Karl-Heinz Stiller
> Steuernummer: 339/5884/0020 - Ust-ID Nr.: DE812927716 - WEEE-Reg.-Nr.
> DE44477193
>
> Diese E-Mail enthlt vertrauliche Informationen. Wenn Sie nicht der
> richtige Adressat sind oder diese E-Mail irrtmlich erhalten haben,
> informieren Sie bitte sofort den Absender und vernichten Sie diese
> E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser
> E-Mail ist nicht gestattet.
>
> This e-mail may contain confidential information. If you are not the
> intended recipient (or have received this e-mail in error) please
> notify the sender immediately and destroy this e-mail. Any
> unauthorised copying, disclosure or distribution of the material in
> this e-mail is strictly forbidden.
>


Andrey.Borzenkov at fujitsu-siemens

Feb 13, 2008, 11:00 PM

Post #12 of 13 (409 views)
Permalink
RE: De-dup'ing Primary Storage [In reply to]

Have you got any pointers regarding this? This is one not quite clear point to me too – whether deduplication affects sequential IO and what ā€œreal lifeā€ impact is.


Д уважением / With best regards / Mit freundlichen Grüβen

---
Andrey Borzenkov
Senior system engineer

________________________________
From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com] On Behalf Of Glenn Dekhayser
Sent: Wednesday, February 13, 2008 5:29 PM
To: Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage

I would be interested in knowing the impact of a-sis on cache hit % though. Since you’re essentially changing pointers to other instances of blocks, those de-duped blocks wouldn’t be located along with the other original blocks, so it could affect the performance of onTap’s read-ahead algorithm.

From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com] On Behalf Of Willeke, Jochen
Sent: Wednesday, February 13, 2008 8:01 AM
To: Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage

Hi Bill,

we have ASIS on two of our 6070's for some time. We did only few tests but could not see any direct impact on the filers performance.
We have not yet used it for LUNs so far, but would be interessted in experience with ASIS on LUNs as well.

Regards

Jochen

________________________________
From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com] On Behalf Of Bill Holland
Sent: Wednesday, February 13, 2008 12:50 PM
To: toasters[at]mathworks.com
Subject: De-dup'ing Primary Storage
To those of you that have implemented Nearstore and A-SIS on your primary storage:

1. Have you seen any difference in overall filer performance?
2. If you have LUNs, how are your space savings on those volumes?

I know that enabling Nearstore does some system tweaking in the background to increase the number of concurrent backup streams that can be running, but I don't know what else it tweaks that may adversely affect performance of a primary storage system. Afterall, it was originally designed to run as a secondary storage platform.
________________________________
Wincor Nixdorf International GmbH
Sitz der Gesellschaft: Paderborn
Registergericht Paderborn HRB 3507
Geschftsfhrer: Eckard Heidloff (Vorsitzender), Stefan Auerbach, Dr. Jrgen Wunram
Vorsitzender des Aufsichtsrats: Karl-Heinz Stiller
Steuernummer: 339/5884/0020 - Ust-ID Nr.: DE812927716 - WEEE-Reg.-Nr. DE44477193

Diese E-Mail enthlt vertrauliche Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtmlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet.

This e-mail may contain confidential information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden.


ggwalker at mindspring

Feb 14, 2008, 5:21 PM

Post #13 of 13 (399 views)
Permalink
RE: De-dup'ing Primary Storage [In reply to]

FYI - we've just completed testing with NFS and dedupe - about 72%
savings (VMDK on NFS is thin provisioned by default anyways, so no big
shock there). About 4% performance reduction, easily acceptable.



One thing to keep in mind - the busier the drives, the sharper the
performance loss (as would be expected): We only have a 21 disk I/O
pool (14D+2P, 7D+2P AGGR), and we're getting about 38MB/s for an 8K
transfer size, 100% random, 25%/75% r/w mix with a 4GB test file per VM
Guest - 12 guests total. Disks are almost maxed out (95% avg I'd say -
disks are upsetting the NVRAM pretty quickly) and adding de-dupe dropped
it to 36.8MB/s. Kicking it up to 15 guests creates 100% disk I/O and
adding de-dupe gives about a 25% performance loss. Testing with 2MB
transfer gives us about 112MB/s, so the testing is pretty subjective -
as is the performance loss.



The thing to keep in mind: performance will _always_ suck when 100%
disk utilization kicks in. The 4% performance loss we've seen for NFS
w/ de-dupe and 7% performance loss with FCP de-dupe when disks are
almost completely at max are cases where more disk I/O would help and
I'd guess that we'd see little to no performance degradation.



We've found that the performance curve for NFS and FCP dedupe are about
the same - the performance for both for a VMWare environment are pretty
close as well.



Hope this info helps some of you...



Glenn



________________________________

From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Glenn Walker
Sent: Wednesday, February 13, 2008 12:50 PM
To: Daniel Keisling; Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage



Just keep in mind that we were using IOMeter to test, not actual real
world workloads - we were also very much disk bound in our testing. If
you are neither using IOMeter (ie, a real-world workload) nor disk
bound, I can't foresee it being a huge problem.



We'll be testing with NFS dedupe a bit later today and I'll gladly share
that info if you want. Same number of disks, so same stipulations
exist.



And because the question did come up, the VMWare server guy didn't build
the VMDKs with thin provisioning, so it may have also impacted the
testing.



Glenn



________________________________

From: Daniel Keisling [mailto:daniel.keisling[at]austin.ppdi.com]
Sent: Wednesday, February 13, 2008 12:37 PM
To: Glenn Walker; Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage



Thanks for the stats, I'll be de-duping VMWare data soon too.



My NetApp storage tech says that read cache peformance will increase
since you're reducing the total number of actual blocks. I have not
heard of any performance degradations with A-SIS, other than filer
overhead (CPU) when SIS is actually de-duplicating. My A-SIS schedules
are during off-peak hours so it's not a concern of mine.


Daniel



________________________________

From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Glenn Walker
Sent: Wednesday, February 13, 2008 9:42 AM
To: Bill Holland; toasters[at]mathworks.com
Subject: RE: De-dup'ing Primary Storage

We've been doing some VMWare testing with FCP LUNs and A-SIS.



We saw a reduction from 471GB to 21GB with only about a 7% reduction in
performance. More than a fair trade-off in my opinion.



Our testing could have had impact on the performance more than the
de-dupe, however.



Glenn



________________________________

From: owner-toasters[at]mathworks.com [mailto:owner-toasters[at]mathworks.com]
On Behalf Of Bill Holland
Sent: Wednesday, February 13, 2008 6:50 AM
To: toasters[at]mathworks.com
Subject: De-dup'ing Primary Storage



To those of you that have implemented Nearstore and A-SIS on your
primary storage:



1. Have you seen any difference in overall filer performance?

2. If you have LUNs, how are your space savings on those volumes?



I know that enabling Nearstore does some system tweaking in the
background to increase the number of concurrent backup streams that can
be running, but I don't know what else it tweaks that may adversely
affect performance of a primary storage system. Afterall, it was
originally designed to run as a secondary storage platform.





______________________________________________________________________
This email transmission and any documents, files or previous email
messages attached to it may contain information that is confidential or
legally privileged. If you are not the intended recipient or a person
responsible for delivering this transmission to the intended recipient,
you are hereby notified that you must not read this transmission and
that any disclosure, copying, printing, distribution or use of this
transmission is strictly prohibited. If you have received this
transmission
in error, please immediately notify the sender by telephone or return
email
and delete the original transmission and its attachments without reading
or saving in any manner.

Netapp toasters RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.