ldidillon at prologue
Apr 27, 2009, 8:34 AM
Post #3 of 3
(692 views)
Permalink
|
|
Re: drbd (8.0.14, 8.0.16) very slow with kernels 2.6.27 to 2.6.29 but ok with kernel 2.6.20
[In reply to]
|
|
I don't test with drbd-8.3 but with 8.0.14, 8.0.15rc and 8.0.16.
I have now tested the write throughput with the following command:
dd if=/dev/zero of=/dbase/test_write_drbd bs=500M count=2
conv=fsync,notrunc
successively in primary/secondary mode and in standalone mode, with
/dev/drbd0 mounted on /dbase
and then without drbd, /dev/sdb1 mounted on /dbase
for each configuration, this command has been repeted 4 or 5 times.
Results for write throughput tests are:
(approx.) time/throughput to write 1GB on /dbase:
Kernel 2.6.20
Kernel 2.6.27
Primary/Secondary
~17s. ~60MB/s
~90s. ~11,6 MB/s
StandAlone
~11s. ~93MB/s
~77s. ~14MB/s
Without drbd
~9s. ~110MB/s
~9s. ~110MB/s
Read performances test don't show significative différences between the
two kernels.
We have 2 IBM x3400 with serveRAID M10 (aacraid driver), 8 processors
(2xquadcores), 16Go memory, 2 x1Gb ethernet card with one dedicated for drbd
one Raid1 for the system one Raid1 for drbd.
This Raid1 for drbd is composed with 2x 140Go disks (15trs/mn) (about
130G for partition sdb1 mounted on /dbase and 5G for sdb2 used for the
metadatas.
I join the drbd.conf . If you want I can join dmesg and lspci for the 2
kernels
We have tested drbd-8.0.14 with kernel 2.6.27 on other machines, we
don't find difference with the 2.6.20 kernel.
I really need some help,
Bests regards
_*drbd.conf:*_
global {
minor-count 4;
dialog-refresh 5; # 5 seconds
# You might disable one of drbdadm's sanity check.
# disable-ip-verification;
usage-count no;
}
resource drbd0 {
protocol C;
handlers {
pri-on-incon-degr "/opt/banana/bin/statusmon --alert DEGRADED; sleep 5 ";
pri-lost-after-sb "/opt/banana/bin/statusmon --alert DEGRADED; sleep 5 ";
split-brain "echo 1 > /var/lib/alinka_engine/status/split-brain";
}
startup {
# Par defaut 0 = attendre infiniment
wfc-timeout 3;
# Pareil si l'on travaille en mode degrade
degr-wfc-timeout 3;
}
disk {
on-io-error detach;
}
syncer {
rate 33M;
al-extents 127;
}
on morinc1 {
device /dev/drbd0;
disk /dev/sdb1;
address 192.168.0.1:7788;
meta-disk /dev/sdb2[0];
}
on morinc2 {
device /dev/drbd0;
disk /dev/sdb1;
address 192.168.0.2:7788;
meta-disk /dev/sdb2[0];
}
}
If I add:
net {
max-epoch-size 8192;
max-buffers 8192;
unplug-watermark 8192;
}
of if I change rate syncer to 100
this don't change nothing....
_*lspci (same on the 2 kernel)*_
00:00.0 Host bridge: Intel Corporation Server Memory Controller Hub (rev b1)
00:02.0 PCI bridge: Intel Corporation Server PCI Express x8 Port 2-3
(rev b1)
00:03.0 PCI bridge: Intel Corporation Server PCI Express x4 Port 3 (rev b1)
00:04.0 PCI bridge: Intel Corporation Server PCI Express x8 Port 4-5
(rev b1)
00:05.0 PCI bridge: Intel Corporation Server PCI Express x4 Port 5 (rev b1)
00:06.0 PCI bridge: Intel Corporation Server PCI Express x8 Port 6-7
(rev b1)
00:07.0 PCI bridge: Intel Corporation Server PCI Express x4 Port 7 (rev b1)
00:10.0 Host bridge: Intel Corporation Server Error Reporting Registers
(rev b1)
00:10.1 Host bridge: Intel Corporation Server Error Reporting Registers
(rev b1)
00:10.2 Host bridge: Intel Corporation Server Error Reporting Registers
(rev b1)
00:11.0 Host bridge: Intel Corporation Reserved Registers (rev b1)
00:13.0 Host bridge: Intel Corporation Reserved Registers (rev b1)
00:15.0 Host bridge: Intel Corporation Server FBD Registers (rev b1)
00:16.0 Host bridge: Intel Corporation Server FBD Registers (rev b1)
00:1c.0 PCI bridge: Intel Corporation Enterprise Southbridge PCI Express
Root Port 1 (rev 09)
00:1c.1 PCI bridge: Intel Corporation Enterprise Southbridge PCI Express
Root Port 2 (rev 09)
00:1d.0 USB Controller: Intel Corporation Enterprise Southbridge UHCI
USB #1 (rev 09)
00:1d.1 USB Controller: Intel Corporation Enterprise Southbridge UHCI
USB #2 (rev 09)
00:1d.2 USB Controller: Intel Corporation Enterprise Southbridge UHCI
USB #3 (rev 09)
00:1d.3 USB Controller: Intel Corporation Enterprise Southbridge UHCI
USB #4 (rev 09)
00:1d.7 USB Controller: Intel Corporation Enterprise Southbridge EHCI
USB (rev 09)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d9)
00:1f.0 ISA bridge: Intel Corporation Enterprise Southbridge LPC (rev 09)
00:1f.1 IDE interface: Intel Corporation Enterprise Southbridge PATA
(rev 09)
00:1f.3 SMBus: Intel Corporation Enterprise Southbridge SMBus (rev 09)
01:00.0 PCI bridge: Intel Corporation Enterprise Southbridge PCI Express
Upstream Port (rev 01)
01:00.3 PCI bridge: Intel Corporation Enterprise Southbridge PCI Express
to PCI-X Bridge (rev 01)
02:00.0 PCI bridge: Intel Corporation Enterprise Southbridge PCI Express
Downstream Port E1 (rev 01)
02:01.0 PCI bridge: Intel Corporation Enterprise Southbridge PCI Express
Downstream Port E2 (rev 01)
03:00.0 RAID bus controller: Adaptec AAC-RAID (Rocket) (rev 02)
04:00.0 PCI bridge: Broadcom: Unknown device 0103 (rev c3)
05:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708
Gigabit Ethernet (rev 12)
15:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E
PCI-Express Fusion-MPT SAS (rev 02)
1a:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5721
Gigabit Ethernet PCI Express (rev 21)
1c:04.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev 02)
Antony MARTINEAU a écrit :
> Hello we have two clusters whith 2 nodes each
> One whith SLES10SP2 ( kernel 2.6.16.60-0.34-xen + drbd 8.3.1)
> One Whith SLES11 (kernel 2.6.27.19-5-xen + drbd 8.3.1).
> No problem whith performance on both
> Your test is not accurate, if you copy file, it is suppose there is a
> FS on drbd devices...
> Could you give the speed directly on block device and after on the
> drbd device.
>
> Have you test whith the latest version of drbd?
> Cordialement,
>
>
> *Antony
> MARTINEAU*
> Service informatique
> Technicien Informatique
> LIPPI Management La Fouillouse
> 16440 Mouthiers sur Boeme
> Tel.: 0545673435
> Fax: 0545673435
> Courriel: _antony.martineau [at] lippi <mailto:antony.martineau [at] lippi>_
> __http://www.lippi.fr_ <http://www.lippi.fr/>
>
>
>
>
>
> De : Lise Didillon <ldidillon [at] prologue>
> A: drbd-user [at] lists
> Date: 06/04/2009 18:38
> Objet : [DRBD-user] drbd (8.0.14, 8.0.16) very slow with kernels
> 2.6.27 to 2.6.29 but ok with kernel 2.6.20
> Envoyé par : drbd-user-bounces [at] lists
>
>
> ------------------------------------------------------------------------
>
>
>
> Hello,
> with drbd-8.0.14 or 8.0.16 on the same machines:
> My tests are "copy big files on the replicated device". This takes for
> a file 8 seconds. instead of 0.2 seconds without drbd if kernel
> version is 2.6.27 or 28 or 29.
> but with kernel 2.6.20 there is no performance problem: it takes
> sensibly the same time to copy this file if drbd is running or not.
> I alway use the same drbd.conf for these tests.
> have you some ideas about why drbd is so slow with kernels 2.6.27 to
> 2.6.29?
> regards,
> Lise Didillon
>
> --
> *Lise DIDILLON*
> Ingénieur d'études, Ligne Produit UseIT
> lise.didillon [at] prologue <mailto:lise.didillon [at] prologue>
>
>
> ------------------------------------------------------------------------
> logo prologue <http://www.prologue.fr/>
> ZA de Courtaboeuf
> 12 Avenue des Tropiques
> BP 73 - 91 943 Les Ulis Cedex
>
> *Tél :*
> +33 1 69 29 39 39
> *Fax :*
> +33 1 69 29 90 43
>
>
>
>
> _______________________________________________
> drbd-user mailing list
> drbd-user [at] lists
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
>
>
>
>
>
>
>
>
>
> Ce message et toutes les pieces jointes sont etablis a l'attention
> exclusive de ses destinataires et sont strictement confidentiels.
> _Pour en savoir plus cliquer ici_ <http://www.lippi.fr/disclaimer.php>
>
> This message and any attachments are confidential to the ordinary user
> of the e-mail address to which it was addressed and may also be
> privileged. _More information_ <http://www.lippi.fr/disclaimer.php>
>
--
Lise DIDILLON
Ingénieur d'études, Ligne Produit UseIT lise.didillon [at] prologue
<mailto:lise.didillon [at] prologue>
------------------------------------------------------------------------
logo prologue <http://www.prologue.fr> ZA de Courtaboeuf
12 Avenue des Tropiques
BP 73 - 91 943 Les Ulis Cedex
Tél : +33 1 69 29 39 39
Fax : +33 1 69 29 90 43
|
logo_prologue_email.gif
