Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: DRBD: Users

DRBD LVM device-mapper error

 

 

DRBD users RSS feed   Index | Next | Previous | View Threaded


yuan.luo at ericsson

Jan 10, 2012, 3:40 AM

Post #1 of 10 (1965 views)
Permalink
DRBD LVM device-mapper error

Hi,

During established DRBD+Pacemaker, I met the problem with the following error in node2. I've googled a lot information, but nothing helped.
In another production environment, everything works. DRBD+Pacemaker, failover OK.

Could you please help me out?

app-ebs02:/dev/mapper # lvcreate -n ipwvol -L 50G ipwdg
WARNING: Locking disabled. Be careful! This could corrupt your metadata.
device-mapper: reload ioctl failed: Invalid argument
Aborting. Failed to activate new LV to wipe the start of it.

app-ebs02:/dev/mapper # dmesg |grep -i "device"
[375349.283710] device-mapper: table: 253:0: linear: dm-linear: Device lookup failed
[375349.283713] device-mapper: ioctl: error adding target to table

Is that because the hard-disk is broken? Thank you in advance.

Best Regards,
YUAN LUO


ff at mpexnet

Jan 10, 2012, 4:13 AM

Post #2 of 10 (1897 views)
Permalink
Re: DRBD LVM device-mapper error [In reply to]

Hi,

On 01/10/2012 12:40 PM, Yuan Luo wrote:
> During established DRBD+Pacemaker, I met the problem with the following
> error in node2. I've googled a lot information, but nothing helped.
> In another production environment, everything works. DRBD+Pacemaker,
> failover OK.

Is node2 Primary when you try doing this? It should be, otherwise you're
bound to fail.

> Could you please help me out?
>
> app-ebs02:/dev/mapper # lvcreate -n ipwvol -L 50G ipwdg
> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
> device-mapper: reload ioctl failed: Invalid argument
> Aborting. Failed to activate new LV to wipe the start of it.

Need more information. What's the output of pvs and lvs?

> Is that because the hard-disk is broken? Thank you in advance.

I disbelieve so.

Cheers,
Felix
_______________________________________________
drbd-user mailing list
drbd-user [at] lists
http://lists.linbit.com/mailman/listinfo/drbd-user


ildefonso.camargo at gmail

Jan 10, 2012, 4:14 AM

Post #3 of 10 (1908 views)
Permalink
Re: DRBD LVM device-mapper error [In reply to]

Hi!

Please, give this info:

+ pvdisplay (output of this command)
+ your current drbd resource configuration

Ildedfonso.

On Tue, Jan 10, 2012 at 7:10 AM, Yuan Luo <yuan.luo [at] ericsson> wrote:
> Hi,
>
> During established DRBD+Pacemaker, I met the problem with the following
> error in node2. I've googled a lot information, but nothing helped.
> In another production environment, everything works. DRBD+Pacemaker,
> failover OK.
>
> Could you please help me out?
>
> app-ebs02:/dev/mapper # lvcreate -n ipwvol -L 50G ipwdg
> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
> device-mapper: reload ioctl failed: Invalid argument
> Aborting. Failed to activate new LV to wipe the start of it.
>
> app-ebs02:/dev/mapper # dmesg |grep -i "device"
> [375349.283710] device-mapper: table: 253:0: linear: dm-linear: Device
> lookup failed
> [375349.283713] device-mapper: ioctl: error adding target to table
>
> Is that because the hard-disk is broken? Thank you in advance.
>
> Best Regards,
> YUAN LUO
>
>
>
> _______________________________________________
> drbd-user mailing list
> drbd-user [at] lists
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
_______________________________________________
drbd-user mailing list
drbd-user [at] lists
http://lists.linbit.com/mailman/listinfo/drbd-user


yuan.luo at ericsson

Jan 10, 2012, 7:34 PM

Post #4 of 10 (1895 views)
Permalink
Re: DRBD LVM device-mapper error [In reply to]

Hi,

Thanks for your reply.

Node2 is DRBD primary. As I mentioned before, in another set of production environment, I've successfully established DRBD(P+S) + Pacemaker and failover works perfect.
So please assume the default DRBD configuration were correct. Ebs02 is primary, ebs01 is secondary. /dev/sda1 is a 300GB physical partition in both nodes and is expected to create a drbd.

ipworks-ebs02:~ # rcdrbd status
drbd driver loaded OK; device status:
version: 8.3.7 (api:88/proto:86-91)
GIT-hash: ea9e28dbff98e331a62bcbcc63a6135808fe2917 build by phil [at] fat-tyr, 2010-01-13 17:17:27
m:res cs ro ds p mounted fstype
0:r0 Connected Primary/Secondary UpToDate/UpToDate C

I also modified /etc/lvm/lvm.conf +line 35 as:
filter = [ "a|^/dev/drbd|", "a/sda/", "a/sdb/", "r/.*/" ]

Now the situation is, pv /dev/drbd0 created ok, vg ipwdg created ok, but lv ipwvol created failed.
ipworks-ebs02:~ # lvcreate -n ipwvol -L 50G ipwdg
WARNING: Locking disabled. Be careful! This could corrupt your metadata.
device-mapper: reload ioctl failed: Invalid argument
Aborting. Failed to activate new LV to wipe the start of it.

ipworks-ebs02:~ # pvs
WARNING: Locking disabled. Be careful! This could corrupt your metadata.
PV VG Fmt Attr PSize PFree
/dev/sda1 ipwdg lvm2 a- 299.98G 299.98G

ipworks-ebs02:~ # pvdisplay
WARNING: Locking disabled. Be careful! This could corrupt your metadata.
--- Physical volume ---
PV Name /dev/sda1
VG Name ipwdg
PV Size 299.99 GB / not usable 2.62 MB
Allocatable yes
PE Size (KByte) 4096
Total PE 76796
Free PE 76796
Allocated PE 0
PV UUID iEopx7-vr09-FKGx-wUWq-hXLm-OVKn-4k3Lq5

ipworks-ebs02:~ # vgdisplay
WARNING: Locking disabled. Be careful! This could corrupt your metadata.
--- Volume group ---
VG Name ipwdg
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 7
VG Access read/write
VG Status resizable
Clustered yes
Shared no
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 299.98 GB
PE Size 4.00 MB
Total PE 76796
Alloc PE / Size 0 / 0
Free PE / Size 76796 / 299.98 GB
VG UUID AATSOL-Fnua-HNwR-n8Ph-waHt-WkMj-k3P7Zy

lvs command output: empty
Of course ;) no lv is created.

This problem is really a little tricky. Thanks in advance.

BRs,
Yuan Luo

P.S. I attached the verbose info of the lvcreate:
The following line is suspecious, but I'm not sure about the root cause. Is it related with Kernel? But in another completely same configured environment, lv creation worked ...

#ioctl/libdm-iface.c:1702 device-mapper: reload ioctl failed: Invalid argument

=========================================
ipworks-ebs02:~ # lvcreate -n ipwvol -L 50G ipwdg -vvvv
#lvmcmdline.c:915 Processing: lvcreate -n ipwvol -L 50G ipwdg -vvvv
#lvmcmdline.c:918 O_DIRECT will be used
#config/config.c:950 Setting global/locking_type to 0
WARNING: Locking disabled. Be careful! This could corrupt your metadata.
#activate/activate.c:362 Getting target version for linear
#ioctl/libdm-iface.c:1685 dm version OF [16384]
#ioctl/libdm-iface.c:1685 dm versions OF [16384]
#activate/activate.c:362 Getting target version for striped
#ioctl/libdm-iface.c:1685 dm versions OF [16384]
#lvcreate.c:302 Setting logging type to disk
#config/config.c:950 Setting activation/mirror_region_size to 512
#lvcreate.c:895 Finding volume group "ipwdg"
#device/dev-io.c:440 Opened /dev/sda1 RW O_DIRECT
#device/dev-io.c:135 /dev/sda1: block size is 512 bytes
#label/label.c:160 /dev/sda1: lvm2 label detected
#cache/lvmcache.c:965 lvmcache: /dev/sda1: now in VG #orphans_lvm2 (#orphans_lvm2)
#format_text/format-text.c:1095 /dev/sda1: Found metadata at 6656 size 656 (in area at 4096 size 192512) for ipwdg (AATSOL-Fnua-HNwR-n8Ph-waHt-WkMj-k3P7Zy)
#cache/lvmcache.c:965 lvmcache: /dev/sda1: now in VG ipwdg with 1 mdas
#cache/lvmcache.c:752 lvmcache: /dev/sda1: setting ipwdg VGID to AATSOLFnuaHNwRn8PhwaHtWkMjk3P7Zy
#cache/lvmcache.c:1002 lvmcache: /dev/sda1: VG ipwdg: Set creation host to ipworks-ebs02.
#device/dev-io.c:440 Opened /dev/sda2 RW O_DIRECT
#device/dev-io.c:135 /dev/sda2: block size is 4096 bytes
#label/label.c:184 /dev/sda2: No label detected
#label/label.c:287 <backtrace>
#device/dev-io.c:486 Closed /dev/sda2
#device/dev-io.c:440 Opened /dev/root RW O_DIRECT
#device/dev-io.c:135 /dev/root: block size is 4096 bytes
#label/label.c:184 /dev/root: No label detected
#label/label.c:287 <backtrace>
#device/dev-io.c:486 Closed /dev/root
#device/dev-io.c:440 Opened /dev/sdb RW O_DIRECT
#device/dev-io.c:135 /dev/sdb: block size is 4096 bytes
#label/label.c:184 /dev/sdb: No label detected
#label/label.c:287 <backtrace>
#device/dev-io.c:486 Closed /dev/sdb
#label/label.c:270 Using cached label for /dev/sda1
#label/label.c:270 Using cached label for /dev/sda1
#format_text/format-text.c:476 Read ipwdg metadata (3) from /dev/sda1 at 6656 size 656
#metadata/pv_manip.c:272 /dev/sda1 0: 0 76796: NULL(0:0)
#format_text/archiver.c:123 Archiving volume group "ipwdg" metadata (seqno 3).
#metadata/lv_manip.c:1822 Creating logical volume ipwvol
#metadata/pv_map.c:47 Allowing allocation on /dev/sda1 start PE 0 length 76796
#metadata/pv_manip.c:272 /dev/sda1 0: 0 12800: ipwvol(0:0)
#metadata/pv_manip.c:272 /dev/sda1 1: 12800 63996: NULL(0:0)
#format_text/format-text.c:566 Writing ipwdg metadata to /dev/sda1 at 7680 len 910
#format_text/format-text.c:662 Pre-Committing ipwdg metadata (4) to /dev/sda1 header at 4096
#format_text/archiver.c:359 Creating volume group backup "/etc/lvm/backup/ipwdg" (seqno 4).
#format_text/format-text.c:852 Writing ipwdg metadata to /etc/lvm/backup/.lvm_ipworks-ebs02_27871_617574249
#format_text/format-text.c:896 Committing ipwdg metadata (4)
#format_text/format-text.c:897 Renaming /etc/lvm/backup/ipwdg.tmp to /etc/lvm/backup/ipwdg
#format_text/format-text.c:662 Committing ipwdg metadata (4) to /dev/sda1 header at 4096
#metadata/metadata.c:2078 Finding volume group for uuid AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG
#label/label.c:270 Using cached label for /dev/sda1
#label/label.c:270 Using cached label for /dev/sda1
#format_text/format-text.c:476 Read ipwdg metadata (4) from /dev/sda1 at 7680 size 910
#metadata/metadata.c:2084 Found volume group "ipwdg"
#activate/activate.c:477 Getting device info for ipwdg-ipwvol
#ioctl/libdm-iface.c:1685 dm info LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG NF [16384]
#ioctl/libdm-iface.c:1685 dm info AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG NF [16384]
#ioctl/libdm-iface.c:1685 dm info ipwdg-ipwvol NF [16384]
#mm/memlock.c:99 Locking memory
#mm/memlock.c:130 memlock_count inc to 1
#activate/dev_manager.c:623 Getting device info for ipwdg-ipwvol [LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG]
#ioctl/libdm-iface.c:1685 dm info LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG OF [16384]
#ioctl/libdm-iface.c:1685 dm info AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG OF [16384]
#ioctl/libdm-iface.c:1685 dm info ipwdg-ipwvol OF [16384]
#activate/dev_manager.c:623 Getting device info for ipwdg-ipwvol-real [LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-real]
#ioctl/libdm-iface.c:1685 dm info LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-real OF [16384]
#ioctl/libdm-iface.c:1685 dm info AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-real OF [16384]
#ioctl/libdm-iface.c:1685 dm info ipwdg-ipwvol-real OF [16384]
#activate/dev_manager.c:623 Getting device info for ipwdg-ipwvol-cow [LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-cow]
#ioctl/libdm-iface.c:1685 dm info LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-cow OF [16384]
#ioctl/libdm-iface.c:1685 dm info AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-cow OF [16384]
#ioctl/libdm-iface.c:1685 dm info ipwdg-ipwvol-cow OF [16384]
#activate/dev_manager.c:623 Getting device info for ipwdg-ipwvol_mlog [LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-_mlog]
#ioctl/libdm-iface.c:1685 dm info LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-_mlog OF [16384]
#ioctl/libdm-iface.c:1685 dm info AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-_mlog OF [16384]
#ioctl/libdm-iface.c:1685 dm info ipwdg-ipwvol_mlog OF [16384]
#activate/dev_manager.c:875 Checking kernel supports striped segment type for ipwvol
#libdm-deptree.c:1210 Creating ipwdg-ipwvol
#ioctl/libdm-iface.c:1685 dm create ipwdg-ipwvol LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG NF [16384]
#libdm-common.c:599 ipwdg-ipwvol: Stacking NODE_ADD (253,0) 0:6 0640
#libdm-deptree.c:1537 Loading ipwdg-ipwvol table
#libdm-deptree.c:1486 Adding target: 0 104857600 linear 8:1 384
#ioctl/libdm-iface.c:1685 dm table (253:0) OF [16384]
#ioctl/libdm-iface.c:1685 dm reload (253:0) NF [16384]
#ioctl/libdm-iface.c:1702 device-mapper: reload ioctl failed: Invalid argument
#libdm-deptree.c:1620 <backtrace>
#activate/dev_manager.c:1141 <backtrace>
#activate/dev_manager.c:1171 <backtrace>
#activate/activate.c:609 <backtrace>
#mm/memlock.c:118 Unlocking memory
#mm/memlock.c:137 memlock_count dec to 0
#libdm-common.c:303 Created /dev/mapper/ipwdg-ipwvol
#lvcreate.c:803 Aborting. Failed to activate new LV to wipe the start of it.
#metadata/metadata.c:2078 Finding volume group for uuid AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG
#label/label.c:270 Using cached label for /dev/sda1
#label/label.c:270 Using cached label for /dev/sda1
#format_text/format-text.c:476 Read ipwdg metadata (4) from /dev/sda1 at 7680 size 910
#metadata/metadata.c:2084 Found volume group "ipwdg"
#activate/activate.c:477 Getting device info for ipwdg-ipwvol
#ioctl/libdm-iface.c:1685 dm info LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG OF [16384]
#mm/memlock.c:99 Locking memory
#mm/memlock.c:130 memlock_count inc to 1
#activate/dev_manager.c:623 Getting device info for ipwdg-ipwvol [LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG]
#ioctl/libdm-iface.c:1685 dm info LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG OF [16384]
#ioctl/libdm-iface.c:1685 dm deps (253:0) OF [16384]
#activate/dev_manager.c:623 Getting device info for ipwdg-ipwvol-real [LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-real]
#ioctl/libdm-iface.c:1685 dm info LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-real OF [16384]
#ioctl/libdm-iface.c:1685 dm info AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-real OF [16384]
#ioctl/libdm-iface.c:1685 dm info ipwdg-ipwvol-real OF [16384]
#activate/dev_manager.c:623 Getting device info for ipwdg-ipwvol-cow [LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-cow]
#ioctl/libdm-iface.c:1685 dm info LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-cow OF [16384]
#ioctl/libdm-iface.c:1685 dm info AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-cow OF [16384]
#ioctl/libdm-iface.c:1685 dm info ipwdg-ipwvol-cow OF [16384]
#activate/dev_manager.c:623 Getting device info for ipwdg-ipwvol_mlog [LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-_mlog]
#ioctl/libdm-iface.c:1685 dm info LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-_mlog OF [16384]
#ioctl/libdm-iface.c:1685 dm info AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG-_mlog OF [16384]
#ioctl/libdm-iface.c:1685 dm info ipwdg-ipwvol_mlog OF [16384]
#ioctl/libdm-iface.c:1685 dm info (253:0) OF [16384]
#libdm-deptree.c:838 Removing ipwdg-ipwvol (253:0)
#ioctl/libdm-iface.c:1685 dm remove (253:0) NF [16384]
#libdm-common.c:615 ipwdg-ipwvol: Stacking NODE_DEL (replaces other stacked ops)
#mm/memlock.c:118 Unlocking memory
#mm/memlock.c:137 memlock_count dec to 0
#libdm-common.c:366 Removed /dev/mapper/ipwdg-ipwvol
#metadata/metadata.c:2078 Finding volume group for uuid AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG
#label/label.c:270 Using cached label for /dev/sda1
#label/label.c:270 Using cached label for /dev/sda1
#format_text/format-text.c:476 Read ipwdg metadata (4) from /dev/sda1 at 7680 size 910
#metadata/metadata.c:2084 Found volume group "ipwdg"
#activate/activate.c:477 Getting device info for ipwdg-ipwvol
#ioctl/libdm-iface.c:1685 dm info LVM-AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG NF [16384]
#ioctl/libdm-iface.c:1685 dm info AATSOLFnuaHNwRn8PhwaHtWkMjk3P7ZyZL6UNMOs5k329EWiEOjj3M1fUadxWdrG NF [16384]
#ioctl/libdm-iface.c:1685 dm info ipwdg-ipwvol NF [16384]
#metadata/pv_manip.c:272 /dev/sda1 0: 0 12800: NULL(0:0)
#metadata/pv_manip.c:272 /dev/sda1 1: 12800 63996: NULL(0:0)
#format_text/format-text.c:566 Writing ipwdg metadata to /dev/sda1 at 8704 len 656
#format_text/format-text.c:662 Pre-Committing ipwdg metadata (5) to /dev/sda1 header at 4096
#format_text/archiver.c:359 Creating volume group backup "/etc/lvm/backup/ipwdg" (seqno 5).
#format_text/format-text.c:852 Writing ipwdg metadata to /etc/lvm/backup/.lvm_ipworks-ebs02_27871_1383614257
#format_text/format-text.c:896 Committing ipwdg metadata (5)
#format_text/format-text.c:897 Renaming /etc/lvm/backup/ipwdg.tmp to /etc/lvm/backup/ipwdg
#format_text/format-text.c:662 Committing ipwdg metadata (5) to /dev/sda1 header at 4096
#device/dev-io.c:486 Closed /dev/sda1
_______________________________________________
drbd-user mailing list
drbd-user [at] lists
http://lists.linbit.com/mailman/listinfo/drbd-user


ildefonso.camargo at gmail

Jan 10, 2012, 8:40 PM

Post #5 of 10 (1888 views)
Permalink
Re: DRBD LVM device-mapper error [In reply to]

On Tue, Jan 10, 2012 at 11:04 PM, Yuan Luo <yuan.luo [at] ericsson> wrote:
> Hi,
>
> Thanks for your reply.
>
> Node2 is DRBD primary. As I mentioned before, in another set of production environment, I've successfully established DRBD(P+S) + Pacemaker and failover works perfect.
> So please assume the default DRBD configuration were correct. Ebs02 is primary, ebs01 is secondary. /dev/sda1 is a 300GB physical partition in both nodes and is expected to create a drbd.
>
> ipworks-ebs02:~ # rcdrbd status
> drbd driver loaded OK; device status:
> version: 8.3.7 (api:88/proto:86-91)
> GIT-hash: ea9e28dbff98e331a62bcbcc63a6135808fe2917 build by phil [at] fat-tyr, 2010-01-13 17:17:27
> m:res cs ro ds p mounted fstype
> 0:r0 Connected Primary/Secondary UpToDate/UpToDate C
>
> I also modified /etc/lvm/lvm.conf +line 35 as:
> filter = [ "a|^/dev/drbd|", "a/sda/", "a/sdb/", "r/.*/" ]
>
> Now the situation is, pv /dev/drbd0 created ok, vg ipwdg created ok, but lv ipwvol created failed.
> ipworks-ebs02:~ # lvcreate -n ipwvol -L 50G ipwdg
> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
> device-mapper: reload ioctl failed: Invalid argument
> Aborting. Failed to activate new LV to wipe the start of it.
>
> ipworks-ebs02:~ # pvs
> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
> PV VG Fmt Attr PSize PFree
> /dev/sda1 ipwdg lvm2 a- 299.98G 299.98G

Ok, here is the problem: it is using the underlaying device directly
(sda1)... It should be using /dev/drbd0 on primary, and volume group
should be invisible on the secondary.

There are some errors on your lvm.conf ... I have not time to dig into
details right now, but I hope this at least put you into the right
direction.

Ildefonso

>
> ipworks-ebs02:~ # pvdisplay
> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
> --- Physical volume ---
> PV Name /dev/sda1
> VG Name ipwdg
> PV Size 299.99 GB / not usable 2.62 MB
> Allocatable yes
> PE Size (KByte) 4096
> Total PE 76796
> Free PE 76796
> Allocated PE 0
> PV UUID iEopx7-vr09-FKGx-wUWq-hXLm-OVKn-4k3Lq5
>
> ipworks-ebs02:~ # vgdisplay
> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
> --- Volume group ---
> VG Name ipwdg
> System ID
> Format lvm2
> Metadata Areas 1
> Metadata Sequence No 7
> VG Access read/write
> VG Status resizable
> Clustered yes
> Shared no
> MAX LV 0
> Cur LV 0
> Open LV 0
> Max PV 0
> Cur PV 1
> Act PV 1
> VG Size 299.98 GB
> PE Size 4.00 MB
> Total PE 76796
> Alloc PE / Size 0 / 0
> Free PE / Size 76796 / 299.98 GB
> VG UUID AATSOL-Fnua-HNwR-n8Ph-waHt-WkMj-k3P7Zy
>
> lvs command output: empty
> Of course ;) no lv is created.
>
> This problem is really a little tricky. Thanks in advance.
>
> BRs,
> Yuan Luo
>
> P.S. I attached the verbose info of the lvcreate:
> The following line is suspecious, but I'm not sure about the root cause. Is it related with Kernel? But in another completely same configured environment, lv creation worked ...
_______________________________________________
drbd-user mailing list
drbd-user [at] lists
http://lists.linbit.com/mailman/listinfo/drbd-user


ildefonso.camargo at gmail

Jan 10, 2012, 10:04 PM

Post #6 of 10 (1892 views)
Permalink
Re: DRBD LVM device-mapper error [In reply to]

Hi,

I'm using LVM volume as underlaying device for DRBD, and on top of
DRBD I have LVM, so, it goes something like this:

sdaX ----- LVM (/dev/mapper/vg_name-lv_drbd_a) ---- DRBD ---- LVM
(vg_a with lv: lv_a)

Anyway, my lvm.conf has these:

filter = [ "r/.*lv_drbd.*/", "r/.*block.*/", "a/.*drbd.*/", "a/sd.*/"]
write_cache_state = 0

I did that filter while half-asleep, but it works.

On Wed, Jan 11, 2012 at 12:21 AM, Yuan Luo <yuan.luo [at] ericsson> wrote:
> Hi Jose,
>
> Thank you for your comment.
> I deleted pv and vg, and created again
> when
> ipworks-ebs02:~ # pvs
> Invalid volume group #orphans_lvm2
>
> ipworks-ebs02:~ # pvs -a
>  WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>  Invalid volume group #orphans_lvm2
>  Invalid volume group #orphans_lvm2
>  Invalid volume group (null)
>  Invalid volume group (null)
>  Invalid volume group (null)
>
> What happened? I think this is the root cause. Could you send me your lvm.conf?
> Thanks.
>
> BRs,
> Yuan Luo
>
> -----Original Message-----
> From: Jose Ildefonso Camargo Tolosa [mailto:ildefonso.camargo [at] gmail]
> Sent: 2012年1月11日 12:38
> To: Yuan Luo
> Subject: Re: [DRBD-user] DRBD LVM device-mapper error
>
> On Tue, Jan 10, 2012 at 11:04 PM, Yuan Luo <yuan.luo [at] ericsson> wrote:
>> Hi,
>>
>> Thanks for your reply.
>>
>> Node2 is DRBD primary. As I mentioned before, in another set of production environment, I've successfully established DRBD(P+S) + Pacemaker and failover works perfect.
>> So please assume the default DRBD configuration were correct. Ebs02 is primary, ebs01 is secondary. /dev/sda1 is a 300GB physical partition in both nodes and is expected to create a drbd.
>>
>> ipworks-ebs02:~ # rcdrbd status
>> drbd driver loaded OK; device status:
>> version: 8.3.7 (api:88/proto:86-91)
>> GIT-hash: ea9e28dbff98e331a62bcbcc63a6135808fe2917 build by
>> phil [at] fat-tyr, 2010-01-13 17:17:27 m:res  cs         ro
>> ds                 p  mounted  fstype 0:r0   Connected
>> Primary/Secondary  UpToDate/UpToDate  C
>>
>> I also modified  /etc/lvm/lvm.conf +line 35 as:
>> filter = [ "a|^/dev/drbd|", "a/sda/", "a/sdb/", "r/.*/" ]
>>
>> Now the situation is, pv /dev/drbd0 created ok, vg ipwdg created ok, but lv ipwvol created failed.
>> ipworks-ebs02:~ # lvcreate -n ipwvol -L 50G ipwdg
>>  WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>>  device-mapper: reload ioctl failed: Invalid argument
>>  Aborting. Failed to activate new LV to wipe the start of it.
>>
>> ipworks-ebs02:~ # pvs
>>  WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>>  PV         VG    Fmt  Attr PSize   PFree
>>  /dev/sda1  ipwdg lvm2 a-   299.98G 299.98G
>
> Ok, here is the problem: it is using the underlaying device directly (sda1)...  It should be using /dev/drbd0 on primary, and volume group should be invisible on the secondary.
>
> There are some errors on your lvm.conf ... I have not time to dig into details right now, but I hope this at least put you into the right direction.
>
> Ildefonso.
>
>>
>> ipworks-ebs02:~ # pvdisplay
>>  WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>>  --- Physical volume ---
>>  PV Name               /dev/sda1
>>  VG Name               ipwdg
>>  PV Size               299.99 GB / not usable 2.62 MB
>>  Allocatable           yes
>>  PE Size (KByte)       4096
>>  Total PE              76796
>>  Free PE               76796
>>  Allocated PE          0
>>  PV UUID               iEopx7-vr09-FKGx-wUWq-hXLm-OVKn-4k3Lq5
>>
>> ipworks-ebs02:~ # vgdisplay
>>  WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>>  --- Volume group ---
>>  VG Name               ipwdg
>>  System ID
>>  Format                lvm2
>>  Metadata Areas        1
>>  Metadata Sequence No  7
>>  VG Access             read/write
>>  VG Status             resizable
>>  Clustered             yes
>>  Shared                no
>>  MAX LV                0
>>  Cur LV                0
>>  Open LV               0
>>  Max PV                0
>>  Cur PV                1
>>  Act PV                1
>>  VG Size               299.98 GB
>>  PE Size               4.00 MB
>>  Total PE              76796
>>  Alloc PE / Size       0 / 0
>>  Free  PE / Size       76796 / 299.98 GB
>>  VG UUID               AATSOL-Fnua-HNwR-n8Ph-waHt-WkMj-k3P7Zy
>>
>> lvs command output: empty
>> Of course ;) no lv is created.
>>
>> This problem is really a little tricky. Thanks in advance.
>>
>> BRs,
>> Yuan Luo
>>
_______________________________________________
drbd-user mailing list
drbd-user [at] lists
http://lists.linbit.com/mailman/listinfo/drbd-user


yuan.luo at ericsson

Jan 11, 2012, 4:24 AM

Post #7 of 10 (1897 views)
Permalink
Re: DRBD LVM device-mapper error [In reply to]

Hi,

The problem seems be solved by cleaning the MBR of the devices on both drbd node
dd if=/dev/null of=/dev/sda1 bs=1M count=1
on both Primary and Secondary node and sync the peer.

Thanks for your help anyway. Good new is the hd works. :D

BRs,
Yuan

________________________________________
From: Jose Ildefonso Camargo Tolosa [ildefonso.camargo [at] gmail]
Sent: Wednesday, January 11, 2012 2:04 PM
To: Yuan Luo
Cc: drbd-user [at] lists
Subject: Re: [DRBD-user] DRBD LVM device-mapper error

Hi,

I'm using LVM volume as underlaying device for DRBD, and on top of
DRBD I have LVM, so, it goes something like this:

sdaX ----- LVM (/dev/mapper/vg_name-lv_drbd_a) ---- DRBD ---- LVM
(vg_a with lv: lv_a)

Anyway, my lvm.conf has these:

filter = [ "r/.*lv_drbd.*/", "r/.*block.*/", "a/.*drbd.*/", "a/sd.*/"]
write_cache_state = 0

I did that filter while half-asleep, but it works.

On Wed, Jan 11, 2012 at 12:21 AM, Yuan Luo <yuan.luo [at] ericsson> wrote:
> Hi Jose,
>
> Thank you for your comment.
> I deleted pv and vg, and created again
> when
> ipworks-ebs02:~ # pvs
> Invalid volume group #orphans_lvm2
>
> ipworks-ebs02:~ # pvs -a
> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
> Invalid volume group #orphans_lvm2
> Invalid volume group #orphans_lvm2
> Invalid volume group (null)
> Invalid volume group (null)
> Invalid volume group (null)
>
> What happened? I think this is the root cause. Could you send me your lvm.conf?
> Thanks.
>
> BRs,
> Yuan Luo
>
> -----Original Message-----
> From: Jose Ildefonso Camargo Tolosa [mailto:ildefonso.camargo [at] gmail]
> Sent: 2012111 12:38
> To: Yuan Luo
> Subject: Re: [DRBD-user] DRBD LVM device-mapper error
>
> On Tue, Jan 10, 2012 at 11:04 PM, Yuan Luo <yuan.luo [at] ericsson> wrote:
>> Hi,
>>
>> Thanks for your reply.
>>
>> Node2 is DRBD primary. As I mentioned before, in another set of production environment, I've successfully established DRBD(P+S) + Pacemaker and failover works perfect.
>> So please assume the default DRBD configuration were correct. Ebs02 is primary, ebs01 is secondary. /dev/sda1 is a 300GB physical partition in both nodes and is expected to create a drbd.
>>
>> ipworks-ebs02:~ # rcdrbd status
>> drbd driver loaded OK; device status:
>> version: 8.3.7 (api:88/proto:86-91)
>> GIT-hash: ea9e28dbff98e331a62bcbcc63a6135808fe2917 build by
>> phil [at] fat-tyr, 2010-01-13 17:17:27 m:res cs ro
>> ds p mounted fstype 0:r0 Connected
>> Primary/Secondary UpToDate/UpToDate C
>>
>> I also modified /etc/lvm/lvm.conf +line 35 as:
>> filter = [ "a|^/dev/drbd|", "a/sda/", "a/sdb/", "r/.*/" ]
>>
>> Now the situation is, pv /dev/drbd0 created ok, vg ipwdg created ok, but lv ipwvol created failed.
>> ipworks-ebs02:~ # lvcreate -n ipwvol -L 50G ipwdg
>> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>> device-mapper: reload ioctl failed: Invalid argument
>> Aborting. Failed to activate new LV to wipe the start of it.
>>
>> ipworks-ebs02:~ # pvs
>> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>> PV VG Fmt Attr PSize PFree
>> /dev/sda1 ipwdg lvm2 a- 299.98G 299.98G
>
> Ok, here is the problem: it is using the underlaying device directly (sda1)... It should be using /dev/drbd0 on primary, and volume group should be invisible on the secondary.
>
> There are some errors on your lvm.conf ... I have not time to dig into details right now, but I hope this at least put you into the right direction.
>
> Ildefonso.
>
>>
>> ipworks-ebs02:~ # pvdisplay
>> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>> --- Physical volume ---
>> PV Name /dev/sda1
>> VG Name ipwdg
>> PV Size 299.99 GB / not usable 2.62 MB
>> Allocatable yes
>> PE Size (KByte) 4096
>> Total PE 76796
>> Free PE 76796
>> Allocated PE 0
>> PV UUID iEopx7-vr09-FKGx-wUWq-hXLm-OVKn-4k3Lq5
>>
>> ipworks-ebs02:~ # vgdisplay
>> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>> --- Volume group ---
>> VG Name ipwdg
>> System ID
>> Format lvm2
>> Metadata Areas 1
>> Metadata Sequence No 7
>> VG Access read/write
>> VG Status resizable
>> Clustered yes
>> Shared no
>> MAX LV 0
>> Cur LV 0
>> Open LV 0
>> Max PV 0
>> Cur PV 1
>> Act PV 1
>> VG Size 299.98 GB
>> PE Size 4.00 MB
>> Total PE 76796
>> Alloc PE / Size 0 / 0
>> Free PE / Size 76796 / 299.98 GB
>> VG UUID AATSOL-Fnua-HNwR-n8Ph-waHt-WkMj-k3P7Zy
>>
>> lvs command output: empty
>> Of course ;) no lv is created.
>>
>> This problem is really a little tricky. Thanks in advance.
>>
>> BRs,
>> Yuan Luo
>>
_______________________________________________
drbd-user mailing list
drbd-user [at] lists
http://lists.linbit.com/mailman/listinfo/drbd-user


ildefonso.camargo at gmail

Jan 11, 2012, 5:01 AM

Post #8 of 10 (1892 views)
Permalink
Re: DRBD LVM device-mapper error [In reply to]

2012/1/11 Yuan Luo <yuan.luo [at] ericsson>:
> Hi,
>
> The problem seems be solved by cleaning the MBR of the devices on both drbd node
> dd if=/dev/null of=/dev/sda1 bs=1M count=1
> on both Primary and Secondary node and sync the peer.
>
> Thanks for your help anyway. Good new is the hd works. :D

Good, but still: your pvdisplay output shows that LVM could be
bypassing drbd, and that's not good.

>
> BRs,
> Yuan
>
> ________________________________________
> From: Jose Ildefonso Camargo Tolosa [ildefonso.camargo [at] gmail]
> Sent: Wednesday, January 11, 2012 2:04 PM
> To: Yuan Luo
> Cc: drbd-user [at] lists
> Subject: Re: [DRBD-user] DRBD LVM device-mapper error
>
> Hi,
>
> I'm using LVM volume as underlaying device for DRBD, and on top of
> DRBD I have LVM, so, it goes something like this:
>
> sdaX ----- LVM (/dev/mapper/vg_name-lv_drbd_a) ---- DRBD ---- LVM
> (vg_a with lv: lv_a)
>
> Anyway, my lvm.conf has these:
>
> filter = [ "r/.*lv_drbd.*/", "r/.*block.*/", "a/.*drbd.*/", "a/sd.*/"]
> write_cache_state = 0
>
> I did that filter while half-asleep, but it works.
>
> On Wed, Jan 11, 2012 at 12:21 AM, Yuan Luo <yuan.luo [at] ericsson> wrote:
>> Hi Jose,
>>
>> Thank you for your comment.
>> I deleted pv and vg, and created again
>> when
>> ipworks-ebs02:~ # pvs
>> Invalid volume group #orphans_lvm2
>>
>> ipworks-ebs02:~ # pvs -a
>> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>> Invalid volume group #orphans_lvm2
>> Invalid volume group #orphans_lvm2
>> Invalid volume group (null)
>> Invalid volume group (null)
>> Invalid volume group (null)
>>
>> What happened? I think this is the root cause. Could you send me your lvm.conf?
>> Thanks.
>>
>> BRs,
>> Yuan Luo
>>
>> -----Original Message-----
>> From: Jose Ildefonso Camargo Tolosa [mailto:ildefonso.camargo [at] gmail]
>> Sent: 2012111 12:38
>> To: Yuan Luo
>> Subject: Re: [DRBD-user] DRBD LVM device-mapper error
>>
>> On Tue, Jan 10, 2012 at 11:04 PM, Yuan Luo <yuan.luo [at] ericsson> wrote:
>>> Hi,
>>>
>>> Thanks for your reply.
>>>
>>> Node2 is DRBD primary. As I mentioned before, in another set of production environment, I've successfully established DRBD(P+S) + Pacemaker and failover works perfect.
>>> So please assume the default DRBD configuration were correct. Ebs02 is primary, ebs01 is secondary. /dev/sda1 is a 300GB physical partition in both nodes and is expected to create a drbd.
>>>
>>> ipworks-ebs02:~ # rcdrbd status
>>> drbd driver loaded OK; device status:
>>> version: 8.3.7 (api:88/proto:86-91)
>>> GIT-hash: ea9e28dbff98e331a62bcbcc63a6135808fe2917 build by
>>> phil [at] fat-tyr, 2010-01-13 17:17:27 m:res cs ro
>>> ds p mounted fstype 0:r0 Connected
>>> Primary/Secondary UpToDate/UpToDate C
>>>
>>> I also modified /etc/lvm/lvm.conf +line 35 as:
>>> filter = [ "a|^/dev/drbd|", "a/sda/", "a/sdb/", "r/.*/" ]
>>>
>>> Now the situation is, pv /dev/drbd0 created ok, vg ipwdg created ok, but lv ipwvol created failed.
>>> ipworks-ebs02:~ # lvcreate -n ipwvol -L 50G ipwdg
>>> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>>> device-mapper: reload ioctl failed: Invalid argument
>>> Aborting. Failed to activate new LV to wipe the start of it.
>>>
>>> ipworks-ebs02:~ # pvs
>>> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>>> PV VG Fmt Attr PSize PFree
>>> /dev/sda1 ipwdg lvm2 a- 299.98G 299.98G
>>
>> Ok, here is the problem: it is using the underlaying device directly (sda1)... It should be using /dev/drbd0 on primary, and volume group should be invisible on the secondary.
>>
>> There are some errors on your lvm.conf ... I have not time to dig into details right now, but I hope this at least put you into the right direction.
>>
>> Ildefonso.
>>
>>>
>>> ipworks-ebs02:~ # pvdisplay
>>> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>>> --- Physical volume ---
>>> PV Name /dev/sda1
>>> VG Name ipwdg
>>> PV Size 299.99 GB / not usable 2.62 MB
>>> Allocatable yes
>>> PE Size (KByte) 4096
>>> Total PE 76796
>>> Free PE 76796
>>> Allocated PE 0
>>> PV UUID iEopx7-vr09-FKGx-wUWq-hXLm-OVKn-4k3Lq5
>>>
>>> ipworks-ebs02:~ # vgdisplay
>>> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
>>> --- Volume group ---
>>> VG Name ipwdg
>>> System ID
>>> Format lvm2
>>> Metadata Areas 1
>>> Metadata Sequence No 7
>>> VG Access read/write
>>> VG Status resizable
>>> Clustered yes
>>> Shared no
>>> MAX LV 0
>>> Cur LV 0
>>> Open LV 0
>>> Max PV 0
>>> Cur PV 1
>>> Act PV 1
>>> VG Size 299.98 GB
>>> PE Size 4.00 MB
>>> Total PE 76796
>>> Alloc PE / Size 0 / 0
>>> Free PE / Size 76796 / 299.98 GB
>>> VG UUID AATSOL-Fnua-HNwR-n8Ph-waHt-WkMj-k3P7Zy
>>>
>>> lvs command output: empty
>>> Of course ;) no lv is created.
>>>
>>> This problem is really a little tricky. Thanks in advance.
>>>
>>> BRs,
>>> Yuan Luo
>>>
_______________________________________________
drbd-user mailing list
drbd-user [at] lists
http://lists.linbit.com/mailman/listinfo/drbd-user


ff at mpexnet

Jan 11, 2012, 5:09 AM

Post #9 of 10 (1880 views)
Permalink
Re: DRBD LVM device-mapper error [In reply to]

Hi,

On 01/11/2012 01:24 PM, Yuan Luo wrote:
> dd if=/dev/null of=/dev/sda1 bs=1M count=1

don't you mean /dev/zero?

/dev/sda1 is unlikely to start in a boot sector, I'm not sure about LVM
metadata though. If this wiped it, I guess you're good(?)

*If* you ever do need to wipe a boot sector, be aware that its 512
byte(!) not 1M (that's 2048 boot sectors' worth of data).

Finally, manipulating the backing devices of a live DRBD is horribly bad
practice as far as I'm concerned.

HTH,
Felix
_______________________________________________
drbd-user mailing list
drbd-user [at] lists
http://lists.linbit.com/mailman/listinfo/drbd-user


yuan.luo at ericsson

Jan 11, 2012, 4:49 PM

Post #10 of 10 (1873 views)
Permalink
Re: DRBD LVM device-mapper error [In reply to]

Hi,

Thanks for your comments. It was my typo, it is /dev/zero
And I was creating pv /dev/drbd0 instead of /dev/sda1.
And I was wrong, the 1M erased is not MBR which is related with booting, but some other header for which part I'm not sure.
But now it works. Thanks.

BRs,
Yuan Luo

-----Original Message-----
From: Felix Frank [mailto:ff [at] mpexnet]
Sent: 2012111 21:10
To: Yuan Luo
Cc: drbd-user
Subject: Re: [DRBD-user] DRBD LVM device-mapper error

Hi,

On 01/11/2012 01:24 PM, Yuan Luo wrote:
> dd if=/dev/null of=/dev/sda1 bs=1M count=1

don't you mean /dev/zero?

/dev/sda1 is unlikely to start in a boot sector, I'm not sure about LVM metadata though. If this wiped it, I guess you're good(?)

*If* you ever do need to wipe a boot sector, be aware that its 512
byte(!) not 1M (that's 2048 boot sectors' worth of data).

Finally, manipulating the backing devices of a live DRBD is horribly bad practice as far as I'm concerned.

HTH,
Felix
_______________________________________________
drbd-user mailing list
drbd-user [at] lists
http://lists.linbit.com/mailman/listinfo/drbd-user

DRBD users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.