amaldonado at pictage
Apr 19, 2012, 8:28 AM
Re: Initial sync stalls forever with many drbd disks (Brian R. Hellman)
> Hi Andrew,
> On 04/13/2012 09:19 AM, Andrew Maldonado wrote:
> > Hey all,
> > I am currently running into an issue using drbd in a xen cluster
> > (managed by ganeti).
> I've worked on Ganeti clusters with 30+vms active w/ DRBD, so I know it
> works. :)
> > When adding drbd instances, I can add up to 17 without issue, but the
> > 18th instance stalls on initial sync:
> > block drbd17: peer( Primary -> Unknown ) conn( SyncTarget ->
> > Disconnecting ) pdsk( UpToDate -> DUnknown )
> > block drbd17: short read expecting header on sock: r=-512
> > block drbd17: meta connection shut down by peer.
> What does the other side say, do the logs mention why it shutdown?
> > block drbd17: asender terminated
> > block drbd17: Terminating asender thread
> > block drbd17: Connection closed
> > block drbd17: conn( Disconnecting -> StandAlone )
> > block drbd17: receiver terminated
> > block drbd17: Terminating receiver thread
> > block drbd17: disk( Inconsistent -> Diskless )
> > block drbd17: drbd_bm_resize called with capacity == 0
> > block drbd17: worker terminated
> > block drbd17: Terminating worker thread
> > I am running Centos 5 xen, drbd 8.3.8. I have tried multiple
> > kernel/drbd(8.3.2/8)/bios combinations to no avail. This behavior is
> > consistent between all nodes (currently 5). I have even changed out the
> > switch the drbd data is transferred on.
> The quick and dirty fix might be upgrading to 8.3.12
The quick and dirty fix worked!
Quickly ran into another issue....had to also increase loop back devices or xen would not start additional instances (added 'options loop max_loop=256' to modprobe.conf) because it couldn't allocate a loop device for the drbd block device.
> > Currently the xen is running with 4GB ram allocated to dom0, with over
> > 2GB free on each node.
> > Do I just have not enough ram allocated to dom0? or am I missing
> > something else.
> >From our blog site:
> "DRBD needs about 32MB RAM per TB storage for its bitmap." So unless
> you have a _really_ big volume you should be OK ;)
> Full site here: http://blogs.linbit.com/p/169/maximum-volume-size/
> Hope that helps,
Yes it did, thanks for the input.
> : Brian Hellman
> : LINBIT | "Your Way to High Availability"
> : 1-503-573-1262 | 1-877-4-LINBIT
> : Web: http://www.linbit.com
> : Twitter: http://www.linbit.com/en/twitter
> : Facebook: http://www.linbit.com/en/facebook
This message, together with any attachments, is intended only for the
use of the individual or entity to which it is addressed. It may contain
information that is confidential and prohibited from disclosure. If you
are not the intended recipient, you are hereby notified that any
dissemination or copying of this message or any attachment is strictly
prohibited. If you have received this item in error, please notify the
original sender and destroy this item, along with any attachments.
drbd-user mailing list
drbd-user [at] lists