Mailing List Archive: Xen: API

xen-api on debian not updating xenstore?

Index | Next | Previous | View Threaded

weber at bigtime-software

May 9, 2012, 10:57 AM

Post #1 of 9 (781 views)
Permalink

xen-api on debian not updating xenstore?

I've been working on getting xen-api working on Debian Wheezy. I followed the installation instructions and have gotten to the point where I can connect to the server with XenCenter and attempt to create a VM. I create a VM from the Debian Squeeze template (or any other for that matter), but I can't get it to start. The error in the log is:

Internal error: the device disappeared from xenstore (frontend (domid=1|kind=vbd|devid=51712); backend (domid=0|kind=vbd|devid=51712))

Poking around a bit I noticed that my xenstore looks suspiciously empty:

root [at] debia:/home/jweber# xenstore-ls
tool = ""
xenstored = ""
local = ""
domain = ""
0 = ""
name = "Domain-0"
memory = ""
dynamic-min = "1168384"
dynamic-max = "1168384"
mh = ""
boot-time = ""
management-mac-address = "00:04:4b:05:56:14"

Seems pretty clear that I messed something up during the install, but I couldn't find any obvious errors in the log files pointing me in the right direction. I've tried reinstalling a couple times now eventually arriving at the same blocking point. Any words of wisdom from the experts would be greatly appreciated.

Thanks,
/John

Dave.Scott at eu

May 10, 2012, 5:53 AM

Post #2 of 9 (742 views)
Permalink

Re: xen-api on debian not updating xenstore? [In reply to]

Hi,

I managed to reproduce a problem with the same symptom as yours.

Try this workaround: log into your debian dom0 and type:

$ sudo xenstore-write /local/domain/0/vm hello

When xapi adds a device it performs a sanity-check that the backend domain exists. It does this by reading the "vm" key from /local/domain/backend-domid. In the case of domain 0, this key isn't being written. Normally xapi will create all the /local/domain/0 entries:

let domain0_setup () =
with_xc_and_xs (fun xc xs ->
let already_setup = try ignore(xs.Xs.read "/local/domain/0/name"); true with _ -> false in
if not already_setup then begin
(* Write an initial neutral target in for domain 0 *)
let di = Xenctrl.domain_getinfo xc 0 in
let memory_actual_kib = Xenctrl.pages_to_kib (Int64.of_nativeint di.Xenctrl.total_memory_pages) in
(* Find domain 0's UUID *)
let uuid = Xapi_inventory.lookup Xapi_inventory._control_domain_uuid in
(* setup xenstore domain 0 for blktap, xentop (CA-24231) *)
xs.Xs.writev "/local/domain/0" [ "name", "Domain-0"; "domid", "0"; "vm", "/vm/" ^ uuid ];

I think some other entity is writing the "name" key but not the "vm" key.

In the latest unstable xapi, most of the xenstore interaction has been delegated to a new service called "xenopsd". We should move this code into xenopsd and make it more robust at the same time.

Cheers,
Dave

From: xen-api-bounces [at] lists [mailto:xen-api-bounces [at] lists] On Behalf Of John Weber
Sent: 09 May 2012 18:58
To: xen-api [at] lists
Subject: [Xen-API] xen-api on debian not updating xenstore?

I've been working on getting xen-api working on Debian Wheezy. I followed the installation instructions and have gotten to the point where I can connect to the server with XenCenter and attempt to create a VM. I create a VM from the Debian Squeeze template (or any other for that matter), but I can't get it to start. The error in the log is:

Internal error: the device disappeared from xenstore (frontend (domid=1|kind=vbd|devid=51712); backend (domid=0|kind=vbd|devid=51712))

Poking around a bit I noticed that my xenstore looks suspiciously empty:

root [at] debia:/home/jweber# xenstore-ls
tool = ""
xenstored = ""
local = ""
domain = ""
0 = ""
name = "Domain-0"
memory = ""
dynamic-min = "1168384"
dynamic-max = "1168384"
mh = ""
boot-time = ""
management-mac-address = "00:04:4b:05:56:14"

Seems pretty clear that I messed something up during the install, but I couldn't find any obvious errors in the log files pointing me in the right direction. I've tried reinstalling a couple times now eventually arriving at the same blocking point. Any words of wisdom from the experts would be greatly appreciated.

Thanks,
/John

weber at bigtime-software

May 11, 2012, 9:25 AM

Post #3 of 9 (738 views)
Permalink

Re: xen-api on debian not updating xenstore? [In reply to]

Dave,

Thanks, adding that key allowed me to create and start a VM successfully. That gets me over that hurdle.

Of course, when I reboot the server that key disappears. I don't see a xenopsd service on my system anywhere... should that get installed as part of xcp-xapi? I noticed a couple other wrinkles like I don't see any entries in the xenstore for the dom0 console and I am unable to connect to the dom0 console via xencenter. Maybe this is all related?

Thanks again,

/John

From: Dave Scott [mailto:Dave.Scott [at] eu]
Sent: Thursday, May 10, 2012 8:54 AM
To: John Weber; xen-api [at] lists
Subject: RE: xen-api on debian not updating xenstore?

Hi,

I managed to reproduce a problem with the same symptom as yours.

Try this workaround: log into your debian dom0 and type:

$ sudo xenstore-write /local/domain/0/vm hello

When xapi adds a device it performs a sanity-check that the backend domain exists. It does this by reading the "vm" key from /local/domain/backend-domid. In the case of domain 0, this key isn't being written. Normally xapi will create all the /local/domain/0 entries:

let domain0_setup () =
with_xc_and_xs (fun xc xs ->
let already_setup = try ignore(xs.Xs.read "/local/domain/0/name"); true with _ -> false in
if not already_setup then begin
(* Write an initial neutral target in for domain 0 *)
let di = Xenctrl.domain_getinfo xc 0 in
let memory_actual_kib = Xenctrl.pages_to_kib (Int64.of_nativeint di.Xenctrl.total_memory_pages) in
(* Find domain 0's UUID *)
let uuid = Xapi_inventory.lookup Xapi_inventory._control_domain_uuid in
(* setup xenstore domain 0 for blktap, xentop (CA-24231) *)
xs.Xs.writev "/local/domain/0" [ "name", "Domain-0"; "domid", "0"; "vm", "/vm/" ^ uuid ];

I think some other entity is writing the "name" key but not the "vm" key.

In the latest unstable xapi, most of the xenstore interaction has been delegated to a new service called "xenopsd". We should move this code into xenopsd and make it more robust at the same time.

Cheers,
Dave

From: xen-api-bounces [at] lists<mailto:xen-api-bounces [at] lists> [mailto:xen-api-bounces [at] lists]<mailto:[mailto:xen-api-bounces [at] lists]> On Behalf Of John Weber
Sent: 09 May 2012 18:58
To: xen-api [at] lists<mailto:xen-api [at] lists>
Subject: [Xen-API] xen-api on debian not updating xenstore?

I've been working on getting xen-api working on Debian Wheezy. I followed the installation instructions and have gotten to the point where I can connect to the server with XenCenter and attempt to create a VM. I create a VM from the Debian Squeeze template (or any other for that matter), but I can't get it to start. The error in the log is:

Internal error: the device disappeared from xenstore (frontend (domid=1|kind=vbd|devid=51712); backend (domid=0|kind=vbd|devid=51712))

Poking around a bit I noticed that my xenstore looks suspiciously empty:

root [at] debia:/home/jweber# xenstore-ls
tool = ""
xenstored = ""
local = ""
domain = ""
0 = ""
name = "Domain-0"
memory = ""
dynamic-min = "1168384"
dynamic-max = "1168384"
mh = ""
boot-time = ""
management-mac-address = "00:04:4b:05:56:14"

Seems pretty clear that I messed something up during the install, but I couldn't find any obvious errors in the log files pointing me in the right direction. I've tried reinstalling a couple times now eventually arriving at the same blocking point. Any words of wisdom from the experts would be greatly appreciated.

Thanks,
/John

mike.mcclurg at citrix

May 17, 2012, 7:09 AM

Post #4 of 9 (716 views)
Permalink

Re: xen-api on debian not updating xenstore? [In reply to]

On 11/05/12 17:25, John Weber wrote:
> Dave,
>
> Thanks, adding that key allowed me to create and start a VM
> successfully. That gets me over that hurdle.
>
> Of course, when I reboot the server that key disappears. I don�t see a
> xenopsd service on my system anywhere� should that get installed as part
> of xcp-xapi? I noticed a couple other wrinkles like I don�t see any
> entries in the xenstore for the dom0 console and I am unable to connect
> to the dom0 console via xencenter. Maybe this is all related?

xenopsd is a new daemon that doesn't exist on Debian yet. The next
release of xapi is almost complete, so we'll start work on creating
Debian packages for the new xapi soon.

Mike

_______________________________________________
Xen-api mailing list
Xen-api [at] lists
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

ptomulik at meil

Jun 28, 2012, 12:13 PM

Post #5 of 9 (629 views)
Permalink

Re: xen-api on debian not updating xenstore? [In reply to]

W dniu 11.05.2012 18:25, John Weber pisze:

> Try this workaround: log into your debian dom0 and type:
>
> $ sudo xenstore-write /local/domain/0/vm hello
>
> When xapi adds a device it performs a sanity-check [...]

Hi,

how I can "automatize" this workaround, so it works after reboot?
I've tried to put these instructions into init.d/* script and also
into rc.local but it prevents somehow xcp-squeezed from starting
properly.

--
Paweł Tomulik

_______________________________________________
Xen-api mailing list
Xen-api [at] lists
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

george.shuklin at gmail

Jun 28, 2012, 1:33 PM

Post #6 of 9 (636 views)
Permalink

Re: xen-api on debian not updating xenstore? [In reply to]

Well, 1st problem I do have around this is xcp is not starting properly
any way at system boot. I've still in research about reason and
solution, but for this exact case I think simplest solution is put this
line or in the /etc/init.d/xcp-xapi, or in /etc/rc.local.

But xcp-xapi start problems (which one is kinda hard to debug due
dependency based boot sequence) is more serious, I think.

On 28.06.2012 23:13, Pawel Tomulik wrote:
>
> W dniu 11.05.2012 18:25, John Weber pisze:
>
>> Try this workaround: log into your debian dom0 and type:
>>
>> $ sudo xenstore-write /local/domain/0/vm hello
>>
>> When xapi adds a device it performs a sanity-check [...]
>
> Hi,
>
> how I can "automatize" this workaround, so it works after reboot?
> I've tried to put these instructions into init.d/* script and also
> into rc.local but it prevents somehow xcp-squeezed from starting
> properly.
>
>

_______________________________________________
Xen-api mailing list
Xen-api [at] lists
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

ptomulik at meil

Jun 28, 2012, 2:24 PM

Post #7 of 9 (632 views)
Permalink

Re: xen-api on debian not updating xenstore? [In reply to]

/etc/rc.local doesn't work (as stated in my previous post).
After reboot, when I do:

service xcp-squeezed status

I always obtain failed (I must do service xcp-squeezed start, and the
daemon starts without problem).

The same is when I put this instruction into my own separate script,
which I start-depend on all xcp daemons (xcp-xapi, xcp-squeezed, xcp-v6d
xcp-fe). Whenever xenstore-write appear in boot scripts, the
xcp-squeezed fails to start (or maybe this is issue with status printing?).

W dniu 28.06.2012 22:33, George Shuklin pisze:
> Well, 1st problem I do have around this is xcp is not starting properly
> any way at system boot. I've still in research about reason and
> solution, but for this exact case I think simplest solution is put this
> line or in the /etc/init.d/xcp-xapi, or in /etc/rc.local.
>
> But xcp-xapi start problems (which one is kinda hard to debug due
> dependency based boot sequence) is more serious, I think.
>
> On 28.06.2012 23:13, Pawel Tomulik wrote:
>>
>> W dniu 11.05.2012 18:25, John Weber pisze:
>>
>>> Try this workaround: log into your debian dom0 and type:
>>>
>>> $ sudo xenstore-write /local/domain/0/vm hello
>>>
>>> When xapi adds a device it performs a sanity-check [...]
>>
>> Hi,
>>
>> how I can "automatize" this workaround, so it works after reboot?
>> I've tried to put these instructions into init.d/* script and also
>> into rc.local but it prevents somehow xcp-squeezed from starting
>> properly.
>>
>>
>

--
Paweł Tomulik

_______________________________________________
Xen-api mailing list
Xen-api [at] lists
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

george.shuklin at gmail

Jun 28, 2012, 3:55 PM

Post #8 of 9 (642 views)
Permalink

Re: xen-api on debian not updating xenstore? [In reply to]

Well, that exactly I'm talking about. If you after start will looks ps
aux|grep xapi, you will see 'hanged' xapi.

Anyway, I'll dig in in near time to see what happens inside.

I've glad to hear it not my personal bug or mistake.

My plan to test:

1) Disable all xcp/xen based services from boot.
2) reboot and try to bring it manually
3) If found buggy behavior - get minimal condition to reproduce
4) If not - add logging strings to every script to get boot sequence
5) repeat it manually

On 29.06.2012 01:24, Pawel Tomulik wrote:
> /etc/rc.local doesn't work (as stated in my previous post).
> After reboot, when I do:
>
> service xcp-squeezed status
>
> I always obtain failed (I must do service xcp-squeezed start, and the
> daemon starts without problem).
>
> The same is when I put this instruction into my own separate script,
> which I start-depend on all xcp daemons (xcp-xapi, xcp-squeezed,
> xcp-v6d xcp-fe). Whenever xenstore-write appear in boot scripts, the
> xcp-squeezed fails to start (or maybe this is issue with status
> printing?).
>
> W dniu 28.06.2012 22:33, George Shuklin pisze:
>> Well, 1st problem I do have around this is xcp is not starting properly
>> any way at system boot. I've still in research about reason and
>> solution, but for this exact case I think simplest solution is put this
>> line or in the /etc/init.d/xcp-xapi, or in /etc/rc.local.
>>
>> But xcp-xapi start problems (which one is kinda hard to debug due
>> dependency based boot sequence) is more serious, I think.
>>
>> On 28.06.2012 23:13, Pawel Tomulik wrote:
>>>
>>> W dniu 11.05.2012 18:25, John Weber pisze:
>>>
>>>> Try this workaround: log into your debian dom0 and type:
>>>>
>>>> $ sudo xenstore-write /local/domain/0/vm hello
>>>>
>>>> When xapi adds a device it performs a sanity-check [...]
>>>
>>> Hi,
>>>
>>> how I can "automatize" this workaround, so it works after reboot?
>>> I've tried to put these instructions into init.d/* script and also
>>> into rc.local but it prevents somehow xcp-squeezed from starting
>>> properly.
>>>
>>>
>>
>
>

_______________________________________________
Xen-api mailing list
Xen-api [at] lists
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

george.shuklin at gmail

Jul 3, 2012, 2:45 AM

Post #9 of 9 (622 views)
Permalink

Re: xen-api on debian not updating xenstore? [In reply to]

Great news: new version of xcp-xapi (1.3.2-7) seems be booting perfectly
without any hangs.

But bug with 'lack of /local/domain/0/vm' is still kinda annoying. I've
added it to /etc/rc.local

Here simple patch wich write correct value to /local/domain/0/vm:

--- /etc/init.d/xcp-xapi.old 2012-06-29 14:51:40.000000000 +0400
+++ /etc/init.d/xcp-xapi 2012-07-03 13:38:14.000000000 +0400
@@ -90,7 +90,9 @@
mkdir -p /var/run/xend/boot
mkdir -p /usr/share/xcp/packages/iso
export OCAMLRUNPARAM=b
-
+ #fix dom0 uuid in xenstore, required for vm startup in xcp-xapi
+ dom0_uuid=`/usr/lib/xcp/bin/list_domains |egrep "^ 0 \|"|awk
'{print $3}'`
+ xenstore-write /local/domain/0/vm $dom0_uuid
rm -f $XAPI_STARTUP_COOKIE $XAPI_INIT_COMPLETE_COOKIE

start-stop-daemon --start --quiet --pidfile $PIDFILE --exec
$DAEMON --test > /dev/null \

But I still fighting with
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=678923

On 29.06.2012 01:24, Pawel Tomulik wrote:
> /etc/rc.local doesn't work (as stated in my previous post).
> After reboot, when I do:
>
> service xcp-squeezed status
>
> I always obtain failed (I must do service xcp-squeezed start, and the
> daemon starts without problem).
>
> The same is when I put this instruction into my own separate script,
> which I start-depend on all xcp daemons (xcp-xapi, xcp-squeezed,
> xcp-v6d xcp-fe). Whenever xenstore-write appear in boot scripts, the
> xcp-squeezed fails to start (or maybe this is issue with status
> printing?).
>
> W dniu 28.06.2012 22:33, George Shuklin pisze:
>> Well, 1st problem I do have around this is xcp is not starting properly
>> any way at system boot. I've still in research about reason and
>> solution, but for this exact case I think simplest solution is put this
>> line or in the /etc/init.d/xcp-xapi, or in /etc/rc.local.
>>
>> But xcp-xapi start problems (which one is kinda hard to debug due
>> dependency based boot sequence) is more serious, I think.
>>
>> On 28.06.2012 23:13, Pawel Tomulik wrote:
>>>
>>> W dniu 11.05.2012 18:25, John Weber pisze:
>>>
>>>> Try this workaround: log into your debian dom0 and type:
>>>>
>>>> $ sudo xenstore-write /local/domain/0/vm hello
>>>>
>>>> When xapi adds a device it performs a sanity-check [...]
>>>
>>> Hi,
>>>
>>> how I can "automatize" this workaround, so it works after reboot?
>>> I've tried to put these instructions into init.d/* script and also
>>> into rc.local but it prevents somehow xcp-squeezed from starting
>>> properly.
>>>
>>>
>>
>
>

_______________________________________________
Xen-api mailing list
Xen-api [at] lists
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads