Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Xen: Devel

no console when using xl toolstack xen 4.1.2

 

 

Xen devel RSS feed   Index | Next | Previous | View Threaded


sam at tacomatelematics

Apr 24, 2012, 1:37 AM

Post #1 of 17 (353 views)
Permalink
no console when using xl toolstack xen 4.1.2

Hello!

I was asking for help on the Freenode channel, and I was pointed here.

I have a situation where, using xl, I can create a functional PV domU, with or without pv-grub, but I cannot access the console. Firing up xend and using xm works without trouble. Since xend and company is being deprecated, I would like to transition to using the xl toolstack.

The system is an Arch Linux system running under VMWare Fusion-- I normally build and test packages this way before I test them on my dev hardware as my dev hardware is a rather loud HP server. I'm compiling Xen in a package, based off of the one in AUR but with some changes (the AUR package doesn't compile pv-grub, for instance). It's Xen 4.1.2, and my linux kernel is the standard Arch Linux kernel version 3.3.2, and it is very near the stock kernel. Clearly, I have done no HVM testing yet.


Here's what it looks like when I try xl:
__START__
[root [at] xentest201 noauto]# xl create -c finnix.cfg
Parsing config file finnix.cfg
Unable to attach console
Daemon running with PID 851
[root [at] xentest201 noauto]# xl console finnix
Unable to attach console
[root [at] xentest201 noauto]# ls
domutest.cfg finnix.cfg
[root [at] xentest201 noauto]# xl create -c domutest.cfg
Parsing config file domutest.cfg
xc: error: panic: xc_dom_bzimageloader.c:556: xc_dom_probe_bzimage_kernel: kernel is not a bzImage: Invalid kernel
Unable to attach console
Daemon running with PID 916
[root [at] xentest201 noauto]# xl console domutest
Unable to attach console
[root [at] xentest201 noauto]#
__END__

Firing up xend, it works just as you would expect.


Here are the configuration files:

finnix.cfg:
__START__
kernel = "/var/finnix/linux64"
ramdisk = "/var/finnix/initrd.xz"
name = "finnix"
memory = "128"
disk = [ 'file:/var/finnix/finnix-104.iso,xvda,r', ]
vif = [ 'bridge=outer0', ]
__END__

domutest.cfg:
__START__
kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
extra = "(hd0)/boot/grub/menu.lst"

memory = 512
vcpus = 4
name = "domutest"
disk = [ "phy:/dev/xtG0/domutest-root,xvda1,w",
"phy:/dev/xtG0/domutest-swap,xvda2,w" ]
vif = [ "bridge=outer0" ]
vfb = [ "type=vnc,vnclisten=0.0.0.0,vncdisplay=5,vncpasswd=smeghead" ]
__END__

I can see finnix ask for a DHCP address, and I can connect to domutest using the VNC vfb and use it as normal, so I've concluded that the domUs are running.


And now for some other information:

xen-hotplug.log
__START__
RTNETLINK answers: Operation not supported
RTNETLINK answers: Operation not supported
__END__

qemu-dm-finnix.log
__START__
domid: 1
Warning: vlan 0 is not connected to host network
-videoram option does not work with cirrus vga device model. Videoram set to 4M.
/home/sam/build/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:628: Init blktap pipes
/home/sam/build/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:603: Created /var/run/tap directory
Could not open /var/run/tap/qemu-read-1
char device redirected to /dev/pts/2
xs_read(): target get error. /local/domain/1/target.
(qemu) (qemu)
__END__

qemu-dm-domutest.log
__START__
domid: 2
Warning: vlan 0 is not connected to host network
-videoram option does not work with cirrus vga device model. Videoram set to 4M.
/home/sam/build/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:628: Init blktap pipes
Could not open /var/run/tap/qemu-read-2
char device redirected to /dev/pts/3
xs_read(): target get error. /local/domain/2/target.
Key lost : keysym=0xffe7(65511)
Key lost : keysym=0xffe7(65511)
__END__



Here's running xl create on domutest with the -v option:
__START__
[root [at] xentest201 noauto]# xl -v create -c domutest.cfg
Parsing config file domutest.cfg
domainbuilder: detail: xc_dom_allocate: cmdline="(hd0)/boot/grub/menu.lst", features="(null)"
domainbuilder: detail: xc_dom_kernel_file: filename="/usr/lib/xen/boot/pv-grub-x86_64.gz"
domainbuilder: detail: xc_dom_malloc_filemap : 1130 kB
domainbuilder: detail: xc_dom_malloc : 14779 kB
domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x11a833 -> 0xe6effa
domainbuilder: detail: xc_dom_boot_xen_init: ver 4.1, caps xen-3.0-x86_64 xen-3.0-x86_32p
domainbuilder: detail: xc_dom_parse_image: called
domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ...
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ...
xc: error: panic: xc_dom_bzimageloader.c:556: xc_dom_probe_bzimage_kernel: kernel is not a bzImage: Invalid kerne
l
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ...
domainbuilder: detail: loader probe OK
xc: detail: elf_parse_binary: phdr: paddr=0x0 memsz=0x99ff60
xc: detail: elf_parse_binary: memory: 0x0 -> 0x99ff60
xc: detail: elf_xen_parse: __xen_guest: "GUEST_OS=Mini-OS,XEN_VER=xen-3.0,VIRT_BASE=0x0,ELF_PADDR_OFFSET=0x0,HYPE
RCALL_PAGE=0x2,LOADER=generic"
xc: detail: elf_xen_parse_guest_info: GUEST_OS="Mini-OS"
xc: detail: elf_xen_parse_guest_info: XEN_VER="xen-3.0"
xc: detail: elf_xen_parse_guest_info: VIRT_BASE="0x0"
xc: detail: elf_xen_parse_guest_info: ELF_PADDR_OFFSET="0x0"
xc: detail: elf_xen_parse_guest_info: HYPERCALL_PAGE="0x2"
xc: detail: elf_xen_parse_guest_info: LOADER="generic"
xc: detail: elf_xen_addr_calc_check: addresses:
xc: detail: virt_base = 0x0
xc: detail: elf_paddr_offset = 0x0
xc: detail: virt_offset = 0x0
xc: detail: virt_kstart = 0x0
xc: detail: virt_kend = 0x99ff60
xc: detail: virt_entry = 0x0
xc: detail: p2m_base = 0xffffffffffffffff
domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_64: 0x0 -> 0x99ff60
domainbuilder: detail: xc_dom_mem_init: mem 512 MB, pages 0x20000 pages, 4k each
domainbuilder: detail: xc_dom_mem_init: 0x20000 pages
domainbuilder: detail: xc_dom_boot_mem_init: called
domainbuilder: detail: x86_compat: guest xen-3.0-x86_64, address size 64
domainbuilder: detail: xc_dom_malloc : 1024 kB
domainbuilder: detail: xc_dom_build_image: called
domainbuilder: detail: xc_dom_alloc_segment: kernel : 0x0 -> 0x9a0000 (pfn 0x0 + 0x9a0 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x0+0x9a0 at 0x7f18be2e9000
xc: detail: elf_load_binary: phdr 0 at 0x0x7f18be2e9000 -> 0x0x7f18bec88f60
domainbuilder: detail: xc_dom_alloc_segment: phys2mach : 0x9a0000 -> 0xaa0000 (pfn 0x9a0 + 0x100 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x9a0+0x100 at 0x7f18be1e9000
domainbuilder: detail: xc_dom_alloc_page : start info : 0xaa0000 (pfn 0xaa0)
domainbuilder: detail: xc_dom_alloc_page : xenstore : 0xaa1000 (pfn 0xaa1)
domainbuilder: detail: xc_dom_alloc_page : console : 0xaa2000 (pfn 0xaa2)
domainbuilder: detail: nr_page_tables: 0x0000ffffffffffff/48: 0x0000000000000000 -> 0x0000ffffffffffff, 1 table(s
)
domainbuilder: detail: nr_page_tables: 0x0000007fffffffff/39: 0x0000000000000000 -> 0x0000007fffffffff, 1 table(s
)
domainbuilder: detail: nr_page_tables: 0x000000003fffffff/30: 0x0000000000000000 -> 0x000000003fffffff, 1 table(s
)
domainbuilder: detail: nr_page_tables: 0x00000000001fffff/21: 0x0000000000000000 -> 0x0000000000bfffff, 6 table(s
)
domainbuilder: detail: xc_dom_alloc_segment: page tables : 0xaa3000 -> 0xaac000 (pfn 0xaa3 + 0x9 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0xaa3+0x9 at 0x7f18be1e0000
domainbuilder: detail: xc_dom_alloc_page : boot stack : 0xaac000 (pfn 0xaac)
domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0xaad000
domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0xc00000
domainbuilder: detail: xc_dom_boot_image: called
domainbuilder: detail: arch_setup_bootearly: doing nothing
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 <= matches
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p
domainbuilder: detail: xc_dom_update_guest_p2m: dst 64bit, pages 0x20000
domainbuilder: detail: clear_page: pfn 0xaa2, mfn 0x5dd0c
domainbuilder: detail: clear_page: pfn 0xaa1, mfn 0x5dd0d
domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0xaa0+0x1 at 0x7f18be1df000
domainbuilder: detail: start_info_x86_64: called
domainbuilder: detail: setup_hypercall_page: vaddr=0x2000 pfn=0x2
domainbuilder: detail: domain builder memory footprint
domainbuilder: detail: allocated
domainbuilder: detail: malloc : 15870 kB
domainbuilder: detail: anon mmap : 0 bytes
domainbuilder: detail: mapped
domainbuilder: detail: file mmap : 1130 kB
domainbuilder: detail: domU mmap : 10920 kB
domainbuilder: detail: arch_setup_bootlate: shared_info: pfn 0x0, mfn 0x587bb
domainbuilder: detail: shared_info_x86_64: called
domainbuilder: detail: vcpu_x86_64: called
domainbuilder: detail: vcpu_x86_64: cr3: pfn 0xaa3 mfn 0x5dd0b
domainbuilder: detail: launch_vm: called, ctxt=0x7fffa9ec8e90
domainbuilder: detail: xc_dom_release: called
Unable to attach console
Daemon running with PID 1395
__END__

And the output of xl info:
__START__
host : xentest2012
release : 3.3.2-1-ARCH
version : #1 SMP PREEMPT Sat Apr 14 09:48:37 CEST 2012
machine : x86_64
nr_cpus : 4
nr_nodes : 1
cores_per_socket : 1
threads_per_core : 1
cpu_mhz : 2403
hw_caps : 0febfbff:20100800:00000000:00000940:80002201:00000000:00000001:00000000
virt_caps :
total_memory : 2047
free_memory : 870
free_cpus : 0
xen_major : 4
xen_minor : 1
xen_extra : .2
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
xen_commandline : dom0_mem=512M loglvl=all guest_loglvl=all com1=115200,8n1 console=com1
cc_compiler : gcc version 4.7.0 20120414 (prerelease) (GCC)
cc_compile_by : sam
cc_compile_domain : (none)
cc_compile_date : Sun Apr 22 19:02:17 PDT 2012
xend_config_format : 4
__END__

The xl logs have a single line in them:

xl-domutest.log:
Waiting for domain domutest (domid 6) to die [pid 1696]

xl-finnix.log :
Waiting for domain finnix (domid 7) to die [pid 1763]


That's what I've got. Any pointers would help. Thanks!


-Sam
_______________________________________________
Xen-devel mailing list
Xen-devel [at] lists
http://lists.xen.org/xen-devel


Ian.Campbell at citrix

Apr 24, 2012, 3:31 AM

Post #2 of 17 (330 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Tue, 2012-04-24 at 09:37 +0100, Sam Mulvey wrote:
> Hello!
>
> I was asking for help on the Freenode channel, and I was pointed here.
>
> I have a situation where, using xl, I can create a functional PV domU,
> with or without pv-grub, but I cannot access the console. Firing up
> xend and using xm works without trouble. Since xend and company is
> being deprecated, I would like to transition to using the xl
> toolstack.

Thanks, it's good to see people trying the transition.

Are you able to try xen-unstable? A lot of this stuff (particularly
relating to pv consoles and pygrub etc) is much improved in the 4.2
version of xl and if not then we should be much more able to fix things
there than in 4.1 (where I suspect any fix would be far too invasive for
a stable backport).

Thanks,
Ian.

> The system is an Arch Linux system running under VMWare Fusion-- I
> normally build and test packages this way before I test them on my dev
> hardware as my dev hardware is a rather loud HP server. I'm
> compiling Xen in a package, based off of the one in AUR but with some
> changes (the AUR package doesn't compile pv-grub, for instance).
> It's Xen 4.1.2, and my linux kernel is the standard Arch Linux kernel
> version 3.3.2, and it is very near the stock kernel. Clearly, I have
> done no HVM testing yet.
>
>
> Here's what it looks like when I try xl:
> __START__
> [root [at] xentest201 noauto]# xl create -c finnix.cfg
> Parsing config file finnix.cfg
> Unable to attach console
> Daemon running with PID 851
> [root [at] xentest201 noauto]# xl console finnix
> Unable to attach console
> [root [at] xentest201 noauto]# ls
> domutest.cfg finnix.cfg
> [root [at] xentest201 noauto]# xl create -c domutest.cfg
> Parsing config file domutest.cfg
> xc: error: panic: xc_dom_bzimageloader.c:556: xc_dom_probe_bzimage_kernel: kernel is not a bzImage: Invalid kernel
> Unable to attach console
> Daemon running with PID 916
> [root [at] xentest201 noauto]# xl console domutest
> Unable to attach console
> [root [at] xentest201 noauto]#
> __END__
>
> Firing up xend, it works just as you would expect.
>
>
> Here are the configuration files:
>
> finnix.cfg:
> __START__
> kernel = "/var/finnix/linux64"
> ramdisk = "/var/finnix/initrd.xz"
> name = "finnix"
> memory = "128"
> disk = [ 'file:/var/finnix/finnix-104.iso,xvda,r', ]
> vif = [ 'bridge=outer0', ]
> __END__
>
> domutest.cfg:
> __START__
> kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
> extra = "(hd0)/boot/grub/menu.lst"
>
> memory = 512
> vcpus = 4
> name = "domutest"
> disk = [ "phy:/dev/xtG0/domutest-root,xvda1,w",
> "phy:/dev/xtG0/domutest-swap,xvda2,w" ]
> vif = [ "bridge=outer0" ]
> vfb = [ "type=vnc,vnclisten=0.0.0.0,vncdisplay=5,vncpasswd=smeghead" ]
> __END__
>
> I can see finnix ask for a DHCP address, and I can connect to domutest using the VNC vfb and use it as normal, so I've concluded that the domUs are running.
>
>
> And now for some other information:
>
> xen-hotplug.log
> __START__
> RTNETLINK answers: Operation not supported
> RTNETLINK answers: Operation not supported
> __END__
>
> qemu-dm-finnix.log
> __START__
> domid: 1
> Warning: vlan 0 is not connected to host network
> -videoram option does not work with cirrus vga device model. Videoram set to 4M.
> /home/sam/build/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:628: Init blktap pipes
> /home/sam/build/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:603: Created /var/run/tap directory
> Could not open /var/run/tap/qemu-read-1
> char device redirected to /dev/pts/2
> xs_read(): target get error. /local/domain/1/target.
> (qemu) (qemu)
> __END__
>
> qemu-dm-domutest.log
> __START__
> domid: 2
> Warning: vlan 0 is not connected to host network
> -videoram option does not work with cirrus vga device model. Videoram set to 4M.
> /home/sam/build/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:628: Init blktap pipes
> Could not open /var/run/tap/qemu-read-2
> char device redirected to /dev/pts/3
> xs_read(): target get error. /local/domain/2/target.
> Key lost : keysym=0xffe7(65511)
> Key lost : keysym=0xffe7(65511)
> __END__
>
>
>
> Here's running xl create on domutest with the -v option:
> __START__
> [root [at] xentest201 noauto]# xl -v create -c domutest.cfg
> Parsing config file domutest.cfg
> domainbuilder: detail: xc_dom_allocate: cmdline="(hd0)/boot/grub/menu.lst", features="(null)"
> domainbuilder: detail: xc_dom_kernel_file: filename="/usr/lib/xen/boot/pv-grub-x86_64.gz"
> domainbuilder: detail: xc_dom_malloc_filemap : 1130 kB
> domainbuilder: detail: xc_dom_malloc : 14779 kB
> domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x11a833 -> 0xe6effa
> domainbuilder: detail: xc_dom_boot_xen_init: ver 4.1, caps xen-3.0-x86_64 xen-3.0-x86_32p
> domainbuilder: detail: xc_dom_parse_image: called
> domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ...
> domainbuilder: detail: loader probe failed
> domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ...
> xc: error: panic: xc_dom_bzimageloader.c:556: xc_dom_probe_bzimage_kernel: kernel is not a bzImage: Invalid kerne
> l
> domainbuilder: detail: loader probe failed
> domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ...
> domainbuilder: detail: loader probe OK
> xc: detail: elf_parse_binary: phdr: paddr=0x0 memsz=0x99ff60
> xc: detail: elf_parse_binary: memory: 0x0 -> 0x99ff60
> xc: detail: elf_xen_parse: __xen_guest: "GUEST_OS=Mini-OS,XEN_VER=xen-3.0,VIRT_BASE=0x0,ELF_PADDR_OFFSET=0x0,HYPE
> RCALL_PAGE=0x2,LOADER=generic"
> xc: detail: elf_xen_parse_guest_info: GUEST_OS="Mini-OS"
> xc: detail: elf_xen_parse_guest_info: XEN_VER="xen-3.0"
> xc: detail: elf_xen_parse_guest_info: VIRT_BASE="0x0"
> xc: detail: elf_xen_parse_guest_info: ELF_PADDR_OFFSET="0x0"
> xc: detail: elf_xen_parse_guest_info: HYPERCALL_PAGE="0x2"
> xc: detail: elf_xen_parse_guest_info: LOADER="generic"
> xc: detail: elf_xen_addr_calc_check: addresses:
> xc: detail: virt_base = 0x0
> xc: detail: elf_paddr_offset = 0x0
> xc: detail: virt_offset = 0x0
> xc: detail: virt_kstart = 0x0
> xc: detail: virt_kend = 0x99ff60
> xc: detail: virt_entry = 0x0
> xc: detail: p2m_base = 0xffffffffffffffff
> domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_64: 0x0 -> 0x99ff60
> domainbuilder: detail: xc_dom_mem_init: mem 512 MB, pages 0x20000 pages, 4k each
> domainbuilder: detail: xc_dom_mem_init: 0x20000 pages
> domainbuilder: detail: xc_dom_boot_mem_init: called
> domainbuilder: detail: x86_compat: guest xen-3.0-x86_64, address size 64
> domainbuilder: detail: xc_dom_malloc : 1024 kB
> domainbuilder: detail: xc_dom_build_image: called
> domainbuilder: detail: xc_dom_alloc_segment: kernel : 0x0 -> 0x9a0000 (pfn 0x0 + 0x9a0 pages)
> domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x0+0x9a0 at 0x7f18be2e9000
> xc: detail: elf_load_binary: phdr 0 at 0x0x7f18be2e9000 -> 0x0x7f18bec88f60
> domainbuilder: detail: xc_dom_alloc_segment: phys2mach : 0x9a0000 -> 0xaa0000 (pfn 0x9a0 + 0x100 pages)
> domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x9a0+0x100 at 0x7f18be1e9000
> domainbuilder: detail: xc_dom_alloc_page : start info : 0xaa0000 (pfn 0xaa0)
> domainbuilder: detail: xc_dom_alloc_page : xenstore : 0xaa1000 (pfn 0xaa1)
> domainbuilder: detail: xc_dom_alloc_page : console : 0xaa2000 (pfn 0xaa2)
> domainbuilder: detail: nr_page_tables: 0x0000ffffffffffff/48: 0x0000000000000000 -> 0x0000ffffffffffff, 1 table(s
> )
> domainbuilder: detail: nr_page_tables: 0x0000007fffffffff/39: 0x0000000000000000 -> 0x0000007fffffffff, 1 table(s
> )
> domainbuilder: detail: nr_page_tables: 0x000000003fffffff/30: 0x0000000000000000 -> 0x000000003fffffff, 1 table(s
> )
> domainbuilder: detail: nr_page_tables: 0x00000000001fffff/21: 0x0000000000000000 -> 0x0000000000bfffff, 6 table(s
> )
> domainbuilder: detail: xc_dom_alloc_segment: page tables : 0xaa3000 -> 0xaac000 (pfn 0xaa3 + 0x9 pages)
> domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0xaa3+0x9 at 0x7f18be1e0000
> domainbuilder: detail: xc_dom_alloc_page : boot stack : 0xaac000 (pfn 0xaac)
> domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0xaad000
> domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0xc00000
> domainbuilder: detail: xc_dom_boot_image: called
> domainbuilder: detail: arch_setup_bootearly: doing nothing
> domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 <= matches
> domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p
> domainbuilder: detail: xc_dom_update_guest_p2m: dst 64bit, pages 0x20000
> domainbuilder: detail: clear_page: pfn 0xaa2, mfn 0x5dd0c
> domainbuilder: detail: clear_page: pfn 0xaa1, mfn 0x5dd0d
> domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0xaa0+0x1 at 0x7f18be1df000
> domainbuilder: detail: start_info_x86_64: called
> domainbuilder: detail: setup_hypercall_page: vaddr=0x2000 pfn=0x2
> domainbuilder: detail: domain builder memory footprint
> domainbuilder: detail: allocated
> domainbuilder: detail: malloc : 15870 kB
> domainbuilder: detail: anon mmap : 0 bytes
> domainbuilder: detail: mapped
> domainbuilder: detail: file mmap : 1130 kB
> domainbuilder: detail: domU mmap : 10920 kB
> domainbuilder: detail: arch_setup_bootlate: shared_info: pfn 0x0, mfn 0x587bb
> domainbuilder: detail: shared_info_x86_64: called
> domainbuilder: detail: vcpu_x86_64: called
> domainbuilder: detail: vcpu_x86_64: cr3: pfn 0xaa3 mfn 0x5dd0b
> domainbuilder: detail: launch_vm: called, ctxt=0x7fffa9ec8e90
> domainbuilder: detail: xc_dom_release: called
> Unable to attach console
> Daemon running with PID 1395
> __END__
>
> And the output of xl info:
> __START__
> host : xentest2012
> release : 3.3.2-1-ARCH
> version : #1 SMP PREEMPT Sat Apr 14 09:48:37 CEST 2012
> machine : x86_64
> nr_cpus : 4
> nr_nodes : 1
> cores_per_socket : 1
> threads_per_core : 1
> cpu_mhz : 2403
> hw_caps : 0febfbff:20100800:00000000:00000940:80002201:00000000:00000001:00000000
> virt_caps :
> total_memory : 2047
> free_memory : 870
> free_cpus : 0
> xen_major : 4
> xen_minor : 1
> xen_extra : .2
> xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p
> xen_scheduler : credit
> xen_pagesize : 4096
> platform_params : virt_start=0xffff800000000000
> xen_changeset : unavailable
> xen_commandline : dom0_mem=512M loglvl=all guest_loglvl=all com1=115200,8n1 console=com1
> cc_compiler : gcc version 4.7.0 20120414 (prerelease) (GCC)
> cc_compile_by : sam
> cc_compile_domain : (none)
> cc_compile_date : Sun Apr 22 19:02:17 PDT 2012
> xend_config_format : 4
> __END__
>
> The xl logs have a single line in them:
>
> xl-domutest.log:
> Waiting for domain domutest (domid 6) to die [pid 1696]
>
> xl-finnix.log :
> Waiting for domain finnix (domid 7) to die [pid 1763]
>
>
> That's what I've got. Any pointers would help. Thanks!
>
>
> -Sam
> _______________________________________________
> Xen-devel mailing list
> Xen-devel [at] lists
> http://lists.xen.org/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel [at] lists
http://lists.xen.org/xen-devel


stefano.stabellini at eu

Apr 24, 2012, 3:37 AM

Post #3 of 17 (333 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Tue, 24 Apr 2012, Sam Mulvey wrote:
> Hello!
>
> I was asking for help on the Freenode channel, and I was pointed here.
>
> I have a situation where, using xl, I can create a functional PV domU, with or without pv-grub, but I cannot access the console. Firing up xend and using xm works without trouble. Since xend and company is being deprecated, I would like to transition to using the xl toolstack.
>
> The system is an Arch Linux system running under VMWare Fusion-- I normally build and test packages this way before I test them on my dev hardware as my dev hardware is a rather loud HP server. I'm compiling Xen in a package, based off of the one in AUR but with some changes (the AUR package doesn't compile pv-grub, for instance). It's Xen 4.1.2, and my linux kernel is the standard Arch Linux kernel version 3.3.2, and it is very near the stock kernel. Clearly, I have done no HVM testing yet.
>
>
> Here's what it looks like when I try xl:
> __START__
> [root [at] xentest201 noauto]# xl create -c finnix.cfg
> Parsing config file finnix.cfg
> Unable to attach console
> Daemon running with PID 851
> [root [at] xentest201 noauto]# xl console finnix
> Unable to attach console
> [root [at] xentest201 noauto]# ls
> domutest.cfg finnix.cfg
> [root [at] xentest201 noauto]# xl create -c domutest.cfg
> Parsing config file domutest.cfg
> xc: error: panic: xc_dom_bzimageloader.c:556: xc_dom_probe_bzimage_kernel: kernel is not a bzImage: Invalid kernel
> Unable to attach console
> Daemon running with PID 916
> [root [at] xentest201 noauto]# xl console domutest
> Unable to attach console
> [root [at] xentest201 noauto]#
> __END__
>
> Firing up xend, it works just as you would expect.
>
>
> Here are the configuration files:
>
> finnix.cfg:
> __START__
> kernel = "/var/finnix/linux64"
> ramdisk = "/var/finnix/initrd.xz"
> name = "finnix"
> memory = "128"
> disk = [ 'file:/var/finnix/finnix-104.iso,xvda,r', ]
> vif = [ 'bridge=outer0', ]
> __END__
>
> domutest.cfg:
> __START__
> kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
> extra = "(hd0)/boot/grub/menu.lst"
>
> memory = 512
> vcpus = 4
> name = "domutest"
> disk = [ "phy:/dev/xtG0/domutest-root,xvda1,w",
> "phy:/dev/xtG0/domutest-swap,xvda2,w" ]
> vif = [ "bridge=outer0" ]
> vfb = [ "type=vnc,vnclisten=0.0.0.0,vncdisplay=5,vncpasswd=smeghead" ]
> __END__
>
> I can see finnix ask for a DHCP address, and I can connect to domutest using the VNC vfb and use it as normal, so I've concluded that the domUs are running.
>
>
> That's what I've got. Any pointers would help. Thanks!

I couldn't find any clues in the logs you posted. Maybe the output of
xenstore-ls would be useful.

_______________________________________________
Xen-devel mailing list
Xen-devel [at] lists
http://lists.xen.org/xen-devel


sam at tacomatelematics

Apr 24, 2012, 4:42 AM

Post #4 of 17 (322 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Apr 24, 2012, at 3:37 AM, Stefano Stabellini wrote:

> I couldn't find any clues in the logs you posted. Maybe the output of
> xenstore-ls would be useful.


Done and done.

tool = ""
xenstored = ""
local = ""
domain = ""
0 = ""
name = "Domain-0"
memory = ""
target = "523904"
static-max = "4294967292"
freemem-slack = "314505"
backend = ""
console = ""
1 = ""
0 = ""
frontend = "/local/domain/1/console"
frontend-id = "1"
online = "0"
state = "5"
domain = "finnix"
protocol = "vt100"
hotplug-status = "connected"
2 = ""
0 = ""
frontend = "/local/domain/2/console"
frontend-id = "2"
online = "0"
state = "5"
domain = "domutest"
protocol = "vt100"
hotplug-status = "connected"
3 = ""
0 = ""
frontend = "/local/domain/3/console"
frontend-id = "3"
online = "0"
state = "5"
domain = "finnix"
protocol = "vt100"
hotplug-status = "connected"
4 = ""
0 = ""
frontend = "/local/domain/4/console"
frontend-id = "4"
online = "0"
state = "5"
domain = "finnix"
protocol = "vt100"
hotplug-status = "connected"
5 = ""
0 = ""
frontend = "/local/domain/5/console"
frontend-id = "5"
online = "0"
state = "5"
domain = "domutest"
protocol = "vt100"
hotplug-status = "connected"
6 = ""
0 = ""
frontend = "/local/domain/6/console"
frontend-id = "6"
online = "1"
state = "4"
domain = "domutest"
protocol = "vt100"
hotplug-status = "connected"
7 = ""
0 = ""
frontend = "/local/domain/7/console"
frontend-id = "7"
online = "1"
state = "4"
domain = "finnix"
protocol = "vt100"
hotplug-status = "connected"
vfb = ""
5 = ""
0 = ""
frontend = "/local/domain/5/device/vfb/0"
frontend-id = "5"
online = "0"
state = "5"
domain = "domutest"
vnc = "1"
vnclisten = "0.0.0.0"
vncpasswd = ""
vncdisplay = "5"
vncunused = "1"
sdl = "0"
opengl = "0"
feature-resize = "1"
hotplug-status = "connected"
request-update = "1"
6 = ""
0 = ""
frontend = "/local/domain/6/device/vfb/0"
frontend-id = "6"
online = "1"
state = "4"
domain = "domutest"
vnc = "1"
vnclisten = "0.0.0.0"
vncpasswd = ""
vncdisplay = "5"
vncunused = "1"
sdl = "0"
opengl = "0"
feature-resize = "1"
hotplug-status = "connected"
request-update = "1"
vkbd = ""
5 = ""
0 = ""
frontend = "/local/domain/5/device/vkbd/0"
frontend-id = "5"
online = "0"
state = "5"
domain = "domutest"
feature-abs-pointer = "1"
hotplug-status = "connected"
6 = ""
0 = ""
frontend = "/local/domain/6/device/vkbd/0"
frontend-id = "6"
online = "1"
state = "4"
domain = "domutest"
feature-abs-pointer = "1"
hotplug-status = "connected"
vbd = ""
6 = ""
51713 = ""
frontend = "/local/domain/6/device/vbd/51713"
physical-device = "fe:2"
params = "/dev/xtG0/domutest-root"
frontend-id = "6"
online = "1"
removable = "0"
bootable = "1"
state = "4"
dev = "xvda1"
type = "phy"
mode = "w"
feature-flush-cache = "0"
discard-secure = "0"
feature-discard = "0"
feature-barrier = "0"
sectors = "6291456"
info = "0"
sector-size = "512"
51714 = ""
frontend = "/local/domain/6/device/vbd/51714"
physical-device = "fe:3"
params = "/dev/xtG0/domutest-swap"
frontend-id = "6"
online = "1"
removable = "0"
bootable = "1"
state = "4"
dev = "xvda2"
type = "phy"
mode = "w"
feature-flush-cache = "0"
discard-secure = "0"
feature-discard = "0"
feature-barrier = "0"
sectors = "1048576"
info = "0"
sector-size = "512"
vif = ""
6 = ""
0 = ""
frontend = "/local/domain/6/device/vif/0"
frontend-id = "6"
online = "1"
state = "4"
script = "/etc/xen/scripts/vif-bridge"
mac = "00:16:3e:0c:49:8e"
bridge = "outer0"
handle = "0"
feature-sg = "1"
feature-gso-tcpv4 = "1"
feature-rx-copy = "1"
feature-rx-flip = "0"
hotplug-status = "connected"
7 = ""
0 = ""
frontend = "/local/domain/7/device/vif/0"
frontend-id = "7"
online = "1"
state = "2"
script = "/etc/xen/scripts/vif-bridge"
mac = "00:16:3e:10:6a:f4"
bridge = "outer0"
handle = "0"
feature-sg = "1"
feature-gso-tcpv4 = "1"
feature-rx-copy = "1"
feature-rx-flip = "0"
hotplug-status = "connected"
qdisk = ""
7 = ""
51712 = ""
frontend = "/local/domain/7/device/vbd/51712"
params = "aio:/var/finnix/finnix-104.iso"
frontend-id = "7"
online = "1"
removable = "0"
bootable = "1"
state = "2"
dev = "xvda"
type = "tap"
mode = "r"
feature-barrier = "1"
info = "4"
sector-size = "512"
sectors = "237712"
hotplug-status = "connected"
device-model = ""
6 = ""
disable_pf = "1"
state = "running"
7 = ""
disable_pf = "1"
state = "running"
6 = ""
vm = "/vm/d7ea3f07-f491-493f-8989-749d7a76a4f4"
name = "domutest"
control = ""
shutdown = ""
platform-feature-multiprocessor-suspend = "1"
device = ""
suspend = ""
event-channel = ""
vbd = ""
51713 = ""
backend = "/local/domain/0/backend/vbd/6/51713"
backend-id = "0"
state = "4"
virtual-device = "51713"
device-type = "disk"
ring-ref = "8"
event-channel = "23"
protocol = "x86_64-abi"
51714 = ""
backend = "/local/domain/0/backend/vbd/6/51714"
backend-id = "0"
state = "4"
virtual-device = "51714"
device-type = "disk"
ring-ref = "9"
event-channel = "24"
protocol = "x86_64-abi"
vif = ""
0 = ""
backend = "/local/domain/0/backend/vif/6/0"
backend-id = "0"
state = "4"
handle = "0"
mac = "00:16:3e:0c:49:8e"
tx-ring-ref = "10"
rx-ring-ref = "11"
event-channel = "25"
request-rx-copy = "1"
feature-rx-notify = "1"
feature-sg = "1"
feature-gso-tcpv4 = "1"
vfb = ""
0 = ""
backend = "/local/domain/0/backend/vfb/6/0"
backend-id = "0"
state = "4"
page-ref = "367826"
event-channel = "27"
protocol = "x86_64-abi"
feature-update = "1"
vkbd = ""
0 = ""
backend = "/local/domain/0/backend/vkbd/6/0"
backend-id = "0"
state = "4"
request-abs-pointer = "1"
page-ref = "284662"
page-gref = "14"
event-channel = "26"
data = ""
cpu = ""
0 = ""
availability = "online"
1 = ""
availability = "online"
2 = ""
availability = "online"
3 = ""
availability = "online"
memory = ""
static-max = "524288"
target = "524288"
videoram = "0"
error = ""
drivers = ""
attr = ""
messages = ""
domid = "6"
store = ""
port = "1"
ring-ref = "369971"
console = ""
backend = "/local/domain/0/backend/console/6/0"
backend-id = "0"
limit = "1048576"
type = "ioemu"
output = "pty"
port = "2"
ring-ref = "369970"
tty = "/dev/pts/2"
vnc-port = "5905"
vnc-listen = "0.0.0.0"
vnc-pass = ""
image = ""
device-model-pid = "1684"
7 = ""
vm = "/vm/bf8f3948-e089-44fd-88be-140479840e93"
name = "finnix"
control = ""
shutdown = ""
platform-feature-multiprocessor-suspend = "1"
device = ""
suspend = ""
event-channel = ""
vbd = ""
51712 = ""
backend = "/local/domain/0/backend/qdisk/7/51712"
backend-id = "0"
state = "1"
virtual-device = "51712"
device-type = "disk"
vif = ""
0 = ""
backend = "/local/domain/0/backend/vif/7/0"
backend-id = "0"
state = "1"
handle = "0"
mac = "00:16:3e:10:6a:f4"
data = ""
cpu = ""
0 = ""
availability = "online"
memory = ""
static-max = "131072"
target = "131072"
videoram = "0"
error = ""
drivers = ""
attr = ""
messages = ""
domid = "7"
store = ""
port = "1"
ring-ref = "265286"
console = ""
backend = "/local/domain/0/backend/console/7/0"
backend-id = "0"
limit = "1048576"
type = "ioemu"
output = "pty"
port = "2"
ring-ref = "265285"
tty = "/dev/pts/3"
image = ""
device-model-pid = "1725"
vm = ""
d7ea3f07-f491-493f-8989-749d7a76a4f4 = ""
uuid = "d7ea3f07-f491-493f-8989-749d7a76a4f4"
name = "domutest"
pool_name = "Pool-0"
image = ""
ostype = "linux"
kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
cmdline = "(hd0)/boot/grub/menu.lst"
start_time = "1335231372.76"
vncpasswd = "\000"
bf8f3948-e089-44fd-88be-140479840e93 = ""
uuid = "bf8f3948-e089-44fd-88be-140479840e93"
name = "finnix"
pool_name = "Pool-0"
image = ""
ostype = "linux"
kernel = "/var/finnix/linux64"
ramdisk = "/var/finnix/initrd.xz"
cmdline = ""
start_time = "1335231379.11"


sam at tacomatelematics

Apr 24, 2012, 4:55 AM

Post #5 of 17 (274 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Apr 24, 2012, at 3:31 AM, Ian Campbell wrote:

> Thanks, it's good to see people trying the transition.
>
> Are you able to try xen-unstable? A lot of this stuff (particularly
> relating to pv consoles and pygrub etc) is much improved in the 4.2
> version of xl and if not then we should be much more able to fix things
> there than in 4.1 (where I suspect any fix would be far too invasive for
> a stable backport).
>
> Thanks,
> Ian.

I can. There's a couple things that have occurred to me that I'd like to try out, but I can certainly give it a try.

The packages I'm working on are eventually destined for a production server. I started out on this process when my production machine started crashing when I'd load up on domUs. You can see <http://www.gossamer-threads.com/lists/xen/users/241986> for more info on that one. That's running a pull from mercurial from some months ago before 4.1.2 was released, so I thought I would make fresh go of Xen, and if I still had the problem I'd feel better about filing bug reports.

So for the purposes of production blah, I thought I'd use the stable releases, but if tracking unstable is preferable, I can do that.

-Sam


Ian.Campbell at citrix

Apr 24, 2012, 5:57 AM

Post #6 of 17 (273 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Tue, 2012-04-24 at 12:55 +0100, Sam Mulvey wrote:
>
> On Apr 24, 2012, at 3:31 AM, Ian Campbell wrote:
>
> > Thanks, it's good to see people trying the transition.
> >
> > Are you able to try xen-unstable? A lot of this stuff (particularly
> > relating to pv consoles and pygrub etc) is much improved in the 4.2
> > version of xl and if not then we should be much more able to fix
> > things
> > there than in 4.1 (where I suspect any fix would be far too invasive
> > for
> > a stable backport).
> >
> > Thanks,
> > Ian.
>
> I can. There's a couple things that have occurred to me that I'd
> like to try out, but I can certainly give it a try.

Thanks.

> The packages I'm working on are eventually destined for a production
> server. I started out on this process when my production machine
> started crashing when I'd load up on domUs. You can see
> <http://www.gossamer-threads.com/lists/xen/users/241986> for more info
> on that one. That's running a pull from mercurial from some months
> ago before 4.1.2 was released,

Are you sure? The stack trace indicates that this is "Xen-4.2-unstable"
not 4.1.x. (note to self, we really ought to print the hg cset in the
panic log if it is available...)

> so I thought I would make fresh go of Xen, and if I still had the
> problem I'd feel better about filing bug reports.
>
>
> So for the purposes of production blah, I thought I'd use the stable
> releases, but if tracking unstable is preferable, I can do that.

For your production uses I think the best advice right now would be to
use 4.1 with xend (unless Stefano can diagnose your xl console issue in
the other subthread). However it would also be useful to evaluate your
use case using xen-unstable and xl so that we can be reasonably sure
that when 4.2 is released you will be able to deploy that without issue.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel [at] lists
http://lists.xen.org/xen-devel


stefano.stabellini at eu

Apr 24, 2012, 6:02 AM

Post #7 of 17 (286 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Tue, 24 Apr 2012, Sam Mulvey wrote:
> tool = ""
>  xenstored = ""
> local = ""
>  domain = ""
>   0 = ""
>    name = "Domain-0"
>    memory = ""
>     target = "523904"
>     static-max = "4294967292"
>     freemem-slack = "314505"
>    backend = ""
>     console = ""
>      1 = ""
>       0 = ""
>        frontend = "/local/domain/1/console"
>        frontend-id = "1"
>        online = "0"
>        state = "5"
>        domain = "finnix"
>        protocol = "vt100"
>        hotplug-status = "connected"
>      2 = ""
>       0 = ""
>        frontend = "/local/domain/2/console"
>        frontend-id = "2"
>        online = "0"
>        state = "5"
>        domain = "domutest"
>        protocol = "vt100"
>        hotplug-status = "connected"
>      3 = ""
>       0 = ""
>        frontend = "/local/domain/3/console"
>        frontend-id = "3"
>        online = "0"
>        state = "5"
>        domain = "finnix"
>        protocol = "vt100"
>        hotplug-status = "connected"
>      4 = ""
>       0 = ""
>        frontend = "/local/domain/4/console"
>        frontend-id = "4"
>        online = "0"
>        state = "5"
>        domain = "finnix"
>        protocol = "vt100"
>        hotplug-status = "connected"
>      5 = ""
>       0 = ""
>        frontend = "/local/domain/5/console"
>        frontend-id = "5"
>        online = "0"
>        state = "5"
>        domain = "domutest"
>        protocol = "vt100"
>        hotplug-status = "connected"
>      6 = ""
>       0 = ""
>        frontend = "/local/domain/6/console"
>        frontend-id = "6"
>        online = "1"
>        state = "4"
>        domain = "domutest"
>        protocol = "vt100"
>        hotplug-status = "connected"
>      7 = ""
>       0 = ""
>        frontend = "/local/domain/7/console"
>        frontend-id = "7"
>        online = "1"
>        state = "4"
>        domain = "finnix"
>        protocol = "vt100"
>        hotplug-status = "connected"
>     vfb = ""
>      5 = ""
>       0 = ""
>        frontend = "/local/domain/5/device/vfb/0"
>        frontend-id = "5"
>        online = "0"
>        state = "5"
>        domain = "domutest"
>        vnc = "1"
>        vnclisten = "0.0.0.0"
>        vncpasswd = ""
>        vncdisplay = "5"
>        vncunused = "1"
>        sdl = "0"
>        opengl = "0"
>        feature-resize = "1"
>        hotplug-status = "connected"
>        request-update = "1"
>      6 = ""
>       0 = ""
>        frontend = "/local/domain/6/device/vfb/0"
>        frontend-id = "6"
>        online = "1"
>        state = "4"
>        domain = "domutest"
>        vnc = "1"
>        vnclisten = "0.0.0.0"
>        vncpasswd = ""
>        vncdisplay = "5"
>        vncunused = "1"
>        sdl = "0"
>        opengl = "0"
>        feature-resize = "1"
>        hotplug-status = "connected"
>        request-update = "1"
>     vkbd = ""
>      5 = ""
>       0 = ""
>        frontend = "/local/domain/5/device/vkbd/0"
>        frontend-id = "5"
>        online = "0"
>        state = "5"
>        domain = "domutest"
>        feature-abs-pointer = "1"
>        hotplug-status = "connected"
>      6 = ""
>       0 = ""
>        frontend = "/local/domain/6/device/vkbd/0"
>        frontend-id = "6"
>        online = "1"
>        state = "4"
>        domain = "domutest"
>        feature-abs-pointer = "1"
>        hotplug-status = "connected"
>     vbd = ""
>      6 = ""
>       51713 = ""
>        frontend = "/local/domain/6/device/vbd/51713"
>        physical-device = "fe:2"
>        params = "/dev/xtG0/domutest-root"
>        frontend-id = "6"
>        online = "1"
>        removable = "0"
>        bootable = "1"
>        state = "4"
>        dev = "xvda1"
>        type = "phy"
>        mode = "w"
>        feature-flush-cache = "0"
>        discard-secure = "0"
>        feature-discard = "0"
>        feature-barrier = "0"
>        sectors = "6291456"
>        info = "0"
>        sector-size = "512"
>       51714 = ""
>        frontend = "/local/domain/6/device/vbd/51714"
>        physical-device = "fe:3"
>        params = "/dev/xtG0/domutest-swap"
>        frontend-id = "6"
>        online = "1"
>        removable = "0"
>        bootable = "1"
>        state = "4"
>        dev = "xvda2"
>        type = "phy"
>        mode = "w"
>        feature-flush-cache = "0"
>        discard-secure = "0"
>        feature-discard = "0"
>        feature-barrier = "0"
>        sectors = "1048576"
>        info = "0"
>        sector-size = "512"
>     vif = ""
>      6 = ""
>       0 = ""
>        frontend = "/local/domain/6/device/vif/0"
>        frontend-id = "6"
>        online = "1"
>        state = "4"
>        script = "/etc/xen/scripts/vif-bridge"
>        mac = "00:16:3e:0c:49:8e"
>        bridge = "outer0"
>        handle = "0"
>        feature-sg = "1"
>        feature-gso-tcpv4 = "1"
>        feature-rx-copy = "1"
>        feature-rx-flip = "0"
>        hotplug-status = "connected"
>      7 = ""
>       0 = ""
>        frontend = "/local/domain/7/device/vif/0"
>        frontend-id = "7"
>        online = "1"
>        state = "2"
>        script = "/etc/xen/scripts/vif-bridge"
>        mac = "00:16:3e:10:6a:f4"
>        bridge = "outer0"
>        handle = "0"
>        feature-sg = "1"
>        feature-gso-tcpv4 = "1"
>        feature-rx-copy = "1"
>        feature-rx-flip = "0"
>        hotplug-status = "connected"
>     qdisk = ""
>      7 = ""
>       51712 = ""
>        frontend = "/local/domain/7/device/vbd/51712"
>        params = "aio:/var/finnix/finnix-104.iso"
>        frontend-id = "7"
>        online = "1"
>        removable = "0"
>        bootable = "1"
>        state = "2"
>        dev = "xvda"
>        type = "tap"
>        mode = "r"
>        feature-barrier = "1"
>        info = "4"
>        sector-size = "512"
>        sectors = "237712"
>        hotplug-status = "connected"
>    device-model = ""
>     6 = ""
>      disable_pf = "1"
>      state = "running"
>     7 = ""
>      disable_pf = "1"
>      state = "running"
>   6 = ""
>    vm = "/vm/d7ea3f07-f491-493f-8989-749d7a76a4f4"
>    name = "domutest"
>    control = ""
>     shutdown = ""
>     platform-feature-multiprocessor-suspend = "1"
>    device = ""
>     suspend = ""
>      event-channel = ""
>     vbd = ""
>      51713 = ""
>       backend = "/local/domain/0/backend/vbd/6/51713"
>       backend-id = "0"
>       state = "4"
>       virtual-device = "51713"
>       device-type = "disk"
>       ring-ref = "8"
>       event-channel = "23"
>       protocol = "x86_64-abi"
>      51714 = ""
>       backend = "/local/domain/0/backend/vbd/6/51714"
>       backend-id = "0"
>       state = "4"
>       virtual-device = "51714"
>       device-type = "disk"
>       ring-ref = "9"
>       event-channel = "24"
>       protocol = "x86_64-abi"
>     vif = ""
>      0 = ""
>       backend = "/local/domain/0/backend/vif/6/0"
>       backend-id = "0"
>       state = "4"
>       handle = "0"
>       mac = "00:16:3e:0c:49:8e"
>       tx-ring-ref = "10"
>       rx-ring-ref = "11"
>       event-channel = "25"
>       request-rx-copy = "1"
>       feature-rx-notify = "1"
>       feature-sg = "1"
>       feature-gso-tcpv4 = "1"
>     vfb = ""
>      0 = ""
>       backend = "/local/domain/0/backend/vfb/6/0"
>       backend-id = "0"
>       state = "4"
>       page-ref = "367826"
>       event-channel = "27"
>       protocol = "x86_64-abi"
>       feature-update = "1"
>     vkbd = ""
>      0 = ""
>       backend = "/local/domain/0/backend/vkbd/6/0"
>       backend-id = "0"
>       state = "4"
>       request-abs-pointer = "1"
>       page-ref = "284662"
>       page-gref = "14"
>       event-channel = "26"
>    data = ""
>    cpu = ""
>     0 = ""
>      availability = "online"
>     1 = ""
>      availability = "online"
>     2 = ""
>      availability = "online"
>     3 = ""
>      availability = "online"
>    memory = ""
>     static-max = "524288"
>     target = "524288"
>     videoram = "0"
>    error = ""
>    drivers = ""
>    attr = ""
>    messages = ""
>    domid = "6"
>    store = ""
>     port = "1"
>     ring-ref = "369971"
>    console = ""
>     backend = "/local/domain/0/backend/console/6/0"
>     backend-id = "0"
>     limit = "1048576"
>     type = "ioemu"
>     output = "pty"
>     port = "2"
>     ring-ref = "369970"
>     tty = "/dev/pts/2"
>     vnc-port = "5905"
>     vnc-listen = "0.0.0.0"
>     vnc-pass = ""
>    image = ""
>     device-model-pid = "1684"
>   7 = ""
>    vm = "/vm/bf8f3948-e089-44fd-88be-140479840e93"
>    name = "finnix"
>    control = ""
>     shutdown = ""
>     platform-feature-multiprocessor-suspend = "1"
>    device = ""
>     suspend = ""
>      event-channel = ""
>     vbd = ""
>      51712 = ""
>       backend = "/local/domain/0/backend/qdisk/7/51712"
>       backend-id = "0"
>       state = "1"
>       virtual-device = "51712"
>       device-type = "disk"
>     vif = ""
>      0 = ""
>       backend = "/local/domain/0/backend/vif/7/0"
>       backend-id = "0"
>       state = "1"
>       handle = "0"
>       mac = "00:16:3e:10:6a:f4"
>    data = ""
>    cpu = ""
>     0 = ""
>      availability = "online"
>    memory = ""
>     static-max = "131072"
>     target = "131072"
>     videoram = "0"
>    error = ""
>    drivers = ""
>    attr = ""
>    messages = ""
>    domid = "7"
>    store = ""
>     port = "1"
>     ring-ref = "265286"
>    console = ""
>     backend = "/local/domain/0/backend/console/7/0"
>     backend-id = "0"
>     limit = "1048576"
>     type = "ioemu"
>     output = "pty"
>     port = "2"
>     ring-ref = "265285"
>     tty = "/dev/pts/3"
>    image = ""
>     device-model-pid = "1725"
> vm = ""
>  d7ea3f07-f491-493f-8989-749d7a76a4f4 = ""
>   uuid = "d7ea3f07-f491-493f-8989-749d7a76a4f4"
>   name = "domutest"
>   pool_name = "Pool-0"
>   image = ""
>    ostype = "linux"
>    kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
>    cmdline = "(hd0)/boot/grub/menu.lst"
>   start_time = "1335231372.76"
>   vncpasswd = "\000"
>  bf8f3948-e089-44fd-88be-140479840e93 = ""
>   uuid = "bf8f3948-e089-44fd-88be-140479840e93"
>   name = "finnix"
>   pool_name = "Pool-0"
>   image = ""
>    ostype = "linux"
>    kernel = "/var/finnix/linux64"
>    ramdisk = "/var/finnix/initrd.xz"
>    cmdline = ""
>   start_time = "1335231379.11"

According to xenstore the console is connected for both domains and you
should be able to access them using /dev/pts/2 and /dev/pts/3.
The only issue I can see is that the disk for "finnix" is not
connected. Maybe an disk open error?


sam at tacomatelematics

Apr 24, 2012, 7:35 AM

Post #8 of 17 (315 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Apr 24, 2012, at 6:02 AM, Stefano Stabellini wrote:

> According to xenstore the console is connected for both domains and you
> should be able to access them using /dev/pts/2 and /dev/pts/3.
> The only issue I can see is that the disk for "finnix" is not
> connected. Maybe an disk open error?

Sure enough, you are right in both cases. Connecting to the pty with screen gives me access to the console.

domutest is an archlinux instance which is using a LVM partition appears to work. The finnix domU is using an ISO file and it does not appear to find it on boot up and I can't force it to find it, so running "xl create" doesn't work in that mode as well as I thought it had. Running it with xend/xm, it works fine

Anything I should be looking at next?

-Sam


sam at tacomatelematics

Apr 24, 2012, 7:50 AM

Post #9 of 17 (315 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

> Are you sure? The stack trace indicates that this is "Xen-4.2-unstable"
> not 4.1.x. (note to self, we really ought to print the hg cset in the
> panic log if it is available)

Not that sure. I was working on this last October, and it looks like 4.1.2 was released right around the time I wrapped up the last set of packages. I was working with 4.1.1, and then I switched to unstable when I ran into trouble getting HVM going.

>
>> so I thought I would make fresh go of Xen, and if I still had the
>> problem I'd feel better about filing bug reports.
>>
>>
>> So for the purposes of production blah, I thought I'd use the stable
>> releases, but if tracking unstable is preferable, I can do that.
>
> For your production uses I think the best advice right now would be to
> use 4.1 with xend (unless Stefano can diagnose your xl console issue in
> the other subthread). However it would also be useful to evaluate your
> use case using xen-unstable and xl so that we can be reasonably sure
> that when 4.2 is released you will be able to deploy that without issue.


Suits me. I may take this opportunity to work on the Xen wiki page for Arch Linux to document start-to-finish domU configuration, so having it ready to roll for 4.2 will be helpful. I've also been submitting patches to make the Xen init scripts fit in Arch's somewhat non-standard init.


-Sam
_______________________________________________
Xen-devel mailing list
Xen-devel [at] lists
http://lists.xen.org/xen-devel


Ian.Campbell at citrix

Apr 24, 2012, 7:57 AM

Post #10 of 17 (278 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Tue, 2012-04-24 at 15:50 +0100, Sam Mulvey wrote:
> I may take this opportunity to work on the Xen wiki page for Arch
> Linux to document start-to-finish domU configuration, so having it
> ready to roll for 4.2 will be helpful.

That would be really great!

This is the page linked from http://wiki.xen.org/wiki/Distro_Resources ?
(you just missed a Xen document day yesterday, not that writing docs
should be restricted to one day a month ;-))

> I've also been submitting patches to make the Xen init scripts fit
> in Arch's somewhat non-standard init.

Sounds good, I guess you've been submitting them to Arch rather than
here (or else I've been asleep at the wheel again)?

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel [at] lists
http://lists.xen.org/xen-devel


stefano.stabellini at eu

Apr 24, 2012, 10:03 AM

Post #11 of 17 (319 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Tue, 24 Apr 2012, Sam Mulvey wrote:
> On Apr 24, 2012, at 6:02 AM, Stefano Stabellini wrote:
>
> According to xenstore the console is connected for both domains and you
> should be able to access them using /dev/pts/2 and /dev/pts/3.
> The only issue I can see is that the disk for "finnix" is not
> connected. Maybe an disk open error?
>
>
> Sure enough, you are right in both cases.   Connecting to the pty with screen gives me access to the console.   

That means there is still a bug in xl or xenconsole because "xl console"
should work.


> domutest is an archlinux instance which is using a LVM partition appears to work.   The finnix domU is using an ISO file
> and it does not appear to find it on boot up and I can't force it to find it, so running "xl create" doesn't work in that
> mode as well as I thought it had.     Running it with xend/xm, it works fine
>
> Anything I should be looking at next?

Is the file you are trying to open on NFS by any chance?
My guess is that the file open is failing, maybe because of an opening flag.
We need to add some tracing to xen_disk.c in qemu to find out.
Could you please apply the appended patch to the qemu tree and then post
the qemu log file again?

---


diff --git a/hw/xen_disk.c b/hw/xen_disk.c
index 6aebb77..5c4828d 100644
--- a/hw/xen_disk.c
+++ b/hw/xen_disk.c
@@ -584,6 +584,7 @@ static int blk_init(struct XenDevice *xendev)
int mode, qflags, have_barriers, info = 0;
char *h = NULL;

+ printf("DEBUG %s %d\n",__func__,__LINE__);
/* read xenstore entries */
if (blkdev->params == NULL) {
blkdev->params = xenstore_read_be_str(&blkdev->xendev, "params");
@@ -598,6 +599,7 @@ static int blk_init(struct XenDevice *xendev)
blkdev->filename = blkdev->params;
}
}
+ printf("DEBUG %s %d filename=%s\n",__func__,__LINE__,blkdev->filename);
if (!strcmp("aio", blkdev->fileproto))
blkdev->fileproto = "raw";
if (blkdev->mode == NULL)
@@ -608,6 +610,7 @@ static int blk_init(struct XenDevice *xendev)
blkdev->dev = xenstore_read_be_str(&blkdev->xendev, "dev");
if (blkdev->devtype == NULL)
blkdev->devtype = xenstore_read_be_str(&blkdev->xendev, "device-type");
+ printf("DEBUG %s %d protocol=%s\n",__func__,__LINE__,blkdev->fileproto);

/* do we have all we need? */
if (blkdev->params == NULL ||
@@ -616,6 +619,7 @@ static int blk_init(struct XenDevice *xendev)
blkdev->dev == NULL)
return -1;

+ printf("DEBUG %s %d\n",__func__,__LINE__);
/* read-only ? */
if (strcmp(blkdev->mode, "w") == 0) {
mode = O_RDWR;
@@ -630,6 +634,7 @@ static int blk_init(struct XenDevice *xendev)
if (blkdev->devtype && !strcmp(blkdev->devtype, "cdrom"))
info |= VDISK_CDROM;

+ printf("DEBUG %s %d\n",__func__,__LINE__);
/* init qemu block driver */
blkdev->index = (blkdev->xendev.dev - 202 * 256) / 16;
blkdev->index = drive_get_index(IF_XEN, 0, blkdev->index);
@@ -640,6 +645,7 @@ static int blk_init(struct XenDevice *xendev)
if (blkdev->bs) {
if (bdrv_open2(blkdev->bs, blkdev->filename, qflags,
bdrv_find_format(blkdev->fileproto)) != 0) {
+ printf("DEBUG %s %d errno=%d\n",__func__,__LINE__,errno);
bdrv_delete(blkdev->bs);
blkdev->bs = NULL;
}
@@ -651,6 +657,7 @@ static int blk_init(struct XenDevice *xendev)
xen_be_printf(&blkdev->xendev, 2, "get configured bdrv (cmdline setup)\n");
blkdev->bs = drives_table[blkdev->index].bdrv;
}
+ printf("DEBUG %s %d\n",__func__,__LINE__);
blkdev->file_blk = BLOCK_SIZE;
blkdev->file_size = bdrv_getlength(blkdev->bs);
if (blkdev->file_size < 0) {


sam at tacomatelematics

Apr 24, 2012, 10:34 AM

Post #12 of 17 (270 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Apr 24, 2012, at 10:03 AM, Stefano Stabellini wrote:

> Is the file you are trying to open on NFS by any chance?

No, it's an XFS filesystem on an LVM partition.

> My guess is that the file open is failing, maybe because of an opening flag.
> We need to add some tracing to xen_disk.c in qemu to find out.
> Could you please apply the appended patch to the qemu tree and then post
> the qemu log file again?

The patch didn't take I can make the changes by hand, but I want to make sure I'm using the right things. So far I'm using the 4.1.2 tarball. Is there a different tree I should be using?

-Sam


sam at tacomatelematics

Apr 24, 2012, 12:10 PM

Post #13 of 17 (267 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Apr 24, 2012, at 10:03 AM, Stefano Stabellini wrote:

> Could you please apply the appended patch to the qemu tree and then post
> the qemu log file again?


Meanwhile, here's said log with patch applied:

[root [at] xentest201 xen]# more qemu-dm-finnix.log
domid: 1
Warning: vlan 0 is not connected to host network
-videoram option does not work with cirrus vga device model. Videoram set to 4M.
/home/sam/build/debug/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:628
: Init blktap pipes
/home/sam/build/debug/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:603
: Created /var/run/tap directory
Could not open /var/run/tap/qemu-read-1
char device redirected to /dev/pts/2
DEBUG blk_init 587
DEBUG blk_init 602 filename=/var/finnix/finnix-104.iso
DEBUG blk_init 613 protocol=raw
DEBUG blk_init 622
DEBUG blk_init 636
DEBUG blk_init 659
xs_read(): target get error. /local/domain/1/target.
(qemu) (qemu)


-Sam


stefano.stabellini at eu

Apr 25, 2012, 8:50 AM

Post #14 of 17 (272 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Tue, 24 Apr 2012, Sam Mulvey wrote:
> On Apr 24, 2012, at 10:03 AM, Stefano Stabellini wrote:
>
> Could you please apply the appended patch to the qemu tree and then post
> the qemu log file again?
>
>
>
> Meanwhile, here's said log with patch applied:
>
> [root [at] xentest201 xen]# more qemu-dm-finnix.log 
> domid: 1
> Warning: vlan 0 is not connected to host network
> -videoram option does not work with cirrus vga device model. Videoram set to 4M.
> /home/sam/build/debug/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:628
> : Init blktap pipes
> /home/sam/build/debug/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:603
> : Created /var/run/tap directory
> Could not open /var/run/tap/qemu-read-1
> char device redirected to /dev/pts/2
> DEBUG blk_init 587
> DEBUG blk_init 602 filename=/var/finnix/finnix-104.iso
> DEBUG blk_init 613 protocol=raw
> DEBUG blk_init 622
> DEBUG blk_init 636
> DEBUG blk_init 659
> xs_read(): target get error. /local/domain/1/target.
> (qemu) (qemu) 

It looks like the disk was opened correctly, maybe more logging will
tell us what is going wrong.
Could you please try the following patch?



diff --git a/hw/xen_backend.c b/hw/xen_backend.c
index 64dc93a..f564667 100644
--- a/hw/xen_backend.c
+++ b/hw/xen_backend.c
@@ -50,7 +50,7 @@ const char *xen_protocol;

/* private */
static TAILQ_HEAD(XenDeviceHead, XenDevice) xendevs = TAILQ_HEAD_INITIALIZER(xendevs);
-static int debug = 0;
+static int debug = 9;

/* ------------------------------------------------------------- */


sam at tacomatelematics

Apr 25, 2012, 11:48 PM

Post #15 of 17 (270 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Apr 25, 2012, at 8:50 AM, Stefano Stabellini wrote:

> It looks like the disk was opened correctly, maybe more logging will
> tell us what is going wrong.
> Could you please try the following patch?
>


domid: 1
Warning: vlan 0 is not connected to host network
-videoram option does not work with cirrus vga device model. Videoram set to 4M.
/home/sam/build/debug/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:628: Init blktap pipes
/home/sam/build/debug/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:603: Created /var/run/tap directory
Could not open /var/run/tap/qemu-read-1
xen be: console-0: backend state: Unknown -> Initialising
char device redirected to /dev/pts/2
xen be: console-0: backend state: Initialising -> InitWait
xen be: console-0: frontend not ready, ignoring
xen be: console-0: bind evtchn port 70
xen be: console-0: ring mfn 412787, remote port 2, local port 70, limit 1048576
xen be: console-0: backend state: InitWait -> Connected
xen be: qdisk-51712: backend state: Unknown -> Initialising
xen be: qdisk-51712: frontend state: Unknown -> Initialising
DEBUG blk_init 587
DEBUG blk_init 602 filename=/var/finnix/finnix-104.iso
DEBUG blk_init 613 protocol=raw
DEBUG blk_init 622
DEBUG blk_init 636
xen be: qdisk-51712: create new bdrv (xenbus setup)
DEBUG blk_init 659
xen be: qdisk-51712: type "tap", fileproto "raw", filename "/var/finnix/finnix-104.iso", size 121708544 (116 MB)
xen be: qdisk-51712: backend state: Initialising -> InitWait
xen be: qdisk-51712: frontend not ready (yet)
xs_read(): target get error. /local/domain/1/target.
xen be: console-0: backend update: state
xen be: console-0: frontend update: tty
xen be: console-0: backend update: hotplug-status
xen be: console-0: backend update: state
xen be: console-0: backend update: state
xen be: qdisk-51712: backend update: state
xen be: qdisk-51712: frontend not ready (yet)
xen be: qdisk-51712: backend update: feature-barrier
xen be: qdisk-51712: frontend not ready (yet)
xen be: qdisk-51712: backend update: info
xen be: qdisk-51712: frontend not ready (yet)
xen be: qdisk-51712: backend update: sector-size
xen be: qdisk-51712: frontend not ready (yet)
xen be: qdisk-51712: backend update: sectors
xen be: qdisk-51712: frontend not ready (yet)
xen be: qdisk-51712: backend update: hotplug-status
xen be: qdisk-51712: frontend not ready (yet)
xen be: qdisk-51712: backend update: state
xen be: qdisk-51712: frontend not ready (yet)
xen be core: displaystate setup failed
(qemu) (qemu) xen be: console-0: backlog piling up, nobody listening?


stefano.stabellini at eu

Apr 26, 2012, 5:18 AM

Post #16 of 17 (318 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Thu, 26 Apr 2012, Sam Mulvey wrote:
> On Apr 25, 2012, at 8:50 AM, Stefano Stabellini wrote:
>
> It looks like the disk was opened correctly, maybe more logging will
> tell us what is going wrong.
> Could you please try the following patch?
>
> domid: 1
> Warning: vlan 0 is not connected to host network
> -videoram option does not work with cirrus vga device model. Videoram set to 4M.
> /home/sam/build/debug/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:628: Init blktap pipes
> /home/sam/build/debug/xen/src/xen-4.1.2/tools/ioemu-qemu-xen/hw/xen_blktap.c:603: Created /var/run/tap directory
> Could not open /var/run/tap/qemu-read-1
> xen be: console-0: backend state: Unknown -> Initialising
> char device redirected to /dev/pts/2
> xen be: console-0: backend state: Initialising -> InitWait
> xen be: console-0: frontend not ready, ignoring
> xen be: console-0: bind evtchn port 70
> xen be: console-0: ring mfn 412787, remote port 2, local port 70, limit 1048576
> xen be: console-0: backend state: InitWait -> Connected
> xen be: qdisk-51712: backend state: Unknown -> Initialising
> xen be: qdisk-51712: frontend state: Unknown -> Initialising
> DEBUG blk_init 587
> DEBUG blk_init 602 filename=/var/finnix/finnix-104.iso
> DEBUG blk_init 613 protocol=raw
> DEBUG blk_init 622
> DEBUG blk_init 636
> xen be: qdisk-51712: create new bdrv (xenbus setup)
> DEBUG blk_init 659
> xen be: qdisk-51712: type "tap", fileproto "raw", filename "/var/finnix/finnix-104.iso", size 121708544 (116 MB)
> xen be: qdisk-51712: backend state: Initialising -> InitWait
> xen be: qdisk-51712: frontend not ready (yet)
> xs_read(): target get error. /local/domain/1/target.
> xen be: console-0: backend update: state
> xen be: console-0: frontend update: tty
> xen be: console-0: backend update: hotplug-status
> xen be: console-0: backend update: state
> xen be: console-0: backend update: state
> xen be: qdisk-51712: backend update: state
> xen be: qdisk-51712: frontend not ready (yet)
> xen be: qdisk-51712: backend update: feature-barrier
> xen be: qdisk-51712: frontend not ready (yet)
> xen be: qdisk-51712: backend update: info
> xen be: qdisk-51712: frontend not ready (yet)
> xen be: qdisk-51712: backend update: sector-size
> xen be: qdisk-51712: frontend not ready (yet)
> xen be: qdisk-51712: backend update: sectors
> xen be: qdisk-51712: frontend not ready (yet)
> xen be: qdisk-51712: backend update: hotplug-status
> xen be: qdisk-51712: frontend not ready (yet)
> xen be: qdisk-51712: backend update: state
> xen be: qdisk-51712: frontend not ready (yet)
> xen be core: displaystate setup failed
> (qemu) (qemu) xen be: console-0: backlog piling up, nobody listening?

It looks like the backend (QEMU) is waiting for the client to connect,
but it never happens.
Are you sure that blkfront is compiled in the guest's kernel?
I fail to see how this could work with XenD but not with Xl.

_______________________________________________
Xen-devel mailing list
Xen-devel [at] lists
http://lists.xen.org/xen-devel


sam at tacomatelematics

Apr 26, 2012, 5:20 AM

Post #17 of 17 (271 views)
Permalink
Re: no console when using xl toolstack xen 4.1.2 [In reply to]

On Apr 26, 2012, at 5:18 AM, Stefano Stabellini wrote:

> It looks like the backend (QEMU) is waiting for the client to connect,
> but it never happens.
> Are you sure that blkfront is compiled in the guest's kernel?
> I fail to see how this could work with XenD but not with Xl.

It loads as a module in the initrd. This guest is Finnix, a livecd which explicitly supports Xen. I use it pretty frequently for testing or occasionally recovery stuff.

http://www.finnix.org/Finnix_for_VPS_providers#Xen


-Sam

Xen devel RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.