Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: MythTV: Users

Back-end Virtualization

 

 

First page Previous page 1 2 3 Next page Last page  View All MythTV users RSS feed   Index | Next | Previous | View Threaded


nateh at thfcom

May 9, 2012, 6:22 PM

Post #1 of 57 (2190 views)
Permalink
Back-end Virtualization

Is 'Anyone' virtualizing their backend?! I cant believe I'm the only person out there who wants to do this... If you are using a VM...please explain how ya have things setup. If you aren't and you feel inclined to contribute please add what you think the best hardware would be to use to accomplish this goal. Personally I think you have to use 64 Bit USB tuner cards/sticks (w/e ya wanna call em...) and then share the USB resource through the base platform thru to the VM. Its totally theoretical, but I have a WinTV-HVR 950 and I want to give it a go. Does Myth recognize these cards in the base code or do I have to mess around with drivers?


jon at whitear

May 9, 2012, 10:10 PM

Post #2 of 57 (2128 views)
Permalink
Re: Back-end Virtualization [In reply to]

----- Original Message -----

> Is ‘Anyone’ virtualizing their backend?! I cant believe I’m the only
> person out there who wants to do this… If you are using a VM…please
> explain how ya have things setup. If you aren’t and you feel
> inclined to contribute please add what you think the best hardware
> would be to use to accomplish this goal. Personally I think you have
> to use 64 Bit USB tuner cards/sticks (w/e ya wanna call em…) and
> then share the USB resource through the base platform thru to the
> VM. Its totally theoretical, but I have a WinTV-HVR 950 and I want
> to give it a go. Does Myth recognize these cards in the base code or
> do I have to mess around with drivers?

I think lots of people are. Mines in a KVM (and has been for a few years.) I use a network attached tuner (HD Homerun) so I have no worries about tuner hardware integration on the backend. Host hardware is somewhat irrelevant, though I have a quad core i5 w/ 12GB RAM. My host is Centos 64-bit while my guest is Ubuntu 11.10 32-bit. haven't got round to migrating the guest to 64-bit yet.

Cheers,

Jon
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


list at mcdermotts

May 9, 2012, 10:27 PM

Post #3 of 57 (2133 views)
Permalink
Re: Back-end Virtualization [In reply to]

> Is ‘Anyone’ virtualizing their backend?!

Yes, I just started an experiment with 0.25.

I have both a HDPVR and a HDhomerun. The HDPVR behaves no better or
worse than outside of a virtualized environment - translation: it
crashes just as often (make sure to use the recommended firmware, I
had enormous problems with the latest firmware and downgraded.) I did
pass the USB port through to the guest OS.

The only strange part that I have really come across are quirks with
ESXi 5 and some unexplained behaviour using a MCE USB blaster with
LIRC. I changed to using the internal HRPVR blaster, so I never dug
into the LIRC strangeness.

I hooked up a PCIE raid controller via pass-trough to the OS so I'd
have raw disk accessible to the backend.

ESXi has been a bit of a disappointment, although it has been more
stable that other experiments I've done with virtual box. ESXi does
do a lot of stuff well. But there is a lot of stuff it does that has
me shaking my head too. For example, only a windows client is
available for the control software, the console access method is being
deprecated, the auto power on for VM's does not work with ESXi 5 patch
1 (you have to hack it internally via unsupported console methods),
sometimes, when placing devices on pass-trough to the client OS, you
cannot revert the change without again hacking in the unsupported
console. Lastly, ESXi does not appear to have a disk health
monitoring feature. You're disk could be crying for help via SMART,
but ESXi won't listen. Puzzling... you get what you pay for I guess.
Oh, and I almost forgot, when you want to clone a VM, you have to
manually copy the files in the datastore - should be a button. (to be
fair, I believe that other management elements alongside of vsphere
can clone easily.)

I also had a problem with custom power policy control within ESXi. My
profile caused ESXi to purple screen when cores were shutdown
unexpectedly - or at least that is my interpretation.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


linux at thehobsons

May 9, 2012, 11:50 PM

Post #4 of 57 (2127 views)
Permalink
Re: Back-end Virtualization [In reply to]

Nathan Hawkins wrote:

>Is 'Anyone' virtualizing their backend?! I cant
>believe I'm the only person out there who wants
>to do thisŠ If you are using a VMŠplease explain
>how ya have things setup.

I used to, but now I have dedicated hardware.

My first foray into MythTV was to run up a Xen PV
guest on my existing server - AMD64 3200+, 4G
RAM, single disk. Did PCI passthrough for a
single DVB-T tuner (HVR 1200 or 1250, something
like that, it's been a while now). Storage was as
"just another LVM volume" on the host.

As you can imagine, this had significant
constraints ! It certainly worked, I could record
2 programs while watching something else as long
as I didn't commflag and there'd still be hiccups
when "other stuff" (such as my internal mail
server) on the machine got busy. I have to add
that during much of this time, we were pre-DSO*,
and we had an old aerial on old cable, so calm
summer evenings we'd lose one or two muxes
altogether for an hour or so either side of
sunset - and it was sometimes difficult to decide
whether it was my setup of the poor signal at
fault for bad recordings.
But it worked, I ran it for some years, and my
only investment to get it going was the tuner
card. Making allowances for the resources I
didn't give it, I was very happy with it.

* DSO = Digital Switch Over. Over in the UK we've
been going through a process of switching from
analogue to digital. Initially the digital muxes
were transmitted as quite low power so as not to
interfere with the analogue channels - leading to
lots of problems if the aerial was anything at
all lacking in performance.

Fast forward to last year.
HP had a cashback offer on their Microservers -
at the time, N31L AMD64 dual core, low power,
room for four SATA drives. With cashback, these
were £140. My plan was to run one box, Xen, and
have MythTV as a PV guest again (but with
dedicated space for storage). However, I found
that with anything above 4GB or RAM in the
system, the HVR1300 tuner card didn't work
properly.
As the cashback offer was still on, and with the
low power two of these would still use less than
my old box, I bought a second - one runs MythTV
backend, the other runs everything else.

Had I been using an external tuner (such as the
HD HomeRun which now has a DVB-T version) then I
might well have ended up with the backend as a
guest under Xen.

--
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


support at drdos

May 10, 2012, 2:44 AM

Post #5 of 57 (2124 views)
Permalink
Re: Back-end Virtualization [In reply to]

A McDermott wrote:
> ESXi has been a bit of a disappointment, although it has been more
> stable that other experiments I've done with virtual box. ESXi does
> do a lot of stuff well.

I agree with everything said. But, for the most part, I'm happy with my
installation. And, mythbackend with PCI Passthough works well for me.

Doug


--
Ben Franklin quote:

"Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety."

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


joe at joenyland

May 10, 2012, 5:46 AM

Post #6 of 57 (2117 views)
Permalink
Re: Back-end Virtualization [In reply to]

-----Original message-----
> From:Doug Lytle <support [at] drdos>
> Sent: Thu 10-May-2012 10:45
> To: Discussion about MythTV <mythtv-users [at] mythtv>
> Subject: Re: [mythtv-users] Back-end Virtualization
>
> I agree with everything said. But, for the most part, I'm happy with my
> installation. And, mythbackend with PCI Passthough works well for me.
>
> Doug

Doug,

I'm running an ESXi 4.1 host at the moment. When you say PCI passthrough works, I'm presuming you mean PCI passthrough of your TV card? If so, what card are you successfully passing through?

I'm looking to do the same thing, but there have been a few reports on the VMware forums that passthrough of video devices doesn't work with ESXi. If you've had some success, I'd like to give it a go, to try to avoid the need for my HDHR and/or a USB type TV card.

Thanks,

Joe
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


rob.verduijn at gmail

May 10, 2012, 6:15 AM

Post #7 of 57 (2116 views)
Permalink
Re: Back-end Virtualization [In reply to]

2012/5/10 Nathan Hawkins <nateh [at] thfcom>:
> Is ‘Anyone’ virtualizing their backend?! I cant believe I’m the only person
> out there who wants to do this… If you are using a VM…please explain how ya
> have things setup. If you aren’t and you feel inclined to contribute please
> add what you think the best hardware would be to use to accomplish this
> goal. Personally I think you have to use 64 Bit USB tuner cards/sticks (w/e
> ya wanna call em…) and then share the USB resource through the base platform
> thru to the VM. Its totally theoretical, but I have a WinTV-HVR 950 and I
> want to give it a go. Does Myth recognize these cards in the base code or do
> I have to mess around with drivers?
>
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://www.mythtv.org/mailman/listinfo/mythtv-users
>

Hello,

I've also tried virtualing my backend with kvm, all worked nice untill
I tried to pass through my hauppauge 150 pci card to the guest.
My host kernel paniced shortly after I booted the guest and in the
best case the virt-manager crapped out with a python error.
Anybody who had more luck with kvm and hauppauge ?

Cheers
Rob
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


support at drdos

May 10, 2012, 6:20 AM

Post #8 of 57 (2119 views)
Permalink
Re: Back-end Virtualization [In reply to]

>> I'm presuming you mean PCI passthrough of your TV card? If so, what card are you successfully passing through?

PVR-500 PCI card


Doug

--
Ben Franklin quote:

"Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety."
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


joe at joenyland

May 10, 2012, 6:27 AM

Post #9 of 57 (2116 views)
Permalink
Re: Back-end Virtualization [In reply to]

-----Original message-----
> From:Doug Lytle <support [at] drdos>
> Sent: Thu 10-May-2012 14:20
> To: Discussion about MythTV <mythtv-users [at] mythtv>
> Subject: Re: [mythtv-users] Back-end Virtualization
>
> >> I'm presuming you mean PCI passthrough of your TV card? If so, what card are you successfully passing through?
>
> PVR-500 PCI card
>
>
> Doug

Thanks Doug. I might give this a go some day.

Cheers,

Joe
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


memmott at gmail

May 10, 2012, 7:09 AM

Post #10 of 57 (2112 views)
Permalink
Re: Back-end Virtualization [In reply to]

On Wed, May 9, 2012 at 9:22 PM, Nathan Hawkins <nateh [at] thfcom> wrote:

> Is ‘Anyone’ virtualizing their backend?! I cant believe I’m the only
> person out there who wants to do this… If you are using a VM…please explain
> how ya have things setup. If you aren’t and you feel inclined to contribute
> please add what you think the best hardware would be to use to accomplish
> this goal. Personally I think you have to use 64 Bit USB tuner cards/sticks
> (w/e ya wanna call em…) and then share the USB resource through the base
> platform thru to the VM. Its totally theoretical, but I have a WinTV-HVR
> 950 and I want to give it a go. Does Myth recognize these cards in the base
> code or do I have to mess around with drivers?****
>
> _______________________________________________
>

My BE started life as a dual Opteron server, which I have P to V'ed and am
now running as a VM on ESXi 5. My tuner is an HDHomeRun Prime. The box is
currently Ubuntu 11.04 with 0.24-1 I think. I plan on upgrading it to
Mythbuntu 12.04 shortly. My storage an an Apple Xserve RAID connected over
fiber. No issues whatsoever.


acstadt at stadt

May 10, 2012, 11:39 AM

Post #11 of 57 (2107 views)
Permalink
Re: Back-end Virtualization [In reply to]

On 10/05/2012 9:15 AM, Rob Verduijn wrote:
> 2012/5/10 Nathan Hawkins<nateh [at] thfcom>:
>> Is ‘Anyone’ virtualizing their backend?! I cant believe I’m the only person
>> out there who wants to do this… If you are using a VM…please explain how ya
>> have things setup. If you aren’t and you feel inclined to contribute please
>> add what you think the best hardware would be to use to accomplish this
>> goal. Personally I think you have to use 64 Bit USB tuner cards/sticks (w/e
>> ya wanna call em…) and then share the USB resource through the base platform
>> thru to the VM. Its totally theoretical, but I have a WinTV-HVR 950 and I
>> want to give it a go. Does Myth recognize these cards in the base code or do
>> I have to mess around with drivers?
>>
>>
>> _______________________________________________
>> mythtv-users mailing list
>> mythtv-users [at] mythtv
>> http://www.mythtv.org/mailman/listinfo/mythtv-users
>>
> Hello,
>
> I've also tried virtualing my backend with kvm, all worked nice untill
> I tried to pass through my hauppauge 150 pci card to the guest.
> My host kernel paniced shortly after I booted the guest and in the
> best case the virt-manager crapped out with a python error.
> Anybody who had more luck with kvm and hauppauge ?
>
> Cheers
> Rob
>
Actually it was the 150 that did me in as well.

Andrew.

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


zarthan at gmail

May 10, 2012, 12:13 PM

Post #12 of 57 (2097 views)
Permalink
Re: Back-end Virtualization [In reply to]

On Thu, May 10, 2012 at 1:39 PM, Andrew Stadt <acstadt [at] stadt> wrote:

> On 10/05/2012 9:15 AM, Rob Verduijn wrote:
>
>> 2012/5/10 Nathan Hawkins<nateh [at] thfcom>:
>>
>>> Is ‘Anyone’ virtualizing their backend?! I cant believe I’m the only
>>> person
>>> out there who wants to do this… If you are using a VM…please explain how
>>> ya
>>> have things setup. If you aren’t and you feel inclined to contribute
>>> please
>>> add what you think the best hardware would be to use to accomplish this
>>> goal. Personally I think you have to use 64 Bit USB tuner cards/sticks
>>> (w/e
>>> ya wanna call em…) and then share the USB resource through the base
>>> platform
>>> thru to the VM. Its totally theoretical, but I have a WinTV-HVR 950 and I
>>> want to give it a go. Does Myth recognize these cards in the base code
>>> or do
>>> I have to mess around with drivers?
>>>
>>>
>>> ______________________________**_________________
>>> mythtv-users mailing list
>>> mythtv-users [at] mythtv
>>> http://www.mythtv.org/mailman/**listinfo/mythtv-users<http://www.mythtv.org/mailman/listinfo/mythtv-users>
>>>
>>> Hello,
>>
>> I've also tried virtualing my backend with kvm, all worked nice untill
>> I tried to pass through my hauppauge 150 pci card to the guest.
>> My host kernel paniced shortly after I booted the guest and in the
>> best case the virt-manager crapped out with a python error.
>> Anybody who had more luck with kvm and hauppauge ?
>>
>> Cheers
>> Rob
>>
>> Actually it was the 150 that did me in as well.
>
> Andrew.
>
>
I do run Myth BE in a VM for testing but I use an HDhomerun (a network
attached tuner) so there are no issues with passthrough. I would highly
recommend the HDHR especially in a virtual setting. As long as there is
sufficient always available CPU and RAM you shouldn't have issues. If you
have multiple VMs that can potentially fight for disk IO you should
consider a hardware RAID controller and if possible split the array.


philledwards at gmail

May 11, 2012, 3:33 AM

Post #13 of 57 (2083 views)
Permalink
Re: Back-end Virtualization [In reply to]

Is anyone here using Xenserver or are all the success stories on VMware?


rob.verduijn at gmail

May 11, 2012, 6:31 AM

Post #14 of 57 (2081 views)
Permalink
Re: Back-end Virtualization [In reply to]

2012/5/11 Phill Edwards <philledwards [at] gmail>:
> Is anyone here using Xenserver or are all the success stories on VMware?
>
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://www.mythtv.org/mailman/listinfo/mythtv-users
>

Well I don't know, some people are a bit vague...they say they have
suc6 but either neglect to mention what hardware they use or what
virtualization is applied.

I could not do pci assignment of my tuner to the guest using the
following hardware/software.
'pci passtrhough' is called 'pci assignment' on kvm.

Virtualization : KVM
Tuner : Hauppauge wintv-pvr-150
Proc : AMD FX(tm)-8120 Eight-Core Processor
MotherBoard : M5A78L-M/USB3
Ram : 16Gb

Rob
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


raymond at wagnerrp

May 11, 2012, 6:35 AM

Post #15 of 57 (2077 views)
Permalink
Re: Back-end Virtualization [In reply to]

On 5/11/2012 09:31, Rob Verduijn wrote:
> Proc : AMD FX(tm)-8120 Eight-Core Processor

Err... four module, eight integer. You cannot compare core to core once
you bring a Bulldozer into the equation, as it has no structure
equivalent to a core.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


nateh at thfcom

May 11, 2012, 8:02 AM

Post #16 of 57 (2074 views)
Permalink
Re: Back-end Virtualization [In reply to]

The Virtual software I have standardized on personally/professionally is Oracle's Virtual Box. It's free and unbelievably well supported. If you are having issues (I think its better than VMWare...) give it a go. The hardware I tend to build for my VM boxes are Intel quad cores with as much memory as my motherboards will handle (typically Kingston or Viking with the aluminum exterior heat sinks). Motherboards are all ASUS (great products, very well supported, last literally years) and I usually go with the absolute cheapest SATA 7200 64 MB cache HDDs I can find but brand new and then set them up in a RAID 5 volume so even if one fails I don't lose my data.

From: mythtv-users-bounces [at] mythtv [mailto:mythtv-users-bounces [at] mythtv] On Behalf Of Phill Edwards
Sent: Friday, May 11, 2012 5:33 AM
To: Discussion about MythTV
Subject: Re: [mythtv-users] Back-end Virtualization


Is anyone here using Xenserver or are all the success stories on VMware?


zarthan at gmail

May 11, 2012, 8:22 AM

Post #17 of 57 (2070 views)
Permalink
Re: Back-end Virtualization [In reply to]

On Fri, May 11, 2012 at 10:02 AM, Nathan Hawkins <nateh [at] thfcom> wrote:

> The Virtual software I have standardized on personally/professionally is
> Oracle’s Virtual Box. It’s free and unbelievably well supported. If you are
> having issues (I think its better than VMWare…) give it a go. The hardware
> I tend to build for my VM boxes are Intel quad cores with as much memory as
> my motherboards will handle (typically Kingston or Viking with the aluminum
> exterior heat sinks). Motherboards are all ASUS (great products, very well
> supported, last literally years) and I usually go with the absolute
> cheapest SATA 7200 64 MB cache HDDs I can find but brand new and then set
> them up in a RAID 5 volume so even if one fails I don’t lose my data.****
>
> ** **
>
> *From:* mythtv-users-bounces [at] mythtv [mailto:
> mythtv-users-bounces [at] mythtv] *On Behalf Of *Phill Edwards
> *Sent:* Friday, May 11, 2012 5:33 AM
> *To:* Discussion about MythTV
>
> *Subject:* Re: [mythtv-users] Back-end Virtualization****
>
> ** **
>
> Is anyone here using Xenserver or are all the success stories on VMware?**
> **
>

I assume you already have the tuner. It only takes a few minutes to install
a Myth VM especially if you use one of the myth enabled distros like
Mythbuntu so give it a try. If you don't already have the tuner HDhomerun
totally eliminates any passthrough issues.


raymond at wagnerrp

May 11, 2012, 8:51 AM

Post #18 of 57 (2069 views)
Permalink
Re: Back-end Virtualization [In reply to]

On 5/9/2012 21:22, Nathan Hawkins wrote:
> Is ‘Anyone’ virtualizing their backend?! I cant believe I’m the only
> person out there who wants to do this…

I've stated my views on virtualization on this mailing list multiple
times in the past, but new people keep joining and asking, so here goes...

Why do you want to want to virtualize your backend? "Because I think it
would be interesting" is a perfectly valid reason here. "Because the
industry uses it and VM vendors tell me it magically makes everything
better" is not. So why does industry use it?

The first reason would be security. There are many mechanisms to
isolate different processes on a single system, but none of them can be
as complete or absolute as full system virtualization. You can run
different application servers, or servers for multiple departments with
different user access rights on one physical system, and if one gets
compromised, there is (almost) no risk to the rest. It can be flushed
and restored from a clean copy with no harm to the other services. If
you are looking to virtualize MythTV for security purposes, your journey
ends here. If you are not running MythTV on a safe, trusted network,
you either shouldn't be running MythTV, or you should be looking into
adding cryptographic authentication into all the various communication
interfaces MythTV uses.

The second reason is high availability. The virtual machine allows you
to save the state of the machine, and in the event of a failure, resume
that state on another piece of hardware. This is really only a crude
route to high availability, as such capability is much more effectively
and efficiently performed by the application itself, such as MySQL
clustering and replication servers. It becomes a question of how
valuable is the application to your needs, and is it valuable enough to
warrant the time and expense developing native support in the
application. If your interest is in HA, you've already thrown far more
money at virtual machines than someone who would be asking a basic
question such as this on an open source mailing list.

And that's it. Those are the two good reasons to run virtual machines
in a production environment.


-- Counter Argument 1 --
What about the need to run on a different hardware architecture or
operating system? Don't. Find software that suits your needs that can
all run on one OS. If it's something critical that you cannot do
without, perhaps that makes it worth dedicating another machine to.


-- Counter Argument 2 --
What about software that refuses to run on anything outside a
pre-defined hardware set, so you define it against a virtual machine to
allow the VM to be mobile? Shame on the company for putting such
restrictive licensing measures on their software. Double shame for
botching it in such a manner that it could be bypassed by simply using a
VM. From an idealist standpoint, shame on the customer for patronizing
such an abusive developer.


-- Counter Argument 3 --
What about software that could crash and take out a system? If you're
doing development, this is a great thing. Any application that you
expect to crash with destructive results should not be used in a
production environment.


-- Counter Argument 4 --
What about ease of management? When trying to run multiple applications
and servers on a single system, you may run into dependency conflicts.
You may update one library for one application, only to find it has
broken another application. Virtual machines let you run multiple
independent installs, with independent dependency sets, to avoid these
issues. This is the big one, and I believe the reason most people
improperly use virtual machines.

This has NOTHING TO DO with virtualization. Virtual machines require
this behavior, but this behavior does not require virtual machines. You
can do the SAME EXACT THING with a simple chroot, without all the
unnecessary complexity and overhead of running a fully virtualized
hardware instance. All your required libraries would be in the
self-contained chroot. The only thing you would have to match for
binary compatibility moving from one machine to the next to the next is
the kernel interfaces, and those interfaces retain backwards
compatibility for a long time. If you really wanted, you could even
implement these on opaque disk images, that would get loopback mounted
where ever you wanted to run them, just like virtual machines.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


linux at thehobsons

May 11, 2012, 9:33 AM

Post #19 of 57 (2066 views)
Permalink
Re: Back-end Virtualization [In reply to]

Raymond Wagner wrote:

>And that's it. Those are the two good reasons to run virtual
>machines in a production environment.

Err, no - that is your opinion of the only 2 good reasons.

I have a third - to allow me to run different systems, with different
kernels (perhaps because I've got some older stuff I haven't got
around to upgrading yet), and I want to be able to upgrade one system
without affecting another. Not a security thing, but a convenience
thing.

And a fourth reason ...
For my knowledge base, I can fire up a new VM to experiment quite
easily. I'd need to go learning yet more new stuff to figure out how
to setup another chroot jail or whatever *properly*.

So I have two reasons for using virtualisation at home, both are
valid, both make sense for me.


And in any case, whatever you or I may think, how someone wants to
run their systems is up to them - not us. This isn't a closed course
world where the vendor tells you what you are and aren't allowed to
do.
By all means point out that running a virtualised backend is
suboptimal, and what the potential issues might be. But don't go
saying it's "wrong" - it's not "wrong", just not how you'd do it.

--
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


memmott at gmail

May 11, 2012, 9:36 AM

Post #20 of 57 (2065 views)
Permalink
Re: Back-end Virtualization [In reply to]

On Fri, May 11, 2012 at 11:51 AM, Raymond Wagner <raymond [at] wagnerrp>wrote:

> On 5/9/2012 21:22, Nathan Hawkins wrote:
>
>> Is ‘Anyone’ virtualizing their backend?! I cant believe I’m the only
>> person out there who wants to do this…
>>
>
> I've stated my views on virtualization on this mailing list multiple times
> in the past, but new people keep joining and asking, so here goes...
>
> Why do you want to want to virtualize your backend? "Because I think it
> would be interesting" is a perfectly valid reason here. "Because the
> industry uses it and VM vendors tell me it magically makes everything
> better" is not. So why does industry use it?
>
> The first reason would be security. There are many mechanisms to isolate
> different processes on a single system, but none of them can be as complete
> or absolute as full system virtualization. You can run different
> application servers, or servers for multiple departments with different
> user access rights on one physical system, and if one gets compromised,
> there is (almost) no risk to the rest. It can be flushed and restored from
> a clean copy with no harm to the other services. If you are looking to
> virtualize MythTV for security purposes, your journey ends here. If you
> are not running MythTV on a safe, trusted network, you either shouldn't be
> running MythTV, or you should be looking into adding cryptographic
> authentication into all the various communication interfaces MythTV uses.
>
> The second reason is high availability. The virtual machine allows you to
> save the state of the machine, and in the event of a failure, resume that
> state on another piece of hardware. This is really only a crude route to
> high availability, as such capability is much more effectively and
> efficiently performed by the application itself, such as MySQL clustering
> and replication servers. It becomes a question of how valuable is the
> application to your needs, and is it valuable enough to warrant the time
> and expense developing native support in the application. If your interest
> is in HA, you've already thrown far more money at virtual machines than
> someone who would be asking a basic question such as this on an open source
> mailing list.
>
> And that's it. Those are the two good reasons to run virtual machines in
> a production environment.
>
>
> -- Counter Argument 1 --
> What about the need to run on a different hardware architecture or
> operating system? Don't. Find software that suits your needs that can all
> run on one OS. If it's something critical that you cannot do without,
> perhaps that makes it worth dedicating another machine to.
>
>
> -- Counter Argument 2 --
> What about software that refuses to run on anything outside a pre-defined
> hardware set, so you define it against a virtual machine to allow the VM to
> be mobile? Shame on the company for putting such restrictive licensing
> measures on their software. Double shame for botching it in such a manner
> that it could be bypassed by simply using a VM. From an idealist
> standpoint, shame on the customer for patronizing such an abusive developer.
>
>
> -- Counter Argument 3 --
> What about software that could crash and take out a system? If you're
> doing development, this is a great thing. Any application that you expect
> to crash with destructive results should not be used in a production
> environment.
>
>
> -- Counter Argument 4 --
> What about ease of management? When trying to run multiple applications
> and servers on a single system, you may run into dependency conflicts. You
> may update one library for one application, only to find it has broken
> another application. Virtual machines let you run multiple independent
> installs, with independent dependency sets, to avoid these issues. This is
> the big one, and I believe the reason most people improperly use virtual
> machines.
>
> This has NOTHING TO DO with virtualization. Virtual machines require this
> behavior, but this behavior does not require virtual machines. You can do
> the SAME EXACT THING with a simple chroot, without all the unnecessary
> complexity and overhead of running a fully virtualized hardware instance.
> All your required libraries would be in the self-contained chroot. The
> only thing you would have to match for binary compatibility moving from one
> machine to the next to the next is the kernel interfaces, and those
> interfaces retain backwards compatibility for a long time. If you really
> wanted, you could even implement these on opaque disk images, that would
> get loopback mounted where ever you wanted to run them, just like virtual
> machines.
>
> ______________________________**_________________
>
>
I like to think I have a valid reason, and mine is this: Consolidation. I
have a server rack in my basement, and prior to my current VM server, I had
several boxes in that rack: 1) Domain controller / DHCP / DNS / file
server. 2) OpenFiler NAS / SAN device. 3) MythTV Backend. 4) ESXi 3.5
server with ~5 VMs on it, doing all sorts of magical things.

One of the problems I had was that my storage was all over the place. I
pointed some of the boxes to my OpenFiler appliance but the storage was
scattered all over the place. Much of this was due to the fact that I'm a
WIndows admin and I really suck at Linux storage administration. At any
rate, I'd have x free terabytes here, y free terabytes there, and it was
just one big disaster.

My server rack now has exactly two "boxes" in it: A PowerEdge 2 x quad-core
Xeon I bought on ebay for $300, and an Apple Xserve RAID (two RAID 5
arrays), connected via fiber. Everything that was physical has been either
replaced or virtualized, and I couldn't be happier with my setup. Having a
VM infrastructure actually enables me to spread roles out to more "servers"
and lessen the chance of an application taking down six others, which I
believe is something Raymond touched on in his post.

Virtualization brings a lot of challenges and is definitely a paradigm
shift. It's not something I'd recommend to use "in production" to anybody
not familiar with the technologies and the layers of abstraction it brings.
But for me the benefits far outweigh the time and effort spent building out
the infrastructure, and I'd do it again in a second.


raymond at wagnerrp

May 11, 2012, 9:43 AM

Post #21 of 57 (2066 views)
Permalink
Re: Back-end Virtualization [In reply to]

On 5/11/2012 12:33, Simon Hobson wrote:
> I have a third - to allow me to run different systems, with different
> kernels (perhaps because I've got some older stuff I haven't got around
> to upgrading yet), and I want to be able to upgrade one system without
> affecting another. Not a security thing, but a convenience thing.

Already mentioned.

On 5/11/2012 11:51, Raymond Wagner wrote:
> -- Counter Argument 1 --
> What about the need to run on a different hardware architecture or
> operating system? Don't. Find software that suits your needs that
> can all run on one OS. If it's something critical that you cannot
> do without, perhaps that makes it worth dedicating another machine
> to.


> And a fourth reason ...
> For my knowledge base, I can fire up a new VM to experiment quite
> easily. I'd need to go learning yet more new stuff to figure out how to
> setup another chroot jail or whatever *properly*.

"experimenting" does not constitute a production environment.



> By all means point out that running a virtualised backend is suboptimal,
> and what the potential issues might be. But don't go saying it's "wrong"
> - it's not "wrong", just not how you'd do it.

It's "wrong" because it's not what it is designed to do. It's like
using a big truck for your daily driver. The truck is designed to haul
things, even though you rarely if ever haul things, but you use it for
your daily commute anyway, not because it's the right tool for the job,
but just because you can.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


support at drdos

May 11, 2012, 9:43 AM

Post #22 of 57 (2072 views)
Permalink
Re: Back-end Virtualization [In reply to]

>> But for me the benefits far outweigh the time and effort spent building out the infrastructure, and I'd do it again in a second.

+1

Doug


--

Ben Franklin quote:

"Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety."


raymond at wagnerrp

May 11, 2012, 10:04 AM

Post #23 of 57 (2069 views)
Permalink
Re: Back-end Virtualization [In reply to]

On 5/11/2012 12:36, Matt Emmott wrote:
>
>
> On Fri, May 11, 2012 at 11:51 AM, Raymond Wagner <raymond [at] wagnerrp
> <mailto:raymond [at] wagnerrp>> wrote:
>> -- Counter Argument 4 --
>> What about ease of management? When trying to run multiple
>> applications and servers on a single system, you may run into
>> dependency conflicts. You may update one library for one
>> application, only to find it has broken another application.
>> Virtual machines let you run multiple independent installs, with
>> independent dependency sets, to avoid these issues. This is the big
>> one, and I believe the reason most people improperly use virtual
>> machines.
>>
>> This has NOTHING TO DO with virtualization. Virtual machines
>> require this behavior, but this behavior does not require virtual
>> machines. You can do the SAME EXACT THING with a simple chroot,
>> without all the unnecessary complexity and overhead of running a
>> fully virtualized hardware instance. All your required libraries
>> would be in the self-contained chroot. The only thing you would
>> have to match for binary compatibility moving from one machine to
>> the next to the next is the kernel interfaces, and those interfaces
>> retain backwards compatibility for a long time. If you really
>> wanted, you could even implement these on opaque disk images, that
>> would get loopback mounted where ever you wanted to run them, just
>> like virtual machines.
>
> Virtualization brings a lot of challenges and is definitely a paradigm
> shift. It's not something I'd recommend to use "in production" to
> anybody not familiar with the technologies and the layers of abstraction
> it brings. But for me the benefits far outweigh the time and effort
> spent building out the infrastructure, and I'd do it again in a second.

There are three different manners that you can run multiple application
servers. You can run them on different physical machines. You can run
them on one physical machine with independent libraries. You can run
them on one physical machine with shared libraries. The paradigm shift
you speak of is shifting to that second one. Moving all these
independent application servers into a single physical machine, but
retaining those independent library sets for ease of management.
Updating one application server would not affect any other through
alteration of shared dependencies.

The point I'm trying to make is that virtualization is not a part of
this shift. The various virtualization vendors have been the primary
force behind the shift, and the data center industry has embraced
virtualization, either due to the added security benefits it offers or
just simple ignorance, but the key feature that everyone in this mailing
list seems to want when they start talking about running virtual
machines is not a feature of virtual machines at all, but a necessary
side effect.

You want to run independent filesystem roots for different applications
for ease of management. You can do that in combination with
virtualization, or without virtualization, making virtualization itself
irrelevant and unnecessary. At best, all the work put into
virtualization mechanisms used by industry just means they have a well
developed tool set for creating them.

In your case, coming from a Windows background, it has the added benefit
of being able to run Windows servers. You likely have a domain
controller running, and possibly Windows web and mail servers. That
goes back to another mentioned argument, is that actually a good thing,
or is it better to consolidate all on servers compatible with a single
shared kernel.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


zarthan at gmail

May 11, 2012, 10:19 AM

Post #24 of 57 (2070 views)
Permalink
Re: Back-end Virtualization [In reply to]

On Fri, May 11, 2012 at 10:51 AM, Raymond Wagner <raymond [at] wagnerrp>wrote:

> On 5/9/2012 21:22, Nathan Hawkins wrote:
>
>> Is ‘Anyone’ virtualizing their backend?! I cant believe I’m the only
>> person out there who wants to do this…
>>
>
> I've stated my views on virtualization on this mailing list multiple times
> in the past, but new people keep joining and asking, so here goes...
>
> Why do you want to want to virtualize your backend? "Because I think it
> would be interesting" is a perfectly valid reason here. "Because the
> industry uses it and VM vendors tell me it magically makes everything
> better" is not. So why does industry use it?
>
> The first reason would be security. There are many mechanisms to isolate
> different processes on a single system, but none of them can be as complete
> or absolute as full system virtualization. You can run different
> application servers, or servers for multiple departments with different
> user access rights on one physical system, and if one gets compromised,
> there is (almost) no risk to the rest. It can be flushed and restored from
> a clean copy with no harm to the other services. If you are looking to
> virtualize MythTV for security purposes, your journey ends here. If you
> are not running MythTV on a safe, trusted network, you either shouldn't be
> running MythTV, or you should be looking into adding cryptographic
> authentication into all the various communication interfaces MythTV


The first reason is to more fully use physical resources. 90 plus percent
of physical machines use less than 10% of the CPU. Today it is very
difficult to right size a server for a single workload. For the vast
majority of servers a single core Atom CPU would be overkill. In a virtual
environment I can tailor the exact needs of a particular workload. If I
need no more than 848 MB of RAM I can assign exactly that. If I require 3
CPU cores I can assign that. I can balance the resource load across all the
servers to further maximize resource utilization.

Up until recently security and the inability to ensure network traffic
security was very much lacking. Today unless you spend the extra money for
all the security measures you do not have any means of protecting or
monitoring inter VM traffic.

>
> The second reason is high availability. The virtual machine allows you to
> save the state of the machine, and in the event of a failure, resume that
> state on another piece of hardware. This is really only a crude route to
> high availability, as such capability is much more effectively and
> efficiently performed by the application itself, such as MySQL clustering
> and replication servers. It becomes a question of how valuable is the
> application to your needs, and is it valuable enough to warrant the time
> and expense developing native support in the application. If your interest
> is in HA, you've already thrown far more money at virtual machines than
> someone who would be asking a basic question such as this on an open source
> mailing list.
>
> And that's it. Those are the two good reasons to run virtual machines in
> a production environment.
>

Reduced datacenter space and power and cooling. A dual multicore CPU server
can easily handle a 20 to one ratio and virtual desktops can go 100 to
one.
During off peak hours I can migrate VM to fewer physical servers and shut
down the extras. When needs change I can run up the extra physical machines
and migrate the VMs back.



> -- Counter Argument 1 --
> What about the need to run on a different hardware architecture or
> operating system? Don't. Find software that suits your needs that can all
> run on one OS. If it's something critical that you cannot do without,
> perhaps that makes it worth dedicating another machine to.
>
> I am primarily a Linux guy but I still have a Windows server or two.
Getting a separate machine just for the Windows servers is totally
wasteful.

>
> -- Counter Argument 2 --
> What about software that refuses to run on anything outside a pre-defined
> hardware set, so you define it against a virtual machine to allow the VM to
> be mobile? Shame on the company for putting such restrictive licensing
> measures on their software. Double shame for botching it in such a manner
> that it could be bypassed by simply using a VM. From an idealist
> standpoint, shame on the customer for patronizing such an abusive developer.
>
> Virtual isn't as unusual as it was and most workloads are totally
acceptable / supported in a virtual environment. Legacy applications that
only run on say NT 4 Windows 95 or even DOS can be migrated (cloned from
physical ) and run as virtual allowing very old hardware to be retired. Not
at all uncommon.



> -- Counter Argument 3 --
> What about software that could crash and take out a system? If you're
> doing development, this is a great thing. Any application that you expect
> to crash with destructive results should not be used in a production
> environment.
>
> Virtualization in the PC world began as a development tool for that
reason. Today it is not only development but allows for large scale
testing.


> -- Counter Argument 4 --
> What about ease of management? When trying to run multiple applications
> and servers on a single system, you may run into dependency conflicts. You
> may update one library for one application, only to find it has broken
> another application. Virtual machines let you run multiple independent
> installs, with independent dependency sets, to avoid these issues. This is
> the big one, and I believe the reason most people improperly use virtual
> machines.
>

It comes down to right sizing the resources of a particular workload. When
all the software runs in a single OS instance a less important process can
hinder the needs of a more important process. It took lots of work to fine
tune some apps to cooperate well together. I make tiny VMs and don't give
them a high priority for non important processes. If they aren't needed
they can be paused or migrated to another physical host.



> This has NOTHING TO DO with virtualization. Virtual machines require this
> behavior, but this behavior does not require virtual machines. You can do
> the SAME EXACT THING with a simple chroot, without all the unnecessary
> complexity and overhead of running a fully virtualized hardware instance.
> All your required libraries would be in the self-contained chroot. The
> only thing you would have to match for binary compatibility moving from one
> machine to the next to the next is the kernel interfaces, and those
> interfaces retain backwards compatibility for a long time. If you really
> wanted, you could even implement these on opaque disk images, that would
> get loopback mounted where ever you wanted to run them, just like virtual
> machines.


I use chrooted images and while useful (running backtrack on my phone) it
doesn't satisfy most of the reasons most people virtualize. There is no
real isolation and if the kernel or module crashes I can loose everything.

The ability to isolate access to resources and tailor that access for
differing needs and differing workloads is the prime reason to virtualize.
In my opinion that applies in the home as well as in the enterprise.


raymond at wagnerrp

May 11, 2012, 10:37 AM

Post #25 of 57 (2055 views)
Permalink
Re: Back-end Virtualization [In reply to]

On 5/11/2012 13:19, Zarthan South wrote:
> On Fri, May 11, 2012 at 10:51 AM, Raymond Wagner <raymond [at] wagnerrp
> <mailto:raymond [at] wagnerrp>> wrote:
>> On 5/9/2012 21:22, Nathan Hawkins wrote:
>>> Is ‘Anyone’ virtualizing their backend?! I cant believe I’m the only
>>> person out there who wants to do this…
>>
>> Why do you want to want to virtualize your backend? "Because I think
>> it would be interesting" is a perfectly valid reason here. "Because
>> the industry uses it and VM vendors tell me it magically makes
>> everything better" is not. So why does industry use it?
>
> The first reason is to more fully use physical resources. 90 plus
> percent of physical machines use less than 10% of the CPU.

No. Contrary to popular belief, modern operating systems are actually
capable of multitasking without the help of virtual machines.

> Up until recently security and the inability to ensure network traffic
> security was very much lacking. Today unless you spend the extra money
> for all the security measures you do not have any means of protecting or
> monitoring inter VM traffic.

And this gets right back to the third paragraph, if you think you need
the level of isolation virtualization provides for the purposes of
security, either you're an absurdly paranoid home user, or you need to
seriously reconsider whether MythTV is a good fit for your commercial use.

>> The second reason is high availability. The virtual machine allows
>> you to save the state of the machine, and in the event of a failure,
>> resume that state on another piece of hardware. This is really only
>> a crude route to high availability, as such capability is much more
>> effectively and efficiently performed by the application itself,
>> such as MySQL clustering and replication servers. It becomes a
>> question of how valuable is the application to your needs, and is it
>> valuable enough to warrant the time and expense developing native
>> support in the application.
>
> Reduced datacenter space and power and cooling. A dual multicore CPU
> server can easily handle a 20 to one ratio and virtual desktops can go
> 100 to one.

As stated, you actually can run multiple programs on a single operating
system these days. All that preemptive multitasking developed in the
1970s is wonderful stuff.

> During off peak hours I can migrate VM to fewer physical servers and
> shut down the extras. When needs change I can run up the extra physical
> machines and migrate the VMs back.

This comes back to high availability, and application clustering. If
you need the ability to migrate a live instance of an application from
one physical machine to another, virtual machines are just about the
only easy route. If instead you are running something like a web server
that has no trouble handling a restart, there is no reason you cannot
simply terminate the instance on the machine you want to power down,
open a new instance on the machine you're migrating to, and update your
load balancers to suit. Virtual machines would not be necessary.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

First page Previous page 1 2 3 Next page Last page  View All MythTV users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.