
mark.kendall at gmail
Mar 22, 2010, 9:04 PM
Post #1 of 34
(3834 views)
Permalink
|
|
MythUI OSD branch and the way forward
|
|
Morning All, There has been some discussion on irc recently regarding the libmythui-osd branch and how to support XVideo (and XvMC) going forward. It's not a straightforward issue and any resolution is likely to have wider implications - so it's about time I tried explaining it in a little more detail and the options I believe we have. Apologies if some of the following is teaching some (or all!) of you to suck eggs. - Background The aim of the branch is to draw the OSD using the libmythui-code. This gives users a consistent look and feel, gives themers significantly more opportunity to theme the osd (and a consistent theme format) and should allow us to remove significant amounts of OSD code. Ideally a win/win/win. My longer term aims have always been broader. By combining the rendering code for the UI and video, I'm targetting a situation in which video is just another UI widget to be toyed with by themers and normal video playback is a just a fullscreen widget. Picture In Picture might become just another OSD UI widget and with some modifications to NuppelVideoPlayer it can become a generic media player (i.e. music and video) with visualisations built in. I guess the extreme case is where the media becomes the background (whether it is livetv, music visualiser, album photo etc) and the UI is alpha blended on top. While admittedly it's all just eye candy, I'd like to think that most people would think it's a good thing! (and for what it's worth, it's probably not that far away) - The Problem All of the above is, in general, pretty straightforward when using OpenGL or Direct3D for the video and the UI. So probably no issue on either Windows or Mac (both of which will also have useable hardware drivers and most users will have sufficiently powerful GPUs). On linux VDPAU will handle the rendering basics well enough but in the future we may want to use OpenGL rendering directly. Any VDPAU capable card also has sufficient OpenGL horsepower. Very little of this will work well with XVideo however (XvMC is even worse) (and in certain situations the hardware drivers just aren't good enough to give the OpenGL performance we need on linux). There are essentially three problems with XVideo:- - XVideo handles video only (or more precisely images in the YUV colorspace and by and large planar formats), there is no API to manipulate the images after they have been converted/scaled and there is no way of reliably and consistently integrating them with other hardware acceleration APIs. - we by default rely on creating a YUV osd image which is then blended into the video frame (in software). This image will usually be a different size to the main UI. As a result we either need to use OSD specific theme elements or reload/resize the main UI. (OpenGL/VDPAU/Direct3D all create the OSD at the screen resolution which should match the UI size - although there will be exceptions that need to be handled) - blending in software is not cheap and as the OSD becomes larger and more complicated, there is a significant performance hit. For example, a number of the newer themes use popups with essentially a fullscreen alphamask (to darken the background) - this is simply not going to work when trying to play h264 1080 material. - Options There are 1001 possible workarounds, hacks and solutions and I've listed a few options below. I do however feel it's worth pointing out that any long term solution that includes support for XVideo will be a compromise for the rest of the UI and hence the rest of the application. Option 1 - Do nothing i.e. do not merge in the branch when complete. Option 2 - Provide XVideo (and XvMC?) OSD support only via Chromakeying. The Chromakey OSD operates at the screen resolution and rendering on screen is not as expensive as the software blend. I think the XVideo chromakey was dropped by Nvidia for 7xxx series cards and later and is by and large still available on Intel. Although i don't know about ATI, this may be a useful short term option for users of 'legacy' cards (but many of the longer term UI features will not be doable). Option 3 - Use specific 'legacy' OSD themes for XVideo to avoid the sizing issue. Far from ideal from a themers perspective and does not provide a long term solution. Option 4 - Force resizing of the main UI when needed. This is going to be extremely expensive and may simply not be doable in certain situations (e.g. guidegrid during playback). XVideo blending will still be expensive and still no future improvements. Option 5 - Force an XVideo overlay On systems without chromakey support, try and render the OSD directly on top of the video frame using the Qt X11 pixmaps directly. I have no idea whether this will work (it is impossible to test until some threading issues have been resolved), though I think there will be blending/compositing issues. Regardless - it will be ugly! There may also be a solution here involving the X composite extension but given we are trying to support older hardware... Option 6 - Drop XVideo/XvMC support and standardise around OpenGL There are numerous ways of approaching this but I would clean up the current OpenGL video code, standardise around GLSL for shader support (UI effects, deinterlacing, colorspace conversion), remove OpenGL code based around immediate mode (for OpenGL ES compatability), provide colourspace conversion only support for ARB_fragment_program (for older OpenGL implementations which will probably not be capable of using any advanced video shaders anyway) and use OpenGL as the 'frontend' for VDPAU. It's probably worth dropping Direct3D support as it offers nothing above and beyond OpenGL and simply represents duplicated code (esp. around supporting HLSL). There is then also a separate debate to be had around the Qt painter.... Option 6 - Drop support in stages Move to Option 2 for the 0.24 release and Option 6 for 0.25. Option 7 - Maintain a legacy branch Cut a stable 'Legacy' branch with XVideo support and move to Option 6 for 0.24 (assuming it's ready). Provide X weeks/months/years of legacy support... Hopefully that's all sufficiently clear. If not, I'll crawl back into my hole and brood on it some more:) In case it isn't obvious, I'd like to move to option 6 as soon as possible - but it's not my decision and I've spent so long working on it that I may have overlooked the obvious... Regards Mark PS Nigel - I've deliberately not mentioned OS X quartz/accelerated support - mostly because I still can't get trunk to compile on Snow Leopard. I do feel however that OS X support should probably be based around OpenGL regardless to maintain consistency across platforms. PPS I've almost finished converting the last of the video code to using the mythui osd. After that the threading in video playback needs to be changed signicantly. This will involve sizeable changes to NuppelVideoPlayer and will take some time. With a fair wind, I'm hoping that we could consider a merge in around 3-4 weeks time. _______________________________________________ mythtv-dev mailing list mythtv-dev [at] mythtv http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev
|