Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: MythTV: Dev

#10305 heads up, mysql.txt removal + config.xml changes + UPnP fixes

 

 

MythTV dev RSS feed   Index | Next | Previous | View Threaded


danielk at cuymedia

May 3, 2012, 8:24 AM

Post #1 of 11 (582 views)
Permalink
#10305 heads up, mysql.txt removal + config.xml changes + UPnP fixes

I've attached the latest patchset to the ticket. I just wanted to
give everyone a heads up in case they had strong opinions on the
config.xml format or want to look at the UPnP backend discovery
changes.

Summary of changes:
* Format of config.xml
+ Database settings have their own top-level instead of being under
<MythFrontend><DefaultBackend>.
+ Wake On Lan settings have been added to config.xml, these
were formerly only settable using mysql.txt.
* UPnP autoconf now waits a full 2 seconds before deciding there
really is only one backend to connect to.
* UPnP resends the discovery packet a few times in auto conf mode.
* config.xml is only rewritten when it changes, and this is done
in a safe manner so a crash will not wipe out your configuration.
* mysql.txt reading has been removed. This does not port over
Wake On Lan settings and all other settings not written to the
file by default are ignored. Setting file functionality is
available using the --override-settings-file option, which
has the added benefit of overriding DB settings instead of just
providing new defaults if the DB doesn't have that setting.

I'll commit tomorrow if there are no objections.

-- Daniel

_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


lvr at softsystem

May 3, 2012, 12:02 PM

Post #2 of 11 (568 views)
Permalink
Re: #10305 heads up, mysql.txt removal + config.xml changes + UPnP fixes [In reply to]

On Thu, 2012-05-03 at 11:24 -0400, Daniel Kristjansson wrote:
> I've attached the latest patchset to the ticket. I just wanted to
> give everyone a heads up in case they had strong opinions on the
> config.xml format or want to look at the UPnP backend discovery
> changes.
>
> Summary of changes:
> * Format of config.xml
> + Database settings have their own top-level instead of being under
> <MythFrontend><DefaultBackend>.
> + Wake On Lan settings have been added to config.xml, these
> were formerly only settable using mysql.txt.
The parsing of xml files is just a total waste of CPU resources. They
add no useful information c.f. a well constructed ini file,

> * UPnP autoconf now waits a full 2 seconds before deciding there
> really is only one backend to connect to.
This is far too long to sit idle. Any reasonable local UPnP server will
respond to a broad/multicast request within a few milliseconds. What
are you waiting for a Sinclair ZX80?

> * UPnP resends the discovery packet a few times in auto conf mode.
> * config.xml is only rewritten when it changes, and this is done
> in a safe manner so a crash will not wipe out your configuration.
> * mysql.txt reading has been removed. This does not port over
> Wake On Lan settings and all other settings not written to the
> file by default are ignored.
There are many 3rd party scripts that rely on mysql.txt. It can contain
the same information as config.xml and takes just a fraction of the CPU
resources to parse. Why not settle for this file alone? Or, what about
a halfway house - a config utility that takes simple command line
queries and returns simple text strings.

> Setting file functionality is
> available using the --override-settings-file option, which
> has the added benefit of overriding DB settings instead of just
> providing new defaults if the DB doesn't have that setting.
This is just a recipe for disaster with some db values coming from a
user provided file that could conflict with settings in the main
database. This resolution should be handled by a common parser within a
myth library. Another good reason for a config utility.

> I'll commit tomorrow if there are no objections.
++objections

--
Lawrence
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


mtdean at thirdcontact

May 3, 2012, 12:21 PM

Post #3 of 11 (563 views)
Permalink
Re: #10305 heads up, mysql.txt removal + config.xml changes + UPnP fixes [In reply to]

On 05/03/2012 03:02 PM, Lawrence Rust wrote:
> On Thu, 2012-05-03 at 11:24 -0400, Daniel Kristjansson wrote:
>> I've attached the latest patchset to the ticket. I just wanted to
>> give everyone a heads up in case they had strong opinions on the
>> config.xml format or want to look at the UPnP backend discovery
>> changes.
>>
>> Summary of changes:
>> * Format of config.xml
>> + Database settings have their own top-level instead of being under
>> <MythFrontend><DefaultBackend>.
>> + Wake On Lan settings have been added to config.xml, these
>> were formerly only settable using mysql.txt.
> The parsing of xml files is just a total waste of CPU resources. They
> add no useful information c.f. a well constructed ini file,

FWIW, I think any machine that can handle running MythTV (or even
scripts that use MythTV) can handle parsing a 500B XML file. Yes, I
realize that's nearly a half kilobyte, but still... ;)

>> * mysql.txt reading has been removed. This does not port over
>> Wake On Lan settings and all other settings not written to the
>> file by default are ignored.
> There are many 3rd party scripts that rely on mysql.txt. It can contain
> the same information as config.xml and takes just a fraction of the CPU
> resources to parse. Why not settle for this file alone? Or, what about
> a halfway house - a config utility that takes simple command line
> queries and returns simple text strings.

The bindings--both Perl and Python (and PHP?)--use only config.xml and
ignore mysql.txt. When it was added (long ago), the decision was made
that it would be used for all future stuff, but we just haven't gotten
around to actually removing the mysql.txt stuff until now.

Whichever file we use, we need only one. I really don't see any reason
why mysql.txt is better than config.xml--it will never contain a lot of
information, so...

Mike
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


lvr at softsystem

May 3, 2012, 12:48 PM

Post #4 of 11 (562 views)
Permalink
Re: #10305 heads up, mysql.txt removal + config.xml changes + UPnP fixes [In reply to]

On Thu, 2012-05-03 at 15:21 -0400, Michael T. Dean wrote:
> On 05/03/2012 03:02 PM, Lawrence Rust wrote:
> > On Thu, 2012-05-03 at 11:24 -0400, Daniel Kristjansson wrote:
> >> I've attached the latest patchset to the ticket. I just wanted to
> >> give everyone a heads up in case they had strong opinions on the
> >> config.xml format or want to look at the UPnP backend discovery
> >> changes.
> >>
> >> Summary of changes:
> >> * Format of config.xml
> >> + Database settings have their own top-level instead of being under
> >> <MythFrontend><DefaultBackend>.
> >> + Wake On Lan settings have been added to config.xml, these
> >> were formerly only settable using mysql.txt.
> > The parsing of xml files is just a total waste of CPU resources. They
> > add no useful information c.f. a well constructed ini file,
>
> FWIW, I think any machine that can handle running MythTV (or even
> scripts that use MythTV) can handle parsing a 500B XML file. Yes, I
> realize that's nearly a half kilobyte, but still... ;)

Why do we bother to optimize our code then? There's always more RAM and
a faster CPU...

> >> * mysql.txt reading has been removed. This does not port over
> >> Wake On Lan settings and all other settings not written to the
> >> file by default are ignored.
> > There are many 3rd party scripts that rely on mysql.txt. It can contain
> > the same information as config.xml and takes just a fraction of the CPU
> > resources to parse. Why not settle for this file alone? Or, what about
> > a halfway house - a config utility that takes simple command line
> > queries and returns simple text strings.
>
> The bindings--both Perl and Python (and PHP?)--use only config.xml and
> ignore mysql.txt. When it was added (long ago), the decision was made
> that it would be used for all future stuff, but we just haven't gotten
> around to actually removing the mysql.txt stuff until now.

php doesn't have(need) specific Myth bindings. It can, and does, use
both file formats.

> Whichever file we use, we need only one. I really don't see any reason
> why mysql.txt is better than config.xml--it will never contain a lot of
> information, so...

Why break so many things when there's no need. It's so easy to 'knock
up' a mysql.txt for a specific occasion.

--
Lawrence
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


raymond at wagnerrp

May 3, 2012, 12:59 PM

Post #5 of 11 (563 views)
Permalink
Re: #10305 heads up, mysql.txt removal + config.xml changes + UPnP fixes [In reply to]

On 5/3/2012 15:48, Lawrence Rust wrote:
> On Thu, 2012-05-03 at 15:21 -0400, Michael T. Dean wrote:
>> On 05/03/2012 03:02 PM, Lawrence Rust wrote:
>>> On Thu, 2012-05-03 at 11:24 -0400, Daniel Kristjansson wrote:
>>>> I've attached the latest patchset to the ticket. I just wanted to
>>>> give everyone a heads up in case they had strong opinions on the
>>>> config.xml format or want to look at the UPnP backend discovery
>>>> changes.
>>>>
>>>> Summary of changes:
>>>> * Format of config.xml
>>>> + Database settings have their own top-level instead of being under
>>>> <MythFrontend><DefaultBackend>.
>>>> + Wake On Lan settings have been added to config.xml, these
>>>> were formerly only settable using mysql.txt.
>>> The parsing of xml files is just a total waste of CPU resources. They
>>> add no useful information c.f. a well constructed ini file,
>> FWIW, I think any machine that can handle running MythTV (or even
>> scripts that use MythTV) can handle parsing a 500B XML file. Yes, I
>> realize that's nearly a half kilobyte, but still... ;)
> Why do we bother to optimize our code then? There's always more RAM and
> a faster CPU...
>
>>>> * mysql.txt reading has been removed. This does not port over
>>>> Wake On Lan settings and all other settings not written to the
>>>> file by default are ignored.
>>> There are many 3rd party scripts that rely on mysql.txt. It can contain
>>> the same information as config.xml and takes just a fraction of the CPU
>>> resources to parse. Why not settle for this file alone? Or, what about
>>> a halfway house - a config utility that takes simple command line
>>> queries and returns simple text strings.
>> The bindings--both Perl and Python (and PHP?)--use only config.xml and
>> ignore mysql.txt. When it was added (long ago), the decision was made
>> that it would be used for all future stuff, but we just haven't gotten
>> around to actually removing the mysql.txt stuff until now.
> php doesn't have(need) specific Myth bindings. It can, and does, use
> both file formats.

Erm...

http://code.mythtv.org/cgit/mythtv/tree/mythtv/bindings/php

Although technically they don't read either form of file, and need to be
told where to go.

>
>> Whichever file we use, we need only one. I really don't see any reason
>> why mysql.txt is better than config.xml--it will never contain a lot of
>> information, so...
> Why break so many things when there's no need. It's so easy to 'knock
> up' a mysql.txt for a specific occasion.

The existing mysql.txt is only of value for people writing scripts in
bash, as it can just be directly sourced without any kind of parsing.
The only value it has for use in bash is when trying to access the
database directly, and if you need to access the database directly, you
can spend the effort writing whatever you're doing in a "proper"
programming language with native MySQL bindings and proper type
handling, rather than something that is just going to clumsily wrap the
`mysql` shell client. Any such language is going to have some form of
XML parser available, and not have any problem dealing with the config.xml.
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


phipps-hutton at sky

May 3, 2012, 11:16 PM

Post #6 of 11 (550 views)
Permalink
Re: #10305 heads up, mysql.txt removal + config.xml changes + UPnP fixes [In reply to]

Quoting Raymond Wagner <raymond [at] wagnerrp>:
> The only value it has for use in bash is when trying to access the
> database directly, and if you need to access the database directly,
> you can spend the effort writing whatever you're doing in a "proper"
> programming language with native MySQL bindings and proper type
> handling, rather than something that is just going to clumsily wrap
> the `mysql` shell client. Any such language is going to have some
> form of XML parser available, and not have any problem dealing with
> the config.xml.

Please don't take offence but the above comes across as snobbish.
"Proper" programming languages come and go, sh is a Posix standard and
is here to stay. That said I don't mind mysql.txt going. I'd rather
have one setup file and if XML means less chance of errors in reading
and writing then that's good. Parsing performance doesn't not matter
for the setup file, robustness does. I'm sure it will be possible to
find a way to read it into a shell script.

Cheers,
Tim.

_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


dekarl at spaetfruehstuecken

May 3, 2012, 11:26 PM

Post #7 of 11 (551 views)
Permalink
Re: #10305 heads up, mysql.txt removal + config.xml changes + UPnP fixes [In reply to]

On 04.05.2012 08:16, Tim Phipps wrote:
> Quoting Raymond Wagner <raymond [at] wagnerrp>:
>> The only value it has for use in bash is when trying to access the
>> database directly, and if you need to access the database directly,
>> you can spend the effort writing whatever you're doing in a "proper"
>> programming language with native MySQL bindings and proper type
>> handling, rather than something that is just going to clumsily wrap
>> the `mysql` shell client. Any such language is going to have some form
>> of XML parser available, and not have any problem dealing with the
>> config.xml.
>
> Please don't take offence but the above comes across as snobbish.
> "Proper" programming languages come and go, sh is a Posix standard and
> is here to stay. That said I don't mind mysql.txt going. I'd rather have
> one setup file and if XML means less chance of errors in reading and
> writing then that's good. Parsing performance doesn't not matter for the
> setup file, robustness does. I'm sure it will be possible to find a way
> to read it into a shell script.

While everything you say is true it doesn't fit together.
Any proper programming language, whichever that may be, will have
a proper way to to parse xml and talk to a SQL database. But the Posix
standard shell will always have to use whatever command line interface
the mysql tools offer.
So if you can trade supporting multiple versions of command line
interfaces in a shell script against saying "no" when asked to rewrite
your script "just because all the cool kids write scripts in Foo now!"
then I'd prefer any proper language that allows me to have a robust
script any time :-)

Regards,
Karl
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


raymond at wagnerrp

May 3, 2012, 11:35 PM

Post #8 of 11 (551 views)
Permalink
Re: #10305 heads up, mysql.txt removal + config.xml changes + UPnP fixes [In reply to]

On 5/4/2012 02:16, Tim Phipps wrote:
> Quoting Raymond Wagner <raymond [at] wagnerrp>:
>> The only value it has for use in bash is when trying to access the
>> database directly, and if you need to access the database directly,
>> you can spend the effort writing whatever you're doing in a "proper"
>> programming language with native MySQL bindings and proper type
>> handling, rather than something that is just going to clumsily wrap
>> the `mysql` shell client. Any such language is going to have some
>> form of XML parser available, and not have any problem dealing with
>> the config.xml.
>
> Please don't take offence but the above comes across as snobbish.
> "Proper" programming languages come and go, sh is a Posix standard and
> is here to stay.

I don't mean to imply that Bourne (Again) is bad, or that it shouldn't
be used, but rather that it should be used for its intended purpose. It
is a shell script, and as such, its intended purpose is to manage
external applications. It has relatively little built in functionality,
as most of is functionality is intended to be provided by calling
external applications. The language is of course Turing complete,
meaning with sufficient effort, you can program it to do anything any
other language could do, but that doesn't mean it will be easier than
doing it in some other language. When you start doing things like
accessing databases, you have long since exceeded the point at which you
should switch to a language with a more expansive tool set.
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


raymond at wagnerrp

May 3, 2012, 11:39 PM

Post #9 of 11 (552 views)
Permalink
Re: #10305 heads up, mysql.txt removal + config.xml changes + UPnP fixes [In reply to]

On 5/4/2012 02:26, Karl Dietz wrote:
> On 04.05.2012 08:16, Tim Phipps wrote:
>> Quoting Raymond Wagner <raymond [at] wagnerrp>:
>>> The only value it has for use in bash is when trying to access the
>>> database directly, and if you need to access the database directly,
>>> you can spend the effort writing whatever you're doing in a "proper"
>>> programming language with native MySQL bindings and proper type
>>> handling, rather than something that is just going to clumsily wrap
>>> the `mysql` shell client. Any such language is going to have some form
>>> of XML parser available, and not have any problem dealing with the
>>> config.xml.
>>
>> Please don't take offence but the above comes across as snobbish.
>> "Proper" programming languages come and go, sh is a Posix standard and
>> is here to stay. That said I don't mind mysql.txt going. I'd rather have
>> one setup file and if XML means less chance of errors in reading and
>> writing then that's good. Parsing performance doesn't not matter for the
>> setup file, robustness does. I'm sure it will be possible to find a way
>> to read it into a shell script.
>
> While everything you say is true it doesn't fit together.
> Any proper programming language, whichever that may be, will have
> a proper way to to parse xml and talk to a SQL database. But the Posix
> standard shell will always have to use whatever command line interface
> the mysql tools offer.

That's technically not true. Given sufficient motivation, and a
masochistic fetish, one could natively implement the MySQL
communications protocol and access the file socket directly, or even use
/dev/tcp and interface with a remote server.
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


danielk at cuymedia

May 4, 2012, 7:33 AM

Post #10 of 11 (541 views)
Permalink
Re: #10305 heads up, mysql.txt removal + config.xml changes + UPnP fixes [In reply to]

On 05/03/2012 03:02 PM, Lawrence Rust wrote:
> This is far too long to sit idle. Any reasonable local UPnP server
> will respond to a broad/multicast request within a few milliseconds.
> What are you waiting for a Sinclair ZX80?

This only happens when run without a config.xml and without the
--prompt command line option. Without changes to our UPnP implementation
the choices are 1 second, 2 seconds or more. With one second we do not
have an opportunity to resend the multicast request so if that packet is
lost. Two seconds is the minimum value we can use if with resends.
Larger values would allow a better chance of a response.

>> * UPnP resends the discovery packet a few times in auto conf mode.
>> * config.xml is only rewritten when it changes, and this is done
>> in a safe manner so a crash will not wipe out your configuration.
>> * mysql.txt reading has been removed. This does not port over
>> Wake On Lan settings and all other settings not written to the
>> file by default are ignored.
> There are many 3rd party scripts that rely on mysql.txt. It can contain
> the same information as config.xml and takes just a fraction of the CPU
> resources to parse. Why not settle for this file alone? Or, what about
> a halfway house - a config utility that takes simple command line
> queries and returns simple text strings.
This is one of the reasons I didn't apply this patch before the 0.25
release.
But config.xml vs mysql.txt isn't up for debate. While I like the simplicity
of mysql.txt, it's more important that we have one initial config file than
the particular format of that file. Which one was a question a few releases
back and the overwhelming preference was for the xml format. The heads up
was in case someone wanted to look at the code and wanted to comment on the
particulars of the xml format or other details.

>> Setting file functionality is
>> available using the --override-settings-file option, which
>> has the added benefit of overriding DB settings instead of just
>> providing new defaults if the DB doesn't have that setting.
> This is just a recipe for disaster with some db values coming from a
> user provided file that could conflict with settings in the main
> database. This resolution should be handled by a common parser within a
> myth library. Another good reason for a config utility.
This isn't a new feature. There were three ways to set settings outside
the DB, after this change there are now two.

-- Daniel
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


keemllib at gmail

May 5, 2012, 7:24 AM

Post #11 of 11 (536 views)
Permalink
Re: #10305 heads up, mysql.txt removal + config.xml changes + UPnP fixes [In reply to]

On 05/03/2012 10:24 AM, Daniel Kristjansson wrote:
> I've attached the latest patchset to the ticket. I just wanted to
> give everyone a heads up in case they had strong opinions on the
> config.xml format ...

> * config.xml is only rewritten when it changes, and this is done
> in a safe manner so a crash will not wipe out your configuration.
...

Scripts using python bindings were found to convert config.xml files
from the new format back to the old. The 'unconverted' file will
have one long line. See: http://code.mythtv.org/trac/ticket/10692

Discussion on the -users channel this morning suggest that using perl
bindings may produce the same result.

--
Bill
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev

MythTV dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.