Mailing List Archive: MythTV: Users

Script to use Schedules Direct with legacy MythTV installations

Index | Next | Previous | View Threaded

jmastron at gmail

Aug 22, 2007, 9:56 PM

Post #1 of 17 (12773 views)
Permalink

Script to use Schedules Direct with legacy MythTV installations

I know we're all anxiously awaiting new releases to natively support
Schedules Direct; I'm planning to upgrade as soon as I can, but in the
meantime I made a small script to semi-automatically run a modified version
of XMLTV's tv_grab_na_dd followed by mythfilldatabase, adjusting the new
program ID format to match the old database. It's not pretty (though it
does nice things like wait for the next suggested run time), and it's
definitely not permanent, but will give me a bit of breathing room to
upgrade on my own schedule. At the very least, you can extract the manual
commands to roll your own solution; it took me some trial and error to
figure out the right command line parameters.

This assumes XMLTV (and tv_grab_na_dd) is already installed (an older
version without the new URL is okay; see modification instructions); this
was true in my ATRPMS "yum install mythtv-suite" following Jarod's guide.
See the help/comments at the beginning for more instructions.

Good luck; I hope this helps a few people.

Josh

#!/usr/bin/perl

sub printhelp {
print <<ENDOFHELP;

SchedulesDirect semi-automatic downloader for legacy MythTV installations
Josh Mastronarde, jmastron[at]gmail.com
8-22-07

Syntax: sdfill <options>
-nodownload Don't download data (use last downloaded file)
-nofill Don't run mythfilldatabase
-once Only run once

This script will grab listings data, modify the episode ID field to match
the old data, and run mythfilldatabase. If -once not specified, it will
loop
waiting for the next suggested time from the last run.

Steps to prepare:

1) Find XMLTV's tv_grab_na_dd (was in /usr/bin for me); copy to
./tv_grab_na_sd

2) Modify ./tv_grab_na_sd; replace the "dd_service" line with:
my $dd_service=
'http://docs.tms.tribune.com/tech/tmsdatadirect/zap2it/xtvd.wsdl';

3) Run: ./tv_grab_na_sd --configure --config-file
~/.xmltv/tv_grab_na_sd.conf"

4) Replace "CA54023" with your lineup id (from the above .conf file; leave
off
the ":-", I think)

If you want to run by hand, the key commands are in "$grabcmd" and
"$mythfillcmd" below, along with the small section of code after "Modify
episode/program IDs" to shorten the IDs.

ENDOFHELP
exit(1);
}

use Time::Local;

# Defaults

$xmlfile1 = "sdlist.xml";
$xmlfile2 = "sdlist_fixed.xml";
$grablog = "grab_log";

$grabcmd = "./tv_grab_na_sd --config-file ~/.xmltv/tv_grab_na_sd.conf " .
"--days 14 --dd-data $xmlfile1 --download-only --list-times";

$mythfillcmd = "mythfilldatabase --dd-file 1 -1 CA54023 $xmlfile2";

# Parse command line arguments
while (@ARGV) {
$arg = shift(@ARGV);
if ($arg eq "-nodownload") {
$nodownload = 1;
} elsif ($arg eq "-nofill") {
$nofill = 1;
} elsif ($arg eq "-h") {
&printhelp;
} elsif ($arg eq "-once") {
$once = 1;
} else {
push(@narg, $arg);
}
}

$| = 1;
print "\n---- SDfill started at ", &prlocal(time()), "\n\n";

# Loop (but only once if -once flag set)

$run = 1;
while ($run) {
$run = 0 if ($once);

# Check for logfile from last grabber run and look for suggested time

if (-e $grablog) {
$suggtime = "";
open(grablog, "<$grablog") || die "Error: Can't open $grablog\n";
while (<grablog>) {
if (($year,$mon,$day,$hour,$min,$sec) = ($_ =~ /^suggestedTime
\|(\d+)\-(\d+)\-(\d+)T(\d+):(\d+):(\d+)Z/)) {
$suggtime = timegm($sec,$min,$hour,$day,$mon-1,$year-1900);
}
}
close(grablog);
if ($suggtime) {
$waitsecs = $suggtime - time();
$waithrs = int($waitsecs/3600);
$waitmins = int($waitsecs/60) % 60;

print "---- Sleeping until suggested ", &prlocal($suggtime), ",
$waithrs:$waitmins from now ----\n\n";
sleep($waitsecs);
}
}

print "\n---- Grab starting at ", &prlocal(time()), "\n\n";

unless ($nodownload) {
unlink("sdlist.xml");
print "Running: $grabcmd\n";
open(f, "$grabcmd 2>&1 |") || die "Error: Can't execute tv_grab\n";
open(grablog, ">$grablog") || die "Error: Can't write $grablog\n";
while (<f>) {
print;
print grablog;
}
close(f);
close(grablog);
}

# Modify episode/program IDs to remove 2 digits to match old format

print "\n\n---- Fixing Episode Number format\n\n";
open(xmlin, "<$xmlfile1") || die "Error: Can't open $xmlfile1";
open(xmlout, ">$xmlfile2") || die "Error: Can't open $xmlfile2";

while (<xmlin>) {
s/(\<program id=\'..)00/$1/;
s/(\<schedule program=\'..)00/$1/;
s/(\<programGenre program=\'..)00/$1/;
s/(\<series\>..)00/$1/;
print xmlout;
}
close(xmlin);
close(xmlout);

# Run mythfilldatabase

unless ($nofill) {
print "Running: $mythfillcmd\n";
open(f, "$mythfillcmd 2>&1 |") || die "Error: Can't execute
mythfilldatabase\n";
while (<f>) {
print;
}
}

print "\n---- Grab done at ", &prlocal(time()), "\n\n";
}

# Helper routine to print nicely formatted local time from a timestamp

sub prlocal {
my($intime) = @_;
my ($sec,$min,$hour,$mday,$mon,$year) = localtime($intime);
return(sprintf("%04d/%02d/%02d %02d:%02d:%02d",
$year+1900,$mon+1,$mday,$hour,$min,$sec));
}

david.schmidt.in.dallas at gmail

Aug 23, 2007, 5:50 AM

Post #2 of 17 (11497 views)
Permalink

Re: Script to use Schedules Direct with legacy MythTV installations [In reply to]

Thanks!!

I will definitely give this a try as soon as I get my subscription! I
(still) have my prototype box on FC3, 0.18.1 and it's also my primary
desktop/word processing/websurfing machine and I *really* didn't want
to rush through an upgrade to F7/0.20.2 with attendant breakage while
the wife is job-hunting!

Hopefully this will keep the old system up until I can quit
procrastinating and build out my production 0.20 system.

On 8/22/07, Josh Mastronarde <jmastron[at]gmail.com> wrote:
> I know we're all anxiously awaiting new releases to natively support
> Schedules Direct; I'm planning to upgrade as soon as I can, but in the
> meantime I made a small script to semi-automatically run a modified version
> of XMLTV's tv_grab_na_dd followed by mythfilldatabase, adjusting the new
> program ID format to match the old database. It's not pretty (though it
> does nice things like wait for the next suggested run time), and it's
> definitely not permanent, but will give me a bit of breathing room to
> upgrade on my own schedule. At the very least, you can extract the manual
> commands to roll your own solution; it took me some trial and error to
> figure out the right command line parameters.
>
> This assumes XMLTV (and tv_grab_na_dd) is already installed (an older
> version without the new URL is okay; see modification instructions); this
> was true in my ATRPMS "yum install mythtv-suite" following Jarod's guide.
> See the help/comments at the beginning for more instructions.
>
> Good luck; I hope this helps a few people.
>
> Josh
>
>
> #!/usr/bin/perl
>
> sub printhelp {
> print <<ENDOFHELP;
>
> SchedulesDirect semi-automatic downloader for legacy MythTV installations
> Josh Mastronarde, jmastron[at]gmail.com
> 8-22-07
>
> Syntax: sdfill <options>
> -nodownload Don't download data (use last downloaded file)
> -nofill Don't run mythfilldatabase
> -once Only run once
>
>
> This script will grab listings data, modify the episode ID field to match
> the old data, and run mythfilldatabase. If -once not specified, it will
> loop
> waiting for the next suggested time from the last run.
>
> Steps to prepare:
>
> 1) Find XMLTV's tv_grab_na_dd (was in /usr/bin for me); copy to
> ./tv_grab_na_sd
>
> 2) Modify ./tv_grab_na_sd; replace the "dd_service" line with:
> my $dd_service=
>
> 'http://docs.tms.tribune.com/tech/tmsdatadirect/zap2it/xtvd.wsdl
> ';
>
> 3) Run: ./tv_grab_na_sd --configure --config-file
> ~/.xmltv/tv_grab_na_sd.conf"
>
> 4) Replace "CA54023" with your lineup id (from the above .conf file; leave
> off
> the ":-", I think)
>
> If you want to run by hand, the key commands are in "$grabcmd" and
> "$mythfillcmd" below, along with the small section of code after "Modify
> episode/program IDs" to shorten the IDs.
>
>
> ENDOFHELP
> exit(1);
> }
>
> use Time::Local;
>
> # Defaults
>
> $xmlfile1 = "sdlist.xml";
> $xmlfile2 = "sdlist_fixed.xml";
> $grablog = "grab_log";
>
> $grabcmd = "./tv_grab_na_sd --config-file ~/.xmltv/tv_grab_na_sd.conf " .
> "--days 14 --dd-data $xmlfile1 --download-only --list-times";
>
> $mythfillcmd = "mythfilldatabase --dd-file 1 -1 CA54023 $xmlfile2";
>
> # Parse command line arguments
> while (@ARGV) {
> $arg = shift(@ARGV);
> if ($arg eq "-nodownload") {
> $nodownload = 1;
> } elsif ($arg eq "-nofill") {
> $nofill = 1;
> } elsif ($arg eq "-h") {
> &printhelp;
> } elsif ($arg eq "-once") {
> $once = 1;
> } else {
> push(@narg, $arg);
> }
> }
>
> $| = 1;
> print "\n---- SDfill started at ", &prlocal(time()), "\n\n";
>
> # Loop (but only once if -once flag set)
>
> $run = 1;
> while ($run) {
> $run = 0 if ($once);
>
> # Check for logfile from last grabber run and look for suggested time
>
> if (-e $grablog) {
> $suggtime = "";
> open(grablog, "<$grablog") || die "Error: Can't open $grablog\n";
> while (<grablog>) {
> if (($year,$mon,$day,$hour,$min,$sec) = ($_ =~
> /^suggestedTime
> \|(\d+)\-(\d+)\-(\d+)T(\d+):(\d+):(\d+)Z/)) {
> $suggtime =
> timegm($sec,$min,$hour,$day,$mon-1,$year-1900);
> }
> }
> close(grablog);
> if ($suggtime) {
> $waitsecs = $suggtime - time();
> $waithrs = int($waitsecs/3600);
> $waitmins = int($waitsecs/60) % 60;
>
> print "---- Sleeping until suggested ", &prlocal($suggtime), ",
> $waithrs:$waitmins from now ----\n\n";
> sleep($waitsecs);
> }
> }
>
>
> print "\n---- Grab starting at ", &prlocal(time()), "\n\n";
>
> unless ($nodownload) {
> unlink("sdlist.xml");
> print "Running: $grabcmd\n";
> open(f, "$grabcmd 2>&1 |") || die "Error: Can't execute tv_grab\n";
> open(grablog, ">$grablog") || die "Error: Can't write $grablog\n";
> while (<f>) {
> print;
> print grablog;
> }
> close(f);
> close(grablog);
> }
>
> # Modify episode/program IDs to remove 2 digits to match old format
>
> print "\n\n---- Fixing Episode Number format\n\n";
> open(xmlin, "<$xmlfile1") || die "Error: Can't open $xmlfile1";
> open(xmlout, ">$xmlfile2") || die "Error: Can't open $xmlfile2";
>
> while (<xmlin>) {
> s/(\<program id=\'..)00/$1/;
> s/(\<schedule program=\'..)00/$1/;
> s/(\<programGenre program=\'..)00/$1/;
> s/(\<series\>..)00/$1/;
> print xmlout;
> }
> close(xmlin);
> close(xmlout);
>
> # Run mythfilldatabase
>
> unless ($nofill) {
> print "Running: $mythfillcmd\n";
> open(f, "$mythfillcmd 2>&1 |") || die "Error: Can't execute
> mythfilldatabase\n";
> while (<f>) {
> print;
> }
> }
>
> print "\n---- Grab done at ", &prlocal(time()), "\n\n";
> }
>
>
> # Helper routine to print nicely formatted local time from a timestamp
>
> sub prlocal {
> my($intime) = @_;
> my ($sec,$min,$hour,$mday,$mon,$year) =
> localtime($intime);
> return(sprintf("%04d/%02d/%02d %02d:%02d:%02d",
> $year+1900,$mon+1,$mday,$hour,$min,$sec));
> }
>
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users[at]mythtv.org
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>
>
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

marc.aronson at comcast

Aug 23, 2007, 11:48 PM

Post #3 of 17 (11443 views)
Permalink

Re: Script to use Schedules Direct with legacy MythTV installations [In reply to]

Thanks for creating this script -- I may use it on 0.19 implementation! I
do have a question: I notice that you are dropping the final 2 digits from
the series id, program id, etc., if they end in "00". Based on other
threads I am reading, it looks like they added 2 digits to the end of both
of these fields, but based on your script, it looks like for the time being
these extra digits are always "00". My conclusion is that this approach will
work until they start actually using those extra digits for something
meaningful. Is that your understanding or is there a reason to believe those
extra digits will never actually be used? Thanks!

Marc

_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

jmastron at gmail

Aug 24, 2007, 1:54 AM

Post #4 of 17 (11445 views)
Permalink

Re: Script to use Schedules Direct with legacy MythTV installations [In reply to]

On 8/23/07, Marc Aronson <marc.aronson[at]comcast.net> wrote:
>
> Thanks for creating this script -- I may use it on 0.19 implementation! I
> do have a question: I notice that you are dropping the final 2 digits from
> the series id, program id, etc., if they end in "00". Based on other
> threads I am reading, it looks like they added 2 digits to the end of both
> of these fields, but based on your script, it looks like for the time
> being
> these extra digits are always "00". My conclusion is that this approach
> will
> work until they start actually using those extra digits for something
> meaningful. Is that your understanding or is there a reason to believe
> those
> extra digits will never actually be used? Thanks!

I'm not assuming they'll never be used, just that they won't be in the
couple of weeks it'll take me to cut over to a new MythTV version
:-) Perhaps a good modification would be to flag if the new digits aren't
00 and email a warning to check it out.

This was all about quick&dirty with the least touching of anything else
(though I got carried away a bit with the next suggested time wait code).

Josh

marc.aronson at comcast

Aug 24, 2007, 3:12 AM

Post #5 of 17 (11421 views)
Permalink

Re: Script to use Schedules Direct with legacy MythTV installations [In reply to]

Josh, thanks for that context -- makes sense!

Marc

-------------- Original message ----------------------
From: "Josh Mastronarde" <jmastron[at]gmail.com>
> On 8/23/07, Marc Aronson <marc.aronson[at]comcast.net> wrote:
> >
> > Thanks for creating this script -- I may use it on 0.19 implementation! I
> > do have a question: I notice that you are dropping the final 2 digits from
> > the series id, program id, etc., if they end in "00". Based on other
> > threads I am reading, it looks like they added 2 digits to the end of both
> > of these fields, but based on your script, it looks like for the time
> > being
> > these extra digits are always "00". My conclusion is that this approach
> > will
> > work until they start actually using those extra digits for something
> > meaningful. Is that your understanding or is there a reason to believe
> > those
> > extra digits will never actually be used? Thanks!
>
>
> I'm not assuming they'll never be used, just that they won't be in the
> couple of weeks it'll take me to cut over to a new MythTV version
> :-) Perhaps a good modification would be to flag if the new digits aren't
> 00 and email a warning to check it out.
>
> This was all about quick&dirty with the least touching of anything else
> (though I got carried away a bit with the next suggested time wait code).
>
> Josh

Attachments:

message-rfc822.eml (3.94 KB)

f-myth-users at media

Aug 30, 2007, 3:15 PM

Post #6 of 17 (11251 views)
Permalink

Script to use Schedules Direct with legacy MythTV installations [In reply to]

I'm having several issues with this. (I'm running 0.18.1 under Ubuntu
Breezy; yes, yes, yes, these are ancient, but upgrading this machine
is fraught with issues and I'm trying to do it on -my- schedule, not DD's.)

Several problems here. First:

> Date: Wed, 22 Aug 2007 21:56:23 -0700
> From: "Josh Mastronarde" <jmastron[at]gmail.com>

> 2) Modify ./tv_grab_na_sd; replace the "dd_service" line with:
> my $dd_service=
> 'http://docs.tms.tribune.com/tech/tmsdatadirect/zap2it/xtvd.wsdl';

Are you sure about that? That looks like an -old- URL. I just
fetched the Breezy version of xmltv (old) and it installed a
tv_grab_na_dd dated 8/30/2005. That version has the -same- URL
you have above; did you perhaps typo the new vs old URL?

(I haven't yet tried to figure out what the -right- URL is, but
I figured I'd check first while I'm poking around.)

Second:

I tried running
tv_grab_na_dd --configure --config-file ~/.xmltv/tv_grab_na_sd.conf
anyway to see what it would do (and in particular whether it would
somehow be grabbing data from SD, not DD, once configured), and it
bombed out:

Use of uninitialized value in pattern match (m//) at
/usr/bin/tv_grab_na_dd line 501.

Time OFFSET Selection (note: not a timeZONE)
It is better to specify +0000 and let the final application
deal with a local conversion (helps with DST issues), but you
can specify a Time Offset if desired.
+0000 UTC
-0400 Eastern Daylight
-0500 Eastern Standard or Central Daylight
-0600 Central Standard or Mountain Daylight
-0700 Mountain Standard or Pacific Daylight
-0800 Pacific Standard

This is while it's trying to compute a TZ offset (that line is
$opt_tz_offset='+0000' if $opt_tz_offset =~ /UTC/i;
), but I don't know why it's having problems.

Third:

If I have two lineups instead of one, what do I do? Run the script
twice? Configure it with alternate ID's? And how do I -find- those
lineup ID's? For now, I can log into labs, or check my existing myth
configuration, but if I was starting from scratch, I'm not sure how
I'd know them---editing my lineups at SD don't show any such lineup
ideas; am I looking in the wrong place?

Tnx for any help...
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

f-myth-users at media

Aug 30, 2007, 3:37 PM

Post #7 of 17 (11247 views)
Permalink

Script to use Schedules Direct with legacy MythTV installations [In reply to]

> Date: Thu, 30 Aug 2007 18:15:19 -0400 (EDT)
> From: f-myth-users[at]media.mit.edu

> I tried running
> tv_grab_na_dd --configure --config-file ~/.xmltv/tv_grab_na_sd.conf
> anyway to see what it would do (and in particular whether it would
> somehow be grabbing data from SD, not DD, once configured), and it
> bombed out:

Err, I take it back, -that's- not where it bombed out: I failed to
notice that it hadn't actually given me a shell prompt back and was
actually sitting at a question (since it was running in an Emacs
buffer).

So I hit return for offset, typed in my "username" as user[at]host,
typed in my password, and -then- it blew out with:

Fetching from DataDirect
Fetched 0 k/bytes in 1 seconds
loading data

WARNING: error parsing DD xml:
mismatched tag at line 1, column 942, byte 942 at
/usr/lib/perl5/XML/Parser.pm line 187

Partial XML follows:
<html><head><title>Apache Tomcat/5.5.17 - Error report</title><style></style> </head><body><h1>HTTP Status 401 - </h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>This request requires HTTP authentication ().</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/5.5.17</h3></body></html>

ERROR: Bad XML from DD, cannot continue

[.Breezy's version of XMLTV is 0.5.40-1ubuntu1, FWIW, though I still
wonder if the URL I mentioned in my previous message is correct.]
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

greg at phaze

Aug 30, 2007, 4:09 PM

Post #8 of 17 (11247 views)
Permalink

Re: Script to use Schedules Direct with legacy MythTV installations [In reply to]

>
> [.Breezy's version of XMLTV is 0.5.40-1ubuntu1, FWIW, though I still
> wonder if the URL I mentioned in my previous message is correct.]

I'm pretty sure the SD support was implemented in 0.5.43 so your version appears to be too old.

But some xmltv experts can jump in if I'm wrong

_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

jmastron at gmail

Aug 30, 2007, 4:15 PM

Post #9 of 17 (11237 views)
Permalink

Re: Script to use Schedules Direct with legacy MythTV installations [In reply to]

On 8/30/07, f-myth-users[at]media.mit.edu <f-myth-users[at]media.mit.edu> wrote:
>
> I'm having several issues with this. (I'm running 0.18.1 under Ubuntu
> Breezy; yes, yes, yes, these are ancient, but upgrading this machine
> is fraught with issues and I'm trying to do it on -my- schedule, not
> DD's.)
>
> Several problems here. First:
>
> > Date: Wed, 22 Aug 2007 21:56:23 -0700
> > From: "Josh Mastronarde" <jmastron[at]gmail.com>
>
> > 2) Modify ./tv_grab_na_sd; replace the "dd_service" line with:
> > my $dd_service=
> > '
> http://docs.tms.tribune.com/tech/tmsdatadirect/zap2it/xtvd.wsdl';
>
> Are you sure about that? That looks like an -old- URL. I just
> fetched the Breezy version of xmltv (old) and it installed a
> tv_grab_na_dd dated 8/30/2005. That version has the -same- URL
> you have above; did you perhaps typo the new vs old URL?
>
> (I haven't yet tried to figure out what the -right- URL is, but
> I figured I'd check first while I'm poking around.)

You're right, I put the old URL in. The correct one is:

my $dd_service=
'
http://docs.tms.tribune.com/tech/tmsdatadirect/schedulesdirect/tvDataDelivery.wsdl
'

I got that from somewhere in the SD Forums (published by one of the
founders).

Josh

f-myth-users at media

Aug 30, 2007, 4:21 PM

Post #10 of 17 (11234 views)
Permalink

Script to use Schedules Direct with legacy MythTV installations [In reply to]

> Date: Thu, 30 Aug 2007 18:37:44 -0400 (EDT)
> From: f-myth-users[at]media.mit.edu

> WARNING: error parsing DD xml:
> mismatched tag at line 1, column 942, byte 942 at
> /usr/lib/perl5/XML/Parser.pm line 187

Since I'd seen mention of an explicit XMLTV version in various mail,
I decided to try flushing Breezy's version and building 0.5.48 from
source. -That- version works---and btw, doesn't require any changes
to URLs. So it looks like Josh's script instructions need some small
tweaks; presumably you need XMLTV that's newer than -some- version,
and if it's between that version and 48, you need a different URL,
and if it's 48 or newer, you can use it unmodified.

I ran it twice, specifying different config files for my two different
lineups, since specifying the same one each time just overwrote one
listing with another.

So now I'm at the point of actually trying it with 0.18.1's mfdb...

(And sorry for the noise, but I guess it won't hurt to have this
in the thread archives somewhere so others don't get tripped up.)
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

mtdean at thirdcontact

Aug 30, 2007, 4:22 PM

Post #11 of 17 (11211 views)
Permalink

Re: Script to use Schedules Direct with legacy MythTV installations [In reply to]

On 08/30/2007 07:09 PM, Greg Estabrooks wrote:
>> [.Breezy's version of XMLTV is 0.5.40-1ubuntu1, FWIW, though I still
>> wonder if the URL I mentioned in my previous message is correct.]
>>
> I'm pretty sure the SD support was implemented in 0.5.43 so your version appears to be too old.
>

Wasn't it 0.5.48 (current version)?

> But some xmltv experts can jump in if I'm wrong
>

Ooops. That's definitely not me.

Mike
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

f-myth-users at media

Aug 30, 2007, 4:23 PM

Post #12 of 17 (11205 views)
Permalink

Script to use Schedules Direct with legacy MythTV installations [In reply to]

> Date: Thu, 30 Aug 2007 16:15:01 -0700
> From: "Josh Mastronarde" <jmastron[at]gmail.com>

> You're right, I put the old URL in. The correct one is:

> my $dd_service=
> '
> http://docs.tms.tribune.com/tech/tmsdatadirect/schedulesdirect/tvDataDelivery.wsdl
> '

Aha! And indeed, that's also the URL in 0.5.48's XMLTV.

Now it's time to actually try -your- script now that I seem to have
XMLTV working. (It complained about SOAP::Lite-0.67 being required,
whereas I have SOAP::Lite-0.60, but -appears- to have worked so far.
Let's hope the complaint is spurious... And if anyone knows
otherwise, speak up...)
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

greg at phaze

Aug 30, 2007, 4:23 PM

Post #13 of 17 (11218 views)
Permalink

Re: Script to use Schedules Direct with legacy MythTV installations [In reply to]

> But some xmltv experts can jump in if I'm wrong

Or I can jump all over me for not reading your entire message before replying :)

_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

f-myth-users at media

Aug 30, 2007, 5:55 PM

Post #14 of 17 (11431 views)
Permalink

Script to use Schedules Direct with legacy MythTV installations [In reply to]

Okay, so here's what I hope is the final scoop on the process:

(a) Upgrade to XMLTV 0.5.48 if you possibly can. If you can't, modify
the dd_service definition to be
http://docs.tms.tribune.com/tech/tmsdatadirect/schedulesdirect/tvDataDelivery.wsdl
instead of
http://docs.tms.tribune.com/tech/tmsdatadirect/zap2it/xtvd.wsdl

If your XMLTV version is older than <some undetermined version>
(and 0.5.40 is certainly too old), it'll barf and you'll -have-
to upgrade it. If you're using a packaged version, you'll
probably have to uninstall xmltv and xmltv-utils, then download
the new version from xmltv.org and build it via "perl Makefile.PL"
in its unpacked directory. NOTE that it will default to -not-
building tv_grab_na_dd (presumably because it was superceded by
Zap2it?) and you'll have to decline the default configuration and
say "yes" at the appropriate spot when it asks you about each
piece. (In Breezy, it also complained that my SOAP::Lite 0.60 was
too old and that 0.67 was required, but I ignored this and it
doesn't seem to have done me any harm. Knock on plastic.)

If you're building from source, you have to do "make" -after- that
"perl Makefile.PL" (think of that command as ./configure :) and
then, if you want to make things tidy for your packing system,
install it using "checkinstall make install" at that point. Note
that the "perl Makefile.PL; make" bit is easy to miss if you're
using to building (e.g.) normal C code; a "make" before the perl
step will just barf 'cause there's no makefile yet---and for some
reason, there's no INSTALL file in the distribution: the build
instructions are in the release notes for the version. Go figure.

(b) If you've installed a package, or if your XMLTV is already
up-to-date and doesn't require the mods for the new URL, you can
then do "tv_grab_na_dd --configure --config-file ~/.xmltv/foo.conf"
but ~/.xmltv MUST EXIST FIRST or it'll blow out on you. (If you've
edited the original script according to Josh's instructions, then
you'll have to do ./tv_grab_na_sd, etc.) If you've been just
using normal DD until now w/o an XMLTV grabber, you won't already
have ~/.xmltv.

(c) The configure step will emit a nasty-looking error message
Use of uninitialized value in pattern match (m//) at /usr/bin/tv_grab_na_dd line 517.
even in the latest version. It's apparently harmless, but it sure
seems like somebody should file a bug report; should be easy
enough to fix. When it prompted me for my timezone offset, I
just took the default (+0000) even though that's nothing near
my timezone; it looks like mfdb is doing the right thing here,
since I compared my listings on my devo machine to the production
machine and they agree about when things are showing.

(d) Remember that your SD username looks like an email address.
(I got -that- step correct right out of the gate. Yay me.)

(e) Allow it to put the password in the file, by typing your password
at it. (You didn't use some -valuable- password for SD, did you?
Remember that it's not an SSL connection, so presumably it's being
blatted out in the clear for anyone to sniff anyway every time you
get new listings. I haven't rigorously checked to see if maybe
the auth is being done w/crypto, but it's easy to just assume it's
insecure and not use a password you care about...)

(f) If you have multiple lineups, you'll have to run the configure
step above more than once, supplying a different --config-file for
each run; otherwise, they'll just overwrite each other. The
lineup ID's were apparently automatically fetched from SD; I
didn't have to do anything special & the config script just
prompted me for which lineup I wanted to configure each time.

(g) If you have more than one lineup, the most convenient thing to do
is to create multiple copies of Josh's script with the correct
lineup in each copy, and call them sequentially. (This will cause
Myth's mfdb to run more than once, but oh well.) Note that
there's quite a pregnant pause with no output when the script
runs; it takes about 3 minutes per lineup for mine to finish, and
there's not much output while it's happening. Use "top" or ls the
current working directory to reassure yourself that things are
actually happening... :) This also means you should be using the
"-once" arg for each call (so you get control back) and should
then be calling all n of them via a script that's called by cron;
yeah, that breaks the recommended-next-download logic, but it's
presumed that this is a short-term stopgap for a small number of
people. (I have no idea if there's enough locking in mfdb for it
to work if you instead let them each run in their own process and
they fire off asynchronously and perhaps collide; I don't feel like
being the practice target if the locking isn't adequate.)

(h) Josh's script has all kinds of hardcoded assumptions in its
$grabcmd line; you'll have to tell it where -your- copy of
tv_grab_na_dd (or _sd, if you've followed his naming convention)
is living if it's not defaultly in your search path (it will be,
if you built a package), and the relevant config file name, and it
had better match the lineup ID you specify in the next line down
in $mythfillcmd. Not rocket science, but...

Thanks for the script, Josh! It's helped take the pressure off
upgrading my ancient Myth boxes, so now I can do so at a convenient
point in various release cycles (probably using packages and when Gutsy
is finally released in a couple months, since many lirc/ivtv/kernel
issues will presumably have calmed down by that point).
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

f-myth-users at media

Aug 30, 2007, 6:07 PM

Post #15 of 17 (11229 views)
Permalink

Script to use Schedules Direct with legacy MythTV installations [In reply to]

Knew I'd forget something...

> Date: Thu, 30 Aug 2007 20:55:23 -0400 (EDT)
> From: f-myth-users[at]media.mit.edu

> Myth's mfdb to run more than once, but oh well.) Note that
> there's quite a pregnant pause with no output when the script
> runs; it takes about 3 minutes per lineup for mine to finish, and
> there's not much output while it's happening.

(This is by contrast to the "one wget per day of data, and per lineup,
with verbose logging" that the old-style DD fetches used.)

> Use "top" or ls the
> current working directory to reassure yourself that things are
> actually happening... :) This also means you should be using the
> "-once" arg for each call (so you get control back) and should
> then be calling all n of them via a script that's called by cron;

..which means you should REMEMBER TO TURN OFF whatever cronjob or myth
job was doing the -old- fetches from DataDirect! Presumably they'll
just break on their own at some point, but they might cause mfdb to
then delete all your listings when they fail to grab any data, or
something like that. Again, better not to find out the hard way...
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

jmastron at gmail

Aug 30, 2007, 8:23 PM

Post #16 of 17 (11216 views)
Permalink

Re: Script to use Schedules Direct with legacy MythTV installations [In reply to]

On 8/30/07, f-myth-users[at]media.mit.edu <f-myth-users[at]media.mit.edu> wrote:
>
>
> (h) Josh's script has all kinds of hardcoded assumptions in its
> $grabcmd line; you'll have to tell it where -your- copy of
> tv_grab_na_dd (or _sd, if you've followed his naming convention)
> is living if it's not defaultly in your search path (it will be,
> if you built a package), and the relevant config file name, and it
> had better match the lineup ID you specify in the next line down
> in $mythfillcmd. Not rocket science, but...
>
> Thanks for the script, Josh! It's helped take the pressure off
> upgrading my ancient Myth boxes, so now I can do so at a convenient
> point in various release cycles (probably using packages and when Gutsy
> is finally released in a couple months, since many lirc/ivtv/kernel
> issues will presumably have calmed down by that point).

I'm glad it helps. Yes, there are a lot of hardcoded assumptions; I tried
to at least move them into variables near the top instead of scattered
throughout, but this was never intended to be a software engineering
masterpiece, just something to tide me over for a couple weeks :-). I
published it as much for anyone who wants to see examples of the tv_grab
and mythfilldatabase commands, and ID modofications needed to use in their
own scripts as anything else.

One thing to note -- the processing section only fixes the episode ID
change; it doesn't do anything for the new/repeat flag changes, and any
other format differences there might be. This might become an issue in a
couple weeks when the new season starts. I'm personally planning to move to
0.20.2 or trunk this weekend to get native support.

Josh

bharnish1004 at gmail

Sep 3, 2007, 12:20 PM

Post #17 of 17 (11130 views)
Permalink

Re: Script to use Schedules Direct with legacy MythTV installations [In reply to]

I want to say thanks for this script. I'm running an old (legacy)
MythTV install (0.18.1) and I'm trying to use the "If it ain't broke,
don't try and fix it" theory with my install.

Also, I noticed that it needs another fix up on the crew members:
s/(\<crew program=\'..)00/$1/;

Also, I wanted to say that it *does* work with XMLTV 0.5.40, I saw
someone mention that they thought that version was too old to work,
but it does.

I saw mention of some new/repeat flag changes, but I thought that was
taken care of on the client side by mfdb instead of coming from dd/sd.
Does anyone have more information on this?

Thanks for the script!
- Brian
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com