Mailing List Archive: MythTV: Users

Getting local-access stations into Schedules Direct

Index | Next | Previous | View Threaded

f-myth-users at media

Oct 28, 2009, 7:38 PM

Post #1 of 13 (306 views)
Permalink

Getting local-access stations into Schedules Direct

None of the local community-access television stations carried by my
cable feed have any listings in Schedules Direct---they're listed as
LOOR003, LOOR016, etc etc but SD has no listings for them.

A couple of them even have some interesting programming on (whether
you're interested in local politics or in refeeds of things like
certain NASA programs, etc), and have at least a week of scheduling
listed on their own websites. But I never think to check 'em out
because, of course, it takes extra effort to go scan the sites, much
less schedule them... :)

So I've got two ways to proceed:
(a) Write a tool to scrape their sites & stuff the info into the
same XML framework mfdb is inhaling. Fragile but obviously
doable if I don't mind some grody code. (And if anyone has
suggestions that beat a cheesy bash/perl script but doesn't
have a silly learning curve, spill... :)
(b) Actually get them to submit their data to Tribune or whoever
is the upstream feeder for SD---assuming they're disciplined
enough to actually do so.

Obviously (b) is the better long-term solution, and would help
everybody in the area. But how do I go about doing this? Is it
documented anywhere what the procedure is to ask a station to start
submitting their data, and (the kicker) do stations have to pay any
money to anyone to do so? I've done some searching both generally
and in the SD pages & forums and haven't seen anything directly
addressing this question.

Thanks!
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

nick.rout at gmail

Oct 29, 2009, 2:38 PM

Post #2 of 13 (274 views)
Permalink

Re: Getting local-access stations into Schedules Direct [In reply to]

On Thu, Oct 29, 2009 at 3:38 PM, <f-myth-users[at]media.mit.edu> wrote:
> None of the local community-access television stations carried by my
> cable feed have any listings in Schedules Direct---they're listed as
> LOOR003, LOOR016, etc etc but SD has no listings for them.
>
> A couple of them even have some interesting programming on (whether
> you're interested in local politics or in refeeds of things like
> certain NASA programs, etc), and have at least a week of scheduling
> listed on their own websites. But I never think to check 'em out
> because, of course, it takes extra effort to go scan the sites, much
> less schedule them... :)
>
> So I've got two ways to proceed:
> (a) Write a tool to scrape their sites & stuff the info into the
> same XML framework mfdb is inhaling. Fragile but obviously
> doable if I don't mind some grody code. (And if anyone has
> suggestions that beat a cheesy bash/perl script but doesn't
> have a silly learning curve, spill... :)
> (b) Actually get them to submit their data to Tribune or whoever
> is the upstream feeder for SD---assuming they're disciplined
> enough to actually do so.
>
> Obviously (b) is the better long-term solution, and would help
> everybody in the area. But how do I go about doing this? Is it
> documented anywhere what the procedure is to ask a station to start
> submitting their data, and (the kicker) do stations have to pay any
> money to anyone to do so? I've done some searching both generally
> and in the SD pages & forums and haven't seen anything directly
> addressing this question.

Option 3: ask if they have an xmltv feed, or persuade them to generate
one. Smaller/community stations are sometimes more amenable to this.

There are hints on writing xmltv generators on the xmltv web site.
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

f-myth-users at media

Nov 1, 2009, 12:55 AM

Post #3 of 13 (254 views)
Permalink

Getting local-access stations into Schedules Direct [In reply to]

> Date: Sat, 31 Oct 2009 13:04:26 -0500
> From: Robert Eden <rmeden[at]yahoo.com>

> How about: http://wiki.xmltv.org/index.php/HowtoWriteAGrabber

Ah, I missed that the first time though. Thanks. But I was more
thinking about how the station -exports- its data to XML, not how
I inhale it; obviously it depends on where there data comes from,
but I presume there are also some tools to take common formats and
produce something reasonable. I can go ask on the xmltv-devel list
if we want to keep it off mythtv (and this depends on the station
being willing to participate at all).

(Though the horrifying thought just occurred to me that if I ask if
they're willing to do this, they're likely to turn around and volunteer
me to do the work. I've done enough theatrical productions to know
how this game gets played... :)

> Feel free to post on the XMLTV-DEVEL list for advice and help. The
> first step is to contact the station and see if they can provide the
> data with redistribution rights or host the data everyone could pick up.
> (and XMLTV grabber could reformat it)

Right. I'll talk to them.

> BTW, if you contact the station and they are willing to provide data for
> redistribution data but not host it, SD or XMLTV.org could do it.

Ah, that's good to know.

> If the station won't provide a raw-data feed, and there is nothing on
> the site's terms of service to exclude a scraper, write one and it can
> be added to XMLTV.

Ok. I didn't see any TOS, but I could well have missed it; obviously
I'll be asking them directly about such things while talking to them
about having them export XML anyway.

> SD can't merge it with the Tribune data, but applications may be able
> to, or you can always use XMLTV's TV_CAT.

Ah! Didn't know about that utility. That seems like it'll make it
easy to tell people how to pick up their data if they make it available.

Thanks!

P.S. So does it cost money to get data into Tribune's feed? My guess
is "yes" simply because I've yet to see a local-access station do so.
(I have several, plus "Educational Access" and "Government Access",
few of which I've ever bothered to watch 'cause, well, do they make
their schedules easy to find? :)
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

jack at greendesk

Nov 1, 2009, 9:20 AM

Post #4 of 13 (248 views)
Permalink

Re: Getting local-access stations into Schedules Direct [In reply to]

f-myth-users[at]media.mit.edu wrote:
> None of the local community-access television stations carried by my
> cable feed have any listings in Schedules Direct
> do stations have to pay any
> money to anyone to do so?

I program two Community Access stations, and yes Tribune has to be paid
to enter station data. The cable co that carries us locally agreed to
pay some years ago, and for years, I submitted a word file (!) for
Tribune to apparently retype into their application. That got updated
every week, then it got passed onto another person at Tribune, they no
longer reply to my email or update the listings I send. So we have
listings on Tribune, but it is a few years old!

I don't have budget for any fancy dedicated application to create the
schedule, it is created in excel, and imported into our playback system
and a linked access table for printed reports. I have been giving
Tribune the excel sheet since they finally told me that would make their
life easier.

_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

nick.rout at gmail

Nov 1, 2009, 12:07 PM

Post #5 of 13 (246 views)
Permalink

Re: Getting local-access stations into Schedules Direct [In reply to]

On Mon, Nov 2, 2009 at 6:20 AM, Jack McGee <jack[at]greendesk.net> wrote:
> f-myth-users[at]media.mit.edu wrote:
>>
>> None of the local community-access television stations carried by my
>> cable feed have any listings in Schedules Direct
>> do stations have to pay any
>> money to anyone to do so?
>
> I program two Community Access stations, and yes Tribune has to be paid to
> enter station data. The cable co that carries us locally agreed to pay some
> years ago, and for years, I submitted a word file (!) for Tribune to
> apparently retype into their application. That got updated every week, then
> it got passed onto another person at Tribune, they no longer reply to my
> email or update the listings I send. So we have listings on Tribune, but it
> is a few years old!
>
> I don't have budget for any fancy dedicated application to create the
> schedule, it is created in excel, and imported into our playback system and
> a linked access table for printed reports. I have been giving Tribune the
> excel sheet since they finally told me that would make their life easier.

I would have thought that exporting the data as csv from excel and
then running it through some xml tool would produce valid xmltv data.

I am not sure about access' ability to export in arbitrary formats,
but there must be some tool to take data from a database and export it
as xml.

>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users[at]mythtv.org
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

rmeden at yahoo

Nov 3, 2009, 6:38 AM

Post #6 of 13 (204 views)
Permalink

Re: Getting local-access stations into Schedules Direct [In reply to]

On 11/1/2009 1:56 AM, f-myth-users[at]media.mit.edu wrote:
> Ah, I missed that the first time though. Thanks. But I was more
> thinking about how the station -exports- its data to XML, not how
> I inhale it; obviously it depends on where there data comes from,
> but I presume there are also some tools to take common formats and
> produce something reasonable.
You would think.... but when Zap2IT Labs shut down and we were looking
into how to recover (before SD) we contacted a number of stations
looking for a direct feed. Basically it's a mess. There is no standard
format! Some have it in word, some excel, some proprietary formats.
XMLTV actually seemed to be the closest thing to a "standard" format! We
were shocked to find that within a TV station, it's often easier/cheaper
for them to buy a Tribune feed than to transfer the data internally!

So Tribune (and presumably the other US source Rovi/Macrovision/Gemstar)
has to retype a lot of the data. You would think the stations would
want to standardize on something (even CSV!) at least for internal use,
but that's not the case. I can think of a few logical reasons for this,
but still find it crazy. The mess is probably one reason why the data
costs so much.

Robert
Schedules Direct
XMLTV

_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

beww at beww

Nov 3, 2009, 7:09 AM

Post #7 of 13 (202 views)
Permalink

Re: Getting local-access stations into Schedules Direct [In reply to]

On Tuesday 03 November 2009 07:38:51 Robert Eden wrote:
> On 11/1/2009 1:56 AM, f-myth-users[at]media.mit.edu wrote:
> > Ah, I missed that the first time though. Thanks. But I was more
> > thinking about how the station -exports- its data to XML, not how
> > I inhale it; obviously it depends on where there data comes from,
> > but I presume there are also some tools to take common formats and
> > produce something reasonable.
>
> You would think.... but when Zap2IT Labs shut down and we were looking
> into how to recover (before SD) we contacted a number of stations
> looking for a direct feed. Basically it's a mess. There is no standard
> format! Some have it in word, some excel, some proprietary formats.
> XMLTV actually seemed to be the closest thing to a "standard" format! We
> were shocked to find that within a TV station, it's often easier/cheaper
> for them to buy a Tribune feed than to transfer the data internally!
>
> So Tribune (and presumably the other US source Rovi/Macrovision/Gemstar)
> has to retype a lot of the data. You would think the stations would
> want to standardize on something (even CSV!) at least for internal use,
> but that's not the case. I can think of a few logical reasons for this,
> but still find it crazy. The mess is probably one reason why the data
> costs so much.

There are two primary suppliers of broadcast "Traffic" systems, BIAS and
Enterprise. These are systems for scheduling (and invoicing and billing etc.)
commercials, and they both interface with the common broadcast automation
systems, producing a file that can be used directly by the station's
automated master control switcher.

Getting BIAS or Enterprise files into formats that can be used by Library
Management Systems or video servers is a real mess, several companies produce
software products to do this, but each one has to be pretty well hand
crafted.

The formats used by BIAS and Enterprise are proprietary, and using them in any
way probably involves payment to the respective vendor.

My experience is that the various departments at a TV station (Traffic,
Engineering, sales, billing etc.) do not talk to each other, and getting any
agreement on file formats would probably be impossible, which is essentially
what you discovered.

I doubt that any Local Access channels use the major vendors like BIAS or
Enterprise, for one thing they do not sell commercials, which is the core
function of the commercial systems.

BTW - "Traffic" does not refer to anything involving the station's parking
lot. It's basically the scheduling of content, and can get pretty complex,
like not having two competing commercials adjacent to each other, and other
such considerations. For example, it used to be a no-no to have two car
commercials next to each other, nowadays it's considered OK to have a foreign
and a domestic car ad adjacent to each other, but not two domestics etc. Some
ad buys require specific time slots, or specific adjacencies, it can get
quite complex. Then there are considerations like killing any airline ads in
a newscast that contains a story about an air crash, or not putting the Ford
spot next to a story about a major Ford recall.

--
Brian Wood
beww[at]beww.org
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

david at segall

Nov 3, 2009, 8:01 AM

Post #8 of 13 (203 views)
Permalink

Re: Getting local-access stations into Schedules Direct [In reply to]

Robert Eden wrote:
> On 11/1/2009 1:56 AM, f-myth-users[at]media.mit.edu wrote:
> You would think the stations would
> want to standardize on something (even CSV!) at least for internal use,
> but that's not the case. I can think of a few logical reasons for this,

What are they?

--
David Segall Email: david[at]segall.net
Profectus Pty Ltd Phone: +61 3 9830 5026
1 Myrtle Road Mobile: +61 4 1756 7246
Canterbury, VIC, 3126 Australia Fax: +61 3 9830 0959
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

rmeden at yahoo

Nov 3, 2009, 8:22 AM

Post #9 of 13 (196 views)
Permalink

Re: Getting local-access stations into Schedules Direct [In reply to]

On 11/3/2009 10:01 AM, David Segall wrote:
>
> Robert Eden wrote:
>> On 11/1/2009 1:56 AM, f-myth-users[at]media.mit.edu wrote:
>> You would think the stations would want to standardize on something
>> (even CSV!) at least for internal use, but that's not the case. I
>> can think of a few logical reasons for this,
>
> What are they?
>
(in addition to Brian's excellent explanation about the traffic systems)...

Maggie does local scheduling and has a pretty schedule template in excel
using merged fields etc. She's always done it that way and doesn't want
to change.

Jane the manager, doesn't like the way the CSV file looks when it's
printed out. Plus, she likes the way Maggy formats it.

Tim took this over from Betty. He doesn't know how to format things
pretty well, but he can edit the template word doc.

Frank in engineering needs a file in a special format for their ATSC
guide. He's not a programmer and can't convert a CSV or Maggie's excel
file. His application can get the data from a consolidator automatically.

(BTW, Maggie and Tim only have info on local programming... Frank needs
network stuff too... and the whole cycle repeats itself)

George the VP agrees this is a mess, but by paying $ to a consolidator,
all the problems goes away. It integrates with Frank's (already
purchased) application (no import step), doesn't require any retraining,
and $ (single lineup) is relatively cheap.

How's that for some plausible reasons?

Robert

_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

gull at gull

Nov 3, 2009, 10:08 AM

Post #10 of 13 (196 views)
Permalink

Re: Getting local-access stations into Schedules Direct [In reply to]

Man, this is taking me back. Fifteen years ago I worked for a very
small public access cable station. It was about as low-tech as you can
get. Part of my job was to make up the schedules, and I'd grid them up
by hand on paper forms. None of our shows were standard lengths so I
got really good at adding up hours and minutes. I also had to manually
switch the tapes and answer the phones, so I had a strong incentive to
get the math right. ;)
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

f-myth-users at media

Nov 3, 2009, 5:38 PM

Post #11 of 13 (172 views)
Permalink

Getting local-access stations into Schedules Direct [In reply to]

> Date: Tue, 03 Nov 2009 08:38:51 -0600
> From: Robert Eden <rmeden[at]yahoo.com>

> On 11/1/2009 1:56 AM, f-myth-users[at]media.mit.edu wrote:
> > Ah, I missed that the first time though. Thanks. But I was more
> > thinking about how the station -exports- its data to XML, not how
> > I inhale it; obviously it depends on where there data comes from,
> > but I presume there are also some tools to take common formats and
> > produce something reasonable.

> You would think.... but when Zap2IT Labs shut down and we were looking
> into how to recover (before SD) we contacted a number of stations
> looking for a direct feed. Basically it's a mess. There is no standard
> format! Some have it in word, some excel, some proprietary formats.
> XMLTV actually seemed to be the closest thing to a "standard" format! We
> were shocked to find that within a TV station, it's often easier/cheaper
> for them to buy a Tribune feed than to transfer the data internally!

*sigh*... :)

I'm actually not all that shocked. We all know that large corporations
often have really badly messed-up internal information flows. Let me
tell you about the couple of times I was trying to get residential
ISDN installed back in the 90's... :)

Though I guess I -am- surprised that, even though there was no
standard -internal- mechanism, Tribune was willing to take the
data in a variety of haphazard formats and then deal with it
themselves. Guess they can charge enough at both ends to make
it worth their while. [.And no doubt knew that asking the stations
to fix it internally was -far- beyond their likely competence...
---and given Brian Wood's hilarious description, that's probably
correct!]

[.If stations are buying Tribune feeds, then last-minute changes might
blindside them internally; I have some that often don't decide what
they're airing on certain days until pretty late---late enough that
the data can't make it into SD in time, at least, so either one slot
is totally wrong, or the entire day is TBA. The History Channel seems
a recent and frequent offender for their Sunday lineups, for example
---often TBA for 12 hours at a stretch.]

I guess this means I won't waste my time trying to find tools to
convert from the stations' formats into XML (assuming they're amenable
in the first place) and will either tell their people where to find
the format documentation, or I'll write the tools myself, depending on
whether they want to cope in-house or draft me to do the work...

> So Tribune (and presumably the other US source Rovi/Macrovision/Gemstar)
> has to retype a lot of the data. You would think the stations would
> want to standardize on something (even CSV!) at least for internal use,
> but that's not the case. I can think of a few logical reasons for this,
> but still find it crazy. The mess is probably one reason why the data
> costs so much.

I'll bet.

[.But my impression was also that Tribune retyped the program
descriptions, at least, even if they would prefer to have the rest of
the data machine-transferrable, so that Tribune owned the copyright to
the feed as a "creative work". Is this incorrect?]

(Some stations provide enormously-detailed program information (e.g.
Sundance) that I wish could soemhow be transferred down into the stuff
that SD provides, but I imagine they either don't want to bundle it in
or there's nowhere logical in the XML to put it, e.g., you still want
the one-liner descriptions for most purposes but it'd be nice to have
"give me Sundance's four paragraphs of history about this film" somewhere
in the UI---but the UI ain't got it 'cause the XML ain't got it. Having
to use their terrible Flash-based website to find it is just too much work. :)

Thanks again!
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

russ.vanwinkle at gmail

Nov 3, 2009, 7:04 PM

Post #12 of 13 (169 views)
Permalink

Re: Getting local-access stations into Schedules Direct [In reply to]

> [.But my impression was also that Tribune retyped the program
> descriptions, at least, even if they would prefer to have the rest of
> the data machine-transferrable, so that Tribune owned the copyright to
> the feed as a "creative work". Is this incorrect?]

When I worked at a local TV station, the schedule we sent out was a
*printout* of an Excel spreadsheet, that the receptionist made up and
mailed to Tribune (and a few other recipients such as area newspapers)
weekly. Any revisions to prior schedules? Text notes on the later
schedules. Not surprisingly, our local newspaper tended to get our
schedule wrong pretty frequently.

Network programming only indicated the show name, Tribune would
presumably get the episode info from the network. Syndicated
programming showed the episode production code, so I'd assume Tribune
had a database to pull the description from for those. The remainder
of our schedule was either paid programming or local news, each of
which would just get a generic description. Either way, a minimum of
"re-typing".

--
Russ D. Van Winkle
_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

rmeden at yahoo

Nov 3, 2009, 9:01 PM

Post #13 of 13 (164 views)
Permalink

Re: Getting local-access stations into Schedules Direct [In reply to]

On 11/3/2009 7:38 PM, f-myth-users[at]media.mit.edu wrote:
>
> [.If stations are buying Tribune feeds, then last-minute changes might
> blindside them internally;
Does the station care about last-minute changes? What does the ATSC
guide look like when things like a football game goes long?
> I guess this means I won't waste my time trying to find tools to
> convert from the stations' formats into XML (assuming they're amenable
> in the first place) and will either tell their people where to find
> the format documentation, or I'll write the tools myself, depending on
> whether they want to cope in-house or draft me to do the work...
>
I'd be shocked if they'll spend the resources to output XMLTV. I'd
consider yourself lucky if they give you a nice CSV

> [.But my impression was also that Tribune retyped the program
> descriptions, at least, even if they would prefer to have the rest of
> the data machine-transferrable, so that Tribune owned the copyright to
> the feed as a "creative work". Is this incorrect?]
>
Not a bad angle.. plausible. If the know they're going to retype it
anyway, they probably don't care about getting it printed/faxed.
> (Some stations provide enormously-detailed program information (e.g.
> Sundance) that I wish could soemhow be transferred down into the stuff
> that SD provides, but I imagine they either don't want to bundle it in
Tribune has *lots* of variations on their products. We may be able to
get it, but I'm not sure it would be worth the cost. (most people
probably wouldn't want to see it). IIRC there is a place in the Data
Direct DTD for it.

Robert

_______________________________________________
mythtv-users mailing list
mythtv-users[at]mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com