Gossamer Forum
Home : Products : Links 2.0 : Customization :

Headline News from Yahoo?

Quote Reply
Headline News from Yahoo?
Check this site yahoosuck at http://www.yahoosuck.com

now how on earth did he display news headlines from the real yahoo on his links home page?

i wanna know real bad!

bill
Quote Reply
Re: Headline News from Yahoo? In reply to
it;s pretty easy with the perl LWP library...


just gotta get the page, parse it and publish it yourself.

here is a quick little hack to get you started. It can be made to be quite elegant....

Code:
#!/usr/bin/perl
## Program Name: fetch.pl

use LWP::Simple;


print "Content-type: text/html\n\n";


&page_header;


########################################
# Get Content
# accepts:
# url to page
# img replacement string for relative addresses, change "pix/img.gif" to "http://www.site.com/pix/img.gif"
# url repalcement string both of these don't happen if set to O
# string to start grabbing at
# string to end with
########################################


&getcontent("http://site.com/page.html",
"0",
"0",
"<start string>",
"<end string>");
}


&page_footer;

## End of program.

sub getcontent{
local ($url,$base_url_img, $base_url_href, $start_string, $end_string) = @_;

unless (defined ($page = get($url))) {
die "There was an error getting URL: $url\n";
}

@page = split(/\n/,$page);

$start=0;
foreach $line (@page){
if ($base_url_img ne "0"){
$line =~ s/src=\"/src=\"http:\/\/$base_url_img/gi;
}
if ($base_url_href ne "0"){
$line =~ s/href=\"/href=\"http:\/\/$base_url_href/gi;
}
if($line =~ /$start_string/){
$start = 1;
}
if($line =~ /$end_string/){
$start = 0;
}
if ($start == 1){
print "$line\n";
}

}
}

sub page_header{
print qq~

<HTML>
<HEAD>
<TITLE>Stolen Stuff</TITLE>
</HEAD>
<BODY>

~;
}

sub page_footer{
print qq~;

</BODY>
</HTML>
~;
}


[This message has been edited by cdavis (edited June 10, 1999).]
Quote Reply
Re: Headline News from Yahoo? In reply to
He could just be using getnews which grabs headlines from dailynews.yahoo.com, it can be set to how many headlines to grab. Its a nice script. Go to Davecentral.com, and search "getnews"

------------------
Quote Reply
Re: Headline News from Yahoo? In reply to
pres

1) Its an invalid call at your site. The source shows you have it <!--#exec cgi="http://www.tylertx.net/links/cgi/getnews.pl"--> , you should on say the computer path like <!--#exec cgi="/links/cgi/getnews.pl"-->

2) You may have to change getnews.pl to getnews.cgi because your running on a document root and .cgi extensions only run in that unless you set it up right with the server.

3) I checked and your not running an index.shtml page. To be able to execute cgi through pages, the extension has to be .shtml or .html-ssi

Eddie,
Thanks for the link, Topnews is better than getnews because its owner updates it. But for now i guess, both cgis won't work because of the major update in Yahoo's news server. So now I use all the services that Excite gives at http://affiliate.excite.com/, like Stock Quotes, horoscopes, headline news, sports, all on your site.
------------------
------------------------------------------
Lavon Russell
LookHard! Search
http://www.lh.yi.org

[This message has been edited by Bmxer (edited July 20, 1999).]
Quote Reply
Re: Headline News from Yahoo? In reply to
NO, NO...

YahooSuck.com are getting there news from isyndicate.com which will provide it free to you too. I use it!

Quote Reply
Re: Headline News from Yahoo? In reply to
Well, i don't like Isyndicate. pres, you have to call it by ssi like <!--#exec cgi="/cgi-bin/getnews.pl"--> or whatever the path is. Just put that where ever you want the bulleted list to show up.

------------------
------------------------------------------
Lavon Russell
LookHard! Search
http://www.lh.yi.org
Quote Reply
Re: Headline News from Yahoo? In reply to
Yahoo just changed their dailynews appearance and the getnews.cgi still grabbing the old html file. It won't print the headers but it will print some sort of server stuff. Go to http://www.imchat.com/ , they have TopNews which will print 10 headers instead of 4. But both scripts must be recoded in order to grab the right headers. Just have to wait then...
Quote Reply
Re: Headline News from Yahoo? In reply to
lavon,

i tried the reference you entered with the full path on my server and inserted it on the home.html page where i wanted the list to come up, but nothing happens after i rebuilt the pages. you can take a look at it at www.tylertx.net/links/ Its still a work in progress as i am installing different mods etc...

thanks

james
Quote Reply
Re: Headline News from Yahoo? In reply to
being new to perl and installing on top of that. i picked up the getnews.pl script. how do i install this to work with my links 2.o webcrawler template as it has a section on the front page to accomodate recent news?

thanks

james
Quote Reply
Re: Headline News from Yahoo? In reply to
the excite content sounds good but it takes you to their site. how can it be set up to generate within a links directory? i want to somehow keep visitors to my site there.
Quote Reply
Re: Headline News from Yahoo? In reply to
There is no way you can keep visitors on your site, unless you use frames and isyndicate, or unless you have a news server like reuters or UPI. All the things, Topnews, getnews, and excite only take links from their (Yahoo or Excites) and puts them on your site.

------------------
------------------------------------------
Lavon Russell
LookHard! Search
http://www.lh.yi.org
Quote Reply
Re: Headline News from Yahoo? In reply to
i've stolen news from insyndicate Smile i used that HARD script that gotze uses.. not sure what it's called but i think it's called WEBPLUCK

jerry
Quote Reply
Re: Headline News from Yahoo? In reply to
actually.. you can.. Smile

it's just like altavista.cgi cept you have "type"..

if type=news then it will "steal" the news from the page specified in the input "page"...

hehe.. it's not a good idea though.. legally that is..

jerry
Quote Reply
Re: Headline News from Yahoo? In reply to
Hi, Just got back from Colorado and saw this site linked to mine.
I created TopNews by modifying GetNews a little. The original GetNews had some problems. I could have done it with LWP but then you would need to have the LWP module installed.
Anyway, its broke again so I have to fix it. Yahoo keeps changing their html around. I'll try and work aroud them a bit so as to keep them guessing. :-)



------------------
Mark P.
imChat Inc.
Quote Reply
Re: Headline News from Yahoo? In reply to
One problem I came accross when trying to use getnews was a permission denied message coming from Yahoo. I think that people are realizing that harvesting information is a major infringement on their intellectual property rights and they are tightening security on their web sites.

------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us
Quote Reply
Re: Headline News from Yahoo? In reply to
Not sure if its that big a deal as long as you don't frame the content when the link is clicked on. They still get their advertising across from the articles. I've put a base target=new tag into the script just in case.

Hope I don't get sued. :-)

Quote Reply
Re: Headline News from Yahoo? In reply to
Just released a new script that will get a nicely formatted weather table based on your zip code.
Don't know what to call it so I just named it weather. You can see it in action at http://imchat.com/start.shtml
This script does use LWP.
Quote Reply
Re: Headline News from Yahoo? In reply to
Had to take the weather script down for awhile. Sorry, but the site it was pulled from sells weather data. I'm doing one based on the National Weather service though. Should be up soon.



------------------
Mark P.
imChat Inc.
Quote Reply
Re: Headline News from Yahoo? In reply to
Best Weather is now up. It gets data from the National Weather Service's hourly weather reports. These are basically text files and they won't be changing the format anytime soon.
The program also uses http-get instead of LWP to make it easier for people to use.



------------------
Mark P.
imChat Inc.
Quote Reply
Re: Headline News from Yahoo? In reply to
is this legal? is it possible that yahoo or the weather service might sue the perosn who is using it? did you ask for permesion to use it?
Quote Reply
Re: Headline News from Yahoo? In reply to
Here is something bizarre about Yahoosuck. Before today, I'd never heard of it. Yet I found my site listed twice in the same subcategory one link right after the other. My sight's URL is http://www.2ants.com/". The only difference was at the end of the description on the first link was "Add your site today!" This is exactly how I submitted my link to Yahoo. Once with "Add your site today" and once without. As far as I can tell Yahoo is rejecting my site as it still isn't listed. I think it looks too much like there's. I wonder how Yahoosuck could possibly be intercepting my additions to Yahoo.
Quote Reply
Re: Headline News from Yahoo? In reply to
theguy,
If imchat uses the INWS its free because that is the same service that software hamweather uses. This could be better for littler sites with less cgi experience. Unless Imchat is making it big.

------------------
-----------------
Lavon Russell
LookHard! Search
www.lh.yi.org
webmaster@lh.yi.org


Quote Reply
Re: Headline News from Yahoo? In reply to
how about yahoo news, is it legal?
i do not think so
Quote Reply
Re: Headline News from Yahoo? In reply to
Its not legal, because they lease the news hubs from reuters, but yahoo doesn't pursuit anyone. They just change the way they get the new. Huh, sort of like msn messenger and aol.

------------------
-----------------
Lavon Russell
LookHard! Search
www.lh.yi.org
webmaster@lh.yi.org


Quote Reply
Re: Headline News from Yahoo? In reply to
Yahoo knows all about people getting their news links and really don't care because it would be too much of a legal hassle for them to pursue and because you still get their site advertising when you click a link. If anything it helps thier advertising campaign. The changes they make to their html aren't to destroy the application, but are merely the types of changes any webmaster makes.

The National Weather service information is totally free to use. Their are companies that make their living off of it by building applications that report the weather. You can get the weather completely free via the radio by hooking your cpu up to a device that you can get the parts from Radio Shack to build. The NWS actually gives you the specs to build it. Then you will have to go through a company to buy software to interpret the data. All I've done is to get rid of the software middleman thats making his living off of free data!!!

With Best Weather you get a totally free app that will cover all the major cities of the continental US. The Weather Underground charges $50 per month for basically the same thing but you only get 10 cities worth of data. If you want forecasts go with HamWeather. I'm not about to try and do any better than they have done with the forecast pages. They also get thier data from IWIN data files. The perfect compliment would be to use the cookie that BW sets to dynamically write the url to HamWeather forecasts and include it as a link at the bottom of the table.



------------------
Mark P.
imChat Inc.