Hello I would like to know where I can get a good script for harvesting news from other sites. I checked the search forum but no such luck on finding a good script for harvesting news from other sites. If anyone knows of any please let me know. Thanks

yeah, i was thinking about doing that with Yahoo, and I don't just want to have the headlines for stocks and no search, so i'll look at different sites searches and pick which one i want to do. Now i have to do something but later. What is the address you put the stockav file at?

[This message has been edited by Bmxer (edited December 27, 1999).]

Check out
http://www.lifelinenews.net
I have them on my site below. Its only the news thing, i did the stock thing myself.

------------------
LookHard Search
http://lookhard.hypermart.net
Lavon Russell

The stock thing on your site, can you get the information from any site? And if so can I get the script for that?

The Stock headline thing i use is www.imchat.com Imchats TopNews Script.
I've only modified so i can get stocks from Yahoo, Altavista, CnetInvestor, Hotbot, and I'm working on Excite. Its not hard to modify, you just change the url and change the html to parse so if you have a different sites stuff you wanna get, thats what you'd do. Here is the one i have on my site with like 8 stocks or something
http://lookhard.hypermart.net/stockav.txt
Change .txt to cgi
Oh cool, i just got this thing to work, called the wealth meter, check it out
http://lookhard.hypermart.net/...ocks/wealthmeter.cgi
It takes the wealth meter from here
http://investor.cnet.com/
and lets me call it with ssi

------------------
LookHard Search
http://lookhard.hypermart.net
Lavon Russell

Thanks Bmxer the script works great. Do you think there is a way you can set a script that can be in a form that when you enter a stock symbol it will get the results from another site and but them on your pages?

i get daily stock information for nasdaq, nyse, and american thrown into my oracle db.. if you guys have DBI/DBD you can remotely access it.. just a simple query..

SELECT Last FROM Nasdaq WHERE ID = 'CMGI'

jerry

Bmxer the address is http://www.liquideffect.com/cgi-bin/stockav.cgi and I found a good site with stock quote results and information. The site is http://www.stockmaster.com if you can get that script to work with that site let me know. Thanks

Cool, i was a little too late reading this, I already modified that Altavista.cgi script to get the results. It works but since they have their urls as / instead of their full url, i'm trying to change it to their address.

From stockmaster
do you want the quotes on the frontpage, or just the search results?

[This message has been edited by Bmxer (edited December 27, 1999).]

See if you can get the results by using a form and then printing the results out on your own page. Thanks

Ok, i won't get to it for a while, at least an hour. But i finished the Cnet one, its very low on design but that doesn't matter now. Its at lookhard.hypermart.net/look-bin/Look/stocks/stockfind.cgi

------------------
LookHard Search
lookhard.hypermart.net
Lavon Russell

[This message has been edited by Bmxer (edited December 29, 1999).]

It works great. Can you tell me how you know where to start pulling the code from and where to stop? I would like to try and write something like that but I don't know much about LWP. I am just catching on to Perl and CGI so any help would be nice. Thanks

um, you just look at the html code, if something is bolded like

Copy this text
If it looks like that on the page
then you would put

Copy this <b>text</b>

All it's doing is getting the page with a variable like
$results
Then it strips down stuff in the variable $results with this s/HTML here//;
// means it's gonna change that code into nothing.
with something like this
$results =~ s/.*HTML here//; That means anything from HTML here and before will be deleted. And this gets rid of anything after it

$results =~ s/HTML here+.*//;
and this gets rid of stuff in between things
like say you had HTML is here
you can just go
$results =~ s/HTML(.+?)here//;
that gets rid of HTML, is, and here.

Just examine the first script i put up. You learn alot about perl by just looking at scripts. When i started with Links in July, I knew no perl or LWP, but about a month later i made the LWP URL Checker mod.


------------------
LookHard Search
http://lookhard.hypermart.net
Lavon Russell

I got my stock search script the way i want it. I'm deciding whether i wanna use templates to customize it or just edit in the cgi.

------------------
LookHard Search
http://lookhard.hypermart.net
Lavon Russell

Getting back to your stockav.cgi script where it says
$news =~ s/.*Top Tech//s;
$news =~ s/Quotes+.*//s;
$news =~ s/<!-- begin dispatch -->+.*//s;
$news =~ s/<br>//sg;
$news =~ s/<\/font><\/b><\/td>(.+?)<\/table>//sg;
so if I wanted to start pulling something from another page I would start with Top Tech and end with <\/font><\/b><\/td> ?

Yeah, or any convenient email that comes after the section of html you want.

------------------
LookHard Search
http://lookhard.hypermart.net
Lavon Russell

Thanks Bmxer I just tried a small script to pull the Today's Markets from Quicken.com and it works great. I didn't know it was that easy!

Yeah, i always look at other scripts when i need to get things done before i try and make a script myself. And those (TopNews, and Altavista.cgi) are two good scripts.

------------------
LookHard Search
http://lookhard.hypermart.net
Lavon Russell