I'm afraid I'm probably being naughty but I decided to post this here as it isn't to do with Links2 Customization....it isn't really to do with Links2 at all infact, however it _could_ be slightly Links2 related due to the recent threads about Google/Dmoz etc.

Anyway, I've just finished the first beta of my Google search script.

It will allow you to grab results from Google and format them into your site. It isn't currently integrated with Links2 but as it already uses templates then it should be fairly easy to port over to Links2 - I'm happy for anyone to do that and post the changes here, or I'll do it myself if I get round to it.

It was tricky getting the parsing right and occassionally on the results pages some unexpected links appear like stock quotes which don't parse properly and so become dead links, but this is very rare.

You can see a demo here:

http://www.wiredon.net/index/fetch.cgi

If anyone is interested in this then I'll post a link so you can download it.

There are still changes to be made and hopefully I can neaten up the code but it seems to work well as it is, but I'll update it every so often.

Sorry for posting this somewhat offtopic post but I just thought that the code could be ported to Links2 eventually.

EDIT: I've added some more search options since I mentioned the script in Bmxers thread the other day

Very Nice.. I did notice a search for web yielded a result: i think for adobe : That had a similiar pages and stock quotes option that were dead. Have you tried to parse those? B/c i had to do wierd parsing for my gofetch script to change some urls to my link and some to retain dmoz.org. Could i help? Or better yet, i'll just wait for the dl and post a change if i make it. Once again, good job
Lavon Russell
LookHard Mods
lavon@lh.links247.net

Yeah it's annoying....if something isn't quite as the script expected in the html then it just ignores it and the link becomes dead.

I may just try to fix that before I make it available.

...or I may just encrypt the code so no-one can see how messy it is lol

I know its in beta but just wanted to say also with my search, everytime i go to the next page the query gets a + added on to it. Even if I skip from 10 to 35 it adds one.
Edit...
Sorry to be posting this many bugs but i know this is a great script and want the least amount of bugs to have to fix after release.
Try this search
http://www.wiredon.net/index/fetch.cgi?do=search&q=*&as_q=&as_epq=&as_oq=&as_eq=
&as_qdr=all&as_occt=any&num=10&lr=&as_dt=i&as_sitesearch=
I don't know if you are parsing the error page or not, but I searched for an asterisk and the broken images and dead links came up.

Lavon Russell
LookHard Mods
lavon@lh.links247.net

All the fore-mentioned bugs including stock quotes, the dodgy error page and the + sign should now be fixed.

Ok. I still get the plus sign though, and i'm sure you parsed out Similiar Pages from the link results, but on this pare:
http://www.wiredon.net/...p;start=110&sa=N I still get it on like two links. Its probably not even worth worrying about.

Lavon Russell
LookHard Mods
lavon@lh.links247.net

Ah I see the problem. I'll fix that in a sec.

I can't see this + sign thing - can you tell me what you are doing/seeing?

Ok, using the latest search i posted above. Go down to where you choose what result page you want. Hover the mouse over any numbers or the next or previous and you'll see there should be 4 + signs after the query. There is already 3 because you'll see when i posted that url, i had them in there. But after you go to the next page, you'll see 5 + signs
Lavon Russell
LookHard Mods
lavon@lh.links247.net

That is so weird.

I've fixed the "Similar Page" thing, but I've looked through all my code and there isn't a + being used/added anywhere....I don't get it.

It won't even strip out with:

$str =~ s,\++,,g;

Sorry to interrupt.

Seems to be a very nice mod, Paul. However, have you checked Google's ToS?

http://www.google.com/terms_of_service.html

Ah but if you get permission first....

Results do not appear in Netscape 4.73.
========================================
Buh Bye!

Cheers,
Me

Have we not had this discussion once already?

Eliot, try parsing results from Google whilst keeping the html perfect, then come back and tell me how you did it

I already had to lock Bmxers thread I don't want to have to lock this one too.

I would prefer you to leave this thread if that is ok.

Paul,

I was simply stating that the search results do not appear and your original reply was rude and "bitchy"...

Anyway, back to the original point of this thread...And again, before I told you the problem was with the search form, which works now, but the search results do not.

I would recommend embedding your parsed results within a nested table or use <xmp></xmp> anchors around the parsed results.

Hope this helps.
========================================
Buh Bye!

Cheers,
Me

I don't know
http://www.wiredon.net/...p;start=110&sa=N
I still get similiar pages on some and i think on the first result on this page, you'll see it says [ More Results from %URL% ]
Also i thought the + thing could have been my fault. Maybe putting a space after my search term. Do you think maybe when you parse the query in the script, there could be a space or something? I looked at the source of the page and it had this where the query field was

<input type="text" name="q" value="usenet: ">

So it's definitely not a plus sign that is the problem.
I would doubt this was the case unless you were using CGI but you are using templates.
Lavon Russell
LookHard Mods
lavon@lh.links247.net

Do you think it is worth carrying on if people need permission to use it?

I don't know. Thats the reason why I stopped work on this when I was doing it. I don't know if it would be a simple yes or no by google. They'd probably want compensation. I know google said do not copy or mirror or whatever their stuff, but an alternative is the uh yahoo google search. If yahoo says nothing about google, and google doesn't specify anything about the yahoo version of google, I would think it wouldn't need permission. But I don't know. It really isn't worth it if you don't necessarily plan on using it yourself. If the overrall plan was to make it to distribute it, google would say no.
Lavon Russell
LookHard Mods
lavon@lh.links247.net

Yeah, great, if you have a few millions to spare! Do you know how much the big search engines pay for access to Inkotomi's database? Nearly all the big search engines out there use the same database and Inkotmoi are making a mint.

You would have to componsate Google a few hundred thousand dollars (IMO) to be able to do this.

- wil

>>Nearly all the big search engines out there use the same database <<

Most of the directories use DMOZ like AOL's directory. Most of the search engines use Inktomi.

Yep, and they pay a good dollar for that service.

Google will absolutely no way let you use their search results without paying them a large sum of money. No way.

- wil

I'm not going to anyway....I gave up on this idea about 10 days ago.