Gossamer Forum
Home : Products : Links 2.0 : Customization :

Please tell me Links can do this ..Please

Quote Reply
Please tell me Links can do this ..Please
Please tell me there is a spidering app that works with links or any other easier way to add ressources to the flat database file other then doing it in filemaker pro and then exporting the records as tab separated and then replacing the tabs with | wich takes for ever......
I know hyperseek doe spidering .... please don't tell me i have to switch to hyperseek for something like that.....
I really need a way to add 2000 records before i can open my sute to the public........

Help
SOS

------------------
www.organizedchaos.net
Quote Reply
Re: Please tell me Links can do this ..Please In reply to
Spidering really doesn't lend itself to something like Links. Links is a web directory engine, not unlike yahoo.

I think what you're looking to do is more of a web index, like altavista or something.

While these are both great tools they are not the same at all.

A directory has had a human hand into it, somone validates all the database entrys, and they generally point to something specific.

A index on the other hand, will look at everything it can find, and then let the user plow through the results.

So I guess the short answer is Links can not do this. Nor will it ever. It's not the same philosophy.

Although, from what I understand there is a way to import a bookmark file, which might make it easier on you.

------------------
--------
Syndicate 23
http://www.netexpress.net/~syn23/
Quote Reply
Re: Please tell me Links can do this ..Please In reply to
Can you tell me more about importing bookmarks and what that is all about ?

------------------
www.organizedchaos.net
Quote Reply
Re: Please tell me Links can do this ..Please In reply to
You will probley want to make a search engine, not a directory. Currently i an starting a new project with a spider i call hikia. It will tak in a url someone sends to it, spider it and then later on spider the rest of the site.

The only problem i have found is that the data bases grow huge. I migh stop this project becuase of that.

Unless you have a in-house server or a co-located server, i would not reconend anyone trying to make their own mini altavista.

i figure about 500 sites per megabite.

Hope this helps.

PS. check out my site.
PSS. if anyone knows a free or relitivly cheap provider that can host 25mb or more tell me. I need a new host, links is getting huge.
Quote Reply
Re: Please tell me Links can do this ..Please In reply to
What is your URL ? you never posted it so how am i supose to go t your siye :P

Happy New Year to everybody

------------------
www.organizedchaos.net
Quote Reply
Re: Please tell me Links can do this ..Please In reply to
I don't completely understand what you mean. Spidering is when a script sends out a "crawler" or "spider" that explores web sites, follows links, and saves the sites to an index automatically - like Alta Vista. A spider on the other hand can't categorize things, add true descriptions, or choose whether it should be included in a directory. Because Links is a categorized directory like Yahoo!, it must be run by humans and not spiders like Yahoo!.

Running a spidering search engine on your own server is also quite difficult, as the spider will go on and on and on. If you do want to establish one, try http://www.xav.com/latest/ , it's the best I've seen.

I also don't understand your comment about writing to the database. The easiest way to add something to the Links database is via the admin script. If you have an existing database, then you could either post a sample up here and someone would be able to tell you how to go about converting it. You say that replacing tabs with | takes forever... it shouldn't, just use any good text editor's search-and-replace function!
Quote Reply
Re: Please tell me Links can do this ..Please In reply to
This bookmarks thing is bugging me as well. About a month ago the Webmaster from Youho posted a messgae saying he had figured the whole thing out. A slew of people asked him to post the code but I don't think he did. Of course I cant find the thread in the archive....

Seriously, can we put our heads together and try to figure this one out? It seems it would save our collective a**es as well as our sanity ;-)

Geoffrey
Quote Reply
Re: Please tell me Links can do this ..Please In reply to
John Cokos (Hyperseek author) here ...

A bit of clarification on the "spider" issue:

Agreed, spidering, in the traditional sense is not recommended for engines like Links or Hyperseek, as we are directories, not search engines. The spider that we ship with Hyperseek is not a traditional spider in that it does not follow every link on a site to fill up your directory. It is designed to grab one page (the exact URL you enter) and put it in your temporary DB.

It's basically a combination of the XAV (mentioned previously) meta tag parser, LWP, and some custom code to fit the Hyperseek database format. There's nothing proprietary about it, so if people are interested, I can let you have the source code to it, so you can hack it to fit into your Links systems.

Just let me know if you want it, and what's the best (accepted) way of putting it up.

John
Quote Reply
Re: Please tell me Links can do this ..Please In reply to
Hi John,

Let me take advantage of your presence here...

I am a Gossamer Links user and have a friend who uses Hyperseek.(http://www.mararts.org/cgi/hyperseek/directory.cgi) .We both run a Martial Arts oriented directory.

At the bottom of my search results page i have the option:
"Try your Search for "query" on:

Altavista - Yahoo - etc..

and want to give my visitors the option to search his database also.

Is this possible? I mean for yahoo i use:
http://search.yahoo.com/bin/search?p=query and my visitors go direct to Yahoo results page for the given query.

What code would i use for Hyperseek?

Thank you

Eduardo Volpi
Quote Reply
Re: Please tell me Links can do this ..Please In reply to
John i would really appreciate that spidering like app which comes wih Hyperseek.... coulf you please email it or upload it to a ftp for me ?


------------------
www.organizedchaos.net
Quote Reply
Re: Please tell me Links can do this ..Please In reply to
I forwarded the code to Alex for his approval. I think he'll make the necessary changes for it to work with the Links Database format and then post it.

If you need it NOW, then email me directly and I'll attach a copy for you.

John (jcokos@ccs.net)
Quote Reply
Re: Please tell me Links can do this ..Please In reply to
Hi Jcokos


John i would really appreciate that spidering like app which comes wih Hyperseek.... coulf you please email it for me ?

my e-mail: weblinker@manaus.br

Tanks!


------------------
[]'s

Lucas Saud - #19815087