Hello all !

I am about to buy Links SQL, but I have some questions for you.

1. How Links SQL work in real life (on PIII 500, 256 MB RAM). How long does it take to search the database with 20 000 links ?

2. Is there any better search engine or links indexing engine. Perhaps search engine similar to altavista technology could be bough (spider).

3. When will you publish new version of Links SQL ? What will be added to the new version ?

4. Is there anything special that I should know before buying Links SQL ?

I am running links 2.0 on my server. I am satisfied with the speed and the features, but now is time to get serious. In the july we shall start working on new site and searchengine will be important pice of new page.

A friend of mine reccomended a Hyperseek searchengine (http://www.hyperseek.com/).

What do you say about Hypperseek verses Links SQL ? I personaly think that Links programmed by Gossamer-Threads are great product. Hyperseek (750$) costs more, but it seems to have more features (spidering...).

I thank you for your answers in advance.

Andrej Lesjak

Hi,

You might want to take a look at the FAQ. The threads on Links peformace have been indexed there.

I would wait until the next release of links before making a decision. Several of us have been beta-testing the spider, and some other features are going to be added.

I've looked at most programs out there, and I have no regrets going with Links.

People have even managed to integrate links with PHP, so you are not limited to perl/cgi at all even.

I don't like the idea of embedding the program code in the HTML files, so I probably will never use PHP for more than an SSI type situation. I believe in separation of church and state, and in program and templates/output.



------------------
POSTCARDS.COM -- Everything Postcards on the Internet www.postcards.com
LinkSQL FAQ: www.postcards.com/FAQ/LinkSQL/

quote: I would wait until the next release of links before making a decision. Several of us have been beta-testing the spider, and some other features are going to be added.


Can you say more about these features ?
Will new version offer the ability to force search result listing, I mean advertising by a keyword.
How about banner rotating system ?
Will I be able to insert a logo next to the link ?
Will new version offer automatic adding to the database ? And after that checking the link accuracy...
Will you work on keywords reporting system (time, results,...) and monitoring the searchengine (how long it takes to search in a database)... ie. advanced reporting.
How about integration with Open source directory or to enable to search in other searchengines ?


Long way to go... but must say that you have done great job so far !!!

I'm not involved in Links development, except to the point where I've sent Alex some really long messages on features and possible implementation. The spider was announced here a couple of months ago, and Alex has not really said what is going to be in the new release, other than odd sentences about what was changed or changing.


>> Can you say more about these features ?

I don't know anything more about them.

>> Will new version offer the ability to
>> force search result listing, I mean
>> advertising by a keyword.
>> How about banner rotating system ?

Alex has said this is going to be in the next release. He said that he was working on it.

>> Will I be able to insert a logo next to the link ?

You can do that now. Check my site:
http://www.postcards.com
and
http://www.postcards.com/portal

>> Will new version offer automatic adding
>> to the database ? And after that checking
>> the link accuracy...

I'm not sure what you mean by that. If you mean not needing to validate the links, you can do that now, by inserting into the Links database rather than the Validate database. You can run a cron job to verify the links, and do it on a staggered basis.

>> Will you work on keywords reporting
>> system (time, results,...) and monitoring
>> the searchengine (how long it takes to
>> search in a database)... ie. advanced
>> reporting.

I'm not sure what you mean. I posted a mod to show the most recent terms. There is a feature in the admin to show the last searched, number of times searched, and how many results returned. Average search times are not logged at this time, but the hooks are in place for it, you'd just have to turn the debugging code back on selectively.

>> How about integration with Open source
>> directory or to enable to search in other
>> searchengines ?

I'm not sure what you mean by this either. You can parse the ODP into Links, and by handling the query term, you can create URL's to allow people to search in other categories. I used to have an Amazon and other search engine option box (based on a template someone posted for Links 2.0) but I removed it when I got rid of all our affiliate programs.

It's not hard to do at all, this is just added to the templates, it's not code at all.

------------------
POSTCARDS.COM -- Everything Postcards on the Internet www.postcards.com
LinkSQL FAQ: www.postcards.com/FAQ/LinkSQL/

Hello all and expecially Pugdog !

Thank you Pugdog for your answers !

<<<QUOTE: >>> Will new version offer automatic adding to the database ? And after that checking the link accuracy...

I'm not sure what you mean by that. If you mean not needing to validate the links, you can do that now, by inserting into the Links database rather than the Validate database. You can run a cron job to verify the links, and do it on a staggered basis. <<<END OF QUOTE>>>

I mean with that that when a user comes to my site and adds a link the link will be automatically inserted to a database, in few seconds enabled to wieved by other people. And after that an admin would be able to check the link accuracy.

<<<QUOTE: >>> >> How about integration with Open source directory or to enable to search in other searchengines ?

I'm not sure what you mean by this either. You can parse the ODP into Links, and by handling the query term, you can create URL's to allow people to search in other categories. I used to have an Amazon and other search engine option box (based on a template someone posted for Links 2.0) but I removed it when I got rid of all our affiliate programs. <<<END OF QUOTE>>>

I mean with that if I am able to import DMOZ data to my searc database (dmoz.org) ? They offer free download of data that list aproximatelly 1,7 M links. Does anyone have experiance with that ?

The folowing example is not advertisement for my site, it is just an example of what you can do with Hyperseek. I wonder if Links SQL offer same features ?

<<< EXAMPLE >>> Go to website http://www.in.be and see on the right side a gray box with two search options. Take for an example the bow "Bedrijven" (dutch for companies) and type the word "hard". There you see the search working on 550.000+ records and the first option show "Hard Work Automation" and shows some customization options. <<<END OF EXAMPLE>>>

Can this be done with Links SQL ?

With other questions I shall wait until the new version is released. I hope it will bring some pretty striking inovations.

The automatic add may or may not be a feature of the new links. As I said, you could do it by making the link be added to the Links database, after the URL is verified, then you could insert the ID into a list or send the admin an email that the new site was added.

As for ODP, I parsed in 35,000 sites. I posted to another thread. The ODP file is 107 Meg at this point, and expands to almost 600 meg. IT taks HOURS to parse in, and it's in English. You'd need to do a germaan/english translation some how to make it work -- or you'll have all english titles.

There are something like 30,000 categories on ODP, so changing them all manually would be impractical.

As for the search, we'll have to wait and see. Alex was working on the search even as the 1.11 version was released. He knows what the competition is doing, and I'm sure he's keeping up with it. He also knows what people have asked for... so we'll have to see

Actually, ODP has 1.7 million listings, and over 250,000 categories (ouch)

http://www.dizzyduck.com is hyperseek, running the whole lot.

You need a NASTY server for that type of database (like 2GB of RAM, and 18GB of hard drive space)

IMO, there already IS a DMOZ, so I'm not in favor of importing it, and putting the crushing load on your server....why not use a remote searcher like anaconda to augment your own, hand picked Links database?

I'm only responding here, because there's an ODP thread in the other links forum, and I'm getting personally, more and more people that want to import it. Just wanted to state an opinion here, that I personally don't see the real need to use someone else's data....


John Cokos

I agree. Most of the links are worthless, and their organization is really, really bad over all.

But, if you are running a targeted site, using a subset of the data to get going is a pretty cool idea. That way you don't have to wait until you have enough traffic to get some links. Once you have a start, the rest come.

There are already too many search engines all trying to do everything. DMOZ was a good idea, but (I am/was an editor there) they left the origins of being human EDITED to just being human SPIDERED. A waste of human time.

This leaves a lot of room for targeted directories that provide really good link information on a subset of data. Ratings, reviews, detailed descriptions, etc.

I'm starting to really understand perl and Links a lot better, and when the next version comes out will probably have a bunch of utilites availabe, including a program that will read through the content.rdf file and break it up by tree. This will speed importing of the targeted subsets. I'm doing that now with a random "split" on the file, but it cuts some categories in half (fortunately none I've needed <G> ).

I just don't see any reason to duplicate ODP. Or any of the other large indexes. The future is in CONTENT (as it's always been) not in spitting out the same regurgitated links listings.

This is what's so great about Links. Most poeple have added to it so it's more than just a links list. My main site uses it to serve up postcards. The flexible database structure allows you to do almost anything with it.

Here, Here !!

Targeted, focused, content-rich directories are the way....

I could not agree more with your reply jcokos.

I think that it is hightime to tell about features in new version, Alex ! People like me are curiouous. When will you publish new version. ?

About new features, Alex wrote something on the following possible project. This would be great to save traffic bandwidth (by browser and proxy caches, instead of all the pages new every day). I don't know if there is or there is not an easy way to implement it:

Build Changed
www.gossamer-threads.com/scripts/forum/resources/Forum9/HTML/000369.html


[This message has been edited by Juan G (edited May 13, 2000).]