Hello,

I'm working on a project where I will need multiple copies of Links SQL to run from one database. In a nutshell, I want to be able to create separate websites from different categories in the database. For example, if I had categories or sub-categories of Cars, Boats, and Motorcycles, is there a way to get an installation of Links to generate pages only for a certain topic and all the topics below that?

Any help would be appreciated.


------------------
--
Travis Burnside
QWK.Net, LLC
http://www.qwk.net

I am sure that the Multiple Category Template Mod located in the LINKS 2.0 section of the Resource Center could be re-written for LINKS SQL.

Hope this helps.

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
* Thinking out of the box (codes) is not only fun, but effective.

Travis,

I think that what you have in mind would require multiple installations of LinkSQL... I guess you'd have to pay for additional licenses.

This is something Alex has not specifically addressed.

It does _not_ require additional installations of Links, if it's done right, but it can be abused to circumvent the licensing since each site _could_ be used individually.

There are two ways I've proposed this to Alex, and I'm not sure which is better.

1) Each "top level" category ie: a category that is not a sub category of another, will generate it's own links tree. The stats on that tree would be saved to a central database. Each tree is built as if it was an entire Links site, with it's own "cool", "new", "ratings" pages. Then, all the site stats are run and a "home" page is built that is one level up from the individual sites. This would allow each sub-tree to have a different look, and maybe serve a different purpose. This could be taken one level further, such that each tree maintained it's own "links" database, so that each tree could have a customized set of links -- classified, shopping, etc.

2) A field is added to the Links table, that is used by the build program, to build separate trees based on the value of that field.

The first way has some advantages, and may eventually be released. The second way also has some advantages, but is not as robust as the first. It's main advantage is running one main site, but being able to break parts of it out -- for example a Kids-only -- area, where only links that are flagged "kids" are built into the tree, but all the links are built into the main tree. They share the _same_ database so the stats are cumulative, only the interface is broken up.

This would have advantages for a Postcards site like ours, where I could set the flag field for all cards that searched out for variations of "July 4th" or "Halloween" and set up a thematic site for the holiday that was completely different from the full site, but was also a part of it -- with one set of links, stats, and such.

Both can be used to run multiple sites, rather than one complex directory, so it's again a question of licensing.

Hello,

Thanks for the input, all.

I'm not really concerned so much about the licensing aspect. If a separate copy of Links is required for each site it runs on, that's fine - Alex has a great product, and he deserves our support.

The practical issues are more what concern me. I'd thought about just writing some code to take a category and all its subcategories and rip them off into a separate database, but that presents problems all its own. Mainly, I'm interested in creating sites that for all intents and purposes are their own and self-contained. But I want these all to operate from the same main database, so that when links are updated or added on the individual sites, they are stored in the main database.

Anyway, I'll keep looking. Thanks!

What you want to do is my Option 2 then.

You need to think about adding a "flag" field to the database, so you can decide what tree they are in, and you could make the flag default to the top-level category if you want.

If you want these to be separate sites, then you probably need to ask Alex about a multiple licensing option.

He might even be able to hack this together for you if you explain what you want.

What I don't understand is _why_ you would want all the links in one database, but have them divided up across many sites?

Are you going to hae one site with all the links, then smaller sites with parts of them?

Come to think about it, your option 2 is probably the most sensible. Unfortunately, I don't have nearly the familiarity with Links required to implement it. If you or anybody here does, I'd love to talk with you about a quick short-term contract assignment.

As to *why* - I'm working on a project where the client wants to be able to create multiple web sites from a central database of links. Rather than having, for example, just a motorcycles section on a large links site - we would like to break that motorcycles section off into a distinct, complete site of its own. That way subject-specific content can be added to the Links output without weighing down a large central site with it.

Normally, that would just be a job for a number of standard, distinct sites and Links installations. But my client wants to keep everything in a central database to facilitate easy categorization and management of Links from a spider.

Really, the *easiest* solution to this, it seems to me, is to have Links modified to be able to accept a category to begin with when building pages, rather than the root level. (I can easily modify the page generation to generate relative links, rather than absolute, to reflect the fact that we're not really starting at the top.) When building the output for different sites, a different set of config files and templates could be swapped in, and voila. Separate site.

Again, if anybody here has the expertise and time to do this (including you, Alex, if you're reading) I'd like to talk with you via e-mail.

Thanks again.


------------------
--
Travis Burnside
QWK.Net, LLC
http://www.qwk.net

Pugdog,

Could you please e-mail me? Your address isn't public and I'd like to discuss a few things off the board.

Thanks.



------------------
--
Travis Burnside
QWK.Net, LLC
http://www.qwk.net

40-50 Hours?

I think I am thinking about what this script should do differently to you guys Isn't it just a case of having the other sites making calls to the database and just printing out the results....

I guess a hack would be quick but doing it right would be hard...

------------------
Michael Bray
Review your webhost or find a new one.
www.webhostarea.com
Links SQL User
------------------

I didn't want to give the impression it's a simple job!

Ball park for this is 30 hours of programmer time, and probably 10-20 hours debugging time, for a total of 40-50 hours of programmer time.

The reason is that there are a few things that have to be worked out, and to do it right is not a trivial job. To try to hack a complex site is penny wise and pound foolish idea.

Before jumping into this I'd spend a good 5-10 hours working on a DETAILED explanation and flow of how you want it to work, and what you want each different layer/section of the site to do, and how you want to manage it. The client needs to CAREFULLY define what they want. Even show mock-ups of what they want (dummy html pages or drawings)

It's a lot easier to program this in from the start, than later on.

Once you do that, getting someone to sit down to code it is a task and a half. When I say 40 hours, that's probably over 3 weeks or more, since there is a lot of "design time" that goes into it that happens during periods away from the computer. Usually the solution hits you when you are on your way out to dinner, to a birthday party, or trying to fall asleep.

It all goes back to the "mythical man month".

I would suggest doing the design now, and finding out what Alex has planned for the next release of Links SQL. It's been a few months without a new version, and that means something is bound to be out soon. He did tell me awhile back that DBSQL.pm was being redesigned, and improved. that's the "key" to this whole thing, since it makes building a site not much more than a bunch of method calls.

If this is something you really want, contact me privately. I'm not cheap, but since it is something I have been thinking about, someone paying the bill would make it happen a lot faster -- meaning, I could justify the time spent making it work.

There are also some details I'd have to work out with Alex, both on licensing and on what he's got planned in the next releases, but I see this as a valuable modification.

It's one I would like to have, but I can't justify it time wise. I'd like to be able to build sub-sites from the main database, by using some sort of selector -- whether it be a keyword or a designated field.

Even to be able to build a site based on the most popular searches, since a pre-generated site with those searches would reduce server load, and make things "look" bigger and faster.

The problem is first -- a hack would be quick -- but it wouldn't be portable, and maintainable.

The second problem is that the way Links is set up to calculate the stats and such, the whole logic would have to be rewritten, and stored in a table so that the various routines could access it. There would have to be a way to recalculate the stats on the fly if the program lost confidence in the stored stats.

Next, because the category tree is the defining structure in Links, and you'd want to have several different category trees, all mapable onto the same tree, you need to a bit more logic control to make sure that all the sub-trees are maintained when the master tree is adjusted. You also would need to have an option to adjust the Master tree if the subtree was adjusted, and that would have to propagate upwards, then downwards to any sub tree that used the categories, since it's possible to have overlapping subtrees.

The links are actually the least of the problem, since they don't care where they are. All they know is the primary category they belong to.

I've wanted to change that so that a link actually only belonged to a "Table" or "Database" rather than a "category". It's category relations were defined exclusively by the Category_Links table.

It's one more level of abstraction, but it would allow (in a larger scheme) for the Category_Links to be the master table, and it could draw links from multiple databases, and integrate them into one site -- or multiple sites.

The advantage of having a link with a primary category, is you could rebuild the "tree" from the links, even if something happened to the Category_Links or Category tables.

In order to do this, I wouldn't want to "hack" it. I'd start at the top, and rewrite the logic of the build process. I'd create extra tables, and develop new relationships, so that an existing database could be imported into the system, and once in it, each link would have a larger set of parameters that could be attached to it, to allow it to be a member of different categories, databases, trees, etc.

Are you following?

The reason I'm thinking like this, is I have a site or set of sites that use 3 different database structures, and I've wanted to pull them into ONE central program, since 80% of the functionality of each site is the same, and the major differences are in the output and database structures. It's much easier to maintain one larger "jump.cgi" than 10 different ones, and one "build.cgi" than 10 different ones, etc. If each of those programs could load the module it needed based on the task it was asked to do, then memory use and efficiency is also improved.

I've done a lot of thinking about this, and really I've been waiting for Alex to release the next generation of code before I dive into it. But I _need_ to get all my sites centrally maintained. (This will work if you are on one server, or set of networked servers, even if you run multiple domains, because all the file systems would be mounted and accessible to Unix directly.).

I'm limiting it now to that -- I'm not going to try to make it work over a distributed network, but I'm sure someone will try <G>.

I don't think this will be hard.... especially if MySQL is used to hold intermediate data and stats that can be passed down through the routines. The big change is that it's a FUNDAMENTAL change to the logic of Links, and I don't want to tackle it until I see that the various modules are really separated out (in the next release) so that I can still use the various modules, but just re-do all the .cgi code.

Hello All,

Let me just jump in and clarify exactly what I'm trying to achieve at this point with this project that Travis is helping me with. Based on the limitations as I understand them, I'd like to replicate the concept used by 4anything.com -- Where the main/homepage is the root/central database of the "tree" and the "all web guides" (see the 4anything.com site) would be considered the "multiple web sites" (notice the 1000+ "all web guides")

I'd like to be able to perform the canonical URL action (like when you click on the "Automotive", link, for example)and have the URL read accordingly, if there's a way to achieve this. The 1,000+, for example, "virtual web sites, or VWS" would be expanded with lots of links over time, and given their own look and feel and unique design. Each VWS would be fed, say 100 - 1500 links each through a spidered database of URL's.

I hope this clarifies what I'm trying to achieve at this point in the project. Make sense? Is the "canonical URL'ing" acheivable?

------------------