Gossamer Forum
Home : Products : DBMan SQL : Development, Plugins and Globals :

search engine friendly db content

Quote Reply
search engine friendly db content
Since so much of my site consists of dynamic content, I'm always looking for ways to be better indexed by the major search engine spiders. Inspired by the search engine friendly templates that GT developed for GForum, I decided to see if I could use mod_rewrite to trick the search spiders into indexing all the records in my database. Turns out this was super easy to do. If anyone's interested, following is a rough outline of how you can accomplish this. You'll obviously need to change all the directory paths and file names so they make sense for your database. For the example, though, let's assume you have a table that contains records on widgets, and that the primary key for that table is 'ID'. We also need to have your search results set up so that passing "detail=1" in the url will get a detailed single record, rather than a loop of all records.

For each table that you want indexed, you need four basic rewrite rules:

RewriteRule widgets$ /cgi-bin/db/db.cgi?db=widgets&do=home [L]
RewriteRule widgets/$ /cgi-bin/db/db.cgi?db=widgets&do=home [L]
RewriteRule widgets/all-widgets.html$ /cgi-bin/db/db.cgi?db=widgets&do=search_results&mh=25&ID=0&ID-opt=> [L]
RewriteRule widgets/detailed/(.+) /cgi-bin/db/db.cgi?db=widgets&do=search_results&detail=1&ID-opt==&ID=$1 [L]

You then need one additional RewriteRule which will be used for all tables you want indexed:

RewriteRule db.cgi /cgi-bin/db/db.cgi?%{QUERY_STRING} [NS,L]

Then to get your widget records indexed, you need to change three urls. The url to the main page about widgets needs to be changed to:


And then you need to insert a link to "list all" or "browse records" - something that tells the user you'll be serving up all the records in the table.


Then finally, in your search_results template, when you're looping through multiple returned records, change the links to the individual records to:


Voila! That should do it.

Now, obviously I'm not really selling any widgets, and my site isn't really at mysite.com. My point is that I've had to change the syntax of the above a bit to make it more generic for this example. It's entirely possible that something got screwed up in the translation. So if anyone tries this out and it doesn't work, that could be the reason. It might just be a matter of tweaking the syntax slightly.

Anyway, don't know if anyone'll be interested in using this idea or not, but I figured it couldn't hurt to put it out there.

Fractured Atlas :: Liberate the Artist
Services: Healthcare, Fiscal Sponsorship, Marketing, Education, The Emerging Artists Fund
Quote Reply
Re: [hennagaijin] search engine friendly db content In reply to
As it turns out, these rules don't quite cut the mustard, because there remains the problem of the speedbar on multiple-page search results. I solved that by writing a template global called <%speedbar_static%> that uses several regex replacement instructions to replace everything up to and including "nh=" with a single "/". That way, the speedbar link to page 7 will look like http://www.mysite.com/widgets/all-widgets/7. Then you just need a new rewrite rule that rewrites those static urls from blahblahblah/widgets/all-widgets/(.*) to blahblahblah/db.cgi?...nh=$1.

A couple of implications...

1. The main list all page now needs to reside at http://www.mysite.com/widgets/all-widgets/1
2. Be careful to make sure you do enough regex replacements in speedbar_static to catch not only the dynamic urls to the numbered pages, but also the [<<]-style jump-to links (this one tripped me up at first).
3. It's obviously possible with a few more rewrite rules to set up "list all" results pages for numerous different search parameters on multiple tables. With a little tinkering, I've now got this working on three different tables with 12 different search parameters. =)

Fractured Atlas :: Liberate the Artist
Services: Healthcare, Fiscal Sponsorship, Marketing, Education, The Emerging Artists Fund