Gossamer Forum
Home : Products : Links 2.0 : Customization :

Simple RSS Feed

Quote Reply
Simple RSS Feed
Hi,

i like to offer an rss feed for my users. is there an easy way to build an rss file with the new links everytime together with the rebuild of the links? The links on the whats new page would be enaugh.

i try the several suggestion in the forum, but no luck.

thanks in advance,
RSS Junkie
Quote Reply
Re: [RSS Junkie] Simple RSS Feed In reply to
Did you look at this? (Attached)


Leonard
aka PerlFlunkie
Quote Reply
Re: [PerlFlunkie] Simple RSS Feed In reply to
Hi Leonard,

yes, i try´d this mod including the enhancements from smatthew in this thread:

http://www.gossamer-threads.com/...orum.cgi?post=282152

it works fine with a few modifications.

But i dont want to produce the feed on the fly, because of the server load, if someone does a DOS attack against the modified search.cgi.

I can modify scripts, but i am not into perl. Do you know how i can change the modified search.cgi, so that it produces a file instead of sending the result to the browser?

Then i could hide the file and start it by cron as example and give the visitores the url the file instead.

cu,

RSS Junkie

Last edited by:

RSS Junkie: Oct 30, 2006, 2:08 PM
Quote Reply
Re: [RSS Junkie] Simple RSS Feed In reply to
Possibly as easy as...

Code:

# Print out the HTML results.
# &site_html_search_results;
open (RSS, ">/rss.txt") or &cgierr("Error in rss_search. Unable to open text file /rss.txt\nReason: $!");
print RSS $link_results;
close RSS;

}

sub search {


This should create the text file (rss.txt) for you, in your root directory. Ten put a call in your template for that file using something like <%RSS%>, and define the tag in site_html_templates.pl, like rss => /rss.txt.

Maybe... My brain has not been up to par lately, so I may be misleading you, but that is the right approach,, anyway. Unsure


Leonard
aka PerlFlunkie
Quote Reply
Re: [PerlFlunkie] Simple RSS Feed In reply to
Hi Leonard,

first, thank you very much for your help.

It works half the way ;-)

After giving it the complete path to the file, the file is produced, but i get an "premature end of..." error.

So the script produces the file, writes the info into it using the template " "rsslink.html" and then end with "premature end of script headers"

Another problem: In the file are only the links defined by "rsslink.html ", but not the header defined by "search_rssresults.html".

cu

RSS Junkie

Last edited by:

RSS Junkie: Oct 30, 2006, 10:14 PM
Quote Reply
Re: [RSS Junkie] Simple RSS Feed In reply to
Hi,

anyone with an idea how i can get it working?

cu,
RSS Junkie
Quote Reply
Re: [RSS Junkie] Simple RSS Feed In reply to
Where did you put the <%RSS%>?


Leonard
aka PerlFlunkie
Quote Reply
Re: [PerlFlunkie] Simple RSS Feed In reply to
Hi Leonard,

i put <%RSS%> at the top of the rsslink.html template. And in site_html_templates.pl, i use RSS => "/rss.txt",

Without the "" i get an syntax error.

On top of every instance of the <item> block i have in the resulting file the string /rss.txt instead of this:

<?xml version="1.0" encoding="iso-8859-1"?>
<rss version="2.0">
<channel>
<title>Company Name</title>
<link>http://www.mysite.com</link>
<description>Feed Description</description>
<language>en</language>
<lastBuildDate><%rssdate%> <%time%> PST</lastBuildDate>
<copyright>copyright holder</copyright>
<managingEditor>my@email.com</managingEditor>
<webMaster>webmaster@email.com</webMaster>
<%if link_results%>
<%link_results%>
<%endif%>
</channel>
</rss>

also the above should only appear only once.

The resulting file looks like this.

/rss.txt
<item>
<title>Title</title>
<pubDate>10 Nov 2006 12:54:44 GMT</pubDate>
<description>Description</description>
<link>http://www.domain.com/Detailed/42257.shtml</link>
</item>

/rss.txt
<item>
<title>Title</title>
<pubDate>10 Nov 2006 12:54:44 GMT</pubDate>
<description>Description</description>
<link>http://www.domain.com/Detailed/42257.shtml</link>
</item>

instead of

<?xml version="1.0" encoding="iso-8859-1"?>
<rss version="2.0">
<channel>
<title>Company Name</title>
<link>http://www.mysite.com</link>
<description>Feed Description</description>
<language>en</language>
<lastBuildDate><%rssdate%> <%time%> PST</lastBuildDate>
<copyright>copyright holder</copyright>
<managingEditor>my@email.com</managingEditor>
<webMaster>webmaster@email.com</webMaster>
<%if link_results%>
<%link_results%>
<%endif%>
</channel>
</rss>

<item>
<title>Title</title>
<pubDate>10 Nov 2006 12:54:44 GMT</pubDate>
<description>Description</description>
<link>http://www.domain.com/Detailed/42257.shtml</link>
</item>

<item>
<title>Title</title>
<pubDate>10 Nov 2006 12:54:44 GMT</pubDate>
<description>Description</description>
<link>http://www.domain.com/Detailed/42257.shtml</link>
</item>

it whould be great if you have an idea what to do.

cu

RSS Junkie
Quote Reply
Re: [RSS Junkie] Simple RSS Feed In reply to
OK, you won't need RSS => "/rss.txt" or <%RSS%>, so remove them.


Near the bottom of links.cfg, add this:

$build_css_url = $build_root_url . "/links.css"; # URL to your CSS file.
$build_rss_url = $build_root_url . "/rss" . $build_extension; # URL to your RSS page.


Change the search.cgi (or is it rsssearch.cgi) to this:

Code:

# Print out the HTML results.
# &site_html_search_results;
open (RSS, ">$build_rss_url") or cgierr ("unable to open RSS page: $build_rss_url. Reason: $!");
print RSS &site_html_search_results;
close RSS;

}

sub search {



If you used the mods in the thread you linked, you might need to use &site_html_search_rssresults instead of &site_html_search_results.

You should now have a page in your root directory called rss.html (or whatever extension you use), that will have the results of search.cgi as a static page. Is that what you were after?


Leonard
aka PerlFlunkie

Last edited by:

PerlFlunkie: Nov 10, 2006, 10:42 AM
Quote Reply
Re: [PerlFlunkie] Simple RSS Feed In reply to
Hi Leonard,

if i use $build_rss_url = $build_root_url . "/rss" . $build_extension; # URL to your RSS page.

i get an error message that the file is missing (BUT with the right path in it, because it works if i make an test call with the browser), but nothing is changed in the file. (i put an empty dummy file there with 777)

if i change it to $build_rss_url = $build_root_path . "/rss" . $build_extension; # URL to your RSS page. than a lonely "1" is writen at the top of the file nothing else, but the whole rss file (complet with anything in it) is writen to the browser also.

This are the subs i use in site_html_templates.pl

sub site_html_rsslink {
# --------------------------------------------------------
# This routine is used to display what a link should look
# like in the rss page.
my %rec = @_;
# Set new and pop to either 1 or 0 for templates.
($rec{'isNew'} eq 'Yes') ? ($rec{'isNew'} = 1) : (delete $rec{'isNew'});
($rec{'isPopular'} eq 'Yes') ? ($rec{'isPopular'} = 1) : (delete $rec{'isPopular'});
if ($rec{'Title'}) { ($rec{'Title'}) =~ s/&/&#38;/g; }
if ($rec{'Title'}) { ($rec{'Title'}) =~ s/</&#60;/g; }
if ($rec{'Title'}) { ($rec{'Title'}) =~ s/>/&#62;/g; }
if ($rec{'Title'}) { ($rec{'Title'}) =~ s/’/'/g; }
if ($rec{'Date'}) { ($rec{'Date'}) =~ s/-/ /g; }
return &load_template ('rsslink.html', {
detailed_url => "$build_detail_url/$rec{'ID'}$build_extension",
%rec,
%globals
});
}

sub site_html_search_rssresults {
# --------------------------------------------------------
# This routine displays the search results.
#
my $term = &urlencode ($in{'query'});
# &html_print_headers;
print "Content-type: application/xml\n\n";
print &load_template ('search_rssresults.html', {
term => $term,
link_results => $link_results,
category_results => $category_results,
next => $next,
cat_hits => $cat_hits,
link_hits => $link_hits,
%in,
%globals
});
}

This is what i have in the modifys search.cgi

# &site_html_search_results;
open (RSS, ">$build_rss_url") or cgierr ("unable to open RSS page: $build_rss_url. Reason: $!");
print RSS &site_html_search_rssresults;
close RSS;

So the file rss.xml is writen now, but only with one single nr "1" at the top and there is a output also the the browser. The output to the browser is complete, as it should be.

I have no idea where the "1" comes from and why it is send also to the browser (complete with anything in it as it should be in the file) also as it should be in the file.

This is strange :-)

cu RSS Junkie
Quote Reply
Re: [RSS Junkie] Simple RSS Feed In reply to
Did you try putting the url in directly? Instead of:

open (RSS, ">$build_rss_url")

use:

open (RSS, ">/rss.html")


Leonard
aka PerlFlunkie
Quote Reply
Re: [PerlFlunkie] Simple RSS Feed In reply to
Hi Leonard,

if i use open (RSS, ">/rss/rss.xml") it get the same error message as if i use $build_root_url

Error Message : unable to open RSS page: /srv/www/htdocs/web5/html/rss/rss.xml. Reason: No such file or directory

The file is there!

And if i use the serverpath ($build_root_path) the "1" is written into it.

If i use open (RSS, ">/srv/www/htdocs/web5/html/rss/rss.xml") istead, i get the (correct) browser output but only the single "1" in rss.xml

cu RSS Junkie

Last edited by:

RSS Junkie: Nov 11, 2006, 12:11 AM
Quote Reply
Re: [RSS Junkie] Simple RSS Feed In reply to
Hi Leonard,

have you any idea how i can manage this. I try everything, but i am totally lost. Especially the perfect output to the browser as i describe above confuses me, because with the same setting i get only one lonely number 1 in the rss file.

Very strange.

cu RSS Junkie
Quote Reply
Re: [RSS Junkie] Simple RSS Feed In reply to
Hi,

i still have the problem. Is there someone with an idea?

cu RSS Junkie
Quote Reply
Re: [RSS Junkie] Simple RSS Feed In reply to
Hello,

i still have this strange problem

if i use this

Code:

open (RSS, ">/srv/www/htdocs/web5/html/rss/rss.xml") or cgierr ("unable to open RSS page: $build_rss_url. Reason: $!");
print RSS &site_html_search_rssresults;
close RSS;


the rrs data is still writen (perfectly!) to the browser. This is strange, because the only call is &site_html_search_rssresults in the routine above, which should be written to a file and not to the browser.

In the file rss.xml is writen only a single "1", in the first line and nothing else.

Any idea how i get the script to write the rss data into rss.xml and not to the browser?

Again, the browser output is perfect with no errors.

Thanks in advance,

RSS Junkie
Quote Reply
Re: [RSS Junkie] Simple RSS Feed In reply to
Possibly...

print RSS "&site_html_search_rssresults";

Unsure


Leonard
aka PerlFlunkie
Quote Reply
Re: [PerlFlunkie] Simple RSS Feed In reply to
Hi Leonard,

with print RSS "&site_html_search_rssresults"; i get an premature end of script headers.

This is my rss.cgi which prints to the browser the perfect output and to the file rss.xml just an single no "1"



Code:
#!/usr/bin/perl
#
# =====================================================================
#
# Form Input:
# 'type' : can be either 'keyword' or 'phrase'.
# 'bool' : can be either 'and' or 'or'.
# 'mh' : the maximum number of hits, can be either 10, 25, 50, 100.
# 'nh' : the page hit we are on.
#
# Setup:
# Make sure the require statement below points to the config file. # Required Librariers
# --------------------------------------------------------
eval {
($0 =~ m,(.*)/[^/]+,) && unshift (@INC, "$1"); # Get the script location: UNIX /
($0 =~ m,(.*)\\[^\\]+,) && unshift (@INC, "$1"); # Get the script location: Windows \ require "admin/links.cfg"; # Change this to full path to links.cfg if you have problems.
require "$db_lib_path/db_utils.pl";
require "$db_lib_path/links.def";
$build_use_templates ?
require "$db_lib_path/site_html_templates.pl" :
require "$db_lib_path/site_html.pl";
};
if ($@) {
print "Content-type: text/plain\n\n";
print "Error including libraries: $@\n[/url]";
print "Make sure they exist, permissions are set properly, and paths are set correctly.";
exit;
}# ========================================================eval { &main; }; # Trap any fatal errors so the program hopefully
if ($@) { &cgierr("fatal error: $@"); } # never produces that nasty 500 server error page.
exit; # There are only two exit calls in the script, here and in in &cgierr. sub main {# --------------------------------------------------------
%in = &parse_form(); # Display the form if called with no input.
(keys %in <= 0) and &site_html_search_form() and return;# Set maximum hits -- default to 25.
local $maxhits = 25;
if ($in{'mh'} && (($in{'mh'} == 10) || ($in{'mh'} == 25) || ($in{'mh'} == 50) || ($in{'mh'} = 100))) {
$maxhits = $in{'mh'};
}# Set search type -- either phrase or keyword. Also build keyword list to search on.
my @search_terms = ();
($in{'type'} eq 'phrase') ?
(@search_terms = ($in{'query'})) :
(@search_terms = split (/\s/, $in{'query'}));# Set boolean connector and next hits page.
my $bool = $in{'bool'} || 'and';
my $nh = $in{'nh'} || 1;# Store the search results here.
local (%link_results, @category_results);# Do the actual search.
my $status = &search (\@search_terms, $bool);
if ($status ne "ok") { &site_html_search_failure ($status); return; }# Return unless we have results.
((keys %link_results > 0) or ($#category_results >= 0)) or
&site_html_search_failure ("Keinen passenden Eintrag gefunden") and return;# The HTML used in the output is stored here.
local ($cat_hits, $link_hits, $category_results, $link_results, $next); # Build the HTML for the category results and store it in "$category_results". Only build the html
# if we are on the first set of link results.
foreach $category (@category_results) {
if ($nh == 1) {
$cat_clean = &build_clean($category);
$linked_title = &build_linked_title ($category);
}
$cat_hits++;
}
$cat_hits ||= 0;
$lowrange = ($nh-1) * $maxhits + 1;
$highrange = $nh * $maxhits;# Go through each category of links returned, and build the HTML. Store in hash %link_output.
SETOFLINKS: foreach $setoflinks (sort keys %link_results) {
my $hits = ($#{$link_results{$setoflinks}} + 1) / ($#db_cols+1);
LINK: for ($i = 0; $i < $hits; $i++) {
$link_hits++;
if (($link_hits <= $highrange) && ($link_hits >= $lowrange)) {
%tmp = &array_to_hash ($i, @{$link_results{$setoflinks}});

$link_output{$setoflinks} .= &site_html_rsslink (%tmp) . "\n"; # RSS Mod }
}
}

# Go through the hash just built, and build the complete link output. Store in $link_results.
foreach $setoflinks (sort keys %link_output) {
$cat_clean = &build_clean ($setoflinks);
$title_linked = &build_linked_title ($setoflinks);
$link_results .= $link_output{$setoflinks};
}
# If we want to bold the search terms...
if ($search_bold) {
foreach $term (@search_terms) {
# This reg expression will do the trick, and doesn't bold things inside <> tags such as
# URL's.
$link_results =~ s,(<[^>]+>)|(\Q$term\E),defined($1) ? $1 : "$2",gie;
$category_results =~ s,(<[^>]+>)|(\Q$term\E),defined($1) ? $1 : "<b>$2</b>",gie;
}
}# If we have to many hits, let's build the next toolbar, and return only the hits we want.
my ($next_hit, $prev_hit, $next_url, $left, $right, $lower, $upper, $i);

if ($link_hits > $maxhits) {# Remove the nh= from the query string.
$next_url = $ENV{'QUERY_STRING'};
$next_url =~ s/\&nh=\d+//;
$next_hit = $nh + 1; $prev_hit = $nh - 1;# Build the next hits toolbar. It seems really complicated as we have to do
# some number crunching to keep track of where we are on the toolbar, and so
# that the toolbar stays centred. # First, set how many pages we have on the left and the right.
$left = $nh; $right = int($numhits/$maxhits) - $nh;
# Then work out what page number we can go above and below.
($left > 7) ? ($lower = $left - 7) : ($lower = 1);
($right > 7) ? ($upper = $nh + 7) : ($upper = int($link_hits/$maxhits) + 1);
# Finally, adjust those page numbers if we are near an endpoint.
(7 - $nh >= 0) and ($upper = $upper + (8 - $nh));
($nh > ($link_hits/$maxhits - 7)) and ($lower = $lower - ($nh - int($link_hits/$maxhits - 7) - 1));
$next = "";# Then let's go through the pages and build the HTML.
($nh > 1) and ($next .= qq~<a href="$build_search_url?$next_url&nh=$prev_hit">[<<]</a> ~);
for ($i = 1; $i <= int($link_hits/$maxhits) + 1; $i++) {
if ($i < $lower) { $next .= " ... "; $i = ($lower-1); next; }
if ($i > $upper) { $next .= " ... "; last; }
($i == $nh) ?
($next .= qq~$i ~) :
($next .= qq~<a href="$build_search_url?$next_url&nh=$i">$i</a> ~);
(($i * $maxhits) >= $link_hits) and last; # Special case if we hit exact.
}
$next .= qq~<a href="$build_search_url?$next_url&nh=$next_hit">[>>]</a> ~ unless ($nh == $i);
}
# Print out the HTML results.
# &site_html_search_results;
open (RSS, ">/srv/www/htdocs/web5/html/rss/rss.xml") or cgierr ("unable to open RSS page: $build_rss_url. Reason: $!");
print RSS &site_html_search_rssresults;
close RSS; }sub search {
# --------------------------------------------------------
# This routine does the actual search of the database.
#
my ($search_terms, $bool) = @_;
my ($regexp, @values, $grand_total, $match, $andmatch, $field, $or_match, %seen, $link, $tmp);# Save the reg expressions to avoid rebuilding.
$or_match = $bool ne 'and';
if ($or_match) {
for (0 .. $#{$search_terms}) {
next if (length ${$search_terms}[$_] < 2); # Skip single letter words.
$tmp .= "m/\Q${$search_terms}[$_]\E/io ||";
}
}
else {
for (0 .. $#{$search_terms}) {
next if (length ${$search_terms}[$_] < 2); # Skip single letter words.
$tmp .= "m/\Q${$search_terms}[$_]\E/io &&";
}
}
chop ($tmp); chop ($tmp);# We can also search by field names.
my @field_search;
for (0 .. $#db_cols) {
exists $in{$db_cols[$_]} and (push (@field_search, $_));
}
if (!$tmp and !@field_search) { return ("Bitte geben Sie einen Suchbegriff ein."); }
if ($tmp) { $regexp = eval "sub { $tmp }"; $@ and &cgierr ("Can't compile reg exp: $tmp! Reason: $@");}# Go through the database.
open (DB, "<$db_file_name") or &cgierr("error in search. unable to open database: $db_file_name. Reason: $!");
flock (DB, 1) if ($db_use_flock);
LINE: while (<DB>) {
/^#/ and next LINE; # Skip comment Lines.
/^\s*$/ and next LINE; # Skip blank lines.
chomp; # Remove trailing new line.
@values = &split_decode($_);
$grand_total++;# Check to see if the link matches.
$match = 0; $andmatch = 1;
if ($regexp) {
FIELD: foreach $field (@search_fields) {
$_ = $values[$field];
$or_match ?
($match = $match || &{$regexp}) :
($match = &{$regexp});
last FIELD if ($match);
}
}# Check to see if the link matches any database fields. Only exact matches
# here.
if ($or_match || $match || !$regexp) {
FIELD: foreach $field (@field_search) {
if ($or_match) {
$match = $match || ($in{$db_cols[$field]} eq $values[$field]);
$match and last FIELD;
}
else {
$match = ($in{$db_cols[$field]} eq $values[$field]);
$match or last FIELD;
}
}
}
$andmatch = $andmatch && $match;# If we have a hit, add it in!
if (($or_match && $match) or $andmatch) {
push (@{$link_results{$values[$db_category]}}, @values);
$numhits++; # We have a match!
} # Check to see if the category matches.
if ($regexp and !$seen{$values[$db_category]}++) {
$match=0; $andmatch = 1;
$_ = $values[$db_category];
$or_match ?
($match = $match || &{$regexp}) :
($match = &{$regexp});
$andmatch = $andmatch && $match; if (($or_match && $match) or $andmatch) {
$numcat++;
push (@category_results, $values[$db_category]);
}
}
}
close DB;# Word is too common, don't try and sort it, can cause problems.
if (($numhits > 50) and (($grand_total * 0.75) < $numhits)) {
return "Suchwort ist zu allgemein.";
}# Sort the results using build_sorthit found in db.pl.
foreach $link ( keys %link_results ) {
@{$link_results{$link}} = &build_sorthit (@{$link_results{$link}});
}
@category_results = sort @category_results;
return "ok";
}sub build_linked_title {
# --------------------------------------------------------
# A little different then the one found in nph-build.cgi as it also
# links up the last field as well. my ($input) = shift; my ($dir, $output, $path, $last);
foreach $dir ((split m!/!, $input)) {
$path .= "/$dir";
$dir = &build_clean ($dir);
$output .= qq|<A HREF="$build_root_url$path/">$dir</A>:|;
}
chop ($output);
return $output;
}
Quote Reply
Re: [RSS Junkie] Simple RSS Feed In reply to
Hi,

> i get an premature end of script headers.

What's in the error.log file?

Did you check to make sure you've closed out this module with:

Code:


1;


This can cause the error you mention... Let me know, I'd love to add this myself, great idea : )

Last edited by:

Brett: Dec 6, 2006, 9:54 PM
Quote Reply
Re: [Brett] Simple RSS Feed In reply to
Hi Brett,

thank you for your idea.

In the log file was "premature end of script headers".

The file is a modification of search.cgi. There is no 1;

I have given up in the moment, because this error is so unlogical. I am "hacking" perl script for many years, but this behavior of the script is very strange. Especialy the perfect browser output and then sending the same to a file it writes only a number 1 into the file. Strange....

RSS Junkie
Quote Reply
Re: [RSS Junkie] Simple RSS Feed In reply to
Hi,

has someone an idea to an external scripts, which just takes the last 10 entrys from the links.db and put it RSS like formattet into a file?

RSSJunkie
Quote Reply
Re: [RSS Junkie] Simple RSS Feed In reply to
i made an solution with an external script. You need cron for that. You can see it here: http://www.perlscripts.de

If someone is interested, just email me.

Best,
Karl