Gossamer Forum
Home : Products : Links 2.0 : Discussions :

SPAM in Links 2.0

Quote Reply
SPAM in Links 2.0
Dear friends,

This is my first message to the Gossamer forums, greetings to all.

Recently, we have been receiving a constant amount of automated SPAM entries into Links 2.0. We installed the following CAPTCHA system (see the following link), including updating the Referrer settings to "@db_referers = ('www.ourdomain.com','ourdomain.com');", and it stopped them.

http://www.gossamer-threads.com/...tring=captch;#296545

Except one. I'm assuming that the SPAMMER was upset with this new CAPTCHA system and now manually types in the entries. The interesting thing is this: on the website access log, there is only one entry showing up when he submits his website links. It appears as such:

Quote:
www.ourdomain.com milenko.idegp.com - - [13/May/2009:13:23:21 -0500] "POST /cgi-bin/links/add.cgi HTTP/1.1" 200 9958 "http://www.ourdomain.com/cgi-bin/links/add.cgi" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"


Somehow, he is bypassing the CAPTCHA system, since normally there should be an entry in the access log such as:

Quote:
"GET /human/26fe40883303ae87c866ca358381cf3b.png"


The referrer host information on his links appear as:

shkarupilo.idegp.com
milenko.idegp.com
talanov.idegp.com

And his email addresses end with i.ua, appears as:

something@i.ua

I've searched those domains at networksolutions and found the IP ranges and put them in the server's host.deny fille. Yet, the SPAM links continue to go through. I hope this information proves useful in finding a resolution for this specific problem and making the world a better place to live in. If anyone has any ideas on how to stop this SPAMMER, please share your solution.

Last edited by:

Zoro: May 14, 2009, 4:47 AM
Quote Reply
Re: [Zoro] SPAM in Links 2.0 In reply to
Been a while since I worked on Links2, but sounds like you just need a way of blocking their domain.

Is it the same domain they keep submitting?

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] SPAM in Links 2.0 In reply to
Andy wrote:
Been a while since I worked on Links2, but sounds like you just need a way of blocking their domain.


Thanks for your message. Yes, indeed, but I'm not sure what domain he's coming from. As I've mentioned, the only piece of information of this SPAMMER's presence in the website logs is this...

Quote:
www.ourdomain.com milenko.idegp.com - - [13/May/2009:13:23:21 -0500] "POST /cgi-bin/links/add.cgi HTTP/1.1" 200 9958 "http://www.ourdomain.com/cgi-bin/links/add.cgi" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"


Andy wrote:
Is it the same domain they keep submitting?


The websites he submits (all fields properly filled out) are all random and are not related to his domain nor are they related to our specific directory. We go to great lengths in making sure people understand what is appropriate to submit to the directory. Unfortunately, the SPAMMER does not see that page since he is not on the website add page. If he was, then the system would have the CAPTCHA and the successful link confirmation page information appear within the logs.

The only common thing in all the websites he submits, is the email addresses, which appear as SomeRandomName@i.ua

As of this morning, three more websites have been added with the following Referrer Hosts:

shkarupilo.idegp.com
pepelats.idegp.com
246-194-124-91.pool.ukrtel.net

Here's the latest entry in the website logs, this one includes the CAPTCHA information:



www.ourdomain.com milenko.idegp.com - - [14/May/2009:09:23:15 -0500] "GET /cgi-bin/links/add.cgi HTTP/1.1" 200 11952 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
www.ourdomain.com milenko.idegp.com - - [14/May/2009:09:23:16 -0500] "GET /human/c8d6a040f9496df3775299fff3fdd7cb.png HTTP/1.1" 200 4166 "http://www.ourdomain.com/cgi-bin/links/add.cgi" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
www.ourdomain.com milenko.idegp.com - - [14/May/2009:09:24:42 -0500] "POST /cgi-bin/links/add.cgi HTTP/1.1" 200 9986 "http://www.ourdomain.com/cgi-bin/links/add.cgi" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"



I have put the entire range of idegp.com IP addresses on the host.deny file. Still, he manages to bypass with a single line of information appearing in the server logs. Thus, what appears as idegp.com in the Referrer Host settings may not actually be that domain.

Last edited by:

Zoro: May 14, 2009, 7:43 AM
Quote Reply
Re: [Zoro] SPAM in Links 2.0 In reply to
Hi,

If the email remains the same, you can do something like this in add.cgi:
Code:
use CGI;
my $IN = new CGI;
if ($IN->param('Contact_Email') =~ /.*\@i.ua/i) {
&site_html_add_success;
exit;
}

..just before:

Code:
eval {
($0 =~ m,(.*)/[^/]+,) && unshift (@INC, "$1"); # Get the script location: UNIX /
($0 =~ m,(.*)\\[^\\]+,) && unshift (@INC, "$1"); # Get the script location: Windows \


That will make him think the record was added, but it won't actually submit it :) Obviously that relies on him still using the *@i.us email address when submitting.

Hope that helps.

NB, this isn't tested - as I don't have a copy of Links2 installed any more - but should work fine.

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] SPAM in Links 2.0 In reply to
Thank you for the email code Andy, I tested it for a few days and found that it motivated them to do more on a daily basis.

Then, I searched the Gossamer Threads for effective ways to stop this SPAMMER and implemented the following:

As previously mentioned, installed Human CAPTCHA and effectively stopped all automatic computer bots SPAMMING the directory. Then updated the following in hopes of stopping the few manual SPAMMERs:

Changed this line in add.cgi...
Code:
if (@db_referers and $ENV{'HTTP_REFERER'}) {


Into...
Code:
if (@db_referers) {


Updated these in add.cgi for more information gathering...
Code:

Remote Host: $ENV{'REMOTE_HOST'}
Remote IP: $ENV{'REMOTE_ADDR'}
Referer: $ENV{'HTTP_REFERER'}
Submitted Using: $ENV{'HTTP_USER_AGENT'};


Lastly, after getting the SPAMMER's remote host and IP information, checked them on Network Solutions and added the entire range in the server's host.deny file, even adding individual specific IP addresses from their submissions. These changes have reduced the 10 a day submissions, down to 5 a day. However, they are still able to get through. Here's the latest submission:
Quote:
Title: Strategic potential of the company
URL: www . all about strategies . com
Category: Health
Description: Realization of strategic decisions changes potential of enterprise, and a return to the previous state of object of management even if it is possible, demands time, resources or efforts.
Contact Name: Janice
Contact Email: janicevenezia@i.ua
Remote Host: shkarupilo.idegp.com
Remote IP: 193.201.100.187
Referer: http://www.ourdomain.com/cgi-bin/links.add.cgi

Submitted Using: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; SU 3.21; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; InfoPath.1);


How are these manual SPAM bastards able to bypass all these precautions and are there any other solutions to keep this beneficial ADD feature available without requiring user registrations or upgrades? I firmly believe right shall ultimately prevail. Any guidance will be appreciated.

Last edited by:

Zoro: May 18, 2009, 8:57 PM
Quote Reply
Re: [Andy] SPAM in Links 2.0 In reply to
Andy wrote:
If the email remains the same, you can do something like this in add.cgi:
Code:

use CGI;
my $IN = new CGI;
if ($IN->param('Contact_Email') =~ /.*\@i.ua/i) {
&site_html_add_success;
exit;
}

..just before:

Code:
eval {
($0 =~ m,(.*)/[^/]+,) && unshift (@INC, "$1"); # Get the script location: UNIX /
($0 =~ m,(.*)\\[^\\]+,) && unshift (@INC, "$1"); # Get the script location: Windows \



That will make him think the record was added, but it won't actually submit it :) Obviously that relies on him still using the *@i.us email address when submitting.


Andy, thanks for the code, I appreciate it very much. I've updated the add.cgi file again with your code and began testing it. When I enter the SPAMMER's link information and specific email address (someone @ i.ua), the page simply refreshes and the entire fields which were typed in do not appear. Basically, the SPAMMER will not see the "Success" confirmation page, but is presented with the same blank add page. I'm using Internet Explorer v.7 and use the html templates in Links 2.0.

Last edited by:

Zoro: May 19, 2009, 9:48 PM
Quote Reply
Re: [Zoro] SPAM in Links 2.0 In reply to
Ok, had to install Links2 and have a play around with this. Finally got it working, by finding this line in add.cgi:

Code:
# Set date variable to today's date.
$in{$db_cols[$db_modified]} = &get_date;

..and adding the following to it:

Code:
# ANDY MOD
if ($in{"Contact Email"} =~ /.*\@i.ua/i) {
&site_html_add_success;
exit;
}
# END ANDY MOD

(the main issue was where we placed the code, and also the _ doesn't exist in Links2, like it does in GossamerLinks)

Hope that helps.

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] SPAM in Links 2.0 In reply to
Andy, thanks for your message, the code worked.

I thought of another idea whereby the SPAMMER doesn't reach the "Success" page, since I don't want them to think they are accomplishing their goals of posting SPAM irregardless to our displayed guidelines. With the current code, it relies on the email address being "i.ua", and then sends them to the success page.

Besides the currently installed and required CAPTCHA on the form, I'd like to ask a simple question with a text box (<input name="Question" value="" size="40">). The code within add.cgi will have a couple of variations of the correct word, i.e. ('word','Word'). Since our community speaks another language, only our users will be able to answer that specific question, even Google Translate won't be able to help them.

In this way, these SPAMMERs will not receive the success page and will have to search for many days or months for the English translation of the word, and if they happen to decipher it, I will change the question on the form and change the answer in add.cgi.

Andy, what do you think of this solution, is it possible?

Last edited by:

Zoro: May 20, 2009, 11:37 AM
Quote Reply
Re: [Zoro] SPAM in Links 2.0 In reply to
Hi,

Not sure - afraid I don't have any time today to look at that. Maybe later in the week.

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] SPAM in Links 2.0 In reply to
   
Great, thanks Andy!

I think having a simple question (<input name="Question" value="" size="40">) after the CAPTCHA information, will deter the remaining manual SPAMMERs and help keep this beneficial add feature available to the visitors. Since the answer (using text or numbers) will be in the code within the add.cgi, they will not be able to pass this security feature.

Ideally, I'd like to be able to use a %Question% in the add.html template to pull the Question and Answer box into the add.cgi, similarly to the way the Human CAPTCHA system works. Thus, future changes to the two "Question and Answer" variables can be easily done within add.cgi and reuploaded as often as needed.

Thanks again. I look forward to hearing from you soon.

Last edited by:

Zoro: May 27, 2009, 7:52 AM
Quote Reply
Re: [Zoro] SPAM in Links 2.0 In reply to
Hi,

You would need someone like PerlFlunkie, who's got quite a bit more experience in Links2 (as I said, I havn't worked with Links2 in about 7 years - as I deal with Gossamer Links SQL now, not the flat-file version)

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] SPAM in Links 2.0 In reply to
Andy wrote:
You would need someone like PerlFlunkie, who's got quite a bit more experience in Links2...

Thanks for your message Andy.

PerlFlunkie, could you please help me with this "Question & Answer" modification?
I appreciate your help very much.
Quote Reply
Re: [Zoro] SPAM in Links 2.0 In reply to
Maybe send him a PM, as he should get an email notificiation of that :)

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] SPAM in Links 2.0 In reply to
Zoro, do you still need this?


Leonard
aka PerlFlunkie
Quote Reply
Re: [PerlFlunkie] SPAM in Links 2.0 In reply to
PerlFlunkie wrote:
Zoro, do you still need this?

Thank you PerlFlunkie for your message.
Yes I still need this as it would be a good additional security feature (see above post #8 and #10). I look forward to adding your code to help keep this useful "Add" feature available.
Quote Reply
Re: [Zoro] SPAM in Links 2.0 In reply to
Took a few hours to find the best way to write it, but this works.

In add.cgi, add the red code. Note you can use as many words as needed in the array @words = qw(4 four FOUR); for variable spellings, just separate them with a space.

Code:
sub process_form {
# --------------------------------------------------------
my ($key, $status, $line, $output);
# Check the referer.
if (@db_referers and $ENV{'HTTP_REFERER'}) {
$found = 0;
foreach (@db_referers) {
$ENV{'HTTP_REFERER'} =~ /$_/i and $found++ and last;
}
if (!$found) {
&site_html_add_failure ("Auto submission is not allowed in this directory. Please visit the site to add your entry.");
return;
}
}
## >>> Security Field mod
## Set security words on next line.
@words = qw(4 four FOUR);
unless(grep /^($in{'question'})$/, @words){
&site_html_add_failure("You entered the wrong security word.") and return;
}
## <<<


Then in you add.html and add_error.html templates, put this:
Code:

<tr><td align="right" valign="top">Security Question:What is the sum of 2 + 2?</td>
<td><input name="question" value="" size="40"></td></tr>


And for some reason the sub site_html_add_failure routine in site_html_templates.pl needs to know what ID means:
Code:
sub site_html_add_failure {
# --------------------------------------------------------
# This routine determines how the add failure page will look like.
my ($errormsg) = shift;
$in{'Category'} ?
($in{'Category'} = qq~<input type=hidden name="Category" value="$in{'Category'}">$in{'Category'}~) :
($in{'Category'} = &build_select_field ("Category"));

&html_print_headers;
print &load_template ('add_error.html', {
ID => $id,
error => $errormsg,
%in,
%globals
});
}


Leonard
aka PerlFlunkie
Quote Reply
Re: [PerlFlunkie] SPAM in Links 2.0 In reply to
Dear PerlFlunkie,

Excellent! By using both of your codes, the Captcha and the Question, the SPAM problems have been resolved.

For others interested in the Captcha solution, the previous website which hosted and developed the Captcha software is no longer available. However, I've attached the zip file of the original Captcha software to this message in hopes of helping others. In summary, use the zip file to get the Captcha code and images, then use PerlFlunkie's code by following his helpful step-by-step instructions.

PerlFlunkie, thank you for all that you do, it makes a difference.

Keep up the great work and spirits!

Zoro

Last edited by:

Zoro: Mar 20, 2011, 10:03 PM