Gossamer Forum
(Advanced Search)
for
Home : Products : Gossamer Links : Development, Plugins and Globals :

Problems with Spider Plugin

May 23, 2003, 2:30 AM
erdmann21
Novice (6 posts)
May 23, 2003, 2:30 AM
Post #1 of 8
Views: 3567
Shortcut
Quote Reply
Problems with Spider Plugin
Hello,

my demonized spider.pl works fine but after 10 minutes there come the following error message:

A fatal error has occured:
Wide character in subroutine entry at ../../GT/HTML.pm line 460.

and the spider.pl stoped. Can anyone help me?

Thans

Hans-Joachim
May 23, 2003, 3:01 AM
Andy
Veteran / Moderator (18441 posts)
Andy's avatar
May 23, 2003, 3:01 AM
Post #2 of 8
Views: 3504
Shortcut
Quote Reply
Re: [erdmann21] Problems with Spider Plugin In reply to
What version of Links SQL are you using? The GT Spider was taken off the market due to a problem in later versions of LSQL.

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
May 23, 2003, 3:12 AM
erdmann21
Novice (6 posts)
May 23, 2003, 3:12 AM
Post #3 of 8
Views: 3509
Shortcut
Quote Reply
Re: [Andy] Problems with Spider Plugin In reply to
Link SQL is Version: 2.1.1
May 23, 2003, 3:16 AM
Andy
Veteran / Moderator (18441 posts)
Andy's avatar
May 23, 2003, 3:16 AM
Post #4 of 8
Views: 3493
Shortcut
Quote Reply
Re: [erdmann21] Problems with Spider Plugin In reply to
Has it been working before? I think v2.1.1, 2.1.2 and 2.1.3 all have problems with running the spider Frown

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
May 23, 2003, 3:28 AM
erdmann21
Novice (6 posts)
May 23, 2003, 3:28 AM
Post #5 of 8
Views: 3530
Shortcut
Quote Reply
Re: [Andy] Problems with Spider Plugin In reply to
No, it not works fine because the linux computer, the LSQL is working, was very slow if the spider was on.

I thought, that would be a problem of the computer.

Thanks for your answer.

Hans-Joachim
May 23, 2003, 4:04 AM
Andy
Veteran / Moderator (18441 posts)
Andy's avatar
May 23, 2003, 4:04 AM
Post #6 of 8
Views: 3483
Shortcut
Quote Reply
Re: [erdmann21] Problems with Spider Plugin In reply to
Your best bet is to talk to GT support at support@gossamer-threads.com.

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
May 31, 2003, 7:53 AM
jaltuve
User (219 posts)
May 31, 2003, 7:53 AM
Post #7 of 8
Views: 3457
Shortcut
Quote Reply
Re: [erdmann21] Problems with Spider Plugin In reply to
The problem you are having is related to foreign characters and the method the spiders uses to fingerprint each record to identify it as unique. If you want to solve the problem You'll need to edit the HTML.pm file and encode using utf8. locate the following lines and update your HTML.pm file. in bold is what you need to add, the last line I think needs to be completely changed.

Code:
use GT::MD5 qw( md5_hex );
use Encode qw(encode_utf8);


Code:
# calculate the MD5 value of the text
$parse{'digest'} = md5_hex(encode_utf8($parse{'bare_text'} || $document));
Jun 2, 2003, 12:43 AM
erdmann21
Novice (6 posts)
Jun 2, 2003, 12:43 AM
Post #8 of 8
Views: 3421
Shortcut
Quote Reply
Re: [jaltuve] Problems with Spider Plugin In reply to
Thanks a lot. It works fine. Smile

Hans-Joachim