Gossamer Forum
Home : Products : Gossamer Links : Discussions :

Internal server error during import

Quote Reply
Internal server error during import
Alex,

I am finally to the stage where I have experimented importing several dozen links from files using Links|Properties|Import. All of my data is being massaged externally in Excel, exported as tab delimited, tabs converted to pipes then imported to Links. Works great until I try to import more than 600 or so links which returns "Internal Server Error" in the main frame. The corresponding error log is shown below.

[Wed Oct 24 00:31:06 2001] [error] Premature end of script headers: /usr/local/etc/httpd/htdocs/surfsafely/cgi-bin/links/admin/admin.cgi

I'm guessing a browser timeout but can't figure out the correct syntax for using nph-import.cgi. The --help is very confusing.

Can you give me the command string I need to execute from SSH which will import and "delete old data first" without timing out?

Thanks,
Quote Reply
Re: [surfsafely] Internal server error during import In reply to
Ten views to this post over three days, and five of them are mine! Guys, I really need to get this large import working. I've got 16,000 links to bring in and I'm really not too keen on the idea of importing them 500 records at a time, thirty two times!

Has anyone looked into this yet?


Quote Reply
Re: [surfsafely] Internal server error during import In reply to
Hi,

Unfortunately nph-import is only for importing from previous Links versions, or dumping/restoring backups. There isn't a command line way to import data yet.

If you don't need to alter category structures, you could use MySQLMan and do:

SELECT * FROM Links INTO OUTFILE '/path/on/your/server/links.txt'

and then edit that file, make changes, and then run:

LOAD DATA INFILE '/path/on/your/server/links.txt' INTO Links

Cheers,

Alex
--
Gossamer Threads Inc.
Quote Reply
Re: [Alex] Internal server error during import In reply to
Alex,

My import database structure is Links.URL|Links.Title|Links.Description|Links.Keywords|
Links.Email|Category.ID|Category.Name

I couldn't care less about any of the other columns and I don't mind exporting and deleting columns I don't need but I MUST be able to import entire links databases WITH category.

Can we make this a reality?

Quote Reply
Re: [Alex] Internal server error during import In reply to
Okay, I installed the Links SQL under an unused virtual domain and restored my existing database to it, then attempted to import the lists I managed outside of Links. I split my import list up into 32 files, 500 rows each. I import them one at a time, repair tables and rebuild index. On the third block of 500 I always get a fatal error. It does not matter which blocks of 500 I take each time, only that it dies on the third block.

This was such a mistake.
___________________________________________
A fatal error has occured:

Can't call method "insert_id" on an undefined value at (eval 4) line 109.

Please enable debugging in setup for more details.

Stack Trace
======================================
Links (7604): Links::environment called at /usr/local/etc/httpd/vhosts/lillypops.com/cgi-bin/links/cgi/admin/Links.pm line 487 with no arguments.Links (7604): Links::fatal called at (eval 4) line 109 with arguments (Can't call method "insert_id" on an undefined value at (eval 4) line 109.
).Links (7604): Links::Admin::editor_import_data called at /usr/local/etc/httpd/vhosts/lillypops.com/cgi-bin/links/cgi/admin/Links/Admin.pm line 40 with arguments (Links::Admin=HASH(0x8059444)).Links (7604): Links::Admin::AUTOLOAD called at /usr/local/etc/httpd/vhosts/lillypops.com/cgi-bin/links/cgi/admin/GT/SQL/Admin.pm line 81 with arguments (Links::Admin=HASH(0x8059444)).Links (7604): GT::SQL::Admin::process called at /usr/local/etc/httpd/vhosts/lillypops.com/cgi-bin/links/cgi/admin/admin.cgi line 78 with arguments (Links::Admin=HASH(0x8059444), def_path, /usr/local/etc/httpd/vhosts/lillypops.com/cgi-bin/links/cgi/admin/defs, cgi, GT::CGI=HASH(0x81ad5f8)).Links (7604): main::db_request called at /usr/local/etc/httpd/vhosts/lillypops.com/cgi-bin/links/cgi/admin/admin.cgi line 56 with no arguments.Links (7604): main::main called at /usr/local/etc/httpd/vhosts/lillypops.com/cgi-bin/links/cgi/admin/admin.cgi line 24 with no arguments.

System Information
======================================
Perl Version: 5.00503
Links SQL Version: 1.2 dev
DBI.pm Version: 1.12
Persistant Env: mod_perl (0) SpeedyCGI (0)
GT::SQL::error = URL can not contain the value '"http://www.familyeducation.com/article/0,1120,21-17920,00.html"'
@INC =
/usr/local/etc/httpd/vhosts/lillypops.com/cgi-bin/links/cgi/admin
/usr/local/lib/perl5/5.00503/i386-bsdos
/usr/local/lib/perl5/5.00503
/usr/local/lib/perl5/site_perl/5.005/i386-bsdos
/usr/local/lib/perl5/site_perl/5.005
.

CGI INPUT
======================================
db => Links
do => editor_import_data
import-delim => |
import-file => tmp/03
import-text =>


ENVIRONMENT
======================================
AUTH_TYPE => Basic
DOCUMENT_ROOT => /usr/local/etc/httpd/vhosts/lillypops.com/htdocs
GATEWAY_INTERFACE => CGI/1.1
GT_TMPDIR => /usr/local/etc/httpd/vhosts/lillypops.com/cgi-bin/links/cgi/admin/tmp
HTTP_ACCEPT => image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
HTTP_ACCEPT_ENCODING => gzip, deflate
HTTP_ACCEPT_LANGUAGE => en-us
HTTP_CONNECTION => Keep-Alive
HTTP_HOST => www.lillypops.com
HTTP_REFERER => http://www.lillypops.com/cgi-bin/links/cgi/admin/admin.cgi?db=Links&do=editor_import_data_form
HTTP_USER_AGENT => Mozilla/4.0 (compatible; MSIE 5.5; Windows 95; MSOCD)
PATH => /bin:/usr/bin:/usr/contrib/bin:/usr/local/bin:/usr/home/newebmas/bin
QUERY_STRING => do=editor_import_data&db=Links&import-file=tmp%2F03&import-text=&import-delim=%7C
REMOTE_ADDR => 24.253.175.191
REMOTE_HOST => c1833271-c.clnvl1.ct.home.com
REMOTE_PORT => 2131
REMOTE_USER => newebmas
REQUEST_METHOD => GET
REQUEST_URI => /cgi-bin/links/cgi/admin/admin.cgi?do=editor_import_data&db=Links&import-file=tmp%2F03&import-text=&import-delim=%7C
SCRIPT_FILENAME => /usr/local/etc/httpd/vhosts/lillypops.com/cgi-bin/links/cgi/admin/admin.cgi
SCRIPT_NAME => /cgi-bin/links/cgi/admin/admin.cgi
SERVER_ADMIN => webmaster@lillypops.com
SERVER_NAME => lillypops.com
SERVER_PORT => 80
SERVER_PROTOCOL => HTTP/1.1
SERVER_SOFTWARE => Apache/1.3.1.1 SSL/1.15 PHP/4.0b2


Quote Reply
Re: [Alex] Update Internal server error during import In reply to
Thread update.

I localized the problem I had importing certain files containing blocks of 500 rows of links to import to 5 files out of 32 by trying each one separately and "Deleting old data first". I "corrected" the problem by running the lists through my parsing program changing the field format to fixed then back to variable again. The resulting output files were all exactly 4 bytes larger than the originals and after this massaging imported correctly. Don't ask me why. All I can tell you is it worked.

Which still does nothing to address the problem that we can not import and rebuild the entire directory without breaking up the import files into byte sized pieces that Links can handle through a web browser without timing out.

Alex, we need a way to do this via telnet that won't time out. Please help.

Quote Reply
Re: [surfsafely] Update Internal server error during import In reply to
Well, so much for that idea. The import looked like it went well on the front end until I did a build afterward. The entire database is now filled with links pointing to http:/// and Links does not think any duplicates exist. As I write this, nph-index is reindexing my backup.

I have had enough. My LinksSQL license is for sale. Best offer takes it. Send your bids linksbids@surfsafely.com. This is not a joke.