Gossamer Forum
Home : Products : Gossamer Links : Discussions :

Import from Excel

Quote Reply
Import from Excel
Hi!

i have read many threads about importing data from excel to links database .. i found all those articles about LOAD DATA INFILE and others but that doesn't help me.
Is there a way to get simple explanation on how to do this .. i mean for a newbie Unsure

p.s. i have purchased Links Suite III for spidering the net and get related links .. they do have converter but only for links 2.0

thanx in advance
Blush

Gregor
Quote Reply
Re: [sc2utp] Import from Excel In reply to
Hi,

I would try Database->Links->Properties->Export Data. Export that to a file, and load it into Excel. Make any changes you want (add new entries, etc), then go to Database->Links->Properties->Import Data and load the file back in again.

Cheers,

Alex
--
Gossamer Threads Inc.
Quote Reply
Re: [Alex] Import from Excel In reply to
Hi!
wonderful ... i'm moving on :)) but (there always have to be BUT) i got error
When i exported to xls format in Excel when i try to open it said SYLK Format Not Suported (windows ME) then i exported in txt and imported txt file to excel .. delimiter .. everything great .. except there is no category no keywords ..some fields where missing.
In link properties i did "Add ALL" but there is no keywords and categories .. thats why i got error on import
"2: Failed validation. Error:
You did not specify a category for this link.
"
.. stucked but i know i'm on the right way:)

thanx


Gregor
Quote Reply
Re: [sc2utp] Import from Excel In reply to
sorry mistype .. when i try to open it in Excel it says "SYLK file format is not valid"

tanx

Gregor
Quote Reply
Re: [Alex] Import from Excel In reply to
Here are my links properties;

Edit Links Table Definition
From here you can change the settings of your Links table:

Position Column
Name Column
Type Not
Null Default Form
Display Form
Type Form
Regex Search
Weight
1 ID INT Yes TEXT ^\d+$
2 Title VARCHAR(100) Yes TEXT 3
3 URL VARCHAR(255) Yes http:// TEXT ^(https?|ftp|news|mailto):\/\/ 1
4 LinkOwner VARCHAR(50) Yes admin TEXT
5 Add_Date DATE Yes 0000-00-00 DATE
6 Mod_Date DATE Yes 0000-00-00 DATE
7 Description TEXT No TEXTAREA 1
8 Hits SMALLINT Yes TEXT ^\d+$
9 isNew ENUM(No, Yes) Yes No SELECT
10 isChanged ENUM(No, Yes) Yes No SELECT
11 isPopular ENUM(No, Yes) Yes No SELECT
12 isValidated ENUM(No, Yes) Yes Yes SELECT
13 Rating FLOAT Yes 0.00 TEXT ^\d+\.?\d*$
14 Votes SMALLINT Yes TEXT ^\d+$
15 Status SMALLINT Yes TEXT ^\-?\d+$
16 Date_Checked DATETIME No DATE
17 Timestmp TIMESTAMP No DATE
18 Recommended INT(11) No TEXT

.. no pasword .. no keywords .. no category .. thats why i cannot import it back when edited in excel ..

any idea ...

thanx

Gregor
Quote Reply
Re: [sc2utp] Import from Excel In reply to
In reply to a private email

Their Import/Export mechanisms still have bugs to be worked out. If you have an existing database of links, go to Links|Properties|Export and export to a file. The new file will be placed in your cgi-bin/links/admin web server directory. The very first line of the exported links is the most important. It is a pipe delimited list of all the database field names exported. You can not choose to export only some fields. It exports all Links table fields plus Category.ID and Category.Name.

Your Excell database will have to have or create on the fly all columns during import. Then, if there's any columns you don't want, you delete those in Excell, massage your data the way you want it and export to a text file again. This time, the first line should contain the same pipe delimited list of database field names but just the ones that you kept, PLUS Category.ID and Category.Name. You can not delete the category information or it will not import back into links. Upload the new file to your cgi-bin/links/admin directory and go to Links|Properties|Import to import the file.

That's how I did it. The problem I have is my original import was done via MySQLMan which allowed me to import records into the Links table WITHOUT category information. Through MySQLMan I can still export my entire Links table but also without category information.

The dilemma I am faced with is I have to figure out how to merge my Links|Properties|Exported file which contains only those links that were categorized with the MySQLMan export that has everything BUT category information so that there are no duplicates and every record gets assigned a Category.ID and Category.Name. I presume I am going to have to do this _IN_ Excell but I am not versed in Excell. I am going to have to learn Excell before I can go the next step. This is becoming a huge pain in the posterior.


Quote Reply
Re: [surfsafely] Import from Excel In reply to
Hi!

Quote:
It exports all Links table fields plus Category.ID and Category.Name.

That what my problem is .. it doesn't export category.ID and categroy.name .. here is a snippet of downladed file (exported)

Code:
ID|Title|URL|LinkOwner|Add_Date|Mod_Date|Description|Hits|isNew|isChanged|isPopular|isValidated|Rating|Votes|Status|Date_Checked|Timestmp|Recommended
2|Kylie Minogue|http://www.kylie.com/|info@kylie.com|2001-09-23|2001-10-03|the official Kylie Minogue site|0|No|No|No|Yes|0.00|0|200|2001-10-10 00:00:00|20011010223824|
3|A*Teens|http://www.a-teens.com/|webmaster@a-teens.com|2001-09-24|2001-09-24|official site about the four Stockholm teenagers performing the songs of ABBA. |2|No|No|No|Yes|0.00|0|200|2001-10-10 00:00:00|20011010223824|

so .. i'm stucked .. if there were categories exported then there should n't be any problem .. i can do in ecxell everything else ...

bye


Gregor
Quote Reply
Re: [sc2utp] Import from Excel In reply to
Hi,

You need to be running 2.0.5 for the import/export to include category information.

Cheers,

Alex
--
Gossamer Threads Inc.
Quote Reply
Re: [Alex] Import from Excel In reply to
so .. there is no way to do this in 2.01

Gregor
Quote Reply
Re: [sc2utp] Import from Excel In reply to
Well there you have it. 2.0.1 will not export Category information. If I remember correctly, 2.0.4 did but that's not the current version. 2.0.5 is and you should definitely upgrade. Just pay close attention to the Table modifications.

P.S. Alex - The text style buttons for the new forum software looks really odd in Navigator 4.77. Something tells me you did not validate your code for Netscape.

Quote Reply
Re: [sc2utp] Import from Excel In reply to
Hi Gregor,

we had this kind of error, too. When you want to work with your data in Excel and you have a text table with pipe (|) or tab delimiter you can avoid this problem by
1. opening the file with editor
2. copy all (CTRL-C)
3. open Excel
4. save the file as name.txt
5. open and import it with excel (set delimiter, properties ...)
Maybe not the best way but it works.

Cheers

Niko
Quote Reply
Re: [Alex] Import from Excel In reply to
Hi Alex

I am just getting into this export/import thing as a means of more efficiently altering cats and subcats. When exporting to a file does it matter what you call the file or can it be anything ( ie I tried zzz) as it would be easy to remember and find at the bottom of my cgi-bin.

Also, I am on a mac using excel for the mac, is there any possibility of this affecting the data so when I import the data gets screewed up?

thanks
Quote Reply
Re: [Alex] Import from Excel In reply to
Hello Alex!

This is also a problem that I am having.

What is desired is to import from a file in an ACSII format.

If there was a choice through a javascript as to which tables needs to be imported, then one could import the entire database or a table.

By having an import possibility from an ASCII file, it would help many users to work the database in Excel or Acess or mdb format.

Having binary format bring a lot of problems.

For e.g. it is difficult to change the table name from the exported binary of a database and restore into anathor database from the backup having a different table name.
Quote Reply
Re: [rajani] Import from Excel In reply to
Hey Rajani, do you find a solution for this problem?
At the moment i think about doing it with PhpMyAdmin;
while it should be easy with one table, i must find a way to pass the ID to the second one.

Maybe it would be the best if i do it with a special php, that insert the things i want.

Robert
Quote Reply
Re: [Robert] Import from Excel In reply to
Hello Robert!

Now, once if your database info came out of MySQL and got into Access for further work, including the category info, then its a nightmare waiting for further work. This may, in many cases necessary.

Therefore, if you have different tables in different a lets say "Normal format" in Access, meaning thereby, having Category information as full name out there, then while insert of a link into the table, it requires that Cat_ID in there. Here you cannot even have the category ID in the access, because when you import then it shall have a new ID. I think this problem has been solved, though not sure.

To get a text file into the database is something that we are all able to do. But the scripts are so designed that it has an internal relationships to aquire handles of the field objects to build RDBMS, which we know why. The problem is not only a simple import but also the related information inserting into other fields into other tables. Thats the most difficult task.

Alex gave a solution, a small sub routine to update the fields with the ID in anathor tables, if I vaguely remember. You could try your luck to find it out in this forum.

If there is a sub_routine that converts that category names to their local IDs during imports and viseversa, that would be the only way to work it out.
Quote Reply
Re: [rajani] Import from Excel In reply to
Ok, here some thoughts at a morning full of headaches after another long night:
First we need an ASCII-File, thatīs clear. Till Excel 2000 there is a problem with export, so we need ACCESS, too, to get this into ASCII.

Then we need a php:

Take the first line, then INSERT all fields, but not ID and CatID as new row into Links; get the ID of this new row;
insert ID from the new row and ID of the Cat into CatLinks.

No problem, so for, isnīt it so?

My problems in thought begin while i need to have import a lot of pictures, too.
I have no relations between cats.

I will start this in a minute; With Fileupload for the ASCII and maybe a web-based administration to get the fields of the file and the cols of the db to pass them to.

Robert

Maybe i should say, that i dont need to import cats or something else; itīs only to import ASCII (or Excel via ASCII)
Quote Reply
Re: [Robert] Import from Excel In reply to
In Reply To:
My problems in thought begin while i need to have import a lot of pictures, too.
I have no relations between cats.

Then you should not have any problem absolutely. Because what you need, the phpAdmin will do it excellent.

The problem is only if you need to fill in the Cat ID somewhere. Or you could also use the default of the field value of the Mysql table, so that when it will not have then it will enter a default value.

Or, am I missunderstanding?

If you need to fill the new Link_ID into anathor table, then you need to develop something, ofcourse. But that would be in the File Upload plug-in, isnt it?

Last edited by:

rajani: May 11, 2002, 6:25 AM
Quote Reply
Re: [rajani] Import from Excel In reply to
Hi Rajani, have done it with php. Its not very comfortable now, but i could import 6000 rows and be very happy not to sit the next two years about it ;-)

Its done something like this:

Import a .txt, take a row, split to values,
then INSERT in Links and ask for the last ID.
Then INSERT this ID and the catid into catlinks.
Maybe i will write it later to a real tool, but at the moment,
im happy to go on with the rest of the site instead.

Robert
Quote Reply
Re: [Robert] Import from Excel In reply to
In Reply To:
... be very happy not to sit the next two years about it ;-)

Carefully chosen....

Hello Robert!

This would also help me a bit. Although I am not too far.

I would appreciate if you could share your work. Wie gehts dein einDeva?

Thanks
Quote Reply
Re: [rajani] Import from Excel In reply to
Hi Rajani,

have just programmed another one to duplicate a link ;-)
Such small things are easy to do for a programmer, but i did it, too (in some hours ;-)
The problem to share it here, is that itīs very special to my needs.
But if you need it, i will send it to you via mail and give some advice i wanted.

or try this as basic information:
Code:


<?php

//SQL
// ---------------------------------
$sql_type = "mysql"; #sql_type
$sql_server = "localhost"; #sql_server
$sql_user = "******"; #sql_user
$sql_pass = "******"; #sql_pass
$sql_db = "****";
$sql_dbL = "Links"; #sql_db Links
$sql_dbCL = "CatLinks"; #sql_db Category

$data = "/***/***/data.txt"; #Path to data.txt


// Fields for my info
// ---------------------------------
$Lcols = "ID, Title, URL, LinkOwner,Add_Date,Mod_Date,Description,Contact_Name ,Contact_Email,
Hits, isNew, isChanged, isPopular, isValidated, Rating, Votes, Status, Date_Checked,
Timestmp, Aufmacher, Text, Link, HTML, Comment, Priority, Wertung, Detailed,
Logo, LogoG, Bild01, Bild02, Bild03, Bild04, Bild05, Media";

$CLcols = "LinkID, CategoryID";

$lncols = "ID,Title,URL,LinkOwner,Add_Date,Mod_Date,Description,Contact_Name,Contact_Email,
Hits,isNew,isChanged,isPopular,isValidated,Rating,Votes,Status,Text";

// Connect SQL
// ---------------------------------
@mysql_pconnect("$sql_server", "$sql_user", "$sql_pass")
or die ("No connect to server!");

@mysql_select_db("$sql_db")
or die ("No connect to db: $sql_db !");

// Get from ASCII

$datei = "$data";
$do = fopen($datei, "r")
or die ("Cant open $datei");

while ($zeile = fgets ($do,4096)):
list($catid, $short, $title, $text) = split('\|',$zeile); # DIVIDED BY -->|<-- !!!!!

I have only four fields to import and melt $short and $tilte to Title !!!

// Ltrim, Spaces Left not neccessary at all
$title = ltrim($title); $text = ltrim($text); $short = ltrim($short);
// Replace ' and "
$title = str_replace("'"," ",$title); $text = str_replace("'"," ",$text);
$title = str_replace("\"","",$title); $text = str_replace("\""," ",$text);

echo "$catid<br>$short - $title<br>$text<br><br>"; # for info
// -- MySQL ---------------------------

$query="INSERT INTO $sql_dbL ($lncols)
VALUES ('',
'$short - $title',
'http://','admin',
'2002-05-11',
'2002-05-11',
'',
'Name','name@domain.de',
'0','No','No','No','Yes','0.00','0','0',
'$text')";

$rs = mysql_query($query);
if ($rs){
echo "<b>$rs</b>"; echo "$query"; echo "<br><br>";
$newid = mysql_insert_id();
} else echo "Error!!!";

$query2="INSERT INTO $sql_dbCL (LinkID, CategoryID)
VALUES ('$newid','$catid')";
$rs2 = mysql_query($query2);
if ($rs2){ echo "<b>$rs2</b>"; echo "$query2"; echo "<br><br>";
}else echo "Error!!!";

endwhile;
fclose($do);
mysql_close();
?>




This was only to import some data from Excel, not to export/import old data from Links.
Itīs not really save or comfortable ...

BTW: If you have problems to export from Excel, do the following:

Change all cells to TEXT!
Save Excel !
Open Access
Import in a new table in a new db.
Then change the sight and change all fields to text or memo.
Export without index and divide with |, no "around" fields.

Itīs theoretical very easy, but every month i have to update 10.000 rows for a client on this way in an old Links 1.13 and while i do it sometimes in 5 minutes, i often stuck on small faults.
Problems like spaces before text, '"&%§$ and others.
But the next php to solve this too, is on the run ;-)

Do you mean "Adeva" ? You mean the old site for Links Mods?
Have just thought about if its worth to build up a new site to get some clients over GT,

while we have some big projects out, build with Links (and a lot of mods ;-)
Try this e.g. http://www.dbl-ev.de
Build with 4 times Links 1.13 and a lot of connection between.

The next generation is in work, still try to have different tables for different cats for additional data. But time is short and i have to do some hours more this night.

Robert