Gossamer Forum
Home : Products : Gossamer Links : Version 1.x :

Text quantity

Quote Reply
Text quantity
Does Links SQL cope with 'detailed' pages which contain quite large amounts of text?

Can anyone suggest what the maximum number of words could be?

Thanks
Quote Reply
Re: Text quantity In reply to
I'm not sure what the limit is, but with a text field, I'm sure it's at least 64k

I've got 18k descriptions now -- check barnes and nobles or etoys.

http://www.postcards.com/...es_that_Support_Us_/

[This message has been edited by pugdog (edited October 27, 1999).]
Quote Reply
Re: Text quantity In reply to
If you define the field as type 'TEXT' it will hold 65k. If you define it as 'MEDIUMTEXT' it will hold ~ 16 MB. If you define it as 'LONGTEXT' you can get ~ 4 GB.

Cheers,

Alex
Quote Reply
Re: Text quantity In reply to
Hello Alex!

Was just thinking of asking you and here you did before me Wink

Well, if I have a TEXT field and the users fill out the detailed page text area. They may use paragraphs and HTML contents.

My question. Will the formatting be lost if it gets into the database? Or if it does, how to retain? For e.g. An ASCII text written in paragraphs, if inserted, will it be stored just the way it was earlier filled in or will be broken by lines?

------------------
rajani











Quote Reply
Re: Text quantity In reply to
Formatting will be preserved, however it won't insert HTML for you. So if a user enters in text with newlines, when displayed in HTML the newlines won't be shown. You'll need to do a s/\n/<BR>/g; in perl to covernt newlines to html <BR> tags.

Cheers,

Alex
Quote Reply
Re: Text quantity In reply to
Hello Alex!

Hey, thats great, good to know. This little missing code was the problem I have had somewhere there. I never could make it work.

Thanks very much.



------------------
rajani











Quote Reply
Re: Text quantity In reply to
Using LONGTEXT or MEDIUMTEXT starts becoming a nightmare!!!

I still have buffer problems with mySQL and I'm not able to manage long texts. According to the mySQL documentation I did the conf change to enlarge the packet buffer but the problem persist... INSERT just refuses records without giving any kind of error.

hmmm...

Does anybody have this problem? I know this is probably a mySQL conf problem rather than Links but I think should be very important to analize this problem and be able to know how you can insert into links large amount of datas like images or documents.




Lepo
Quote Reply
Re: Text quantity In reply to
Hi Lepo,

This could be a problem with Links SQL. Quite often I do a SELECT * which means it has to retrieve a lot of data and store it in memory (if using MEDIUMTEXT OR LONGTEXT types). You might be better off saving the information to a file and storing the filename.

Cheers,

Alex
Quote Reply
Re: Text quantity In reply to
Alex,

On that issue, I've noticed you do suck a lot of data -- some of which isn't necessary. You've looked for things that would improve the utility of the program -- and intelligent field selection would be one -- for instance, not sucking the "description" fields if they aren't needed, would save the most memory.

Because you don't know ahead of times what fields are going to be used, but you do know what fields are NOT going to be used, you could pre-process the hash to remove fields that are NOT used, before asking for the records with a Select $fields rather than a Select *

I've been thinking about that a lot, and that seems to be the #1 limitation that I'm going to hit, and if I'm going to hit it, others are going to eventually too.

Another way would be to split the links table up into essential fields and extra fields. That way the select * would only grab the eseential fields, and when necessary could do the lookup for the other fields.

I haven't looked too deeply into this, since it's not something I'm going to need/run into in the forseeable future, but it's something I have to plan for, and start thinking about in terms of table splits/joins.

I've been hoping that this falls into the 'maturity' issues, and develops as LinkSQL develops. Once the features become stable, they can be tweaked for speed.

Quote Reply
Re: Text quantity In reply to
Unfortunately I don't think it's quite that easy. Because I want to make the program as expandable as possible, I want someone to be add a field to the Links table, and have it immediately show up in the link.html template. They shouldn't have to go into the code and hunt for a select statement to make it available.

Also, I don't think I do know which fields are not going to be used before hand. I suppose parsing the appropriate template before hand to pull out what tags are used is an option. I'm not that convinced that the performance hit is that significant on most sites.

For those with large fields > 50kb, I think once built in file attachment support is available, it will allow people to easily and transparently store large amounts of data without any performance hit.

Cheers,

Alex
Quote Reply
Re: Text quantity In reply to
Alex,

you don't really need to know what fields are in the templates... the offending fields are most likely description or 'Text' fields.

It's a little reworking of the logic to fetch the information when needed, rather than up front.

As I said, this is something that will mature as the product settles down. Things will become more evident that right now are obscured by the rush to get the new program features working.

There is a limit to physical memory on a system, but CPU keeps increasing, so it's going to get easier to work with the 'identifier' fields, then pick out the other fields when you need them. This part is beyond me at this point -- and hopefully when I run into it Links will either have matured, or I'll have a separate database server running something awesome Wink

Quote Reply
Re: Text quantity In reply to
What I'm saying though, is I can't not fetch the Description field, as without looking at the template before hand, it might be used. To change it so that the information is fetched when the template is parsed would hurt more then help I'd suspect.

Cheers,

Alex