Alex,
On that issue, I've noticed you do suck a lot of data -- some of which isn't necessary. You've looked for things that would improve the utility of the program -- and intelligent field selection would be one -- for instance, not sucking the "description" fields if they aren't needed, would save the most memory.
Because you don't know ahead of times what fields are going to be used, but you do know what fields are NOT going to be used, you could pre-process the hash to remove fields that are NOT used, before asking for the records with a Select $fields rather than a Select *
I've been thinking about that a lot, and that seems to be the #1 limitation that I'm going to hit, and if I'm going to hit it, others are going to eventually too.
Another way would be to split the links table up into essential fields and extra fields. That way the select * would only grab the eseential fields, and when necessary could do the lookup for the other fields.
I haven't looked too deeply into this, since it's not something I'm going to need/run into in the forseeable future, but it's something I have to plan for, and start thinking about in terms of table splits/joins.
I've been hoping that this falls into the 'maturity' issues, and develops as LinkSQL develops. Once the features become stable, they can be tweaked for speed.
On that issue, I've noticed you do suck a lot of data -- some of which isn't necessary. You've looked for things that would improve the utility of the program -- and intelligent field selection would be one -- for instance, not sucking the "description" fields if they aren't needed, would save the most memory.
Because you don't know ahead of times what fields are going to be used, but you do know what fields are NOT going to be used, you could pre-process the hash to remove fields that are NOT used, before asking for the records with a Select $fields rather than a Select *
I've been thinking about that a lot, and that seems to be the #1 limitation that I'm going to hit, and if I'm going to hit it, others are going to eventually too.
Another way would be to split the links table up into essential fields and extra fields. That way the select * would only grab the eseential fields, and when necessary could do the lookup for the other fields.
I haven't looked too deeply into this, since it's not something I'm going to need/run into in the forseeable future, but it's something I have to plan for, and start thinking about in terms of table splits/joins.
I've been hoping that this falls into the 'maturity' issues, and develops as LinkSQL develops. Once the features become stable, they can be tweaked for speed.
