Gossamer Forum
Home : Products : Gossamer Links : Discussions :

Image Gallery/Picture Post 4

Quote Reply
Image Gallery/Picture Post 4
I think I figured out how to solve one of the problems. A big problem is dealing with uploads on a busy system. On an admin-only, or a few editor systems, it's not much of a problem. But on busy systems it would be.

Assumption #1 for the first releases:
This is a postcards/picture-post program where user-uploads are anticipated (rather than a gallery or an admin-only upload situation). (Gallery management features will be added later once the parts work).

I propose:

1) give each user a directory <%username%> (this means restrictive usernames)
2) All a users uploads go into that directory, and thumbnails are in <%username%>/thumbnails. This allows one /uploads directory, or /uploads/a, /uploads/b etc type divisions.
3) files are given the LinkID as their stored name. A user can see their aborted files if their directory has un-attached files in it, and figuring out who an aborted file belongs to is then really easy.
4) Not every user will upload files. Not every user will have a directory. If a directory exists for that <%username%> a user has uploaded files.
5) This will dove-tail with the 'attachments' modification when/if Alex/GT decides what that will look like. Attachments can go into one central area or into <%user%> or <%category%> directories. More functions will be added with the "gallery maintain" routines, but we have to start somewhere.
6) Image/physical-disk layout will most likely _not_ resemble the presentation-interface in any way. It's more and more obvious that this is the way to go, and if you want the disk to look like the interface, just upload it that way, but the interface will not be limited by the disk-layout.

Comments... Remember, this system has to be scalable, so what might seem like over-kill on a site with 100 or even 2000 links, will be a necessity on a site with 20,000.



PUGDOG®
PUGDOG® Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
I like this a lot, it seems a really sensible way of doing it.

what do you mean by aborted files? is this when the upload dosen't finish? how do we know the file is aborted?

not really much to add, you seem to have it all very worked out.

http://www.ASciFi.com/ - The Science Fiction Portal
Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
incomplete uploads should be handled by the browser/protocol (at least I haven't found garbage files around).

More likely, it will be files that get uploaded, but do not get attached to a database record, or get detached somehow.

These will be files that do not have a LinkID as the upload name.

PUGDOG®
PUGDOG® Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
Ok,

I think it's stable and cleaned up enough to show off the file uploads.

http://postcards.com/...est/test_upload.html

Please, don't abuse it. The error reporting is not great, but these routines don't need it. Most just return good/bad or appear to succeed without doing anything -- but input (file names) are checked before being used. The big errors (which you won't see any of) are already tied into the Links error system, and use the default 'error' template.

The uploads will handle as many as the protocol will allow (at least up to 10), but I've limited it to two for the demo.

Next step will require you to get an ID on the system before you can do anything, since all of this will be tied to a 'user' record of some sort. No anonymous uploads.

PUGDOG®
PUGDOG® Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
Please hold the system open; if YOU mean you wont have anonymous uploads, means not all others see that in the same way.
I have projects with the use of registering, i have projects without; and i think the webmaster should decide what he wants, if you decide it for them, you will loose value of your mod.

Robert

Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
i have to agree, i can see the use of anonomyous uploads in some occassions, especially when all uploads must be validated anyway.

http://www.ASciFi.com/ - The Science Fiction Portal
Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
cool it works well, being able to do more than one at once is very useful.

I assume that the thumbnail size is a config option with options such as "width sclaing to x", "height scaling to x"

also is there a way to prevent someone uploading say a 50,10000 image. The reason would be say that you just shrink everything in proportion to the width as 50, then this would come out as a 10,000 image. If you can get the pixel size during the upload then an error that the images dimensions are too big could be very useful.

http://www.ASciFi.com/ - The Science Fiction Portal
Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
If I leave in anonymous uploads, it will not be a "feature" per se. It will be an "option" and will be a separate category unto itself. It's _not_ going to be integrated into the whole system. The whole system depends on user registration for too many things -- including security and LEGALITY.

There is nothing to prevent someone from uploading illegal material to your server. In some areas that's almost 'defacto' guilt. Without a good audit trail to prove how it got there, it can lead to problems. Even with "preview" or "verification" it can lead to problems. I've been doing this since about 1984, and have been involved in some of the initial "problems" and "resolutions" that came out in electronic forums etc.

I realize that most people don't think this way, but I do. It's a major loop hole I'd like to tighten up BEFORE the program gets done.

There are virtually _NO_ systems out there you can do anonymous uploads to. mostly for the legal reasons, but as I get into it deeper, it also makes tracking and tagging files _much_ _much_easier to have a userID or name associated with them.

For what good reason would you need an anonymous upload that can't be handled with a registered upload?


PUGDOG®
PUGDOG® Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
>> being able to do more than one at once is very useful.

Yes it is... for "attachments" it would either insert them into the AttachLink database using LinkID_alpha, or something similar.

>> I assume that the thumbnail size is a config option with options such
>> as "width sclaing to x", "height scaling to x"

Of course. Give it a try. Right now, I _think_ I set it to 140 x 140 (the pass it a single X value for 'square' doesn't work properly - at least on solaris). But, my own version uses 140 x 250. I'll set the defaults back, so you can see what happens if you upload odd-sized images. They are fit "into" that slot, not distorted to that size. NOTE: I will _NOT_ change that behaviour!! I _hate_ distorted images!!

>> also is there a way to prevent someone uploading say a 50,10000 image.
>> The reason would be say that you just shrink everything in proportion to
>> the width as 50, then this would come out as a 10,000 image. If you can get
>> the pixel size during the upload then an error that the images dimensions
>> are too big could be very useful.

I could add some checking for that, but you still have to upload the image, since you can only check the image after it's uploaded. I'm going to try to check filesizes "during" upload -- but I don't know if that is possible, or how that works. Those are fine points after the program is fleshed out. What that means, is after a file is transferred, I can check it's sizes and delete it if it's too big. But, "where" is that file during transfer? Can someone try and crash your server by uploading a 6000 MB file? That I don't know. It is part of the HTTP upload protocol and CGI.pm -- I'll have to figure it out down the road.

But, the problem is clip-art sites will have "lines" that are maybe 1000x longer then they are wide.

I guess I could add a user-definable height/width ratio -- with 1:5 and 5:1 being the defaults. that should cover 99% of legitimate photographic images.


PUGDOG®
PUGDOG® Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
Sounds good pugdog. The line thing is not essential really if images are validated, it would only be if they are not as any table would be ruined. I imagine i will almost certainly require validation but still useful.

Arr... the file size check, that is more of a worry. It happens after uploading... Yes i can see that a problem, it could so easily be abused with 1GB uploads wasting both space and bandwidth. Not sure what would happy if half way through an upload someone ran out of space on say a virtual account, if the script needs any disk space to run then it would not be able to delete the file.. that could be messy.

I know there are lots of upload scripts about, do you think they all do their file size checking after upload? i don't think so, so perhaps have a look there. Even if it is not possible to know the size of the file before upload, it may perhaps be possible to upload the file until the max kb is hit and then abort the transfer. Of course i don't know but it is really pretty important.

Oh and yes, i hate distortion, simply no point of it. I like the "fit it into a square" approach. Sensible.

http://www.ASciFi.com/ - The Science Fiction Portal
Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
Here's the problem. even with a javascript or java utility to check it, a user can still by-pass any checks. The only way to really check is during the filestream upload. What I have to see, is _where_ that actually takes place, and if it can be aborted. I'm sure it can -- I just have to figure out the 'safe' way to do it.



PUGDOG®
PUGDOG® Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
not sure at all but this line might be all you need?

$CGI::POST_MAX=1024 * $limit;

with $limit number of kb.

i think that should then just stop the upload if the max size is reached?

don't quote me on it...

http://www.ASciFi.com/ - The Science Fiction Portal
Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
That sounds like it's a good guess <G>

I'm starting to integrate into Links, so the upload now requires a registration. Please don't bother with bug reports yet ;)



PUGDOG®
PUGDOG® Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
I'm really using GT::CGI so I think I found the place to change there.

I asked Alex for confirmation of this, and if buffer size and max reads can be passed parameters.


PUGDOG®
PUGDOG® Enterprises, Inc.
FAQ: http://pugdog.com/FAQ


Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
That depends on the site and the users, some users are to stupid to understand all they need.
But i think its the problem of every single webmaster what he wants to do.
The programmer should give them every opportunity that´s thinkable.
But it´s ok, meanwhile im ready to programm the smaller things alone, so i think i will modify the script to my needs.
Robert

Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
As i told you the last months, links could be much more than only a search-engine; while you talk about improving your search-engines or found out that links could be used as a TGP, we use links since a long time as a search-engine, a document-management-system, a tgp and a toplist; another one still with links2 (Ascii) as a full operational cart-system. Another links is used as a database for partnerprogramms with user and programm-registry ;-)

We are no programmers at all, but we take a lot of programms and melt what we need to the scripts we need.
With this experience we see that a script should give you all possible ways to choose and not only that what the programmer means as the only right way.

Robert

Quote Reply
Re: Image Gallery/Picture Post 4 In reply to
Robert, what exactly are you talking about? Pugdog is creating this as a mod with loads of configuration options avaliable to change. He could easily make it just for himself and i am sure some of the things he is adding/changing he probably won't even use himself.

The only thing he has said he does not want to do is destort the image and he probably said that as more of a joke than anything else, but i can't really see why anyone would want to do it anyway.

As for modifying links for multiple uses he has also been doing that for years, and is doing it again. You are obviously free to modify his mod anyway.

It just sounds like you are angry/critisisng someone and i have no idea why or who! Also remember that Pugdog does not work for gossamer-threads or anything, he is making a mod for a program as it is!!!

http://www.ASciFi.com/ - The Science Fiction Portal