aw at ice-sa
Feb 7, 2012, 1:34 PM
Post #16 of 21
mike cardeiro wrote:
>> From: André Warnier <aw [at] ice-sa>
>> Very good. And by the same mechanism, you are already avoiding the problem with
>> all the silly (and sometimes dangerous) names that people can give to their
> that is precisely why I did it this way.
You would be surprised at the number of people who don't think of that issue until after
they have put their system in place (not even mentioning the question of international
character sets and their representation in the filesystem).
>> So if your files in the filesystem only have an unique-id as a name, then you
>> also have a download script, and this download script can also show the users a
>> nice page with the list of files that they can download, right ? and the links
>> that are shown are each in turn a call to your download script, to download the
>> file that they choose, right ?
I would be tempted to ask what was your original problem then ?
>> And another question : how many files maximum can end up being stored in that
>> upload/download directory ?
> generally I do it by size. so a user can upload say 5 gigs worth of files and the upload script will query the database for a sum of filesizes of all of that users files to see it they still have space. This is where I am really exited about the prospect of doing this in mod_perl. in the past, if you were uploading a 1 gig file but did not have space on your account you would not get an error until the entire file is uploaded. I am thinking with the way mod_perl works I can query the db for that user before the entire request (upload) is made and throw an error immediately (though I could be wrong)
> I guess another thought I need is not only the size of files but the actual number of files...I suppose there is a choking point when there are just too many files in one directory.
Yes, that is why is was asking, because it is also something that many people ignore at
first. It used to be worse, but even with current OS'es, there is still a limit as to how
many files can be in a directory, before any access to it becomes extremely sluggish. I'd
say that a limit of 2000 files or so is certainly not to be crossed. Under Windows, I
would say much less, or else even viewing a directory with Explorer becomes painful.
Since you are giving each file a unique id however, you can decide to split the files into
a subdirectory structure based on different substrings within that id, to split your files
into a reasonable amount of separate (sub-)directories. You can decide on a 2-level,
3-level structure or more, depending on how many files you think you are going to have
over the next 5 years or so. Fortunately, disks are now large enough and filesystems smart
enough that you do not need to worry too quickly about space.
If you are planning to go into the hundreds of thousands of files, then you need a smarter
system, because otherwise you will have trouble keeping track, doing backups, splitting
these files logically for various purposes etc..
Now another question : apart from cgi-bin scripts which run much faster under mod_perl,
have you already tried writing mod_perl-based Apache modules ?
(because that's where the really interesting things start, with mod_perl).