Gossamer Forum
Home : Products : Others : Fileman :

Check upload file size before executing command

Quote Reply
Check upload file size before executing command
One big problem we have found with file uploads is that the program doesn't check file size against available free space before jumping right into an upload.

So for really big files, you might get halfway through an upload (15 minutes or more) before the browser craps out on you with a failure message of one kind or another (usually ERR_FREE_SPC, but we've seen other more generic errors).

If anyone has a hacked version of Commands.pm that does either a file size check against available free space, and/or a progress indicator, let me know. Otherwise I'm going to have to try to hack it myself...

Unless GT has any kind of release date scheduled for the next version. My client has basically invested about $650 in license and development costs for a product that they basically find useless since they deal with a lot of large uploads (video).

Thanks, john
Quote Reply
Re: [yawny] Check upload file size before executing command In reply to
I'd be interested to know how you'd like a server side script (fileman) to find out a client side file size.
Quote Reply
Re: [Paul] Check upload file size before executing command In reply to
Well, look, I'm not the application's programmer, I'm just explaining things from the client's perspective.

If, as GT mentioned in a previous post, a progress indicator is in fact planned for a future release, then they have in mind some way to roughly determine file size before beginning upload.

Looking at the code...ok, so it reads data from the remote file into a buffer variable 1024 bytes at a time, and each time checks the buffer against available free space. Each time it reads a new 1024 byte block it also writes to the OUTFILE handle on the server. Additionally, if you're in ASCII mode, it performs a linefeed substitution.

So...you could try to read the remote file quickly into a variable first (without writing, without the linefeed substitution, without the size comparison), then compare total file size against available space when done. Perhaps increasing the block size would speed up the while loop too.

Of course this would add significantly to overall execution time, since you're reading in the whole thing twice, but hopefully the quick read part could be, well, quick. There might also be Perl modules to help with this: e.g. the application could request stat-type information on the file from the remote operating system.

In our case the client is saying that if they have to wait a minute or two to calculate available space before beginning upload, they would accept that as a trade to avoid the 15-minute upload that aborts mid-way through.

Obviously if it takes 15 minutes to calculate file size then this is not a serious solution.

For smaller files it's not really an issue since the difference between two seconds and four seconds is not nearly as great as the difference between 15 minutes and 30 minutes.

This could also be an user-specific setting: "Check available space before uploading."

I'd be happy to hear about all the reasons why this is wrong-headed, I'm just trying to find a solution for my client's complaints. (So that I can get paid, for one thing.)

- john
Quote Reply
Re: [yawny] Check upload file size before executing command In reply to
Quote:
So...you could try to read the remote file quickly into a variable first

It's not going to be any more quicker than printing it to the file a bit at a time, you still have to wait for the whole file to upload before you know it's size.

Quote:
There might also be Perl modules to help with this: e.g. the application could request stat-type information on the file from the remote operating system.

It's not possible with perl. Perl is server side...you can't find the size of a file on a users computer with server side code.

It can however be done with javascript.

Last edited by:

Paul: May 31, 2003, 2:06 PM
Quote Reply
Re: [yawny] Check upload file size before executing command In reply to
Hello yawny,

Linux has file limits, so you may want to look at them before you blame fileman for the errors.

Code:


Users can have a quota or the OS can have a default file size max set.



[bob@hostmybutt]$ quota -v bob <enter>



Disk quotas for user bob (uid 111):
Filesystem blocks quota limit grace files quota limit grace
/dev/ab/111111 7601 25600 25600 990 0 0



Seems user bob has used 7,601 kb [7.6 Mb ] of his quota of 25,600 kb [25.6 Mb ] !



So bob has a limit right now of uploading a file to 17,999 kb [ 17.9 Mb ]



Does your system have a user limit ??



Linux also has a default or unlimited max file size depending on the OS version.



[bob@hostmybutt]$ ulimit -a <enter>



core file size (blocks) 0
data seg size (kbytes) unlimited
file size (blocks) unlimited
max locked memory (kbytes) unlimited
max memory size (kbytes) unlimited
open files 8192
pipe size (512 bytes) 8
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 7168
virtual memory (kbytes) unlimited


file size (blocks) unlimited or 25600 ???



Does your system have a default limit set ???


Hope this helps. Sorry about the groovy empty spaces.

cornball
Quote Reply
Re: [cornball] Check upload file size before executing command In reply to
No, it's not a Linux server. Thanks anyway tho :|
Quote Reply
Re: [yawny] Check upload file size before executing command In reply to
Hello yawny,

What OS and web Server are you running under then ?

I am just interested as there is a way around this I am sure but with guessing I will fail to guess the right answer.

Thanks

cornball perl -Tcw