Gossamer Forum
Home : General : Internet Technologies :

safe maximum number of files per directory

Quote Reply
safe maximum number of files per directory
I'm running Debian Linux, and I'm wondering what a reasonable maximum number of files is for a single directory. Does anyone have any experience with this? The best answer I've been able to find so far is that the filesystem won't allow more than 2^15 subdirectories in a single directory, but I can't find any information on limits for number of files.

Many thanks for any advice!

Fractured Atlas :: Liberate the Artist
Services: Healthcare, Fiscal Sponsorship, Marketing, Education, The Emerging Artists Fund
Quote Reply
Re: [hennagaijin] safe maximum number of files per directory In reply to
It really depends on what file system you're using. I don't remember any numbers, but with ext2/3, with a large number (a few thousand?) files in a directory, it will slow things down a lot. reiserfs on the other hand, can handle many more files in a directory at a faster speed.

Adrian
Quote Reply
Re: [hennagaijin] safe maximum number of files per directory In reply to
I wouldn't recomend more then 20k files in a dir on ext2/3. If you are on reiserfs, you can have a lot more though (2^31 files in a directory) while still getting decent performance.

Cheers,

Alex
--
Gossamer Threads Inc.
Quote Reply
Re: [Alex] safe maximum number of files per directory In reply to
Thanks Alex. Maybe this is a dumb question, but how do I find out which filesystem I have? Is there a command I can run from shell to reveal that info?

Fractured Atlas :: Liberate the Artist
Services: Healthcare, Fiscal Sponsorship, Marketing, Education, The Emerging Artists Fund
Quote Reply
Re: [hennagaijin] safe maximum number of files per directory In reply to
Try: `df -hT` to see the file types. If that doesn't work, try `cat /etc/fstab` to see what file system you are using (third column).

Cheers,

Alex
--
Gossamer Threads Inc.
Quote Reply
Re: [Alex] safe maximum number of files per directory In reply to
Looks like ext3 it is... So I'll keep it under 20k per directory, then (I'm experimenting with caching lots of dynamic content). Thanks for the tip!

Fractured Atlas :: Liberate the Artist
Services: Healthcare, Fiscal Sponsorship, Marketing, Education, The Emerging Artists Fund
Quote Reply
Re: [hennagaijin] safe maximum number of files per directory In reply to
You'll want it less than that as well if you are worried about performance. ext3 does linear scans when accessing files in a directory (so it slows down the more files you have in a single directory).

Cheers,

Alex
--
Gossamer Threads Inc.
Quote Reply
Re: [Alex] safe maximum number of files per directory In reply to
Yes, thank you, Alex...that will help with one of the websites I am maintaining, which will pose an issue when we 10,000's (and possibly 1,000,000's) of photos uploaded. I'll have to take a look at our architecture and re-evaluate how and where we store photos.

BTW: I could not obtain the file system info with the commands you recommended. It is a managed server running Linux RedHat 7.0. I'll have to ask our hosting company what the file system is. Thanks for the tip.
========================================
Buh Bye!

Cheers,
Me

Last edited by:

Stealth: Aug 31, 2003, 8:55 PM
Quote Reply
Re: [Stealth] safe maximum number of files per directory In reply to
You could also try:

cat /proc/mounts

and look at the third column which should tell you the file system type.

Cheers,

Alex
--
Gossamer Threads Inc.
Quote Reply
Re: [Stealth] safe maximum number of files per directory In reply to
The "-T" option should show you the filesystem you're using. For example:
Code:
Filesystem Type Size Used Avail Use% Mounted on
/dev/hda1 reiserfs 21G 5.2G 15G 26% /
/dev/hda2 reiserfs 36G 6.5G 29G 19% /home

cat /proc/mounts also give you similar information:
Code:
rootfs / rootfs rw 0 0
/dev/root / reiserfs rw,noatime,nodiratime 0 0
/dev/hda2 /home reiserfs rw,noatime,nodiratime 0 0

Adrian