Aug 24, 2011
tom

Storing files in a directory… are there limits?

Question

I am using CentOS 5 with Plesk 9 (64-bit), I am running a site where users will be uploading pictures. With a 64 bit os, are there any limits to how many files I can store? All I care about is performance, and serving up the files. I’d prefer not to have 4 directories deep of scattered files. However, I am hoping, that at some point I could have 200-300 thousand images.

Answer

If you are using ext3, I found this quote (warning: spanish speaking site)

“There is a limit of 32k (32768)
subdirectories in a single directory,
a limitation likely of only academic
interest, as many people don’t even
have that many files (though huge mail
servers may need to keep that in
mind). The ext2 inode specification
allows for over 100 trillion files to
reside in a single directory”

Further reading showed that ext3 doesn’t have a 32K limitation, which can be empirically proven with

a=0; i=1; while [ $a == 0 ]; do touch $i; a=$?; let i++; done

but it does have a 32K folder limit for folder, which can be tested with

a=0; i=1; while [ $a == 0 ]; do mkdir $i; a=$?; let i++; done

This (unfounded) claim says that

ReiserFS has no trouble at all with
hundreds of thousands of files in a
single directory. flabdablet – February 1, 2007

This question from sister site stackoverflow.com could help too.

In general:

  • There is a limit to the amount of directories,
  • You should keep your files/directories under 32K, but can go a lot further,
  • The file system you are using does matter.

Related posts:

  1. optimal folder structure for storing 100k files on a USB drive
  2. How many files can I have directly under a directory in ext3?
  3. Bash snippet to move all files in a directory into that directory
  4. Max total files in a directory in FreeBSD 6 ufs?
  5. Maximum Number of Files in a Single Directory for Netapp NFS mounts on Linux

Leave a comment