CODEWITHSUNDEEP
amazon-web-servicesamazon-efs
linuxfever
linuxfever

Reputation: 3823

AWS EFS: Max number of files allowed in a folder

I am trying to find out the maximum number of files I can store in a folder when using the AWS EFS file system. The total number of files I am interested in is around 2.5 million, and each file's size is about 10KB - 20KB. I've found this relevant question which suggests I run df -i. Unfortunately, when I run it, I get:

Filesystem                                 Inodes IUsed  IFree IUse% Mounted on
udev                                        55824   303  55521    1% /dev
tmpfs                                       58849   467  58382    1% /run
/dev/nvme0n1p1                            1024000 75489 948511    8% /
tmpfs                                       58849     1  58848    1% /dev/shm
tmpfs                                       58849     3  58846    1% /run/lock
tmpfs                                       58849    18  58831    1% /sys/fs/cgroup
fs-XXXXXXXX.efs.XXXXXXXXX.amazonaws.com:/       0     0      0     - /mnt/efs
tmpfs                                       58849    10  58839    1% /run/user/1000

As you can see, the file system I am interested in (fs-XXXX...) mounted on /mnt/efs gives me zero for the number of Inodes and I'm not sure what to do next. The AWS documentation on the limits (here) doesn't specify such a limit. Any ideas would be much appreciated!

Upvotes: 6

Views: 3964

Answers (1)

Rodrigo Murillo
Rodrigo Murillo

Reputation: 13638

As you noted, I don't see any mention of file count limits in the AWS EFS Limits doc https://docs.aws.amazon.com/efs/latest/ug/limits.html

These limits guides are a good way to know about hard service limits that would affect any implementation. If it's not mentioned here, there likely is no such limit.

Also note, EFS implements NFSv4, which is a network file sharing protocol. There is some good discussion of why NFS itself would have no such inherent limit. See https://superuser.com/questions/700102/file-count-limit-for-nfs-mounted-drive and See also https://aws.amazon.com/efs/

Finally, here is a good discussion on file system limits How many files can I put in a directory?

I would assume that the file system back end on EFS has to be ext4. As noted above, an ext4 filesystem has no limit to the number of files in a directory.

Based on the above, I believe is there is no such limit in EFS. It should handle your file requirements.

Upvotes: 4

Related Questions