I tried this simple test: to create one million files in an empty directory, using EXT2, FAT, NTFS, ReiserFS and PicoStorage.
- Linux Ext2/Ext3 (SUSE 9.0)
- after creating about 40,000 entries, the creation of new entries gradually slows down to a crawl. The creation of 40,000 entries takes about 1minute. I conclude that on ext2/ext3 it's impossible to create one million entries in a directory. (this is because the time for creating a new entry grows quadratically WRT the number of entries already existing in the directory).
- WindowXP FAT
- the creation of new entries seems to gradually slow down. After creating about 65,000 entries it becomes impossible to create new entries (perhaps some FAT limit?). The creation of the 64K entries took about 2 minutes. I conclude that on FAT it's impossible to create one million entries in a directory.
- WindowsXP NTFS
- It seems NTFS doesn't have the quadratic slowdown shown by Ext2 and FAT, but it's rather slow (very approximatively 1minute for 100,000 entries). Anyway I need more data about NTFS's performance, because on this test the computer had to be rebooted because of a hang-up caused by Windows Explorer looking at the big directory :).
- Linux ReiserFS (SUSE 8.0)
- the creation of one million entries took about 1minute. The conclussion is that ReiserFS supports well big directories, and I'm impressed with the performance.
- the creation of one million empty files takes about 15seconds when using compression, and about 10seconds whithout compression. The storage file (which contains everything, directory entries and inodes) has about 8MB with compression, and 42MB without compression. PicoStorage was about 6 times faster then ReiserFS.
I know this 'benchmark' is not exact enough. It can also be argued that it's comparing apples with oranges, because a classical filesystem has a whole different set of capabilities then PicoStorage. The point of these measurements is to show that PicoStorage offers a solution which, for some specific needs, is better then a using a classical filesystem.