Performance drop with Large block size(1MB) with SDCard

[iloop2020]

Hi,

I found a read performance drop with 1MB block size, after a number of new files are created.

I measured the performance drop, there are extra sector_read() call, especially on Block0 and Block1. Delete those newly created files don't recover the performance.

Thank you for advise.

[iloop2020]

Hi,

Here is the procdure to reproduce the issue:

Settings:
BLOCK_SIZE:1048576
BLOCK_COUNT:4095
CACHE_SIZE:1048576

Create 1024 file:

        for i in $(seq 1 1024);
        do
                dd if=/dev/random of=disk/test$i.img bs=16 count=1 oflag=dsync
        done

Will get stuck and the filesystem corruption.

Thank you for your help.

[geky]

Hi @iloop2020, thanks for creating an issue.

This is a known issue with littlefs, it scales pretty terribly with large block sizes ($O(n^2)$).

What is your block device? Some devices support a range of erase sizes, with varying names which can get a bit confusing.

metadata_max may also be useful for artificially limiting the size of metadata blocks to prevent a performance cliff:

littlefs/lfs.h

Lines 271 to 275 in d01280e

	// Optional upper limit on total space given to metadata pairs in bytes. On
	// devices with large blocks (e.g. 128kB) setting this to a low size (2-8kB)
	// can help bound the metadata compaction time. Must be <= block_size.
	// Defaults to block_size when zero.
	lfs_size_t metadata_max;

These issues may have a bit more info: #214, #502

[iloop2020]

Hi @geky ,

Thank you very much for your replay and support.

Block devices info:
We are using the SDCard as the block device, the sector szie of 512 bytes.

Question:
For our case with 1MB block size, what is the suggested metadata_max ?

Thank you.

[geky]

Hi @iloop2020,

I don't have very useful numbers, so it's hard to say without a bit of trial and error.

A good starting value might be 4KiB, and decreasing if you still notice performance issues.