When I try to write the same file, I found the dir->off will increase without any decrease

[XiuChengZhao]

When I try to write the same file, I found the dir->off will increase

I only try to write the same content to the file, and found the dir->off continue to increase.

Does it is the normal phenomenon？

I think it do not need to increase the dir->off, because I only write the same content

my test code:

    for (int i = 0; i < 21; ++i) {
        int32 stringId[] = {22, 23, 24, 25, 26, 27, 29, 30, 31, 31, 32, 25, 22, 23, 24, 25, 26, 27, 29, 30, 31};
        char *setting_value = NULL;
        int setting_value_len = 0;    
        char *file_name = (char *)QCLI_MALLOC(16);
        #define TC_SETTINGS_FILE_SUFFIX1         ("_Setting.txt")
        lfs_file_t file;
        lfs_dir_t dir;
        memset(file_name, 0, 16);
        sprintf(file_name, "%03ld%s", stringId[i], TC_SETTINGS_FILE_SUFFIX1);

        switch (i)
        {
        case 0:
        case 2:
        case 3:
        case 6:
        case 12:
        case 15:
        case 18:
            setting_value = (char *)QCLI_MALLOC(1);
            setting_value[0] = 0xFF;
            setting_value_len = 1;
            break;
        case 1:
        case 13:
            setting_value = NULL;
            setting_value_len = 0;
            break;
        case 4:
        case 5:
        case 7:
        case 16:
        case 17:
        case 19:
        case 20:
            setting_value = (char *)QCLI_MALLOC(1);
            setting_value[0] = 0x01;
            setting_value_len = 1;
            break;
        case 8:
            setting_value = (char *)QCLI_MALLOC(8);
            setting_value[0] = 0x01;
            setting_value[1] = 0xdf;
            setting_value[2] = 0x06;
            setting_value[3] = 0x20;
            setting_value[4] = 0x58;
            setting_value[5] = 0x15;
            setting_value[6] = 0x20;
            setting_value[7] = 0x28;
            setting_value_len = 8;
            break;
        case 9:
        case 10:
            setting_value = (char *)QCLI_MALLOC(1);
            setting_value[0] = 0x31;
            setting_value_len = 1;
            break;
        case 11:
            setting_value = (char *)QCLI_MALLOC(1);
            setting_value[0] = 0x30;
            setting_value_len = 1;
            break;
        case 14:
            setting_value = (char *)QCLI_MALLOC(7);
            setting_value[0] = 0xff;
            setting_value[1] = 0x6e;
            setting_value[2] = 0x67;
            setting_value[3] = 0x2e;
            setting_value[4] = 0x74;
            setting_value[5] = 0x78;
            setting_value[6] = 0x74;
            setting_value_len = 7;
            break;

        default:
            break;
        }    

        if (setting_value != NULL) {
            FILESYSTEM_PRINTF("file_name:%s, setting_value[0]:%d, setting_value_len:%d\r\n", file_name, setting_value[0], setting_value_len);
        } else {
            FILESYSTEM_PRINTF("setting_value is NULL");
        }

        lfs_file_open(&lfs, &file, file_name, LFS_O_CREAT | LFS_O_RDWR);
        lfs_file_sync(&lfs, &file);
        
        FILESYSTEM_PRINTF("start writing data to file...");
        lfs_ssize_t write_size = lfs_file_write(&lfs, &file, setting_value, setting_value_len);
        FILESYSTEM_PRINTF("start closing the file...");
        lfs_file_close(&lfs, &file);

        FILESYSTEM_PRINTF("remove the file...");
        lfs_remove(&lfs, file_name);

        QCLI_FREE(file_name);
        file_name = NULL;

        if (NULL != setting_value) {
            QCLI_FREE(setting_value);
            setting_value = NULL;
        }
        setting_value_len = 0;

The impact of this is after write to the limit of block size, it will do compact action.
Actually, it is not expected. So Could you help confirm about that？

[geky]

Hi @XiuChengZhao, can confirm this is expected behavior.

dir->off is the current position of the log containing the file's metadata. Every time the file's metadata is updated, the new metadata is appended to the log, and old metadata is only removed when the block is full (the compaction operation you noticed).

This is how logs work, and is a consequence of the limitations of flash. In flash, writes are a two part operation: erase + program. Programs are relatively cheap, but erases are very expensive. So it's more efficient to append new metadata to a block than to rewrite the entire block. The downside is this moves more work into the compaction step, but it makes metadata updates more efficient on average.

This is a common in any filesystem targeting flags (SPIFFS, JFFS2, etc).