Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize Volume.refresh flow #169

Open
nirs opened this issue May 9, 2022 · 0 comments
Open

Optimize Volume.refresh flow #169

nirs opened this issue May 9, 2022 · 0 comments
Labels
enhancement Enhancing the system by adding new feature or improving performance or reliability storage virt

Comments

@nirs
Copy link
Member

nirs commented May 9, 2022

When finishing extend flow we refresh the volume using storage API: Volume.refresh.

Refreshing takes 25% of total extend time, and creates lots of I/O since lvm
must read entire vg metadata twice.

Volume.refresh does:

  • produce a storage domain, possibly blocking for long time if the storage domain cache is in the middle of a rescan, possibly because of unrelated storage domain.
  • produce a volume, validating that the volume exists by calling lvm.getLV()
  • call volume.refreshVolume()
  • call lvm.refreshLVs(), running "lvchange --refresh vg-name/lv-name"

In the logs we see that for every call we run "lvs" - reading entire vg metadata
from storage. The lvs command always succeeds with a warning:

WARNING: Cannot find matching striped segment for bafd0f16-9aba-4f9f-ba90-46d3b8a29157/b91914e5-8446-4ff8-a143-a3f37cf23188.

The warning is likely expected and can be suppressed, but we really should not run
lvs for every call.

All this flow is pointless since call this from running VM context. We know
that the logical volume exist and active, so we should really call only:

lvm.refreshLVs(self.sdUUID, [self.volUUID])

Volume.refresh() does nothing for file volume, so callers should not call this API
with file volumes.

Example log:

2022-05-10 00:43:32,953+0300 INFO  (mailbox-hsm/2) [vdsm.api] START refreshVolume(sdUUID='bafd0f16-9aba-4f9f-ba90-46d3b8a29157', spUUID='fa20ea36-c317-49a6-af6d-afaa5b1561df', imgUUID='a710d32b-eb03-4f19-8049-9803be251f85', volUUID='b91914e5-8446-4ff8-a143-a3f37cf23188') from=internal, task_id=62cbdc9d-9ecb-4607-b620-5e7c564465aa (api:48)

2022-05-10 00:43:33,034+0300 WARN  (mailbox-hsm/2) [storage.lvm] Command ['/usr/sbin/lvm', 'lvs', '--devices', '/dev/mapper/0QEMU_QEMU_HARDDISK_data-fc-01,/dev/mapper/0QEMU_QEMU_HARDDISK_data-fc-02,/dev/mapper/0QEMU_QEMU_HARDDISK_data-fc-03', '--config', 'devices {  preferred_names=["^/dev/mapper/"]  ignore_suspended_devices=1  write_cache_state=0  disable_after_error_count=3    hints="none"  obtain_device_list_from_udev=0 } global {  prioritise_write_locks=1  wait_for_locks=1  use_lvmpolld=1 } backup {  retain_min=50  retain_days=0 }', '--noheadings', '--units', 'b', '--nosuffix', '--separator', '|', '--ignoreskippedcluster', '-o', 'uuid,name,vg_name,attr,size,seg_start_pe,devices,tags', 'bafd0f16-9aba-4f9f-ba90-46d3b8a29157'] succeeded with warnings: ['  WARNING: Cannot find matching striped segment for bafd0f16-9aba-4f9f-ba90-46d3b8a29157/b91914e5-8446-4ff8-a143-a3f37cf23188.'] (lvm:358)

2022-05-10 00:43:33,035+0300 INFO  (mailbox-hsm/2) [storage.lvm] Refreshing LVs (vg=bafd0f16-9aba-4f9f-ba90-46d3b8a29157, lvs=('b91914e5-8446-4ff8-a143-a3f37cf23188',)) (lvm:1735)

2022-05-10 00:43:33,136+0300 INFO  (mailbox-hsm/2) [vdsm.api] FINISH refreshVolume return=None from=internal, task_id=62cbdc9d-9ecb-4607-b620-5e7c564465aa (api:54)

Fix:

  • Call lvm directly bypassing storage layer
@nirs nirs added enhancement Enhancing the system by adding new feature or improving performance or reliability storage virt labels May 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancing the system by adding new feature or improving performance or reliability storage virt
Projects
None yet
Development

No branches or pull requests

1 participant