Feature suggestion: labels for VMs and disks (and networks)

[pavelanni]

Description

It would be helpful to have labels added to VM instances and disks. Example uses:

• Add roles to servers to use when creating multi-node clusters. E.g., for Kubernetes clusters, we could have control-plane and worker nodes with appropriate labels
• Add creation time and delete-after time to be able to delete VM instances and disks automatically

... and others, similar to what cloud providers do.

It could be implemented as a JSON file in the instance directory or as a BoltDB file, which supports the locking mechanism (using https://github.com/etcd-io/bbolt, of course).

To show the instance (or disk) labels, you would run limactl list <instance> --show-labels.

Labels could be added to the YAML config file before starting the instance or using limactl edit. Also, they could be added using the --add-labels flag with limactl create or limactl start.

Appreciate your feedback.

[jandubois]

I would want to see some actual use cases where having labels would make a significant difference.

Right now you can use the param mechanism to add any custom metadata to your instance you want.

For disks the only thing I can think of would be a naming scheme. Again, I would need to see an actual use case to figure out if adding labels would be the best way to support it.

I can see that adding the instance creation date to the limactl ls --json output could be useful. I would set it to the creation time of the $LIME_HOME/$INSTANCE/lima-version file, which will never be modified by Lima once it has been created.

[pavelanni]

Right now you can use the param mechanism to add any custom metadata to your instance you want.

I tried adding param: with role: controlplane to my YAML and got an error message that it's not used by anything in that YAML. So, it looks like I can't add arbitrary labels to an instance to use them outside of it. I understand that the param mechanism is used within that instance configuration YAML file in other fields such as provision, portForward, copyToHost, etc. Please correct me if I'm wrong.

I would want to see some actual use cases where having labels would make a significant difference.

Here is what I do with Lima. I create several instances and install a K3s cluster into them. To install the cluster, I use Ansible playbooks. I created a wrapper tool that takes a config file in YAML that contains instance configs and their roles, creates YAML files for each instance, starts them, and, at the same time, creates an Ansible inventory file with instance roles in it. So Ansible knows where to install the control plane parts and where to install workers. So, without my wrapper, Ansible wouldn't know instance roles. If I had labels, I could simply poll my instances, figure out their roles and IPs, and create an Ansible inventory for the instances that are already running. Even more, I could figure out which cluster they belong to in case I have more than one cluster. Currently, I use instance names for that, for example, adding -cp to the name of the control plane node. Not very robust.
There are other uses for labels, similar to how we use labels in cloud providers or Kubernetes clusters. You could group servers; you could find out something about the server without ssh-ing into it and even without starting it. Some might want to run something like "start all configured instances with this label."

Let's do this: I'll add it to my wrapper tool and show what I have in mind.

Thanks for your quick answer!

[jandubois]

I tried adding param: with role: controlplane to my YAML and got an error message that it's not used by anything in that YAML. So, it looks like I can't add arbitrary labels to an instance to use them outside of it

Yes, that is currently true. I always felt that the restriction that all param values must be used in the template expansions is too strict, and think we may eventually get rid of it.

You can also use env to set labels via /etc/environment:

❯ limactl start --tty=false --set '.env.role="controlplane"'
…
❯ limactl shell default printenv role
controlplane
❯ lima printenv role
controlplane
❯ limactl ls default --json | jq -r .config.env.role
controlplane

The last command will work even when the instance is not running.

[jandubois]

Some might want to run something like "start all configured instances with this label."

With the env mechanism I showed above you could start all your controlplane instances like this:

for inst in $(limactl ls --json | jq -r 'select(.config.env.role == "controlplane").name'); do
    limactl start $inst
done

Use 'select(.config.env.role == "controlplane" and .status == "Stopped").name') to only start instances that are currently stopped.

[pavelanni]

Great, thanks! Let me try that.

[jandubois]

@pavelanni Can this issue be closed?

[pavelanni]

Yes, thank you for the idea with env vars!

[AkihiroSuda]

$ limactl create --name foo1 --set '.param.category="foo"' 
? Creating an instance "foo1" Proceed with the current configuration
FATA[0002] field `param` key "category" is not used in any provision, probe, copyToHost, or portForward

😞

Do we want to use env rather than param to emulate labels?

[AkihiroSuda]

Having proper "labels" might be still useful if we want to support them in non-VM objects such as disks (as suggested by OP), networks, and other possible objects in future

[jandubois]

Do we want to use env rather than param to emulate labels?

I think conceptually params are closer to labels, and they are not automatically exposed to all processes running inside the guest.

Having proper "labels" might be still useful if we want to support them in non-VM objects such as disks (as suggested by OP), networks, and other possible objects in future

Yes, in that case having one kind of metadata that applies to all kinds of objects would be preferable.

Still feels a bit like a feature in search of a use-case to me, but I don't have strong objections; I just want us to consider if we really need 3 somewhat overlapping mechanisms.

[AkihiroSuda]

Can we just rename "params" to "labels", if almost nobody has started to use params yet?