ioc rename with stricter rules than create

[urosgruber]

While doing ioc rename ioc/consul-agent-0 ioc/consul-client-0
I got error message
Invalid jail name: Names have to begin and end with an alphanumeric character
Looks like mismatch in validation

[gronke]

@urosgruber it is not possible rename jails across iocage root datasets, which is why it must not be defined in the target name. Anyhow there are a few learnings from your experience we can make:

• The error message needs to be specific and should mention whether the source or target name did not match the requirements
• Renaming from a iocage root dataset to the same one should be accepted input
• The error message when renaming across iocage root datasets needs to be descriptive or
  • We can export and import the jail (without the compression step) to the target dataset. This would imply that the release gets fetched if it does not exist on the remote dataset or we develop a mechanism that transfers the existing one before importing the jail

[urosgruber]

Not sure if this is related but I tried without dataset and it failed with device busy message, I checked mounts and since jail (template) was stopped I thought ok I'll unmount. After that I get this on every command I use.

Traceback (most recent call last):
  File "/usr/local/bin/ioc", line 10, in <module>
    sys.dd:exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/ioc/destroy.py", line 101, in cli
    logger=logger
  File "/usr/local/lib/python3.6/site-packages/iocage/Jails.py", line 105, in __iter__
    for jail in iterator:
  File "/usr/local/lib/python3.6/site-packages/iocage/ListableResource.py", line 105, in __iter__
    resource = self._get_resource_from_dataset(child_dataset)
  File "/usr/local/lib/python3.6/site-packages/iocage/ListableResource.py", line 131, in _get_resource_from_dataset
    return self._create_resource_instance(dataset)
  File "/usr/local/lib/python3.6/site-packages/iocage/Jails.py", line 86, in _create_resource_instance
    zfs=self.zfs
  File "/usr/local/lib/python3.6/site-packages/iocage/Jail.py", line 367, in __init__
    self.config.read(data=self.read_config(), skip_on_error=True)
  File "/usr/local/lib/python3.6/site-packages/iocage/Resource.py", line 332, in read_config
    data = self.config_handler.read()  # type: typing.Dict[str, typing.Any]
  File "/usr/local/lib/python3.6/site-packages/iocage/Jail.py", line 2045, in __getattribute__
    return object.__getattribute__(self, key)
  File "/usr/local/lib/python3.6/site-packages/iocage/Resource.py", line 338, in config_handler
    handler = object.__getattribute__(self, f"config_{self.config_type}")
  File "/usr/local/lib/python3.6/site-packages/iocage/Jail.py", line 2045, in __getattribute__
    return object.__getattribute__(self, key)
  File "/usr/local/lib/python3.6/site-packages/iocage/Resource.py", line 247, in config_type
    self._config_type = self._detect_config_type()
  File "/usr/local/lib/python3.6/site-packages/iocage/Resource.py", line 260, in _detect_config_type
    if self.config_json.exists is True:
  File "/usr/local/lib/python3.6/site-packages/iocage/Config/Prototype.py", line 124, in exists
    return os.path.isfile(self.file)
  File "/usr/local/lib/python3.6/site-packages/iocage/Config/Dataset.py", line 62, in file
    return str(os.path.join(self.dataset.mountpoint, self._file))
  File "/usr/local/lib/python3.6/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

So ioc is unusable at that stage. After mounting dataset back with zfs mount it started to work again. Maybe everything around this and rename is related to the fact I'm trying to rename template jail from which I already created new jails.

[gronke]

iocage does not attempt to mount your datasets if they are unmounted, which explains the TypeError: expected str, bytes or os.PathLike object, not NoneType.

It should be possible though to rename a template, even though datasets were cloned from the templates root dataset (deletion does not work though). @urosgruber you could try to manually rename the template - if you succeed, please let me know which commands you executed to do so, so that we can compare the steps to the renaming feature of libiocage.

[urosgruber]

Here are the steps I did with the output

root@:/ioc/jails # ioc stop consul-client-0
root@:/ioc/jails # zfs rename sys/ioc/jails/consul-agent-template sys/ioc/jails/consul-client-template
cannot unmount '/ioc/jails/consul-client-0/root': Device busy

root@:/ioc/jails # zfs unmount sys/ioc/jails/consul-agent-template
root@:/ioc/jails # zfs unmount sys/ioc/jails/consul-client-0
cannot unmount '/ioc/jails/consul-client-0/root': Device busy

root@:/ioc/jails # zfs unmount sys/ioc/jails/consul-client-0/root
cannot unmount '/ioc/jails/consul-client-0/root': Device busy

root@:/ioc/jails # cd
root@:~ # zfs unmount sys/ioc/jails/consul-client-0/root
root@:~ # zfs unmount sys/ioc/jails/consul-client-0
root@:~ # zfs rename sys/ioc/jails/consul-agent-template sys/ioc/jails/consul-client-template
root@:~ # zfs mount sys/ioc/jails/consul-client-template
root@:~ # zfs mount sys/ioc/jails/consul-client-template/root
root@:~ # zfs mount sys/ioc/jails/consul-client-0
root@:~ # zfs mount sys/ioc/jails/consul-client-0/root
root@:~ # ioc start ioc start ioc/consul-client-0

After that it looks like rename was ok, but as you can see from the commands this was not possible without stopping jail created from this template. I know that this might not be even good idea to rename the template but rather creating new one and then recreate all existing jails from that new template.

And note. Not sure why device was busy while I was inside /ioc/jails folder.

[gronke]

The fact that consul-client-0 was running should not affect your ability to rename the template it was created from. The only relation they have is caused by the clone operation with sys/ioc/jails/consul-agent-template/root being the origin of sys/ioc/jails/consul-client-0/root.

I don't get why you were unable to zfs unmount sys/ioc/jails/consul-client-0/root while being in the /ioc/jails directory. Nevertheless no interaction (umount) with the consul-client-0 jail's datasets should be required to rename the template.

@problame this seems to be your area of expertise. Do you mind telling us what we did wrong?

[problame]

There shouldn't be a reason if (and as far as I can tell this is the case) /ioc/jails is not the mountpoint of any of the unmounted filesystems.
My only guess is that the stop operation returned too early (i.e. some process still running inside the dying jail for a few secs after ioc stop exited)

[gronke]

My only guess is that the stop operation returned too early (i.e. some process still running inside the dying jail for a few secs after ioc stop exited)

There are no operations pending after stopping a jail. @problame thanks for commenting. @urosgruber could you please further investigate and help me to reproduce the behavior?

[urosgruber]

@gronke will do. Just let me first find the issue with some networking to unblock my team. I'll do a fresh install in case there is some problem on current environment.

[igalic]

My only guess is that the stop operation returned too early (i.e. some process still running inside the dying jail for a few secs after ioc stop exited)

There are no operations pending after stopping a jail. @problame thanks for commenting.

and yet, we have to wait for zfs to settle after a stop in our regression tests, before we can delete the zfs dataset

[urosgruber]

@gronke ok here it is. So foo1 was created from bar-template There was no problem with me being inside /ioc/jails. But as you can see stop on all jails created from the template is inevitable. I think the only missing thing is to support heaving dataset in the name to be able to define which template to use and as you noted above to check if source and target datasets are the same.

root:/ioc/jails # ioc start foo1
Jail 'foo1' is already running
root:/ioc/jails # ioc rename bar-template moo-template
Traceback (most recent call last):
  File "/usr/local/bin/ioc", line 10, in <module>
    sys.dd:exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/ioc/rename.py", line 55, in cli
    print_function(ioc_jail.rename(name))
  File "/usr/local/lib/python3.6/site-packages/iocage/Jail.py", line 2091, in rename
    return list(JailGenerator.rename(self, *args, **kwargs))
  File "/usr/local/lib/python3.6/site-packages/iocage/Jail.py", line 1061, in rename
    raise e
  File "/usr/local/lib/python3.6/site-packages/iocage/Jail.py", line 1058, in rename
    raise event.error
  File "/usr/local/lib/python3.6/site-packages/iocage/Storage.py", line 123, in _rename_dataset
    dataset.rename(new_dataset_name)
  File "libzfs.pyx", line 2211, in libzfs.ZFSObject.rename
libzfs.ZFSException: Device busy
root:/ioc/jails # ioc stop foo1
[+] JailDestroy@ioc/foo1: OK [0.056s]
foo1 stopped
root:/ioc/jails # ioc rename bar-template moo-template
[+] JailRename@ioc/bar-template: OK [0.113s]
  [+] ZFSDatasetRename@sys/ioc/jails/bar-template: OK [0.029s]
  [+] ZFSSnapshotRename@sys/ioc/jails/moo-template: OK [0.035s]
  [+] JailFstabUpdate@ioc/moo-template: OK [0.048s]
root:/ioc/jails # ioc start foo1
[+] JailResolverConfig: OK [0.001s]
[+] JailLaunch@ioc/foo1: OK [0.179s]
foo1 running as JID 68

I also tried using export and import to a different dataset and it didn't work. But that is different issue I believe. I just wanted to have same template on both datasets instead of running provisioning script over blank template.

[gronke]

This snippet reproduces the issue without iocage being involved:

root@freebsd-11-2 /tmp # zfs create zroot/source
root@freebsd-11-2 /tmp # zfs snapshot zroot/source@snap
root@freebsd-11-2 /tmp # zfs clone zroot/source@snap zroot/target 
root@freebsd-11-2 /tmp # mount -t nullfs /zroot/target /tmp/foo
root@freebsd-11-2 /tmp # zfs rename zroot/source zroot/source2
cannot unmount '/zroot/target': Device busy

When a jail is created from a template it is cloned from the templates root dataset. As demonstrated above the NullFS mount of the target dataset prevents renaming the source dataset. Any ideas how to solve this?