Repository navigation
RuntimeError: expected scalar type Float but found BFloat16 #31
Description
what i input :
ns-train gaussctrl --load-checkpoint unedited_models/bear/splatfacto/2024-12-20_113611/nerfstudio_models/step-000029999.ckpt --experiment-name bear --output-dir outputs --pipeline.datamanager.data data/bear --pipeline.edit_prompt "a photo of a polar bear in the forest" --pipeline.reverse_prompt "a photo of a bear statue in the forest" --pipeline.guidance_scale 5 --pipeline.chunk_size 3 --pipeline.langsam_obj 'bear' --viewer.quit-on-train-completion True
Loading pipeline components...: 100%|█████████████████████████████████████████████████████████████████| 7/7 [00:01<00:00, 5.32it/s]
Attempting to load checkpoint from: unedited_models/bear/splatfacto/2024-12-20_113611/nerfstudio_models/step-000029999.ckpt
Done loading Nerfstudio checkpoint from
unedited_models/bear/splatfacto/2024-12-20_113611/nerfstudio_models/step-000029999.ckpt
Rendering view 0
Traceback (most recent call last):
File "/root/data1/miniconda3/envs/gsctrl/bin/ns-train", line 8, in
sys.exit(entrypoint())
^^^^^^^^^^^^
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/nerfstudio/scripts/train.py", line 262, in entrypoint
main(
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/nerfstudio/scripts/train.py", line 247, in main
launch(
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/nerfstudio/scripts/train.py", line 189, in launch
main_func(local_rank=0, world_size=world_size, config=config)
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/nerfstudio/scripts/train.py", line 99, in train_loop
trainer.setup()
File "/root/data1/code/gaussctrl/gaussctrl/gc_trainer.py", line 76, in setup
self.pipeline.render_reverse()
File "/root/data1/code/gaussctrl/gaussctrl/gc_pipeline.py", line 134, in render_reverse
rendered_image = self._model.get_outputs_for_camera(current_cam)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/root/data1/code/gaussctrl/gaussctrl/gc_model.py", line 232, in get_outputs_for_camera
outs = self.get_outputs(camera.to(self.device))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/data1/code/gaussctrl/gaussctrl/gc_model.py", line 140, in get_outputs
self.xys, depths, self.radii, conics, num_tiles_hit, cov3d = project_gaussians( # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/gsplat/project_gaussians.py", line 59, in project_gaussians
return _ProjectGaussians.apply(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/torch/autograd/function.py", line 574, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/gsplat/project_gaussians.py", line 109, in forward
) = _C.project_gaussians_forward(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/gsplat/cuda/init.py", line 9, in call_cuda
return getattr(_C, name)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: expected scalar type Float but found BFloat16
How can I solve this problem? Thanks a lot!
Activity
jingwu2121 commentedon Feb 18, 2025
Hi, please try manually set the variable type to float16 in
like this
yusskkt commentedon Feb 19, 2025
Thank you very much for your reply! I changed the variable to float16 and tried to run it again, and then an error occurred.
change：
current_cam = self.datamanager.cameras[cam_idx].to(self.device).to(torch.float16)
error：
Attempting to load checkpoint from: unedited_models/bear/splatfacto/2024-12-20_113611/nerfstudio_models/step-000029999.ckpt
Done loading Nerfstudio checkpoint from
unedited_models/bear/splatfacto/2024-12-20_113611/nerfstudio_models/step-000029999.ckpt
Rendering view 0
Traceback (most recent call last):
File "/root/data1/miniconda3/envs/gsctrl/bin/ns-train", line 8, in
sys.exit(entrypoint())
^^^^^^^^^^^^
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/nerfstudio/scripts/train.py", line 262, in entrypoint
main(
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/nerfstudio/scripts/train.py", line 247, in main
launch(
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/nerfstudio/scripts/train.py", line 189, in launch
main_func(local_rank=0, world_size=world_size, config=config)
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/nerfstudio/scripts/train.py", line 99, in train_loop
trainer.setup()
File "/root/data1/code/gaussctrl/gaussctrl/gc_trainer.py", line 76, in setup
self.pipeline.render_reverse()
File "/root/data1/code/gaussctrl/gaussctrl/gc_pipeline.py", line 132, in render_reverse
current_cam = self.datamanager.cameras[cam_idx].to(self.device).to(torch.float16)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/nerfstudio/utils/tensor_dataclass.py", line 265, in to
return self._apply_fn_to_fields(lambda x: x.to(device))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/nerfstudio/utils/tensor_dataclass.py", line 310, in _apply_fn_to_fields
return dataclasses.replace(self_dc, **new_fields)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/dataclasses.py", line 1503, in replace
return obj.class(**changes)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/nerfstudio/cameras/cameras.py", line 152, in init
self.height = self._init_get_height_width(height, self.cy)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/data1/miniconda3/envs/gsctrl/lib/python3.11/site-packages/nerfstudio/cameras/cameras.py", line 245, in _init_get_height_width
assert not torch.is_floating_point(h_w), f"height and width tensor must be of type int, not: {h_w.dtype}"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: height and width tensor must be of type int, not: torch.float16
zengjl03 commentedon Mar 5, 2025
I have same problem, could you tell me how to solve it? thanks!
PotatoBigRoom commentedon Mar 13, 2025
I have the same problem,could you please tell me how to solve it?
yusskkt commentedon Apr 2, 2025
sorry,i can't solve it
manolisfosteris commentedon Mar 10, 2026
Hey, I had the same problem and my understanding is that this is caused by a dtype mismatch between the splatfacto checkpoint (which stores tensors in BFloat16) and gsplat's project_gaussians(), which expects Float32.
Two changes in gc_model.py fix it:
1. Explicitly cast all tensors to float32 before calling project_gaussians():
means_crop = means_crop.float()
scales_crop = scales_crop.float()
quats_crop = quats_crop.float()
opacities_crop = opacities_crop.float()
colors_crop = colors_crop.float()
viewmat = viewmat.float()
projmat = projmat.float()
2. Wrap the get_outputs() call in get_render_outputs() with autocast(enabled=False) to prevent AMP from re-introducing BFloat16:
with torch.cuda.amp.autocast(enabled=False):
outs = self.get_outputs(camera.to(self.device))
Without the second fix, PyTorch's Automatic Mixed Precision can silently cast the tensors back to BFloat16 inside get_outputs(), undoing the explicit casts.
HuangZhongTian1009 commentedon Mar 18, 2026
Has everyone solved this problem，“RuntimeError: expected scalar type Float but found BFloat16 ”
zengjl03 commentedon Mar 18, 2026
#31 (comment) you could have a try to solve the problem using this method, i conduct perfectly before.
HuangZhongTian1009 commentedon Mar 18, 2026
After completing the conversion of the first variable, an error is still reported.
Does the second one also need to be modified?
zengjl03 commentedon Mar 18, 2026
maybe you can convert scalar type to Float manually everywhere it's mentioned. It's also important thing that your machine could support BFloat16. There are several months since i taken my attention to this work, so it's hard for me to give more details for you. Best wishes.