sparse_embeddings is never initialized when both points and boxes are None

[ShadiZaki]

sparse_embeddings is never initialized when both points and boxes are set to None, causing the following error when the forward method of the PromptEncoder module is called:

File "/opt/conda/lib/python3.10/site-packages/segment_anything_fast/predictor.py", line 155, in predict masks, iou_predictions, low_res_masks = self.predict_torch( File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/segment_anything_fast/predictor.py", line 223, in predict_torch sparse_embeddings, dense_embeddings = self.model.prompt_encoder( File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/segment_anything_fast/modeling/prompt_encoder.py", line 170, in forward if sparse_embeddings.is_nested: UnboundLocalError: local variable 'sparse_embeddings' referenced before assignment

[cpuhrsch]

Hello @ShadiZaki - yes we only specialize on a subset of the API as needed for the experiments. It should be possible to extend the code to work on more, but we've started work on making SAM2 faster in https://github.com/pytorch/ao/tree/1d75c8fb46c58ac1f6ed641f93ba6a0ca78b33e8/examples/sam2_amg_server