Enhanced SAM2 Napari Plugin

Note

This is a work in progress as part of a live project within the SEAI (Software Engineering & AI) STP. If you come across this repo and are interested please contact Cameron Shand or Fiona Young.

A plugin for interactive use of SAM2 for segmentation and tracking.

---

This napari plugin was generated with copier using the napari-plugin-template.

Features/Why this SAM2 plugin?

• SAM2 improvements:
  • Low-memory mode allows easier use of SAM2 for longer videos/larger stacks by avoiding GPU memory accumulation issue
• No hard CUDA/GPU requirement (though recommended!)
• Intuitive usage, utilizing keybinds to speed up annotation
  • Includes proper resetting of model state when adding/removing prompts for existing objects
• Prompt import/export for working across sessions

Installation

While we include napari in the requirements, it is recommended to first install Napari in the best way for your system, for which the official documentation has some pointers.

You can install napari-sam2 from this repo via:

pip install git+https://github.com/FrancisCrickInstitute/napari_sam2.git

Crick Installation

If you're at the Crick, and want to use napari-sam2 (before it's been handily wrapped into a premade app), here's what you need to do.

1. Start an OnDemand Nemo Desktop session on the GPU partition
2. Create a conda environment and install what's needed (first time only)
   1. conda create -n napari_sam2 python=3.11
   2. conda activate napari_sam2
   3. conda install nvidia/label/cuda-12.6.0::cuda-toolkit
   4. pip install 'napari[all]'
   5. pip install git+https://github.com/FrancisCrickInstitute/napari_sam2.git
3. Load modules we need (every time)
   1. ml cuDNN/9.5.0.50-CUDA-12.6.0 VirtualGL
4. Run napari: vglrun napari

If you have any issues with the above, likely a Disk quota exceeded, you need to ensure that your .conda and .cache folders in your home directory are symlinked somewhere with more space. Ask HPC if you are unsure how to do this.

Any other issues, Crick users can email/Slack us or join the #sam2-napari-plugin Slack channel.

Usage

For guidance on best usage and further detail on features such as the "low-memory mode", please see the Usage Guide.

Existing To-Dos

• Add support for bounding box prompt input
• Add support for adding masks to guide prompts (low priority)
• Tutorial video on usage
• Completed documentation and usage guide

Contributing

Contributions are very welcome. Tests can be run with tox, please ensure the coverage at least stays the same before you submit a pull request.

License

Distributed under the terms of the MIT license, "napari-sam2" is free and open source software

Issues

If you encounter any problems, please file an issue along with a detailed description.