AI Mask Prototype

Prototype based on Meta's Segment Anything Model implemented in pure C/C++ using GGML tensor library for machine learning.

Description

The prototype currently supports only the ViT-B SAM model checkpoint.

Build

Clone repository

git clone [email protected]:andriiryzhkov/ai_mask_prototype.git
cd ai_mask_prototype
git submodule update --init --recursive

Install python dependencies in a virtual environment

poetry install

Download PTH model

curl --create-dirs --output-dir weights -O https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth

Convert PTH model to ggml

poetry run python scripts/convert-pth-to-ggml.py weights/sam_vit_b_01ec64.pth checkpoints/ 1

Build

./build.sh

or

mkdir build && cd build
cmake -G Ninja ..
cmake --build . --config Release -j 8

Run

Run command line inference

./build/bin/sam_cli -t 12 -i ./images/in/example1.jpg -o ./images/out/example1 -p "2070, 1170, 1" -m ./weights/sam_vit_b-ggml-model-f16.bin 
./build/bin/sam_cli -t 12 -i ./images/in/example2.jpg -o ./images/out/example2 -p "650, 700, 1" -m ./weights/sam_vit_b-ggml-model-f16.bin 

or

./build/bin/sam_cli.exe -t 12 -i ./images/in/example1.jpg -o ./images/out/example1 -p "2070, 1170, 1" -m ./weights/sam_vit_b-ggml-model-f16.bin 
./build/bin/sam_cli.exe -t 12 -i ./images/in/example2.jpg -o ./images/out/example2 -p "650, 700, 1" -m ./weights/sam_vit_b-ggml-model-f16.bin 

GTK3 application

cd ./build/bin && ./sam_gui

or

cd ./build/bin && ./sam_gui.exe

Sample images

1. Photo by Aaron Doucett on Unsplash
2. Photo by Anoir Chafik on Unsplash

License

GPL3

References

• ggml
• SAM
• SAM demo
• sam.cpp