Segment-Anything-Model-2: Optimized for Qualcomm Devices

SAM 2, the successor to Meta's Segment Anything Model (SAM), is a cutting-edge tool designed for comprehensive object segmentation in both images and videos. It excels in handling complex visual data through a unified, promptable model architecture that supports real-time processing and zero-shot generalization.

This is based on the implementation of Segment-Anything-Model-2 found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime	Precision	Chipset	SDK Versions	Download
ONNX	float	Universal	QAIRT 2.42, ONNX Runtime 1.24.1	Download
ONNX	w8a8	Universal	QAIRT 2.42, ONNX Runtime 1.24.1	Download
QNN_DLC	float	Universal	QAIRT 2.43	Download
QNN_DLC	w8a8	Universal	QAIRT 2.43	Download
TFLITE	w8a8	Universal	QAIRT 2.43, TFLite 2.17.0	Download

For more device-specific assets and performance metrics, visit Segment-Anything-Model-2 on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

Custom weights (e.g., fine-tuned checkpoints)
Custom input shapes
Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for Segment-Anything-Model-2 on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.semantic_segmentation

Model Stats:

Model checkpoint: sam2.1_hiera_t
Input resolution: 720p (720x1280)
Number of parameters (SAM2Encoder): 33.5M
Model size (SAM2Encoder) (float): 128 MB
Number of parameters (SAM2Decoder): 6.22M
Model size (SAM2Decoder) (float): 23.7 MB

Performance Summary

Model	Runtime	Precision	Chipset	Inference Time (ms)	Peak Memory Range (MB)	Primary Compute Unit
SAM2Decoder	ONNX	float	Snapdragon® X2 Elite	3.276 ms	15 - 15 MB	NPU
SAM2Decoder	ONNX	float	Snapdragon® 8 Gen 3 Mobile	4.71 ms	16 - 333 MB	NPU
SAM2Decoder	ONNX	float	Qualcomm® QCS8550 (Proxy)	6.772 ms	16 - 25 MB	NPU
SAM2Decoder	ONNX	float	Snapdragon® 8 Elite For Galaxy Mobile	3.472 ms	1 - 232 MB	NPU
SAM2Decoder	ONNX	float	Snapdragon® 8 Elite Gen 5 Mobile	3.146 ms	17 - 242 MB	NPU
SAM2Decoder	ONNX	w8a8	Snapdragon® X2 Elite	5.431 ms	4 - 4 MB	NPU
SAM2Decoder	ONNX	w8a8	Snapdragon® X Elite	12.473 ms	4 - 4 MB	NPU
SAM2Decoder	ONNX	w8a8	Snapdragon® 8 Gen 3 Mobile	9.18 ms	4 - 268 MB	NPU
SAM2Decoder	ONNX	w8a8	Qualcomm® QCS6490	130.615 ms	51 - 60 MB	CPU
SAM2Decoder	ONNX	w8a8	Qualcomm® QCS8550 (Proxy)	11.935 ms	2 - 9 MB	NPU
SAM2Decoder	ONNX	w8a8	Qualcomm® QCM6690	58.195 ms	48 - 69 MB	CPU
SAM2Decoder	ONNX	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	6.758 ms	2 - 240 MB	NPU
SAM2Decoder	ONNX	w8a8	Snapdragon® 7 Gen 4 Mobile	50.849 ms	50 - 67 MB	CPU
SAM2Decoder	ONNX	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	5.289 ms	4 - 264 MB	NPU
SAM2Decoder	QNN_DLC	float	Snapdragon® X2 Elite	3.049 ms	16 - 16 MB	NPU
SAM2Decoder	QNN_DLC	float	Snapdragon® X Elite	5.448 ms	16 - 16 MB	NPU
SAM2Decoder	QNN_DLC	float	Snapdragon® 8 Gen 3 Mobile	3.652 ms	3 - 232 MB	NPU
SAM2Decoder	QNN_DLC	float	Qualcomm® QCS8550 (Proxy)	5.241 ms	16 - 55 MB	NPU
SAM2Decoder	QNN_DLC	float	Qualcomm® QCS8450 (Proxy)	8.865 ms	4 - 225 MB	NPU
SAM2Decoder	QNN_DLC	float	Qualcomm® SA8295P	7.997 ms	0 - 214 MB	NPU
SAM2Decoder	QNN_DLC	float	Snapdragon® 8 Elite For Galaxy Mobile	2.913 ms	4 - 213 MB	NPU
SAM2Decoder	QNN_DLC	float	Snapdragon® 8 Elite Gen 5 Mobile	2.525 ms	4 - 201 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Snapdragon® X2 Elite	1.287 ms	4 - 4 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Snapdragon® X Elite	2.226 ms	4 - 4 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Snapdragon® 8 Gen 3 Mobile	1.294 ms	0 - 102 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® QCS8275 (Proxy)	3.868 ms	4 - 82 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® QCS8550 (Proxy)	1.916 ms	4 - 6 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® SA8775P	2.543 ms	4 - 84 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® QCM6690	9.886 ms	4 - 265 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® QCS8450 (Proxy)	2.703 ms	0 - 102 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® SA7255P	3.868 ms	4 - 82 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® SA8295P	2.961 ms	4 - 81 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	1.03 ms	1 - 83 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Snapdragon® 7 Gen 4 Mobile	2.605 ms	4 - 209 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	0.902 ms	1 - 106 MB	NPU
SAM2Decoder	TFLITE	float	Snapdragon® 8 Gen 3 Mobile	3.754 ms	0 - 239 MB	NPU
SAM2Decoder	TFLITE	float	Qualcomm® QCS8275 (Proxy)	12.854 ms	0 - 210 MB	NPU
SAM2Decoder	TFLITE	float	Qualcomm® QCS8550 (Proxy)	5.319 ms	0 - 7 MB	NPU
SAM2Decoder	TFLITE	float	Qualcomm® SA8775P	6.761 ms	0 - 210 MB	NPU
SAM2Decoder	TFLITE	float	Qualcomm® SA7255P	12.854 ms	0 - 210 MB	NPU
SAM2Decoder	TFLITE	float	Qualcomm® SA8295P	7.998 ms	0 - 221 MB	NPU
SAM2Decoder	TFLITE	float	Snapdragon® 8 Elite Gen 5 Mobile	2.526 ms	0 - 206 MB	NPU
SAM2Decoder	TFLITE	w8a8	Snapdragon® 8 Gen 3 Mobile	9.253 ms	0 - 260 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® QCS8275 (Proxy)	20.902 ms	0 - 210 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® QCS8550 (Proxy)	12.179 ms	0 - 22 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® SA8775P	13.809 ms	0 - 213 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® QCM6690	46.634 ms	14 - 208 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® QCS8450 (Proxy)	15.662 ms	0 - 241 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® SA7255P	20.902 ms	0 - 210 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® SA8295P	14.232 ms	0 - 222 MB	NPU
SAM2Decoder	TFLITE	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	6.926 ms	0 - 226 MB	NPU
SAM2Decoder	TFLITE	w8a8	Snapdragon® 7 Gen 4 Mobile	15.913 ms	8 - 204 MB	NPU
SAM2Decoder	TFLITE	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	5.416 ms	0 - 239 MB	NPU
SAM2Encoder	ONNX	float	Snapdragon® X2 Elite	99.568 ms	71 - 71 MB	NPU
SAM2Encoder	ONNX	float	Snapdragon® 8 Gen 3 Mobile	145.472 ms	45 - 3853 MB	NPU
SAM2Encoder	ONNX	float	Qualcomm® QCS8550 (Proxy)	196.682 ms	0 - 88 MB	NPU
SAM2Encoder	ONNX	float	Snapdragon® 8 Elite For Galaxy Mobile	111.964 ms	33 - 1668 MB	NPU
SAM2Encoder	ONNX	float	Snapdragon® 8 Elite Gen 5 Mobile	92.277 ms	44 - 2275 MB	NPU
SAM2Encoder	ONNX	w8a8	Snapdragon® X2 Elite	42.391 ms	51 - 51 MB	NPU
SAM2Encoder	ONNX	w8a8	Snapdragon® X Elite	89.486 ms	47 - 47 MB	NPU
SAM2Encoder	ONNX	w8a8	Snapdragon® 8 Gen 3 Mobile	60.75 ms	11 - 1996 MB	NPU
SAM2Encoder	ONNX	w8a8	Qualcomm® QCS6490	2191.807 ms	766 - 790 MB	CPU
SAM2Encoder	ONNX	w8a8	Qualcomm® QCS8550 (Proxy)	88.469 ms	0 - 59 MB	NPU
SAM2Encoder	ONNX	w8a8	Qualcomm® QCM6690	1332.829 ms	659 - 673 MB	CPU
SAM2Encoder	ONNX	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	59.235 ms	9 - 1792 MB	NPU
SAM2Encoder	ONNX	w8a8	Snapdragon® 7 Gen 4 Mobile	1257.884 ms	648 - 663 MB	CPU
SAM2Encoder	ONNX	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	39.538 ms	12 - 1882 MB	NPU
SAM2Encoder	QNN_DLC	float	Snapdragon® X2 Elite	96.624 ms	12 - 12 MB	NPU
SAM2Encoder	QNN_DLC	float	Snapdragon® X Elite	201.908 ms	12 - 12 MB	NPU
SAM2Encoder	QNN_DLC	float	Snapdragon® 8 Gen 3 Mobile	146.731 ms	12 - 2001 MB	NPU
SAM2Encoder	QNN_DLC	float	Qualcomm® QCS8550 (Proxy)	204.181 ms	12 - 25 MB	NPU
SAM2Encoder	QNN_DLC	float	Qualcomm® QCS8450 (Proxy)	397.56 ms	13 - 1952 MB	NPU
SAM2Encoder	QNN_DLC	float	Qualcomm® SA8295P	303.317 ms	0 - 1619 MB	NPU
SAM2Encoder	QNN_DLC	float	Snapdragon® 8 Elite For Galaxy Mobile	113.902 ms	12 - 1673 MB	NPU
SAM2Encoder	QNN_DLC	float	Snapdragon® 8 Elite Gen 5 Mobile	89.4 ms	6 - 1645 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Snapdragon® X2 Elite	81.192 ms	3 - 3 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Snapdragon® X Elite	85.347 ms	3 - 3 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Snapdragon® 8 Gen 3 Mobile	57.192 ms	3 - 2046 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® QCS8275 (Proxy)	149.071 ms	3 - 1666 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® QCS8550 (Proxy)	82.989 ms	3 - 7 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® SA8775P	85.343 ms	3 - 1666 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® QCM6690	1626.356 ms	3 - 1891 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® QCS8450 (Proxy)	141.404 ms	3 - 2022 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® SA7255P	149.071 ms	3 - 1666 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® SA8295P	95.021 ms	3 - 1670 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	42.151 ms	3 - 1652 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Snapdragon® 7 Gen 4 Mobile	110.451 ms	3 - 1790 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	83.639 ms	3 - 1690 MB	NPU
SAM2Encoder	TFLITE	float	Snapdragon® 8 Gen 3 Mobile	146.865 ms	15 - 2040 MB	NPU
SAM2Encoder	TFLITE	float	Qualcomm® QCS8275 (Proxy)	457.938 ms	16 - 1692 MB	NPU
SAM2Encoder	TFLITE	float	Qualcomm® QCS8550 (Proxy)	203.986 ms	0 - 4 MB	NPU
SAM2Encoder	TFLITE	float	Qualcomm® SA8775P	910.158 ms	5 - 1681 MB	NPU
SAM2Encoder	TFLITE	float	Qualcomm® SA7255P	457.938 ms	16 - 1692 MB	NPU
SAM2Encoder	TFLITE	float	Qualcomm® SA8295P	295.729 ms	16 - 1660 MB	NPU
SAM2Encoder	TFLITE	float	Snapdragon® 8 Elite Gen 5 Mobile	95.65 ms	16 - 2353 MB	NPU
SAM2Encoder	TFLITE	w8a8	Snapdragon® 8 Gen 3 Mobile	61.048 ms	14 - 1996 MB	NPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® QCS8275 (Proxy)	150.292 ms	4 - 1740 MB	NPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® QCS8550 (Proxy)	88.305 ms	4 - 8 MB	NPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® SA8775P	88.47 ms	4 - 1739 MB	NPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® QCM6690	2324.338 ms	1 - 253 MB	CPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® QCS8450 (Proxy)	149.64 ms	4 - 2047 MB	NPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® SA7255P	150.292 ms	4 - 1740 MB	NPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® SA8295P	101.399 ms	4 - 1759 MB	NPU
SAM2Encoder	TFLITE	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	58.304 ms	4 - 1784 MB	NPU
SAM2Encoder	TFLITE	w8a8	Snapdragon® 7 Gen 4 Mobile	1270.122 ms	64 - 316 MB	CPU
SAM2Encoder	TFLITE	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	39.36 ms	4 - 1862 MB	NPU

License

The license for the original implementation of Segment-Anything-Model-2 can be found here.

References

Community

Join our AI Hub Slack community to collaborate, post questions and learn more about on-device AI.
For questions or feedback please reach out to us.

Downloads last month: -; Downloads are not tracked for this model. How to track

Paper for qualcomm/Segment-Anything-Model-2

SAM 2: Segment Anything in Images and Videos

Paper • 2408.00714 • Published Aug 1, 2024 • 120