Update README.md
Browse filesAdd reference to original repository.
Change model_name and revision to point to the updated model weights.
README.md
CHANGED
|
@@ -9,19 +9,18 @@ datasets:
|
|
| 9 |
tags:
|
| 10 |
- mosaicML
|
| 11 |
- sharded
|
| 12 |
-
-
|
| 13 |
---
|
| 14 |
|
| 15 |
-
# mpt-7b-
|
| 16 |
|
| 17 |
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
|
|
|
| 21 |
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
Please refer to the previously linked repo for details on usage/implementation/etc. This model was downloaded from the original repo under Apache-2.0 and is redistributed under the same license.
|
| 25 |
|
| 26 |
|
| 27 |
## Basic Usage
|
|
@@ -40,12 +39,12 @@ Load the model:
|
|
| 40 |
import torch
|
| 41 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 42 |
|
| 43 |
-
model_name = '
|
| 44 |
model = AutoModelForCausalLM.from_pretrained(
|
| 45 |
model_name,
|
| 46 |
torch_dtype=torch.bfloat16,
|
| 47 |
trust_remote_code=True,
|
| 48 |
-
revision='
|
| 49 |
device_map='auto',
|
| 50 |
load_in_8bit=False, # install bitsandbytes then set to true for 8-bit
|
| 51 |
)
|
|
|
|
| 9 |
tags:
|
| 10 |
- mosaicML
|
| 11 |
- sharded
|
| 12 |
+
- instruct
|
| 13 |
---
|
| 14 |
|
| 15 |
+
# mpt-7b-instruct: sharded
|
| 16 |
|
| 17 |
|
| 18 |
+
This is a version of the [mpt-7b-instruct](https://huggingface.co/mosaicml/mpt-7b-instruct) model, sharded to 2 GB chunks for low-RAM loading (i.e. Colab).
|
| 19 |
+
The weights are stored in `bfloat16` so in theory you can run this on CPU, though it may take forever.
|
| 20 |
+
Original code and credits go to [mpt-7b-storywriter-sharded](https://huggingface.co/ethzanalytics/mpt-7b-storywriter-sharded).
|
| 21 |
+
See the [community discussion](https://huggingface.co/ethzanalytics/mpt-7b-storywriter-sharded/discussions/2) on how to replicate this.
|
| 22 |
|
| 23 |
+
Please refer to the previously linked repo for details on usage/implementation/etc. This model was downloaded from the original repo under Apache-2.0 and is redistributed under the same license.
|
|
|
|
|
|
|
| 24 |
|
| 25 |
|
| 26 |
## Basic Usage
|
|
|
|
| 39 |
import torch
|
| 40 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 41 |
|
| 42 |
+
model_name = 'jprafael/mpt-7b-instruct-sharded'
|
| 43 |
model = AutoModelForCausalLM.from_pretrained(
|
| 44 |
model_name,
|
| 45 |
torch_dtype=torch.bfloat16,
|
| 46 |
trust_remote_code=True,
|
| 47 |
+
revision='8d8911ad980f48f8a791e5f5876dea891dcbc064', # optional, but a good idea
|
| 48 |
device_map='auto',
|
| 49 |
load_in_8bit=False, # install bitsandbytes then set to true for 8-bit
|
| 50 |
)
|