Instructions to use modilify/Modilify-Mk2-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use modilify/Modilify-Mk2-mlx with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("modilify/Modilify-Mk2-mlx") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use modilify/Modilify-Mk2-mlx with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "modilify/Modilify-Mk2-mlx"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "modilify/Modilify-Mk2-mlx" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use modilify/Modilify-Mk2-mlx with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "modilify/Modilify-Mk2-mlx"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "modilify/Modilify-Mk2-mlx" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "modilify/Modilify-Mk2-mlx", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use modilify/Modilify-Mk2-mlx with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "modilify/Modilify-Mk2-mlx"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default modilify/Modilify-Mk2-mlx
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use modilify/Modilify-Mk2-mlx with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "modilify/Modilify-Mk2-mlx"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "modilify/Modilify-Mk2-mlx" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Configure OpenClaw
# Install OpenClaw:
npm install -g openclaw@latest# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
--auth-choice custom-api-key \
--custom-base-url http://127.0.0.1:8080/v1 \
--custom-model-id "modilify/Modilify-Mk2-mlx" \
--custom-provider-id mlx-lm \
--custom-compatibility openai \
--custom-text-input \
--accept-risk \
--skip-healthRun OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"Modilify Mk2 MLX
Modilify Mk2 MLX is a self-contained Apple Silicon inference release of Modilify Mk2 step 4400. The text-only DiffusionGemma 26B-A4B backbone revises a rolling 256-token canvas and commits a prefix at each denoising step when its confidence and entropy meet the commit policy. Working GDN2 memory follows canvas revisions. Persistent GDN2 memory updates when tokens are committed.
| Property | Value |
|---|---|
| Architecture | schema25 compact_gdn2_v2 |
| Weights | 16 Safetensors shards, about 48.23 GiB |
| Precision | 1,297 BF16 tensors and 26 FP32 tensors |
| Runtime | MLX Metal on Apple Silicon |
Run locally
Use Python 3.12 on Apple Silicon. Allow unified memory for the weights, prompt cache, rolling state and generation workspace.
cd ~/Modilify-Mk2-mlx
python -m pip install -r requirements.txt
python inference.py --prompt "Why is the sky blue?" --stream
The CLI loads the model and tokenizer from the script directory by default. Move the entire directory together to use it elsewhere. --model /path/to/Modilify-Mk2-mlx selects a local copy of this release. The loader verifies the manifest and all 16 shard checksums before generation. After dependency installation, loading and inference work offline.
--no-thinking disables the thinking prompt. --seed defaults to 42. --max-new-tokens and --max-denoising-steps both default to 256. Reaching either limit can leave a response unfinished. Streaming emits text as tokens are committed.
The runtime computes router probabilities and expert aggregation in FP32, then returns the aggregate in BF16 before the next backbone operation. Seeds reproduce results within this runtime version. The precision correction can change generated tokens relative to the earlier MLX release.
Python
from modilify_mk2_mlx import load
model = load()
result = model.generate(
[{"role": "user", "content": "Explain how rainbows form."}],
enable_thinking=False,
max_new_tokens=256,
max_denoising_steps=256,
seed=42,
)
print(result.text)
print(result.stop_reason, result.denoise_steps)
Pass system, user, and assistant messages in conversation order. result.token_ids contains the committed response tokens. To stream, pass on_text=callback. The callback receives text increments that concatenate to result.text. A seed makes requests reproducible in the same MLX environment. The MLX and PyTorch releases use separate random number streams.
License
This model is available for non-commercial research and experimentation under the Modilify Non-Commercial Model License 1.0. See NOTICE.md for attribution and upstream terms.
- Downloads last month
- 46
Quantized
Start the MLX server
# Install MLX LM: uv tool install mlx-lm# Start a local OpenAI-compatible server: mlx_lm.server --model "modilify/Modilify-Mk2-mlx"