aipster/Phi-4-mini-instruct-abliterated-q4f16_1-MLC
Text Generation • Updated • 4
AIpster
company
AI & ML interests
None defined yet.
Recent Activity
Organization Card
AIpster
An independent think tank on artificial intelligence, society, and the future of thought.
We're a collective of computer science friends from the late '90s who turned a WhatsApp group into a laboratory for exploring what AI is doing to how we work, build, and think.
What we do here
This Hugging Face organization is where we publish the artifacts of our exploration — models, datasets, and tools that come out of the experiments we write about on our blog.
We're not a company. We don't sell anything. We build things to understand them, then share what we learned.
Focus areas
- 🔬 Small specialist models — distillation, fine-tuning, and the art of making tiny models punch above their weight
- 🧭 Prompt engineering & routing — how prompts become infrastructure, not just text
- 🛠️ Local LLM workflows — what 96 GB of VRAM can (and can't) do
- 🤖 Coding agents & automation — how AI is reshaping software development from the inside out
- 📖 AI & society — the uncomfortable conversations the industry would rather skip
What you'll find here
Models
DevRouter-1.5B — our first release. A tiny prompt router that reads a raw developer prompt and returns a single JSON decision: a cleaned-up
rewrite, an intent / complexity classification, a suggested model-tier route, and the context the prompt forgot to include. Built on Qwen2.5-Coder-1.5B (Apache 2.0) and distilled from a
stronger teacher, it holds ~96% valid-JSON and runs at ~280 tokens/s on a single RTX 3090 — small enough to sit in front of your real models and triage every prompt in 1–3 seconds.
- 🧠 aipster/DevRouter-1.5B — fp16 weights (transformers / vLLM)
- 📦 aipster/DevRouter-1.5B-GGUF — Q8_0 + F16, plug-n-play with Ollama / llama.cpp
And one honest caveat, because we ship those too: Q6 and below quantizations break its JSON. A small model doing strict structured output is far more fragile than the "Q4 is fine" rule of thumb suggests — ship Q8_0 or F16.
Phi-4-mini-instruct abliterated · MLC q4f16_1 — an uncensored Phi-4-mini (3.8B), abliterated to remove refusal behavior and quantized to 4-bit MLC format for in-browser inference via WebGPU. This is the model that powers the top-tier slot in our Local AI Playground — a browser tool where LLMs run entirely on your GPU, no server involved. Built from huihui-ai's abliteration of Microsoft's Phi-4-mini (MIT), converted with MLC-LLM's source-built toolchain after the pip nightly broke on linux. ~2.1 GB download, ~3.4 GB VRAM. Sovereignty as a feature, not a slogan.
- 🧠 aipster/Phi-4-mini-instruct-abliterated-q4f16_1-MLC — MLC q4f16_1 weights (WebLLM / MLC-LLM)
Datasets
Coming soon — curated and synthetic datasets from our distillation experiments, released alongside the models that use them.
Spaces
Coming soon — interactive demos of our experiments.
Read our work
- 📝 Blog
Articles
- 🧪 How we built our distillation pipeline (coming soon)
- 🔍 Four GPUs, Two Weeks, and the Uncomfortable Truth About Local LLMs
- 🤖 I Stopped Learning n8n. I Just Told My Coding Agent What I Wanted
- 💸 Two Hours to Mass Extinction: What Coding Agents Mean for the Open-Core Business Model
- 🤔 Six Fallacies That Break Agentic AI Systems (And How to Design Around Them)
- 🕵🏻 The AI "Web Search" That Quietly Decides What You're Allowed to Find (part 2 is already here)
Up-to-date information
Every day, we provide a summary of what is shaking the AI ecossystem up. It include.
Tutorials
Local LLM series
This series deal with running LLMs locally and some theory behind LLMs.
- 📚 Introduction - Provide some introductory tips on how to get start in the world of local LLMs;
- 📚 The Brain, the Engine, and Your First Llama on Ollama - Teaches how to make your first model online. Also deal with how transformers work and the relation between a model and an inference engine.
- 📚 From terminal to a ChatGPT style chat - Teaches how to install Open WebUI to provide a Chat GPT like interface. Delve into multimodality and how a model can seamlessly understand text, audio, and images.
- 📚 The facts and the reason - Teaches how RAG works and the impact of context in the token probabilities
- 📚 Does size really matter? - Teaches what, besides parameter count, impact how well a model perform
- 📚 It's just text - Creates an adhoc tool calling scheme for educational purposes
Philosophy
We build to understand. We share to learn together.
Everything we publish here is open. Code, weights, datasets, methodology — including the failures. Especially the failures.
Get in touch
- 🌐 Website: aipster.com
- 📬 Email: contact@aipster.com
Independent. Curious. Slightly skeptical.
datasets 0
None public yet