---

language:
- en
tags:
- duoneural
- litert
- edge
- gguf
- on-device
- smollm

• smol

• tiny

• litert

• edge

• instruct base_model: HuggingFaceTB/SmolLM2-135M-Instruct pipeline_tag: text-generation license: apache-2.0

  SmolLM2-135M-Instruct-LiteRT

  SmolLM2 135M Instruct — ultra-tiny on-device assistant (~90MB) — converted for mobile and edge deployment by DuoNeural.

  • Source model: HuggingFaceTB/SmolLM2-135M-Instruct
  • Format: GGUF Q4_K_M (llama.cpp-compatible)
  • File size: 105 MB
  • Quantization: 4-bit K-mean (Q4_K_M) — excellent accuracy/size trade-off for edge devices
  • Target platforms: Android, iOS, desktop edge inference
  • Converted: 2026-05-06 06:08:19 by Archon / DuoNeural

  Usage

  llama.cpp (CLI)

  ./llama-cli -m SmolLM2-135M-Instruct-LiteRT_Q4_K_M.gguf -n 512 --temp 0.7
  

  Google AI Edge / MediaPipe (Android/iOS)

  This GGUF is compatible with MLC-LLM and llama.cpp Android bindings for on-device inference. For use with Google Edge Gallery, convert to .task bundle using MediaPipe LLM conversion tools.

  Python via llama-cpp-python

  from llama_cpp import Llama
  
  llm = Llama(
      model_path="SmolLM2-135M-Instruct-LiteRT_Q4_K_M.gguf",
      n_ctx=2048,
      n_threads=4,
      verbose=False,
  )
  
  response = llm.create_chat_completion(
      messages=[
          {"role": "system", "content": "You are a helpful assistant."},
          {"role": "user", "content": "Hello! How can you help me today?"},
      ]
  )
  print(response["choices"][0]["message"]["content"])
  

  Ollama

  ollama run hf.co/DuoNeural/SmolLM2-135M-Instruct-LiteRT
  

  About the Conversion

  Converted using llama.cpp GGUF pipeline with CUDA acceleration. Source weights downloaded from HuggingFace, converted to F16 GGUF, then quantized to Q4_K_M.

  ---

  DuoNeural

  DuoNeural is an open AI research lab — human + AI in collaboration.

  Platform	Link
  HuggingFace	huggingface.co/DuoNeural
  Website	duoneural.com
  GitHub	github.com/DuoNeural
  X / Twitter	@DuoNeural
  Email	duoneural@proton.me
  Newsletter	duoneural.beehiiv.com
  Support	buymeacoffee.com/duoneural

  DuoNeural Research Publications

  Title	DOI
  Nano-CTM: Ternary Continuous Thought Machines with Thought-Space Self-Prediction for Efficient Iterative Reasoning	10.5281/zenodo.19775622
  Recurrence as World Model: CTM Learns Implicit Belief States in Partially Observable Physical Environments	10.5281/zenodo.19810620
  Per-Object Slot Decomposition for Scalable Neural World Modeling: When Does Attention Beat Mean-Field?	10.5281/zenodo.19846804
  The Dynamical Horizon Principle: CTM Gates Converge to the Predictability Limit of Dynamical Systems	10.5281/zenodo.19952612

  Open access, CC BY 4.0. Authored by Archon, Jesse Caldwell, Aura — DuoNeural.

  Research Team

  • Jesse — Vision, hardware, direction
  • Archon — Lab Director, post-training, abliteration, experiments
  • Aura — Research AI, literature synthesis, novel proposals

  Subscribe to the lab newsletter at duoneural.beehiiv.com for model drops before they go anywhere else.