Filter by type:
Filter by tags:
minicpm5-1b-claude-opus-fable5-v2-thinking
# MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking
GGUF quantizations for local deployment: **MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-GGUF**
中文说明
**MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking** is a compact 1B **Thinking** language model built on openbmb/MiniCPM5-1B. Compared with V1, this V2 release is further fine-tuned on **Fable 5** data with a stronger focus on **tool calling / function calling**, while also improving **coding** and **instruction-following**. It keeps MiniCPM5's native Thinking chat template and XML tool-call format.
Previous version: **MiniCPM5-1B-Claude-Opus-Fable5-Thinking** (V1)
For llama.cpp / Ollama / LM Studio deployment, see the **GGUF repository**.
## Overview
## Capabilities
- **Tool calling (enhanced in V2)** — more reliable XML / function-calling style tool use on top of MiniCPM5's native format
- **Coding** — code generation, debugging, and software-engineering-style tasks
- **Instruction following** — more reliable adherence to user prompts and structured constraints
- **Thinking mode** — chain-of-thought reasoning via the MiniCPM5 chat template
- **Long context** — up to **128K tokens** (131,072 tokens per `config.json`)
...
Repository: localaiLicense: apache-2.0
bonsai-8b-1bit
Bonsai 8B (PrismML) is an end-to-end 1-bit language model built on the Qwen3-8B dense
architecture (GQA, SwiGLU, RoPE, RMSNorm, 36 layers, 65,536 context). Every weight is a
single sign bit (`-scale` / `+scale`) with one FP16 scale per group of 128 weights, for
an effective 1.125 bits/weight and a ~1.15 GB footprint (14.2x smaller than FP16) while
matching full-precision 8B instruct models at ~70.5 average across 6 benchmark
categories.
The Q1_0 quantization is only decodable by the PrismML llama.cpp fork, so this entry
runs on LocalAI's `bonsai` backend (that fork), not the stock `llama-cpp` backend.
License: Apache 2.0.
Repository: localaiLicense: apache-2.0
bonsai-27b-1bit
Bonsai 27B (PrismML) is a full 27B-class reasoning model in end-to-end 1-bit weights,
derived from the Qwen3.6-27B hybrid-attention backbone (~75% linear attention, 262K
context). At a true 1.125 bits/weight it deploys in ~3.9 GB (~14.2x smaller than FP16)
while retaining 89.5% of FP16 intelligence across 15 thinking-mode benchmarks (math
91.66, coding 81.88). Ships an optional 4-bit vision tower (mmproj) for image input,
included here.
The Q1_0_g128 weights and hybrid-attention kernels are only in the PrismML llama.cpp
fork, so this runs on LocalAI's `bonsai` backend. A GPU is recommended. License: Apache 2.0.
Repository: localaiLicense: apache-2.0
minicpm5-1b-claude-opus-fable5-thinking
# MiniCPM5-1B-Claude-Opus-Fable5-Thinking
GGUF quantizations for local deployment: **MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF**
中文说明
**MiniCPM5-1B-Claude-Opus-Fable5-Thinking** is a compact 1B **Thinking** language model built on openbmb/MiniCPM5-1B. It is further fine-tuned on **Fable 5** data to improve **coding** and **instruction-following** while keeping MiniCPM5's native Thinking chat template and tool-call format.
For llama.cpp / Ollama / LM Studio deployment, see the **GGUF repository**.
## Overview
## Capabilities
- **Coding** — code generation, debugging, and software-engineering-style tasks
- **Instruction following** — more reliable adherence to user prompts and structured constraints
- **Thinking mode** — chain-of-thought reasoning via the MiniCPM5 chat template
- **Tool calling** — inherits MiniCPM5's XML tool-call format
- **Long context** — up to **128K tokens** (131,072 tokens per `config.json`)
## Quick start
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking"
...
Repository: localaiLicense: apache-2.0
serenity-26b-a4b
.mc-wrap{background:#0d1117;color:#c9d1d9;font-family:'Inter',sans-serif;max-width:920px;margin:0 auto;padding:24px;border-radius:16px;box-sizing:border-box}
.mc-wrap *{box-sizing:border-box}
.mc-wrap h1,.mc-wrap h2,.mc-wrap h3,.mc-wrap h4{color:#e6edf3;border:none}
.mc-wrap p{color:#c9d1d9}
.mc-wrap strong{color:#7ee8d0}
.mc-wrap a{color:#7ee8d0;text-decoration:none}
.mc-wrap ul{list-style:none;padding-left:0;margin:0}
.mc-wrap li{color:#c9d1d9;margin-bottom:8px;padding-left:4px}
.mc-wrap code{background:#161b22;color:#7ee8d0;padding:2px 8px;border-radius:4px;font-family:'JetBrains Mono',monospace;font-size:.88em;border:1px solid rgba(126,232,208,.15)}
.mc-hdr{text-align:center;padding:40px 32px;background:#0d1117;border:1px solid #21262d;border-radius:24px;margin-bottom:20px;position:relative;overflow:hidden}
.mc-hdr::before{content:'';position:absolute;top:0;left:0;right:0;height:3px;background:linear-gradient(135deg,#7ee8d0,#a78bfa,#c4b5fd)}
.mc-name{font-family:'Space Grotesk',sans-serif;font-size:2.8em;font-weight:800;margin:0;letter-spacing:-.02em;background:linear-gradient(135deg,#7ee8d0,#a78bfa,#c4b5fd);-webkit-background-clip:text;-webkit-text-fill-color:transparent;backg
...
Repository: localaiLicense: apache-2.0
gemma-4-31b-it-qat-q4_0
Gemma 4 31B is the largest dense multimodal (text + image) instruction-tuned model in the Gemma 4 family from Google DeepMind, optimized with Quantization-Aware Training (QAT) to preserve bfloat16-level quality while dramatically reducing the memory required to load the model. This is the official Google Q4_0 GGUF, shipped with its multimodal projector.
License: Apache 2.0 | Authors: Google DeepMind
Repository: localaiLicense: apache-2.0
gemma-4-31b-it-qat-mtp
Gemma 4 31B IT QAT (Google DeepMind), the largest dense model in the family, paired with its Multi-Token Prediction (MTP) drafter head for speculative decoding on the llama.cpp backend. The Q4_K_XL target carries the full multimodal (text + image) model; the small `mtp-gemma-4-31B-it` head predicts several tokens ahead which the target verifies in parallel, accelerating generation with no change to output quality. Dense models like 31B are the sizes that benefit most from MTP.
The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here. It uses the upstream `gemma4-assistant` architecture registered by llama.cpp PR #23398, so it loads on stock llama.cpp without any patch.
License: Apache 2.0 | Authors: Google DeepMind (target/drafter checkpoints), Unsloth (GGUF conversion)
Repository: localaiLicense: apache-2.0
lfm2.5-8b-a1b
Try LFM •
Docs •
LEAP •
Discord
# LFM2.5-8B-A1B
LFM2.5 is a new family of hybrid models designed for on-device deployment. It builds on the LFM2 architecture with extended pre-training and reinforcement learning.
- **On-device personal assistant**: Designed to power real-life applications, chaining tool calls, and following complex instructions on all devices.
- **Compressed performance**: Competitive with much larger dense and MoE models on instruction following and agentic tasks.
- **Unmatched throughput**: Fastest in its size class on both CPU and GPU inference, with day-one support for llama.cpp, MLX, vLLM, and SGLang.
Find more information about LFM2.5-8B-A1B in our blog post.
**AA-Omniscience Index (higher is better) rewards correct answers and penalizes hallucinations. Scores range from -100 to 100. See more results on Artificial Analysis.*
## 🗒️ Model Details
LFM2.5-8B-A1B is a general-purpose text-only model with the following features:
...
Repository: localaiLicense: other
gemma-4-31b-it
Google Gemma 4 31B-IT is the largest dense model in the Gemma 4 family with 31B parameters. It handles text and image input, generating text output, with a 256K context window and support for 140+ languages. Provides the highest quality outputs in the Gemma 4 lineup, well-suited for complex reasoning, summarization, and image understanding tasks.
Repository: localaiLicense: apache-2.0
baidu_ernie-4.5-21b-a3b-thinking
Over the past three months, we have continued to scale the thinking capability of ERNIE-4.5-21B-A3B, improving both the quality and depth of reasoning, thereby advancing the competitiveness of ERNIE lightweight models in complex reasoning tasks. We are pleased to introduce ERNIE-4.5-21B-A3B-Thinking, featuring the following key enhancements:
Significantly improved performance on reasoning tasks, including logical reasoning, mathematics, science, coding, text generation, and academic benchmarks that typically require human expertise.
Efficient tool usage capabilities.
Enhanced 128K long-context understanding capabilities.
Note: This version has an increased thinking length. We strongly recommend its use in highly complex reasoning tasks. ERNIE-4.5-21B-A3B-Thinking is a text MoE post-trained model, with 21B total parameters and 3B activated parameters for each token.
Repository: localaiLicense: apache-2.0
liquidai_lfm2-8b-a1b
LFM2 is a new generation of hybrid models developed by Liquid AI, specifically designed for edge AI and on-device deployment. It sets a new standard in terms of quality, speed, and memory efficiency.
We're releasing the weights of our first MoE based on LFM2, with 8.3B total parameters and 1.5B active parameters.
LFM2-8B-A1B is the best on-device MoE in terms of both quality (comparable to 3-4B dense models) and speed (faster than Qwen3-1.7B).
Code and knowledge capabilities are significantly improved compared to LFM2-2.6B.
Quantized variants fit comfortably on high-end phones, tablets, and laptops.
Repository: localaiLicense: lfm1.0
qwen3-8b-jailbroken
This jailbroken LLM is released strictly for academic research purposes in AI safety and model alignment studies. The author bears no responsibility for any misuse or harm resulting from the deployment of this model. Users must comply with all applicable laws and ethical guidelines when conducting research.
A jailbroken Qwen3-8B model using weight orthogonalization[1].
Implementation script: https://gist.github.com/cooperleong00/14d9304ba0a4b8dba91b60a873752d25
[1]: Arditi, Andy, et al. "Refusal in language models is mediated by a single direction." arXiv preprint arXiv:2406.11717 (2024).
Repository: localaiLicense: apache-2.0
minicpm5-1b
MiniCPM5-1B is a compact 1B-parameter language model based on the LLaMA architecture. Despite its small size, it achieves competitive performance among sub-2B models, making it ideal for edge deployment and resource-constrained environments.
Repository: localaiLicense: apache-2.0
gemma-3-1b-it
google/gemma-3-1b-it is a large language model with 1 billion parameters. It is part of the Gemma family of open, state-of-the-art models from Google, built from the same research and technology used to create the Gemini models. Gemma 3 models are multimodal, handling text and image input and generating text output, with open weights for both pre-trained variants and instruction-tuned variants. These models have multilingual support in over 140 languages, and are available in more sizes than previous versions. They are well-suited for a variety of text generation and image understanding tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them in environments with limited resources such as laptops, desktops or your own cloud infrastructure, democratizing access to state of the art AI models and helping foster innovation for everyone.
Repository: localaiLicense: gemma
huihui-ai_gemma-3-1b-it-abliterated
This is an uncensored version of google/gemma-3-1b-it created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens
Repository: localaiLicense: gemma
amoral-gemma3-1b-v2
Core Function:
Produces analytically neutral responses to sensitive queries
Maintains factual integrity on controversial subjects
Avoids value-judgment phrasing patterns
Response Characteristics:
No inherent moral framing ("evil slop" reduction)
Emotionally neutral tone enforcement
Epistemic humility protocols (avoids "thrilling", "wonderful", etc.)
Repository: localaiLicense: apache-2.0
falcon3-1b-instruct
Falcon3 family of Open Foundation Models is a set of pretrained and instruct LLMs ranging from 1B to 10B parameters.
This repository contains the Falcon3-1B-Instruct. It achieves strong results on reasoning, language understanding, instruction following, code and mathematics tasks. Falcon3-1B-Instruct supports 4 languages (English, French, Spanish, Portuguese) and a context length of up to 8K.
Repository: localaiLicense: falcon-llm-license
falcon3-1b-instruct-abliterated
This is an uncensored version of tiiuae/Falcon3-1B-Instruct created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.
Repository: localaiLicense: falcon-llm-license
granite-3.0-1b-a400m-instruct
Granite 3.0 language models are a new set of lightweight state-of-the-art, open foundation models that natively support multilinguality, coding, reasoning, and tool usage, including the potential to be run on constrained compute resources. All the models are publicly released under an Apache 2.0 license for both research and commercial use. The models' data curation and training procedure were designed for enterprise usage and customization in mind, with a process that evaluates datasets for governance, risk and compliance (GRC) criteria, in addition to IBM's standard data clearance process and document quality checks.
Granite 3.0 includes 4 different models of varying sizes:
Dense Models: 2B and 8B parameter models, trained on 12 trillion tokens in total.
Mixture-of-Expert (MoE) Models: Sparse 1B and 3B MoE models, with 400M and 800M activated parameters respectively, trained on 10 trillion tokens in total.
Accordingly, these options provide a range of models with different compute requirements to choose from, with appropriate trade-offs with their performance on downstream tasks. At each scale, we release a base model — checkpoints of models after pretraining, as well as instruct checkpoints — models finetuned for dialogue, instruction-following, helpfulness, and safety.
Repository: localaiLicense: apache-2.0
moe-girl-1ba-7bt-i1
A finetune of OLMoE by AllenAI designed for roleplaying (and maybe general usecases if you try hard enough).
PLEASE do not expect godliness out of this, it's a model with 1 billion active parameters. Expect something more akin to Gemma 2 2B, not Llama 3 8B.
Repository: localaiLicense: apache-2.0
llama-3.2-1b-instruct:q4_k_m
The Meta Llama 3.2 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction-tuned generative models in 1B and 3B sizes (text in/text out). The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. They outperform many of the available open source and closed chat models on common industry benchmarks.
Model Developer: Meta
Model Architecture: Llama 3.2 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.
Repository: localaiLicense: llama3.2
Page 1