Filter by type:
Filter by tags:
qwen3.8-flash-next-unsloth-ud-q5_k_xl
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
Unsloth's Dynamic quantization (UD-Q5_K_XL, 5-bit, close to Q8_0) (about 159
GB to download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 108 GB of RAM to keep every expert in memory; with less, the
mapped experts are read from the SSD while it answers, which works and is much
slower. Keep the files on an SSD. Needs a GPU the strata backend supports;
NVIDIA with 12 GB of VRAM or more is the class Strata tests. Runs only on the
experimental strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF files are the Unsloth quantization; the vision projector is
ISTA-DASLab's file for the original model; Strata is by Niko1221. The MTP
draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.5-0.8b-artemis-q5
TurinTech Artemis Q5_K_M quantization of Qwen3.5 0.8B for text chat,
reasoning, and tool use. This text-only build omits the vision encoder
and MTP layer. Uses the embedded chat template and an 8192-token context.
Repository: localaiLicense: apache-2.0
qwen3.5-2b-artemis-q5-k-m
TurinTech Artemis Q5_K_M quantization of Qwen3.5 2B for local tool calls,
maths, and instruction following. Accepts image input through the
bundled F16 vision projector. Uses the embedded chat template and an
8192-token context.
Repository: localaiLicense: apache-2.0
qwen3.5-4b-artemis-q5-k-m
TurinTech Artemis Q5_K_M quantization of Qwen3.5 4B for local chat and
tool calls. Accepts image input through the bundled F16 vision
projector. Uses the embedded chat template and an 8192-token context.
Repository: localaiLicense: apache-2.0
qwen3.5-4b-artemis-q5-k-s
TurinTech Artemis Q5_K_S quantization of Qwen3.5 4B for local chat and
tool calls. Accepts image input through the bundled F16 vision
projector. Uses the embedded chat template and an 8192-token context.
Repository: localaiLicense: apache-2.0
qwen3.5-9b-artemis-q5-k-s
TurinTech Artemis Q5_K_S quantization of Qwen3.5 9B for local chat and
tool calls. Accepts image input through the bundled F16 vision
projector. Uses the embedded chat template and an 8192-token context.
Repository: localaiLicense: apache-2.0
dirk-qwen3.8-27b-q5
Dirk in the higher-quality Q5_K_XL GGUF format, with MTP speculative
decoding and the shared F16 vision projector for multimodal prompts.
Repository: localaiLicense: apache-2.0
huihui-qwen3.8-27b-abliterated-q5
Huihui Qwen3.8 27B in Q5_K GGUF format, with the shared BF16 vision
projector and MTP speculative decoding through llama.cpp.
Repository: localaiLicense: apache-2.0
ornith-1.5-9b-q5
Ornith-1.5-9B in the Q5_K_M GGUF format, with the shared BF16
vision projector for multimodal prompts.
Repository: localaiLicense: mit
ornith-1.5-35b-a3b-q5
Ornith-1.5-35B-A3B in the Q5_K_M GGUF format, with the shared
BF16 vision projector for multimodal prompts.
Repository: localaiLicense: mit
nex-n2.5-mini-q5
Nex-N2.5-mini is a 35B-parameter Qwen3.5 MoE model for coding, tool use,
and computer and browser tasks with image input. This Q5_K_M GGUF build
includes the F16 vision projector and uses the embedded chat template.
Repository: localaiLicense: apache-2.0
tiel-coder-35b-a3b-q5-mtp
Tiel-Coder-35B-A3B in Q5_K_XL format with MTP speculative decoding and a
BF16 vision projector.
Repository: localaiLicense: mit
sharp-spark-x2.5-4b-q5
Sharp-Spark is an imatrix quantization of XHToken's Spark-X2.5-4B text model
with an adjusted chat template for coding. This Q5_K_XL build uses the
embedded Sharp-Spark template and a 32K-token default context.
Repository: localaiLicense: apache-2.0
qwen3.8-35b-a3b-distill-q5
Empero's Qwen3.8 distillation into Qwen3.6-35B-A3B has 35B total
parameters and about 3B active per token. This Q5_K_M GGUF build
uses the embedded reasoning template and includes the F16 vision
projector. Vision is inherited from the base and was not evaluated
by the publisher.
Repository: localaiLicense: apache-2.0
gemma-4-12b-it-q5
Gemma 4 12B IT is Google's 12B instruction-tuned model for chat,
reasoning, tool use, and image understanding. This Unsloth Q5_K_M
GGUF build includes the F16 vision projector and uses the embedded
chat template. It defaults to a 32,768-token context.
Repository: localaiLicense: apache-2.0
gemma-4-e4b-hauhaucs-aggressive-q5
HauhauCS Aggressive is an uncensored derivative of Google's Gemma 4 E4B
instruction model. This Q5_K_M GGUF supports text chat and image input
through the publisher's bundled F16 multimodal projector.
Uses the embedded chat template and the publisher's sampling settings.
The default context is 8192 tokens. The publisher declares the Gemma license.
Repository: localaiLicense: gemma
lythri-7b-a4b-q5
Lythri 7B A4B is a Gemma 4 E4B fine-tune for conversational companionship,
with 7.46B total parameters and 4.5B active parameters.
This Q5_K_M GGUF build runs text chat through llama.cpp with the embedded
chat template and a 32,768-token context.
Repository: localaiLicense: apache-2.0
zero-gemma4-e4b-openzero-q5-k-m
Zero Gemma4-E4B OpenZero is a Gemma 4 E4B instruction fine-tune for
local coding, research, chat, and agentic workflows. It was trained on
2,033 curated OpenZero examples and is distributed as one merged Q5_K_M
GGUF file.
License: OpenZero Community Source v1; see the model repository for terms.
Repository: localaiLicense: other
lfm2.5-24b-a2b-q5
LiquidAI LFM2.5-24B-A2B is a text-only reasoning model with 24B total
parameters and 2.45B active parameters for multilingual chat and tool use.
This entry uses the official Q5_K_M GGUF quantization.
Repository: localaiLicense: other
neohorse-1-4b-q5
NeoHorse-1-4B is TokenRhythm's text-only Qwen3.5-4B fine-tune for coding,
reasoning, and agentic tasks. This official Q5_K_M GGUF build uses the
embedded Jinja chat template and a 32K-token default context.
Repository: localaiLicense: apache-2.0
qwen3-reranker-0.6b
The Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. Building upon the dense foundational models of the Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining.
**Exceptional Versatility**: The embedding model has achieved state-of-the-art performance across a wide range of downstream application evaluations. The 8B size embedding model ranks No.1 in the MTEB multilingual leaderboard (as of June 5, 2025, score 70.58), while the reranking model excels in various text retrieval scenarios.
**Comprehensive Flexibility**: The Qwen3 Embedding series offers a full spectrum of sizes (from 0.6B to 8B) for both embedding and reranking models, catering to diverse use cases that prioritize efficiency and effectiveness. Developers can seamlessly combine these two modules. Additionally, the embedding model allows for flexible vector definitions across all dimensions, and both embedding and reranking models support user-defined instructions to enhance performance for specific tasks, languages, or scenarios.
**Multilingual Capability**: The Qwen3 Embedding series offer support for over 100 languages, thanks to the multilingual capabilites of Qwen3 models. This includes various programming languages, and provides robust multilingual, cross-lingual, and code retrieval capabilities.
**Qwen3-Reranker-0.6B** has the following features:
- Model Type: Text Reranking
- Supported Languages: 100+ Languages
- Number of Paramaters: 0.6B
- Context Length: 32k
- Quantization: q4_K_M, q5_0, q5_K_M, q6_K, q8_0, f16
Repository: localaiLicense: apache-2.0
Page 1