Model Gallery

8 models from 1 repositories

Filter by type:

Filter by tags:

bonsai-8b-1bit
Bonsai 8B (PrismML) is an end-to-end 1-bit language model built on the Qwen3-8B dense architecture (GQA, SwiGLU, RoPE, RMSNorm, 36 layers, 65,536 context). Every weight is a single sign bit (`-scale` / `+scale`) with one FP16 scale per group of 128 weights, for an effective 1.125 bits/weight and a ~1.15 GB footprint (14.2x smaller than FP16) while matching full-precision 8B instruct models at ~70.5 average across 6 benchmark categories. The Q1_0 quantization is only decodable by the PrismML llama.cpp fork, so this entry runs on LocalAI's `bonsai` backend (that fork), not the stock `llama-cpp` backend. License: Apache 2.0.

Repository: localaiLicense: apache-2.0

ternary-bonsai-8b
Ternary Bonsai 8B (PrismML) is a 1.58-bit ternary language model on the Qwen3-8B dense architecture. Each weight takes a value from {-1, 0, +1} with one shared FP16 scale per group of 128 weights (GGUF Q2_0, ~2.18 GB deployed, 7.5x smaller than FP16). The extra zero state recovers more of the full-precision model than the 1-bit build: it ranks 2nd among compared 6-9B models at 75.5 average despite being ~1/8th their size. Q2_0 is the recommended, ternary-lossless variant. The Q2_0 kernels are only in the PrismML llama.cpp fork, so this runs on LocalAI's `bonsai` backend. License: Apache 2.0.

Repository: localaiLicense: apache-2.0

ternary-bonsai-8b-q2-g64
Ternary Bonsai 8B (PrismML), GGUF Q2_0 with group-64 packing (each FP16 scale shared across 64 weights instead of 128). Slightly larger (~2.31 GB) but matches llama.cpp's native 64-value Q2_0 block layout. Runs on LocalAI's `bonsai` backend. License: Apache 2.0.

Repository: localaiLicense: apache-2.0

ternary-bonsai-8b-pq2
Ternary Bonsai 8B (PrismML), GGUF PQ2_0 (packed Q2_0) ternary variant (~2.18 GB). Same {-1, 0, +1} weight alphabet as Q2_0. Runs on LocalAI's `bonsai` backend. License: Apache 2.0.

Repository: localaiLicense: apache-2.0

bonsai-27b-1bit
Bonsai 27B (PrismML) is a full 27B-class reasoning model in end-to-end 1-bit weights, derived from the Qwen3.6-27B hybrid-attention backbone (~75% linear attention, 262K context). At a true 1.125 bits/weight it deploys in ~3.9 GB (~14.2x smaller than FP16) while retaining 89.5% of FP16 intelligence across 15 thinking-mode benchmarks (math 91.66, coding 81.88). Ships an optional 4-bit vision tower (mmproj) for image input, included here. The Q1_0_g128 weights and hybrid-attention kernels are only in the PrismML llama.cpp fork, so this runs on LocalAI's `bonsai` backend. A GPU is recommended. License: Apache 2.0.

Repository: localaiLicense: apache-2.0

ternary-bonsai-27b
Ternary Bonsai 27B (PrismML) is the quality-oriented operating point of the Bonsai 27B family: full 27B-class reasoning in ternary {-1, 0, +1} weights on the Qwen3.6-27B hybrid-attention backbone (262K context). At a true 1.71 bits/weight it deploys in ~7.2 GB (GGUF Q2_0_g128) and retains 95% of FP16 intelligence (80.49 average across 15 thinking-mode benchmarks) - a higher score than a conventional IQ2_XXS build at less than two-thirds its footprint. Ships an optional 4-bit vision tower (mmproj), included. The Q2_0 weights and hybrid-attention kernels are only in the PrismML llama.cpp fork, so this runs on LocalAI's `bonsai` backend. A GPU is recommended. License: Apache 2.0.

Repository: localaiLicense: apache-2.0

ternary-bonsai-27b-pq2
Ternary Bonsai 27B (PrismML), GGUF PQ2_0 (packed Q2_0) ternary variant (~7.17 GB) with the 4-bit vision tower (mmproj) included. Runs on LocalAI's `bonsai` backend. License: Apache 2.0.

Repository: localaiLicense: apache-2.0

ternary-bonsai-27b-q2-g64
Ternary Bonsai 27B (PrismML), GGUF Q2_0 with group-64 packing (~7.59 GB), matching llama.cpp's native 64-value Q2_0 block layout, with the 4-bit vision tower (mmproj) included. Runs on LocalAI's `bonsai` backend. License: Apache 2.0.

Repository: localaiLicense: apache-2.0