Quantization - DEV Community

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

GINIGEN AI

Nov 11

Cutting 64 of 256 Experts from a GGUF by Byte Copy: What the Headers Say About GLM-5.3-SLIM on One B200

#ai #machinelearning #edgeai #quantization

13 min read

Private Local AI

Nov 10

Model quantization, read carefully: what the bit-widths really cost you

#ai #performance #storage #quantization

6 min read

Private Local AI

Nov 10

GSQ and RCO: the non-uniform quantization now sitting in GGUF files

#ai #performance #opensource #quantization

7 min read

Suresh Kumar Pallapothu

Nov 10

Day 4: Optimizing Model Weights — Quantization Basics (GGUF, AWQ) for Efficient Inference

#ai #quantization #aiintermediateseries

4 min read

Nov 10

llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff

#quantization #vllm #fp8 #awq

9 min read

Nov 8

2비트라는 이름의 GGUF가 3비트 이름의 파일보다 컸습니다: 공개 Darwin-180B 양자화 파일 6개를 헤더까지 열어 보니

#ai #machinelearning #llm #quantization

4 min read

GINIGEN AI

Nov 8

One Omni Model, Two GGUF Files: What LFM3-Omni-2B Really Costs in RAM, from the Tensor Headers Up

#ai #machinelearning #edgeai #quantization

12 min read

Aleksei Romanov for g factor

Nov 5

Quantization, Explained: From FP32 to INT4, and What Each Precision Costs You

#quantization #machinelearning #llm #ai

11 min read

mycute

Nov 9

We tried to build a 0.1-bit model. It worked on one task and failed on another.

#llm #machinelearning #quantization #opensource

13 min read

GINIGEN AI

Nov 2

A 763B MoE at "2 Bits": Where the Bytes Really Go in the GSQ-RCO GGUFs, and Why 100% Recovery Sits Inside the Noise

#ai #machinelearning #edgeai #quantization

15 min read

Huilin Ye

Oct 30

FP8 Cut Our TPOT by 25%. At High Concurrency, It Also Changed the Answers.

#llm #vllm #quantization #benchmarking

3 min read

amrit

Oct 26

Why T Quant is the Secret Weapon for Mobile AI in 2026

#diffusionllm #quantization #mobileai #ai

6 min read

GINIGEN AI

Oct 19

LFM2.5-24B-A2B on Your Own Hardware: The RAM, KV Cache and Bandwidth Budget

#ai #machinelearning #edgeai #quantization

11 min read

amrit

Oct 12

How One Startup Cut Diffusion‑LLM Inference Costs by 45 % – No Calibration Needed

#ai #quantization #diffusionllm

6 min read

Oct 12

Model Distillation & Quantization: Running 8B Parameter Models at a Fraction of Cloud Cost

#modeldistillation #quantization #vllm #awq

10 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.