DEV Community

#cuda

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
How vLLM CUDA Kernels Write and Read the Paged KV Cache

How vLLM CUDA Kernels Write and Read the Paged KV Cache

Comments 2
5 min read
Can a QAT 12B Fit an 8 GB GPU? Gemma 4 12B on an RTX 3050

Can a QAT 12B Fit an 8 GB GPU? Gemma 4 12B on an RTX 3050

8
Comments
15 min read
Zero-Cloud AI Code Review: Connecting Self-Hosted GitLab to Local CUDA LLM Runtimes

Zero-Cloud AI Code Review: Connecting Self-Hosted GitLab to Local CUDA LLM Runtimes

Comments
4 min read
Reverse-Engineering NVIDIA: Modifying a CUDA binary

Reverse-Engineering NVIDIA: Modifying a CUDA binary

Comments
4 min read
I wrote 21 CUDA kernels on a free GPU. Here's what the profiler taught me.

I wrote 21 CUDA kernels on a free GPU. Here's what the profiler taught me.

1
Comments
5 min read
Gemma 4 at Over 70 Tokens/s on a 2021 Laptop's 4 GB GPU: The Live Demo, Step by Step

Gemma 4 at Over 70 Tokens/s on a 2021 Laptop's 4 GB GPU: The Live Demo, Step by Step

9
Comments 2
12 min read
Gemma 4 on a Tesla T4, Part 3: Int4 Embeddings Serve E2B in 2.86 GiB at 2.30x bf16

Gemma 4 on a Tesla T4, Part 3: Int4 Embeddings Serve E2B in 2.86 GiB at 2.30x bf16

9
Comments 2
11 min read
A 4 GB Laptop GPU vs a 6-Core CPU on Gemma 4, Re-Measured in ABBA Order: 4.1x

A 4 GB Laptop GPU vs a 6-Core CPU on Gemma 4, Re-Measured in ABBA Order: 4.1x

10
Comments 2
11 min read
Gemma 4 on a Tesla T4, Part 2: The Minimum GCE VM and a Script to Drive It

Gemma 4 on a Tesla T4, Part 2: The Minimum GCE VM and a Script to Drive It

13
Comments
13 min read
Finding a Random Island with Geometry and CUDA

Finding a Random Island with Geometry and CUDA

Comments
4 min read
Gemma 4 on a Tesla T4: QAT Weights Decode 1.79x Faster Than bf16

Gemma 4 on a Tesla T4: QAT Weights Decode 1.79x Faster Than bf16

8
Comments 1
9 min read
Same nvJPEG2000, different numbers: timer boundaries and frames in flight

Same nvJPEG2000, different numbers: timer boundaries and frames in flight

Comments
13 min read
Qwen3.8 27B at 256K: 50 TPS on a 24 GB GPU

Qwen3.8 27B at 256K: 50 TPS on a 24 GB GPU

1
Comments 1
11 min read
Understanding GPU Memory: VRAM, Bandwidth, and Why Your Model Won't Fit

Understanding GPU Memory: VRAM, Bandwidth, and Why Your Model Won't Fit

1
Comments
2 min read
Gemma 4 on an 2021 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6

Gemma 4 on an 2021 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6

11
Comments 3
13 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
⌂ 换网址