DEV Community

#cuda

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Swizzle Runtime v0.1.0: How We Reached 209 tok/s and 43ms TTFT on a 4GB Laptop GPU in Pure Rust

Swizzle Runtime v0.1.0: How We Reached 209 tok/s and 43ms TTFT on a 4GB Laptop GPU in Pure Rust

9
Comments 5
2 min read
Project EREN: My Journey to Building a CUDA-Powered LLM Inference Engine

Project EREN: My Journey to Building a CUDA-Powered LLM Inference Engine

1
Comments
4 min read
Jensen is out. CUDA isn't. A Developer's Guide to Everything Nvidia Built

Jensen is out. CUDA isn't. A Developer's Guide to Everything Nvidia Built

8
Comments 4
25 min read
Qwen3.8-Flash-Next (125B) on an RTX 3060 12 GB with 30 GB RAM: 50-70 tok/s

Qwen3.8-Flash-Next (125B) on an RTX 3060 12 GB with 30 GB RAM: 50-70 tok/s

1
Comments 2
3 min read
Amazon: New TorchNeuron tool challenges Nvidia's dominance

Amazon: New TorchNeuron tool challenges Nvidia's dominance

1
Comments
2 min read
Running NVIDIA's new 120B Nemotron-3.5-Super on an 8 GB GPU — skipping the zeros, and finding the real bottleneck

Running NVIDIA's new 120B Nemotron-3.5-Super on an 8 GB GPU — skipping the zeros, and finding the real bottleneck

Comments
7 min read
One Word, Two GPUs: Why Gemma 4 Answers the Same Question Differently on a Laptop and a Tesla T4

Hardware differences subtly alter benchmark grades

One Word, Two GPUs: Why Gemma 4 Answers the Same Question Differently on a Laptop and a Tesla T4

30
Comments 7
12 min read
Running vLLM natively on Windows

Running vLLM natively on Windows

Comments
6 min read
Running the 510 GB DeepSeek-V4.1-Flash on an 8 GB GPU — and three bugs that never raise an error

Running the 510 GB DeepSeek-V4.1-Flash on an 8 GB GPU — and three bugs that never raise an error

Comments
8 min read
Running a 133 GB MoE model on an 8 GB GPU at 11 tokens/s by streaming experts from NVMe

Running a 133 GB MoE model on an 8 GB GPU at 11 tokens/s by streaming experts from NVMe

Comments
6 min read
GPU Kernels, Part 2: Teaching a 27B Model to Write Triton Kernels with SFT and a GPU Gym

GPU Kernels, Part 2: Teaching a 27B Model to Write Triton Kernels with SFT and a GPU Gym

Comments
21 min read
Size Your Model to Where You Run It: Gemma 4 E2B and E4B on an AWS L4 and Two Small GPUs

Size Your Model to Where You Run It: Gemma 4 E2B and E4B on an AWS L4 and Two Small GPUs

8
Comments
10 min read
TensorRT Was Fast. My 1080p RTSP Pipeline Wasn't.

TensorRT Was Fast. My 1080p RTSP Pipeline Wasn't.

Comments
5 min read
Gemma 4 on a 4 GB GPU in Plain PyTorch: From Out of Memory to Faster Than llama.cpp

Includes a tricky broadcast K/V debugging story

Gemma 4 on a 4 GB GPU in Plain PyTorch: From Out of Memory to Faster Than llama.cpp

7
Comments 3
13 min read
Size Your Model to Where You Run It: Gemma 4 E2B and E4B on an AWS L4 and Two Small GPUs

Budget GPUs achieve surprisingly high performance

Size Your Model to Where You Run It: Gemma 4 E2B and E4B on an AWS L4 and Two Small GPUs

3
Comments 2
10 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
⌂ 换网址