DEV Community

#llamacpp

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Ollama vs. llama.cpp on the Same GGUF: The 15% Speed Gap Was the Defaults, Not the Engine
maruhana profile

Ollama vs. llama.cpp on the Same GGUF: The 15% Speed Gap Was the Defaults, Not the Engine

#ai #llm #ollama #llamacpp
Comments
7 min read
Jan open source ChatGPT alternative: offline local AI
spanobserver profile

Jan open source ChatGPT alternative: offline local AI

#llamacpp #llm #localai #mcp
Comments
7 min read
How to Use a Remote Ollama or llama.cpp Server From a Desktop AI App on Windows in 2026
alichherawalla profile

How to Use a Remote Ollama or llama.cpp Server From a Desktop AI App on Windows in 2026

#ollama #llamacpp #windows #ai
Comments
6 min read
Six Serving Engines, One 8 GB GPU: Gemma 4 E2B on an RTX 3050
xbill profile

Six Serving Engines, One 8 GB GPU: Gemma 4 E2B on an RTX 3050

#gemma #vllm #llamacpp #nvidia
22
Comments 1
12 min read
Extending llama.cpp to Support Reflection Beam-1.0-501B-A23B: Handling Complex MoE and Attention Architectures
pdrojack profile

Extending llama.cpp to Support Reflection Beam-1.0-501B-A23B: Handling Complex MoE and Attention Architectures

#llamacpp #mixtureofexperts #attention #modelconversion
Comments
3 min read
Running MoE LLMs Larger Than Your RAM: A Practical Guide to mmap, Expert Offloading and SSD Streaming with llama.cpp
eme_gug_0821b41b948be6516 profile

Running MoE LLMs Larger Than Your RAM: A Practical Guide to mmap, Expert Offloading and SSD Streaming with llama.cpp

#llm #llamacpp #moe #performance
Comments
8 min read
One Word, Two GPUs: Why Gemma 4 Answers the Same Question Differently on a Laptop and a Tesla T4

Hardware differences subtly alter benchmark grades
xbill profile

One Word, Two GPUs: Why Gemma 4 Answers the Same Question Differently on a Laptop and a Tesla T4

#gemma #llamacpp #cuda #nvidia
21
Comments 6
12 min read
Four GPUs, One Quantization Ladder: Gemma 4 E2B From a 4 GB Laptop to an NVIDIA L4
xbill profile

Four GPUs, One Quantization Ladder: Gemma 4 E2B From a 4 GB Laptop to an NVIDIA L4

#gemma #llamacpp #quantization #nvidia
13
Comments 2
11 min read
llama-server's prompt cache reuses KV computed under a different LoRA scale
homelabpm profile

llama-server's prompt cache reuses KV computed under a different LoRA scale

#llamacpp #llm #selfhosted #lora
Comments
6 min read
Performance vs Quality: Gemma 4 E2B Quantization on an NVIDIA L4 and a Tesla T4
xbill profile

Performance vs Quality: Gemma 4 E2B Quantization on an NVIDIA L4 and a Tesla T4

#gemma #llamacpp #aws #quantization
Comments
11 min read
Running 100B+ MoE Models on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp
eme_gug_0821b41b948be6516 profile

Running 100B+ MoE Models on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp

#llm #llamacpp #gpu #machinelearning
Comments
8 min read
Running 100B+ MoE LLMs on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp
eme_gug_0821b41b948be6516 profile

Running 100B+ MoE LLMs on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp

#llm #llamacpp #gpu #selfhosted
Comments
7 min read
Accuracy vs Speed: Gemma 4 E2B Quantization Tradeoffs, BF16 to Q2_K

Hacktoberfest Open-Source AI Challenge: Ghost in the Machine Submission 👻🌼
xbill profile

Accuracy vs Speed: Gemma 4 E2B Quantization Tradeoffs, BF16 to Q2_K

#devchallenge #hf26challenge #gemma #llamacpp
11
Comments 10
15 min read
When Chat Templates Go Wrong
pielounw profile

When Chat Templates Go Wrong

#llm #opensource #ai #llamacpp
Comments
7 min read
Size Your Model to Where You Run It: Gemma 4 E2B and E4B on an AWS L4 and Two Small GPUs
xbill profile

Size Your Model to Where You Run It: Gemma 4 E2B and E4B on an AWS L4 and Two Small GPUs

#gemma #llamacpp #aws #cuda
8
Comments
10 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
⌂ 换网址