How accurate are the VRAM numbers?

They're weights + KV cache at your chosen quant and context length. Real deployments need 1-2GB headroom for CUDA context, framework overhead, and memory fragmentation — especially on Windows and with long contexts. Treat them as a floor, not a ceiling.

Can I use it without a GPU?

Yes. There's a CPU-only path that estimates RAM requirements and throughput based on memory bandwidth class. Expect single-digit tokens/s for anything above ~8B on DDR4 dual-channel.

How is the KV cache calculated?

Per-layer KV heads × head dim × 2 (K and V) × context length × bytes-per-element, summed across layers. GQA/MQA architectures get correctly reduced KV footprints — which is why a Qwen 2.5 14B at 32K context shows under 1GB while older MHA models balloon past 3GB.

Why does a 141B MoE like Mixtral 8x22B show ~85GB VRAM but high tokens/s?

All experts must be resident in memory (total param count drives VRAM), but only 2 experts activate per token, so compute — and therefore tokens/s — tracks active params (~44B), not the full 141B. Same logic applies to DeepSeek, Qwen MoE, and GPT-OSS.

Which inference engines are modeled for throughput?

llama.cpp/Ollama, LM Studio, KoboldCpp, MLX (oMLX/vMLX), vLLM, TGI, and SGLang. Expect llama.cpp-family engines to dominate on consumer single-GPU and Apple Silicon, while vLLM and SGLang win at batch>1 on datacenter cards.

Directory/Search/LLM Inference/Run This LLM

Run This LLM

Run This LLM is a hardware compatibility calculator for local language models. Enter your GPU VRAM, system RAM, RAM type, hardware vendor (NVIDIA, AMD, Apple Silicon, Intel Arc, Snapdragon) and inference stack (llama.cpp, vLLM, MLX, TGI, SGLang) to see which of 506 indexed models will run on GPU, run CPU-offloaded, or won't run at all — with tokens/sec estimates and VRAM broken down by weights vs KV cache. The index spans text, code, reasoning, vision, audio, image, video, 3D, embedding, reranker and time-series models, with benchmark columns (MMLU, HumanEval, GPQA, SWE-bench, MTEB) for side-by-side comparison. Quantization, context length and KV cache precision are adjustable, so you model real deployment tradeoffs instead of guessing. A free Chrome sidebar extension gives the same lookup without tab switching; a paid custom-build service specs and ships a machine for you.

Visit website

Added on

Nov 4, 2026

506 modelsVRAM calculatorKV cache mathChrome extensionOllama/llama.cppMulti-modalBenchmark sortingCustom builds

Product information

Everything worth knowing about Run This LLM

The complete picture — from everyday features to the technical detail builders and IT folks dig for.

Core features

What it actually does

What Can I Run? (hardware-first lookup)

Enter your GPU VRAM, system RAM, RAM type, and hardware backend and get a full breakdown of which of the 506 indexed models run fully on GPU, which run with CPU offload, and which simply won't fit.

What Do I Need? (model-first lookup)

Pick a model and the site tells you the minimum VRAM and system RAM required, split into weights plus KV cache overhead, so you know exactly what to buy before you buy it.

506-model index across every modality

Text, code, reasoning, vision, medical, finance, legal, science, bio/genomics, embedding, reranker, image gen, video gen, 3D gen, audio, and time series models — each with VRAM footprint and estimated tokens per second for your hardware.

Benchmark comparison table

Sort and compare models head-to-head on MMLU, HumanEval, MATH, GPQA, SWE-bench Pro, Terminal-Bench, HLE, MTEB, BEIR, GenEval, and VBench instead of guessing from a name and parameter count.

Free Chrome sidebar extension

Look up hardware requirements for any model directly from your browser sidebar while you're reading a model card or release post — no tab switching, no signup.

Technical capabilities

KV cache and context-length math

Requirements aren't just model weights. The calculator models context length from 2K up to 1000K and KV cache precision (FP16, Q8, Q4) separately, then adds them to the weight footprint — which is why a '4.8 GB' model can actually need 5.3 GB or far more at long context.

Runtime- and backend-aware estimation

Adjusts the estimate for your inference stack (llama.cpp/Ollama, LM Studio, KoboldCpp, MLX LM, vLLM, TGI, SGLang), your quantization (Q4, Q8, FP16), and your hardware backend (NVIDIA CUDA, NVIDIA Unified/DGX/Jetson, AMD ROCm, AMD Strix Halo, Snapdragon X, Intel Arc SYCL, Apple Silicon, CPU only), plus single- vs quad-channel DDR4/DDR5 memory bandwidth effects.

Must watch videos

Your Hardware Can Run AI — You Just Need the Right Model

22:40

Tutorial22:40

Your Hardware Can Run AI — You Just Need the Right Model

Not sure which AI model fits your hardware? In this video, I'll show you exactly how to find LLMs that match your GPU, CPU, and RAM — no wasted downloads, no frustrating crashes. 🔧 What you'll learn: How to read your hardware specs for LLM compatibility Where to find and filter models that fit your setup The tools and sites I use to match hardware to models 00:00 intro 01:17 runthisllm.com 07:05 Setting up Gemma 4 MoE 13:00 Temperature and Systemprompt 14:10 Code testing 16:32 Puzzle test 20:54 Conclusion Whether you're running a high-end GPU or a modest laptop, there's an LLM out there for you — let's find it. 🔔 Subscribe for more local AI guides 👍 Like if this saved you time 💬 Drop your specs in the comments — I'll help you find a match! #LocalAI #LLM #RunAILocally #AIForEveryone #OllamaLLM

Day 3 | How to Build Vision AI Agents: Screenshots, VLMs & Browser Control

21:00

Tutorial21:00

Day 3 | How to Build Vision AI Agents: Screenshots, VLMs & Browser Control

Build a vision AI agent in Python that can see a webpage, take screenshots and act on what it observes. This is Course #3 of the Build AI Agents series — we move from text-only agents to agents that use vision language models (VLMs) to understand images and control a real browser. You'll download and process images with Python, feed them to a vision model, then automate Chrome with Selenium and Helium so the agent can find elements, close pop-ups, scroll, take screenshots and remember what it saw between steps. WHAT YOU'LL LEARN • What a vision agent is and why VLMs matter • Downloading and processing images with requests • Sending images to a vision model from a Python agent • Browser automation with Selenium and Helium • Finding page elements by text and handling missing ones • Screenshots, timing delays and converting PNG output • Agent memory: observations carried between steps • Giving an agent tools to click, scroll and navigate • What to do when the agent fails (and how to work around it) Prerequisites: Courses #1 and #2 — the agent loop and tool calling are already covered there. CODE https://github.com/KNehe/ai_agents FULL COURSE (playlist) https://www.youtube.com/playlist?list=PLIGyG15z2KLk OpenAI Billing https://platform.openai.com/settings/organization/billing/overview Can my machine run this LLM? https://runthisllm.com/

Day 1 | How to Get Started with Building AI Agents: Ollama, Hugging Face & OpenAI

1:13:41

Tutorial1:13:41

Day 1 | How to Get Started with Building AI Agents: Ollama, Hugging Face & OpenAI

Learn how to build AI agents with Python from scratch — no prior agent experience needed. This first course sets the foundation for the whole series: what agents actually are, how LLMs power them, how tools and actions work, and how to get your environment running with Ollama, Hugging Face and OpenAI. You'll understand the agent workflow (thought → action → observation), why an LLM is the "brain" of an agent, how messages are structured, and how to turn a plain Python function into a tool an agent can call. We finish by building a flight-search agent step by step in a Jupyter notebook. You should be comfortable with basic Python before starting. WHAT YOU'LL LEARN • The course roadmap and how the four units build on each other • How AI agents work: tools, actions and the agent loop • LLM fundamentals: transformers, embeddings, tokens and context length • Messages: system, user and assistant roles explained • Tools = Python functions, decorators, and tool calling • Thought / chain-of-thought, JSON agents vs code agents • Full environment setup with Ollama, Hugging Face, OpenAI and Jupyter • How to check if your machine can run a local LLM • Build a working flight-search agent end to end CODE https://github.com/KNehe/ai_agents FULL COURSE (playlist) https://www.youtube.com/playlist?list=PLIGyG15z2KLk OpenAI Billing https://platform.openai.com/settings/organization/billing/overview Can my machine run this LLM? https://runthisllm.com/ If this helped, like and subscribe — Unit 2 builds directly on it. #AIAgents #Python #LangChain

Who uses it

Real use cases, no hype

GPU Purchase Sanity Check

Verify the card you're about to buy actually fits your target model at your target context length.

Best Model for Your VRAM

Filter 506 models down to what fits in 8, 12, 24, or 80 GB without offloading.

Quantization Tradeoffs

Compare Q4 vs Q8 vs FP16 VRAM cost against the quality hit you're willing to eat.

MoE Deployment Sizing

Understand why a 141B Mixtral needs 85GB resident but only runs at active-param speed.

Inference Stack Comparison

Estimate throughput differences between llama.cpp, vLLM, MLX, and TGI for the same model.

Multimodal VRAM Scouting

Check fit for SDXL, FLUX, Wan video, Whisper, and TTS pipelines before downloading 20GB.

Custom Rig Spec'ing

Commission a build tuned to your exact model, context, and concurrency needs.

What's great

506 models indexed — text, code, vision, audio, video, 3D, embeddings
Instant verdict: runs on GPU, CPU-offloads, or won't run at all
Free Chrome extension and PWA for sidebar lookups
No signup, no paywall, no fluff — pure spec lookup

Technical strengths

Honest MoE accounting: total weights resident vs active params for tokens/s
KV-cache precision modeling (FP16/Q8/Q4) with context-length scaling
Covers llama.cpp, vLLM, TGI, SGLang, MLX, LM Studio, KoboldCpp plus CUDA/ROCm/SYCL/Metal backends

Where it falls short

Tokens/s figures are modeled estimates, not live benchmarks
No pricing transparency on the custom-build upsell
Assumes Q4_K_M by default; other quant formats loosely modeled
Benchmark columns (MMLU, HumanEval) have no release dates or versioning

Technical limitations

No batch-size or multi-user concurrency modeling for vLLM/TGI
Ignores driver, CUDA context, and framework overhead headroom per stack
Single-user throughput only — datacenter serving numbers will diverge sharply

For technical folks

The deep-dive specs

Models indexed

506

Modalities covered

Text, code, image, video, audio, 3D, embeddings, rerankers, time series

Quantization options

Q4 (smallest), Q8 (balanced), FP16 (full)

Context length range

2K to 1000K

KV cache precision

FP16, Q8, Q4

Supported inference runtimes

llama.cpp / Ollama, LM Studio, KoboldCpp, MLX LM (oMLX/vMLX), vLLM, TGI (HuggingFace), SGLang

Supported hardware backends

NVIDIA (CUDA), NVIDIA Unified (DGX / Jetson), AMD (ROCm), AMD Strix Halo (Unified), Snapdragon X (Unified), Intel Arc (SYCL), Apple Silicon, CPU only

RAM configurations modeled

Auto (from size), DDR4 Single-Channel, DDR4 Dual-Channel, DDR5 Dual-Channel, DDR5 Quad-Channel, DDR5 8-Channel

Benchmark columns

MMLU, HumanEval, MATH, GPQA, SWE-bench Pro, Terminal-Bench, HLE, MTEB, BEIR, GenEval, VBench

Default quant for index

Q4_K_M

FAQ

Includes technical Q&A

Yes. The model database, hardware calculator, and Chrome extension are free and account-free. The site monetizes through a custom machine build service if you want someone else to spec and ship the rig.

Traffic Insights

Monthly Visits

—

November 2026

Growth Rate

vs last month

Dominance

in Search

Top Country

—

Top Countries Breakdown

—

Growth Trend

Stable

Traffic has remained stable.

Pricing overview

Unknown Pricing

Run This LLM Pricing

Run This LLM uses a unknown pricing structure. Real-time rates, custom enterprise quotas, and plan details are managed directly on their official platform.

Check Pricing on Run This LLM

Reviews