How much power does running Ollama or a local AI model actually use?

The inference burst itself is surprisingly cheap — a few seconds of prompt processing draws roughly the same wattage as gaming. The real cost is idle time. A GPU like an RTX 3080 or 4070 sitting in 'ready' mode with Ollama loaded can draw 20–80 W continuously, even when you haven't typed a single prompt. Over 23 hours of idle per day that adds up to several kWh, which shows up clearly on your electricity bill by month-end.

What is the GPU power limit trick and does it slow down AI inference?

The GPU power limit trick uses the nvidia-smi command — specifically 'nvidia-smi -pl [watts]' — to cap the maximum power your GPU is allowed to draw. Setting a limit around 60–70% of TDP (e.g., 150 W on a 250 W card) cuts idle and peak draw significantly while barely affecting inference speed: LLM token generation is mostly memory-bandwidth-bound, not compute-bound, so the speed drop is often under 5%. It's one of the highest-ROI power tweaks for local AI setups.

Why does my PC refuse to sleep when Ollama is running?

Ollama and similar local AI servers (LM Studio, ComfyUI) hold a power sleep request open so that long-running inference jobs aren't interrupted. Windows sees this as an active workload and blocks automatic sleep. You can confirm this by running 'powercfg /requests' in an elevated command prompt — it will list which process is holding the sleep blocker. The challenge is that simply overriding the blocker might kill a job mid-way, so you need smarter scheduling that sleeps only when the AI server is genuinely idle.

How can I run local AI jobs overnight without leaving my PC on and burning electricity all night?

The key is conditional sleep: let the PC stay awake while a job is genuinely running, then allow it to sleep automatically once the workload finishes. You can set this up by combining a sleep-blocker tool (or a batch script that calls SetThreadExecutionState) with a post-job hook that releases the blocker. PowerDoze lets you define rules — for example, 'if Ollama.exe is in the active process list, hold awake; otherwise sleep after 10 minutes of idle' — so you get proper overnight job runs without manually managing power settings every time.

It's Not the Seconds You Prompt, It's the 23 Hours It Idles: The Truth About Local AI's Power Bill

You type a question into your own chat model, hit enter, and a few seconds later there's an answer — private, free, no token meter ticking. You lean back feeling pretty clever. You stopped renting AI; you own it now.

Then the electricity bill shows up, and it's gone up more than a handful of seconds a day should explain. Here's the twist: those bursts of typing barely cost a thing. The real charge is racking up in the hours you're not even at the keyboard.

This one's about where the watts actually hide — the idle "kept warm, just in case" hours, the one GPU trick that cuts power without slowing your answers, and how to stop choosing between letting the PC sleep and watching an overnight job die.

Where the watts actually go

A local-AI machine lives in three states, and the expensive one isn't the one you'd guess:

Active inference. A high-end card at stock settings can pull 300–450 W while generating (diffusion and prompt processing push the high end). Sounds bad — but each request only takes seconds.
Model loaded, GPU idle. Between requests, the model can sit resident in VRAM — for minutes or indefinitely, depending on your keep-alive settings. The GPU idles noticeably higher than an empty desktop — and a multi-monitor, high-refresh setup raises that floor further.
Waiting, 24/7. If the box serves an API for your phone, your editor plugin, or Home Assistant, it can never sleep — so the whole machine (CPU, fans, board, drives) idles around the clock.

Here's the uncomfortable math: unless you're batch-generating all day, your "AI server" spends well over 90% of its life in states two and three, drawing power to do nothing. The inference you actually wanted is a rounding error next to the idle hours you didn't think about.

The expensive part isn't the few seconds you spend asking it something — it's the 23 hours it spends idling, waiting for you to ask. Like a home water heater: you assume the bill is the ten minutes of shower, but most of it is the tank reheating itself to scalding all day long, just in case you wander in. Local AI is the same — the inference you use is a rounding error; the bill comes from a full day of "kept warm, just in case."

The power-limit trick (you lose less speed than you think)

r/LocalLLaMA has known this one for a while: LLM inference is mostly memory-bandwidth-bound, not compute-bound. Cap the GPU's power and the clocks drop, but tokens/sec barely moves. Community tests on cards like the 3090 and 4090 consistently find that cutting the power limit by 25–35% costs only a few percent in tokens/sec — while cutting heat and fan noise along with the watts.

LLM inference is like a queue at a food counter: the bottleneck is how fast the serving window moves (memory bandwidth), not how many cooks are in the back (compute). So you can send a third of the cooks home (cut power 25–35%) and the food comes out almost as fast — because the cooks were never the holdup. What you save is the wages of all those idle cooks (the watts) and the heat they gave off; the meal still arrives on time.

The manual version, in an admin shell:

nvidia-smi -pl 300 — cap the card at 300 W (your card's valid range: nvidia-smi -q -d POWER).
Benchmark your own setup before and after — five minutes with your usual model tells you the real cost. Image generation is more compute-bound than LLMs, so expect a bigger hit there and cap less aggressively.
Note: the limit resets on reboot, so it needs a startup task to stick.

The sleep dilemma (the other half of the problem)

Power limits handle the watts. Sleep is where Windows gets actively hostile to local AI:

Long jobs die. A fine-tune, a big batch run, an overnight embedding job — Windows' sleep timer counts keyboard and mouse, not CUDA. No input for 30 minutes → "idle" → sleep → hours of GPU work gone.
Serving dies too. If the box sleeps, your Ollama endpoint goes dark and every client breaks.

Windows decides whether you're "using the PC" from one thing only: is someone wiggling a keyboard or mouse? Trouble is, your GPU grinding through an overnight training run doesn't count as "in use" — like a guard who only watches the front door for footsteps, misses the workshop out back running full tilt all night, and flips the master switch at closing time, erasing hours of work.

So everyone applies the same fix: disable sleep entirely. Which is how a privacy decision quietly becomes a machine that burns full idle power forever. Each step was rational; the sum is the worst of both worlds.

The static-settings treadmill, AI edition

You can wire all of this up by hand — a startup task for nvidia-smi, sleep disabled, maybe a scheduled script that tightens the power limit at night and loosens it for daytime use. It works, the same way the homelab version works: until the day you're gaming at 8 PM on a GPU still capped for overnight serving, or a Windows update quietly resets something and you find out a month later on the electricity bill.

Automating it

This is exactly the kind of problem PowerDoze exists for. The pieces, mapped to local AI:

Keep-awake whitelist — put ollama.exe, koboldcpp.exe, or your training script's process on it. The PC won't sleep while they're running, and you can turn normal sleep back on for the other 20 hours a day. Long jobs survive; idle nights don't burn.
Power modes with an NVIDIA power limit built in — bundle GPU watts, CPU cap, and cooling policy into modes, then schedule them: full power during the day, a capped "serving mode" overnight. The GPU limit is re-applied automatically after every reboot.
Analytics (part of Pro) — estimated watts, kWh, and cost over time, so "did capping the GPU actually matter?" gets a number instead of a feeling.

Honest boundary: PowerDoze doesn't schedule your AI jobs, doesn't touch VRAM or model loading, and doesn't make inference faster. It manages the part around the AI — sleep, power limits, schedules, and measurement — which happens to be where the waste lives.

Schedules, power modes, the whitelist, and monitoring are free, no account required. Everything runs locally — which, if you're running local AI for privacy reasons, is presumably a feature you care about.

Who doesn't need this

If your models run in the cloud, your power bill is someone else's problem. If you do inference on a laptop, the battery already polices waste. And if your AI box is a dedicated headless Linux server, PowerDoze can't help — it's Windows-only; nvidia-smi plus a systemd timer is your friend. This is for the most common setup of all: the Windows desktop that is both your daily machine and your AI box — the one that needs full power at 2 PM and almost none at 2 AM, and currently gets full power at both.

Download PowerDoze free →

Nisonxi

I'm Nisonxi, the developer behind PowerDoze. I built it because my own Windows desktop idled all day at near-full power and no existing tool could read the situation and switch on its own. This blog is my notebook from the journey.

About the author and PowerDoze →