Local AI: What's the Power Bill?

If you're thinking about running large language models locally, have you considered how much electricity they're going to consume? We often talk about GPU prices, VRAM, and inference performance, but electricity is another cost that can add up, especially if you're running multiple GPUs for several hours every day. In this video, we'll use an estimated residential electricity rate of 15 cents per kilowatt-hour, which is a reasonable illustrative rate for the Pacific Northwest region of the United States. Actual rates vary depending on your utility provider and location. We'll compare several hardware configurations, from older NVIDIA GPUs to the latest RTX 5090, along with more power-efficient alternatives like AMD Strix Halo and Apple Silicon. And we'll look at what these systems cost to operate for a few hours per day, what happens if you leave them running continuously, and how electricity costs compare with cloud inference services. Let's start with the NVIDIA RTX 5090. The RTX 5090 comes with 32 gigabytes of VRAM and has a maximum graphics power rating of 575 watts. However, that doesn't mean it constantly consumes 575 watts during inference. Depending on the model, inference engine, and workload, actual consumption can be considerably lower. For our calculations, we'll assume a complete RTX 5090 workstation consumes approximately 700 watts under a relatively heavy workload. That includes the GPU, CPU, memory, motherboard, and other components. At 15 cents per kilowatt-hour, this works out to approximately 11 cents per hour. If you run the system for four hours every day, your monthly electricity cost would be around 13 dollars. At eight hours per day, that increases to approximately 25 dollars per month. Now, there's another interesting scenario. What if we reduce the RTX 5090's power limit? Some users report that power limiting their GPUs can substantially reduce electricity consumption without significantly affecting inference performance. For example, suppose we configure the RTX 5090 workstation to consume an average of 450 watts instead of 700 watts. Electricity consumption drops by approximately 36 percent. Now, four hours of daily operation costs around eight dollars per month, while eight hours costs approximately 16 dollars. Of course, the actual performance impact depends on the model and inference engine. You would need to benchmark your particular setup to determine whether the reduced power consumption is worthwhile. Next, let's look at older NVIDIA GPUs. A GTX 1080 comes with 8 gigabytes of VRAM. If we combine two GTX 1080 cards, we have 16 gigabytes of aggregate VRAM, although that memory isn't automatically pooled into one unified allocation. For this configuration, let's assume the entire workstation consumes approximately 600 watts. Running it for four hours per day would cost around 11 dollars per month. At eight hours per day, that increases to approximately 22 dollars. Now consider a workstation with four RTX 2080 cards. Each RTX 2080 has 8 gigabytes of VRAM, giving us 32 gigabytes of aggregate GPU memory. That's the same total VRAM capacity as a single RTX 5090, although the memory is distributed across four separate GPUs. And there's another important difference. The RTX 2080 is based on a much older architecture, so having the same aggregate VRAM doesn't mean it delivers comparable inference performance. Let's assume this four-GPU workstation consumes approximately 1,150 watts. At four hours per day, electricity costs around 21 dollars per month. At eight hours per day, that increases to approximately 41 dollars. That's noticeably more expensive than our RTX 5090 scenarios. And depending on the model and inference software, the older four-GPU system might also deliver significantly lower inference performance. This highlights an important distinction. When evaluating local LLM hardware, we shouldn't look at electricity consumption alone. We also need to consider how many tokens the system generates for each watt of electricity consumed. Now, let's compare these systems with more power-efficient alternatives. One interesting option is AMD's Strix Halo platform. Some configurations support 128 gigabytes of unified system memory, shared between the CPU and integrated GPU. That doesn't mean all 128 gigabytes are necessarily available to the GPU, but it allows substantially larger models to run without requiring a traditional high-VRAM discrete graphics card. In one community example, a Strix Halo system reportedly consumed approximately 120 watts while generating around 30 tokens per second for prose and 50 tokens per second for code. At that power level, running the system for eight hours per day would cost approximately four dollars and 32 cents per month. Apple Silicon is another alternative. Apple's unified memory architecture allows the CPU and GPU to access the same memory pool, making it attractive for running larger quantized models. For our comparison, let's assume an Apple Silicon workstation consumes approximately 150 watts during inference. At four hours per day, electricity costs around two dollars and 70 cents per month. At eight hours per day, that's approximately five dollars and 40 cents. Of course, these power figures are illustrative. Different Apple Silicon models can have very different performance characteristics and electricity consumption. And while these systems can be extremely power-efficient, they aren't necessarily faster than an RTX 5090. That's where the relationship between power consumption and inference speed becomes particularly interesting. Let's consider what happens if we leave these systems running continuously. For a stock RTX 5090 workstation consuming 700 watts, running 24 hours a day would cost approximately 76 dollars per month, or 920 dollars per year. A power-limited RTX 5090 workstation consuming 450 watts would cost around 49 dollars per month, or 591 dollars per year. Our two-GTX-1080 workstation would cost approximately 65 dollars per month. And the four-RTX-2080 workstation would cost approximately 124 dollars per month, or more than 1,500 dollars per year. By comparison, a 120-watt Strix Halo system would cost approximately 13 dollars per month, while our hypothetical 150-watt Apple Silicon workstation would cost around 16 dollars. These calculations assume the systems continuously consume their estimated workload power. In reality, an idle machine usually consumes considerably less. But the differences become significant when systems are operating continuously. Now, let's look at another way of measuring electricity costs. Instead of asking how much electricity a GPU consumes per hour, what if we calculate the electricity required to generate one million tokens? For this example, we'll assume a workstation consumes 400 watts while generating tokens. At 20 tokens per second, generating one million tokens takes approximately 14 hours. The electricity cost would be around 83 cents. At 50 tokens per second, generating one million tokens takes approximately five and a half hours, costing around 33 cents. At 100 tokens per second, that drops to approximately 17 cents. And at 200 tokens per second, generating one million tokens takes less than an hour and a half, costing just eight cents. This is an important observation. A faster GPU might consume more electricity per second, but it can also finish the same workload much sooner. Consequently, a more powerful GPU can sometimes consume less total electricity for the same number of generated tokens. These figures only consider continuous output-token generation. Real agentic workloads also involve prompt processing, idle time, tool execution, and repeated requests. But they provide a useful starting point for comparing local inference with cloud services. For example, Cerebras offers Qwen 3.8 27B inference at approximately 99 cents per million input tokens and one dollar and 49 cents per million output tokens. If a local workstation generates one million output tokens at 50 tokens per second while consuming 400 watts, the electricity cost would be approximately 33 cents. That's considerably lower than the cloud output-token price. However, there's an important catch. Electricity isn't the only cost of local inference. You also have to purchase the hardware. An RTX 5090 workstation can cost several thousand dollars, while a high-memory Apple Silicon or Strix Halo system can also represent a substantial upfront investment. And cloud services don't require you to purchase or maintain your own GPUs. For someone running local models occasionally, cloud inference might actually be cheaper. But for someone generating millions of tokens regularly, local inference can become increasingly attractive, especially if they already own the hardware. There are also benefits that aren't purely financial. Local inference provides greater control over your models, more privacy, and independence from cloud API pricing or subscription limits. And if you're running models on sensitive documents or proprietary code, those benefits might matter more than a few dollars in electricity costs. So, what can we conclude from these comparisons? For a single modern GPU running a few hours every day, electricity is usually a relatively small expense. Even a powerful RTX 5090 workstation might add only 10 to 25 dollars per month to your electricity bill under the usage scenarios we've discussed. But for multi-GPU workstations running continuously, particularly those using older hardware, electricity can become a meaningful operating expense. Power limiting and undervolting can help, but the most useful metric is ultimately how much work your system completes per unit of electricity. And remember that electricity prices vary considerably by region. Someone paying 30 cents per kilowatt-hour would face twice the costs we've calculated here. The best approach is to measure actual power consumption at the wall, benchmark your inference throughput, and calculate your cost per million tokens. Then compare that with cloud inference pricing, while accounting for the cost of purchasing and maintaining your hardware. Ultimately, running local LLMs isn't free. But depending on your hardware, electricity prices, and usage patterns, it can still be surprisingly inexpensive. And for many enthusiasts, the privacy, flexibility, and control of running models locally may be worth far more than the electricity they consume.