On a server, power and cooling are someone else’s problem, handled by the rack. On an edge device, they are your problem, and they often decide the design long before model accuracy does. A model that runs beautifully on a development board can fail in a sealed enclosure in summer.

Two Budgets, Not One

Power and heat are linked but not identical, and each needs its own budget.

The power budget is how much energy the device can draw. For battery products it determines runtime. For wired products it is limited by the supply, the connector or the standard being used, such as power delivered over a network cable. Either way it is a hard ceiling.

The thermal budget is how much heat the enclosure can shed without the chip getting too hot. Essentially all electrical power consumed becomes heat, so the thermal budget is bounded by the power you draw, the surface area, the airflow and the ambient temperature.

You can satisfy one budget and violate the other. A device may draw little average power yet overheat because the heat concentrates in a small sealed volume.

Peak, Sustained and Average

Specifications often quote a single power figure. You need three.

• Peak power is the short-term maximum, which the supply and any regulators must handle without voltage dips.
• Sustained power is what the system draws under continuous load, which determines steady-state temperature.
• Average power over the duty cycle determines battery life.

Designs fail when these are confused. A part rated for a certain power in a datasheet may reach that figure only briefly before thermal limits force it to slow down.

Thermal Throttling in Practice

Most edge chips protect themselves by reducing clock speed when they get too hot. This is thermal throttling. It prevents damage, but it changes performance, and the effect is sneaky: a short benchmark looks great, while a long run slows down as the enclosure heats up.

For an application with a latency requirement, throttling can push response times past the limit after the device has been running for a while. The device passes a quick test and fails in the field.

So test over time, at the worst ambient temperature the product must support, in the real enclosure, with the real workload. Anything less measures a different system.

A Hypothetical Example

Imagine an outdoor camera running a detection model continuously. On the bench, with an open board in a cool room, it hits its target frame rate with room to spare. In the sealed housing on a warm day, the temperature climbs for an hour and then the chip begins throttling, dropping frame rate below the requirement.

Possible responses, from cheap to expensive:

1. Reduce workload: lower input resolution, skip frames, or run detection only when motion is present.
2. Use a smaller or quantized model that does the same job with less computation.
3. Improve heat transfer: a thermal pad to the housing, a larger heat spreader, or a metal enclosure used as a heatsink.
4. Choose a more efficient accelerator.

Notice that the first two are software changes. Many thermal problems are best solved by doing less work, not by cooling harder.

Levers for Reducing Power and Heat

Lever	Effect	Watch out for
Duty cycling (run inference only when needed)	Lowers average power and heat	Added wake-up latency
Smaller or quantized model	Less compute and memory traffic	Accuracy must be revalidated
Lower input resolution or frame rate	Proportionally less work	May miss small or fast objects
Dedicated accelerator instead of CPU	Better efficiency per inference	Operator support limits
Power modes or frequency limits	Caps peak and sustained power	Reduces peak throughput
Better heat path to enclosure	Higher sustained performance	Mechanical design and cost

How to Measure

Good numbers come from simple, disciplined measurement.

1. Measure at the supply, not just the chip. Include regulators, memory, sensors, radios and any other components. The accelerator is often not the largest consumer once a radio is active.
2. Record power across the whole duty cycle, including idle, wake-up and inference phases.
3. Log temperature and clock speed together during a long run, so you can see exactly when throttling begins.
4. Test in the final enclosure and at the hottest ambient condition the product will face, with margin.
5. Repeat with realistic data, since inputs can change how much work the model does.

Use the board’s documented power modes, and check the vendor’s current documentation for how they behave, because defaults are sometimes tuned for demos rather than products.

Battery-Powered Considerations

For battery devices, energy per inference matters more than peak speed. Counter-intuitively, finishing quickly at a higher power and then sleeping deeply can use less total energy than running slowly, depending on the chip’s idle behaviour. The only reliable way to know is to measure both patterns on your hardware.

Also account for battery behaviour at temperature extremes. Capacity and allowable discharge rate vary with temperature, so cold and hot conditions both deserve testing, and the vendor’s cell documentation will say what to expect.

Frequently Asked Questions

Why does my device get slower after running for a while? Probably thermal throttling. Log temperature and clock speed together to confirm.

Is a fan always the answer? Not necessarily. Fans add power, noise, dust ingress risk and a failure point. Passive designs that conduct heat into the enclosure are often preferable for sealed products.

Does lower power always mean lower heat? Roughly, since consumed power becomes heat. But where the heat concentrates and how well it can escape also matter.

Should I size the supply for peak or sustained power? For peak, with margin, because voltage dips during bursts can cause resets or errors. Sustained power determines thermal design.

How much margin should I allow? Enough that the worst realistic condition still works. Decide the margin from your test results rather than a generic rule.

Next Step

Put your device in its final enclosure, run the real workload continuously at your hottest expected ambient temperature, and log power, temperature and clock speed. The point where performance starts to fall tells you exactly how much budget you actually have.