That raised an obvious question: if consumers are entering the same details, why don’t comparison tools agree?
According to an Electricity Authority spokesperson, differences largely come down to the data and assumptions behind the estimates.
“The main reasons are the underlying dataset and model used to produce estimates of power usage and savings results.”
Billy allows users either to upload a recent bill or answer questions about their household and energy use. Its estimation model uses household and consumption data from 27,000 New Zealand households collected in late 2025.
The Electricity Authority said some plans reward households for shifting electricity use to cheaper times of day – known as time-of-use pricing – and that different tools may apply different assumptions about how much households are realistically able or willing to change their habits.
Billy allows users to indicate whether they are open to changing when they use electricity and reflects estimated levels of flexibility based on household information and usage patterns.
The methodologies of power comparison tools mean there are bound to be some differences in results. Image / Supplied
Consumer NZ’s Powerswitch approaches the challenge differently.
Manager of the platform Paul Fuge said Powerswitch’s comparison engine has been developed and refined over more than 25 years and estimates household electricity use either from uploaded bill data or household profiling.
Where customers do not provide bill information, Powerswitch applies an Annual Average Profile for residential consumption and adjusts that estimate using factors such as household size, location and household demand drivers including heating and hot water.
Fuge said more information improves precision, but does not necessarily alter retailer rankings.
“Using actual consumption data from a household’s bill will always produce the most accurate cost projections,” he said, although Consumer NZ’s analysis suggests additional detail does not typically change the relative ranking of retailers.
One unexpected difference was not just the recommendations themselves but how the platforms handled missing information.
Powerswitch gathered more household information before generating results, while Billy produced an initial estimate and then offered opportunities to refine recommendations with more detailed inputs afterwards.
That difference may partly explain why comparison tools can feel inconsistent. They are not necessarily asking the same questions at the same stage or making the same assumptions when information is incomplete.
Chat GPT didn’t really want to get too drawn into which power companies were cheapest – and redirected us to more formal power comparison tools. Photo / Silas Stein, dpa, via Getty Images
Then there was ChatGPT.
When asked the same question using the same household assumptions, ChatGPT responded that there was not enough information to confidently recommend the cheapest provider without actual electricity usage data.
Instead, it suggested using specialist comparison platforms and requested additional information including electricity consumption (kWh), contract status and whether the household was interested in bundled services.
That response highlighted a key distinction. While AI could explain which factors influence electricity costs, it did not attempt to estimate household consumption or calculate likely annual spending.
There was another point of agreement across the different approaches: switching is not always as simple as choosing the lowest number.
The Electricity Authority recommends checking whether you are already on a fixed-term contract before switching, as some plans may include break fees.
It also notes that some pricing structures depend on having the right meter setup. Time-of-use plans often require smart meter data and may only deliver savings if households can realistically shift usage.
Consumer NZ made a similar point, noting that realised savings depend partly on whether household behaviour matches the assumptions built into the comparison.
So what should consumers actually do?
The answer from all three approaches turned out to be surprisingly similar: better inputs generally lead to better outputs.
Both Billy and Powerswitch recommend uploading an actual bill where possible rather than relying solely on estimated household behaviour.
Comparison tools, it turns out, are less like calculators and more like forecasts: the more complete the information, the more useful the estimate.
Herald contributor Nikki Birrell has worked in food and travel publishing for nearly 20 years.