Electricity tariffs and variability in market prices

This is the first of several articles based on studies of electricity markets in the UK, Europe and the USA which I will be publishing on this Substack over the next 2-3 months. This article is extracted from a paper titled Variability in Market Prices and Options for Electricity Tariffs published by the Renewable Energy Foundation at January 2025. You can access the full version of the paper here.

Price variability is an inherent feature of electricity markets. It reflects and signals variations in the marginal cost of running different generating plants to meet variations in the demand for electricity – either within days or weeks or across seasons. Wholesale markets for power have highlighted the extent of such variability but they have not created or exacerbated such variations. The figure below shows average within-day variation in market prices in the UK wholesale market in the last quarter of 2023.

As shown in the next figure this variability – measured by the monthly coefficient of variation of prices - has been increasing steadily over time.

In the past – and still in some parts of the world - it was expected that vertically-integrated electricity companies would absorb this price variability. This was feasible while they had monopoly control of electricity systems, but once the electricity sector was unbundled to separate generation, networks and energy supply it became difficult or impossible to sustain the pooling which allowed monopoly utilities to suppress price variability.

Even when electricity systems are run by monopoly utilities, they have a strong incentive to offer multiperiod tariffs to encourage electricity users – both businesses and households – to shift some or most of their usage from periods when generation costs tend to be high to period when they tend to be low. Hence, peak/off-peak tariffs (Economy 7 tariffs in the UK) are offered almost everywhere.

The main constraint on the type of variable tariffs offered has been the capabilities of electricity meters. In many countries 50 years ago, it was necessary to install separate meters and electricity circuits for peak and off-peak use. The issue was partly one of cost. Large customers – primarily businesses - could choose to install more sophisticated meters, but they were not seen as being economic for smaller users.

This constraint was gradually removed with the introduction of more sophisticated electronic “smart” meters with network connections in place of older mechanical meters. For example, the Italian electricity utility Enel implemented a nationwide program of meter upgrades in the early 2000s using meters that have measure consumption in 3 periods as standard.

There have been two primary reasons for countries to promote multiperiod tariffs, which lead to different tariff structures. The first is the availability of large amounts of baseload generation with (very) low variable costs. This was often nuclear power but, in some countries, hydro power or thermal plants burning lignite or very cheap coal would also qualify. In such cases, the goal was to increase demand during the night when it might otherwise fall below the output from plants whose output could not easily be reduced. The second was the need to operate plants with high variable costs to meet peak demand, either for air conditioning or heating, during periods of high or low temperatures.

The first reason was generally more important in Europe, especially NW Europe where air conditioning demand is low. The second reason was important in North America and Asia where air conditioning is more widespread. However, in most developed countries three developments have increased the underlying variability in market prices and pressure to pass at least some of that variability through to energy consumers.

First, the rapid increase in generation from intermittent renewable generators – primarily solar and wind plants – has increased the medium-term variability of wholesale prices. This trend is clearly apparent in the GB market and may be observed by focusing on what I call market generation. There are several reasons for this. Many renewable generators are embedded, i.e. connected to distribution networks, so that grid demand is total demand minus embedded generation. Grid-connected renewable generators have zero marginal costs and most receive subsidies which mean that they can earn an operating margin even if market prices are zero or negative. Hence, intermittent and subsidised renewable generation (ISG) is always dispatched before other forms of generation.

The residue – i.e. total demand minus the sum of embedded generation and ISG – is market generation which responds to and determines market prices. The higher the level of market generation, whether due to higher total demand or lower non-market generation, the higher on average will be the level of market prices. Over the period from 2015 the variability of market generation and, thus, market prices has nearly doubled. The variability in non-market generation has remained constant but as non-market generation has displaced market generation its variability has an increasing impact on market prices.

Similar trends are visible in other European countries, most notably Germany. The German wholesale market has experienced an increasing frequency of negative market prices. These are a perverse consequence of subsidies, particularly to small solar producers, which encourage renewable generators to continue exporting power to the grid even when market prices are low or negative. Negative market prices are merely the most visible manifestation of increasing price variability and are becoming more frequent both in the UK and in markets linked to Germany.

The second development has been the separation of energy supply from generation which means that an increasing proportion of electricity supplied to final customers is either traded on power markets or is purchased on terms that are affected by wholesale market prices. Many final customers may prefer to pay an electricity price that does not vary by time of day and is fixed for anything from 3 to 12 months or longer. To offer such contracts energy suppliers must pay for price insurance, either by hedging or entering into power purchase agreements. With increasing variability of market prices, the cost of price insurance increases and this cost is passed on to final consumers.

During the early period after the liberalisation of energy markets, some generators took the view that the combination of generation and energy supply provided a natural hedge for the variability in energy prices. Low market prices for output were offset by higher margins in their energy supply businesses and vice-versa. Over time this model was undermined by the growth in subsidised renewable generation whose revenues were less affected by market prices and by the entry of competing energy suppliers with limited or no associated generation. Energy supply is now seen as a very competitive and often unprofitable business that is avoided by many companies who invest in and operate both renewable and thermal generation facilities.

The third development has been the gradual switch to smart meters. The UK is behind European countries such as Italy, Spain and all of Scandinavia in this respect. One major benefit of smart meters for energy suppliers is the elimination of manual meter reading. This allows energy suppliers to introduce flexible or dynamic prices that are linked in various ways to market prices.

In both Italy and Spain, many final customers are on flexible multiperiod tariffs under which (a) the standard price they pay is linked to a monthly index of the market price, and (b) multipliers are applied to this standard price for the prices in peak (a multiplier > 1) and off-peak (a multiplier < 1) periods. Customers pay either a fixed daily charge or a per kWh to cover network costs, the supplier’s costs, and various levies and taxes. Such tariffs are more complicated than the familiar UK tariff but they are much more transparent about how market prices translate to what is paid by final customers. From the perspective of energy suppliers, such flexible tariffs pass through a large portion of market risk to customers and reduce the cost of market insurance that would otherwise be built into fixed prices.

Most Scandinavian countries have gone further by promoting the adoption of dynamic pricing. In this case final customers pay a price per unit of electricity used that is equal to the wholesale market price for that period. In addition, they pay separate charges to cover network and other costs based on total monthly consumption and the capacity connection. Dynamic pricing means that final consumers are fully exposed to market price variability but typically the average price paid is much lower than customers in the UK pay. Energy suppliers bear volume risk but not market price risk.

Another consideration is that the increase in heavily subsidised renewable generation has greatly increased the gap between market prices and average tariffs charged by electricity suppliers. In countries where flexible and dynamic pricing is widespread, regulators put great weight in ensuring transparency about how electricity bills are made up. By contrast, Ofgem and UK governments have focused on the headline composite price per kWh used, allegedly because this facilitates simple comparisons and thus competition.

Perhaps coincidentally, the emphasis on a single final price hides the extent to which UK electricity bills are primarily determined by levies on consumption and network charges, both of which have risen rapidly because of the growth in renewable generation. Other countries have no difficulty in ensuring that comparison websites show the expected total cost of electricity consumption, even when tariffs are far more transparent.

The issue facing UK policymakers is that plans to decarbonise the electricity system by 2030 will certainly increase the variability of market prices. Preserving the current regulatory arrangement of setting a cap on prices every quarter will incur higher insurance costs, pushing up the premium over average market prices. This mechanism may not be viable if the hedging market does not have sufficient capacity. Thus, moving to flexible prices in which prices are reset every month with a defined link to the average market price in the previous month would make sense. Currently, his option is limited by the slow progress of the program to install smart meters in all customer premises.

The fiasco that is the government’s smart meter program goes beyond technical issues. It has reinforced the public’s general distrust of energy suppliers linked to their record of poor customer service and inept administration. The distrust fuels suspicion that smart meters may be abused to ration electricity or to monitor its use during episodes of constrained supplies. Even though such concerns are distorted and exaggerated, they are fed by an approach to developing policies for the electricity market which relies heavily on highly optimistic assumptions reinforced by PR and lobbying.

Despite the resistance from policymakers and public suspicion, the increase in the variability of market prices that will accompany the commitment to increase the share of total generation supplied by intermittent renewables is likely to force a transition from fixed tariffs to flexible multiperiod and even dynamic tariffs. The question, then, is whether this change can be managed properly and presented to the public as a reasonable option.

Given the pattern of short-sighted and incompetent policymaking in the energy sector over the last two decades, this would be asking for a radical shift. It is probably too much to hope for significant improvements in the next 5 or even 10 years. As Sam Freedman and many others have argued the UK is patently a failed state and its decision-making processes are unlikely to change without some major crisis.[1] Hence, the evidence and analysis in this paper is presented partly for future reference when the issues discussed here come to the forefront of wider public debate, and partly to promote a wider understanding of how electricity markets work.

[1] See Sam Freedman – Failed State: How Nothing Works and How We Fix It, London: Macmillan, 2024. The diagnoses offered by Freedman are a long way from practical remedies that can be implemented over the objections of those who have little to gain from radical change.