Renewable Energy Power Grid

Renewable Energy Meets Power Grid Operations

[Total: 4    Average: 5/5]

The electric power industry faces much turmoil in the coming decades.  The business model of of the electric utility company (public or private) may not survive. In addition, the power needs of high density cities may diverge significantly from non-urban areas causing political turmoil and technical hurdles. 

The increasing share of renewable energy will be a major factor in the turmoil.  It can be viewed as a coming of age. When the percentage contributions of solar & wind to the power grid were small enough, it didn’t matter how they behaved.  But as their share grows, renewables become significant, and ultimately dominant.

There’s  nothing new about turmoil, or upheavals.  Technologists are sometimes proud of the disruptive nature of their inventions.  But in the case of electric power, we must never stumble in the reliable supply of electricity, not even for brief periods.  A total revamp while maintaining continuous operation is like reupholstering the seats in your car as you drive down the highway.

I am talking about the future.  No I do not own a crystal ball. My approach is to describe the essential properties of a reliable power grid that must remain invariant despite turmoil, conflict and invention.  If renewables are to dominate the future, then renewables much take over these necessary functions. This article may be a bit wordy, but bear with me.

Design and operation of the bulk power grid has been my career. My examples come from the State of New York, USA.   In other states and countries and other utility structures, the nomenclature and ownership details may seem very different, but the things I describe here are analogous to power grids anywhere in the world.


This article discusses the bulk power system and the wholesale markets, not distribution not retail, not personal use or personal costs, or personal production.  To understand this article, you must think of energy supply at the level of continents, not individual homes or neighborhoods or even individual countries.

Operating the Grid — Balance Generation and Load

The power grid is demand driven.  Customers create demand, for example by turning on a light.  The grid must satisfy this demand instantaneously, creating a balance:  generation=load+losses. For purposes this article, I will neglect losses and say generation=load.  It would be correct to say production=consumption, but I prefer generation=load. Our foundational physics principle is the is conservation of energy.

The power grid stores no energy, so this balance must apply instantaneously.  Energy storage we can treat as external devices attached to the grid.  They are not part of the grid itself. In an earlier Insights article, I discussed maintaining the balance over a wide range of time intervals; the longest interval ##10^{18}## times longer than the shortest.  In this article we’ll zero on on a mere ##10^{5}## range of time intervals from about a minute to a year.

So that’s it.  Customers demand power whenever and wherever they want to.  The industry must assure that generation matches load and deliver that energy to the customer’s location in all reasonable circumstances at an affordable price.  Pretty simple huh?

The Solution: Find the Optimum Allocation Among Multiple Sources

It is necessary to have multiple resources and multiple ways to meet the demand.  Each resource has different properties, a geographical location, and a price curve.   Therefore, the technical problem is reduced to simple optimization. Find the optimum combination of resources to make generation=load subject to constraints.   The familiar Simplex method is widely used to do that. We repeatedly solve that problem once every hour for the next 24 hours, once every 15 minutes for the next hour, and once every 5 minutes for the next 15 minutes, and in a limited sense once every 2 seconds for the coming 5 minutes.

Now let’s talk about the scale of the problem.  The New York grid is a network with roughly 20K nodes and 100K branches.  It is a linear passive electric circuit. The nonlinear boundary conditions are the power flows entering/leaving the grid at each node.  Ohm’s Law, Kirchoff’s Laws and matrix methods are the primary tools used to solve it. During optimization, every vertex considered must calculate the voltages at all nodes, and the power flows in all branches.   

The geographical and electrical locations are critically dominant  factors. We can and we do generate electricity near Hudson’s Bay in Canada that is consumed in NYC, but we can’t do it as reliably as power generated locally on the island of Manhattan.  

As a prerequisite to the circuit solution, we need to decide what circuit is being solved.  To do that, we run a state estimator program. Think of a large and geographically distributed data acquisition system with partially overlapping measurements.   Add to that,  manually entered parameter values which may be inaccurate. It is inevitable that some data is incorrect or missing. Even circuit breaker statuses can be misreported.  Entire remote telemetry facilities can be wiped out. The data will never be perfect, but the calculations need to be executed nevertheless. The state estimator provides us with a best guess for the actual circuit and boundary conditions that need solving; a guess that completely obeys Ohm’s Law and Kirchoff’s Laws.

Now think of constraints.  Every node has a maximum and minimum voltage.  Every generating source has a max and min power and a max ramp rate of change.  Every transmission line has a weather-dependent max power rating, and a short-term overload capability.  About 150K constraints may not be violated.  

Next come contingencies.   Every branch in the network and every generating source may fail during the time period considered.  We apply all contingencies to every possible solution and verify that no constraints are violated. If a violation is predicted, that solution is discarded.  We apply all contingencies singly to the entire state, all combinations of contingencies taken 2 at a time to NYC, and all combinations taken 3 at a time for the island of Manhattan.   All together about 250K contingencies are evaluated for every vertex in the Simplex solution. Remember that each if those 250K calculations requires solving for 20K voltages and 100K power flows.  It makes for a lot of number crunching.

Sometimes constraints can consider emissions.  For example, there used to be some older power plants in NYC that could burn dirty oil, or cleaner gas.  Naturally, it was policy to use gas as much as possible. But one contingency was loss of the gas pipeline.  So in  rare cases, the plants were ordered to burn dirty oil if it was determined that the grid could not survive loss of that gas pipeline.  The optimum solution would calculate the maximum amount of dependency on gas.

After all that complexity, what is the objective function to be optimized?  It is single scalar number; total cost.

The Energy Auction

Costs are determined by an auction.  Each seller and each buyer provides one or more bids; each bid for a specified MWh quantity, and a dollar price.  The bids are ordered and cumulative curves a drawn.  The intersection of the curves is the solution.  The dotted lines in the figure show the clearing price and the clearing quantity.  All bids to the left of the clearing quantity are accepted and all are paid the clearing price regardless of their bid price.  All bids to the right are rejected; those sellers and those buyers go home empty-handed. 

The Energy Auction

That auction method is the same as stock markets use to determine stock prices and quantity bought and sold.

Note that part of the sell curve is negative. Remember that for later in this article.

It is easy to visualize prices fluctuating in response to changes in the supply curve or the demand curve.  During heat waves, demand is high. During economic recessions, demand is low. When a power plant breaks down, the supply curve shifts.  Solar and wind supply for tomorrow depend on tomorrow’s weather forecast. Price volatility depends on how steep the slopes of those curves are. For example, gasoline prices all over the USA increased about 10% in the past month because of mechanical problems at a single refinery.  There is no defined limit to how close to vertical those curves can be.

Of course in real time, utilities have no flexibility; they must buy whatever retail power customers demand.  However, the biggest auction is held the day before, so the utility has the option to buy some energy in advance or buy the rest later.  

Other Vital Services

Reserves:  Things break.  We must have spares and backups for the times when they do break unexpectedly.   Therefore we require power reserves on the order of 15-20% of load. Reserves are generation capacity that is paid to be ready but to not actually generate.  In NY, we buy reserves of three types: instantaneous, 15 minute notice, 30 minute notice. The price paid for reserves is determined by a separate auction.

Frequency control: Any unbalance between generation and load+losses results in the frequency going up or down.  Some fraction of generating capacity must be available to follow that second-by-second in closed loop control to keep frequency reasonably constant.  A generator must run below 100% capacity to be able to respond up/down on command. Prices for frequency response are determined by a separate auction.  

Voltage Support: This is mostly a question of reactive power; VARS.  The network may need plus or minus VAR generation at strategic points to help maintain voltages within limits. The price paid for voltage support is determined by a separate auction.  

Black Start: After a blackout (horrors!!!) the grid must be restarted.  That is not necessarily easy. After the 1965 NE blackout, power plants on Long Island had to burn office furniture to light the fires.  Most power plants need electricity to operate their controls. The grid needs sources capable of starting without outside help, and with enough capacity to energize the entire grid, so that other generators can be started and loads reconnected.  The price paid for black start capability is determined by a separate auction.

Demand Response: To balance generation=load+losses, it should make no difference whether we increase generation by 1MW or decrease load by 1MW.  A few customers, who have the flexibility to put their loads under computer control of the grid operator are paid handsome fees to do so.  For example, mining and rock crushing. Some of them can crush enough rock for a year’s product with only 1 month operation of the crushers.  That gives them great flexibility to schedule when they consume electricity. In the future, demand response providers may participate directly in the energy auction.

ICAP:  ICAP is short for installed capacity.  It is essential that there is sufficient generating capacity ready to bid in the energy markets for each region.  If utilities own all the resources, government requires them to provide adequate ICAP. If not, government requires them to purchase it from third parties.  Yet another separate auction can be used to buy and sell ICAP.

Suppose New York is short on ICAP for next year.  Third party investors may be sour on NY, thinking that profits are too low there to be profitable.  What can be done to motivate them to build a new power plant in time for next year? Bribe them of course.  ICAP payments are the bribe. There have been legal fights over whether states have the authority to mandate that certain percentages of ICAP be renewable.  There are also legal fights in some places about the eligibility of solar and wind for ICAP payments.

What is the difference between reserves and ICAP?  Mostly the time scale. Reserves are bought for tomorrow, ICAP for next year.

Renewables Take Over

Energy/Reserves/Frequency Control/Voltage Support/Black Start/Demand Response: those are the essential invariant services of the grid.  Heretofore they have been provided by conventional means, especially hydro and steam power plants. Note that all services except energy require that the provider produce less than the maximum possible power.  

As solar and wind grow as a fraction of the total, they must assume a share of the responsibilities.   If they take over 100% as many people dream of, they have 100% of the responsibility. Opinions about what fraction they should be are irrelevant.

We already have a problem.  Remember the negative portion of the sell curve?   That is real, and in certain locations every once in a while the clearing price goes negative.  Generators must pay for the privilege of producing power. Nuclear plants make negative bids because they can not shut down for just one hour and restart for the next hour when power demand is higher.   Solar and wind generators can make negative bids because they can make a profit with subsidies alone even if they pay to the grid for the privilege. Negative prices used to be rare, but it is happening more and more often in more places and that threatens the whole financial model.

If you understood this article, you see how the technical operation of the grid is intimately co-dependent on the economics and financial auctions.  Auctions have a critical assumption — the cost-versus-production-curves must have nonzero slopes. That does not apply well to wind and solar. If negative prices become too frequent, the auction system collapses and we must find something other than money to be the objective function for optimization.  What could that something be? I have no idea. But the prospect of such a fundamental change threatens disruption, and disruptions in the delivery of energy are unacceptable.

Even experimentation and change with fundamental markets are risky.  We are talking about hundreds of billions of dollars per year. The incentive to steal, cheat, or manipulate is huge.  If you remember, we already saw what happened in 2000 with disastrous rolling brownouts in California.   There, the crooks proved to be more clever than the market designers, and much more clever than the regulators.   Innovation and prudence do not mix well for vital services.

In most contexts, I am a stalwart advocate of change, but in this case change scares the hell out of me.   My fears apply equally to regulated, deregulated, public, and private electric industries.

Comment Thread Here

53 replies
Newer Comments »
  1. PeterDonis
    PeterDonis says:
    Prior to 140 years ago, the reliable supply of electricity was zero, yet everyone seemed to survive.

    At a much lower population, with a much lower standard of living, with much slower communications…

    There are still places today where there is not a reliable supply of electricity. I would not want to live in any of those places.

  2. PeterDonis
    PeterDonis says:
    The article notes that negative bids are becoming more frequent and that this is a potential issue. I’m curious about two things:

    (1) Are there reasons for negative bids other than the two mentioned in the article (nuclear not being able to shut down/start up quickly, and solar/wind being subsidized)?

    (2) Would solar/wind ever be able to make negative bids if they were not subsidized?

  3. anorlunda
    anorlunda says:
    (1) Are there reasons for negative bids other than the two mentioned in the article (nuclear not being able to shut down/start up quickly, and solar/wind being subsidized)?

    Any kind of "must run" situation. Perhaps the chairman is coming to see a demo of the new features.

    But in terms of repeatable behavior, those are the only two cases I can think of.

    A zero bid is more common. We call that "price takers" I want to sell at any reasonable price, not matter what the price. We do the same when selling stock. We give an order "sell at market" which means I want them sold regardless of market price. But in the stock market we never see negative prices.

    (2) Would solar/wind ever be able to make negative bids if they were not subsidized?

    No. It is a direct consequence of subsidies. I thought I said that in the article.

    But it does point that subsidies destabilize the markets and that as solar&wind grow to a bigger fraction, the subsidies must cease at some point.

    But even without subsidies, a flat price curve where costs do not vary in proportion to power produced is also destabilizing of the markets. That is pretty much true of wind & solar. Once installed and ready, the incremental cost to make one more watt is zero.

    Thanks Peter, I may revise the article to more clearly separate subsidies versus flat curves.

    Similarly, the ICAP market is destabilized by state mandates regarding the percentage of renewable power. If I have some solar, and if the state mandates that solar bids must win, then I can set the price as high as I want. The only thing that would prevent that would be competition from other solar and that the sum of all solar capacity exceeds 100% of needs.

    All viable markets, stocks, energy, pork futures, …, require sufficient competition so that:

    1. More goods are offered for sale than needed. Some buyers and some sellers much go home empty handed at the end of the day. Market power or monopoly occurs when there is too little competition, so that some bids must be accepted. Subsidies and mandates distort and destabilize markets. Market stability is a continuous variable, not a binary choice.
    2. In the long term, all participants in the market make sufficient profit that they remain in the market. If they don’t they may abandon this business and invest their money in something else. Think of it on a personal level. You can invest your retirement money in a power project, or you could buy Apple stock. Nobody can command your choice. If you don’t expect the power project to make a profit, you’ll choose Apple.
  4. Derek Bolton
    Derek Bolton says:

    Synchronous condensers, batteries and pumped hydro can all be mixed with renewables to provide some of the missing functions. Is there anything that can’t cover?
    The need for such fixes does not rise linearly with the incursion of renewables, I believe. E.g. I’ve heard they could go to 40% with little other modification to the grid. Do you have any projections for how the costs of renewables and storage will fall, and how that will shift the least-cost balance? (I think it is reasonable to allow for some continued subsidy to represent the social cost of carbon).

  5. anorlunda
    anorlunda says:
    I lived off grid for 12 years myself. I had 205 watts of PV panels. It was plenty.

    So I’m envisioning homes, or at least certain appliances, in the near future, should have very similar features.

    Yes indeed. Some home loads, notably hot water heating, have great flexibility in when we schedule them.

    But please stick with the actual article topic:

    This article discusses the bulk power system and the wholesale markets, not distribution not retail, not personal use or personal costs, or personal production. To understand this article, you must think of energy supply at the level of continents, not individual homes or neighborhoods or even individual countries.

  6. PeterDonis
    PeterDonis says:
    I would disagree that the cost to make one more watt is zero.

    You left out the key qualifier he gave: "once installed and ready". He is talking about the marginal cost per additional watt generated, not the average cost per watt obtained by dividing the capital cost by the total number of watts generated. His point is that once you’ve built the thing, the capital cost is a sunk cost; it’s there no matter how many watts you generate. But the additional cost of generating more watts is zero. That means the bidding strategy of someone who has built this kind of system will be very different from the bidding strategy of someone who has built a system with a nonzero marginal cost to generate additional watts (for example, the cost of the additional fuel needed to generate those watts).

  7. PeterDonis
    PeterDonis says:
    o me though, this actually amplifies the capital cost factor, because that is the only operating expense

    Capital cost is not operating expense. Capital cost is what you pay up front.

    that and maintenance

    Maintenance is a genuine operating expense, yes, and it means the marginal cost of generating an additional watt is not exactly zero; there is some positive marginal cost. But it’s still much, much smaller than for a type of plant that requires fuel, and its relationship to the number of watts generated is also much less straightforward.

    If the capital cost could be so easily shrugged off as "sunk", solar wouldn’t need subsidies!

    You’re missing the point. The fact that the capital cost is sunk does not mean it can be "shrugged off". It means that it’s already been spent before the plant starts operating, and it is the same whether the plant generates zero watts or a billion watts.

    Nor does that mean the capital cost has no impact on the bidding strategies that are used in the markets @anorlunda described. Obviously it must have an impact, since those costs, as I’ve just said, cannot be just "shrugged off". Unless, of course, they were subsidized–which means that subsidies do in fact make a big difference.

  8. PeterDonis
    PeterDonis says:
    I would assume someone somewhere made a business case to front $ to build a plant, maybe its a bank that wants monthly interest payments, so in the case of solar, you have basically fixed overhead that you have to pay regardless of the generated output

    If the plant was subsidized, that would make a big difference to this business case, wouldn’t it?

    so if you generate less for a given time period, those watts are more expensive.

    No, they’re not, because the cost you are paying is not for generating the watts; it’s paying off the capital investment, and, as you yourself point out, that amount is the same regardless of how many watts the plant generates. So, again, the marginal cost of an additional watt, which is what @anorlunda was talking about, is zero (if we leave out maintenance cost). The average cost per watt will be different depending on how many watts are generated, but that’s not the important cost if you’re trying to understand bidding strategies and the stability of the energy markets.

  9. PeterDonis
    PeterDonis says:
    form a business perspective you get best value is generating at 100% capacity

    You can’t always generate at 100% capacity. First, the sun isn’t always shining; second, the grid can’t always accept what you are generating. That’s the whole point of @anorlunda‘s article, to explain how the process of allocating the load (power demand) among the various possible sources (power suppliers) actually works. It’s a lot more complicated than you appear to be assuming.

    The specific point I’ve been trying to make (or rather reinforce, since @anorlunda originally made it) is that the optimal strategy in the bidding process @anorlunda describes for a supplier of solar power is very different from the optimal strategy for a supplier of, say, natural gas or nuclear power, and a key factor that makes the difference is that the marginal cost to the solar power supplier of each additional watt generated is zero. The cost you keep talking about is the average cost, but that is not the cost that plays a primary role in determining the optimal strategy; the marginal cost is.

    Are you telling me that the overhead, which is to say paying all the bills needed to keep your doors open, does not factor into the cost per W?

    It does not affect the marginal cost per watt, as I’ve repeatedly explained. I’m beginning to wonder whether you have actually read my posts.

    Bidding strategy is something else

    As I said above, bidding strategy is a lot more complicated that you appear to be assuming. The article goes into this in some detail. I’m beginning to wonder whether you have actually read the article.

    A nuke plant could conceivably build a dump load so that when the grid does not want that power they could literally burn it off, or pump store hydro, rather than paying to deliver power.

    In principle they could, yes. But they don’t. Why do you think that is?

Newer Comments »

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply