Grim Day on the Texas Power Grid

  • Thread starter anorlunda
  • Start date
  • Featured
  • #1
anorlunda
Staff Emeritus
Insights Author
10,403
7,626
  • This is an enormous failure, comparable to the "Great Northeast Blackout of 1965".
  • Rolling blackouts are being imposed at a time when millions of residential customers need home heating.
  • The oil refineries shut down. Expect gasoline/diesel shortages and price spikes nation wide in a few days.
  • You can actually see blackouts as negative steps in the green curve below.
  • Wholesale prices throughout the state are roughly 2000% higher than normal.
  • There is no way to import bulk power to most of the state from outside.

I don't have enough data to diagnose with confidence. The possibilities are:
  1. It is a failure of policy -- not enough reserves required.
  2. It is a failure of planning -- the needed reserves underforecasted.
  3. An extremely unlikely combination of random events.
But like the 65 blackout, or the Space Shuttle Challenger disaster, I'm sure this event will be thoroughly investigated and reported in the next year. For now, just add it to the heap of calamities heaped on us in 2020 and 2021. 17 year locusts in the SE USA are due to emerge soon.

source: ercot.com
1613424868037.png
 
  • Like
  • Informative
Likes Fisherman199, Greg Bernhardt, dlgoff and 3 others

Answers and Replies

  • #2
Klystron
Gold Member
1,016
1,542
This link seems a bit frivolous for the EE forum but the Houston Chronicle article describes the human toll of the cold temperatures coupled with widespread power outages. Interviews mention poorly insulated homes and difficulty keeping small children and elder citizens warm without sufficient electricity.
 
  • #3
nsaspook
Science Advisor
1,145
1,985
It's bad all over.

https://www.news9.com/story/602ab8a...-pulls-back-to-energy-emergency-alert-level-2
OKLAHOMA -
The Southwest Power Pool officials said enough power has been generated to shift Oklahoma back to an Energy Emergency Alert level 2.
The SPP first declared an Energy Emergency Alert level 3, which is the highest alert level, on Sunday, meaning that its “operating reserves are below the required minimum,” officials said in a press release.
According to The Southwest Power Pool’s website, it “manages the electric grid across 17 central and western U.S. states and provides energy services on a contract basis to customers.” This includes Oklahoma.
According to a press release, the Southwest Power Pool has “directed its member utilities to be prepared to implement controlled interruptions of service if necessary.”

https://www.kshb.com/news/local-news/utility-companies-initiate-controlled-outages
As a result, the grid operator is directing its member utilities to implement controlled interruptions of service to prevent more widespread and uncontrolled outages.

“In our history as a grid operator, this is an unprecedented event and marks the first time SPP has ever had to call for controlled interruptions of service,” Southwest Power Pool Executive Vice President and Chief Operating Officer Lanny Nickell said in a statement. “It’s a last resort the we understand puts a burden on our member utilities and the customers they serve, but it’s a step we’re consciously taking to prevent circumstances from getting worse, which could result in uncontrolled outages of even greater magnitude.”

The Southwest Power Pool includes all of Kansas and Oklahoma along with portions of western Missouri, Arkansas, Iowa, Louisiana, Minnesota, Montana, Nebraska, New Mexico, North Dakota, South Dakota, Texas and Wyoming. The pool also has contracts with power providers in Arizona, Colorado and Utah.
 
  • Informative
Likes dlgoff and Klystron
  • #5
nsaspook
Science Advisor
1,145
1,985
WOW! I'm stunned.

I'm tempted to speculate, but I'll wait for more official reports instead.

https://www.kptv.com/news/nearly-30...cle_e411d1d8-6f8a-11eb-b9f9-a78d1ea140ad.html
(KPTV) – Widespread power outages persisted for a third day Monday after a weekend of winter weather slammed northwest Oregon.

Hundreds of thousands of Oregonians didn’t have power Saturday, Sunday and now Monday.

On Saturday, Gov. Kate Brown issued a state of emergency for nine counties during the winter storm.
 
  • #6
DaveE
Science Advisor
Gold Member
2,302
1,977
I don't have enough data to diagnose with confidence. The possibilities are:
  1. It is a failure of policy -- not enough reserves required.
  2. It is a failure of planning -- the needed reserves underforecasted.
  3. An extremely unlikely combination of random events.
4. A normal feature of efficient systems.

Resiliency costs money that people don't want to spend because they only see the benefits in extreme circumstances. In truly "free market capitalism" this would be exaggerated by short term optimization. This is one reason utility monopolies must be regulated. Even so, with perfect planning, it raises the question of what is the optimal design. Does it really make sense to pay for a system that never fails, that always has excess capacity?

In California, power shortages are a nearly yearly problem, with regulators working with large power consumers to help manage peak load demands with pricing incentives. I wonder to what extent they have done that sort of thing in the Southwest Pool? I don't recall a significant history of power shortages there.
 
  • #7
Klystron
Gold Member
1,016
1,542
Here is a photo montage of effects of the current weather provided by WaPo.
 
  • #8
anorlunda
Staff Emeritus
Insights Author
10,403
7,626
Does it really make sense to pay for a system that never fails, that always has excess capacity?
There are national standards set by NERC (North American Reliability Corporation) that everyone follows. The last time I checked, the standard was 10 years mean time between outages that affect 2 or more million people. That's a long way from a system that never fails.

There are many lower level requirements, one of which is "adequate" reserves (you would say excess capacity). In NY, the reserve requirement is roughly 20% of peak demand. A rolling blackout means that reserves were not adequate by definition.

In my home state (NY), we had blackouts in NYC in the years 1965, 1977, and 2003. That is 3 times in 38 years, pretty close to the NERC standard.

I've complained before about California not meeting its reserve obligations.

On the news tonight, they mentioned frozen wind turbines out of service. Texas has a huge amount of wind power (21,190 MW) compared to 73,308 MW record peak demand . But it's still too early to finger the causes with confidence.

ABC news also hinted that the rolling blackouts may grow to include everything directly north of Texas up to Canada. It boggles my mind.
 
  • Like
  • Informative
Likes Fisherman199, nsaspook, russ_watters and 1 other person
  • #10
nsaspook
Science Advisor
1,145
1,985
  • #11
2,411
711
nasaspook said, " Outages typically last from 10 to 45 minutes for residential neighborhoods and small businesses, but the exact response would vary by transmission company, according to protocols for emergency alerts from ERCOT. " I talked to a friend (a physicist) in Round Rock, TX (just north of Austin) who told me the power had just come back on at his home after an 8 hour outage.
 
  • #12
dlgoff
Science Advisor
Gold Member
4,320
2,592
Rolling blackouts are being imposed at a time when millions of residential customers need home heating.
:oldcry: :oldgrumpy: Really needed here. Last night -10°F tonight expected to be -15°F.
 
  • #13
13,577
7,566
nasaspook said, " Outages typically last from 10 to 45 minutes for residential neighborhoods and small businesses, but the exact response would vary by transmission company, according to protocols for emergency alerts from ERCOT. " I talked to a friend (a physicist) in Round Rock, TX (just north of Austin) who told me the power had just come back on at his home after an 8 hour outage.

Yeah, progress moves so fast in Texas that 10-45 mins can seem like 8hrs.
 
  • Like
Likes Klystron and nsaspook
  • #14
anorlunda
Staff Emeritus
Insights Author
10,403
7,626
My son is a plumber. He says that all the online plumbing forums are swamped today with posts about frozen pipes in Texas.

When temperatures are below freezing, that is the worst possible time to lose power. I feel sorry for the victims in those areas.
 
  • Like
Likes Fisherman199, ChemAir, Klystron and 1 other person
  • #15
13,577
7,566
Texas has multiple issues. Houses are built with minimal insulation against the cold with lots of window area and air conditioning for the summer heat. The grid model is natural gas heating for the winter and natural gas for electricity for the summer to augment the grid.

Texas also isolates its grid from the rest of the nation to avoid federal regulation and hence operators didn’t conform to federal guidelines for extreme cold weather events.
 
  • Informative
Likes bhobba and phinds
  • #16
nsaspook
Science Advisor
1,145
1,985
EuN7zFoXYAAsSWL.jpeg
 
  • Haha
  • Like
Likes PhDeezNutz, jedishrfu and anorlunda
  • #17
BWV
1,088
1,160
Simple story about no political will to spend money to winterize generators for rare storms
 
  • Informative
  • Like
Likes DaveE and nsaspook
  • #18
2,411
711
Simple story about no political will to spend money to winterize generators for rare storms
This sound pretty cynical. For any group of people, there are only finite resources available. Those resources will always be allocated to deal with the more probable events, with less to the rare events. Would you really have it done some other way?
 
  • #19
anorlunda
Staff Emeritus
Insights Author
10,403
7,626
I won't accept any simple explanations until I read more details from the investigation. It is analogous to a plane crash. There can be a chain of events and multiple contributing causes.
 
  • Like
Likes bhobba, russ_watters and phinds
  • #20
nsaspook
Science Advisor
1,145
1,985
“Welcome to every winter in the northeast US” is the simple explanation for blackouts in sub-tropical Texas during a rare winter event for them.

The chain of events and multiple contributing causes will be much the same for every management and engineering task designed to only happen once in maybe 10-20 years instead of every year. Dry runs, simulations, load testing with long idle/offline/low usage standby and backup systems will always be a prime failure point when compared to actually using those systems operationally on a regular basis to reduce full tempo operational failures.

No excuses, it's just the reality IMO with complex systems with finite resources.

https://www.energy.gov/sites/prod/files/oeprod/DocumentsandMedia/PreliminaryDisturbanceReport.pdf
https://www.energy.gov/sites/prod/files/oeprod/DocumentsandMedia/8-14-03-outage-announcement4.pdf
 
  • Like
Likes bhobba, atyy and russ_watters
  • #21
anorlunda
Staff Emeritus
Insights Author
10,403
7,626
No excuses, it's just the reality IMO with complex systems with finite resources.
That would make it a common mode failure, and the reliability planning should have accounted for that when calculating reserve requirements.

The costs to eliminate common mode failures may be unbounded, but the cost of continued operation despite them can be finite.
 
  • Like
Likes bhobba and atyy
  • #22
2,411
711
I won't accept any simple explanations until I read more details from the investigation. It is analogous to a plane crash. There can be a chain of events and multiple contributing causes.
Sadly, I think that this will most likely come down to a lot of finger pointing and not much more.
 
  • #23
nsaspook
Science Advisor
1,145
1,985
That would make it a common mode failure, and the reliability planning should have accounted for that when calculating reserve requirements.

The costs to eliminate common mode failures may be unbounded, but the cost of continued operation despite them can be finite.

All true in theory but every flood, fire, cold snap, heat wave, etc ... proves our ability to predict future cascading failure modes to be limited. In my engineering experience unused reserves become useless reserves very quickly. (offline UPS systems during power loss is a typical example)

The state is rightly focused on and likely prepared for extended yearly extreme heat events (heat elimination in homes, businesses and power generation) at the expense of rarer extreme cold events that require the polar opposite of heat retention.
 
  • #24
anorlunda
Staff Emeritus
Insights Author
10,403
7,626
All true in theory but every flood, fire, cold snap, heat wave, etc ... proves our ability to predict future cascading failure modes to be limited. In my engineering experience unused reserves become useless reserves very quickly. (offline UPS systems during power loss is a typical example)
That's a false comparison.

The grid operator pays for reserve by the MWh with real money. To qualify to receive the checks, the providers must comply with the grid operator's requirements and audits.

Often the reserves are in the form of unsold generating capacity. Suppose you have a 500 MW plant. You bid to supply energy to the grid. Your bids are in several blocks, quantity 1 MWh at price1, and quantity 2 at price 2. Some bids may be accepted and some rejected. The rejected capacity can then be offered for sale as reserve capacity. If accepted, they are paid the price for reserves.

The following day (or the following hour) both of your block 1 and block 2 bids for energy may be accepted and the provider actually has to provide the energy.

In NY, we pay different prices for reserves that can be ready to go in 30 minutes, 15 minutes, or 0 seconds (the last one is called spinning reserves). There are also auctions for other services a power plant can provide such as voltage support, frequency regulation, black-start capability, and installed capacity.

So comparing those reserves with a backup diesel that sits unused and neglected until needed is a false comparison.
 
  • Informative
Likes bhobba, DaveE and berkeman
  • #26
nsaspook
Science Advisor
1,145
1,985
https://www.argusleader.com/story/n...innesota-xcel-sioux-valley-energy/6763081002/

There is precious little power to pipe into Texas even if it was connected to the national grid.
Power was back on for all of South Dakota early Tuesday afternoon after rolling blackouts caused by "unprecedented demand" on the grid led to outages throughout the eastern part of the state in the morning.

As of 10:30 a.m. on Tuesday, the Southwest Power Pool turned power back on in parts of South Dakota and Minnesota that experienced rolling blackouts.

Power stations in eastern South Dakota and western Minnesota were being de-energized for up to an hour Tuesday morning on a rolling basis, according to East River Electric spokesman Chris Studer.
...
He noted that some customers were rightly upset about losing power, but stressed the importance of conserving energy in the short term and explained why that's vital.

"The power grid is a remarkable machine that operates under set engineering guidelines that protect it from physics. The system, while strong, must operate within strict parameters. When a situation arises in which the grid is pressed beyond those operating parameters, that once-strong system quickly becomes very delicate. Simply put, the system starts to implode and it will shut down in very damaging fashion. When this happens it can cause a cascading effect that will produce outages across the entire power grid. At that point, those outages will not be of a short duration. Those outages will be days on end as equipment is fixed and then the grid is slowly brought back online and synchronized to the proper operating parameters before it can be used again," McCarthy said.
 
  • #27
Baluncore
Science Advisor
2021 Award
10,886
5,193
  • Like
Likes Mondayman, Klystron, sophiecentaur and 1 other person
  • #28
nsaspook
Science Advisor
1,145
1,985
Power grids are filled with holes that all line up on a regular basis.

https://en.wikipedia.org/wiki/List_of_major_power_outages
At 2:44:16 AM on March 13, all was well and power engineers at Hydro-Quebec resigned themselves to yet another night of watching loads come and go during the off-peak hours. The rest of the world had finished enjoying the dance of the aurora borealis, and were slumbering peacefully, preparing for another day’s work the next day. The engineers didn’t know, however, that for the last half-hour, their entire system had been under attack by powerful Earth currents.
http://www.solarstorms.org/SWChapter1.html
One second later, at 2:44:17 AM, these currents found a weak spot in the power grid of the Hydro-Quebec Power Authority. A 100-ton, static VAR capacitor Number 12 at the Chibougamau sub-station tripped and went off-line as harmonic currents induced by the electrojet flowing overhead, caused protective relays to sense overload conditions. The loss of voltage regulation at Chibougamau caused power swings and a reduction of power generation in the 735,000-volt La Grande transmission network. At 2:44:19 AM, a second capacitor followed suit at the same station. 150 kilometers away at the Albanel and Nemiskau stations, four more capacitors went off-line at 2:44:46. The last to fall at 2:45:16 AM was a static VAR capacitor at the Laverendrye complex to the south of Chibougamau. The fate of the network had been sealed in barely 59 seconds as the entire 9,460-megawatt output from Hydro-Quebec’s La Grande Hydroelectric Complex found itself without proper regulation.
 
  • Like
Likes 256bits and OCR
  • #29
anorlunda
Staff Emeritus
Insights Author
10,403
7,626
Well, it seems that everyone else is commenting on this event without waiting for the report, so I guess I will too. I'm using an article from a source that I trust to check sources, and to not sensationalize -- wsj.com.
https://www.wsj.com/articles/texas-...arket-incentives-11613777856?mod=hp_lead_pos7

The core problem: Power providers can reap rewards by supplying electricity to Texas customers, but they aren’t required to do it and face no penalties for failing to deliver during a lengthy emergency.
The phrase "power providers" must be read carefully. They mean power plants, that sell energy to power utilities. Power utilities that sell energy retail to consumers are punished by the PUC (Public Utilities Commission http://www.puc.texas.gov/) for failing to deliver.

Later in the article, they mention the "capacity market" that Texas does not have, but other states do have. In NY, we call that ICAP. ICAP pays generators for being capable of delivering energy, and bidding in the daily markets. The purpose of ICAP is to prevent bad guys like Enron from boosting prices by withholding capacity, or faking malfunctions. If ICAP payments are substantial, power generators want the payments badly. The key relevant here is, what do the power plants need to do to qualify for ICAP payments? Thinks like winterproofing for example.

Critics say ICAP pays providers for doing nothing, and Texas doesn't have it. But I agree with the WSJ that the lack of ICAP in Texas is a major blunder.

The system broke down this week when 185 generating units, including gas and coal-fired power plants, tripped offline during the brunt of the storm. Wind turbines in West Texas froze as well, and a nuclear unit near the Gulf of Mexico went down for more than 48 hours. Another problem emerged: Some power plants lost their pipeline supply of gas and couldn’t generate electricity even if they wanted to capture the high prices.
The grid operator said that about 46 gigawatts of natural gas, coal and wind generation wasn’t working—roughly 40% of what it had expected to be available.
That's a staggering number, 40%.

But it wasn't just power plants. Natural gas pipelines, oil refineries, and city water systems, also shut down because of freezing. [EDIT: Therefore, some consumers who heat their homes with gas were left to freeze. Customers can provide for gas furnaces that can operate without grid electricity.]

Natural gas prices.jpg


Within the competitive Texas power market, there is a strong incentive for generators to keep costs down to recoup their investments. The rapid buildout of wind and solar power, which are now among the cheapest sources of electricity, have pushed prices even lower in recent years, making it more difficult for gas and coal plants to compete.
That touches on the destabilization of markets that I talked about in the 2019 PF Insights article. https://www.physicsforums.com/insights/renewable-energy-meets-power-grid-operations/

EDIT: One more point.
https://www.foxnews.com/us/some-tex...kyrocket-as-high-as-17000-during-winter-storm
Some Texans' electricity bills skyrocket as high as $17,000 during winter storm
Retail consumers should never be exposed to the volatility of wholesale energy prices. Sweden learned that many years ago when some consumers were similarly harmed when they chose to buy electricity on a time variable price. Consumers can never be adequately informed about the risks to enable knowledgeable consent.

That is not the same thing as utility peak/off-peak tiered pricing. Those tiers have published prices.

The system is supposed to be that power utilities buy energy on the volatile wholesale market. Then they sell to retail consumers, with the retail rates set by the PUC. Typically, retail rates stay fixed for 1-2 years.
It is community organizers who advocate becoming middlemen, buying power wholesale and selling it retail. They argue that helps keep the profits within the community. They escape regulation by the PUC. Of course, the organizers are never responsible for paying the bills when prices skyrocket. IMO, those middlemen should never be allowed.
 
Last edited:
  • Like
Likes Astronuc, russ_watters and Dr.D
  • #30
1,198
779
Someone already pointed this out here. I recently talked with a friend of mine who happens to be a nuke plant operator in Alabama, he said that the main problem for Texas is the fact that they are not interconnected to the national grid which itself seems to be made from two halves namely the east and west.
Surely enough I live in the northern part of Europe and such weather as is in Texas now is just fun and games for us here. We had like -25Celsius recently and nobody even bothered. The reason is simple we have reinforced concrete buildings with heat insulation and pretty much everything everywhere is made for such weather so no problem.

What I see in Texas is that it's a state that enjoys sunshine and warm weather pretty much and everything is built accordingly. Whenever I see the private house construction videos from US south side I see thin structures that are not made for any real thermal insulation for longer than a few hours.
Now bad thermal insulation aside, a state that is not interconnected and has very few baseload 24/7 plants of which as I understood some had been offline for whatever reason it's no miracle that when everyone switches on their heater the grid drops.

I can be wrong here but my own two conclusions from what I know here are such:

1) A smaller grid with too small reserve capacity is more likely to fail than a larger one with more backups, this I think is pretty much a universal axiom

2) whenever a substantial portion of the grid is made by renewables like solar and wind one also has to have failsafe backups that can act as base loads much like a nuke plant has its diesel's always ready.

So that when your wind turbines ice up and are stopped or your solar turns to snowstorm and your natural gas fails for whatever reason you can switch on a fast acting reserve. From what I know and contrary to popular belief coal can be a good backup. Pumped hydro or hydro can also save the day.Actually natural gas I think is also a good reserve as well as active , not sure how much of it Texas has and in what condition
All in all I think it isn't/wasn't a wise strategy to close down and stop building nuclear plants as base loads.

This seems more like a long term political planning and ideology problem that is now manifesting as a technical problem.

I can draw some similarities to Fukushima, I looked at some history a while back and saw that Japan has had tsunami's like that before , but when they built Fukushima they concluded that because in the past 50 years it did not happen it is unlikely to happen in the next 50 of the plant operating lifetime. Well that short sighted planning turned out to cost a tremendous fortune along the road.

All they had to do is locate those diesles higher up on the shore and seal them better. I'm sure a minor investment compared to the consequences.
Even with our modern meteorology we cannot foresee dramatic events with 100% accuracy and big enough failsafe time to adequately prepare so being ready in advance and having a failsafe reserve is always the smart move.
I think it costs and that is the real problem why things get built less sturdy
 
Last edited:

Related Threads on Grim Day on the Texas Power Grid

  • Last Post
Replies
6
Views
2K
  • Last Post
Replies
6
Views
3K
  • Last Post
Replies
3
Views
2K
  • Last Post
2
Replies
46
Views
3K
Replies
8
Views
1K
Replies
16
Views
907
  • Last Post
Replies
6
Views
1K
  • Last Post
Replies
15
Views
979
Replies
2
Views
753
Replies
25
Views
5K
Top