Problems with the Dreamliner battery

[nsaspook]

http://www.ntsb.gov/news/2013/130127.html

Here is a NTSB media brief from a few days ago. http://www.youtube.com/watch?v=wVMkt3sFwh0&feature=youtu.be

My comments in a earlier post about the lack of circuit board protection is related to this: http://www.newsdaily.com/stories/bre90o0gk-us-boeing-dreamliner-japan-circuitboard/

Typically on a system that monitors a possible hazardous process there is a containment space and isolated protected controller area for the monitor and control electronics so if there is a problem the actual controller can be examined and not be turned into a useless cinder of carbon.

 

[Dotini]

http://seattletimes.com/html/businesstechnology/2020241385_787deadbatteriesxml.html

An interesting article from the Seattle Times aerospace reporter notes that 100 to 150 of the lithium-ion 787 batteries have been returned to the manufacturer, Yuasa. The troubles with them at first sight seem separate from the two failures which have grounded the fleet, but the electrical system which monitors and controls the batteries is being closely scrutinized by the NTSB and other agencies.

Respectfully submitted,
Steve

 

[nsaspook]

The Cessna Citation CJ4 battery:
http://saeaero.saejournals.org/content/3/1/149.abstract

A year later:
http://www.aero-news.net/index.cfm?do=main.textpost&id=00580ffe-b33b-44ab-9db0-3e6ad0ca4f19

 

[jim hardy]

From that FAA airworthiness directive linked by spook:

The service information describes procedures for replacing lithium-ion main aircraft batteries, ..., with Ni-Cad or lead acid batteries. The agency is issuing this AD following evaluation of all the relevant information and determined the unsafe condition described previously is likely to exist or develop in other products of the same type design.

well, some engineering manager is taking his lumps."Cutting Edge Technology" can cut one's career short.old jim

 

[nsaspook]

This is a complete WAG but maybe we are seeing the effects of strong EM fields on the metal foils that are in tight coils inside each cell when operated at high energy combined with high density levels. Looking at the length of the foils from the Dreamliner cells they would seem to have a fairly large inductance. Rapid current changes from loads or from being charged at high variable rates could be causing some sort of voltage or current non-uniformity inside the cells from the magnetic field effects.

 

[AlephZero]

It emerged on Wednesday [Jan 30] that ANA, the largest operator of the 787, had problems with the lithium-ion batteries on its Dreamliners before the emergency landing on January 16.

ANA replaced Dreamliner batteries on 10 occasions last year because of faults with this equipment or related components, although the airline and Boeing said safety was not compromised.

... from http://www.ft.com/cms/s/0/79906f2e-6ac8-11e2-9871-00144feab49a.html#axzz2JV7ZNnGX

I'm sticking with my original "best guess": there won't be a quick fix for this.

An interestng choice of words in the FT article: Boeing is "assuming" (not "forecasting"!) no significant financial impact from all this. Elsewhere, I've seen an estimate that a grounding to the end of 2013 would knock about $7bn off their 2013 earnings, plus knock-on effects on canceled orders and other future business.

EDIT: Those 10 battery incidents at ANA were on a fleet of just 17 aircraft

 

[jim hardy]

This is a complete WAG but maybe we are seeing the effects of strong EM fields on the metal foils that are in tight coils inside each cell when operated at high energy combined with high density levels. Looking at the length of the foils from the Dreamliner cells they would seem to have a fairly large inductance. Rapid current changes from loads or from being charged at high variable rates could be causing some sort of voltage or current non-uniformity inside the cells from the magnetic field effects.

Batteries have a ripple current rating, just like capacitors. Manufacturers caution about ripple in the charger output.

Ripple current
Batteries, as DC devices, prefer to have only DC imposed
on them. The charger’s job is to convert AC into DC but
no charger is 100% efficient. Frequently, filters are added
to chargers to remove the AC current from the DC output.
The AC current on the DC is called ripple current. Battery
manufacturers have stated that more than about 5 A rms
of ripple for every 100 Ah of battery capacity can lead to
premature failure due to internal heating. Ripple voltage is
not a concern since it is the heating effect of the ripple current
that damages batteries. The 5% ripple current figure is
a rough estimate and depends also on the ambient temperature.
Ripple current can increase slowly as the electronic
components in the charger age.

http://www.artec-ingenieria.com/pdf/...uide_en_LR.pdf

Clearly, high crest factor waveform has more heating value than sinewave.
And high frequency would shift that load to the foil nearest the terminals for reason you stated.

I once had to tame some loads - we had SCR style inverters modulating 135 volt 1000 AH batteries to the tune of 100 volt spikes. Their currrent draw was not constant, they drew it in huge gulps. We added one microfarad per milliamp across inverter inputs to calm the battery bus.

So what you suggest is possible.
Ripple and its heating can come either from charger or load.
I'd wager the charger is well behaved,
but the loads are a wag for me, too. In fact an Un-Scientific one, USWAG.

 

[Dotini]

http://seattletimes.com/html/businesstechnology/2020275838_boeingoutsourcingxml.html

The latest in a series of Seattle Times aerospace reporter articles into the 787 electrical woes.

As an off-and-on employee of The Boeing Company from 1968 to 2005, in various positions on all commercial airplane programs, I can relate to the sentiments expressed in the article. The Company may indeed be embroiled in something of a fiasco, but I deeply hope and believe it has the ability and time it needs to recover. My pension may depend upon it!

Respectfully submitted,
Steve

 

[nsaspook]

This seems more related to the current problem.

http://seattletimes.com/html/businesstechnology/2020241162_787battery29xml.html?prmid=obinsource

EaglePicher’s key test — proving that a battery explosion is contained within the box — is one such certification test pre-agreed as satisfying the FAA’s conditions.

The company’s website contends that overcharge explosion tests on its battery were repeated successfully multiple times and concludes that “even during this worst-case scenario, the (battery) is able to contain a thermal event.”
...
However, according to a detailed account of the 787’s battery-fire protection system provided by Sinnett, Boeing’s containment plan did not envisage confining the accident entirely inside the battery box.

Sinnett said Boeing had to demonstrate to the FAA that it had multiple redundant safety mechanisms that ruled out the worst-case scenario that EaglePicher’s test simulates: an overcharged battery explosion

 

[nsaspook]

http://www.thestate.com/2013/02/05/2619386/japan-787-probe-finds-thermal.html#.URGs-uDlAW0

The Japan Transportation Safety Board said that CAT scans and other analysis found damage to all eight cells in the battery that overheated on the All Nippon Airways 787 on Jan. 16, which prompted an emergency landing and probes by both U.S. and Japanese aviation safety regulators.

They also found signs of short-circuiting and "thermal runaway," a chemical reaction in which rising temperature causes progressively hotter temperatures. U.S. investigators found similar evidence in the battery that caught fire last month on a Japan Airlines 787 parked in Boston.

http://mainichi.jp/english/english/newsselect/news/20130206p2g00m0dm001000c.html

All eight cells of the battery installed in the ANA plane showed heat-caused damage, Norihiro Goto, chairman of the Japan Transport Safety Board, said at a press conference.

"Cells 3 and 6 were severely damaged and Cells 1, 2, 7 and 8 were swollen or deformed," Goto said.

He also said that the positive electrode of Cell 3 was found to have experienced particularly severe damage, and wiring inside some cells melted.

 

[AlephZero]

The problem is, they are doing the CT scans etc after a lot of secondary damage occurred. The hard part is figuring out what was the primary cause.

To use a different example, this

looks impressive, but was not very relevant to the investigation compared with looking at what actually broke (or half of it, to be pedantic):
http://www.smarteraircharter.com/news/wp-content/uploads/2010/11/qantas-a380-engine-failure-part-300x225.jpg

After painstakingly dissecting a number of batteries, examining associated electronic parts, and analyzing information from flight-data recorders, NTSB experts and their Japanese counterparts haven't been able to pinpoint any specific component, automated subsystem or software application that appears to offer hope of finding answers.

... from http://online.wsj.com/article/SB10001424127887324761004578284392368343774.html

Having apparently got nowhere after a month of ground testing, Boeing have asked the FAA to approve some 787 flight tests to try to diagnose the problem. Flgiht testing to certify a fix is one thing. Flgiht testing to reproduce an on-board fire hazard is something else. I wonder that the FAA will make of the request

 

[nsaspook]

The problem is, they are doing the CT scans etc after a lot of secondary damage occurred. The hard part is figuring out what was the primary cause.

The failure modes in these cases seem almost identical, a chemically driven positive temperature feedback loop that once started seems to be impossible to stop by all present electronic control measures and so far there seems to be little data on the precursors that start it from all the monitoring data on the system. That leads me to believe the problem is related to what Elon Musk describes (large cells in close contact, thermal and/or electromagnetic effects).

"They [Boeing] believe they have this under control, although I think there is a fundamental safety issue with the architecture of a pack with large cells," writes Musk in an email. "It is much harder to maintain an even temperature in a large cell, as the distance from the center of the cell to the edge is much greater, which increases the risk of thermal runaway."

 

[AlephZero]

A (lack of) progress report from the NTSB investigation: http://www.bbc.co.uk/news/business-21373593
I guess somebody in Boeing is saying "ouch", after those comments.

Re the "test flights", The FAA has granted Boeing permisson for one flight to return a 787 from a "paint job" in Texas back to Everett (i.e. not a test flight as such, though there are conditions imposed on monitoring the batteries while in flight).

Other news: the Europoean air safety agency (EASA) has been invited to join the US investigations. And Polish airline LOT, which has a 787 grounded in Chicago, is applying to the FAA via the EASA, for permission to fly it back home (presumably the fuel cost is less than the long term parking charges!)

 

[AlephZero]

FAA approval for 787 test flights: http://www.faa.gov/news/press_releases/news_story.cfm?newsId=14295

 

[Ptero]

The person I consider to have the best investigative performance to date on this issue is Celina Mikolajczak.

Lithium-Ion Batteries Hazard and Use Assessment
Final Report Prepared by:
Celina Mikolajczak, PE
Michael Kahn, PhD
Kevin White, PhD
Richard Thomas Long, PE
Exponent Failure Analysis Associates, Inc.
© July 2011 Fire Protection Research Foundation
http://www.nfpa.org/assets/files/PDF/Research/RFLithiumIonBatteriesHazard.pdf

I will be glad to provide my opinions on what is contained in this rather damning document, but I am most interested in yours. It is clear to me that this technology is not mature enough in manufacture to be allowed in commercial aircraft. The absolute inability to identify point shorts and dendrite growth, or to prevent Li plating on the anode, the concerns with aging increasing liklihood of shorting, the acceptance of flammable electrolyte, the outgassing of flammable gas and the impossibility of extinguishing electrolyte fires with halon, the cascading effects of thermal runaway events and the inadequate "solution" of boxing this ticking timebomb of a battery in a titanium box vented to the slipstream screams not just of engineering incompetence but of another political mandate gone sour. I do not know if that last bit is true, but I do see Steven Chu scrambling for the exit as Airbus reconsiders Li-Ion deployment.

A kludge (or kluge) is a workaround, a quick-and-dirty solution, a clumsy, inelegant, difficult to extend, hard to maintain yet effective and quick solution to a problem, and a rough synonym to the terms "jury rig", "Jugaad" or "jerry rig". -- Wikipedia

Photos of Japan Airlines January 7 incident, battery pack and cell damage
-- Slide #13: NOTE THE SHORTING DAMAGE THROUGH THE TITANIUM CONTAINMENT! The failure of NTSB's mandated kludge may have been the strongest reason to ground the B-787 fleet. (The melting point of titanium is 3000 F.)
NTSB PDF February 7, 2013
http://www.ntsb.gov/investigations/2013/boeing_787/JAL_B-787_2-7-13.pdf

 

[jim hardy]

Thanks Ptero i forwarded that nfpa paper to a friend at Boeing.

Disasters are always a string of small things stacked up, they topple like Dominoes. That was Ernie Gann's premise in "Fate is the Hunter".
I saw same principle at work in nuclear industry.

In my opinion that is a significant domino.

 

[OmCheeto]

What's this? No one has quoted wiki yet?

Groundings

The focus of the review will be on the safety of the lithium-ion batteries made of lithium cobalt oxide (LiCo). The 787 battery contract was signed in 2005,[195] when LiCo batteries were the only type of lithium aerospace battery available, but since then newer and safer[299] types (such as LiFePO), which provide less reaction energy during thermal runaway, have become available.[193][300] FAA approved a 787 battery in 2007 with nine "special conditions".[301][302] A battery approved by FAA (through Mobile Power Solutions) was made by Rose Electronics using Kokam cells;[303] the batteries installed in the 787 are made by Yuasa[191].

bolding mine

Interesting. But I understand how contracts go. Something better comes along, but we've signed a contract to buy the inherently more dangerous battery.

I changed one of the reference links[193 original ref link], as the original was in some incomprehensible language(Norsk I think). My link points to the following image:

http://www.tu.no/incoming/2013/01/16/1200013182.jpg/ALTERNATES/w620f/1200013182.jpg

hmmm... What was the name of that guy who was so enthralled with LiFePO batteries, that he lost a small fortune investing in a company that made them? What was his name?



----------------------------
Ok to delete, as I am aware that I'm being a, "told you so", kind of smart***

 

[AlephZero]

That last page of the NFPA paper says it is a literature survey about transportation and storage of Li batteries, not about their actual use.

This may be more relevant in showing how faults develop can during use - e.g Fig 5 showing that heat release was delayed for 20 hours after the simulated fault was created.
http://www.evdl.org/docs/li_fault_detection.pdf

If something "goes bang" when you poke it with a sharp stick, there's a fair chance you will speculate there could be a cause and effect mechanism. If it goes bang a day later, for no apparent reason, that's not so easy to understand.

 

[OmCheeto]

That last page of the NFPA paper says it is a literature survey about transportation and storage of Li batteries, not about their actual use.

This may be more relevant in showing how faults develop can during use - e.g Fig 5 showing that heat release was delayed for 20 hours after the simulated fault was created.
http://www.evdl.org/docs/li_fault_detection.pdf

If something "goes bang" when you poke it with a sharp stick, there's a fair chance you will speculate there could be a cause and effect mechanism. If it goes bang a day later, for no apparent reason, that's not so easy to understand.

TLDR. But yes, bang, is not a pleasant sound.

Dec 8, 2011
General Motors Co. (GM), maker of the Chevrolet Volt plug-in hybrid that is the subject of a federal safety probe, is moving to a less volatile battery chemistry for its Chevy Spark electric car going on sale in 2013.
GM is using phosphate-based lithium ion batteries from Waltham, Massachusetts-based A123 Systems Inc. (AONE) that are less likely to burn than other lithium chemistry, including that used in the Volt model introduced last year, said battery experts and suppliers.

Not that their manganese based LIOH battery was that bad. Didn't that Volt catch fire 2 weeks after it had been crash tested?

 

[AlephZero]

Not that their manganese based LIOH battery was that bad. Didn't that Volt catch fire 2 weeks after it had been crash tested?

At which point, one of my engineering mentors would repeat his favorite "development engineering is hard" parable:

Ug the caveman set off to the woods to pick some nuts and berries, but found the path blocked by a large dinosaur, asleep in the sun.

Shouting at the dinosaur had no effect, so Ug went back to his cave, found a sharp stick, and poked the dinosaur in the posterior.

Nothing happened for a couple of hours. Then, the dinosaur got up, took two steps backwards, and sat down on top of Ug, crushing him to death.

Now, the moral of this story is this: the dinosaur did not attack Ug because of the poke from the sharp stick. It was actually responding to a tap on the nose that somebody had given it three weeks earlier.

 

[Ptero]

"Didn't that Volt catch fire 2 weeks after it had been crash tested?"

In this instance, the battery pack had sustained damage and the battery coolant had drained out. The fire occurred, I think, as you state, two weeks later and was blamed by all investigators on a mechanically-damaged cell that initiated the thermal runaway, likely due to a combination of physical damage and a lack of coolant. GM subsequently retrofitted the battery armor on the Volt, beefing it up, and the US DOT accepted this as an effective fix and canceled the investigation. Personally, I do not consider this anything but a kludge based in inadequate investigation. But this was the accident where GM and the NHTSA learned that these battery packs could not be left to sit idle after an accident like their conventional kin, so I am inclined to cut them some slack as a giant and ponderous organization, despite the fact that they failed to listen to some of their more knowledgeable people. However, I am also not satisfied that the entire truth has been told - because of politics - yet there have been no more fires, and no deaths or injuries that I know of, so perhaps I am being too skeptical in regard to magnesium-type Li-Ion, regardless of the fact that it, too, contains flammable electrolyte. Gasoline, diesel, ethanol, propane, natural gas and hydrogen, after all, are also flammable and can ignite in accidents. But spontaneous ignition and thermal runaway, as has occurred in smaller consumer cells and in aviation cargo, or ignition during charging or discharging, or most dramatically with the thermal runaways of Cobalt Li-Ion that resulted in the B-787 fleet grounding, should not be acceptable, imo. Cobalt Li-Ion remains a unique and frightening story. There are serious and unresolved physics problems here - perhaps even fundamental physics barriers that cannot be overcome and extend across the gamut of Li-Ion (this is my personal belief - what do you think?). I do not understand why Cobalt Li-Ion was not entirely abandoned by the aviation industry when the currently insurmountable problems came to light, unless politics played a role. In commercial transport history, it has become de rigueur that virtually all latent design threats to aircraft operations have been immediately addressed and resolved. The implementation of design features with known dangerous faults is unusual in the extreme. That said, I remain suspicious of undetectable dendrite formation in all (flammable electrolyte) Li-Ion battery types so I am keeping a close eye on pure EV as well as hybrid batteries as they age.

 

[jim hardy]

"...There are serious and unresolved physics problems here - perhaps even fundamental physics barriers that cannot be overcome and extend across the gamut of Li-Ion (this is my personal belief - what do you think?)...

I think using big Li-Co batteries is like playing with dynamite.

That said, mea culpa - when i was in college early 1960's every small town hardware store carried dynamite. It was easier to buy dynamite than firecrackers. On a dull Sunday afternoon we boys not infrequently went out to the local rock quarry where we learned to move boulders and uproot trees.
We thought nothing of driving around with a half dozen sticks of the stuff under our car seat.

But dynamite has a fairly high activation energy - you can even hold a match to it and it just fizzles a bit. Unlike those Li-Co batteries, which need only be modestly heated or modestly overcharged to go pyrotechnic on you.

I think i said it earlier - i won't sit in a car seat with one of those batteries underneath my butt. Their low activation energy is a weak latch on a tiger cage.

And that's what i think.

I'm sorry i can't be more scientific about it.

old jim

 

[jim hardy]

I do not know how word 'batteries' in previous post became a link. I tried to edit it out, something not in my control gave it that vigilink attribute.

Odd - it's only a link when i am not logged in.
old jim

"never trust a computer with anything important."

 

[Ptero]

Talked to a pilot who flies for a carrier that had a bunch of B-787s grounded. He said several flight crews had reported overheating battery packs in flight. We may have been very lucky that we didn't lose one over the Pacific.

I haven't heard that anywhere else, btw.

 

[jim hardy]

There's been some trouble. Probably one would have to play golf or go fishing with pilots and airplane mechanics to get the real story .

http://www.aviationweek.com/Article.aspx?id=/article-xml/awx_01_30_2013_p0-542761.xml&p=1

Japan’s two biggest airlines replaced lithium-ion batteries on their Boeing Co 787 Dreamliners in the months before separate incidents led to the technologically advanced aircraft being grounded worldwide due to battery problems.

Comments from both All Nippon Airways, the new Boeing jetliner’s biggest customer to date, and Japan Airlines Co Ltd point to reliability issues with the batteries long before a battery caught fire on a JAL 787 at Boston’s airport and a second battery was badly charred and melted on an ANA domestic flight that was forced into an emergency landing.

ANA said it changed 10 batteries on its 787s last year, but did not inform accident investigators in the United States because the incidents, including five batteries that had unusually low charges, did not compromise the plane’s safety, spokesman Ryosei Nomura said on Wednesday.

JAL also replaced batteries on the 787 “on a few occasions”, said spokeswoman Sze Hunn Yap, declining to be more specific on when units were replaced or whether these were reported to authorities.

ANA did, however, inform Boeing of the faults that began in May, and returned the batteries to their manufacturer, GS Yuasa Corp. ...

 

[AlephZero]

Probably one would have to play golf or go fishing with pilots and airplane mechanics to get the real story.

It's all shared information between the people who can actually make some use of it. Given the amount of air traffic world wide, you shouldn't be too surprised that there are "incidents" somewhere every day of the week, but most of them are no more "news" than the fact that one of your car tires got a puncture and had to be fixed - unless the tire blew and the car ends up upside down in a ditch, of course, which might make it into a "news story". And then some news reporter with a PhD in basket weaving discovers there are a million car tire puctures every year in the USA - shock horror, panic, "car tires are unsafe", etc, etc ...

FWIW we once had quite an argument with the FAA which started from "what's going on here - we AREN'T getting any reports of a particular type of failure from your engines. What are you hiding?", The answer was "nothing", but it took a while to convince them that our desgins were sufficiently different from the competition that they really never did fail that way.

(Note for pedants: I've no idea whether "a million" is the right number or not).

 

[jim hardy]

I didnt mean to suggest a surreptitious coverup or conspiracy.. Sorry.

More along the line firsthand observations are always the best.

Information in a corporation flows much like water in a stream, and any cowboy will tell you : "Always drink upstream of the herd".

old jim

 

[AlephZero]

I didn't think you were suggesting any conspiracy. But the information doesn't just go to the airilne or the plane maker. There are committees and safety boards run by organizations like the FAA and NTSB that have representatives from all the major players, and they get to see data across the whole industry, not just for their own products.

These can end up like Longfellow's poen - "Though the mills of God grind slowly, yet they grind exceeding small". For example a committee set up to investigate this http://en.wikipedia.org/wiki/United_Airlines_Flight_232 back in 1989 is still grinding away at improving inspection methods. One reason it's a slow process is the small amount of data to analyse - it gets about one new "data point" (i.e. somebody detects a potential problem at the manufacturinig stage) per year, and of course some of those turn out to be false positives.

But you won't find that level of information in the public domain, unless somebody leaks it.

 

[AlephZero]

Polish airline LOT announces plans to reschedule summer services, keeping its 787s grounded until October. http://seattletimes.com/html/businesstechnology/2020353578_apeupolanddreamliner.html

Airbus rules out Li batteries on the A350. http://www.bbc.co.uk/news/business-21477126

 

[nsaspook]

Wow, that is really bad news for Boeing if they can't completely nail this problem down to a fixable root cause. I sure hope Boeing has a 'Plan B' that won't take a year of downtime to qualify a new electrical sub-system.