Boeing How Safe is the Boeing 737 Max's MCAS System?

[gmax137]

The Ford Pinto

Boy that takes me back. In the late 1970's I drove a Pinto... with Firestone 500 tires .

Following up on @anorlunda 's post and the nuclear biz, we do a lot of work with fault-trees and "probablilistic risk assessment" - PRA. This isn't used so much to actually drive design of the plants (most of which were designed & built before the advent of the PRA approach). The PRA is used nowadays to provide "insights" into what is important and what isn't so important. So it can support decisions on surveillance frequencies (ie, spend more time testing the important stuff; don't wear equipment out by testing it), and determining severity of nonconformance/noncompliance issues.

I was told, the nuclear PRA work and approaches started out following similar approaches developed in the aviation industry. So I have a hard time squaring that with the idea that the airplanes have unidentified single point vulnerabilities.

 

[PeterDonis]

Earlier in this thread (too many posts, can't find it) I learned that Boeing's scheme was to have two independent strings of logic for MCAS.

Yes, but as I understand it, only one of them is actually affecting the operation of the plane at any given time, so only its failure rate is relevant for assessing flight safety.

With X1, we have added protection against single failure of A1 or B1, but we introduce a new common mode failure X1 that could make both strings fail simultaneously.

But if only one string is actually affecting the plane's operation, the only failure probability that is relevant is the failure probability at that string's final output. So the relevant question is not whether introducing the new failure mode at X1 increases or decreases the overall probability of at least one of the A or B strings failing; the relevant question is whether introducing the new failure mode at X1 increases or decreases the probability of failure of the string, A or B, that is actually affecting the plane's operation. And one would expect that it is very likely to decrease the probability of that one string failing, because the reduction in failure probability from comparing outputs A1 and B1 is likely to be much greater than the increase in failure probability from the chance of the comparison at X1 having an error.

 

[PeterDonis]

MCAS was explicitly judged to be less risky in case of failure than it should have been. Badly written software or not, the one sensor-one computer architecture was insufficiently robust for the criticality and power(severity of failure) of the system.

Yes, agreed.

 

[anorlunda]

Yes, but as I understand it, only one of them is actually affecting the operation of the plane at any given time, so only its failure rate is relevant for assessing flight safety.

That was my understanding too from earlier in this thread. That exactly matches the first graphic posted in #539. That scheme is arguably the one with the highest probability of having at least one string functional.

So my wager is that the original design engineers argued for fully independent strings, with no contaminating cross connections. But somehow, switchover from A string to B string when needed never appeared in the operating procedures. Without a switchover, what is the point of a second string in the first place? Spare parts? I am unaware of any other case where they carry spare parts on board the aircraft without the possibility of using those spares during a flight.

Nevertheless, we should strive for the same discipline we use in SR, GR and QM. None of us should be making factual assertions about a conclusion that must be calculated.

 

[PeterDonis]

That exactly matches the first graphic posted in #539. That scheme is arguably the one with the highest probability of having at least one string functional.

Not if including the crossover X1 decreases the overall probability of failure, by incorporating input from both AoA sensors (if we assume that nodes A1 and B1 are the two sensors) in order to detect sensor failure or unreliability, and that improvement outweighs the impact of adding the additional failure mode associated with X1 itself making an error.

switchover from A string to B string

How would the system decide to make such a switchover without doing some kind of comparison between the two?

Also, to be clear, the kind of redundant sensor configuration I was thinking of would not switch over from one string to the other; it would just compare the two sensors, and if they didn't agree within some tolerance, the automated system would simply be disabled, and the flight crew would see some kind of warning in the cockpit telling them the system was disabled. (The "AoA agree" cockpit indicator that is in the additional package that US carriers like Southwest paid for would be a similar cockpit indicator, if it were actually connected to an automatic function that disabled MCAS when the AoA sensors did not agree.) One could imagine more complicated algorithms to detect sensor failure or unreliability, but you are correct that more complexity means more possibilities for failure, so such algorithms would have to be evaluated on that basis.

 

[anorlunda]

that improvement outweighs the impact of adding the additional failure mode associated with X1 itself making an error.

Sorry Peter, but that's an assertion of fact that needs a citation.

One could imagine more complicated algorithms to detect sensor failure or unreliability, but you are correct that more complexity means more possibilities for failure, so such algorithms would have to be evaluated on that basis.

That's inconsistent. You are saying more complicated algorithms (like my 3rd graphic) have to be evaluated but that your preferred scheme (like my 2nd graphic) does not need evaluation.

 

[PeterDonis]

that's an assertion of fact that needs a citation.

I said "if". I agree we don't know for sure whether and in what cases it would be true; we would have to actually collect real world data. I was only saying that I don't agree with your statement that "arguably" the configuration with the highest probability of having at least one string functional is the one with no crossovers at all. I was giving an argument for a different expectation that I think is more likely: that there are ways to add crossover nodes that, while they do introduce new failure modes, decrease the overall probability of failure by allowing comparison of multiple items as an error check.

You are saying more complicated algorithms (like my 3rd graphic) have to be evaluated but that your preferred scheme (like my 2nd graphic) does not need evaluation.

I was not saying that simpler algorithms don't need evaluation. What I was saying is that in evaluating any algorithm, the complexity of the algorithm will be a factor since more complexity means more possibilities for error.

 

[Anderson-Paul]

After having read the House Committee Report(s) concerning the circumstances surrounding the crash of two Boeing 737 Max Airliners. Along with the destruction of both airliners, was a horrendous loss of human life. This House Report is one of many Federal Government reports ( FAA, NTSB etc. ) directed at both cause and failure of parts and systems etc. thorough their investigation phase.

During my research into the demise of both airliners, I found everything from the opinions of the general public, to memos and reports written by company employees. One such report was written by both company test pilots. Both had experienced problems with the MCAS system, and both reported the problems to the company. The problem is, the information the company received, fell on deaf ears. No attempt had been made to correct the MCAS system Issues.
I was most concerned about the conflict between engineering and management , over whether or not to re-engine a fifty year old design, that had been through four cycles of modification change. The discussion between 737 Max project engineering and project management was this, all appeared to oppose a re-engine attempt, on a fifty year old design. On the other hand, both the board members and top level management, approved
going ahead with a fifty year old design. The opposition were all for a new “Clean Sheet“ design. As a pilot and retired research and development engineer for P&W, I would also have opposed using a fifty year old design.

The fact that Boeing had decided to replace the CFM 56-7, used on their previous Boeing 737NG, for the CFM Leap 1B, Really got my attention. To begin with, the CFM Leap 1B is much larger diagonally. There are thrust and weight differences, but the diameter is the problem. They were given orders in spite of their opposition. The installation crews had lots of difficulty fitting the larger engine. To install the engine, the engine must be moved forward and upward on their pylons. Engineering was well aware of the fact, such an installation would directly effect the weight and balance factor, which would alter the flight characteristics of the airliner. Knowing this should have caused the company to altar their plans. MCAS is nothing more than a fix to get by. I can assure you, I will not be one of the passengers on a 737 Max no matter what they call it. If an airplane is within weight and balance standards, the thumb adjustments on the yoke should be enough. All of this highly advanced technology, tends to create issues that can be deadly. A light touch on the controls of my aircraft is more than enough.
One thing I would suggest, is 100 or so hours of aerobatic training. My training saved my life Some years ago, when a pilot nearly struct my airplane in flight. I rolled over inverted as the other pilot nearly collided with me. I was upright in a flash, and happy to be alive, while the other fella was in my airspace.

 

[DaveE]

MCAS is nothing more than a fix to get by.

First I 100% agree with your opinions of Boeing management (FAA too, IMO). But my understanding is that the "fix to get by" was to avoid pilot training, and perhaps a new type rating, to deal with stall recovery characteristics that were significantly different from the previous versions. This appears to me to be essentially an exercise in putting poorly designed, tested, and approved software in the flight control systems to make planes easier to sell. For example, a flight control system that essentially overrides untrained pilots based on a single AOA sensor. Honestly, I wouldn't be upset if someone went to jail, but that won't happen since the FAA said it was ok to do, and since no one had sole responsibility for such a stupid decision.

However, with modified (i.e. limited) SW, a redundant AOA system, and newly required type-specific pilot training, I don't agree that this airplane is unsafe. In fact, because of the scrutiny this design has been subjected to, I actually have more confidence. My guess is that at the first master caution light 90% of pilots will be going to the stab trim cut-out switches (ok, just kidding). If I was to choose an airplane to be scared of, I might go with a 787 built in S. Carolina, they seem to have some QA issues there.

Also, while I know that you know more than I about flying, I am not convinced that aerobatic training in a small aircraft is very applicable to large transport aircraft. I would like to hear from someone that has experience with both, like any of the numerous commercial pilots that learn in military fighters. My understanding is that big planes maneuver more slowly.

 

[FactChecker]

Military airplanes have flight controls with some abilities that a pilot can not match. They can react faster. They also have some safety features to make sure that the pilot (even some excellent ones) do not crash. There have been examples where the chase plane of an experimental plane followed the experimental plane right into the ground because the chase pilot was so busy watching the experimental plane that he did not notice where he was going. Those were expert pilots. There is also an Automatic Ground Collision Avoidance System (AGCAS) on the F-16, F-22, and F-35 that has saved lives.

 

[russ_watters]

Military airplanes have flight controls with some abilities that a pilot can not match. They can react faster. They also have some safety features to make sure that the pilot (even some excellent ones) do not crash.

I'd amplify that to say that there are some military planes that are unflyable without their flight control avionics. As-in; you lose power, you eject, period. Clearly, an airliner can't be like that, though; it has to be flyable un-augmented. I'm sure you know that, I'm just pointing it out as a setup for later discussion:

There are thrust and weight differences, but the diameter is the problem. They were given orders in spite of their opposition...Engineering was well aware of the fact, such an installation would directly effect the weight and balance factor, which would alter the flight characteristics of the airliner. Knowing this should have caused the company to altar their plans. MCAS is nothing more than a fix to get by. I can assure you, I will not be one of the passengers on a 737 Max no matter what they call it. If an airplane is within weight and balance standards, the thumb adjustments on the yoke should be enough. All of this highly advanced technology, tends to create issues that can be deadly. A light touch on the controls of my aircraft is more than enough.

First I 100% agree with your opinions of Boeing management (FAA too, IMO). But my understanding is that the "fix to get by" was to avoid pilot training, and perhaps a new type rating, to deal with stall recovery characteristics that were significantly different from the previous versions...

However, with modified (i.e. limited) SW, a redundant AOA system, and newly required type-specific pilot training, I don't agree that this airplane is unsafe.

I'm on the same page as DaveE. While I don't think I have seen it explicitly stated, my understanding is the plane is still flyable with the automatic trim/stability augmentation features turned-off. Exactly how much more difficult it is to fly it is, I don't know, and I suppose that could matter if it is very difficult to control un-augmented. But I think it is important to recognize that the direct cause of the crashes was a faulty avionics system design, not faulty aerodynamics/handling.

So I also would have no qualms about flying in one.

During my research into the demise of both airliners, I found everything from the opinions of the general public, to memos and reports written by company employees. One such report was written by both company test pilots. Both had experienced problems with the MCAS system, and both reported the problems to the company. The problem is, the information the company received, fell on deaf ears. No attempt had been made to correct the MCAS system Issues.
I was most concerned about the conflict between engineering and management , over whether or not to re-engine a fifty year old design, that had been through four cycles of modification change. The discussion between 737 Max project engineering and project management was this, all appeared to oppose a re-engine attempt, on a fifty year old design. On the other hand, both the board members and top level management, approved
going ahead with a fifty year old design. The opposition were all for a new “Clean Sheet“ design. As a pilot and retired research and development engineer for P&W, I would also have opposed using a fifty year old design.

That part of it -- the businessmen vs engineers vs regulators part -- will certainly be debated in engineering ethics classes for decades. The fact that all of that is intertwined with a clear system design flaw makes it more complicated, but to me they are really separate issues. There are pros and cons to a clean-sheet, and it's not just about money. That 50 year old design has proven reliable and safe, and a clean-sheet will eliminate that track record and inject new risks into the system/process. The more new systems you design, the more opportunities you have for design flaws.

 

[FactChecker]

I'd amplify that to say that there are some military planes that are unflyable without their flight control avionics. As-in; you lose power, you eject, period. Clearly, an airliner can't be like that, though; it has to be flyable un-augmented. I'm sure you know that, I'm just pointing it out as a setup for later discussion:

Exactly. That is why those flight control systems have quad-redundancy with cross-checking and fault management. And there are backup emergency power systems. This all makes it hard for me to imagine the lack of redundancy in the Max MCAS system, especially if it is true that some relevant situation displays were changed from being available by default to available only for an extra cost.

I'm on the same page as DaveE. While I don't think I have seen it explicitly stated, my understanding is the plane is still flyable with the automatic trim/stability augmentation features turned-off. Exactly how much more difficult it is to fly it is, I don't know, and I suppose that could matter if it is very difficult to control un-augmented. But I think it is important to recognize that the direct cause of the crashes was a faulty avionics system design, not faulty aerodynamics/handling.

It sounded like the pilots were fighting against the faulty control system but had less authority (i.e. only for a shorter time). So they lost the fight.

So I also would have no qualms about flying in one.

Likewise. In fact, even the odds of a crash in the flawed system were fairly small.

That part of it -- the businessmen vs engineers vs regulators part -- will certainly be debated in engineering ethics classes for decades. The fact that all of that is intertwined with a clear system design flaw makes it more complicated, but to me they are really separate issues. There are pros and cons to a clean-sheet, and it's not just about money. That 50 year old design has proven reliable and safe, and a clean-sheet will eliminate that track record and inject new risks into the system/process. The more new systems you design, the more opportunities you have for design flaws.

Good point. A clean start would be extremely expensive and introduce a lot of unnecessary risk.[/quote]

 

[PeterDonis]

my understanding is the plane is still flyable with the automatic trim/stability augmentation features turned-off

It is, but the pilot would need to understand the handling characteristics--specifically, that the engines will create a pitch up moment that increases with increasing angle of attack. That makes it easy to stall the plane. It also means the stick force as a function of angle of attack will not be what is usually expected.

 

[PeterDonis]

That 50 year old design has proven reliable and safe

With the original engines, yes. The problem is that with the original engines, that 50 year old design is no longer competitive on fuel economy, but with the new engines, pilots need to be retrained. MCAS was an attempt on Boeing's part to avoid having to have pilots retrained; that attempt has failed. That significantly changes the cost-benefit analysis of trying to re-use the old design vs. doing a new design.

 

[hutchphd]

There are pros and cons to a clean-sheet, and it's not just about money. That 50 year old design has proven reliable and safe, and a clean-sheet will eliminate that track record and inject new risks into the system/process. The more new systems you design, the more opportunities you have for design flaws.

That is absolutely true. But if I understand the facts correctly, the design of MCAS was driven by requirements to eliminate pilot retraining at the expense of one more nontrivial layer of system complexity. That is the truly reprehensible action. Not the desire to maintain a good design. The fact that it was so badly implemented just adds to the sorrow

 

[russ_watters]

With the original engines, yes.

With or without the original engines. There's also wings, fuselage, landing gear, doors, windows, etc. That was my point: a lot of parts can be kept/not re-designed, all of which contribute to the safety record.

The problem is that with the original engines, that 50 year old design is no longer competitive on fuel economy, but with the new engines, pilots need to be retrained. MCAS was an attempt on Boeing's part to avoid having to have pilots retrained; that attempt has failed. That significantly changes the cost-benefit analysis of trying to re-use the old design vs. doing a new design.

It does, but only because they designed a buggy system. If they'd designed MCAS better from the start, we probably would never have heard of it.

 

[russ_watters]

That is absolutely true. But if I understand the facts correctly, the design of MCAS was driven by requirements to eliminate pilot retraining at the expense of one more nontrivial layer of system complexity. That is the truly reprehensible action.

I'm not following. What about that is reprehensible?

 

[PeterDonis]

a lot of parts can be kept/not re-designed, all of which contribute to the safety record

Yes, agreed. My intended point was about something different; see below.

If they'd designed MCAS better from the start, we probably would never have heard of it.

If they'd done things right from the start, the 737 MAX would probably not have existed because the pilot retraining would have been cost prohibitive; after all, the whole point of MCAS was to avoid that cost. Doing things right would have meant not avoiding that cost.

Or, alternatively, if Boeing had been willing to spend the amount of money it would have taken to do an MCAS-type system right and accept the costs of pilot retraining that would go along with that, there might have been more gains to be had from that investment by investing in a new design.

 

[russ_watters]

If they'd done things right from the start, the 737 MAX would probably not have existed because the pilot retraining would have been cost prohibitive; after all, the whole point of MCAS was to avoid that cost. Doing things right would have meant not avoiding that cost.

Or, alternatively, if Boeing had been willing to spend the amount of money it would have taken to do an MCAS-type system right and accept the costs of pilot retraining that would go along with that, there might have been more gains to be had from that investment by investing in a new design.

Ok, so isn't this speculation that if the FAA had taken a closer look at the system they would have mandated pilot training go along with it? Maybe the would have, maybe not.

Also, I don't think it's just "pilot retraining", but rather the type rating that is the issue. Even today, as far as I can tell, 737 pilots today do not need a new type rating to fly the 737 Max.
https://en.wikipedia.org/wiki/Boeing_737_MAX_certification

"pilot retraining" can be as minimal as an hour-long course on an ipad.

The part about the implementation cost of MCAS I agree with though. Ultimately what was done involved a major re-design of the flight control architecture to use sensors differently (shared instead of dedicated to different computers). How that would have affected the economics I don't know, though. Remember; we're talking different economics between the purchase price of the plane and the cost and complexity of implementation of a fleet for the airline.

 

[hutchphd]

I'm not following. What about that is reprehensible?

The inevitable result of these ongoing tradeoffs is to eventually produce an engineering product that is too labyrinthine to analyze. Folks understood this and did it anyway or perhaps were truly clueless apparatchik. It is reprehensible either way, all the more for a company with a legacy of engineering excellence in an industry that demands nothing less.

 

[anorlunda]

Sigh, long threads are tedious because things posted earlier may be forgotten.

This all makes it hard for me to imagine the lack of redundancy in the Max MCAS system

Earlier in this thread this was discussed. The Max had fully redundant A and B strings, with manual switching between them. In the accident planes, if the pilots had switched from A to B, the crashes may have been avoided.

The complaint is that A string has one AOA sensor, and B has one AOA sensor, but some people complain that both strings should have access to both sensors without manual switching. In that sense, A and B strings would no longer be fully independent. Cross-connections between strings introduce mutual dependencies and new kinds of common mode failures.

Boeing's design can be criticized, but it is unfair to characterize is as lack of redundancy.

If they'd done things right from the start, the 737 MAX would probably not have existed

Or, alternatively, if Boeing had been willing to spend the amount of money it would have taken to do an MCAS-type system right

The still better alternative, was also mentioned upthread. If I remember right, it was the option to redesign the landing gear to allow higher ground clearance instead of moving the engines forward. It was mentioned upthread that the new landing gear design had been completed for the 737-MAX-10 but not used in 737-MAX-9, thus leading to MCAS.

 

[russ_watters]

The inevitable result of these ongoing tradeoffs is to eventually produce an engineering product that is too labyrinthine to analyze.

I think that's a stretch, and really weird to apply a such a harsh value judgement such as "reprehensible" to it. How would you even measure such a thing? Ultimately the 737 Max is back in service and it still has MCAS. I don't know if the 737 Max is more or less complex than an A380 or 787, but I suspect it is substantially less complex. It should have been possible to make it work the first time.

 

[hutchphd]

I don't really understand your argument. Yes they have cobbled together a fix for the Frankenstein and it will rise. But 346 people are dead. My argument is that the proximate cause should not be substituted for the fundamental cause. The proximate cause was bad design verification and validation; the fundamental cause was choosing profit over good design practice. The proximate cause mistake is regrettable but the fundamental cause mistake is reprehensible.

 

[russ_watters]

I don't really understand your argument.

It's more confusion than an argument. I don't understand applying any value judgement at all to the idea of complexity. Systems can be simple or complex or really really complex. I've never thought of judging simple systems as good/moral, and complex systems as bad/immoral. It just makes no sense to me.

I judge morality of decisions, based on the calculus behind them. E.G., if you said you believed Boeing consciously made a decision they expected would kill people and chose to do it anyway because they'd profit, I'd consider that very bad, and would understand the judgement of "reprehensible". So...

Yes they have cobbled together a fix for the Frankenstein and it will rise. But 346 people are dead. My argument is that the proximate cause should not be substituted for the fundamental cause. The proximate cause was bad design verification and validation; the fundamental cause was choosing profit over good design practice. The proximate cause mistake is regrettable but the fundamental cause mistake is reprehensible.

Ahh -- so that's it: it's not about complexity, it's about a choice of profit over good design practice. Yes, that I can see. I don't know that we have a good handle on the details of the process, but I understand it is possible to imagine they knowingly cut corners or ignored clear signs of an issue. If that's true, that would be really bad.

But I don't know that we really know what went into the decision-making and I prefer to make positive assumptions where I don't know. And here, complacency and impatience would explain it. I judge this way in part because I'm sure they would know that a plane with a significant/fundamental flaw would likely crash and would likely undermine any profit motive. And because we've seen complacency and performance pressure (similar to, but not quite the same as a profit motive) in action before. So for now I choose to believe that the decisions were made with the expectation that they would be unlikely to substantially impact safety.

So I don't necessarily agree, mostly because I just don't know the details of the motivations/decision-making, but at least now I understand it.

 

[PeterDonis]

isn't this speculation that if the FAA had taken a closer look at the system they would have mandated pilot training go along with it?

No. At least, it's not speculation on my part. I'm just going by what we now know to be the primary driver of MCAS within Boeing: the desire to avoid pilot retraining, since they believed (correctly, as far as I can tell) that their airline customers would not buy the plane if they had to pay to retrain their pilots, and Boeing could not afford to pay for the pilot retraining themselves in order to sweeten the deal so the airlines would accept it.

I don't think it's just "pilot retraining", but rather the type rating that is the issue.

From what I understand, the cost of a new type rating would have been far less than the cost of pilot retraining. So even if a new type rating would have been required, I don't think that requirement was the primary driver of Boeing's thinking.

 

[FactChecker]

Sigh, long threads are tedious because things posted earlier may be forgotten.Earlier in this thread this was discussed. The Max had fully redundant A and B strings, with manual switching between them. In the accident planes, if the pilots had switched from A to B, the crashes may have been avoided.

Neither system had the necessary redundancy designed into that system. It is wrong to expect the pilot to know when to switch when the display that he needed to know what was going on was removed. Giving so much authority to a non-redundant system that even ignored contrary pilot input was especially unwise.

The complaint is that A string has one AOA sensor, and B has one AOA sensor, but some people complain that both strings should have access to both sensors without manual switching.

Absolutely!

In that sense, A and B strings would no longer be fully independent. Cross-connections between strings introduce mutual dependencies and new kinds of common mode failures.

Boeing's design can be criticized, but it is unfair to characterize is as lack of redundancy.

If a flight-critical system is given full authority, there should be cross-comparisons of the sensors and logic for a discrepancy. Some airplanes where the flight control is so flight-critical has a third flight control as a tie-breaker and even has a fourth flight control system as a back-up. I admit that designing such a system is a lot more work, but that is what a flight-critical system with full authority requires -- especially if it is going to over-ride contrary pilot inputs.

 

[russ_watters]

From what I understand, the cost of a new type rating would have been far less than the cost of pilot retraining.

I think you have it backwards. Or, rather, "pilot retraining" can be a little retraining whereas a new type rating is a lot of retraining.

 

[PeterDonis]

the option to redesign the landing gear to allow higher ground clearance

IIRC this was a nonstarter because airline customers would have had to rework jetways and their maintenance infrastructure, all of which were designed for the 737's existing ground clearance. For Southwest, in particular, I can imagine that cost would have been a deal breaker; their business model relies heavily on fast turnaround, which in turn relies on every piece of that turnaround being fine tuned for optimum efficiency around the existing 737 footprint and ground clearance.

 

[PeterDonis]

I think you have it backwards.

Possibly I do; it's been a while since I looked at this and I may be misremembering things.

 

[russ_watters]

Possibly I do; it's been a while since I looked at this and I may be misremembering things.

From the wiki linked in my prior post:

In the U.S., the MAX shares a compatible type rating throughout the Boeing 737 series.[28] The impetus for Boeing to build the 737 MAX was serious competition from the Airbus A320neo, which was a threat to win a major order for aircraft from American Airlines, a traditional customer for Boeing airplanes.[29] Boeing decided to update its 737, designed in the 1960s, rather than designing a clean sheet aircraft, which would have cost much more and taken years longer. Boeing's goal was to ensure the 737 MAX would not need a new type rating, which would require significant additional pilot training, adding unacceptably to the overall cost of the airplane for customers.