Electrical interference in work environment

[ZedHead]

Hi!

I've been searching all over for a recommendation to my problem, but I've come to the conclusion it'd be better to ask.

I am a controls engineer, design and produce a lot of large scale automation products. Our system uses a basic PLC system tied to a plant management server rack running automation control software.

Now that i laid that out, here is the situation. One of my customers has a room that is about 100' x 80' dimensions. All of our process takes place in this room. There are two overhead cranes with magnets, tables with an electromagnetic system, and many many servos moving around. The site is in Connecticut which is known for having "dirty power". However, we have placed our own transformer inside the room to clean the power.

The problem is... Electronics are non stop dying. I replace 2 servers, 2 more die. Networking gear is constantly dying and PLCs are always acting up. I've even noticed walking around the room my cell phone screen will flash at times. Exiting the room required you to swipe your hand on a sensor, it always gives me a massive static shock on my fingers.

Does anyone have a theory of what could be happening here? I went to school for Software Engineering and studied Physics a bit... but I am hitting the end of my electricity knowledge on this guy

I've never seen so many computer components go bad at once.

Extra info: This room is inside a massive concrete manufacturing plant. They have electric ovens firing off outside the room for processing materials. A lot of high power stuff.

 

[meBigGuy]

1. Static (ESD) voltages will kill electronic systems. Just handing a PCB to a fellow employee without doing the ESD-Handshake first will destroy the card. If you spark to a board it is either gone, or on its way out.

2. Also, heavy machinery switching on and off will create huge line transients that will also kill systems. You can rent a powerline monitor to look for transients. When big motors turn off there can be huge transients.

You need a good UPS power conditioner to run the system, and need to treat the area around the computer equipment to eliminate ESD. You can spray rugs or floors to reduce ESD generation. If you were really careful, you would wear grounding wrist-straps when working on the servers. Research ESD control. You can also measure ESD.

One clue might be the nature of the failures. For example, if power supplies are going out, then line transients might be the primary culprit.

 

[ZedHead]

Thanks for the prompt reply!

I should have mentioned the servers are running off two massive UPS power conditioners. Also, i always wear an ESD bracelet when messing with computer hardware... learned my lesson in my teen years.

It's interesting you mention line transients. The computer power supplies haven't gone bad themselves, but the automation power supplies do very frequently. The are not plugged into a UPS and are just power supplies to the PLC control cabinets. I've been on site for about 2 weeks and have seen 3 of these 24v supplies just die.

Now you mention ESD, what would cause that in the room? How could you measure for it so i can show hard data to my customer? I talked with the operators on each shift and they have experienced static shock daily. Usually when touching fiberglass materials and other processing materials. I myself have only seen shocks when exiting the room using the hand-swipe to open the door.. its a pretty large static discharge

 

[phyzguy]

You might check the humidity in the room. If you want to keep ESD down, you need to keep the room humidity above about 35-40%.

 

[meBigGuy]

There are meters to measure ESD. Does the room have rugs? Some tile floors will generate ESD. There are sprays to minimize it. You could add mats to the walkways.

Are there interfaces between the computers and external equipment where grounding issues could be a problem? Maybe some isolation is needed there also.There are books on the subject too. One I have is an earlier version of https://www.amazon.com/dp/0470397047/?tag=pfamazon01-20 There may be better ones, but I'd recommend you do some reading.

 

[jim hardy]

"Die" is a pretty nondescriptive term.


Do power supplies go up in smoke? Or are input cards ceasing to work?

Electronics industry learned hard lessons in 1960's when solid state relays made their way into substations
and the IEEE Surge Withstand Capability standard was written.
That would be a place to start.


The basic physics is when a large current is interrupted a rapidly changing magnetic field results.
It will induce a brief but substantial voltage into any loop of wire that's nearby.
If a fragile FET multiplexer is on either end of that loop it will 'feel' that voltage.

Approach to correcting that is
1. Pay attention to wire routing, minimize enclosed area of wiring loops. Use twisted pair for signal leads.
2. Suppress surges with overvoltage absorber. Typical IEEE SWC circuitry uses combination of series inductance and parallel avalanche devices.

http://www.denverpels.org/Downloads/Denver_PELS_20070918_Hesterman_Voltage_Surge_Immunity.pdf

Do your I/O cards have transient protection ?