Solving DAQ Freeze Issues with Fail-Safe Alarm Device

[Yeti08]

I'll try to keep this concise. I am trying to build a alarm system for the DAQ system in my lab that will be software (LabVIEW) based and will replace the current hardware tripped alarms. Right now I am working on a final fail-safe alarm to protect against the freezing of the "alarm server" computer which sets off a telephone alarm (Sensaphone Express II) via a USB DAQ. As a mechanical engineer I'm not all that familiar with all the IC components available, and my electronics technician is more or less brand new.

If the computer freezes, the USB DAQ will also freeze in whatever state it is in. To make this an alarm condition, my first thought was to have a digital counter that would count to some threshold at which it would set off an alarm (currently an alarm is triggered by the signal changing from 5V to 0V). The USB DAQ could then send a reset signal every so often, so if the computer or DAQ froze the reset signal would not be sent and the counter would trip the alarm. My technician also had the idea of using an IC, I believe it was an LM555, which she explained could accept a pulsed signal and if it went outside the determined frequency, it could change its output from 5V to 0V, thus tripping the alarm.

Does anyone here have any suggestions or criticisms for either of these methods? Hopefully this makes some sense to you. Thanks for bearing with me.

 

[mgb_phys]

it's called a watchdog timer,
You have a separate counter that counts down for 60 seconds (or whatever interval you can tolerate) at the end it reboots the computer and possibly triggers the alarm.
Your software resets the counter every 30seconds or so.

They are often builtin to industrial control and embedded PCs, or your buy a plug-in card, or you could make one that connected to a serial port pretty easily.

 

[Yeti08]

Thanks for that. I've used software watchdog timers, but I didn't think to look for a hardware one. Now I just have to find one that will do what I want. Thanks.

 

[f95toli]

It is perhaps worth pointing out that there is a good reason why you rarely see the words "Labview" and "failsafe" in the same sentence...
Labview Realtime might work, but I would personally NEVER trust Labview for anything critical.
Also, how do you protect yourself from only Labview freezing (with the rest of the computer still running)?
It is not exactly unusual for Labview to just stop responding (it doesn't happen very day, but it DOES happen fairly regularly).

 

[Yeti08]

I've actually had fairly good luck with my LabVIEW programs; some have run months to a year without problems and they only had to be shut down for unrelated issues. That said, I have seen it freeze before, and that is precisely why I need this failsafe. This is a critical operation that has roughly $20M of equipment running. I have failsafes at every other plausible point of failure and I need one for the main LabVIEW alarm computer (there are several other computers feeding data to a central server). Having something after the DAQ that trips the alarm would, I believe, protect against the computer/OS freezing, LabVIEW freezing, and any sort of DAQ problem. A physical watchdog timer between the DAQ and alarm system should do that unless you can see any other holes in my plan.

 

[f95toli]

$20M!
If you insist on using Labview you should at least get Labview Realtime and run it on a RT OS (NOT Windows).
If the programs are small enough you could even run them off an FPGA (the FPGA DAQs from NI are actually quite nice and easy to program using Labview).
And yes, sometimes you are lucky and Labview works well; but neither "plain" Labview nor Windows are meant for this kind of application.

 

[Yeti08]

To be fair that is the total value for all the 10 or so current stations/experiments; all wouldn't be lost from a single failure. And this is really just something that I'm working with what I have. LabVIEW was being used before I arrived on existing computers (running Windows 2000 no less) and changing things where I work is a PITA (software is a nightmare to procure and hardware must be on an approved list, some things can take years to do if ever). I know that this is a less than ideal situation, but that is why I'm trying to cover all the bases. If a "client" computer misses a read an alarm is triggered; if the "server" computer freezes or shuts down an alarm is triggered; if the DAQ that triggers the alarm fails an alarm is triggered. These alarms make a phone call to personnel, one of whom is on call 24/7. If you can think of where or what can fail that I'm missing I really am open to suggestions, and that's why I'm trying to get some other peoples' inputs before I actually implement this plan.

On a side note, human error has been my most dangerous risk so far. In the past two months I've had a technician accidentally turn off alarms and equipment that, if someone else hadn't noticed, would have resulted in a lot of damage. These are things that the software alarm system would prevent. I would also have the hardware and software alarm systems running side-by-side for quite some time before being comfortable enough to leave anything soley to the software.

 

[JACK01]

I use Labview quite a lot and for any critical application create an external watchdog circuit in hardware. Labview then outputs a 20Hz (or so not critical) pulse to it using a digital I/O line. This bit of code runs in its own loop in th elabview program. If computer/labview crashes the pulse stops and wathdog circuit trips. Its important that the watchdog circuit has a relay on it that is only active when the pulse is present, hence fail safe. If you have a look at CNC zone forums you should find a suitable schematic for watchdog circuit. If any problms let me know.

Cheers

Jack

 

[nathanmc]

If you need a lot of outputs like the Express II but are looking for a rack mountable http://www.surewarning.com" I'd check out Absolute Automation. They have the IMS-4000 which is expandable and rack mountable.