As far as live tests go: in my opinion, one has to do a full power cutover test from street to generator on a regular basis (although not once a week for sure) to be sure that the data center responds properly. If you never test that your ATS switches between power properly, that your UPS’ can handle their load and that the generator powers on properly and stays up during load, you’re just asking to get blown out of the water in case of a real emergency. That being said, these live tests are best done with staff on-hand capable of troubleshooting any problems that arise and also at such a time that load on the systems is at their lowest so as to cause the least amount of downtime should a failure occur. Ideally, you should also be ready for a cutover to the DR site.

As far as the redundant cooling…tell me about it. Kinda crazy not to have that.

So if the tests fail, everyone dies.

That’s… umm… different than the way most places I’ve been to do it. I won’t even get into the non-redundant cooling :)

You’re bad juju, helly.