Quantcast
Channel: THWACK: Message List
Viewing all articles
Browse latest Browse all 20518

Re: What’s Your Config Error Nightmare?

$
0
0

When I worked for a national sports media group, I was tasked to extend the network for our radio division to a new transmitter they were testing.  This was on a Sunday; I had been given the project on a Friday, and the customer expected it up by Monday morning.  This required us to bypass most typical change controls, as I was the only network engineer working over the weekend.  During my rush to get the new switch online, I neglected to make sure I configured the Etherchannel on BOTH sides of the link (and we weren't using LACP).  Eventually, broadcast traffic looping through the network managed to kill all the clients...during a baseball playoff game...during a national commercial break.  Thanks to some quick thinking by the board operator, they only lost one national commercial...at an estimated loss of revenue of about $100k.

 

I was very happy to still have my job after that event took place...luckily, I had a boss who understood that mistakes are sometimes made (and since change controls were bypassed to get the switch online, I was not the only one to blame, even though I made the configuration error).


Viewing all articles
Browse latest Browse all 20518

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>