Dallas is our busiest location. We have many, many cabinets there filled with wonderfully shiny servers (all black, as befits our Kiwi origins). That setup has grown somewhat organically since 2005. Our core networking wiring was starting to be a little scary. And network capacity (ports and bandwidth was staring to be an issue).
So for our 10th year in Dallas we opted to rebuild our network from scratch. Get new, faster networking gear; tidily re-cable everything with fiber; simplify things as much as possible; and increase our uplink capacity.
Back in February we added a new core/distribution layer using Cisco Nexus 7 equipment. This gear lets us use lots (and lots) of 10G and 40G connections. And it does a pretty decent job at moving network packets around, too. The switchover from the old core to the new core went smoothly (thank you to those networking guys involved there!)
The new core gear has allowed us to increase our data center uplink capacity. (And we have plans to triple the capacity in the next few weeks).
We will also be switching from gigabit links between cabinets and our core networking to 10 gigabit fibre links. We have already done this for most of our cabinets, and the remainder should be done in the next few weeks (as the switches and cabling we ordered arrive). A number of cabinets have also had their switches upgraded (from non-fiber friendly models to newer models).
Since even 10G is not fast enough for everyone, in some cabinets we are installing 40G switches. With 10G access ports to each server. This gives us a few options for network attached storage and for some of our enterprise customers with particular private networking requirements.
The new Dallas networking setup should future proof us for a few more years.
Like all good network setups we hope you never have to pay it another thought!