By Vishwas Manral, Distinguished Technologist, HP Networking, Advanced Technology Group
At the recent VMworld 2011 in Las Vegas, VXLAN made headlines when VMware CTO Steve Harrod called it “a new vision and approach.” Just in case you were hiding in some remote corner of the world and missed it, you can read this: VMware, Cisco stretch virtual LANs across the heavens: VXLAN virtualizes Layer 3 networks.
This topic has generally been beaten to death by various bloggers. Most blog posts have rightly gone to great lengths to elaborate the technology and its advantages. However I see one critical aspect that has been totally neglected. Let us examine the cost angle of the technology here. This should hopefully help you see some of the hard facts and make an informed decision about using the solution.
VXLAN is a technology that allows the server to do Layer-3 tunneling of server Ethernet traffic, even before it hits the first networking device. This contrasts with the Edge Virtual Bridging (EVB) approach many vendors have taken. This requires signaling between the server and the top-of-rack (ToR) switch with all the tunneling actually being done on the switch. The two approaches sound similar but have a very different cost dynamics.
Examining economies of scale
In the VXLAN context, when the switch moves into the server (vSwitch), the server needs a bigger, costlier CPU to achieve the networking functionality that may be required, such as switching, routing or security. (Think of all the ACLs and encryption that may be required to be done on the vSwitch when the ToR becomes opaque to application traffic.
As each vSwitch handles traffic only for that server, it needs to be sufficiently provisioned for CPU cycles to take care of traffic spikes that may occur. This means a lot of the CPU cycles stay idle in most cases, resulting in considerable additional costs. (Other interesting questions like scalability and management also need to be tackled but are probably best served with a separate blog post.)
However, if the switching, ACL’s and encryption/ decryption are all done in a common place, where the ToR handles traffic for all the servers attached to it, we gain the “economies of scale.”
Most current ToRs can handle the traffic at peak rates. However, by putting the heavy functionality on the ToR hardware, we gain economies of scale—hence optimizing costs. This is exactly what the EVB approach intends to achieve, by passing the load onto the central switch that handles traffic on behalf of multiple servers.
The EVB idea is nothing new. We have done a similar thing in the WLAN architecture in the enterprise where we have moved from architecture of multiple thick costly APs to a model where we have a few costly controllers and a lot of light APs.
What to keep in mind: VXLAN vs. EVB
To summarize, with each individual server provisioned for peak traffic, the cost of the server goes up. In data centers where there are a considerably larger number of servers than networking ToRs, the costs can considerably proliferate.
BTW, we should not forget about the ongoing associated power and cooling costs. The VXLAN solution can lead to considerably higher costs when compared to the EVB solution.
This is not to say VXLAN is never useful. There are certain advantages in the VXLAN solutions too. But before rushing in, vendors trying to use the solution should look at the cost they may have to incur with the increased server usage or increase in the number of servers that may result when the VXLAN solution is implemented.
What are your thoughts on the VXLAN topic?