A lot of people have been asking me what I think of the recently released Tolly report comparing the bandwidth of the HP and Cisco blade solutions.
The short answer is, I don’t think much of it. It’s technically sound, and the the conclusions it reaches are perfectly reasonable – for the conditions of the tests they performed. In keeping with Tolly’s charter, the tests were repeatable, documented, and indisputable. The problem is, the results of the testing only tell half the story. The *conclusions* they reach, on the other hand, aren’t as defensible.
It’s really not necessary to get into a point by point rebuttal. At least not for me. I’m sure Cisco will be along any minute to do just that.
The facts that Tolly established during the report aren’t groundbreaking or surprising. Essentially, the tests were built to demonstrate the oversubscription of links between UCS chassis and Fabric Interconnects, which Cisco is quite willing to disclose at 2:1. These tests were paired with HP comparisons of blade-to-blade traffic on the same VLAN, which in HP architectures keeps the traffic local to the chassis. The interesting thing there is that if the traffic were between the same blades but in different VLANs, the Cisco and HP solutions would have performed identically (assuming the same aggregation-layer architecture). What makes that interesting is that the Tolly report’s figures depend on a specific configuration and test scenario – the Cisco values won’t change (or get any worse) no matter how you change the variables. The HP values will vary widely.
And that, my friends, is where I see the true benefit of Cisco’s architecture. Change the conditions of the test repeatedly, and you’ll get the same results.
I’m not faulting Tolly in this case. Not at all. They were asked to test a certain set of conditions, and did so thoroughly and presumably accurately. It’s just that you can’t take that set of data and use it to make any kind of useful comparison between the platforms. The real world is much more complicated than a strictly controlled set of test objectives. Do we really think that HP went to Tolly and asked “We want a fair fight?” Of course not.