Tag: vnlink

Placement of mezzanine adapters in full width blades

During a discussion recently with some other UCS-savvy folks, I realized that there may be some confusion in how to place mezzanine adapters in full width blades when you’re only using one adapter.

First, a quick review of how UCS pins mezzanine ports to uplinks.   I’ll skip the one uplink option, since all ports use the single uplink.

In a two-uplink configuration, all odd numbered slots use uplink #1, while all even numbered slots use uplink #2.  (Easy to remember – odds go to the odd uplink, evens go to the even uplnk).   Since a full-width blade occupies two horizontally adjacent slots, each full width blade will contain both an even and an odd numbered mezzanine slot.  Let’s look at the slot numbering – this diagram shows half width blades.

Consider the case of 4 full width blades, all with a single mezzanine card in the “left” (as viewed from the front) slot.   In a two-uplink configuration, only the first uplink would be used – since all odd numbered slots are pinned to uplink #1.   Uplink #2 would be completely unused.

In a four uplink configuration, the pinning is as follows:

Ports 1,5 -> Uplink 1

Ports 2,6 -> Uplink 2

Ports 3,7 -> Uplink 3

Ports 4,8 -> Uplink 4

Consider the same previous example.   In the four-uplink configuration, only uplinks 1 and 3 are used, as these are the uplinks pinned to the odd-numbered mezzanine slots.   Uplinks 2 and 4 would remain idle.

When using full width blades and single mezzanine cards, it’s important to consider the other blades in the chassis when selecting where to install the mezzanine card.   Individual configurations will vary, depending on the number of half and full width blades in use.

If using all full width blades with single mezzanine cards, two of the blades should have the mezzanine card in the left slot and two blades should have the mezzanine card in the right slot.   By properly selecting the mezzanine placement, such as first blade (slots 1 and 2) in the left (1) position, the second blade (slots 3 and 4) in the left (3) position, third blade (slots 5 and 6) in the right (6) position, and the fourth blade (slots 7 and 8) in the right (8) position, you achieve proper usage of all four uplinks.   The inverse is also functionally identical, populating mezzanine slots 2, 4, 5, 7.

For a two-uplink configuration, any combination of mezzanine layout is acceptable as long as two blades have mezzanine cards in the left slots, and two have mezzanine cards in the right slots.  If beginning with a two-uplink configuration, it would be wise to use the four-uplink layout to minimize future reconfiguration if expanding to four uplinks.

What’s the difference between VN-Link and VN-Tag?

This is a question that constantly comes up in the classes and discussions that I’m involved in.

Part of the problem is that Cisco’s own documentation tends towards the marketing and less towards the technology.  Because of that, even a lot of Cisco folks are confused on exactly what is meant by each term.  Hopefully, this short summary will help.

VN-Link is a marketing umbrella term used by Cisco to describe any number of approaches to providing physical network type visibility to non-physical or non-directly attached devices.  This could mean virtual machines, could mean virtual interfaces on a remote interface card (a la Cisco’s Virtual Interface Card – aka Palo), or could mean physical interfaces on a non-switching remote device such as the Nexus 2000-series devices.

Probably the easiest way to group these is by the existence of a “Virtual Ethernet Port” on a switching device that is controlled like a physical port, but doesn’t directly map to a local physical switch port.

There are two approaches currently in use that fall under the VN-Link umbrella.

The Cisco Nexus 1000V switch, which is a software-only Cisco-branded switch that rides on top of VMware’s vNetwork Distributed Switch (DVS), is considered VN-Link because it provides virtual machine level visibility and granularity in network configuration and control.  Each virtual machine receives a “virtual Ethernet port” on the 1000V, which can be configured and controlled just like a physical Ethernet port would on a standard switch.

The Nexus 5000/2000 combination and the Cisco UCS Fabric Interconnect/IO Module combination both use an additional header in Ethernet frames called VN-Tag, which uniquely identifies some remote port which will receive a virtual Ethernet port on the local switch (50xx or 61xx).  This causes the Nexus 2000 or UCS IO Module to act as a remote line card to the host device, and doesn’t have to be managed individually.   All switching happens in the host device (50xx or 61xx).  The same VN-Tag technology is used by the Cisco Virtual Interface Card (VIC, or Palo) to identify the virtual interfaces being supported by the card.  With this tag added into the Ethernet frame, the host device (50xx or 61xx) can uniquely identify the source port (virtual or physical) and apply policy or configuration to it.

Note that the Nexus 1000V does not perform VN-Tag’ing – it simply doesn’t need to in order to meet it’s objective of providing VM-level visibility and control.

So both of these approaches meet the same architectural goal, while doing so with very different technologies.  Even so, they both fall under the same VN-Link umbrella.  Don’t confuse VN-Link, the “goal”, with the implementation.