Nexus 5000 FCoE Notes

Just some notes from reading about FCoE on the 5K.

I'm using this great reference from Cisco for these notes:
http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/mkt_ops_guides/513_n1_1/n5k_ops_fcoe.pdf

Some terms:

ENode - this is the FCoE capable Ethernet endpoint - usually a CNA.
VNP - Virtual NPV port.  The NP-port proxies fabric login (FLOGI) requests from end stations and converts them to Fabric Discoveries (FDISC).

I'll first start of with NPV. NPV can be enabled two ways on the 5K.

1. Feature FCOE and feature NPV - this requires a reload and I don't see any reason to use it.
2. Feature FCOE-NPV - this does not require a reload.

NPV proxies the FIP messages and FLOGI from the CNA/endpoint.   

Here are the types of ports used in the flow starting from the perspective of the CNA:
1.  The port on the 5K facing the CNA is a VF - virtual F port
2.  The port on the 5K facing the upstream NPIV enabled devices is a VNP
3.  The port on the NPIV enabled device is a VF port.

For the VFC interfaces, the NPV 5K has two sets; one set faces the CNA's as described above are F ports.  VSAN separation is done by simply assigning the correct VSAN to the interface. The upstream VFC is listed as a VNP port but in actuality it is a VTNP port as it trunks multiple VSAN's.

Here is a sample configuration from the Cisco document

switch(config)# feature fcoe-npv
switch(config)# vsan database
switch(config-vsan-db)# vsan 1-2
switch(config)# vlan 10
switch(config-vlan)# fcoe vsan 1
switch(config)# vlan 20
switch(config-vlan)# fcoe vsan 2
switch(config)# interface eth1/1
switch(config-if)# switchport mode trunk
switch(config)# interface Eth1/2
switch(config-if)# switchport mode trunk
switch(config)# interface vfc1
switch(config-if)# bind interface eth1/1
switch(config-if)# switchport trunk allowed vsan 1
switch(config-if)# no shut
switch(config)# interface vfc2
switch(config-if)# bind interface eth1/2
switch(config-if)# switchport trunk allowed vsan 2
switch(config-if)# no shut
switch(config)# interface vfc101
switch(config-if)# switchport trunk allowed vsan 10
switch(config-if)# switchport trunk allowed vsan add 20
switch(config-if)# no shut
switch(config)# vsan database
switch(config-vsan-db)# vsan 1 interface vfc1, vfc101
switch(config-vsan-db)# vsan 2 interface vfc2

FEX and FCoE

The NX-OS version on the lab blueprint (5.1.3) supports Enhanced VPC for FCoE.  The configuration guide warns that the same FEX should not be associated with multiple N5K's.

" Beginning with the Cisco NX-OS Release 5.1(3)N1(1), Cisco Nexus 5000 Series devices support FCoE on Enhanced vPC (EVPC). With EVPC topologies, it is important to ensure that FCoE traffic does not get forwarded from the FEXs to more than one Cisco Nexus 5000 Series device, because it violates the SAN fabric isolation rule.

This behavior is implemented by associating each FEX with only one Cisco Nexus 5000 Series device. The Cisco Nexus 2000 and Cisco Nexus 5000 pair is part of only one FCoE SAN fabric."


nexus5000-sanA(config)# fex 101
nexus5000-sanA(config-fex)# fcoe
nexus5000-sanB(config)# fex 102
nexus5000-sanB(config-fex)# fcoe