FCoE on UCS/Nexus

This post is based on a Gold Lab that I did.  The purpose is to capture the basics of Multihop FCoE.

Multihop FCoE in this document is where FCoE is used both in the southbound and northbound directions from the point of view of the upstream Switch connected to FI. The lab used NPIV mode on the Nexus 5000. A port-channel was created from UCS to the FI and a a single link from the FI to the Nexus 5000.

Configuration Steps

Step 1 - Create the port-channels in UCS for the UCS to FI connectivity.  Very simple and I'm not going to show a screen shot.

Step 2 - Enable the NPIV feature and create the FCoE VSAN


Pod7-5548-A(config)# feature fcoe
Pod7-5548-A(config)# feature npiv
Pod7-5548-A(config)# vsan database
Pod7-5548-A(config-vsan-db)# vsan  77
Pod7-5548-A(config-vsan-db)# vl 77
Pod7-5548-A(config-vlan)# fcoe vsan 77

Pod7-5548-A# sh vlan fcoe

Original VLAN ID        Translated VSAN ID      Association State
----------------        ------------------      -----------------

      77                        77               Operational


Step 3 - Create and FCoE connection to UCS on top on the Ethernet

Pod7-5548-A(config)# int vfc 1
Pod7-5548-A(config-if)# sw mod f
Pod7-5548-A(config-if)# bind in po 12
Pod7-5548-A(config-if)# no sh
Pod7-5548-A(config-if)# sh int vfc1
vfc1 is trunking (Not all VSANs UP on the trunk)
    Bound interface is port-channel12
    Hardware is Ethernet
    Port WWN is 20:00:54:7f:ee:39:97:bf
    Admin port mode is F, trunk mode is on
    snmp link state traps are enabled
    Port mode is TF
    Port vsan is 1
    Trunk vsans (admin allowed and active) (1,77)
    Trunk vsans (up)                       ()
    Trunk vsans (isolated)                 ()
    Trunk vsans (initializing)             (1,77)


Step 4 - Create an FCoE connection up to the core

Pod7-5548-A(config-if)# int vfc2
Pod7-5548-A(config-if)# sw mo e
Pod7-5548-A(config-if)# bind int e1/15
Pod7-5548-A(config-if)# no sh
Pod7-5548-A(config-if)# sh int vfc2
vfc2 is trunking (Not all VSANs UP on the trunk)
    Bound interface is Ethernet1/15
    Hardware is Ethernet
    Port WWN is 20:01:54:7f:ee:39:97:bf
    Admin port mode is E, trunk mode is on
    snmp link state traps are enabled
    Port mode is TE
    Port vsan is 1
    Trunk vsans (admin allowed and active) (1,77)
    Trunk vsans (up)                       ()
    Trunk vsans (isolated)                 ()
    Trunk vsans (initializing)             (1,77)

Step 5 - Enable FC trunking on UCS for both Fabrics


Step 6- Create FCoE VSANs on both Fabrics



 Step 7 - Create FCoE port-channels

 
The VFC facing UCS now shows the VSAN up

Pod7-5548-A# sh int vfc1
vfc1 is trunking (Not all VSANs UP on the trunk)
    Bound interface is port-channel12
    Hardware is Ethernet
    Port WWN is 20:00:54:7f:ee:39:97:bf
    Admin port mode is F, trunk mode is on
    snmp link state traps are enabled
    Port mode is TF
    Port vsan is 1
    Trunk vsans (admin allowed and active) (1,77)
    Trunk vsans (up)                       (77)
    Trunk vsans (isolated)                 ()
    Trunk vsans (initializing)             (1)
    1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
    1 minute output rate 512 bits/sec, 64 bytes/sec, 0 frames/sec
      7 frames input, 900 bytes
        0 discards, 0 errors
      118 frames output, 17856 bytes
        0 discards, 0 errors
    last clearing of "show interface" counters never
    Interface last changed at Fri Jul  5 16:54:49 2013

 Step 8 - Provide SAN to ESX

Click on the vHBA under the service profile and create a vHBA for both the A and B chassis.