NX-OS VEM Physical and Virtual Ports

VEM (Virtual Ethernet Module)

VSM (Virtual Supervisor Module)

Nexus 1000v supports the following

-2 VSMs (High Availability)

-64 VEMs

-512 Active VLANs

-2048 Ports (Eth + vEth) 

-256 Port Channels

VEM supports the following

-216 Ports (vETH

-32 Physical NICs

-8 Port Channels

NX-OS Fibre Channel Module

The Nexus 5000 can run in two modes

Fabric Mode - The Nexus 5000 switch module runs as a typical switch in a fibre channel network. 

NPV (N-Port Virtualization) Mode - Does not operate as a typical FC switch. Operates like a NPIV enabled host within a fabric.

NX-OS FCoE Ports

Nexus 5000 Feature that needs to be licensed.

FCoE (Fibre Channel over Ethernet) - is leveraged to further unify I/O.  FCoE allows fibre channel to operate over ethernet by encapsulating Fibre Channel into ethernet. 

FCoE Ports

Virutal N_Port (VN_Port) - Node ports which exist on hosts or storage arrays and connect to a FC fabric. Operates over Ethernet links.

Virtual F_Port (VF_Port) - Switch or director ports that connct to node ports. Operates over Ethernet links.

Virtual E_Port (VE_Port) - Expansion port that is used to inter-connect two FC switches together. When two swithes are connected they form an ISL (interswitch link) Operates over Ethernet links.

NX-OS Virtual SPAN

Virtual SPAN empowers a network administrator to SPAN more than 1 VLAN and enables the network administrator the ability to selectively chose which VLAN goes to what destination SPAN port. Example: A network administrator wants to SPAN a trunk port with VLAN 10, 20, and 30 but wants to send VLAN 10 to SPAN port ethernet 1/1, send VLAN 20 to SPAN port ethernet 1/2. and send VLAN 30 to SPAN port ethernet 1/3. Virtual SPAN enables that flexibility. This also helps reduce the number of SPAN sessions required. 

NX-OS SPAN

Nexus 7000/5000 SPAN Sessions

SPAN Session Limit - 18

Nexus 1000V

SPAN Session Limit (SPAN and ERSPAN) - 64

Nexus 5000 SPAN Sessions

Can SPAN Ethernet,Fibre Channel,PortChannel,SAN PortChannel,VLAN,VSAN (Virtual Storage Area Network)

NX-OS ISSU

ISSU (In-Service Software Update) - Provides the ability to upgrade software without disrupting operations. The system performs the following steps to ensure a non disruptive upgrade:

1. Active and Standby Supervisors and all line cards BIOS's are upgraded
2. Standby Supervisor is upgraded and rebooted
3. Once the Standby Supervisor comes online with the upgraded version of NX-OS a stateful switchover is performed. All control plane traffic is now running on the former Standby Supervisor which is now the Active Supervisor.
4. The new Standby Supervisor (former Active Supervisor) is now upgraded.
5. Upgrade is performed on the Line Cards on are a time and then reloaded. The reload is non disruptive and is only performed on the CPU, no data plane components are impacted.
6. CMP (Connectivity Management Processor) on both Supervisors are upgraded. 

NX-OS Stateful Switchover

Having redundant supervisors and a software architecture like NX-OS provides the ability to switchover to the redundant supervisor. Common reasons to fail-over include:

-ISSU (In Service Software Update)

-System Manager Initiated

-User Initiated

To manually switchover perform the following:

#system switchover

NX-OS 1000V Installation

Nexus 1000V can be installed within VMware using two methods

-Manual Installation

-Nexus 1000V Installer

When using an ISO image use the following settings for the VM

VMType: Other 64-bit Linux

1 Processor

2GB RAM

3 NICs

Minimum 3GB SCSI Disk

LSILogic adapter

Reserve 2 GB RAM for the VM

Configure VM Network adapters and attach ISO, power on

You can use and OVA/OVF (Open Virtualization Appliance/Open Virtualization Format) file to perform the install

Note: There is a Nexus 1000V plug-in that needs to be registered into VMware Virtual Center

NX-OS VEM Port Types

VEM Virtual Ethernet Module supports 3 port types

Virtual Nic - Three types of virtual NIC types are supported in VMware

• virutal NIC (vnic) - Physical port of an ESC host which is plugged into an switch. Assigned to a VM
• virtual kernal NIC(vmknic) - bound to a virtual ethernet port and used by the hypervisor for management, iSCSI, NFS, VMotion, and other network access that may be needed by the kernel.
• vswif - Service console network interface, virtual management port and mapped to veth within the Nexus 1000V switch. vswif0 is the first service console created.

Virtual Ethernet (vETH) port - this is where the virtual cable is plugged into from the VM, veth are assigned to port groups and represent a port on the Nexus 1000V Distributed Virtual Switch

 Local Virtual Ethernet (lvEth) port - Dynamically selected for vEth ports needed on a host. Local vEths ports do not move and are addressable by module/port number.

VEM Physical Ethernet supports 3 port types

VMware NIC

Uplink port

Ethernet port

NX-OS GOLD

GOLD - Generic Online Diagnostics empowers support staff to become proactive instead of reactive. GOLD helps identify hardware failures before they happen. This is not new to Cisco products but powerful feature to have included in the Nexus platform. GOLD tests and verifies the functionality of components at various times which enables support staff to become proactive. Tests that are performed can be executed with no system impact running in the background and other tests need to be run in a controlled environment as they may be disruptive to production.

GOLD suite of diagnostics include

-Bootup Diagnostics

-Runtime Diagnostics

-On-Demand Diagnostics

NX-OS Port Profiles

Port-Profiles can be used to streamline the configuration of ports that have a common configuration. You must create a port-profile with common settings and apply it to an interface.

Create a port-profile

(config)#port-profile PORTS

(config-ppm)#switchport

(config-ppm)#switchport mode access

(config-ppm)#spanning-tree port type edge

(config-ppm)#spanning-tree bpdufilter enable

(config-ppm)#no shutdown

(config-ppm)#state enabled

Apply to an interface

(config)#interface ethernet 1/1

(config-if)#inherit port-profile PORTS

NX-OS 802.1D-2004 (Dispute Mechanism)

Dispute Mechanism can prevent loops in the following scenarios 

-unidirectional links

-port-channel misconfiguration

This feature is enabled by default on the Nexus platform and cannot be disabled.

NNI

Recently I was asked to explain what an NNI was and I thought that this would make for a great topic for my blog

NNI - Network Network Interface. I will be explaining NNI in a GMPLS (Global MPLS) scenario.

The EMEA MPLS Provider is offering services but the Asia locations have to pay for expensive connectivity options over long distances. This increases latency and costs even when the Asia customers want to communicate with other Asia customers such as Hong Kong and Singapore.

The EMEA MPLS Provider partners with a Third Party MPLS Provider which provides a local MPLS network and has local presence in each center reducing the overall costs. This provides low latency connectivity options within the region. The EMEA MPLS Provider and the Third Party MPLS Provider create an NNI connection with each other and form a GMPLS network.

 

NX-OS User Modes

EXEC Mode - When you log in you are placed into the EXEC Mode. Commands include:

-show

-clear

-Other commands that perform actions that do not save into the device configuration

Global Configuration Mode - global commands that affect the device as a whole

to enter this mode enter "configure terminal". This also enables you to enter more specific configuration modes.

Interface Configuration  Command Mode (sub-mode example this is not the only sub-mode available)

(config)#interface ethernet 1/1

(config-if)#

Note: You do not need to specify speeds like you do in IOS such as "interface fastethernet 1/1" or "interface gigabitethernet 1/1", you just specify ethernet as the speed is determined by NX-OS and displayed in the respective "show" commands.

NX-OS Licensing

You must install a license from Cisco, you must copy the license file to flash.

To show the license file installed do the following

"show license host-id"

To install a license file

"install license bootflash:license_file.lic"

To install a 120 day grace period license for testing - Caution should be taken as the configuration is wiped automatically at the end of the grace period.

"license grace-period"

PPDIOO

Cisco PPDIOO Methodology

If you were to break up the process by job function what would be the result?

Job Functions

Architecture

Design Engineering

Deployment Engineering

Operations

Best Practices Part 1 - Layer 2 Spanning-Tree

The topology depicted in the diagrams is used to help demonstrate data flow during failure and to provide discussion around best practices and may not be necessarily be configured as optimal as possible. I will provide examples in a series of blogs that will provide alternate technical solutions that follow best practice guidelines.

Topology Image

Normal Data Path Flow

 Data Path Flow Root Fail

Data Path Flow-Access Trunk Fail

Data Path Flow Router Fail

Spanning-Tree mode Rapid-PVST (802.1w) or MST (802.1s) - I will show more about load balancing techniques leveraging each of these technologies in "Layer 2 Spanning-Tree Best Practices Part-2" Deterministic blocked ports - in this example we know exactly which ports are going to be blocked by STP. All redundant connections to the secondary root bridge will be blocked. Cisco also recommends that you do not exceed STP diameter of seven hops. Ensure that you hard configure your Root and Secondary Root bridges. Ensure that you only allow required VLAN's over the trunks to ensure you are not running unnecessary STP instances.

Features to leverage include:
Access Layer
-portfast
-bdpuguard
-disable DTP
-loopguard
-etherchannel Guard

Distribution Layer
-root and secondary root placement
-root guard
-disable DTP
-etherchannel Guard

Leverage EtherChannel to reduce the number of ports that need to transition from blocking to forwarding state when leveraging multiple links.

EtherChannel Ports
-EtherChannel Guard

TURN

TURN - Traversal Using Relay NAT - allows a device that is behind a firewall or NAT (Symmetric NAT or better known as Bi Directional NAT) device to receive incoming data leveraging TCP or UDP. TURN will most likely provide connectivity to the client but it does come at a high price to the provider. STUN is generally used first and TURN is used as a last resort.

The host sitting behind the NAT device is called a TURN client which connects the the TURN server on the public internet which acts as a relay. The TURN client communicates and arranges with the TURN server to have the server relay the packets to the desired peer. When the TURN client and peer want to communicate, the communication from the TURN client to the TURN Server is encapsulated within a TURN MESSAGE. The communication between the peer and the TURN server is not encapsulated.

TURN - RFC 5766

STUN

STUN - Session Traversal Utilities for NAT (RFC 5389) - used in NAT Traversal for applications real-time video, voice, messaging and other IP communications that are interactive.

STUN works with the following types of NAT
- Full cone NAT
- Restricted cone NAT
- Part Restricted cone NAT

STUN does not work with bi-directional NAT (Symmetric NAT). TURN works better with this type of NAT

STUN works as follows
- Client (OS or application) on a private network sends a "binding request" to the STUN server on the public internet.
- STUN Server sends "success response" that contains an IP address and PORT as observed from the the STUN servers. (After the Client has been natted)

Once the client is aware of its external IP address and port number it uses this external IP address and port number when communicating to its peers. This allows its peers to establish communications to the device which would otherwise not be accomplished since the client is on a private IP network.

Standard Ports for STUN
UDP/TCP 3478
TLS 5349