NSX Controllers and VXLAN Cluster Installation

On to the next stage of deployment we need to setup our NSX Controllers and push out the VXLAN host modules and setup information to begin network virtualization.  These NSX Controllers assist the hosts in setting up the NSX Edge distributed router services across the hosts participating the NSX deployment.

To begin we should setup an IP Pool for your NSX Controllers to automatically assign out.   Login to the vCenter Web Client, select the "Networking & Security" option on the left hand side bar from the home screen.  From here select the "NSX Managers" option.

From here select the IP of your NSX Manager and then select the Manage tab, and the Grouping Objects subsection, select the IP Pools grouping object, and click on the + icon to add a new IP Pool.

You should run the NSX controllers in sets of odd numbers starting from 3 and up.  So allocate a pool with a minimum of three addresses.  The Add Pool process can also be done from the directly from the next section of the NSX Controllers deployment by clicking on the "Add Pool" option within the Add Controller dialog.

With you're IP Pool configured we'll move onto defining and deploying the NSX Controllers.
You can either click on the Back button inside the vCenter WebClient or goto Home and back to "Networking & Security".  Choose Installation from the left hand side menu now.

We'll need to wait while the first controller is deploying.   Any attempt to add another during deployment may cause the current to fail.   Wait until the controller deploys, powers up and then lists with a status of Normal with a green checkmark.  Once the first controller is up continue adding controllers. Add at least three controllers; for maximum availability and failover place each controller on a separate datastore.  Deploy controllers in sets of odd numbers.

Once you're controllers are prepared, we'll be moving onto the next tab Host Preparation. This portion of the installation will push out the ESXi host side components for the Distributed Firewall, and VXLAN services. 

 Just click on Install here and let NSX do it's magic to configure the hosts, go grab something to drink.
Once you're cluster has completed setup of the NSX package and the Distributed Firewall package you will see our familiar green check mark under both Installation status, and Firewall.  The installation field also keeps track of the version of the components installed.

The next step requires a dvSwitch, so we're going to configure one, add hosts to it, and then come back to the installation of the VXLAN service. Click on the Home icon in the vCenter Web Client and then select the Networking inventory section.

From the Networking screen right click on your Datacenter and select New distributed switch

Set the Switch name.

 Set the distributed switch version, both 5.5, and 5.1 are useable.

 Set the number of uplinks to match exactly to the number of uplinks you have.  This is a demo / proof of concept lab so I am only utilizing one uplink.  In production it's highly recommended to have two uplinks for maximum availability.  If this Distribute switch is only going to be utilized for VXLAN purposes I recommend not creating the default port group.  As when VXLAN is prepared a portgroup is automatically created for the vmknic VTEPs and any new Logical Switch created within will also have it's own portgroup generated.

Finish up and move on to Adding hosts.

Right click on your new dvSwitch and select Add or Manage Hosts

Select Add for the operation type, and click on next.

 Click on the green + New Hosts button if you are missing hosts from this list click the incompatible hosts list and you will be provided with a window and information about any hosts any why they cannot join.   Select all the hosts you want to admin to the distributed switch.

If you are a lucky man, all your hosts are the exact same hardware with all their NIC connected the same way and you can use template mode to join the hosts.

 

Next select the host to use as a template.

Choose which tasks we are going to perform.  We only need to assign uplinks, so we can remove the option to manage VMKernel Adapters.

 Set the uplinks for your template host, and then hit the Apply to all

 

 vCenter will now asses the impact!  Since we are not moving VMKernel Adapters or VM, and as long as you're not stealing a physical NIC that is already backing another distributed or standard switch that holds a VMKernel Adapter your screen should be telling you No impact.

 The final victory lap of setting up the distributed switch, after we're done here we can move back to the Networking & Security inventory section from the vCenter Web client home screen to begin configuration for VXLAN.

Back to the vCenter WebClient Home screen and back into Networking & Security and our Installation section, and click on the Configure option under the VXLAN column.  I have already created another IP Pool through the same process we went through to create an NSX Controller IP Pool, once again the Add Pool option is also available in the IP Pool drop down.  Be keen to match the VMKNic Teaming Policy with one that matches your uplinks.  I have chosen explicit failover as my VXLAN will be going over a single link.  Options are:

All Distributed Switch Versions:

Fail Over: which will give you an explicit fail over order and use a Active / Standby set up for all uplinks

Static Etherchannel:  which will give you an Active/Active Route by IP Hash for uplinks

Load Balance - SRCID: utilizes the normal route based on originating virtual port id 
Load Balance - SRCMAC: utilizes the alternate load balance algorithm based on source MAC address;

Distributed Switch 5.1:
LACP Active: which will give you route by IP Hash for uplinks
LACP Passive: which will give you route by IP Hash for uplinks; but the physical switch must be in Active state, or the Link will not be brought online.

Distributed Switch 5.5:
Enhanced LACP: Utilizes the Enhanced LACP configuration your vSphere 5.5 distributed Switch

 If you set your Recent Task panel to show All Users' Tasks you should see the the an Add Virtual NIC Task for each host, along with an Add Distributed Port Group, and many Update opaque data tasks.

If you switch over to the Logical Network Preparation tab, and the VXLAN Transport section you should now see all hosts listing with a green check mark and the VMKnic IP's that have been set up as VTEP endpoints.

Next move onto the Segment ID section, and define a Segment ID Pool, set a range that is equal to the number of VXLAN you expect to deploy.  You only need to define a Multicast range if you are using ESXi 5.1 hosts or plan on using Hybrid mode.  I will only be setting up Unicast mode so I have no configured any multicast informaiton.

Next move onto the Transport Zones section and click on the Green + to add a new Transport zone.  Choose the type of Transport method, Multicast which is compatible with 5.5, and 5.1 and is required if you have any 5.1 hosts.  Unicast which is compatible with 5.5 only, and Hybrid which uses both Multicast and Unicast. Select the cluster and distributed switch pairs you wish to join this Transport Zone.

We're now done with the VXLAN set up.  The final tab in the Installation section is Service Deployments this is used to push out VMware Endpoint, and VMware Data Security.  Also as third party services are registered with NSX you will be able to deploy them from this tab.  I will discuss these services in later posts.

To being adding VXLAN Logical Switches, select the Logical Switches section, and click on the green +.

If you switch over to the Networking view to check out your Distributed Switch you should now see two distributed port groups defined. vxw-vmknicPg-<dvs-##>-<vlanid>-<UUID>, and a vxw-<dvs-##>-virtualwire-<vtepgroup>-sid-<segmentid>-<logical switch name>

These break down to mean:
vxw is for VXLAN Wire
dvs-## is the dvs Managed object reference identifier
vlanid is the VLAN the VXLAN VTEPs are running in
UUID is a randomized unique identifier
vtepgroup is the VTEP group id the VXLAN is defined in
segmentid is the assigned segment id in use by this VXLAN wire
logical switch name is the name you assigned to the Logical switch

Next post I will cover setting up an NSX Edge as a routing point out of the VXLAN networks.