Monday, November 9, 2015
Do you like PowerCLI and NSX? Add Layer 3 VDR/LDR capabilities to esxcli / PowerCLI
Long time no write. I've been working with a quite a few customers recently and have been working on developing PowerCLI checks so that customers do not have to login to SSH terminals for every host to run NSX Sanity Checks. Remember to "Connect-VIServer" first!
PowerCLI version of "net-vdl2 -l"
netvdl_query.ps1
This will provide Format-Table outputs of Esxcli responses from all connected hosts in the targeted clusters. It lets you verify your VNI's are connected, online, and that controller connections are proper.
Namespaces queried:
$esxcli.network.vswitch.dvs.vmware.vxlan.get()
$esxcli.network.vswitch.dvs.vmware.vxlan.list()
$esxcli.network.vswitch.dvs.vmware.vxlan.network.list(vxlanvds)
$esxcli.network.ip.interface.list("vxlan")
$esxcli.network.ip.interface.ipv4.get()
PowerCLI C:\pcli\nsx> .\netvdl2_query.ps1 Physical
hivemind.united.earth
VXLAN Global States:
ControlplaneOutOfSync UDPport
--------------------- -------
No 8472
VXLAN VDS:
GatewayIP GatewayMAC MTU NetworkCount SegmentID VDSID VDSName VmknicCount
--------- ---------- --- ------------ --------- ----- ------- -----------
192.168.24.1 ff:ff:ff:ff:ff:ff 9000 9 192.168.24.0 28 2c 38 50 93 cc 28 37-8e 56 33 cd 7c 1a ea cf nsxswitch 2
VTEP VMKNIC:
Enabled ExternalID MACAddress MTU Name NetstackInstance OpaqueNetworkID OpaqueNetworkType PortID Portgroup
------- ---------- ---------- --- ---- ---------------- --------------- ----------------- ------ ---------
true N/A 00:50:56:6f:ca:99 9000 vmk1 vxlan N/A N/A 67108979 N/A
true N/A 00:50:56:68:17:93 9000 vmk2 vxlan N/A N/A 67108980 N/A
AddressType DHCPDNS IPv4Address IPv4Broadcast IPv4Netmask Name
----------- ------- ----------- ------------- ----------- ----
STATIC false 192.168.24.4 192.168.24.255 255.255.255.0 vmk1
STATIC false 192.168.24.7 192.168.24.255 255.255.255.0 vmk2
VXLAN VNI:
ARPEntryCount ControlPlane ControllerConnection MACEntryCount MulticastIP PortCount VXLANID
------------- ------------ -------------------- ------------- ----------- --------- -------
0 Enabled (multicast proxy,ARP proxy) 192.168.1.90 (up) 2 N/A (headend replication) 1 5001
0 Enabled (multicast proxy,ARP proxy) 192.168.1.90 (up) 2 N/A (headend replication) 2 5000
0 Enabled (multicast proxy,ARP proxy) 192.168.1.90 (up) 2 N/A (headend replication) 2 5006
0 Enabled (multicast proxy,ARP proxy) 192.168.1.90 (up) 2 N/A (headend replication) 1 5007
0 Enabled (multicast proxy,ARP proxy) 192.168.1.90 (up) 0 N/A (headend replication) 1 5002
0 Disabled 0.0.0.0 (down) 0 0.0.0.0 1 5004
0 Disabled 0.0.0.0 (down) 0 0.0.0.0 1 5005
0 Enabled (multicast proxy,ARP proxy) 192.168.1.90 (up) 2 N/A (headend replication) 2 5003
0 Enabled (multicast proxy,ARP proxy) 192.168.1.90 (up) 1 N/A (headend replication) 2 5009
Next up is a PowerCLI check to make sure a vdrPort exists on your hosts, Please note this script defaults to using "nsxswitch" as the DV Switch Name to Check, and VNI 5000, you can alter the script or specify the switch name and VNI manually as parameters. This script is useful in making sure your hosts haven't encountered the "Would Block" issue which can affect both VTEP, and LIF creation.
This verifies that a VDR instance has at least been created and was my initial work around to the net-vdr tool not being exposed by esxcli. It specifically looks for the "vdrPort" instance on your DV Switch.
vdrPort_check.ps1
PowerCLI C:\pcli\nsx> .\vdrPort_check.ps1 Physical nsxswitch 5001
hivemind.united.earth
1 vdrPort was found
hivemind2.united.earth
1 vdrPort was found
hivemind3.united.earth
1 vdrPort was found
Now to the bread and butter of the post. ESXCLI-NETVDR, have you wanted insight into your LDR/VDR instances on your hosts via PowerCLI but been left hanging because the Layer 3 information is no where to be found? The following VIB will let you get what you need! This allows access to a specific subset of net-vdr commands within ESXCLI, and more important extended into PowerCLI via the Get-ESXCLI object. Unfortunately at this time it is only CommunitySupported.
esxcli-netvdr.vib
It creates the following namespaces under ESXCLI:
network.vdr
network.vdr.nbr
network.vdr.route
network.vdr.lif
The following commands:
network.vdr.list
network.vdr.lif.list
network.vdr.nbr.clear
network.vdr.nbr.list
network.vdr.route.list
Here it is in action from an SSH session, but this is not what it's for, it's for PowerCLI
[root@hivemind:~] esxcli network vdr list
default+edge-15
VdrName: default+edge-15
VdrId: 0x00001389
NumberOfLifs: 1
NumberOfRoutes: 1
State: Enabled
ControllerIP: 192.168.1.90
ControlPlaneIP: 192.168.1.20
ControlPlaneActive: Yes
NumUniqueNexthops: 0
GenerationNumber: 0
EdgeActive: Yes
default+edge-4
VdrName: default+edge-4
VdrId: 0x00001388
NumberOfLifs: 7
NumberOfRoutes: 10
State: Enabled
ControllerIP: 192.168.1.90
ControlPlaneIP: 192.168.1.20
ControlPlaneActive: Yes
NumUniqueNexthops: 1
GenerationNumber: 0
EdgeActive: No
So how can I use these mystical commands via PowerCLI, once the VIB is installed make sure to restart management agents, or reboot your host. The ESXCLI namespaces won't be available until hostd is restarted. In all my examples below I will query my LDR instance "default+edge-4"
Start off by connecting to your vCenter "Connect-VIServer"
Then grab an ESXCLI Object from a host.
PowerCLI C:\pcli\nsx> Connect-VIServer cosmos
Name Port User
---- ---- ----
cosmos 443 NADESICO\dieth
PowerCLI C:\pcli\nsx> $esxcli = Get-ESXCLI -VMHost hivemind.united.earth
PowerCLI C:\pcli\nsx> $esxcli.network.vdr.list()
ControlPlaneActive : Yes
ControlPlaneIP : 192.168.1.20
ControllerIP : 192.168.1.90
EdgeActive : Yes
GenerationNumber : 0
NumUniqueNexthops : 0
NumberOfLifs : 1
NumberOfRoutes : 1
State : Enabled
VdrId : 0x00001389
VdrName : default+edge-15
ControlPlaneActive : Yes
ControlPlaneIP : 192.168.1.20
ControllerIP : 192.168.1.90
EdgeActive : No
GenerationNumber : 0
NumUniqueNexthops : 1
NumberOfLifs : 7
NumberOfRoutes : 10
State : Enabled
VdrId : 0x00001388
VdrName : default+edge-4
Want a list of just your VdrNames?
PowerCLI C:\pcli\nsx> $esxcli.network.vdr.list() | Select-Object VdrName
VdrName
-------
default+edge-15
default+edge-4
How about all the connected logical interfaces? (I've selected a subset of the columns to fit into the blog)
PowerCLI C:\pcli\nsx> $esxcli.network.vdr.lif.list("default+edge-4") | FT -Property LIFName,ID,ConnectedDvs,State,Mode -AutoSize
LIFName ID ConnectedDvs State Mode
------- -- ------------ ----- ----
13880000000f Vxlan:5000 nsxswitch Enabled Routing, Distributed, Internal
138800000002 Vxlan:5001 nsxswitch Enabled Routing, Distributed, Uplink
13880000000c Vxlan:5005 nsxswitch Enabled Routing, Distributed, Internal
13880000000b Vxlan:5004 nsxswitch Enabled Routing, Distributed, Internal
13880000000a Vxlan:5002 nsxswitch Enabled Routing, Distributed, Internal
13880000000e Vxlan:5007 nsxswitch Enabled Routing, Distributed, Internal
13880000000d Vxlan:5006 nsxswitch Enabled Routing, Distributed, Internal
How about all the neighbor information for a specific LIF? (Again I've selected a subset)
PowerCLI C:\pcli\nsx> $esxcli.network.vdr.nbr.list("138800000002","default+edge-4") | FT -Property Network,Interface,Mac,SrcPort -AutoSize
Network Interface Mac SrcPort
------- --------- --- -------
172.16.10.2 138800000002 02:50:56:56:44:52 0
172.16.10.1 138800000002 00:50:56:b8:1a:d0 67108868
What if I want all LIFs neighbor information?
PowerCLI C:\pcli\nsx> $esxcli.network.vdr.lif.list("default+edge-4") | %{ $esxcli.network.vdr.nbr.list($_.LIFName,"default+edge-4") } | FT Network,Interface,Mac,SrcPort -AutoSize
Network Interface Mac SrcPort
------- --------- --- -------
172.16.254.3 13880000000f 00:50:56:b8:68:36 67108998
172.16.254.1 13880000000f 02:50:56:56:44:52 0
172.16.10.2 138800000002 02:50:56:56:44:52 0
172.16.10.1 138800000002 00:50:56:b8:1a:d0 67108868
172.16.5.1 13880000000c 02:50:56:56:44:52 0
172.16.4.1 13880000000b 02:50:56:56:44:52 0
172.16.12.1 13880000000a 02:50:56:56:44:52 0
172.16.19.2 13880000000e 00:50:56:b8:48:48 67108868
172.16.19.1 13880000000e 02:50:56:56:44:52 0
172.16.8.1 13880000000d 02:50:56:56:44:52 0
172.16.8.2 13880000000d 00:50:56:b8:33:92 67108984
Finally what if I want to clear the neighbor information?
PowerCLI C:\pcli\nsx> $esxcli.network.vdr.lif.list("default+edge-4") | %{ $_.LIFName ; $esxcli.network.vdr.nbr.clear($_.LIFName,"default+edge-4") }
13880000000f
Complete
138800000002
Complete
13880000000c
Complete
13880000000b
Complete
13880000000a
Complete
13880000000e
Complete
13880000000d
Complete
Alternatively to clear neighbor information on a per LIF basis:
PowerCLI C:\pcli\nsx> $esxcli.network.vdr.nbr.clear("13880000000f","default+edge-4")
Complete
Last but not least list all routes available via from your specific LDR/VDR instances:
PowerCLI C:\pcli\nsx> $esxcli.network.vdr.route.list("default+edge-4") | FT -AutoSize
Destination Flags Gateway GenMask Interface Origin Ref Uptime
----------- ----- ------- ------- --------- ------ --- ------
0.0.0.0 UG 172.16.10.1 0.0.0.0 138800000002 AUTO 1 551207
172.16.4.0 UCI 0.0.0.0 255.255.255.0 13880000000b MANUAL 1 1768288
172.16.5.0 UCI 0.0.0.0 255.255.255.0 13880000000c MANUAL 1 1768288
172.16.7.0 UG 172.16.10.1 255.255.255.0 138800000002 AUTO 1 551207
172.16.8.0 UCI 0.0.0.0 255.255.255.0 13880000000d MANUAL 1 1768288
172.16.10.0 UCI 0.0.0.0 255.255.255.0 138800000002 MANUAL 1 1768288
172.16.12.0 UCI 0.0.0.0 255.255.255.0 13880000000a MANUAL 1 1768288
172.16.19.0 UCI 0.0.0.0 255.255.255.0 13880000000e MANUAL 11 1768288
172.16.254.0 UCI 0.0.0.0 255.255.255.0 13880000000f MANUAL 1 1768288
192.168.1.0 UG 172.16.10.1 255.255.255.0 138800000002 AUTO 6 551207
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment