Table of Contents
HPE Synergy and Aruba CX 10000 Networking Deployment Guide
10000 Switch Series Software Documentation
Aruba cx 10000 series datasheet
HPE Synergy and Aruba CX 10000 NetworkingDeployment Guide
HPE Aruba Networking CX 10000 Distributed Services Switch
Contents
Introduction
The Aruba CX 10000 Series Switch with Pensando represents a new category of data center switches that combines best-of-breed Aruba data center L2/3 switching with the industry's only, fully programmable data-processing unit (DPU). The Pensando Elba DPU is able to deliver stateful software-defined services inline, at scale, with wire-rate performance and orders of magnitude scale and performance improvements over traditional data center L2/3 switches at a fraction of their TCO. The Pensando Policy and Services Manager (PSM) is a distributed system, leveraging an intent-based model that delivers network and security policy to CX 10000 DPUs at the edge. Aruba Fabric Composer is an intelligent, API-driven, software-defined orchestration solution that simplifies and accelerates network provisioning, security management and day-to-day operations across rack-scale compute and storage infrastructure. PSM integration allows network security policies to be configured directly from Aruba Fabric Composer. Aruba Fabric Composer together with the Pensando Policy and Services Manager (PSM) provides a distributed system, leveraging an intent-based model that delivers network and security policy to Aruba CX 10000 DPUs at the edge. This document provides networking guidance when building out a solution that leverages an HPE Synergy enclosure connected to an Aruba CX 10000 network managed by Aruba Fabric Composer and PSM.
Overview
As shown in Figure 1, Aruba Fabric Composer provides underlay and overlay automation for the Data Center fabric comprising Aruba CX 8325 Spines and CX 10000 Leafs which connect to rack mount servers and blade server enclosures such as HPE Synergy.
*Figure 1. Data Center with CX 8325 Spines, CX 10000 Leafs and HPE Synergy*
For network redundancy and traffic load sharing, HPE Synergy Virtual Connect modules connect to the Aruba CX 10000s via a Virtual Switching Extension (VSX) Link Aggregation Group (LAG). “Tunnel” mode would be used on Virtual Connect to simplify and minimize network configuration within HPE OneView. In addition, when Aruba Fabric Composer (AFC) is integrated with VMware vCenter, you can easily create vSphere Distributed Switches (vDS), PVLAN, port groups and visualize those network connections between the physical switches, Virtual Connect ports, vSwitches, Virtual Machines (VMs). The Aruba CX 10000 network security policies provide east/west firewall capabilities between Servers, Virtual Machines (VMs) or Containers within a VLAN or across different network VLAN/subnets. If micro-segmentation within the same hypervisor or across different hypervisors and within the same subnet is required for network isolation and security policies, Private VLAN (PVLAN) can be utilized. This guide uses VMware vSphere Distributed Switch on HPE Synergy compute modules as an example.
Prerequisites
Before connecting and configuring an HPE Synergy enclosure to the Aruba CX 10000 network, the following are recommended:
- Aruba CX 10000s loaded and booted up with code version 10.09.0010 (minimum)
- Ports between devices should be cabled up and connected without errors
- A separate Out Of Band (OOB) network for management
- Aruba CX 6300 switches are suitable for the OOB network
- Mgmt ports of the Aruba CX 8325/10000 switches, Aruba Fabric Composer and PSM should be connected to the OOB network and have IP reachability between each other
- Aruba Fabric Composer 6.2 (minimum) should be functional, refer to Aruba Support Portal
- This guide is focused on HPE Synergy with Aruba CX 10000 switches, refer to Aruba Fabric Composer documentation for Aruba Fabric Composer high availability deployment recommendations
- PSM 1.29.2-T-11 (minimum) should be functional, refer to Aruba Support Portal
- This guide is focused on HPE Synergy with Aruba CX 10000 switches. Please refer to PSM documentation for PSM high availability deployment recommendations
- HPE OneView and HPE Synergy should be functional, refer to HPE Synergy Documentation Quick Links for more information on HPE Synergy networking
- For reference, this guide used firmware 1.7.1.1001 for interconnects, firmware 6.4.00 for Oneview and enclosure bundles
- For flow-logging to appear properly within PSM, make sure that the time set within the attached PC/VM is set to the same time as PSM.
Detailed Topology
As this guide is focused on HPE Synergy with Aruba CX 10000 switches, the detailed topology as shown in Figure 2 will be used and referenced for the rest of this guide.
*Figure 2. Data Center rack with Aruba CX 10000 Leafs and HPE Synergy*
3 VLANs are used in this guide:
- VLAN 200 – Primary PVLAN
- Int VLAN 200 (10.10.200.254/24) on Aruba CX 10000s will function as default gateways for VLAN 200 and VLAN 201.
- Suitable for VMs that do not require L2 micro-segmentation on the 10.10.200.0/24 subnet.
- VLAN 201 – Isolated PVLAN
- Suitable for VMs that require L2 micro-segmentation on the 10.10.200.0/24 subnet.
- VLAN 202 – Normal VLAN (Non PVLAN)
- Interface VLAN 202 (10.10.202.254/24) on Aruba CX 10000s will function as default gateways for VLAN 202.
- Suitable for VMs that do not require L2 micro-segmentation on the 10.10.202.0/24 subnet.
For network redundancy and traffic load sharing, 1 x 40G uplink on each HPE Synergy Virtual Connect module connects to a pair of Aruba CX 10000s via VSX LAG. LACP is only required between Aruba CX 10000s and the Virtual Connect modules. The vDS doesn't require LACP to Virtual Connect modules. When “Tunnel” mode is configured on Virtual Connect, there is no requirement to create VLANs or PVLANs in OneView. Note that connectivity options from the Synergy enclosure to the Aruba CX 10000 switches will vary based on the VC module in use and the constraints of the environment. Depending on the VC module chosen users could use 40G or 100G interfaces, but they could also use 4x10G or 4x25G interfaces to connect to each ToR switch. Please refer to the Synergy site for more details on VC modules available - https://www.hpe.com/us/en/integratedsystems/synergy.html
To implement east/west firewall capabilities between VMs on different subnets, security policies configured on Aruba Fabric Composer are pushed down via PSM to the Aruba CX 10000 DPUs to permit or deny desired traffic. The remainder of this guide walks through the tasks required for this deployment to be successful and should be done sequentially.
Task 1: AOS-CX Switches
Using console access, enter the base configuration below required on AOS-CX switches for Aruba Fabric Composer management, modify hostnames and IPs as required.
configure hostname CX10000-R1RU33-SW1 interface mgmt ip static 10.251.X.5/24 default-gateway 10.251.X.254
If you need ports changed from 25g to 10g, you will need to modify the interface-group as required.
system interface-group 1 speed 10g !interface group 1 contains ports 1/1/1-1/1/4 system interface-group 5 speed 10g !interface group 5 contains ports 1/1/17-1/1/20
When managed by Aruba Fabric Composer, the majority of Aruba CX 10000 features can be configured from the Aruba Fabric Composer GUI. To ensure all traffic is inspected by the security policy, this should also be added at the global level.
no ip icmp redirect
Task 2: Aruba Fabric Composer
This section provides guidance on automating the network fabric, validating connectivity to the Aruba CX switches, PSM, and integration with vSphere/PSM.
###### Guided Setup After logging into Aruba Fabric Composer, click the guided network setup (green icon with 3 …) on the right. Click on “SWITCHES” to discover new switches, enter IPs and desired passwords to be pushed down to the switches. The switches should be added but unassigned. Click on “FABRIC” on the guided network setup to add a fabric, enter desired info and click “APPLY”.
###### Switch to Fabric Association After the switches are discovered, you can assign switches to the fabric by clicking on “ASSIGN SWITCH TO FABRIC” in the guided setup workflow on the right. Add the switches, select a fabric, desired role, initialize the ports and click “ADD” and then “APPLY”. After a minute or so, the switches should appear healthy and synced with the assigned fabric. Users can continue to follow the workflow on the right and now add NTP/DNS configurations. The configs are straight forward so this guide continues to the VSX setup portion of the workflow.
###### VSX Configuration Click “VSX CONFIGURATION” in the guided setup workflow on the right. Select “Automatically generate VSX Pairs” and click “NEXT”. Add desired name and click “NEXT”. Use default values and click “NEXT”. Enter desired keepalive interface mode and add an address pool. Click “ADD” to provide a name for the address pool, and then click “NEXT”. Add a subnet range and then click “NEXT” and then “APPLY”. Click “NEXT”. For Keepalive setting, keep as default values, and click “NEXT”. Leave the Linkup timer as default and click “ADD” to add a MAC Address Resource Pool, or simply add the desired MAC Address range needed. Click “NEXT”. Review the parameters and click “APPLY”. VSX should now be operational (you may need to click refresh for AFC to reflect that). Make sure that parameters are operational, in_sync, and peer_established.
###### Persona Configuration Click on “Configurations” > “Ports” > “Ports” > (icon) > select desired switch on top right drop down menu. Users can select multiple switches, if desired – in this example select both Aruba CX 10000 switches. Select the ports facing the server (both 1/1/49 in this example) and click > Actions > Port Type. Select the “Access” port type and then click “OK”.
###### VSX LAG Configuration Click on “Configurations” > “Ports” > “Link Aggregation Groups” > “ACTIONS” > “Add”. Select a single LAG option and click “NEXT”. Enter a desired name, LAG number and then click “NEXT”. Select the VSX switches/ports connected to Synergy Virtual Connect and click “NEXT”. Select the switches and enter the desired values and click “NEXT”. Enter a native VLAN, desired VLANs to be allowed, and click “NEXT” and “APPLY”. VSX LAG info can be seen if you expand it out.
###### VRF and SVI Configuration Click on “Configurations” > “Routing” > “VRF” > “ACTIONS” > “Add”. Enter the desired name and click “NEXT”. You can either “Apply” the config to all switches or select specific switches and click “NEXT”. “L3 VNI” and “Route Targets” are not required if VXLAN is not used, so for this example click “NEXT” on both screens to proceed. Review the parameters and click “APPLY”. After the desired VRF is created, select that VRF and then click on “ACTIONS” > “IP Interfaces”. Under “IP Interfaces”, click on “ACTIONS” > “Add”. Enter your desired VLAN, switches, and subnet. Enter your desired SVI IP range, active gateway IP and MAC. “Enable Local Proxy ARP” should be selected for primary PVLAN SVI (VLAN 200) to allow VMs on the same isolated PVLAN to communicate if desired via the security policy. “Enable Local Proxy ARP” is not required for normal VLAN 202. Click “NEXT”. Enter an optional name, description and then click “NEXT”. Review the parameters and then click “APPLY”. SVIs should now be operational, repeat for other desired SVIs. In this guide, only VLANs 200 and 202 require SVIs. VLAN 201 is an isolated PVLAN and uses VLAN 200 primary PVLAN as the SVI for the 10.10.200.0/24 subnet.
###### PVLAN Configuration Click on “Configurations” > “Ports” > “PVLANS” > “ACTIONS” > “Add”. Give the PVLAN config a name and then click “NEXT”. Select the desired switches and then click “NEXT”. Enter the Primary VLAN 200 and then click “NEXT”. Enter the Isolated VLAN 201 and then click “NEXT”. Click “NEXT” on the “Secondary Ports” screen. Review and click “APPLY”.
###### PSM Integration The Pensando integration allows you to:
- Configure security policies in Aruba Fabric Composer
- Or configure security policies in PSM
- This may be required in scenarios where the security and network team are separate entities and the networking team is not allowed to manage the security policies
In the guided setup, select “DISTRIBUTED SERVICES” > “Pensando PSM”. Enter the required values. Click “VALIDATE” and if successful, click “NEXT”. Select the fabric, then both check box options, click “NEXT”, review and then click “APPLY”. The connected status should now be shown. You can verify REST API connectivity between the Aruba CX 10000 switch and PSM is operational by clicking on “CLI Commands” next to “Guide Setup” icon. Select your fabric or switches, type your desired “show” command, hit enter and click “RUN”. The Aruba CX 10000 swicthes should be shown as “admitted” into PSM.
###### VMware vCenter Integration The vSphere integration will allow you to view vSphere hosts in Aruba Fabric Composer, automatically deploy network configurations based on VM deployment on vSphere and visualize network connections between physical switches, Virtual Connect ports, vSwitches and VMs. Click on “Configurations” > “Integrations” > “VMware vSphere” > “ACTIONS” > “Add”. Enter the required values. Click “Validate” and if successful, then click “NEXT”. Select all 3 options and then click “NEXT”. Select “Discovery protocols” and then click “NEXT”. Review the parameters and then click “APPLY”. The connected status should be shown.
Task 3: HPE Synergy Networking
This section provides guidance on networks, logical interconnects, and server profiles to connect to the Aruba CX 10000 switches.
###### Networks From OneView, select “NETWORKING” > “Networks” and create a network with VLAN set to “Tunnel”.
###### Logical Interconnects Within “Networking” > “Logical Interconnect Groups”, click “Edit”, then select “Add uplink set”. Create an uplink set with the “Type” set to “Tunnel” and add the desired uplink ports. From “Networking” > “Logical Interconnects”, you will notice an inconsistency error. Select “Actions” > “Update from group” to fix it. If VSX LAG is configured on the Aruba CX 10000s from the previous section, you can validate the LACP LAG is operational between Virtual Connect and the CX 10000s under the “Uplinks Sets” section. The interconnects should be green with “Linked active” state, “LACP activity” and the attached switch neighbor info should also be seen.
###### Server Profiles Edit your “Server Profile Template” and “Add Connection” with “Network” set to “Tunnel-uplink”. “Tunnel-uplink” connection should be created. From “Server Profiles”, you will see it is inconsistent with its “server profile template”, you can fix it by clicking on “Update from template”. You can update it by selecting “Update from template after power is off” and “Momentary press”. If desired, VMs should be migrated to another hypervisor before powering the compute module off. Repeat this step for all server profiles that require the “Tunnel” uplink. Once the server profile update is complete, you will be able to check the MAC address of the “Tunnel” uplink by clicking on “Edit” > “Connections”. This MAC will show up in vSphere as a physical NIC.
Task 4: VMware vDS
After Aruba Fabric Composer is integrated with VMware vSphere, from Aruba Fabric Composer you will be able to:
- Create VMware vDS and assign VMNICs
- Create PVLANs in vDS
In Aruba Fabric Composer, click on “Visualizations” > “Hosts” > select your desired hypervisors in the bottom selection window and unselect undesired hypervisors. You should see the new VMNIC with mac address matching the previous screenshot in OneView, that is the VMNIC that should be assigned into the vDS. To create the vDS, click on the desired hypervisor > “Create Microsegmentation”. Input the desired names and select desired NICs. Multiple VMINCs on the same host can be selected if available. Only the VMNIC MAC assigned to the “tunnel uplink” is required. Click “NEXT”. Enter the Primary PVLAN (200), Isolated PVLAN (201), and click “ADD”, click NEXT“ and then “APPLY”. Once done, you should be able to see the vDS, VMNIC and port group. On other desired hypervisors, select “Update Microsegmentation” to add additional hypervisors and VMNICs to the vDS. Once done, you should be able to see the vDS, VMNIC and port group. You will need to create non PVLAN port groups directly in vCenter by clicking on the “Networking” icon > right click desired vDS > “Distributed Port Group” > “New Distributed Port Group”. Enter the desired values, click “NEXT” and then “FINISH”. The next step is to update the VM network adapters to the desired port group in vCenter. This example shows a VM with 10.10.200.2/24 IP assigned to an isolated PVLAN VLAN 201 port group. This example shows a VM with 10.10.202.2/24 IP assigned to the normal VLAN 202 port group. Repeat and assign other VMs to their desired port group. For visualizations in Aruba Fabric Composer to display correctly, the ESXi hypervisors should have unique host names, DNS domain and DNS server configured under host > Networking > TCP/IP Configuration > Edit > Default. After the VMs are attached to port groups, the “Visualizations” pane should be updated with VMNIC/vDS/port group/VM links. In addition, the VMNIC links to the VC module and the Aruba CX 10000 switchports should also be shown. If LACP is required on the Aruba Fabric Composer created vDS pointed towards the switches, you can enable LACP support by selecting the vDS > “Upgrade” > “Enhance LACP Support” > “Next” > “Finish”. This is only applicable if the vDS is used on rack mount servers, if the vDS is only used within a Synergy enclosure, LACP is not required as LACP is enabled at the Virtual Connect level.
Task 5: Distributed Services
This section provides guidance on deploying distributed services on Aruba Fabric Composer, PSM and the Aruba CX 10000 switches.
Network Configuration Next step is to add VLANs to be redirected/inspected by the Aruba CX 10000 DPUs. Click on guided setup icon > “DISTRIBUTED SERVICES” > select the desired VRF > “CONFIGURE NETWORKS” > “ACTIONS” > “Add”. Add desired name and click “NEXT”. Add desired VLAN, click “NEXT” and “APPLY”. Repeat for all desired VLANs with SVIs. VLANs 200 and 202 are configured with SVIs used in this guide. VLAN 201 is an isolated PVLAN that utilizes SVI 200 and doesn't need to be created here.
###### Distributed Firewall Configuration Click on guided setup icon > “DISTRIBUTED SERVICES” > select desired VRF > “CONFIGURE POLICY” > “ACTIONS” > “Add”. Enter the desired policy name and then click “NEXT”. Select “Distributed Firewall” and then click “NEXT”. Select “ACTIONS” > “Add” > “New”. This example permits RDP traffic from Web Tier to App Tier, e.g. VM (10.10.200.3/32) to VM (10.10.202.2/32). Click “NEXT”. Select the desired action and click “NEXT”. Select “ADD” to add new source endpoint group. Enter the desired name and click next. Select, “ADD” desired VM, click “NEXT” and then “APPLY”. Click “ADD” on the destination endpoint group. Name the endpoint group and then click “NEXT”. Select, add desired VM (this example shows you can add multiple endpoints), click “NEXT” and then “APPLY”. Click “NEXT” once both desired source and destination endpoint groups are added. Select existing “Service Qualifier” if applicable, or click “ADD” to add a new service qualifier and then click “NEXT”. Review and click “APPLY”. Take note there is an implicit deny at the end of the policy. You can add more rules to permit traffic by clicking on “Actions” > “Add” > “New”. An “ICMP-permit” rule is used in this example to allow VMs to check network connectivity. Select the desired action and then click “NEXT”. Leave the endpoint groups empty to match on “any”, and then click “NEXT”. Select the desired service qualifier, click “NEXT”, review and then click “APPLY”. Review, add additional rules, e.g. allow Web to App desired traffic and then click “NEXT”. Take note the policy is applied to the entire network fabric (all Aruba CX 10000 switches), you will not be able to select only specific CX 10000s. Select
- Desired Fabric
- Desired direction
- Egress refers to policy applied from workload perspective, we are trying to permit outbound traffic egress from Web Tier to App Tier, e.g. VM (10.10.200.3/32) to VM (10.10.202.2/32)
- Desired VRF
- Desired Networks
Click on “ADD”. Click “NEXT” and then “APPLY”. The configured policy should be seen as healthy. You can validate the security policy is pushed down to PSM and CX 10000 DPUs in PSM GUI > “Tenants” > “Security Policies” > policy name. And the policy is attached to desired networks in PSM > “Tenants” > “Networks”.
Task 6: Security Policy Validation & FW Logging
VMs on the same isolated PVLAN normally do not have network connectivity between each other. However, with local proxy arp enabled on VLAN 200, traffic between VMs on the same isolated PVLAN can be subjected to a security policy for traffic to be allowed or denied. With the security policy in place, we can verify 10.10.200.3 VM in WebTier is able to ping to VMs in the same subnet (10.10.200.0/24) and in the AppTier (10.10.202.0/24) subnet due to the “ICMP-permit” rule. It is also able to connect via RDP to the 10.10.202.2 VM due to the “RDP-permit” rule. Due to the implicit deny rule, RDP between VMs on the same subnet are denied as expected. You can view hitcounts towards each rule in Aruba Fabric Composer by clicking on “Configurations” > “Policy” > “Policies” > ”…“ > “Rules”. Expand each rule out to view hitcount. You can view hitcounts towards each rule in PSM by clicking on “Tenants” > “Security Policies” > “Policy” > hover over a rule. This example shows 2 hits to RDP-permit in DSM 1/2. To enable Firewall logging, from within Aruba Fabric Composer, click on Configuration > System > Firewall Log. Then select “ACTIONS”, and then click “Add”. The Firewall Log Configuration wizard will open up. Provide a name for the new Firewall Log Policy and then click “NEXT”. You can now either choose to deploy this Firewall Log Policy to the whole fabric, or if desired, you can select specific switches to apply the Firewall Log Policy. Click “NEXT”. Next you can now choose the Site Facility, the Severity (All, Allow, Deny), and the preferred Format. When desired parameters have been chosen click “NEXT”, and then “Apply”. If you log into the PSM GUI directly and click on Tenants > Firewall Export Policies, you will now see the new logging policy that was just created.
Appendix
CX10000-1 Configs and Verification Commands
You can use these commands to verify desired VLANs are redirected to DSM, LACP is functional and MACs, ARPs are learnt as expected, full configs are provided for reference.
10000-RU33-SW1# sh dsm 1/1 redirect Distributed Services Modules 1/1 ==================================== Filter information No VLAN redirect configured to Distributed Services module 10000-RU33-SW1# sh dsm 1/2 redirect Distributed Services Modules 1/2 ==================================== Filter information VLANs: 200-202 10000-RU33-SW1# sh lacp int State abbreviations : A - Active P - Passive F - Aggregable I - Individual S - Short-timeout L - Long-timeout N - InSync O - OutofSync C - Collecting D - Distributing X - State m/c expired E - Default neighbor state Actor details of all interfaces: ---------------------------------------------------------------------------------- Intf Aggr Port Port State System-ID System Aggr Forwarding Name Id Pri Pri Key State State State ---------------------------------------------------------------------------------- 1/1/49 lag10(mc) 1049 1 ALFNCD 02:00:00:00:01:00 65534 10 up 1/1/48 lag256 49 1 ALFNCD 04:90:81:00:36:56 65534 256 up Partner details of all interfaces: ---------------------------------------------------------------------------------- Intf Aggr Port Port State System-ID System Aggr Name Id Pri Pri Key ---------------------------------------------------------------------------------- 1/1/49 lag10(mc) 250 128 ASFNCD f4:03:43:60:a4:d8 32768 21 1/1/48 lag256 49 1 ALFNCD 04:90:81:00:33:4a 65534 256 10000-RU33-SW1# sh mac-ad MAC age-time : 300 seconds Number of MAC addresses : 14 MAC Address VLAN Type Port -------------------------------------------------------------- 00:50:56:8e:30:05 10 dynamic 1/1/17 00:50:56:a2:10:a8 20 dynamic lag256 00:50:56:8e:d0:4f 20 dynamic lag256 00:50:56:8e:1e:cb 20 dynamic 1/1/17 00:50:56:8e:92:30 20 dynamic lag256 04:90:81:00:33:4a 200 dynamic lag256 00:50:56:8e:7c:e7 200 dynamic pv lag10 00:50:56:8e:84:7a 200 dynamic pv lag10 04:90:81:00:33:4a 201 dynamic lag256 00:50:56:8e:7c:e7 201 dynamic lag10 00:50:56:8e:84:7a 201 dynamic lag10 04:90:81:00:33:4a 202 dynamic lag256 00:50:56:8e:b9:48 202 dynamic lag10 00:50:56:8e:36:8f 202 dynamic lag10 10000-RU-33-SW1# show arp vrf Synergy IPv4 Address MAC Port Physical Port State VRF ------------------------------------------------------------------------------------------------------- 10.10.202.2 00:50:56:8e:b9:48 vlan202 lag10 reachable Synergy 10.10.202.3 00:50:56:8e:36:8f vlan202 lag10 reachable Synergy 10.10.200.240 04:90:81:00:33:4a vlan200 lag256 reachable Synergy 10.10.202.240 04:90:81:00:33:4a vlan202 lag256 reachable Synergy 10.10.200.3 00:50:56:8e:84:7a vlan200 lag10 reachable Synergy 10.10.200.2 00:50:56:8e:7c:e7 vlan200 lag10 reachable Synergy Total Number Of ARP Entries Listed: 6. -------------------------------------------------------------------------------------------------------- 10000-RU33-SW1# sh run Current configuration: ! !Version ArubaOS-CX DL.10.09.0010 !export-password: default hostname 10000-RU-33-SW1 user admin group administrators password ciphertext AQBapYYxwdkONC4Sev+y+b04Fd0cjpymGqnlCM3LhbDcWe3qYgAAABXl9SsYxNtZG+srlqp3cbElDSoow9j3gCfvJDfHB hrvHMipUH9e1HgOlG9JdpqZksDdVrM0Pjo1zikAKATkOWTdST+bvBjD2+4GQHgefUJw1PlZkh5S7kFEea+geJIwN63d user afc_admin group administrators password ciphertext AQBapTpF3DW4Dzf95Cn2ycp7tQxBzppatqV12DYzpB3py+hKYgAAAOuKA2gWHrCe3Kc1LIleiPzyeZR7eWEaZf0ZwImpP SmINdJDz9kKcTcjZpZgK6/u0IAn8qni4+iBgE/3xgMPXn0yEJXCQ07LFJ2R+UsVgxLsbWvf6LCEtGvPrvLhfnYX3UJv no ip icmp redirect profile leaf vrf Synergy ! ! ! ! ! ssh server vrf mgmt psm host 10.10.10.150 vrf mgmt vlan 1,10,20 vlan 200 private-vlan primary vlan 201 private-vlan isolated primary-vlan 200 vlan 202 interface mgmt no shutdown ip static 10.10.10.213/24 default-gateway 10.10.10.254 system interface-group 1 speed 10g !interface group 1 contains ports 1/1/1-1/1/4 system interface-group 5 speed 10g !interface group 5 contains ports 1/1/17-1/1/20 system interface-group 10 speed 10g !interface group 10 contains ports 1/1/37-1/1/40 system interface-group 12 speed 10g !interface group 12 contains ports 1/1/45-1/1/48 interface lag 10 multi-chassis no shutdown description provisioned no routing vlan trunk native 1 vlan trunk allowed 1,200-202 lacp mode active lacp fallback lacp rate slow interface lag 256 no shutdown description ISL no routing vlan trunk native 1 vlan trunk allowed all lacp mode active lacp rate slow qos trust cos interface 1/1/1 no shutdown mtu 9198 ip mtu 9198 interface 1/1/2 no shutdown mtu 9198 ip mtu 9198 interface 1/1/3 no shutdown mtu 9198 ip mtu 9198 interface 1/1/4 no shutdown mtu 9198 ip mtu 9198 interface 1/1/5 no shutdown mtu 9198 ip mtu 9198 interface 1/1/6 no shutdown mtu 9198 ip mtu 9198 interface 1/1/7 no shutdown mtu 9198 ip mtu 9198 interface 1/1/8 no shutdown mtu 9198 ip mtu 9198 interface 1/1/9 no shutdown mtu 9198 ip mtu 9198 interface 1/1/10 no shutdown mtu 9198 ip mtu 9198 interface 1/1/11 no shutdown mtu 9198 ip mtu 9198 interface 1/1/12 no shutdown mtu 9198 ip mtu 9198 interface 1/1/13 no shutdown mtu 9198 ip mtu 9198 interface 1/1/14 no shutdown mtu 9198 ip mtu 9198 interface 1/1/15 no shutdown mtu 9198 ip mtu 9198 interface 1/1/16 no shutdown mtu 9198 ip mtu 9198 interface 1/1/17 no shutdown mtu 9198 no routing vlan trunk native 1 tag vlan trunk allowed 10,20 interface 1/1/18 no shutdown mtu 9198 ip mtu 9198 interface 1/1/19 no shutdown mtu 9198 ip mtu 9198 interface 1/1/20 no shutdown mtu 9198 ip mtu 9198 interface 1/1/21 no shutdown mtu 9198 ip mtu 9198 interface 1/1/22 no shutdown mtu 9198 ip mtu 9198 interface 1/1/23 no shutdown mtu 9198 ip mtu 9198 interface 1/1/24 no shutdown mtu 9198 ip mtu 9198 interface 1/1/25 no shutdown mtu 9198 ip mtu 9198 interface 1/1/26 no shutdown mtu 9198 ip mtu 9198 interface 1/1/27 no shutdown mtu 9198 ip mtu 9198 interface 1/1/28 no shutdown mtu 9198 ip mtu 9198 interface 1/1/29 no shutdown mtu 9198 ip mtu 9198 interface 1/1/30 no shutdown mtu 9198 ip mtu 9198 interface 1/1/31 no shutdown mtu 9198 ip mtu 9198 interface 1/1/32 no shutdown mtu 9198 ip mtu 9198 interface 1/1/33 no shutdown mtu 9198 ip mtu 9198 interface 1/1/34 no shutdown mtu 9198 ip mtu 9198 interface 1/1/35 no shutdown mtu 9198 ip mtu 9198 interface 1/1/36 no shutdown mtu 9198 ip mtu 9198 interface 1/1/37 no shutdown mtu 9198 ip mtu 9198 interface 1/1/38 no shutdown mtu 9198 ip mtu 9198 interface 1/1/39 no shutdown mtu 9198 ip mtu 9198 interface 1/1/40 no shutdown mtu 9198 ip mtu 9198 interface 1/1/41 no shutdown mtu 9198 ip mtu 9198 interface 1/1/42 no shutdown mtu 9198 ip mtu 9198 interface 1/1/43 no shutdown mtu 9198 ip mtu 9198 interface 1/1/44 no shutdown mtu 9198 ip mtu 9198 interface 1/1/45 no shutdown mtu 9198 ip mtu 9198 interface 1/1/46 no shutdown persona access mtu 9198 qos trust cos ip mtu 9198 ip address 192.168.10.2/31 interface 1/1/47 no shutdown mtu 9198 ip mtu 9198 interface 1/1/48 no shutdown persona access mtu 9198 lag 256 interface 1/1/49 no shutdown persona access mtu 9198 lag 10 interface 1/1/50 no shutdown persona uplink mtu 9198 ip mtu 9198 interface 1/1/51 no shutdown persona uplink mtu 9198 ip mtu 9198 interface 1/1/52 no shutdown persona uplink mtu 9198 ip mtu 9198 interface 1/1/53 no shutdown persona uplink mtu 9198 ip mtu 9198 interface 1/1/54 no shutdown persona uplink mtu 9198 ip mtu 9198 interface vlan 200 vrf attach Synergy ip mtu 9198 ip address 10.10.200.241/24 active-gateway ip mac 00:00:00:00:00:01 active-gateway ip 10.10.200.254 ip local-proxy-arp interface vlan 202 vrf attach Synergy ip mtu 9198 ip address 10.10.202.241/24 active-gateway ip mac 00:00:00:00:00:02 active-gateway ip 10.10.202.254 ip local-proxy-arp vsx system-mac 02:00:00:00:01:00 inter-switch-link lag 256 role secondary keepalive peer 192.168.10.3 source 192.168.10.2 no split-recovery vsx-sync vsx-global ! ! ! ! ! https-server vrf mgmt
CX10000-2 Configs and Verification Commands
You can use these commands to verify desired VLANs are redirected to DSM, LACP is functional and MACs, ARPs are learnt as expected, full configs are provided for reference.
10000-RU34-SW2# sh dsm 1/1 redirect Distributed Services Modules 1/1 ==================================== Filter information No VLAN redirect configured to Distributed Services module 10000-RU34-SW2# sh dsm 1/2 redirect Distributed Services Modules 1/2 ==================================== Filter information VLANs: 200-202 10000-RU34-SW2# sh lacp int State abbreviations : A - Active P - Passive F - Aggregable I - Individual S - Short-timeout L - Long-timeout N - InSync O - OutofSync C - Collecting D - Distributing X - State m/c expired E - Default neighbor state Actor details of all interfaces: ---------------------------------------------------------------------------------- Intf Aggr Port Port State System-ID System Aggr Forwarding Name Id Pri Pri Key State State State ---------------------------------------------------------------------------------- 1/1/49 lag10(mc) 49 1 ALFNCD 02:00:00:00:01:00 65534 10 up 1/1/48 lag256 49 1 ALFNCD 04:90:81:00:33:4a 65534 256 up Partner details of all interfaces: ---------------------------------------------------------------------------------- Intf Aggr Port Port State System-ID System Aggr Name Id Pri Pri Key ---------------------------------------------------------------------------------- 1/1/49 lag10(mc) 112 128 ASFNCD f4:03:43:60:a4:d8 32768 21 1/1/48 lag256 49 1 ALFNCD 04:90:81:00:36:56 65534 256 10000-RU34-SW2# sh mac-ad MAC age-time : 300 seconds Number of MAC addresses : 13 MAC Address VLAN Type Port -------------------------------------------------------------- 00:50:56:a2:10:a8 20 dynamic 1/1/18 00:50:56:8e:d0:4f 20 dynamic 1/1/17 00:50:56:8e:1e:cb 20 dynamic lag256 00:50:56:8e:92:30 20 dynamic 1/1/17 04:90:81:00:36:56 200 dynamic lag256 00:50:56:8e:7c:e7 200 dynamic pv lag10 00:50:56:8e:84:7a 200 dynamic pv lag10 04:90:81:00:36:56 201 dynamic lag256 00:50:56:8e:7c:e7 201 dynamic lag10 00:50:56:8e:84:7a 201 dynamic lag10 04:90:81:00:36:56 202 dynamic lag256 00:50:56:8e:36:8f 202 dynamic lag10 00:50:56:8e:b9:48 202 dynamic lag10 10000-RU34-SW2# sh arp vrf Synergy IPv4 Address MAC Port Physical Port State VRF --------------------------------------------------------------------------------------------- 10.10.202.3 00:50:56:8e:36:8f vlan202 lag10 reachable Synergy 10.10.200.251 04:90:81:00:36:56 vlan200 lag256 reachable Synergy Total Number Of ARP Entries Listed: 2. --------------------------------------------------------------------------------------------- 10000-RU34-SW2# sh run Current configuration: ! !Version ArubaOS-CX DL.10.09.0010 !export-password: default hostname 10000-RU34-SW2 user admin group administrators password ciphertext AQBapfGrz5kq6he5ykpcx4YR1KlJh13fWgVDCSdybQvHf5UhYgAAAE5u3cuwvp8FBs8yTvjLEDGTBi5uGjrQo22ur/4G5 7yjX6K5yhmcK33PG/g+hLs1NqozFFRx+S52ozvyKegnCXjs3piV4D/D5EKd01P8YeEZbv920OGcoXPLau6Ws8MiFKgk user afc_admin group administrators password ciphertext AQBapS0Y4qS+NoaDC7C/qqGXB832EdF1A3/pSbsyx9RVlIhYYgAAAKHMMJ1XZ0JTwal8hvnFzMn52WtloGsB0+wRQNEF+ lFz04nnJuvGUy5zDtm/9dBLqg3ExgKxJIn4N1cHHHVWDy/7+upkCgY70LgGE7mEVFEC4wCqh596BZiNlHmTUq661vZo no ip icmp redirect profile leaf vrf Synergy ! ! ! ! ! ssh server vrf mgmt psm host 10.10.10.150 vrf mgmt vlan 1,10,20-21 vlan 200 private-vlan primary vlan 201 private-vlan isolated primary-vlan 200 vlan 202 interface mgmt no shutdown ip static 10.10.10.212/24 default-gateway 10.10.10.254 system interface-group 1 speed 10g !interface group 1 contains ports 1/1/1-1/1/4 system interface-group 5 speed 10g !interface group 5 contains ports 1/1/17-1/1/20 system interface-group 10 speed 10g !interface group 10 contains ports 1/1/37-1/1/40 system interface-group 12 speed 10g !interface group 12 contains ports 1/1/45-1/1/48 interface lag 10 multi-chassis no shutdown description provisioned no routing vlan trunk native 1 vlan trunk allowed 1,200-202 lacp mode active lacp fallback lacp rate slow interface lag 256 no shutdown description ISL no routing vlan trunk native 1 vlan trunk allowed all lacp mode active lacp rate slow qos trust cos interface 1/1/1 no shutdown mtu 9198 ip mtu 9198 interface 1/1/2 no shutdown mtu 9198 ip mtu 9198 interface 1/1/3 no shutdown mtu 9198 ip mtu 9198 interface 1/1/4 no shutdown mtu 9198 ip mtu 9198 interface 1/1/5 no shutdown mtu 9198 ip mtu 9198 interface 1/1/6 no shutdown mtu 9198 ip mtu 9198 interface 1/1/7 no shutdown mtu 9198 ip mtu 9198 interface 1/1/8 no shutdown mtu 9198 ip mtu 9198 interface 1/1/9 no shutdown mtu 9198 ip mtu 9198 interface 1/1/10 no shutdown mtu 9198 ip mtu 9198 interface 1/1/11 no shutdown mtu 9198 ip mtu 9198 interface 1/1/12 no shutdown mtu 9198 ip mtu 9198 interface 1/1/13 no shutdown mtu 9198 ip mtu 9198 interface 1/1/14 no shutdown mtu 9198 ip mtu 9198 interface 1/1/15 no shutdown mtu 9198 ip mtu 9198 interface 1/1/16 no shutdown mtu 9198 ip mtu 9198 interface 1/1/17 no shutdown mtu 9198 no routing vlan trunk native 1 tag vlan trunk allowed 10,20 interface 1/1/18 no shutdown mtu 9198 no routing vlan trunk native 1 tag vlan trunk allowed 20-21 interface 1/1/19 no shutdown mtu 9198 ip mtu 9198 interface 1/1/20 no shutdown mtu 9198 ip mtu 9198 interface 1/1/21 no shutdown mtu 9198 ip mtu 9198 interface 1/1/22 no shutdown mtu 9198 ip mtu 9198 interface 1/1/23 no shutdown mtu 9198 ip mtu 9198 interface 1/1/24 no shutdown mtu 9198 ip mtu 9198 interface 1/1/25 no shutdown mtu 9198 ip mtu 9198 interface 1/1/26 no shutdown mtu 9198 ip mtu 9198 interface 1/1/27 no shutdown mtu 9198 ip mtu 9198 interface 1/1/28 no shutdown mtu 9198 ip mtu 9198 interface 1/1/29 no shutdown mtu 9198 ip mtu 9198 interface 1/1/30 no shutdown mtu 9198 ip mtu 9198 interface 1/1/31 no shutdown mtu 9198 ip mtu 9198 interface 1/1/32 no shutdown mtu 9198 ip mtu 9198 interface 1/1/33 no shutdown mtu 9198 ip mtu 9198 interface 1/1/34 no shutdown mtu 9198 ip mtu 9198 interface 1/1/35 no shutdown mtu 9198 ip mtu 9198 interface 1/1/36 no shutdown mtu 9198 ip mtu 9198 interface 1/1/37 no shutdown mtu 9198 ip mtu 9198 interface 1/1/38 no shutdown mtu 9198 ip mtu 9198 interface 1/1/39 no shutdown mtu 9198 ip mtu 9198 interface 1/1/40 no shutdown mtu 9198 ip mtu 9198 interface 1/1/41 no shutdown mtu 9198 ip mtu 9198 interface 1/1/42 no shutdown mtu 9198 ip mtu 9198 interface 1/1/43 no shutdown mtu 9198 ip mtu 9198 interface 1/1/44 no shutdown mtu 9198 ip mtu 9198 interface 1/1/45 no shutdown mtu 9198 ip mtu 9198 interface 1/1/46 no shutdown mtu 9198 qos trust cos description Keep alive Interface 10000-RU34-SW2 ip mtu 9198 ip address 192.168.10.3/31 interface 1/1/47 no shutdown mtu 9198 ip mtu 9198 interface 1/1/48 no shutdown mtu 9198 lag 256 interface 1/1/49 no shutdown persona access mtu 9198 lag 10 interface 1/1/50 no shutdown persona uplink mtu 9198 ip mtu 9198 interface 1/1/51 no shutdown persona uplink mtu 9198 ip mtu 9198 interface 1/1/52 no shutdown persona uplink mtu 9198 ip mtu 9198 interface 1/1/53 no shutdown persona uplink mtu 9198 ip mtu 9198 interface 1/1/54 no shutdown persona uplink mtu 9198 ip mtu 9198 interface vlan 200 vrf attach Synergy ip mtu 9198 ip address 10.10.200.240/24 active-gateway ip mac 00:00:00:00:00:01 active-gateway ip 10.10.200.254 ip local-proxy-arp interface vlan 202 vrf attach Synergy ip mtu 9198 ip address 10.10.202.240/24 active-gateway ip mac 00:00:00:00:00:02 active-gateway ip 10.10.202.254 ip local-proxy-arp vsx system-mac 02:00:00:00:01:00 inter-switch-link lag 256 role primary keepalive peer 192.168.10.2 source 192.168.10.3 no split-recovery vsx-sync vsx-global ! ! ! ! ! https-server vrf mgmt