Antonio Perez

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== HPE Synergy and Aruba CX 10000 Networking Deployment Guide ======

----
----

**[[https://arubanetworking.hpe.com/techdocs/AOS-CX/help_portal/Content/ArubaTopics/Switches/10000.htm|10000 Switch Series Software Documentation]]**
----
----


{{ youtube>8YdaHFuEYOg? }}

{{ youtube>MIuST5eKmf4? }}

{{ youtube>MC5GO3dc_3Y }}

{{ youtube>N-C2oMOSfI8? }}


----
----
**{{ :aruba_networks:switch:aruba-cx-10000-series-datasheet.pdf |Aruba cx 10000 series datasheet}}**

{{pdfjs 46em >:aruba_networks:switch:aruba-cx-10000-series-datasheet.pdf }}
----
----

**{{ :aruba_networks:switch:10000_user_guide.pdf |HPE Synergy and Aruba CX 10000 NetworkingDeployment Guide}}**

{{pdfjs 46em >:aruba_networks:switch:10000_user_guide.pdf }}

----
----
**{{ :aruba_networks:switch:igsg_10000.pdf |HPE Aruba Networking CX 10000 Distributed Services Switch}}**

{{pdfjs 46em >:aruba_networks:switch:igsg_10000.pdf }}

----
----


===== Contents =====
  * [[#introduction|Introduction]]
  * [[#overview|Overview]]
  * [[#prerequisites|Prerequisites]]
  * [[#detailed-topology|Detailed Topology]]
  * [[#task-1-aos-cx-switches|Task 1: AOS-CX Switches]]
  * [[#task-2-aruba-fabric-composer|Task 2: Aruba Fabric Composer]]
    * [[#guided-setup|Guided Setup]]
    * [[#switch-to-fabric-association|Switch to Fabric Association]]
    * [[#vsx-configuration|VSX Configuration]]
    * [[#persona-configuration|Persona Configuration]]
    * [[#vsx-lag-configuration|VSX LAG Configuration]]
    * [[#vrf-and-svi-configuration|VRF and SVI Configuration]]
    * [[#pvlan-configuration|PVLAN Configuration]]
    * [[#psm-integration|PSM Integration]]
    * [[#vmware-vcenter-integration|VMware vCenter Integration]]
  * [[#task-3-hpe-synergy-networking|Task 3: HPE Synergy Networking]]
    * [[#networks|Networks]]
    * [[#logical-interconnects|Logical Interconnects]]
    * [[#server-profiles|Server Profiles]]
  * [[#task-4-vmware-vds|Task 4: VMware vDS]]
  * [[#task-5-distributed-services|Task 5: Distributed Services]]
    * [[#distributed-firewall-configuration|Distributed Firewall Configuration]]
  * [[#task-6-security-policy-validation--fw-logging|Task 6: Security Policy Validation & FW Logging]]
  * [[#appendix|Appendix]]
    * [[#cx10000-1-configs-and-verification-commands|CX10000-1 Configs and Verification Commands]]
    * [[#cx10000-2-configs-and-verification-commands|CX10000-2 Configs and Verification Commands]]

----

===== Introduction =====
The Aruba CX 10000 Series Switch with Pensando represents a new category of data center switches that combines best-of-breed Aruba data center L2/3 switching with the industry's only, fully programmable data-processing unit (DPU). The Pensando Elba DPU is able to deliver stateful software-defined services inline, at scale, with wire-rate performance and orders of magnitude scale and performance improvements over traditional data center L2/3 switches at a fraction of their TCO. The Pensando Policy and Services Manager (PSM) is a distributed system, leveraging an intent-based model that delivers network and security policy to CX 10000 DPUs at the edge. Aruba Fabric Composer is an intelligent, API-driven, software-defined orchestration solution that simplifies and accelerates network provisioning, security management and day-to-day operations across rack-scale compute and storage infrastructure. PSM integration allows network security policies to be configured directly from Aruba Fabric Composer. Aruba Fabric Composer together with the Pensando Policy and Services Manager (PSM) provides a distributed system, leveraging an intent-based model that delivers network and security policy to Aruba CX 10000 DPUs at the edge. This document provides networking guidance when building out a solution that leverages an HPE Synergy enclosure connected to an Aruba CX 10000 network managed by Aruba Fabric Composer and PSM.

===== Overview =====
As shown in Figure 1, Aruba Fabric Composer provides underlay and overlay automation for the Data Center fabric comprising Aruba CX 8325 Spines and CX 10000 Leafs which connect to rack mount servers and blade server enclosures such as HPE Synergy.

*Figure 1. Data Center with CX 8325 Spines, CX 10000 Leafs and HPE Synergy*

For network redundancy and traffic load sharing, HPE Synergy Virtual Connect modules connect to the Aruba CX 10000s via a Virtual Switching Extension (VSX) Link Aggregation Group (LAG). "Tunnel" mode would be used on Virtual Connect to simplify and minimize network configuration within HPE OneView. In addition, when Aruba Fabric Composer (AFC) is integrated with VMware vCenter, you can easily create vSphere Distributed Switches (vDS), PVLAN, port groups and visualize those network connections between the physical switches, Virtual Connect ports, vSwitches, Virtual Machines (VMs). The Aruba CX 10000 network security policies provide east/west firewall capabilities between Servers, Virtual Machines (VMs) or Containers within a VLAN or across different network VLAN/subnets. If micro-segmentation within the same hypervisor or across different hypervisors and within the same subnet is required for network isolation and security policies, Private VLAN (PVLAN) can be utilized. This guide uses VMware vSphere Distributed Switch on HPE Synergy compute modules as an example.

===== Prerequisites =====
Before connecting and configuring an HPE Synergy enclosure to the Aruba CX 10000 network, the following are recommended:
  * Aruba CX 10000s loaded and booted up with code version 10.09.0010 (minimum)
  * Ports between devices should be cabled up and connected without errors
  * A separate Out Of Band (OOB) network for management
    * Aruba CX 6300 switches are suitable for the OOB network
    * Mgmt ports of the Aruba CX 8325/10000 switches, Aruba Fabric Composer and PSM should be connected to the OOB network and have IP reachability between each other
  * Aruba Fabric Composer 6.2 (minimum) should be functional, refer to Aruba Support Portal
    * This guide is focused on HPE Synergy with Aruba CX 10000 switches, refer to Aruba Fabric Composer documentation for Aruba Fabric Composer high availability deployment recommendations
  * PSM 1.29.2-T-11 (minimum) should be functional, refer to Aruba Support Portal
    * This guide is focused on HPE Synergy with Aruba CX 10000 switches. Please refer to PSM documentation for PSM high availability deployment recommendations
  * HPE OneView and HPE Synergy should be functional, refer to HPE Synergy Documentation Quick Links for more information on HPE Synergy networking
    * For reference, this guide used firmware 1.7.1.1001 for interconnects, firmware 6.4.00 for Oneview and enclosure bundles
  * For flow-logging to appear properly within PSM, make sure that the time set within the attached PC/VM is set to the same time as PSM.

===== Detailed Topology =====
As this guide is focused on HPE Synergy with Aruba CX 10000 switches, the detailed topology as shown in Figure 2 will be used and referenced for the rest of this guide.

*Figure 2. Data Center rack with Aruba CX 10000 Leafs and HPE Synergy*

3 VLANs are used in this guide:
  * **VLAN 200 – Primary PVLAN**
    * Int VLAN 200 (10.10.200.254/24) on Aruba CX 10000s will function as default gateways for VLAN 200 and VLAN 201.
    * Suitable for VMs that do not require L2 micro-segmentation on the 10.10.200.0/24 subnet.
  * **VLAN 201 – Isolated PVLAN**
    * Suitable for VMs that require L2 micro-segmentation on the 10.10.200.0/24 subnet.
  * **VLAN 202 – Normal VLAN (Non PVLAN)**
    * Interface VLAN 202 (10.10.202.254/24) on Aruba CX 10000s will function as default gateways for VLAN 202.
    * Suitable for VMs that do not require L2 micro-segmentation on the 10.10.202.0/24 subnet.

For network redundancy and traffic load sharing, 1 x 40G uplink on each HPE Synergy Virtual Connect module connects to a pair of Aruba CX 10000s via VSX LAG. LACP is only required between Aruba CX 10000s and the Virtual Connect modules. The vDS doesn't require LACP to Virtual Connect modules. When "Tunnel" mode is configured on Virtual Connect, there is no requirement to create VLANs or PVLANs in OneView. Note that connectivity options from the Synergy enclosure to the Aruba CX 10000 switches will vary based on the VC module in use and the constraints of the environment. Depending on the VC module chosen users could use 40G or 100G interfaces, but they could also use 4x10G or 4x25G interfaces to connect to each ToR switch. Please refer to the Synergy site for more details on VC modules available - https://www.hpe.com/us/en/integratedsystems/synergy.html

To implement east/west firewall capabilities between VMs on different subnets, security policies configured on Aruba Fabric Composer are pushed down via PSM to the Aruba CX 10000 DPUs to permit or deny desired traffic. The remainder of this guide walks through the tasks required for this deployment to be successful and should be done sequentially.

===== Task 1: AOS-CX Switches =====
Using console access, enter the base configuration below required on AOS-CX switches for Aruba Fabric Composer management, modify hostnames and IPs as required.
<code>
configure
hostname CX10000-R1RU33-SW1
interface mgmt
 ip static 10.251.X.5/24
 default-gateway 10.251.X.254
</code>

If you need ports changed from 25g to 10g, you will need to modify the interface-group as required.
<code>
system interface-group 1 speed 10g !interface group 1 contains ports 1/1/1-1/1/4
system interface-group 5 speed 10g !interface group 5 contains ports 1/1/17-1/1/20
</code>

When managed by Aruba Fabric Composer, the majority of Aruba CX 10000 features can be configured from the Aruba Fabric Composer GUI. To ensure all traffic is inspected by the security policy, this should also be added at the global level.
<code>
no ip icmp redirect
</code>

===== Task 2: Aruba Fabric Composer =====
This section provides guidance on automating the network fabric, validating connectivity to the Aruba CX switches, PSM, and integration with vSphere/PSM.

###### Guided Setup
After logging into Aruba Fabric Composer, click the guided network setup (green icon with 3 ...) on the right.
Click on "SWITCHES" to discover new switches, enter IPs and desired passwords to be pushed down to the switches.
The switches should be added but unassigned. Click on "FABRIC" on the guided network setup to add a fabric, enter desired info and click "APPLY".

###### Switch to Fabric Association
After the switches are discovered, you can assign switches to the fabric by clicking on "ASSIGN SWITCH TO FABRIC" in the guided setup workflow on the right.
Add the switches, select a fabric, desired role, initialize the ports and click "ADD" and then "APPLY".
After a minute or so, the switches should appear healthy and synced with the assigned fabric.
Users can continue to follow the workflow on the right and now add NTP/DNS configurations. The configs are straight forward so this guide continues to the VSX setup portion of the workflow.

###### VSX Configuration
Click "VSX CONFIGURATION" in the guided setup workflow on the right.
Select "Automatically generate VSX Pairs" and click "NEXT".
Add desired name and click "NEXT".
Use default values and click "NEXT".
Enter desired keepalive interface mode and add an address pool. Click "ADD" to provide a name for the address pool, and then click "NEXT".
Add a subnet range and then click "NEXT" and then "APPLY". Click "NEXT".
For Keepalive setting, keep as default values, and click "NEXT".
Leave the Linkup timer as default and click "ADD" to add a MAC Address Resource Pool, or simply add the desired MAC Address range needed. Click "NEXT".
Review the parameters and click "APPLY".
VSX should now be operational (you may need to click refresh for AFC to reflect that). Make sure that parameters are operational, in_sync, and peer_established.

###### Persona Configuration
Click on "Configurations" > "Ports" > "Ports" > (icon) > select desired switch on top right drop down menu. Users can select multiple switches, if desired – in this example select both Aruba CX 10000 switches.
Select the ports facing the server (both 1/1/49 in this example) and click > Actions > Port Type.
Select the "Access" port type and then click "OK".

###### VSX LAG Configuration
Click on "Configurations" > "Ports" > "Link Aggregation Groups" > "ACTIONS" > "Add".
Select a single LAG option and click "NEXT".
Enter a desired name, LAG number and then click "NEXT".
Select the VSX switches/ports connected to Synergy Virtual Connect and click "NEXT".
Select the switches and enter the desired values and click "NEXT".
Enter a native VLAN, desired VLANs to be allowed, and click "NEXT" and "APPLY".
VSX LAG info can be seen if you expand it out.

###### VRF and SVI Configuration
Click on "Configurations" > "Routing" > "VRF" > "ACTIONS" > "Add". Enter the desired name and click "NEXT".
You can either "Apply" the config to all switches or select specific switches and click "NEXT".
"L3 VNI" and "Route Targets" are not required if VXLAN is not used, so for this example click "NEXT" on both screens to proceed.
Review the parameters and click "APPLY".
After the desired VRF is created, select that VRF and then click on "ACTIONS" > "IP Interfaces". Under "IP Interfaces", click on "ACTIONS" > "Add".
Enter your desired VLAN, switches, and subnet.
Enter your desired SVI IP range, active gateway IP and MAC. "Enable Local Proxy ARP" should be selected for primary PVLAN SVI (VLAN 200) to allow VMs on the same isolated PVLAN to communicate if desired via the security policy. "Enable Local Proxy ARP" is not required for normal VLAN 202. Click "NEXT".
Enter an optional name, description and then click "NEXT".
Review the parameters and then click "APPLY".
SVIs should now be operational, repeat for other desired SVIs. In this guide, only VLANs 200 and 202 require SVIs. VLAN 201 is an isolated PVLAN and uses VLAN 200 primary PVLAN as the SVI for the 10.10.200.0/24 subnet.

###### PVLAN Configuration
Click on "Configurations" > "Ports" > "PVLANS" > "ACTIONS" > "Add".
Give the PVLAN config a name and then click "NEXT".
Select the desired switches and then click "NEXT".
Enter the Primary VLAN 200 and then click "NEXT".
Enter the Isolated VLAN 201 and then click "NEXT".
Click "NEXT" on the "Secondary Ports" screen.
Review and click "APPLY".

###### PSM Integration
The Pensando integration allows you to:
  * Configure security policies in Aruba Fabric Composer
  * Or configure security policies in PSM
    * This may be required in scenarios where the security and network team are separate entities and the networking team is not allowed to manage the security policies

In the guided setup, select "DISTRIBUTED SERVICES" > "Pensando PSM".
Enter the required values.
Click "VALIDATE" and if successful, click "NEXT".
Select the fabric, then both check box options, click "NEXT", review and then click "APPLY".
The connected status should now be shown.
You can verify REST API connectivity between the Aruba CX 10000 switch and PSM is operational by clicking on "CLI Commands" next to "Guide Setup" icon.
Select your fabric or switches, type your desired "show" command, hit enter and click "RUN". The Aruba CX 10000 swicthes should be shown as "admitted" into PSM.

###### VMware vCenter Integration
The vSphere integration will allow you to view vSphere hosts in Aruba Fabric Composer, automatically deploy network configurations based on VM deployment on vSphere and visualize network connections between physical switches, Virtual Connect ports, vSwitches and VMs. Click on "Configurations" > "Integrations" > "VMware vSphere" > "ACTIONS" > "Add".
Enter the required values.
Click "Validate" and if successful, then click "NEXT".
Select all 3 options and then click "NEXT".
Select "Discovery protocols" and then click "NEXT".
Review the parameters and then click "APPLY". The connected status should be shown.

===== Task 3: HPE Synergy Networking =====
This section provides guidance on networks, logical interconnects, and server profiles to connect to the Aruba CX 10000 switches.

###### Networks
From OneView, select "NETWORKING" > "Networks" and create a network with VLAN set to "Tunnel".

###### Logical Interconnects
Within "Networking" > "Logical Interconnect Groups", click "Edit", then select "Add uplink set". Create an uplink set with the "Type" set to "Tunnel" and add the desired uplink ports.
From "Networking" > "Logical Interconnects", you will notice an inconsistency error. Select "Actions" > "Update from group" to fix it. If VSX LAG is configured on the Aruba CX 10000s from the previous section, you can validate the LACP LAG is operational between Virtual Connect and the CX 10000s under the "Uplinks Sets" section. The interconnects should be green with "Linked active" state, "LACP activity" and the attached switch neighbor info should also be seen.

###### Server Profiles
Edit your "Server Profile Template" and "Add Connection" with "Network" set to "Tunnel-uplink".
"Tunnel-uplink" connection should be created.
From "Server Profiles", you will see it is inconsistent with its "server profile template", you can fix it by clicking on "Update from template".
You can update it by selecting "Update from template after power is off" and "Momentary press". If desired, VMs should be migrated to another hypervisor before powering the compute module off.
Repeat this step for all server profiles that require the "Tunnel" uplink. Once the server profile update is complete, you will be able to check the MAC address of the "Tunnel" uplink by clicking on "Edit" > "Connections". This MAC will show up in vSphere as a physical NIC.

===== Task 4: VMware vDS =====
After Aruba Fabric Composer is integrated with VMware vSphere, from Aruba Fabric Composer you will be able to:
  * Create VMware vDS and assign VMNICs
  * Create PVLANs in vDS

In Aruba Fabric Composer, click on "Visualizations" > "Hosts" > select your desired hypervisors in the bottom selection window and unselect undesired hypervisors. You should see the new VMNIC with mac address matching the previous screenshot in OneView, that is the VMNIC that should be assigned into the vDS.
To create the vDS, click on the desired hypervisor > "Create Microsegmentation".
Input the desired names and select desired NICs. Multiple VMINCs on the same host can be selected if available. Only the VMNIC MAC assigned to the "tunnel uplink" is required. Click "NEXT".
Enter the Primary PVLAN (200), Isolated PVLAN (201), and click "ADD", click NEXT" and then "APPLY".
Once done, you should be able to see the vDS, VMNIC and port group.
On other desired hypervisors, select "Update Microsegmentation" to add additional hypervisors and VMNICs to the vDS.
Once done, you should be able to see the vDS, VMNIC and port group.
You will need to create non PVLAN port groups directly in vCenter by clicking on the "Networking" icon > right click desired vDS > "Distributed Port Group" > "New Distributed Port Group".
Enter the desired values, click "NEXT" and then "FINISH".
The next step is to update the VM network adapters to the desired port group in vCenter. This example shows a VM with 10.10.200.2/24 IP assigned to an isolated PVLAN VLAN 201 port group.
This example shows a VM with 10.10.202.2/24 IP assigned to the normal VLAN 202 port group. Repeat and assign other VMs to their desired port group. For visualizations in Aruba Fabric Composer to display correctly, the ESXi hypervisors should have unique host names, DNS domain and DNS server configured under host > Networking > TCP/IP Configuration > Edit > Default.
After the VMs are attached to port groups, the "Visualizations" pane should be updated with VMNIC/vDS/port group/VM links.
In addition, the VMNIC links to the VC module and the Aruba CX 10000 switchports should also be shown. If LACP is required on the Aruba Fabric Composer created vDS pointed towards the switches, you can enable LACP support by selecting the vDS > "Upgrade" > "Enhance LACP Support" > "Next" > "Finish".
This is only applicable if the vDS is used on rack mount servers, if the vDS is only used within a Synergy enclosure, LACP is not required as LACP is enabled at the Virtual Connect level.

===== Task 5: Distributed Services =====
This section provides guidance on deploying distributed services on Aruba Fabric Composer, PSM and the Aruba CX 10000 switches.

**Network Configuration**  
Next step is to add VLANs to be redirected/inspected by the Aruba CX 10000 DPUs. Click on guided setup icon > "DISTRIBUTED SERVICES" > select the desired VRF > "CONFIGURE NETWORKS" > "ACTIONS" > "Add".
Add desired name and click "NEXT".
Add desired VLAN, click "NEXT" and "APPLY".
Repeat for all desired VLANs with SVIs. VLANs 200 and 202 are configured with SVIs used in this guide. VLAN 201 is an isolated PVLAN that utilizes SVI 200 and doesn't need to be created here.

###### Distributed Firewall Configuration
Click on guided setup icon > "DISTRIBUTED SERVICES" > select desired VRF > "CONFIGURE POLICY" > "ACTIONS" > "Add".
Enter the desired policy name and then click "NEXT".
Select "Distributed Firewall" and then click "NEXT". Select "ACTIONS" > "Add" > "New".
This example permits RDP traffic from Web Tier to App Tier, e.g. VM (10.10.200.3/32) to VM (10.10.202.2/32). Click "NEXT". Select the desired action and click "NEXT".
Select "ADD" to add new source endpoint group. Enter the desired name and click next.
Select, "ADD" desired VM, click "NEXT" and then "APPLY". Click "ADD" on the destination endpoint group.
Name the endpoint group and then click "NEXT". Select, add desired VM (this example shows you can add multiple endpoints), click "NEXT" and then "APPLY".
Click "NEXT" once both desired source and destination endpoint groups are added. Select existing "Service Qualifier" if applicable, or click "ADD" to add a new service qualifier and then click "NEXT".
Review and click "APPLY".
Take note there is an implicit deny at the end of the policy. You can add more rules to permit traffic by clicking on "Actions" > "Add" > "New".
An "ICMP-permit" rule is used in this example to allow VMs to check network connectivity. Select the desired action and then click "NEXT".
Leave the endpoint groups empty to match on "any", and then click "NEXT". Select the desired service qualifier, click "NEXT", review and then click "APPLY".
Review, add additional rules, e.g. allow Web to App desired traffic and then click "NEXT".
Take note the policy is applied to the entire network fabric (all Aruba CX 10000 switches), you will not be able to select only specific CX 10000s. Select
  * Desired Fabric
  * Desired direction
    * Egress refers to policy applied from workload perspective, we are trying to permit outbound traffic egress from Web Tier to App Tier, e.g. VM (10.10.200.3/32) to VM (10.10.202.2/32)
  * Desired VRF
  * Desired Networks  
Click on "ADD".
Click "NEXT" and then "APPLY". The configured policy should be seen as healthy.
You can validate the security policy is pushed down to PSM and CX 10000 DPUs in PSM GUI > "Tenants" > "Security Policies" > policy name.
And the policy is attached to desired networks in PSM > "Tenants" > "Networks".

===== Task 6: Security Policy Validation & FW Logging =====
VMs on the same isolated PVLAN normally do not have network connectivity between each other. However, with local proxy arp enabled on VLAN 200, traffic between VMs on the same isolated PVLAN can be subjected to a security policy for traffic to be allowed or denied. With the security policy in place, we can verify 10.10.200.3 VM in WebTier is able to ping to VMs in the same subnet (10.10.200.0/24) and in the AppTier (10.10.202.0/24) subnet due to the "ICMP-permit" rule. It is also able to connect via RDP to the 10.10.202.2 VM due to the "RDP-permit" rule.
Due to the implicit deny rule, RDP between VMs on the same subnet are denied as expected.
You can view hitcounts towards each rule in Aruba Fabric Composer by clicking on "Configurations" > "Policy" > "Policies" > "..." > "Rules". Expand each rule out to view hitcount.
You can view hitcounts towards each rule in PSM by clicking on "Tenants" > "Security Policies" > "Policy" > hover over a rule. This example shows 2 hits to RDP-permit in DSM 1/2.
To enable Firewall logging, from within Aruba Fabric Composer, click on Configuration > System > Firewall Log. Then select "ACTIONS", and then click "Add". The Firewall Log Configuration wizard will open up.
Provide a name for the new Firewall Log Policy and then click "NEXT".
You can now either choose to deploy this Firewall Log Policy to the whole fabric, or if desired, you can select specific switches to apply the Firewall Log Policy. Click "NEXT".
Next you can now choose the Site Facility, the Severity (All, Allow, Deny), and the preferred Format. When desired parameters have been chosen click "NEXT", and then "Apply".
If you log into the PSM GUI directly and click on Tenants > Firewall Export Policies, you will now see the new logging policy that was just created.

===== Appendix =====

==== CX10000-1 Configs and Verification Commands ====
You can use these commands to verify desired VLANs are redirected to DSM, LACP is functional and MACs, ARPs are learnt as expected, full configs are provided for reference.

<code>
10000-RU33-SW1# sh dsm 1/1 redirect
Distributed Services Modules 1/1
====================================
Filter information
No VLAN redirect configured to Distributed Services module

10000-RU33-SW1# sh dsm 1/2 redirect
Distributed Services Modules 1/2
====================================
Filter information
VLANs: 200-202

10000-RU33-SW1# sh lacp int

State abbreviations :
A - Active
P - Passive
F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync
O - OutofSync
C - Collecting D - Distributing
X - State m/c expired
E - Default neighbor state

Actor details of all interfaces:
----------------------------------------------------------------------------------
Intf        Aggr      Port Port State  System-ID           System Aggr Forwarding
Name        Id   Pri  Pri  Key  State                      State        State
----------------------------------------------------------------------------------
1/1/49      lag10(mc) 1049 1    ALFNCD 02:00:00:00:01:00  65534 10     up
1/1/48      lag256    49   1    ALFNCD 04:90:81:00:36:56  65534 256    up

Partner details of all interfaces:
----------------------------------------------------------------------------------
Intf        Aggr      Port Port State  System-ID           System Aggr
Name        Id   Pri  Pri  Key                             
----------------------------------------------------------------------------------
1/1/49      lag10(mc) 250  128  ASFNCD f4:03:43:60:a4:d8  32768 21
1/1/48      lag256    49   1    ALFNCD 04:90:81:00:33:4a  65534 256

10000-RU33-SW1# sh mac-ad

MAC age-time       : 300 seconds
Number of MAC addresses : 14

MAC Address           VLAN Type    Port
--------------------------------------------------------------
00:50:56:8e:30:05     10   dynamic 1/1/17
00:50:56:a2:10:a8     20   dynamic lag256
00:50:56:8e:d0:4f     20   dynamic lag256
00:50:56:8e:1e:cb     20   dynamic 1/1/17
00:50:56:8e:92:30     20   dynamic lag256
04:90:81:00:33:4a     200  dynamic lag256
00:50:56:8e:7c:e7     200  dynamic pv lag10
00:50:56:8e:84:7a     200  dynamic pv lag10
04:90:81:00:33:4a     201  dynamic lag256
00:50:56:8e:7c:e7     201  dynamic lag10
00:50:56:8e:84:7a     201  dynamic lag10
04:90:81:00:33:4a     202  dynamic lag256
00:50:56:8e:b9:48     202  dynamic lag10
00:50:56:8e:36:8f     202  dynamic lag10

10000-RU-33-SW1# show arp vrf Synergy

IPv4 Address   MAC                 Port      Physical Port State      VRF
-------------------------------------------------------------------------------------------------------
10.10.202.2    00:50:56:8e:b9:48   vlan202   lag10         reachable  Synergy
10.10.202.3    00:50:56:8e:36:8f   vlan202   lag10         reachable  Synergy
10.10.200.240  04:90:81:00:33:4a   vlan200   lag256        reachable  Synergy
10.10.202.240  04:90:81:00:33:4a   vlan202   lag256        reachable  Synergy
10.10.200.3    00:50:56:8e:84:7a   vlan200   lag10         reachable  Synergy
10.10.200.2    00:50:56:8e:7c:e7   vlan200   lag10         reachable  Synergy

Total Number Of ARP Entries Listed: 6.
--------------------------------------------------------------------------------------------------------

10000-RU33-SW1# sh run
Current configuration:
!
!Version ArubaOS-CX DL.10.09.0010
!export-password: default
hostname 10000-RU-33-SW1
user admin group administrators password ciphertext AQBapYYxwdkONC4Sev+y+b04Fd0cjpymGqnlCM3LhbDcWe3qYgAAABXl9SsYxNtZG+srlqp3cbElDSoow9j3gCfvJDfHB
hrvHMipUH9e1HgOlG9JdpqZksDdVrM0Pjo1zikAKATkOWTdST+bvBjD2+4GQHgefUJw1PlZkh5S7kFEea+geJIwN63d
user afc_admin group administrators password ciphertext AQBapTpF3DW4Dzf95Cn2ycp7tQxBzppatqV12DYzpB3py+hKYgAAAOuKA2gWHrCe3Kc1LIleiPzyeZR7eWEaZf0ZwImpP
SmINdJDz9kKcTcjZpZgK6/u0IAn8qni4+iBgE/3xgMPXn0yEJXCQ07LFJ2R+UsVgxLsbWvf6LCEtGvPrvLhfnYX3UJv
no ip icmp redirect
profile leaf
vrf Synergy
!
!
!
!
!
ssh server vrf mgmt
psm
 host 10.10.10.150 vrf mgmt
vlan 1,10,20
vlan 200
 private-vlan primary
vlan 201
 private-vlan isolated primary-vlan 200
vlan 202
interface mgmt
 no shutdown
 ip static 10.10.10.213/24
 default-gateway 10.10.10.254
system interface-group 1 speed 10g !interface group 1 contains ports 1/1/1-1/1/4
system interface-group 5 speed 10g !interface group 5 contains ports 1/1/17-1/1/20
system interface-group 10 speed 10g !interface group 10 contains ports 1/1/37-1/1/40
system interface-group 12 speed 10g !interface group 12 contains ports 1/1/45-1/1/48
interface lag 10 multi-chassis
 no shutdown
 description provisioned
 no routing
 vlan trunk native 1
 vlan trunk allowed 1,200-202
 lacp mode active
 lacp fallback
 lacp rate slow
interface lag 256
 no shutdown
 description ISL
 no routing
 vlan trunk native 1
 vlan trunk allowed all
 lacp mode active
 lacp rate slow
 qos trust cos
interface 1/1/1
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/2
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/3
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/4
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/5
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/6
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/7
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/8
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/9
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/10
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/11
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/12
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/13
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/14
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/15
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/16
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/17
 no shutdown
 mtu 9198
 no routing
 vlan trunk native 1 tag
 vlan trunk allowed 10,20
interface 1/1/18
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/19
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/20
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/21
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/22
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/23
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/24
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/25
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/26
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/27
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/28
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/29
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/30
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/31
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/32
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/33
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/34
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/35
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/36
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/37
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/38
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/39
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/40
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/41
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/42
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/43
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/44
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/45
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/46
 no shutdown
 persona access
 mtu 9198
 qos trust cos
 ip mtu 9198
 ip address 192.168.10.2/31
interface 1/1/47
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/48
 no shutdown
 persona access
 mtu 9198
 lag 256
interface 1/1/49
 no shutdown
 persona access
 mtu 9198
 lag 10
interface 1/1/50
 no shutdown
 persona uplink
 mtu 9198
 ip mtu 9198
interface 1/1/51
 no shutdown
 persona uplink
 mtu 9198
 ip mtu 9198
interface 1/1/52
 no shutdown
 persona uplink
 mtu 9198
 ip mtu 9198
interface 1/1/53
 no shutdown
 persona uplink
 mtu 9198
 ip mtu 9198
interface 1/1/54
 no shutdown
 persona uplink
 mtu 9198
 ip mtu 9198
interface vlan 200
 vrf attach Synergy
 ip mtu 9198
 ip address 10.10.200.241/24
 active-gateway ip mac 00:00:00:00:00:01
 active-gateway ip 10.10.200.254
 ip local-proxy-arp
interface vlan 202
 vrf attach Synergy
 ip mtu 9198
 ip address 10.10.202.241/24
 active-gateway ip mac 00:00:00:00:00:02
 active-gateway ip 10.10.202.254
 ip local-proxy-arp
vsx
 system-mac 02:00:00:00:01:00
 inter-switch-link lag 256
 role secondary
 keepalive peer 192.168.10.3 source 192.168.10.2
 no split-recovery
 vsx-sync vsx-global
!
!
!
!
!
https-server vrf mgmt
</code>

----

==== CX10000-2 Configs and Verification Commands ====
You can use these commands to verify desired VLANs are redirected to DSM, LACP is functional and MACs, ARPs are learnt as expected, full configs are provided for reference.

<code>
10000-RU34-SW2# sh dsm 1/1 redirect
Distributed Services Modules 1/1
====================================
Filter information
No VLAN redirect configured to Distributed Services module

10000-RU34-SW2# sh dsm 1/2 redirect
Distributed Services Modules 1/2
====================================
Filter information
VLANs: 200-202

10000-RU34-SW2# sh lacp int

State abbreviations :
A - Active
P - Passive
F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync
O - OutofSync
C - Collecting D - Distributing
X - State m/c expired
E - Default neighbor state

Actor details of all interfaces:
----------------------------------------------------------------------------------
Intf        Aggr      Port Port State  System-ID           System Aggr Forwarding
Name        Id   Pri  Pri  Key  State                      State        State
----------------------------------------------------------------------------------
1/1/49      lag10(mc) 49   1    ALFNCD 02:00:00:00:01:00  65534 10     up
1/1/48      lag256    49   1    ALFNCD 04:90:81:00:33:4a  65534 256    up

Partner details of all interfaces:
----------------------------------------------------------------------------------
Intf        Aggr      Port Port State  System-ID           System Aggr
Name        Id   Pri  Pri  Key                             
----------------------------------------------------------------------------------
1/1/49      lag10(mc) 112  128  ASFNCD f4:03:43:60:a4:d8  32768 21
1/1/48      lag256    49   1    ALFNCD 04:90:81:00:36:56  65534 256

10000-RU34-SW2# sh mac-ad

MAC age-time       : 300 seconds
Number of MAC addresses : 13

MAC Address           VLAN Type    Port
--------------------------------------------------------------
00:50:56:a2:10:a8     20   dynamic 1/1/18
00:50:56:8e:d0:4f     20   dynamic 1/1/17
00:50:56:8e:1e:cb     20   dynamic lag256
00:50:56:8e:92:30     20   dynamic 1/1/17
04:90:81:00:36:56     200  dynamic lag256
00:50:56:8e:7c:e7     200  dynamic pv lag10
00:50:56:8e:84:7a     200  dynamic pv lag10
04:90:81:00:36:56     201  dynamic lag256
00:50:56:8e:7c:e7     201  dynamic lag10
00:50:56:8e:84:7a     201  dynamic lag10
04:90:81:00:36:56     202  dynamic lag256
00:50:56:8e:36:8f     202  dynamic lag10
00:50:56:8e:b9:48     202  dynamic lag10

10000-RU34-SW2# sh arp vrf Synergy

IPv4 Address   MAC                 Port      Physical Port State      VRF
---------------------------------------------------------------------------------------------
10.10.202.3    00:50:56:8e:36:8f   vlan202   lag10         reachable  Synergy
10.10.200.251  04:90:81:00:36:56   vlan200   lag256        reachable  Synergy

Total Number Of ARP Entries Listed: 2.
---------------------------------------------------------------------------------------------

10000-RU34-SW2# sh run
Current configuration:
!
!Version ArubaOS-CX DL.10.09.0010
!export-password: default
hostname 10000-RU34-SW2
user admin group administrators password ciphertext AQBapfGrz5kq6he5ykpcx4YR1KlJh13fWgVDCSdybQvHf5UhYgAAAE5u3cuwvp8FBs8yTvjLEDGTBi5uGjrQo22ur/4G5
7yjX6K5yhmcK33PG/g+hLs1NqozFFRx+S52ozvyKegnCXjs3piV4D/D5EKd01P8YeEZbv920OGcoXPLau6Ws8MiFKgk
user afc_admin group administrators password ciphertext AQBapS0Y4qS+NoaDC7C/qqGXB832EdF1A3/pSbsyx9RVlIhYYgAAAKHMMJ1XZ0JTwal8hvnFzMn52WtloGsB0+wRQNEF+
lFz04nnJuvGUy5zDtm/9dBLqg3ExgKxJIn4N1cHHHVWDy/7+upkCgY70LgGE7mEVFEC4wCqh596BZiNlHmTUq661vZo
no ip icmp redirect
profile leaf
vrf Synergy
!
!
!
!
!
ssh server vrf mgmt
psm
 host 10.10.10.150 vrf mgmt
vlan 1,10,20-21
vlan 200
 private-vlan primary
vlan 201
 private-vlan isolated primary-vlan 200
vlan 202
interface mgmt
 no shutdown
 ip static 10.10.10.212/24
 default-gateway 10.10.10.254
system interface-group 1 speed 10g !interface group 1 contains ports 1/1/1-1/1/4
system interface-group 5 speed 10g !interface group 5 contains ports 1/1/17-1/1/20
system interface-group 10 speed 10g !interface group 10 contains ports 1/1/37-1/1/40
system interface-group 12 speed 10g !interface group 12 contains ports 1/1/45-1/1/48

interface lag 10 multi-chassis
 no shutdown
 description provisioned
 no routing
 vlan trunk native 1
 vlan trunk allowed 1,200-202
 lacp mode active
 lacp fallback
 lacp rate slow

interface lag 256
 no shutdown
 description ISL
 no routing
 vlan trunk native 1
 vlan trunk allowed all
 lacp mode active
 lacp rate slow
 qos trust cos

interface 1/1/1
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/2
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/3
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/4
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/5
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/6
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/7
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/8
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/9
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/10
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/11
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/12
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/13
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/14
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/15
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/16
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/17
 no shutdown
 mtu 9198
 no routing
 vlan trunk native 1 tag
 vlan trunk allowed 10,20
interface 1/1/18
 no shutdown
 mtu 9198
 no routing
 vlan trunk native 1 tag
 vlan trunk allowed 20-21
interface 1/1/19
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/20
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/21
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/22
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/23
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/24
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/25
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/26
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/27
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/28
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/29
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/30
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/31
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/32
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/33
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/34
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/35
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/36
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/37
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/38
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/39
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/40
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/41
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/42
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/43
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/44
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/45
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/46
 no shutdown
 mtu 9198
 qos trust cos
 description Keep alive Interface 10000-RU34-SW2
 ip mtu 9198
 ip address 192.168.10.3/31
interface 1/1/47
 no shutdown
 mtu 9198
 ip mtu 9198
interface 1/1/48
 no shutdown
 mtu 9198
 lag 256
interface 1/1/49
 no shutdown
 persona access
 mtu 9198
 lag 10
interface 1/1/50
 no shutdown
 persona uplink
 mtu 9198
 ip mtu 9198
interface 1/1/51
 no shutdown
 persona uplink
 mtu 9198
 ip mtu 9198
interface 1/1/52
 no shutdown
 persona uplink
 mtu 9198
 ip mtu 9198
interface 1/1/53
 no shutdown
 persona uplink
 mtu 9198
 ip mtu 9198
interface 1/1/54
 no shutdown
 persona uplink
 mtu 9198
 ip mtu 9198
interface vlan 200
 vrf attach Synergy
 ip mtu 9198
 ip address 10.10.200.240/24
 active-gateway ip mac 00:00:00:00:00:01
 active-gateway ip 10.10.200.254
 ip local-proxy-arp
interface vlan 202
 vrf attach Synergy
 ip mtu 9198
 ip address 10.10.202.240/24
 active-gateway ip mac 00:00:00:00:00:02
 active-gateway ip 10.10.202.254
 ip local-proxy-arp
vsx
 system-mac 02:00:00:00:01:00
 inter-switch-link lag 256
 role primary
 keepalive peer 192.168.10.2 source 192.168.10.3
 no split-recovery
 vsx-sync vsx-global
!
!
!
!
!
https-server vrf mgmt
</code>