GPON ONU Profile Match state is Mismatch

An Huawei GPON ONU connected to a GPON port of an Huawei MA5800 OLT or MA5600T OLT can go online successfully, but the queried Match state of the ONU is displayed as mismatch by running the display ont info command on the OLT.

Context

The ONU matching status indicates whether the actual ONU capability is the same as the service profile bound to the ONU. The status includes: initial, mismatch, and match. To some extent, the matching status is determined by the ONU running status and configuration status.

The matching status of the ONU can be queried only when the ONU running status is online. The matching status is match when the actual hardware capability of ONU is the same as the ONU service profile bound with the ONU. Otherwise, the status is mismatch.
In other configuration states, the matching status is initial.
The ONU matching status does not affect the normal forwarding of the service flow, and only indicates whether the actual ONU capability is the same as the service profile bound to the ONU.

In practice, ONUs in the offline state are bulk pre-configured on the OLT to facilitate service provisioning. An ONU service profile and an ONU line profile are specified during such configurations. The ONU profiles together can be regarded as a virtual ONU. Subsequent services are configured based on this virtual ONU. Inconsistency between the capability set configured in the ONU profiles and the actual ONU capabilities involves the following two situations:

The configured capability set outmatches the actual ONU capabilities. If the ONU is bound to such ONU profiles, ONU configurations will fail to match when the ONU goes online.
The configured capability set undermatches the actual ONU capabilities. In this case, the ONU capabilities that are not covered by the ONU profiles will fail to be configured or applied.

Location Method

When the queried Match state of the ONU is displayed as mismatch, locate the fault according to the following procedure:

Check whether the capability set configured in the ONU service profile matches the actual ONU capabilities.

Procedure

Run the display ont capability command to query the actual ONU capabilities. According to the data plan, modify the current ONU profiles, or bind matching ONU profiles to the ONU.
- If this problem occurs on all the ONUs of the same type, the configurations of the ONU profiles may be incorrect.
  - If the OLT works in the distributed mode, the profiles that are bound to the ONU cannot be modified or deleted. In this case, bind matching ONU profiles to the ONU.
  - If the OLT works in the profile mode:
    1. Run the display ont-srvprofile command to query the information about the ONU service profile and run the display ont-lineprofile command to query the information about the ONU line profile.
    2. Modify the ONU profiles by referring to Configuring a GPON ONT Profile in the Commissioning and Configuration Guide.
- If this problem occurs on only one ONU, it is suggested to bind matching ONU profiles to the ONU.
  - If the OLT works in the distributed mode:
    1. Run the display ont-profile command to query the current ONU profiles that are configured on the OLT.
    2. If the OLT does not have matching ONU profiles, run the ont-profile add command to add matching ONU profiles.
    3. Run the ont modify command to bind the ONU profiles to the ONU.
  - If the OLT works in the profile mode:
    1. Run the display ont-srvprofile command to query the information about the ONU service profile and run the display ont-lineprofile command to query the information about the ONU line profile.
    2. If the OLT does not have matching ONU profiles, add matching ONU profiles by referring to Configuring a GPON ONT Profile in the Commissioning and Configuration Guide.
    3. In the GPON mode of the OLT, run the ont modify command to bind the ONU profiles to the ONU.
Check whether Match state of the ONU is displayed as match.
- If Match state of the ONU is displayed as match, go to 4.
- If Match state of the ONU is displayed as mismatch, proceed to 3.
Connect Thunder-link.com for technical support.
The fault is rectified.

This case mainly talks about the ICMP delay caused by the packet processing mechanism of the CE switch.

Problem Description

The CE12808 switch is connected to the CRS_Router Cisco switch through four Layer 3 interfaces. After the interconnection, it's found that the ICMP delay between the two devices is long.

imgDownload?uuid=bd70c794892a43068942105

To check whether the link is normal, we make an ICMP test with large Packets and see the following on Huawei & Cisco.

1. From Huawei to Cisco :

~Huawei]ping -m 30 -t 8000 -c 100 -s 9152 -a 192.168.23.2 -f 192.168.23.1

PING 192.168.23.1: 9152 data bytes, press CTRL_C to break

Reply from 192.168.23.1: bytes=9152 Sequence=1 ttl=255 time=5 ms

Reply from 192.168.23.1: bytes=9152 Sequence=2 ttl=255 time=6 ms

Reply from 192.168.23.1: bytes=9152 Sequence=3 ttl=255 time=3 ms

Reply from 192.168.23.1: bytes=9152 Sequence=4 ttl=255 time=3 ms

Reply from 192.168.23.1: bytes=9152 Sequence=9 ttl=255 time=17 ms

Reply from 192.168.23.1: bytes=9152 Sequence=10 ttl=255 time=59 ms

Reply from 192.168.23.1: bytes=9152 Sequence=11 ttl=255 time=93 ms

Reply from 192.168.23.1: bytes=9152 Sequence=12 ttl=255 time=93 ms

Reply from 192.168.23.1: bytes=9152 Sequence=13 ttl=255 time=93 ms

Reply from 192.168.23.1: bytes=9152 Sequence=14 ttl=255 time=92 ms

Reply from 192.168.23.1: bytes=9152 Sequence=15 ttl=255 time=93 ms

...

Reply from 192.168.23.1: bytes=9152 Sequence=24 ttl=255 time=92 ms

Reply from 192.168.23.1: bytes=9152 Sequence=25 ttl=255 time=93 ms

--- 192.168.23.1 ping statistics ---

100 packet(s) transmitted

100 packet(s) received

0.00% packet loss

round-trip min/avg/max = 3/84/94 ms

2. From Cisco to Huawei :

Cisco# ping 192.168.24.2 source 192.168.24.1 size 9180 donotfrag count 30000 interval 5

Fri Jul 1 03:50:57.844 GMT

Type escape sequence to abort.

Sending 30000, 9180-byte ICMP Echos to 192.168.24.2, timeout is 2 seconds:

Success rate is 100 percent (153/153), round-trip min/avg/max = 1/90/94 ms

There seems to be some ISSUE on the link interconnect because the delay should not exceed 15 or 20 ms.

Handling Process

Before checking the issue we need to be clear about MTU for Cisco & Huawei:

A. Cisco Interface MTU is 9194: The ICMP test packet length is 9180 bytes through Cisco CRS Ios XR not consider 14 bytes for Ethernet on ICMP Test.

B. Huawei Interface MTU is 9180: The ICMP test packet length is 9152 bytes through Huawei does not consider 14 bytes for Ethernet, 8 bytes for ICMP, and 6 bytes for VLAN.

1. At the beginning of the issue, we thought the issue was on Optical Fibers and GE 10G Transceivers, so our plan was to verify physical links and made physical loops for verifying the Transceiver. After this, the issue continues.

[~Huawei]display interface 10GE8/0/46 transceiver verbose | in DBm

Current RX Power (dBm) :-2.79

Default RX Power High Threshold (dBm) :2.50

Default RX Power Low Threshold (dBm) :-16.40

Current TX Power (dBm) :-3.75

Default TX Power High Threshold (dBm) :2.50

Default TX Power Low Threshold (dBm) :-10.20

Cisco Ten 0/4/0/0 <<>> Huawei 10GE5/0/46

Cisco#sh controllers tenGigE 0/4/0/0 phy | in dBm

Fri Jul 1 03:18:00.095 GMT

Tx power: -3 dBm (4307 in units of 0.1 uW)

Rx power: -5 dBm (2597 in units of 0.1 uW)

<Huawei>display interface 10GE5/0/46 transceiver verbose | in dBm

Current RX Power (dBm) :-3.90

Default RX Power High Threshold (dBm) :2.50

Default RX Power Low Threshold (dBm) :-16.40

Current TX Power (dBm) :-3.26

Default TX Power High Threshold (dBm) :2.50

Default TX Power Low Threshold (dBm) :-10.20

2. Verify the packets for ICMP statistics and check if there is some issue with processing :

<Huawei>disp current-configuration configuration include-default | i fast

undo arp fast-reply disable

undo icmp echo-reply fast disable

[~Huawei]display icmp fast-reply statistics slot 5

------------------------ Display ICMP Statistics -------------------------------

Received packets:

request packets: 0

invalid request packets: 0

failed to get vrf: 0

destination is not host ip: 0

failed to get ctrl word: 0

Send packets:

successful reply packets: 0

failed reply packets: 0

[~Huawei]display icmp fast-reply statistics slot 8

------------------------ Display ICMP Statistics -------------------------------

Received packets:

request packets: 0

invalid request packets: 0

failed to get vrf: 0

destination is not host ip: 0

failed to get ctrl word: 0

Send packets:

successful reply packets: 0

failed reply packets: 0

[~Huawei]display icmp fast-reply statistics slot 1

------------------------ Display ICMP Statistics -------------------------------

Received packets:

request packets: 0

invalid request packets: 0

failed to get vrf: 0

destination is not host ip: 0

failed to get ctrl word: 0

Send packets:

successful reply packets: 0

failed reply packets: 0

--------------------------------------------------------------------------------

[~Huawei]display icmp fast-reply statistics slot 4

------------------------ Display ICMP Statistics -------------------------------

Received packets:

request packets: 0

invalid request packets: 0

failed to get vrf: 0

destination is not host ip: 0

failed to get ctrl word: 0

Send packets:

successful reply packets: 0

failed reply packets: 0

--------------------------------------------------------------------------------

Root Cause

The Ping Packet Processing on a CE Switch is like follows :

imgDownload?uuid=218fdfcb05dc45c28d3e53b

1. The green line in the preceding figure shows ping packet processing. The forwarding plane sends ping packets to the control plane through the CPU. The processing latency mainly includes the latency on the forwarding plane and the latency on the control plane. The latency on the forwarding plane is at the microsecond level and can be ignored. The latency on the control plane is determined by the CPU usage. If the CPU usage is high, the latency increases.

2. The red lines show service data flow processing. Service data packets are directly forwarded by the forwarding plane and are not processed by the control plane. The latency is at the microsecond level.

Analysis of Latency and Jitter During Ping Packet Transmission Between CE and Cisco Switches :

Ping packet processing is as follows:

1. The packet receiving process on the CE switch is as follows:

Control plane of the Cisco switch -> packet sending by the CPU of the Cisco switch -> forwarding plane of the Cisco switch -> forwarding plane of the CE switch -> packet receiving by the CPU of the CE switch -> control plane of the CE switch

2. The packet forwarding process on the CE switch is as follows:

Control plane of the CE switch -> packet sending by the CPU of the CE switch -> forwarding plane of the CE switch -> forwarding plane of the Cisco switch -> packet receiving by the CPU of the Cisco switch -> control plane of the Cisco switch.

Solution

The latency of ping packets mainly includes the processing latency on the Cisco switch and the processing latency on the CE switch. The processing latency on the CE switch mainly includes the latency on the forwarding plane and the latency on the control plane.

The latency on the forwarding plane is at the microsecond level and can be ignored. The latency on the control plane is determined by the CPU usage. If the CPU usage is high, the latency increases.

Because of the CPU performance and software calculation priority of the CE switch, a little latency and jitter may occur for ping packets that need to be processed by the CPU. However, data packet forwarding is not affected on the CE switch because data packets are forwarded by the hardware and software processing is not involved.

Suggestions and Summary

As we mentioned this issue is just related to IMCP Test, not with Traffic of Services.

That's true because when we put traffic on the Internet there is no issue with users.

Before the ICMP test, ensure that the test does not affect service traffic processing.

Daisy Thunder-link

Monday, June 27, 2022

Troubleshooting of GPON ONU Profile Match state is Mismatch

GPON ONU Profile Match state is Mismatch

Context

Location Method

Procedure

Sunday, June 12, 2022

Why ICMP delay with jumbo frames between Huawei and Cisco switch?

Problem Description

Handling Process

Root Cause

Solution

Suggestions and Summary