MPLS network issues can be caused by problems in different network layers. Troubleshooting can begin at any layer, but having a consistent method with an understanding of the function of each layer usually is the most effective and reliable. An efficient method of troubleshooting is to verify MPLS functionality at each layer, starting with the physical layer. By troubleshooting from one layer to the next, the issue can then be isolated and resolved.
Start with the basics, check the physical layer first. It’s generally the quickest and easiest to troubleshoot. Confirm the routers are connected and the interfaces are up by running ‘show interface’ commands and ping commands. If a physical layer issue is found, fix it then verify the LSP state.
Data Link Layer
When troubleshooting at this layer, you’ll be focusing on the encapsulation mode (PPP or HDLC). If an issue is found at this layer, it is generally a mismatch in the encapsulation type, PPP options, or keepalive frames. Troubleshooting should be done on the ingress, egress and transit routers. If an issue is found, resolve the issue, then verify the LSP state again.
When checking IP connectivity, confirm the router interfaces have the correct IP addresses and established the neighbor adjacencies for the interior gateway protocol (IGP), such as OSPF or IS-IS. First, verify the IGP configuration.
For OSPF, run the appropriate ‘show’ commands for OSPF to verify neighbor state. When checking OSPF, verify the router interfaces are in the correct area, and the loopback interface is configured and being used for OSPF. LSP (Label Switched Path) will not work correctly if the loopback interface is not configured for OSPF.
IS-IS is independent of the IP layer so it doesn’t matter whether the IP layer or IS-IS is checked first. Run the appropriate ‘show’ commands to verify IS-IS adjacencies. If a router is configured for a specific level, it doesn’t communicate with routers on a different level. For example, a level 2 router can communicate with other level 2 routers and other autonomous systems, but will not form an adjacency with a level 1 router. If a problem is found, resolve it, and then verify the LSP state.
RSVP and MPLS Layers
If the problem is still not resolved, it’s time to check the RSVP (Resource Reservation Protocol) and MPLS layers. Use the appropriate ‘show rsvp’ commands to check the dynamic RSVP signaling, correct configuration of the interfaces for RSVP, and RSVP neighbors are established. Checking the logs for RSVP-related events is also beneficial to narrow down the issue.
When checking the MPLS layer, run the appropriate ‘show mpls’ commands to verify the LSP is up and functioning. Other helpful commands to run are ‘show route,’ traceroute, and ping. For routers where the traceroute command is MPLS aware, the traceroute output will help you verify that packets are being sent over LSP. If the traceroute does not include the MPLS label in the output, that means the traffic is not using the LSP. If a problem is found, take steps to resolve it, then check the LSP state.
The final layer to check is the BGP (Border Gateway Protocol) layer. Because BGP uses MPLS LSP to forward traffic, the LSP must be established first. That is why it’s a good practice to troubleshoot the lower layers first and confirm the LSP state is established before troubleshooting BGP. When troubleshooting at this layer, verify the next hop is the LSP, and that the route exists and is active. The BGP next hop should be the LSP egress address. Otherwise the BGP traffic won’t use LSP. Using an MPLS aware traceroute can confirm if the LSP is being used to reach the next BGP hop. If the traceroute output includes the MPLS label, then the traffic is using the LSP. If the MPLS label is not included in the output, then the traffic is using the IGP layer to reach the next hop. If an issue is found, make the appropriate changes then verify the LSP again.
Final Troubleshooting Tip
There is one additional troubleshooting tip that can apply to any networking issue in addition to specific MPLS troubleshooting.
When a problem arises on the network, it’s always a good idea to find out when the last configuration changes were made. Often a recent change has an undesirable effect on the network. Ideally, there is a centralized logging server to check for configuration changes. Otherwise checking the logs on each router will need to be done. If not already configured, it’s a good idea to setup configuration tracking. Configuration tracking can be configured on the routers themselves, or through a monitoring or syslog server. Knowing what changes are being made on your MPLS network will make troubleshooting much quicker when issues arise.