No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

HUAWEI Firewall How to Analyze IPSec Failures

Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
How to Analyze IPSec Failures

How to Analyze IPSec Failures

Introduction

This document provides in-depth analysis of the IKEv1 and IKEv2 negotiation processes, IPsec packet forwarding process, and IPsec working principles. Based on principle analysis, this document provides the troubleshooting method to help you locate faults and learn the causes behind the faults.

For details on common solutions to IPSec VPN failure and consulting issues, see Troubleshooting IPSec Issues that describes a checklist of common procedures that you might try before you begin to troubleshoot a connection and call Huawei Technical Support.

Prerequisites

  • It is recommended that you have knowledge of IPSec VPN configuration on HUAWEI firewalls.
  • This document uses Huawei USG6000 series firewall products of V5 version as an example. There may be differences in the implementation of different products and versions. Please refer to the specific version of the product documentation.
  • In this document, FW is short for Firewall.
  • In this document, public IP addresses may be used and are for reference only unless otherwise specified.

IPSec Failure Analysis

IPSec Tunnel Setup Failure

ISAKMP Packet Format

ISAKMP Packet Format

Internet Security Association and Key Management Protocol (ISAKMP) are basis of IKE. IKE uses ISAKMP packets for security association (SA) negotiation, key exchange, and peer identity verification.

Figure 1-1 shows the ISAKMP packet format.

Figure 1-1 ISAKMP packet format
IP Header

Source address: indicates the IP address that initiates IKE negotiation. It can be an interface IP address or an IP address configured using the tunnel local command.

Destination address: indicates the IP address of the remote device. It is configured using the remote-address command.

UDP Header

IKE uses port 500 to initiate negotiation and respond to negotiation. In the NAT traversal scenario, the gateway converts the detected port number 500 to 4500.

ISAKMP Header

Initiator's Cookie (SPI): specifies a number used by the initiator to uniquely identify an IKE SA. The SPI cannot be 0.

Responder's Cookie (SPI): specifies a number used by the responder to uniquely identify an IKE SA. The SPI in the first message is 0, and in later messages cannot be 0.

Next Payload: indicates the type of next payload in a message. If the current payload is the last one in a message, the value of this field is 0. This field provides the linking capability between payloads. To add a new payload to the end of a message, specify the type of the new payload at the end of the previous payload.

Table1 and Table2 list the next payload types in IKEv1 and IKEv2.

Table 1-1 Next payloads in IKEv1

Next Payload

Abbreviation

Value

Function

None

-

0

Indicate that the current payload is the last one in the message.

Security Association

SA

1

Negotiate an IKE or IPSec proposal.

Proposal

P

2

Negotiate the security protocol and security mechanism used by an SA.

Transform

T

3

Transmit SA attributes for negotiating the proposal supported by both parties.

Key Exchange

KE

4

Exchange the Diffie-Hellman (DH) public number as part of a DH key exchange.

Identification

IDi, IDr

5

Send an identity ID for access control. There is no need to match this payload in certificate authentication.

Certificate

CERT

6

Transmit the certificate or other authentication-related information.

Certificate-Request

CERTREQ

7

Request the preferred certificate.

Hash

HASH_I, HASH_R

8

Verify integrity of an IKE message, or authenticate the peer.

Signature

SIG_I, SIG_R

9

Verify integrity of an IKE message and provide the non-repudiation service.

Norigication

N

10

Announce errors and status changes.

Nonce

Ni, Nr

11

Transmit the temporary random number.

Delete

D

12

Indicate that the SA with the specified SPI has been deleted.

Vendor ID

V

13

Indicate that the sender can accept protocol extensions.

Properties

-

14

Indicate an attribute payload.

NAT Discovery

NAT-D

20

Determine whether the device is connected to a NAT device.

NAT Original Address

NAT-OA

21

Specify the original address of an IPSec session party.

Reserved

-

15-127

Reserved.

Private Use

-

128-255

Private use.

Table 1-2 Next payloads in IKEv2

Next Payload

Abbreviation

Value

Function

No Next Payload

-

0

Indicate that the current payload is the last one in the message.

Reserved

-

1-32

Reserved.

Security Association

SA

33

Negotiate an IKE or IPSec proposal.

Key Exchange

KE

34

Exchange the DH public number as part of a DH key exchange.

Identification-Initiator

IDi

35

Send an identity ID for access control. There is no need to match this payload in certificate authentication.

Identification-Responder

IDr

36

Certificate

CERT

37

Transmit the certificate or other authentication-related information.

Certificate-Request

CERTREQ

38

Request the preferred certificate.

Authentication

Auth

39

Send the authentication method and data (hash values).

Nonce

Ni, Nr

40

Transmit the temporary random number.

Notify

N

41

Announce errors and status changes.

Delete

D

42

Indicate that the SA with the specified SPI has been deleted.

Vendor ID

V

43

Indicate that the sender can accept protocol extensions. Currently, this payload is used to negotiate the NAT-T capability.

Traffic Selector-Initiator

TSi

44

Specify data flows to be protected by IPSec.

Traffic Selector-Responder

TSr

45

Encrypted

E

46

Transmit encrypted values of other payloads.

Configuration

CP

47

Include CFG_REQUEST, CFG_REPLAY, CFG_SET, and CFG_ACK for address allocation request and reply.

Extensible-Authentication

EAP

48

Carry Extensible Authentication Protocol (EAP) information.

Reserved to

IANA

49-127

Reserved.

Private Use

-

128-255

Private use.

Maj Ver/Min Ver: identifies the ISAKMP version by numbers of the Maj Ver and Min Ver fields. In IKE, the two fields are combined.

Version: In IKEv1, the value of this field is 1.0. In IKEv2, the value of this field is 2.0.

Exchange Type: indicates the exchange type. This field constrains the payloads sent in each message and orderings of messages in an exchange.

Table3 and Table4 list exchange types in IKEv1 and IKEv2.

Table 1-3 Exchange types in IKEv1

Exchange Type

Value

None

0

Base

1

Identity protection

2

Authentication only

3

Aggressive

4

Informational

5

Future use

6-31

DOI specific use

32-239

Private use

240-255

Table 1-4 Exchange types in IKEv2

Exchange Type

Value

Reserved

0-33

IKE_SA_INIT

34

IKE_SA_AUTH

35

Create_CHILD_SA

36

Informational

37

Reserved to IANA

38-239

Reserved for Private Use

240-255

Flags: indicates specific options for ISAKMP exchange. For flags in this field, the meanings of each bit starting from the least significant bit are as follows:

  • IKEv1: Bit 0 is the encryption bit. When set to 1, it means the payload is encrypted. Bit 1 is the commit bit and, if set, it ensures that encrypted material is not received prior to SA establishment. Bit 2 is the authentication bit and implies, if set, that the payload will be only authenticated and not encrypted. The other bits are not defined, and must be set to 0.
  • IKEv2: Bit 3, if set to 1, indicates the initiator. Bit 4 indicates IKEv2 and must be set to 0. Bit 5, if set to 1, indicates the responder. The other bits are not defined, and must be set to 0.

Message ID: The value of this field is 0 in phase 1 and a random number generated by the initiator in phase 2. As the unique identifier of a message, it identifies the protocol status in phase 2 negotiation.

Message Length: indicates the length of the total ISAKM message, which is the header plus the payloads.

Negotiation Process in IKEv1 Phase 1

The purpose of IKEv1 phase 1 negotiation is to establish an IKE SA. After an IKE SA is established, all the Internet Security Association and Key Management Protocol (ISAKMP) messages transmitted between two IKE peers will be encrypted and authenticated. The secure tunnel established in phase 1 enables IKE peers to communicate securely in phase 2.

IKEv1 phase 1 supports two negotiation modes:

Negotiation Process in Main Mode

The main mode uses six ISAKMP messages to implement three bidirectional exchanges, as shown in Figure 1-2.

  1. Policy exchange: messages (1) and (2)
  2. Key information exchange: messages (3) and (4)
  3. Identity and authentication information exchange: messages (5) and (6)
Figure 1-2 Negotiation process in main mode (pre-shared key authentication)

Policy Exchange

Message (1): The initiator sends an SA payload containing one or more IKE proposals to negotiate an IPSec proposal. The SA carries parameters, such as the encryption algorithm, authentication method, authentication algorithm, Diffie-Hellman (DH) group, and IKE SA lifetime.

Message (2): The responder finds the first matched IKE proposal and sends an SA payload to indicate that it accepts the negotiated IKE proposal.

Figure 1-3 Messages (1) and (2)

Key Information Exchange

Messages (3) and (4): The initiator and responder exchange DH public number (KE payload) and temporary random number (Ni, Nr). Ni and Nr are used to calculate the encryption key and authentication key.

Figure 1-4 Messages (3) and (4)

Identity and Authentication Information Exchange

Messages (5) and (6): The initiator and responder exchange the identity ID (ID payload) and verify hash values (AUTH payload). Information transmitted in messages (5) and (6) is encrypted using the key generated based on key information exchanged in messages (3) and (4), so identity information is protected. As the ID payload is encrypted in main mode, you cannot view its value in the obtained packet header.

Figure 1-5 Messages (5) and (6)

Negotiation Process in Aggressive Mode

In aggressive mode, only three messages are used in the exchange process, as shown in Figure 1-6.

Messages (1) and (2) are used to negotiate IKE proposal and exchange the Diffie-Hellman public number, mandatory auxiliary information, and identity information. Message (2) also contains the identity information sent by the responder to the initiator for authentication. Message (3) is used by the responder to authenticate the initiator.

Figure 1-6 Negotiation process in aggressive mode (pre-shared key authentication)

Message (1): The initiator sends an SA payload containing one IKE proposal. In aggressive mode, the SA payload contains only one proposal (encryption algorithm, authentication method, authentication algorithm, DH group, and IKE SA lifetime). The responder can accept or reject this proposal. This message also carries the DH public number (KE payload), temporary random number (Ni payload), and identity ID (ID payload). Identity information is not encrypted in aggressive mode, so you can view the information in the obtained packet header.

Figure 1-7 Message (1)

Message (2): The responder sends an SA payload containing the recommended IKE proposal. This message also carries the DH public number (KE payload), temporary random number (Nr payload), identity ID (ID payload), and hash value (AUTH payload).

Figure 1-8 Message (2)

Message (3): The initiator sends this message to verify the hash value and confirm negotiation success.

Figure 1-9 Message (3)
Summary

After IKEv1 phase 1 negotiation is complete, run the display ike sa command to view the IKE SA establishment. If Flag is displayed as RD or RD|ST, IKE SAs are established successfully. The value ST indicates that the local end initiates SA negotiation.

<sysname> display ike sa 
IKE SA information :  
      Conn-ID      Peer                 VPN    Flag(s)        Phase 
  ------------------------------------------------------------------ 
      16           2.1.1.1:500                 RD|ST          v1:2 
      14           2.1.1.1:500                 RD|ST          v1:1 
                                                
   Number of IKE SA : 2 
  ------------------------------------------------------------------- 
  RD--READY   ST--STAYALIVE   RL--REPLACED   FD--FADING   TO--TIMEOUT 
  HRT--HEARTBEAT   LKG--LAST KNOWN GOOD SEQ NO.   BCK--BACKED UP 
  M--ACTIVE   S--STANDBY   A--ALONE  NEG--NEGOTIATING  

If negotiation fails, the output of this command is empty, the Flag parameter is empty, or the Peer parameter is displayed as 0.0.0.0.

If parameters such as the IKE proposal and pre-shared key on two ends of a tunnel do not match, an IKE SA cannot be established. Common debugging information for inconsistent IKE parameters is as follows:

  • The IKE proposals configured on two ends are different.
    <sysname> debugging ikev1 error
     Aug 17 2017 14:29:15.800.1 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 17:5773 Message from peer 2.1.1.1: Got NOTIFY of type NO_PROPOSAL_CHOS 
     EN
      
     Aug 17 2017 14:29:15.800.3 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 0:9946 Ikev1 error-info record(peer address: 2.1.1.1, error reason: ph 
     ase1 proposal mismatch,list number: 11). 
  • The pre-shared keys configured on two ends are different.
    <sysname> debugging ikev1 error
     Aug 17 2017 15:18:09.800.1 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 17:5773 Message from peer 2.1.1.1: Got NOTIFY of type PAYLOAD_MALFORME
     D  
    <sysname> debugging ikev1 error
     Aug 17 2017 15:24:53.940.2 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 17:1053 Message from peer 1.1.1.1: Invalid Next Payload of Type 60 in  
     Payload Type 5
                                                                                      
     Aug 17 2017 15:24:53.940.3 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 17:6847 Message from peer 1.1.1.1: dropping Message due to notificatio 
     n type INVALID_PAYLOAD_TYPE
          
     Aug 17 2017 15:24:53.940.4 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 17:4538 Message from peer 1.1.1.1: Message Sort Error Occurred
                            
     Aug 17 2017 15:24:53.940.5 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 0:9946 Ikev1 error-info record(peer address: 1.1.1.1, error reason: ma 
     lformed payload,list number: 200). 
  • The remote-address command is not configured on one end.
    <sysname> debugging ikev1 error
     Aug 17 2017 16:13:33.940.3 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 0:9946 Ikev1 error-info record(peer address: 1.1.1.1, error reason: pe 
     er address mismatch,list number: 200).                                           
                                                                                      
     Aug 17 2017 16:13:33.940.4 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 0:3595 Phase 1 Exchange: ike peer configuration not found for peer "1. 
     1.1.1"
  • The remote-id parameter value on the local end differs from the name of the remote end.
    <sysname> debugging ikev1 error
     Aug 17 2017 16:25:01.850.2 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 25:6956 ERROR - Received remote-name(fw2) does not match with peer rem 
     ote-name(fw3)
                                                                                      
     Aug 17 2017 16:25:01.850.3 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 0:9946 Ikev1 error-info record(peer address: 2.1.1.1, error reason: co 
     nfig ID mismatch,list number: 148). 

Negotiation Process in IKEv1 Phase 2

In IKEv1 phase 2, IPSec SAs are established for data transmission. In this phase, quick exchange mode is used and the exchanged payloads are encrypted.

Negotiation Process

Figure 1-10 shows the exchange process in quick mode.

Figure 1-10 Negotiation process in IKEv1 phase 2

Messages (1) and (2): The two messages are used to negotiate an IPSec proposal (SA payload) and negotiate the DH key group (KE payload) used in the perfect forward secrecy (PFS) function. The two messages also exchange the identity ID (ID payload, which is optional) and hash value for integrity verification (AUTH payload).

IDi and IDr are included in the ID payload, and are used to exchange the traffic selector. They ensure that data flows protected by both parties are the same. (This function is similar to that of the TS payload in IKEv2.)

Figure 1-11 Messages (1) and (2)

Message (3): The initiator sends this message to verify the hash value and confirm negotiation success. The packet format is similar to the format of message (1).

Summary

After IKEv1 phase 2 negotiation is complete, run the display ike sa command to view the IPSec SA establishment in phase 2. If Flag is displayed as RD or RD|ST, IPSec SAs are established successfully. The value ST indicates that the local end initiates SA negotiation.

<sysname> display ike sa 
IKE SA information :  
      Conn-ID      Peer                 VPN    Flag(s)        Phase 
  ------------------------------------------------------------------ 
      16           2.1.1.1:500                 RD|ST          v1:2 
      14           2.1.1.1:500                 RD|ST          v1:1 
                                                
   Number of IKE SA : 2 
  ------------------------------------------------------------------- 
  RD--READY   ST--STAYALIVE   RL--REPLACED   FD--FADING   TO--TIMEOUT 
  HRT--HEARTBEAT   LKG--LAST KNOWN GOOD SEQ NO.   BCK--BACKED UP 
  M--ACTIVE   S--STANDBY   A--ALONE  NEG--NEGOTIATING  

If negotiation fails, the Flag parameter in the command output is empty.

Inconsistent IPSec proposals, PFS algorithms, and ACL rules on two ends may lead to the IPSec SA establishment failure. Packets in phase 2 are encrypted, so you cannot check the IPSec proposal configuration by obtaining the packet headers.

Common debugging information for inconsistent IPSec parameters is as follows:

  • ACL rules on two ends do not match.
    <sysname> debugging ikev1 error
     Aug 17 2017 17:35:52.350.1 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 17:5773 Message from peer 1.1.1.1: Got NOTIFY of type INVALID_ID_INFOR 
     MATION
                                                                                      
     Aug 17 2017 17:35:52.350.2 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 0:9946 Ikev1 error-info record(peer address: 1.1.1.1, error reason: fl 
     ow mismatch,list number: 200).  
    <sysname> debugging ipsec error
     Aug 17 2017 17:35:17.800.1 sysname IPSEC/7/IPSEC-DEBUG:  
     [IPSEC-Error] acl mismatch.(Ifindex=[7], SeqNum=[10],PeerAddress=[2.1.1.1], Peer 
     Port=[500])   
  • The IPSec proposals or PFS algorithms on two ends are different.
    <sysname> debugging ikev1 error
     Aug 18 2017 09:28:45.350.2 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 0:9946 Ikev1 error-info record(peer address: 2.1.1.1, error reason: ph 
     ase2 proposal or pfs mismatch,list number: 200).                                 
                                                                                      
     Aug 18 2017 09:28:45.350.3 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 17:6847 Message from peer 2.1.1.1: dropping Message due to notificatio 
     n type NO_PROPOSAL_CHOSEN
    <sysname> debugging ipsec error
     Aug 18 2017 09:29:26.800.1 sysname IPSEC/7/IPSEC-DEBUG: 
     [IPSEC-Error] ipsec proposal or pfs mismatch.(Ifindex=[7], SeqNum=[10],PeerAddre 
     ss=[2.1.1.1], PeerPort=[500]) 

IKEv2 Negotiation Process

Negotiation Process

IKEv1 goes through two phases to establish a pair of IPSec SAs: "main mode + quick mode" or "aggressive mode + quick mode". When IKEv1 phase 1 uses the main mode, IKE peers exchange at least nine messages. When IKEv1 phase 1 uses the aggressive mode, IKE peers exchange at least six messages. IKEv2, without the main mode or aggressive mode, establishes an IKE SA and a pair of IPSec SAs by exchanging four messages at two times. Figure 1-12 shows the IKEv2 negotiation process.

Figure 1-12 IKEv2 negotiation process (pre-shared key authentication)

Messages (1) and (2): In the IKE_SA_INIT exchange, the initiator and responder negotiate an IKE proposal (SA payload) and exchange the temporary random number (Ni or Nr payload) and DH public number (KE payload).

Figure 1-13 Messages (1) and (2)

Messages (3) and (4): In the IKE_AUTH exchange, the two parties continue to exchange the identity ID (ID payload) and hash value (AUTH payload), negotiate the IPSec proposal (SA payload), and determine data flows to be protected (TS payload). The N payload is used for error announcement. After negotiation, an IKE SA and a pair of IPSec SAs are established.

Figure 1-14 Messages (3) and (4)

Summary

After IKEv2 negotiation is complete, run the display ike sa command to view the IKE SA establishment. If Flag is displayed as RD or RD|ST, IKE SAs are established successfully. The value ST indicates that the local end initiates SA negotiation.

<sysname> display ike sa 
IKE SA information :  
      Conn-ID      Peer                 VPN    Flag(s)        Phase 
  ------------------------------------------------------------------ 
      16           2.1.1.1:500                 RD|ST          v2:2 
      14           2.1.1.1:500                 RD|ST          v2:1 
                                                
   Number of IKE SA : 2 
  ------------------------------------------------------------------- 
  RD--READY   ST--STAYALIVE   RL--REPLACED   FD--FADING   TO--TIMEOUT 
  HRT--HEARTBEAT   LKG--LAST KNOWN GOOD SEQ NO.   BCK--BACKED UP 
  M--ACTIVE   S--STANDBY   A--ALONE  NEG--NEGOTIATING  

If negotiation fails, the following may occur when you run the display ike sa command:

  • If phase 1 negotiation fails, the command output is empty, the Flag parameter is empty, or the Peer parameter is displayed as 0.0.0.0.
  • If phase 2 negotiation fails, information about phase 1 is normally displayed; information about phase 2 is empty or the Flag parameter is empty.

Inconsistent IKE proposals, pre-shared keys, IPSec proposals, PFS algorithms, and ACL rules on two ends may lead to the IKE SA or IPSec SA establishment failure.

Common debugging information for inconsistent IPSec parameters is as follows:

  • The IKE proposals configured on two ends are different.
    <sysname> debugging ikev2 error
     Aug 18 2017 14:51:00.410.2 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 56:8841 Ikev2 error Info record(peer address: 2.1.1.1, error reason: p 
     hase1 proposal mismatch,list number: 200).   
  • The pre-shared keys configured on two ends are different.
    <sysname> debugging ikev2 error
     Aug 18 2017 15:02:25.540.2 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 63:4087 Authentication failed for the peer 2.1.1.1
      
     Aug 18 2017 15:02:25.540.4 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 56:8841 Ikev2 error Info record(peer address: 2.1.1.1, error reason: a 
     uthentication fail,list number: 4). 
  • The remote-id parameter value on the local end differs from the name of the remote end.
    <sysname> debugging ikev2 error
     Aug 18 2017 15:39:32.50.4 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 27:27568 ERROR - Peer remote-name(fw3) does not match with
                                                                                      
     Aug 18 2017 15:39:32.50.5 FW1 IKE/7/IKE_Debug: 
     IKE_ERROR 56:8841 Ikev2 error Info record(peer address: 2.1.1.1, error reason: c 
     onfig ID mismatch,list number: 151). 
  • The remote-address command is not configured on one end.
    <sysname> debugging ikev2 error
     Aug 18 2017 15:23:04.400.1 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 56:8841 Ikev2 error Info record(peer address: 2.1.1.1, error reason: p 
     eer address mismatch,list number: 107).  
  • ACL rules on two ends do not match.
    <sysname> debugging ikev2 error
     Aug 18 2017 15:27:40.750.1 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 56:8841 Ikev2 error Info record(peer address: 1.1.1.1, error reason: f 
     low mismatch,list number: 200).  
    <sysname> debugging ipsec error
     Aug 18 2017 15:29:46.650.1 sysname IPSEC/7/IPSEC-DEBUG: 
     [IPSEC-Error] acl mismatch.(Ifindex=[7], SeqNum=[0],PeerAddress=[1.1.1.1], PeerP 
     ort=[500]) 
  • The IPSec proposals or PFS algorithms on two ends are different.
    <sysname> debugging ikev2 error
     Aug 19 2017 09:41:40.800.2 sysname IKE/7/IKE_Debug: 
     IKE_ERROR 56:8841 Ikev2 error Info record(peer address: 2.1.1.1, error reason: p 
     hase2 proposal or pfs mismatch,list number: 200).    
    <sysname> debugging ipsec error
     Aug 19 2017 09:44:25.760.2 sysname IPSEC/7/IPSEC-DEBUG:                              
     [IPSEC-Error] ipsec proposal or pfs mismatch.(Ifindex=[7], SeqNum=[10],PeerAddre 
     ss=[2.1.1.1], PeerPort=[500])  

IKE Negotiation with NAT Traversal

When a NAT gateway is deployed between devices on two ends of an IPSec tunnel, IKE automatically completes NAT-T capability negotiation, NAT gateway detection, and SA establishment.

IKEv1 Negotiation with NAT Traversal

In the NAT traversal scenario using IKEv1, the following must be detected in phase 1 negotiation:

  • Support for NAT traversal (NAT-T capability)

    IKEv1 uses the VID payload to determine whether NAT traversal is supported.

  • Presence of a NAT gateway

    IKEv1 uses the NAT-D payload to detect whether a NAT gateway exists between two IKE peers and determine its location (finding the device connecting to the NAT gateway) if available.

As the NAT gateway changes the source port number in IKE UDP packets, the responder must be able to process IKE packets whose source port number is not 500.

Figure 1-15 shows the negotiation process of IKEv1 phase 1 in aggressive mode with NAT traversal.

The negotiation process in main mode with NAT traversal is omitted here. The process is similar to Negotiation Process in Main Mode, with only the following differences: The VID payload is added to messages (1) and (2), and the NAT-D payload is added to messages (3) and (4).

Figure 1-15 Negotiation process of IKEv1 phase 1 in aggressive mode with NAT traversal (pre-shared key authentication)

Message (1): The initiator inserts the VID payload to an IKE message to inform the receiver that it supports NAT traversal. If IKEv1 messages sent by both parties contain the VID payload, both parties support NAT-T. In this case, negotiation continues. Otherwise, the process comes to an end.

Figure 1-16 Message (1)

Message (2): The responder inserts the VID payload to an IKE message to inform the receiver that it supports NAT traversal. The responder also adds two NAT-D payloads to the message. The first NAT-D payload contains the IP address of the IKE peer and the port's hash value; the second NAT-D payload contains the IP address of the local end and the port's hash value. Both the initiator and responder calculate the hash values and compare them with local ones. The device with inconsistent hash values is connected to the NAT gateway.

Figure 1-17 Message (2)

Message (3): After confirming NAT-T support and detecting NAT gateway, the port number in subsequent UDP packets is modified to 4500 if a NAT gateway is detected.

Figure 1-18 Message (3)

The non-ESP marker field (four all-0 bytes) is added under UDP encapsulation of IPSec packets to distinguish from encapsulation of ESP packets.

In IKEv1 phase 2 SA negotiation, whether NAT traversal is used and the encapsulation mode (UDP-Encapsulated-tunnel or UDP-Encapsulated-transport) for NAt traversal must be confirmed. Then, subsequently transmitted ESP packets are encapsulated using UDP (as the AH protocol does not support NAT traversal). After an ESP packet is encapsulated using UDP, the packet does not contain the non-ESP marker field. Instead, this field is replaced by the SPI field, which consists non-0 bytes.

IKEv2 Negotiation with NAT Traversal

In the NAT traversal scenario using IKEv2, the initiator and responder add two N payloads (next to the Ni or Nr payload) in IKE_SA_INIT exchange. One message type is NAT_DETECTION_SOURCE_IP, which identifies the initiator's IP address. The other message type is NAT_DETECTION_DESTINATION_IP, which identifies the responder's IP address.

Figure 1-19 shows the IKEv2 negotiation process with NAT traversal.

Figure 1-19 IKEv2 negotiation process with NAT traversal (pre-shared key authentication)

Messages (1) and (2): Two N payloads are inserted to an IKE message. The first N payload contains the IP address of the local end and the port's hash value; the second N payload contains the IP address of the IKE peer and the port's hash value. Both the initiator and responder calculate the hash values and compare them with local ones. The device with inconsistent hash values is connected to the NAT gateway.

Figure 1-20 Messages (1) and (2)

Messages (3) and (4): After the IKE_SA_INIT exchange, the port number of UDP packets is modified to 4500 if a NAT device is detected.

Figure 1-21 Messages (3) and (4)

Summary

The NAT traversal function must be enabled on both ends to ensure IKE negotiation success.

After IKE negotiation is complete, run the display ike sa command to view the SA establishment. If Flag is displayed as RD or RD|ST, IKE SAs are established successfully. The value ST indicates that the local end initiates SA negotiation.

<sysname> display ike sa 
IKE SA information :  
      Conn-ID      Peer                 VPN    Flag(s)        Phase 
  ------------------------------------------------------------------ 
      16           2.1.1.1:4500                RD|ST          v2:2 
      14           2.1.1.1:4500                RD|ST          v2:1 
                                                
   Number of IKE SA : 2 
  ------------------------------------------------------------------- 
  RD--READY   ST--STAYALIVE   RL--REPLACED   FD--FADING   TO--TIMEOUT 
  HRT--HEARTBEAT   LKG--LAST KNOWN GOOD SEQ NO.   BCK--BACKED UP 
  M--ACTIVE   S--STANDBY   A--ALONE  NEG--NEGOTIATING  

Abnormal Services After Successful IPSec Tunnel Setup

IPSec Packet Forwarding Process

Figure 1-22 shows the IPSec packet forwarding process.

Figure 1-22 IPSec packet forwarding process

In the FW processing procedure, IPSec processes packets after NAT, routing, and security policies. It must be ensured that no NAT policy processes IPSec protected packets, and the packets can match a route and security policy to be forwarded to an interface to which an IPSec policy is applied. The following requirements must be met:

  1. The packet arriving at the FW does not match the server map table and reverse server map table established by the NAT server. Otherwise, the destination address of the packet will be converted.
  2. The packet arriving at the FW does not match the DNAT policy. Otherwise, the destination address of the packet will be converted.
  3. The routing table must have a route (usually a default route) to the private network of the IKE peer. The outbound interface in the route must have an IPSec policy applied. If the packet does not match any route in the routing table, the packet will be discarded. If the outbound interface in the matching route does not have an IPSec policy applied, the packet will not be processed by the IPSec module. Instead, the outbound interface sends the packet in plain text.
  4. Typically, IPSec data flows between zones. Therefore, interzone packet filtering must be enabled between the source zone (to which the intranet interface belongs) and destination zone (to which the IPSec policy applied extranet interface belongs). Otherwise, the packet will be discarded.
  5. SNAT may or may not be applied to the packet passing interzone packet filtering. If the packet matches an interzone NAT policy in SNAT, the source IP address of the packet will be converted. In this case, the security ACL filters the packet by the converted source IP address. If the packet does not match any interzone NAT policy, it is sent to the IPSec module directly.
  6. The packet entering the IPSec module is protected only if it matches a security ACL. Otherwise, the packet will be discarded.
NOTE:

Data flows triggering IKE negotiation follow the preceding procedure too. If a data flow fails in any stage along this procedure, it will not trigger IKE negotiation.

IPSec Working Principles

The security policy database (SPD) is basis for establishment of IPSec SAs, which defines data flows to be protected by IPSec. The security association database (SAD) is used to store all attributes of the IPSec SAs.

This section describes IPSec working principles using point-to-point unidirectional data transmission (in tunnel mode) as an example. In Figure 1-23, FW1 is the branch gateway; FW2 is the simulated carrier device; FW3 is the headquarters gateway. PC1 and PC2 are private network hosts in the branch and headquarters. IPSec is deployed on FW1 and FW3 to protect communication between the network segments where PC1 and PC2 reside. IPSec encrypts packets transmitted between GE0/0/2 on FW1 and GE0/0/2 on FW3.

Figure 1-23 IPSec VPN working principles
  • FW1
    1. PC1 sends an IP packet with PC2's address as the destination address to FW1.
    2. After receiving this packet from PC1, FW1 searches for a route to PC2 in the routing table.
    3. FW1 finds that the outbound interface is GE0/0/2, and forwards the packet to GE0/0/2.
    4. After the packet reaches the outbound interface with the IPSec policy applied, it checks the packet based on packet characteristics to determine whether the packet should be protected by IPSec (using an ACL). If so, the outbound interface encapsulates the packet based on negotiated IPSec parameters. The steps are as follows:
      1. The outbound interface searches the SPD and determines whether to process this packet using IPSec based on the source and destination addresses.

        □ If the action is permit, the packet needs to be processed by the IPSec module.

        □ If the action is drop, the packet does not need to be processed by the IPSec module.

      2. The outbound interface searches for an SA based on the destination address.

        □ If a matching entry is found, go to step c. (In manual mode, the SA and SAD have been generated after the configuration is complete.)

        □ If no matching entry is found, the outbound interface initiates IKE negotiation. After successful negotiation, two engaged devices generate their own SA and SAD.

      3. Based on the SPI, the outbound interface searches for a matching IPSec SA in the SAD. Then, it encrypts the packet, calculates the hash value, and encapsulates the packet based on the parameters of the SA.
    5. The outbound interface GE0/0/2 forwards the encapsulated packet along the found route.
  • FW2

    FW2 forwards the packet based on the routing information.

  • FW3
    1. FW3 receives the packet on GE0/0/2, and finds the destination address being its own IP address. So, FW3 sends the packet to the network layer to determine whether it is an IPSec packet (IP protocol number: ESP 50; AH 51). If so, the IPSec module decrypts and decapsulates the packet. The steps are as follows:
      1. (Optional) The IPSec module searches the SPD and performs pre-IPSec check on the (unencrypted) IP packet.

        If the packet needs to be encrypted but not, the packet will be discarded.

      2. The IPSec module searches the SAD and SA based on the SPI, destination IP address, and protocol number. Then, it decapsulates, verifies, and decrypts the packet based on the SA parameter settings.
      3. (Optional) The IPSec module searches the SPD and performs post-IPSec check on the decrypted IP packet.

        If the packet not requiring encryption is encrypted, the packet will be discarded.

    2. After packet decapsulation, FW3 searches for the outbound interface based on the destination address. It finds that the destination address is in the same network segment as the local outbound interface, so it forwards the packet through GE0/0/1 based on the ARP table.

IPSec Service Interruption

In Figure 1-24, an IPSec tunnel is successfully established between FWs.

Figure 1-24 Point-to-point IPSec networking

If PCs in the headquarters and branch cannot access each other, perform the check in the following sequence:

  1. Can packets sent by PC1 reach the gateway FW1?

    PC1 must have a reachable route to FW1. That is, you can ping the IP address 10.1.1.1 of FW1 on PC1.

  2. Can packets sent by PC1 be transmitted from a trust zone to an untrust zone?

    FW1 matches received packets with the routing table and then security policies for interzone packet filtering. So, the following requirements must be met:

    • FW1 has a reachable route (the default route also applies) to the network segment 10.2.1.0/24, and the outbound interface (GE0/0/2) in the route has an IPSec policy applied.
    • Interzone packet filtering for 10.1.1.0/24 in the trust zone to 1.1.1.0/24 in the untrust zone is enabled on FW1.
  3. Can packets enter the IPSec module for processing?

    Ensure that packets are sent to the IPSec module by referring to IPSec Working Principles.

  4. Ensure that the session corresponding to the IPSec tunnel is not aged on the NAT device.
  5. Ensure that IPSec packets are not denied by the carrier?
  6. Can packets reaching FW2 be forwarded from the untrust zone to the trust zone?

    Interzone packet filtering for 2.1.1.0/24 in the untrust zone to 10.2.1.0/24 in the trust zone is enabled on FW2.

  7. Ensure that IPSec packets are not discarded.

    When IPSec uses the SHA2-384 and SHA2-512 algorithms, IPSec packets will be discarded if the encryption and decryption methods on two ends are different. In this case, run the ipsec sha2 compatible enable command to change the SHA-2 encryption and decryption methods on two ends to be the same.

    However, when the FW is connected to an AR router and IPSec uses the SHA2-256 algorithm, IPSec packets will be discarded if the encryption and decryption methods on the two ends are different. In this case, run the ipsec authentication sha2 compatible enable command on the AR router to change the SHA-2 encryption and decryption methods on two ends to be the same.

  8. Can packets reaching FW2 be forwarded to PC2?

    FW2 must have a reachable route to PC2.

Poor IPSec Service Quality

In Figure 1-25, an IPSec tunnel is successfully established between FWs.

Figure 1-25 Point-to-point IPSec networking
Impact by CPU Performance
  • IPSec service quality deterioration caused by CPU resource consumption during packet fragmentation
    • By maximum transmission unit (MTU) value

      The MTU value supported by a link is determined by the smallest MTU value of an interface on this link, while the interface MTU value varies with the interface type. (For example, the default MTU value of an Ethernet interface is 1500 bytes.) If the size of a packet exceeds the interface MTU, the device fragments the encrypted packet before sending it. The receiver needs to reassemble all the fragments of an IP packet before decrypting it. Fragmentation and reassembly consume CPU resources.

      In the IPSec packet encapsulation process, the cost increases each time the IPSec module encapsulates the received original IP packets. The increased cost varies with the encapsulation protocol. For details, see Table1. Assume that the cost increases by 80 bytes after IPSec processing. Packets containing more than 1420 bytes will carry more than 1500 bytes after IPSec encapsulation; therefore, they need to be fragmented. When most packets in data flows are large packets exceeding 1420 bytes, a large number of CPU resources are consumed. This will lead to decrease of the IPSec service access speed and service quality deterioration.

      Table 1-5 Cost of bytes by protocol

      Protocol

      Increased Bytes

      ESP

      Default value: 56

      The increased cost of ESP depends on the encryption algorithm and whether an authentication algorithm is used.

      AH

      24

      GRE

      24

      NAT-T

      8

      L2TP

      12

      PPPoE

      8

      IPSec tunnel mode

      20

      TCP

      8

  • By Transmission Control Protocol maximum segment size (TCP MSS)

    If the total packet length (MSS plus various types of costs, including the TCP packet header, IP packet header, and IPSec packet header) exceeds the link MTU, data packets will be fragmented. Fragmentation will lead to consumption of more CPU resources. In addition, encryption and decryption of packet fragments also consume CPU resources of devices on the transmission link. Too much CPU resource consumption will lead to data packet loss.

    Some upper layer applications (such as the application layer protocol HTTP) will set the Don't Fragment (DF) flag of IP packets to valid to prevent TCP packet fragmentation. If the DF flag is set to valid but the interface MTU is smaller than the MSS, the device will discard packets because it cannot forcibly fragment the TCP packets.

    • IPSec service quality deterioration caused by CPU resource consumption by other features

      Features such as IPSec, content security, and attack defense consume large CPU resources. Therefore, if IPSec, content security, and other features are enabled simultaneously, the IPSec service access speed will become slow. The larger the data volume, the more obvious the problem.

Impact by Internet Quality

IPSec is used to secure communication between virtual networks over the Internet. The transmission quality of the Internet has a direct import on the IPSec service quality. You can deploy the VPN in bypass mode to determine whether the poor IPSec quality is caused by the poor transmission quality on the Internet. If the transmission quality on the Internet is poor, contact the carrier. The Internet may encounter the following problems:

  • The Internet encounters network congestion, leading to packet loss or network interruption. Packet loss has a huge impact on the quality of voice and video services, and obviously slows down web browsing and file transfer. Network interruption will directly lead to service interruption.
  • The carrier denies packets of specific types, for example, UDP packets. If so, IKE negotiation will fail in the NAT traversal scenario.
  • The carrier disables ports such as ports 500 and 4500 used by the IPSec service. If port 500 is disabled, IKE negotiation will fail. If port 4500 is disabled, IKE negotiation will fail in the NAT traversal scenario.
Download
Updated: 2019-06-03

Document ID: EDOC1100086052

Views: 614

Downloads: 25

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next