Firewall rejects SNMP GET request messages

Publication Date:  2016-07-31 Views:  300 Downloads:  0
Issue Description

Firewall rejects any SNMP GET request messages coming from NMS.

Customer set the Loopback interface as the source interface for traps and it works fine (SNMP 162 outbound). However, the firewall will reject every SNMP message (SNMP 161 inbound) sent by our NMSs. Here you can see the Loopback 0 configuration, the snmp-agent configuration and the security policy rule that controls the incoming packets towards the firewall, as well as the ACL and the IP address sets used:

interface LoopBack0
description INBAND_MGMT
alias INBAND_MGMT_29004_FW
ip address X.X.1.53 255.255.255.255
ospf cost 1
ospf network-type p2p


snmp-agent local-engineid 000007DB7F0000016FD75838
snmp-agent community read  xxxxxxxxxxx
snmp-agent community write  xxxxxxxxxxx
snmp-agent sys-info contact xxxxxxx
snmp-agent sys-info location xxxxxxxxxxxxx
snmp-agent sys-info version v3
snmp-agent group v3 v3group privacy  read-view v3view write-view v3view notify-view v3view acl  2020
snmp-agent target-host trap address udp-domain x.x.30.6 params securityname usg6650 v3  privacy
snmp-agent target-host trap address udp-domain x.x.30.6 params securityname usg6650 v3  privacy
snmp-agent target-host trap address udp-domain x.x.16.6 params securityname usg6650 v3  privacy
snmp-agent target-host trap address udp-domain x.x.6.6  params securityname usg6650 v3  privacy
snmp-agent mib-view included v3view iso
snmp-agent usm-user v3 usg6650 v3group  authentication-mode sha xxxxxxxxxx
snmp-agent trap enable ipsec
snmp-agent trap enable l2tp
snmp-agent trap enable configuration
snmp-agent trap enable system
snmp-agent trap enable standard
snmp-agent trap enable srm
snmp-agent trap source LoopBack0

acl number 2020
description SNMP_MGMT
rule 5 permit source xx.x.6.0 0.0.0.255
rule 10 permit source x.x.22.0 0.0.0.255
rule 15 permit source x.x.30.0 0.0.0.255
rule 100 deny

rule name "INBAND MGMT UNTRUST > LOCAL"
  description Inbound Inband MGMT from NMS 
  policy logging
  source-zone untrust
  destination-zone local
  source-address address-set INBAND_MGMT
  source-address address-set NMS_41
  action permit

[core-29004-n1-00-FW2]display ip address-set verbose INBAND_MGMT item
08:18:00  2016/06/02
Address-set: INBAND_MGMT
Type: object
Item number(s): 1
Reference number(s): 2
Item(s):
address 0 x.x.30.0 mask 24

[core-29004-n1-00-FW2]display ip address-set verbose MV_NMS_41  item
08:18:27  2016/06/02
Address-set: xMV_NMS_41
Type: group x
Item number(s): 2
Reference number(s): 2
Item(s):
address 0 address-set NMS1
address 1 address-set NMS2

[core-29004-n1-00-FW2]display ip address-set verbose NMS1 item
08:18:47  2016/06/02
Address-set: NMS1
Type: object
Item number(s): 1
Reference number(s): 1
Item(s):
address 0 x.x.6.16 mask 32
[core-29004-n1-00-FW2]display ip address-set verbose NMS2 item
08:18:51  2016/06/02
Address-set: NMS2
Type: object
Item number(s): 1
Reference number(s): 1
Item(s):
address 0 x.x.6.17 mask 32

Firewall version

display version
12:04:39  2016/06/03
Huawei Versatile Security Platform Software
Software Version: USG6600 V100R001C30  (VRP (R) Software, Version 5.30)
Copyright (C) 2013-2015 Huawei Technologies Co., Ltd..
USG6650 uptime is 4 weeks, 3 days, 1 hour, 11 minutes

Alarm Information
[root@nms1 ~]# snmpwalk -v3 -l authPriv -u usg6650 -x AES -X xxxxxx -a SHA -A xxxxxx  x.x.1.53:161 IF-MIB::ifAlias
^C
[root@nms1 ~]# snmpwalk -v3 -l authPriv -u usg6650 -x AES -X xxxxxx -a SHA -A xxxxxx x.x .1.53:161 IF-MIB::ifAlias
snmpwalk: Timeout

Handling Process

Firstly I will check if the connectivity between USG and NMS is achieve. Ping and SSH was permitted and from NMS we were able to PING and do SSH.

[root@nms1 ~]# ping x.x.1.53
PING x.x.1.53 (x.x.1.53) 56(84) bytes of data.
64 bytes from x.x.1.53: icmp_seq=1 ttl=252 time=6.40 ms
64 bytes from x.x.1.53: icmp_seq=2 ttl=252 time=6.27 ms

--- x.x.1.53 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4005ms
rtt min/avg/max/mdev = 5.939/6.173/6.409/0.204 ms
[root@nms1 ~]# telnet x.x.1.53 22
Trying x.x.1.53...
Connected to x.x.1.53.

Then I decided to perform some tests in our lab. For me, with current configuration from customer I wasn't able to make traps even work.

By checking further the SNMP configuration I manage to find the root-cause that is related to securityname value. Currently customer is using “usg6650” , this doesn't contain any extra characters and should be the same as the user. Into the product documentation it states clearly that security name must contain at least 3 types of characters. For "usg6650" we have only 2.


So I asked customer to adjust the configuration like below
 
snmp-agent group v3 v3group privacy  read-view v3view write-view v3view notify-view v3view
snmp-agent target-host trap address udp-domain x.16.30.6 params securityname Admin@123 v3  privacy
snmp-agent target-host trap address udp-domain x.18.30.6 params securityname Admin@123  v3  privacy
snmp-agent target-host trap address udp-domain x.31.16.6 params securityname Admin@123  v3  privacy
snmp-agent target-host trap address udp-domain x.31.6.6 params securityname Admin@123 v3  privacy
snmp-agent mib-view included v3view iso
snmp-agent usm-user v3 Admin@123 v3group  authentication-mode sha xxxxxxxxxxxxxxxxxx

But this is not all, customer need to adjust the interface that connects to NMS and add "service manage snmp permit" with other services like ping, ssh, https.

#
interface Vlanif1000
ip address x.x.1.1 255.255.255.252
ospf cost 1
ospf bfd enable
service-manage https permit
service-manage ping permit
service-manage ssh permit
service-manage snmp permit
#

With this last configuration, SNMP started to work as expected, but there is one more thing, customer spot it that actually no security policy is matched when SNMP is performing the query.

The explanation is related to service-manage snmp permit command which actually bypass the security policies in order to provide quick configuration for some specific services.

So customer disable "service-manage snmp permit" command in order to force firewall to process the SNMP packets through security policies. But this made SNMP service to fail again, we are back on square one.

I decided to perform firewall traffic statistic and this actually help us to find the root-cause.

1. Create an ACL to match the communication flow between NMS and USG.

#                                                                                                                                  
acl number 3999                                                                                                                    
rule 10 permit ip source x.x.30.6 0 destination x.x.1.53 0                                                                 
rule 20 permit ip source x.x.1.53 0 destination x.x.30.6 0                                                                 


2. Enable statistics on the firewall.

<HUAWEI>system-view                                                                                                                
14:26:41  2016/06/03                                                                                                               
Enter system view, return user view with Ctrl+Z.                                                                                   
[HUAWEI]diag                                                                                                                       
[HUAWEI]diagnose 
[HUAWEI-diagnose]firewall statistic acl 3999 enable

3. Try to walk the OID again.

4. Check the firewall statistics, copy the output and send it to me.
[HUAWEI-diagnose]display firewall statistic acl 

5. Close statistics.
[HUAWEI-diagnose]undo firewall statistic                                                                                           
14:29:58  2016/06/03                                                                                                               
Stop the ACL statistic 

Root Cause

The firewall statistics showed that the traffic was dismissed by the interface security manager (IF_SERVICE_MANAGER_FILTER)

Solution

According to the logs, snmpwalk operation was dismissed by the interface service manager. Last time customer only disable snmp service from the interface, but it looks like the firewall behavior is to filter SNMP completely, it will not send it to security policy processing.

In order to push snmp packets to be processed by the security policy you will need to disable service manage entirely. Check below.
 

<sysname> system-view
[sysname] interface GigabitEthernet 1/0/1
[sysname-GigabitEthernet1/0/1]undo service-manage enable


Discard detail information:
  IF_SERVICE_MANAGER_PACKET_FILTER:     6

Suggestions

"Service-manage enable" commands are only used to quickly configure, SSH, Telnet, HTTPS access to firewall in order to avoid configuring security policy, just for easy management. The recommendation is to control all traffic flows coming to the firewall through security policies which offer better granularity, control and security.

I hope you enjoy this case! please rated if you like it.

END