Customer uses URL filter for both http and https in his firewall USG6300 version V500R001C50, and he noticed that one website doesn’t show up completely, the site is messed up and scrambled in way that no contents show up, the site is News website and there is no filter under this category, the filter is under other categories.
No alarm reported for this case.
Again the site is News website, and there is no filter for this category, we can see some parts of the site appear, others parts are messed up, we tried to put this website in the white list to avoid filter and it doesn’t work also, but when we remove the filter of all categories the website come up and shows correctly, so we noticed the issue was on the URL filter, then we started filtering category by category until we noticed the Gambling category is affecting the website, once this category is enabled the website is messed up.
Customer doesn’t want this category to be accessible in his network and still wants to see the whole content of the News Website.
We started analyzing the packets between the end user and Web server and specially Get message which is used to extract the URL address in case of https website to compare it with the filter content and we noticed that the web server of this News website sends one gambling website as part of its content as advertisement.
Below is URL log from Firewall and we can see the firewall blocks this website.
Web server sends advertisement includes one website which part of filtered categories and firewall doesn’t allow this website from complete the loading.
The best solution to let website owner to stop sending this URL to the firewall during page loading, but as workaround solution we put this website in the white list of this filter and then the users are able to see this website News, regarding the gambling website in the white list we put its IP address in deny Security Policy to deny users from accessing it.
Customized URL filter allow the gambling website.
Deny Policy to deny gambling website.
The News website works good.
The Gambling website is not working due to Security policy filter.
Trace the packets and check URL log to see what type of websites that might be sent through the loading session and do check Get message which usually firewall uses to extract URL address for https.