Tuesday, February 11, 2014

Packet’s Don’t Lie

 

Doing a customer migration from one firewall to another firewall. Customer has web-server on inside LAN. Old firewall doing port-forwarding to inside server.

Had a small maintenance window.

So I setup 1:1 static NAT (entire IP protocol – no TCP/UDP filtering) on the new firewall.

Afterwards we change the Default-Gateway on the web-server and verify connections sourced from the Internet.

Untitled

Open Wireshark to see what in the heck is going on on…

web-server-problem

Packet #105, #106, and #107 are the TCP-3WAY Handshake. Because packet #106 was sent from x.x.x.x back to my laptop, this tells me 1) the firewall ACL rules I setup were correct and 2) the default-gateway setting on inside web-server was setup correct.


Notice packet #108. This is the HTTP GET request my laptop sends to x.x.x.x.

Notice that my laptop continues to retransmit this HTTP GET request... because x.x.x.x never sends a response.

Something wrong with the inside web-server. Firewall is fine.

After attempting unsuccessfully to make this work several times, we NATed with a different public IP and it started working fine. Weird. Several days later we switched out an HP Ethernet switch in between the firewall and the ISP. So we speculate that either ISP (Verizon) was somehow intercepting TCP port 80 traffic or there was some strange Layer3 problem on a Layer2 HP switch?!?

We could never get to the bottom of why but after changing the public IP for the port-forwarding, everything worked fine.