Friday, October 23, 2015

Ubiquiti Networks Outdoor P2P AP

 

Customer has requirement to establish a connection from MDF to new office on same property but about 100 yards away. What to do? Multi-mode fiber cost prohibitive.

Decided to try Ubiquiti Networks outdoor APs. Was very impressed. Implemented two of the inexpensive ($150 per unit) UniFi AP Outdoor+

Ubiquiti_UniFi_Outdoor_AP__UAP-Outdoor__802.11n_MiMo_main_product

Setup at first was a little tricky… before physically mounting on roof-top with line-of-sight between two points, need to first stage configuration by plugging both units into an Ethernet switch with Internet access. Download Ubiquiti’s ‘Unfi’ software which detects devices (the devices will acquire a DHCP address if a server exists but the Unifi software seems to detect them over a multicast address). Once both devices have been provisioned as independent APs, put a label on the far side AP, and unplug it from the Ethernet LAN. The Unifi software detects the disconnect and then you can configure the disconnected AP in a P2P topology (what Ubiquiti calls an (‘uplink’). After uplinking, the AP will operate in both modes concurrently… as an AP for nearby WiFi clients as well as a P2P endpoint with the other Unifi Outdoor+.

Here’s a very helpful video for uplink configuration… https://www.youtube.com/watch?v=16r9CoTdRqo

Also, knowing what LED status means comes in very handy as these guys can think slowly leading to confusion…

image

Thursday, September 17, 2015

Practical Use for Disabling Layer2 Keepalives…

 

Retail store has no Ethernet device plugged into LAN interface of MPSL router. How to test from Corp Core Switch to verify connectivity to store?

KEEP-ALIVES-ENABLED-DISABLED

BGP Fail-Over!!

 

Interesting observation… retail store has a main MPLS connection and a backup connection over the Internet. When the main connection goes down, the CE router immediately injects floating-static route with AD of 21 into RIB and re-routes traffic to neighboring Cisco ASA Firewall programmed for IPSEC backup to Service-Provider.

However, there is packet-loss when the CE router goes down… for (3) minutes…

A

B

Monday, August 24, 2015

MPLS Service Provider Migration

Working on a large (40 site) MPLS migration from Verizon to Windstream.

First couple sites were a little bumpy but I think we got a good design to roll-out now…

image

Tuesday, August 11, 2015

High-Availability Baby!!

 

Just implemented sweet HA solution for customer. Dual Cisco ASA 5525-X is Active/Standby configuration… Core Switch is dual 3850’s in Stackwise configuration… SQL environment is iSCSI on dual 3850’s in StackWise configuration as well…

 

image

Below is the concept…

image

Tuesday, April 28, 2015

Cloud App Performance Problem

 

So I get a call from a customer complaining that their cloud-based production application is slooooow.

image

Customer believes the problem is “the network”.

So I load up Wireshark on the customer’s PC and ask the them to open the application and login.

image

After typing in their username and password, the computer sat there thinking for almost (3) minutes.

Afterwards, I analyzed the Wireshark trace.

Sitelink-Slow-Logon-Analysis

I noticed…

Packet#66: (100) seconds after the login process began the customer’s PC ends conversation (FIN,ACK) (BAD - WHY DOES CLIENT SOFTWARE WAIT 100 SECONDS INTO LOGON AND THEN DECIDE TO END CONVERSATION?)

Packet#69: another (50) seconds later customer PC decides to initiate another conversation (TCP SYN) (BAD - WHY DOES CLIENT SOFTWARE WAIT ANOTHER 50 SECONDS AND THEN BEGIN A NEW CONVERSATION?)

All the while what the customer saw on their screen was the logon dialog-box sitting there thinking and then they received a message stating they were successfully logged in.

Root cause of the issue is clearly NOT the network but rather slow PC Client Software.

Opened up a trouble-ticket with the cloud-app service provider. We’re waiting to hear back.

Tuesday, April 14, 2015

Interesting Cisco ASA Firewall HairPin Issue

 

So I setup hair-pinning for customer.Need to hit public IP of inside server sourced from the inside. I then verify with a PING (public IP’s obfuscated)…

1

Looks good to me… So I open up a web-browser to browse to the server and it fails…

2

Hmmm… how is THAT possible?!?

So I load up Wireshark and to see what is happening on the wire. And the first thing I notice is that Wireshark sees the ECHO Requests and corresponding Replies as different conversations…

Independent ECHO Requests:

3

Independent ECHO Replies:

4

The above screen-shots show (2) different conversations. It is not supposed to look like that. It is supposed to look they are a part of the same conversations as seen below…

Inter-dependent ECHO Replies / Requests:

5

And then it hit me… how is it possible that the computer is displaying at the command-line successful replies when the corresponding ECHO Reply is being sourced from an IP that was not the original IP destined for in the PING Request?!?!

This is why… ICMP ECHO Requests ARE NOT “connection-oriented” like TCP packets. ICMP does not require acknowledgements. So basically the computer at the command line will show ECHO Replies all day long even if there is not an ECHO Request.

So then I realized that the Cisco ASA does not do an ICMP Redirect (which a router does do). In other words, I assumed because I was getting replies

that what was happening was the ASA was saying to the computer… “Yo… wait a minute… I’m going to re-direct you from the public IP you’re trying to talk to to the private IP you really need to talk to” and that is why successful ECHO Replies.

THIS IS NOT THE CASE. The reason why ECHO Replies is because ICMP is not a “connection-oriented” protocol like TCP and it will receive ECHO Replies all day long without corresponding ECHO Requests.

The solution to the problem was to delete a static route on the servers default-gateway (which was NOT the Firewall) because it was circumventing the Firewall and routing traffic to the client an alternate path.

Windows 7/8 Personal Firewall

 

Don’t forget that Windows Personal Firewall can cause false-negatives.

Today needed to verify some hair-pin NAT configuration on a Cisco ASA 5515-X.

1

Hmm… should be getting a reply on that last hop.

On older versions of Windows, even if Personal Firewall was turned off so long as the ping originated on the Windows PC the Personal Firewall would permit the echo-reply but this is not the case with new versions of Windows.

So, let’s turn off Personal Firewall on my Windows 8 laptop…

2

… now let’s re-verify our trace…

3

That looks better!!

Wednesday, January 7, 2015

VoIP Tagged DHCP Broadcast Madness

 

OK so here’s a really weird one…

Got a customer with a Star-2-Start VOIP system.

images

Customer hires me to replace existing Netgear switching with new Cisco Small Business switching.

download

So I install the new switch and set all switchports untagged for the data VLAN and tagged for the voice VLAN (Star-2-Star wants to always use VLAN #41).

All the phones come up just fine.

All the Grandstream ATA’s (GXW4004/8) aren’t working

 

download (1)

The Grandstreams (configured for 802.1q tagging in VLAN 41) are acquiring IPs from the customers Microsoft DHCP server!

The phones (also configured for 802.1q tagging in VLAN 41) are correctly acquiring IPS from the customers Star-2-Star server!

The fix was to HTTP in to the Cisco switch and “exclude” VLAN 41 from being tagged on the switchport the Microsoft DHCP server was plugged into!

Below is the definition right out of the Cisco switch admin guide:

Excluded—The interface is currently not a member of the VLAN. This is the default for all the ports and LAGs. The port can join the VLAN through GVRP registration.

Bizzare that phones tagged DHCP requests successfully hit the Star-2-Star server but the Grandstreams not.