Finding vlan settings on HP Procurve switch

Finding what vlans are set on a switch port is a very needed thing for almost any config changes in Procurve software. This entry shows a quick way to check the vlans both tagged/untagged on a procurve. This works for all procurve I believe, but I am testing on a J9773A 2530 switch. This is a simple entry but might help someone out.

To show vlans associated with the ports the command “show vlan ports X” can be used, and to find out more info like tagged/untagged you can add the “detail” to the command to get more info. For example to get info for port 1

Show vlan ports 1

show-vlan

And more info:

show vlan ports 1 detail

show-vlan-detail

Upgrading HP J9299A HP 2520-24G VIA TFTP

The old Procurve switch line is very long in the tooth, but I run across them all the time. In this case its a 2520 switch that is in very bad need a of a firmware upgrade. I will detail where to go to get the firmware, and how to use TFTP to upload it. In my case, the web interface is having all kinds of java issues, and TFTP is just easier.

First lets get the software – it can be found at :

https://h10145.www1.hpe.com/support/SupportLookUp.aspx

Next, I download the firmware for the J9299A, and it seems the latest is from 2016. I download it, put it on my TFTP server which has full network connectivity to the switch.

SSH to the switch, and run these commands

If you get the error ” SFTP must be disabled before enabling tftp. ” you will need to run the “no ip ssh filetransfer” command first before enabling tftp client.

config t
no ip ssh filetransfer — This allows TFTP, if you enable this, and use SCP or SFTP then no need for TFTP
tftp client
exit
copy tftp flash 10.10.16.5 J_15_09_0028.swi primary

It will then write this to the primary flash. Next I will tell it to boot this firmware.

boot system flash primary  – Now it will reboot with the latest firmware
AH-POE-Top# show ver
Image stamp:
/ws/swbuildm/J_rel_hartford_qaoff/code/build/walle(J_rel_hartford_qaoff)
Aug 23 2016 08:57:14
J.15.09.0028
1791
Boot Image: Primary

 

 

 

Ruckus ICX VLAN range command

Thought this might be useful for anyone who needs to make a lot of changes quickly to many vlans. In ICX vlans, and vlan interfaces (routing interfaces) are different. In this example I will show how to quickly edit each vlan in a range and modify STP settings.

Sacred-CORE#show vlan brief

System-max vlan Params: Max(4095) Default(1024) Current(1024)
Default vlan Id :999
Total Number of Vlan Configured :29
VLANs Configured :2 32 50 to 51 128 132 136 138 140 142 144 146 150 152 154 156 160 162 164 220 224 240 250 255 324 332 350 500 999

As you can see we have a lot of Vlans, so lets modify the STP settings for all of them. I will just copy the vlans listed above and basically paste them in since I want to modify all vlans.

Sacred-CORE#config t
Sacred-CORE(config)#vlan 2 32 50 to 51 128 132 136 138 140 142 144 146 150 152 154 156 160 162 164 220 224 240 250 255 324 332 350 500
Sacred-CORE(config-mvlan-2*500)#spanning-tree 802-1w

Sacred-CORE(config-mvlan-2*500)#spanning-tree 802-1w priority 4096

Sacred-CORE(config-mvlan-2*500)#

Bam! now we have modifed the setting for all vlans. The syntax of the command is

vlan X X X X – so lets do vlan 2, 128 and 136

vlan 2 128 136   — thats it.

 

802.11 SSID Overhead and impact

When deploying wireless networks one of the best practices is to limit the SSIDs used within the WLAN. The reason we want to limit the amount of SSIDs, is the SSID announcement is sent through a Beacon frame. This frame is a broadcast to all stations. This happens every 100 MS or 102.4 MS according to how busy the medium is. In all documentation and great blogs I have not found a lot of visual examples of what actually happens, So I thought this might be a cool blog entry to write.

On top of the constant beacon broadcasting of the APs, when a client wants to join, it will send a probe request to the broadcast address of ff:ff:ff:ff:ff:ff and all APs will respond with a probe response frame directly to the station informing the client of the SSID and its supported capabilities. Beacons and probe request/response frames are the reason we need to limit the amount of SSIDs. For instance (and I will show this below) if you had 1 AP with 5 SSIDs, with 1 client trying to connect, the AP would be sending somewhere around 50 Beacons in 1 seconds, and 5 probe responses. This is might not seem like a lot of air time used at first, but think if you had 5 clients trying to connect at the same time, or maybe 2 or more APs on the same channel, with 5 clients trying to connect at the same time. Then the airtime usage goes up huge. Since wireless is half-duplex this takes increases the time it takes for clients to do what they need to do on the spectrum.

To show the results, I am using the Ekahua sidekick with spectrum analysis. I will start with 1 AP broadcasting 1 ssid, and then add another AP on the same channel. From there I will increase the SSIDs and show the airtime usage.  I hope to show why it is important to keep SSID overhead to a minimum. The gear used will be a Ruckus Zondirector, Ruckus 7982, and R710 APs. I am lucky enough to have an area with very little neighboring wifi networks, so I am testing with 1 wireless client in range of the AP to limit, and show the probe request/response traffic. I am broadcasting the test SSIDs on channel 11 and channel 157.

First lets look at just 1 AP with one SSID broadcasting.

As you can see from above, very little spectrum usage. Again, no clients are connected to an SSID, and only one device would be doing probe requests.

I did a packet capture next, check out the analysis data and the size of the capture. I only let the capture run for 1 minute (60 seconds). The size of the file was roughly 165 Kb. The red in the graph below represents beacon frames, and the dots/green are probe requests vs probe responses.

1-SSID-1-AP-wire

I then added the other AP on the exact same channels, still only broadcasting 1 SSID. Spectrum usage definitely increased but only by very little. Below are the spectrum density graphs and Wireshark IO graph.

 

2aps-1-ssid

Now, lets increase things a bit – I will start broadcasting 4 SSIDs from 1 AP.

Spectrum usage from 1 AP broadcasting 4 SSIDs:

Below is 2 APs broadcasting 4 SSIDs – all on the same channels.

When comparing the wireshark IO graph we can really see how many Beacon frames are sent and compare those between 1 and 2 APs. Another cool thing to note is the size of the wireless PCAPs – each running for 1 minute, the 1 AP broadcasting 4 SSIDs size was 662 KB, and 2 APs broadcasting 4 SSIDs each was just about double at 1200 KB. In the graphs below, red indicates the number of Beacon frames.

4-SSID-1-AP-wire

2 APs – 4 SSIDs:

4-ssid-2-ap

 

After all the testing, we have a big increase from just adding 1 SSID and an even bigger increase from having another AP broadcasting on the same channels. This isn’t ground breaking but cool to see in action. I have seen many many clients have at least 6 SSIDs broadcasting, and this give some good illustration on why we need to reduce the amount of SSIDs in use. There are a few tools out to calculate the SSID overhead – tools such as http://www.revolutionwifi.net.

Check out the below picture comparing 1 SSID to 4 SSIDs just from 1 AP for 2.4 and 5 gig.

 

.

 

 

College dorm Wi-Fi issues

Recently I was asked to look at a local college that is experiencing issues in the evening with wireless users. In this example the complaints from students who live in the dorm are obscure and lacking details. They range from “Wifi goes out” to “Video’s buffer”, and “Its just so slooooow”. So, with this little information known we decided to go over and check out the spectrum to see what was going on.

First I talked with the admins to find out about the building, construction, Bandwidth/networking and how students use the dorm wireless. From our conversation I found that the dorm has dedicated internet of around 400 MBits and monitoring is show it busy but not maxed out. Students are working throughout the day in their dorms, and complaints pretty much only come in at night. Construction is new, but lots of brick – signal should not propagate to far (this is important for later).  The building which is 5 floors, stands by itself with little sources of wifi interference outside of internal APs.

We then went on site and did a passive survey with spectrum analysis to check coverage, interference,  and if the spectrum was busy.  Wifi coverage with was great, lots of APs to cover needed space, maybe too many APs (we will get to that later). When conducting the spectrum analysis —- WHOA! its not busy, but look at the signals that are seen. Below shows an screen shot of the spectrum using Ekahau.

eevator 3

My first thought was that’s a lot of APs! I looked at our layout and we only had about 6 APs in this area, but I see many more than that. So the idea that the building materials (brick) would attenuate the enough that we would not see APs on other floors was incorrect. This gives a lot of credit to on site surveys before installing APs – building materials might be different and react different then we think sometimes.   I was seeing APs 2 floors above with usable signal. Another thing to notice that there is plenty of open spectrum in the 5 gig range, why are all the channels being used close to each other and at the beginning of the spectrum? At this client Ruckus wireless is being used and after checking documentation and asking vendor reps, seems like it might be a firmware thing. Also notice that we have a lot of APs on the same channel that can hear each other, this would cause contention on the channel.

This shows the importance of always doing a site survey and checking spectrum to see what AP coverage is.

To help resolve the issues we created a channel plan to statically set channels and spread them throughout the spectrum, also lowering channel width to 20 MHZ since we have so much AP coverage between floors. By doing these two things we were able to decrease channel contention and give our users a better wireless experience.

One additional setting we modified was to block broadcast/multicast from clients. When doing a packet capture on the spectrum we also noticed that around 30 percent of the traffic was just MDNS. We created ACLs to block this traffic.

 

 

Cisco – Event Management to enable backup interface

I work with a lot of sites that are multi-homed or have a backup connection that, even though its backup still needs to be used at the same time for load balancing.  Recently I was able to have a site that had a very slow MPLS link that was being moved to only backup – and a new SD-WAN link has been installed as primary.  The MPLS link will go away after were sure the SD-WAN option works well.

The only time we want this link to become active is if SD-WAN fails. My first thought is no problem, just  use a routing protocol and a static route – the routing protocol will be our preferred method, and we can just raise the admin distance of the default route to make it a backup. One problem in this scenario, the MPLS uses dynamic routing (EIGRP), and many other locations still use the MPLS for primary, or secondary connections, so we cannot even have this link up (admin state) or the MPLS will advertise the network.

So, to recap what I need is a preferred route to our SD-WAN, and if SD-WAN fails, bring up the backup connection admin state and move traffic to it. Then of course, auto fix everything if SD-WAN comes back online. No problem! Cisco’s Event Manager to the rescue. Cisco’s Embedded Event Manager (EEM) (Copied From Cisco) –is a distributed and customized approach to event detection and recovery offered directly in a Cisco IOS device. EEM offers the ability to monitor events and take informational, corrective, or any desired EEM action when the monitored events occur or when a threshold is reached. An EEM policy is an entity that defines an event and the actions to be taken when that event occurs. When creating EEM scripts, you have two options TCL or CLI – in this case I am using just CLI.

Using EEM on the 3850 core I was able to detect if the default route learned VIA OSPF was removed from the routing table – if that happened, then run the command “No shut” on my MPLS uplink and EIGRP would take over. If the default route was learned at some point through OSPF when things were corrected – then run the command “Shut” on my MPLS uplink. This worked extremely well in combination with Link detection on the Fortigate, and OSPF default route distribution.

The Fortigate is my SD-WAN device, and my default gateway for the network.  I am using link detection to test HTTP access to google. If my WAN interface cannot get a response from google.com in my set time (5 attempts, with 5 seconds between each attempt) then it will remove the default route from my routing table. When this happens the route will be removed from redistribution in OSPF, and removed from the Cisco core. EEM sees this event, and does my list of commands. Below shows the layout, and code to get this going. Interface 0/24 is my MPLS uplink.

Layout2

First I made sure that the MPLS interface was  shutdown and OSPF was up, and I was receiving the redistributed default route from the Fortigate.

config t

event manager applet MPLS-UP
event routing network 0.0.0.0/0 type remove protocol OSPF
action 1 cli command “enable”
action 2 cli command “config t”
action 3 cli command “int gig 1/0/24”
action 4 cli command “no shut”
action 5 cli command “exit”

event manager applet MPLS-DOWN
event routing network 0.0.0.0/0 type add protocol OSPF
action 1 cli command “enable”
action 2 cli command “config t”
action 3 cli command “int gig 1/0/24”
action 4 cli command “shut”
action 5 cli command “exit”

You can check status and history of events by using the show event manager commands.

show

During a failure of my ISP, everything worked great. The default route was removed from OSPF, which caused an event that EEM matched – then it enabled the MPLS interface, and all routes/default was learned VIA EIGRP and the MPLS. When internet was restored, the MPLS interface was shutdown, and all traffic started flowing over SD-WAN.

 

Cisco – Flex links

Cisco Flex links give the ability to have a layer2 redundant connection, or pair of connections configured as an Etherchannel for a primary link. This is an active/passive setup where if the primary connection’s link status goes down, the Flex link will become active, and if the primary comes back it will go into a standby mode and not take back primary functionality unless told to do so with preemption commands. STP is disabled automatically on Flex Links so no need to bother with Portfast.

Flex links started in code a long while back, and not sure how I missed them. I have needed this functionality before if I was connecting to a backup Firewall, or some device that Spanning-tree would have issues with. This options gives a great way to have a backup link configured if you just need it to become active if something happens to the primary link. In the below scenario I have a Fortigate firewall and Cisco 3560 switch. Port FA 0/1 is my primary and goes to Port 1 of the FGT, and Port 12 is the backup port for this link.

Layout

image

Switch config:

interface FastEthernet0/1
description “Connected to Fortigate”
switchport trunk encapsulation dot1q
switchport mode trunk
switchport backup interface Fa0/12
spanning-tree portfast

interface FastEthernet0/12
description “Connected to Fortigate – Backup”
switchport trunk encapsulation dot1q
switchport mode trunk

Below shows that status of the Flex link

status-primary

Notice that the primary state is active and up. Now, I will cause a physical port state change by unplugging the interface and see how many pings/time it takes to failover.

After unplugging the primary connection, the link light of port 12 instantly came on and went green, I didn’t even lose a ping to my switch. All mac/uplinks moved over the backup port but no loss. Below shows the status after. Notice that the backup state is up, not the primary. The primary port after plugging it back up is amber like a blocked port, and both interfaces port status show up, even though the primary port at this point is not forwarding traffic.

status

 

 

 

Protected: Upgrading firmware on Dell 4128-ON

This content is password protected. To view it please enter your password below:

Fortinac PXE DHCP boot options

Fortinac is built on top of CentOS and is a great product. Recently I needed to have default or isolated vlan support PXE booting as well as isolation. This way if a computer is being imaged we don’t have to worry about hard coding ports with vlans, etc. This is important because the NAC cannot look at the client prior to the OS install.

These settings were added to the dhcpd.conf – they would work for any implementation running dhcpd not just Fortinac.

Below is the conf that works.

# Sample /etc/dhcpd.conf

authoritative;
log-facility local6;
ddns-update-style none;
allow bootp;
allow booting;
class “authenticated_clients”
{
match pick-first-value (option dhcp-client-identifier, hardware);
}
# Isolation Scope ISOL_Isolation_blackhole
subnet 172.16.172.0 netmask 255.255.252.0 {
range 172.16.172.10 172.16.175.200;
default-lease-time 28800;
max-lease-time 86400;
option domain-name “blackhole.local”;
option domain-name-servers 172.16.172.254;
option broadcast-address 172.16.175.255;
option routers 172.16.172.1;
next-server PXE-SERVER-IP;
filename “SMSBoot\\x64\\file1.efi”; — NOTICE the two slashes that represent the file path – I was missing this, and of course it could not find the file.

The options for next-server and file-name were the options needed to push PXE settings over.

Restart the service after saving the configuration to dhcpd.conf

802.11 Finding the MCS rate in Wireshark

One great way to troubleshoot wireless issues that clients might be having is to perform a packet capture or use some kind of wireless analysis tool. In this case I have an Ekahau sidekick with Capture. Before this tool I used Ubuntu, Wireshark, and a compatible wireless adapter to capture packets. The MCS or Modulation and Coding Scheme value indicates a lot about the client. For example, the MCS rate is as really a reference number given to the combination of  – amount of Spatial streams, Modulation type and coding scheme. Analyzing the MCS helps when we need to troubleshoot the common problem of “its slow” while everything looks good.

I look at the MCS rate when troubleshooting client problems for a lot of reasons. Since the MCS rate is a reference number describing how much data can be sent over the wireless medium, it gives us an idea of the wireless environment. If the spectrum has lots of contention/interference or the client has low RSSI then the MCS will reflect this. If clients have lots of retrys, or a low RSSI then it will lower the modulation scheme, and therefore lower the MCS rate.

Lets take a look at a 802.11ac packet in Wireshark and break down what each section means. The MCS is located in the Data frame under the 802.11 Radio information section.

packet1

This packet is captured from a Data frame, you will notice the MCS rates are only captured on data packets – Lots of reasons for this. Most management/control frames are sent with the lowest data rate possible to allow them to be understood by clients far away, or if the network is very congested.

In this example we are looking at an 802.11ac frame, and we see right away the MCS index is 7  – from looking at the MCS chart we see that means modulation:64 QAM, which a number 5/6 . The 5/6 is the actual FEC or Forward error correction coding rate, which is given in fraction form. 5/6 equals to 0.83333.. which means that 83% of the data stream can be used to send real data. 5/6 is the highest coding rate.

We can see that the host has 2 Spatial streams, is using a Short guard interval , 40 MHz channel and the data rate is 300 – the data rate is  the highest capable with these settings. Notice the RSSI is -44 and we are just using 64 QAM, you really have to be close and have a clear spectrum to achieve 256 QAM.

Below  is the table of MCS rates for 802.11AC/N – Compliments of WLANPROS.com

802.11n-and-802.11ac-MCS-Chart-2019-1536x1187