Overtime we forget things, especially Shared secret radius keys. This is pretty common, and I run into it a lot. For example – lets say a you setup NPS (Network Policy Server) and a Wireless controller for 802.1x auth, or a ASA doing radius authentication years ago. Some how or another that key was lost – no worries, you can get that back from the NPS server itself.
In just a few simple steps you can get that key back. So lets start by opening up NPS and then selecting “Radius Clients and Servers” and dropping down “Radius Clients”
In this example I am using a Ruckus Smartzone – lets say I forget the password. I can just right click on the client and select “Save and apply as Template.
Next we can create a new radius client by right clicking on “Radius Clients” and once the client info pops up to fill in, we will select to create it from the template, and select the template we made.
To see the *** Password, uncheck the box “Select and existing template” and then select the “Generate” Radio button – and bam! there is the PSK.
This entry details how to create a static route in both the GUI and CLI of the Fortigate firewall. Specifically I am using FortiOS 6.2.4 but its pretty much been the same for years.
Lets start by talking through the things that will be needed to create the static route.
– Subnet – this is what we want to route to, for a default route its 0.0.0.0/0 but if we wanted a more specific route, lets say to 192.168.100.0/24.
– Destination Interface – Next hop interface we want to send traffic out of.
– Gateway address – Directly connected interface neighbor that we want the next hop for 192.168.100.0/24 to be.
– Administrative Distance– is a feature used by routers to select the best path to a destination when multiple paths to the same destination are present. Lowest AD wins and will be placed in the routing table.
Advanced option – Priority – To build on AD definition – What if two routes exist in the routing table to the same destination with the same AD? This is where Priority comes in. Lowest priority wins. By selecting a priority you can have multiple routes to the same destination in the routing table, but one would be preferred over the other. This comes in very hand for Reverse Path forwarding issues.
So after all that’s said, we need to route 192.168.100.0/24 to our LAN interface with a next hop of 192.168.1.2.
First lets create this in the GUI. Navigate to network – static routes – and create a new one.
Now we will just insert the needed info. I am leaving the AD at 10 – which is default.
Press OK – and Bam! route created. We can check that the route has been created and is the routing table by going to monitor – routing monitor.
Next lets do the same thing in CLI.
First route creation. When you create the route edit the next available sequence number. In this case its 46.
You can see if your route is in the routing table in CLI by running the command “get router info routing-table all” but in this case I am using the static option, and grepping just what I need to see.
Finding what vlans are set on a switch port is a very needed thing for almost any config changes in Procurve software. This entry shows a quick way to check the vlans both tagged/untagged on a procurve. This works for all procurve I believe, but I am testing on a J9773A 2530 switch. This is a simple entry but might help someone out.
To show vlans associated with the ports the command “show vlan ports X” can be used, and to find out more info like tagged/untagged you can add the “detail” to the command to get more info. For example to get info for port 1
The old Procurve switch line is very long in the tooth, but I run across them all the time. In this case its a 2520 switch that is in very bad need a of a firmware upgrade. I will detail where to go to get the firmware, and how to use TFTP to upload it. In my case, the web interface is having all kinds of java issues, and TFTP is just easier.
First lets get the software – it can be found at :
Next, I download the firmware for the J9299A, and it seems the latest is from 2016. I download it, put it on my TFTP server which has full network connectivity to the switch.
SSH to the switch, and run these commands
If you get the error ” SFTP must be disabled before enabling tftp. ” you will need to run the “no ip ssh filetransfer” command first before enabling tftp client.
config t no ip ssh filetransfer —This allows TFTP, if you enable this, and use SCP or SFTP then no need for TFTP tftp client exit copy tftp flash 10.10.16.5 J_15_09_0028.swi primary
It will then write this to the primary flash. Next I will tell it to boot this firmware.
boot system flash primary – Now it will reboot with the latest firmware AH-POE-Top# show ver Image stamp: /ws/swbuildm/J_rel_hartford_qaoff/code/build/walle(J_rel_hartford_qaoff) Aug 23 2016 08:57:14 J.15.09.0028 1791 Boot Image: Primary
Thought this might be useful for anyone who needs to make a lot of changes quickly to many vlans. In ICX vlans, and vlan interfaces (routing interfaces) are different. In this example I will show how to quickly edit each vlan in a range and modify STP settings.
Sacred-CORE#show vlan brief
System-max vlan Params: Max(4095) Default(1024) Current(1024) Default vlan Id :999 Total Number of Vlan Configured :29 VLANs Configured :2 32 50 to 51 128 132 136 138 140 142 144 146 150 152 154 156 160 162 164 220 224 240 250 255 324 332 350 500 999
As you can see we have a lot of Vlans, so lets modify the STP settings for all of them. I will just copy the vlans listed above and basically paste them in since I want to modify all vlans.
When deploying wireless networks one of the best practices is to limit the SSIDs used within the WLAN. The reason we want to limit the amount of SSIDs, is the SSID announcement is sent through a Beacon frame. This frame is a broadcast to all stations. This happens every 100 MS or 102.4 MS according to how busy the medium is. In all documentation and great blogs I have not found a lot of visual examples of what actually happens, So I thought this might be a cool blog entry to write.
On top of the constant beacon broadcasting of the APs, when a client wants to join, it will send a probe request to the broadcast address of ff:ff:ff:ff:ff:ff and all APs will respond with a probe response frame directly to the station informing the client of the SSID and its supported capabilities. Beacons and probe request/response frames are the reason we need to limit the amount of SSIDs. For instance (and I will show this below) if you had 1 AP with 5 SSIDs, with 1 client trying to connect, the AP would be sending somewhere around 50 Beacons in 1 seconds, and 5 probe responses. This is might not seem like a lot of air time used at first, but think if you had 5 clients trying to connect at the same time, or maybe 2 or more APs on the same channel, with 5 clients trying to connect at the same time. Then the airtime usage goes up huge. Since wireless is half-duplex this takes increases the time it takes for clients to do what they need to do on the spectrum.
To show the results, I am using the Ekahua sidekick with spectrum analysis. I will start with 1 AP broadcasting 1 ssid, and then add another AP on the same channel. From there I will increase the SSIDs and show the airtime usage. I hope to show why it is important to keep SSID overhead to a minimum. The gear used will be a Ruckus Zondirector, Ruckus 7982, and R710 APs. I am lucky enough to have an area with very little neighboring wifi networks, so I am testing with 1 wireless client in range of the AP to limit, and show the probe request/response traffic. I am broadcasting the test SSIDs on channel 11 and channel 157.
First lets look at just 1 AP with one SSID broadcasting.
As you can see from above, very little spectrum usage. Again, no clients are connected to an SSID, and only one device would be doing probe requests.
I did a packet capture next, check out the analysis data and the size of the capture. I only let the capture run for 1 minute (60 seconds). The size of the file was roughly 165 Kb. The red in the graph below represents beacon frames, and the dots/green are probe requests vs probe responses.
I then added the other AP on the exact same channels, still only broadcasting 1 SSID. Spectrum usage definitely increased but only by very little. Below are the spectrum density graphs and Wireshark IO graph.
Now, lets increase things a bit – I will start broadcasting 4 SSIDs from 1 AP.
Spectrum usage from 1 AP broadcasting 4 SSIDs:
Below is 2 APs broadcasting 4 SSIDs – all on the same channels.
When comparing the wireshark IO graph we can really see how many Beacon frames are sent and compare those between 1 and 2 APs. Another cool thing to note is the size of the wireless PCAPs – each running for 1 minute, the 1 AP broadcasting 4 SSIDs size was 662 KB, and 2 APs broadcasting 4 SSIDs each was just about double at 1200 KB. In the graphs below, red indicates the number of Beacon frames.
2 APs – 4 SSIDs:
After all the testing, we have a big increase from just adding 1 SSID and an even bigger increase from having another AP broadcasting on the same channels. This isn’t ground breaking but cool to see in action. I have seen many many clients have at least 6 SSIDs broadcasting, and this give some good illustration on why we need to reduce the amount of SSIDs in use. There are a few tools out to calculate the SSID overhead – tools such as http://www.revolutionwifi.net.
Check out the below picture comparing 1 SSID to 4 SSIDs just from 1 AP for 2.4 and 5 gig.
Recently I was asked to look at a local college that is experiencing issues in the evening with wireless users. In this example the complaints from students who live in the dorm are obscure and lacking details. They range from “Wifi goes out” to “Video’s buffer”, and “Its just so slooooow”. So, with this little information known we decided to go over and check out the spectrum to see what was going on.
First I talked with the admins to find out about the building, construction, Bandwidth/networking and how students use the dorm wireless. From our conversation I found that the dorm has dedicated internet of around 400 MBits and monitoring is show it busy but not maxed out. Students are working throughout the day in their dorms, and complaints pretty much only come in at night. Construction is new, but lots of brick – signal should not propagate to far (this is important for later). The building which is 5 floors, stands by itself with little sources of wifi interference outside of internal APs.
We then went on site and did a passive survey with spectrum analysis to check coverage, interference, and if the spectrum was busy. Wifi coverage with was great, lots of APs to cover needed space, maybe too many APs (we will get to that later). When conducting the spectrum analysis —- WHOA! its not busy, but look at the signals that are seen. Below shows an screen shot of the spectrum using Ekahau.
My first thought was that’s a lot of APs! I looked at our layout and we only had about 6 APs in this area, but I see many more than that. So the idea that the building materials (brick) would attenuate the enough that we would not see APs on other floors was incorrect. This gives a lot of credit to on site surveys before installing APs – building materials might be different and react different then we think sometimes. I was seeing APs 2 floors above with usable signal. Another thing to notice that there is plenty of open spectrum in the 5 gig range, why are all the channels being used close to each other and at the beginning of the spectrum? At this client Ruckus wireless is being used and after checking documentation and asking vendor reps, seems like it might be a firmware thing. Also notice that we have a lot of APs on the same channel that can hear each other, this would cause contention on the channel.
This shows the importance of always doing a site survey and checking spectrum to see what AP coverage is.
To help resolve the issues we created a channel plan to statically set channels and spread them throughout the spectrum, also lowering channel width to 20 MHZ since we have so much AP coverage between floors. By doing these two things we were able to decrease channel contention and give our users a better wireless experience.
One additional setting we modified was to block broadcast/multicast from clients. When doing a packet capture on the spectrum we also noticed that around 30 percent of the traffic was just MDNS. We created ACLs to block this traffic.
I work with a lot of sites that are multi-homed or have a backup connection that, even though its backup still needs to be used at the same time for load balancing. Recently I was able to have a site that had a very slow MPLS link that was being moved to only backup – and a new SD-WAN link has been installed as primary. The MPLS link will go away after were sure the SD-WAN option works well.
The only time we want this link to become active is if SD-WAN fails. My first thought is no problem, just use a routing protocol and a static route – the routing protocol will be our preferred method, and we can just raise the admin distance of the default route to make it a backup. One problem in this scenario, the MPLS uses dynamic routing (EIGRP), and many other locations still use the MPLS for primary, or secondary connections, so we cannot even have this link up (admin state) or the MPLS will advertise the network.
So, to recap what I need is a preferred route to our SD-WAN, and if SD-WAN fails, bring up the backup connection admin state and move traffic to it. Then of course, auto fix everything if SD-WAN comes back online. No problem! Cisco’s Event Manager to the rescue. Cisco’s Embedded Event Manager (EEM) (Copied From Cisco) –is a distributed and customized approach to event detection and recovery offered directly in a Cisco IOS device. EEM offers the ability to monitor events and take informational, corrective, or any desired EEM action when the monitored events occur or when a threshold is reached. An EEM policy is an entity that defines an event and the actions to be taken when that event occurs. When creating EEM scripts, you have two options TCL or CLI – in this case I am using just CLI.
Using EEM on the 3850 core I was able to detect if the default route learned VIA OSPF was removed from the routing table – if that happened, then run the command “No shut” on my MPLS uplink and EIGRP would take over. If the default route was learned at some point through OSPF when things were corrected – then run the command “Shut” on my MPLS uplink. This worked extremely well in combination with Link detection on the Fortigate, and OSPF default route distribution.
The Fortigate is my SD-WAN device, and my default gateway for the network. I am using link detection to test HTTP access to google. If my WAN interface cannot get a response from google.com in my set time (5 attempts, with 5 seconds between each attempt) then it will remove the default route from my routing table. When this happens the route will be removed from redistribution in OSPF, and removed from the Cisco core. EEM sees this event, and does my list of commands. Below shows the layout, and code to get this going. Interface 0/24 is my MPLS uplink.
First I made sure that the MPLS interface was shutdown and OSPF was up, and I was receiving the redistributed default route from the Fortigate.
You can check status and history of events by using the show event manager commands.
During a failure of my ISP, everything worked great. The default route was removed from OSPF, which caused an event that EEM matched – then it enabled the MPLS interface, and all routes/default was learned VIA EIGRP and the MPLS. When internet was restored, the MPLS interface was shutdown, and all traffic started flowing over SD-WAN.
Cisco Flex links give the ability to have a layer2 redundant connection, or pair of connections configured as an Etherchannel for a primary link. This is an active/passive setup where if the primary connection’s link status goes down, the Flex link will become active, and if the primary comes back it will go into a standby mode and not take back primary functionality unless told to do so with preemption commands. STP is disabled automatically on Flex Links so no need to bother with Portfast.
Flex links started in code a long while back, and not sure how I missed them. I have needed this functionality before if I was connecting to a backup Firewall, or some device that Spanning-tree would have issues with. This options gives a great way to have a backup link configured if you just need it to become active if something happens to the primary link. In the below scenario I have a Fortigate firewall and Cisco 3560 switch. Port FA 0/1 is my primary and goes to Port 1 of the FGT, and Port 12 is the backup port for this link.
Notice that the primary state is active and up. Now, I will cause a physical port state change by unplugging the interface and see how many pings/time it takes to failover.
After unplugging the primary connection, the link light of port 12 instantly came on and went green, I didn’t even lose a ping to my switch. All mac/uplinks moved over the backup port but no loss. Below shows the status after. Notice that the backup state is up, not the primary. The primary port after plugging it back up is amber like a blocked port, and both interfaces port status show up, even though the primary port at this point is not forwarding traffic.