Dell S4128F-ON port issues

Recently have been working with the S4128 switches. These have been great, and the price point is amazing.

The device comes with 2 ports that can be 10/40/or 100 Gig interfaces (given media). I needed to connect the port via a 40 gig DAC to a Dell server. When I plugged this in, a “show interface eth 1/1/26” would show the interface up, show the DAC model number and then would say “Protocol down”. I thought at first this could be mismatching vlans, etc. But after a few minutes found it had to be a negotiation issue.

After some checking I found this in the config:

interface breakout 1/1/25 map 100g-1x
interface breakout 1/1/26 map 100g-1x

Interface 1/1/26 is my connection to the server. After some digging with Dell we have to modify this. I ran the following command:

interface breakout 1/1/26 map 40g-1x

After running that command a sub-interface showed up. A “Show interface status” presents the following Eth 1/1/26:1. After configuring the sub interface as needed all seemed to work great.

I will post images of interface status later.

Advertisements

Ruckus SMZ – Disabling TLS 1.0

A client recently had an issue where a security audit found ciphers supported within HTTPS that are insecure. These ciphers were TLS 1.0 and TLS 1.2. The audit found these issues on the web interface of the Smartzone, nothing to do with EAP or WiFi authentication. . After trying quite a few things I decided to open a ticket with Ruckus support. They instructed me to run the following commands on the SmartZone to disable it:

vszh-50#debug

vszh-50(debug)#

vszh-50(debug)# no tlsv1

This seemed to fix the issue. The web service (Tomcat) restarts and takes about 5 minutes before you can log back into the SMZ again.

Redundant network design using a Ruckus P300 as a backup link

This is a design I need a few weeks ago to help with a redundancy issue. Currently we have a client that occupies two buildings separated by about 500 hundred feet. Soon they will start construction to add a structure right in the middle , connecting the two buildings. But guess what runs right in the middle of this area? The fiber connecting the two buildings. We are thinking that the construction will most certainly cut the fiber causing an outage, whether planned or not.

We decided to have a backup wireless bridge link to help with redundancy. Ruckus’s P300 AC bridges works great, and that is what we decided to do .

Currently the link between the buildings is a Layer2 Trunk, and we are routing over Vlan 254 which traverses the trunk. OSPF is used to advertise each building’s local subnets, and redistribute the default route.

The goal is that routing/layer 2 will only come active on the wireless bridge in case of a failure in the Fiber connections. So Spanning-tree will block all vlans other than the native 200 – going through the bridge. If there is a failure, those vlans will come online over the bridge, routing will come up, and all should work great.

The switching/routing that is used is a Nexus 9500 and 3850 stack.

 

Bridges2

 

To accomplish the above, we enable OSPF on vlan 254, and make sure all routing is correct – including redistribution. Vlan 254 our routing vlan is allowed along with a few other vlans – At some point this will be fixed and we will only route over this link, but for now we have to stretch (I know, not the best practice).  Building 1 is currently the STP root for all vlans stretching over layer 2 link.  The Spanning-tree path cost is increased on links connecting to the bridge, and special commands are enabled on the wireless bridge to disable the Ruckus loop detection mechanism. This is very important because it will stop STP from flowing by default. Follow this link to help with that:

https://usermanual.wiki/Ruckus/P30010010957ReleaseNotesRevA20170721.1820034031/help.

After the bridge was setup, path cost modified, and Cisco port configs set correctly – it is time to test. First we needed to make sure STP was indeed blocking the vlans that were needed. Yes! STP is blocking the redundant path.

6040-blocked

Testing/Results

We tested fail over in two ways. 1 – just shutting down fiber links in CLI, and 2- physically unplugging the links. During fail over we saw that 2 pings were lost and then they were back up. I actually thought that OSPF drop, and then re-converge, but that did not happen. Instead, since the Hello-Dead timers were never reached, OSPF never dropped – fail over time was much better than I expected. The only way I could really tell we had failed over was a small increase in latency, and of course we were limited to around 300 Mbits.

Some notes on this – Make 100% sure that the Ruckus loop detection is disabled before even starting actual bridge configuration. Also create some kind of alerts VIA Prtg/Solarwinds/Cacti to send an alert if links go down, or their is a big increase in bandwidth on wireless bridges.

 

 

Ruckus P300 Bridge- Spanning-tree issue

I wanted to create a backup link for a network using a P300 bridge. The current network has two 10 gig links going between two buildings, but construction is set to start soon, that could cut the fiber stretching between. One option was to use the P300 bridges to create a backup link between the two building, which would become active in case of a failure in the fiber links.

We are currently stretching maybe 12 vlans between the buildings. The goal was to have all data go over the 2×10 gig links, and Spanning-tree block the other vlans from using the Bridge. I increased the STP port cost on each side and brought up the bridges – and both fiber links and bridge were forwarding, causing a loop. It took a bit to understand, but according to Ruckus their Gateway bridge detection mechanism basically stops STP and LACP from forwarding. I found the below help doc from Ruckus which gave the command to disable this feature.

https://usermanual.wiki/Ruckus/P30010010957ReleaseNotesRevA20170721.1820034031/help

After disabling – Bam! Vlans are blocked VIA STP.

The command to disable this feature in CLI:

rkscli: set meshcfg loop_detect disable

Below is the topology

Top

I modified the Path cost to something crazy high – 5000 with this command.

interface TenGigabitEthernet1/0/15
description “Connected to Ruckus Bridge”
switchport trunk native vlan 200
switchport trunk allowed vlan 5,10,12,14,15,20,22,40,150,200,254,1337
switchport mode trunk
auto qos trust
spanning-tree cost 5000

Then as soon as I disabled the loop Detect STP shutdown the secondary link for each vlan that the switch was not root for. .

For example, Building 1 is root for all vlans except 10, 254 and 12. So, Building 1 blocked those vlans from traversing port 15 which is the bridge.

6040-blocked

Failover took just seconds to work.

 

 

Ruckus ICX Radius logins

I refer back to these commands a lot and thought they might help someone else. This will allow the Ruckus or Brocade ICX switches to authenticate to a radius server for logins to the device.

aaa authentication web-server default radius local enable
aaa authentication login default radius local enable
aaa authentication login privilege-mode

radius-server host 1.1.1.1 auth-port 1812 acct-port 1813 default key $aWdAblUmc3JuVSY9Z1k= dot1x

A few things to note about this. I am setting the web-server login and SSH logins to use radius, then if radius is not available use local authentication settings.

The login privilege-mode command bypasses the enable password and logs be straight in a privileged.

Ruckus ICX untagged vlan port config

I have been working with Brocade ICX and now Ruckus ICX for a few years now. They are awesome switches.

I was asked a couple of times about something that was happening when someone would try and set the untagged or access vlan on a port. They would get this error:

error – port ethe x/x/x are not member of default vlan

The reason we were getting this error is because other vlans were attached to port as either untagged or tagged. To put a port into a vlan other than default as ‘untagged’ we need to make sure no other vlans are bound to that port. To do this we can check what vlans are attached to the port. In this scenario my default vlan is 999. It would be 1 on a switch that it was not manually changed on.

switch#show vlan br eth 1/1/3

Port 1/1/3 is a member of 2 VLANs

VLANs 32 48

Untagged VLAN : 999

Tagged VLANs : 32 48

Great, so now we know its untagged 999 (default) but tagged those 2 other ports. We need to remove the tags of 32 and 48 on this port before we can add it untagged into vlan 16 – which is the goal

switch#config t

switch(config)#vlan 32

switch(config-vlan-32)#no tag eth 1/1/3

switch(config-vlan-32)#vlan 48

switch(config-vlan-48)#no tag eth 1/1/3

Voice-vlan is cleared on port 1/1/3

switch(config-vlan-48)#

switch(config-vlan-48)#exit
switch(config)#vlan 16

switch(config-vlan-16)#unt eth 1/1/3

Added untagged port(s) ethe 1/1/3 to port-vlan 16.

switch(config-vlan-16)#exit

switch(config)#exit

switch#show vlan br eth 1/1/3

Port 1/1/3 is a member of 1 VLANs

VLANs 16

Untagged VLAN : 16

Tagged VLANs :

 

Thats it! now we are untagged or access in vlan 16. But wait! what if we wanted to have it be a trunk port to allow vlans 32/48 and be native 16. Then we would use the ‘Dual port’ command with the modification of the untagged vlan like this:

dual mode 16   — means untagged 16, but allow whatever vlans are tagged to pass. Of course vlans 16,32,48 would need to be tagged on the port first. I will write another entry about that.

Fortigate – Ping and Traceroute options

Within the Fortigate firewall you can modify many ping and traceroute options to suite what needs you might have. For example, if you need to modify the source IP address for a ping or trace you have that option and many more. Both ping and traceroute are crucial network troubleshooting  tools. Many times I need to ping through a VPN tunnel  using my internal interface, which is in the encryption domain to make sure the tunnel is up and working. or make sure the source of my ping or traceroute are on a local subnet to rule out routing/gateway issues.

The list below shows the table of ping options available:

Fortigate-Firewal # exe ping-options
adaptive-ping Adaptive ping <enable|disable>.
data-size Integer value to specify datagram size in bytes.
df-bit Set DF bit in IP header <yes | no>.
interface Auto | <outgoing interface>.
interval Integer value to specify seconds between two pings.
pattern Hex format of pattern, e.g. 00ffaabb.
repeat-count Integer value to specify how many times to repeat PING.
reset Reset settings.
source Auto | <source interface IP>.
timeout Integer value to specify timeout in seconds.
tos IP type-of-service option.
ttl Integer value to specify time-to-live.
validate-reply Validate reply data <yes | no>.
view-settings View the current settings for PING option.

So to highlight a few of these options – Lets modify the source address we are pinging from, increase the amount of pings and then show the settings to confirm all is set.

Fortigate-Firewall# exe ping-options source 192.168.1.1
Fortigate-Firewall# exe ping-options repeat-count 1000
Fortigate-Firewall# exe ping-options view-settings

Ping Options:
Repeat Count: 1000
Data Size: 56
Timeout: 2
Interface: auto
Interval: 1
TTL: 64
TOS: 0
DF bit: unset
Source Address: 192.168.1.1
Pattern:
Pattern Size in Bytes: 0
Validate Reply: no
Adaptive Ping: disable

I removed the default settings it spits out for brevity. That’s it though, we now have changed the source and the repeat count. Lots of other cool settings like ToS and size available.

Now on for Traceroute – You have less options, but the main two that I use – modifying the source IP or interface and setting the amount of hops it will go.

Fortigate-Firewall # exe traceroute-options
device Auto | <ifname>.
queries Integer value to specify number of queries per hop.
source Auto | <source interface IP>.
view-settings View the current options of traceroute.

Lets set the source for the traceroute to 192.168.1.1 and then check the settings.

Fortigate-Firewall# exe traceroute-options source 192.168.1.1

Fortigate-Firewall# exe traceroute-options view-settings
Traceroute Options:
Number of probes per hop: 3
Source Address: 192.168.1.1
Device: auto

Thats it! now we are modifying our source IP for both ping and Traceroute.

Cisco ASA – E-SMTP

I recently had an issue with a Office 365 deployment. This was a hybrid deployment, and as we were trying to start syncing to Office 365 we were getting an error in our logs :

(Retry : Must issue a STARTTLS command first)

This was causing mail not to flow. This site happened to be behind a Cisco ASA. After some research the cause of this problem is that the ASA is inspecting ESMTP in the policy map, which is stripping the STARTTLS packets.

To bypass this you have to create a new policy map to make sure the parameters of the inspection allow encrypted connections – the default is to not allow this, and it does this by removing the STARTTLS packets.

First lets create our policy map

policy-map type inspect esmtp esmtp_map
parameters
allow-tls action log

Then we will modify the global policy and match this policy for inspection. In the following example, the service policy is all default – the global_policy.

policy-map global_policy
class inspection_default

inspect esmtp esmtp_map

Thats it – now it will allow TLS . I had one other issue when applying I kept getting this error – ERROR: Inspect configuration of this type exists, first remove
that configuration and then add the new configuration

I was getting this error because by default the “Inspect esmtp” option is enabled in the global_policy map. So first remove this inspection and then reapply.  After modifying the ESMTP parameter mail started flowing. The follow shows that whole step.

show run

policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect esmtp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
!

config t

NHS-ASA(config-pmap)# policy-map global_policy
NHS-ASA(config-pmap)# class inspection_default
NHS-ASA(config-pmap-c)# inspect esmtp esmtp_map
ERROR: Inspect configuration of this type exists, first remove
that configuration and then add the new configuration

NHS-ASA(config-pmap-c)# no inspect esmtp
NHS-ASA(config-pmap-c)# inspect esmtp esmtp_map

show run :

policy-map type inspect esmtp esmtp_map
parameters
allow-tls action log
policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
inspect esmtp esmtp_map
!
service-policy global_policy global

 

 

802.11 – WIFI IFs

Inter frame spacing is some of the magic in WiFI. Its also one of the more confusing aspects of studying and understanding how WMM, and processes like Point coordination function work. Inter frame spacing is used to help avoid collisions on the wireless spectrum.

When a station wants to send a frame there are two methods that have to be OK before it can do this. 1 – Physical Carrier sense. Physical Carrier Sense is the process of checking to see if the wireless medium is busy. It does this my checking the wireless medium for RF energy, if it is above a certain threshold then it will wait to send.

The second check is Virtual Carrier sense. This method looks at passing wireless frames and sets a local MAC layer timer called the Network allocation vector or NAV to the amount of time found in the passing frame’s Duration field. When this reaches 0 it attempt a Physical Carrier sense. If the NAV is 0 and Physical Carrier sense shows RF energy below its threshold – the station will then wait the length of a Interframe space, then wait through the random backoff algorithm time (decided by the contention window), run through the carrier sense process again – then will actually send the frame.

The importance of the IFS is that the station has to wait that given amount of time. In the section below all the different IFS are listed with the its length of time. When studying for the CWAP exam I had a hard time remembering them, so hence the blog entry.

Once the station has the okay to send (Carrier sense is good) then it will wait the given IFS length. This is a big deal because not only does the STA wait the length of time before attempting to send the frame, but it can be used to give priority to certain frames over others. We do this by having that station wait less time then others based on the type of frame they are sending. By Priority I mean it asks the client to wait a smaller amount of time before sending the packet with a higher priority – This is called Probabilistic priority.

The following is a quick list of interframe spacing methods and are ordered by the duration time – shortest to longest time.

RIFS – Reduced IFS – Shorter than SIFS. Came out in 802.11n , to help minimize overhead. RIFS are used when one transmitter is sending packets when no SIFS reponses are expected. This IFS is not used in 802.11ac.

SIFS – Short Interframe Spacing – Shortest of all IFs . Used for ACKs, CTS in response to RTS, and the first data frame following the CTS.

PIFS – Point Coordination interframe space. – Used by the Point Coordination function to gain access to the medium. Used when switching from Distributed Coordination to Point Coordination

DIFS – DCF (Distribution coordination function) Interframe spacing– used with normal data frames. After the completion of sending a data frame the duration of a DIFS must be waited before attempting to send another.

AIFS – Arbitration IFS – Used in a kind of QoS manner. When a QoS station send data frames it waits an AIFS. Also used on a few control frames such as PS-Poll, CTS when not responding to a RTS (Thats a SIFS), BlockACKs, and RTS frames. A neat thing abou the AIFS is that the time is configurable on the AP, and then transmitted to clients by the Beacon frame.

EIFS – Extended IFS – Extended for a reason – When a frame is received with errors or corrupted the receiver will respond with  EIFS which tells the sender to wait for a longer period so it can see if the packet was delivered correctly. Once this happens, it will start go back to using a normal DIFS or AIFS .

Inter frame spacing gives us another mechanism to help avoid collisions in 802.11. It does this by having the Station wait a given amount of time (IFS) according to the type frame being sent.  We do this by utilizing Carrier sense, random back off timer, IFS and contention windows.

802.11 Spectrum Analysis – useful graphs

Spectrum analysis in 802.11 design is extremely important. Detecting which channels are in use in 2.4 and 5 gig spectrum’s as well as the channel density is a great help when channel planning or troubleshooting wifi issues. There are lots of graphs within any spectrum analyzing software.  Below are a few that I feel really help narrow down issues within spectrum.

FFT Graphs

FFT stands for Fast Fourier Transform – Which is an algorithm that samples signals over a period of time. One of the most usable graphs when looking at the spectrum is the FFT density graph and the real time FFT graph. I am currently using Fluke’s or now NetScouts SpectrumXT.

The below images shows the FFT of our office. As you can easily see just from the FFT graphs that channel 124 and 128 is heavily in use. This is because of a few reason – 1 we are using 40 MHZ channels so channel 124 and 128 are combined.  2 – the AP using these channels are the closest to me, the other APs are pretty far away.

fft-density

So why is this important during a survey or just checking to see whats going on – its important because it shows the amount of energy used on those channels. An abundance of energy will cause slowness and clients to defer sending as well as packet loss. Understanding and viewing spectrum is also extremely important in channel selection and design.

So lets compare the above with the Real time FFT – the Real time graph changes as the name states very often. But the Density and Real Time should be pretty similar. For channel planning purposes you can easily recognize that channels 124 through 136 should not be used. Lots of channels are open and can be used in this range that will not cause any interference.

FFT

 

Spectrogram

The next graph is the famous Spectrogram. The Spectrogram graph shows the amplitude of a signal over time. Basically it shows how busy and strong the signals are on that channel. These are also know as sweep or waterfall graphs. This graph shows how much signal density is on that channel. For example, if you have a channel maybe 104 that has traffic on it (seen by FFT), but its usage is very low (Spectrogram) then we know that that channel even though used would have very low interference, where as Channel 124 has lots of traffic and usage.

In the following graph the darker color shows more usage, notice Channel 124 which my STA is on.

Spectrogram.

The following graph shows the spectrum after my station starts downloading a huge file on WIFI. Check out the difference in usage.

spec-after-download

 

There are a lot of spectrum analysis graphs some are very helpful. These are two graphs that I find that help a lot to pinpoint the usage and channel use/layout of a wireless network. There are many more helpful graphs such as the wifi integrated graphs that can collect retries, Channel SNR, etc.