Blog

  • How to Become a NetApp ONTAP CLI Ninja

    How to Become a NetApp ONTAP CLI Ninja

    April 3rd, 2018
    Read More

    Over the past decade or so I’ve spent quite a few hours of my life staring at the NetApp ONTAP command line interface. 3840 hours, to be exact.  I did the math…8 hours per week, 48 weeks per year, over the last 10 years comes to 3840 hours (OK, not exact, but I’d argue a very conservative estimate).  I’ve worked on countless different versions of ONTAP containing multiple revisions of various commands.  As with anything you put 3840 hours into, I picked up a few shortcuts and tricks along the way.  My goal with this post is to share some of this knowledge to help you become more efficient in your role.  The tips below will only be relevant to those of you running Clustered Data ONTAP (aka CDOT, or just “ONTAP" now).  If you’re still running a NetApp system with a 7-mode (7m) version of ONTAP, it’s time to upgrade (*cough* we can help *cough*). 

    Almost to the fun stuff -  but first a few disclaimers.  Depending on your skill level, some of these may seem basic, but they get more advanced as you go through, I promise.  Nomenclature:

    • CLI commands will be italicized (volume would represent typing the word volume at the CLI)
    • Specific key presses will be bold (enter would represent pressing the enter key)
    • Variables will be indicated with <> brackets (<volume-name> would be a placeholder for an actual volume name)
    • Some commands require the use of special characters (! and { } and * for example).  Type these exactly as you see them displayed here.
    • All commands are always capitalization sensitive so pay close attention to upper/lower-case letters.

    Important caution: Any commands shown in the following post are for demonstration purposes only and should always be modified accordingly and used carefully.  Do not run any of the procedures below without thorough testing and if you do not fully understand the consequences.  Please contact a representative at H.A. Inc. if you need assistance with components of your infrastructure as it relates to this posting.

    Tip 1 – Use the tab key to autocomplete commands

    It’s somewhat difficult for me to show this via static screenshots and text but I think this feature is somewhat self-explanatory.  Not only will autocomplete fill commands in, it will (in most cases) fill in variables for you too.  For example, if you type the following:

    ne tab in tab sh tab -vs tab Cl tab enter

    you get the following command and associated output:

     

    Tip 2 – Consolidate output with rows 0

    You will find that some commands have a large output with some line wrapping that will require you to hit space or enter to continue.  This is inconvenient and makes a mess of the log if you’re trying to capture the output.  The default number of rows displayed can be set to infinite by entering the following command.  This will be reset back to default every time you login.

    rows 0

                    Output of a command with rows set to 10:

    Output of the same command with rows set to 0:

    Tip 3 – Use the -fields operator to show precisely what you need

    As shown by the network interface show outputs above, you can get a lot of good basic info about the LIFs using the basic show command.  But what if I wanted to show just the IP addresses and also add the allowed data protocol for each LIF? Add -fields then a comma separated list of fields you’d like to show to the command as shown below to customize the output:

    network interface show -fields address,data-protocol

    Tip 4 – Make your output CSV friendly with set -showseperator ","

    The ouput of tip 3 above is nice but not very friendly for copying out for use elsewhere (a script, a report, etc) due to inconsistent spacing and line wrapping.  Formatting the output with commas as separators is a huge help with this.  The default separator character can be set to a comma by entering the following command.  This will be reset back to default every time you login.

    set -showseperator ","

    Tip 5 – Use the wildcard * operator

    Use the asterisk as a wild card character at the beginning, end, or anywhere in the middle of the item you’re searching for (results will differ depending on location).  Let’s say I have a bunch of volumes but I’d like to get a report of volumes with “nfs” in the name:

    Or, I only want to see the volumes that are located on all aggregates that have “SSD” in the name:

    Or, how about I want to search the event log for all entries with “Dump” in the event column that occurred on “3/21/2018”

    Tip 6 – Use the NOT ! operator

    Use the exclamation point as a NOT operator for searches.  Let’s say I want to show all volumes that are not currently online:

     

    We can also combine ! with * to show all volumes that do not have “root” anywhere in the name of the volumes:

    Tip 7 – Use the greater/less than operators

    Use the greater than and less than operators during searches that involve sizes.  Let’s say I want to show all volumes with a size of more than 1GB:

    Tip 8 – Use curly brackets for extended queries

    Use curly brackets to perform a query in order to determine items to perform further action on.  For example, let’s say I want to modify only LIFs that meet the following specific criteria:

    • auto-revert set to false
    • not iscsi or fcp data-protocol

    Tip 9 – Use set -confirmations off to auto-confirm all end-user prompts

    Certain commands require the user to confirm for each occurrence.  When running a handful of commands in a row (as is often the case for repetitive/scriptable tasks), it is cumbersome to have to acknowledge after every single line.  Using set -confirmations off disables these confirmations.  This is potentially risky as the confirmations are often provided as a safety mechanism to ensure the action you are about to take is really what you intend so use this one carefully.

    Deleting a volume with set -confirmations on:

    Deleting a volume with set -confirmations off you can see the user is not prompted to confirm this action:

    Tip 10 – Naming naming naming

    As shown with tips 1-10, the powerful capabilities of the CLI really reinforces the importance of good, consistent naming conventions.  Develop good naming conventions, document them well, share them with your team, and stick to them.

  • The Network Engineer’s Favorite Dummy Address, 1.1.1.1, is Now Live on the Internet

    The Network Engineer’s Favorite Dummy Address, 1.1.1.1, is Now Live on the Internet

    March 29th, 2018
    Read More

    Most network engineers have, at some point in their career, used a “dummy” IP address of 1.1.1.1 for some reason or another. Maybe it was a placeholder in a configuration template, or maybe it was used for some sort of loopback or private connection that should never see the light of day such as a firewall failover link. Most of us have made this IP faux pas at least once.

     

    In fact, way back in 2010, when IANA allocated the 1.0.0.0/8 prefix to APNIC (the Asia-Pacific regional Internet number authority) work was done by APNIC’s research and development group to assess the feasibility of using various prefixes within the 1.0.0.0/8 block. The result of their study found that there was a substantial amount of random, garbage Internet traffic constantly flowing toward several of the prefixes within that range. Notably, IP addresses 1.1.1.1, 1.0.0.1, and 1.2.3.4 bore the brunt of this traffic barrage. The recommendation from APNIC’s R&D group at that time was not to allocate these prefixes to any Internet users due to the massive volume of background junk traffic always being directed at them.

     

    Now, fast forward to 2018. April Fool’s Day 2018, to be exact, and everyone’s favorite dummy IP addresses 1.1.1.1 and 1.0.0.1 are now live on the Internet providing a new free, privacy-oriented public DNS service. This is no prank, though, so how did it happen? Both Cloudflare (the company behind the new DNS service) and APNIC have put up blog posts discussing this joint project. The gist is that Cloudflare wanted to provide a public DNS service that, unlike Google’s 8.8.8.8 service, was not tracking/logging user activity. Because Cloudflare is already a web-scale service that provides content delivery network (CDN) services, Distributed Denial-of-Service (DDoS) attack mitigation, and other web services they were well-suited to both deal with, and study, the flood of background traffic going 1.1.1.1 and 1.0.0.1. APNIC agreed to provide these addresses to Cloudflare (for an initial period of 5 years) to home their DNS service at, so that APNIC and Cloudflare could study and gain insight from both the aggregated DNS query data and the rest of the background traffic.

     

    Should you start using 1.1.1.1 for your DNS resolution? The answer is that you could, but just like Google’s DNS service or Level3’s old standby of 4.2.2.2 there’s really no insight or control to be gained from the IT administrator’s perspective when using these upstream resolvers. There’s also no SLA guaranteeing uptime or performance. If you’d like to use reliable SLA-protected DNS services to provide an additional layer of security or content control for your users, H.A. strongly recommends looking at Cisco’s Umbrella DNS security service. Cisco Umbrella offers IT administrators the ability to provide security and web content filtering policy as well as protection against DNS-leakage for on-premises and remote users at the site, AD group, or user level, while providing statistics and activity reporting for the IT admin. For more information about Cisco Umbrella, contact your H.A. account manager.

     

    While this new 1.1.1.1 DNS service is interesting and may provide valuable Internet traffic research, there are a couple of take-aways that network engineers should keep in mind:

     

    1. We really shouldn’t ever use addresses that could be allocated on the Internet, but were not issued to us to use, for any reason. For private links such as firewall failover clustering, a piece of the RFC1918 IP space is best, or using addresses in the RFC3927 (the 169.254.0.0/16 link-local “auto-configuration” range) should also be acceptable.
       
    2. Now that 1.1.1.1 and 1.0.0.1 are live, we should avoid using them for documentation or examples. Actually, we never should have done this because the addresses were always valid as “real” Internet addresses, but there’s even more reason to avoid it now.
      If you need IP addresses to use in templates, or for examples or documentation, it’s best to use one of the three “TEST-NET” blocks defined in RFC5735. These include 192.0.2.0/24, 198.51.100.0/24, and 203.0.113.0/24.
       
    3. Be aware that some commercial products may have used 1.1.1.1 for some purpose, and that functionality could conflict with using the actual 1.1.1.1 DNS service now. An example of this is Cisco wireless LAN controllers which use 1.1.1.1 as the “virtual” interface address for serving guest portal splash pages. As a result of this, it’s not uncommon to see companies with DNS entries in their public DNS zones for 1.1.1.1 mapping to “guest.company.com” or something similar.

      Additionally, if using a Cisco Wireless LAN Controller for DHCP on a guest wireless network, the end client’s DHCP server will appear to be 1.1.1.1. In this case you would want to avoid assigning 1.1.1.1 for the guests’ DNS server since the WLC will intercept requests to 1.1.1.1 thinking it’s intended for it.
       
    4. We continue to scrape the bottom of the barrel for IPv4 address space and the activation of certain IP blocks that were always assumed to be de facto off-limits continues to prove this out. If you have not yet begun investigating IPv6 readiness in your network, it’s time to start that process.
  • High Availability, Inc. Named One of 2018 Tech Elite Solution Providers by CRN®

    High Availability, Inc. Named One of 2018 Tech Elite Solution Providers by CRN®

    March 27th, 2018
    Read More

    High Availability. Inc, announced yesterday that CRN®, a brand of The Channel Company, has named High Availability, Inc. to its 2018 Tech Elite 250 list. This annual list honors an exclusive group of North American IT solution providers that have earned the highest number of advanced technical certifications from leading technology suppliers, scaled to their company size.

    To compile the annual list, The Channel Company’s research group and CRN editors work together to identify the most customer-beneficial technical certifications in the North American IT channel. Companies who have obtained these elite designations— which enable solution providers to deliver premium products, services and customer support—are then selected from a pool of online applicants.

     “Being named to CRN’s Tech Elite 250 list is no small feat,” said Bob Skelley, CEO of The Channel Company. “These companies have distinguished themselves with multiple, top-level IT certifications, specializations and partner program designations from the industry’s most prestigious technology providers. Their pursuit of deep expertise and broader skill sets in a wide range of technologies and IT practices demonstrates an impressive commitment to elevating their businesses—and to providing the best possible customer experience.”  

    “We are thrilled to be included in the 2018 CRN Tech Elite 250,” said Steve Eisenhart, Chief Executive Officer of High Availability, Inc. “Over the last 10 years we have invested more in engineering than all other departments combined.  We see immense value when it comes to on-going education, trainings and certifications for existing and emerging technology partners.  We are continuously investing in additional engineering talent to support our expansion into new practice areas like Security, Cloud and Managed Services.   We understand the value our engineers bring to our end user community and are committed to doing all we can to deliver top-notch support and service.”

    Coverage of the Tech Elite 250 will be featured in the April issue of CRN, and online at www.crn.com/techelite250.

  • High Availability, Inc. Named Top Workplace in Philadelphia

    High Availability, Inc. Named Top Workplace in Philadelphia

    March 22nd, 2018
    Read More

    The Philadelphia Media Network and The Chamber of Commerce for Greater Philadelphia has officially recognized High Availability, Inc. as a Philadelphia Top Workplace for 2018. High Availability, Inc. was among 125 companies honored at an awards ceremony that took place last week. High Availability, Inc. was ranked #22 in the “Small Companies” category, which recognizes organizations with less than 150 employees.

    To be eligible for this prestigious designation, all nominated organizations must participate in a company-wide survey facilitated by Energage, a leading research firm focused on company culture and workplace improvement.  With over 47,000 organizations surveyed since 2006, Energage is the most trusted solution for empowering organizations to work better together.

    The Energage survey consists of four main factors; alignment, effectiveness, connection, and management. Alignment focuses on the direction and ethics of the organization, effectiveness assesses overall corporate communication and project execution, connection evaluates employee appreciation and job potential, and, lastly, the manager section reviews the employee’s manager and corporate leadership.

    “The entire team at High Availability, Inc. is honored to be named as one of the top workplaces in Philadelphia,” said Steve Eisenhart, Chief Executive Officer of High Availability, Inc. “Our #1 priority at H.A., Inc. is building and maintaining the right culture.  It is the foundation that allows us to increase productivity, attract talent, and most importantly create a positive and successful work environment.  I am convinced that it is our culture that has fueled our rapid growth over the last decade.  Making this list is a tremendous accomplishment and something that means the world to all of us at H.A., Inc!

    Click Here for more information about working for High Availability, Inc. and to view open positions.

  • Accessing Virtual Machine Consoles from the High Availability Inc. Customer Portal

    Accessing Virtual Machine Consoles from the High Availability Inc. Customer Portal

    March 15th, 2018
    Read More

    The High Availability Inc. Customer Portal gives users the ability to access a virtual machine's console using our web interface. The Portal features two different ways to open a console.
    The first way to access the Virtual Machine console is through the Device Management section of the portal. From the home page, click the ‘Devices Management’ link on the left-hand sidebar, then click ‘Devices’.


     
    The first icon in the row of ‘Shortcuts’ will open the console in a separate window for that corresponding device. The console button may be opened if the icon is green. The icon will appear as disabled, with a gray background, if console access to that device is unavailable. The button may also be red, which indicates that the device is powered off.
     
    Virtual Machine Consoles may also be accessed by clicking on a device name on the ‘Device Management’ page. The portal will then open the management page for the selected device. The individual device page features device information, data graphs, alert and ticket history. The console can be opened by clicking on the ‘Open Console’ tab in the top menu, to the right of the device name.


    For questions regarding the customer portal, or to request a user account, please contact us at support@hainc.com.
    https://my.haservices.com

  • VXLAN EVPN Multihoming

    VXLAN EVPN Multihoming

    March 9th, 2018
    Read More

    Introduction 

    Cisco Nexus platforms support a feature called VPC (virtual port channel) which a pair of switches act as a single switch for redundancy and both of switches will function as an active switch.

    For Cisco Nexus 9000 in VXLAN EVPN environments, two solutions are supported:

    • Traditional VPC

    • BGP EVPN 

    Traditional VPC use consistency checking, which is a mechanism used by both switches as VPC pair to exchange configuration information and verify compatibility. BGP EVPN lacks this consistency check. BGP EVPN relies on LACP protocol to detect any miss-configuration.

    This approach eliminates the MCT link used by traditional VPC and offers more flexibility.

    BGP EVPN Multihoming:

    Terminology:

    • EVI: The EVPN instance (EVI) is represented by the virtual network identifier (VNI).

    • MAC-VRF: The MAC Virtual Routing and Forwarding (MAC-VRF) instance is a container for housing the virtual forwarding table for MAC addresses.

    • ES: The Ethernet segment (ES) is a set of bundled links.

    • ESI: The Ethernet segment identifier (ESI) represents each Ethernet segment uniquely across the network.

    Guidelines and limitations

    • EVPN multihoming is supported on the Cisco Nexus 9300 Series switches only and it is not supported on the Cisco Nexus 9300-EX and 9500 Series switches. The Cisco Nexus 9500 Series switches can be used as Spine switches but they cannot be used as VTEPs.

    • EVPN multihoming is not supported on FEX.

    • EVPN multihoming is supported with multihoming to two switches only.

    • Cisco recommends enabling LACP on ES PO.

    EVPN Multihoming redundancy group

    The following figure shows a dual-homed topology in which VTEPs Leaf1 and Leaf 2 are distributed anycast VXLAN gateways performing IRB (integrated Routing and Bridging). Host 2 is connected to a switch which is dual-homed to both Leaf1 and Leaf2.

    EVPN Multihoming

    The switch is not aware of that the bundle is configured on two different switches. But the Leaf 1 and Leaf 2 must be aware of that they are part of the same bundle.

    To make the VTEPs aware that they belong to the same bundle link, Cisco NXOS uses the ESI and system MAC address configured on the port-channel interface.

    Ethernet Segment Identifier 

    It is a 10-byte value for bundled link that they share with multihomed neighbor. 

    LACP Bundling

    LCAP can detect ESI misconfiguration and the port-channel bundle. The LACP on the leaf switches will send the ESI configured MAC address in the hello message sent to the access switch. LACP is not mandatory with ESI. A given ESI interface shares the same ESI across the VTEPs in the group.

    EVPN Multihoming with LACP

    It is recommended to run LACP between VTEPs and access devices because LCAP BPDs have a mechanism to detect and act on the ESI misconfiguration.

    Layer 2 Gateway STP

    L2G-STP builds a loop-free topology. The STP root always must be the VXLAN Fabric. A bridge ID for STP consists of a MAC and bridge priority. For VXLAN fabric the system automatically assigns the VTEPs with a MAC address from the reserved range (c84c.75fa.6000), so each switch uses the same MAC address for the bridge ID emulating a single logical pseudo root. The L2G-STP is disabled by default.

    All L2G-STP vlans should be set to a lower priority than the fabric edge switches. Or you can set the leaf switches L2G-STP priority to 0.

    BGP EVPN Multihoming Configuration steps:
    • Enable multihoming globally: evpn esi multihoming
    • Enable BGP maximum paths. This setting enables equal-cost multipath (ECMP) for host routes. Otherwise, host routes will have only 1 VTEP as the next hop. 
      • maximum-paths ibgp x
      • maximum-paths x
    • Enable core links. This setting tracks uplink interfaces to the core. If all uplinks are down, local Ethernet segment–based ports will be shut down or suspended. This setting is used mainly to avoid black-holing south-to-north traffic when no uplinks are available.
      • evpn multihoming core-tracking
    • Configure the Ethernet segment.
      • interface port-channel x
      • ethernet-segment <es-id>
      • system-mac <es-system-mac>
    • Configure TCAM.
      • hardware access-list tcam region vpc-convergence 256
      • hardware access-list tcam region arp-ether 256 

     

    Sample Configuration

     

    Leaf 1 configuration:

    evpn esi multihoming ⇒enable the feature

    Interface X/Y ⇒  leaf switch connection to core 

    evpn multihoming core-tracking ⇒ enable core link tracking

    interface port-channel XXX  ⇒ port channel link to downstream switch

    ethernet-segment 2011 ⇒ ESI

        system-mac 0000.0000.2011 ⇒ system MAC

    Leaf 2 configuration:

    evpn esi multihoming ⇒ enable the feature

    Interface X/Y ⇒  leaf switch connection to core 

    evpn multihoming core-tracking ⇒ enable core link tracking

    interface port-channel XXX  ⇒ port channel link to downstream switch

    ethernet-segment 2011 ⇒ ESI

        system-mac 0000.0000.2011 ⇒ system MAC

    References:

    • Cisco VXLAN EVPN Multihoming white paper

    • Cisco Nexus 9000 series NX-OS VXLAN Configuration

  • Replacing a Defective UCS 6324 Fabric Interconnect

    Replacing a Defective UCS 6324 Fabric Interconnect

    February 28th, 2018
    Read More

    Occasionally, UCS hardware does go bad and recently I had the opportunity to replace a defective UCS 6324 Fabric Interconnect.

    First, some background on the hardware…

    The UCS 6324 Fabric Interconnect (FI) is a core component of Cisco's UCS mini system and extends the UCS architecture into smaller domains while providing the same unified server and networking capability as in the full-scale UCS solution.  The UCS 6324 Fabric-Interconnect provides embedded connectivity in a small UCS domain for up to 20 servers.

    Unlike the traditional Fabric Interconnects, the UCS 6324 Fabric Interconnects are installed directly into the UCS 5108 chassis, where the IO modules are installed.  (see diagram below)

    Because the 6324 Fabric Interconnects are installed directly into the chassis as part of the UCS mini design, and there are no IO modules required, there is no need for FEX cabling or L1/L2 cables between FIs.

    Before we begin to remove the defective hardware (in our case Fabric-Interconnect A), if there are QSFP licenses installed on the defective FI, we should contact TAC to have them transfer the port-licenses to the replacement Fabric Interconnect.  Since our 6324 Fabric Interconnect was not purchased with additional port-licenses, we're good here.

    Other considerations before we begin:

    • Cabling connected to the ports on the 6324 Fabric Interconnects should be labeled.

    • Obtain a full-state and configurational backup of the 6324 Fabric interconnects.

    • Verify the cluster state of the Fabric Interconnects

     

    To verify the cluster state of the existing Fabric Interconnects, SSH to UCS Manager (UCSM) and run the command "show cluster extended-state".  We're looking for the "HA Ready" message between the 2 Fabric Interconnects.

     

    PA-UCS-6324-A# show cluster extended-state 

    Cluster Id: 0xe144f0c272fc11e7-0x9678002a6a88c401

    Start time: Tue Nov 21 22:08:26 2017

    Last election time: Tue Nov 21 22:29:48 2017

    A: UP, PRIMARY

    B: UP, SUBORDINATE

     

    A: memb state UP, lead state PRIMARY, mgmt services state: UP

    B: memb state UP, lead state SUBORDINATE, mgmt services state: UP

       heartbeat state PRIMARY_OK

    INTERNAL NETWORK INTERFACES:

    eth1, UP

    eth2, UP

    HA READY

    Detailed state of the device selected for HA storage:

    Chassis 1, serial: FOX1743H1ZN, state: active

    PA-UCS-6324-A#

     

    Once we have verified and performed the tasks listed above, we are ready to start the replacement of the defective Fabric Interconnect.

    Step #1:  

    The next task is to perform a fabric evacuation on the defective Fabric Interconnect.  The caveat here is that the fabric evacuation can ONLY be performed on the subordinate Fabric Interconnect, so if the defective Fabric Interconnect is currently the primary Fabric Interconnect, we will need to change the cluster lead over to the secondary Fabric Interconnect.  This can be accomplished by switching to local management and running the command, "cluster lead b".  Be sure to run the "show cluster state" command to verify the cluster state as "HA Ready" before proceeding.

     

    PA-UCS-6324-A# sh cluster state

    Cluster Id: 0xe144f0c272fc11e7-0x9678002a6a88c401

     

    A: UP, PRIMARY

    B: UP, SUBORDINATE

    HA READY

    PA-UCS-6324-A# connect local

    Cisco Nexus Operating System (NX-OS) Software

    TAC support: http://www.cisco.com/tac

    Copyright (c) 2009, Cisco Systems, Inc. All rights reserved.

    The copyrights to certain works contained in this software are

    owned by other third parties and used and distributed under

    license. Certain components of this software are licensed under

    the GNU General Public License (GPL) version 2.0 or the GNU

    Lesser General Public License (LGPL) Version 2.1. A copy of each

    such license is available at

    http://www.opensource.org/licenses/gpl-2.0.php and

    http://www.opensource.org/licenses/lgpl-2.1.php

    PA-UCS-6324-A(local-mgmt)#

    PA-UCS-6324-A(local-mgmt)# cluster lead b

    If the system is at 'infrastructure firmware' auto-install 'pending user Ack' stage,this action will result in upgrading and rebooting current primary. Please check the outstanding faults (scope monitoring <enter> show new-faults) and make sure the data-paths on FI-B are established properly before making it primary to ensure there is no data outage.

    Do you want to continue? (yes/no):yes

    Cluster Id: 0xe144f0c272fc11e7-0x9678002a6a88c401

    PA-UCS-6324-A(local-mgmt)# sh cluster state

    Cluster Id: 0xe144f0c272fc11e7-0x9678002a6a88c401

    A: UP, SUBORDINATE

    B: UP, PRIMARY

    HA READY

    PA-UCS-6324-A(local-mgmt)#

     

    To evacuate the vEths and vHBAs from the defective Fabric Interconnect, navigate to Equipment tab -> Fabric Interconnects/Fabric-Interconnect A.  

    Note that "Admin Evac Mode" can be enabled, even if the subordinate FI is to be replaced/removed because the UCSM configuration is kept on both FIs as a cluster.  

    Once the Fabric-Interconnect A has been evacuated, and the vEths and vHBAs have been moved over to Fabric-Interconnect B, we can now safely remove the defective Fabric-Interconnect.

    Step #2:

    The next step is to physically swap the defective Fabric Interconnect with the replacement FI.  Once the replacement FI has been inserted into the slot, it takes roughly 2 minutes for the FI to power-up.

    Step #3:

    After the replacement Fabric Interconnect has been powered up, the FI will boot and run POST.  Once this is completed, the Fabric Interconnect will run through a series configurational dialog messages.

    The FI should then detect the presence of a peer Fabric-Interconnect and will prompt us to add the replacement FI to the cluster.  This process also copies the configuration details over to the replacement FI.  The setup process then compares the firmware version of the replacement FI to the active Fabric-Interconnect.  If the firmware versions are not identical, the configurational dialog will prompt for an upgrade to match the peer FI version.  This process takes around 5-10 minutes.

     

    Enter the configuration method. (console/gui) ? 

      Enter the configuration method. (console/gui) ? console

      Installer has detected the presence of a peer Fabric interconnect. This Fabric interconnect will be added to the cluster. Continue (y/n) ? yes

      Enter the admin password of the peer Fabric interconnect: 

        Connecting to peer Fabric interconnect... done

        Retrieving config from peer Fabric interconnect... done

        Installer has determined that the peer Fabric Interconnect is running a different firmware version than the local Fabric. Cannot join cluster.

        Local Fabric Interconnect

          UCSM version     : 3.0(2c)

          Kernel version   : 5.0(3)N2(3.02c)

          System version   : 5.0(3)N2(3.02c)

          local_model_no   : Mini

        Peer Fabric Interconnect

          UCSM version     : 3.2(2b)

          Kernel version   : 5.0(3)N2(3.22a)

          System version   : 5.0(3)N2(3.22a)

          peer_model_no    : Mini

      Do you wish to update firmware on this Fabric Interconnect to the Peer's version? (y/n): y

    Updating firmware of Fabric Interconnect....... [ Please don't press Ctrl+c while updating firmware ]

     Updating images 

     Please wait for firmware update to complete.... 

     Checking the Compatibility of new Firmware..... [ Please don't Press ctrl+c ]. 

    Verifying image bootflash:/installables/switch/ucs-mini-k9-kickstart.5.0.3.N2.3.22a.bin for boot variable "kickstart".

    [####################] 100% -- SUCCESS

    Verifying image bootflash:/installables/switch/ucs-mini-k9-system.5.0.3.N2.3.22a.bin for boot variable "system".

    [####################] 100% -- SUCCESS

    Verifying image type.

    [####################] 100% -- SUCCESS

    Extracting "system" version from image bootflash:/installables/switch/ucs-mini-k9-system.5.0.3.N2.3.22a.bin.

    [####################] 100% -- SUCCESS

    Extracting "kickstart" version from image bootflash:/installables/switch/ucs-mini-k9-kickstart.5.0.3.N2.3.22a.bin.

    [####################] 100% -- SUCCESS

    Extracting "bios" version from image bootflash:/installables/switch/ucs-mini-k9-system.5.0.3.N2.3.22a.bin.

    [####################] 100% -- SUCCESS

    Performing module support checks.

    [####################] 100% -- SUCCESS

    Notifying services about system upgrade.

    [####################] 100% -- SUCCESS

    Compatibility check is done:

    Module  bootable          Impact  Install-type  Reason

    ------  --------  --------------  ------------  ------

         1       yes      disruptive         reset  Incompatible image

    Images will be upgraded according to following table:

    Module             Image         Running-Version             New-Version  Upg-Required

    ------  ----------------  ----------------------  ----------------------  ------------

         1            system         5.0(3)N2(3.02c)         5.0(3)N2(3.22a)           yes

         1         kickstart         5.0(3)N2(3.02c)         5.0(3)N2(3.22a)           yes

         1              bios                v1.022.0                v1.022.0            no

         1         power-seq                    v1.0                    v1.0            no

    Switch will be reloaded for disruptive upgrade.

    Install is in progress, please wait.

    Performing runtime checks.

    [####################] 100% -- SUCCESS

    Setting boot variables.

    [####################] 100% -- SUCCESS

    Performing configuration copy.

    [####################] 100% -- SUCCESS

    Install has been successful.

     

     

    Step #4:

    Once the replacement FI has been upgraded, it will now prompt us to enter the management IP information.  After the management IP information has been configured on the replacement FI, the Fabric Interconnect will reboot two more times and will come online in approximately 10 minutes.

     

     

     Firmware Updation Successfully Completed. Please wait to enter the IP address 

        Peer Fabric interconnect Mgmt0 IPv4 Address: 172.17.10.72

        Peer Fabric interconnect Mgmt0 IPv4 Netmask: 255.255.255.0

        Cluster IPv4 address          : 172.17.10.70

        Peer FI is IPv4 Cluster enabled. Please Provide Local Fabric Interconnect Mgmt0 IPv4 Address  

      Physical Switch Mgmt0 IP address : 172.17.10.71

      Apply and save the configuration (select 'no' if you want to re-enter)? (yes/no): yes

      Applying configuration. Please wait.

      Configuration file - Ok

     FI will now reboot after which you can access the switch.

    WARNING: There is unsaved configuration!!!

    WARNING: This command will reboot the system

     

    After the reboots have been completed, the replacement FI should be good to go.  To verify IP reachability, console into the replacement FI and verify that it can reach the peer Fabric-Interconnect and its gateway.

     

    PA-UCS-6324-A(local-mgmt)# ping 172.17.10.72

    PING 172.17.10.72 (172.17.10.72) from 172.17.10.71 : 56(84) bytes of data.

    64 bytes from 172.17.10.72: icmp_seq=1 ttl=64 time=5.94 ms

    64 bytes from 172.17.10.72: icmp_seq=2 ttl=64 time=0.108 ms

    64 bytes from 172.17.10.72: icmp_seq=3 ttl=64 time=0.109 ms

    64 bytes from 172.17.10.72: icmp_seq=4 ttl=64 time=0.105 ms

    64 bytes from 172.17.10.72: icmp_seq=5 ttl=64 time=0.111 ms

    64 bytes from 172.17.10.72: icmp_seq=6 ttl=64 time=0.103 ms

    ^C

    --- 172.17.10.72 ping statistics ---

    6 packets transmitted, 6 received, 0% packet loss, time 5015ms

    rtt min/avg/max/mdev = 0.103/1.079/5.942/2.174 ms

    PA-UCS-6324-A(local-mgmt)# ping 172.17.10.1

    PING 172.17.10.1 (172.17.10.1) from 172.17.10.71 : 56(84) bytes of data.

    64 bytes from 172.17.10.1: icmp_seq=1 ttl=255 time=0.541 ms

    64 bytes from 172.17.10.1: icmp_seq=2 ttl=255 time=0.581 ms

    64 bytes from 172.17.10.1: icmp_seq=3 ttl=255 time=0.569 ms

    64 bytes from 172.17.10.1: icmp_seq=4 ttl=255 time=0.648 ms

    64 bytes from 172.17.10.1: icmp_seq=5 ttl=255 time=0.578 ms

    64 bytes from 172.17.10.1: icmp_seq=6 ttl=255 time=0.567 ms

    ^C

    --- 172.17.10.1 ping statistics ---

    6 packets transmitted, 6 received, 0% packet loss, time 4996ms

    rtt min/avg/max/mdev = 0.541/0.580/0.648/0.042 ms

    PA-UCS-6324-A(local-mgmt)#

     

    Step #5:

    Once the UCS Manager is up, http into UCSM and set the "Admin Evac Mode" to "off".  This can be configured under to Equipment tab -> Fabric Interconnects/Fabric-Interconnect A.

     

    Step #6:

    As a final step, switch the cluster lead to Fabric-Interconnect A by running the following commands:

     

    connect local-mgmt b

    show cluster state

    cluster lead a

     

    PA-UCS-6324-A# connect local-mgmt b

    Cisco Nexus Operating System (NX-OS) Software

    TAC support: http://www.cisco.com/tac

    Copyright (c) 2009, Cisco Systems, Inc. All rights reserved.

    The copyrights to certain works contained in this software are

    owned by other third parties and used and distributed under

    license. Certain components of this software are licensed under

    the GNU General Public License (GPL) version 2.0 or the GNU

    Lesser General Public License (LGPL) Version 2.1. A copy of each

    such license is available at

    http://www.opensource.org/licenses/gpl-2.0.php and

    http://www.opensource.org/licenses/lgpl-2.1.php

    PA-UCS-6324-B(local-mgmt)# show cluster state

    Cluster Id: 0xe144f0c272fc11e7-0x9678002a6a88c401

    B: UP, PRIMARY

    A: UP, SUBORDINATE

    HA READY

    PA-UCS-6324-B(local-mgmt)# cluster lead a

    If the system is at 'infrastructure firmware' auto-install 'pending user Ack' stage,this action will result in upgrading and rebooting current primary. Please check the outstanding faults (scope monitoring <enter> show new-faults) and make sure the data-paths on FI-A are established properly before making it primary to ensure there is no data outage.

    Do you want to continue? (yes/no):yes

    Cluster Id: 0xe144f0c272fc11e7-0x9678002a6a88c401

    PA-UCS-6324-B(local-mgmt)#

    PA-UCS-6324-B(local-mgmt)# sh cluster state

    Cluster Id: 0xe144f0c272fc11e7-0x9678002a6a88c401

    B: UP, SUBORDINATE

    A: UP, PRIMARY

    HA READY

    PA-UCS-6324-B(local-mgmt)#

     

    Congratulations!  You have successfully replaced a defective UCS 6324 Fabric Interconnect.

  • Tips for Meraki WLAN Deployment Part 3: Working with Floor Plans

    Tips for Meraki WLAN Deployment Part 3: Working with Floor Plans

    February 15th, 2018
    Read More

    This is the 3rd part in our series on Meraki WLAN deployment tips. The first article covered some helpful hints for assigning IP addressing to your Meraki access points. The second post had tips for AP naming, tagging, and the important of installation photos.

     

    In this final part in the series, we will discuss adding a floor plan to your Meraki WLAN deployment. Why do we need to do this? What value does it provide? What cool things can we do once we have a floor plan in our deployment? Read on to find out!

     

    To start with, let’s cover a basic fact: Adding floor plans to your Meraki WLAN deployment is completely optional. And there are at least a few cases where they are unnecessary. For example, if your deployment is covering a large outdoor space, the default Google Maps satellite view will work fine. Or if you have a small remote office with a single access point, there’s not a ton of value in having a floor plan imported. However, if your deployment involves multiple access points in the interior of a building, then adding a floor plan gives you some nice benefits.

     

    First, a populated floor plan lets you easily see AP locations and some quick stats of those APs. Take, for example, this floor plan display from the H.A. office in Audubon, PA:

     

     

    This screen is found at the Wireless > Map & floor plan menu item. As you can see, we can easily identify where our APs are physically located. Each AP also has a number overlaid on it. This is the current number of wireless clients associated with that access point. This can be useful for gauging client density, although there is a better tool for that which we will explore later.

     

    Mousing over one of the APs brings up a link to that AP’s Dashboard page, like this:

     

    Clicking the PA-AP07 link will take me to the details page for that AP.

     

    Likewise, if I want to see where a specific AP is physically deployed, I can go to that AP’s details page in Dashboard, and then go to the Location tab. The AP will be displayed on the supplied floor plan. In this example, we can easily find that PA-AP03 is the one in our reception area. If notes or tags were not added when APs were deployed, this can be very helpful!

     

     

    OK, so we can locate an AP on the map. That’s handy. What else? Well, what if we want to locate a client device within our environment? That’s easy too, once floor plans are deployed. Simply navigate to a client’s details page by going to Network wide > Clients, and then clicking on a wireless client in the list. The client detail page will be displayed, and the users most-likely location will be indicated on the floor plan:

     

     

    In this case, the user’s iPhone is believed to most likely be at the location of the blue dot. The light blue circle is the circle of uncertainty, this is the total area that the client could be in, but the system thinks the location of the blue spot is most likely. If the system can get a tighter triangulation on the user, the circle of uncertainty will get smaller to indicate higher confidence. In this eample, I know the location the device is indicated as being is exactly where this user’s cubicle is, so the system has located this user’s phone with a high degree of accuracy. This can be convenient if you need to hunt down a device/user/guest that isn’t behaving well, or if you’re asked to help locate a missing device.

     

    Locating a device in this way isn’t always perfect, but it gives you a good place to start if you’re trying to locate someone/something.

     

    Another useful thing you can do with the floor plan is see your wireless channel settings visually. If you navigate to Wireless > Radio settings, you will see a list of all access points and their assigned channels for the current network. If you hit the “Map” toggle, you will see the APs on the floor plan with their assigned channels as labels, like this:

     

     

     

    This can be helpful for manual channel planning to ensure you’ve got the maximum possible separation between APs on the same channel or adjacent channels, or even to review automatic channel assignments to ensure the system is not making a bad choice in assigning APs to certain channels.

     

    Now, one of my favorite uses of the floor plan: the location heatmap. This feature, which is found at Wireless > Location heatmap in the Dashboard, shows you a color-coded heatmap of where wireless clients are physically located in your environment.

     

     

    What can we use this for? It’s a nice real-time display of how many devices are in certain areas of your environment. It can be used to see where high-density regions might be, or if an unusual congregation of devices (and thus people) has popped up. And notice the “Play” button and timeline slider in the upper-left corner of the screen? That lets you play back an hour-by-hour view of the heatmap to see how client density has changed over time. For an office setting, that’s nifty. In a retail or industrial setting, this can be highly valuable business intelligence, letting you see where customers tend to spend time in a retail store, or by seeing patterns in movement of fork trucks in a warehouse during shifts.

     

    As you can see, taking advantage of the Dashboard features that can utilize floor plans can give you some nice perks in your Meraki deployment.

     

    Now that you have some idea of why you want floor plans configured in your Meraki WLAN networks, how do you set them up? Fortunately Meraki has already documented that procedure better than I could (with animated GIFs and everything!), so head on over to their site for the how-to.

     

    If you’re interested in a Meraki-based WLAN deployment, or need help tuning an existing deployment to get the most out of it, reach out to your High Availability Account Manager today. We’d be happy to help you!

     

Join the High Availability, Inc. Mailing List

Subscribe