Mission

Create meaningful metadata to enrich customer device information delivered as a micro-service. Expose out this new metadata and allow consumption through exiting and new applications. The combination of the business importance and technical role of the device gives us a better understanding of the device's Place in Network.


Device Importance

Evaluate running features on customer equipment and weight the importance of devices based on running features. An SME walk of global feature data to determine "important" features was performed. Feature information is fed from Mimir for all devices for a company key. Devices are then weighted and ranked based on their operating system type and significant features. Categories of "low", "medium", "high", and "critical" are used to equate importance to the device. This information is available via a RESTful API.


Role in Network

Create on-the-fly supervised Random Forest models from extracted configurations. Categorize devices based on user specified roles from configuration and feature information by predicting against the training data. A RESTful API exposes the categorization predictions for devices and company keys.


How it Works
Training the Models

NCEs help train data models with supervised learning. By aligning devices with a particular role type, we can build a profile of configuration factors and ratios that are common across devices. We use a 80/20 split to build the model and then test the devices excluded from the training data to determine their role. The output of the model is delivered with a CSV file of prediction and feature extraction information, histogram of device classification, feature importance breakdown, clustering visualizations of roles, and a decision tree breakdown of how the algorithm determined which feature correlated to particular roles. When running an individual model, information for the customer is stored in the Role in Network API and can be then consumed from other applications and tools.

Feature Extraction

A "design intent" approach was used to develop an algorithm that looks at the configured state of a device to profile the functional role. By using how the device was configured, not only can we determine with features are in use by the device, but also the responsibility of the device. The algorithm defines service and configuration boundaries of devices and helps us cluster elements and features into role categories.
The Profile of the device is built off the following characteristics:

  • Features of the device
  • Importance features from the Device Importance API
  • Count of "up" interfaces by interface type
  • Ratio of "up" interfaces by interface type
  • Count of interfaces by switchport mode
  • Ratio of switchport modes by interface
  • Count of interfaces by interface speed
  • Count of interfaces by OSPF area
  • Count of interfaces in EIGRP
  • Count of FHRP protocols
  • Count of HSRP groups
  • Count of ibgp and ebgp Peers
  • Count of public and private peer AS
  • Count of BGP peers by address family
  • Count of VLANs
  • Count of SVIs
  • Ratio of VLANs to SVIs
  • Count of VLANs to STP by STP priority
  • Count of interfaces configured to use root guard
  • Count of interfaces by Service Policy, and direction
  • Count and length of applied ACLs
  • Count and priority of VPCs
  • Count of applied Route Maps
  • Count of static routes
  • Count of Netflow destinations
  • Applied QoS Statistics
  • Count of interfaces by ISIS levels
  • Route Redistributions
  • Count of applied and configured VRFs
  • Count of interfaces with port security enabled
  • Count of interfaces using dot1x authentication
  • Interface count by configured MTU
  • Count of interfaces with BFD enabled
  • BGP next-hop-self, aggregate addresses, maximum-prefix remove-private-as
  • Ratio and count of public IP addresses to private IP addresses
  • Count of interfaces with a voice vlan
  • Count of interfaces trusting QoS markings


Predicting Device Roles
Single Device Prediction

A single device's role can be predicted using the global model, or an industry specific model. A breakdown of classification results based on all role types contained in the model are displayed. This data is only displayed on-screen and is not saved to the Role in Network API.

Predicting all Devices for a Customer

Applying the prediction to all devices for a customer works in a similar fashion. The engineer can select to use the global model, or an industry specific model. After the model has run, a classification report is generated, along with a CSV file of predictions and features, feature importance breakdown, and role classification histogram. By default, this data is not stored automatically in the Role in Network API. The engineer can validate that the classification of devices is good, and can click the publish link to save the results to the API.

Publishing results to the Role in Network API

This option is abstracted from the training and running phase to allow an engineer to test a customer against multiple models before storing information in the API. Upon validation that the classification was successful, clicking the publish link will remove any previous entries for a customer and replace the data with the predicted classification results.