And Now For Something Completely Different
Last week I gave a talk to the Networking Field Day crew about something I’m calling Software Defined Intelligence, or SDI. SDI is a new interdisciplinary approach that integrates Compute, Storage, Networking, Security, Energy, and IoT (and I’m sure many other things) with Machine Learning. Like many “big” ideas, the some of the components of SDI may not themselves be new; rather it is the combination of machine learning with networking which creates a new discipline and which is novel here. From what we have already seen, we know that machine learning is having (and will continue to have) a dramatic effect on the way we build, operate and monetize networks, data centers, as well as diverse technologies including mobile handsets and just about everything else. Indeed, as renowned venture capitalist Vinod Khosla has opined “In the next 20 years, machine learning will have more impact than mobile has.”. While Mr. Khosla’s statement is perhaps a bit hyperbolic, few who are familiar with machine learning doubt its potential impact (complete with the hypothetical apocalyptic down sides [think Skynet] described by noted physicist Stephen Hawking and his colleagues).
Machine Learning is the foundational and enabling technology of SDI, and as such data science and “big data” are an inherent part of SDI. Andrew Ng, former Stanford professor and Coursea co-founder (now at Baidu), puts a finer point on it:
A trained learning algorithm (e.g., neural network, boosting, decision tree, SVM, …) is very complex. But the learning algorithm itself is usually very simple. The complexity of the trained algorithm comes from the data, not the algorithm.
This is a radically different paradigm than the programming approaches that we might have learned in school (or otherwise be used to), in which we write programs that process data to produce output. In the case of machine learning, the output plus data gives us a program that can predict outputs (sometimes called regression) given a new input, classify new inputs (is this an image of a cat or a dog or a motorcycle?), and many other tasks. This difference is depicted in cartoon form in Figure 1.
Of course, tasks like regression, classification and correlation analysis are key functions of any network analytics engine. Another key attribute of any analytics engine is whether it’s actions are proactive or reactive. To understand the difference, consider the example of DDOS mitigation. Today’s DDOS mitigation solutions are largely reactive: We collect SFLOW (and/or other kinds of) data and analyze it at some analytics backend; if we find that we’re seeing a DDOS attack we change the network’s configuration to deal with the offending flows in whatever way the network administrator sees fit. On the other hand, the SDI approach is proactive. We use the configurations, telemetry and flow data to predict the probability that a DDOS is about to occur. This is an instance of a class of techniques that are sometimes collectively called “Predictive Security”.
What Was The Promise Of SDN Anyway?
The promise of SDN has always been, among other things, to enable us to build much more intelligent networks. SDI is a framework that is designed to do just this: it takes advantage of the programmability and software orientation of technologies like SDN and combines it with Machine Learning to enable powerful new class of intelligent networks. These networks are based not only on SDN but also on Software Defined Compute, Storage, Security, Energy, IoT and beyond.
By the way, if you want to write some machine learning code to solve a problem you have or just to explore what is possible, there are many great open source libraries such as scikit-learn and pylearn2 (again, among many others), written in almost whatever language you like. Personally I like python; see these code snippets for a quickstart on how you can build a support vector machine with various kernels or an artificial neural network that is trained with backpropagation. Finally, the slides and video from last week’s talk can be found on the respective links.
There’s a lot to say on this topic but I wanted to get people thinking about the idea of using machine learning in conjunction with SDN (orchestration, …) to build a new and powerful kind of network intelligence. I’ll give you more detail on machine learning theory (and how this all works) as well as more use cases in my next blog on this topic.