Thursday, 18 January 2018 15:15

How to Differentiate Machine ..

Rate this item
(0 votes)

How to Differentiate Machine Learning From Dressed-up BI?

Machine learning is the use of computing resources that have the ability to learn without being explicitly programmed — that is, acquire and apply knowledge and skills that maximize the chance of success. That definition of machine learning, provided courtesy of Rob Clyde, vice-chair of the ISACAboard of directors and board director and executive chair to White Cloud Security is a pretty standard explanation. Or here’s another. Machine learning is a cognitive system that has the potential to learn from interactions and then deliver evidence-based answers to a problem.

Dressed Up As Machine Learning

But you’d never know that from the many vendors that purport to offer a machine-learning based application that is really something else — usually a dressed-up business intelligence solution against which SQL queries are run. Such vendors are prevalent, says Padraig Stapleton, VP of Engineering at Argyle Data, "Machine learning and AI are two terms that are abused a lot. You talk to a vendor and it will tell you that it has machine learning or AI but then you talk further to them about how the applications really works and it becomes clear that they do not."

Some executives like to throw around the names of particular machine learning algorithms, such as deep learning, XGBoost, random forest or boosted trees, according to Colin Priest, director of Product Marketing for DataRobot. But deep learning, one of his examples, tends to perform best on images and sounds or voices, he said — it doesn’t perform as well on mainstream business. 

Priest offered yet another definition for true machine learning: it positively impacts business operations so long as it satisfies three prerequisites: business/domain knowledge, access to data and an understanding of that data. "One of the things we realized early on is that there is a lot of market uncertainty about what AI actually is and what machine learning actually is," he said.

How To Tell The Difference

Mere definitions, though, only go so far in helping companies distinguish the pretenders from the real thing. Especially as so many companies have come to believe that any new application worthwhile purchasing must have machine learning as part of its make-up. "I think a lot of companies think that if whatever they buy doesn’t have machine learning then they are buying old, obsolete technology," Stapleton said. They are ready to believe, in other words, when a vendor says its product has machine learning. To help companies better tell what exactly it is that they are buying, these experts share some pointers.

It’s all about the data. Understand exactly what it is the application is doing with the data, Stapleton said. "Is your system capable of adapting to the changes in the underlying dataset, or is it subject to a rigid set of rules and thresholds that have been configured to your business? Be careful to distinguish mere automation from true, intelligent decision-making that detects and responds to patterns in data and adapts to changes over time, says Prasad Chalasani, senior vice president of Data Science at MediaMath. "In general, a system can be said to be ‘learning’ only if its behavior is not explicitly coded, but rather its actions, decisions, recommendations, etc. are able to improve automatically as it is exposed to increasing amounts of data."

He offers another example of a real-time bidding system. If that system is able to figure out over time that visitors to the page in California are between the ages of 30 and 40 and are more likely to click on a Tesla ad — and adjusts bid prices accordingly — then such a system can be said to be truly employing machine-learning. There is no explicit hard-coded rule that could prescribe that behavior, he said.

Another obvious example of AI would be self-driving vehicles. "They [self-driving cars] have to navigate new roads every day, under different weather conditions, moving through traffic and pedestrians that they have never seen before and yet they can," says Chris Nicholson, CEO of Skymind

How much customization is necessary? If an application is truly based on machine learning there will be a lot of customization involved, Stapleton said. "There is a data component piece, and then choosing of the right algorithm that will be applied to that dataset and adding some relevant features." If the application in question is really dressed-up BI or something similar then not much customization will be necessary. "It is very tell-tale if a vendor says, ‘We can install our solution out-of-the-box, start processing the data and give you results" over a short period of time.'"

Talk to the vendor. Really dig into the company and who its people are, Stapleton says, "Understand who the engineering team is and what their background is. A proper machine learning application is built combining access to data, access to domain expertise and access to the data scientists who developed the algorithms." 

Dima Stopel, co-founder and VP of R&D at Twistlock offered other questions to throw at a vendor. 


  • What is the exact machine learning algorithm that you use? 
  • Why did you choose it? 
  • How large is the learning data set and what are the most significant input features? 
  • How long does your system learn before providing any value? 

"If a vendor can provide good answers to the above questions, I'd say they use machine learning or at least try to," says Stopel.


Read 167 times Last modified on Thursday, 18 January 2018 15:26
More in this category: « Connect and Network What is COBIT? »

About Us


Chapter Members and Visitors,

Welcome to our ISACA, Great Houston Chapter website!

We recently held our Annual General Meeting and one day gratis Security and Audit Conference July 21. We have not offered a gratis one day conference in a while, although all of our conferences are quite reasonable, and it was appreciated. As someone noted, “Free is always good!”

We had a good turnout, around 125, with approximately 85 staying to the final bell; it did help that we gave away several $100 certificates at the close of the day.

We had several folks who gave significantly of their time to both plan and serve our members and guests during the conference, and we will acknowledge them shortly on our website. However, one person I want to thank now is Mel McQueary, with HCC, who was able to help us rent the auditorium at their West Loop South Campus. We received many compliments on the space itself, and the location.


More About Us

What We Do

Our aim is to sponsor local educational seminars and workshops, conduct regular chapter meetings, and help to further promote and elevate the visibility of the IS audit, control and security profession throughout the area. 

 We conduct chapter meetings the third Thursday of the month that typically includes a morning or afternoon training along with a luncheon meeting/training.  We also sponsor SIG group meetings on the same day.

Local seminars are held in the spring and fall that include topics of high relevance to our membership community. Certification training is scheduled before each ISACA exam date based on interest level. 

We partner with other organizations to provide additional educational opportunities for our greater Houston area membership. Please visit our chapter EVENTS page frequently for the latest educational and networking opportunities.