Table of Contents
Introduction
Why Machine Learning? Why do you want to understand Machine Learning? How does it matter to your life? If you don't master it, why do you at least need to understand the basics of Machine Learning? The answer to all these questions is very simple. It’s because Machine Learning on a day to day basis is becoming bigger and it is knowingly or unknowingly part of our life, so it is important to know what it is. We’ll try to answer all the basic questions related to machine learning in this and the following articles on Machine Learning to figure out what Machine Learning is and what can be achieved with it.
Series
We’ll try to cover the topic and Machine Learning concepts including terminology in the form of a series. This being the first article, it will focus on high level machine learning concepts that could be followed by understanding machine learning in detail with how it works. Here are the articles in the machine learning series:
- Introduction to Machine Learning
- Machine learning processes and scenarios
- Machine learning : Deep dive
Machine Learning
So, what is machine learning? To be honest, there is no definition of machine learning that everyone knows, but it is a concept and there are more core concepts related to this. The one important area where we could apply machine learning is to find patterns in data. Then those patterns are used to predict the future and make decisions. For example, on a Learning Management System (L.M.S.) one can identify that one what chapter, content, image a learner or reader spent more time reading and what interests an individual as a reader. One can apply machine learning concepts to identify each and every action and the time spent by the reader on the learning content, and accordingly, the content creator will know in which area of the content are people more interested, and what has to be improved or should be focused on. Another such example could be to detect an internet banking fraud.
If there is continuous fraud happening with the funds transfer via internet banking and we have the complete data, we could find out the pattern involved through which we can identify that where are the loop holes or hack prone areas of the application. So it’s all about patterns and predicting the results and future on the basis of those patterns. Therefore, we can say that machine learning plays an important role in data mining, image processing and language processing. Machine learning could not always provide a correct analysis or could not always provide an accurate result based on the analysis but it gives a predictive model based on historical data to make decisions. The more data the more result oriented predictions could be made.
Patterns
Let’s take a simple example of reading and learning. We learn by reading, but how do we read? We read by identifying patterns, we identify the letters, the patterns of the alphabets to form words. Then we get adapted to that pattern by seeing it again and again. Therefore, we learn to read and eventually read to learn. Now, coming back to the net banking fraud that I mentioned, suppose we have data for the transactions happened through net banking, we can create a predictive analysis model and try to know the future. Let’s suppose we have five records for transactions.
As shown in the above image, we have the data of customer name, amount and the fraudulent transactions.
When we see the data and try to identify the pattern, we could not really get anything out except from the conclusion that all those customers whose name starts with "J" have fraudulent transactions. But this is not something on which we can rely upon as this is kind of vague predictions. So the limitation here for us to identify the correct pattern is data or one can say limited/less data. We cannot conclude that why the two transactions are likely to be fraudulent. Now suppose we have more data as shown below.
So, we have almost 16 transaction records with us now with more details like the location of the transaction and the age of the customer as well. If we closely consider the data and try to find out a pattern, we find that all the fraudulent transactions have been made in China, the age group of the customers who fall in fraudulent transactions is between 20 to 25 and the amount for which fraud has been detected is always more than $20000. So we can say that we found a pattern here, but again, can we rely on this analysis and make a prediction? Do we have enough data for the analysis? Maybe not. We need more and more transaction data to actually predict the behavior based on patterns. But when it comes to millions of transaction records, it would be then cumbersome for us to find the right pattern with the naked eye (or manually). Therefore, we need software to do all the analysis and so machine learning comes into picture.
Machine Learning: High Level View
At a high level, Machine Learning could be understood in a way as shown in the following diagram.
We initially start with lots of data, the data that contains patterns. That data gets inside machine learning logic and algorithm to find the pattern or patterns. A predictive model is the outcome of the machine learning algorithm process. A model is typically the business logic that identifies the probable patterns with new data. Applications are used to supply data to the model to know if the model identifies the known pattern with the new data. In the case that we took, new data could be data of more transactions. Probable patterns mean that a model should come up with predictive patterns to check if the transactions are really fraudulent.
Machine Learning: The Buzz
We see machine learning picking up the pace and is very popular now days. The question is why, but there are a lot of reasons to answer this question. Firstly, if we see machine learning process we figure out that to correctly do machine learning; we need to feed it with lots of data. The data could be structured or un-structured, and the good thing is that we have that data. In this era of Big Data we not only have a lot of data but we also have the computation power to process that data. Not only this, we talk about cloud computing which requires operative and implied machine learning algorithms, which we also have. All these things were not available earlier, but they are now. Another question is, who does machine learning interest? Who are the people who actually care about machine learning? The people in a nut shell could be divided into three major groups. First is business owners, they always need solutions to solve their business needs and business problems. For example, to check the fraudulent transactions, to check how likely their customers are to switch to other providers. Machine learning thus adds value to the business by providing better solutions and accurate predictions. The bigger an organization is, the better and faster business they own and therefore the need of those business solutions always persists. The best thing is that the business owners are also ready to pay for the business solutions. So, we see machine learning is already beyond limitations.
Another group is software developers. They care about machine learning so that they could build good applications to solve business problems. The applications as we saw in the machine learning model are dependent upon predictive models created out of machine learning algorithms to make better predictions. So software developers do not actually need to worry about the models or they do not have to really be an expert on models, using machine learning they can build smart applications which just use those models.
The third group of people is data scientists who actually need effective and handy tools. Data scientists are the people who really care and have a deep knowledge of statistics, machine learning tools/software’s, who really are domain experts. Data scientists are very limited and expensive because again they help businesses grow by solving business problems and growing business means saving and earning more money. So data scientists in themselves are a value to any business, as they are a complete package of knowledge, statistics, problem domain and machine learning. There is another group that we did not talked about, and that is vendors who make machine learning offerings and services.
Since machine learning is referred to many times as predictive analysis, there are vendors who offer analytics, there is SAAS, RapidMiner and others too who are the part of this space.
Machine learning is more of an added value on to conventional analytics of data mining in most of the ways. That’s the reason why companies like SAP, Oracle, Microsoft and IBM offer such machine learning products because they know that a big money is involved in this space, and they know that this is in and part of growing market. On the other hand when we see the companies like Amazon, Microsoft providing cloud offerings, we see the machine learning offering already living there.
R Programming
"R" is a programming language an environment, the beauty is that it is an open source programming language. Following is the definition from r-project
R is a language and environment for statistical computing and graphics. It is a GNU project which is similar to the S language and environment which was developed at Bell Laboratories (formerly AT&T, now Lucent Technologies) by John Chambers and colleagues. R can be considered as a different implementation of S. There are some important differences, but much code written for S runs unaltered under R.
R provides a wide variety of statistical (linear and nonlinear modelling, classical statistical tests, time-series analysis, classification, clustering, …) and graphical techniques, and is highly extensible. The S language is often the vehicle of choice for research in statistical methodology, and R provides an Open Source route to participation in that activity.
The definition is self-explanatory about this language, R supports machine learning, statistics computation and a lot more. It contains packages to solve various sorts of machine learning problems. R is supported by a lot of commercial machine learning offerings. R is not alone, but Python also is equally popular in this area and also contains packages more or less similar to R to solve business problems.
Conclusion
To handle an intelligent conversation and to have a high level knowledge of machine learning, we acquired theoretically a good knowledge. We can say that using machine learning we can find patterns, and then create a model that recognizes those patterns with new iterative data, and last but not the least, machine learning can really help any organization to grow.
References
- R-Project
- Pluralsight Course - Understanding Machine Learning
