Types of Machine Learning Explained: Supervised, Unsupervised and Reinforcement Learning

Introduction into the field of Machine Learning.

Machine learning has grown to be a key element in today’s tech landscape which in turn is transforming how companies go about their day to day functions, how we as a society access info and how digital products attend to user needs. From sorting out email spam, product recommendations, security systems against fraud, to automated customer service, machine learning can get machines to carry out complex processes which in other settings would need a very large amount of specific coding. Also a part of artificial intelligence that makes it such that computers can train themselves from data given them, to recognize relationships between different elements and do better over time which doesn’t include a programmer to give the computer detailed instructions for each and every instance. As opposed to the classic set of fixed rules, in machine learning the model uses lots of different cases, statistics, and by the process of learning from experience is able to come up with what will most likely happen in new cases. As more and more players in the tech world are adopting artificial intelligence into their work, at the same time also seeing the value in knowing the fundamentals of what is the category of machine learning out there — this has become a very important issue for students, professionals, business owners, and tech enthusiasts.

Supervised learning, unsupervised learning, and reinforcement learning. Although all of these approaches fall under the umbrella of machine learning, what they do and how they do it is very different. Supervised learning which uses labeled examples to predict outcomes, unsupervised learning which identifies patterns in data without the use of predefined labels, and reinforcement learning which learns from interaction with an environment and produces actions which are either rewarded or penalized. Also each of these has its pros and cons which makes some better suited for certain tasks. What also plays a role is that each has different applications and which to use is dependent on what you are trying to achieve. In this article we look at how each of these approaches works, we also look at real world applications of them and what factors into which one to choose.

Forms of Machine Learning 

The present three primary forms of machine learning which in fact categorize the different manners computer systems learn from provided data or experience. In supervised learning systems learn that are presented with relationships of inputs which are given some certain outputs, unsupervised learning in this case looks to research data sets which in turn present unknown elements, then develop models to explain those issues and finally reinforcement learning we see where the focus is developed on creating a decision making model by the system as it reports and learns from continuous interaction with the environment. Also that these are not really just different terms for the same type of function rather each includes a distinct what the system is to achieve, what is the process to go about it, and how to evaluate success. For example a financial institution may use supervised learning which has already categorized some past transactions as fraud or non-fraud to predict future fraudulent activities. Also a store may use an unsupervised learning approach to find out what are the different customer segments which are not at present known. Also we have robots that are put into a new environment which will use reinforcement learning to better its performance based on the results of the actions it takes.

Which approach to take in a machine learning project depends on the issue at hand, the quality and amount of data you have, and what results you are looking to achieve. In supervised learning we see that it is very useful in cases where we have access to large sets of historical data which we also have the correct answers to. Unsupervised learning on the other hand is best for when a company has large data sets which they do not yet know how to classify or what relationships exist within them. Also when a system has to make a series of decisions which will in turn produce a certain outcome that is to be learned from, reinforcement learning is the way to go. Also it is not at all atypical for some real world projects to use a combination of these approaches. For example, a recommendation system may use supervised learning to predict what a customer will buy and unsupervised learning to put that customer in a group of people with similar buying habits. By recognizing these facts we build a better base for how machine learning can be applied to real world decision making.

Supervised Learning: Learning from Labeled Data.

Supervised learning is a type of machine learning which uses labeled training data. A labeled dataset includes examples of inputs with their corresponding correct outputs. During the training phase the algorithm goes over these examples which in turn allows the model to learn a relationship between the input data and the right results. The goal is to use that relationship to make accurate predictions with new data the model hasn’t seen before. For instance a model which is put together to identify fraudulent bank transactions may be given info on transaction amounts, locations, times, account action, and past fraud reports. By looking at the characteristics of these transactions which led to the identified fraud the model puts together a picture which it uses to determine if future transactions are fraudulent. While the models aren’t always 100% correct in their predictions a well trained model can greatly improve an organization’s ability to identify suspicious activity as compared to doing it all via manual review.

Supervised learning is very much at the mercy of the quality and representativeness of its training data. If the data includes wrong labels, leaves out important cases, or has historical biases the resulting model will put out poor predictions. Also models must be put to the test with data which did not feature in the training set to see if they perform well in new and unknown situations. By task different developers may report accuracy, precision, recall or other performance metrics. In the case of fraud detection for example it is important to identify fraudulent transactions but at the same time we don’t want to flag up low risk transactions which will just end up being innocent. Thus an appropriate evaluation method must include the 2 which types of error are more serious rather than report only a single performance score.

What is Supervised Learning about?

Supervised learning we see as a process which starts with the collection and preparation of a dataset that includes input variables and their known results. As developers work through the data they clean it, impute as needed for missing values, choose which features to use, and break the data into training and evaluation sets. The algorithm we put in play then uses the training data to fine tune its internal parameters which in turn improves the accuracy of the predictions. This is a guide with a loss function that measures the model’s performance by the difference between what it predicts and what is actually seen. We train the algorithms to reduce this error while at the same time we do not want the model to become too complex which may cause it to do poorly on data it has not seen before. Once training is finished we put the model to the test with separate examples, and may go back to the drawing board with the model’s design or parameters if the results are not what we are looking for. After we are happy with the results of the testing the model is put out into the world to make predictions on new information.

Classification and regression. In classification we put things into categories like which emails are spam, which transactions are fraudulent, and which customers are going to cancel their subscription. In regression we are after a number: will it rain tomorrow, what the property price will be, at what time the delivery will arrive. Also included in this are common supervised learning algorithms which we use decision trees, logit regression, linear regression, random forests, support vector machines, and neural networks. What algorithm you choose depends on the make up of your data, the issue’s complexity, how easy it is to explain what the model is doing and also what computing resources you have at your disposal. No matter which algorithm you pick, the main idea is the same: the model is trained on labeled data and then used to put new cases into action.

Examples of Supervised Learning.

Fraud detection is a very large field for supervised learning which is enabled by the fact that financial institutions keep history of investigated and classified transactions. A model may use features like transaction frequency, payment amount, device info, and what is out of the ordinary for that account’s activity. As new transactions come in the system puts out a probability that it is fraud which in turn may put some cases in a queue for more look at. This can help financial institutions to better focus their investigations, react to new risks, and see a reduction in losses. At the same time fraud trends do change over time and at times we may not have confirmed labels for fraud right away. What we see is that companies must track how the models are doing, update the training data, and put in place automatic prediction which works with the right security measures.

Supervised learning also includes recommendation systems, health care analysis, customer service, and demand forecasting. An online retailer may train a model which uses past interactions, product info, and purchase results to put forth which products a customer may also be interested in. A hospital may use — within proper governance structures large sets of medical data to predict certain health outcomes, which in turn supports staff in their decision making. Also we see that companies can predict which customers will leave, what the future revenue will be, and which support issues are which. These examples show out that when you have history which includes the results you are trying to predict, supervised learning does very well. That said, the approach breaks down when labeled data is at a premium, very expensive to get, or tough to validate. In such cases unsupervised learning or other methods may present a better option.

Unsupervised Learning: Identifying Invisible Trends in Data.

Unsupervised learning is a type of machine learning which brings out structures, similarities and relationships in data without the use of pre determined output labels. The model does not learn from examples which have correct answers included, instead it looks at what is present to determine how each record plays with the other. This process is to identify what natural groups are present, what the outliers are, what the recurring actions are, or what relationships existed which may not have been known before analysis. For instance a business may have access to thousands of customer records which include purchase history, what they browsed, product preferences and spending trends. If the business does not have a set out way to put its customers into groups it can use an unsupervised learning algorithm to put together groups which have similar traits. These results in turn will support marketing decisions, service improvements, product development and also further research.

The issue of identifying what are to be meaningful relationships without the use of predetermined labels is at the heart of pattern discovery which in turn is a broad term for the task of finding out useful regularities and structures within information. In machine learning we have unsupervised techniques which allow researchers and businesses to look at large data sets before they determine which issues to look into in great detail. But a found pattern is not at once meant to be taken as meaningful, accurate or useful. Algorithms may put forward groups which are a result of how the data was collected and not of true customer behavior and also some reported relationships may be random. Human input, statistical validation, and knowledge of the application area still play a large role in what we make of the results. Unsupervised learning thus is best put forth as a tool for data exploration and insight generation as opposed to a sure fire way that every put forth structure has practical value.

What is Unsupervised Learning?

Unsupervised learning starts out with a data set which has input variables in it but doesn’t present the right answer for each example. Developers prepare the data, pick out which features to use and which algorithm best suits what they are trying to achieve. The model looks at what is similar, what is different, the distribution, or what are the relationships within the data. Per model type this may see which records are similar and put them in groups, reduce the number of dimensions required to represent the data, or find which observations are very different from the rest. As there are no pre given labels which are the correct answers this issue presents a greater challenge when it comes to evaluating an unsupervised model as compared to a supervised model. Developers may use cluster quality measures, do stability tests, use visualisation, apply domain expertise, or look at business results which come out of it to determine if the results are useful.

Several types of algorithms are put forward in unsupervised learning. Clustering which is a practice of putting observations into groups based on their similarities, and dimensionality reduction which transform data into a more manageable set of important features. Also we see association rule learning which is to identify items or events that tend to happen together and anomaly detection which is for the identification of the odd man out that require further research. Popular methods include K-means clustering, hierarchical clustering, DBSCAN, principal component analysis and some autoencoder based approaches. Each method brings its own set of issues and benefits. For instance K means requires you to specify the number of clusters to look for which also may not be known in advance and also is at the mercy of the scale of the input features. Dimensionality reduction does make the data easier to work with but also may get rid of important information. It is a great challenge to choose the right technique which is based on what your data is like and what you are trying to achieve from the analysis.

Examples of Unsupervised Learning.

Customer segmentation is a primary business application of unsupervised learning. We see that companies have a wide range of customers which have different needs, interests, purchasing trends, and levels of interaction. Instead of a one size fits all approach a business may use clustering to determine groups of like minded customers. We may find that one group is into very affordable products, another into premium items, and a third into heavy navigation which in turn results in no purchase. From these we may develop more relevant promotions, improve the overall customer experience, and better put marketing resources to use. Also these groups are not permanent as customer behavior does change over time. It is thus recommended that businesses review their segments at regular intervals and not to base assumptions of individual customers’ actions totally on what group they are in.

Unsupervised learning also plays a role in cybersecurity, manufacturing, document analysis, and recommendation systems. We see that in security which an anomaly detection model puts out of what is out of the ordinary which in turn allows security teams to take a look at what may be a threat. In manufacturing atypical sensor reports may point to that some equipment may need a closer look but we also have to do more research to see what is the issue. Also for recommendation systems we see that they use grouping and identifying similar features in what certain groups of users like or products that share the same traits. Also we note that researchers use dimensionality reduction to put in simpler terms very complex sets of data which in turn makes them easier to present. These are examples of how unsupervised learning helps organizations to go through information which we don’t have pre-determined answers to. At the same time though it is also important to note that the results of this type of learning often require very close look into what is presented and that what is found does not in fact prove cause and effect.

Reinforcement Learning: Learning from Feedback which includes Rewards and Penalties.

Reinforcement learning is a machine learning field in which an agent learns from interaction with the environment. As opposed to having a set of labeled answers at hand or the ability to determine groupings within present info, the agent puts forth actions and gets a response regarding the results of said actions. This response is usually in the form of a numeric reward or penalty which in turn helps the agent to figure out what behaviors play out best in the long term. For example a robot which is learning to get around a room will see positive input for getting to the destination in an efficient manner and negative for issues like crashing into things or unproductive movement. Over time through many interactions the agent develops a policy which is a plan for what action to take in what situation. Also the goal is usually to do what is best for the total reward over time as opposed to what may present the greatest return from a single action.

Reinforcement learning has a number of key elements. An agent in this context is a decision making system, at the same time there is the environment into which it interacts. A state reports to the agent what the present situation is, and an action is what the agent does. A reward reports back to the agent of how well an action did, also we have a policy which is the agent’s strategy for what to do next. Also some models will use a value function which is a way to estimate the long term benefit of being in a certain state or performing a certain action. These elements put together are what allow reinforcement learning to work in situations where present decisions also play a role in what options are available in the future. But also it is a fact that for learning to happen agents must interact a great deal and also it is possible for an agent to develop what we do not want it to do if the reward it is given does not truly represent what we want it to achieve.

What is Reinforcement Learning about?

In a typical reinforcement learning setting an agent reports to the present state and from that chooses an action as per its policy. The environment in turn changes to a new state and gives out a reward or penalty. The agent notes this interaction and uses it to improve its strategy which in turn causes it to learn what actions may lead to better results over time. This goes on for many iterations till the agent has a policy which does well enough at what we want it to do. Also unlike supervised learning which requires the right answer for each problem, in reinforcement learning the agent has to find out what is good behavior via feedback, exploration, and evaluation. Also this approach is very much to the point when actions play out over many steps.

An action may not show its value at first but in the long term it may prove very beneficial. In what ways do Reinforcement learning algorithms like Q learning and policy gradient methods which they do is what they present. Also in different ways do these algorithms address the issues at hand. What deep reinforcement learning does is it puts reinforcement learning together with deep neural networks which in turn enables us to deal with complex data or large action sets. As training is resource intensive and somewhat of a hit or miss process, developers tend to use simulations, controlled settings and safety features before putting a trained model out into the real world.

Uses of Reinforcement Learning.

Robotics also presents a good case of how reinforcement learning plays out in complex decision making. A robot for example may have to learn the action which best moves an object, which path to take around barriers, how to preserve balance, or what is the best way to co-ordinate its actions as it goes about a task. We as developers can put the robot through a simulation where it gets feedback on its performance. Once the model displays appropriate behavior we put it to the test in real world settings which we control. Simulation also reduces the cost of experiment and at the same time minimizes physical risk during the early stages of training. That said, what is learned in a sim may not translate perfectly to the real world due to issues like different levels of friction, lighting, sensor accuracy, or mechanical reactions. Thus it is a very careful validation which is required before we put these reinforcement learning systems into the field where failure may damage equipment or put people in danger.

Reinforcement learning is used in a variety of fields including resource allocation, traffic management, industrial control, and certain recommendation and advertising systems. For instance an energy management system may learn to what degree it should adjust equipment settings which in turn balances energy use, operating costs, and demand. Also a traffic control system may learn which signal strategies to use that will in turn reduce traffic congestion across a network of roads. In recommendation systems reinforcement learning may also play a role in which recommendations to put forth based on how the user reacts which in turn influences future decisions. But in each of these uses care must be taken in the design of the reward system and in what is monitored. A system which only rewards for immediate action may in fact prioritize attention getting content which in turn may not be in the best interest of the user in the long run. Also an energy control that only looks at short term savings may in fact do a disservice to other very important operational issues. Reinforcement learning works best when the rewards structure reflects the full range of the task’s goals and constraints.

Key Features of Different Types of Machine Learning.

Supervised, unsupervised, and reinforcement learning is for the most part different in terms of the type of feedback they get as well as what they are trying to achieve. In supervised learning we use labeled examples to train a model for input to output mapping. In unsupervised learning we work with data that does not have pre-defined results and we try to put forth a structure which is relevant. Reinforcement learning is one in which learning happens from performed actions’ feedback from the environment, usually we see that the aim is to do what will produce the highest aggregate reward over time. Also these differences play a role in what kind of data is used, how the training is conducted, the ways in which we evaluate the models, and which issues each approach is best at. While all of these are part of machine learning, it is not enough to choose which based on which is the most popular or which is the most advanced technologically. What you should do is pick the one that best fits the issue at hand.

The below table we present which also includes the main features and uses.

In practice what we see is that the difference between which approaches to use becomes more apparent when the same business issue is looked at from different angles. A retailer may use supervised learning to predict which customer will buy what product, unsupervised learning to identify customer groups, and reinforcement learning to improve the product recommendation sequence. Also a financial institution may train a supervised model to put past fraud transactions into categories and use unsupervised methods to look at atypical transaction trends. Reinforcement learning may play a role in a different kind of issue related to a series of decisions and long term goals but it isn’t to say it is the best for all types of fraud classification. In determining the issue at hand, the data at your disposal, and the results of what goes wrong is more important than in which branch of AI you put your solution.

How to Pick the Right Machine Learning Approach.

Selecting the right machine learning technique is a function of clearly defining the issue at hand. Businesses and developers must determine the result they are after, the type of data at their disposal, and what decisions the model will make. If past examples are present which include correct answers and the goal is to predict new outcomes, supervised learning is the way to go. If the goal is to look at a set of data, to find groups of customers or to see what is out of the ordinary without predetermined labels, then unsupervised learning may be what you use. If the system has to improve via interaction and the quality of its decisions is based on long term results, reinforcement learning may be what you require. These points also help in the avoidance of the common error of over complicating a solution which in fact requires a more simple approach.

Other issues to note are around data quality, available computing resources, development costs, interpretability, privacy and the risks of poor decisions. In supervised learning we put in a great deal of effort into accurate data labeling, in unsupervised we may get out hard to analyze and validate results. Also it is reported that for unsupervised results we may have to do extensive work to make sense of what we get out, and for reinforcement we may spend a great deal of time designing the reward function. Also it is worthwhile for organizations to look at whether simpler stats methods or rules based systems will serve their purpose before they jump into a machine learning project. Once a model is picked out it should be looked at in terms of how it performs against relevant metrics, monitored after going live, and made better when the data or environment changes. In high impact applications humans in the loop and appropriate safety measures are even more so a must. For a machine learning project to be a success it is not just the algorithm that matters but also having good data, sound evaluation, responsible rollout, and continuous improvement.

How the different types of Machine Learning can work together

Although put forth separately, supervised, unsupervised, and reinforcement learning in practice are often used together. A company will begin by using unsupervised learning for its value in exploring customer data and present a picture of different market segments. Then out of those it may implement supervised learning to predict which of the customers in each group will perform best as a customer, if sufficient labeled examples present themselves. At a later time if the need is to actually have the business actors play out a scenario and get better at it over time that is when reinforcement learning comes into play. That said, combining the methods while it does produce a more full featured solution, also increases the development, testing, and maintenance complexity. Each element has to add something to the table and the whole system must be looked at in terms of how well it supports the business’ goals.

In another example we see in the field of cybersecurity which also uses a combination of supervised classification with anomaly detection. A supervised model which is fed labeled examples is able to identify what are known categories of malicious activity, at the same time an unsupervised method may put forth that which does not fit into the usual patterns. Analysts look at this data, which comes out of the models, to determine if what is presented is indeed a threat and also to use what is proven to be a case of a threat to improve what goes into the training data. Also we see reinforcement learning which plays a role in some adaptive control tasks although it’s a which plays out that its use is dependent on the environment and the risks of the experiment. The large picture is that these three approaches are in fact best used as complimentary tools which do not necessarily have to be used in isolation from each other. By studying what each of them do best and also what they do not do so well, organizations may put together machine learning solutions which address different elements of an issue at hand and also at the same time maintain proper watch over what is going on and evaluate results.

Issues in Machine Learning.

Although there is great promise in machine learning, it also brings out issues which in turn may question the output’s accuracy and value. Data quality is the primary issue we face as poor data results from missing info, incorrect entries, different formats, and non-representative samples all of which can degrade model performance. In supervised learning we see that if past data used for training has bias then the model will too, also when feature selection is poor in unsupervised models we get into faulty groupings. Also in reinforcement learning we note that if what is being rewarded is not the true objective then the behavior learned may not be what was intended at all. These issues put forth the fact that machine learning is not a black box solution which always gives right answers. What we as developers have to do is know our data, check our assumptions, put forward other perspectives which may explain things in another way, and look at how errors play out for the end users or organizations that will be affected by the results.

Other issues are around compute resources, privacy issues, and that which complex models put forth are hard to explain. That which we see perform well in a test may in live settings prove to be of lower accuracy. What we see is that which does well in tests may not do so once customer behavior, fraud trends, or operational settings have changed. Thus it is that organizations should run continuous watch on live systems, protect personal info, document key decisions, and put in place what is correct for failure. Also of value is human input which in large part plays into financial decisions, health care, public services, or what which affect physical safety. While machine learning does bring speed and consistency to analysis, it also does not do away with the need for accountability and good judgment. In the roll out of these solutions we see a need for a balance between auto and watchful human input which is to not let the benefits of predictive and adaptive tech go to waste due to avoidable risks.

Conclusion 

Supervised learning which includes unsupervised and reinforcement learning is a taxonomy of how we train machines from data and experience. In supervised learning we use labeled examples to predict outcomes and it does very well in fraud detection, classification and forecasting. Unsupervised learning looks at data without predetermined target labels to put forth patterns, to group similar observations together and to identify what is out of the ordinary which in turn is very useful in customer segmentation and exploratory analysis. Reinforcement learning is on the other hand about learning what actions to take via interaction with an environment which gives feedback in terms of rewards and penalties which in turn play out in robotics, control systems and sequential decision making. Each of these approaches has its own set of benefits but also issues which must be taken into account before putting them to use.

These differences which we study better enable students, business owners, developers, and technology enthusiasts to make informed decisions related to machine learning projects. What works best is based on the issue at hand, the data available, the result we are after, and the issues of which we must be aware if we get it wrong. In some tasks we require reliable labeled examples, in others we benefit from that which is unknown and which the model may discover, and some we put to the model which must learn from a series of decisions and long term feedback. In many cases we see value in combining approaches which each serve a clear purpose. As machine learning continues to grow we will see which and how to apply these three approaches as an important skill in the recognition of artificial intelligence and in the responsible use of it.