{"id":30873,"date":"2020-11-30T21:10:23","date_gmt":"2020-12-01T03:10:23","guid":{"rendered":"http:\/\/pzd.hmy.temporary.site\/?p=30873"},"modified":"2020-11-30T21:10:23","modified_gmt":"2020-12-01T03:10:23","slug":"introduction-for-decision-tree","status":"publish","type":"post","link":"https:\/\/datascienceplus.com\/introduction-for-decision-tree\/","title":{"rendered":"Introduction for Decision Tree"},"content":{"rendered":"

Decision Tree falls under supervised machine learning, as the name suggests it is a tree-like structure that helps us to make decisions based on certain conditions. A decision tree can help us to solve both regression and classification problems.<\/p>\n

What is Classification?<\/h3>\n

Classification is the process of dividing the data into different categories or groups by giving certain labels. For Example; categorize the transaction data based on whether the transaction is Fraud or Genuine. If we take the present epidemic as an example based on the symptoms like fever, cold and cough we categorize the patient as suffering from covid or not.<\/p>\n

What is Regression?<\/h3>\n

Regression is a process to get the predictions which is a continuous value. For example; prediction the weight or predicting the sales or profit of the company etc.
\nA gentle introduction to the Decision tree:
\nA decision tree is a graphical representation that helps us to make decisions based on certain conditions. <\/p>\n

For Example; making a decision whether to watch a movie or not.
\n\"\"<\/a><\/p>\n

Important terminology in the decision tree:<\/h3>\n

Node:<\/b><\/p>\n

A decision tree is made up of several nodes:<\/p>\n

1.Root Node: A Root Node represents the entire data and the starting point of the tree. From the above example the
\n First Node where we are checking the first condition, whether the movie belongs to Hollywood or not that is the
\n Rood node from which the entire tree grows
\n 2.Leaf Node: A Leaf Node is the end node of the tree, which can\u2019t split into further nodes.
\n From the above example \u2018watch movie\u2019 and \u2018Don\u2019t watch \u2018are leaf nodes.
\n 3.Parent\/Child Nodes: A Node that splits into a further node will be the parent node for the successor nodes. The
\n nodes which are obtained from the previous node will be child nodes for the above node. <\/p>\n

Branches:<\/b><\/p>\n

Branches are the arrows which is a connection between nodes, it represents a flow from the starting\/Root node to the leaf node.<\/p>\n

How to select an attribute to create the tree or split the node:
\nWe use criteria to select attribute which helps us to split the data into partitions.<\/p>\n

Here are the most important and useful methods to select the node for splitting the data <\/p>\n

Information Gain:<\/b><\/p>\n

In the process of selecting an attribute that gives more information about the data, we select the attribute for splitting further from which we get the highest information gain. For calculating Information gain we use matric
\nEntropy.<\/p>\n

\t Information from attribute = \u2211p(x). Entropy (x)<\/p>\n

Here, x represents a class in the attribute
\nInformation Gain for any attribute = total entropy \u2013 Information from attribute after splitting
\nEntropy:
\nEntropy is used to measure the Impurity and disorder in the dataset<\/p>\n

\t\t\tEntropy = – \u2211 p(y). log2 p(y)<\/p>\n

Here, y represents the class in the target variable<\/p>\n

Gini Index:<\/b><\/p>\n

Gini Index is also called Gini Impurity which calculates the probability of an attribute that is randomly selected.<\/p>\n

R code<\/h2>\n

The dataset that we are looking into:
\n\"\"<\/a><\/p>\n

We are going to build a decision tree model to decide whether to play outside or not.
\nNow we will build a decision tree on the above dataset.
\nTo build a decision tree, first we have to select an attribute that gives highest information among all the attributes.<\/p>\n

Calculating Total Entropy<\/h3>\n
\r\nView(dataset)\r\n## changing the data into factors type \r\ndata = data.frame(lapply(dataset,factor))\r\nsummary(data) ## summary of the data \r\n### Claculating Total Entropy\r\ntable(data$Play)\r\n## p(Yes)*log2 p(Yes)-p(No)*log2 p(No)\r\nTotalEntropy= -(9\/14)*log2(9\/14)-(5\/14)*log2(5\/14)\r\nTotalEntropy\r\n> View(dataset)\r\n> ## changing the data into factors type \r\n> data = data.frame(lapply(dataset,factor))\r\n> summary(data) ## summary of the data \r\n     Outlook  Temperature   Humidity     wind    Play  \r\n Overcast:4   Cold:4      High  :7   Strong:6   No :5  \r\n Rainy   :5   Hot :4      Normal:7   Weak  :8   Yes:9  \r\n Sunny   :5   Mild:6                                   \r\n> ### Claculating Total Entropy\r\n> table(data$Play)\r\n\r\n No Yes \r\n  5   9 \r\n> ## p(Yes)*log2 p(Yes)-p(No)*log2 p(No)\r\n> TotalEntropy= -(9\/14)*log2(9\/14)-(5\/14)*log2(5\/14)\r\n> TotalEntropy\r\n[1] 0.940286\r\n<\/em>\r\n<\/pre>\n
Calculate Entropy for each class in Outlook and Information Gain for the Outlook<\/p>\n
\r\ntable(data$Play)\r\n## p(Yes)*log2 p(Yes)-p(No)*log2 p(No)\r\nTotalEntropy= -(9\/14)*log2(9\/14)-(5\/14)*log2(5\/14)\r\nTotalEntropy\r\n## filtering Outlook data to calculate entropy\r\nlibrary(dplyr)\r\n## Calculate Entropy for Outlook\r\nOutlook_Rainy = data.frame(filter(select(data,Outlook,Play),Outlook=='Rainy'))\r\n\r\nView(Outlook_Rainy)\r\nEntropy_Rainy = -(3\/5)*log2(3\/5)-(2\/5)*log2(2\/5)\r\nEntropy_Rainy\r\n\r\nOutlook_Overcast = data.frame(filter(select(data,Outlook,Play),Outlook=='Overcast'))\r\nView(Outlook_Overcast)\r\nEntropy_Overcast=-(4\/4)*log2(4\/4)-0 ## since we don't have any No values\r\nEntropy_Overcast\r\n\r\nOutlook_Sunny = data.frame(filter(select(data,Outlook,Play),Outlook=='Sunny'))\r\nView(Outlook_Sunny)\r\nEntropy_Sunny = -(2\/5)*log2(2\/5)-(3\/5)*log2(3\/5)\r\nEntropy_Sunny\r\n\r\n# calculating Information for outlook\r\n### Info = summation(p(x)*Entropy(x))\r\n\r\nOutlook_Info = ((5\/14)*Entropy_Rainy)+((4\/14)*Entropy_Overcast)+((5\/14)*Entropy_Sunny)\r\nOutlook_Info\r\n## Information gain\r\n## Info_gain = Total Entropy- Outlook_info\r\nInfo_gain1 = TotalEntropy - Outlook_Info\r\nInfo_gain1\r\n> table(data$Play)\r\n\r\n No Yes \r\n  5   9 \r\n> ## p(Yes)*log2 p(Yes)-p(No)*log2 p(No)\r\n> TotalEntropy= -(9\/14)*log2(9\/14)-(5\/14)*log2(5\/14)\r\n> TotalEntropy\r\n[1] 0.940286\r\n> ## filtering Outlook data to calculate entropy\r\n> library(dplyr)\r\n> ## Calculate Entropy for Outlook\r\n> Outlook_Rainy = data.frame(filter(select(data,Outlook,Play),Outlook=='Rainy'))\r\n> View(Outlook_Rainy)\r\n> Entropy_Rainy = -(3\/5)*log2(3\/5)-(2\/5)*log2(2\/5)\r\n> Entropy_Rainy\r\n[1] 0.9709506\r\n> Outlook_Overcast = data.frame(filter(select(data,Outlook,Play),Outlook=='Overcast'))\r\n> View(Outlook_Overcast)\r\n> Entropy_Overcast=-(4\/4)*log2(4\/4)-0 ## since we don't have any No values\r\n> Entropy_Overcast\r\n[1] 0\r\n> Outlook_Sunny = data.frame(filter(select(data,Outlook,Play),Outlook=='Sunny'))\r\n> View(Outlook_Sunny)\r\n> Entropy_Sunny = -(2\/5)*log2(2\/5)-(3\/5)*log2(3\/5)\r\n> Entropy_Sunny\r\n[1] 0.9709506\r\n> Outlook_Info = ((5\/14)*Entropy_Rainy)+((4\/14)*Entropy_Overcast)+((5\/14)*Entropy_Sunny)\r\n> Outlook_Info\r\n[1] 0.6935361\r\n> ## Information gain\r\n> ## Info_gain = Total Entropy- Outlook_info\r\n> Info_gain1 = TotalEntropy - Outlook_Info\r\n> Info_gain1\r\n[1] 0.2467498\r\n<\/em>\r\n<\/pre>\n
Same way calculating entropy and Information gain for all remaining columns<\/p>\n
\"\"<\/a><\/p>\n
From the Above table Outlook has highest information gain, so the first attribute at the root node is Outlook<\/p>\n
\"\"<\/a><\/p>\n
From the above diagram, we can observe that Overcast has only the Yes class. So, there is no need for further splitting. But for the Rainy and Sunny Contains both Yes and No. Again the same process repeats.<\/p>\n
Till now, we have seen the manual process<\/p>\n
Here is the R code for Building a Decision Tree Model using C5.0 function and plot of the Decision Tree.<\/p>\n
\r\nlibrary(C50)\r\n\r\n## Syntax  C5.0(Input_Columns, Target)\r\n\r\nmodel = C5.0(data[,1:4],data$Play)\r\nplot(model)\r\n<\/pre>\n
\"\"<\/a><\/p>\n
Resource Article: simple-linear-regression<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
Decision Tree falls under supervised machine learning, as the name suggests it is a tree-like structure that helps us to make decisions based on certain conditions. A decision tree can help us to solve both regression and classification problems. What is Classification? Classification is the process of dividing the data into different categories or groups […]<\/p>\n","protected":false},"author":6265,"featured_media":30951,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[459],"class_list":["post-30873","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-basic-statistics","tag-decision-trees"],"views":11712,"_links":{"self":[{"href":"https:\/\/datascienceplus.com\/wp-json\/wp\/v2\/posts\/30873","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/datascienceplus.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datascienceplus.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datascienceplus.com\/wp-json\/wp\/v2\/users\/6265"}],"replies":[{"embeddable":true,"href":"https:\/\/datascienceplus.com\/wp-json\/wp\/v2\/comments?post=30873"}],"version-history":[{"count":0,"href":"https:\/\/datascienceplus.com\/wp-json\/wp\/v2\/posts\/30873\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/datascienceplus.com\/wp-json\/wp\/v2\/media\/30951"}],"wp:attachment":[{"href":"https:\/\/datascienceplus.com\/wp-json\/wp\/v2\/media?parent=30873"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datascienceplus.com\/wp-json\/wp\/v2\/categories?post=30873"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datascienceplus.com\/wp-json\/wp\/v2\/tags?post=30873"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}