{"id":5203,"date":"2019-11-01T18:57:33","date_gmt":"2019-11-01T13:27:33","guid":{"rendered":"https:\/\/code4developers.com\/?p=5203"},"modified":"2019-11-01T18:57:33","modified_gmt":"2019-11-01T13:27:33","slug":"exploratory-data-analysis","status":"publish","type":"post","link":"https:\/\/code4developers.com\/exploratory-data-analysis\/","title":{"rendered":"Exploratory Data Analysis"},"content":{"rendered":"

Machine learning is an application of AI(Artificial Intelligence) that makes computers to learn themselves from given data without being explicitly programmed. Now days computers are much powerful that they can easily be trained with much amount of data with so much minimum time. As a data scientist it is also mandatory that one have to know how the data is varying, how the data is categorized and how distributed. With the help of Exploratory Data Analysis(EDA) we get conclusions about the data that human can observe with the help of graphs, charts and values.<\/p>\n

Definition<\/span><\/h4>\n

Exploratory Data Analysis refers to the critical process of performing initial investigations on data so as to discover patterns,to spot anomalies,to test hypothesis and to check assumptions with the help of summary statistics and graphical representations.<\/p>\n

Explanation of EDA with sample iris dataset:<\/h4>\n

I am taking iris dataset as a sample dataset and performing EDA. Iris dataset contains four features:<\/p>\n

    \n
  1. \n
      \n
    1. sepal_length<\/li>\n
    2. sepal_width<\/li>\n
    3. petal_length<\/li>\n
    4. petal_width and 3 classes\n
        \n
      1. setosa<\/li>\n
      2. verginica<\/li>\n
      3. versicolor<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n

        Please click\u00a0here<\/a> to get more information about iris dataset.<\/p>\n

        First step is to import required libraries and than read data files.<\/strong><\/p>\n

        import pandas as pd\r\nimport seaborn as sns\r\nimport matplotlib.pyplot as plt\r\nimport numpy as np\r\niris = pd.read_csv(\u201ciris.csv\u201d)<\/pre>\n
        Now we have to show number of raws and columns in dataframe, shape provides that functionality. After that we have to figure out which columns that dataframes contains. dataframe.columns returns list of columns that dataframe contains.<\/p>\n
        print(\u201cShape of dataframe is\u201d, iris.shape)\r\nprint(\u201cColumns of dataframe are\u201d , iris.columns)<\/pre>\n
        <\/div>\n
        \"Exploratory<\/div>\n
        <\/div>\n
        We have to observe how the data is, so we have to display initial first raws.head() function provides that functionality.<\/p>\n
        \n
        iris.head()<\/pre>\n
        \"DA<\/p>\n
        Data contains null values , so we have to fill that null blocks with some values. So first of all we have to figure out how much null each columns contains.<\/p>\n
        iris.isna().sum()<\/pre>\n<\/div>\n
        \"DA<\/p>\n
        Here species is target column , means we have to classified data in spices. So we have to observe which unique species exists in data frame.<\/p>\n
        iris[\u201cspecies\u201d].value_counts()<\/pre>\n
        \"Exploratory<\/p>\n
        2 D Scatter plot<\/strong><\/h4>\n
        Two-dimensional scatterplots visualize a relation (correlation) between two variables X and Y . Individual data points are represented in two-dimensional space, where axes represent the variables . The two coordinates that determine the location of each point correspond to its specific values on the two variables.<\/p>\n
        sns.set_style(\u201cwhitegrid\u201d);\r\nsns.FacetGrid(iris, hue=\u201dspecies\u201d, size=4) \\\r\n.map(plt.scatter, \u201csepal_length\u201d, \u201csepal_width\u201d) \\\r\n.add_legend();\r\nplt.show();<\/pre>\n
        \n
        \n
        \n
        \"Exploratory<\/div>\n
        Here different different species are displayed by different different colours. We can conclude that with the help of 2d scatter plot we can observe how different different categories are seprated from each other by two continuous features. So we have to draw multiple 2d scatterplot between each attribute. Pairplot solve this problem ! Pairplot plots 2D scatter plot between each 2 attrbutes.<\/div>\n<\/div>\n<\/div>\n<\/figure>\n
        sns.set_style(\u201cwhitegrid\u201d);\r\nsns.pairplot(iris, hue=\u201dspecies\u201d, size=3);\r\n\r\n\r\nplt.show()<\/pre>\n
        \n
        \n
        \n
        \n
        \"Exploratory<\/div>\n
        <\/div>\n
        Observations:<\/strong><\/div>\n
        petal_length and petal_width are the most useful features to identify various flower types. While Setosa can be easily identified (linearly seperable), Virnica and Versicolor have some overlap (almost linearly seperable). We can find \u201clines\u201d and \u201cif-else\u201d conditions to build a simple model to classify the flower types.<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/figure>\n
        <\/div>\n
        Histogram<\/span><\/div>\n
        \n
        \n
        \n
        A\u00a0histogram is an accurate representation of the distribution of numerical data. It is an estimate of the probability distribution of a continuous variable. It differs from a bar graph, in the sense that a bar graph relates two variables, but a histogram relates only one.<\/p>\n
        sns.FacetGrid(iris, hue=\u201dspecies\u201d, size=5) \\\r\n.map(sns.distplot, \u201cpetal_length\u201d) \\\r\n.add_legend();\r\nplt.show();<\/pre>\n
        \n
        \n
        \n
        \"Exploratory<\/div>\n<\/div>\n<\/div>\n<\/figure>\n
        sns.FacetGrid(iris, hue=\u201dspecies\u201d, size=5) \\\r\n.map(sns.distplot, \u201cpetal_width\u201d) \\\r\n.add_legend();\r\nplt.show();<\/pre>\n
        \n
        \n
        \n
        \"Exploratory<\/div>\n<\/div>\n<\/div>\n<\/figure>\n
        sns.FacetGrid(iris, hue=\u201dspecies\u201d, size=5) \\\r\n.map(sns.distplot, \u201csepal_length\u201d) \\\r\n.add_legend();\r\nplt.show();<\/pre>\n
        \n
        \n
        \n
        \"Exploratory<\/div>\n<\/div>\n<\/div>\n<\/figure>\n
        sns.FacetGrid(iris, hue=\u201dspecies\u201d, size=5) \\\r\n.map(sns.distplot, \u201csepal_width\u201d) \\\r\n.add_legend();\r\nplt.show();<\/pre>\n
        \n
        \n
        \n
        \"Exploratory<\/div>\n<\/div>\n<\/div>\n<\/figure>\n
        PDF and CDF<\/h4>\n