{"id":2149,"date":"2019-07-07T13:59:36","date_gmt":"2019-07-07T13:59:36","guid":{"rendered":"https:\/\/machinelearningplus.com\/?p=2149"},"modified":"2022-08-30T12:07:11","modified_gmt":"2022-08-30T12:07:11","slug":"vector-autoregression-examples-python","status":"publish","type":"post","link":"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/","title":{"rendered":"Vector Autoregression (VAR) &#8211; Comprehensive Guide with Examples in Python"},"content":{"rendered":"<p><em>Vector Autoregression (VAR) is a forecasting algorithm that can be used when two or more time series influence each other. That is, the relationship between the time series involved is bi-directional. In this post, we will see the concepts, intuition behind VAR models and see a comprehensive and correct method to train and forecast VAR models in python using <code>statsmodels<\/code>.<\/em><\/p>\n<p><img decoding=\"async\" src=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Vector_Autoregression_Feature.jpg\">Vector Autoregression (VAR) &#8211; Comprehensive Guide with Examples in Python. Photo by Kyran Low.<\/img><\/p>\n<h2 id=\"content\">Content<\/h2>\n<ol>\n<li>Introduction<\/li>\n<li>Intuition behind VAR Model Formula<\/li>\n<li>Building a VAR model in Python<\/li>\n<li>Import the datasets<\/li>\n<li>Visualize the Time Series<\/li>\n<li>Testing Causation using Granger&#8217;s Causality Test<\/li>\n<li>Cointegration Test<\/li>\n<li>Split the Series into Training and Testing Data<\/li>\n<li>Check for Stationarity and Make the Time Series Stationary<\/li>\n<li>How to Select the Order (P) of VAR model<\/li>\n<li>Train the VAR Model of Selected Order(p)<\/li>\n<li>Check for Serial Correlation of Residuals (Errors) using Durbin Watson Statistic<\/li>\n<li>How to Forecast VAR model using statsmodels<\/li>\n<li>Train the VAR Model of Selected Order(p)<\/li>\n<li>Invert the transformation to get the real forecast<\/li>\n<li>Plot of Forecast vs Actuals<\/li>\n<li>Evaluate the Forecasts<\/li>\n<li>Conclusion<\/li>\n<\/ol>\n<h2 id=\"1introduction\">1. Introduction<\/h2>\n<p>First, what is Vector Autoregression (VAR) and when to use it?<\/p>\n<p>Vector Autoregression (VAR) is a multivariate forecasting algorithm that is used when two or more <a href=\"https:\/\/machinelearningplus.com\/time-series\/time-series-analysis-python\/\" rel=\"noopener noreferrer\" target=\"_blank\">time series<\/a> influence each other. <\/p>\n<p>That means, the basic requirements in order to use VAR are:<\/p>\n<ol>\n<li>You need at least two time series (variables) <\/li>\n<li>The time series should influence each other.<\/li>\n<\/ol>\n<p>Alright. So why is it called &#8216;Autoregressive&#8217;?<\/p>\n<p>It is considered as an Autoregressive model because, each variable (Time Series) is modeled as a function of the past values, that is the predictors are nothing but the lags (time delayed value) of the series.<\/p>\n<p>Ok, so how is VAR different from other Autoregressive models like AR, ARMA or ARIMA?<\/p>\n<p>The primary difference is those models are uni-directional, where, the predictors influence the Y and not vice-versa. Whereas, Vector Auto Regression (VAR) is bi-directional. That is, the variables influence each other. <\/p>\n<p>We will go more in detail in the next section.<\/p>\n<p>In this article you will gain a clear understanding of:<\/p>\n<ul>\n<li>Intuition behind VAR Model formula<\/li>\n<li>How to check the bi-directional relationship using Granger Causality<\/li>\n<li>Procedure to building a VAR model in Python<\/li>\n<li>How to determine the right order of VAR model<\/li>\n<li>Interpreting the results of VAR model<\/li>\n<li>How to generate forecasts to original scale of time series<\/li>\n<\/ul>\n<h2 id=\"2intuitionbehindvarmodelformula\">2. Intuition behind VAR Model Formula<\/h2>\n<p>If you remember in <a href=\"https:\/\/machinelearningplus.com\/time-series\/arima-model-time-series-forecasting-python\/\">Autoregression models<\/a>, the time series is modeled as a linear combination of it&#8217;s own lags. That is, the past values of the series are used to forecast the current and future.<\/p>\n<p>A typical AR(p) model equation looks something like this:<\/p>\n<p><a href=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_ARp_Model-min.png\"><img fetchpriority=\"high\" decoding=\"async\" src=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_ARp_Model-min-640x89.png\" alt=\"AR(p) Model - Equation\" width=\"640\" height=\"89\" class=\"size-shareaholic-thumbnail wp-image-2153\" srcset=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_ARp_Model-min-640x89.png 640w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_ARp_Model-min-300x42.png 300w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_ARp_Model-min-768x106.png 768w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_ARp_Model-min-100x14.png 100w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_ARp_Model-min.png 794w\" sizes=\"(max-width: 640px) 100vw, 640px\" \/><\/a><\/p>\n<p>where &alpha; is the intercept, a constant and &beta;1, &beta;2 till &beta;p are the coefficients of the lags of Y till order p. <\/p>\n<p>Order &#8216;p&#8217; means, up to p-lags of Y is used and they are the predictors in the equation. The &epsilon;_{t} is the error, which is considered as white noise.<\/p>\n<p>Alright. So, how does a VAR model&#8217;s formula look like?<\/p>\n<p>In the VAR model, each variable is modeled as a <strong>linear combination of past values of itself and the past values of other variables in the system<\/strong>. Since you have multiple time series that influence each other, it is modeled as a system of equations with one equation per variable (time series). <\/p>\n<p>That is, if you have 5 time series that influence each other, we will have a system of 5 equations.<\/p>\n<p>Well, how is the equation exactly framed?<\/p>\n<p>Let&#8217;s suppose, you have two variables (Time series) Y1 and Y2, and you need to forecast the values of these variables at time (t).<\/p>\n<p>To calculate Y1(t), VAR will use the past values of both Y1 as well as Y2. Likewise, to compute Y2(t), the past values of both Y1 and Y2 be used.<\/p>\n<p>For example, the system of equations for a VAR(1) model with two time series (variables `Y1` and `Y2`) is as follows:<\/p>\n<p><a href=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR1_Model-min.png\"><img decoding=\"async\" src=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR1_Model-min.png\" alt=\"Formula for VAR(1) model with two Y&#039;s\" width=\"744\" height=\"150\" class=\"size-full wp-image-2157\" srcset=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR1_Model-min.png 744w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR1_Model-min-300x60.png 300w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR1_Model-min-640x129.png 640w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR1_Model-min-100x20.png 100w\" sizes=\"(max-width: 744px) 100vw, 744px\" \/><\/a><\/p>\n<p>Where, Y{1,t-1}  and Y{2,t-1} are the first lag of time series Y1 and Y2 respectively.<\/p>\n<p>The above equation is referred to as a VAR(1) model, because, each equation is of order 1, that is, it contains up to one lag of each of the predictors (Y1 and Y2).<\/p>\n<p>Since the Y terms in the equations are interrelated, the Y&#8217;s are considered as endogenous variables, rather than as exogenous predictors. <\/p>\n<p>Likewise, the second order VAR(2) model for two variables would include up to two lags for each variable (Y1 and Y2).<\/p>\n<p><a href=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model-min.png\"><img decoding=\"async\" src=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model-min.png\" alt=\"VAR(2) model with Two Y&#039;s\" width=\"1184\" height=\"146\" class=\"size-full wp-image-2158\" srcset=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model-min.png 1184w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model-min-300x37.png 300w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model-min-768x95.png 768w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model-min-1024x126.png 1024w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model-min-640x79.png 640w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model-min-100x12.png 100w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model-min-865x107.png 865w\" sizes=\"(max-width: 1184px) 100vw, 1184px\" \/><\/a><\/p>\n<p>Can you imagine what a second order VAR(2) model with three variables (Y1, Y2 and Y3) would look like?<\/p>\n<p><a href=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model_with_three_Ys-min.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model_with_three_Ys-min.png\" alt=\"VAR(2) model with three Y&#039;s\" width=\"1184\" height=\"167\" class=\"size-full wp-image-2159\" srcset=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model_with_three_Ys-min.png 1184w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model_with_three_Ys-min-300x42.png 300w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model_with_three_Ys-min-768x108.png 768w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model_with_three_Ys-min-1024x144.png 1024w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model_with_three_Ys-min-640x90.png 640w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model_with_three_Ys-min-100x14.png 100w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Equation_VAR2_Model_with_three_Ys-min-865x122.png 865w\" sizes=\"(max-width: 1184px) 100vw, 1184px\" \/><\/a><\/p>\n<p>As you increase the number of time series (variables) in the model the system of equations become larger.<\/p>\n<h2 id=\"3buildingavarmodelinpython\">3. Building a VAR model in Python<\/h2>\n<p>The procedure to build a VAR model involves the following steps:<\/p>\n<ol>\n<li>Analyze the time series characteristics<\/li>\n<li>Test for causation amongst the time series<\/li>\n<li>Test for stationarity <\/li>\n<li>Transform the series to make it stationary, if needed<\/li>\n<li>Find optimal order (p)<\/li>\n<li>Prepare training and test datasets<\/li>\n<li>Train the model<\/li>\n<li>Roll back the transformations, if any.<\/li>\n<li>Evaluate the model using test set<\/li>\n<li>Forecast to future<\/li>\n<\/ol>\n<pre><code class=\"python language-python\">import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\n# Import Statsmodels\nfrom statsmodels.tsa.api import VAR\nfrom statsmodels.tsa.stattools import adfuller\nfrom statsmodels.tools.eval_measures import rmse, aic\n<\/code><\/pre>\n<h2 id=\"4importthedatasets\">4. Import the datasets<\/h2>\n<p>For this article let&#8217;s use the time series used in Yash P Mehra&#8217;s 1994 article: &#8220;Wage Growth and the Inflation Process: An Empirical Approach&#8221;.<\/p>\n<p>This dataset has the following 8 quarterly time series:<\/p>\n<pre>\n1. rgnp  : Real GNP.\n2. pgnp  : Potential real GNP.\n3. ulc   : Unit labor cost.\n4. gdfco : Fixed weight deflator for personal consumption expenditure excluding food and energy.\n5. gdf   : Fixed weight GNP deflator.\n6. gdfim : Fixed weight import deflator.\n7. gdfcf : Fixed weight deflator for food in personal consumption expenditure.\n8. gdfce : Fixed weight deflator for energy in personal consumption expenditure.\n<\/pre>\n<p>Let&#8217;s import the data.<\/p>\n<pre><code class=\"python language-python\">filepath = 'https:\/\/raw.githubusercontent.com\/selva86\/datasets\/master\/Raotbl6.csv'\ndf = pd.read_csv(filepath, parse_dates=['date'], index_col='date')\nprint(df.shape)  # (123, 8)\ndf.tail()\n<\/code><\/pre>\n<p><a href=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Multi_dimensional_time_series_VAR-min.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Multi_dimensional_time_series_VAR-min.png\" alt=\"Input Time Series Data for VAR model\" width=\"1134\" height=\"460\" class=\"aligncenter size-full wp-image-2161\" srcset=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Multi_dimensional_time_series_VAR-min.png 1134w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Multi_dimensional_time_series_VAR-min-300x122.png 300w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Multi_dimensional_time_series_VAR-min-768x312.png 768w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Multi_dimensional_time_series_VAR-min-1024x415.png 1024w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Multi_dimensional_time_series_VAR-min-640x260.png 640w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Multi_dimensional_time_series_VAR-min-100x41.png 100w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Multi_dimensional_time_series_VAR-min-865x351.png 865w\" sizes=\"(max-width: 1134px) 100vw, 1134px\" \/><\/a><\/p>\n<h2 id=\"5visualizethetimeseries\">5. Visualize the Time Series<\/h2>\n<pre><code class=\"python language-python\"># Plot\nfig, axes = plt.subplots(nrows=4, ncols=2, dpi=120, figsize=(10,6))\nfor i, ax in enumerate(axes.flatten()):\n    data = df[df.columns[i]]\n    ax.plot(data, color='red', linewidth=1)\n    # Decorations\n    ax.set_title(df.columns[i])\n    ax.xaxis.set_ticks_position('none')\n    ax.yaxis.set_ticks_position('none')\n    ax.spines[\"top\"].set_alpha(0)\n    ax.tick_params(labelsize=6)\n\nplt.tight_layout();\n<\/code><\/pre>\n<figure id=\"attachment_2150\" aria-describedby=\"caption-attachment-2150\" style=\"width: 1184px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/actuals_VAR.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/actuals_VAR.png\" alt=\"Multi Dimensional Time Series for VAR model\" width=\"1184\" height=\"705\" class=\"size-full wp-image-2150\" srcset=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/actuals_VAR.png 1184w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/actuals_VAR-300x179.png 300w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/actuals_VAR-768x457.png 768w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/actuals_VAR-1024x610.png 1024w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/actuals_VAR-640x381.png 640w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/actuals_VAR-100x60.png 100w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/actuals_VAR-865x515.png 865w\" sizes=\"(max-width: 1184px) 100vw, 1184px\" \/><\/a><figcaption id=\"caption-attachment-2150\" class=\"wp-caption-text\">Actual Multi Dimensional Time Series for VAR model<\/figcaption><\/figure>\n<p>Each of the series have a fairly similar trend patterns over the years except for <code>gdfce<\/code> and <code>gdfim<\/code>, where a different pattern is noticed starting in 1980.<\/p>\n<p>Alright, next step in the analysis is to check for causality amongst these series. The Granger&#8217;s Causality test and the Cointegration test can help us with that.<\/p>\n<h2 id=\"6testingcausationusinggrangerscausalitytest\">6. Testing Causation using Granger&#8217;s Causality Test<\/h2>\n<p>The basis behind Vector AutoRegression is that each of the time series in the system influences each other. That is, you can predict the series with past values of itself along with other series in the system.<\/p>\n<p>Using <a href=\"https:\/\/machinelearningplus.com\/time-series\/granger-causality-test\/\" rel=\"noopener\" target=\"_blank\">Granger&#8217;s Causality Test<\/a>, it&#8217;s possible to test this relationship before even building the model.<\/p>\n<p>So what does Granger&#8217;s Causality really test?<\/p>\n<p>Granger&#8217;s causality tests the null hypothesis that the coefficients of past values in the regression equation is zero. <\/p>\n<p>In simpler terms, the past values of time series (X) do not cause the other series (Y). So, if the p-value obtained from the test is lesser than the significance level of 0.05, then, you can safely reject the null hypothesis.<\/p>\n<p>The below code implements the Granger&#8217;s Causality test for all possible combinations of the time series in a given dataframe and stores the p-values of each combination in the output matrix.<\/p>\n<pre><code class=\"python language-python\">from statsmodels.tsa.stattools import grangercausalitytests\nmaxlag=12\ntest = 'ssr_chi2test'\ndef grangers_causation_matrix(data, variables, test='ssr_chi2test', verbose=False):    \n    \"\"\"Check Granger Causality of all possible combinations of the Time series.\n    The rows are the response variable, columns are predictors. The values in the table \n    are the P-Values. P-Values lesser than the significance level (0.05), implies \n    the Null Hypothesis that the coefficients of the corresponding past values is \n    zero, that is, the X does not cause Y can be rejected.\n\n    data      : pandas dataframe containing the time series variables\n    variables : list containing names of the time series variables.\n    \"\"\"\n    df = pd.DataFrame(np.zeros((len(variables), len(variables))), columns=variables, index=variables)\n    for c in df.columns:\n        for r in df.index:\n            test_result = grangercausalitytests(data[[r, c]], maxlag=maxlag, verbose=False)\n            p_values = [round(test_result[i+1][0][test][1],4) for i in range(maxlag)]\n            if verbose: print(f'Y = {r}, X = {c}, P Values = {p_values}')\n            min_p_value = np.min(p_values)\n            df.loc[r, c] = min_p_value\n    df.columns = [var + '_x' for var in variables]\n    df.index = [var + '_y' for var in variables]\n    return df\n\ngrangers_causation_matrix(df, variables = df.columns)        \n<\/code><\/pre>\n<p><a href=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Grangers-Causality-Test-Results-Matrix-min.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Grangers-Causality-Test-Results-Matrix-min.png\" alt=\"Grangers Causality Test - Results p-value Matrix\" width=\"1184\" height=\"564\" class=\"aligncenter size-full wp-image-2162\" srcset=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Grangers-Causality-Test-Results-Matrix-min.png 1184w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Grangers-Causality-Test-Results-Matrix-min-300x143.png 300w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Grangers-Causality-Test-Results-Matrix-min-768x366.png 768w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Grangers-Causality-Test-Results-Matrix-min-1024x488.png 1024w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Grangers-Causality-Test-Results-Matrix-min-640x305.png 640w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Grangers-Causality-Test-Results-Matrix-min-100x48.png 100w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Grangers-Causality-Test-Results-Matrix-min-865x412.png 865w\" sizes=\"(max-width: 1184px) 100vw, 1184px\" \/><\/a><\/p>\n<p>So how to read the above output?<\/p>\n<p>The row are the Response (Y) and the columns are the predictor series (X). <\/p>\n<p>For example, if you take the value 0.0003 in (row 1, column 2), it refers to the p-value of <code>pgnp_x<\/code> causing <code>rgnp_y<\/code>. Whereas, the 0.000 in (row 2, column 1) refers to the p-value of <code>rgnp_y<\/code> causing <code>pgnp_x<\/code>. <\/p>\n<p>So, how to interpret the p-values?<\/p>\n<p>If a given p-value is &lt; significance level (0.05), then, the corresponding X series (column) causes the Y (row).<\/p>\n<p>For example, P-Value of 0.0003 at (row 1, column 2) represents the p-value of the Grangers Causality test for <code>pgnp_x<\/code> causing <code>rgnp_y<\/code>, which is less that the significance level of 0.05. <\/p>\n<p>So, you can reject the null hypothesis and conclude <code>pgnp_x<\/code> causes <code>rgnp_y<\/code>.<\/p>\n<p>Looking at the P-Values in the above table, you can pretty much observe that all the variables (time series) in the system are interchangeably causing each other.<\/p>\n<p>This makes this system of multi time series a good candidate for using VAR models to forecast. <\/p>\n<p>Next, let&#8217;s do the Cointegration test.<\/p>\n<h2 id=\"7cointegrationtest\">7. Cointegration Test<\/h2>\n<p>Cointegration test helps to establish the presence of a statistically significant connection between two or more time series.  <\/p>\n<p>But, what does Cointegration mean?<\/p>\n<p>To understand that, you first need to know what is <em>&#8216;order of integration&#8217; (d)<\/em>.<\/p>\n<p>Order of integration(d) is nothing but the number of differencing required to make a non-stationary time series stationary.<\/p>\n<p>Now, when you have two or more time series, and there exists a linear combination of them that has an order of integration (d) less than that of the individual series, then the collection of series is said to be cointegrated.<\/p>\n<p>Ok?<\/p>\n<p>When two or more time series are cointegrated, it means they have a long run, statistically significant relationship. <\/p>\n<p>This is the basic premise on which Vector Autoregression(VAR) models is based on. So, it&#8217;s fairly common to implement the cointegration test before starting to build VAR models.<\/p>\n<p>Alright, So how to do this test?<\/p>\n<p>Soren Johanssen in his <a href=\"https:\/\/www.jstor.org\/stable\/2938278?seq=1#page_scan_tab_contents\">paper (1991)<\/a> devised a procedure to implement the cointegration test.  <\/p>\n<p>It is fairly straightforward to implement in python&#8217;s <code>statsmodels<\/code>, as you can see below.<\/p>\n<pre><code class=\"python language-python\">from statsmodels.tsa.vector_ar.vecm import coint_johansen\n\ndef cointegration_test(df, alpha=0.05): \n    \"\"\"Perform Johanson's Cointegration Test and Report Summary\"\"\"\n    out = coint_johansen(df,-1,5)\n    d = {'0.90':0, '0.95':1, '0.99':2}\n    traces = out.lr1\n    cvts = out.cvt[:, d[str(1-alpha)]]\n    def adjust(val, length= 6): return str(val).ljust(length)\n\n    # Summary\n    print('Name   ::  Test Stat &gt; C(95%)    =&gt;   Signif  \\n', '--'*20)\n    for col, trace, cvt in zip(df.columns, traces, cvts):\n        print(adjust(col), ':: ', adjust(round(trace,2), 9), \"&gt;\", adjust(cvt, 8), ' =&gt;  ' , trace &gt; cvt)\n\ncointegration_test(df)\n<\/code><\/pre>\n<p>Results:<\/p>\n<pre><code>Name   ::  Test Stat &gt; C(95%)    =&gt;   Signif  \n ----------------------------------------\nrgnp   ::  248.0     &gt; 143.6691  =&gt;   True\npgnp   ::  183.12    &gt; 111.7797  =&gt;   True\nulc    ::  130.01    &gt; 83.9383   =&gt;   True\ngdfco  ::  85.28     &gt; 60.0627   =&gt;   True\ngdf    ::  55.05     &gt; 40.1749   =&gt;   True\ngdfim  ::  31.59     &gt; 24.2761   =&gt;   True\ngdfcf  ::  14.06     &gt; 12.3212   =&gt;   True\ngdfce  ::  0.45      &gt; 4.1296    =&gt;   False\n<\/code><\/pre>\n<h2 id=\"8splittheseriesintotrainingandtestingdata\">8. Split the Series into Training and Testing Data<\/h2>\n<p>Splitting the dataset into training and test data. <\/p>\n<p>The VAR model will be fitted on <code>df_train<\/code> and then used to forecast the next 4 observations. These forecasts will be compared against the actuals present in test data.<\/p>\n<p>To do the comparisons, we will use multiple forecast accuracy metrics, as seen later in this article.<\/p>\n<pre><code class=\"python language-python\">nobs = 4\ndf_train, df_test = df[0:-nobs], df[-nobs:]\n\n# Check size\nprint(df_train.shape)  # (119, 8)\nprint(df_test.shape)  # (4, 8)\n<\/code><\/pre>\n<h2 id=\"9checkforstationarityandmakethetimeseriesstationary\">9. Check for Stationarity and Make the Time Series Stationary<\/h2>\n<p>Since the VAR model requires the time series you want to forecast to be stationary, it is customary to check all the time series in the system for stationarity.<\/p>\n<p>Just to refresh, a stationary time series is one whose characteristics like mean and variance does not change over time.<\/p>\n<p>So, how to test for stationarity?<\/p>\n<p>There is a suite of tests called unit-root tests. The popular ones are:<\/p>\n<ol>\n<li><a href=\"https:\/\/machinelearningplus.com\/time-series\/augmented-dickey-fuller-test\/\" rel=\"noopener noreferrer\" target=\"_blank\">Augmented Dickey-Fuller Test (ADF Test)<\/a><\/li>\n<li><a href=\"https:\/\/machinelearningplus.com\/time-series\/kpss-test-for-stationarity\/\" rel=\"noopener noreferrer\" target=\"_blank\">KPSS test<\/a><\/li>\n<li>Philip-Perron test<\/li>\n<\/ol>\n<p>Let&#8217;s use the ADF test for our purpose.<\/p>\n<p>By the way, if a series is found to be non-stationary, you make it stationary by differencing the series once and repeat the test again until it becomes stationary.<\/p>\n<p>Since, differencing reduces the length of the series by 1 and since all the time series has to be of the same length, you need to difference all the series in the system if you choose to difference at all.<\/p>\n<p>Got it?<\/p>\n<p>Let&#8217;s implement the ADF Test.<\/p>\n<p>First, we implement a nice function (<code>adfuller_test()<\/code>) that writes out the results of the ADF test for any given time series and implement this function on each series one-by-one.<\/p>\n<pre><code class=\"python language-python\">def adfuller_test(series, signif=0.05, name='', verbose=False):\n    \"\"\"Perform ADFuller to test for Stationarity of given series and print report\"\"\"\n    r = adfuller(series, autolag='AIC')\n    output = {'test_statistic':round(r[0], 4), 'pvalue':round(r[1], 4), 'n_lags':round(r[2], 4), 'n_obs':r[3]}\n    p_value = output['pvalue'] \n    def adjust(val, length= 6): return str(val).ljust(length)\n\n    # Print Summary\n    print(f'    Augmented Dickey-Fuller Test on \"{name}\"', \"\\n   \", '-'*47)\n    print(f' Null Hypothesis: Data has unit root. Non-Stationary.')\n    print(f' Significance Level    = {signif}')\n    print(f' Test Statistic        = {output[\"test_statistic\"]}')\n    print(f' No. Lags Chosen       = {output[\"n_lags\"]}')\n\n    for key,val in r[4].items():\n        print(f' Critical value {adjust(key)} = {round(val, 3)}')\n\n    if p_value &lt;= signif:\n        print(f\" =&gt; P-Value = {p_value}. Rejecting Null Hypothesis.\")\n        print(f\" =&gt; Series is Stationary.\")\n    else:\n        print(f\" =&gt; P-Value = {p_value}. Weak evidence to reject the Null Hypothesis.\")\n        print(f\" =&gt; Series is Non-Stationary.\")    \n<\/code><\/pre>\n<p>Call the <code>adfuller_test()<\/code> on each series.<\/p>\n<pre><code class=\"python language-python\"># ADF Test on each column\nfor name, column in df_train.iteritems():\n    adfuller_test(column, name=column.name)\n    print('\\n')\n<\/code><\/pre>\n<p>Results:<\/p>\n<pre><code>    Augmented Dickey-Fuller Test on \"rgnp\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = 0.5428\n No. Lags Chosen       = 2\n Critical value 1%     = -3.488\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.9861. Weak evidence to reject the Null Hypothesis.\n =&gt; Series is Non-Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"pgnp\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = 1.1556\n No. Lags Chosen       = 1\n Critical value 1%     = -3.488\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.9957. Weak evidence to reject the Null Hypothesis.\n =&gt; Series is Non-Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"ulc\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = 1.2474\n No. Lags Chosen       = 2\n Critical value 1%     = -3.488\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.9963. Weak evidence to reject the Null Hypothesis.\n =&gt; Series is Non-Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdfco\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = 1.1954\n No. Lags Chosen       = 3\n Critical value 1%     = -3.489\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.996. Weak evidence to reject the Null Hypothesis.\n =&gt; Series is Non-Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdf\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = 1.676\n No. Lags Chosen       = 7\n Critical value 1%     = -3.491\n Critical value 5%     = -2.888\n Critical value 10%    = -2.581\n =&gt; P-Value = 0.9981. Weak evidence to reject the Null Hypothesis.\n =&gt; Series is Non-Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdfim\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -0.0799\n No. Lags Chosen       = 1\n Critical value 1%     = -3.488\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.9514. Weak evidence to reject the Null Hypothesis.\n =&gt; Series is Non-Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdfcf\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = 1.4395\n No. Lags Chosen       = 8\n Critical value 1%     = -3.491\n Critical value 5%     = -2.888\n Critical value 10%    = -2.581\n =&gt; P-Value = 0.9973. Weak evidence to reject the Null Hypothesis.\n =&gt; Series is Non-Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdfce\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -0.3402\n No. Lags Chosen       = 8\n Critical value 1%     = -3.491\n Critical value 5%     = -2.888\n Critical value 10%    = -2.581\n =&gt; P-Value = 0.9196. Weak evidence to reject the Null Hypothesis.\n =&gt; Series is Non-Stationary.\n<\/code><\/pre>\n<p>The ADF test confirms none of the time series is stationary. Let&#8217;s difference all of them once and check again.<\/p>\n<pre><code class=\"python language-python\"># 1st difference\ndf_differenced = df_train.diff().dropna()\n<\/code><\/pre>\n<p>Re-run ADF test on each differenced series.<\/p>\n<pre><code class=\"python language-python\"># ADF Test on each column of 1st Differences Dataframe\nfor name, column in df_differenced.iteritems():\n    adfuller_test(column, name=column.name)\n    print('\\n')\n<\/code><\/pre>\n<pre><code>    Augmented Dickey-Fuller Test on \"rgnp\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -5.3448\n No. Lags Chosen       = 1\n Critical value 1%     = -3.488\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.0. Rejecting Null Hypothesis.\n =&gt; Series is Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"pgnp\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -1.8282\n No. Lags Chosen       = 0\n Critical value 1%     = -3.488\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.3666. Weak evidence to reject the Null Hypothesis.\n =&gt; Series is Non-Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"ulc\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -3.4658\n No. Lags Chosen       = 1\n Critical value 1%     = -3.488\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.0089. Rejecting Null Hypothesis.\n =&gt; Series is Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdfco\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -1.4385\n No. Lags Chosen       = 2\n Critical value 1%     = -3.489\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.5637. Weak evidence to reject the Null Hypothesis.\n =&gt; Series is Non-Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdf\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -1.1289\n No. Lags Chosen       = 2\n Critical value 1%     = -3.489\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.7034. Weak evidence to reject the Null Hypothesis.\n =&gt; Series is Non-Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdfim\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -4.1256\n No. Lags Chosen       = 0\n Critical value 1%     = -3.488\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.0009. Rejecting Null Hypothesis.\n =&gt; Series is Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdfcf\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -2.0545\n No. Lags Chosen       = 7\n Critical value 1%     = -3.491\n Critical value 5%     = -2.888\n Critical value 10%    = -2.581\n =&gt; P-Value = 0.2632. Weak evidence to reject the Null Hypothesis.\n =&gt; Series is Non-Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdfce\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -3.1543\n No. Lags Chosen       = 7\n Critical value 1%     = -3.491\n Critical value 5%     = -2.888\n Critical value 10%    = -2.581\n =&gt; P-Value = 0.0228. Rejecting Null Hypothesis.\n =&gt; Series is Stationary.\n<\/code><\/pre>\n<p>After the first difference, Real Wages (Manufacturing) is still not stationary. It&#8217;s critical value is between 5% and 10% significance level.<\/p>\n<p>All of the series in the VAR model should have the same number of observations.<\/p>\n<p>So, we are left with one of two choices.<\/p>\n<p>That is, either proceed with 1st differenced series or difference all the series one more time. <\/p>\n<pre><code class=\"python language-python\"># Second Differencing\ndf_differenced = df_differenced.diff().dropna()\n<\/code><\/pre>\n<p>Re-run ADF test again on each second differenced series.<\/p>\n<pre><code class=\"python language-python\"># ADF Test on each column of 2nd Differences Dataframe\nfor name, column in df_differenced.iteritems():\n    adfuller_test(column, name=column.name)\n    print('\\n')\n<\/code><\/pre>\n<p>Results:<\/p>\n<pre><code>    Augmented Dickey-Fuller Test on \"rgnp\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -9.0123\n No. Lags Chosen       = 2\n Critical value 1%     = -3.489\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.0. Rejecting Null Hypothesis.\n =&gt; Series is Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"pgnp\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -10.9813\n No. Lags Chosen       = 0\n Critical value 1%     = -3.488\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.0. Rejecting Null Hypothesis.\n =&gt; Series is Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"ulc\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -8.769\n No. Lags Chosen       = 2\n Critical value 1%     = -3.489\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.0. Rejecting Null Hypothesis.\n =&gt; Series is Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdfco\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -7.9102\n No. Lags Chosen       = 3\n Critical value 1%     = -3.49\n Critical value 5%     = -2.887\n Critical value 10%    = -2.581\n =&gt; P-Value = 0.0. Rejecting Null Hypothesis.\n =&gt; Series is Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdf\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -10.0351\n No. Lags Chosen       = 1\n Critical value 1%     = -3.489\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.0. Rejecting Null Hypothesis.\n =&gt; Series is Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdfim\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -9.4059\n No. Lags Chosen       = 1\n Critical value 1%     = -3.489\n Critical value 5%     = -2.887\n Critical value 10%    = -2.58\n =&gt; P-Value = 0.0. Rejecting Null Hypothesis.\n =&gt; Series is Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdfcf\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -6.922\n No. Lags Chosen       = 5\n Critical value 1%     = -3.491\n Critical value 5%     = -2.888\n Critical value 10%    = -2.581\n =&gt; P-Value = 0.0. Rejecting Null Hypothesis.\n =&gt; Series is Stationary.\n\n\n    Augmented Dickey-Fuller Test on \"gdfce\" \n    -----------------------------------------------\n Null Hypothesis: Data has unit root. Non-Stationary.\n Significance Level    = 0.05\n Test Statistic        = -5.1732\n No. Lags Chosen       = 8\n Critical value 1%     = -3.492\n Critical value 5%     = -2.889\n Critical value 10%    = -2.581\n =&gt; P-Value = 0.0. Rejecting Null Hypothesis.\n =&gt; Series is Stationary.\n<\/code><\/pre>\n<p>All the series are now stationary. <\/p>\n<p>Let&#8217;s prepare the training and test datasets.<\/p>\n<h2 id=\"10howtoselecttheorderpofvarmodel\">10. How to Select the Order (P) of VAR model<\/h2>\n<p>To select the right order of the VAR model, we iteratively fit increasing orders of VAR model and pick the order that gives a model with least AIC. <\/p>\n<p>Though the usual practice is to look at the AIC, you can also check other best fit comparison estimates of <code>BIC<\/code>, <code>FPE<\/code> and <code>HQIC<\/code>.<\/p>\n<pre><code class=\"python language-python\">model = VAR(df_differenced)\nfor i in [1,2,3,4,5,6,7,8,9]:\n    result = model.fit(i)\n    print('Lag Order =', i)\n    print('AIC : ', result.aic)\n    print('BIC : ', result.bic)\n    print('FPE : ', result.fpe)\n    print('HQIC: ', result.hqic, '\\n')\n<\/code><\/pre>\n<p>Results:<\/p>\n<pre><code>Lag Order = 1\nAIC :  -1.3679402315450664\nBIC :  0.3411847146588838\nFPE :  0.2552682517347198\nHQIC:  -0.6741331335699554 \n\nLag Order = 2\nAIC :  -1.621237394447824\nBIC :  1.6249432095295848\nFPE :  0.2011349437137139\nHQIC:  -0.3036288826795923 \n\nLag Order = 3\nAIC :  -1.7658008387012791\nBIC :  3.0345473163767833\nFPE :  0.18125103746164364\nHQIC:  0.18239143783963296 \n\nLag Order = 4\nAIC :  -2.000735164470318\nBIC :  4.3712151376540875\nFPE :  0.15556966521481097\nHQIC:  0.5849359332771069 \n\nLag Order = 5\nAIC :  -1.9619535608363954\nBIC :  5.9993645622420955\nFPE :  0.18692794389114886\nHQIC:  1.268206331178333 \n\nLag Order = 6\nAIC :  -2.3303386524829053\nBIC :  7.2384526890885805\nFPE :  0.16380374017443664\nHQIC:  1.5514371669548073 \n\nLag Order = 7\nAIC :  -2.592331352347129\nBIC :  8.602387254937796\nFPE :  0.1823868583715414\nHQIC:  1.9483069621146551 \n\nLag Order = 8\nAIC :  -3.317261976458205\nBIC :  9.52219581032303\nFPE :  0.15573163248209088\nHQIC:  1.8896071386220985 \n\nLag Order = 9\nAIC :  -4.804763125958631\nBIC :  9.698613139231597\nFPE :  0.08421466682671915\nHQIC:  1.0758291640834052 \n<\/code><\/pre>\n<p>In the above output, the AIC drops to lowest at lag 4, then increases at lag 5 and then continuously drops further. <\/p>\n<p>Let&#8217;s go with the lag 4 model.<\/p>\n<p>An alternate method to choose the order(p) of the VAR models is to use the <code>model.select_order(maxlags)<\/code> method. <\/p>\n<p>The selected order(p) is the order that gives the lowest &#8216;AIC&#8217;, &#8216;BIC&#8217;, &#8216;FPE&#8217; and &#8216;HQIC&#8217; scores. <\/p>\n<pre><code class=\"python language-python\">x = model.select_order(maxlags=12)\nx.summary()\n<\/code><\/pre>\n<p><a href=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Order_Selection_Table-min.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Order_Selection_Table-min-640x897.png\" alt=\"How to select the order of the VAR model\" width=\"640\" height=\"897\" class=\"aligncenter size-shareaholic-thumbnail wp-image-2152\" srcset=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Order_Selection_Table-min-640x897.png 640w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Order_Selection_Table-min-214x300.png 214w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Order_Selection_Table-min-100x140.png 100w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Order_Selection_Table-min.png 706w\" sizes=\"(max-width: 640px) 100vw, 640px\" \/><\/a><\/p>\n<p>According to FPE and HQIC, the optimal lag is observed at a lag order of 3. <\/p>\n<p>I, however, don&#8217;t have an explanation for why the observed AIC and BIC values differ when using <code>result.aic<\/code> versus as seen using <code>model.select_order()<\/code>.<\/p>\n<p>Since the explicitly computed AIC is the lowest at lag 4, I choose the selected order as 4.<\/p>\n<h2 id=\"11trainthevarmodelofselectedorderp\">11. Train the VAR Model of Selected Order(p)<\/h2>\n<pre><code class=\"python language-python\">model_fitted = model.fit(4)\nmodel_fitted.summary()\n<\/code><\/pre>\n<p>Results:<\/p>\n<pre><code>  Summary of Regression Results   \n==================================\nModel:                         VAR\nMethod:                        OLS\nDate:           Sat, 18, May, 2019\nTime:                     11:35:15\n--------------------------------------------------------------------\nNo. of Equations:         8.00000    BIC:                    4.37122\nNobs:                     113.000    HQIC:                  0.584936\nLog likelihood:          -905.679    FPE:                   0.155570\nAIC:                     -2.00074    Det(Omega_mle):       0.0200322\n--------------------------------------------------------------------\nResults for equation rgnp\n===========================================================================\n              coefficient       std. error           t-stat            prob\n---------------------------------------------------------------------------\nconst            2.430021         2.677505            0.908           0.364\nL1.rgnp         -0.750066         0.159023           -4.717           0.000\nL1.pgnp         -0.095621         4.938865           -0.019           0.985\nL1.ulc          -6.213996         4.637452           -1.340           0.180\nL1.gdfco        -7.414768        10.184884           -0.728           0.467\nL1.gdf         -24.864063        20.071245           -1.239           0.215\nL1.gdfim         1.082913         4.309034            0.251           0.802\nL1.gdfcf        16.327252         5.892522            2.771           0.006\nL1.gdfce         0.910522         2.476361            0.368           0.713\nL2.rgnp         -0.568178         0.163971           -3.465           0.001\nL2.pgnp         -1.156201         4.931931           -0.234           0.815\nL2.ulc         -11.157111         5.381825           -2.073           0.038\nL2.gdfco         3.012518        12.928317            0.233           0.816\nL2.gdf         -18.143523        24.090598           -0.753           0.451\nL2.gdfim        -4.438115         4.410654           -1.006           0.314\nL2.gdfcf        13.468228         7.279772            1.850           0.064\nL2.gdfce         5.130419         2.805310            1.829           0.067\nL3.rgnp         -0.514985         0.152724           -3.372           0.001\nL3.pgnp        -11.483607         5.392037           -2.130           0.033\nL3.ulc         -14.195308         5.188718           -2.736           0.006\nL3.gdfco       -10.154967        13.105508           -0.775           0.438\nL3.gdf         -15.438858        21.610822           -0.714           0.475\nL3.gdfim        -6.405290         4.292790           -1.492           0.136\nL3.gdfcf         9.217402         7.081652            1.302           0.193\nL3.gdfce         5.279941         2.833925            1.863           0.062\nL4.rgnp         -0.166878         0.138786           -1.202           0.229\nL4.pgnp          5.329900         5.795837            0.920           0.358\nL4.ulc          -4.834548         5.259608           -0.919           0.358\nL4.gdfco        10.841602        10.526530            1.030           0.303\nL4.gdf         -17.651510        18.746673           -0.942           0.346\nL4.gdfim        -1.971233         4.029415           -0.489           0.625\nL4.gdfcf         0.617824         5.842684            0.106           0.916\nL4.gdfce        -2.977187         2.594251           -1.148           0.251\n===========================================================================\n\nResults for equation pgnp\n===========================================================================\n              coefficient       std. error           t-stat            prob\n---------------------------------------------------------------------------\nconst            0.094556         0.063491            1.489           0.136\nL1.rgnp         -0.004231         0.003771           -1.122           0.262\nL1.pgnp          0.082204         0.117114            0.702           0.483\nL1.ulc          -0.097769         0.109966           -0.889           0.374\n\n(... TRUNCATED because of long output....)\n(... TRUNCATED because of long output....)\n(... TRUNCATED because of long output....)\n\nCorrelation matrix of residuals\n             rgnp      pgnp       ulc     gdfco       gdf     gdfim     gdfcf     gdfce\nrgnp     1.000000  0.248342 -0.668492 -0.160133 -0.047777  0.084925  0.009962  0.205557\npgnp     0.248342  1.000000 -0.148392 -0.167766 -0.134896  0.007830 -0.169435  0.032134\nulc     -0.668492 -0.148392  1.000000  0.268127  0.327761  0.171497  0.135410 -0.026037\ngdfco   -0.160133 -0.167766  0.268127  1.000000  0.303563  0.232997 -0.035042  0.184834\ngdf     -0.047777 -0.134896  0.327761  0.303563  1.000000  0.196670  0.446012  0.309277\ngdfim    0.084925  0.007830  0.171497  0.232997  0.196670  1.000000 -0.089852  0.707809\ngdfcf    0.009962 -0.169435  0.135410 -0.035042  0.446012 -0.089852  1.000000 -0.197099\ngdfce    0.205557  0.032134 -0.026037  0.184834  0.309277  0.707809 -0.197099  1.000000\n<\/code><\/pre>\n<h2 id=\"12checkforserialcorrelationofresidualserrorsusingdurbinwatsonstatistic\">12. Check for Serial Correlation of Residuals (Errors) using Durbin Watson Statistic<\/h2>\n<p>Serial correlation of residuals is used to check if there is any leftover pattern in the residuals (errors). <\/p>\n<p>What does this mean to us?<\/p>\n<p>If there is any correlation left in the residuals, then, there is some pattern in the time series that is still left to be explained by the model. In that case, the typical course of action is to either increase the order of the model or induce more predictors into the system or look for a different algorithm to model the time series. <\/p>\n<p>So, checking for serial correlation is to ensure that the model is sufficiently able to explain the variances and patterns in the time series.<\/p>\n<p>Alright, coming back to topic.<\/p>\n<p>A common way of checking for serial correlation of errors can be measured using the Durbin Watson&#8217;s Statistic.<\/p>\n<p><a href=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Durbin_Watson_Statistic_Formula-min.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Durbin_Watson_Statistic_Formula-min-300x113.png\" alt=\"Durbin Watson Statistic - Formula\" width=\"300\" height=\"113\" class=\"aligncenter size-medium wp-image-2164\" srcset=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Durbin_Watson_Statistic_Formula-min-300x113.png 300w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Durbin_Watson_Statistic_Formula-min-100x38.png 100w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/Durbin_Watson_Statistic_Formula-min.png 482w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>The value of this statistic can vary between 0 and 4. The closer it is to the value 2, then there is no significant serial correlation. The closer to 0, there is a positive serial correlation, and the closer it is to 4 implies negative serial correlation.<\/p>\n<pre><code class=\"python language-python\">from statsmodels.stats.stattools import durbin_watson\nout = durbin_watson(model_fitted.resid)\n\nfor col, val in zip(df.columns, out):\n    print(adjust(col), ':', round(val, 2))\n<\/code><\/pre>\n<p>Results:<\/p>\n<pre><code>rgnp   : 2.09\npgnp   : 2.02\nulc    : 2.17\ngdfco  : 2.05\ngdf    : 2.25\ngdfim  : 1.99\ngdfcf  : 2.2\ngdfce  : 2.17\n<\/code><\/pre>\n<p>The serial correlation seems quite alright. Let&#8217;s proceed with the forecast.<\/p>\n<h2 id=\"13howtoforecastvarmodelusingstatsmodels\">13. How to Forecast VAR model using statsmodels<\/h2>\n<p>In order to forecast, the VAR model expects up to the lag order number of observations from the past data. <\/p>\n<p>This is because, the terms in the VAR model are essentially the lags of the various time series in the dataset, so you need to provide it as many of the previous values as indicated by the lag order used by the model.<\/p>\n<pre><code class=\"python language-python\"># Get the lag order\nlag_order = model_fitted.k_ar\nprint(lag_order)  #&gt; 4\n\n# Input data for forecasting\nforecast_input = df_differenced.values[-lag_order:]\nforecast_input\n<\/code><\/pre>\n<pre><code>4\n\narray([[ 13.5,   0.1,   1.4,   0.1,   0.1,  -0.1,   0.4,  -2. ],\n       [-23.6,   0.2,  -2. ,  -0.5,  -0.1,  -0.2,  -0.3,  -1.2],\n       [ -3.3,   0.1,   3.1,   0.5,   0.3,   0.4,   0.9,   2.2],\n       [ -3.9,   0.2,  -2.1,  -0.4,   0.2,  -1.5,   0.9,  -0.3]])\n<\/code><\/pre>\n<p>Let&#8217;s forecast.<\/p>\n<pre><code class=\"python language-python\"># Forecast\nfc = model_fitted.forecast(y=forecast_input, steps=nobs)\ndf_forecast = pd.DataFrame(fc, index=df.index[-nobs:], columns=df.columns + '_2d')\ndf_forecast\n<\/code><\/pre>\n<p><a href=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Forecasts_raw.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Forecasts_raw-1024x251.png\" alt=\"Raw forecasts from the VAR model\" width=\"865\" height=\"212\" class=\"aligncenter size-large wp-image-2166\" srcset=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Forecasts_raw-1024x251.png 1024w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Forecasts_raw-300x74.png 300w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Forecasts_raw-768x188.png 768w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Forecasts_raw-640x157.png 640w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Forecasts_raw-100x25.png 100w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Forecasts_raw-865x212.png 865w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR_Forecasts_raw.png 1184w\" sizes=\"(max-width: 865px) 100vw, 865px\" \/><\/a><\/p>\n<p>The forecasts are generated but it is on the scale of the training data used by the model. So, to bring it back up to its original scale, you need to de-difference it as many times you had differenced the original input data. <\/p>\n<p>In this case it is two times.<\/p>\n<h2 id=\"14invertthetransformationtogettherealforecast\">14. Invert the transformation to get the real forecast<\/h2>\n<pre><code class=\"python language-python\">def invert_transformation(df_train, df_forecast, second_diff=False):\n    \"\"\"Revert back the differencing to get the forecast to original scale.\"\"\"\n    df_fc = df_forecast.copy()\n    columns = df_train.columns\n    for col in columns:        \n        # Roll back 2nd Diff\n        if second_diff:\n            df_fc[str(col)+'_1d'] = (df_train[col].iloc[-1]-df_train[col].iloc[-2]) + df_fc[str(col)+'_2d'].cumsum()\n        # Roll back 1st Diff\n        df_fc[str(col)+'_forecast'] = df_train[col].iloc[-1] + df_fc[str(col)+'_1d'].cumsum()\n    return df_fc\n<\/code><\/pre>\n<pre><code class=\"python language-python\">df_results = invert_transformation(train, df_forecast, second_diff=True)        \ndf_results.loc[:, ['rgnp_forecast', 'pgnp_forecast', 'ulc_forecast', 'gdfco_forecast',\n                   'gdf_forecast', 'gdfim_forecast', 'gdfcf_forecast', 'gdfce_forecast']]\n<\/code><\/pre>\n<p><a href=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR-Forecasts-min.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR-Forecasts-min-1024x198.png\" alt=\"VAR Forecasts\" width=\"865\" height=\"167\" class=\"aligncenter size-large wp-image-2165\" srcset=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR-Forecasts-min-1024x198.png 1024w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR-Forecasts-min-300x58.png 300w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR-Forecasts-min-768x149.png 768w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR-Forecasts-min-640x124.png 640w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR-Forecasts-min-100x19.png 100w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR-Forecasts-min-865x167.png 865w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/VAR-Forecasts-min.png 1184w\" sizes=\"(max-width: 865px) 100vw, 865px\" \/><\/a><\/p>\n<p>The forecasts are back to the original scale. Let&#8217;s plot the forecasts against the actuals from test data.<\/p>\n<h2 id=\"15plotofforecastvsactuals\">15. Plot of Forecast vs Actuals<\/h2>\n<pre><code class=\"python language-python\">fig, axes = plt.subplots(nrows=int(len(df.columns)\/2), ncols=2, dpi=150, figsize=(10,10))\nfor i, (col,ax) in enumerate(zip(df.columns, axes.flatten())):\n    df_results[col+'_forecast'].plot(legend=True, ax=ax).autoscale(axis='x',tight=True)\n    df_test[col][-nobs:].plot(legend=True, ax=ax);\n    ax.set_title(col + \": Forecast vs Actuals\")\n    ax.xaxis.set_ticks_position('none')\n    ax.yaxis.set_ticks_position('none')\n    ax.spines[\"top\"].set_alpha(0)\n    ax.tick_params(labelsize=6)\n\nplt.tight_layout();\n<\/code><\/pre>\n<figure id=\"attachment_2151\" aria-describedby=\"caption-attachment-2151\" style=\"width: 1184px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/forecast_vs_actuals_VAR.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/forecast_vs_actuals_VAR.png\" alt=\"Forecast vs Actuals comparison of VAR model\" width=\"1184\" height=\"1184\" class=\"size-full wp-image-2151\" srcset=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/forecast_vs_actuals_VAR.png 1184w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/forecast_vs_actuals_VAR-150x150.png 150w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/forecast_vs_actuals_VAR-300x300.png 300w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/forecast_vs_actuals_VAR-768x768.png 768w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/forecast_vs_actuals_VAR-1024x1024.png 1024w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/forecast_vs_actuals_VAR-640x640.png 640w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/forecast_vs_actuals_VAR-144x144.png 144w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/forecast_vs_actuals_VAR-100x100.png 100w, https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/forecast_vs_actuals_VAR-865x865.png 865w\" sizes=\"(max-width: 1184px) 100vw, 1184px\" \/><\/a><figcaption id=\"caption-attachment-2151\" class=\"wp-caption-text\">Forecast vs Actuals comparison of VAR model<\/figcaption><\/figure>\n<h2 id=\"16evaluatetheforecasts\">16. Evaluate the Forecasts<\/h2>\n<p>To evaluate the forecasts, let&#8217;s compute a comprehensive set of metrics, namely, the MAPE, ME, MAE, MPE, RMSE, corr and minmax.<\/p>\n<pre><code class=\"python language-python\">from statsmodels.tsa.stattools import acf\ndef forecast_accuracy(forecast, actual):\n    mape = np.mean(np.abs(forecast - actual)\/np.abs(actual))  # MAPE\n    me = np.mean(forecast - actual)             # ME\n    mae = np.mean(np.abs(forecast - actual))    # MAE\n    mpe = np.mean((forecast - actual)\/actual)   # MPE\n    rmse = np.mean((forecast - actual)**2)**.5  # RMSE\n    corr = np.corrcoef(forecast, actual)[0,1]   # corr\n    mins = np.amin(np.hstack([forecast[:,None], \n                              actual[:,None]]), axis=1)\n    maxs = np.amax(np.hstack([forecast[:,None], \n                              actual[:,None]]), axis=1)\n    minmax = 1 - np.mean(mins\/maxs)             # minmax\n    return({'mape':mape, 'me':me, 'mae': mae, \n            'mpe': mpe, 'rmse':rmse, 'corr':corr, 'minmax':minmax})\n\nprint('Forecast Accuracy of: rgnp')\naccuracy_prod = forecast_accuracy(df_results['rgnp_forecast'].values, df_test['rgnp'])\nfor k, v in accuracy_prod.items():\n    print(adjust(k), ': ', round(v,4))\n\nprint('\\nForecast Accuracy of: pgnp')\naccuracy_prod = forecast_accuracy(df_results['pgnp_forecast'].values, df_test['pgnp'])\nfor k, v in accuracy_prod.items():\n    print(adjust(k), ': ', round(v,4))\n\nprint('\\nForecast Accuracy of: ulc')\naccuracy_prod = forecast_accuracy(df_results['ulc_forecast'].values, df_test['ulc'])\nfor k, v in accuracy_prod.items():\n    print(adjust(k), ': ', round(v,4))\n\nprint('\\nForecast Accuracy of: gdfco')\naccuracy_prod = forecast_accuracy(df_results['gdfco_forecast'].values, df_test['gdfco'])\nfor k, v in accuracy_prod.items():\n    print(adjust(k), ': ', round(v,4))\n\nprint('\\nForecast Accuracy of: gdf')\naccuracy_prod = forecast_accuracy(df_results['gdf_forecast'].values, df_test['gdf'])\nfor k, v in accuracy_prod.items():\n    print(adjust(k), ': ', round(v,4))\n\nprint('\\nForecast Accuracy of: gdfim')\naccuracy_prod = forecast_accuracy(df_results['gdfim_forecast'].values, df_test['gdfim'])\nfor k, v in accuracy_prod.items():\n    print(adjust(k), ': ', round(v,4))\n\nprint('\\nForecast Accuracy of: gdfcf')\naccuracy_prod = forecast_accuracy(df_results['gdfcf_forecast'].values, df_test['gdfcf'])\nfor k, v in accuracy_prod.items():\n    print(adjust(k), ': ', round(v,4))\n\nprint('\\nForecast Accuracy of: gdfce')\naccuracy_prod = forecast_accuracy(df_results['gdfce_forecast'].values, df_test['gdfce'])\nfor k, v in accuracy_prod.items():\n    print(adjust(k), ': ', round(v,4))\n<\/code><\/pre>\n<pre><code>Forecast Accuracy of: rgnp\nmape   :  0.0192\nme     :  79.1031\nmae    :  79.1031\nmpe    :  0.0192\nrmse   :  82.0245\ncorr   :  0.9849\nminmax :  0.0188\n\nForecast Accuracy of: pgnp\nmape   :  0.0005\nme     :  2.0432\nmae    :  2.0432\nmpe    :  0.0005\nrmse   :  2.146\ncorr   :  1.0\nminmax :  0.0005\n\nForecast Accuracy of: ulc\nmape   :  0.0081\nme     :  -1.4947\nmae    :  1.4947\nmpe    :  -0.0081\nrmse   :  1.6856\ncorr   :  0.963\nminmax :  0.0081\n\nForecast Accuracy of: gdfco\nmape   :  0.0033\nme     :  0.0007\nmae    :  0.4384\nmpe    :  0.0\nrmse   :  0.5169\ncorr   :  0.9407\nminmax :  0.0032\n\nForecast Accuracy of: gdf\nmape   :  0.0023\nme     :  0.2554\nmae    :  0.29\nmpe    :  0.002\nrmse   :  0.3392\ncorr   :  0.9905\nminmax :  0.0022\n\nForecast Accuracy of: gdfim\nmape   :  0.0097\nme     :  -0.4166\nmae    :  1.06\nmpe    :  -0.0038\nrmse   :  1.0826\ncorr   :  0.807\nminmax :  0.0096\n\nForecast Accuracy of: gdfcf\nmape   :  0.0036\nme     :  -0.0271\nmae    :  0.4604\nmpe    :  -0.0002\nrmse   :  0.5286\ncorr   :  0.9713\nminmax :  0.0036\n\nForecast Accuracy of: gdfce\nmape   :  0.0177\nme     :  0.2577\nmae    :  1.72\nmpe    :  0.0031\nrmse   :  2.034\ncorr   :  0.764\nminmax :  0.0175\n<\/code><\/pre>\n<h2 id=\"17conclusion\">17. Conclusion<\/h2>\n<p>In this article we covered VAR from scratch beginning from the intuition behind it, interpreting the formula, causality tests, finding the optimal order of the VAR model, preparing the data for forecasting, build the model, checking for serial autocorrelation, inverting the transform to get the actual forecasts, plotting the results and computing the accuracy metrics. <\/p>\n<p>Hope you enjoyed reading this as much as I did writing it. I will see you in the next one.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Vector Autoregression (VAR) is a forecasting algorithm that can be used when two or more time series influence each other. That is, the relationship between the time series involved is bi-directional. In this post, we will see the concepts, intuition behind VAR models and see a comprehensive and correct method to train and forecast VAR [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":2174,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"site-sidebar-layout":"default","site-content-layout":"default","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"default","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[68],"tags":[85,103,22,86,69,84],"class_list":["post-2149","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-time-series","tag-grangers-causality","tag-machine-learning","tag-python","tag-statsmodels","tag-time-series","tag-vector-autoregression"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.0 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Vector Autoregression (VAR) - Comprehensive Guide with Examples in Python - machinelearningplus<\/title>\n<meta name=\"description\" content=\"Learn Vector Autoregression (VAR) for multivariate time series forecasting. Covers the intuition, math, model training, diagnostics, and forecasting with Python statsmodels.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Vector Autoregression (VAR) - Comprehensive Guide with Examples in Python - machinelearningplus\" \/>\n<meta property=\"og:description\" content=\"Learn Vector Autoregression (VAR) for multivariate time series forecasting. Covers the intuition, math, model training, diagnostics, and forecasting with Python statsmodels.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/\" \/>\n<meta property=\"og:site_name\" content=\"machinelearningplus\" \/>\n<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/rtipaday\/\" \/>\n<meta property=\"article:published_time\" content=\"2019-07-07T13:59:36+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-08-30T12:07:11+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/vector_autoregression_examples.png\" \/>\n\t<meta property=\"og:image:width\" content=\"560\" \/>\n\t<meta property=\"og:image:height\" content=\"315\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Selva Prabhakaran\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@https:\/\/twitter.com\/R_Programming\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Selva Prabhakaran\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"27 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/time-series\\\/vector-autoregression-examples-python\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/time-series\\\/vector-autoregression-examples-python\\\/\"},\"author\":{\"name\":\"Selva Prabhakaran\",\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/#\\\/schema\\\/person\\\/510885c0515804366fa644c38258391e\"},\"headline\":\"Vector Autoregression (VAR) &#8211; Comprehensive Guide with Examples in Python\",\"datePublished\":\"2019-07-07T13:59:36+00:00\",\"dateModified\":\"2022-08-30T12:07:11+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/time-series\\\/vector-autoregression-examples-python\\\/\"},\"wordCount\":2515,\"commentCount\":40,\"publisher\":{\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/time-series\\\/vector-autoregression-examples-python\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/machinelearningplus.com\\\/wp-content\\\/uploads\\\/2019\\\/07\\\/vector_autoregression_examples.png\",\"keywords\":[\"Grangers Causality\",\"Machine Learning\",\"Python\",\"statsmodels\",\"Time Series\",\"Vector Autoregression\"],\"articleSection\":[\"Time Series\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/machinelearningplus.com\\\/time-series\\\/vector-autoregression-examples-python\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/time-series\\\/vector-autoregression-examples-python\\\/\",\"url\":\"https:\\\/\\\/machinelearningplus.com\\\/time-series\\\/vector-autoregression-examples-python\\\/\",\"name\":\"Vector Autoregression (VAR) - Comprehensive Guide with Examples in Python - machinelearningplus\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/time-series\\\/vector-autoregression-examples-python\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/time-series\\\/vector-autoregression-examples-python\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/machinelearningplus.com\\\/wp-content\\\/uploads\\\/2019\\\/07\\\/vector_autoregression_examples.png\",\"datePublished\":\"2019-07-07T13:59:36+00:00\",\"dateModified\":\"2022-08-30T12:07:11+00:00\",\"description\":\"Learn Vector Autoregression (VAR) for multivariate time series forecasting. Covers the intuition, math, model training, diagnostics, and forecasting with Python statsmodels.\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/machinelearningplus.com\\\/time-series\\\/vector-autoregression-examples-python\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/time-series\\\/vector-autoregression-examples-python\\\/#primaryimage\",\"url\":\"https:\\\/\\\/machinelearningplus.com\\\/wp-content\\\/uploads\\\/2019\\\/07\\\/vector_autoregression_examples.png\",\"contentUrl\":\"https:\\\/\\\/machinelearningplus.com\\\/wp-content\\\/uploads\\\/2019\\\/07\\\/vector_autoregression_examples.png\",\"width\":560,\"height\":315,\"caption\":\"vector autoregression model feature image\"},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/#website\",\"url\":\"https:\\\/\\\/machinelearningplus.com\\\/\",\"name\":\"machinelearningplus\",\"description\":\"Learn Data Science (AI \\\/ ML) Online\",\"publisher\":{\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/machinelearningplus.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/#organization\",\"name\":\"machinelearningplus\",\"url\":\"https:\\\/\\\/machinelearningplus.com\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/machinelearningplus.com\\\/wp-content\\\/uploads\\\/2022\\\/05\\\/MachineLearningplus-logo.svg\",\"contentUrl\":\"https:\\\/\\\/machinelearningplus.com\\\/wp-content\\\/uploads\\\/2022\\\/05\\\/MachineLearningplus-logo.svg\",\"width\":348,\"height\":36,\"caption\":\"machinelearningplus\"},\"image\":{\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/#\\\/schema\\\/person\\\/510885c0515804366fa644c38258391e\",\"name\":\"Selva Prabhakaran\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/machinelearningplus.com\\\/wp-content\\\/litespeed\\\/avatar\\\/a994280177da541405c016f593e86ea7.jpg?ver=1783622267\",\"url\":\"https:\\\/\\\/machinelearningplus.com\\\/wp-content\\\/litespeed\\\/avatar\\\/a994280177da541405c016f593e86ea7.jpg?ver=1783622267\",\"contentUrl\":\"https:\\\/\\\/machinelearningplus.com\\\/wp-content\\\/litespeed\\\/avatar\\\/a994280177da541405c016f593e86ea7.jpg?ver=1783622267\",\"caption\":\"Selva Prabhakaran\"},\"description\":\"Selva is an experienced Data Scientist and leader, specializing in executing AI projects for large companies. Selva started machinelearningplus to make Data Science \\\/ ML \\\/ AI accessible to everyone. The website enjoys 4 Million+ readership. His courses, lessons, and videos are loved by hundreds of thousands of students and practitioners.\",\"sameAs\":[\"https:\\\/\\\/localhost:8080\\\/\",\"https:\\\/\\\/www.facebook.com\\\/rtipaday\\\/\",\"https:\\\/\\\/x.com\\\/https:\\\/\\\/twitter.com\\\/R_Programming\"],\"url\":\"https:\\\/\\\/machinelearningplus.com\\\/author\\\/selva86\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Vector Autoregression (VAR) - Comprehensive Guide with Examples in Python - machinelearningplus","description":"Learn Vector Autoregression (VAR) for multivariate time series forecasting. Covers the intuition, math, model training, diagnostics, and forecasting with Python statsmodels.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/","og_locale":"en_US","og_type":"article","og_title":"Vector Autoregression (VAR) - Comprehensive Guide with Examples in Python - machinelearningplus","og_description":"Learn Vector Autoregression (VAR) for multivariate time series forecasting. Covers the intuition, math, model training, diagnostics, and forecasting with Python statsmodels.","og_url":"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/","og_site_name":"machinelearningplus","article_author":"https:\/\/www.facebook.com\/rtipaday\/","article_published_time":"2019-07-07T13:59:36+00:00","article_modified_time":"2022-08-30T12:07:11+00:00","og_image":[{"width":560,"height":315,"url":"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/vector_autoregression_examples.png","type":"image\/png"}],"author":"Selva Prabhakaran","twitter_card":"summary_large_image","twitter_creator":"@https:\/\/twitter.com\/R_Programming","twitter_misc":{"Written by":"Selva Prabhakaran","Est. reading time":"27 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/#article","isPartOf":{"@id":"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/"},"author":{"name":"Selva Prabhakaran","@id":"https:\/\/machinelearningplus.com\/#\/schema\/person\/510885c0515804366fa644c38258391e"},"headline":"Vector Autoregression (VAR) &#8211; Comprehensive Guide with Examples in Python","datePublished":"2019-07-07T13:59:36+00:00","dateModified":"2022-08-30T12:07:11+00:00","mainEntityOfPage":{"@id":"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/"},"wordCount":2515,"commentCount":40,"publisher":{"@id":"https:\/\/machinelearningplus.com\/#organization"},"image":{"@id":"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/#primaryimage"},"thumbnailUrl":"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/vector_autoregression_examples.png","keywords":["Grangers Causality","Machine Learning","Python","statsmodels","Time Series","Vector Autoregression"],"articleSection":["Time Series"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/","url":"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/","name":"Vector Autoregression (VAR) - Comprehensive Guide with Examples in Python - machinelearningplus","isPartOf":{"@id":"https:\/\/machinelearningplus.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/#primaryimage"},"image":{"@id":"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/#primaryimage"},"thumbnailUrl":"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/vector_autoregression_examples.png","datePublished":"2019-07-07T13:59:36+00:00","dateModified":"2022-08-30T12:07:11+00:00","description":"Learn Vector Autoregression (VAR) for multivariate time series forecasting. Covers the intuition, math, model training, diagnostics, and forecasting with Python statsmodels.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/machinelearningplus.com\/time-series\/vector-autoregression-examples-python\/#primaryimage","url":"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/vector_autoregression_examples.png","contentUrl":"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2019\/07\/vector_autoregression_examples.png","width":560,"height":315,"caption":"vector autoregression model feature image"},{"@type":"WebSite","@id":"https:\/\/machinelearningplus.com\/#website","url":"https:\/\/machinelearningplus.com\/","name":"machinelearningplus","description":"Learn Data Science (AI \/ ML) Online","publisher":{"@id":"https:\/\/machinelearningplus.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/machinelearningplus.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/machinelearningplus.com\/#organization","name":"machinelearningplus","url":"https:\/\/machinelearningplus.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/machinelearningplus.com\/#\/schema\/logo\/image\/","url":"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2022\/05\/MachineLearningplus-logo.svg","contentUrl":"https:\/\/machinelearningplus.com\/wp-content\/uploads\/2022\/05\/MachineLearningplus-logo.svg","width":348,"height":36,"caption":"machinelearningplus"},"image":{"@id":"https:\/\/machinelearningplus.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/machinelearningplus.com\/#\/schema\/person\/510885c0515804366fa644c38258391e","name":"Selva Prabhakaran","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/machinelearningplus.com\/wp-content\/litespeed\/avatar\/a994280177da541405c016f593e86ea7.jpg?ver=1783622267","url":"https:\/\/machinelearningplus.com\/wp-content\/litespeed\/avatar\/a994280177da541405c016f593e86ea7.jpg?ver=1783622267","contentUrl":"https:\/\/machinelearningplus.com\/wp-content\/litespeed\/avatar\/a994280177da541405c016f593e86ea7.jpg?ver=1783622267","caption":"Selva Prabhakaran"},"description":"Selva is an experienced Data Scientist and leader, specializing in executing AI projects for large companies. Selva started machinelearningplus to make Data Science \/ ML \/ AI accessible to everyone. The website enjoys 4 Million+ readership. His courses, lessons, and videos are loved by hundreds of thousands of students and practitioners.","sameAs":["https:\/\/localhost:8080\/","https:\/\/www.facebook.com\/rtipaday\/","https:\/\/x.com\/https:\/\/twitter.com\/R_Programming"],"url":"https:\/\/machinelearningplus.com\/author\/selva86\/"}]}},"_links":{"self":[{"href":"https:\/\/machinelearningplus.com\/wp-json\/wp\/v2\/posts\/2149","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/machinelearningplus.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/machinelearningplus.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/machinelearningplus.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/machinelearningplus.com\/wp-json\/wp\/v2\/comments?post=2149"}],"version-history":[{"count":0,"href":"https:\/\/machinelearningplus.com\/wp-json\/wp\/v2\/posts\/2149\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/machinelearningplus.com\/wp-json\/wp\/v2\/media\/2174"}],"wp:attachment":[{"href":"https:\/\/machinelearningplus.com\/wp-json\/wp\/v2\/media?parent=2149"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/machinelearningplus.com\/wp-json\/wp\/v2\/categories?post=2149"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/machinelearningplus.com\/wp-json\/wp\/v2\/tags?post=2149"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}