This project is for the comparison of the Mathematica and R implementation and utilization of software monads for the specification and execution of Quantile Regression workflows.

The project was aimed to mirror and aid the workshop "Quantile Regression Workshop" of the meetup Orlando Machine Learning and Data Science.

This project was also used for the presentation: "Intro to Quantile Regression Workflows (Boston useR Group)" of the meetup Greater Boston useR Group (R Programming Language). (That particular event was co-hosted by ODSC.)

The Mathematica / Wolfram Language monad QRMon is implemented with the package "MonadicQuantileRegression.m", [AAp1], and it is based on the package "QuantileRegression.m", [AAp2]. (Below this software monad is referenced as QRMon-WL.)

The R monad is also named QRMon, it is implemented with the package "QRMon-R", [AAp3], and it is based on the Quantile Regression package quantreg, [RKp1, RK1]; it also utilizes the "pipeline" package magrittr. (Below this software monad is referenced as QRMon-R.)

The goals of this workshop are to introduce the theoretical background of Quantile Regression (QR), to demonstrate QR's practical (and superior) abilities to deal with "real life" time series data, and to teach how to rapidly create QR workflows using Mathematica or R.

• Quantile Regression (QR) theory (7-10 min)

  • Brief outline

  • Warm-up examples

  • Linear programming problem formulation

    • Just a sketch

  • Analogies between Quantile Regression and Neural Networks

• Monadic programming (5 min)

  • Why use it?

  • What are the main benefits?

  • Monads vs pipelines

  • Why apply it to QR?

• Introduction to QRMon (15 min)

  • Installing the packages

  • Review of the look-and-feel

• Main benefits of QR (first wave examples) (30-40 min)

  • All done through monadic pipelines (workflows)

  • Time series analysis

  • Conditional PDF’s and CDF’s

  • Outlier detection

  • Simulation of time series

• QR workflows (second wave examples) (30 min)

  • Using StackExchange questions (and data)

  • Data cleaning)

  • Envelopes

  • Picking points of interest

  • Brute force identification of expansion terms

  • Finding peaks

  • Data generation

For introduction to Quantile Regression see the YouTube video "Quantile Regression—Theory, Implementations, and Applications".

Video of the "Quantile Regression Workflows" presentation at Boston useR! Meetup 2019.04.18.

Here is a video for using the QRMon-WL monad.

Here is a video for using the QRMon-R monad.

• Follow the GitHub Markdown article "A monad for Quantile Regression workflows".

• Examine QRMon-WL's unit test files:

  • MonadicQuantileRegression-Unit-Tests.wlt and

  • MonadicQuantileRegressionRandomPipelinesUnitTests.m.

• Download/follow the introductory notebook "Separation by regression quantiles" (html version).

• Read/follow the vignette "Rapid making of Quantile Regression workflows".

• Examine QRMon-R's unit test files.

• If using Mathematica (and having an internet connection) the financial and weather data can obtained with the functions FinancialData and WeatherData.

• I using R: the QRMon-R package comes with three sample datasets (distribution, financial, and weather data.)

• The data used in the workshop examples can be obtained from this repository.

  • See the sub-directory "Data".

Here is a monadic pipeline in Mathematica:

qrmon =
   QRMonUnit[distData]⟹
   QRMonEchoDataSummary⟹
   QRMonQuantileRegression[12]⟹
   QRMonPlot;

Here is a monadic pipeline in R:

 qrmon <-
   QRMonUnit( dfTemperatureData ) %>%
   QRMonEchoDataSummary() %>%
   QRMonQuantileRegression( df = 16, degree = 3, quantiles = seq(0.1,0.9,0.2) ) %>%
   QRMonPlot( datePlotQ = TRUE, dateOrigin = "1900-01-01" )  

Because of the specialized algorithms developed by Roger Koenker et al. the package quantreg, [RKp1], produces Quantile Regression fits much faster than the package [AAp3]. (The latter uses the built-in Mathematica functions LinearProgramming, Minimize, and NMinimize. LinearProgramming is the fastest.)

TBD...

TBD...

[AAp1] Anton Antonov, Monadic Quantile Regression Mathematica package, (2018), MathematicaForPrediction at GitHub.

[AAp2] Anton Antonov, Quantile Regression Mathematica package, (2014), MathematicaForPrediction at GitHub.

[AAp3] Anton Antonov, Quantile Regression workflows monad in R, (2018), QRMon-R at GitHub.

[RKp1] Roger Koenker et al., "quantreg: Quantile Regression", (2018).

[AA1] Anton Antonov, "Monad code generation and extension", (2017), MathematicaForPrediction at GitHub.

[AA2] Anton Antonov, "Quantile regression through linear programming", (2013), MathematicaForPrediction at WordPress.

[AA3] Anton Antonov, "Quantile regression with B-splines", (2014), MathematicaForPrediction at WordPress.

[AA4] Anton Antonov, "Estimation of conditional density distributions", (2014), MathematicaForPrediction at WordPress.

[AA5] Anton Antonov, "Finding local extrema in noisy data using Quantile Regression", (2015), MathematicaForPrediction at WordPress.

[AA6] Anton Antonov, A monad for Quantile Regression workflows, (2018), MathematicaForPrediction at WordPress.

[Wk1] Wikipedia entry, Monad.

[Wk2] Wikipedia entry, Quantile Regression/

[Wk3] Wikipedia entry, Chebyshev polynomials/

[CN1] Brian S. Code and Barry R. Noon, "A gentle introduction to quantile regression for ecologists", (2003). Frontiers in Ecology and the Environment. 1 (8): 412-420. doi:10.2307/3868138.

[RK1] Roger Koenker, "Quantile Regression in R: a vignette", (2018).

[RK2] Roger Koenker, Quantile Regression, ‪Cambridge University Press, 2005‬.