Category Archives: mean

December 2, 2009 · 7:45 pm

Mean in the context of Mean-Reversion

I want a running mean estimator that acts as a mode through mean reversion cycles of target amplitude or frequency.   The key characteristics should be:

  • adaptation to local volatility
    • determination of diffusion related squared return
    • determination of jump related squared return
    • determination as to how much of the jump should be absorbed into the mean
  • model of mean reversion
    • calibrated to a desired long-run rate of reversion
    • allowance for changes in reversion constant and reversion to long run
  • model of mean
    • autoregressive
    • innovations scaled by sigma term (with MR component and jumps removed)
  • recursive backward estimation of ML
    • implicitly decide how innovation is distributed amongst mean, mean-reversion, and noise

A SDE-based Approach
The model is an expanded variant of the familiar Ornstein–Uhlenbeck process, with specialized mean-reversion, mean, and volatility processes.   It also attempts to correct for jumps.    Let’s start with the following SDEs (in continuous time):

Variance
There are many approaches to modeling volatility (all with issues).   Initially I had though to use a predictive model based on:

  • intensity process (based on “first exit” duration)
    This is a very complex process.  First approximations have been to use ACD, a family of AR models for duration.   ACD models perform very poorly on HF data however.    It seems that a markov chain model recognizing the patterns will be most appropriate.
  • amplitude process
    The amplitudes of squared returns seem to follow a largely AR process.   This seems fairly well behaved.

Before fully committing to a complex volatility model thought its makes sense to first try with a non-predictive measure of realized variance.  I will use:

The choice of α determines the degree of smoothing with previous values based on how local (and noisy) we want this function to be.   For example, here is the estimate with a smoothing factor of 60 and a threshold of 3e-5:

Discretising
Using Ito’s lemma we discretise the processes as follows:

Simplifying the volatility term in S(t), we first determine the variance of the SDE:

We reorganize as follows:

Putting it together
We can now model this discretely as a state-space based filter, searching for parameters that fit a-posteriori idealized view on the mode and mean-reversion process.   Post-parameterization, the process can be used in real-time to provide an estimate of the mode.

Final Notes
As you may have seen I took a (useful) 2-3 week diversion before coming back to the SDE based approach.   This is not a final model by any means, but I think a a solid starting point.    The purpose of the above is as a one of a number of factors in a multi-factor  strategy that want to optimize further.

Leave a comment

Filed under mean, state-space-models, statistics, stochatistic, volatility

November 6, 2009 · 8:02 am

Mode of the Signal Envelope

One thing that struck me as clever with the HHT was the use of projecting a spline across the minima and maxima for a given harmonic.   In effect this defines the envelope for the series for a given harmonic (level of decomposition).   A posteri, the mean or mode should be more or less equivalent to the average of the envelope splines.   Interesting!

This is a very appropriate way to model the mean within the context of mean-reversion (ie oscillations around the mode within an envelope).   Instead of trying to model the mean directly as a stochastic process, why not model the envelope — this is more appropriate as we can fit the envelope into our view of mean reversion.

Version 1
I used a regressor to estimate the mean and connected minima and maxima with a spline for the envelope.  The approach has issues (such as what sort of bias does the mean regressor have with respect to the data).   There are some issues below:

Picture 1

Version 2
I took a dfference approach, estimating the inflection points with a regressing “oscillator”  (in green) and determining the mid-points between minima and maxima to produce a spline representing the mode (blue).   So far looks good.   Edge cases, consolidation, and jumps need to be considered:

Picture 2

More on this later.

Leave a comment

Filed under mean, regression, signal-processing, statistics, technical-analysis

November 4, 2009 · 7:46 pm

Intrinsic Mode Function (Basis Decomposition)

Norden Huang did a very interesting talk at CERN a few months ago “A New Method for Non-linear and Non-stationary Time Series Analysis: The Hilbert Spectral Analysis“.   I found this through Max Dama’s blog (thanks).

Huang proposes a new approach to signal decomposition for non-linear, non-stationary signals.   In the presentation he walks through the issues with Fourier, Wavelet, and Poincare analysis.   The issues with each of these approaches is that they either miss the dynamics in the time domain (fourier, poincare) or miss features in the frequency domain (wavelet).   He further goes on to show that, though Hilbert space analysis presents both the time and frequency domain, the results are skewed by non-stationarity.

My Approach
Before I get into Huang’s approach, let me detail the approach I have been using.   I had explored wavelets a few years ago and realized their limitations.   Wavelets with orthogonal bases lose features in the timeseries due to the 2^n partitioning of the signal.   Features that are aligned at the center of the 2^n partitions are captured most accurately and those on the fringes with diminished accuracy (or not at all).   I designed an empirical basis in response to this:

let X <- <signal vector>
let residual <- X
for (n,rho) in <successive values approximating frequencies 2^n>
{
    # compute data derived basis function
    basis[n] <- <penalized least squares spline> (residual, rho)
    # compute residual
    residual <- residual - basis[n]
}

The above uses a penalized least squares spline as a (near-orthogonal) basis function, decomposing the signal at successive frequencies.   The decomposition has little cross-correlation so works out to be near orthogonal for the most part.

Basis functions in my approach:

Picture 5

Various stages of recomposition:

Picture 3

The basis functions are each optimal least-squares fits and have very little in the way of artifacts.   The downside of this basis function is that there is non-locality, in that earlier parts of the signal influence the basis function in later parts (usually in a non-intrusive way) and is expensive to compute at high frequency.

HHT (Hilbert-Huang Transform)
Huang’s approach is quite clever and importantly, parsimonious.   The approach was motivated by the issues one gets with the Hilbert tranform when dealing with a non-stationary timeseries.   What if we could create an empirical basis function that creates bases that are centered around the maxima/minima and represent the mode.   The algorithm is as follows:

let X <- <signal vector>
decomp[1] <- X
for (i in 2:<maximum bases for this signal>)
{
    # determine local maxima and minima in current component
    maxima <- <locate maxima> (decomp[i-1])
    minima <- <locate minima> (decomp[i-1])

    # compute splines through maxima & minima
    maxspline <- spline (maxima)
    minspline <- spline (minima)

    # compute basis function
    mean[i] <- (minspline + maxspline) / 2
    decomp[i] <- decomp[i-1] - mean
}

Basis functions in Huang’s approach:

Picture 6

Various stages of recomposition with Huang’s approach:

Picture 4

The resulting basis function has some interesting properties such as the functions are centered around the origin with a long term mean approaching zer0 and have locality.   HHT is especially good at capturing non-stationary high-frequency oscillations with fidelity.

There are, however, some noticable artifacts due to the natural-spline approach (notice exaggerated pertubations in the curve near abrupt price movements).   This approach fairs less well with abrupt shifts and at timeseries boundaries then the approach I have been using.

There are some aspects of this approach, though, that may make this useful in analysing the mean.  More on this later.

Leave a comment

Filed under mean, signal-processing

November 4, 2009 · 1:01 pm

The “Mean”, take 2

Thinking about the prior post, am not satisfied with the approach in that it does not quantify properties of the mean and price relative to the mean in a way that is explicit.

Let’s examine various properties the system should represent:

  1. Integral of Y[t] – μ[t] should be close to 0
    This implies that the mean’s course is balanced between time/distance spent below the mean and above the mean.
  2. average max amplitude should meet a target amplitude
    We want a predictable mean reversion process.   One approach to this is ensuring that the mean is such that it allows for some average max deviation.   We will be working with an evolving  distribution of amplitudes to modify the behavior of the mean.
  3. Should be smooth and continuous with exception of jumps
    That is we minimize the integral of μ”(t)^2 in some ratio with other constraints.   Alternatively we could require a AR(p) process to provide continuity with prior μ(t) observations.

A-posteriori Approach
This is a relatively simple problem to solve after-the-fact, where we find a regressor f(t) that meets the above requirements.   The tricky aspect is in the observation of minima and maxima of the price difference in a way that we can integrate into a system of equations.

Assuming we have the regressor f(t), and a function that evaluates the average amplitude over the regressor Ea (f), we can express as:

Picture 3

The problem with this is that though it is optimal for the data set over which it is evaluated it is unlikely to be optimal relative to future values of Yi.  Observe how the regressors differ (the red with the original data series and the green relative to an additional hour of data):

Picture 2

The initial regressor (red) is no longer optimal given another hr of data.  The green regressor now represents the optimum.   This analysis points in the direction of determining an “online” estimate of the mean which works probabilistically.

Online Approach
I do not yet have a concrete solution in mind, so what follows is a train of thought:

  • use an evolving stochastic cubic system with constraints to guide coefficient processes (what constraints from above and how)
  • alternatively model the mean on an autoregressive process with innovations in proportion to a running variance estimate.   Equating variance and level-duration arrive at a formulation for the amount of “innovation” and therefore deviation from the mean allowed.
  • Another variant of the above approach is to adjust the AR coefficients to respond to changes in volatility.

I’ll update as the ideas mature …

Leave a comment

Filed under mean, signal-processing, statistics, stochatistic

November 3, 2009 · 7:16 pm

The “Mean” in the context of a mean-reverting process

We know that price processes exhibit mean reversion on multiple timescales.   Supposing one wants to measure the distance between the price X[t] and the mean E[X[t]].   The mean of a price process is particularly difficult to model as the process is non-stationary.    The standard approach in options modeling has been to model in terms of a near-stationary process such as log returns .

I am interested in the empirical price distance to the mean.   One could look at the sum of log returns as a proxy to price, but empirical returns are very noisy and the mean-reversion aspect would be lost.    This works out in a continuous model, but wold require some smoothing to make sense in an empirical model.

The “Mean”
Focusing again on the mean, in a non-stationary process, we can take different views on the mean depending on the timescale of mean reversion we are interested in focusing on.   Here is an example of different means through a price series (taken ex-post as a penalized least squares regression):

Picture 1

The mean for a random variable is defined as the expectation of the variable E[X].  That’s fine, except that E[x] is model dependent and also dependent on our desired mean-reversion timescale.  In the example above all of the regression lines represent reasonable values for the mean and are optimal in a least squares sense, so which “mean” do we choose and what process generates it?

Modeling Issues
Ultimately would like to come up with an empirical, discrete, state system that allows me to estimate the mean as a hidden state from a maximum likelihood point of view.   Just as with “volatility”, the mean is not truely observable.  The model needs express the following:

  1. represent mean reversion timescale of interest
  2. deal with jumps / regime change in the mean
  3. mean is a function of time
  4. variance is a function of time

Models
A common place to start with mean-reversion models is the Ornstein-Uhlenbeck process:

Picture 11

Note that this is expressed in terms of a long-run mean (constant) μ, a mean-reversion coefficient (constant) κ, and standard deviation (constant) σ.  Also note that this is a process modeling the change in price (usually log change).

The Hull-White model went a step further to allow the mean and standard deviation to be a function of time.   Finally the extended Vasicek model allows for the mean-reversion constant to vary with time:

Picture 5

Ok, a few problems.   We have not addressed the jump aspect of price movement and also need to convert this into a discrete empirical process.   One way of introducing a jump component is modifying the above formulation to include a jump process dQ (note that I have adjusted the mean-reversion process to be a constant to allow us to solve for μ(t) later:

Picture 7

This is not the most sophisticated model, however, for the purposes of modeling the mean should prove useful.

Discrete form
Let’s ignore the jump component for the moment, and arrive at a discrete formulation for the Gaussian component in the mixture.   A general gaussian markov process has the form:

Picture 1

we get the following solution:

Picture 2

For our equation this works out to:

Picture 3

or between times t-Δt and t:

Picture 4

We now have enough to formulate a state system.   We will assume the change in volatility or mean over Δt to be small enough so that can assume volatility and mean are constant over the integral for Δt.

First we need to decide on a process to express variance.   For daily returns we can use GARCH(1,1), and for intraday will need an alternate model:

Picture 7

Within the context of a discrete increment Δt within our mean reverting SDE, the volatility scales as:

Picture 8

We express the mean as an AR(2) process, but within the larger state system so that it accepts “innovations” from the mean reversion process (via recursive feedback):

Picture 9

We can easily express this state system in a particle filter to find the ML parameters (assuming constant κ).

Handling Jumps
Given that we are not using this SDE in a predictive fashion, we can introduce jumps as a point process or alternatively as a cumulator variable as we observe jumps in the market.   So for instance, could reexpress the mean process as:

Picture 10

Conclusion
Our goal here was to determine an accurate measure of the mean, one that we can consider optimal from a model / maximum likelihood point of view.   I now need to implement this state system in a particle filter and determine how effective it is.   Some enhancements may be required.    This is a work in progress, look for further updates …

Leave a comment

Filed under mean, stochatistic